Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Allow sampling in the backgound #94

Merged
merged 6 commits into from
Mar 18, 2024

Conversation

aseyboldt
Copy link
Member

@aseyboldt aseyboldt commented Mar 15, 2024

Allow sampling in the background:

import pymc as pm
import nutpie

with pm.Model() as model:
    pm.Normal("x", shape=10_000)

compiled = nutpie.compile_pymc_model(model)

sampler = nutpie.sample(compiled, blocking=False)

try:
    sampler.wait(timeout=2)
except TimeoutError:
    pass

#sampler.abort()
trace = sampler.finalize()

cc @ricardoV94

Also fixes #90

@aseyboldt aseyboldt force-pushed the background-sampling branch 3 times, most recently from 68110b3 to bdd60ff Compare March 15, 2024 13:06
@aseyboldt aseyboldt force-pushed the background-sampling branch from bdd60ff to 7412e1a Compare March 15, 2024 13:13
Copy link
Member

@ricardoV94 ricardoV94 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neat

@ricardoV94
Copy link
Member

Is there a way to probe if the sampling is finished without interrupting or blocking? Not clear from skimming the code

@aseyboldt
Copy link
Member Author

Is there a way to probe if the sampling is finished without interrupting or blocking? Not clear from skimming the code

Now there is a is_finished property...

@aseyboldt aseyboldt force-pushed the background-sampling branch from a24dffa to 9a33176 Compare March 18, 2024 10:56
@aseyboldt
Copy link
Member Author

And now also sampler.pause() and sampler.resume() and sampler.wait(timeout=seconds).

A form of sampler.inspect() that returns the partial trace that's been produced so far would stilll be nice, but I'll leave that for later.

@aseyboldt
Copy link
Member Author

@ricardoV94 If you have a minute, this should be ready for review with the new changes again.

@ricardoV94
Copy link
Member

ricardoV94 commented Mar 18, 2024

Wanna test some of those methods in the new test?


Would be nice to setup some docs to show this functionality, but need not be in this PR.
The only docs that exist right now are the README :D
Perhaps a pymc-examples is also a good place.

Copy link
Member

@ricardoV94 ricardoV94 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is awesome. The inspect thing will also be great in the future!

@aseyboldt aseyboldt merged commit cfceaf0 into pymc-devs:main Mar 18, 2024
7 checks passed
@aseyboldt aseyboldt deleted the background-sampling branch March 18, 2024 16:21
@AlexAndorra
Copy link
Contributor

This is super cool @aseyboldt !!
I just started playing around with this, but I think the finalize doesn't exist, so you can't do trace = sampler.finalize() to store the trace once the sampler is done.

What I ended up doing is:

compiled_model = nutpie.compile_pymc_model(model)
sampler = nutpie.sample(compiled_model, blocking=False, save_warmup=False, store_unconstrained=False)
idata.extend(sampler.abort())

That works, but feels weird from a user perspective.

Is this a bug? I'm happy to open an issue if that's helpful

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

After chains in warmup go to zero, show how many chains are still sampling instead
3 participants