Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

force_reload=True on a single dataset makes minitree twice #48

Open
JelleAalbers opened this issue Dec 7, 2016 · 0 comments
Open

force_reload=True on a single dataset makes minitree twice #48

JelleAalbers opened this issue Dec 7, 2016 · 0 comments

Comments

@JelleAalbers
Copy link
Contributor

JelleAalbers commented Dec 7, 2016

We must first make the minitrees from the first dataset given to hax.minitrees.load, since the dask multiprocessing only works if it knows which variables to expect. Since these both get the force_reload option, using a single dataset with minitrees.load and force_reload causes a double remake.

Not a big issue but hints at a bigger problem: having to make the minitrees to know which variables it will produce. This is inconvenient for other reasons too (see e.g. #47).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant