Automatic fix and detection of partial installations, option to bundle extra wheels for simpler offline and monorepo builds #169
Tremeschin
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi, I've got a fork with two interesting features that adds extra safety and flexibility for monorepos and offline installations.
Pardon the length, I like to add reasoning and context!
Partial installation detection
Problem: Whenever pyapp is closed mid-way an installation, the heuristic in
ensure_ready()
of checking for the existence of!app::install_dir().is_dir()
fails to notice this partial installation, and subsequent runs gives module not found (as it's not.. installed). The only way out is doing aself restore
or manually deleting the installation path.While this is trivial for us programmers, giving such instructions to (Windows) users are annoying and error-prone
Solution: A
ready.flag
file is created in the root of the installation afterinstall_project()
, andensure_ready()
logic now checks for its existence beforematerialize()
and creates it after so. The file works like a checkpoint ensuring previous runs reached the end of the installation correctly! :)I'm not sure if this is the best place to put it, maybe calling
self restore
at the beginning ofrun_project()
which also works, but could break isolation compatibilityIn the same idea, we can save the current binary hash to a file and call
self restore
if a same-version but different contents release was run (mostly in dev mode), I also have coded it, but within Python this time, and it works perfectly.Bundling extra wheels
Problem: In a monorepo scenario like mine, I'm having to bundle all sub-projects packages into a monolithic wheel in
PYAPP_PROJECT_PATH
with[tool.hatch.build.targets.wheel]
, and definingmain-lib[all]
dependency to be installed only for executable releases (dynamically editingpyproject.toml
pre-build), this is clunky to say the least.Solution: While the previous works, having an option to bundle multiple local wheels like in
PYAPP_PROJECT_PATH="/A.whl;/B.whl;C.whl.."
would solve this issue, just list all wheels out ofuv build --wheel --all
and do a singlepip install *.whl
in pyapp when bootstrapping the pyapp project.The effects and possibilities of this are:
uv tree
, we pretty much created an easier method for making offline distributionsuv
upstreams and promotes it a lot:main-lib[all]
, as the subprojects defines optional deps of the bundledmain-lib
The important thing is to pip install all bundled wheels in a single go, as individual wheels would look for pypi for deps not already installed! So joining
EXTRA_WHEELS
andPROJECT_PATH
could do it, we don't need to modifyPROJECT_PATH
, keeping current compatibility of how it works in essence.Thanks a lot for the work and attention, I hope you consider adding these mechanisms as they'd help me a lot ✨
You can base-off the work in my proof of concept no problem if you do it!
Beta Was this translation helpful? Give feedback.
All reactions