Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix distribution embeds and update docs #80

Merged
merged 3 commits into from
Feb 29, 2024
Merged

Conversation

nmay231
Copy link
Contributor

@nmay231 nmay231 commented Jan 22, 2024

Closes #79

I updated the docs with what would have (hopefully) prevented any of my confusion.

@nmay231

This comment was marked as resolved.

@nmay231 nmay231 marked this pull request as draft January 23, 2024 04:29
@nmay231

This comment was marked as resolved.

@nmay231 nmay231 marked this pull request as ready for review January 24, 2024 06:37
@ofek
Copy link
Owner

ofek commented Jan 24, 2024

I don't have time to review this today but can you please make sure the change is correct? I'm almost positive that the empty string is required to indicate that there is no source at runtime.

@nmay231
Copy link
Contributor Author

nmay231 commented Jan 24, 2024

Certainly!

During runtime, there is no check at all; it blindly tries downloading the source if no embed exists. During build-time, the behavior is identical to providing PYAPP_DISTRIBUTION_SOURCE and PYAPP_DISTRIBUTION_EMBED except that PYAPP_DISTRIBUTION_PIP_AVAILABLE gets set to true; that's a simple fix though. There is no other .is_empty() check that changes behavior.

To clarify what I should do, I need to know how you intend PYAPP_DISTRIBUTION_PATH to be used. Do you intend *_PATH to only be one of the standalone (or pypy) builds, or do you expect someone to zip their local python builds and package that? My personal usecase was to cache the download of the build(s) to allow faster iteration.

In any case, I think we should deprecate *_PATH and allow *_SOURCE to be a file. Then any branching on embed vs download would check if *_SOURCE is a url.

Sidenote: I remember why I got the error I was getting earlier.

Error: download failed: ../dist/python.tar.gz

I was curling the tar file without -L and getting an empty file because of url redirects. distribution.rs:materialize then assumed since there was no embedding (empty file) that it has to download from a url.

@nmay231
Copy link
Contributor Author

nmay231 commented Feb 10, 2024

I updated to fix the issue with PYAPP_DISTRIBUTION_PIP_AVAILABLE.

The only issues that remain are:

  • Do you want to deprecate PYAPP_DISTRIBUTION_PATH? (the reason why is explained above)
  • How would we automatically test this and future changes?

You have the test.yml workflow that runs cowsay on a matrix of OSs and architectures, but that's obvisouly not enough on its own since this bug went unnoticed.

I could probably add tests, but I'm unfortunately no longer using pyapp for the project I'm working on right now so I'm less motivated to work on this contribution. However, I'll try not to leave this issue unresolved.

@ofek
Copy link
Owner

ofek commented Feb 29, 2024

Thanks for your patience here! I'm about to push a change and merge this.

  • Yes the option is intended for any prebuilt distribution as long as it's compressed with a known format.
  • Currently, I don't want to deprecate the option because I like the explicitness of options for URLs vs paths. I'm open to changing my mind however.
  • Testing is a good question! I honestly haven't had time to write proper tests so no worries, my bad.
  • I'm going to remove the pip available change because it's possible that custom distributions that folks provide do not have pip already installed. I will update the documentation to make it more explicit that it would be a performance benefit in that case.

Copy link
Owner

@ofek ofek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much!

@ofek ofek merged commit 2eba793 into ofek:master Feb 29, 2024
12 checks passed
github-actions bot pushed a commit that referenced this pull request Feb 29, 2024
@ofek
Copy link
Owner

ofek commented Mar 1, 2024

FYI I see what was happening and the pip available conditional you had there was indeed necessary so I added that back. I need more sleep... and to write tests lol

@nmay231
Copy link
Contributor Author

nmay231 commented Mar 1, 2024

Lol, we all could probably use a bit more sleep :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cannot use PYAPP_DISTRIBUTION_PATH (at least with locally embedded projects)
2 participants