Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Acquire a reference count for manifests being read. #463

Open
nfachan opened this issue Feb 6, 2025 · 0 comments
Open

Acquire a reference count for manifests being read. #463

nfachan opened this issue Feb 6, 2025 · 0 comments

Comments

@nfachan
Copy link
Collaborator

nfachan commented Feb 6, 2025

The way it works now is:

  • The ArtifactGatherer will start a manifest read on behalf of any job requests it gets, but only after the manifest has been read into cache.
  • If there are multiple clients that all submit jobs using the same manifest simultaneously, they will each attempt to download the manifest independently. However, once the first one succeeds and asks to read the manifest, the manifest read will be shared by any other requests that come in.
  • There is currently no way to cancel a manifest read.
  • So, it's possible that a client may disconnect while a manifest read is outstanding. In this case, there will be no reference count held in the cache for the manifest, and in theory, the cache may remove the manifest while it's being read.
@nfachan nfachan added this to the 0.13.0 milestone Feb 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant