Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve the file release process #746

Open
5 tasks
rebkwok opened this issue Jan 20, 2025 · 0 comments
Open
5 tasks

Improve the file release process #746

rebkwok opened this issue Jan 20, 2025 · 0 comments
Assignees

Comments

@rebkwok
Copy link
Contributor

rebkwok commented Jan 20, 2025

Files are released to job-server by

  1. Creating a job-server release with a list of requested files (created the File instance, but doesn't actually upload it)
  2. Calling the job-server upload endpoint for each individual file

We have now updated the job-server releases API so that re-uploads are not blocked. This has (mostly) fixed the issues seen in this thread. However, this release contains 134 files. After the re-upload fixes, some of those files were uploaded on another attempt to release from airlock. It then seems to have encountered an unhandled error which has stopped the last 12 or so files being uploaded. The only solution is to try again to release from Airlock.

There are a few things that would make this experience better:

  1. Have job-server return a non-error message if a file has already been uploaded. Currently it's returning 400s which Airlock catches, but are misleading when looking for problems in honeycomb
    2) Check the response from job-server and record on the Airlock ReleaseFileMetadata whether the file has been uploaded or not. This means we can avoid sending every already-uploaded file to job-server again if we need to re-attempt a release
    3) Catch errors from uploading files so we can retry and/or continue to reupload the remaining files
    4) Report something better than "Release failed" in Airlock (if we do Automate deployment to test backend #3 we should be able to report X of Y files were released)

We also have #468 for releasing files asynchronously

#765 will fix #468, which in turn addresses most of points 2-4 above. Still remaining:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant