Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update docs to reflect the asynchronous uploads #779

Merged
merged 5 commits into from
Feb 14, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion docs/explanation/notifications.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,13 @@ Notifications are sent at various stags during the release request workflow. Not
- GitHub issue closed
- Slack notification to output checkers

## Files released
## Request approved for release

- GitHub issue updated
- Slack notification to output checkers
- Author notified by email

## Request released (all files successsully uploaded)

- GitHub issue closed
- Author notified by email
14 changes: 10 additions & 4 deletions docs/explanation/workflow-and-permissions.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,9 +41,14 @@ SUBMITTED. It is owned by the output checkers. The researcher can no longer add
withdraw files, or comment on file groups. It is now the turn of the output checkers
to work on the request, reviewing files and asking questions where necessary.

A release request moves into APPROVED status when an output checker has clicked the
button to release the files. At this stage, the files have been scheduled for release
and upload to the jobs site. Once all files have been uploaded, the request will
automatically move to the RELEASED status.

The RELEASED, REJECTED and WITHDRAWN statuses are considered final states. When a
release request is in one of these statuses, it cannot be edited by any user, and
it cannot be moved into any other status.
it cannot be moved into any other status.

```mermaid
flowchart TD
Expand All @@ -62,10 +67,11 @@ flowchart TD
C -- Return prior to full review --> E
D -- Consolidation --> E
E -- Updates and responds to questions --> B
D -- All files approved ----> F(RELEASED)
D -- All files approved ----> F(APPROVED)
D -- Not approved ----> G(REJECTED)
A --> H(WITHDRAWN)
E --> H
F(APPROVED) -- Files uploaded ----> H(RELEASED)
A --> I(WITHDRAWN)
E --> I
end
madwort marked this conversation as resolved.
Show resolved Hide resolved

classDef author fill:#ff9,stroke:#333;
Expand Down
19 changes: 16 additions & 3 deletions docs/how-tos/release-files.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,23 @@ independent review has been completed and the files can be released.

Click the "Release Files" button to start the release process.

The release request [transitions to the "Approved" state](../reference/request-states.md). Once all files have been uploaded to the jobs site, the
release request will move to the "Released" state.
The release request [transitions to the "Approved" status](../reference/request-states.md). In
the background, the files will be uploaded to the jobs site. While this happens, you will see
a count of uploaded files on the request overview page, and an in-progress spinner icon that
indicates the files are still uploading.

![Files released](../screenshots/files_released.png)
![Request approved, upload in progress](../screenshots/request_approved_upload_in_progress.png)

Airlock will attempt to upload each output file a limited number of times. If all file
uploads have been attempted, and some files have failed to release, you will see a failed
icon next to the uploaded file count. The "Re-release files" button is now enabled, and you can attempt to re-release. This will re-try the uploads for any files that have failed.

![Request approved, upload failed](../screenshots/request_approved_upload_failed.png)


Once all files have been uploaded to the jobs site, the release request will [move to the "Released" status](../reference/request-states.md).

![Request released and all files uploaded](../screenshots/request_released.png)


!!!info "Files excluded from release"
Expand Down
Binary file modified docs/screenshots/add_file_button.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/screenshots/context_and_controls.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/screenshots/context_modal.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/screenshots/file_approved.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/screenshots/file_group.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/screenshots/file_review.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/screenshots/file_update.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/screenshots/file_update_modal.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed docs/screenshots/files_released.png
Binary file not shown.
Binary file modified docs/screenshots/markdown_comment_blockquote_1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/screenshots/markdown_comment_blockquote_2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/screenshots/markdown_comment_bold_1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/screenshots/markdown_comment_bold_2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/screenshots/markdown_comment_italics_2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/screenshots/markdown_comment_ordered_list_2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/screenshots/multiselect_update.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/screenshots/ready_to_release.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/screenshots/request_independent_review_file_icons.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/screenshots/request_overview.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/screenshots/request_released.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/screenshots/request_reviewed_file_icons.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/screenshots/request_tree.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/screenshots/request_tree_post_voting.png
Binary file modified docs/screenshots/returned_request_comments.png
Binary file modified docs/screenshots/returned_tree.png
Binary file modified docs/screenshots/reviewed_request_comments.png
Binary file modified docs/screenshots/submit_request.png
Binary file modified docs/screenshots/submit_review.png
Binary file modified docs/screenshots/submitted_request.png
Binary file modified docs/screenshots/submitted_review.png
Binary file modified docs/screenshots/withdraw_request.png
Binary file modified docs/screenshots/withdraw_request_modal.png
Binary file modified docs/screenshots/withdrawn_file.png
Binary file modified docs/screenshots/workspace_directory_content.png
Binary file modified docs/screenshots/workspace_directory_view.png
45 changes: 43 additions & 2 deletions tests/functional/test_docs_screenshots.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,18 @@ def get_user_data():
return author, user_dicts


def move_mouse_from(page):
"""
Move the mouse away from any elements it might be hovering over so that we
don't screenshot tooltips.
"""
# Note there's a locator.dispatch_event('mouseleave') method, which does move the
# pointer off the button, but it doesn't get rid of the tooltip until you move the
# mouse again. Moving it to 0, 0 ensures that it doesn't accidentally land on some
# other element that has a tooltip
page.mouse.move(0, 0)


@pytest.mark.skipif(
os.getenv("RUN_SCREENSHOT_TESTS") is None,
reason="screenshot tests skipped; set RUN_SCREENSHOT_TESTS env variable",
Expand Down Expand Up @@ -239,7 +251,10 @@ def do_review(screenshot=True):
page.screenshot(path=settings.SCREENSHOT_DIR / "submit_review.png")

page.locator("#submit-review-button").click()

if screenshot:
# move mouse off button to avoid screenshotting tooltips
move_mouse_from(page)
page.screenshot(path=settings.SCREENSHOT_DIR / "submitted_review.png")

# Login as output checker and visit pages
Expand Down Expand Up @@ -350,13 +365,39 @@ def do_review_and_approve(username):
release_files_stubber(release_request)
page.goto(live_server.url + release_request.get_url())

# Move the mouse off the button so we don't screenshot the tooltip
move_mouse_from(page)
page.screenshot(path=settings.SCREENSHOT_DIR / "ready_to_release.png")

# Approve for release
page.locator("#release-files-button").click()
# Make sure we've waited for the files to be released

expect(page.locator("body")).to_contain_text(
"Files have been released and will be uploaded to jobs.opensafely.org"
)
page.screenshot(path=settings.SCREENSHOT_DIR / "files_released.png")
# Move the mouse off the button so we don't screenshot the tooltip
move_mouse_from(page)
page.screenshot(
path=settings.SCREENSHOT_DIR / "request_approved_upload_in_progress.png"
)

# Progress the release request to all uploads failed
for relpath in release_request.output_files():
for _ in range(settings.UPLOAD_MAX_ATTEMPTS):
bll.register_file_upload_attempt(release_request, relpath)

page.goto(live_server.url + release_request.get_url())
page.screenshot(path=settings.SCREENSHOT_DIR / "request_approved_upload_failed.png")

# Progress the release request to all uploads complete and released
release_request = factories.refresh_release_request(release_request)
checker_user = factories.create_user(**user_dicts["checker1"])
for relpath in release_request.output_files():
for _ in range(settings.UPLOAD_MAX_ATTEMPTS):
bll.register_file_upload(release_request, relpath, checker_user)
bll.set_status(release_request, RequestStatus.RELEASED, checker_user)
page.goto(live_server.url + release_request.get_url())
page.screenshot(path=settings.SCREENSHOT_DIR / "request_released.png")


@pytest.mark.skipif(
Expand Down