Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: set intermediate and final output files #129

Merged
merged 23 commits into from
Dec 8, 2023

Conversation

deliaBlue
Copy link
Collaborator

@deliaBlue deliaBlue commented Dec 2, 2023

Fixes #86

  • add a directory to store the intermediate files in config_template.yaml and config_schema.json
  • add the intermediate files path in all the snakemake files
  • add the onfailure and onsuccess handlers in the main Snakefile so intermediate files are not lost if the workflow fails and removed otherwise.
  • redirect the intermediate files to the new directory in the sub-workflows
  • add the section "Expected output files" in the main README.md with a brief explanation of the files that are kept by default and how to keep the intermediate ones
  • update the finish rules in pipeline_documentaion.md
  • update the rule graph
  • add the CLI option --no-hooks to all the tests in order to keep checking all the files in the CI tests
  • update the expected output to match the new location of the intermediate files

@deliaBlue deliaBlue self-assigned this Dec 2, 2023
@deliaBlue deliaBlue linked an issue Dec 2, 2023 that may be closed by this pull request
@deliaBlue deliaBlue changed the title build: clean/order final output files in snakefile feat: set intermediate and final output files Dec 2, 2023
@deliaBlue deliaBlue requested a review from uniqueg December 2, 2023 19:08
uniqueg
uniqueg previously approved these changes Dec 3, 2023
Copy link
Member

@uniqueg uniqueg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good work :) Couple of comments, but shouldn't be all that much

README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Show resolved Hide resolved
config/config_schema.json Outdated Show resolved Hide resolved
config/config_schema.json Outdated Show resolved Hide resolved
workflow/Snakefile Show resolved Hide resolved
workflow/Snakefile Outdated Show resolved Hide resolved
uniqueg
uniqueg previously approved these changes Dec 7, 2023
Copy link
Member

@uniqueg uniqueg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll approve this now. You can see how/whether you address (or not) the two open conversations and then merge.

@deliaBlue deliaBlue merged commit 57c6710 into dev Dec 8, 2023
8 checks passed
@deliaBlue deliaBlue deleted the 86-cleanorder-final-output-files-in-snakefile branch December 8, 2023 15:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Clean/order final output files in Snakefile
2 participants