Remove internal workflows and scripts #742
Conversation
Delete a few more files; update vapor Move more stuff Move out igv docker, ApplyManualVariantFilter, and some unused input templates
mwalker174
force-pushed
the
mw_release_cleanup
branch
from
8588022 to
353c384
Compare
There was a problem hiding this comment.
This looks generally good. I double-checked all the WDLs for remaining dependencies and they were fine.
My biggest question is how we want to maintain utility workflows in the internal repo that depend on tasks in the main repo - ex. ConcatTextFiles and SubsetVcfBySamples rely on TasksMakeCohortVcf and Utils respectively, and as such are currently broken in gatk-sv-internal. Is it worthwhile to copy the tasks WDLs over, or to create an InternalTasks.wdl to copy specific tasks? I don't like to duplicate code but with two repos I think there isn't much of an alternative.
| @@ -1,7 +0,0 @@ | |||
| { | |||
There was a problem hiding this comment.
It looks like none of the JSON templates got moved to the new repo. I'm not sure any of the current ones are super necessary to keep around, but we might want the ability/setup to store and validate them in the future
There was a problem hiding this comment.
Right, I did move example jsons over but not templates because they could get outdated. I think if we ever wanted to restore the templates we would just go back in the git history here instead.
| @@ -1,313 +0,0 @@ | |||
| #!/bin/python | |||
There was a problem hiding this comment.
We have shared these scripts with other groups and have links to them in the onboarding doc & slides. We'll need to update our onboarding materials and we may want to put a note in the README or somewhere to check the internal repo
There was a problem hiding this comment.
Yes we should update internal onboarding documentation. gatk-sv-interrnal is a private repo though, so we should not point users to it.
There was a problem hiding this comment.
I updated the link in the onboarding doc
There was a problem hiding this comment.
Oh I missed that it's private. I think it needs to be public so we can use it with dockstore
There was a problem hiding this comment.
My mistake, it is public. I still wouldn't want to refer users there since some code may not be maintained
| @@ -1,117 +0,0 @@ | |||
| #!/bin/python | |||
There was a problem hiding this comment.
I noticed you're only keeping the v2 scripts in the new internal repo - in which case this one can be deleted as get_cromwell_resource_usage2.sh renders it obsolete
There was a problem hiding this comment.
Obsolete or redundant? It seems like they do similar things but it might be a good idea to have a python implementation available.
| @@ -1,132 +0,0 @@ | |||
| #!/bin/python | |||
There was a problem hiding this comment.
This script is referred to in website/docs/advanced/build_ref_panel.md - should update docs accordingly
There was a problem hiding this comment.
Yes okay, I am deleting the documentation on another branch mw_ss_mode_v1, which replaces reference panel generation with a Terra method, and copied it into the script header over in the other repo.
| @@ -1,43 +0,0 @@ | |||
| version 1.0 | |||
|
|
|||
| import "TasksMakeCohortVcf.wdl" as tasks | |||
There was a problem hiding this comment.
What's the plan for utility workflows like this that we may want to use again, but rely on WDLs in the gatk-sv repo that haven't been moved to gatk-sv-internal? ie. here ConcatTextFiles needs TasksMakeCohortVcf.wdl - but we may not want to maintain two versions
There was a problem hiding this comment.
Good point, I will copy those dependencies over into gatk-sv-internal since it's possible that they could get desynced with the dependent WDLs over time. I'm not worried about having a little redundant code over there if they are just a few core utility tasks.
| @@ -1,159 +0,0 @@ | |||
| version 1.0 | |||
There was a problem hiding this comment.
Mentioned in website/docs/gs/input_files.md so should update docs
There was a problem hiding this comment.
Done. I think that the ID generating script is critical so I have moved it into /scripts/inputs and updated the link in the doc.
Moves workflows and scripts considered legacy code or that are otherwise not necessary to run gatk-sv. All of these files have been relocated to gatk-sv-internal.
Two WDLs had to be modified:
Some workflows may still be in regular use. They will still be available through Dockstore in previous versions.
We may also consider moving str related code out as well in a future PR.