Argo data submission [email protected]

[edsu7]

Addressing the following items:

• Update the song/score URLs in configuration file for RDPC Prod/QA/Dev cutover to Cumulus #92
  • changed song and score URLs from collab to cumulus
  • updated score

### Test QA
nextflow run ../main.nf -params-file local-test-job-5-InputBam.json --skip_duplicate_check -profile rdpc_qa,docker --score_mem 4 --score_cpus 4 --api_token ${QA_TOKEN}

### Test prod
nextflow run ../main.nf \
--skip_duplicate_check \
-profile rdpc,docker \
--score_mem 4 --score_cpus 4 \
--api_token ${PROD_TOKEN} \
--study_id TEST-CA \
--experiment_info_tsv input/prod_experiment-fq.v2.tsv \
--read_group_info_tsv input/prod_read_group-fq.v2.tsv \
--file_info_tsv input/prod_file-fq.v2.tsv \
--download_mode local \
--publish_dir outdir

• Add non-committal run #86

nextflow run ../main.nf -params-file local-test-job-5-InputBam.json --skip_duplicate_check -profile rdpc_qa,docker --score_mem 4 --score_cpus 4 --api_token ${QA_TOKEN} --dry_run true

• Force mandatory usage of profiles #87

% nextflow run ../main.nf -params-file local-test-job-5-InputBam.json --skip_duplicate_check -profile rdpc_qa,docker --score_mem 4 --score_cpus 4
#N E X T F L O W  ~  version 22.10.6
#Launching `../main.nf` [peaceful_hodgkin] DSL2 - revision: d1e7ce6cd4
#WARN: Access to undefined parameter `api_download_token` -- Initialise it to a default value eg. `params.api_download_token = some_value`
#Error SONG parameters detected but missing token params. `--api_token` or `api_upload_token` and `api_download_token` must be supplied when uploading.

% nextflow run ../main.nf -params-file local-test-job-5-InputBam.json --skip_duplicate_check --api_token NNNNN --score_mem 4 --score_cpus 4
#N E X T F L O W  ~  version 22.10.6
#Launching `../main.nf` [loving_hamilton] DSL2 - revision: d1e7ce6cd4
#Error Missing profile. `-profile` must be specified with the engines :`docker` or `singularity`.


% nextflow run ../main.nf -params-file local-test-job-5-InputBam.json --skip_duplicate_check -profile docker --api_token NNNNN --score_mem 4 --score_cpus 4
#N E X T F L O W  ~  version 22.10.6
#Launching `../main.nf` [happy_pesquet] DSL2 - revision: d1e7ce6cd4
#Error Missing profile. `-profile` must be specified with the rdpc environments : `rdpc_qa`,`rdpc_dev`, or `rdpc`.

[lindaxiang]

@edsu7 can we bump the version to 1.1.0 as this version contains the url changes to backend infrastructure which are breaking changes.

[edsu7]

For WFPM, would it be easier to start a new branch as 1.1.0?

[lindaxiang]

for sure, that's the only way I guess.

[lindaxiang]

We mentioned using -preview get a dry run in docs: https://docs.icgc-argo.org/docs/submission/submitting-molecular-data#quick-start
Can we use skip_upload instead here to differentiate them?

[lindaxiang]

When we run the following command:
nextflow run -preview main.nf -params-file example-params.local.json
it will require us to set profile and provide api_token as well.
Any thoughts we can skip doing this if we run with -preview?

[edsu7]

@lindaxiang start new pull request in #94.
See changes there. Will close this one once 1.1.0 is approved.