diff --git a/docs/data-models/clinicalanalysis.md b/docs/data-models/clinicalanalysis.md index cf753ada1a0..caea2bd3365 100644 --- a/docs/data-models/clinicalanalysis.md +++ b/docs/data-models/clinicalanalysis.md @@ -69,63 +69,33 @@ You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1 | **attributes**
*Map*
|

You can use this field to store any other information, keep in mind this is not indexed so you cannot search by attributes.

| | **status**
*Status*
|

Object status.

| -### ClinicalPriorityAnnotation -You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/study/configuration/ClinicalPriorityAnnotation.java). +### ClinicalAudit +You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/clinical/ClinicalAudit.java). | Field | Description | | :--- | :--- | -| **id**
*String*
|

Object ID is a mandatory parameter when creating a new one, this ID cannot be changed at the moment.

| -| **description**
*String*
|

Users may provide a description for the entry.

| -| **rank**
*int*
|

ClinicalPriorityAnnotation rank.

| -| **date**
*String*
|

ClinicalPriorityAnnotation date.

| +| **author**
*String*
|

Audit author

| +| **action**
*Action*
|

Enum action that can have the values CREATE_CLINICAL_ANALYSIS, CREATE_INTERPRETATION, UPDATE_CLINICAL_ANALYSIS, DELETE_CLINICAL_ANALYSIS, UPDATE_INTERPRETATION, REVERT_INTERPRETATION, CLEAR_INTERPRETATION, MERGE_INTERPRETATION, SWAP_INTERPRETATION and DELETE_INTERPRETATION

| +| **message**
*String*
|

Audit message

| +| **date**
*String*
|

Date of the audit

| -### ClinicalReport -You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/clinical/ClinicalReport.java). +### ClinicalAnalysisQualityControl +You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/clinical/ClinicalAnalysisQualityControl.java). | Field | Description | | :--- | :--- | -| **title**
*String*
|

Report title.

| -| **overview**
*String*
|

Report overview.

| -| **discussion**
*String*
|

Report discussion.

| -| **logo**
*String*
|

Report logo.

| -| **signedBy**
*String*
|

Indicates who has signed the report.

| -| **signature**
*String*
|

Report signature.

| -| **date**
*String*
|

Report date.

| +| **summary**
*QualityControlSummary*
|

ClinicalAnalysisQualityControl summary that can have the values HIGH, MEDIUM, LOW, DISCARD, NEEDS_REVIEW, UNKNOWN.

| +| **comments**
*List<ClinicalComment>*
|

List of ClinicalAnalysisQualityControl comments.

| +| **comments**
*List<String>*
|

List of ClinicalAnalysisQualityControl files.

| -### Disorder -You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/clinical/Disorder.java). +### FlagAnnotation +You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/common/FlagAnnotation.java). | Field | Description | | :--- | :--- | -| **id**
*String*
|

Id to identify the object

| -| **name**
*String*
|

Object name

| +| **id**
*String*
|

Object ID is a mandatory parameter when creating a new one, this ID cannot be changed at the moment.

| | **description**
*String*
|

Users may provide a description for the entry.

| -| **source**
*String*
|

Ontology source

| -| **url**
*String*
|

Ontology url

| -| **attributes**
*Map*
|

Dictionary that can be customised by users to store any additional information users may require..

| - -### Panel -You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/panel/Panel.java). - -| Field | Description | -| :--- | :--- | -| **uuid**
*String*
|

Unique 32-character identifier assigned automatically by OpenCGA.

| -| **release**
*int*
|

An integer describing the current data release.

| -| **version**
*int*
|

OpenCGA version of this panel, this is incremented when the panel is updated.

| -| **~~author~~**
*String*

_Deprecated_ |

Author of the panel.

| -| **status**
*[Status](https://docs.opencga.opencb.org/data-models/clinicalanalysis#status)*
|

Panel status can have the values READY or DELETED.

| -| **studyUid**
*long*
|

Panel reference to study.

| -| **uid**
*long*
|

Panel reference to study.

| - -### ClinicalComment -You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/clinical/ClinicalComment.java). - -| Field | Description | -| :--- | :--- | -| **author**
*String*
|

Clinical comment author

| -| **message**
*String*
|

Clinical comment message

| -| **tags**
*List<String>*
|

List of tags for the clinical comment

| -| **date**
*String*
|

Date of the clinical comment

| +| **date**
*String*
|

FlagAnnotation date.

| ### Interpretation You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/clinical/Interpretation.java). @@ -138,15 +108,6 @@ You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1 | **internal**
*[InterpretationInternal](https://docs.opencga.opencb.org/data-models/clinicalanalysis#interpretationinternal)*
|

Internal field for manage the object.

| | **release**
*int*
|

An integer describing the current data release.

| -### ClinicalAnalysisQualityControl -You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/clinical/ClinicalAnalysisQualityControl.java). - -| Field | Description | -| :--- | :--- | -| **summary**
*QualityControlSummary*
|

ClinicalAnalysisQualityControl summary that can have the values HIGH, MEDIUM, LOW, DISCARD, NEEDS_REVIEW, UNKNOWN.

| -| **comments**
*List<ClinicalComment>*
|

List of ClinicalAnalysisQualityControl comments.

| -| **comments**
*List<String>*
|

List of ClinicalAnalysisQualityControl files.

| - ### ClinicalConsentAnnotation You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/study/configuration/ClinicalConsentAnnotation.java). @@ -175,34 +136,63 @@ You can find the Java code [here](https://github.com/opencb/biodata/tree/develop | **assignedBy**
*String*
|

Assigned by field

| | **date**
*String*
|

Date of the clinical analyst

| -### FlagAnnotation -You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/common/FlagAnnotation.java). +### Disorder +You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/clinical/Disorder.java). + +| Field | Description | +| :--- | :--- | +| **id**
*String*
|

Id to identify the object

| +| **name**
*String*
|

Object name

| +| **description**
*String*
|

Users may provide a description for the entry.

| +| **source**
*String*
|

Ontology source

| +| **url**
*String*
|

Ontology url

| +| **attributes**
*Map*
|

Dictionary that can be customised by users to store any additional information users may require..

| + +### Panel +You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/panel/Panel.java). + +| Field | Description | +| :--- | :--- | +| **uuid**
*String*
|

Unique 32-character identifier assigned automatically by OpenCGA.

| +| **release**
*int*
|

An integer describing the current data release.

| +| **version**
*int*
|

OpenCGA version of this panel, this is incremented when the panel is updated.

| +| **~~author~~**
*String*

_Deprecated_ |

Author of the panel.

| +| **status**
*[Status](https://docs.opencga.opencb.org/data-models/clinicalanalysis#status)*
|

Panel status can have the values READY or DELETED.

| +| **studyUid**
*long*
|

Panel reference to study.

| +| **uid**
*long*
|

Panel reference to study.

| + +### ClinicalPriorityAnnotation +You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/study/configuration/ClinicalPriorityAnnotation.java). | Field | Description | | :--- | :--- | | **id**
*String*
|

Object ID is a mandatory parameter when creating a new one, this ID cannot be changed at the moment.

| | **description**
*String*
|

Users may provide a description for the entry.

| -| **date**
*String*
|

FlagAnnotation date.

| +| **rank**
*int*
|

ClinicalPriorityAnnotation rank.

| +| **date**
*String*
|

ClinicalPriorityAnnotation date.

| -### ClinicalAudit -You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/clinical/ClinicalAudit.java). +### ClinicalComment +You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/clinical/ClinicalComment.java). | Field | Description | | :--- | :--- | -| **author**
*String*
|

Audit author

| -| **action**
*Action*
|

Enum action that can have the values CREATE_CLINICAL_ANALYSIS, CREATE_INTERPRETATION, UPDATE_CLINICAL_ANALYSIS, DELETE_CLINICAL_ANALYSIS, UPDATE_INTERPRETATION, REVERT_INTERPRETATION, CLEAR_INTERPRETATION, MERGE_INTERPRETATION, SWAP_INTERPRETATION and DELETE_INTERPRETATION

| -| **message**
*String*
|

Audit message

| -| **date**
*String*
|

Date of the audit

| +| **author**
*String*
|

Clinical comment author

| +| **message**
*String*
|

Clinical comment message

| +| **tags**
*List<String>*
|

List of tags for the clinical comment

| +| **date**
*String*
|

Date of the clinical comment

| -### InterpretationInternal -You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/clinical/InterpretationInternal.java). +### ClinicalReport +You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/clinical/ClinicalReport.java). | Field | Description | | :--- | :--- | -| **status**
*[InterpretationStatus](https://docs.opencga.opencb.org/data-models/clinicalanalysis#interpretationstatus)*
|

State of the interpretation that can have the values READY, DELETED, NOT_REVIEWED, UNDER_REVIEW, REVIEWED and REJECTED.

| -| **status**
*[Status](https://docs.opencga.opencb.org/data-models/clinicalanalysis#status)*
|

Status of the internal object.

| -| **registrationDate**
*String*
|

Registration date of the internal object.

| -| **lastModified**
*String*
|

Date of the last modification of the internal object.

| +| **title**
*String*
|

Report title.

| +| **overview**
*String*
|

Report overview.

| +| **discussion**
*String*
|

Report discussion.

| +| **logo**
*String*
|

Report logo.

| +| **signedBy**
*String*
|

Indicates who has signed the report.

| +| **signature**
*String*
|

Report signature.

| +| **date**
*String*
|

Report date.

| ### ClinicalConsentParam You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/study/configuration/ClinicalConsentParam.java). @@ -218,6 +208,16 @@ You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1 You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/common/Status.java). +### InterpretationInternal +You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/clinical/InterpretationInternal.java). + +| Field | Description | +| :--- | :--- | +| **status**
*[InterpretationStatus](https://docs.opencga.opencb.org/data-models/clinicalanalysis#interpretationstatus)*
|

State of the interpretation that can have the values READY, DELETED, NOT_REVIEWED, UNDER_REVIEW, REVIEWED and REJECTED.

| +| **status**
*[Status](https://docs.opencga.opencb.org/data-models/clinicalanalysis#status)*
|

Status of the internal object.

| +| **registrationDate**
*String*
|

Registration date of the internal object.

| +| **lastModified**
*String*
|

Date of the last modification of the internal object.

| + ### InterpretationStatus You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/clinical/InterpretationStatus.java). diff --git a/docs/data-models/cohort.md b/docs/data-models/cohort.md index 020b345bc1e..c1f1f554ac8 100644 --- a/docs/data-models/cohort.md +++ b/docs/data-models/cohort.md @@ -37,10 +37,6 @@ You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1 | **internal**
*[CohortInternal](https://docs.opencga.opencb.org/data-models/cohort#cohortinternal)*
|

Internal field for manage the object.

| | **attributes**
*Map*
|

You can use this field to store any other information, keep in mind this is not indexed so you cannot search by attributes.

| -### Status -You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/common/Status.java). - - ### CohortInternal You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/cohort/CohortInternal.java). @@ -49,3 +45,7 @@ You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1 | **status**
*[Status](https://docs.opencga.opencb.org/data-models/cohort#status)*
|

Status of the internal object.

| | **registrationDate**
*String*
|

Registration date of the internal object.

| | **lastModified**
*String*
|

Date of the last modification of the internal object.

| + +### Status +You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/common/Status.java). + diff --git a/docs/data-models/family.md b/docs/data-models/family.md index 92f60a9091c..c35d327ff1c 100644 --- a/docs/data-models/family.md +++ b/docs/data-models/family.md @@ -47,18 +47,17 @@ You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1 | **release**
*int*
|

An integer describing the current data release.

| | **version**
*int*
|

Autoincremental version assigned to the registered entry. By default, updates does not create new versions. To enable versioning, users must set the `incVersion` flag from the /update web service when updating the document.

| -### FamilyQualityControl -You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/family/FamilyQualityControl.java). +### Disorder +You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/clinical/Disorder.java). | Field | Description | | :--- | :--- | -| **relatedness**
*List<RelatednessReport>*
|

Reports of family relationship.

| -| **files**
*List<String>*
|

File IDs related to the quality control.

| -| **comments**
*List<ClinicalComment>*
|

Comments related to the quality control.

| - -### Status -You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/common/Status.java). - +| **id**
*String*
|

Id to identify the object

| +| **name**
*String*
|

Object name

| +| **description**
*String*
|

Users may provide a description for the entry.

| +| **source**
*String*
|

Ontology source

| +| **url**
*String*
|

Ontology url

| +| **attributes**
*Map*
|

Dictionary that can be customised by users to store any additional information users may require..

| ### FamilyInternal You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/family/FamilyInternal.java). @@ -69,6 +68,19 @@ You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1 | **registrationDate**
*String*
|

Registration date of the internal object.

| | **lastModified**
*String*
|

Date of the last modification of the internal object.

| +### Status +You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/common/Status.java). + + +### FamilyQualityControl +You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/family/FamilyQualityControl.java). + +| Field | Description | +| :--- | :--- | +| **relatedness**
*List<RelatednessReport>*
|

Reports of family relationship.

| +| **files**
*List<String>*
|

File IDs related to the quality control.

| +| **comments**
*List<ClinicalComment>*
|

Comments related to the quality control.

| + ### Phenotype You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/clinical/Phenotype.java). @@ -83,17 +95,15 @@ You can find the Java code [here](https://github.com/opencb/biodata/tree/develop | **url**
*String*
|

Ontology url

| | **attributes**
*Map*
|

Dictionary that can be customised by users to store any additional information users may require..

| -### Disorder -You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/clinical/Disorder.java). +### RelatednessReport +You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/clinical/qc/RelatednessReport.java). | Field | Description | | :--- | :--- | -| **id**
*String*
|

Id to identify the object

| -| **name**
*String*
|

Object name

| -| **description**
*String*
|

Users may provide a description for the entry.

| -| **source**
*String*
|

Ontology source

| -| **url**
*String*
|

Ontology url

| -| **attributes**
*Map*
|

Dictionary that can be customised by users to store any additional information users may require..

| +| **method**
*String*
|

Method of the relatedness report

| +| **maf**
*String*
|

Minor allele frequency to filter variants, e.g.: 1kg_phase3:CEU>0.35, cohort:ALL>0.05

| +| **scores**
*List<RelatednessScore>*
|

Relatedness scores for pair of samples

| +| **files**
*List<String>*
|

List of files of Relatedness Report

| ### ClinicalComment You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/clinical/ClinicalComment.java). @@ -104,13 +114,3 @@ You can find the Java code [here](https://github.com/opencb/biodata/tree/develop | **message**
*String*
|

Clinical comment message

| | **tags**
*List<String>*
|

List of tags for the clinical comment

| | **date**
*String*
|

Date of the clinical comment

| - -### RelatednessReport -You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/clinical/qc/RelatednessReport.java). - -| Field | Description | -| :--- | :--- | -| **method**
*String*
|

Method of the relatedness report

| -| **maf**
*String*
|

Minor allele frequency to filter variants, e.g.: 1kg_phase3:CEU>0.35, cohort:ALL>0.05

| -| **scores**
*List<RelatednessScore>*
|

Relatedness scores for pair of samples

| -| **files**
*List<String>*
|

List of files of Relatedness Report

| diff --git a/docs/data-models/file.md b/docs/data-models/file.md index 48477ff4adc..dc43068bfc6 100644 --- a/docs/data-models/file.md +++ b/docs/data-models/file.md @@ -65,8 +65,8 @@ You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1 | **internal**
*[FileInternal](https://docs.opencga.opencb.org/data-models/file#fileinternal)*
|

Internal field for manage the object.

| | **attributes**
*Map*
|

You can use this field to store any other information, keep in mind this is not indexed so you cannot search by attributes.

| -### FileQualityControl -You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/file/FileQualityControl.java). +### Status +You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/common/Status.java). ### Software @@ -81,6 +81,18 @@ You can find the Java code [here](https://github.com/opencb/biodata/tree/develop | **website**
*String*
|

Software website

| | **params**
*Map*
|

Software params

| +### FileExperiment +You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/file/FileExperiment.java). + + +### FileQualityControl +You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/file/FileQualityControl.java). + + +### FileRelatedFile +You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/file/FileRelatedFile.java). + + ### URI You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/java/net/URI.java). @@ -99,26 +111,6 @@ You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1 | **registrationDate**
*String*
|

Registration date of the internal object.

| | **lastModified**
*String*
|

Date of the last modification of the internal object.

| -### Status -You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/common/Status.java). - - -### FileExperiment -You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/file/FileExperiment.java). - - -### FileRelatedFile -You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/file/FileRelatedFile.java). - - -### FileInternalVariant -You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/file/FileInternalVariant.java). - - -### FileStatus -You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/file/FileStatus.java). - - ### MissingSamples You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/file/MissingSamples.java). @@ -127,6 +119,14 @@ You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1 | **existing**
*List<String>*
|

List of existing samples.

| | **nonExisting**
*List<String>*
|

List of non existing samples.

| +### FileInternalVariant +You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/file/FileInternalVariant.java). + + ### FileInternalAlignment You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/file/FileInternalAlignment.java). + +### FileStatus +You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/file/FileStatus.java). + diff --git a/docs/data-models/individual.md b/docs/data-models/individual.md index 0fd1ba2be78..7eaf3827b11 100644 --- a/docs/data-models/individual.md +++ b/docs/data-models/individual.md @@ -63,29 +63,42 @@ You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1 | **release**
*int*
|

An integer describing the current data release.

| | **version**
*int*
|

Autoincremental version assigned to the registered entry. By default, updates does not create new versions. To enable versioning, users must set the `incVersion` flag from the /update web service when updating the document.

| -### OntologyTermAnnotation -You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/core/OntologyTermAnnotation.java). +### Location +You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/individual/Location.java). + +| Field | Description | +| :--- | :--- | +| **address**
*String*
|

Location address.

| +| **postalCode**
*String*
|

Location postal code.

| +| **city**
*String*
|

Location city.

| +| **state**
*String*
|

Location state.

| +| **country**
*String*
|

Location country.

| + +### Phenotype +You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/clinical/Phenotype.java). | Field | Description | | :--- | :--- | | **id**
*String*
|

Id to identify the object

| +| **ageOfOnset**
*String*
|

Indicates the age of on set of the phenotype

| +| **status**
*Status*
|

Status of phenotype OBSERVED, NOT_OBSERVED, UNKNOWN

| | **name**
*String*
|

Object name

| | **description**
*String*
|

Users may provide a description for the entry.

| | **source**
*String*
|

Ontology source

| | **url**
*String*
|

Ontology url

| | **attributes**
*Map*
|

Dictionary that can be customised by users to store any additional information users may require..

| -### IndividualPopulation -You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/individual/IndividualPopulation.java). +### IndividualInternal +You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/individual/IndividualInternal.java). | Field | Description | | :--- | :--- | -| **name**
*String*
|

Name of the individual population.

| -| **subpopulation**
*String*
|

Subpopulation of the individual population.

| -| **description**
*String*
|

Description of the individual population.

| +| **status**
*[Status](https://docs.opencga.opencb.org/data-models/individual#status)*
|

Status of the internal object.

| +| **registrationDate**
*String*
|

Registration date of the internal object.

| +| **lastModified**
*String*
|

Date of the last modification of the internal object.

| -### SexOntologyTermAnnotation -You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/core/SexOntologyTermAnnotation.java). +### Disorder +You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/clinical/Disorder.java). | Field | Description | | :--- | :--- | @@ -96,33 +109,24 @@ You can find the Java code [here](https://github.com/opencb/biodata/tree/develop | **url**
*String*
|

Ontology url

| | **attributes**
*Map*
|

Dictionary that can be customised by users to store any additional information users may require..

| -### IndividualQualityControl -You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/individual/IndividualQualityControl.java). - -| Field | Description | -| :--- | :--- | -| **inferredSexReports**
*List<InferredSexReport>*
|

List of inferred sex reports, it depends on the method (currently by coverage ratio).

| -| **sampleRelatednessReport**
*[SampleRelatednessReport](https://docs.opencga.opencb.org/data-models/individual#samplerelatednessreport)*
|

Reports of samples relatedness.

| -| **mendelianErrorReports**
*List<MendelianErrorReport>*
|

Mendelian errors.

| -| **files**
*List<String>*
|

File IDs related to the quality control.

| -| **author**
*List<ClinicalComment>*
|

Comments related to the quality control.

| - -### Phenotype -You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/clinical/Phenotype.java). +### OntologyTermAnnotation +You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/core/OntologyTermAnnotation.java). | Field | Description | | :--- | :--- | | **id**
*String*
|

Id to identify the object

| -| **ageOfOnset**
*String*
|

Indicates the age of on set of the phenotype

| -| **status**
*Status*
|

Status of phenotype OBSERVED, NOT_OBSERVED, UNKNOWN

| | **name**
*String*
|

Object name

| | **description**
*String*
|

Users may provide a description for the entry.

| | **source**
*String*
|

Ontology source

| | **url**
*String*
|

Ontology url

| | **attributes**
*Map*
|

Dictionary that can be customised by users to store any additional information users may require..

| -### Disorder -You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/clinical/Disorder.java). +### Status +You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/common/Status.java). + + +### SexOntologyTermAnnotation +You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/core/SexOntologyTermAnnotation.java). | Field | Description | | :--- | :--- | @@ -133,29 +137,25 @@ You can find the Java code [here](https://github.com/opencb/biodata/tree/develop | **url**
*String*
|

Ontology url

| | **attributes**
*Map*
|

Dictionary that can be customised by users to store any additional information users may require..

| -### IndividualInternal -You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/individual/IndividualInternal.java). +### IndividualQualityControl +You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/individual/IndividualQualityControl.java). | Field | Description | | :--- | :--- | -| **status**
*[Status](https://docs.opencga.opencb.org/data-models/individual#status)*
|

Status of the internal object.

| -| **registrationDate**
*String*
|

Registration date of the internal object.

| -| **lastModified**
*String*
|

Date of the last modification of the internal object.

| +| **inferredSexReports**
*List<InferredSexReport>*
|

List of inferred sex reports, it depends on the method (currently by coverage ratio).

| +| **sampleRelatednessReport**
*[SampleRelatednessReport](https://docs.opencga.opencb.org/data-models/individual#samplerelatednessreport)*
|

Reports of samples relatedness.

| +| **mendelianErrorReports**
*List<MendelianErrorReport>*
|

Mendelian errors.

| +| **files**
*List<String>*
|

File IDs related to the quality control.

| +| **author**
*List<ClinicalComment>*
|

Comments related to the quality control.

| -### Location -You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/individual/Location.java). +### IndividualPopulation +You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/individual/IndividualPopulation.java). | Field | Description | | :--- | :--- | -| **address**
*String*
|

Location address.

| -| **postalCode**
*String*
|

Location postal code.

| -| **city**
*String*
|

Location city.

| -| **state**
*String*
|

Location state.

| -| **country**
*String*
|

Location country.

| - -### Status -You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/common/Status.java). - +| **name**
*String*
|

Name of the individual population.

| +| **subpopulation**
*String*
|

Subpopulation of the individual population.

| +| **description**
*String*
|

Description of the individual population.

| ### MendelianErrorReport You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/clinical/qc/MendelianErrorReport.java). @@ -165,10 +165,6 @@ You can find the Java code [here](https://github.com/opencb/biodata/tree/develop You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/clinical/qc/InferredSexReport.java). -### SampleRelatednessReport -You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/clinical/qc/SampleRelatednessReport.java). - - ### ClinicalComment You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/clinical/ClinicalComment.java). @@ -178,3 +174,7 @@ You can find the Java code [here](https://github.com/opencb/biodata/tree/develop | **message**
*String*
|

Clinical comment message

| | **tags**
*List<String>*
|

List of tags for the clinical comment

| | **date**
*String*
|

Date of the clinical comment

| + +### SampleRelatednessReport +You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/clinical/qc/SampleRelatednessReport.java). + diff --git a/docs/data-models/job.md b/docs/data-models/job.md index 884de4870df..a0b09fecf45 100644 --- a/docs/data-models/job.md +++ b/docs/data-models/job.md @@ -59,6 +59,18 @@ You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1 | **study**
*[JobStudyParam](https://docs.opencga.opencb.org/data-models/job#jobstudyparam)*
|

Job study.

| | **attributes**
*Map*
|

You can use this field to store any other information, keep in mind this is not indexed so you cannot search by attributes.

| +### JobInternal +You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/job/JobInternal.java). + +| Field | Description | +| :--- | :--- | +| **status**
*[ExecutionStatus](https://docs.opencga.opencb.org/data-models/job#executionstatus)*
|

Job internal status can have the values PENDING, QUEUED, RUNNING, DONE, ERROR, UNKNOWN, REGISTERING, UNREGISTERED, ABORTED, DELETED.

| +| **webhook**
*[JobInternalWebhook](https://docs.opencga.opencb.org/data-models/job#jobinternalwebhook)*
|

Job internal Webhook.

| +| **events**
*List<Event>*
|

Events of the internal job.

| +| **status**
*[Status](https://docs.opencga.opencb.org/data-models/job#status)*
|

Status of the internal object.

| +| **registrationDate**
*String*
|

Registration date of the internal object.

| +| **lastModified**
*String*
|

Date of the last modification of the internal object.

| + ### ExecutionResult You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/tools/result/ExecutionResult.java). @@ -85,18 +97,6 @@ You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1 | **type**
*Type*
|

Tool info type can have the values OPERATION and ANALYSIS.

| | **resource**
*Resource*
|

Tool info resource can have the values AUDIT, USER, PROJECT, STUDY, FILE, SAMPLE, JOB, INDIVIDUAL, COHORT, DISEASE_PANEL, FAMILY, CLINICAL_ANALYSIS, INTERPRETATION, VARIANT, ALIGNMENT, CLINICAL, EXPRESSION, RGA and FUNCTIONAL.

| -### JobInternal -You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/job/JobInternal.java). - -| Field | Description | -| :--- | :--- | -| **status**
*[ExecutionStatus](https://docs.opencga.opencb.org/data-models/job#executionstatus)*
|

Job internal status can have the values PENDING, QUEUED, RUNNING, DONE, ERROR, UNKNOWN, REGISTERING, UNREGISTERED, ABORTED, DELETED.

| -| **webhook**
*[JobInternalWebhook](https://docs.opencga.opencb.org/data-models/job#jobinternalwebhook)*
|

Job internal Webhook.

| -| **events**
*List<Event>*
|

Events of the internal job.

| -| **status**
*[Status](https://docs.opencga.opencb.org/data-models/job#status)*
|

Status of the internal object.

| -| **registrationDate**
*String*
|

Registration date of the internal object.

| -| **lastModified**
*String*
|

Date of the last modification of the internal object.

| - ### JobStudyParam You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/job/JobStudyParam.java). @@ -116,8 +116,12 @@ You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1 | **source**
*Source*
|

Executor info source can have the values FILE, PARQUET_FILE, MONGODB, HBASE, STORAGE.

| | **framework**
*Framework*
|

Executor info framework can have the values LOCAL, MAP_REDUCE, SPARK.

| -### Date -You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/java/util/Date.java). +### ExecutionStatus +You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/common/Enums/ExecutionStatus.java). + + +### URI +You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/java/net/URI.java). ### ToolStep @@ -131,8 +135,12 @@ You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1 | **status**
*Type*
|

Executor status can have the values PENDING, RUNNING, DONE and ERROR.

| | **attributes**
*ObjectMap*
|

You can use this field to store any other information, keep in mind this is not indexed so you cannot search by attributes.

| -### URI -You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/java/net/URI.java). +### Date +You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/java/util/Date.java). + + +### Status +You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/common/Status.java). ### JobInternalWebhook @@ -143,14 +151,6 @@ You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1 | **webhook**
*[URL](https://docs.opencga.opencb.org/data-models/job#url)*
|

Webhook URL.

| | **status**
*Map*
|

Webhook status map can have the values SUCCESS or ERROR.

| -### Status -You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/tools/result/Status.java). - - -### ExecutionStatus -You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/common/Enums/ExecutionStatus.java). - - ### URL You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/java/net/URL.java). diff --git a/docs/data-models/project.md b/docs/data-models/project.md index b27571132ee..4d291e0ad2d 100644 --- a/docs/data-models/project.md +++ b/docs/data-models/project.md @@ -37,10 +37,6 @@ You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1 | **release**
*[ProjectInternal](https://docs.opencga.opencb.org/data-models/project#projectinternal)*
|

An integer describing the current data release.

| | **attributes**
*Map*
|

You can use this field to store any other information, keep in mind this is not indexed so you cannot search by attributes.

| -### ProjectOrganism -You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/project/ProjectOrganism.java). - - ### ProjectInternal You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/project/ProjectInternal.java). @@ -52,6 +48,10 @@ You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1 | **registrationDate**
*String*
|

Registration date of the internal object.

| | **lastModified**
*String*
|

Date of the last modification of the internal object.

| +### ProjectOrganism +You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/project/ProjectOrganism.java). + + ### Status You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/common/Status.java). diff --git a/docs/data-models/sample.md b/docs/data-models/sample.md index 7f48a61ba24..6284da8c0e3 100644 --- a/docs/data-models/sample.md +++ b/docs/data-models/sample.md @@ -51,18 +51,6 @@ You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1 | **version**
*int*
|

Autoincremental version assigned to the registered entry. By default, updates does not create new versions. To enable versioning, users must set the `incVersion` flag from the /update web service when updating the document.

| | **internal**
*[SampleInternal](https://docs.opencga.opencb.org/data-models/sample#sampleinternal)*

_since_: 2.0 |

Sample internal information.

| -### SampleCollection -You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/sample/SampleCollection.java). - -| Field | Description | -| :--- | :--- | -| **from**
*List<OntologyTermAnnotation>*
|

OntologyTermAnnotation list.

| -| **type**
*String*
|

Type of the sample collection.

| -| **quantity**
*String*
|

Quantity collected for the sample.

| -| **method**
*String*
|

Describes which method was used to collect the sample.

| -| **date**
*String*
|

Date when the sample was collected.

| -| **attributes**
*Map*
|

Attributes of the sample collection.

| - ### SampleProcessing You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/sample/SampleProcessing.java). @@ -76,10 +64,6 @@ You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1 | **date**
*String*
|

Date when the sample was processed in the lab.

| | **attributes**
*Map*
|

Attributes of the processing.

| -### Status -You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/common/Status.java). - - ### SampleInternal You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/sample/SampleInternal.java). @@ -90,14 +74,17 @@ You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1 | **registrationDate**
*String*
|

Registration date of the internal object.

| | **lastModified**
*String*
|

Date of the last modification of the internal object.

| -### SampleQualityControl -You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/sample/SampleQualityControl.java). +### SampleCollection +You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/sample/SampleCollection.java). | Field | Description | | :--- | :--- | -| **files**
*List<String>*
|

Files used for the quality control of the sample.

| -| **comments**
*List<ClinicalComment>*
|

Comments for the quality control of the sample.

| -| **variant**
*[SampleVariantQualityControlMetrics](https://docs.opencga.opencb.org/data-models/sample#samplevariantqualitycontrolmetrics)*
|

Describes variant quality control.

| +| **from**
*List<OntologyTermAnnotation>*
|

OntologyTermAnnotation list.

| +| **type**
*String*
|

Type of the sample collection.

| +| **quantity**
*String*
|

Quantity collected for the sample.

| +| **method**
*String*
|

Describes which method was used to collect the sample.

| +| **date**
*String*
|

Date when the sample was collected.

| +| **attributes**
*Map*
|

Attributes of the sample collection.

| ### ExternalSource You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/common/ExternalSource.java). @@ -124,17 +111,18 @@ You can find the Java code [here](https://github.com/opencb/biodata/tree/develop | **url**
*String*
|

Ontology url

| | **attributes**
*Map*
|

Dictionary that can be customised by users to store any additional information users may require..

| -### OntologyTermAnnotation -You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/core/OntologyTermAnnotation.java). +### Status +You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/common/Status.java). + + +### SampleQualityControl +You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/sample/SampleQualityControl.java). | Field | Description | | :--- | :--- | -| **id**
*String*
|

Id to identify the object

| -| **name**
*String*
|

Object name

| -| **description**
*String*
|

Users may provide a description for the entry.

| -| **source**
*String*
|

Ontology source

| -| **url**
*String*
|

Ontology url

| -| **attributes**
*Map*
|

Dictionary that can be customised by users to store any additional information users may require..

| +| **files**
*List<String>*
|

Files used for the quality control of the sample.

| +| **comments**
*List<ClinicalComment>*
|

Comments for the quality control of the sample.

| +| **variant**
*[SampleVariantQualityControlMetrics](https://docs.opencga.opencb.org/data-models/sample#samplevariantqualitycontrolmetrics)*
|

Describes variant quality control.

| ### RgaIndex You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/common/RgaIndex.java). @@ -144,16 +132,6 @@ You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1 | **status**
*Status*
|

Status of the Rga index NOT_INDEXED, INDEXED, INVALID_PERMISSIONS, INVALID_METADATA, INVALID.

| | **date**
*String*
|

Date of Rga index.

| -### SampleVariantQualityControlMetrics -You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/sample/SampleVariantQualityControlMetrics.java). - -| Field | Description | -| :--- | :--- | -| **variantStats**
*List<SampleQcVariantStats>*
|

Variant stats for the quality control of the sample.

| -| **signatures**
*List<Signature>*
|

Signature for the quality control of the sample.

| -| **genomePlot**
*[GenomePlot](https://docs.opencga.opencb.org/data-models/sample#genomeplot)*
|

Genome plot for the quality control of the sample.

| -| **files**
*List<String>*
|

File for the quality control metrics of the sample.

| - ### ClinicalComment You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/clinical/ClinicalComment.java). @@ -164,18 +142,27 @@ You can find the Java code [here](https://github.com/opencb/biodata/tree/develop | **tags**
*List<String>*
|

List of tags for the clinical comment

| | **date**
*String*
|

Date of the clinical comment

| -### Signature -You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/clinical/qc/Signature.java). +### OntologyTermAnnotation +You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/core/OntologyTermAnnotation.java). | Field | Description | | :--- | :--- | | **id**
*String*
|

Id to identify the object

| +| **name**
*String*
|

Object name

| | **description**
*String*
|

Users may provide a description for the entry.

| -| **query**
*ObjectMap*
|

Map for query

| -| **type**
*String*
|

Signature type SNV, INDEL...

| -| **counts**
*List<GenomeContextCount>*
|

List of GenomeContextCount

| -| **files**
*List<String>*
|

List of files of signature

| -| **fitting**
*[SignatureFitting](https://docs.opencga.opencb.org/data-models/sample#signaturefitting)*
|

Signature fitting

| +| **source**
*String*
|

Ontology source

| +| **url**
*String*
|

Ontology url

| +| **attributes**
*Map*
|

Dictionary that can be customised by users to store any additional information users may require..

| + +### SampleVariantQualityControlMetrics +You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/sample/SampleVariantQualityControlMetrics.java). + +| Field | Description | +| :--- | :--- | +| **variantStats**
*List<SampleQcVariantStats>*
|

Variant stats for the quality control of the sample.

| +| **signatures**
*List<Signature>*
|

Signature for the quality control of the sample.

| +| **genomePlot**
*[GenomePlot](https://docs.opencga.opencb.org/data-models/sample#genomeplot)*
|

Genome plot for the quality control of the sample.

| +| **files**
*List<String>*
|

File for the quality control metrics of the sample.

| ### GenomePlot You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/clinical/qc/GenomePlot.java). @@ -198,15 +185,18 @@ You can find the Java code [here](https://github.com/opencb/biodata/tree/develop | **stats**
*SampleVariantStats*
|

Stats result set

| | **sampleId**
*String*
|

Stats result set

| -### GenomePlotConfig -You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/clinical/qc/GenomePlotConfig.java). +### Signature +You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/clinical/qc/Signature.java). | Field | Description | | :--- | :--- | -| **title**
*String*
|

Title of the genome plot configuration

| -| **density**
*String*
|

Density of the genome plot configuration

| -| **generalQuery**
*Map*
|

Map for the general query of the genome plot configuration

| -| **tracks**
*List<GenomePlotTrack>*
|

List of GenomePlotTrack

| +| **id**
*String*
|

Id to identify the object

| +| **description**
*String*
|

Users may provide a description for the entry.

| +| **query**
*ObjectMap*
|

Map for query

| +| **type**
*String*
|

Signature type SNV, INDEL...

| +| **counts**
*List<GenomeContextCount>*
|

List of GenomeContextCount

| +| **files**
*List<String>*
|

List of files of signature

| +| **fitting**
*[SignatureFitting](https://docs.opencga.opencb.org/data-models/sample#signaturefitting)*
|

Signature fitting

| ### SignatureFitting You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/clinical/qc/SignatureFitting.java). @@ -220,6 +210,16 @@ You can find the Java code [here](https://github.com/opencb/biodata/tree/develop | **coeff**
*double*
|

Coefficient of the fitting signature

| | **file**
*String*
|

Files of the fitting signature

| +### GenomePlotConfig +You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/clinical/qc/GenomePlotConfig.java). + +| Field | Description | +| :--- | :--- | +| **title**
*String*
|

Title of the genome plot configuration

| +| **density**
*String*
|

Density of the genome plot configuration

| +| **generalQuery**
*Map*
|

Map for the general query of the genome plot configuration

| +| **tracks**
*List<GenomePlotTrack>*
|

List of GenomePlotTrack

| + ### GenomeContextCount You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/clinical/qc/Signature/GenomeContextCount.java). diff --git a/docs/data-models/study.md b/docs/data-models/study.md index 47b46a0510b..9e2a1a9ed3f 100644 --- a/docs/data-models/study.md +++ b/docs/data-models/study.md @@ -71,6 +71,42 @@ You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1 | **additionalInfo**
*List<AdditionalInfo>*
|

Dictionary that can be customised by users to store any additional information users may require.

| | **attributes**
*Map*
|

You can use this field to store any other information, keep in mind this is not indexed so you cannot search by attributes.

| +### VariableSet +You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/study/VariableSet.java). + + +### AdditionalInfo +You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/common/AdditionalInfo.java). + +| Field | Description | +| :--- | :--- | +| **id**
*String*
|

Object ID is a mandatory parameter when creating a new one, this ID cannot be changed at the moment.

| +| **name**
*String*
|

Name of the .

| +| **description**
*String*
|

Users may provide a description for the entry.

| +| **type**
*String*
|

Type of the additional info.

| +| **attributes**
*Map*
|

You can use this field to store any other information, keep in mind this is not indexed so you cannot search by attributes.

| + +### Panel +You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/panel/Panel.java). + +| Field | Description | +| :--- | :--- | +| **uuid**
*String*
|

Unique 32-character identifier assigned automatically by OpenCGA.

| +| **release**
*int*
|

An integer describing the current data release.

| +| **version**
*int*
|

OpenCGA version of this panel, this is incremented when the panel is updated.

| +| **~~author~~**
*String*

_Deprecated_ |

Author of the panel.

| +| **status**
*[Status](https://docs.opencga.opencb.org/data-models/study#status)*
|

Panel status can have the values READY or DELETED.

| +| **studyUid**
*long*
|

Panel reference to study.

| +| **uid**
*long*
|

Panel reference to study.

| + +### StudyType +You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/study/StudyType.java). + +| Field | Description | +| :--- | :--- | +| **id**
*String*
|

Object ID is a mandatory parameter when creating a new one, this ID cannot be changed at the moment.

| +| **description**
*String*
|

Users may provide a description for the entry.

| + ### StudyInternal You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/study/StudyInternal.java). @@ -83,8 +119,13 @@ You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1 | **registrationDate**
*String*
|

Registration date of the internal object.

| | **lastModified**
*String*
|

Date of the last modification of the internal object.

| -### Status -You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/common/Status.java). +### Group +You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/study/Group.java). + + +### Enum Entity +_Enumeration class._ +You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/common/Enums/Entity.java). ### StudyNotification @@ -94,26 +135,9 @@ You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1 | :--- | :--- | | **webhook**
*[URL](https://docs.opencga.opencb.org/data-models/study#url)*
|

Url of the study notification.

| -### StudyType -You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/study/StudyType.java). - -| Field | Description | -| :--- | :--- | -| **id**
*String*
|

Object ID is a mandatory parameter when creating a new one, this ID cannot be changed at the moment.

| -| **description**
*String*
|

Users may provide a description for the entry.

| - -### Panel -You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/panel/Panel.java). +### URI +You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/java/net/URI.java). -| Field | Description | -| :--- | :--- | -| **uuid**
*String*
|

Unique 32-character identifier assigned automatically by OpenCGA.

| -| **release**
*int*
|

An integer describing the current data release.

| -| **version**
*int*
|

OpenCGA version of this panel, this is incremented when the panel is updated.

| -| **~~author~~**
*String*

_Deprecated_ |

Author of the panel.

| -| **status**
*[Status](https://docs.opencga.opencb.org/data-models/study#status)*
|

Panel status can have the values READY or DELETED.

| -| **studyUid**
*long*
|

Panel reference to study.

| -| **uid**
*long*
|

Panel reference to study.

| ### PermissionRule You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/study/PermissionRule.java). @@ -125,10 +149,6 @@ You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1 | **members**
*List<String>*
|

List of members of the permission rule.

| | **permissions**
*List<String>*
|

List of permissions of the permission rule.

| -### Group -You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/study/Group.java). - - ### ExternalSource You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/common/ExternalSource.java). @@ -140,34 +160,14 @@ You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1 | **source**
*String*
|

Source ...

| | **url**
*String*
|

Source ID

| -### URI -You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/java/net/URI.java). - - -### Enum Entity -_Enumeration class._ -You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/common/Enums/Entity.java). - - -### VariableSet -You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/study/VariableSet.java). +### Status +You can find the Java code [here](https://github.com/opencb/biodata/tree/develop/biodata-models/src/main/java/org/opencb/biodata/models/common/Status.java). -### AdditionalInfo -You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/common/AdditionalInfo.java). +### URL +You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/java/net/URL.java). -| Field | Description | -| :--- | :--- | -| **id**
*String*
|

Object ID is a mandatory parameter when creating a new one, this ID cannot be changed at the moment.

| -| **name**
*String*
|

Name of the .

| -| **description**
*String*
|

Users may provide a description for the entry.

| -| **type**
*String*
|

Type of the additional info.

| -| **attributes**
*Map*
|

You can use this field to store any other information, keep in mind this is not indexed so you cannot search by attributes.

| ### StudyIndex You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/study/StudyIndex.java). - -### URL -You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/java/net/URL.java). - diff --git a/docs/data-models/user.md b/docs/data-models/user.md index 5f8800ed109..f8d3a96945c 100644 --- a/docs/data-models/user.md +++ b/docs/data-models/user.md @@ -37,14 +37,14 @@ You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1 | **filters**
*List<UserFilter>*
|

A List with related filters.

| | **attributes**
*Map*
|

You can use this field to store any other information, keep in mind this is not indexed so you cannot search by attributes.

| -### UserInternal -You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/user/UserInternal.java). - - ### UserQuota You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/user/UserQuota.java). +### UserInternal +You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/user/UserInternal.java). + + ### Account You can find the Java code [here](https://github.com/opencb/opencga/tree/issue-1806/opencga-core/src/main/java/org/opencb/opencga/core/models/user/Account.java). diff --git a/opencga-analysis/pom.xml b/opencga-analysis/pom.xml index 28882a5bda5..e6b218db1f7 100644 --- a/opencga-analysis/pom.xml +++ b/opencga-analysis/pom.xml @@ -22,7 +22,7 @@ org.opencb.opencga opencga - 2.2.0-rc2 + 2.2.0 ../pom.xml diff --git a/opencga-analysis/src/main/java/org/opencb/opencga/analysis/variant/VariantExportTool.java b/opencga-analysis/src/main/java/org/opencb/opencga/analysis/variant/VariantExportTool.java index 3ff0de615de..b2e17820e0b 100644 --- a/opencga-analysis/src/main/java/org/opencb/opencga/analysis/variant/VariantExportTool.java +++ b/opencga-analysis/src/main/java/org/opencb/opencga/analysis/variant/VariantExportTool.java @@ -41,7 +41,7 @@ public VariantExportTool setQuery(Query query) { } public VariantExportTool setOutputFormat(VariantWriterFactory.VariantOutputFormat outputFormat) { - toolParams.setOutputFormat(outputFormat.toString()); + toolParams.setOutputFileFormat(outputFormat.toString()); return this; } @@ -60,14 +60,14 @@ protected void check() throws Exception { super.check(); toolParams.updateParams(params); - if (StringUtils.isEmpty(toolParams.getOutputFormat())) { - toolParams.setOutputFormat(VariantWriterFactory.VariantOutputFormat.VCF.toString()); + if (StringUtils.isEmpty(toolParams.getOutputFileFormat())) { + toolParams.setOutputFileFormat(VariantWriterFactory.VariantOutputFormat.VCF.toString()); } if (toolParams.getLimit() != null && toolParams.getLimit() == 0) { toolParams.setLimit(null); } - outputFormat = VariantWriterFactory.toOutputFormat(toolParams.getOutputFormat(), toolParams.getOutputFileName()); + outputFormat = VariantWriterFactory.toOutputFormat(toolParams.getOutputFileFormat(), toolParams.getOutputFileName()); if (outputFormat.isPlain()) { outputFormat = outputFormat.withGzip(); } diff --git a/opencga-analysis/src/main/java/org/opencb/opencga/analysis/variant/manager/VariantStorageManager.java b/opencga-analysis/src/main/java/org/opencb/opencga/analysis/variant/manager/VariantStorageManager.java index 41860583032..51316c271d9 100644 --- a/opencga-analysis/src/main/java/org/opencb/opencga/analysis/variant/manager/VariantStorageManager.java +++ b/opencga-analysis/src/main/java/org/opencb/opencga/analysis/variant/manager/VariantStorageManager.java @@ -69,6 +69,7 @@ import org.opencb.opencga.core.models.study.StudyAclEntry; import org.opencb.opencga.core.response.OpenCGAResult; import org.opencb.opencga.core.response.VariantQueryResult; +import org.opencb.opencga.core.tools.ToolParams; import org.opencb.opencga.storage.core.StorageEngineFactory; import org.opencb.opencga.storage.core.StoragePipelineResult; import org.opencb.opencga.storage.core.exceptions.StorageEngineException; @@ -387,6 +388,14 @@ public void sampleIndexAnnotate(String study, List samples, ObjectMap pa }); } + public DataResult> familyIndexUpdate(String study, + ObjectMap params, String token) + throws CatalogException, StorageEngineException { + return secureOperation(VariantFamilyIndexOperationTool.ID, study, params, token, engine -> { + return engine.familyIndexUpdate(study, params); + }); + } + public DataResult> familyIndex(String study, List familiesStr, boolean skipIncompleteFamilies, ObjectMap params, String token) throws CatalogException, StorageEngineException { @@ -407,7 +416,6 @@ public DataResult> familyIndex(String study, List familiesS trios.addAll(catalogUtils.getTriosFromFamily(study, family, metadataManager, skipIncompleteFamilies, token)); } } - return engine.familyIndex(study, trios, params); }); } @@ -501,7 +509,7 @@ public OpenCGAResult configureSampleIndex(String studyStr, SampleIndexConfi return secureOperation("configure", studyStr, new ObjectMap(), token, engine -> { sampleIndexConfiguration.validate(); String studyFqn = getStudyFqn(studyStr, token); - engine.getMetadataManager().addSampleIndexConfiguration(studyFqn, sampleIndexConfiguration); + engine.getMetadataManager().addSampleIndexConfiguration(studyFqn, sampleIndexConfiguration, true); catalogManager.getStudyManager() .setVariantEngineConfigurationSampleIndex(studyStr, sampleIndexConfiguration, token); @@ -509,10 +517,14 @@ public OpenCGAResult configureSampleIndex(String studyStr, SampleIndexConfi return new OpenCGAResult<>(0, new ArrayList<>(), 0, new ArrayList<>(), 0); } else { // If changes, launch sample-index-run - VariantSampleIndexParams params = + ToolParams params = new VariantSampleIndexParams(Collections.singletonList(ParamConstants.ALL), true, true, false); - return catalogManager.getJobManager().submit(studyFqn, VariantSampleIndexOperationTool.ID, null, - params.toParams(STUDY_PARAM, studyFqn), token); + Job job = catalogManager.getJobManager().submit(studyFqn, VariantSampleIndexOperationTool.ID, null, + params.toParams(STUDY_PARAM, studyFqn), token).first(); + params = new VariantFamilyIndexParams(Collections.emptyList(), false, true, false); + Job job2 = catalogManager.getJobManager().submit(studyFqn, VariantFamilyIndexOperationTool.ID, null, + params.toParams(STUDY_PARAM, studyFqn), token).first(); + return new OpenCGAResult<>(0, new ArrayList<>(), 2, Arrays.asList(job, job2), 0); } }); } diff --git a/opencga-analysis/src/main/java/org/opencb/opencga/analysis/variant/operations/VariantFamilyIndexOperationTool.java b/opencga-analysis/src/main/java/org/opencb/opencga/analysis/variant/operations/VariantFamilyIndexOperationTool.java index 0fc66c8b055..ca1e25111cc 100644 --- a/opencga-analysis/src/main/java/org/opencb/opencga/analysis/variant/operations/VariantFamilyIndexOperationTool.java +++ b/opencga-analysis/src/main/java/org/opencb/opencga/analysis/variant/operations/VariantFamilyIndexOperationTool.java @@ -19,9 +19,11 @@ import org.apache.commons.collections4.CollectionUtils; import org.opencb.commons.datastore.core.DataResult; import org.opencb.commons.datastore.core.Event; -import org.opencb.opencga.core.tools.annotations.Tool; -import org.opencb.opencga.core.models.operations.variant.VariantFamilyIndexParams; +import org.opencb.opencga.core.api.ParamConstants; import org.opencb.opencga.core.models.common.Enums; +import org.opencb.opencga.core.models.operations.variant.VariantFamilyIndexParams; +import org.opencb.opencga.core.tools.annotations.Tool; +import org.opencb.opencga.core.tools.annotations.ToolParams; import java.util.List; @@ -33,29 +35,43 @@ public class VariantFamilyIndexOperationTool extends OperationTool { public static final String DESCRIPTION = "Build the family index"; private String study; - private VariantFamilyIndexParams variantFamilyIndexParams; + + @ToolParams + protected VariantFamilyIndexParams variantFamilyIndexParams; @Override protected void check() throws Exception { super.check(); - variantFamilyIndexParams = VariantFamilyIndexParams.fromParams(VariantFamilyIndexParams.class, params); study = getStudyFqn(); - if (CollectionUtils.isEmpty(variantFamilyIndexParams.getFamily())) { - throw new IllegalArgumentException("Empty list of families"); + List list = variantFamilyIndexParams.getFamily(); + if (variantFamilyIndexParams.isUpdateIndex()) { + if (list.size() > 1 || list.size() == 1 && !list.get(0).equals(ParamConstants.ALL)) { + throw new IllegalArgumentException("Unaccepted parameter \"family\" when updating index."); + } + } else { + if (CollectionUtils.isEmpty(list)) { + throw new IllegalArgumentException("Empty list of families"); + } } + } @Override protected void run() throws Exception { step(() -> { - DataResult> trios = variantStorageManager.familyIndex( - study, - variantFamilyIndexParams.getFamily(), - variantFamilyIndexParams.isSkipIncompleteFamilies(), - params, - token); + DataResult> trios; + if (variantFamilyIndexParams.isUpdateIndex()) { + trios = variantStorageManager.familyIndexUpdate(study, params, token); + } else { + trios = variantStorageManager.familyIndex( + study, + variantFamilyIndexParams.getFamily(), + variantFamilyIndexParams.isSkipIncompleteFamilies(), + params, + token); + } if (trios.getEvents() != null) { for (Event event : trios.getEvents()) { addEvent(event.getType(), event.getMessage()); diff --git a/opencga-analysis/src/main/java/org/opencb/opencga/analysis/variant/operations/VariantStorageMetadataRepairTool.java b/opencga-analysis/src/main/java/org/opencb/opencga/analysis/variant/operations/VariantStorageMetadataRepairTool.java index 559ceac77c5..a8e9ba5b528 100644 --- a/opencga-analysis/src/main/java/org/opencb/opencga/analysis/variant/operations/VariantStorageMetadataRepairTool.java +++ b/opencga-analysis/src/main/java/org/opencb/opencga/analysis/variant/operations/VariantStorageMetadataRepairTool.java @@ -141,7 +141,7 @@ private void rebuildSampleFileIds(VariantStorageMetadataManager metadataManager, metadataManager.updateSampleMetadata(studyId, sampleId, sampleMetadata -> { logger.info("Repair sample {}. Had {} fileIds instead of {}.", sampleMetadata.getName(), sampleMetadata.getFiles().size(), fileIds.size()); - return sampleMetadata.setFiles(fileIds); + sampleMetadata.setFiles(fileIds); }); } } diff --git a/opencga-analysis/src/test/java/org/opencb/opencga/analysis/variant/VariantAnalysisTest.java b/opencga-analysis/src/test/java/org/opencb/opencga/analysis/variant/VariantAnalysisTest.java index 9cd67964000..1450d3ca753 100644 --- a/opencga-analysis/src/test/java/org/opencb/opencga/analysis/variant/VariantAnalysisTest.java +++ b/opencga-analysis/src/test/java/org/opencb/opencga/analysis/variant/VariantAnalysisTest.java @@ -18,7 +18,6 @@ import org.apache.commons.io.FileUtils; import org.apache.commons.lang3.mutable.MutableInt; -import org.apache.hadoop.conf.Configuration; import org.hamcrest.CoreMatchers; import org.junit.AfterClass; import org.junit.Assume; @@ -530,7 +529,7 @@ public void testExportVep() throws Exception { assertEquals("22,1,5", variantExportParams.getRegion()); variantExportParams.setCt("lof"); variantExportParams.setOutputFileName("chr1-5-22"); - variantExportParams.setOutputFormat(VariantWriterFactory.VariantOutputFormat.ENSEMBL_VEP.name()); + variantExportParams.setOutputFileFormat(VariantWriterFactory.VariantOutputFormat.ENSEMBL_VEP.name()); toolRunner.execute(VariantExportTool.class, variantExportParams.toObjectMap(), outDir, null, token); } diff --git a/opencga-app/app/misc/clients/rest_client_generator.py b/opencga-app/app/misc/clients/rest_client_generator.py index faef13ff9a2..a1cd6738ceb 100644 --- a/opencga-app/app/misc/clients/rest_client_generator.py +++ b/opencga-app/app/misc/clients/rest_client_generator.py @@ -1,11 +1,10 @@ +import json import os import re import sys from abc import ABC, abstractmethod from datetime import datetime -import json - class RestClientGenerator(ABC): @@ -27,9 +26,12 @@ def __init__(self, rest_api_file, output_dir): self.endpoints = { 'users/{user}/filters/{filterId}/update': {'method_name': 'update_filter'}, 'ga4gh/reads/{study}/{file}': {'method_name': 'fetch_reads'}, - 'analysis/clinical/{clinicalAnalysis}/interpretation/{interpretationId}/merge': {'method_name': 'merge_interpretation'}, - 'analysis/clinical/{clinicalAnalysis}/interpretation/{interpretationId}/update': {'method_name': 'update_interpretation'}, - 'analysis/clinical/{clinicalAnalysis}/interpretation/{interpretations}/delete': {'method_name': 'delete_interpretation'} + 'analysis/clinical/{clinicalAnalysis}/interpretation/{interpretationId}/merge': { + 'method_name': 'merge_interpretation'}, + 'analysis/clinical/{clinicalAnalysis}/interpretation/{interpretationId}/update': { + 'method_name': 'update_interpretation'}, + 'analysis/clinical/{clinicalAnalysis}/interpretation/{interpretations}/delete': { + 'method_name': 'delete_interpretation'} } self.categories = { 'Users': 'User', @@ -190,7 +192,7 @@ def get_method_name(self, endpoint, category): elif len(items) == 4: # e.g. /{apiVersion}/operation/variant/sample/genotype/index if not self.any_arg(items): - method_name = '_'.join([items[3], items[1], items[2]]) + method_name = '_'.join([items[0], items[1], items[2], items[3]]) # /{apiVersion}/analysis/clinical/{clinicalAnalysis}/interpretation/{interpretationId}/merge elif self.all_arg([items[0], items[2]]) and not self.any_arg([items[1], items[3]]): method_name = '_'.join([items[3], items[1]]) @@ -257,9 +259,12 @@ def parse_resources(self, category, endpoint): self.action = '' else: subpath = endpoint['path'].replace('/{apiVersion}/', '') - resources = re.findall('([a-zA-Z0-9\/]+)(\/\{[a-zA-Z0-9]+\})?(\/[a-zA-Z0-9]+)?(\/\{[a-zA-Z0-9]+\})?(\/[a-zA-Z0-9\/]+)', subpath) + resources = re.findall( + '([a-zA-Z0-9\/]+)(\/\{[a-zA-Z0-9]+\})?(\/[a-zA-Z0-9]+)?(\/\{[a-zA-Z0-9]+\})?(\/[a-zA-Z0-9\/]+)', + subpath) if resources: - [self.category, self.id1, self.subcategory, self.id2, self.action] = resources if type(resources[0]) != tuple else list(resources[0]) + [self.category, self.id1, self.subcategory, self.id2, self.action] = resources if type( + resources[0]) != tuple else list(resources[0]) if self.id1.startswith("/"): self.id1 = self.id1[2:-1] diff --git a/opencga-app/pom.xml b/opencga-app/pom.xml index c5b59a3997e..d5bc2679e33 100644 --- a/opencga-app/pom.xml +++ b/opencga-app/pom.xml @@ -22,7 +22,7 @@ org.opencb.opencga opencga - 2.2.0-rc2 + 2.2.0 ../pom.xml diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/CliOptionsParser.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/CliOptionsParser.java index 30600a7c5a0..89e92b50fe0 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/CliOptionsParser.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/CliOptionsParser.java @@ -7,6 +7,7 @@ import java.util.ArrayList; import java.util.Collections; import java.util.List; +import java.util.stream.Collectors; /** * Created on 08/09/17. @@ -54,15 +55,12 @@ protected void printMainUsage() { } protected void printCommands(JCommander commander) { - int pad = commander.getCommands().keySet().stream().mapToInt(String::length).max().orElse(0); - // Set padding between 14 and 40 - pad = Math.max(14, pad); - pad = Math.min(40, pad); + // Calculate the padding needed and add 10 extra spaces to get some left indentation + int padding = 10 + commander.getCommands().keySet().stream().mapToInt(String::length).max().orElse(0); - List cmds = new ArrayList(commander.getCommands().keySet()); - Collections.sort(cmds); + List cmds = commander.getCommands().keySet().stream().sorted().collect(Collectors.toList()); for (String key : cmds) { - PrintUtils.printCommandHelpFormattedString(key, commander.getCommandDescription(key)); + PrintUtils.printCommandHelpFormattedString(padding, key, commander.getCommandDescription(key)); } } diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/GeneralCliOptions.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/GeneralCliOptions.java index 48d03109cd9..cfeff8f4af2 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/GeneralCliOptions.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/GeneralCliOptions.java @@ -61,7 +61,7 @@ public static class GeneralOptions { * This class contains all those common parameters available for all 'subcommands' */ public static class CommonCommandOptions { - @Parameter(names = {"--out-format", "--OF"}, description = "Output format. one of {JSON, JSON_PRETTY, TEXT, YAML}", arity = 1) + @Parameter(names = {"--of", "--output-format"}, description = "Output format. one of {JSON, JSON_PRETTY, TEXT, YAML}", arity = 1) public String outputFormat = "TEXT"; @Parameter(names = {"-h", "--help"}, description = "Print this help", help = true) diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/internal/executors/VariantInternalCommandExecutor.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/internal/executors/VariantInternalCommandExecutor.java index bcbe4cce3b6..56d6ce9a20e 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/internal/executors/VariantInternalCommandExecutor.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/internal/executors/VariantInternalCommandExecutor.java @@ -516,6 +516,7 @@ private void familyIndex() VariantFamilyIndexParams params = new VariantFamilyIndexParams( cliOptions.family, cliOptions.overwrite, + cliOptions.update, cliOptions.skipIncompleteFamilies); toolRunner.execute(VariantFamilyIndexOperationTool.class, diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/internal/options/JobCommandOptions.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/internal/options/JobCommandOptions.java index 3f425a27de8..f17a66d8f7a 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/internal/options/JobCommandOptions.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/internal/options/JobCommandOptions.java @@ -6,6 +6,7 @@ import com.beust.jcommander.ParametersDelegate; import org.opencb.opencga.app.cli.GeneralCliOptions; import org.opencb.opencga.app.cli.internal.InternalCliOptionsParser; +import org.opencb.opencga.app.cli.main.parent.ParentJobsCommandExecutor; import org.opencb.opencga.core.api.ParamConstants; @Parameters(commandNames = {"job"}, commandDescription = "Implement several job tasks") diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/internal/options/VariantCommandOptions.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/internal/options/VariantCommandOptions.java index 93f1c3327cf..1e0b42ed447 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/internal/options/VariantCommandOptions.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/internal/options/VariantCommandOptions.java @@ -539,6 +539,9 @@ public class FamilyIndexCommandOptions extends GeneralCliOptions.StudyOption { @Parameter(names = {"--overwrite"}, description = "Overwrite existing values") public boolean overwrite = false; + @Parameter(names = {"--update"}, description = "Update family index") + public boolean update = false; + @Parameter(names = {"--skip-incomplete-families"}, description = "Do not process incomplete families.") public boolean skipIncompleteFamilies = false; diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/OpenCgaCompleter.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/OpenCgaCompleter.java index d1349085bb5..5ebd4c66481 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/OpenCgaCompleter.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/OpenCgaCompleter.java @@ -1,5 +1,5 @@ /* -* Copyright 2015-2022-03-15 OpenCB +* Copyright 2015-2022-03-25 OpenCB * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -32,12 +32,12 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 +* Autogenerated on: 2022-03-25 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. -* Command line version: 2.2.0-rc2-SNAPSHOT -* Command line commit: 9160ac7368cfd4011dc8764856956dd793c77c96 +* Command line version: 2.2.0-SNAPSHOT +* Command line commit: dfe705195791e3286a61838762d29ee845a5c875 */ public abstract class OpenCgaCompleter implements Completer { @@ -52,7 +52,7 @@ public abstract class OpenCgaCompleter implements Completer { .map(Candidate::new) .collect(toList()); - private List projectsList = asList( "create","search","aggregationStats","info","studies","update") + private List projectsList = asList( "create","search","aggregationStats","info","incRelease","studies","update") .stream() .map(Candidate::new) .collect(toList()); @@ -62,7 +62,7 @@ public abstract class OpenCgaCompleter implements Completer { .map(Candidate::new) .collect(toList()); - private List clinicalList = asList( "update-acl","update-clinical-configuration","create","distinct","distinct-interpretation","search-interpretation","info-interpretation","run-interpreter-cancerTiering","run-interpreter-team","run-interpreter-tiering","run-interpreter-zetta","aggregationStats-rga","query-rga-gene","summary-rga-gene","run-rga-index","query-rga-individual","summary-rga-individual","query-rga-variant","summary-rga-variant","search","actionable-variant","query-variant","acl","delete","update","info","create-interpretation","delete-interpretation","update-interpretation") + private List clinicalList = asList( "update-acl","update-clinical-configuration","create","distinct","distinct-interpretation","search-interpretation","info-interpretation","run-interpreter-cancerTiering","run-interpreter-team","run-interpreter-tiering","run-interpreter-zetta","aggregationStats-rga","query-rga-gene","summary-rga-gene","run-rga-index","query-rga-individual","summary-rga-individual","query-rga-variant","summary-rga-variant","search","actionable-variant","query-variant","acl","delete","update","info","create-interpretation","clear-interpretation","delete-interpretation","revert-interpretation","update-interpretation") .stream() .map(Candidate::new) .collect(toList()); @@ -72,12 +72,12 @@ public abstract class OpenCgaCompleter implements Completer { .map(Candidate::new) .collect(toList()); - private List individualsList = asList( "update-acl","aggregationStats","load-annotationSets","create","distinct","search","acl","delete","info","update","relatives") + private List individualsList = asList( "update-acl","aggregationStats","load-annotationSets","create","distinct","search","acl","delete","info","update","update-annotationSets-annotations","relatives") .stream() .map(Candidate::new) .collect(toList()); - private List familiesList = asList( "update-acl","aggregationStats","load-annotationSets","create","distinct","search","acl","delete","info","update") + private List familiesList = asList( "update-acl","aggregationStats","load-annotationSets","create","distinct","search","acl","delete","info","update","update-annotationSets-annotations") .stream() .map(Candidate::new) .collect(toList()); @@ -87,12 +87,12 @@ public abstract class OpenCgaCompleter implements Completer { .map(Candidate::new) .collect(toList()); - private List samplesList = asList( "update-acl","aggregationStats","load-annotationSets","create","distinct","load","search","acl","delete","info","update") + private List samplesList = asList( "update-acl","aggregationStats","load-annotationSets","create","distinct","load","search","acl","delete","info","update","update-annotationSets-annotations") .stream() .map(Candidate::new) .collect(toList()); - private List alignmentsList = asList( "run-bwa","run-coverage-index","run-qc-geneCoverageStats","query-coverage","ratio-coverage","stats-coverage","run-deeptools","run-fastqc","run-index","run-picard","run-qc","query","run-samtools") + private List alignmentsList = asList( "run-bwa","run-coverage-index","coverage-qc-geneCoverageStats-run","query-coverage","ratio-coverage","stats-coverage","run-deeptools","run-fastqc","run-index","run-picard","run-qc","query","run-samtools") .stream() .map(Candidate::new) .collect(toList()); @@ -102,22 +102,22 @@ public abstract class OpenCgaCompleter implements Completer { .map(Candidate::new) .collect(toList()); - private List studiesList = asList( "update-acl","create","search","acl","aggregationStats","info","search-audit","groups","update-groups","update-groups-users","permissionRules","update-permissionRules","run-templates","delete-templates","update","variableSets","update-variableSets","update-variableSets-variables") + private List studiesList = asList( "update-acl","create","search","acl","aggregationStats","info","search-audit","groups","update-groups","update-groups-users","permissionRules","update-permissionRules","run-templates","upload-templates","delete-templates","update","variableSets","update-variableSets","update-variableSets-variables") .stream() .map(Candidate::new) .collect(toList()); - private List filesList = asList( "update-acl","aggregationStats","load-annotationSets","create","distinct","fetch","link","run-link","run-postlink","search","acl","delete","info","unlink","update","download","grep","head","image","refresh","tail","list","tree") + private List filesList = asList( "update-acl","aggregationStats","load-annotationSets","create","distinct","fetch","link","run-link","run-postlink","search","upload","acl","delete","info","unlink","update","update-annotationSets-annotations","download","grep","head","image","refresh","tail","list","tree") .stream() .map(Candidate::new) .collect(toList()); - private List operationsList = asList( "configure-cellbase","aggregate-variant","delete-variant-annotation","index-variant-annotation","save-variant-annotation","configure-variant","delete-variant","aggregate-variant-family","index-variant-family","index-variant","launcher-variant-index","run-variant-julie","repair-variant-metadata","synchronize-variant-metadata","delete-variant-sample","index-variant-sample","delete-variant-score","index-variant-score","secondaryIndex-variant","delete-variant-secondaryIndex","delete-variant-stats","index-variant-stats","delete-variant-study") + private List operationsList = asList( "configure-cellbase","aggregate-variant","delete-variant-annotation","index-variant-annotation","save-variant-annotation","configure-variant","delete-variant","aggregate-variant-family","index-variant-family","index-variant","launcher-variant-index","run-variant-julie","repair-variant-metadata","synchronize-variant-metadata","delete-variant-sample","index-variant-sample","variant-sample-index-configure","delete-variant-score","index-variant-score","secondaryIndex-variant","delete-variant-secondaryIndex","delete-variant-stats","index-variant-stats","delete-variant-study") .stream() .map(Candidate::new) .collect(toList()); - private List cohortsList = asList( "update-acl","aggregationStats","load-annotationSets","create","distinct","generate","search","acl","delete","info","update") + private List cohortsList = asList( "update-acl","aggregationStats","load-annotationSets","create","distinct","generate","search","acl","delete","info","update","update-annotationSets-annotations") .stream() .map(Candidate::new) .collect(toList()); diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/OpencgaCliOptionsParser.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/OpencgaCliOptionsParser.java index d66fa347754..016c9aa8c5e 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/OpencgaCliOptionsParser.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/OpencgaCliOptionsParser.java @@ -1,5 +1,5 @@ /* -* Copyright 2015-2022-03-15 OpenCB +* Copyright 2015-2022-03-25 OpenCB * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -26,12 +26,12 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 +* Autogenerated on: 2022-03-25 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. -* Command line version: 2.2.0-rc2-SNAPSHOT -* Command line commit: 9160ac7368cfd4011dc8764856956dd793c77c96 +* Command line version: 2.2.0-SNAPSHOT +* Command line commit: dfe705195791e3286a61838762d29ee845a5c875 */ public class OpencgaCliOptionsParser extends ParentCliOptionsParser { @@ -106,6 +106,7 @@ public OpencgaCliOptionsParser() { projectsSubCommands.addCommand("search", projectsCommandOptions.searchCommandOptions); projectsSubCommands.addCommand("aggregationstats", projectsCommandOptions.aggregationStatsCommandOptions); projectsSubCommands.addCommand("info", projectsCommandOptions.infoCommandOptions); + projectsSubCommands.addCommand("increlease", projectsCommandOptions.incReleaseCommandOptions); projectsSubCommands.addCommand("studies", projectsCommandOptions.studiesCommandOptions); projectsSubCommands.addCommand("update", projectsCommandOptions.updateCommandOptions); @@ -151,7 +152,9 @@ public OpencgaCliOptionsParser() { analysisClinicalSubCommands.addCommand("update", analysisClinicalCommandOptions.updateCommandOptions); analysisClinicalSubCommands.addCommand("info", analysisClinicalCommandOptions.infoCommandOptions); analysisClinicalSubCommands.addCommand("interpretation-create", analysisClinicalCommandOptions.createInterpretationCommandOptions); + analysisClinicalSubCommands.addCommand("interpretation-clear", analysisClinicalCommandOptions.clearInterpretationCommandOptions); analysisClinicalSubCommands.addCommand("interpretation-delete", analysisClinicalCommandOptions.deleteInterpretationCommandOptions); + analysisClinicalSubCommands.addCommand("interpretation-revert", analysisClinicalCommandOptions.revertInterpretationCommandOptions); analysisClinicalSubCommands.addCommand("interpretation-update", analysisClinicalCommandOptions.updateInterpretationCommandOptions); jobsCommandOptions = new JobsCommandOptions(commonCommandOptions, jCommander); @@ -170,6 +173,7 @@ public OpencgaCliOptionsParser() { jobsSubCommands.addCommand("update", jobsCommandOptions.updateCommandOptions); jobsSubCommands.addCommand("log-head", jobsCommandOptions.headLogCommandOptions); jobsSubCommands.addCommand("log-tail", jobsCommandOptions.tailLogCommandOptions); + jobsSubCommands.addCommand("log", jobsCommandOptions.logCommandOptions); individualsCommandOptions = new IndividualsCommandOptions(commonCommandOptions, jCommander); jCommander.addCommand("individuals", individualsCommandOptions); @@ -184,6 +188,7 @@ public OpencgaCliOptionsParser() { individualsSubCommands.addCommand("delete", individualsCommandOptions.deleteCommandOptions); individualsSubCommands.addCommand("info", individualsCommandOptions.infoCommandOptions); individualsSubCommands.addCommand("update", individualsCommandOptions.updateCommandOptions); + individualsSubCommands.addCommand("annotationsets-annotations-update", individualsCommandOptions.updateAnnotationSetsAnnotationsCommandOptions); individualsSubCommands.addCommand("relatives", individualsCommandOptions.relativesCommandOptions); familiesCommandOptions = new FamiliesCommandOptions(commonCommandOptions, jCommander); @@ -199,6 +204,7 @@ public OpencgaCliOptionsParser() { familiesSubCommands.addCommand("delete", familiesCommandOptions.deleteCommandOptions); familiesSubCommands.addCommand("info", familiesCommandOptions.infoCommandOptions); familiesSubCommands.addCommand("update", familiesCommandOptions.updateCommandOptions); + familiesSubCommands.addCommand("annotationsets-annotations-update", familiesCommandOptions.updateAnnotationSetsAnnotationsCommandOptions); usersCommandOptions = new UsersCommandOptions(commonCommandOptions, jCommander); jCommander.addCommand("users", usersCommandOptions); @@ -229,13 +235,14 @@ public OpencgaCliOptionsParser() { samplesSubCommands.addCommand("delete", samplesCommandOptions.deleteCommandOptions); samplesSubCommands.addCommand("info", samplesCommandOptions.infoCommandOptions); samplesSubCommands.addCommand("update", samplesCommandOptions.updateCommandOptions); + samplesSubCommands.addCommand("annotationsets-annotations-update", samplesCommandOptions.updateAnnotationSetsAnnotationsCommandOptions); analysisAlignmentCommandOptions = new AnalysisAlignmentCommandOptions(commonCommandOptions, jCommander); jCommander.addCommand("alignments", analysisAlignmentCommandOptions); JCommander analysisAlignmentSubCommands = jCommander.getCommands().get("alignments"); analysisAlignmentSubCommands.addCommand("bwa-run", analysisAlignmentCommandOptions.runBwaCommandOptions); analysisAlignmentSubCommands.addCommand("coverage-index-run", analysisAlignmentCommandOptions.runCoverageIndexCommandOptions); - analysisAlignmentSubCommands.addCommand("qc-genecoveragestats-run", analysisAlignmentCommandOptions.runQcGeneCoverageStatsCommandOptions); + analysisAlignmentSubCommands.addCommand("coverage-qc-genecoveragestats-run", analysisAlignmentCommandOptions.coverageQcGeneCoverageStatsRunCommandOptions); analysisAlignmentSubCommands.addCommand("coverage-query", analysisAlignmentCommandOptions.queryCoverageCommandOptions); analysisAlignmentSubCommands.addCommand("coverage-ratio", analysisAlignmentCommandOptions.ratioCoverageCommandOptions); analysisAlignmentSubCommands.addCommand("coverage-stats", analysisAlignmentCommandOptions.statsCoverageCommandOptions); @@ -268,12 +275,12 @@ public OpencgaCliOptionsParser() { studiesSubCommands.addCommand("permissionrules", studiesCommandOptions.permissionRulesCommandOptions); studiesSubCommands.addCommand("permissionrules-update", studiesCommandOptions.updatePermissionRulesCommandOptions); studiesSubCommands.addCommand("templates-run", studiesCommandOptions.runTemplatesCommandOptions); + studiesSubCommands.addCommand("templates-upload", studiesCommandOptions.uploadTemplatesCommandOptions); studiesSubCommands.addCommand("templates-delete", studiesCommandOptions.deleteTemplatesCommandOptions); studiesSubCommands.addCommand("update", studiesCommandOptions.updateCommandOptions); studiesSubCommands.addCommand("variablesets", studiesCommandOptions.variableSetsCommandOptions); studiesSubCommands.addCommand("variablesets-update", studiesCommandOptions.updateVariableSetsCommandOptions); studiesSubCommands.addCommand("variablesets-variables-update", studiesCommandOptions.updateVariableSetsVariablesCommandOptions); - studiesSubCommands.addCommand("templates-upload", studiesCommandOptions.templatesUploadCommandOptions); filesCommandOptions = new FilesCommandOptions(commonCommandOptions, jCommander); jCommander.addCommand("files", filesCommandOptions); @@ -288,11 +295,13 @@ public OpencgaCliOptionsParser() { filesSubCommands.addCommand("link-run", filesCommandOptions.runLinkCommandOptions); filesSubCommands.addCommand("postlink-run", filesCommandOptions.runPostlinkCommandOptions); filesSubCommands.addCommand("search", filesCommandOptions.searchCommandOptions); + filesSubCommands.addCommand("upload", filesCommandOptions.uploadCommandOptions); filesSubCommands.addCommand("acl", filesCommandOptions.aclCommandOptions); filesSubCommands.addCommand("delete", filesCommandOptions.deleteCommandOptions); filesSubCommands.addCommand("info", filesCommandOptions.infoCommandOptions); filesSubCommands.addCommand("unlink", filesCommandOptions.unlinkCommandOptions); filesSubCommands.addCommand("update", filesCommandOptions.updateCommandOptions); + filesSubCommands.addCommand("annotationsets-annotations-update", filesCommandOptions.updateAnnotationSetsAnnotationsCommandOptions); filesSubCommands.addCommand("download", filesCommandOptions.downloadCommandOptions); filesSubCommands.addCommand("grep", filesCommandOptions.grepCommandOptions); filesSubCommands.addCommand("head", filesCommandOptions.headCommandOptions); @@ -301,7 +310,6 @@ public OpencgaCliOptionsParser() { filesSubCommands.addCommand("tail", filesCommandOptions.tailCommandOptions); filesSubCommands.addCommand("list", filesCommandOptions.listCommandOptions); filesSubCommands.addCommand("tree", filesCommandOptions.treeCommandOptions); - filesSubCommands.addCommand("upload", filesCommandOptions.uploadCommandOptions); operationsVariantStorageCommandOptions = new OperationsVariantStorageCommandOptions(commonCommandOptions, jCommander); jCommander.addCommand("operations", operationsVariantStorageCommandOptions); @@ -322,6 +330,7 @@ public OpencgaCliOptionsParser() { operationsVariantStorageSubCommands.addCommand("variant-metadata-synchronize", operationsVariantStorageCommandOptions.synchronizeVariantMetadataCommandOptions); operationsVariantStorageSubCommands.addCommand("variant-sample-delete", operationsVariantStorageCommandOptions.deleteVariantSampleCommandOptions); operationsVariantStorageSubCommands.addCommand("variant-sample-index", operationsVariantStorageCommandOptions.indexVariantSampleCommandOptions); + operationsVariantStorageSubCommands.addCommand("variant-sample-index-configure", operationsVariantStorageCommandOptions.variantSampleIndexConfigureCommandOptions); operationsVariantStorageSubCommands.addCommand("variant-score-delete", operationsVariantStorageCommandOptions.deleteVariantScoreCommandOptions); operationsVariantStorageSubCommands.addCommand("variant-score-index", operationsVariantStorageCommandOptions.indexVariantScoreCommandOptions); operationsVariantStorageSubCommands.addCommand("variant-secondaryindex", operationsVariantStorageCommandOptions.secondaryIndexVariantCommandOptions); @@ -344,6 +353,7 @@ public OpencgaCliOptionsParser() { cohortsSubCommands.addCommand("delete", cohortsCommandOptions.deleteCommandOptions); cohortsSubCommands.addCommand("info", cohortsCommandOptions.infoCommandOptions); cohortsSubCommands.addCommand("update", cohortsCommandOptions.updateCommandOptions); + cohortsSubCommands.addCommand("annotationsets-annotations-update", cohortsCommandOptions.updateAnnotationSetsAnnotationsCommandOptions); } public AnalysisVariantCommandOptions getAnalysisVariantCommandOptions() { diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/OpencgaMain.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/OpencgaMain.java index b793fc1ee36..6541ebc5fcb 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/OpencgaMain.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/OpencgaMain.java @@ -23,6 +23,8 @@ import org.opencb.opencga.app.cli.main.shell.Shell; import org.opencb.opencga.app.cli.main.utils.CommandLineUtils; import org.opencb.opencga.catalog.exceptions.CatalogAuthenticationException; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import java.util.Arrays; import java.util.Locale; @@ -34,6 +36,7 @@ */ public class OpencgaMain { + private static final Logger logger = LoggerFactory.getLogger(OpencgaMain.class); public static Mode mode = Mode.CLI; public static Shell shell; public static Level logLevel = Level.OFF; @@ -48,6 +51,7 @@ public static void main(String[] args) { checkLogLevel(args); checkMode(args); CommandLineUtils.printLog(Arrays.toString(args)); + logger.debug(Arrays.toString(args)); try { if (Mode.SHELL.equals(getMode())) { executeShell(args); @@ -56,7 +60,7 @@ public static void main(String[] args) { } } catch (Exception e) { CommandLineUtils.error("Failed to initialize OpenCGA CLI " + e.getMessage(), e); - e.printStackTrace(); + logger.error("Failed to initialize OpenCGA CLI " + e.getMessage(), e); } } @@ -67,22 +71,25 @@ private static void checkMode(String[] args) { setMode(Mode.CLI); } CommandLineUtils.printLog("Execution mode " + getMode()); + logger.debug("Execution mode " + getMode()); } - private static void checkLogLevel(String[] args) { if (ArrayUtils.contains(args, "--log-level")) { + String level = ""; try { - String level = args[ArrayUtils.indexOf(args, "--log-level") + 1].toLowerCase(Locale.ROOT); - setLogLevel(getNormalizedLogLevel(level)); + level = args[ArrayUtils.indexOf(args, "--log-level") + 1].toLowerCase(Locale.ROOT); + Level logLevel = getNormalizedLogLevel(level); + setLogLevel(logLevel); + + logger.debug("Console verbose mode: " + logLevel); } catch (Exception e) { setLogLevel(Level.SEVERE); CommandLineUtils.error("Invalid log level. Valid values are INFO, WARN, DEBUG, ERROR", e); + logger.error("Invalid log level " + level + ": Valid values are INFO, WARN, DEBUG, ERROR", e); System.exit(0); } } - - } private static Level getNormalizedLogLevel(String level) { @@ -96,7 +103,7 @@ private static Level getNormalizedLogLevel(String level) { case "warn": return Level.WARNING; case "error": - case "sever": + case "severe": return Level.SEVERE; default: return Level.OFF; @@ -123,13 +130,16 @@ public static void executeShell(String[] args) { } // Create a shell executor instance shell = new Shell(options); - CommandLineUtils.printLog("Shell created "); + CommandLineUtils.debug("Shell created "); + logger.debug("Shell created "); // Launch execute command to begin the execution shell.execute(); } catch (CatalogAuthenticationException e) { CommandLineUtils.printLog("Failed to initialize shell", e); + logger.error("Failed to initialize shell", e); } catch (Exception e) { CommandLineUtils.printLog("Failed to execute shell", e); + logger.error("Failed to execute shell", e); } } @@ -143,23 +153,24 @@ private static String[] normalizePasswordArgs(String[] args, String s) { return args; } - public static String[] parseCliParams(String[] args) throws CatalogAuthenticationException { + public static String[] parseCliParams(String[] args) { CommandLineUtils.printLog("Executing " + CommandLineUtils.argsToString(args)); if (CommandLineUtils.isNotHelpCommand(args)) { if (ArrayUtils.contains(args, "--user-password")) { normalizePasswordArgs(args, "--user-password"); } } - CommandLineUtils.printLog("CLI PARSED PARAMS ::: " + CommandLineUtils.argsToString(args)); + CommandLineUtils.debug("CLI parsed params ::: " + CommandLineUtils.argsToString(args)); + logger.debug("CLI parsed params ::: " + CommandLineUtils.argsToString(args)); String shortcut = CommandLineUtils.getShortcut(args); args = CommandLineUtils.processShortCuts(args); - if (args != null) { - CommandLineUtils.printLog("PROCESS SHORTCUTS RESULT ::: " + CommandLineUtils.argsToString(args)); + CommandLineUtils.debug("Process shortcut result ::: " + CommandLineUtils.argsToString(args)); + logger.debug("Process shortcut result ::: " + CommandLineUtils.argsToString(args)); } else { - CommandLineUtils.printLog("IS SHORTCUT " + shortcut); + CommandLineUtils.debug("Is shortcut " + shortcut); + logger.debug("Is shortcut " + shortcut); } - return args; } diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/AnalysisAlignmentCommandExecutor.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/AnalysisAlignmentCommandExecutor.java index e4aad2a469f..f5cdf875fe2 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/AnalysisAlignmentCommandExecutor.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/AnalysisAlignmentCommandExecutor.java @@ -35,7 +35,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 +* Autogenerated on: 2022-03-25 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -44,7 +44,7 @@ /** * This class contains methods for the Analysis - Alignment command line. - * OpenCGA version: 2.2.0-rc2-SNAPSHOT + * OpenCGA version: 2.2.0-SNAPSHOT * PATH: /{apiVersion}/analysis/alignment */ public class AnalysisAlignmentCommandExecutor extends OpencgaCommandExecutor { @@ -72,8 +72,8 @@ public void execute() throws Exception { case "coverage-index-run": queryResponse = runCoverageIndex(); break; - case "qc-genecoveragestats-run": - queryResponse = runQcGeneCoverageStats(); + case "coverage-qc-genecoveragestats-run": + queryResponse = coverageQcGeneCoverageStatsRun(); break; case "coverage-query": queryResponse = queryCoverage(); @@ -126,13 +126,13 @@ private RestResponse runBwa() throws Exception { queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } BwaWrapperParams bwaWrapperParams = new BwaWrapperParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(bwaWrapperParams)); @@ -164,13 +164,13 @@ private RestResponse runCoverageIndex() throws Exception { queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } CoverageIndexParams coverageIndexParams = new CoverageIndexParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(coverageIndexParams)); @@ -181,18 +181,21 @@ private RestResponse runCoverageIndex() throws Exception { } else { ((CoverageIndexParams)coverageIndexParams) .setFile(commandOptions.file) - .setWindowSize(commandOptions.windowSize) - .setOverwrite(commandOptions.overwrite); + .setWindowSize(commandOptions.windowSize); + + if (commandOptions.overwrite != null){ + ((CoverageIndexParams)coverageIndexParams).setOverwrite(commandOptions.overwrite); + } } return openCGAClient.getAlignmentClient().runCoverageIndex(coverageIndexParams, queryParams); } - private RestResponse runQcGeneCoverageStats() throws Exception { + private RestResponse coverageQcGeneCoverageStatsRun() throws Exception { - logger.debug("Executing runQcGeneCoverageStats in Analysis - Alignment command line"); + logger.debug("Executing coverageQcGeneCoverageStatsRun in Analysis - Alignment command line"); - AnalysisAlignmentCommandOptions.RunQcGeneCoverageStatsCommandOptions commandOptions = analysisAlignmentCommandOptions.runQcGeneCoverageStatsCommandOptions; + AnalysisAlignmentCommandOptions.CoverageQcGeneCoverageStatsRunCommandOptions commandOptions = analysisAlignmentCommandOptions.coverageQcGeneCoverageStatsRunCommandOptions; ObjectMap queryParams = new ObjectMap(); queryParams.putIfNotEmpty("study", commandOptions.study); @@ -200,13 +203,13 @@ private RestResponse runQcGeneCoverageStats() throws Exception { queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } AlignmentGeneCoverageStatsParams alignmentGeneCoverageStatsParams = new AlignmentGeneCoverageStatsParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(alignmentGeneCoverageStatsParams)); @@ -221,7 +224,7 @@ private RestResponse runQcGeneCoverageStats() throws Exception { .setOutdir(commandOptions.outdir); } - return openCGAClient.getAlignmentClient().runQcGeneCoverageStats(alignmentGeneCoverageStatsParams, queryParams); + return openCGAClient.getAlignmentClient().coverageQcGeneCoverageStatsRun(alignmentGeneCoverageStatsParams, queryParams); } private RestResponse queryCoverage() throws Exception { @@ -239,8 +242,8 @@ private RestResponse queryCoverage() throws Exception { queryParams.putIfNotEmpty("range", commandOptions.range); queryParams.putIfNotNull("windowSize", commandOptions.windowSize); queryParams.putIfNotNull("splitResults", commandOptions.splitResults); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getAlignmentClient().queryCoverage(commandOptions.file, queryParams); @@ -261,8 +264,8 @@ private RestResponse ratioCoverage() throws Exception { queryParams.putIfNotNull("onlyExons", commandOptions.onlyExons); queryParams.putIfNotNull("windowSize", commandOptions.windowSize); queryParams.putIfNotNull("splitResults", commandOptions.splitResults); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getAlignmentClient().ratioCoverage(commandOptions.file1, commandOptions.file2, queryParams); @@ -277,8 +280,8 @@ private RestResponse statsCoverage() throws Exception { ObjectMap queryParams = new ObjectMap(); queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotNull("threshold", commandOptions.threshold); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getAlignmentClient().statsCoverage(commandOptions.file, commandOptions.gene, queryParams); @@ -296,13 +299,13 @@ private RestResponse runDeeptools() throws Exception { queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } DeeptoolsWrapperParams deeptoolsWrapperParams = new DeeptoolsWrapperParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(deeptoolsWrapperParams)); @@ -331,13 +334,13 @@ private RestResponse runFastqc() throws Exception { queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } FastqcWrapperParams fastqcWrapperParams = new FastqcWrapperParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(fastqcWrapperParams)); @@ -366,13 +369,13 @@ private RestResponse runIndex() throws Exception { queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } AlignmentIndexParams alignmentIndexParams = new AlignmentIndexParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(alignmentIndexParams)); @@ -382,8 +385,11 @@ private RestResponse runIndex() throws Exception { objectMapper.writeValue(new java.io.File(commandOptions.jsonFile), alignmentIndexParams); } else { ((AlignmentIndexParams)alignmentIndexParams) - .setFile(commandOptions.file) - .setOverwrite(commandOptions.overwrite); + .setFile(commandOptions.file); + + if (commandOptions.overwrite != null){ + ((AlignmentIndexParams)alignmentIndexParams).setOverwrite(commandOptions.overwrite); + } } return openCGAClient.getAlignmentClient().runIndex(alignmentIndexParams, queryParams); @@ -401,13 +407,13 @@ private RestResponse runPicard() throws Exception { queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } PicardWrapperParams picardWrapperParams = new PicardWrapperParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(picardWrapperParams)); @@ -436,13 +442,13 @@ private RestResponse runQc() throws Exception { queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } AlignmentQcParams alignmentQcParams = new AlignmentQcParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(alignmentQcParams)); @@ -456,9 +462,12 @@ private RestResponse runQc() throws Exception { .setBedFile(commandOptions.bedFile) .setDictFile(commandOptions.dictFile) .setSkip(commandOptions.skip) - .setOverwrite(commandOptions.overwrite) .setOutdir(commandOptions.outdir); + if (commandOptions.overwrite != null){ + ((AlignmentQcParams)alignmentQcParams).setOverwrite(commandOptions.overwrite); + } + } return openCGAClient.getAlignmentClient().runQc(alignmentQcParams, queryParams); } @@ -489,8 +498,8 @@ private RestResponse query() throws Exception { queryParams.putIfNotNull("forceMDField", commandOptions.forceMDField); queryParams.putIfNotNull("binQualities", commandOptions.binQualities); queryParams.putIfNotNull("splitResults", commandOptions.splitResults); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getAlignmentClient().query(commandOptions.file, queryParams); @@ -508,13 +517,13 @@ private RestResponse runSamtools() throws Exception { queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } SamtoolsWrapperParams samtoolsWrapperParams = new SamtoolsWrapperParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(samtoolsWrapperParams)); diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/AnalysisClinicalCommandExecutor.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/AnalysisClinicalCommandExecutor.java index 489065c2c1d..3267711f8e3 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/AnalysisClinicalCommandExecutor.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/AnalysisClinicalCommandExecutor.java @@ -64,7 +64,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 +* Autogenerated on: 2022-03-25 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -73,7 +73,7 @@ /** * This class contains methods for the Analysis - Clinical command line. - * OpenCGA version: 2.2.0-rc2-SNAPSHOT + * OpenCGA version: 2.2.0-SNAPSHOT * PATH: /{apiVersion}/analysis/clinical */ public class AnalysisClinicalCommandExecutor extends OpencgaCommandExecutor { @@ -176,9 +176,15 @@ public void execute() throws Exception { case "interpretation-create": queryResponse = createInterpretation(); break; + case "interpretation-clear": + queryResponse = clearInterpretation(); + break; case "interpretation-delete": queryResponse = deleteInterpretation(); break; + case "interpretation-revert": + queryResponse = revertInterpretation(); + break; case "interpretation-update": queryResponse = updateInterpretation(); break; @@ -200,13 +206,13 @@ private RestResponse updateAcl() throws Exception { ObjectMap queryParams = new ObjectMap(); queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotNull("propagate", commandOptions.propagate); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } ClinicalAnalysisAclUpdateParams clinicalAnalysisAclUpdateParams = new ClinicalAnalysisAclUpdateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(clinicalAnalysisAclUpdateParams)); @@ -231,13 +237,13 @@ private RestResponse updateClinicalConfiguration() throws Exception { ObjectMap queryParams = new ObjectMap(); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } ClinicalAnalysisStudyConfiguration clinicalAnalysisStudyConfiguration = new ClinicalAnalysisStudyConfiguration(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(clinicalAnalysisStudyConfiguration)); @@ -260,8 +266,8 @@ private RestResponse create() throws Exception { queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotNull("skipCreateDefaultInterpretation", commandOptions.skipCreateDefaultInterpretation); queryParams.putIfNotNull("includeResult", commandOptions.includeResult); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } @@ -297,10 +303,14 @@ private RestResponse create() throws Exception { FamilyParam familyParam= new FamilyParam(); invokeSetter(familyParam, "id", commandOptions.familyId); - ClinicalAnalysis.Type typeParam = ClinicalAnalysis.Type.valueOf(commandOptions.type); + ClinicalAnalysis.Type typeParam = null; + if (commandOptions.type != null) { + typeParam = ClinicalAnalysis.Type.valueOf(commandOptions.type); + + } ClinicalAnalysisCreateParams clinicalAnalysisCreateParams = new ClinicalAnalysisCreateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(clinicalAnalysisCreateParams)); @@ -316,7 +326,6 @@ private RestResponse create() throws Exception { .setDisorder(disorderReferenceParam) .setProband(probandParam) .setFamily(familyParam) - .setPanelLock(commandOptions.panelLock) .setAnalyst(clinicalAnalystParam) .setReport(clinicalReport) .setInterpretation(interpretationCreateParams) @@ -326,6 +335,10 @@ private RestResponse create() throws Exception { .setPriority(priorityParam) .setStatus(statusParam); + if (commandOptions.panelLock != null){ + ((ClinicalAnalysisCreateParams)clinicalAnalysisCreateParams).setPanelLock(commandOptions.panelLock); + } + } return openCGAClient.getClinicalAnalysisClient().create(clinicalAnalysisCreateParams, queryParams); } @@ -362,8 +375,8 @@ private RestResponse distinct() throws Exception { queryParams.putIfNotEmpty("status", commandOptions.status); queryParams.putIfNotEmpty("internalStatus", commandOptions.internalStatus); queryParams.putIfNotNull("deleted", commandOptions.deleted); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getClinicalAnalysisClient().distinct(commandOptions.field, queryParams); @@ -390,8 +403,8 @@ private RestResponse distinctInterpretation() throws Exception { queryParams.putIfNotEmpty("status", commandOptions.status); queryParams.putIfNotEmpty("internalStatus", commandOptions.internalStatus); queryParams.putIfNotEmpty("release", commandOptions.release); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getClinicalAnalysisClient().distinctInterpretation(commandOptions.field, queryParams); @@ -423,8 +436,8 @@ private RestResponse searchInterpretation() throws Exception { queryParams.putIfNotEmpty("status", commandOptions.status); queryParams.putIfNotEmpty("internalStatus", commandOptions.internalStatus); queryParams.putIfNotEmpty("release", commandOptions.release); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getClinicalAnalysisClient().searchInterpretation(queryParams); @@ -442,8 +455,8 @@ private RestResponse infoInterpretation() throws Exception { queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotEmpty("version", commandOptions.version); queryParams.putIfNotNull("deleted", commandOptions.deleted); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getClinicalAnalysisClient().infoInterpretation(commandOptions.interpretations, queryParams); @@ -461,13 +474,13 @@ private RestResponse runInterpreterCancerTiering() throws Exception { queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } CancerTieringInterpretationAnalysisParams cancerTieringInterpretationAnalysisParams = new CancerTieringInterpretationAnalysisParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(cancerTieringInterpretationAnalysisParams)); @@ -478,8 +491,11 @@ private RestResponse runInterpreterCancerTiering() throws Exception { } else { ((CancerTieringInterpretationAnalysisParams)cancerTieringInterpretationAnalysisParams) .setClinicalAnalysis(commandOptions.clinicalAnalysis) - .setDiscardedVariants(splitWithTrim(commandOptions.discardedVariants)) - .setPrimary(commandOptions.primary); + .setDiscardedVariants(splitWithTrim(commandOptions.discardedVariants)); + + if (commandOptions.primary != null){ + ((CancerTieringInterpretationAnalysisParams)cancerTieringInterpretationAnalysisParams).setPrimary(commandOptions.primary); + } } return openCGAClient.getClinicalAnalysisClient().runInterpreterCancerTiering(cancerTieringInterpretationAnalysisParams, queryParams); @@ -497,13 +513,13 @@ private RestResponse runInterpreterTeam() throws Exception { queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } TeamInterpretationAnalysisParams teamInterpretationAnalysisParams = new TeamInterpretationAnalysisParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(teamInterpretationAnalysisParams)); @@ -515,8 +531,11 @@ private RestResponse runInterpreterTeam() throws Exception { ((TeamInterpretationAnalysisParams)teamInterpretationAnalysisParams) .setClinicalAnalysis(commandOptions.clinicalAnalysis) .setPanels(splitWithTrim(commandOptions.panels)) - .setFamilySegregation(commandOptions.familySegregation) - .setPrimary(commandOptions.primary); + .setFamilySegregation(commandOptions.familySegregation); + + if (commandOptions.primary != null){ + ((TeamInterpretationAnalysisParams)teamInterpretationAnalysisParams).setPrimary(commandOptions.primary); + } } return openCGAClient.getClinicalAnalysisClient().runInterpreterTeam(teamInterpretationAnalysisParams, queryParams); @@ -534,14 +553,18 @@ private RestResponse runInterpreterTiering() throws Exception { queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } - ClinicalProperty.Penetrance penetranceParam = ClinicalProperty.Penetrance.valueOf(commandOptions.penetrance); + ClinicalProperty.Penetrance penetranceParam = null; + if (commandOptions.penetrance != null) { + penetranceParam = ClinicalProperty.Penetrance.valueOf(commandOptions.penetrance); + + } TieringInterpretationAnalysisParams tieringInterpretationAnalysisParams = new TieringInterpretationAnalysisParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(tieringInterpretationAnalysisParams)); @@ -552,8 +575,11 @@ private RestResponse runInterpreterTiering() throws Exception { } else { ((TieringInterpretationAnalysisParams)tieringInterpretationAnalysisParams) .setClinicalAnalysis(commandOptions.clinicalAnalysis) - .setPanels(splitWithTrim(commandOptions.panels)) - .setPrimary(commandOptions.primary); + .setPanels(splitWithTrim(commandOptions.panels)); + + if (commandOptions.primary != null){ + ((TieringInterpretationAnalysisParams)tieringInterpretationAnalysisParams).setPrimary(commandOptions.primary); + } } return openCGAClient.getClinicalAnalysisClient().runInterpreterTiering(tieringInterpretationAnalysisParams, queryParams); @@ -571,13 +597,13 @@ private RestResponse runInterpreterZetta() throws Exception { queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } ZettaInterpretationAnalysisParams zettaInterpretationAnalysisParams = new ZettaInterpretationAnalysisParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(zettaInterpretationAnalysisParams)); @@ -630,14 +656,20 @@ private RestResponse runInterpreterZetta() throws Exception { .setFunctionalScore(commandOptions.functionalScore) .setClinical(commandOptions.clinical) .setClinicalSignificance(commandOptions.clinicalSignificance) - .setClinicalConfirmedStatus(commandOptions.clinicalConfirmedStatus) .setCustomAnnotation(commandOptions.customAnnotation) .setPanel(commandOptions.panel) .setPanelModeOfInheritance(commandOptions.panelModeOfInheritance) .setPanelConfidence(commandOptions.panelConfidence) .setPanelRoleInCancer(commandOptions.panelRoleInCancer) - .setTrait(commandOptions.trait) - .setPrimary(commandOptions.primary); + .setTrait(commandOptions.trait); + + if (commandOptions.clinicalConfirmedStatus != null){ + ((ZettaInterpretationAnalysisParams)zettaInterpretationAnalysisParams).setClinicalConfirmedStatus(commandOptions.clinicalConfirmedStatus); + } + + if (commandOptions.primary != null){ + ((ZettaInterpretationAnalysisParams)zettaInterpretationAnalysisParams).setPrimary(commandOptions.primary); + } } return openCGAClient.getClinicalAnalysisClient().runInterpreterZetta(zettaInterpretationAnalysisParams, queryParams); @@ -673,8 +705,8 @@ private RestResponse aggregationStatsRga() throws Exception { queryParams.putIfNotEmpty("populationFrequency", commandOptions.populationFrequency); queryParams.putIfNotEmpty("consequenceType", commandOptions.consequenceType); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getClinicalAnalysisClient().aggregationStatsRga(commandOptions.field, queryParams); @@ -716,8 +748,8 @@ private RestResponse queryRgaGene() throws Exception { queryParams.putIfNotEmpty("populationFrequency", commandOptions.populationFrequency); queryParams.putIfNotEmpty("consequenceType", commandOptions.consequenceType); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getClinicalAnalysisClient().queryRgaGene(queryParams); @@ -754,8 +786,8 @@ private RestResponse summaryRgaGene() throws Exception { queryParams.putIfNotEmpty("populationFrequency", commandOptions.populationFrequency); queryParams.putIfNotEmpty("consequenceType", commandOptions.consequenceType); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getClinicalAnalysisClient().summaryRgaGene(queryParams); @@ -774,13 +806,13 @@ private RestResponse runRgaIndex() throws Exception { queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); queryParams.putIfNotNull("auxiliarIndex", commandOptions.auxiliarIndex); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } RgaAnalysisParams rgaAnalysisParams = new RgaAnalysisParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(rgaAnalysisParams)); @@ -829,8 +861,8 @@ private RestResponse queryRgaIndividual() throws Exception queryParams.putIfNotEmpty("populationFrequency", commandOptions.populationFrequency); queryParams.putIfNotEmpty("consequenceType", commandOptions.consequenceType); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getClinicalAnalysisClient().queryRgaIndividual(queryParams); @@ -867,8 +899,8 @@ private RestResponse summaryRgaIndividual() throws queryParams.putIfNotEmpty("populationFrequency", commandOptions.populationFrequency); queryParams.putIfNotEmpty("consequenceType", commandOptions.consequenceType); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getClinicalAnalysisClient().summaryRgaIndividual(queryParams); @@ -910,8 +942,8 @@ private RestResponse queryRgaVariant() throws Exception { queryParams.putIfNotEmpty("populationFrequency", commandOptions.populationFrequency); queryParams.putIfNotEmpty("consequenceType", commandOptions.consequenceType); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getClinicalAnalysisClient().queryRgaVariant(queryParams); @@ -948,8 +980,8 @@ private RestResponse summaryRgaVariant() throws Except queryParams.putIfNotEmpty("populationFrequency", commandOptions.populationFrequency); queryParams.putIfNotEmpty("consequenceType", commandOptions.consequenceType); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getClinicalAnalysisClient().summaryRgaVariant(queryParams); @@ -992,8 +1024,8 @@ private RestResponse search() throws Exception { queryParams.putIfNotEmpty("status", commandOptions.status); queryParams.putIfNotEmpty("internalStatus", commandOptions.internalStatus); queryParams.putIfNotNull("deleted", commandOptions.deleted); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getClinicalAnalysisClient().search(queryParams); @@ -1008,8 +1040,8 @@ private RestResponse actionableVariant() throws Exception { ObjectMap queryParams = new ObjectMap(); queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotEmpty("sample", commandOptions.sample); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getClinicalAnalysisClient().actionableVariant(queryParams); @@ -1082,8 +1114,8 @@ private RestResponse queryVariant() throws Exception { queryParams.putIfNotEmpty("panelFeatureType", commandOptions.panelFeatureType); queryParams.putIfNotNull("panelIntersection", commandOptions.panelIntersection); queryParams.putIfNotEmpty("trait", commandOptions.trait); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getClinicalAnalysisClient().queryVariant(queryParams); @@ -1099,8 +1131,8 @@ private RestResponse acl() throws Exception { queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotEmpty("member", commandOptions.member); queryParams.putIfNotNull("silent", commandOptions.silent); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getClinicalAnalysisClient().acl(commandOptions.clinicalAnalyses, queryParams); @@ -1115,8 +1147,8 @@ private RestResponse delete() throws Exception { ObjectMap queryParams = new ObjectMap(); queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotNull("force", commandOptions.force); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getClinicalAnalysisClient().delete(commandOptions.clinicalAnalyses, queryParams); @@ -1133,8 +1165,8 @@ private RestResponse update() throws Exception { queryParams.putIfNotEmpty("exclude", commandOptions.exclude); queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotNull("includeResult", commandOptions.includeResult); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } @@ -1164,10 +1196,14 @@ private RestResponse update() throws Exception { FamilyParam familyParam= new FamilyParam(); invokeSetter(familyParam, "id", commandOptions.familyId); - ClinicalAnalysis.Type typeParam = ClinicalAnalysis.Type.valueOf(commandOptions.type); + ClinicalAnalysis.Type typeParam = null; + if (commandOptions.type != null) { + typeParam = ClinicalAnalysis.Type.valueOf(commandOptions.type); + + } ClinicalAnalysisUpdateParams clinicalAnalysisUpdateParams = new ClinicalAnalysisUpdateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(clinicalAnalysisUpdateParams)); @@ -1180,10 +1216,8 @@ private RestResponse update() throws Exception { .setId(commandOptions.id) .setDescription(commandOptions.description) .setDisorder(disorderReferenceParam) - .setPanelLock(commandOptions.panelLock) .setProband(probandParam) .setFamily(familyParam) - .setLocked(commandOptions.locked) .setAnalyst(clinicalAnalystParam) .setReport(clinicalReport) .setCreationDate(commandOptions.creationDate) @@ -1192,6 +1226,14 @@ private RestResponse update() throws Exception { .setPriority(priorityParam) .setStatus(statusParam); + if (commandOptions.panelLock != null){ + ((ClinicalAnalysisUpdateParams)clinicalAnalysisUpdateParams).setPanelLock(commandOptions.panelLock); + } + + if (commandOptions.locked != null){ + ((ClinicalAnalysisUpdateParams)clinicalAnalysisUpdateParams).setLocked(commandOptions.locked); + } + } return openCGAClient.getClinicalAnalysisClient().update(commandOptions.clinicalAnalyses, clinicalAnalysisUpdateParams, queryParams); } @@ -1207,8 +1249,8 @@ private RestResponse info() throws Exception { queryParams.putIfNotEmpty("exclude", commandOptions.exclude); queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotNull("deleted", commandOptions.deleted); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getClinicalAnalysisClient().info(commandOptions.clinicalAnalysis, queryParams); @@ -1226,8 +1268,8 @@ private RestResponse createInterpretation() throws Exception { queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotNull("setAs", commandOptions.setAs); queryParams.putIfNotNull("includeResult", commandOptions.includeResult); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } @@ -1243,7 +1285,7 @@ private RestResponse createInterpretation() throws Exception { invokeSetter(clinicalAnalystParam, "id", commandOptions.analystId); InterpretationCreateParams interpretationCreateParams = new InterpretationCreateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(interpretationCreateParams)); @@ -1265,6 +1307,21 @@ private RestResponse createInterpretation() throws Exception { return openCGAClient.getClinicalAnalysisClient().createInterpretation(commandOptions.clinicalAnalysis, interpretationCreateParams, queryParams); } + private RestResponse clearInterpretation() throws Exception { + + logger.debug("Executing clearInterpretation in Analysis - Clinical command line"); + + AnalysisClinicalCommandOptions.ClearInterpretationCommandOptions commandOptions = analysisClinicalCommandOptions.clearInterpretationCommandOptions; + + ObjectMap queryParams = new ObjectMap(); + queryParams.putIfNotEmpty("study", commandOptions.study); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + } + + return openCGAClient.getClinicalAnalysisClient().clearInterpretation(commandOptions.clinicalAnalysis, commandOptions.interpretations, queryParams); + } + private RestResponse deleteInterpretation() throws Exception { logger.debug("Executing deleteInterpretation in Analysis - Clinical command line"); @@ -1274,13 +1331,28 @@ private RestResponse deleteInterpretation() throws Exception { ObjectMap queryParams = new ObjectMap(); queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotEmpty("setAsPrimary", commandOptions.setAsPrimary); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getClinicalAnalysisClient().deleteInterpretation(commandOptions.clinicalAnalysis, commandOptions.interpretations, queryParams); } + private RestResponse revertInterpretation() throws Exception { + + logger.debug("Executing revertInterpretation in Analysis - Clinical command line"); + + AnalysisClinicalCommandOptions.RevertInterpretationCommandOptions commandOptions = analysisClinicalCommandOptions.revertInterpretationCommandOptions; + + ObjectMap queryParams = new ObjectMap(); + queryParams.putIfNotEmpty("study", commandOptions.study); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + } + + return openCGAClient.getClinicalAnalysisClient().revertInterpretation(commandOptions.clinicalAnalysis, commandOptions.interpretation, commandOptions.version, queryParams); + } + private RestResponse updateInterpretation() throws Exception { logger.debug("Executing updateInterpretation in Analysis - Clinical command line"); @@ -1293,8 +1365,8 @@ private RestResponse updateInterpretation() throws Exception { queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotNull("setAs", commandOptions.setAs); queryParams.putIfNotNull("includeResult", commandOptions.includeResult); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } @@ -1310,7 +1382,7 @@ private RestResponse updateInterpretation() throws Exception { invokeSetter(clinicalAnalystParam, "id", commandOptions.analystId); InterpretationUpdateParams interpretationUpdateParams = new InterpretationUpdateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(interpretationUpdateParams)); diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/AnalysisVariantCommandExecutor.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/AnalysisVariantCommandExecutor.java index 93823c34d0c..dbd17453bf0 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/AnalysisVariantCommandExecutor.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/AnalysisVariantCommandExecutor.java @@ -62,7 +62,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 +* Autogenerated on: 2022-03-25 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -71,7 +71,7 @@ /** * This class contains methods for the Analysis - Variant command line. - * OpenCGA version: 2.2.0-rc2-SNAPSHOT + * OpenCGA version: 2.2.0-SNAPSHOT * PATH: /{apiVersion}/analysis/variant */ public class AnalysisVariantCommandExecutor extends OpencgaCommandExecutor { @@ -257,8 +257,8 @@ private RestResponse aggregationStats() throws Exception { queryParams.putIfNotEmpty("customAnnotation", commandOptions.customAnnotation); queryParams.putIfNotEmpty("trait", commandOptions.trait); queryParams.putIfNotEmpty("field", commandOptions.field); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getVariantClient().aggregationStats(queryParams); @@ -303,13 +303,13 @@ private RestResponse runCircos() throws Exception { ObjectMap queryParams = new ObjectMap(); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } CircosAnalysisParams circosAnalysisParams = new CircosAnalysisParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(circosAnalysisParams)); @@ -336,8 +336,8 @@ private RestResponse deleteCohortStats() throws Exception { ObjectMap queryParams = new ObjectMap(); queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotEmpty("cohort", commandOptions.cohort); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getVariantClient().deleteCohortStats(queryParams); @@ -351,8 +351,8 @@ private RestResponse infoCohortStats() throws Exception { ObjectMap queryParams = new ObjectMap(); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getVariantClient().infoCohortStats(commandOptions.cohort, queryParams); @@ -370,13 +370,13 @@ private RestResponse runCohortStats() throws Exception { queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } CohortVariantStatsAnalysisParams cohortVariantStatsAnalysisParams = new CohortVariantStatsAnalysisParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(cohortVariantStatsAnalysisParams)); @@ -388,10 +388,13 @@ private RestResponse runCohortStats() throws Exception { ((CohortVariantStatsAnalysisParams)cohortVariantStatsAnalysisParams) .setCohort(commandOptions.cohort) .setSamples(splitWithTrim(commandOptions.samples)) - .setIndex(commandOptions.index) .setSampleAnnotation(commandOptions.sampleAnnotation) .setOutdir(commandOptions.outdir); + if (commandOptions.index != null){ + ((CohortVariantStatsAnalysisParams)cohortVariantStatsAnalysisParams).setIndex(commandOptions.index); + } + } return openCGAClient.getVariantClient().runCohortStats(cohortVariantStatsAnalysisParams, queryParams); } @@ -411,13 +414,13 @@ private RestResponse runExport() throws Exception { queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } VariantExportParams variantExportParams = new VariantExportParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(variantExportParams)); @@ -429,14 +432,12 @@ private RestResponse runExport() throws Exception { ((VariantExportParams)variantExportParams) .setOutdir(commandOptions.outdir) .setOutputFileName(commandOptions.outputFileName) - .setOutputFormat(commandOptions.outputFormat) + .setOutputFileFormat(commandOptions.outputFileFormat) .setVariantsFile(commandOptions.variantsFile) .setInclude(commandOptions.bodyInclude) .setExclude(commandOptions.bodyExclude) - .setCompress(commandOptions.compress) .setLimit(commandOptions.limit) .setSkip(commandOptions.skip) - .setSummary(commandOptions.summary) .setSavedFilter(commandOptions.savedFilter) .setChromosome(commandOptions.chromosome) .setReference(commandOptions.reference) @@ -468,7 +469,6 @@ private RestResponse runExport() throws Exception { .setCohortStatsMgf(commandOptions.cohortStatsMgf) .setMissingAlleles(commandOptions.missingAlleles) .setMissingGenotypes(commandOptions.missingGenotypes) - .setAnnotationExists(commandOptions.annotationExists) .setScore(commandOptions.score) .setPolyphen(commandOptions.polyphen) .setSift(commandOptions.sift) @@ -483,9 +483,27 @@ private RestResponse runExport() throws Exception { .setProteinKeyword(commandOptions.proteinKeyword) .setDrug(commandOptions.drug) .setCustomAnnotation(commandOptions.customAnnotation) - .setUnknownGenotype(commandOptions.unknownGenotype) - .setSampleMetadata(commandOptions.sampleMetadata) - .setSort(commandOptions.sort); + .setUnknownGenotype(commandOptions.unknownGenotype); + + if (commandOptions.compress != null){ + ((VariantExportParams)variantExportParams).setCompress(commandOptions.compress); + } + + if (commandOptions.summary != null){ + ((VariantExportParams)variantExportParams).setSummary(commandOptions.summary); + } + + if (commandOptions.annotationExists != null){ + ((VariantExportParams)variantExportParams).setAnnotationExists(commandOptions.annotationExists); + } + + if (commandOptions.sampleMetadata != null){ + ((VariantExportParams)variantExportParams).setSampleMetadata(commandOptions.sampleMetadata); + } + + if (commandOptions.sort != null){ + ((VariantExportParams)variantExportParams).setSort(commandOptions.sort); + } } return openCGAClient.getVariantClient().runExport(variantExportParams, queryParams); @@ -503,8 +521,8 @@ private RestResponse genotypesFamily() throws Exception { queryParams.putIfNotEmpty("clinicalAnalysis", commandOptions.clinicalAnalysis); queryParams.putIfNotNull("penetrance", commandOptions.penetrance); queryParams.putIfNotEmpty("disorder", commandOptions.disorder); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getVariantClient().genotypesFamily(commandOptions.modeOfInheritance, queryParams); @@ -522,13 +540,13 @@ private RestResponse runFamilyQc() throws Exception { queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } FamilyQcAnalysisParams familyQcAnalysisParams = new FamilyQcAnalysisParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(familyQcAnalysisParams)); @@ -561,8 +579,8 @@ private RestResponse deleteFile() throws Exception { queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotEmpty("file", commandOptions.file); queryParams.putIfNotNull("resume", commandOptions.resume); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getVariantClient().deleteFile(queryParams); @@ -580,13 +598,13 @@ private RestResponse runGatk() throws Exception { queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } GatkWrapperParams gatkWrapperParams = new GatkWrapperParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(gatkWrapperParams)); @@ -615,13 +633,13 @@ private RestResponse runGenomePlot() throws Exception { queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } GenomePlotAnalysisParams genomePlotAnalysisParams = new GenomePlotAnalysisParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(genomePlotAnalysisParams)); @@ -653,15 +671,23 @@ private RestResponse runGwas() throws Exception { queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } - GwasConfiguration.Method methodParam = GwasConfiguration.Method.valueOf(commandOptions.method); - GwasConfiguration.FisherMode fisherModeParam = GwasConfiguration.FisherMode.valueOf(commandOptions.fisherMode); + GwasConfiguration.Method methodParam = null; + if (commandOptions.method != null) { + methodParam = GwasConfiguration.Method.valueOf(commandOptions.method); + + } + GwasConfiguration.FisherMode fisherModeParam = null; + if (commandOptions.fisherMode != null) { + fisherModeParam = GwasConfiguration.FisherMode.valueOf(commandOptions.fisherMode); + + } GwasAnalysisParams gwasAnalysisParams = new GwasAnalysisParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(gwasAnalysisParams)); @@ -672,7 +698,6 @@ private RestResponse runGwas() throws Exception { } else { ((GwasAnalysisParams)gwasAnalysisParams) .setPhenotype(commandOptions.phenotype) - .setIndex(commandOptions.index) .setIndexScoreId(commandOptions.indexScoreId) .setCaseCohort(commandOptions.caseCohort) .setCaseCohortSamplesAnnotation(commandOptions.caseCohortSamplesAnnotation) @@ -682,6 +707,10 @@ private RestResponse runGwas() throws Exception { .setControlCohortSamples(splitWithTrim(commandOptions.controlCohortSamples)) .setOutdir(commandOptions.outdir); + if (commandOptions.index != null){ + ((GwasAnalysisParams)gwasAnalysisParams).setIndex(commandOptions.index); + } + } return openCGAClient.getVariantClient().runGwas(gwasAnalysisParams, queryParams); } @@ -698,13 +727,13 @@ private RestResponse runIndex() throws Exception { queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } VariantIndexParams variantIndexParams = new VariantIndexParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(variantIndexParams)); @@ -715,18 +744,10 @@ private RestResponse runIndex() throws Exception { } else { ((VariantIndexParams)variantIndexParams) .setFile(commandOptions.file) - .setResume(commandOptions.resume) .setOutdir(commandOptions.outdir) - .setTransform(commandOptions.transform) - .setGvcf(commandOptions.gvcf) - .setNormalizationSkip(commandOptions.normalizationSkip) .setReferenceGenome(commandOptions.referenceGenome) .setFailOnMalformedLines(commandOptions.failOnMalformedLines) - .setFamily(commandOptions.family) - .setSomatic(commandOptions.somatic) - .setLoad(commandOptions.load) .setLoadSplitData(commandOptions.loadSplitData) - .setLoadMultiFileData(commandOptions.loadMultiFileData) .setLoadSampleIndex(commandOptions.loadSampleIndex) .setLoadArchive(commandOptions.loadArchive) .setLoadHomRef(commandOptions.loadHomRef) @@ -735,13 +756,60 @@ private RestResponse runIndex() throws Exception { .setIncludeSampleData(commandOptions.includeSampleData) .setMerge(commandOptions.merge) .setDeduplicationPolicy(commandOptions.deduplicationPolicy) - .setCalculateStats(commandOptions.calculateStats) .setAggregationMappingFile(commandOptions.aggregationMappingFile) - .setAnnotate(commandOptions.annotate) - .setAnnotator(commandOptions.annotator) - .setOverwriteAnnotations(commandOptions.overwriteAnnotations) - .setIndexSearch(commandOptions.indexSearch) - .setSkipIndexedFiles(commandOptions.skipIndexedFiles); + .setAnnotator(commandOptions.annotator); + + if (commandOptions.resume != null){ + ((VariantIndexParams)variantIndexParams).setResume(commandOptions.resume); + } + + if (commandOptions.transform != null){ + ((VariantIndexParams)variantIndexParams).setTransform(commandOptions.transform); + } + + if (commandOptions.gvcf != null){ + ((VariantIndexParams)variantIndexParams).setGvcf(commandOptions.gvcf); + } + + if (commandOptions.normalizationSkip != null){ + ((VariantIndexParams)variantIndexParams).setNormalizationSkip(commandOptions.normalizationSkip); + } + + if (commandOptions.family != null){ + ((VariantIndexParams)variantIndexParams).setFamily(commandOptions.family); + } + + if (commandOptions.somatic != null){ + ((VariantIndexParams)variantIndexParams).setSomatic(commandOptions.somatic); + } + + if (commandOptions.load != null){ + ((VariantIndexParams)variantIndexParams).setLoad(commandOptions.load); + } + + if (commandOptions.loadMultiFileData != null){ + ((VariantIndexParams)variantIndexParams).setLoadMultiFileData(commandOptions.loadMultiFileData); + } + + if (commandOptions.calculateStats != null){ + ((VariantIndexParams)variantIndexParams).setCalculateStats(commandOptions.calculateStats); + } + + if (commandOptions.annotate != null){ + ((VariantIndexParams)variantIndexParams).setAnnotate(commandOptions.annotate); + } + + if (commandOptions.overwriteAnnotations != null){ + ((VariantIndexParams)variantIndexParams).setOverwriteAnnotations(commandOptions.overwriteAnnotations); + } + + if (commandOptions.indexSearch != null){ + ((VariantIndexParams)variantIndexParams).setIndexSearch(commandOptions.indexSearch); + } + + if (commandOptions.skipIndexedFiles != null){ + ((VariantIndexParams)variantIndexParams).setSkipIndexedFiles(commandOptions.skipIndexedFiles); + } } return openCGAClient.getVariantClient().runIndex(variantIndexParams, queryParams); @@ -759,13 +827,13 @@ private RestResponse runIndividualQc() throws Exception { queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } IndividualQcAnalysisParams individualQcAnalysisParams = new IndividualQcAnalysisParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(individualQcAnalysisParams)); @@ -796,13 +864,13 @@ private RestResponse runInferredSex() throws Exception { queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } InferredSexAnalysisParams inferredSexAnalysisParams = new InferredSexAnalysisParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(inferredSexAnalysisParams)); @@ -831,8 +899,8 @@ private RestResponse queryKnockoutGene() throws Exception { queryParams.putIfNotNull("skip", commandOptions.skip); queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotEmpty("job", commandOptions.job); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getVariantClient().queryKnockoutGene(queryParams); @@ -849,8 +917,8 @@ private RestResponse queryKnockoutIndividual() throws Exce queryParams.putIfNotNull("skip", commandOptions.skip); queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotEmpty("job", commandOptions.job); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getVariantClient().queryKnockoutIndividual(queryParams); @@ -868,13 +936,13 @@ private RestResponse runKnockout() throws Exception { queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } KnockoutAnalysisParams knockoutAnalysisParams = new KnockoutAnalysisParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(knockoutAnalysisParams)); @@ -891,9 +959,15 @@ private RestResponse runKnockout() throws Exception { .setConsequenceType(commandOptions.consequenceType) .setFilter(commandOptions.filter) .setQual(commandOptions.qual) - .setSkipGenesFile(commandOptions.skipGenesFile) - .setOutdir(commandOptions.outdir) - .setIndex(commandOptions.index); + .setOutdir(commandOptions.outdir); + + if (commandOptions.skipGenesFile != null){ + ((KnockoutAnalysisParams)knockoutAnalysisParams).setSkipGenesFile(commandOptions.skipGenesFile); + } + + if (commandOptions.index != null){ + ((KnockoutAnalysisParams)knockoutAnalysisParams).setIndex(commandOptions.index); + } } return openCGAClient.getVariantClient().runKnockout(knockoutAnalysisParams, queryParams); @@ -911,13 +985,13 @@ private RestResponse runMendelianError() throws Exception { queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } MendelianErrorAnalysisParams mendelianErrorAnalysisParams = new MendelianErrorAnalysisParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(mendelianErrorAnalysisParams)); @@ -952,8 +1026,8 @@ private RestResponse metadata() throws Exception { queryParams.putIfNotEmpty("includeSample", commandOptions.includeSample); queryParams.putIfNotEmpty("include", commandOptions.include); queryParams.putIfNotEmpty("exclude", commandOptions.exclude); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getVariantClient().metadata(queryParams); @@ -982,8 +1056,8 @@ private RestResponse queryMutationalSignature() throws Exception { queryParams.putIfNotEmpty("panelRoleInCancer", commandOptions.panelRoleInCancer); queryParams.putIfNotNull("panelIntersection", commandOptions.panelIntersection); queryParams.putIfNotNull("fitting", commandOptions.fitting); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getVariantClient().queryMutationalSignature(queryParams); @@ -1001,13 +1075,13 @@ private RestResponse runMutationalSignature() throws Exception { queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } MutationalSignatureAnalysisParams mutationalSignatureAnalysisParams = new MutationalSignatureAnalysisParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(mutationalSignatureAnalysisParams)); @@ -1021,9 +1095,12 @@ private RestResponse runMutationalSignature() throws Exception { .setId(commandOptions.id) .setDescription(commandOptions.description) .setRelease(commandOptions.release) - .setFitting(commandOptions.fitting) .setOutdir(commandOptions.outdir); + if (commandOptions.fitting != null){ + ((MutationalSignatureAnalysisParams)mutationalSignatureAnalysisParams).setFitting(commandOptions.fitting); + } + } return openCGAClient.getVariantClient().runMutationalSignature(mutationalSignatureAnalysisParams, queryParams); } @@ -1040,13 +1117,13 @@ private RestResponse runPlink() throws Exception { queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } PlinkWrapperParams plinkWrapperParams = new PlinkWrapperParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(plinkWrapperParams)); @@ -1146,8 +1223,8 @@ private RestResponse query() throws Exception { queryParams.putIfNotEmpty("panelFeatureType", commandOptions.panelFeatureType); queryParams.putIfNotNull("panelIntersection", commandOptions.panelIntersection); queryParams.putIfNotEmpty("trait", commandOptions.trait); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getVariantClient().query(queryParams); @@ -1165,13 +1242,13 @@ private RestResponse runRelatedness() throws Exception { queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } RelatednessAnalysisParams relatednessAnalysisParams = new RelatednessAnalysisParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(relatednessAnalysisParams)); @@ -1203,13 +1280,13 @@ private RestResponse runRvtests() throws Exception { queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } RvtestsWrapperParams rvtestsWrapperParams = new RvtestsWrapperParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(rvtestsWrapperParams)); @@ -1255,8 +1332,8 @@ private RestResponse aggregationStatsSample() throws Exception { queryParams.putIfNotEmpty("clinicalSignificance", commandOptions.clinicalSignificance); queryParams.putIfNotNull("clinicalConfirmedStatus", commandOptions.clinicalConfirmedStatus); queryParams.putIfNotEmpty("field", commandOptions.field); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getVariantClient().aggregationStatsSample(queryParams); @@ -1274,13 +1351,13 @@ private RestResponse runSampleEligibility() throws Exception { queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } SampleEligibilityAnalysisParams sampleEligibilityAnalysisParams = new SampleEligibilityAnalysisParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(sampleEligibilityAnalysisParams)); @@ -1291,9 +1368,12 @@ private RestResponse runSampleEligibility() throws Exception { } else { ((SampleEligibilityAnalysisParams)sampleEligibilityAnalysisParams) .setQuery(commandOptions.query) - .setIndex(commandOptions.index) .setCohortId(commandOptions.cohortId); + if (commandOptions.index != null){ + ((SampleEligibilityAnalysisParams)sampleEligibilityAnalysisParams).setIndex(commandOptions.index); + } + } return openCGAClient.getVariantClient().runSampleEligibility(sampleEligibilityAnalysisParams, queryParams); } @@ -1310,8 +1390,8 @@ private RestResponse runSampleQc() throws Exception { queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } @@ -1343,7 +1423,7 @@ private RestResponse runSampleQc() throws Exception { invokeSetter(annotationVariantQueryParams, "clinicalConfirmedStatus", commandOptions.variantStatsQueryClinicalConfirmedStatus); SampleQcAnalysisParams sampleQcAnalysisParams = new SampleQcAnalysisParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(sampleQcAnalysisParams)); @@ -1381,8 +1461,8 @@ private RestResponse querySample() throws Exception { queryParams.putIfNotEmpty("variant", commandOptions.variant); queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotEmpty("genotype", commandOptions.genotype); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getVariantClient().querySample(queryParams); @@ -1400,13 +1480,13 @@ private RestResponse runSample() throws Exception { queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } SampleVariantFilterParams sampleVariantFilterParams = new SampleVariantFilterParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(sampleVariantFilterParams)); @@ -1418,7 +1498,6 @@ private RestResponse runSample() throws Exception { ((SampleVariantFilterParams)sampleVariantFilterParams) .setGenotypes(splitWithTrim(commandOptions.genotypes)) .setSample(splitWithTrim(commandOptions.sample)) - .setSamplesInAllVariants(commandOptions.samplesInAllVariants) .setMaxVariants(commandOptions.maxVariants) .setId(commandOptions.id) .setRegion(commandOptions.region) @@ -1428,7 +1507,6 @@ private RestResponse runSample() throws Exception { .setPanelModeOfInheritance(commandOptions.panelModeOfInheritance) .setPanelConfidence(commandOptions.panelConfidence) .setPanelRoleInCancer(commandOptions.panelRoleInCancer) - .setPanelIntersection(commandOptions.panelIntersection) .setCohortStatsRef(commandOptions.cohortStatsRef) .setCohortStatsAlt(commandOptions.cohortStatsAlt) .setCohortStatsMaf(commandOptions.cohortStatsMaf) @@ -1446,6 +1524,14 @@ private RestResponse runSample() throws Exception { .setClinicalSignificance(commandOptions.clinicalSignificance) .setClinicalConfirmedStatus(commandOptions.clinicalConfirmedStatus); + if (commandOptions.samplesInAllVariants != null){ + ((SampleVariantFilterParams)sampleVariantFilterParams).setSamplesInAllVariants(commandOptions.samplesInAllVariants); + } + + if (commandOptions.panelIntersection != null){ + ((SampleVariantFilterParams)sampleVariantFilterParams).setPanelIntersection(commandOptions.panelIntersection); + } + } return openCGAClient.getVariantClient().runSample(sampleVariantFilterParams, queryParams); } @@ -1472,8 +1558,8 @@ private RestResponse querySampleStats() throws Exception { queryParams.putIfNotNull("clinicalConfirmedStatus", commandOptions.clinicalConfirmedStatus); queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotNull("filterTranscript", commandOptions.filterTranscript); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getVariantClient().querySampleStats(commandOptions.sample, queryParams); @@ -1491,13 +1577,13 @@ private RestResponse runSampleStats() throws Exception { queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } SampleVariantStatsAnalysisParams sampleVariantStatsAnalysisParams = new SampleVariantStatsAnalysisParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(sampleVariantStatsAnalysisParams)); @@ -1510,12 +1596,18 @@ private RestResponse runSampleStats() throws Exception { .setSample(splitWithTrim(commandOptions.sample)) .setIndividual(splitWithTrim(commandOptions.individual)) .setOutdir(commandOptions.outdir) - .setIndex(commandOptions.index) - .setIndexOverwrite(commandOptions.indexOverwrite) .setIndexId(commandOptions.indexId) .setIndexDescription(commandOptions.indexDescription) .setBatchSize(commandOptions.batchSize); + if (commandOptions.index != null){ + ((SampleVariantStatsAnalysisParams)sampleVariantStatsAnalysisParams).setIndex(commandOptions.index); + } + + if (commandOptions.indexOverwrite != null){ + ((SampleVariantStatsAnalysisParams)sampleVariantStatsAnalysisParams).setIndexOverwrite(commandOptions.indexOverwrite); + } + } return openCGAClient.getVariantClient().runSampleStats(sampleVariantStatsAnalysisParams, queryParams); } @@ -1533,13 +1625,13 @@ private RestResponse runStatsExport() throws Exception { queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } VariantStatsExportParams variantStatsExportParams = new VariantStatsExportParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(variantStatsExportParams)); @@ -1553,7 +1645,7 @@ private RestResponse runStatsExport() throws Exception { .setOutput(commandOptions.output) .setRegion(commandOptions.region) .setGene(commandOptions.gene) - .setOutputFormat(commandOptions.outputFormat); + .setOutputFileFormat(commandOptions.outputFileFormat); } return openCGAClient.getVariantClient().runStatsExport(variantStatsExportParams, queryParams); @@ -1571,13 +1663,13 @@ private RestResponse runStats() throws Exception { queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } VariantStatsAnalysisParams variantStatsAnalysisParams = new VariantStatsAnalysisParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(variantStatsAnalysisParams)); diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/CohortsCommandExecutor.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/CohortsCommandExecutor.java index 599cd077ffe..521faa0b5fc 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/CohortsCommandExecutor.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/CohortsCommandExecutor.java @@ -37,7 +37,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 +* Autogenerated on: 2022-03-25 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -46,7 +46,7 @@ /** * This class contains methods for the Cohorts command line. - * OpenCGA version: 2.2.0-rc2-SNAPSHOT + * OpenCGA version: 2.2.0-SNAPSHOT * PATH: /{apiVersion}/cohorts */ public class CohortsCommandExecutor extends OpencgaCommandExecutor { @@ -101,6 +101,9 @@ public void execute() throws Exception { case "update": queryResponse = update(); break; + case "annotationsets-annotations-update": + queryResponse = updateAnnotationSetsAnnotations(); + break; default: logger.error("Subcommand not valid"); break; @@ -118,13 +121,13 @@ private RestResponse updateAcl() throws Exception { ObjectMap queryParams = new ObjectMap(); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } CohortAclUpdateParams cohortAclUpdateParams = new CohortAclUpdateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(cohortAclUpdateParams)); @@ -160,8 +163,8 @@ private RestResponse aggregationStats() throws Exception { queryParams.putIfNotEmpty("annotation", commandOptions.annotation); queryParams.putIfNotNull("default_values", commandOptions.default_values); queryParams.putIfNotEmpty("field", commandOptions.field); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getCohortClient().aggregationStats(queryParams); @@ -177,13 +180,13 @@ private RestResponse loadAnnotationSets() throws Exception { queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotNull("parents", commandOptions.parents); queryParams.putIfNotEmpty("annotationSetId", commandOptions.annotationSetId); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } TsvAnnotationParams tsvAnnotationParams = new TsvAnnotationParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(tsvAnnotationParams)); @@ -212,8 +215,8 @@ private RestResponse create() throws Exception { queryParams.putIfNotEmpty("variableSet", commandOptions.variableSet); queryParams.putIfNotEmpty("variable", commandOptions.variable); queryParams.putIfNotNull("includeResult", commandOptions.includeResult); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } @@ -221,10 +224,14 @@ private RestResponse create() throws Exception { invokeSetter(statusParams, "id", commandOptions.statusId); invokeSetter(statusParams, "name", commandOptions.statusName); invokeSetter(statusParams, "description", commandOptions.statusDescription); - Enums.CohortType typeParam = Enums.CohortType.valueOf(commandOptions.type); + Enums.CohortType typeParam = null; + if (commandOptions.type != null) { + typeParam = Enums.CohortType.valueOf(commandOptions.type); + + } CohortCreateParams cohortCreateParams = new CohortCreateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(cohortCreateParams)); @@ -268,8 +275,8 @@ private RestResponse distinct() throws Exception { queryParams.putIfNotEmpty("samples", commandOptions.samples); queryParams.putIfNotEmpty("numSamples", commandOptions.numSamples); queryParams.putIfNotEmpty("release", commandOptions.release); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getCohortClient().distinct(commandOptions.field, queryParams); @@ -299,8 +306,8 @@ private RestResponse generate() throws Exception { queryParams.putIfNotEmpty("release", commandOptions.release); queryParams.putIfNotNull("snapshot", commandOptions.snapshot); queryParams.putIfNotNull("includeResult", commandOptions.includeResult); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } @@ -308,10 +315,14 @@ private RestResponse generate() throws Exception { invokeSetter(statusParams, "body_id", commandOptions.statusBodyId); invokeSetter(statusParams, "name", commandOptions.statusName); invokeSetter(statusParams, "description", commandOptions.statusDescription); - Enums.CohortType typeParam = Enums.CohortType.valueOf(commandOptions.type); + Enums.CohortType typeParam = null; + if (commandOptions.type != null) { + typeParam = Enums.CohortType.valueOf(commandOptions.type); + + } CohortGenerateParams cohortGenerateParams = new CohortGenerateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(cohortGenerateParams)); @@ -360,8 +371,8 @@ private RestResponse search() throws Exception { queryParams.putIfNotEmpty("samples", commandOptions.samples); queryParams.putIfNotEmpty("numSamples", commandOptions.numSamples); queryParams.putIfNotEmpty("release", commandOptions.release); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getCohortClient().search(queryParams); @@ -377,8 +388,8 @@ private RestResponse acl() throws Exception { queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotEmpty("member", commandOptions.member); queryParams.putIfNotNull("silent", commandOptions.silent); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getCohortClient().acl(commandOptions.cohorts, queryParams); @@ -392,8 +403,8 @@ private RestResponse delete() throws Exception { ObjectMap queryParams = new ObjectMap(); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getCohortClient().delete(commandOptions.cohorts, queryParams); @@ -411,8 +422,8 @@ private RestResponse info() throws Exception { queryParams.putIfNotNull("flattenAnnotations", commandOptions.flattenAnnotations); queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotNull("deleted", commandOptions.deleted); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getCohortClient().info(commandOptions.cohorts, queryParams); @@ -429,8 +440,8 @@ private RestResponse update() throws Exception { queryParams.putIfNotEmpty("exclude", commandOptions.exclude); queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotNull("includeResult", commandOptions.includeResult); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } @@ -438,10 +449,14 @@ private RestResponse update() throws Exception { invokeSetter(statusParams, "id", commandOptions.statusId); invokeSetter(statusParams, "name", commandOptions.statusName); invokeSetter(statusParams, "description", commandOptions.statusDescription); - Enums.CohortType typeParam = Enums.CohortType.valueOf(commandOptions.type); + Enums.CohortType typeParam = null; + if (commandOptions.type != null) { + typeParam = Enums.CohortType.valueOf(commandOptions.type); + + } CohortUpdateParams cohortUpdateParams = new CohortUpdateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(cohortUpdateParams)); @@ -461,4 +476,30 @@ private RestResponse update() throws Exception { } return openCGAClient.getCohortClient().update(commandOptions.cohorts, cohortUpdateParams, queryParams); } + + private RestResponse updateAnnotationSetsAnnotations() throws Exception { + + logger.debug("Executing updateAnnotationSetsAnnotations in Cohorts command line"); + + CohortsCommandOptions.UpdateAnnotationSetsAnnotationsCommandOptions commandOptions = cohortsCommandOptions.updateAnnotationSetsAnnotationsCommandOptions; + + ObjectMap queryParams = new ObjectMap(); + queryParams.putIfNotEmpty("study", commandOptions.study); + queryParams.putIfNotNull("action", commandOptions.action); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + } + + + ObjectMap objectMap = new ObjectMap(); + if (commandOptions.jsonDataModel) { + RestResponse res = new RestResponse<>(); + res.setType(QueryType.VOID); + PrintUtils.println(getObjectAsJSON(objectMap)); + return res; + } else if (commandOptions.jsonFile != null) { + ObjectMapper objectMapper = new ObjectMapper(); + objectMapper.writeValue(new java.io.File(commandOptions.jsonFile), objectMap); + } return openCGAClient.getCohortClient().updateAnnotationSetsAnnotations(commandOptions.cohort, commandOptions.annotationSet, objectMap, queryParams); + } } \ No newline at end of file diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/DiseasePanelsCommandExecutor.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/DiseasePanelsCommandExecutor.java index 67f2c7c1ccc..46743e09b10 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/DiseasePanelsCommandExecutor.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/DiseasePanelsCommandExecutor.java @@ -28,7 +28,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 +* Autogenerated on: 2022-03-25 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -37,7 +37,7 @@ /** * This class contains methods for the Disease Panels command line. - * OpenCGA version: 2.2.0-rc2-SNAPSHOT + * OpenCGA version: 2.2.0-SNAPSHOT * PATH: /{apiVersion}/panels */ public class DiseasePanelsCommandExecutor extends OpencgaCommandExecutor { @@ -100,13 +100,13 @@ private RestResponse updateAcl() throws Exception { ObjectMap queryParams = new ObjectMap(); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } PanelAclUpdateParams panelAclUpdateParams = new PanelAclUpdateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(panelAclUpdateParams)); @@ -136,13 +136,13 @@ private RestResponse create() throws Exception { queryParams.putIfNotEmpty("source", commandOptions.source); queryParams.putIfNotEmpty("id", commandOptions.id); queryParams.putIfNotNull("includeResult", commandOptions.includeResult); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } PanelCreateParams panelCreateParams = new PanelCreateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(panelCreateParams)); @@ -186,8 +186,8 @@ private RestResponse distinct() throws Exception { queryParams.putIfNotEmpty("acl", commandOptions.acl); queryParams.putIfNotEmpty("release", commandOptions.release); queryParams.putIfNotNull("snapshot", commandOptions.snapshot); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getDiseasePanelClient().distinct(commandOptions.field, queryParams); @@ -222,8 +222,8 @@ private RestResponse search() throws Exception { queryParams.putIfNotEmpty("acl", commandOptions.acl); queryParams.putIfNotEmpty("release", commandOptions.release); queryParams.putIfNotNull("snapshot", commandOptions.snapshot); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getDiseasePanelClient().search(queryParams); @@ -239,8 +239,8 @@ private RestResponse acl() throws Exception { queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotEmpty("member", commandOptions.member); queryParams.putIfNotNull("silent", commandOptions.silent); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getDiseasePanelClient().acl(commandOptions.panels, queryParams); @@ -254,8 +254,8 @@ private RestResponse delete() throws Exception { ObjectMap queryParams = new ObjectMap(); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getDiseasePanelClient().delete(commandOptions.panels, queryParams); @@ -273,8 +273,8 @@ private RestResponse info() throws Exception { queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotEmpty("version", commandOptions.version); queryParams.putIfNotNull("deleted", commandOptions.deleted); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getDiseasePanelClient().info(commandOptions.panels, queryParams); @@ -292,13 +292,13 @@ private RestResponse update() throws Exception { queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotNull("incVersion", commandOptions.incVersion); queryParams.putIfNotNull("includeResult", commandOptions.includeResult); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } PanelUpdateParams panelUpdateParams = new PanelUpdateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(panelUpdateParams)); diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/FamiliesCommandExecutor.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/FamiliesCommandExecutor.java index 0aadcec4e8e..820d8b30a7e 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/FamiliesCommandExecutor.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/FamiliesCommandExecutor.java @@ -36,7 +36,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 +* Autogenerated on: 2022-03-25 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -45,7 +45,7 @@ /** * This class contains methods for the Families command line. - * OpenCGA version: 2.2.0-rc2-SNAPSHOT + * OpenCGA version: 2.2.0-SNAPSHOT * PATH: /{apiVersion}/families */ public class FamiliesCommandExecutor extends OpencgaCommandExecutor { @@ -97,6 +97,9 @@ public void execute() throws Exception { case "update": queryResponse = update(); break; + case "annotationsets-annotations-update": + queryResponse = updateAnnotationSetsAnnotations(); + break; default: logger.error("Subcommand not valid"); break; @@ -115,13 +118,13 @@ private RestResponse updateAcl() throws Exception { ObjectMap queryParams = new ObjectMap(); queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotNull("propagate", commandOptions.propagate); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } FamilyAclUpdateParams familyAclUpdateParams = new FamilyAclUpdateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(familyAclUpdateParams)); @@ -161,8 +164,8 @@ private RestResponse aggregationStats() throws Exception { queryParams.putIfNotEmpty("annotation", commandOptions.annotation); queryParams.putIfNotNull("default_values", commandOptions.default_values); queryParams.putIfNotEmpty("field", commandOptions.field); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getFamilyClient().aggregationStats(queryParams); @@ -178,13 +181,13 @@ private RestResponse loadAnnotationSets() throws Exception { queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotNull("parents", commandOptions.parents); queryParams.putIfNotEmpty("annotationSetId", commandOptions.annotationSetId); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } TsvAnnotationParams tsvAnnotationParams = new TsvAnnotationParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(tsvAnnotationParams)); @@ -212,8 +215,8 @@ private RestResponse create() throws Exception { queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotEmpty("members", commandOptions.members); queryParams.putIfNotNull("includeResult", commandOptions.includeResult); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } @@ -223,7 +226,7 @@ private RestResponse create() throws Exception { invokeSetter(statusParams, "description", commandOptions.statusDescription); FamilyCreateParams familyCreateParams = new FamilyCreateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(familyCreateParams)); @@ -270,8 +273,8 @@ private RestResponse distinct() throws Exception { queryParams.putIfNotEmpty("acl", commandOptions.acl); queryParams.putIfNotEmpty("release", commandOptions.release); queryParams.putIfNotNull("snapshot", commandOptions.snapshot); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getFamilyClient().distinct(commandOptions.field, queryParams); @@ -308,8 +311,8 @@ private RestResponse search() throws Exception { queryParams.putIfNotEmpty("acl", commandOptions.acl); queryParams.putIfNotEmpty("release", commandOptions.release); queryParams.putIfNotNull("snapshot", commandOptions.snapshot); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getFamilyClient().search(queryParams); @@ -325,8 +328,8 @@ private RestResponse acl() throws Exception { queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotEmpty("member", commandOptions.member); queryParams.putIfNotNull("silent", commandOptions.silent); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getFamilyClient().acl(commandOptions.families, queryParams); @@ -340,8 +343,8 @@ private RestResponse delete() throws Exception { ObjectMap queryParams = new ObjectMap(); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getFamilyClient().delete(commandOptions.families, queryParams); @@ -360,8 +363,8 @@ private RestResponse info() throws Exception { queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotEmpty("version", commandOptions.version); queryParams.putIfNotNull("deleted", commandOptions.deleted); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getFamilyClient().info(commandOptions.families, queryParams); @@ -380,8 +383,8 @@ private RestResponse update() throws Exception { queryParams.putIfNotNull("incVersion", commandOptions.incVersion); queryParams.putIfNotNull("updateRoles", commandOptions.updateRoles); queryParams.putIfNotNull("includeResult", commandOptions.includeResult); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } @@ -391,7 +394,7 @@ private RestResponse update() throws Exception { invokeSetter(statusParams, "description", commandOptions.statusDescription); FamilyUpdateParams familyUpdateParams = new FamilyUpdateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(familyUpdateParams)); @@ -412,4 +415,31 @@ private RestResponse update() throws Exception { } return openCGAClient.getFamilyClient().update(commandOptions.families, familyUpdateParams, queryParams); } + + private RestResponse updateAnnotationSetsAnnotations() throws Exception { + + logger.debug("Executing updateAnnotationSetsAnnotations in Families command line"); + + FamiliesCommandOptions.UpdateAnnotationSetsAnnotationsCommandOptions commandOptions = familiesCommandOptions.updateAnnotationSetsAnnotationsCommandOptions; + + ObjectMap queryParams = new ObjectMap(); + queryParams.putIfNotEmpty("study", commandOptions.study); + queryParams.putIfNotNull("action", commandOptions.action); + queryParams.putIfNotNull("incVersion", commandOptions.incVersion); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + } + + + ObjectMap objectMap = new ObjectMap(); + if (commandOptions.jsonDataModel) { + RestResponse res = new RestResponse<>(); + res.setType(QueryType.VOID); + PrintUtils.println(getObjectAsJSON(objectMap)); + return res; + } else if (commandOptions.jsonFile != null) { + ObjectMapper objectMapper = new ObjectMapper(); + objectMapper.writeValue(new java.io.File(commandOptions.jsonFile), objectMap); + } return openCGAClient.getFamilyClient().updateAnnotationSetsAnnotations(commandOptions.family, commandOptions.annotationSet, objectMap, queryParams); + } } \ No newline at end of file diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/FilesCommandExecutor.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/FilesCommandExecutor.java index 269d509f0ca..2f28f5bc66e 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/FilesCommandExecutor.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/FilesCommandExecutor.java @@ -51,7 +51,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 +* Autogenerated on: 2022-03-25 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -60,7 +60,7 @@ /** * This class contains methods for the Files command line. - * OpenCGA version: 2.2.0-rc2-SNAPSHOT + * OpenCGA version: 2.2.0-SNAPSHOT * PATH: /{apiVersion}/files */ public class FilesCommandExecutor extends ParentFilesCommandExecutor { @@ -112,6 +112,9 @@ public void execute() throws Exception { case "search": queryResponse = search(); break; + case "upload": + queryResponse = upload(); + break; case "acl": queryResponse = acl(); break; @@ -127,6 +130,9 @@ public void execute() throws Exception { case "update": queryResponse = update(); break; + case "annotationsets-annotations-update": + queryResponse = updateAnnotationSetsAnnotations(); + break; case "download": queryResponse = download(); break; @@ -151,9 +157,6 @@ public void execute() throws Exception { case "tree": queryResponse = tree(); break; - case "upload": - queryResponse = upload(); - break; default: logger.error("Subcommand not valid"); break; @@ -171,13 +174,13 @@ private RestResponse updateAcl() throws Exception { ObjectMap queryParams = new ObjectMap(); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } FileAclUpdateParams fileAclUpdateParams = new FileAclUpdateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(fileAclUpdateParams)); @@ -222,8 +225,8 @@ private RestResponse aggregationStats() throws Exception { queryParams.putIfNotEmpty("annotation", commandOptions.annotation); queryParams.putIfNotNull("default_values", commandOptions.default_values); queryParams.putIfNotEmpty("field", commandOptions.field); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getFileClient().aggregationStats(queryParams); @@ -239,13 +242,13 @@ private RestResponse loadAnnotationSets() throws Exception { queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotNull("parents", commandOptions.parents); queryParams.putIfNotEmpty("annotationSetId", commandOptions.annotationSetId); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } TsvAnnotationParams tsvAnnotationParams = new TsvAnnotationParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(tsvAnnotationParams)); @@ -270,8 +273,8 @@ private RestResponse create() throws Exception { ObjectMap queryParams = new ObjectMap(); queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotNull("parents", commandOptions.parents); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } @@ -286,12 +289,24 @@ private RestResponse create() throws Exception { invokeSetter(software, "repository", commandOptions.softwareRepository); invokeSetter(software, "commit", commandOptions.softwareCommit); invokeSetter(software, "website", commandOptions.softwareWebsite); - File.Type typeParam = File.Type.valueOf(commandOptions.type); - File.Format formatParam = File.Format.valueOf(commandOptions.format); - File.Bioformat bioformatParam = File.Bioformat.valueOf(commandOptions.bioformat); + File.Type typeParam = null; + if (commandOptions.type != null) { + typeParam = File.Type.valueOf(commandOptions.type); + + } + File.Format formatParam = null; + if (commandOptions.format != null) { + formatParam = File.Format.valueOf(commandOptions.format); + + } + File.Bioformat bioformatParam = null; + if (commandOptions.bioformat != null) { + bioformatParam = File.Bioformat.valueOf(commandOptions.bioformat); + + } FileCreateParams fileCreateParams = new FileCreateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(fileCreateParams)); @@ -352,8 +367,8 @@ private RestResponse distinct() throws Exception { queryParams.putIfNotEmpty("acl", commandOptions.acl); queryParams.putIfNotNull("deleted", commandOptions.deleted); queryParams.putIfNotEmpty("release", commandOptions.release); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getFileClient().distinct(commandOptions.field, queryParams); @@ -371,13 +386,13 @@ private RestResponse fetch() throws Exception { queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } FileFetch fileFetch = new FileFetch(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(fileFetch)); @@ -403,8 +418,8 @@ private RestResponse link() throws Exception { ObjectMap queryParams = new ObjectMap(); queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotNull("parents", commandOptions.parents); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } @@ -414,7 +429,7 @@ private RestResponse link() throws Exception { invokeSetter(statusParams, "description", commandOptions.statusDescription); FileLinkParams fileLinkParams = new FileLinkParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(fileLinkParams)); @@ -447,13 +462,13 @@ private RestResponse runLink() throws Exception { queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } FileLinkToolParams fileLinkToolParams = new FileLinkToolParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(fileLinkToolParams)); @@ -465,8 +480,11 @@ private RestResponse runLink() throws Exception { ((FileLinkToolParams)fileLinkToolParams) .setUri(splitWithTrim(commandOptions.uri)) .setPath(commandOptions.path) - .setDescription(commandOptions.description) - .setParents(commandOptions.parents); + .setDescription(commandOptions.description); + + if (commandOptions.parents != null){ + ((FileLinkToolParams)fileLinkToolParams).setParents(commandOptions.parents); + } } return openCGAClient.getFileClient().runLink(fileLinkToolParams, queryParams); @@ -484,13 +502,13 @@ private RestResponse runPostlink() throws Exception { queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } PostLinkToolParams postLinkToolParams = new PostLinkToolParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(postLinkToolParams)); @@ -546,13 +564,21 @@ private RestResponse search() throws Exception { queryParams.putIfNotEmpty("acl", commandOptions.acl); queryParams.putIfNotNull("deleted", commandOptions.deleted); queryParams.putIfNotEmpty("release", commandOptions.release); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getFileClient().search(queryParams); } + protected RestResponse upload() throws Exception { + + logger.debug("Executing upload in Files command line"); + + return super.upload(); + + } + private RestResponse acl() throws Exception { logger.debug("Executing acl in Files command line"); @@ -563,8 +589,8 @@ private RestResponse acl() throws Exception { queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotEmpty("member", commandOptions.member); queryParams.putIfNotNull("silent", commandOptions.silent); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getFileClient().acl(commandOptions.files, queryParams); @@ -579,8 +605,8 @@ private RestResponse delete() throws Exception { ObjectMap queryParams = new ObjectMap(); queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotNull("skipTrash", commandOptions.skipTrash); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getFileClient().delete(commandOptions.files, queryParams); @@ -598,8 +624,8 @@ private RestResponse info() throws Exception { queryParams.putIfNotNull("flattenAnnotations", commandOptions.flattenAnnotations); queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotNull("deleted", commandOptions.deleted); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getFileClient().info(commandOptions.files, queryParams); @@ -613,8 +639,8 @@ private RestResponse unlink() throws Exception { ObjectMap queryParams = new ObjectMap(); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getFileClient().unlink(commandOptions.files, queryParams); @@ -631,8 +657,8 @@ private RestResponse update() throws Exception { queryParams.putIfNotEmpty("exclude", commandOptions.exclude); queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotNull("sampleIdsAction", commandOptions.sampleIdsAction); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } @@ -657,11 +683,19 @@ private RestResponse update() throws Exception { invokeSetter(software, "repository", commandOptions.softwareRepository); invokeSetter(software, "commit", commandOptions.softwareCommit); invokeSetter(software, "website", commandOptions.softwareWebsite); - File.Format formatParam = File.Format.valueOf(commandOptions.format); - File.Bioformat bioformatParam = File.Bioformat.valueOf(commandOptions.bioformat); + File.Format formatParam = null; + if (commandOptions.format != null) { + formatParam = File.Format.valueOf(commandOptions.format); + + } + File.Bioformat bioformatParam = null; + if (commandOptions.bioformat != null) { + bioformatParam = File.Bioformat.valueOf(commandOptions.bioformat); + + } FileUpdateParams fileUpdateParams = new FileUpdateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(fileUpdateParams)); @@ -687,6 +721,32 @@ private RestResponse update() throws Exception { return openCGAClient.getFileClient().update(commandOptions.files, fileUpdateParams, queryParams); } + private RestResponse updateAnnotationSetsAnnotations() throws Exception { + + logger.debug("Executing updateAnnotationSetsAnnotations in Files command line"); + + FilesCommandOptions.UpdateAnnotationSetsAnnotationsCommandOptions commandOptions = filesCommandOptions.updateAnnotationSetsAnnotationsCommandOptions; + + ObjectMap queryParams = new ObjectMap(); + queryParams.putIfNotEmpty("study", commandOptions.study); + queryParams.putIfNotNull("action", commandOptions.action); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + } + + + ObjectMap objectMap = new ObjectMap(); + if (commandOptions.jsonDataModel) { + RestResponse res = new RestResponse<>(); + res.setType(QueryType.VOID); + PrintUtils.println(getObjectAsJSON(objectMap)); + return res; + } else if (commandOptions.jsonFile != null) { + ObjectMapper objectMapper = new ObjectMapper(); + objectMapper.writeValue(new java.io.File(commandOptions.jsonFile), objectMap); + } return openCGAClient.getFileClient().updateAnnotationSetsAnnotations(commandOptions.file, commandOptions.annotationSet, objectMap, queryParams); + } + private RestResponse download() throws Exception { logger.debug("Executing download in Files command line"); @@ -695,8 +755,8 @@ private RestResponse download() throws Exception { ObjectMap queryParams = new ObjectMap(); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getFileClient().download(commandOptions.file, queryParams); @@ -713,8 +773,8 @@ private RestResponse grep() throws Exception { queryParams.putIfNotEmpty("pattern", commandOptions.pattern); queryParams.putIfNotNull("ignoreCase", commandOptions.ignoreCase); queryParams.putIfNotNull("maxCount", commandOptions.maxCount); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getFileClient().grep(commandOptions.file, queryParams); @@ -730,8 +790,8 @@ private RestResponse head() throws Exception { queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotNull("offset", commandOptions.offset); queryParams.putIfNotNull("lines", commandOptions.lines); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getFileClient().head(commandOptions.file, queryParams); @@ -745,8 +805,8 @@ private RestResponse image() throws Exception { ObjectMap queryParams = new ObjectMap(); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getFileClient().image(commandOptions.file, queryParams); @@ -760,8 +820,8 @@ private RestResponse refresh() throws Exception { ObjectMap queryParams = new ObjectMap(); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getFileClient().refresh(commandOptions.file, queryParams); @@ -776,8 +836,8 @@ private RestResponse tail() throws Exception { ObjectMap queryParams = new ObjectMap(); queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotNull("lines", commandOptions.lines); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getFileClient().tail(commandOptions.file, queryParams); @@ -796,8 +856,8 @@ private RestResponse list() throws Exception { queryParams.putIfNotNull("skip", commandOptions.skip); queryParams.putIfNotNull("count", commandOptions.count); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getFileClient().list(commandOptions.folder, queryParams); @@ -814,8 +874,8 @@ private RestResponse tree() throws Exception { queryParams.putIfNotEmpty("exclude", commandOptions.exclude); queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotNull("maxDepth", commandOptions.maxDepth); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getFileClient().tree(commandOptions.folder, queryParams); diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/IndividualsCommandExecutor.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/IndividualsCommandExecutor.java index c8f0d83497e..18095ce071e 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/IndividualsCommandExecutor.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/IndividualsCommandExecutor.java @@ -41,7 +41,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 +* Autogenerated on: 2022-03-25 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -50,7 +50,7 @@ /** * This class contains methods for the Individuals command line. - * OpenCGA version: 2.2.0-rc2-SNAPSHOT + * OpenCGA version: 2.2.0-SNAPSHOT * PATH: /{apiVersion}/individuals */ public class IndividualsCommandExecutor extends OpencgaCommandExecutor { @@ -102,6 +102,9 @@ public void execute() throws Exception { case "update": queryResponse = update(); break; + case "annotationsets-annotations-update": + queryResponse = updateAnnotationSetsAnnotations(); + break; case "relatives": queryResponse = relatives(); break; @@ -123,13 +126,13 @@ private RestResponse updateAcl() throws Exception { ObjectMap queryParams = new ObjectMap(); queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotNull("propagate", commandOptions.propagate); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } IndividualAclUpdateParams individualAclUpdateParams = new IndividualAclUpdateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(individualAclUpdateParams)); @@ -175,8 +178,8 @@ private RestResponse aggregationStats() throws Exception { queryParams.putIfNotEmpty("annotation", commandOptions.annotation); queryParams.putIfNotNull("default_values", commandOptions.default_values); queryParams.putIfNotEmpty("field", commandOptions.field); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getIndividualClient().aggregationStats(queryParams); @@ -192,13 +195,13 @@ private RestResponse loadAnnotationSets() throws Exception { queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotNull("parents", commandOptions.parents); queryParams.putIfNotEmpty("annotationSetId", commandOptions.annotationSetId); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } TsvAnnotationParams tsvAnnotationParams = new TsvAnnotationParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(tsvAnnotationParams)); @@ -226,8 +229,8 @@ private RestResponse create() throws Exception { queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotEmpty("samples", commandOptions.samples); queryParams.putIfNotNull("includeResult", commandOptions.includeResult); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } @@ -260,11 +263,19 @@ private RestResponse create() throws Exception { invokeSetter(ontologyTermAnnotation, "description", commandOptions.ethnicityDescription); invokeSetter(ontologyTermAnnotation, "source", commandOptions.ethnicitySource); invokeSetter(ontologyTermAnnotation, "url", commandOptions.ethnicityUrl); - IndividualProperty.KaryotypicSex karyotypicSexParam = IndividualProperty.KaryotypicSex.valueOf(commandOptions.karyotypicSex); - IndividualProperty.LifeStatus lifeStatusParam = IndividualProperty.LifeStatus.valueOf(commandOptions.lifeStatus); + IndividualProperty.KaryotypicSex karyotypicSexParam = null; + if (commandOptions.karyotypicSex != null) { + karyotypicSexParam = IndividualProperty.KaryotypicSex.valueOf(commandOptions.karyotypicSex); + + } + IndividualProperty.LifeStatus lifeStatusParam = null; + if (commandOptions.lifeStatus != null) { + lifeStatusParam = IndividualProperty.LifeStatus.valueOf(commandOptions.lifeStatus); + + } IndividualCreateParams individualCreateParams = new IndividualCreateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(individualCreateParams)); @@ -282,13 +293,16 @@ private RestResponse create() throws Exception { .setModificationDate(commandOptions.modificationDate) .setLocation(location) .setEthnicity(ontologyTermAnnotation) - .setParentalConsanguinity(commandOptions.parentalConsanguinity) .setPopulation(individualPopulation) .setDateOfBirth(commandOptions.dateOfBirth) .setKaryotypicSex(karyotypicSexParam) .setLifeStatus(lifeStatusParam) .setStatus(statusParams); + if (commandOptions.parentalConsanguinity != null){ + ((IndividualCreateParams)individualCreateParams).setParentalConsanguinity(commandOptions.parentalConsanguinity); + } + } return openCGAClient.getIndividualClient().create(individualCreateParams, queryParams); } @@ -326,8 +340,8 @@ private RestResponse distinct() throws Exception { queryParams.putIfNotEmpty("acl", commandOptions.acl); queryParams.putIfNotEmpty("release", commandOptions.release); queryParams.putIfNotNull("snapshot", commandOptions.snapshot); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getIndividualClient().distinct(commandOptions.field, queryParams); @@ -372,8 +386,8 @@ private RestResponse search() throws Exception { queryParams.putIfNotEmpty("acl", commandOptions.acl); queryParams.putIfNotEmpty("release", commandOptions.release); queryParams.putIfNotNull("snapshot", commandOptions.snapshot); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getIndividualClient().search(queryParams); @@ -389,8 +403,8 @@ private RestResponse acl() throws Exception { queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotEmpty("member", commandOptions.member); queryParams.putIfNotNull("silent", commandOptions.silent); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getIndividualClient().acl(commandOptions.individuals, queryParams); @@ -405,8 +419,8 @@ private RestResponse delete() throws Exception { ObjectMap queryParams = new ObjectMap(); queryParams.putIfNotNull("force", commandOptions.force); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getIndividualClient().delete(commandOptions.individuals, queryParams); @@ -425,8 +439,8 @@ private RestResponse info() throws Exception { queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotEmpty("version", commandOptions.version); queryParams.putIfNotNull("deleted", commandOptions.deleted); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getIndividualClient().info(commandOptions.individuals, queryParams); @@ -444,8 +458,8 @@ private RestResponse update() throws Exception { queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotNull("incVersion", commandOptions.incVersion); queryParams.putIfNotNull("includeResult", commandOptions.includeResult); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } @@ -478,11 +492,19 @@ private RestResponse update() throws Exception { invokeSetter(ontologyTermAnnotation, "description", commandOptions.ethnicityDescription); invokeSetter(ontologyTermAnnotation, "source", commandOptions.ethnicitySource); invokeSetter(ontologyTermAnnotation, "url", commandOptions.ethnicityUrl); - IndividualProperty.KaryotypicSex karyotypicSexParam = IndividualProperty.KaryotypicSex.valueOf(commandOptions.karyotypicSex); - IndividualProperty.LifeStatus lifeStatusParam = IndividualProperty.LifeStatus.valueOf(commandOptions.lifeStatus); + IndividualProperty.KaryotypicSex karyotypicSexParam = null; + if (commandOptions.karyotypicSex != null) { + karyotypicSexParam = IndividualProperty.KaryotypicSex.valueOf(commandOptions.karyotypicSex); + + } + IndividualProperty.LifeStatus lifeStatusParam = null; + if (commandOptions.lifeStatus != null) { + lifeStatusParam = IndividualProperty.LifeStatus.valueOf(commandOptions.lifeStatus); + + } IndividualUpdateParams individualUpdateParams = new IndividualUpdateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(individualUpdateParams)); @@ -498,17 +520,47 @@ private RestResponse update() throws Exception { .setMother(individualReferenceParam) .setCreationDate(commandOptions.creationDate) .setModificationDate(commandOptions.modificationDate) - .setParentalConsanguinity(commandOptions.parentalConsanguinity) .setLocation(location) .setEthnicity(ontologyTermAnnotation) .setPopulation(individualPopulation) .setDateOfBirth(commandOptions.dateOfBirth) .setStatus(statusParams); + if (commandOptions.parentalConsanguinity != null){ + ((IndividualUpdateParams)individualUpdateParams).setParentalConsanguinity(commandOptions.parentalConsanguinity); + } + } return openCGAClient.getIndividualClient().update(commandOptions.individuals, individualUpdateParams, queryParams); } + private RestResponse updateAnnotationSetsAnnotations() throws Exception { + + logger.debug("Executing updateAnnotationSetsAnnotations in Individuals command line"); + + IndividualsCommandOptions.UpdateAnnotationSetsAnnotationsCommandOptions commandOptions = individualsCommandOptions.updateAnnotationSetsAnnotationsCommandOptions; + + ObjectMap queryParams = new ObjectMap(); + queryParams.putIfNotEmpty("study", commandOptions.study); + queryParams.putIfNotNull("action", commandOptions.action); + queryParams.putIfNotNull("incVersion", commandOptions.incVersion); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + } + + + ObjectMap objectMap = new ObjectMap(); + if (commandOptions.jsonDataModel) { + RestResponse res = new RestResponse<>(); + res.setType(QueryType.VOID); + PrintUtils.println(getObjectAsJSON(objectMap)); + return res; + } else if (commandOptions.jsonFile != null) { + ObjectMapper objectMapper = new ObjectMapper(); + objectMapper.writeValue(new java.io.File(commandOptions.jsonFile), objectMap); + } return openCGAClient.getIndividualClient().updateAnnotationSetsAnnotations(commandOptions.individual, commandOptions.annotationSet, objectMap, queryParams); + } + private RestResponse relatives() throws Exception { logger.debug("Executing relatives in Individuals command line"); @@ -521,8 +573,8 @@ private RestResponse relatives() throws Exception { queryParams.putIfNotNull("flattenAnnotations", commandOptions.flattenAnnotations); queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotNull("degree", commandOptions.degree); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getIndividualClient().relatives(commandOptions.individual, queryParams); diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/JobsCommandExecutor.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/JobsCommandExecutor.java index ebd67dc735d..8637d5193b4 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/JobsCommandExecutor.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/JobsCommandExecutor.java @@ -16,6 +16,8 @@ import org.opencb.opencga.app.cli.main.options.JobsCommandOptions; +import org.opencb.opencga.app.cli.main.parent.ParentJobsCommandExecutor; + import org.opencb.opencga.core.models.job.Job; import org.opencb.opencga.core.models.job.JobRetryParams; import org.opencb.commons.datastore.core.FacetField; @@ -36,7 +38,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 +* Autogenerated on: 2022-03-25 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -45,15 +47,15 @@ /** * This class contains methods for the Jobs command line. - * OpenCGA version: 2.2.0-rc2-SNAPSHOT + * OpenCGA version: 2.2.0-SNAPSHOT * PATH: /{apiVersion}/jobs */ -public class JobsCommandExecutor extends OpencgaCommandExecutor { +public class JobsCommandExecutor extends ParentJobsCommandExecutor { private JobsCommandOptions jobsCommandOptions; public JobsCommandExecutor(JobsCommandOptions jobsCommandOptions) throws CatalogAuthenticationException { - super(jobsCommandOptions.commonCommandOptions); + super(jobsCommandOptions.commonCommandOptions,jobsCommandOptions); this.jobsCommandOptions = jobsCommandOptions; } @@ -106,6 +108,9 @@ public void execute() throws Exception { case "log-tail": queryResponse = tailLog(); break; + case "log": + queryResponse = log(); + break; default: logger.error("Subcommand not valid"); break; @@ -122,7 +127,7 @@ private RestResponse updateAcl() throws Exception { JobsCommandOptions.UpdateAclCommandOptions commandOptions = jobsCommandOptions.updateAclCommandOptions; JobAclUpdateParams jobAclUpdateParams = new JobAclUpdateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(jobAclUpdateParams)); @@ -164,8 +169,8 @@ private RestResponse aggregationStats() throws Exception { queryParams.putIfNotEmpty("release", commandOptions.release); queryParams.putIfNotNull("default_values", commandOptions.default_values); queryParams.putIfNotEmpty("field", commandOptions.field); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getJobClient().aggregationStats(queryParams); @@ -179,8 +184,8 @@ private RestResponse create() throws Exception { ObjectMap queryParams = new ObjectMap(); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } @@ -190,10 +195,14 @@ private RestResponse create() throws Exception { ExecutionResult executionResult= new ExecutionResult(); invokeSetter(executionResult, "id", commandOptions.resultId); - Enums.Priority priorityParam = Enums.Priority.valueOf(commandOptions.priority); + Enums.Priority priorityParam = null; + if (commandOptions.priority != null) { + priorityParam = Enums.Priority.valueOf(commandOptions.priority); + + } JobCreateParams jobCreateParams = new JobCreateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(jobCreateParams)); @@ -243,8 +252,8 @@ private RestResponse distinct() throws Exception { queryParams.putIfNotEmpty("acl", commandOptions.acl); queryParams.putIfNotEmpty("release", commandOptions.release); queryParams.putIfNotNull("deleted", commandOptions.deleted); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getJobClient().distinct(commandOptions.field, queryParams); @@ -262,13 +271,13 @@ private RestResponse retry() throws Exception { queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } JobRetryParams jobRetryParams = new JobRetryParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(jobRetryParams)); @@ -278,8 +287,11 @@ private RestResponse retry() throws Exception { objectMapper.writeValue(new java.io.File(commandOptions.jsonFile), jobRetryParams); } else { ((JobRetryParams)jobRetryParams) - .setJob(commandOptions.job) - .setForce(commandOptions.force); + .setJob(commandOptions.job); + + if (commandOptions.force != null){ + ((JobRetryParams)jobRetryParams).setForce(commandOptions.force); + } } return openCGAClient.getJobClient().retry(jobRetryParams, queryParams); @@ -316,31 +328,19 @@ private RestResponse search() throws Exception { queryParams.putIfNotEmpty("acl", commandOptions.acl); queryParams.putIfNotEmpty("release", commandOptions.release); queryParams.putIfNotNull("deleted", commandOptions.deleted); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getJobClient().search(queryParams); } - private RestResponse top() throws Exception { + protected RestResponse top() throws Exception { logger.debug("Executing top in Jobs command line"); - JobsCommandOptions.TopCommandOptions commandOptions = jobsCommandOptions.topCommandOptions; - - ObjectMap queryParams = new ObjectMap(); - queryParams.putIfNotNull("limit", commandOptions.limit); - queryParams.putIfNotEmpty("study", commandOptions.study); - queryParams.putIfNotEmpty("internalStatus", commandOptions.internalStatus); - queryParams.putIfNotEmpty("priority", commandOptions.priority); - queryParams.putIfNotEmpty("userId", commandOptions.userId); - queryParams.putIfNotEmpty("toolId", commandOptions.toolId); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); - } + return super.top(); - return openCGAClient.getJobClient().top(queryParams); } private RestResponse acl() throws Exception { @@ -364,8 +364,8 @@ private RestResponse delete() throws Exception { ObjectMap queryParams = new ObjectMap(); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getJobClient().delete(commandOptions.jobs, queryParams); @@ -382,8 +382,8 @@ private RestResponse info() throws Exception { queryParams.putIfNotEmpty("exclude", commandOptions.exclude); queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotNull("deleted", commandOptions.deleted); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getJobClient().info(commandOptions.jobs, queryParams); @@ -400,13 +400,13 @@ private RestResponse update() throws Exception { queryParams.putIfNotEmpty("exclude", commandOptions.exclude); queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotNull("includeResult", commandOptions.includeResult); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } JobUpdateParams jobUpdateParams = new JobUpdateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(jobUpdateParams)); @@ -417,8 +417,11 @@ private RestResponse update() throws Exception { } else { ((JobUpdateParams)jobUpdateParams) .setDescription(commandOptions.description) - .setTags(splitWithTrim(commandOptions.tags)) - .setVisited(commandOptions.visited); + .setTags(splitWithTrim(commandOptions.tags)); + + if (commandOptions.visited != null){ + ((JobUpdateParams)jobUpdateParams).setVisited(commandOptions.visited); + } } return openCGAClient.getJobClient().update(commandOptions.jobs, jobUpdateParams, queryParams); @@ -435,8 +438,8 @@ private RestResponse headLog() throws Exception { queryParams.putIfNotNull("offset", commandOptions.offset); queryParams.putIfNotNull("lines", commandOptions.lines); queryParams.putIfNotEmpty("type", commandOptions.type); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getJobClient().headLog(commandOptions.job, queryParams); @@ -452,8 +455,8 @@ private RestResponse tailLog() throws Exception { queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotNull("lines", commandOptions.lines); queryParams.putIfNotEmpty("type", commandOptions.type); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getJobClient().tailLog(commandOptions.job, queryParams); diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/MetaCommandExecutor.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/MetaCommandExecutor.java index ef8ea4b19f5..7f6de7866d2 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/MetaCommandExecutor.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/MetaCommandExecutor.java @@ -24,7 +24,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 +* Autogenerated on: 2022-03-25 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -33,7 +33,7 @@ /** * This class contains methods for the Meta command line. - * OpenCGA version: 2.2.0-rc2-SNAPSHOT + * OpenCGA version: 2.2.0-SNAPSHOT * PATH: /{apiVersion}/meta */ public class MetaCommandExecutor extends OpencgaCommandExecutor { diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/OpencgaCommandExecutor.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/OpencgaCommandExecutor.java index 276ae6e141d..55d7d904f3f 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/OpencgaCommandExecutor.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/OpencgaCommandExecutor.java @@ -20,6 +20,7 @@ import com.fasterxml.jackson.databind.ObjectMapper; import org.apache.commons.lang3.StringUtils; import org.opencb.commons.datastore.core.ObjectMap; +import org.opencb.commons.utils.DataModelsUtils; import org.opencb.opencga.app.cli.CommandExecutor; import org.opencb.opencga.app.cli.GeneralCliOptions; import org.opencb.opencga.app.cli.main.io.*; @@ -33,7 +34,6 @@ import org.slf4j.LoggerFactory; import java.io.IOException; -import java.lang.reflect.Field; import java.lang.reflect.Method; import java.nio.charset.StandardCharsets; import java.util.ArrayList; @@ -70,10 +70,14 @@ public static List splitWithTrim(String value) { } public static List splitWithTrim(String value, String separator) { - String[] splitFields = value.split(separator); - List result = new ArrayList<>(splitFields.length); - for (String s : splitFields) { - result.add(s.trim()); + List result = null; + if (value != null) { + String[] splitFields = value.split(separator); + + result = new ArrayList<>(splitFields.length); + for (String s : splitFields) { + result.add(s.trim()); + } } return result; } @@ -208,24 +212,12 @@ public OpencgaCommandExecutor setOpenCGAClient(OpenCGAClient openCGAClient) { } public String getObjectAsJSON(Object o) throws Exception { - ObjectMapper mapper = new ObjectMapper(); - retrieveDeepObject(o); - //Convert object to JSON string and pretty print - String jsonInString = mapper.writerWithDefaultPrettyPrinter().writeValueAsString(o); - return jsonInString; - } - - private void retrieveDeepObject(Object o) { + String jsonInString = "Data model not found."; try { - Field[] fields = o.getClass().getDeclaredFields(); - for (Field field : fields) { - if (!field.getType().isPrimitive()) { - Class cl = Class.forName(field.getType().getCanonicalName()); - invokeSetter(o, field.getName(), cl.newInstance()); - } - } - } catch (Exception e) { - + jsonInString = DataModelsUtils.dataModelToJsonString(o.getClass()); + } catch (Throwable e) { + e.printStackTrace(); } + return jsonInString; } } diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/OperationsVariantStorageCommandExecutor.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/OperationsVariantStorageCommandExecutor.java index 06b6175f286..39ed37108d3 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/OperationsVariantStorageCommandExecutor.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/OperationsVariantStorageCommandExecutor.java @@ -47,7 +47,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 +* Autogenerated on: 2022-03-25 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -56,7 +56,7 @@ /** * This class contains methods for the Operations - Variant Storage command line. - * OpenCGA version: 2.2.0-rc2-SNAPSHOT + * OpenCGA version: 2.2.0-SNAPSHOT * PATH: /{apiVersion}/operation */ public class OperationsVariantStorageCommandExecutor extends OpencgaCommandExecutor { @@ -126,6 +126,9 @@ public void execute() throws Exception { case "variant-sample-index": queryResponse = indexVariantSample(); break; + case "variant-sample-index-configure": + queryResponse = variantSampleIndexConfigure(); + break; case "variant-score-delete": queryResponse = deleteVariantScore(); break; @@ -169,7 +172,7 @@ private RestResponse configureCellbase() throws Exception { CellBaseConfiguration cellBaseConfiguration = new CellBaseConfiguration(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(cellBaseConfiguration)); @@ -198,13 +201,13 @@ private RestResponse aggregateVariant() throws Exception { queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } VariantAggregateParams variantAggregateParams = new VariantAggregateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(variantAggregateParams)); @@ -212,10 +215,15 @@ private RestResponse aggregateVariant() throws Exception { } else if (commandOptions.jsonFile != null) { ObjectMapper objectMapper = new ObjectMapper(); objectMapper.writeValue(new java.io.File(commandOptions.jsonFile), variantAggregateParams); - } else { - ((VariantAggregateParams)variantAggregateParams) - .setOverwrite(commandOptions.overwrite) - .setResume(commandOptions.resume); + } else {; + + if (commandOptions.overwrite != null){ + ((VariantAggregateParams)variantAggregateParams).setOverwrite(commandOptions.overwrite); + } + + if (commandOptions.resume != null){ + ((VariantAggregateParams)variantAggregateParams).setResume(commandOptions.resume); + } } return openCGAClient.getVariantOperationClient().aggregateVariant(variantAggregateParams, queryParams); @@ -251,13 +259,13 @@ private RestResponse indexVariantAnnotation() throws Exception { queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); queryParams.putIfNotEmpty("project", commandOptions.project); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } VariantAnnotationIndexParams variantAnnotationIndexParams = new VariantAnnotationIndexParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(variantAnnotationIndexParams)); @@ -270,12 +278,18 @@ private RestResponse indexVariantAnnotation() throws Exception { .setOutdir(commandOptions.outdir) .setOutputFileName(commandOptions.outputFileName) .setAnnotator(commandOptions.annotator) - .setOverwriteAnnotations(commandOptions.overwriteAnnotations) .setRegion(commandOptions.region) - .setCreate(commandOptions.create) .setLoad(commandOptions.load) .setCustomName(commandOptions.customName); + if (commandOptions.overwriteAnnotations != null){ + ((VariantAnnotationIndexParams)variantAnnotationIndexParams).setOverwriteAnnotations(commandOptions.overwriteAnnotations); + } + + if (commandOptions.create != null){ + ((VariantAnnotationIndexParams)variantAnnotationIndexParams).setCreate(commandOptions.create); + } + } return openCGAClient.getVariantOperationClient().indexVariantAnnotation(variantAnnotationIndexParams, queryParams); } @@ -295,7 +309,7 @@ private RestResponse saveVariantAnnotation() throws Exception { VariantAnnotationSaveParams variantAnnotationSaveParams = new VariantAnnotationSaveParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(variantAnnotationSaveParams)); @@ -320,13 +334,13 @@ private RestResponse configureVariant() throws Exception { ObjectMap queryParams = new ObjectMap(); queryParams.putIfNotEmpty("project", commandOptions.project); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } ObjectMap objectMap = new ObjectMap(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(objectMap)); @@ -349,13 +363,13 @@ private RestResponse deleteVariant() throws Exception { queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } VariantFileDeleteParams variantFileDeleteParams = new VariantFileDeleteParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(variantFileDeleteParams)); @@ -365,8 +379,11 @@ private RestResponse deleteVariant() throws Exception { objectMapper.writeValue(new java.io.File(commandOptions.jsonFile), variantFileDeleteParams); } else { ((VariantFileDeleteParams)variantFileDeleteParams) - .setFile(splitWithTrim(commandOptions.file)) - .setResume(commandOptions.resume); + .setFile(splitWithTrim(commandOptions.file)); + + if (commandOptions.resume != null){ + ((VariantFileDeleteParams)variantFileDeleteParams).setResume(commandOptions.resume); + } } return openCGAClient.getVariantOperationClient().deleteVariant(variantFileDeleteParams, queryParams); @@ -384,13 +401,13 @@ private RestResponse aggregateVariantFamily() throws Exception { queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } VariantAggregateFamilyParams variantAggregateFamilyParams = new VariantAggregateFamilyParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(variantAggregateFamilyParams)); @@ -401,8 +418,11 @@ private RestResponse aggregateVariantFamily() throws Exception { } else { ((VariantAggregateFamilyParams)variantAggregateFamilyParams) .setSamples(splitWithTrim(commandOptions.samples)) - .setGapsGenotype(commandOptions.gapsGenotype) - .setResume(commandOptions.resume); + .setGapsGenotype(commandOptions.gapsGenotype); + + if (commandOptions.resume != null){ + ((VariantAggregateFamilyParams)variantAggregateFamilyParams).setResume(commandOptions.resume); + } } return openCGAClient.getVariantOperationClient().aggregateVariantFamily(variantAggregateFamilyParams, queryParams); @@ -420,13 +440,13 @@ private RestResponse indexVariantFamily() throws Exception { queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } VariantFamilyIndexParams variantFamilyIndexParams = new VariantFamilyIndexParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(variantFamilyIndexParams)); @@ -436,9 +456,19 @@ private RestResponse indexVariantFamily() throws Exception { objectMapper.writeValue(new java.io.File(commandOptions.jsonFile), variantFamilyIndexParams); } else { ((VariantFamilyIndexParams)variantFamilyIndexParams) - .setFamily(splitWithTrim(commandOptions.family)) - .setOverwrite(commandOptions.overwrite) - .setSkipIncompleteFamilies(commandOptions.skipIncompleteFamilies); + .setFamily(splitWithTrim(commandOptions.family)); + + if (commandOptions.overwrite != null){ + ((VariantFamilyIndexParams)variantFamilyIndexParams).setOverwrite(commandOptions.overwrite); + } + + if (commandOptions.updateIndex != null){ + ((VariantFamilyIndexParams)variantFamilyIndexParams).setUpdateIndex(commandOptions.updateIndex); + } + + if (commandOptions.skipIncompleteFamilies != null){ + ((VariantFamilyIndexParams)variantFamilyIndexParams).setSkipIncompleteFamilies(commandOptions.skipIncompleteFamilies); + } } return openCGAClient.getVariantOperationClient().indexVariantFamily(variantFamilyIndexParams, queryParams); @@ -456,13 +486,13 @@ private RestResponse indexVariant() throws Exception { queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } VariantIndexParams variantIndexParams = new VariantIndexParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(variantIndexParams)); @@ -473,18 +503,10 @@ private RestResponse indexVariant() throws Exception { } else { ((VariantIndexParams)variantIndexParams) .setFile(commandOptions.file) - .setResume(commandOptions.resume) .setOutdir(commandOptions.outdir) - .setTransform(commandOptions.transform) - .setGvcf(commandOptions.gvcf) - .setNormalizationSkip(commandOptions.normalizationSkip) .setReferenceGenome(commandOptions.referenceGenome) .setFailOnMalformedLines(commandOptions.failOnMalformedLines) - .setFamily(commandOptions.family) - .setSomatic(commandOptions.somatic) - .setLoad(commandOptions.load) .setLoadSplitData(commandOptions.loadSplitData) - .setLoadMultiFileData(commandOptions.loadMultiFileData) .setLoadSampleIndex(commandOptions.loadSampleIndex) .setLoadArchive(commandOptions.loadArchive) .setLoadHomRef(commandOptions.loadHomRef) @@ -493,13 +515,60 @@ private RestResponse indexVariant() throws Exception { .setIncludeSampleData(commandOptions.includeSampleData) .setMerge(commandOptions.merge) .setDeduplicationPolicy(commandOptions.deduplicationPolicy) - .setCalculateStats(commandOptions.calculateStats) .setAggregationMappingFile(commandOptions.aggregationMappingFile) - .setAnnotate(commandOptions.annotate) - .setAnnotator(commandOptions.annotator) - .setOverwriteAnnotations(commandOptions.overwriteAnnotations) - .setIndexSearch(commandOptions.indexSearch) - .setSkipIndexedFiles(commandOptions.skipIndexedFiles); + .setAnnotator(commandOptions.annotator); + + if (commandOptions.resume != null){ + ((VariantIndexParams)variantIndexParams).setResume(commandOptions.resume); + } + + if (commandOptions.transform != null){ + ((VariantIndexParams)variantIndexParams).setTransform(commandOptions.transform); + } + + if (commandOptions.gvcf != null){ + ((VariantIndexParams)variantIndexParams).setGvcf(commandOptions.gvcf); + } + + if (commandOptions.normalizationSkip != null){ + ((VariantIndexParams)variantIndexParams).setNormalizationSkip(commandOptions.normalizationSkip); + } + + if (commandOptions.family != null){ + ((VariantIndexParams)variantIndexParams).setFamily(commandOptions.family); + } + + if (commandOptions.somatic != null){ + ((VariantIndexParams)variantIndexParams).setSomatic(commandOptions.somatic); + } + + if (commandOptions.load != null){ + ((VariantIndexParams)variantIndexParams).setLoad(commandOptions.load); + } + + if (commandOptions.loadMultiFileData != null){ + ((VariantIndexParams)variantIndexParams).setLoadMultiFileData(commandOptions.loadMultiFileData); + } + + if (commandOptions.calculateStats != null){ + ((VariantIndexParams)variantIndexParams).setCalculateStats(commandOptions.calculateStats); + } + + if (commandOptions.annotate != null){ + ((VariantIndexParams)variantIndexParams).setAnnotate(commandOptions.annotate); + } + + if (commandOptions.overwriteAnnotations != null){ + ((VariantIndexParams)variantIndexParams).setOverwriteAnnotations(commandOptions.overwriteAnnotations); + } + + if (commandOptions.indexSearch != null){ + ((VariantIndexParams)variantIndexParams).setIndexSearch(commandOptions.indexSearch); + } + + if (commandOptions.skipIndexedFiles != null){ + ((VariantIndexParams)variantIndexParams).setSkipIndexedFiles(commandOptions.skipIndexedFiles); + } } return openCGAClient.getVariantOperationClient().indexVariant(variantIndexParams, queryParams); @@ -517,8 +586,8 @@ private RestResponse launcherVariantIndex() throws Exception { queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } @@ -553,7 +622,7 @@ private RestResponse launcherVariantIndex() throws Exception { invokeSetter(variantIndexParams, "skipIndexedFiles", commandOptions.indexParamsSkipIndexedFiles); VariantFileIndexJobLauncherParams variantFileIndexJobLauncherParams = new VariantFileIndexJobLauncherParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(variantFileIndexJobLauncherParams)); @@ -565,11 +634,17 @@ private RestResponse launcherVariantIndex() throws Exception { ((VariantFileIndexJobLauncherParams)variantFileIndexJobLauncherParams) .setName(commandOptions.name) .setDirectory(commandOptions.directory) - .setResumeFailed(commandOptions.resumeFailed) - .setIgnoreFailed(commandOptions.ignoreFailed) .setMaxJobs(commandOptions.maxJobs) .setIndexParams(variantIndexParams); + if (commandOptions.resumeFailed != null){ + ((VariantFileIndexJobLauncherParams)variantFileIndexJobLauncherParams).setResumeFailed(commandOptions.resumeFailed); + } + + if (commandOptions.ignoreFailed != null){ + ((VariantFileIndexJobLauncherParams)variantFileIndexJobLauncherParams).setIgnoreFailed(commandOptions.ignoreFailed); + } + } return openCGAClient.getVariantOperationClient().launcherVariantIndex(variantFileIndexJobLauncherParams, queryParams); } @@ -589,7 +664,7 @@ private RestResponse runVariantJulie() throws Exception { JulieParams julieParams = new JulieParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(julieParams)); @@ -600,8 +675,11 @@ private RestResponse runVariantJulie() throws Exception { } else { ((JulieParams)julieParams) .setCohorts(splitWithTrim(commandOptions.cohorts)) - .setRegion(commandOptions.region) - .setOverwrite(commandOptions.overwrite); + .setRegion(commandOptions.region); + + if (commandOptions.overwrite != null){ + ((JulieParams)julieParams).setOverwrite(commandOptions.overwrite); + } } return openCGAClient.getVariantOperationClient().runVariantJulie(julieParams, queryParams); @@ -621,7 +699,7 @@ private RestResponse repairVariantMetadata() throws Exception { VariantStorageMetadataRepairToolParams variantStorageMetadataRepairToolParams = new VariantStorageMetadataRepairToolParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(variantStorageMetadataRepairToolParams)); @@ -650,13 +728,13 @@ private RestResponse synchronizeVariantMetadata() throws Exception { queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } VariantStorageMetadataSynchronizeParams variantStorageMetadataSynchronizeParams = new VariantStorageMetadataSynchronizeParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(variantStorageMetadataSynchronizeParams)); @@ -685,13 +763,13 @@ private RestResponse deleteVariantSample() throws Exception { queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } VariantSampleDeleteParams variantSampleDeleteParams = new VariantSampleDeleteParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(variantSampleDeleteParams)); @@ -701,9 +779,15 @@ private RestResponse deleteVariantSample() throws Exception { objectMapper.writeValue(new java.io.File(commandOptions.jsonFile), variantSampleDeleteParams); } else { ((VariantSampleDeleteParams)variantSampleDeleteParams) - .setSample(splitWithTrim(commandOptions.sample)) - .setForce(commandOptions.force) - .setResume(commandOptions.resume); + .setSample(splitWithTrim(commandOptions.sample)); + + if (commandOptions.force != null){ + ((VariantSampleDeleteParams)variantSampleDeleteParams).setForce(commandOptions.force); + } + + if (commandOptions.resume != null){ + ((VariantSampleDeleteParams)variantSampleDeleteParams).setResume(commandOptions.resume); + } } return openCGAClient.getVariantOperationClient().deleteVariantSample(variantSampleDeleteParams, queryParams); @@ -721,13 +805,13 @@ private RestResponse indexVariantSample() throws Exception { queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } VariantSampleIndexParams variantSampleIndexParams = new VariantSampleIndexParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(variantSampleIndexParams)); @@ -737,15 +821,50 @@ private RestResponse indexVariantSample() throws Exception { objectMapper.writeValue(new java.io.File(commandOptions.jsonFile), variantSampleIndexParams); } else { ((VariantSampleIndexParams)variantSampleIndexParams) - .setSample(splitWithTrim(commandOptions.sample)) - .setBuildIndex(commandOptions.buildIndex) - .setAnnotate(commandOptions.annotate) - .setOverwrite(commandOptions.overwrite); + .setSample(splitWithTrim(commandOptions.sample)); + + if (commandOptions.buildIndex != null){ + ((VariantSampleIndexParams)variantSampleIndexParams).setBuildIndex(commandOptions.buildIndex); + } + + if (commandOptions.annotate != null){ + ((VariantSampleIndexParams)variantSampleIndexParams).setAnnotate(commandOptions.annotate); + } + + if (commandOptions.overwrite != null){ + ((VariantSampleIndexParams)variantSampleIndexParams).setOverwrite(commandOptions.overwrite); + } } return openCGAClient.getVariantOperationClient().indexVariantSample(variantSampleIndexParams, queryParams); } + private RestResponse variantSampleIndexConfigure() throws Exception { + + logger.debug("Executing variantSampleIndexConfigure in Operations - Variant Storage command line"); + + OperationsVariantStorageCommandOptions.VariantSampleIndexConfigureCommandOptions commandOptions = operationsVariantStorageCommandOptions.variantSampleIndexConfigureCommandOptions; + + ObjectMap queryParams = new ObjectMap(); + queryParams.putIfNotEmpty("study", commandOptions.study); + queryParams.putIfNotNull("skipRebuild", commandOptions.skipRebuild); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + } + + + SampleIndexConfiguration sampleIndexConfiguration = new SampleIndexConfiguration(); + if (commandOptions.jsonDataModel) { + RestResponse res = new RestResponse<>(); + res.setType(QueryType.VOID); + PrintUtils.println(getObjectAsJSON(sampleIndexConfiguration)); + return res; + } else if (commandOptions.jsonFile != null) { + ObjectMapper objectMapper = new ObjectMapper(); + objectMapper.writeValue(new java.io.File(commandOptions.jsonFile), sampleIndexConfiguration); + } return openCGAClient.getVariantOperationClient().variantSampleIndexConfigure(sampleIndexConfiguration, queryParams); + } + private RestResponse deleteVariantScore() throws Exception { logger.debug("Executing deleteVariantScore in Operations - Variant Storage command line"); @@ -761,8 +880,8 @@ private RestResponse deleteVariantScore() throws Exception { queryParams.putIfNotEmpty("name", commandOptions.name); queryParams.putIfNotNull("resume", commandOptions.resume); queryParams.putIfNotNull("force", commandOptions.force); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getVariantOperationClient().deleteVariantScore(queryParams); @@ -780,13 +899,13 @@ private RestResponse indexVariantScore() throws Exception { queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } VariantScoreIndexParams variantScoreIndexParams = new VariantScoreIndexParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(variantScoreIndexParams)); @@ -800,8 +919,11 @@ private RestResponse indexVariantScore() throws Exception { .setCohort1(commandOptions.cohort1) .setCohort2(commandOptions.cohort2) .setInput(commandOptions.input) - .setInputColumns(commandOptions.inputColumns) - .setResume(commandOptions.resume); + .setInputColumns(commandOptions.inputColumns); + + if (commandOptions.resume != null){ + ((VariantScoreIndexParams)variantScoreIndexParams).setResume(commandOptions.resume); + } } return openCGAClient.getVariantOperationClient().indexVariantScore(variantScoreIndexParams, queryParams); @@ -820,13 +942,13 @@ private RestResponse secondaryIndexVariant() throws Exception { queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); queryParams.putIfNotEmpty("project", commandOptions.project); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } VariantSecondaryIndexParams variantSecondaryIndexParams = new VariantSecondaryIndexParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(variantSecondaryIndexParams)); @@ -837,8 +959,11 @@ private RestResponse secondaryIndexVariant() throws Exception { } else { ((VariantSecondaryIndexParams)variantSecondaryIndexParams) .setRegion(commandOptions.region) - .setSample(splitWithTrim(commandOptions.sample)) - .setOverwrite(commandOptions.overwrite); + .setSample(splitWithTrim(commandOptions.sample)); + + if (commandOptions.overwrite != null){ + ((VariantSecondaryIndexParams)variantSecondaryIndexParams).setOverwrite(commandOptions.overwrite); + } } return openCGAClient.getVariantOperationClient().secondaryIndexVariant(variantSecondaryIndexParams, queryParams); @@ -857,8 +982,8 @@ private RestResponse deleteVariantSecondaryIndex() throws Exception { queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotEmpty("samples", commandOptions.samples); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getVariantOperationClient().deleteVariantSecondaryIndex(queryParams); @@ -876,13 +1001,13 @@ private RestResponse deleteVariantStats() throws Exception { queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } VariantStatsDeleteParams variantStatsDeleteParams = new VariantStatsDeleteParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(variantStatsDeleteParams)); @@ -892,8 +1017,11 @@ private RestResponse deleteVariantStats() throws Exception { objectMapper.writeValue(new java.io.File(commandOptions.jsonFile), variantStatsDeleteParams); } else { ((VariantStatsDeleteParams)variantStatsDeleteParams) - .setCohort(splitWithTrim(commandOptions.cohort)) - .setForce(commandOptions.force); + .setCohort(splitWithTrim(commandOptions.cohort)); + + if (commandOptions.force != null){ + ((VariantStatsDeleteParams)variantStatsDeleteParams).setForce(commandOptions.force); + } } return openCGAClient.getVariantOperationClient().deleteVariantStats(variantStatsDeleteParams, queryParams); @@ -911,13 +1039,13 @@ private RestResponse indexVariantStats() throws Exception { queryParams.putIfNotEmpty("jobDescription", commandOptions.jobDescription); queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } VariantStatsIndexParams variantStatsIndexParams = new VariantStatsIndexParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(variantStatsIndexParams)); @@ -929,10 +1057,16 @@ private RestResponse indexVariantStats() throws Exception { ((VariantStatsIndexParams)variantStatsIndexParams) .setCohort(splitWithTrim(commandOptions.cohort)) .setRegion(commandOptions.region) - .setOverwriteStats(commandOptions.overwriteStats) - .setResume(commandOptions.resume) .setAggregationMappingFile(commandOptions.aggregationMappingFile); + if (commandOptions.overwriteStats != null){ + ((VariantStatsIndexParams)variantStatsIndexParams).setOverwriteStats(commandOptions.overwriteStats); + } + + if (commandOptions.resume != null){ + ((VariantStatsIndexParams)variantStatsIndexParams).setResume(commandOptions.resume); + } + } return openCGAClient.getVariantOperationClient().indexVariantStats(variantStatsIndexParams, queryParams); } @@ -949,13 +1083,13 @@ private RestResponse deleteVariantStudy() throws Exception { queryParams.putIfNotEmpty("jobDependsOn", commandOptions.jobDependsOn); queryParams.putIfNotEmpty("jobTags", commandOptions.jobTags); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } VariantStudyDeleteParams variantStudyDeleteParams = new VariantStudyDeleteParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(variantStudyDeleteParams)); @@ -963,9 +1097,11 @@ private RestResponse deleteVariantStudy() throws Exception { } else if (commandOptions.jsonFile != null) { ObjectMapper objectMapper = new ObjectMapper(); objectMapper.writeValue(new java.io.File(commandOptions.jsonFile), variantStudyDeleteParams); - } else { - ((VariantStudyDeleteParams)variantStudyDeleteParams) - .setResume(commandOptions.resume); + } else {; + + if (commandOptions.resume != null){ + ((VariantStudyDeleteParams)variantStudyDeleteParams).setResume(commandOptions.resume); + } } return openCGAClient.getVariantOperationClient().deleteVariantStudy(variantStudyDeleteParams, queryParams); diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/ProjectsCommandExecutor.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/ProjectsCommandExecutor.java index efa0751f4ee..0fdc6820fad 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/ProjectsCommandExecutor.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/ProjectsCommandExecutor.java @@ -28,7 +28,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 +* Autogenerated on: 2022-03-25 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -37,7 +37,7 @@ /** * This class contains methods for the Projects command line. - * OpenCGA version: 2.2.0-rc2-SNAPSHOT + * OpenCGA version: 2.2.0-SNAPSHOT * PATH: /{apiVersion}/projects */ public class ProjectsCommandExecutor extends OpencgaCommandExecutor { @@ -71,6 +71,9 @@ public void execute() throws Exception { case "info": queryResponse = info(); break; + case "increlease": + queryResponse = incRelease(); + break; case "studies": queryResponse = studies(); break; @@ -104,7 +107,7 @@ private RestResponse create() throws Exception { invokeSetter(projectOrganism, "assembly", commandOptions.organismAssembly); ProjectCreateParams projectCreateParams = new ProjectCreateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(projectCreateParams)); @@ -147,8 +150,8 @@ private RestResponse search() throws Exception { queryParams.putIfNotEmpty("modificationDate", commandOptions.modificationDate); queryParams.putIfNotEmpty("internalStatus", commandOptions.internalStatus); queryParams.putIfNotEmpty("attributes", commandOptions.attributes); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getProjectClient().search(queryParams); @@ -185,6 +188,14 @@ private RestResponse info() throws Exception { return openCGAClient.getProjectClient().info(commandOptions.projects, queryParams); } + private RestResponse incRelease() throws Exception { + + logger.debug("Executing incRelease in Projects command line"); + + ProjectsCommandOptions.IncReleaseCommandOptions commandOptions = projectsCommandOptions.incReleaseCommandOptions; + return openCGAClient.getProjectClient().incRelease(commandOptions.project); + } + private RestResponse studies() throws Exception { logger.debug("Executing studies in Projects command line"); @@ -218,7 +229,7 @@ private RestResponse update() throws Exception { invokeSetter(projectOrganism, "assembly", commandOptions.organismAssembly); ProjectUpdateParams projectUpdateParams = new ProjectUpdateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(projectUpdateParams)); diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/SamplesCommandExecutor.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/SamplesCommandExecutor.java index f71a88af25e..3a38c14f965 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/SamplesCommandExecutor.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/SamplesCommandExecutor.java @@ -39,7 +39,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 +* Autogenerated on: 2022-03-25 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -48,7 +48,7 @@ /** * This class contains methods for the Samples command line. - * OpenCGA version: 2.2.0-rc2-SNAPSHOT + * OpenCGA version: 2.2.0-SNAPSHOT * PATH: /{apiVersion}/samples */ public class SamplesCommandExecutor extends OpencgaCommandExecutor { @@ -103,6 +103,9 @@ public void execute() throws Exception { case "update": queryResponse = update(); break; + case "annotationsets-annotations-update": + queryResponse = updateAnnotationSetsAnnotations(); + break; default: logger.error("Subcommand not valid"); break; @@ -120,13 +123,13 @@ private RestResponse updateAcl() throws Exception { ObjectMap queryParams = new ObjectMap(); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } SampleAclUpdateParams sampleAclUpdateParams = new SampleAclUpdateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(sampleAclUpdateParams)); @@ -169,8 +172,8 @@ private RestResponse aggregationStats() throws Exception { queryParams.putIfNotEmpty("annotation", commandOptions.annotation); queryParams.putIfNotNull("default_values", commandOptions.default_values); queryParams.putIfNotEmpty("field", commandOptions.field); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getSampleClient().aggregationStats(queryParams); @@ -186,13 +189,13 @@ private RestResponse loadAnnotationSets() throws Exception { queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotNull("parents", commandOptions.parents); queryParams.putIfNotEmpty("annotationSetId", commandOptions.annotationSetId); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } TsvAnnotationParams tsvAnnotationParams = new TsvAnnotationParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(tsvAnnotationParams)); @@ -219,8 +222,8 @@ private RestResponse create() throws Exception { queryParams.putIfNotEmpty("exclude", commandOptions.exclude); queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotNull("includeResult", commandOptions.includeResult); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } @@ -250,7 +253,7 @@ private RestResponse create() throws Exception { invokeSetter(externalSource, "url", commandOptions.sourceUrl); SampleCreateParams sampleCreateParams = new SampleCreateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(sampleCreateParams)); @@ -268,9 +271,12 @@ private RestResponse create() throws Exception { .setSource(externalSource) .setProcessing(sampleProcessing) .setCollection(sampleCollection) - .setSomatic(commandOptions.somatic) .setStatus(statusParams); + if (commandOptions.somatic != null){ + ((SampleCreateParams)sampleCreateParams).setSomatic(commandOptions.somatic); + } + } return openCGAClient.getSampleClient().create(sampleCreateParams, queryParams); } @@ -320,8 +326,8 @@ private RestResponse distinct() throws Exception { queryParams.putIfNotEmpty("statsBiotypeCount", commandOptions.statsBiotypeCount); queryParams.putIfNotEmpty("statsClinicalSignificanceCount", commandOptions.statsClinicalSignificanceCount); queryParams.putIfNotEmpty("statsConsequenceTypeCount", commandOptions.statsConsequenceTypeCount); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getSampleClient().distinct(commandOptions.field, queryParams); @@ -336,8 +342,8 @@ private RestResponse load() throws Exception { ObjectMap queryParams = new ObjectMap(); queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotEmpty("variableSet", commandOptions.variableSet); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getSampleClient().load(commandOptions.file, queryParams); @@ -395,8 +401,8 @@ private RestResponse search() throws Exception { queryParams.putIfNotEmpty("statsBiotypeCount", commandOptions.statsBiotypeCount); queryParams.putIfNotEmpty("statsClinicalSignificanceCount", commandOptions.statsClinicalSignificanceCount); queryParams.putIfNotEmpty("statsConsequenceTypeCount", commandOptions.statsConsequenceTypeCount); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getSampleClient().search(queryParams); @@ -412,8 +418,8 @@ private RestResponse acl() throws Exception { queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotEmpty("member", commandOptions.member); queryParams.putIfNotNull("silent", commandOptions.silent); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getSampleClient().acl(commandOptions.samples, queryParams); @@ -430,8 +436,8 @@ private RestResponse delete() throws Exception { queryParams.putIfNotEmpty("emptyFilesAction", commandOptions.emptyFilesAction); queryParams.putIfNotNull("deleteEmptyCohorts", commandOptions.deleteEmptyCohorts); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getSampleClient().delete(commandOptions.samples, queryParams); @@ -451,8 +457,8 @@ private RestResponse info() throws Exception { queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotEmpty("version", commandOptions.version); queryParams.putIfNotNull("deleted", commandOptions.deleted); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getSampleClient().info(commandOptions.samples, queryParams); @@ -470,8 +476,8 @@ private RestResponse update() throws Exception { queryParams.putIfNotEmpty("study", commandOptions.study); queryParams.putIfNotNull("incVersion", commandOptions.incVersion); queryParams.putIfNotNull("includeResult", commandOptions.includeResult); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } @@ -501,7 +507,7 @@ private RestResponse update() throws Exception { invokeSetter(externalSource, "url", commandOptions.sourceUrl); SampleUpdateParams sampleUpdateParams = new SampleUpdateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(sampleUpdateParams)); @@ -519,10 +525,40 @@ private RestResponse update() throws Exception { .setSource(externalSource) .setProcessing(sampleProcessing) .setCollection(sampleCollection) - .setSomatic(commandOptions.somatic) .setStatus(statusParams); + if (commandOptions.somatic != null){ + ((SampleUpdateParams)sampleUpdateParams).setSomatic(commandOptions.somatic); + } + } return openCGAClient.getSampleClient().update(commandOptions.samples, sampleUpdateParams, queryParams); } + + private RestResponse updateAnnotationSetsAnnotations() throws Exception { + + logger.debug("Executing updateAnnotationSetsAnnotations in Samples command line"); + + SamplesCommandOptions.UpdateAnnotationSetsAnnotationsCommandOptions commandOptions = samplesCommandOptions.updateAnnotationSetsAnnotationsCommandOptions; + + ObjectMap queryParams = new ObjectMap(); + queryParams.putIfNotEmpty("study", commandOptions.study); + queryParams.putIfNotNull("action", commandOptions.action); + queryParams.putIfNotNull("incVersion", commandOptions.incVersion); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + } + + + ObjectMap objectMap = new ObjectMap(); + if (commandOptions.jsonDataModel) { + RestResponse res = new RestResponse<>(); + res.setType(QueryType.VOID); + PrintUtils.println(getObjectAsJSON(objectMap)); + return res; + } else if (commandOptions.jsonFile != null) { + ObjectMapper objectMapper = new ObjectMapper(); + objectMapper.writeValue(new java.io.File(commandOptions.jsonFile), objectMap); + } return openCGAClient.getSampleClient().updateAnnotationSetsAnnotations(commandOptions.sample, commandOptions.annotationSet, objectMap, queryParams); + } } \ No newline at end of file diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/StudiesCommandExecutor.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/StudiesCommandExecutor.java index b11ceead704..f973d0c2861 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/StudiesCommandExecutor.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/StudiesCommandExecutor.java @@ -55,7 +55,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 +* Autogenerated on: 2022-03-25 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -64,7 +64,7 @@ /** * This class contains methods for the Studies command line. - * OpenCGA version: 2.2.0-rc2-SNAPSHOT + * OpenCGA version: 2.2.0-SNAPSHOT * PATH: /{apiVersion}/studies */ public class StudiesCommandExecutor extends ParentStudiesCommandExecutor { @@ -125,6 +125,9 @@ public void execute() throws Exception { case "templates-run": queryResponse = runTemplates(); break; + case "templates-upload": + queryResponse = uploadTemplates(); + break; case "templates-delete": queryResponse = deleteTemplates(); break; @@ -140,9 +143,6 @@ public void execute() throws Exception { case "variablesets-variables-update": queryResponse = updateVariableSetsVariables(); break; - case "templates-upload": - queryResponse = templatesUpload(); - break; default: logger.error("Subcommand not valid"); break; @@ -159,7 +159,7 @@ private RestResponse updateAcl() throws Exception { StudiesCommandOptions.UpdateAclCommandOptions commandOptions = studiesCommandOptions.updateAclCommandOptions; StudyAclUpdateParams studyAclUpdateParams = new StudyAclUpdateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(studyAclUpdateParams)); @@ -200,7 +200,7 @@ private RestResponse create() throws Exception { invokeSetter(statusParams, "description", commandOptions.statusDescription); StudyCreateParams studyCreateParams = new StudyCreateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(studyCreateParams)); @@ -341,7 +341,7 @@ private RestResponse updateGroups() throws Exception { GroupCreateParams groupCreateParams = new GroupCreateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(groupCreateParams)); @@ -369,7 +369,7 @@ private RestResponse updateGroupsUsers() throws Exception { GroupUpdateParams groupUpdateParams = new GroupUpdateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(groupUpdateParams)); @@ -404,7 +404,7 @@ private RestResponse updatePermissionRules() throws Exception { PermissionRule permissionRule = new PermissionRule(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(permissionRule)); @@ -430,6 +430,14 @@ protected RestResponse runTemplates() throws Exception { } + protected RestResponse uploadTemplates() throws Exception { + + logger.debug("Executing uploadTemplates in Studies command line"); + + return super.uploadTemplates(); + + } + private RestResponse deleteTemplates() throws Exception { logger.debug("Executing deleteTemplates in Studies command line"); @@ -438,8 +446,8 @@ private RestResponse deleteTemplates() throws Exception { ObjectMap queryParams = new ObjectMap(); queryParams.putIfNotEmpty("study", commandOptions.study); - if(queryParams.get("study")==null && OpencgaMain.isShellMode()){ - queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); + if (queryParams.get("study") == null && OpencgaMain.isShellMode()) { + queryParams.putIfNotEmpty("study", sessionManager.getSession().getCurrentStudy()); } return openCGAClient.getStudyClient().deleteTemplates(commandOptions.study, commandOptions.templateId, queryParams); @@ -467,7 +475,7 @@ private RestResponse update() throws Exception { invokeSetter(statusParams, "description", commandOptions.statusDescription); StudyUpdateParams studyUpdateParams = new StudyUpdateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(studyUpdateParams)); @@ -512,7 +520,7 @@ private RestResponse updateVariableSets() throws Exception { VariableSetCreateParams variableSetCreateParams = new VariableSetCreateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(variableSetCreateParams)); @@ -524,10 +532,16 @@ private RestResponse updateVariableSets() throws Exception { ((VariableSetCreateParams)variableSetCreateParams) .setId(commandOptions.id) .setName(commandOptions.name) - .setUnique(commandOptions.unique) - .setConfidential(commandOptions.confidential) .setDescription(commandOptions.description); + if (commandOptions.unique != null){ + ((VariableSetCreateParams)variableSetCreateParams).setUnique(commandOptions.unique); + } + + if (commandOptions.confidential != null){ + ((VariableSetCreateParams)variableSetCreateParams).setConfidential(commandOptions.confidential); + } + } return openCGAClient.getStudyClient().updateVariableSets(commandOptions.study, variableSetCreateParams, queryParams); } @@ -541,10 +555,14 @@ private RestResponse updateVariableSetsVariables() throws Exception ObjectMap queryParams = new ObjectMap(); queryParams.putIfNotNull("action", commandOptions.action); - Variable.VariableType typeParam = Variable.VariableType.valueOf(commandOptions.type); + Variable.VariableType typeParam = null; + if (commandOptions.type != null) { + typeParam = Variable.VariableType.valueOf(commandOptions.type); + + } Variable variable = new Variable(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(variable)); @@ -558,14 +576,20 @@ private RestResponse updateVariableSetsVariables() throws Exception .setName(commandOptions.name) .setCategory(commandOptions.category) .setDefaultValue(commandOptions.defaultValue) - .setRequired(commandOptions.required) - .setMultiValue(commandOptions.multiValue) .setAllowedValues(splitWithTrim(commandOptions.allowedValues)) .setAllowedKeys(splitWithTrim(commandOptions.allowedKeys)) .setRank(commandOptions.rank) .setDependsOn(commandOptions.dependsOn) .setDescription(commandOptions.description); + if (commandOptions.required != null){ + ((Variable)variable).setRequired(commandOptions.required); + } + + if (commandOptions.multiValue != null){ + ((Variable)variable).setMultiValue(commandOptions.multiValue); + } + } return openCGAClient.getStudyClient().updateVariableSetsVariables(commandOptions.study, commandOptions.variableSet, variable, queryParams); } diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/UsersCommandExecutor.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/UsersCommandExecutor.java index 42f28247215..00eedb879f2 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/UsersCommandExecutor.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/executors/UsersCommandExecutor.java @@ -39,7 +39,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 +* Autogenerated on: 2022-03-25 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -48,7 +48,7 @@ /** * This class contains methods for the Users command line. - * OpenCGA version: 2.2.0-rc2-SNAPSHOT + * OpenCGA version: 2.2.0-SNAPSHOT * PATH: /{apiVersion}/users */ public class UsersCommandExecutor extends ParentUsersCommandExecutor { @@ -119,7 +119,7 @@ private RestResponse create() throws Exception { UsersCommandOptions.CreateCommandOptions commandOptions = usersCommandOptions.createCommandOptions; UserCreateParams userCreateParams = new UserCreateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(userCreateParams)); @@ -154,7 +154,7 @@ private RestResponse password() throws Exception { UsersCommandOptions.PasswordCommandOptions commandOptions = usersCommandOptions.passwordCommandOptions; PasswordChangeParams passwordChangeParams = new PasswordChangeParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(passwordChangeParams)); @@ -209,7 +209,7 @@ private RestResponse updateConfigs() throws Exception { ConfigUpdateParams configUpdateParams = new ConfigUpdateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(configUpdateParams)); @@ -273,7 +273,7 @@ private RestResponse update() throws Exception { UserUpdateParams userUpdateParams = new UserUpdateParams(); - if (commandOptions.jsonViewTemplate) { + if (commandOptions.jsonDataModel) { RestResponse res = new RestResponse<>(); res.setType(QueryType.VOID); PrintUtils.println(getObjectAsJSON(userUpdateParams)); diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/io/TextOutputWriter.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/io/TextOutputWriter.java index 4e3460f06eb..678c94233f1 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/io/TextOutputWriter.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/io/TextOutputWriter.java @@ -402,6 +402,7 @@ private void printRecursiveTree(List fileTreeList, StringBuilder sb, S sb.append(" ["); if (file.getInternal() != null && file.getInternal().getStatus() != null + && file.getInternal().getStatus().getName() != null && !READY.equals(file.getInternal().getStatus().getName())) { sb.append(file.getInternal().getStatus().getName()).append(", "); } diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/AnalysisAlignmentCommandOptions.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/AnalysisAlignmentCommandOptions.java index 9e61f080c22..3e67f3fa216 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/AnalysisAlignmentCommandOptions.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/AnalysisAlignmentCommandOptions.java @@ -14,7 +14,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 +* Autogenerated on: 2022-03-25 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -23,7 +23,7 @@ /** * This class contains methods for the Analysis - Alignment command line. - * OpenCGA version: 2.2.0-rc2-SNAPSHOT + * OpenCGA version: 2.2.0-SNAPSHOT * PATH: /{apiVersion}/analysis/alignment */ @Parameters(commandNames = {"alignments"}, commandDescription = "Analysis - Alignment commands") @@ -34,7 +34,7 @@ public class AnalysisAlignmentCommandOptions { public RunBwaCommandOptions runBwaCommandOptions; public RunCoverageIndexCommandOptions runCoverageIndexCommandOptions; - public RunQcGeneCoverageStatsCommandOptions runQcGeneCoverageStatsCommandOptions; + public CoverageQcGeneCoverageStatsRunCommandOptions coverageQcGeneCoverageStatsRunCommandOptions; public QueryCoverageCommandOptions queryCoverageCommandOptions; public RatioCoverageCommandOptions ratioCoverageCommandOptions; public StatsCoverageCommandOptions statsCoverageCommandOptions; @@ -53,7 +53,7 @@ public AnalysisAlignmentCommandOptions(CommonCommandOptions commonCommandOptions this.commonCommandOptions = commonCommandOptions; this.runBwaCommandOptions = new RunBwaCommandOptions(); this.runCoverageIndexCommandOptions = new RunCoverageIndexCommandOptions(); - this.runQcGeneCoverageStatsCommandOptions = new RunQcGeneCoverageStatsCommandOptions(); + this.coverageQcGeneCoverageStatsRunCommandOptions = new CoverageQcGeneCoverageStatsRunCommandOptions(); this.queryCoverageCommandOptions = new QueryCoverageCommandOptions(); this.ratioCoverageCommandOptions = new RatioCoverageCommandOptions(); this.statsCoverageCommandOptions = new StatsCoverageCommandOptions(); @@ -76,8 +76,8 @@ public class RunBwaCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "study", required = false, arity = 1) public String study; @@ -119,8 +119,8 @@ public class RunCoverageIndexCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "study", required = false, arity = 1) public String study; @@ -147,8 +147,8 @@ public class RunCoverageIndexCommandOptions { public Boolean overwrite; } - @Parameters(commandNames = {"qc-genecoveragestats-run"}, commandDescription ="Compute gene coverage stats for a given alignment file and a list of genes") - public class RunQcGeneCoverageStatsCommandOptions { + @Parameters(commandNames = {"coverage-qc-genecoveragestats-run"}, commandDescription ="Compute gene coverage stats for a given alignment file and a list of genes") + public class CoverageQcGeneCoverageStatsRunCommandOptions { @ParametersDelegate public CommonCommandOptions commonOptions = commonCommandOptions; @@ -156,8 +156,8 @@ public class RunQcGeneCoverageStatsCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "study", required = false, arity = 1) public String study; @@ -283,8 +283,8 @@ public class RunDeeptoolsCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "study", required = false, arity = 1) public String study; @@ -317,8 +317,8 @@ public class RunFastqcCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "study", required = false, arity = 1) public String study; @@ -351,8 +351,8 @@ public class RunIndexCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "study", required = false, arity = 1) public String study; @@ -385,8 +385,8 @@ public class RunPicardCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "study", required = false, arity = 1) public String study; @@ -419,8 +419,8 @@ public class RunQcCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "study", required = false, arity = 1) public String study; @@ -532,8 +532,8 @@ public class RunSamtoolsCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "study", required = false, arity = 1) public String study; diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/AnalysisClinicalCommandOptions.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/AnalysisClinicalCommandOptions.java index 5f83c2bd6fc..885a862dc18 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/AnalysisClinicalCommandOptions.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/AnalysisClinicalCommandOptions.java @@ -14,7 +14,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 +* Autogenerated on: 2022-03-25 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -23,7 +23,7 @@ /** * This class contains methods for the Analysis - Clinical command line. - * OpenCGA version: 2.2.0-rc2-SNAPSHOT + * OpenCGA version: 2.2.0-SNAPSHOT * PATH: /{apiVersion}/analysis/clinical */ @Parameters(commandNames = {"clinical"}, commandDescription = "Analysis - Clinical commands") @@ -59,7 +59,9 @@ public class AnalysisClinicalCommandOptions { public UpdateCommandOptions updateCommandOptions; public InfoCommandOptions infoCommandOptions; public CreateInterpretationCommandOptions createInterpretationCommandOptions; + public ClearInterpretationCommandOptions clearInterpretationCommandOptions; public DeleteInterpretationCommandOptions deleteInterpretationCommandOptions; + public RevertInterpretationCommandOptions revertInterpretationCommandOptions; public UpdateInterpretationCommandOptions updateInterpretationCommandOptions; @@ -94,7 +96,9 @@ public AnalysisClinicalCommandOptions(CommonCommandOptions commonCommandOptions, this.updateCommandOptions = new UpdateCommandOptions(); this.infoCommandOptions = new InfoCommandOptions(); this.createInterpretationCommandOptions = new CreateInterpretationCommandOptions(); + this.clearInterpretationCommandOptions = new ClearInterpretationCommandOptions(); this.deleteInterpretationCommandOptions = new DeleteInterpretationCommandOptions(); + this.revertInterpretationCommandOptions = new RevertInterpretationCommandOptions(); this.updateInterpretationCommandOptions = new UpdateInterpretationCommandOptions(); } @@ -108,8 +112,8 @@ public class UpdateAclCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -139,8 +143,8 @@ public class UpdateClinicalConfigurationCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -155,8 +159,8 @@ public class CreateCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--include", "-I"}, description = "Fields included in the response, whole JSON path must be provided", required = false, arity = 1) public String include; @@ -481,8 +485,8 @@ public class RunInterpreterCancerTieringCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -518,8 +522,8 @@ public class RunInterpreterTeamCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -558,8 +562,8 @@ public class RunInterpreterTieringCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -598,8 +602,8 @@ public class RunInterpreterZettaCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -1031,8 +1035,8 @@ public class RunRgaIndexCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -1734,8 +1738,8 @@ public class UpdateCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--include", "-I"}, description = "Fields included in the response, whole JSON path must be provided", required = false, arity = 1) public String include; @@ -1847,8 +1851,8 @@ public class CreateInterpretationCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--include", "-I"}, description = "Fields included in the response, whole JSON path must be provided", required = false, arity = 1) public String include; @@ -1895,6 +1899,28 @@ public class CreateInterpretationCommandOptions { @Parameter(names = {"--status-id"}, description = "The body web service id parameter", required = false, arity = 1) public String statusId; + } + @Parameters(commandNames = {"interpretation-clear"}, commandDescription ="Clear the fields of the main interpretation of the Clinical Analysis") + public class ClearInterpretationCommandOptions { + + @ParametersDelegate + public CommonCommandOptions commonOptions = commonCommandOptions; + + @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) + public String jsonFile; + + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; + + @Parameter(names = {"--study", "-s"}, description = "[[user@]project:]study ID", required = false, arity = 1) + public String study; + + @Parameter(names = {"--interpretations"}, description = "Interpretation IDs of the Clinical Analysis", required = true, arity = 1) + public String interpretations; + + @Parameter(names = {"--clinical-analysis"}, description = "Clinical analysis ID", required = true, arity = 1) + public String clinicalAnalysis; + } @Parameters(commandNames = {"interpretation-delete"}, commandDescription ="Delete interpretation") public class DeleteInterpretationCommandOptions { @@ -1914,6 +1940,31 @@ public class DeleteInterpretationCommandOptions { @Parameter(names = {"--set-as-primary"}, description = "Interpretation id to set as primary from the list of secondaries in case of deleting the actual primary one", required = false, arity = 1) public String setAsPrimary; + } + @Parameters(commandNames = {"interpretation-revert"}, commandDescription ="Revert to a previous interpretation version") + public class RevertInterpretationCommandOptions { + + @ParametersDelegate + public CommonCommandOptions commonOptions = commonCommandOptions; + + @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) + public String jsonFile; + + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; + + @Parameter(names = {"--study", "-s"}, description = "[[user@]project:]study ID", required = false, arity = 1) + public String study; + + @Parameter(names = {"--clinical-analysis"}, description = "Clinical analysis ID", required = true, arity = 1) + public String clinicalAnalysis; + + @Parameter(names = {"--interpretation"}, description = "Interpretation ID", required = true, arity = 1) + public String interpretation; + + @Parameter(names = {"--version"}, description = "Version to revert to", required = true, arity = 1) + public Integer version; + } @Parameters(commandNames = {"interpretation-update"}, commandDescription ="Update interpretation fields") public class UpdateInterpretationCommandOptions { @@ -1924,8 +1975,8 @@ public class UpdateInterpretationCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--include", "-I"}, description = "Fields included in the response, whole JSON path must be provided", required = false, arity = 1) public String include; diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/AnalysisVariantCommandOptions.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/AnalysisVariantCommandOptions.java index 73a83f83142..2ed7bf331e6 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/AnalysisVariantCommandOptions.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/AnalysisVariantCommandOptions.java @@ -14,7 +14,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 +* Autogenerated on: 2022-03-25 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -23,7 +23,7 @@ /** * This class contains methods for the Analysis - Variant command line. - * OpenCGA version: 2.2.0-rc2-SNAPSHOT + * OpenCGA version: 2.2.0-SNAPSHOT * PATH: /{apiVersion}/analysis/variant */ @Parameters(commandNames = {"variant"}, commandDescription = "Analysis - Variant commands") @@ -283,8 +283,8 @@ public class RunCircosCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "study", required = false, arity = 1) public String study; @@ -334,8 +334,8 @@ public class RunCohortStatsCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -377,8 +377,8 @@ public class RunExportCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--include", "-I"}, description = "Fields included in the response, whole JSON path must be provided", required = false, arity = 1) public String include; @@ -410,8 +410,8 @@ public class RunExportCommandOptions { @Parameter(names = {"--output-file-name"}, description = "The body web service outputFileName parameter", required = false, arity = 1) public String outputFileName; - @Parameter(names = {"--output-format"}, description = "The body web service outputFormat parameter", required = false, arity = 1) - public String outputFormat; + @Parameter(names = {"--output-file-format"}, description = "The body web service outputFileFormat parameter", required = false, arity = 1) + public String outputFileFormat; @Parameter(names = {"--variants-file"}, description = "The body web service variantsFile parameter", required = false, arity = 1) public String variantsFile; @@ -616,8 +616,8 @@ public class RunFamilyQcCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -684,8 +684,8 @@ public class RunGatkCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "study", required = false, arity = 1) public String study; @@ -718,8 +718,8 @@ public class RunGenomePlotCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -761,8 +761,8 @@ public class RunGwasCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -825,8 +825,8 @@ public class RunIndexCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -937,8 +937,8 @@ public class RunIndividualQcCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -977,8 +977,8 @@ public class RunInferredSexCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -1052,8 +1052,8 @@ public class RunKnockoutCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "study", required = false, arity = 1) public String study; @@ -1110,8 +1110,8 @@ public class RunMendelianErrorCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -1239,8 +1239,8 @@ public class RunMutationalSignatureCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -1285,8 +1285,8 @@ public class RunPlinkCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "study", required = false, arity = 1) public String study; @@ -1554,8 +1554,8 @@ public class RunRelatednessCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -1597,8 +1597,8 @@ public class RunRvtestsCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "study", required = false, arity = 1) public String study; @@ -1704,8 +1704,8 @@ public class RunSampleEligibilityCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -1741,8 +1741,8 @@ public class RunSampleQcCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -1896,8 +1896,8 @@ public class RunSampleCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -2063,8 +2063,8 @@ public class RunSampleStatsCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -2115,8 +2115,8 @@ public class RunStatsExportCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--project", "-p"}, description = "Project [user@]project where project can be either the ID or the alias", required = false, arity = 1) public String project; @@ -2148,8 +2148,8 @@ public class RunStatsExportCommandOptions { @Parameter(names = {"--gene"}, description = "The body web service gene parameter", required = false, arity = 1) public String gene; - @Parameter(names = {"--output-format"}, description = "The body web service outputFormat parameter", required = false, arity = 1) - public String outputFormat; + @Parameter(names = {"--output-file-format"}, description = "The body web service outputFileFormat parameter", required = false, arity = 1) + public String outputFileFormat; } @Parameters(commandNames = {"stats-run"}, commandDescription ="Compute variant stats for any cohort and any set of variants.") @@ -2161,8 +2161,8 @@ public class RunStatsCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/CohortsCommandOptions.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/CohortsCommandOptions.java index 8ea10d73349..45c2d782faa 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/CohortsCommandOptions.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/CohortsCommandOptions.java @@ -14,7 +14,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 +* Autogenerated on: 2022-03-25 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -23,7 +23,7 @@ /** * This class contains methods for the Cohorts command line. - * OpenCGA version: 2.2.0-rc2-SNAPSHOT + * OpenCGA version: 2.2.0-SNAPSHOT * PATH: /{apiVersion}/cohorts */ @Parameters(commandNames = {"cohorts"}, commandDescription = "Cohorts commands") @@ -43,6 +43,7 @@ public class CohortsCommandOptions { public DeleteCommandOptions deleteCommandOptions; public InfoCommandOptions infoCommandOptions; public UpdateCommandOptions updateCommandOptions; + public UpdateAnnotationSetsAnnotationsCommandOptions updateAnnotationSetsAnnotationsCommandOptions; public CohortsCommandOptions(CommonCommandOptions commonCommandOptions, JCommander jCommander) { @@ -60,6 +61,7 @@ public CohortsCommandOptions(CommonCommandOptions commonCommandOptions, JCommand this.deleteCommandOptions = new DeleteCommandOptions(); this.infoCommandOptions = new InfoCommandOptions(); this.updateCommandOptions = new UpdateCommandOptions(); + this.updateAnnotationSetsAnnotationsCommandOptions = new UpdateAnnotationSetsAnnotationsCommandOptions(); } @@ -72,8 +74,8 @@ public class UpdateAclCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -143,8 +145,8 @@ public class LoadAnnotationSetsCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -174,8 +176,8 @@ public class CreateCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--include", "-I"}, description = "Fields included in the response, whole JSON path must be provided", required = false, arity = 1) public String include; @@ -287,8 +289,8 @@ public class GenerateCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--include", "-I"}, description = "Fields included in the response, whole JSON path must be provided", required = false, arity = 1) public String include; @@ -505,8 +507,8 @@ public class UpdateCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--include", "-I"}, description = "Fields included in the response, whole JSON path must be provided", required = false, arity = 1) public String include; @@ -550,5 +552,30 @@ public class UpdateCommandOptions { @Parameter(names = {"--status-description"}, description = "The body web service description parameter", required = false, arity = 1) public String statusDescription; + } + @Parameters(commandNames = {"annotationsets-annotations-update"}, commandDescription ="Update annotations from an annotationSet") + public class UpdateAnnotationSetsAnnotationsCommandOptions { + + @ParametersDelegate + public CommonCommandOptions commonOptions = commonCommandOptions; + + @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) + public String jsonFile; + + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; + + @Parameter(names = {"--cohort"}, description = "Cohort ID", required = true, arity = 1) + public String cohort; + + @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) + public String study; + + @Parameter(names = {"--annotation-set"}, description = "AnnotationSet ID to be updated.", required = true, arity = 1) + public String annotationSet; + + @Parameter(names = {"--action"}, description = "Action to be performed: ADD to add new annotations; REPLACE to replace the value of an already existing annotation; SET to set the new list of annotations removing any possible old annotations; REMOVE to remove some annotations; RESET to set some annotations to the default value configured in the corresponding variables of the VariableSet if any.", required = false, arity = 1) + public String action; + } } \ No newline at end of file diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/DiseasePanelsCommandOptions.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/DiseasePanelsCommandOptions.java index 64215ed612c..3ddb788d170 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/DiseasePanelsCommandOptions.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/DiseasePanelsCommandOptions.java @@ -14,7 +14,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 +* Autogenerated on: 2022-03-25 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -23,7 +23,7 @@ /** * This class contains methods for the Disease Panels command line. - * OpenCGA version: 2.2.0-rc2-SNAPSHOT + * OpenCGA version: 2.2.0-SNAPSHOT * PATH: /{apiVersion}/panels */ @Parameters(commandNames = {"panels"}, commandDescription = "Disease Panels commands") @@ -66,8 +66,8 @@ public class UpdateAclCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -94,8 +94,8 @@ public class CreateCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--include", "-I"}, description = "Fields included in the response, whole JSON path must be provided", required = false, arity = 1) public String include; @@ -331,8 +331,8 @@ public class UpdateCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--include", "-I"}, description = "Fields included in the response, whole JSON path must be provided", required = false, arity = 1) public String include; diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/FamiliesCommandOptions.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/FamiliesCommandOptions.java index ce986fb3e0e..8631656462e 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/FamiliesCommandOptions.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/FamiliesCommandOptions.java @@ -14,7 +14,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 +* Autogenerated on: 2022-03-25 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -23,7 +23,7 @@ /** * This class contains methods for the Families command line. - * OpenCGA version: 2.2.0-rc2-SNAPSHOT + * OpenCGA version: 2.2.0-SNAPSHOT * PATH: /{apiVersion}/families */ @Parameters(commandNames = {"families"}, commandDescription = "Families commands") @@ -42,6 +42,7 @@ public class FamiliesCommandOptions { public DeleteCommandOptions deleteCommandOptions; public InfoCommandOptions infoCommandOptions; public UpdateCommandOptions updateCommandOptions; + public UpdateAnnotationSetsAnnotationsCommandOptions updateAnnotationSetsAnnotationsCommandOptions; public FamiliesCommandOptions(CommonCommandOptions commonCommandOptions, JCommander jCommander) { @@ -58,6 +59,7 @@ public FamiliesCommandOptions(CommonCommandOptions commonCommandOptions, JComman this.deleteCommandOptions = new DeleteCommandOptions(); this.infoCommandOptions = new InfoCommandOptions(); this.updateCommandOptions = new UpdateCommandOptions(); + this.updateAnnotationSetsAnnotationsCommandOptions = new UpdateAnnotationSetsAnnotationsCommandOptions(); } @@ -70,8 +72,8 @@ public class UpdateAclCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -156,8 +158,8 @@ public class LoadAnnotationSetsCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -187,8 +189,8 @@ public class CreateCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--include", "-I"}, description = "Fields included in the response, whole JSON path must be provided", required = false, arity = 1) public String include; @@ -445,8 +447,8 @@ public class UpdateCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--include", "-I"}, description = "Fields included in the response, whole JSON path must be provided", required = false, arity = 1) public String include; @@ -496,5 +498,33 @@ public class UpdateCommandOptions { @Parameter(names = {"--status-description"}, description = "The body web service description parameter", required = false, arity = 1) public String statusDescription; + } + @Parameters(commandNames = {"annotationsets-annotations-update"}, commandDescription ="Update annotations from an annotationSet") + public class UpdateAnnotationSetsAnnotationsCommandOptions { + + @ParametersDelegate + public CommonCommandOptions commonOptions = commonCommandOptions; + + @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) + public String jsonFile; + + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; + + @Parameter(names = {"--family"}, description = "Family id", required = true, arity = 1) + public String family; + + @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) + public String study; + + @Parameter(names = {"--annotation-set"}, description = "AnnotationSet ID to be updated.", required = true, arity = 1) + public String annotationSet; + + @Parameter(names = {"--action"}, description = "Action to be performed: ADD to add new annotations; REPLACE to replace the value of an already existing annotation; SET to set the new list of annotations removing any possible old annotations; REMOVE to remove some annotations; RESET to set some annotations to the default value configured in the corresponding variables of the VariableSet if any.", required = false, arity = 1) + public String action; + + @Parameter(names = {"--inc-version"}, description = "Create a new version of family", required = false, arity = 1) + public Boolean incVersion; + } } \ No newline at end of file diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/FilesCommandOptions.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/FilesCommandOptions.java index c38d4cf1d8b..b3bee1f1ec2 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/FilesCommandOptions.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/FilesCommandOptions.java @@ -16,7 +16,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 +* Autogenerated on: 2022-03-25 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -25,7 +25,7 @@ /** * This class contains methods for the Files command line. - * OpenCGA version: 2.2.0-rc2-SNAPSHOT + * OpenCGA version: 2.2.0-SNAPSHOT * PATH: /{apiVersion}/files */ @Parameters(commandNames = {"files"}, commandDescription = "Files commands") @@ -42,11 +42,13 @@ public class FilesCommandOptions extends ParentFilesCommandOptions { public RunLinkCommandOptions runLinkCommandOptions; public RunPostlinkCommandOptions runPostlinkCommandOptions; public SearchCommandOptions searchCommandOptions; + public UploadCommandOptions uploadCommandOptions; public AclCommandOptions aclCommandOptions; public DeleteCommandOptions deleteCommandOptions; public InfoCommandOptions infoCommandOptions; public UnlinkCommandOptions unlinkCommandOptions; public UpdateCommandOptions updateCommandOptions; + public UpdateAnnotationSetsAnnotationsCommandOptions updateAnnotationSetsAnnotationsCommandOptions; public DownloadCommandOptions downloadCommandOptions; public GrepCommandOptions grepCommandOptions; public HeadCommandOptions headCommandOptions; @@ -70,11 +72,13 @@ public FilesCommandOptions(CommonCommandOptions commonCommandOptions, JCommander this.runLinkCommandOptions = new RunLinkCommandOptions(); this.runPostlinkCommandOptions = new RunPostlinkCommandOptions(); this.searchCommandOptions = new SearchCommandOptions(); + this.uploadCommandOptions = new UploadCommandOptions(); this.aclCommandOptions = new AclCommandOptions(); this.deleteCommandOptions = new DeleteCommandOptions(); this.infoCommandOptions = new InfoCommandOptions(); this.unlinkCommandOptions = new UnlinkCommandOptions(); this.updateCommandOptions = new UpdateCommandOptions(); + this.updateAnnotationSetsAnnotationsCommandOptions = new UpdateAnnotationSetsAnnotationsCommandOptions(); this.downloadCommandOptions = new DownloadCommandOptions(); this.grepCommandOptions = new GrepCommandOptions(); this.headCommandOptions = new HeadCommandOptions(); @@ -95,8 +99,8 @@ public class UpdateAclCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -193,8 +197,8 @@ public class LoadAnnotationSetsCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -224,8 +228,8 @@ public class CreateCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -388,8 +392,8 @@ public class FetchCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--job-id"}, description = "Job ID. It must be a unique string within the study. An ID will be autogenerated automatically if not provided.", required = false, arity = 1) public String jobId; @@ -422,8 +426,8 @@ public class LinkCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -465,8 +469,8 @@ public class RunLinkCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -505,8 +509,8 @@ public class RunPostlinkCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -715,8 +719,8 @@ public class UpdateCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--include", "-I"}, description = "Fields included in the response, whole JSON path must be provided", required = false, arity = 1) public String include; @@ -811,6 +815,31 @@ public class UpdateCommandOptions { @Parameter(names = {"--status-description"}, description = "The body web service description parameter", required = false, arity = 1) public String statusDescription; + } + @Parameters(commandNames = {"annotationsets-annotations-update"}, commandDescription ="Update annotations from an annotationSet") + public class UpdateAnnotationSetsAnnotationsCommandOptions { + + @ParametersDelegate + public CommonCommandOptions commonOptions = commonCommandOptions; + + @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) + public String jsonFile; + + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; + + @Parameter(names = {"--file"}, description = "File id, name or path. Paths must be separated by : instead of /", required = true, arity = 1) + public String file; + + @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) + public String study; + + @Parameter(names = {"--annotation-set"}, description = "AnnotationSet ID to be updated.", required = true, arity = 1) + public String annotationSet; + + @Parameter(names = {"--action"}, description = "Action to be performed: ADD to add new annotations; REPLACE to replace the value of an already existing annotation; SET to set the new list of annotations removing any possible old annotations; REMOVE to remove some annotations; RESET to set some annotations to the default value configured in the corresponding variables of the VariableSet if any.", required = false, arity = 1) + public String action; + } @Parameters(commandNames = {"download"}, commandDescription ="Download file") public class DownloadCommandOptions { diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/IndividualsCommandOptions.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/IndividualsCommandOptions.java index 1a047517eca..e951726012e 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/IndividualsCommandOptions.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/IndividualsCommandOptions.java @@ -14,7 +14,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 +* Autogenerated on: 2022-03-25 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -23,7 +23,7 @@ /** * This class contains methods for the Individuals command line. - * OpenCGA version: 2.2.0-rc2-SNAPSHOT + * OpenCGA version: 2.2.0-SNAPSHOT * PATH: /{apiVersion}/individuals */ @Parameters(commandNames = {"individuals"}, commandDescription = "Individuals commands") @@ -42,6 +42,7 @@ public class IndividualsCommandOptions { public DeleteCommandOptions deleteCommandOptions; public InfoCommandOptions infoCommandOptions; public UpdateCommandOptions updateCommandOptions; + public UpdateAnnotationSetsAnnotationsCommandOptions updateAnnotationSetsAnnotationsCommandOptions; public RelativesCommandOptions relativesCommandOptions; @@ -59,6 +60,7 @@ public IndividualsCommandOptions(CommonCommandOptions commonCommandOptions, JCom this.deleteCommandOptions = new DeleteCommandOptions(); this.infoCommandOptions = new InfoCommandOptions(); this.updateCommandOptions = new UpdateCommandOptions(); + this.updateAnnotationSetsAnnotationsCommandOptions = new UpdateAnnotationSetsAnnotationsCommandOptions(); this.relativesCommandOptions = new RelativesCommandOptions(); } @@ -72,8 +74,8 @@ public class UpdateAclCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -176,8 +178,8 @@ public class LoadAnnotationSetsCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -207,8 +209,8 @@ public class CreateCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--include", "-I"}, description = "Fields included in the response, whole JSON path must be provided", required = false, arity = 1) public String include; @@ -573,8 +575,8 @@ public class UpdateCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--include", "-I"}, description = "Fields included in the response, whole JSON path must be provided", required = false, arity = 1) public String include; @@ -678,6 +680,34 @@ public class UpdateCommandOptions { @Parameter(names = {"--status-description"}, description = "The body web service description parameter", required = false, arity = 1) public String statusDescription; + } + @Parameters(commandNames = {"annotationsets-annotations-update"}, commandDescription ="Update annotations from an annotationSet") + public class UpdateAnnotationSetsAnnotationsCommandOptions { + + @ParametersDelegate + public CommonCommandOptions commonOptions = commonCommandOptions; + + @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) + public String jsonFile; + + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; + + @Parameter(names = {"--individual"}, description = "Individual ID, name or UUID", required = true, arity = 1) + public String individual; + + @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) + public String study; + + @Parameter(names = {"--annotation-set"}, description = "AnnotationSet ID to be updated.", required = true, arity = 1) + public String annotationSet; + + @Parameter(names = {"--action"}, description = "Action to be performed: ADD to add new annotations; REPLACE to replace the value of an already existing annotation; SET to set the new list of annotations removing any possible old annotations; REMOVE to remove some annotations; RESET to set some annotations to the default value configured in the corresponding variables of the VariableSet if any.", required = false, arity = 1) + public String action; + + @Parameter(names = {"--inc-version"}, description = "Create a new version of individual", required = false, arity = 1) + public Boolean incVersion; + } @Parameters(commandNames = {"relatives"}, commandDescription ="Get individual relatives") public class RelativesCommandOptions { diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/JobsCommandOptions.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/JobsCommandOptions.java index 1f2d31fc653..90e57cb6e61 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/JobsCommandOptions.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/JobsCommandOptions.java @@ -7,6 +7,8 @@ import java.util.List; +import org.opencb.opencga.app.cli.main.parent.ParentJobsCommandOptions; + import static org.opencb.opencga.app.cli.GeneralCliOptions.*; @@ -14,7 +16,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 +* Autogenerated on: 2022-03-25 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -23,14 +25,12 @@ /** * This class contains methods for the Jobs command line. - * OpenCGA version: 2.2.0-rc2-SNAPSHOT + * OpenCGA version: 2.2.0-SNAPSHOT * PATH: /{apiVersion}/jobs */ @Parameters(commandNames = {"jobs"}, commandDescription = "Jobs commands") -public class JobsCommandOptions { +public class JobsCommandOptions extends ParentJobsCommandOptions { - public JCommander jCommander; - public CommonCommandOptions commonCommandOptions; public UpdateAclCommandOptions updateAclCommandOptions; public AggregationStatsCommandOptions aggregationStatsCommandOptions; @@ -49,8 +49,7 @@ public class JobsCommandOptions { public JobsCommandOptions(CommonCommandOptions commonCommandOptions, JCommander jCommander) { - this.jCommander = jCommander; - this.commonCommandOptions = commonCommandOptions; + super(commonCommandOptions,jCommander); this.updateAclCommandOptions = new UpdateAclCommandOptions(); this.aggregationStatsCommandOptions = new AggregationStatsCommandOptions(); this.createCommandOptions = new CreateCommandOptions(); @@ -76,8 +75,8 @@ public class UpdateAclCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--members"}, description = "Comma separated list of user or group ids", required = true, arity = 1) public String members; @@ -162,8 +161,8 @@ public class CreateCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -275,8 +274,8 @@ public class RetryCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--job-id"}, description = "Job ID. It must be a unique string within the study. An ID will be autogenerated automatically if not provided.", required = false, arity = 1) public String jobId; @@ -378,31 +377,6 @@ public class SearchCommandOptions { @Parameter(names = {"--deleted"}, description = "Boolean to retrieve deleted entries", required = false, arity = 1) public Boolean deleted; - } - @Parameters(commandNames = {"top"}, commandDescription ="Provide a summary of the running jobs") - public class TopCommandOptions { - - @ParametersDelegate - public CommonCommandOptions commonOptions = commonCommandOptions; - - @Parameter(names = {"--limit"}, description = "Maximum number of jobs to be returned", required = false, arity = 1) - public Integer limit; - - @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) - public String study; - - @Parameter(names = {"--internal-status"}, description = "Filter by internal status", required = false, arity = 1) - public String internalStatus; - - @Parameter(names = {"--priority"}, description = "Priority of the job", required = false, arity = 1) - public String priority; - - @Parameter(names = {"--user-id"}, description = "User that created the job", required = false, arity = 1) - public String userId; - - @Parameter(names = {"--tool-id"}, description = "Tool ID executed by the job", required = false, arity = 1) - public String toolId; - } @Parameters(commandNames = {"acl"}, commandDescription ="Return the acl of the job. If member is provided, it will only return the acl for the member.") public class AclCommandOptions { @@ -464,8 +438,8 @@ public class UpdateCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--include", "-I"}, description = "Fields included in the response, whole JSON path must be provided", required = false, arity = 1) public String include; diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/MetaCommandOptions.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/MetaCommandOptions.java index 5ddf1ca7a18..a77a54feaef 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/MetaCommandOptions.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/MetaCommandOptions.java @@ -14,7 +14,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 +* Autogenerated on: 2022-03-25 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -23,7 +23,7 @@ /** * This class contains methods for the Meta command line. - * OpenCGA version: 2.2.0-rc2-SNAPSHOT + * OpenCGA version: 2.2.0-SNAPSHOT * PATH: /{apiVersion}/meta */ @Parameters(commandNames = {"meta"}, commandDescription = "Meta commands") diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/OperationsVariantStorageCommandOptions.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/OperationsVariantStorageCommandOptions.java index 4ab2e75b729..12532ec2049 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/OperationsVariantStorageCommandOptions.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/OperationsVariantStorageCommandOptions.java @@ -14,7 +14,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 +* Autogenerated on: 2022-03-25 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -23,7 +23,7 @@ /** * This class contains methods for the Operations - Variant Storage command line. - * OpenCGA version: 2.2.0-rc2-SNAPSHOT + * OpenCGA version: 2.2.0-SNAPSHOT * PATH: /{apiVersion}/operation */ @Parameters(commandNames = {"operations"}, commandDescription = "Operations - Variant Storage commands") @@ -48,6 +48,7 @@ public class OperationsVariantStorageCommandOptions { public SynchronizeVariantMetadataCommandOptions synchronizeVariantMetadataCommandOptions; public DeleteVariantSampleCommandOptions deleteVariantSampleCommandOptions; public IndexVariantSampleCommandOptions indexVariantSampleCommandOptions; + public VariantSampleIndexConfigureCommandOptions variantSampleIndexConfigureCommandOptions; public DeleteVariantScoreCommandOptions deleteVariantScoreCommandOptions; public IndexVariantScoreCommandOptions indexVariantScoreCommandOptions; public SecondaryIndexVariantCommandOptions secondaryIndexVariantCommandOptions; @@ -77,6 +78,7 @@ public OperationsVariantStorageCommandOptions(CommonCommandOptions commonCommand this.synchronizeVariantMetadataCommandOptions = new SynchronizeVariantMetadataCommandOptions(); this.deleteVariantSampleCommandOptions = new DeleteVariantSampleCommandOptions(); this.indexVariantSampleCommandOptions = new IndexVariantSampleCommandOptions(); + this.variantSampleIndexConfigureCommandOptions = new VariantSampleIndexConfigureCommandOptions(); this.deleteVariantScoreCommandOptions = new DeleteVariantScoreCommandOptions(); this.indexVariantScoreCommandOptions = new IndexVariantScoreCommandOptions(); this.secondaryIndexVariantCommandOptions = new SecondaryIndexVariantCommandOptions(); @@ -96,8 +98,8 @@ public class ConfigureCellbaseCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--project", "-p"}, description = "Project [user@]project where project can be either the ID or the alias", required = false, arity = 1) public String project; @@ -124,8 +126,8 @@ public class AggregateVariantCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--job-id"}, description = "Job ID. It must be a unique string within the study. An ID will be autogenerated automatically if not provided.", required = false, arity = 1) public String jobId; @@ -183,8 +185,8 @@ public class IndexVariantAnnotationCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--job-id"}, description = "Job ID. It must be a unique string within the study. An ID will be autogenerated automatically if not provided.", required = false, arity = 1) public String jobId; @@ -238,8 +240,8 @@ public class SaveVariantAnnotationCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--job-id"}, description = "Job ID. It must be a unique string within the study. An ID will be autogenerated automatically if not provided.", required = false, arity = 1) public String jobId; @@ -269,8 +271,8 @@ public class ConfigureVariantCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--project", "-p"}, description = "Project [user@]project where project can be either the ID or the alias", required = false, arity = 1) public String project; @@ -288,8 +290,8 @@ public class DeleteVariantCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--job-id"}, description = "Job ID. It must be a unique string within the study. An ID will be autogenerated automatically if not provided.", required = false, arity = 1) public String jobId; @@ -322,8 +324,8 @@ public class AggregateVariantFamilyCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--job-id"}, description = "Job ID. It must be a unique string within the study. An ID will be autogenerated automatically if not provided.", required = false, arity = 1) public String jobId; @@ -359,8 +361,8 @@ public class IndexVariantFamilyCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--job-id"}, description = "Job ID. It must be a unique string within the study. An ID will be autogenerated automatically if not provided.", required = false, arity = 1) public String jobId; @@ -383,6 +385,9 @@ public class IndexVariantFamilyCommandOptions { @Parameter(names = {"--overwrite"}, description = "The body web service overwrite parameter", required = false, arity = 1) public Boolean overwrite; + @Parameter(names = {"--update-index"}, description = "The body web service updateIndex parameter", required = false, arity = 1) + public Boolean updateIndex; + @Parameter(names = {"--skip-incomplete-families"}, description = "The body web service skipIncompleteFamilies parameter", required = false, arity = 1) public Boolean skipIncompleteFamilies; @@ -396,8 +401,8 @@ public class IndexVariantCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--job-id"}, description = "Job ID. It must be a unique string within the study. An ID will be autogenerated automatically if not provided.", required = false, arity = 1) public String jobId; @@ -508,8 +513,8 @@ public class LauncherVariantIndexCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--job-id"}, description = "Job ID. It must be a unique string within the study. An ID will be autogenerated automatically if not provided.", required = false, arity = 1) public String jobId; @@ -635,8 +640,8 @@ public class RunVariantJulieCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--job-id"}, description = "Job ID. It must be a unique string within the study. An ID will be autogenerated automatically if not provided.", required = false, arity = 1) public String jobId; @@ -672,8 +677,8 @@ public class RepairVariantMetadataCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--job-id"}, description = "Job ID. It must be a unique string within the study. An ID will be autogenerated automatically if not provided.", required = false, arity = 1) public String jobId; @@ -703,8 +708,8 @@ public class SynchronizeVariantMetadataCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--job-id"}, description = "Job ID. It must be a unique string within the study. An ID will be autogenerated automatically if not provided.", required = false, arity = 1) public String jobId; @@ -737,8 +742,8 @@ public class DeleteVariantSampleCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--job-id"}, description = "Job ID. It must be a unique string within the study. An ID will be autogenerated automatically if not provided.", required = false, arity = 1) public String jobId; @@ -774,8 +779,8 @@ public class IndexVariantSampleCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--job-id"}, description = "Job ID. It must be a unique string within the study. An ID will be autogenerated automatically if not provided.", required = false, arity = 1) public String jobId; @@ -804,6 +809,25 @@ public class IndexVariantSampleCommandOptions { @Parameter(names = {"--overwrite"}, description = "The body web service overwrite parameter", required = false, arity = 1) public Boolean overwrite; + } + @Parameters(commandNames = {"variant-sample-index-configure"}, commandDescription ="Update SampleIndex configuration") + public class VariantSampleIndexConfigureCommandOptions { + + @ParametersDelegate + public CommonCommandOptions commonOptions = commonCommandOptions; + + @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) + public String jsonFile; + + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; + + @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) + public String study; + + @Parameter(names = {"--skip-rebuild"}, description = "Skip sample index re-build", required = false, arity = 1) + public Boolean skipRebuild; + } @Parameters(commandNames = {"variant-score-delete"}, commandDescription ="Remove a variant score in the database") public class DeleteVariantScoreCommandOptions { @@ -845,8 +869,8 @@ public class IndexVariantScoreCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--job-id"}, description = "Job ID. It must be a unique string within the study. An ID will be autogenerated automatically if not provided.", required = false, arity = 1) public String jobId; @@ -891,8 +915,8 @@ public class SecondaryIndexVariantCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--job-id"}, description = "Job ID. It must be a unique string within the study. An ID will be autogenerated automatically if not provided.", required = false, arity = 1) public String jobId; @@ -956,8 +980,8 @@ public class DeleteVariantStatsCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -990,8 +1014,8 @@ public class IndexVariantStatsCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -1033,8 +1057,8 @@ public class DeleteVariantStudyCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--job-id"}, description = "Job ID. It must be a unique string within the study. An ID will be autogenerated automatically if not provided.", required = false, arity = 1) public String jobId; diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/ProjectsCommandOptions.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/ProjectsCommandOptions.java index 18f76dec674..6932e647a2b 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/ProjectsCommandOptions.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/ProjectsCommandOptions.java @@ -14,7 +14,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 +* Autogenerated on: 2022-03-25 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -23,7 +23,7 @@ /** * This class contains methods for the Projects command line. - * OpenCGA version: 2.2.0-rc2-SNAPSHOT + * OpenCGA version: 2.2.0-SNAPSHOT * PATH: /{apiVersion}/projects */ @Parameters(commandNames = {"projects"}, commandDescription = "Projects commands") @@ -36,6 +36,7 @@ public class ProjectsCommandOptions { public SearchCommandOptions searchCommandOptions; public AggregationStatsCommandOptions aggregationStatsCommandOptions; public InfoCommandOptions infoCommandOptions; + public IncReleaseCommandOptions incReleaseCommandOptions; public StudiesCommandOptions studiesCommandOptions; public UpdateCommandOptions updateCommandOptions; @@ -48,6 +49,7 @@ public ProjectsCommandOptions(CommonCommandOptions commonCommandOptions, JComman this.searchCommandOptions = new SearchCommandOptions(); this.aggregationStatsCommandOptions = new AggregationStatsCommandOptions(); this.infoCommandOptions = new InfoCommandOptions(); + this.incReleaseCommandOptions = new IncReleaseCommandOptions(); this.studiesCommandOptions = new StudiesCommandOptions(); this.updateCommandOptions = new UpdateCommandOptions(); @@ -62,8 +64,8 @@ public class CreateCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--include", "-I"}, description = "Fields included in the response, whole JSON path must be provided", required = false, arity = 1) public String include; @@ -197,6 +199,22 @@ public class InfoCommandOptions { @Parameter(names = {"--projects"}, description = "Comma separated list of projects [user@]project up to a maximum of 100", required = true, arity = 1) public String projects; + } + @Parameters(commandNames = {"increlease"}, commandDescription ="Increment current release number in the project") + public class IncReleaseCommandOptions { + + @ParametersDelegate + public CommonCommandOptions commonOptions = commonCommandOptions; + + @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) + public String jsonFile; + + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; + + @Parameter(names = {"--project", "-p"}, description = "Project [user@]project where project can be either the ID or the alias", required = true, arity = 1) + public String project; + } @Parameters(commandNames = {"studies"}, commandDescription ="Fetch all the studies contained in the project") public class StudiesCommandOptions { @@ -229,8 +247,8 @@ public class UpdateCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--include", "-I"}, description = "Fields included in the response, whole JSON path must be provided", required = false, arity = 1) public String include; diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/SamplesCommandOptions.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/SamplesCommandOptions.java index 08fb6fb0333..cc0bd17d9fa 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/SamplesCommandOptions.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/SamplesCommandOptions.java @@ -14,7 +14,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 +* Autogenerated on: 2022-03-25 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -23,7 +23,7 @@ /** * This class contains methods for the Samples command line. - * OpenCGA version: 2.2.0-rc2-SNAPSHOT + * OpenCGA version: 2.2.0-SNAPSHOT * PATH: /{apiVersion}/samples */ @Parameters(commandNames = {"samples"}, commandDescription = "Samples commands") @@ -43,6 +43,7 @@ public class SamplesCommandOptions { public DeleteCommandOptions deleteCommandOptions; public InfoCommandOptions infoCommandOptions; public UpdateCommandOptions updateCommandOptions; + public UpdateAnnotationSetsAnnotationsCommandOptions updateAnnotationSetsAnnotationsCommandOptions; public SamplesCommandOptions(CommonCommandOptions commonCommandOptions, JCommander jCommander) { @@ -60,6 +61,7 @@ public SamplesCommandOptions(CommonCommandOptions commonCommandOptions, JCommand this.deleteCommandOptions = new DeleteCommandOptions(); this.infoCommandOptions = new InfoCommandOptions(); this.updateCommandOptions = new UpdateCommandOptions(); + this.updateAnnotationSetsAnnotationsCommandOptions = new UpdateAnnotationSetsAnnotationsCommandOptions(); } @@ -72,8 +74,8 @@ public class UpdateAclCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -164,8 +166,8 @@ public class LoadAnnotationSetsCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) public String study; @@ -195,8 +197,8 @@ public class CreateCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--include", "-I"}, description = "Fields included in the response, whole JSON path must be provided", required = false, arity = 1) public String include; @@ -643,8 +645,8 @@ public class UpdateCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--include", "-I"}, description = "Fields included in the response, whole JSON path must be provided", required = false, arity = 1) public String include; @@ -733,5 +735,33 @@ public class UpdateCommandOptions { @Parameter(names = {"--status-description"}, description = "The body web service description parameter", required = false, arity = 1) public String statusDescription; + } + @Parameters(commandNames = {"annotationsets-annotations-update"}, commandDescription ="Update annotations from an annotationSet") + public class UpdateAnnotationSetsAnnotationsCommandOptions { + + @ParametersDelegate + public CommonCommandOptions commonOptions = commonCommandOptions; + + @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) + public String jsonFile; + + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; + + @Parameter(names = {"--sample"}, description = "Sample ID", required = true, arity = 1) + public String sample; + + @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = false, arity = 1) + public String study; + + @Parameter(names = {"--annotation-set"}, description = "AnnotationSet ID to be updated.", required = true, arity = 1) + public String annotationSet; + + @Parameter(names = {"--action"}, description = "Action to be performed: ADD to add new annotations; REPLACE to replace the value of an already existing annotation; SET to set the new list of annotations removing any possible old annotations; REMOVE to remove some annotations; RESET to set some annotations to the default value configured in the corresponding variables of the VariableSet if any.", required = false, arity = 1) + public String action; + + @Parameter(names = {"--inc-version"}, description = "Create a new version of sample", required = false, arity = 1) + public Boolean incVersion; + } } \ No newline at end of file diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/StudiesCommandOptions.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/StudiesCommandOptions.java index 89d50cc8d68..61593773d53 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/StudiesCommandOptions.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/StudiesCommandOptions.java @@ -16,7 +16,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 +* Autogenerated on: 2022-03-25 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -25,7 +25,7 @@ /** * This class contains methods for the Studies command line. - * OpenCGA version: 2.2.0-rc2-SNAPSHOT + * OpenCGA version: 2.2.0-SNAPSHOT * PATH: /{apiVersion}/studies */ @Parameters(commandNames = {"studies"}, commandDescription = "Studies commands") @@ -45,6 +45,7 @@ public class StudiesCommandOptions extends ParentStudiesCommandOptions { public PermissionRulesCommandOptions permissionRulesCommandOptions; public UpdatePermissionRulesCommandOptions updatePermissionRulesCommandOptions; public RunTemplatesCommandOptions runTemplatesCommandOptions; + public UploadTemplatesCommandOptions uploadTemplatesCommandOptions; public DeleteTemplatesCommandOptions deleteTemplatesCommandOptions; public UpdateCommandOptions updateCommandOptions; public VariableSetsCommandOptions variableSetsCommandOptions; @@ -68,6 +69,7 @@ public StudiesCommandOptions(CommonCommandOptions commonCommandOptions, JCommand this.permissionRulesCommandOptions = new PermissionRulesCommandOptions(); this.updatePermissionRulesCommandOptions = new UpdatePermissionRulesCommandOptions(); this.runTemplatesCommandOptions = new RunTemplatesCommandOptions(); + this.uploadTemplatesCommandOptions = new UploadTemplatesCommandOptions(); this.deleteTemplatesCommandOptions = new DeleteTemplatesCommandOptions(); this.updateCommandOptions = new UpdateCommandOptions(); this.variableSetsCommandOptions = new VariableSetsCommandOptions(); @@ -85,8 +87,8 @@ public class UpdateAclCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--members"}, description = "Comma separated list of user or group ids", required = true, arity = 1) public String members; @@ -113,8 +115,8 @@ public class CreateCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--include", "-I"}, description = "Fields included in the response, whole JSON path must be provided", required = false, arity = 1) public String include; @@ -354,8 +356,8 @@ public class UpdateGroupsCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = true, arity = 1) public String study; @@ -379,8 +381,8 @@ public class UpdateGroupsUsersCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = true, arity = 1) public String study; @@ -417,8 +419,8 @@ public class UpdatePermissionRulesCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = true, arity = 1) public String study; @@ -461,8 +463,8 @@ public class UpdateCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--include", "-I"}, description = "Fields included in the response, whole JSON path must be provided", required = false, arity = 1) public String include; @@ -529,8 +531,8 @@ public class UpdateVariableSetsCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = true, arity = 1) public String study; @@ -563,8 +565,8 @@ public class UpdateVariableSetsVariablesCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or UUID", required = true, arity = 1) public String study; diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/UsersCommandOptions.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/UsersCommandOptions.java index 51b58dd2d34..6d4b1072769 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/UsersCommandOptions.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/options/UsersCommandOptions.java @@ -16,7 +16,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 +* Autogenerated on: 2022-03-25 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -25,7 +25,7 @@ /** * This class contains methods for the Users command line. - * OpenCGA version: 2.2.0-rc2-SNAPSHOT + * OpenCGA version: 2.2.0-SNAPSHOT * PATH: /{apiVersion}/users */ @Parameters(commandNames = {"users"}, commandDescription = "Users commands") @@ -69,8 +69,8 @@ public class CreateCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--id"}, description = "The body web service id parameter", required = true, arity = 1) public String id; @@ -97,8 +97,8 @@ public class PasswordCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--user", "-u"}, description = "The body web service user parameter", required = false, arity = 1) public String user; @@ -151,8 +151,8 @@ public class UpdateConfigsCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--user", "-u"}, description = "User ID", required = true, arity = 1) public String user; @@ -218,8 +218,8 @@ public class UpdateCommandOptions { @Parameter(names = {"--json-file"}, description = "File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.", required = false, arity = 1) public String jsonFile; - @Parameter(names = {"--json-view-template"}, description = "Show example of file structure for body data.", help = true, arity = 0) - public Boolean jsonViewTemplate= false; + @Parameter(names = {"--json-data-model"}, description = "Show example of file structure for body data.", help = true, arity = 0) + public Boolean jsonDataModel = false; @Parameter(names = {"--include", "-I"}, description = "Fields included in the response, whole JSON path must be provided", required = false, arity = 1) public String include; diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/parent/ParentJobsCommandExecutor.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/parent/ParentJobsCommandExecutor.java new file mode 100644 index 00000000000..f6b196b23b0 --- /dev/null +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/parent/ParentJobsCommandExecutor.java @@ -0,0 +1,66 @@ +/* + * Copyright 2015-2017 OpenCB + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.opencb.opencga.app.cli.main.parent; + +import org.opencb.commons.datastore.core.Query; +import org.opencb.opencga.app.cli.GeneralCliOptions; +import org.opencb.opencga.app.cli.main.executors.OpencgaCommandExecutor; +import org.opencb.opencga.app.cli.main.options.JobsCommandOptions; +import org.opencb.opencga.app.cli.main.utils.JobsLog; +import org.opencb.opencga.app.cli.main.utils.JobsTopManager; +import org.opencb.opencga.catalog.db.api.JobDBAdaptor; +import org.opencb.opencga.catalog.exceptions.CatalogAuthenticationException; +import org.opencb.opencga.core.api.ParamConstants; +import org.opencb.opencga.core.models.job.JobTop; +import org.opencb.opencga.core.response.QueryType; +import org.opencb.opencga.core.response.RestResponse; + +public abstract class ParentJobsCommandExecutor extends OpencgaCommandExecutor { + + private final JobsCommandOptions jobsCommandOptions; + + public ParentJobsCommandExecutor(GeneralCliOptions.CommonCommandOptions options, + JobsCommandOptions jobsCommandOptions) throws CatalogAuthenticationException { + super(options); + this.jobsCommandOptions = jobsCommandOptions; + } + + protected RestResponse top() throws Exception { + JobsCommandOptions.TopCommandOptions c = jobsCommandOptions.topCommandOptions; + + Query query = new Query(); + query.putIfNotEmpty(JobDBAdaptor.QueryParams.STUDY.key(), c.study); + query.putIfNotEmpty(ParamConstants.JOB_TOOL_ID_PARAM, c.toolId); + query.putIfNotEmpty(ParamConstants.INTERNAL_STATUS_PARAM, c.internalStatus); + query.putIfNotEmpty(ParamConstants.JOB_USER_PARAM, c.userId); + query.putIfNotEmpty(ParamConstants.JOB_PRIORITY_PARAM, c.priority); + query.putAll(c.commonOptions.params); + + new JobsTopManager(openCGAClient, query, c.iterations, c.jobsLimit, c.delay, c.plain, c.columns).run(); + RestResponse res = new RestResponse<>(); + res.setType(QueryType.VOID); + return res; + } + + protected RestResponse log() throws Exception { + JobsCommandOptions.LogCommandOptions c = jobsCommandOptions.logCommandOptions; + new JobsLog(openCGAClient, c, System.out).run(); + RestResponse res = new RestResponse<>(); + res.setType(QueryType.VOID); + return res; + } + +} diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/parent/ParentJobsCommandOptions.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/parent/ParentJobsCommandOptions.java new file mode 100644 index 00000000000..4e7f23e6b39 --- /dev/null +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/parent/ParentJobsCommandOptions.java @@ -0,0 +1,102 @@ +package org.opencb.opencga.app.cli.main.parent; + +import com.beust.jcommander.JCommander; +import com.beust.jcommander.Parameter; +import com.beust.jcommander.Parameters; +import com.beust.jcommander.ParametersDelegate; +import org.opencb.opencga.core.api.ParamConstants; + +import static org.opencb.opencga.app.cli.GeneralCliOptions.CommonCommandOptions; +import static org.opencb.opencga.core.tools.result.Status.Type.RUNNING; +/* + * WARNING: AUTOGENERATED CODE + * + * This code was generated by a tool. + * Autogenerated on: 2021-10-11 + * + * Manual changes to this file may cause unexpected behavior in your application. + * Manual changes to this file will be overwritten if the code is regenerated. + */ + +/** + * This class contains methods for the Files command line. OpenCGA version: 2.2.0-SNAPSHOT PATH: /{apiVersion}/files + */ +@Parameters(commandNames = {"jobs"}, commandDescription = "Jobs commands") +public class ParentJobsCommandOptions { + + public JCommander jCommander; + public CommonCommandOptions commonCommandOptions; + public TopCommandOptions topCommandOptions; + public LogCommandOptions logCommandOptions; + + public ParentJobsCommandOptions(CommonCommandOptions commonCommandOptions, JCommander jCommander) { + this.jCommander = jCommander; + this.commonCommandOptions = commonCommandOptions; + topCommandOptions = new TopCommandOptions(); + logCommandOptions = new LogCommandOptions(); + } + + @Parameters(commandNames = {"top"}, commandDescription = "Provide a view of jobs activity in real time.") + public class TopCommandOptions { + @ParametersDelegate + public CommonCommandOptions commonOptions = commonCommandOptions; + + @Parameter(names = {"-d", "--delay"}, description = "Delay between iterations in seconds", arity = 1) + public int delay = 2; + + @Parameter(names = {"--plain"}, description = "Plain representation, without dependencies.", arity = 0) + public boolean plain; + + @Parameter(names = {"--columns"}, description = "Output columns to print." + + " [ID, TOOL_ID, STATUS, EVENTS, STUDY, SUBMISSION, PRIORITY, RUNNING_TIME, START, END, INPUT, OUTPUT, OUTPUT_DIRECTORY]") + public String columns; + + @Parameter(names = { "--iterations"}, description = "Exit after N iterations", arity = 1) + public Integer iterations; + + @Parameter(names = {"-n", "--jobs"}, description = "Number of jobs to print", arity = 1) + public Integer jobsLimit; + + @Parameter(names = {"--tool-id"}, description = ParamConstants.JOB_TOOL_ID_DESCRIPTION, arity = 1) + public String toolId; + + @Parameter(names = {"--user-id"}, description = ParamConstants.JOB_USER_DESCRIPTION, arity = 1) + public String userId; + + @Parameter(names = {"--priority"}, description = ParamConstants.JOB_PRIORITY_DESCRIPTION, arity = 1) + public String priority; + + @Parameter(names = {"--internal-status"}, description = ParamConstants.JOB_STATUS_DESCRIPTION, arity = 1) + public String internalStatus; + + @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or" + + " UUID", required = false, arity = 1) + public String study; + } + + @Parameters(commandNames = {"log"}, commandDescription = "Upload a physical local file to catalog.") + public class LogCommandOptions { + + @ParametersDelegate + public CommonCommandOptions commonOptions = commonCommandOptions; + + @Parameter(names = {"--job"}, description = ParamConstants.JOB_ID_DESCRIPTION + " or 'running' to print all running jobs.") + public String job = "running"; + + @Parameter(names = {"--type"}, description = "Log file to be shown (stdout or stderr)") + public String type = "stderr"; + + @Parameter(names = {"-f", "--follow"}, description = "Output appended data as the file grows", arity = 0) + public boolean follow; + + @Parameter(names = {"-n", "--tail"}, description = "Output the last lines NUM lines.", arity = 1) + public Integer tailLines; + + @Parameter(names = {"-d", "--delay"}, description = "Delay between iterations in seconds", arity = 1) + public int delay = 2; + + @Parameter(names = {"--study", "-s"}, description = "Study [[user@]project:]study where study and project can be either the ID or" + + " UUID", required = false, arity = 1) + public String study; + } +} \ No newline at end of file diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/parent/ParentStudiesCommandExecutor.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/parent/ParentStudiesCommandExecutor.java index 6aca044706f..072fd6ad5c0 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/parent/ParentStudiesCommandExecutor.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/parent/ParentStudiesCommandExecutor.java @@ -60,9 +60,9 @@ protected RestResponse runTemplates() throws Exception { return openCGAClient.getStudyClient().runTemplates(c.study, templateParams, params); } - protected RestResponse templatesUpload() throws Exception { + protected RestResponse uploadTemplates() throws Exception { logger.debug("Upload template file"); - StudiesCommandOptions.TemplatesUploadCommandOptions c = studiesCommandOptions.templatesUploadCommandOptions; + StudiesCommandOptions.UploadTemplatesCommandOptions c = studiesCommandOptions.uploadTemplatesCommandOptions; ObjectMap params = new ObjectMap(); diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/parent/ParentStudiesCommandOptions.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/parent/ParentStudiesCommandOptions.java index c93720fe1fb..988c4cdae87 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/parent/ParentStudiesCommandOptions.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/parent/ParentStudiesCommandOptions.java @@ -24,19 +24,19 @@ public class ParentStudiesCommandOptions { public JCommander jCommander; public CommonCommandOptions commonCommandOptions; - public TemplatesUploadCommandOptions templatesUploadCommandOptions; + public UploadTemplatesCommandOptions uploadTemplatesCommandOptions; public RunTemplatesCommandOptions templateRunCommandOptions; public ParentStudiesCommandOptions(CommonCommandOptions commonCommandOptions, JCommander jCommander) { this.jCommander = jCommander; this.commonCommandOptions = commonCommandOptions; - this.templatesUploadCommandOptions = new TemplatesUploadCommandOptions(); + this.uploadTemplatesCommandOptions = new UploadTemplatesCommandOptions(); this.templateRunCommandOptions = new RunTemplatesCommandOptions(); } @Parameters(commandNames = {"templates-upload"}, commandDescription = "Upload template") - public class TemplatesUploadCommandOptions { + public class UploadTemplatesCommandOptions { @ParametersDelegate public CommonCommandOptions commonOptions = commonCommandOptions; diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/parent/ParentUsersCommandOptions.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/parent/ParentUsersCommandOptions.java index 3b4fcef4df5..6a17e4f9451 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/parent/ParentUsersCommandOptions.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/parent/ParentUsersCommandOptions.java @@ -51,8 +51,7 @@ public class LoginCommandOptions { @Parameter(names = {"-u", "--user"}, description = "User id, this must be unique in OpenCGA", required = true, arity = 1) public String user; - @Parameter(names = {"-p", "--password"}, description = "User password (DO NOT TYPE THE PASSWORD BELOW, YOU WILL " - + "BE ASKED WHEN YOU PRESS THE ENTER KEY)", arity = 0, required = true, password = true) + @Parameter(names = {"-p", "--password"}, description = "User password", arity = 0, required = true, password = true) public String password; @Parameter(names = {"--refresh-token"}, description = "The body web service refreshToken parameter", arity = 1) diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/processors/CommandProcessor.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/processors/CommandProcessor.java index a70c6d129c8..2a2db426a52 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/processors/CommandProcessor.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/processors/CommandProcessor.java @@ -17,6 +17,8 @@ import org.opencb.opencga.core.models.study.Study; import org.opencb.opencga.core.response.OpenCGAResult; import org.opencb.opencga.core.response.RestResponse; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import java.util.ArrayList; import java.util.List; @@ -25,6 +27,7 @@ public class CommandProcessor { + private static final Logger logger = LoggerFactory.getLogger(CommandProcessor.class); public void process(String[] args) { OpencgaCliOptionsParser cliOptionsParser = new OpencgaCliOptionsParser(); @@ -33,19 +36,23 @@ public void process(String[] args) { //2. Parse params of options files cliOptionsParser.parse(args); if (!ArrayUtils.isEmpty(args)) { - CommandLineUtils.debug("PARSED OPTIONS ::: " + ArrayUtils.toString(args)); + CommandLineUtils.debug("PARSED OPTIONS ::: " + CommandLineUtils.argsToString(args)); + logger.debug("PARSED OPTIONS ::: " + CommandLineUtils.argsToString(args)); try { // 3. Check if a command has been provided is valid String parsedCommand = cliOptionsParser.getCommand(); - CommandLineUtils.debug("COMMAND TO EXECUTE ::: " + ArrayUtils.toString(args)); + CommandLineUtils.debug("COMMAND TO EXECUTE ::: " + CommandLineUtils.argsToString(args)); + logger.debug("COMMAND TO EXECUTE ::: " + CommandLineUtils.argsToString(args)); if (cliOptionsParser.isValid(parsedCommand)) { // 4. Get command executor from ExecutorProvider CommandLineUtils.debug("COMMAND AND SUBCOMMAND ARE VALID"); + logger.debug("COMMAND AND SUBCOMMAND ARE VALID"); String parsedSubCommand = cliOptionsParser.getSubCommand(); OpencgaCommandExecutor commandExecutor = ExecutorProvider.getOpencgaCommandExecutor(cliOptionsParser, parsedCommand); // 5. Execute parsed command with executor provided using CommandProcessor Implementation - CommandLineUtils.debug("EXECUTING ::: " + ArrayUtils.toString(args)); + CommandLineUtils.debug("EXECUTING ::: " + CommandLineUtils.argsToString(args)); + logger.debug("EXECUTING ::: " + CommandLineUtils.argsToString(args)); if (commandExecutor != null) { try { @@ -54,10 +61,11 @@ public void process(String[] args) { loadSessionStudies(commandExecutor); } catch (Exception ex) { CommandLineUtils.error("Execution error: " + ex.getMessage(), ex); - //System.exit(1); + logger.error("Execution error: " + ex.getMessage(), ex); } } else { cliOptionsParser.printUsage(); + logger.error("Command Executor NULL"); System.exit(1); } } else { @@ -66,8 +74,11 @@ public void process(String[] args) { } catch (ParameterException e) { printWarn("\n" + e.getMessage()); cliOptionsParser.printUsage(); + logger.error("Parameter error: " + e.getMessage(), e); } catch (CatalogAuthenticationException e) { printWarn("\n" + e.getMessage()); + logger.error(e.getMessage(), e); + } } } @@ -75,6 +86,8 @@ public void process(String[] args) { } catch (Exception e) { CommandLineUtils.error(e); cliOptionsParser.printUsage(); + logger.error(e.getMessage(), e); + } } @@ -85,6 +98,8 @@ public void loadSessionStudies(OpencgaCommandExecutor commandExecutor) { if (!StringUtils.isEmpty(session.getToken()) && !SessionManager.NO_TOKEN.equals(session.getToken())) { CommandLineUtils.debug("Loading session studies using token: " + session.getToken()); + logger.debug("Loading session studies using token: " + + session.getToken()); OpenCGAClient openCGAClient = commandExecutor.getOpenCGAClient(); try { @@ -103,10 +118,12 @@ public void loadSessionStudies(OpencgaCommandExecutor commandExecutor) { if (CollectionUtils.isNotEmpty(studies)) { //Save all the user studies in session CommandLineUtils.debug("Retrieved studies: " + studies); + logger.debug("Retrieved studies: " + studies); session.setStudies(studies); CommandLineUtils.debug("Session studies: " + session.getStudies()); + logger.debug("Session studies: " + session.getStudies()); //If the List of studies retrieved doesn't contain the current study, must select the new one if (!studies.contains(session.getCurrentStudy())) { @@ -122,6 +139,7 @@ public void loadSessionStudies(OpencgaCommandExecutor commandExecutor) { } } CommandLineUtils.debug("Current study: " + session.getCurrentStudy()); + logger.debug("Current study: " + session.getCurrentStudy()); // If none is found, save the first one as the current study if (!find) { @@ -133,12 +151,12 @@ public void loadSessionStudies(OpencgaCommandExecutor commandExecutor) { } } catch (Exception e) { CommandLineUtils.error("Failure reloading studies ", e); - e.printStackTrace(); - CommandLineUtils.error("Session studies: " + commandExecutor.getSessionManager().getSession().getStudies().toString(), e); - CommandLineUtils.error("Current study: " + commandExecutor.getSessionManager().getSession().getCurrentStudy(), e); + logger.error("Failure reloading studies ", e); } CommandLineUtils.debug("Session studies: " + commandExecutor.getSessionManager().getSession().getStudies().toString()); CommandLineUtils.debug("Current study: " + commandExecutor.getSessionManager().getSession().getCurrentStudy()); + logger.debug("Session studies: " + commandExecutor.getSessionManager().getSession().getStudies().toString()); + logger.debug("Current study: " + commandExecutor.getSessionManager().getSession().getCurrentStudy()); } } diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/shell/Shell.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/shell/Shell.java index eaa27d495d9..be7fc39fe5d 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/shell/Shell.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/shell/Shell.java @@ -59,6 +59,7 @@ private LineReader getTerminal() { defaultHistory.save(); } catch (IOException e) { CommandLineUtils.error("Failed to save terminal history", e); + logger.error("Failed to save terminal history", e); } })); reader = LineReaderBuilder.builder() @@ -68,6 +69,7 @@ private LineReader getTerminal() { .build(); } catch (Exception e) { CommandLineUtils.error("Failed to create terminal ", e); + logger.error("Failed to create terminal ", e); } @@ -119,6 +121,9 @@ public void execute() throws Exception { CommandLineUtils.error("OpenCGA execution error ", e); CommandLineUtils.debug("sessionManager:" + sessionManager); CommandLineUtils.debug("getCliSession:" + sessionManager.getSession()); + logger.error("OpenCGA execution error ", e); + logger.debug("sessionManager:" + sessionManager); + logger.debug("getCliSession:" + sessionManager.getSession()); } } @@ -217,11 +222,15 @@ public void setValidatedCurrentStudy(String arg) { } } catch (ClientException e) { CommandLineUtils.error(e); + logger.error(e.getMessage(), e); + } catch (IOException e) { CommandLineUtils.error(e); + logger.error(e.getMessage(), e); } } else { printError("Client not available"); + logger.error("Client not available"); } } else { printWarn("To set a study you must be logged in"); diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/utils/CommandLineUtils.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/utils/CommandLineUtils.java index 62779e9ba67..71943a77dd5 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/utils/CommandLineUtils.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/utils/CommandLineUtils.java @@ -6,6 +6,7 @@ import org.opencb.opencga.app.cli.main.OpencgaMain; import org.opencb.opencga.core.common.GitRepositoryState; +import java.util.Arrays; import java.util.List; import java.util.logging.Level; @@ -14,7 +15,6 @@ public class CommandLineUtils { - public static String getVersionString() { String res = PrintUtils.getKeyValueAsFormattedString("\tOpenCGA CLI version: ", "\t" + GitRepositoryState.get().getBuildVersion() + "\n"); res += PrintUtils.getKeyValueAsFormattedString("\tGit version:", "\t\t" + GitRepositoryState.get().getBranch() + " " + GitRepositoryState.get().getCommitId() + "\n"); @@ -48,7 +48,6 @@ private static void printLevel(String message, Exception e, Level level) { } public static void printLog(String message, Exception e) { - if (OpencgaMain.getLogLevel().equals(Level.FINE)) { PrintUtils.printDebug(message); } else if (OpencgaMain.getLogLevel().equals(Level.INFO)) { @@ -96,7 +95,6 @@ public static void debug(String s) { CommandLineUtils.printLevel(s, Level.FINE); } - public static String[] processShortCuts(String[] args) { OpencgaCliOptionsParser cliOptionsParser = new OpencgaCliOptionsParser(); switch (getShortcut(args)) { @@ -166,10 +164,16 @@ public static void printArgs(String[] args) { } public static String argsToString(String[] args) { - String sArgs = ArrayUtils.toString(args); + + String[] res = Arrays.copyOf(args, args.length); + if (ArrayUtils.contains(res, "--password") && (ArrayUtils.indexOf(res, "--password") + 1) < res.length) { + res[(ArrayUtils.indexOf(res, "--password") + 1)] = "********"; + } + String sArgs = ArrayUtils.toString(res); sArgs = sArgs.replaceAll("\\{", ""); sArgs = sArgs.replaceAll("}", ""); sArgs = sArgs.replaceAll(",", " "); + return sArgs; } } diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/utils/JobsLog.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/utils/JobsLog.java new file mode 100644 index 00000000000..8e715c94027 --- /dev/null +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/utils/JobsLog.java @@ -0,0 +1,238 @@ +/* + * Copyright 2015-2020 OpenCB + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.opencb.opencga.app.cli.main.utils; + +import com.google.common.base.Stopwatch; +import org.opencb.commons.datastore.core.ObjectMap; +import org.opencb.commons.datastore.core.QueryOptions; +import org.opencb.opencga.app.cli.main.options.JobsCommandOptions; +import org.opencb.opencga.catalog.db.api.JobDBAdaptor; +import org.opencb.opencga.client.exceptions.ClientException; +import org.opencb.opencga.client.rest.OpenCGAClient; +import org.opencb.opencga.core.api.ParamConstants; +import org.opencb.opencga.core.models.file.FileContent; +import org.opencb.opencga.core.models.job.Job; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.io.PrintStream; +import java.util.HashMap; +import java.util.Iterator; +import java.util.Map; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.atomic.AtomicInteger; + +import static org.opencb.opencga.core.models.common.Enums.ExecutionStatus.RUNNING; + +public class JobsLog { + private static final int BATCH_SIZE = ParamConstants.MAXIMUM_LINES_CONTENT; + + private final Logger logger = LoggerFactory.getLogger(this.getClass().toString()); + private final OpenCGAClient openCGAClient; + private final JobsCommandOptions.LogCommandOptions logCommandOptions; + private final Map jobs = new HashMap<>(); + private final Map printedLines = new HashMap<>(); + private final ObjectMap params; + private final int maxLines; + private final PrintStream out; + private final boolean logAllRunningJobs; + private final boolean logMultipleJobs; + private String lastFile = null; + public static final int MAX_ERRORS = 3; + + public JobsLog(OpenCGAClient openCGAClient, JobsCommandOptions.LogCommandOptions logCommandOptions, PrintStream out) { + this.openCGAClient = openCGAClient; + this.logCommandOptions = logCommandOptions; + this.out = out; + this.openCGAClient.setThrowExceptionOnError(true); + + params = new ObjectMap(ParamConstants.STUDY_PARAM, logCommandOptions.study) + .append("type", logCommandOptions.type); + + if (logCommandOptions.follow || logCommandOptions.tailLines == null || logCommandOptions.tailLines < 0) { + maxLines = Integer.MAX_VALUE; + } else { + maxLines = logCommandOptions.tailLines; + } + + logAllRunningJobs = logCommandOptions.job.equalsIgnoreCase(RUNNING); + logMultipleJobs = logAllRunningJobs || logCommandOptions.job.contains(","); + } + + public void run() throws ClientException, InterruptedException { + if (logMultipleJobs) { + openCGAClient.getJobClient() + .search(new ObjectMap(ParamConstants.STUDY_PARAM, logCommandOptions.study) + .append(JobDBAdaptor.QueryParams.INTERNAL_STATUS_ID.key(), RUNNING) + .append(QueryOptions.INCLUDE, "id")) + .allResults() + .forEach(job -> jobs.put(job.getId(), null)); + } else { + for (String job : logCommandOptions.job.split(",")) { + jobs.put(job, null); + } + } + + Stopwatch timer = Stopwatch.createStarted(); + while (!jobs.isEmpty()) { + if (timer.elapsed(TimeUnit.MINUTES) > 5) { + openCGAClient.refresh(); + timer.reset().start(); + } + + Iterator iterator = jobs.keySet().iterator(); + while (iterator.hasNext()) { + String job = iterator.next(); + boolean eof = jobLogLoop(job); + if (eof) { + iterator.remove(); + printedLines.remove(job); + } + } + if (logAllRunningJobs) { + final int waitForNewJobsSecs = 60; // Seconds to wait to find new jobs, if list is empty + int i = 0; + + // Update list of running jobs + do { + secureOp(() -> openCGAClient.getJobClient() + .search(new ObjectMap(ParamConstants.STUDY_PARAM, logCommandOptions.study) + .append(JobDBAdaptor.QueryParams.INTERNAL_STATUS_ID.key(), RUNNING) + .append(QueryOptions.INCLUDE, "id")) + .allResults() + .forEach(job -> jobs.putIfAbsent(job.getId(), null))); + i++; + if (jobs.isEmpty()) { + // Sleep if there are jobs left. + logger.debug("Sleep"); + Thread.sleep(TimeUnit.SECONDS.toMillis(1)); + } + } while (jobs.isEmpty() && i < waitForNewJobsSecs); + } + if (!jobs.isEmpty()) { + // Sleep if there are jobs left. + logger.debug("Sleep"); + Thread.sleep(TimeUnit.SECONDS.toMillis(logCommandOptions.delay)); + } + } + } + + + private boolean jobLogLoop(String jobId) throws ClientException { + boolean eof = true; + AtomicInteger printedLines = this.printedLines.computeIfAbsent(jobId, k -> new AtomicInteger()); + FileContent content = jobs.get(jobId); + while (logCommandOptions.follow || printedLines.get() < maxLines) { + ObjectMap params = new ObjectMap(this.params); + if (content == null) { + // First file content + if (logCommandOptions.tailLines == null || logCommandOptions.tailLines < 0) { + // Undefined tail. Print all. + params.append("lines", BATCH_SIZE); + content = secureOp(() -> openCGAClient.getJobClient().headLog(jobId, params).firstResult()); + } else { + params.append("lines", logCommandOptions.tailLines); + content = secureOp(() -> openCGAClient.getJobClient().tailLog(jobId, params).firstResult()); + } + } else { + params.put("lines", Math.min(maxLines - printedLines.get(), BATCH_SIZE)); + params.put("offset", content.getOffset()); + content = secureOp(() -> openCGAClient.getJobClient().headLog(jobId, params).firstResult()); + } + jobs.put(jobId, content); + printedLines.addAndGet(printContent(content)); + + // Read fewer lines than expected. Check EOF + if (content.getLines() < params.getInt("lines")) { + if (logCommandOptions.follow) { + // Check job status + Job job = secureOp(() -> openCGAClient.getJobClient().info(jobId, new ObjectMap(ParamConstants.STUDY_PARAM, logCommandOptions.study)).firstResult()); + if (job.getInternal().getStatus().getName().equals(RUNNING)) { + // The job is still running. eof=false and break + eof = false; + break; + } else { + // If the job is not running. Trust the content.eof + eof = content.isEof(); + break; + } + } else { + // End of file + if (content.isEof()) { + eof = true; + break; + } + } + } + } + return eof; + } + + interface Op { + R apply() throws ClientException; + } + + interface OpConsumer { + void apply() throws ClientException; + + default Op toOp() { + return () -> { + apply(); + return null; + }; + } + } + + private void secureOp(OpConsumer op) throws ClientException { + secureOp(op.toOp()); + } + + private T secureOp(Op op) throws ClientException { + int errors = 0; + while (true) { + try { + return op.apply(); + } catch (Exception e) { + errors++; + if (errors > MAX_ERRORS) { + logger.error("Got " + errors + " consecutive errors trying to print Jobs Log"); + throw e; + } + } + } + } + + private int printContent(FileContent content) { + if (!content.getContent().isEmpty()) { + if (logMultipleJobs) { + String fileId = content.getFileId().substring(content.getFileId().lastIndexOf("/") + 1); + if (!fileId.equals(lastFile)) { + out.println(); + out.println("==> " + fileId + " <=="); + } + lastFile = fileId; + } + out.print(content.getContent()); + if (!content.getContent().endsWith("\n")) { + out.println(); + } + out.flush(); + } + return content.getLines(); + } + +} diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/utils/JobsTopManager.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/utils/JobsTopManager.java new file mode 100644 index 00000000000..6e93e318a1a --- /dev/null +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/utils/JobsTopManager.java @@ -0,0 +1,231 @@ +/* + * Copyright 2015-2020 OpenCB + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.opencb.opencga.app.cli.main.utils; + +import com.google.common.base.Stopwatch; +import org.apache.commons.lang3.StringUtils; +import org.opencb.commons.datastore.core.Query; +import org.opencb.commons.datastore.core.QueryOptions; +import org.opencb.opencga.app.cli.main.io.Table; +import org.opencb.opencga.app.cli.main.io.TextOutputWriter; +import org.opencb.opencga.catalog.db.api.JobDBAdaptor; +import org.opencb.opencga.client.exceptions.ClientException; +import org.opencb.opencga.client.rest.OpenCGAClient; +import org.opencb.opencga.core.common.GitRepositoryState; +import org.opencb.opencga.core.models.common.Enums; +import org.opencb.opencga.core.models.job.Job; +import org.opencb.opencga.core.models.job.JobInternal; +import org.opencb.opencga.core.models.job.JobTop; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.io.ByteArrayOutputStream; +import java.io.PrintStream; +import java.time.Instant; +import java.util.*; +import java.util.concurrent.TimeUnit; +import java.util.stream.Collectors; + +import static org.opencb.opencga.app.cli.main.io.TextOutputWriter.JobColumns.*; + +public class JobsTopManager { + + public static final int MAX_ERRORS = 4; + private final Logger logger = LoggerFactory.getLogger(this.getClass()); + + private final OpenCGAClient openCGAClient; + private final Query baseQuery; + private final int iterations; + private final int jobsLimit; + private final long delay; + private final boolean plain; + + // FIXME: Use an intermediate buffer to prepare the table, and print in one system call to avoid flashes + private final ByteArrayOutputStream buffer; + private final Table jobTable; + private PrintStream bufferStream; + + public JobsTopManager(OpenCGAClient openCGAClient, Query query, Integer iterations, Integer jobsLimit, long delay, boolean plain) { + this(openCGAClient, query, iterations, jobsLimit, delay, plain, parseColumns(null)); + } + + public JobsTopManager(OpenCGAClient openCGAClient, Query query, Integer iterations, Integer jobsLimit, long delay, boolean plain, String columns) { + this(openCGAClient, query, iterations, jobsLimit, delay, plain, parseColumns(columns)); + } + + public JobsTopManager(OpenCGAClient openCGAClient, Query query, Integer iterations, Integer jobsLimit, long delay, boolean plain, List columns) { + this.openCGAClient = openCGAClient; + this.baseQuery = new Query(query) + .append(QueryOptions.SORT, JobDBAdaptor.QueryParams.CREATION_DATE.key()) + .append(QueryOptions.ORDER, QueryOptions.DESCENDING); + this.buffer = new ByteArrayOutputStream(); + this.iterations = iterations == null || iterations <= 0 ? -1 : iterations; + if (jobsLimit == null || jobsLimit <= 0) { + String lines = System.getenv("LINES"); + if (StringUtils.isNumeric(lines)) { + int HEADER_SIZE = 9; + this.jobsLimit = Integer.parseInt(lines) - HEADER_SIZE; + } else { + this.jobsLimit = 20; + } + } else { + this.jobsLimit = jobsLimit; + } + this.delay = delay < 0 ? 2 : delay; + this.plain = plain; + + buffer.reset(); + bufferStream = new PrintStream(buffer); + + Table.TablePrinter tablePrinter = new Table.JAnsiTablePrinter(bufferStream); + jobTable = new Table<>(tablePrinter); + for (TextOutputWriter.JobColumns column : columns) { + jobTable.addColumn(column.getColumnSchema()); + } + jobTable.setMultiLine(false); + + } + + private static List parseColumns(String columnsStr) { + if (StringUtils.isBlank(columnsStr) || columnsStr.equalsIgnoreCase("default")) { + return Arrays.asList(ID, TOOL_ID, STATUS, STUDY, SUBMISSION, PRIORITY, RUNNING_TIME, STEP, START, END); + } else { + List columns = new LinkedList<>(); + for (String c : columnsStr.split(",")) { + columns.add(TextOutputWriter.JobColumns.valueOf(c.toUpperCase())); + } + return columns; + } + } + + public void run() throws ClientException, InterruptedException { + Stopwatch timer = Stopwatch.createStarted(); + int iteration = 0; + int errors = 0; + openCGAClient.setThrowExceptionOnError(true); + while (iterations != iteration) { + try { + iteration++; + if (timer.elapsed(TimeUnit.MINUTES) > 5) { + openCGAClient.refresh(); + timer.reset().start(); + } + + print(openCGAClient.getJobClient().top(baseQuery).firstResult()); + + Thread.sleep(TimeUnit.SECONDS.toMillis(this.delay)); + // Reset errors counter + errors = 0; + } catch (InterruptedException e) { + // Do not ignore InterruptedException!! + throw e; + } catch (Exception e) { + errors++; + if (errors > MAX_ERRORS) { + logger.error("Got " + errors + " consecutive errors trying to print Jobs Top"); + throw e; + } + } + } + } + + public void print(JobTop top) { + buffer.reset(); + List jobList = processJobs(top.getJobs()); + jobTable.updateTable(jobList); + + jobTable.restoreCursorPosition(); + jobTable.println("OpenCGA jobs TOP"); + jobTable.println(" Version " + GitRepositoryState.get().getBuildVersion()); + jobTable.println(" " + TextOutputWriter.SIMPLE_DATE_FORMAT.format(Date.from(Instant.now()))); + jobTable.println(); + jobTable.print(Enums.ExecutionStatus.RUNNING + ": " + top.getStats().getRunning() + ", "); + jobTable.print(Enums.ExecutionStatus.QUEUED + ": " + top.getStats().getQueued() + ", "); + jobTable.print(Enums.ExecutionStatus.PENDING + ": " + top.getStats().getPending() + ", "); + jobTable.print(Enums.ExecutionStatus.DONE + ": " + top.getStats().getDone() + ", "); + jobTable.print(Enums.ExecutionStatus.ERROR + ": " + top.getStats().getError() + ", "); + jobTable.print(Enums.ExecutionStatus.ABORTED + ": " + top.getStats().getAborted()); + jobTable.println(); + jobTable.println(); + jobTable.printTable(); + + bufferStream.flush(); + System.out.print(buffer); + } + + private List processJobs(List jobs) { + List jobList = new LinkedList<>(); + jobs.sort(Comparator.comparing(j -> j.getInternal().getStatus().getName().equals(Enums.ExecutionStatus.RUNNING) ? 0 : 1)); + jobs = trimJobs(jobs); + + int jobDependsMax = 5; + for (Job job : jobs) { + jobList.add(job); + if (job.getDependsOn() != null && !job.getDependsOn().isEmpty()) { + List dependsOn = job.getDependsOn(); + dependsOn.removeIf(Objects::isNull); + if (dependsOn.size() > jobDependsMax) { + int size = dependsOn.size(); + TreeMap byType = dependsOn + .stream() + .collect(Collectors.groupingBy( + j -> j.getInternal().getStatus().getName(), + TreeMap::new, + Collectors.summingInt(j -> 1))); + int maxStatus = byType.keySet().stream().mapToInt(String::length).max().orElse(0); + dependsOn = new ArrayList<>(byType.size()); + for (Map.Entry entry : byType.entrySet()) { + dependsOn.add(new Job() + .setId(StringUtils.rightPad(entry.getKey(), maxStatus) + " : " + entry.getValue() + "/" + size) + .setInternal(new JobInternal(new Enums.ExecutionStatus(entry.getKey())))); + } + } + if (!plain) { + for (int i = 0; i < dependsOn.size(); i++) { + Job auxJob = dependsOn.get(i); + if (i + 1 < dependsOn.size()) { + auxJob.setId("├── " + auxJob.getId()); + } else { + auxJob.setId("└── " + auxJob.getId()); + } + jobList.add(auxJob); + } + } + } + } + + jobList = trimJobs(jobList); + + return jobList; + } + + private List trimJobs(List jobs) { + if (jobs.size() <= jobsLimit) { + return jobs; + } + List jobList = jobs.subList(0, jobsLimit); + for (int i = jobList.size() - 1; i >= 0; i--) { + if (jobList.get(i).getId().startsWith("└") || (!jobList.get(i).getId().startsWith("├") && (jobList.get(i).getDependsOn() == null || jobList.get(i).getDependsOn().isEmpty()))) { + break; + } + jobList.remove(i); + } + return jobList; + + } + +} \ No newline at end of file diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/utils/LoginUtils.java b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/utils/LoginUtils.java index 3d03dd5d966..26f765ea409 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/utils/LoginUtils.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/cli/main/utils/LoginUtils.java @@ -12,30 +12,87 @@ public class LoginUtils { - public static String[] forceLogin(String[] args) { + public static String[] parseLoginCommand(String[] args) { + CommandLineUtils.debug("LOGIN COMMAND: " + CommandLineUtils.argsToString(args)); + + // 1. Make sure first argument is 'login' + if (!args[0].equals("login")) { + printUsage(args); + return null; + } + + // 2. There are only four valid lengths and formats accepted: + // 1. login + // 2. login imedina + // 3. login --host host + // 4. login imedina --host host + switch(args.length) { + case 2: + if (args[1].startsWith("-")) { + printUsage(args); + return null; + } + break; + case 3: + if (!args[1].equals("--host")) { + printUsage(args); + return null; + } + break; + case 4: + if (args[1].startsWith("-") || !args[2].equals("--host")) { + printUsage(args); + return null; + } + break; + default: + if (args.length > 4) { + printUsage(args); + return null; + } + break; + } + + // Add 'users' as first argument + args = ArrayUtils.addFirst(args, "users"); + + if (args.length == 2 || args.length == 4 ) { + // We need to ask for the user in 2 of the 4 possible cases here: + // 1. users login + // 2. users login --host host + return loginUser(args); + } else { + // Two possible cases here: + // 1. users login imedina + // 2. users login imedina --host host + String user = args[2]; + args = ArrayUtils.remove(args, 2); + return loginUser(args, user); + } + } + + private static String[] loginUser(String[] args) { String user = System.console().readLine(format("\nEnter your user: ", PrintUtils.Color.GREEN)).trim(); CommandLineUtils.debug("Login user " + user); return loginUser(args, user); } - public static String[] loginUser(String[] args, String user) { + private static String[] loginUser(String[] args, String user) { String password = ""; if (System.console() != null) { char[] passwordArray = System.console().readPassword(format("\nEnter your password: ", PrintUtils.Color.GREEN)); password = new String(passwordArray).trim(); } else { - BufferedReader reader = new BufferedReader(new InputStreamReader( - System.in)); - try { + CommandLineUtils.debug("System.console() is null"); + try (BufferedReader reader = new BufferedReader(new InputStreamReader(System.in))) { password = reader.readLine(); } catch (IOException e) { e.printStackTrace(); } - CommandLineUtils.debug("Console null "); } + if (CommandLineUtils.isValidUser(user)) { - args = ArrayUtils.addAll(args, "-u", user); - args = ArrayUtils.addAll(args, "--password", password); + args = ArrayUtils.addAll(args, "-u", user, "--password", password); CommandLineUtils.debug(ArrayUtils.toString(args)); } else { println(PrintUtils.format("Invalid user name: ", Color.RED) + PrintUtils.format(user, Color.DEFAULT)); @@ -45,45 +102,7 @@ public static String[] loginUser(String[] args, String user) { return args; } - - public static String[] parseLoginCommand(String[] args) { - //adds in position 0 command "users" - - CommandLineUtils.debug("LOGIN COMMAND: " + CommandLineUtils.argsToString(args)); - if ("login".equals(args[0]) && (ArrayUtils.contains(args, "-u") || ArrayUtils.contains(args, "-u"))) { - printUsage(args); - return null; - } - if ("login".equals(args[0]) && (args[1].startsWith("-")) && (!"--host".equals(args[1]))) { - printUsage(args); - return null; - } - - args = ArrayUtils.addAll(new String[]{"users"}, args); - - //case opencga.sh login OR [opencga][demo@project:study]login - if (args.length == 2 && "login".equals(args[1])) { - return forceLogin(args); - } - - //CASES - //case opencga.sh login --host ...... OR [opencga][demo@project:study]login --host ...... - //case opencga.sh login user1 [.....] OR [opencga][demo@project:study]login user1 [.....] - if (args.length > 2 && "login".equals(args[1])) { - if (args[2].startsWith("-")) { - return forceLogin(args); - } else { - String user = args[2]; - args = ArrayUtils.remove(args, 2); - return loginUser(args, user); - } - } - - return args; - } - private static void printUsage(String[] args) { - PrintUtils.println(""); if (!isHelp(args)) { PrintUtils.println(PrintUtils.getKeyValueAsFormattedString("Invalid Command: ", CommandLineUtils.argsToString(args))); @@ -103,5 +122,5 @@ private static boolean isHelp(String[] args) { || ArrayUtils.contains(args, "help"); } - } + diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/migrations/v2_1_0/storage/DefaultSampleIndexConfiguration.java b/opencga-app/src/main/java/org/opencb/opencga/app/migrations/v2_1_0/storage/DefaultSampleIndexConfiguration.java index 7295eed79b9..301b510266b 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/migrations/v2_1_0/storage/DefaultSampleIndexConfiguration.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/migrations/v2_1_0/storage/DefaultSampleIndexConfiguration.java @@ -86,7 +86,8 @@ protected void run() throws Exception { if (CollectionUtils.isEmpty(studyMetadata.getSampleIndexConfigurations())) { studyMetadata.setSampleIndexConfigurations(new ArrayList<>()); studyMetadata.getSampleIndexConfigurations().add( - new StudyMetadata.SampleIndexConfigurationVersioned(sampleIndexConfiguration, 1, Date.from(Instant.now()))); + new StudyMetadata.SampleIndexConfigurationVersioned(sampleIndexConfiguration, 1, Date.from(Instant.now()), + StudyMetadata.SampleIndexConfigurationVersioned.Status.ACTIVE)); } else { int size = studyMetadata.getSampleIndexConfigurations().size(); studyMetadata.getSampleIndexConfigurations().get(size - 1).setConfiguration(sampleIndexConfiguration); diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/migrations/v2_2_0/catalog/FixFamilyReferences.java b/opencga-app/src/main/java/org/opencb/opencga/app/migrations/v2_2_0/catalog/FixFamilyReferences.java new file mode 100644 index 00000000000..5ffe8471598 --- /dev/null +++ b/opencga-app/src/main/java/org/opencb/opencga/app/migrations/v2_2_0/catalog/FixFamilyReferences.java @@ -0,0 +1,69 @@ +package org.opencb.opencga.app.migrations.v2_2_0.catalog; + +import com.mongodb.client.model.Projections; +import com.mongodb.client.model.UpdateOneModel; +import org.bson.Document; +import org.opencb.opencga.catalog.db.mongodb.MongoDBAdaptorFactory; +import org.opencb.opencga.catalog.migration.Migration; +import org.opencb.opencga.catalog.migration.MigrationTool; + +import java.util.*; + +import static com.mongodb.client.model.Filters.eq; + +@Migration(id = "fix_family_references_in_individual", + description = "Fix Family references, #TASK-489", version = "2.2.0", + language = Migration.MigrationLanguage.JAVA, + domain = Migration.MigrationDomain.CATALOG, + date = 20220324) +public class FixFamilyReferences extends MigrationTool { + + @Override + protected void run() throws Exception { + // Get all the family ids: studyUid - {familyIds} + Map> familyIds = new HashMap<>(); + queryMongo(MongoDBAdaptorFactory.FAMILY_COLLECTION, new Document(), + Projections.include(Arrays.asList("id", "studyUid")), (doc) -> { + String studyId = doc.get("studyUid", Number.class).toString(); + if (!familyIds.containsKey(studyId)) { + familyIds.put(studyId, new HashSet<>()); + } + familyIds.get(studyId).add(doc.getString("id")); + }); + + migrateCollection(MongoDBAdaptorFactory.INDIVIDUAL_COLLECTION, + new Document("familyIds", new Document("$ne", Collections.emptyList())), + Projections.include("familyIds", "studyUid"), + (individual, bulk) -> { + String studyId = individual.get("studyUid", Number.class).toString(); + + if (!familyIds.containsKey(studyId)) { + return; + } + + boolean changed = false; + List newIds = new ArrayList<>(); + List iFamilyIds = individual.getList("familyIds", String.class); + + // Store all existing familyIds in newIds so non-existing ones are removed + for (String iFamilyId : iFamilyIds) { + if (familyIds.get(studyId).contains(iFamilyId)) { + newIds.add(iFamilyId); + } else { + // Found a non-existing familyId, so we'll need to apply the update ! + changed = true; + } + } + + if (changed) { + bulk.add(new UpdateOneModel<>( + eq("_id", individual.get("_id")), + new Document("$set", new Document("familyIds", newIds)) + ) + ); + } + } + ); + } + +} diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/migrations/v2_2_0/catalog/ImproveClinicalAnalysisQualityControl.java b/opencga-app/src/main/java/org/opencb/opencga/app/migrations/v2_2_0/catalog/ImproveClinicalAnalysisQualityControl.java index f6350a84e8d..4014fbf1f8e 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/migrations/v2_2_0/catalog/ImproveClinicalAnalysisQualityControl.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/migrations/v2_2_0/catalog/ImproveClinicalAnalysisQualityControl.java @@ -42,7 +42,14 @@ private void improvementsClinicalAnalysis() { String summary = qc.get("summary", String.class); String text = qc.get("comment", String.class); String user = qc.get("user", String.class); - Date date = qc.containsKey("date") ? new Date(qc.get("date", Number.class).longValue()) : null; + Date date = null; + try { + if (qc.containsKey("date")) { + date = new Date(qc.get("date", Number.class).longValue()); + } + } catch (Exception e) { + // empty catch block + } // if EXCELLENT, GOOD, NORMAL, BAD, UNKNOWN String newSummary = null; diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/migrations/v2_2_0/catalog/issues_1853_1855/IndividualOntologyTermMigration.java b/opencga-app/src/main/java/org/opencb/opencga/app/migrations/v2_2_0/catalog/issues_1853_1855/IndividualOntologyTermMigration.java index 60b31ef481e..b505c4b7d9c 100644 --- a/opencga-app/src/main/java/org/opencb/opencga/app/migrations/v2_2_0/catalog/issues_1853_1855/IndividualOntologyTermMigration.java +++ b/opencga-app/src/main/java/org/opencb/opencga/app/migrations/v2_2_0/catalog/issues_1853_1855/IndividualOntologyTermMigration.java @@ -47,10 +47,10 @@ protected void run() throws Exception { ); MongoCollection collection = getMongoCollection(MongoDBAdaptorFactory.INDIVIDUAL_COLLECTION); - collection.dropIndex(new Document().append("sex", 1).append("studyUid", 1)); - collection.dropIndex(new Document().append("ethnicity", 1).append("studyUid", 1)); - collection.createIndex(new Document().append("sex.id", 1).append("studyUid", 1), new IndexOptions().background(true)); - collection.createIndex(new Document().append("ethnicity.id", 1).append("studyUid", 1), new IndexOptions().background(true)); + dropIndex(collection, new Document().append("sex", 1).append("studyUid", 1)); + dropIndex(collection, new Document().append("ethnicity", 1).append("studyUid", 1)); + createIndex(collection, new Document().append("sex.id", 1).append("studyUid", 1), new IndexOptions().background(true)); + createIndex(collection, new Document().append("ethnicity.id", 1).append("studyUid", 1), new IndexOptions().background(true)); } } diff --git a/opencga-app/src/main/java/org/opencb/opencga/app/migrations/v2_2_0/storage/UpdateSampleIndexStatus.java b/opencga-app/src/main/java/org/opencb/opencga/app/migrations/v2_2_0/storage/UpdateSampleIndexStatus.java new file mode 100644 index 00000000000..1bc4b2406c9 --- /dev/null +++ b/opencga-app/src/main/java/org/opencb/opencga/app/migrations/v2_2_0/storage/UpdateSampleIndexStatus.java @@ -0,0 +1,133 @@ +package org.opencb.opencga.app.migrations.v2_2_0.storage; + +import org.opencb.commons.ProgressLogger; +import org.opencb.opencga.app.migrations.StorageMigrationTool; +import org.opencb.opencga.catalog.migration.Migration; +import org.opencb.opencga.storage.core.metadata.VariantStorageMetadataManager; +import org.opencb.opencga.storage.core.metadata.models.SampleMetadata; +import org.opencb.opencga.storage.core.metadata.models.StudyMetadata; +import org.opencb.opencga.storage.core.metadata.models.TaskMetadata; +import org.opencb.opencga.storage.core.variant.VariantStorageEngine; + +import java.util.Iterator; + +@Migration(id = "update_sample_index_status_1901", + description = " Improve sample-index multi-schema management. #1901", version = "2.2.0", + language = Migration.MigrationLanguage.JAVA, + domain = Migration.MigrationDomain.STORAGE, + date = 20220302) +public class UpdateSampleIndexStatus extends StorageMigrationTool { + + // Deprecated to avoid confusion with actual "SAMPLE_INDEX_STATUS" + protected static final String SAMPLE_INDEX_ANNOTATION_STATUS_OLD = "sampleIndex"; + + protected static final String SAMPLE_INDEX_STATUS = "sampleIndexGenotypes"; + protected static final String SAMPLE_INDEX_VERSION = "sampleIndexGenotypesVersion"; + protected static final String SAMPLE_INDEX_ANNOTATION_STATUS = "sampleIndexAnnotation"; + protected static final String SAMPLE_INDEX_ANNOTATION_VERSION = "sampleIndexAnnotationVersion"; + + private static final String FAMILY_INDEX_STATUS = "family_index"; + + private final String markKey = "migration_" + getAnnotation().id(); + private final int markValue = getAnnotation().patch(); + + @Override + protected void run() throws Exception { + + for (String project : getVariantStorageProjects()) { + VariantStorageEngine engine = getVariantStorageEngineByProject(project); + if (!engine.getStorageEngineId().equals("hadoop")) { + logger.info("Skip project '{}'. VariantStorageEngine not in hadoop."); + continue; + } + VariantStorageMetadataManager metadataManager = engine.getMetadataManager(); + for (String study : metadataManager.getStudyNames()) { + int studyId = metadataManager.getStudyId(study); + Iterator it = metadataManager.sampleMetadataIterator(studyId); + ProgressLogger progressLogger = new ProgressLogger("Update samples from study " + study); + progressLogger.setBatchSize(1000); + int read = 0; + int modified = 0; + while (it.hasNext()) { + SampleMetadata sampleMetadata = it.next(); + if (needsMigration(sampleMetadata)) { + metadataManager.updateSampleMetadata(studyId, sampleMetadata.getId(), + this::updateSampleMetadata); + modified++; + } + read++; + progressLogger.increment(1); + } + logger.info("Finish update samples from study {}. {}/{} samples updated", study, modified, read); + } + } + } + + protected void updateSampleMetadata(SampleMetadata sampleMetadata) { + if (needsMigration(sampleMetadata)) { + addVersionToSampleIndexStatus(sampleMetadata); + renameOldSampleIndexAnnotationStatus(sampleMetadata); + addVersionToSampleIndexAnnotationStatus(sampleMetadata); + addVersionToFamilyIndexStatus(sampleMetadata); +// removeOldSampleIndexStatus(sampleMetadata); + addUpdatedMark(sampleMetadata); + } + } + + protected boolean needsMigration(SampleMetadata sampleMetadata) { + return sampleMetadata.isIndexed() && sampleMetadata.getAttributes().getInt(markKey, -1) != markValue; + } + + protected void addVersionToSampleIndexStatus(SampleMetadata sampleMetadata) { + // This is a new status. In case of missing value (null), assume it's READY + TaskMetadata.Status status = sampleMetadata.getStatus(SAMPLE_INDEX_STATUS, TaskMetadata.Status.READY); + if (status != null) { + int version = sampleMetadata.getAttributes().getInt(SAMPLE_INDEX_VERSION, -1); + if (version == -1) { + version = StudyMetadata.DEFAULT_SAMPLE_INDEX_VERSION; + } + sampleMetadata.setSampleIndexStatus(status, version); + } + } + + protected void renameOldSampleIndexAnnotationStatus(SampleMetadata sampleMetadata) { + TaskMetadata.Status annotationStatus = sampleMetadata.getStatus(SAMPLE_INDEX_ANNOTATION_STATUS, null); + if (annotationStatus == null) { + TaskMetadata.Status annotationOldStatus = sampleMetadata.getStatus(SAMPLE_INDEX_ANNOTATION_STATUS_OLD, null); + if (annotationOldStatus != null) { + // Use old as new status + sampleMetadata.setStatus(SAMPLE_INDEX_ANNOTATION_STATUS, annotationOldStatus); + } + } + } + + protected void addVersionToSampleIndexAnnotationStatus(SampleMetadata sampleMetadata) { + TaskMetadata.Status annotationStatus = sampleMetadata.getStatus(SAMPLE_INDEX_ANNOTATION_STATUS, null); + if (annotationStatus != null) { + int version = sampleMetadata.getAttributes().getInt(SAMPLE_INDEX_ANNOTATION_VERSION, -1); + if (version == -1) { + version = StudyMetadata.DEFAULT_SAMPLE_INDEX_VERSION; + } + sampleMetadata.setSampleIndexAnnotationStatus(annotationStatus, version); + } + } + + private void addVersionToFamilyIndexStatus(SampleMetadata sampleMetadata) { + if (sampleMetadata.getStatus(FAMILY_INDEX_STATUS) == TaskMetadata.Status.READY) { + sampleMetadata.setFamilyIndexStatus(TaskMetadata.Status.READY, StudyMetadata.DEFAULT_SAMPLE_INDEX_VERSION); + } + } + + private void removeOldSampleIndexStatus(SampleMetadata sampleMetadata) { + sampleMetadata.getStatus().remove(SAMPLE_INDEX_STATUS); + sampleMetadata.getStatus().remove(SAMPLE_INDEX_ANNOTATION_STATUS_OLD); + sampleMetadata.getStatus().remove(SAMPLE_INDEX_ANNOTATION_STATUS); + sampleMetadata.getStatus().remove(FAMILY_INDEX_STATUS); + } + + + private void addUpdatedMark(SampleMetadata sampleMetadata) { + sampleMetadata.getAttributes().put(markKey, markValue); + } + +} diff --git a/opencga-app/src/test/java/org/opencb/opencga/app/cli/JSONTest.java b/opencga-app/src/test/java/org/opencb/opencga/app/cli/JSONTest.java new file mode 100644 index 00000000000..7db0c08e0f6 --- /dev/null +++ b/opencga-app/src/test/java/org/opencb/opencga/app/cli/JSONTest.java @@ -0,0 +1,17 @@ +package org.opencb.opencga.app.cli; + +import org.junit.Test; +import org.opencb.commons.utils.DataModelsUtils; +import org.opencb.opencga.core.models.sample.SampleUpdateParams; + +public class JSONTest { + + + @Test + public void testJSONBean() throws Exception { + SampleUpdateParams sup = new SampleUpdateParams(); + String json = DataModelsUtils.dataModelToJsonString(sup.getClass()); + System.out.println(json); + } + +} diff --git a/opencga-app/src/test/java/org/opencb/opencga/app/migrations/v2_2_0/storage/UpdateSampleIndexStatusTest.java b/opencga-app/src/test/java/org/opencb/opencga/app/migrations/v2_2_0/storage/UpdateSampleIndexStatusTest.java new file mode 100644 index 00000000000..05214755fac --- /dev/null +++ b/opencga-app/src/test/java/org/opencb/opencga/app/migrations/v2_2_0/storage/UpdateSampleIndexStatusTest.java @@ -0,0 +1,99 @@ +package org.opencb.opencga.app.migrations.v2_2_0.storage; + +import org.junit.Before; +import org.junit.Test; +import org.opencb.opencga.storage.core.metadata.models.SampleMetadata; +import org.opencb.opencga.storage.core.metadata.models.TaskMetadata; + +import static org.junit.Assert.*; + +public class UpdateSampleIndexStatusTest { + + private UpdateSampleIndexStatus migrationTool; + private SampleMetadata sm; + + @Before + public void setUp() throws Exception { + migrationTool = new UpdateSampleIndexStatus(); + } + + @Test + public void testUpdateUnindexed() { + sm = new SampleMetadata(1, 1, "1"); + assertFalse(migrationTool.needsMigration(sm)); + migrationTool.updateSampleMetadata(sm); + + check(null, null); + } + + @Test + public void testUpdateIndexedWithoutSampleIndex() { + sm = new SampleMetadata(1, 1, "1") + .setIndexStatus(TaskMetadata.Status.READY) + .setStatus(UpdateSampleIndexStatus.SAMPLE_INDEX_STATUS, TaskMetadata.Status.NONE); + assertTrue(migrationTool.needsMigration(sm)); + migrationTool.updateSampleMetadata(sm); + assertFalse(migrationTool.needsMigration(sm)); + + check(null, null); + } + + @Test + public void testUpdateUnversioned() { + sm = new SampleMetadata(1, 1, "1") + .setIndexStatus(TaskMetadata.Status.READY) + .setStatus(UpdateSampleIndexStatus.SAMPLE_INDEX_ANNOTATION_STATUS_OLD, TaskMetadata.Status.READY); + assertTrue(migrationTool.needsMigration(sm)); + migrationTool.updateSampleMetadata(sm); + assertFalse(migrationTool.needsMigration(sm)); + + check(1, 1); + } + + @Test + public void testUpdateVersionedPartial() { + sm = new SampleMetadata(1, 1, "1") + .setIndexStatus(TaskMetadata.Status.READY) + .setStatus(UpdateSampleIndexStatus.SAMPLE_INDEX_ANNOTATION_STATUS_OLD, TaskMetadata.Status.READY) + .putAttribute(UpdateSampleIndexStatus.SAMPLE_INDEX_VERSION, 2); + assertTrue(migrationTool.needsMigration(sm)); + migrationTool.updateSampleMetadata(sm); + assertFalse(migrationTool.needsMigration(sm)); + + check(2, 1); + } + + @Test + public void testUpdateVersioned() { + sm = new SampleMetadata(1, 1, "1") + .setIndexStatus(TaskMetadata.Status.READY) + .setStatus(UpdateSampleIndexStatus.SAMPLE_INDEX_ANNOTATION_STATUS_OLD, TaskMetadata.Status.READY) + .putAttribute(UpdateSampleIndexStatus.SAMPLE_INDEX_VERSION, 3) + .putAttribute(UpdateSampleIndexStatus.SAMPLE_INDEX_ANNOTATION_VERSION, 2); + assertTrue(migrationTool.needsMigration(sm)); + migrationTool.updateSampleMetadata(sm); + assertFalse(migrationTool.needsMigration(sm)); + + check(3, 2); + + } + + private void check(Integer sampleIndexVersion, Integer sampleIndexAnnotationVersion) { + System.out.println("sm = " + sm.toString()); + assertEquals(sampleIndexVersion, sm.getSampleIndexVersion()); + assertEquals(sampleIndexAnnotationVersion, sm.getSampleIndexAnnotationVersion()); + + if (sampleIndexVersion != null) { + for (int i = 0; i < sampleIndexVersion - 1; i++) { + assertEquals(TaskMetadata.Status.NONE, sm.getSampleIndexStatus(i)); + } + assertEquals(TaskMetadata.Status.READY, sm.getSampleIndexStatus(sampleIndexVersion)); + } + if (sampleIndexAnnotationVersion != null) { + for (int i = 0; i < sampleIndexAnnotationVersion - 1; i++) { + assertEquals(TaskMetadata.Status.NONE, sm.getSampleIndexAnnotationStatus(i)); + } + assertEquals(TaskMetadata.Status.READY, sm.getSampleIndexAnnotationStatus(sampleIndexAnnotationVersion)); + } + } +} \ No newline at end of file diff --git a/opencga-catalog/pom.xml b/opencga-catalog/pom.xml index 1f4263aa272..b34d74e2a71 100644 --- a/opencga-catalog/pom.xml +++ b/opencga-catalog/pom.xml @@ -22,7 +22,7 @@ org.opencb.opencga opencga - 2.2.0-rc2 + 2.2.0 ../pom.xml diff --git a/opencga-catalog/src/main/java/org/opencb/opencga/catalog/db/mongodb/FamilyMongoDBAdaptor.java b/opencga-catalog/src/main/java/org/opencb/opencga/catalog/db/mongodb/FamilyMongoDBAdaptor.java index 408be00ad16..7d9f4425204 100644 --- a/opencga-catalog/src/main/java/org/opencb/opencga/catalog/db/mongodb/FamilyMongoDBAdaptor.java +++ b/opencga-catalog/src/main/java/org/opencb/opencga/catalog/db/mongodb/FamilyMongoDBAdaptor.java @@ -356,7 +356,29 @@ OpenCGAResult privateUpdate(ClientSession clientSession, Family family, Document familyUpdate = parseAndValidateUpdateParams(clientSession, parameters, tmpQuery).toFinalUpdateDocument(); if (CollectionUtils.isNotEmpty(parameters.getAsList(QueryParams.MEMBERS.key()))) { - updateFamilyReferenceInIndividuals(clientSession, family, parameters.getAsList(QueryParams.MEMBERS.key(), Map.class)); + List newIndividuals = parameters.getAsList(QueryParams.MEMBERS.key(), Map.class); + Set newIndividualIds = newIndividuals.stream().map(i -> (String) i.get(IndividualDBAdaptor.QueryParams.ID.key())) + .collect(Collectors.toSet()); + + Set currentIndividualIds = family.getMembers().stream().map(Individual::getId).collect(Collectors.toSet()); + + // Obtain new members to be added to the family + List missingIndividualIds = new ArrayList<>(); + for (String newIndividualId : newIndividualIds) { + if (!currentIndividualIds.contains(newIndividualId)) { + missingIndividualIds.add(newIndividualId); + } + } + + // Obtain members to remove from family + List oldIndividualIds = new ArrayList<>(); + for (String currentIndividualId : currentIndividualIds) { + if (!newIndividualIds.contains(currentIndividualId)) { + oldIndividualIds.add(currentIndividualId); + } + } + + updateFamilyReferenceInIndividuals(clientSession, family, missingIndividualIds, oldIndividualIds); } if (familyUpdate.isEmpty() && result.getNumUpdated() == 0) { @@ -391,34 +413,16 @@ OpenCGAResult privateUpdate(ClientSession clientSession, Family family, return endWrite(tmpStartTime, 1, 1, events); } - private void updateFamilyReferenceInIndividuals(ClientSession clientSession, Family family, List individuals) + private void updateFamilyReferenceInIndividuals(ClientSession clientSession, Family family, List newIndividuals, + List removeIndividuals) throws CatalogParameterException, CatalogDBException, CatalogAuthorizationException { - List newIndividualIds = new ArrayList<>(); - List individualIdsToRemove = new ArrayList<>(); - - Set finalMemberSet = individuals.stream().map(i -> String.valueOf(i.get(IndividualDBAdaptor.QueryParams.ID.key()))) - .collect(Collectors.toSet()); - Set currentIndividualIds = family.getMembers().stream().map(Individual::getId).collect(Collectors.toSet()); - - for (Map individual : individuals) { - String individualId = String.valueOf(individual.get(IndividualDBAdaptor.QueryParams.ID.key())); - if (!currentIndividualIds.contains(individualId)) { - newIndividualIds.add(individualId); - } - } - if (!newIndividualIds.isEmpty()) { - dbAdaptorFactory.getCatalogIndividualDBAdaptor().updateFamilyReferences(clientSession, family.getStudyUid(), newIndividualIds, + if (CollectionUtils.isNotEmpty(newIndividuals)) { + dbAdaptorFactory.getCatalogIndividualDBAdaptor().updateFamilyReferences(clientSession, family.getStudyUid(), newIndividuals, family.getId(), ParamUtils.BasicUpdateAction.ADD); } - - for (Individual member : family.getMembers()) { - if (!finalMemberSet.contains(member.getId())) { - individualIdsToRemove.add(member.getId()); - } - } - if (!individualIdsToRemove.isEmpty()) { + if (CollectionUtils.isNotEmpty(removeIndividuals)) { dbAdaptorFactory.getCatalogIndividualDBAdaptor().updateFamilyReferences(clientSession, family.getStudyUid(), - individualIdsToRemove, family.getId(), ParamUtils.BasicUpdateAction.REMOVE); + removeIndividuals, family.getId(), ParamUtils.BasicUpdateAction.REMOVE); } } @@ -752,10 +756,23 @@ OpenCGAResult privateDelete(ClientSession clientSession, Document family logger.debug("Family uid '{}' version '{}' deleted from main SAMPLE collection", familyUid, sampleVersion); } + // Remove family references + removeFamilyReferences(clientSession, familyDocument); + logger.debug("Family {}({}) deleted", familyId, familyUid); return endWrite(tmpStartTime, 1, 0, 0, 1, Collections.emptyList()); } + private void removeFamilyReferences(ClientSession clientSession, Document familyDocument) + throws CatalogParameterException, CatalogDBException, CatalogAuthorizationException { + // Remove family reference from individuals + Family family = familyConverter.convertToDataModelType(familyDocument); + if (CollectionUtils.isNotEmpty(family.getMembers())) { + List members = family.getMembers().stream().map(Individual::getId).collect(Collectors.toList()); + updateFamilyReferenceInIndividuals(clientSession, family, Collections.emptyList(), members); + } + } + @Override public OpenCGAResult restore(long id, QueryOptions queryOptions) throws CatalogDBException { throw new NotImplementedException("Not yet implemented"); diff --git a/opencga-catalog/src/main/java/org/opencb/opencga/catalog/managers/FamilyManager.java b/opencga-catalog/src/main/java/org/opencb/opencga/catalog/managers/FamilyManager.java index a2e1fb66ef1..adf8196fa03 100644 --- a/opencga-catalog/src/main/java/org/opencb/opencga/catalog/managers/FamilyManager.java +++ b/opencga-catalog/src/main/java/org/opencb/opencga/catalog/managers/FamilyManager.java @@ -489,7 +489,8 @@ public OpenCGAResult delete(String studyStr, List familyIds, ObjectMap p FamilyAclEntry.FamilyPermissions.DELETE); } - // TODO: Check if the family is used in a clinical analysis. At this point, it can be deleted no matter what. + // Check family can be deleted + checkCanBeDeleted(study, family); // Delete the family result.append(familyDBAdaptor.delete(family)); @@ -511,6 +512,17 @@ public OpenCGAResult delete(String studyStr, List familyIds, ObjectMap p return endResult(result, ignoreException); } + private void checkCanBeDeleted(Study study, Family family) throws CatalogException { + Query query = new Query() + .append(ClinicalAnalysisDBAdaptor.QueryParams.STUDY_UID.key(), study.getUid()) + .append(ClinicalAnalysisDBAdaptor.QueryParams.FAMILY_UID.key(), family.getUid()); + OpenCGAResult result = clinicalDBAdaptor.get(query, ClinicalAnalysisManager.INCLUDE_CLINICAL_IDS); + if (result.getNumResults() > 0) { + String clinicalIds = result.getResults().stream().map(ClinicalAnalysis::getId).collect(Collectors.joining(", ")); + throw new CatalogException("Family {" + family.getId() + "} in use in Clinical Analyses: {" + clinicalIds + "}"); + } + } + @Override public OpenCGAResult delete(String studyStr, Query query, QueryOptions options, String token) throws CatalogException { return delete(studyStr, query, options, false, token); diff --git a/opencga-catalog/src/main/java/org/opencb/opencga/catalog/managers/JobManager.java b/opencga-catalog/src/main/java/org/opencb/opencga/catalog/managers/JobManager.java index ce58e78f787..1ef906d5680 100644 --- a/opencga-catalog/src/main/java/org/opencb/opencga/catalog/managers/JobManager.java +++ b/opencga-catalog/src/main/java/org/opencb/opencga/catalog/managers/JobManager.java @@ -427,7 +427,13 @@ public OpenCGAResult retry(String studyStr, JobRetryParams jobRetry, Enums. if (jobRetry.getParams() != null) { params.putAll(jobRetry.getParams()); } - return submit(studyStr, job.getTool().getId(), priority, params, jobId, jobDescription, jobDependsOn, jobTags, token); + HashMap attributes = new HashMap<>(); + attributes.put("retry_from", jobRetry.getJob()); + if (StringUtils.isEmpty(jobDescription)) { + jobDescription = "Retry from job '" + jobRetry.getJob() + "'"; + } + return submit(studyStr, job.getTool().getId(), priority, params, jobId, jobDescription, jobDependsOn, jobTags, + attributes, token); } else { throw new CatalogException("Unable to retry job with status " + job.getInternal().getStatus().getId()); } @@ -459,6 +465,13 @@ public OpenCGAResult submitProject(String projectStr, String toolId, Enums. public OpenCGAResult submit(String studyStr, String toolId, Enums.Priority priority, Map params, String jobId, String jobDescription, List jobDependsOn, List jobTags, String token) throws CatalogException { + return submit(studyStr, toolId, priority, params, jobId, jobDescription, jobDependsOn, jobTags, null, token); + } + + public OpenCGAResult submit(String studyStr, String toolId, Enums.Priority priority, Map params, String jobId, + String jobDescription, List jobDependsOn, List jobTags, + Map attributes, String token) + throws CatalogException { String userId = userManager.getUserId(token); Study study = catalogManager.getStudyManager().resolveId(studyStr, userId); @@ -485,7 +498,7 @@ public OpenCGAResult submit(String studyStr, String toolId, Enums.Priority job.setDependsOn(jobDependsOn != null ? jobDependsOn.stream().map(j -> new Job().setId(j)).collect(Collectors.toList()) : Collections.emptyList()); - + job.setAttributes(attributes); try { autoCompleteNewJob(study, job, token); diff --git a/opencga-catalog/src/main/java/org/opencb/opencga/catalog/managers/UserManager.java b/opencga-catalog/src/main/java/org/opencb/opencga/catalog/managers/UserManager.java index 51e604dba09..4c4a7e15149 100644 --- a/opencga-catalog/src/main/java/org/opencb/opencga/catalog/managers/UserManager.java +++ b/opencga-catalog/src/main/java/org/opencb/opencga/catalog/managers/UserManager.java @@ -36,11 +36,8 @@ import org.opencb.opencga.catalog.io.CatalogIOManager; import org.opencb.opencga.catalog.utils.ParamUtils; import org.opencb.opencga.catalog.utils.UuidUtils; - -import org.opencb.opencga.core.common.PasswordUtils; - import org.opencb.opencga.core.api.ParamConstants; - +import org.opencb.opencga.core.common.PasswordUtils; import org.opencb.opencga.core.common.TimeUtils; import org.opencb.opencga.core.config.AuthenticationOrigin; import org.opencb.opencga.core.config.Configuration; @@ -205,7 +202,7 @@ public OpenCGAResult create(User user, String password, @Nullable String t try { if (!PasswordUtils.isStrongPassword(password)) { - throw new CatalogException("Invalid password. Check password strength for user " + user); + throw new CatalogException("Invalid password. Check password strength for user " + user.getId()); } if (user.getProjects() != null && !user.getProjects().isEmpty()) { throw new CatalogException("Creating user and projects in a single transaction is forbidden"); diff --git a/opencga-catalog/src/main/java/org/opencb/opencga/catalog/migration/MigrationTool.java b/opencga-catalog/src/main/java/org/opencb/opencga/catalog/migration/MigrationTool.java index bfe9065b804..62da70c5cad 100644 --- a/opencga-catalog/src/main/java/org/opencb/opencga/catalog/migration/MigrationTool.java +++ b/opencga-catalog/src/main/java/org/opencb/opencga/catalog/migration/MigrationTool.java @@ -184,7 +184,7 @@ protected final void migrateCollection(MongoCollection inputCollection List> list = new ArrayList<>(BATCH_SIZE); ProgressLogger progressLogger = new ProgressLogger("Execute bulk update").setBatchSize(BATCH_SIZE); - try (MongoCursor it = inputCollection.find(query).projection(projection).cursor()) { + try (MongoCursor it = inputCollection.find(query).noCursorTimeout(true).projection(projection).cursor()) { while (it.hasNext()) { Document document = it.next(); migrateFunc.accept(document, list); diff --git a/opencga-catalog/src/test/java/org/opencb/opencga/TestParamConstants.java b/opencga-catalog/src/test/java/org/opencb/opencga/TestParamConstants.java new file mode 100644 index 00000000000..0ac6d8bff98 --- /dev/null +++ b/opencga-catalog/src/test/java/org/opencb/opencga/TestParamConstants.java @@ -0,0 +1,8 @@ +package org.opencb.opencga; + +public class TestParamConstants { + + public static final String ADMIN_PASSWORD = "4dMiNiStR4t0R."; + public static final String PASSWORD = "p4sSw0rD."; + +} diff --git a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/auth/authorization/CatalogAuthorizationManagerTest.java b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/auth/authorization/CatalogAuthorizationManagerTest.java index 2c58e19d77d..eb42da8df7b 100644 --- a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/auth/authorization/CatalogAuthorizationManagerTest.java +++ b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/auth/authorization/CatalogAuthorizationManagerTest.java @@ -25,6 +25,7 @@ import org.opencb.commons.datastore.core.Query; import org.opencb.commons.datastore.core.QueryOptions; import org.opencb.commons.test.GenericTest; +import org.opencb.opencga.TestParamConstants; import org.opencb.opencga.catalog.db.api.SampleDBAdaptor; import org.opencb.opencga.catalog.exceptions.CatalogAuthorizationException; import org.opencb.opencga.catalog.exceptions.CatalogDBException; @@ -71,7 +72,6 @@ public class CatalogAuthorizationManagerTest extends GenericTest { private final String studyAdminUser2 = "studyAdmin2"; private final String memberUser = "member_member"; private final String externalUser = "external"; - private final String password = "1234"; private final String groupAdmin = "@admins"; private final String groupMember = "@analyst"; @@ -130,20 +130,20 @@ public void before() throws Exception { CatalogManagerExternalResource.clearCatalog(configuration); catalogManager = new CatalogManager(configuration); - catalogManager.installCatalogDB("dummy", "admin", "opencga@admin.com", "", true); + catalogManager.installCatalogDB("dummy", "4dMiNiStR4t0R.", "opencga@admin.com", "", true); fileManager = catalogManager.getFileManager(); - catalogManager.getUserManager().create(ownerUser, ownerUser, "email@ccc.ccc", password, "ASDF", null, Account.AccountType.FULL, null); - catalogManager.getUserManager().create(studyAdminUser1, studyAdminUser1, "email@ccc.ccc", password, "ASDF", null, Account.AccountType.FULL, null); - catalogManager.getUserManager().create(studyAdminUser2, studyAdminUser2, "email@ccc.ccc", password, "ASDF", null, Account.AccountType.FULL, null); - catalogManager.getUserManager().create(memberUser, memberUser, "email@ccc.ccc", password, "ASDF", null, Account.AccountType.FULL, null); - catalogManager.getUserManager().create(externalUser, externalUser, "email@ccc.ccc", password, "ASDF", null, Account.AccountType.FULL, null); + catalogManager.getUserManager().create(ownerUser, ownerUser, "email@ccc.ccc", TestParamConstants.PASSWORD, "ASDF", null, Account.AccountType.FULL, null); + catalogManager.getUserManager().create(studyAdminUser1, studyAdminUser1, "email@ccc.ccc", TestParamConstants.PASSWORD, "ASDF", null, Account.AccountType.FULL, null); + catalogManager.getUserManager().create(studyAdminUser2, studyAdminUser2, "email@ccc.ccc", TestParamConstants.PASSWORD, "ASDF", null, Account.AccountType.FULL, null); + catalogManager.getUserManager().create(memberUser, memberUser, "email@ccc.ccc", TestParamConstants.PASSWORD, "ASDF", null, Account.AccountType.FULL, null); + catalogManager.getUserManager().create(externalUser, externalUser, "email@ccc.ccc", TestParamConstants.PASSWORD, "ASDF", null, Account.AccountType.FULL, null); - ownerSessionId = catalogManager.getUserManager().login(ownerUser, password).getToken(); - studyAdmin1SessionId = catalogManager.getUserManager().login(studyAdminUser1, password).getToken(); - studyAdmin2SessionId = catalogManager.getUserManager().login(studyAdminUser2, password).getToken(); - memberSessionId = catalogManager.getUserManager().login(memberUser, password).getToken(); - externalSessionId = catalogManager.getUserManager().login(externalUser, password).getToken(); + ownerSessionId = catalogManager.getUserManager().login(ownerUser, TestParamConstants.PASSWORD).getToken(); + studyAdmin1SessionId = catalogManager.getUserManager().login(studyAdminUser1, TestParamConstants.PASSWORD).getToken(); + studyAdmin2SessionId = catalogManager.getUserManager().login(studyAdminUser2, TestParamConstants.PASSWORD).getToken(); + memberSessionId = catalogManager.getUserManager().login(memberUser, TestParamConstants.PASSWORD).getToken(); + externalSessionId = catalogManager.getUserManager().login(externalUser, TestParamConstants.PASSWORD).getToken(); p1 = catalogManager.getProjectManager().create("p1", "p1", null, "Homo sapiens", null, "GRCh38", INCLUDE_RESULT, ownerSessionId).first().getId(); @@ -356,7 +356,7 @@ public void removeMemberFromNonBelongingGroup() throws CatalogException { @Test public void addExistingUserToRole() throws CatalogException { String newUser = "newUser"; - catalogManager.getUserManager().create(newUser, newUser, "email@ccc.ccc", password, "ASDF", null, Account.AccountType.FULL, null); + catalogManager.getUserManager().create(newUser, newUser, "email@ccc.ccc", TestParamConstants.PASSWORD, "ASDF", null, Account.AccountType.FULL, null); StudyAclParams aclParams = new StudyAclParams("", AuthorizationManager.ROLE_ANALYST); catalogManager.getStudyManager().updateAcl(Arrays.asList(studyFqn), newUser, aclParams, ParamUtils.AclAction.ADD, ownerSessionId); } @@ -365,7 +365,7 @@ public void addExistingUserToRole() throws CatalogException { @Test public void addExistingUserToRole2() throws CatalogException { String newUser = "newUser"; - catalogManager.getUserManager().create(newUser, newUser, "email@ccc.ccc", password, "ASDF", null, Account.AccountType.FULL, null); + catalogManager.getUserManager().create(newUser, newUser, "email@ccc.ccc", TestParamConstants.PASSWORD, "ASDF", null, Account.AccountType.FULL, null); thrown.expect(CatalogAuthorizationException.class); thrown.expectMessage("Only owners or administrative users"); StudyAclParams aclParams = new StudyAclParams("", AuthorizationManager.ROLE_ANALYST); @@ -375,7 +375,7 @@ public void addExistingUserToRole2() throws CatalogException { @Test public void addExistingGroupToRole() throws CatalogException { String newUser = "newUser"; - catalogManager.getUserManager().create(newUser, newUser, "email@ccc.ccc", password, "ASDF", null, Account.AccountType.FULL, null); + catalogManager.getUserManager().create(newUser, newUser, "email@ccc.ccc", TestParamConstants.PASSWORD, "ASDF", null, Account.AccountType.FULL, null); String group = "@newGroup"; // catalogManager.addUsersToGroup(studyFqn, group, newUser, studyAdmin1SessionId); catalogManager.getStudyManager().createGroup(studyFqn, group, Collections.singletonList(newUser), studyAdmin1SessionId); @@ -606,8 +606,8 @@ public void readNonSharedFile() throws CatalogException { @Test public void readFileNoStudyMember() throws CatalogException, IOException { String newUser = "newUser"; - catalogManager.getUserManager().create(newUser, newUser, "asda@mail.com", password, "org", 1000L, Account.AccountType.FULL, null); - String sessionId = catalogManager.getUserManager().login(newUser, password).getToken(); + catalogManager.getUserManager().create(newUser, newUser, "asda@mail.com", TestParamConstants.PASSWORD, "org", 1000L, Account.AccountType.FULL, null); + String sessionId = catalogManager.getUserManager().login(newUser, TestParamConstants.PASSWORD).getToken(); thrown.expect(CatalogAuthorizationException.class); catalogManager.getFileManager().get(studyFqn, data, null, sessionId); } @@ -616,8 +616,8 @@ public void readFileNoStudyMember() throws CatalogException, IOException { public void readFileSharedForGroup() throws CatalogException, IOException { // Add a new user to a new group String newUser = "newUser"; - catalogManager.getUserManager().create(newUser, newUser, "asda@mail.com", password, "org", 1000L, Account.AccountType.FULL, null); - String sessionId = catalogManager.getUserManager().login(newUser, password).getToken(); + catalogManager.getUserManager().create(newUser, newUser, "asda@mail.com", TestParamConstants.PASSWORD, "org", 1000L, Account.AccountType.FULL, null); + String sessionId = catalogManager.getUserManager().login(newUser, TestParamConstants.PASSWORD).getToken(); String newGroup = "@external"; // catalogManager.addUsersToGroup(studyFqn, "@external", newUser, ownerSessionId); catalogManager.getStudyManager().createGroup(studyFqn, newGroup, Collections.singletonList(newUser), ownerSessionId); @@ -646,8 +646,8 @@ public void readFileForbiddenForUser() throws CatalogException { public void readFileForbiddenForGroupSharedWithUser() throws CatalogException, IOException { // Add a new user to a new group String newUser = "newUser"; - catalogManager.getUserManager().create(newUser, newUser, "asda@mail.com", password, "org", 1000L, Account.AccountType.FULL, null); - String sessionId = catalogManager.getUserManager().login(ownerUser, password).getToken(); + catalogManager.getUserManager().create(newUser, newUser, "asda@mail.com", TestParamConstants.PASSWORD, "org", 1000L, Account.AccountType.FULL, null); + String sessionId = catalogManager.getUserManager().login(ownerUser, TestParamConstants.PASSWORD).getToken(); String newGroup = "@external"; // catalogManager.addUsersToGroup(studyFqn, "@external", newUser, ownerSessionId); catalogManager.getStudyManager().createGroup(studyFqn, newGroup, Collections.singletonList(newUser), ownerSessionId); @@ -706,8 +706,8 @@ public void createNonSharedFile() throws CatalogException { @Test public void createFileNoStudyMember() throws CatalogException, IOException { String newUser = "newUser"; - catalogManager.getUserManager().create(newUser, newUser, "asda@mail.com", password, "org", 1000L, Account.AccountType.FULL, null); - String sessionId = catalogManager.getUserManager().login(newUser, password).getToken(); + catalogManager.getUserManager().create(newUser, newUser, "asda@mail.com", TestParamConstants.PASSWORD, "org", 1000L, Account.AccountType.FULL, null); + String sessionId = catalogManager.getUserManager().login(newUser, TestParamConstants.PASSWORD).getToken(); thrown.expect(CatalogAuthorizationException.class); catalogManager.getFileManager().createFolder(studyFqn, Paths.get("data/my_folder/").toString(), false, null, QueryOptions.empty(), sessionId); @@ -767,8 +767,8 @@ public void readSampleExplicitForbidden() throws CatalogException { @Test public void readSampleExternalUser() throws CatalogException, IOException { String newUser = "newUser"; - catalogManager.getUserManager().create(newUser, newUser, "asda@mail.com", password, "org", 1000L, Account.AccountType.FULL, null); - String sessionId = catalogManager.getUserManager().login(newUser, password).getToken(); + catalogManager.getUserManager().create(newUser, newUser, "asda@mail.com", TestParamConstants.PASSWORD, "org", 1000L, Account.AccountType.FULL, null); + String sessionId = catalogManager.getUserManager().login(newUser, TestParamConstants.PASSWORD).getToken(); thrown.expect(CatalogAuthorizationException.class); catalogManager.getSampleManager().get(studyFqn, smp2.getId(), null, sessionId); } @@ -793,8 +793,8 @@ public void readSampleForbiddenForExternalUser() throws CatalogException { public void readSampleSharedForGroup() throws CatalogException, IOException { // Add a new user to a new group String newUser = "newUser"; - catalogManager.getUserManager().create(newUser, newUser, "asda@mail.com", password, "org", 1000L, Account.AccountType.FULL, null); - String sessionId = catalogManager.getUserManager().login(ownerUser, password).getToken(); + catalogManager.getUserManager().create(newUser, newUser, "asda@mail.com", TestParamConstants.PASSWORD, "org", 1000L, Account.AccountType.FULL, null); + String sessionId = catalogManager.getUserManager().login(ownerUser, TestParamConstants.PASSWORD).getToken(); String newGroup = "@external"; // catalogManager.addUsersToGroup(studyFqn, "@external", newUser, ownerSessionId); catalogManager.getStudyManager().createGroup(studyFqn, newGroup, Collections.singletonList(newUser), ownerSessionId); @@ -821,8 +821,8 @@ public void readSampleSharedForOthers() throws CatalogException { public void readSampleSharedForOthersNotWithStudyPermissions() throws CatalogException, IOException { // Add a new user to a new group String newUser = "newUser"; - catalogManager.getUserManager().create(newUser, newUser, "asda@mail.com", password, "org", 1000L, Account.AccountType.FULL, null); - String sessionId = catalogManager.getUserManager().login(ownerUser, password).getToken(); + catalogManager.getUserManager().create(newUser, newUser, "asda@mail.com", TestParamConstants.PASSWORD, "org", 1000L, Account.AccountType.FULL, null); + String sessionId = catalogManager.getUserManager().login(ownerUser, TestParamConstants.PASSWORD).getToken(); catalogManager.getStudyManager().updateGroup(studyFqn, "@members", ParamUtils.BasicUpdateAction.ADD, new GroupUpdateParams(Collections.singletonList(newUser)), ownerSessionId); diff --git a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/AbstractManagerTest.java b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/AbstractManagerTest.java index 45a4b6e9538..775e3d52857 100644 --- a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/AbstractManagerTest.java +++ b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/AbstractManagerTest.java @@ -24,6 +24,7 @@ import org.opencb.commons.datastore.core.ObjectMap; import org.opencb.commons.datastore.core.QueryOptions; import org.opencb.commons.test.GenericTest; +import org.opencb.opencga.TestParamConstants; import org.opencb.opencga.catalog.exceptions.CatalogException; import org.opencb.opencga.core.api.ParamConstants; import org.opencb.opencga.core.models.common.AnnotationSet; @@ -47,7 +48,6 @@ public class AbstractManagerTest extends GenericTest { - public final static String PASSWORD = "asdf"; @Rule public ExpectedException thrown = ExpectedException.none(); @@ -88,13 +88,13 @@ public void setUp() throws IOException, CatalogException { public void setUpCatalogManager(CatalogManager catalogManager) throws IOException, CatalogException { - catalogManager.getUserManager().create("user", "User Name", "mail@ebi.ac.uk", PASSWORD, "", null, Account.AccountType.FULL, null); - catalogManager.getUserManager().create("user2", "User2 Name", "mail2@ebi.ac.uk", PASSWORD, "", null, Account.AccountType.FULL, null); - catalogManager.getUserManager().create("user3", "User3 Name", "user.2@e.mail", PASSWORD, "ACME", null, Account.AccountType.FULL, null); + catalogManager.getUserManager().create("user", "User Name", "mail@ebi.ac.uk", TestParamConstants.PASSWORD, "", null, Account.AccountType.FULL, null); + catalogManager.getUserManager().create("user2", "User2 Name", "mail2@ebi.ac.uk", TestParamConstants.PASSWORD, "", null, Account.AccountType.FULL, null); + catalogManager.getUserManager().create("user3", "User3 Name", "user.2@e.mail", TestParamConstants.PASSWORD, "ACME", null, Account.AccountType.FULL, null); - token = catalogManager.getUserManager().login("user", PASSWORD).getToken(); - sessionIdUser2 = catalogManager.getUserManager().login("user2", PASSWORD).getToken(); - sessionIdUser3 = catalogManager.getUserManager().login("user3", PASSWORD).getToken(); + token = catalogManager.getUserManager().login("user", TestParamConstants.PASSWORD).getToken(); + sessionIdUser2 = catalogManager.getUserManager().login("user2", TestParamConstants.PASSWORD).getToken(); + sessionIdUser3 = catalogManager.getUserManager().login("user3", TestParamConstants.PASSWORD).getToken(); project1 = catalogManager.getProjectManager().create("1000G", "Project about some genomes", "", "Homo sapiens", null, "GRCh38", INCLUDE_RESULT, token).first().getId(); diff --git a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/CatalogManagerExternalResource.java b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/CatalogManagerExternalResource.java index e93f86806bd..148a5352338 100644 --- a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/CatalogManagerExternalResource.java +++ b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/CatalogManagerExternalResource.java @@ -23,6 +23,7 @@ import org.opencb.commons.datastore.core.DataStoreServerAddress; import org.opencb.commons.datastore.mongodb.MongoDataStore; import org.opencb.commons.datastore.mongodb.MongoDataStoreManager; +import org.opencb.opencga.TestParamConstants; import org.opencb.opencga.catalog.exceptions.CatalogException; import org.opencb.opencga.core.common.TimeUtils; import org.opencb.opencga.core.common.UriUtils; @@ -73,7 +74,7 @@ public void before() throws Exception { Files.createDirectory(opencgaHome); } catalogManager = new CatalogManager(configuration); - catalogManager.installCatalogDB("dummy", "admin", "opencga@admin.com", "", true); + catalogManager.installCatalogDB("dummy", TestParamConstants.ADMIN_PASSWORD, "opencga@admin.com", "", true); catalogManager.close(); // FIXME!! Should not need to create again the catalogManager // Have to create again the CatalogManager, as it has a random "secretKey" inside diff --git a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/CatalogManagerTest.java b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/CatalogManagerTest.java index 0109f91e364..f32a1c4503e 100644 --- a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/CatalogManagerTest.java +++ b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/CatalogManagerTest.java @@ -28,6 +28,8 @@ import org.opencb.biodata.models.core.SexOntologyTermAnnotation; import org.opencb.biodata.models.pedigree.IndividualProperty; import org.opencb.commons.datastore.core.*; +import org.opencb.opencga.TestParamConstants; +import org.opencb.opencga.catalog.auth.authorization.AuthorizationManager; import org.opencb.opencga.catalog.db.api.*; import org.opencb.opencga.catalog.exceptions.*; import org.opencb.opencga.catalog.utils.Constants; @@ -80,7 +82,7 @@ public void createStudyFailMoreThanOneProject() throws CatalogException { @Test public void testAdminUserExists() throws Exception { - String token = catalogManager.getUserManager().loginAsAdmin("admin").getToken(); + String token = catalogManager.getUserManager().loginAsAdmin(TestParamConstants.ADMIN_PASSWORD).getToken(); assertEquals("opencga", catalogManager.getUserManager().getUserId(token)); } @@ -88,14 +90,14 @@ public void testAdminUserExists() throws Exception { public void testCreateExistingUser() throws Exception { thrown.expect(CatalogException.class); thrown.expectMessage(containsString("already exists")); - catalogManager.getUserManager().create("user", "User Name", "mail@ebi.ac.uk", PASSWORD, "", null, Account.AccountType.FULL, null); + catalogManager.getUserManager().create("user", "User Name", "mail@ebi.ac.uk", TestParamConstants.PASSWORD, "", null, Account.AccountType.FULL, null); } @Test public void testCreateAnonymousUser() throws Exception { thrown.expect(CatalogParameterException.class); thrown.expectMessage(containsString("reserved")); - catalogManager.getUserManager().create(ParamConstants.ANONYMOUS_USER_ID, "User Name", "mail@ebi.ac.uk", PASSWORD, "", null, + catalogManager.getUserManager().create(ParamConstants.ANONYMOUS_USER_ID, "User Name", "mail@ebi.ac.uk", TestParamConstants.PASSWORD, "", null, Account.AccountType.FULL, null); } @@ -103,13 +105,13 @@ public void testCreateAnonymousUser() throws Exception { public void testCreateRegisteredUser() throws Exception { thrown.expect(CatalogParameterException.class); thrown.expectMessage(containsString("reserved")); - catalogManager.getUserManager().create(ParamConstants.REGISTERED_USERS, "User Name", "mail@ebi.ac.uk", PASSWORD, "", null, + catalogManager.getUserManager().create(ParamConstants.REGISTERED_USERS, "User Name", "mail@ebi.ac.uk", TestParamConstants.PASSWORD, "", null, Account.AccountType.FULL, null); } @Test public void testLogin() throws Exception { - catalogManager.getUserManager().login("user", PASSWORD); + catalogManager.getUserManager().login("user", TestParamConstants.PASSWORD); thrown.expect(CatalogAuthenticationException.class); thrown.expectMessage(allOf(containsString("Incorrect"), containsString("password"))); @@ -193,7 +195,7 @@ public void testModifyUser() throws CatalogException, InterruptedException, IOEx catalogManager.getUserManager().update("user", params, null, token); catalogManager.getUserManager().update("user", new ObjectMap("email", newEmail), null, token); - catalogManager.getUserManager().changePassword("user", PASSWORD, newPassword); + catalogManager.getUserManager().changePassword("user", TestParamConstants.PASSWORD, newPassword); List userList = catalogManager.getUserManager().get("user", new QueryOptions(QueryOptions .INCLUDE, Arrays.asList(UserDBAdaptor.QueryParams.NAME.key(), UserDBAdaptor.QueryParams.EMAIL.key(), @@ -208,8 +210,8 @@ public void testModifyUser() throws CatalogException, InterruptedException, IOEx assertEquals(userPost.getAttributes().get(entry.getKey()), entry.getValue()); } - catalogManager.getUserManager().changePassword("user", newPassword, PASSWORD); - catalogManager.getUserManager().login("user", PASSWORD); + catalogManager.getUserManager().changePassword("user", newPassword, TestParamConstants.PASSWORD); + catalogManager.getUserManager().login("user", TestParamConstants.PASSWORD); try { params = new ObjectMap(); @@ -364,7 +366,7 @@ public void importLdapGroups() throws CatalogException, IOException { @Test public void createEmptyGroup() throws CatalogException { - catalogManager.getUserManager().create("test", "test", "test@mail.com", "test", null, 100L, Account.AccountType.GUEST, null); + catalogManager.getUserManager().create("test", "test", "test@mail.com", TestParamConstants.PASSWORD, null, 100L, Account.AccountType.GUEST, null); catalogManager.getStudyManager().createGroup("user@1000G:phase1", "group_cancer_some_thing_else", null, token); catalogManager.getStudyManager().updateGroup("user@1000G:phase1", "group_cancer_some_thing_else", ParamUtils.BasicUpdateAction.ADD, new GroupUpdateParams(Collections.singletonList("test")), token); @@ -372,7 +374,7 @@ public void createEmptyGroup() throws CatalogException { @Test public void testAssignPermissions() throws CatalogException { - catalogManager.getUserManager().create("test", "test", "test@mail.com", "test", null, 100L, Account.AccountType.GUEST, null); + catalogManager.getUserManager().create("test", "test", "test@mail.com", TestParamConstants.PASSWORD, null, 100L, Account.AccountType.GUEST, null); catalogManager.getStudyManager().createGroup("user@1000G:phase1", "group_cancer_some_thing_else", Collections.singletonList("test"), token); @@ -381,7 +383,7 @@ public void testAssignPermissions() throws CatalogException { new StudyAclParams("", "view_only"), ParamUtils.AclAction.SET, token); assertTrue(permissions.first().containsKey("@group_cancer_some_thing_else")); - String token = catalogManager.getUserManager().login("test", "test").getToken(); + String token = catalogManager.getUserManager().login("test", TestParamConstants.PASSWORD).getToken(); DataResult studyDataResult = catalogManager.getStudyManager().get("user@1000G:phase1", QueryOptions.empty(), token); assertEquals(1, studyDataResult.getNumResults()); assertTrue(studyDataResult.first().getAttributes().isEmpty()); @@ -956,7 +958,7 @@ public void submitJobWithoutPermissions() throws CatalogException { // Grant view permissions, but no EXECUTION permission catalogManager.getStudyManager().updateAcl(Collections.singletonList(studyFqn), "user3", - new StudyAclParams("", "view-only"), ParamUtils.AclAction.SET, token); + new StudyAclParams("", AuthorizationManager.ROLE_VIEW_ONLY), ParamUtils.AclAction.SET, token); try { catalogManager.getJobManager().submit(studyFqn, "variant-index", Enums.Priority.MEDIUM, new ObjectMap(), sessionIdUser3); @@ -979,7 +981,7 @@ public void submitJobWithPermissions() throws CatalogException { // Grant view permissions, but no EXECUTION permission catalogManager.getStudyManager().updateAcl(Collections.singletonList(studyFqn), "user3", - new StudyAclParams(StudyAclEntry.StudyPermissions.EXECUTE_JOBS.name(), "view-only"), ParamUtils.AclAction.SET, token); + new StudyAclParams(StudyAclEntry.StudyPermissions.EXECUTE_JOBS.name(), AuthorizationManager.ROLE_VIEW_ONLY), ParamUtils.AclAction.SET, token); OpenCGAResult search = catalogManager.getJobManager().submit(studyFqn, "variant-index", Enums.Priority.MEDIUM, new ObjectMap(), sessionIdUser3); @@ -991,7 +993,7 @@ public void submitJobWithPermissions() throws CatalogException { public void deleteJobTest() throws CatalogException { // Grant view permissions, but no EXECUTION permission catalogManager.getStudyManager().updateAcl(Collections.singletonList(studyFqn), "user3", - new StudyAclParams(StudyAclEntry.StudyPermissions.EXECUTE_JOBS.name(), "view-only"), ParamUtils.AclAction.SET, token); + new StudyAclParams(StudyAclEntry.StudyPermissions.EXECUTE_JOBS.name(), AuthorizationManager.ROLE_VIEW_ONLY), ParamUtils.AclAction.SET, token); OpenCGAResult search = catalogManager.getJobManager().submit(studyFqn, "variant-index", Enums.Priority.MEDIUM, new ObjectMap(), sessionIdUser3); diff --git a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/ClinicalAnalysisManagerTest.java b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/ClinicalAnalysisManagerTest.java index 974e10adfc8..9afcbde4463 100644 --- a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/ClinicalAnalysisManagerTest.java +++ b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/ClinicalAnalysisManagerTest.java @@ -37,6 +37,7 @@ import org.opencb.commons.datastore.core.Query; import org.opencb.commons.datastore.core.QueryOptions; import org.opencb.commons.test.GenericTest; +import org.opencb.opencga.TestParamConstants; import org.opencb.opencga.catalog.db.api.ClinicalAnalysisDBAdaptor; import org.opencb.opencga.catalog.db.api.InterpretationDBAdaptor; import org.opencb.opencga.catalog.exceptions.CatalogException; @@ -73,7 +74,6 @@ public class ClinicalAnalysisManagerTest extends GenericTest { - public final static String PASSWORD = "asdf"; public final static String STUDY = "user@1000G:phase1"; @Rule public ExpectedException thrown = ExpectedException.none(); @@ -96,10 +96,10 @@ public void setUp() throws IOException, CatalogException { public void setUpCatalogManager(CatalogManager catalogManager) throws IOException, CatalogException { - catalogManager.getUserManager().create("user", "User Name", "mail@ebi.ac.uk", PASSWORD, "", null, Account.AccountType.FULL, null); - sessionIdUser = catalogManager.getUserManager().login("user", PASSWORD).getToken(); + catalogManager.getUserManager().create("user", "User Name", "mail@ebi.ac.uk", TestParamConstants.PASSWORD, "", null, Account.AccountType.FULL, null); + sessionIdUser = catalogManager.getUserManager().login("user", TestParamConstants.PASSWORD).getToken(); - catalogManager.getUserManager().create("user2", "User Name2", "mail2@ebi.ac.uk", PASSWORD, "", null, Account.AccountType.GUEST, + catalogManager.getUserManager().create("user2", "User Name2", "mail2@ebi.ac.uk", TestParamConstants.PASSWORD, "", null, Account.AccountType.GUEST, null); String projectId = catalogManager.getProjectManager().create("1000G", "Project about some genomes", "", "Homo sapiens", @@ -978,7 +978,7 @@ public void updateInterpretationComments() throws CatalogException { @Test public void assignPermissions() throws CatalogException { ClinicalAnalysis clinicalAnalysis = createDummyEnvironment(true, false).first(); - catalogManager.getUserManager().create("external", "User Name", "external@mail.com", PASSWORD, "", null, + catalogManager.getUserManager().create("external", "User Name", "external@mail.com", TestParamConstants.PASSWORD, "", null, Account.AccountType.GUEST, null); OpenCGAResult>> aclResult = catalogManager.getClinicalAnalysisManager().getAcls(STUDY, diff --git a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/FamilyManagerTest.java b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/FamilyManagerTest.java index 2b5944ac531..06c1504727c 100644 --- a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/FamilyManagerTest.java +++ b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/FamilyManagerTest.java @@ -30,6 +30,7 @@ import org.opencb.commons.datastore.core.Query; import org.opencb.commons.datastore.core.QueryOptions; import org.opencb.commons.test.GenericTest; +import org.opencb.opencga.TestParamConstants; import org.opencb.opencga.catalog.db.api.FamilyDBAdaptor; import org.opencb.opencga.catalog.db.api.IndividualDBAdaptor; import org.opencb.opencga.catalog.exceptions.CatalogAuthorizationException; @@ -64,7 +65,6 @@ */ public class FamilyManagerTest extends GenericTest { - public final static String PASSWORD = "asdf"; public final static String STUDY = "user@1000G:phase1"; @Rule public ExpectedException thrown = ExpectedException.none(); @@ -86,8 +86,8 @@ public void setUp() throws IOException, CatalogException { } public void setUpCatalogManager(CatalogManager catalogManager) throws CatalogException { - catalogManager.getUserManager().create("user", "User Name", "mail@ebi.ac.uk", PASSWORD, "", null, Account.AccountType.FULL, null); - sessionIdUser = catalogManager.getUserManager().login("user", PASSWORD).getToken(); + catalogManager.getUserManager().create("user", "User Name", "mail@ebi.ac.uk", TestParamConstants.PASSWORD, "", null, Account.AccountType.FULL, null); + sessionIdUser = catalogManager.getUserManager().login("user", TestParamConstants.PASSWORD).getToken(); String projectId = catalogManager.getProjectManager().create("1000G", "Project about some genomes", "", "Homo sapiens", null, "GRCh38", INCLUDE_RESULT, sessionIdUser).first().getId(); @@ -144,6 +144,97 @@ public void createFamily() throws CatalogException { assertTrue("Father id not associated to any children", fatherIdUpdated); } + @Test + public void deleteFamilyTest() throws CatalogException { + DataResult familyDataResult = createDummyFamily("Martinez-Martinez", true); + String familyId = familyDataResult.first().getId(); + + assertEquals(1, familyDataResult.getNumResults()); + assertEquals(5, familyDataResult.first().getMembers().size()); + for (Individual member : familyDataResult.first().getMembers()) { + assertEquals(1, member.getFamilyIds().size()); + assertEquals(familyId, member.getFamilyIds().get(0)); + } + + catalogManager.getFamilyManager().delete(STUDY, Collections.singletonList(familyId), QueryOptions.empty(), sessionIdUser); + try { + catalogManager.getFamilyManager().get(STUDY, familyId, QueryOptions.empty(), sessionIdUser); + fail("Family should not exist"); + } catch (CatalogException e) { + // empty block + } + + List members = familyDataResult.first().getMembers().stream().map(Individual::getId).collect(Collectors.toList()); + OpenCGAResult result = catalogManager.getIndividualManager().get(STUDY, members, QueryOptions.empty(), sessionIdUser); + + for (Individual member : result.getResults()) { + assertTrue(member.getFamilyIds().isEmpty()); + } + } + + @Test + public void deleteWithClinicalAnalysisTest() throws CatalogException { + Sample sample = new Sample().setId("sample1"); + catalogManager.getSampleManager().create(STUDY, sample, QueryOptions.empty(), sessionIdUser); + + sample = new Sample().setId("sample2"); + catalogManager.getSampleManager().create(STUDY, sample, QueryOptions.empty(), sessionIdUser); + + sample = new Sample().setId("sample3"); + catalogManager.getSampleManager().create(STUDY, sample, QueryOptions.empty(), sessionIdUser); + + sample = new Sample().setId("sample4"); + catalogManager.getSampleManager().create(STUDY, sample, QueryOptions.empty(), sessionIdUser); + + Individual individual = new Individual() + .setId("proband") + .setDisorders(Collections.singletonList(new Disorder().setId("disorder"))); + catalogManager.getIndividualManager().create(STUDY, individual, Arrays.asList("sample1", "sample2"), QueryOptions.empty(), sessionIdUser); + + individual = new Individual().setId("father"); + catalogManager.getIndividualManager().create(STUDY, individual, Arrays.asList("sample3"), QueryOptions.empty(), sessionIdUser); + + Family family = new Family().setId("family"); + catalogManager.getFamilyManager().create(STUDY, family, Arrays.asList("proband", "father"), QueryOptions.empty(), sessionIdUser); + + family.setMembers(Arrays.asList( + new Individual().setId("proband").setSamples(Collections.singletonList(new Sample().setId("sample2"))), + new Individual().setId("father").setSamples(Collections.singletonList(new Sample().setId("sample3"))) + )); + + ClinicalAnalysis clinicalAnalysis = new ClinicalAnalysis() + .setId("clinical") + .setProband(new Individual().setId("proband")) + .setFamily(family) + .setLocked(true) + .setType(ClinicalAnalysis.Type.FAMILY); + catalogManager.getClinicalAnalysisManager().create(STUDY, clinicalAnalysis, QueryOptions.empty(), sessionIdUser); + + try { + catalogManager.getFamilyManager().delete(STUDY, Collections.singletonList(family.getId()), QueryOptions.empty(), sessionIdUser); + fail("Clinical is locked. It should not delete anything"); + } catch (CatalogException e) { + System.out.println(e.getMessage()); + // empty block + } + + OpenCGAResult result = catalogManager.getFamilyManager().get(STUDY, family.getId(), QueryOptions.empty(), sessionIdUser); + assertEquals(1, result.getNumResults()); + + catalogManager.getClinicalAnalysisManager().update(STUDY, clinicalAnalysis.getId(), + new ClinicalAnalysisUpdateParams() + .setLocked(false), + QueryOptions.empty(), sessionIdUser); + + try { + catalogManager.getFamilyManager().delete(STUDY, Collections.singletonList(family.getId()), QueryOptions.empty(), sessionIdUser); + fail("Clinical is not locked. It should not delete anything either"); + } catch (CatalogException e) { + System.out.println(e.getMessage()); + // empty block + } + } + @Test public void updateFamilyReferencesInIndividualTest() throws CatalogException { DataResult familyDataResult = createDummyFamily("Martinez-Martinez", true); @@ -372,8 +463,8 @@ public void testRoles() throws CatalogException { public void testPropagateFamilyPermission() throws CatalogException { createDummyFamily("Martinez-Martinez", true); - catalogManager.getUserManager().create("user2", "User Name", "mail@ebi.ac.uk", PASSWORD, "", null, Account.AccountType.GUEST, null); - String token = catalogManager.getUserManager().login("user2", PASSWORD).getToken(); + catalogManager.getUserManager().create("user2", "User Name", "mail@ebi.ac.uk", TestParamConstants.PASSWORD, "", null, Account.AccountType.GUEST, null); + String token = catalogManager.getUserManager().login("user2", TestParamConstants.PASSWORD).getToken(); try { familyManager.get(STUDY, "Martinez-Martinez", QueryOptions.empty(), token); @@ -433,8 +524,8 @@ public void testPropagateFamilyPermission() throws CatalogException { public void getFamilyWithOnlyAllowedMembers2() throws CatalogException, IOException { createDummyFamily("Martinez-Martinez", true); - catalogManager.getUserManager().create("user2", "User Name", "mail@ebi.ac.uk", PASSWORD, "", null, Account.AccountType.GUEST, null); - String token = catalogManager.getUserManager().login("user2", PASSWORD).getToken(); + catalogManager.getUserManager().create("user2", "User Name", "mail@ebi.ac.uk", TestParamConstants.PASSWORD, "", null, Account.AccountType.GUEST, null); + String token = catalogManager.getUserManager().login("user2", TestParamConstants.PASSWORD).getToken(); try { familyManager.get(STUDY, "Martinez-Martinez", QueryOptions.empty(), token); diff --git a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/FileManagerTest.java b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/FileManagerTest.java index b15b1b5a55d..7fb1081c84c 100644 --- a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/FileManagerTest.java +++ b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/FileManagerTest.java @@ -26,6 +26,7 @@ import org.opencb.commons.datastore.core.ObjectMap; import org.opencb.commons.datastore.core.Query; import org.opencb.commons.datastore.core.QueryOptions; +import org.opencb.opencga.TestParamConstants; import org.opencb.opencga.catalog.db.api.FileDBAdaptor; import org.opencb.opencga.catalog.db.api.SampleDBAdaptor; import org.opencb.opencga.catalog.db.api.StudyDBAdaptor; @@ -145,9 +146,9 @@ public void testLinkCram() throws CatalogException { @Test public void testLinkAnalystUser() throws CatalogException { - catalogManager.getUserManager().create("analyst", "analyst", "a@mail.com", "analyst", "", 200000L, Account.AccountType.GUEST, null); + catalogManager.getUserManager().create("analyst", "analyst", "a@mail.com", TestParamConstants.PASSWORD, "", 200000L, Account.AccountType.GUEST, null); catalogManager.getStudyManager().updateAcl(studyFqn, "analyst", new StudyAclParams("", "analyst"), ParamUtils.AclAction.SET, token); - String analystToken = catalogManager.getUserManager().login("analyst", "analyst").getToken(); + String analystToken = catalogManager.getUserManager().login("analyst", TestParamConstants.PASSWORD).getToken(); String reference = getClass().getResource("/biofiles/cram/hg19mini.fasta").getFile(); File referenceFile = fileManager.link(studyFqn, Paths.get(reference).toUri(), "", null, analystToken).first(); @@ -157,9 +158,9 @@ public void testLinkAnalystUser() throws CatalogException { @Test public void testLinkUserWithNoWritePermissions() throws CatalogException { - catalogManager.getUserManager().create("view_user", "view_user", "a@mail.com", "view_user", "", 200000L, Account.AccountType.GUEST, null); + catalogManager.getUserManager().create("view_user", "view_user", "a@mail.com", TestParamConstants.PASSWORD, "", 200000L, Account.AccountType.GUEST, null); catalogManager.getStudyManager().updateAcl(studyFqn, "view_user", new StudyAclParams("", "view_only"), ParamUtils.AclAction.SET, token); - String analystToken = catalogManager.getUserManager().login("view_user", "view_user").getToken(); + String analystToken = catalogManager.getUserManager().login("view_user", TestParamConstants.PASSWORD).getToken(); String reference = getClass().getResource("/biofiles/cram/hg19mini.fasta").getFile(); diff --git a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/PanelManagerTest.java b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/PanelManagerTest.java index 3684843c10f..d202e8867df 100644 --- a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/PanelManagerTest.java +++ b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/PanelManagerTest.java @@ -25,6 +25,7 @@ import org.opencb.commons.datastore.core.DataResult; import org.opencb.commons.datastore.core.QueryOptions; import org.opencb.commons.test.GenericTest; +import org.opencb.opencga.TestParamConstants; import org.opencb.opencga.catalog.exceptions.CatalogException; import org.opencb.opencga.core.api.ParamConstants; import org.opencb.opencga.core.models.panel.Panel; @@ -39,7 +40,6 @@ public class PanelManagerTest extends GenericTest { - public final static String PASSWORD = "asdf"; private String studyFqn = "user@1000G:phase1"; @Rule public ExpectedException thrown = ExpectedException.none(); @@ -60,12 +60,12 @@ public void setUp() throws IOException, CatalogException { catalogManager = catalogManagerResource.getCatalogManager(); panelManager = catalogManager.getPanelManager(); setUpCatalogManager(catalogManager); - adminToken = catalogManager.getUserManager().loginAsAdmin("admin").getToken(); + adminToken = catalogManager.getUserManager().loginAsAdmin(TestParamConstants.ADMIN_PASSWORD).getToken(); } private void setUpCatalogManager(CatalogManager catalogManager) throws IOException, CatalogException { - catalogManager.getUserManager().create("user", "User Name", "mail@ebi.ac.uk", PASSWORD, "", null, Account.AccountType.FULL, null); - sessionIdUser = catalogManager.getUserManager().login("user", PASSWORD).getToken(); + catalogManager.getUserManager().create("user", "User Name", "mail@ebi.ac.uk", TestParamConstants.PASSWORD, "", null, Account.AccountType.FULL, null); + sessionIdUser = catalogManager.getUserManager().login("user", TestParamConstants.PASSWORD).getToken(); String projectId = catalogManager.getProjectManager().create("1000G", "Project about some genomes", "", "Homo sapiens", null, "GRCh38", INCLUDE_RESULT, sessionIdUser).first().getId(); diff --git a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/ProjectManagerTest.java b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/ProjectManagerTest.java index ed40802edc6..859f08145bb 100644 --- a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/ProjectManagerTest.java +++ b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/ProjectManagerTest.java @@ -25,6 +25,7 @@ import org.opencb.commons.datastore.core.Query; import org.opencb.commons.datastore.core.QueryOptions; import org.opencb.commons.test.GenericTest; +import org.opencb.opencga.TestParamConstants; import org.opencb.opencga.catalog.db.api.ProjectDBAdaptor; import org.opencb.opencga.catalog.exceptions.CatalogAuthorizationException; import org.opencb.opencga.catalog.exceptions.CatalogException; @@ -45,7 +46,6 @@ */ public class ProjectManagerTest extends GenericTest { - public final static String PASSWORD = "asdf"; @Rule public ExpectedException thrown = ExpectedException.none(); @@ -72,13 +72,13 @@ public void setUp() throws IOException, CatalogException { } public void setUpCatalogManager(CatalogManager catalogManager) throws IOException, CatalogException { - catalogManager.getUserManager().create("user", "User Name", "mail@ebi.ac.uk", PASSWORD, "", null, Account.AccountType.FULL, null); - catalogManager.getUserManager().create("user2", "User2 Name", "mail2@ebi.ac.uk", PASSWORD, "", null, Account.AccountType.FULL, null); - catalogManager.getUserManager().create("user3", "User3 Name", "user.2@e.mail", PASSWORD, "ACME", null, Account.AccountType.FULL, null); + catalogManager.getUserManager().create("user", "User Name", "mail@ebi.ac.uk", TestParamConstants.PASSWORD, "", null, Account.AccountType.FULL, null); + catalogManager.getUserManager().create("user2", "User2 Name", "mail2@ebi.ac.uk", TestParamConstants.PASSWORD, "", null, Account.AccountType.FULL, null); + catalogManager.getUserManager().create("user3", "User3 Name", "user.2@e.mail", TestParamConstants.PASSWORD, "ACME", null, Account.AccountType.FULL, null); - sessionIdUser = catalogManager.getUserManager().login("user", PASSWORD).getToken(); - sessionIdUser2 = catalogManager.getUserManager().login("user2", PASSWORD).getToken(); - sessionIdUser3 = catalogManager.getUserManager().login("user3", PASSWORD).getToken(); + sessionIdUser = catalogManager.getUserManager().login("user", TestParamConstants.PASSWORD).getToken(); + sessionIdUser2 = catalogManager.getUserManager().login("user2", TestParamConstants.PASSWORD).getToken(); + sessionIdUser3 = catalogManager.getUserManager().login("user3", TestParamConstants.PASSWORD).getToken(); project1 = catalogManager.getProjectManager().create("1000G", "Project about some genomes", "", "Homo sapiens", null, "GRCh38", INCLUDE_RESULT, sessionIdUser).first().getId(); diff --git a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/SampleManagerTest.java b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/SampleManagerTest.java index 69cd5c93cdd..19907398d76 100644 --- a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/SampleManagerTest.java +++ b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/SampleManagerTest.java @@ -31,6 +31,7 @@ import org.opencb.commons.datastore.core.ObjectMap; import org.opencb.commons.datastore.core.Query; import org.opencb.commons.datastore.core.QueryOptions; +import org.opencb.opencga.TestParamConstants; import org.opencb.opencga.catalog.auth.authorization.AuthorizationManager; import org.opencb.opencga.catalog.db.api.DBIterator; import org.opencb.opencga.catalog.db.api.IndividualDBAdaptor; @@ -2363,12 +2364,12 @@ public void updateIndividualFromSample() throws CatalogException { @Test public void getSharedProject() throws CatalogException, IOException { - catalogManager.getUserManager().create("dummy", "dummy", "asd@asd.asd", "dummy", "", 50000L, + catalogManager.getUserManager().create("dummy", "dummy", "asd@asd.asd", TestParamConstants.PASSWORD, "", 50000L, Account.AccountType.GUEST, null); catalogManager.getStudyManager().updateGroup(studyFqn, "@members", ParamUtils.BasicUpdateAction.ADD, new GroupUpdateParams(Collections.singletonList("dummy")), token); - String token = catalogManager.getUserManager().login("dummy", "dummy").getToken(); + String token = catalogManager.getUserManager().login("dummy", TestParamConstants.PASSWORD).getToken(); DataResult queryResult = catalogManager.getProjectManager().getSharedProjects("dummy", QueryOptions.empty(), token); assertEquals(1, queryResult.getNumResults()); @@ -2388,8 +2389,8 @@ public void smartResolutorStudyAliasFromAnonymousUser() throws CatalogException @Test public void checkRegisteredUserPermissions() throws CatalogException { - catalogManager.getUserManager().create("dummy", "dummy", "asd@asd.asd", "dummy", "", 50000L, Account.AccountType.GUEST, null); - String token = catalogManager.getUserManager().login("dummy", "dummy").getToken(); + catalogManager.getUserManager().create("dummy", "dummy", "asd@asd.asd", TestParamConstants.PASSWORD, "", 50000L, Account.AccountType.GUEST, null); + String token = catalogManager.getUserManager().login("dummy", TestParamConstants.PASSWORD).getToken(); catalogManager.getStudyManager().updateGroup(studyFqn, "@members", ParamUtils.BasicUpdateAction.ADD, new GroupUpdateParams(Collections.singletonList(ParamConstants.REGISTERED_USERS)), this.token); @@ -2401,8 +2402,8 @@ public void checkRegisteredUserPermissions() throws CatalogException { @Test public void checkRegisteredUserPermissions2() throws CatalogException { - catalogManager.getUserManager().create("dummy", "dummy", "asd@asd.asd", "dummy", "", 50000L, Account.AccountType.GUEST, null); - String token = catalogManager.getUserManager().login("dummy", "dummy").getToken(); + catalogManager.getUserManager().create("dummy", "dummy", "asd@asd.asd", TestParamConstants.PASSWORD, "", 50000L, Account.AccountType.GUEST, null); + String token = catalogManager.getUserManager().login("dummy", TestParamConstants.PASSWORD).getToken(); catalogManager.getStudyManager().updateGroup(studyFqn, "@members", ParamUtils.BasicUpdateAction.ADD, new GroupUpdateParams(Collections.singletonList(ParamConstants.REGISTERED_USERS)), this.token); @@ -2449,8 +2450,8 @@ public void checkAnonymousUserPermissions2() throws CatalogException { assertTrue(catalogManager.getSampleManager().search(studyFqn, new Query(), new QueryOptions(), "").getNumResults() > 0); assertTrue(catalogManager.getFileManager().search(studyFqn, new Query(), new QueryOptions(), "").getNumResults() > 0); - catalogManager.getUserManager().create("dummy", "dummy", "asd@asd.asd", "dummy", "", 50000L, Account.AccountType.GUEST, null); - String token = catalogManager.getUserManager().login("dummy", "dummy").getToken(); + catalogManager.getUserManager().create("dummy", "dummy", "asd@asd.asd", TestParamConstants.PASSWORD, "", 50000L, Account.AccountType.GUEST, null); + String token = catalogManager.getUserManager().login("dummy", TestParamConstants.PASSWORD).getToken(); studyResult = catalogManager.getStudyManager().get(studyFqn, QueryOptions.empty(), token); assertEquals(1, studyResult.getNumResults()); assertTrue(catalogManager.getSampleManager().search(studyFqn, new Query(), new QueryOptions(), token).getNumResults() > 0); diff --git a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/StudyManagerTest.java b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/StudyManagerTest.java index b4392bc4b7d..2f06b8da687 100644 --- a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/StudyManagerTest.java +++ b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/managers/StudyManagerTest.java @@ -22,6 +22,7 @@ import org.opencb.commons.datastore.core.ObjectMap; import org.opencb.commons.datastore.core.Query; import org.opencb.commons.datastore.core.QueryOptions; +import org.opencb.opencga.TestParamConstants; import org.opencb.opencga.catalog.exceptions.CatalogAuthorizationException; import org.opencb.opencga.catalog.exceptions.CatalogException; import org.opencb.opencga.catalog.utils.AvroToAnnotationConverter; @@ -260,11 +261,11 @@ public void emptyGroupTest() throws CatalogException { Group first = catalogManager.getStudyManager().getGroup(studyFqn, "@test", token).first(); assertNotNull(first.getUserIds()); - catalogManager.getUserManager().create("dummy", "dummy", "dummy@mail.com", "dummy", "", 0L, Account.AccountType.GUEST, token); + catalogManager.getUserManager().create("dummy", "dummy", "dummy@mail.com", TestParamConstants.PASSWORD, "", 0L, Account.AccountType.GUEST, token); catalogManager.getStudyManager().createGroup(studyFqn, "@test2", Collections.singletonList("dummy"), token); catalogManager.getStudyManager().updateAcl(studyFqn, "@test2", new StudyAclParams("", "view_only"), ParamUtils.AclAction.ADD, token); - String dummyToken = catalogManager.getUserManager().login("dummy", "dummy").getToken(); + String dummyToken = catalogManager.getUserManager().login("dummy", TestParamConstants.PASSWORD).getToken(); OpenCGAResult search = catalogManager.getFileManager().search(studyFqn, new Query(), new QueryOptions(), dummyToken); assertTrue(search.getNumResults() > 0); } diff --git a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/migration/MigrationManagerTest.java b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/migration/MigrationManagerTest.java index a3783bfe480..343799ed41e 100644 --- a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/migration/MigrationManagerTest.java +++ b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/migration/MigrationManagerTest.java @@ -7,6 +7,7 @@ import org.opencb.commons.datastore.core.Query; import org.opencb.commons.datastore.core.QueryOptions; import org.opencb.commons.datastore.mongodb.MongoDBCollection; +import org.opencb.opencga.TestParamConstants; import org.opencb.opencga.catalog.db.mongodb.MongoDBAdaptorFactory; import org.opencb.opencga.catalog.exceptions.CatalogException; import org.opencb.opencga.catalog.managers.AbstractManagerTest; @@ -130,7 +131,7 @@ public void setUp() throws java.io.IOException, CatalogException { @Test public void testMigration() throws Exception { MigrationManager migrationManager = catalogManager.getMigrationManager(); - String token = catalogManager.getUserManager().loginAsAdmin("admin").getToken(); + String token = catalogManager.getUserManager().loginAsAdmin(TestParamConstants.ADMIN_PASSWORD).getToken(); List> pendingMigrations = migrationManager.getPendingMigrations("0.0.1", token); assertEquals(0, pendingMigrations.size()); @@ -212,7 +213,7 @@ public void testMigration() throws Exception { @Test public void testManualMigrations() throws CatalogException, IOException { - String token = catalogManager.getUserManager().loginAsAdmin("admin").getToken(); + String token = catalogManager.getUserManager().loginAsAdmin(TestParamConstants.ADMIN_PASSWORD).getToken(); MigrationRun migrationRun = catalogManager.getMigrationManager().runManualMigration("0.2.1", "test4-1-manual", Paths.get(""), new ObjectMap("key", "OtherValue"), token); assertEquals(MigrationRun.MigrationStatus.ERROR, migrationRun.getStatus()); @@ -223,7 +224,7 @@ public void testManualMigrations() throws CatalogException, IOException { @Test public void testMigrationsWithJobs() throws CatalogException, IOException { - String token = catalogManager.getUserManager().loginAsAdmin("admin").getToken(); + String token = catalogManager.getUserManager().loginAsAdmin(TestParamConstants.ADMIN_PASSWORD).getToken(); catalogManager.getMigrationManager().runManualMigration("0.2.1", "test4-1-manual", Paths.get(""), new ObjectMap("key", "value"), token); diff --git a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/stats/solr/AbstractSolrManagerTest.java b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/stats/solr/AbstractSolrManagerTest.java index 47ff81e28d7..6b24a37dc7b 100644 --- a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/stats/solr/AbstractSolrManagerTest.java +++ b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/stats/solr/AbstractSolrManagerTest.java @@ -22,6 +22,7 @@ import org.junit.rules.ExpectedException; import org.opencb.commons.datastore.core.QueryOptions; import org.opencb.commons.test.GenericTest; +import org.opencb.opencga.TestParamConstants; import org.opencb.opencga.catalog.db.api.DBAdaptor; import org.opencb.opencga.catalog.exceptions.CatalogException; import org.opencb.opencga.catalog.managers.CatalogManager; @@ -43,8 +44,6 @@ public class AbstractSolrManagerTest extends GenericTest { - public final static String PASSWORD = "password"; - @Rule public ExpectedException thrown = ExpectedException.none(); @@ -82,17 +81,17 @@ public void after() throws CatalogException { public void setUpCatalogManager(CatalogManager catalogManager) throws IOException, CatalogException { - catalogManager.getUserManager().create("owner", "Owner", "owner@mail.com", PASSWORD, "", null, Account.AccountType.FULL, null); - catalogManager.getUserManager().create("admin1", "Admin", "admin@mail.com", PASSWORD, "", null, Account.AccountType.GUEST, null); - catalogManager.getUserManager().create("user1", "User Name", "mail@ebi.ac.uk", PASSWORD, "", null, Account.AccountType.GUEST, null); - catalogManager.getUserManager().create("user2", "User2 Name", "mail2@ebi.ac.uk", PASSWORD, "", null, Account.AccountType.GUEST, null); - catalogManager.getUserManager().create("user3", "User3 Name", "user.2@e.mail", PASSWORD, "ACME", null, Account.AccountType.GUEST, null); - - sessionIdOwner = catalogManager.getUserManager().login("owner", PASSWORD).getToken(); - sessionIdAdmin = catalogManager.getUserManager().login("admin1", PASSWORD).getToken(); - sessionIdUser = catalogManager.getUserManager().login("user1", PASSWORD).getToken(); - sessionIdUser2 = catalogManager.getUserManager().login("user2", PASSWORD).getToken(); - sessionIdUser3 = catalogManager.getUserManager().login("user3", PASSWORD).getToken(); + catalogManager.getUserManager().create("owner", "Owner", "owner@mail.com", TestParamConstants.PASSWORD, "", null, Account.AccountType.FULL, null); + catalogManager.getUserManager().create("admin1", "Admin", "admin@mail.com", TestParamConstants.PASSWORD, "", null, Account.AccountType.GUEST, null); + catalogManager.getUserManager().create("user1", "User Name", "mail@ebi.ac.uk", TestParamConstants.PASSWORD, "", null, Account.AccountType.GUEST, null); + catalogManager.getUserManager().create("user2", "User2 Name", "mail2@ebi.ac.uk", TestParamConstants.PASSWORD, "", null, Account.AccountType.GUEST, null); + catalogManager.getUserManager().create("user3", "User3 Name", "user.2@e.mail", TestParamConstants.PASSWORD, "ACME", null, Account.AccountType.GUEST, null); + + sessionIdOwner = catalogManager.getUserManager().login("owner", TestParamConstants.PASSWORD).getToken(); + sessionIdAdmin = catalogManager.getUserManager().login("admin1", TestParamConstants.PASSWORD).getToken(); + sessionIdUser = catalogManager.getUserManager().login("user1", TestParamConstants.PASSWORD).getToken(); + sessionIdUser2 = catalogManager.getUserManager().login("user2", TestParamConstants.PASSWORD).getToken(); + sessionIdUser3 = catalogManager.getUserManager().login("user3", TestParamConstants.PASSWORD).getToken(); Project project = catalogManager.getProjectManager().create("1000G", "Project about some genomes", "", "Homo sapiens", null, "GRCh38", INCLUDE_RESULT, sessionIdOwner).first(); diff --git a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/stats/solr/CatalogSolrManagerTest.java b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/stats/solr/CatalogSolrManagerTest.java index c85fa901e14..bc5d7d6076b 100644 --- a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/stats/solr/CatalogSolrManagerTest.java +++ b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/stats/solr/CatalogSolrManagerTest.java @@ -23,6 +23,7 @@ import org.junit.Before; import org.junit.Test; import org.opencb.commons.datastore.core.*; +import org.opencb.opencga.TestParamConstants; import org.opencb.opencga.catalog.db.api.*; import org.opencb.opencga.catalog.db.mongodb.CohortMongoDBAdaptor; import org.opencb.opencga.catalog.db.mongodb.FileMongoDBAdaptor; @@ -202,7 +203,7 @@ public void testCanViewIfAnonymousHavePermissions() throws CatalogException { CatalogSolrManager.SAMPLE_SOLR_COLLECTION); // We create a new user - catalogManager.getUserManager().create("user4", "User4 Name", "user.2@e.mail", PASSWORD, "ACME", null, Account.AccountType.GUEST, null); + catalogManager.getUserManager().create("user4", "User4 Name", "user.2@e.mail", TestParamConstants.PASSWORD, "ACME", null, Account.AccountType.GUEST, null); // We query facets with the new user. That user should be able to see the information because anonymous can see it as well. DataResult facet = catalogSolrManager.facetedQuery(study, CatalogSolrManager.SAMPLE_SOLR_COLLECTION, diff --git a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/utils/CatalogFileUtilsTest.java b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/utils/CatalogFileUtilsTest.java index 58433aea206..f998cb46c7a 100644 --- a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/utils/CatalogFileUtilsTest.java +++ b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/utils/CatalogFileUtilsTest.java @@ -24,6 +24,7 @@ import org.opencb.commons.datastore.core.DataStoreServerAddress; import org.opencb.commons.datastore.core.QueryOptions; import org.opencb.commons.datastore.mongodb.MongoDBConfiguration; +import org.opencb.opencga.TestParamConstants; import org.opencb.opencga.catalog.exceptions.CatalogException; import org.opencb.opencga.catalog.managers.CatalogManager; import org.opencb.opencga.catalog.managers.CatalogManagerExternalResource; @@ -77,11 +78,11 @@ public void before() throws CatalogException, IOException, URISyntaxException { CatalogManagerExternalResource.clearCatalog(configuration); catalogManager = new CatalogManager(configuration); - catalogManager.installCatalogDB("dummy", "admin", "opencga@admin.com", "", true); + catalogManager.installCatalogDB("dummy", TestParamConstants.ADMIN_PASSWORD, "opencga@admin.com", "", true); //Create USER - catalogManager.getUserManager().create("user", "name", "mi@mail.com", "asdf", "", null, Account.AccountType.FULL, null); - userSessionId = catalogManager.getUserManager().login("user", "asdf").getToken(); + catalogManager.getUserManager().create("user", "name", "mi@mail.com", TestParamConstants.PASSWORD, "", null, Account.AccountType.FULL, null); + userSessionId = catalogManager.getUserManager().login("user", TestParamConstants.PASSWORD).getToken(); // adminSessionId = catalogManager.login("admin", "admin", "--").getResults().get(0).getString("sessionId"); String projectId = catalogManager.getProjectManager().create("proj", "proj", "", "Homo sapiens", null, "GRCh38", INCLUDE_RESULT, userSessionId).getResults().get(0).getId(); diff --git a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/utils/CatalogSampleAnnotationsLoaderTest.java b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/utils/CatalogSampleAnnotationsLoaderTest.java index b8017a05a2c..0af8e2ef273 100644 --- a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/utils/CatalogSampleAnnotationsLoaderTest.java +++ b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/utils/CatalogSampleAnnotationsLoaderTest.java @@ -26,6 +26,7 @@ import org.opencb.commons.datastore.core.Query; import org.opencb.commons.datastore.core.QueryOptions; import org.opencb.commons.test.GenericTest; +import org.opencb.opencga.TestParamConstants; import org.opencb.opencga.catalog.exceptions.CatalogException; import org.opencb.opencga.catalog.managers.CatalogManager; import org.opencb.opencga.core.config.Configuration; @@ -63,10 +64,10 @@ public static void beforeClass() throws IOException, CatalogException, URISyntax configuration.getAdmin().setAlgorithm("HS256"); catalogManager = new CatalogManager(configuration); if (catalogManager.existsCatalogDB()) { - String token = catalogManager.getUserManager().loginAsAdmin("admin").getToken(); + String token = catalogManager.getUserManager().loginAsAdmin(TestParamConstants.ADMIN_PASSWORD).getToken(); catalogManager.deleteCatalogDB(token); } - catalogManager.installCatalogDB("dummy", "admin", "opencga@admin.com", "", true); + catalogManager.installCatalogDB("dummy", TestParamConstants.ADMIN_PASSWORD, "opencga@admin.com", "", true); loader = new CatalogSampleAnnotationsLoader(catalogManager); String pedFileName = "20130606_g1k.ped"; @@ -74,8 +75,8 @@ public static void beforeClass() throws IOException, CatalogException, URISyntax pedigree = loader.readPedigree(pedFileURL.getPath()); userId = "user1"; - catalogManager.getUserManager().create(userId, userId, "asdasd@asd.asd", userId, "", -1L, Account.AccountType.FULL, null); - sessionId = catalogManager.getUserManager().login(userId, userId).getToken(); + catalogManager.getUserManager().create(userId, userId, "asdasd@asd.asd", TestParamConstants.PASSWORD, "", -1L, Account.AccountType.FULL, null); + sessionId = catalogManager.getUserManager().login(userId, TestParamConstants.PASSWORD).getToken(); Project project = catalogManager.getProjectManager().create("def", "default", "", "Homo sapiens", null, "GRCh38", new QueryOptions(), sessionId).getResults().get(0); Study study = catalogManager.getStudyManager().create(project.getFqn(), "def", null, "default", "", null, null, null, null, null, sessionId).getResults().get(0); diff --git a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/utils/FileMetadataReaderTest.java b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/utils/FileMetadataReaderTest.java index 42cc7c40a53..4f971e3030c 100644 --- a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/utils/FileMetadataReaderTest.java +++ b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/utils/FileMetadataReaderTest.java @@ -21,6 +21,7 @@ import org.junit.Rule; import org.junit.Test; import org.opencb.commons.datastore.core.QueryOptions; +import org.opencb.opencga.TestParamConstants; import org.opencb.opencga.catalog.exceptions.CatalogException; import org.opencb.opencga.catalog.managers.CatalogManager; import org.opencb.opencga.catalog.managers.CatalogManagerExternalResource; @@ -54,7 +55,6 @@ public class FileMetadataReaderTest { @Rule public CatalogManagerExternalResource catalogManagerExternalResource = new CatalogManagerExternalResource(); - public static final String PASSWORD = "asdf"; private CatalogManager catalogManager; private String sessionIdUser; private Project project; @@ -72,8 +72,8 @@ public class FileMetadataReaderTest { public void setUp() throws IOException, CatalogException, URISyntaxException { catalogManager = catalogManagerExternalResource.getCatalogManager(); - catalogManager.getUserManager().create("user", "User Name", "mail@ebi.ac.uk", PASSWORD, "", null, Account.AccountType.FULL, null); - sessionIdUser = catalogManager.getUserManager().login("user", PASSWORD).getToken(); + catalogManager.getUserManager().create("user", "User Name", "mail@ebi.ac.uk", TestParamConstants.PASSWORD, "", null, Account.AccountType.FULL, null); + sessionIdUser = catalogManager.getUserManager().login("user", TestParamConstants.PASSWORD).getToken(); project = catalogManager.getProjectManager().create("1000G", "Project about some genomes", "", "Homo sapiens", null, "GRCh38", INCLUDE_RESULT, sessionIdUser).first(); study = catalogManager.getStudyManager().create(project.getId(), "phase1", null, "Phase 1", "Done", null, null, null, null, INCLUDE_RESULT, sessionIdUser).first(); diff --git a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/utils/FileScannerTest.java b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/utils/FileScannerTest.java index 34c31648a24..ec6b2e1cb64 100644 --- a/opencga-catalog/src/test/java/org/opencb/opencga/catalog/utils/FileScannerTest.java +++ b/opencga-catalog/src/test/java/org/opencb/opencga/catalog/utils/FileScannerTest.java @@ -23,6 +23,7 @@ import org.opencb.commons.datastore.core.DataResult; import org.opencb.commons.datastore.core.Query; import org.opencb.commons.datastore.core.QueryOptions; +import org.opencb.opencga.TestParamConstants; import org.opencb.opencga.catalog.db.api.FileDBAdaptor; import org.opencb.opencga.catalog.exceptions.CatalogException; import org.opencb.opencga.catalog.io.IOManager; @@ -58,7 +59,6 @@ public class FileScannerTest { @Rule public CatalogManagerExternalResource catalogManagerExternalResource = new CatalogManagerExternalResource(); - public static final String PASSWORD = "asdf"; private CatalogManager catalogManager; private String sessionIdUser; private File folder; @@ -72,8 +72,8 @@ public class FileScannerTest { public void setUp() throws IOException, CatalogException { catalogManager = catalogManagerExternalResource.getCatalogManager(); - catalogManager.getUserManager().create("user", "User Name", "mail@ebi.ac.uk", PASSWORD, "", null, Account.AccountType.FULL, null); - sessionIdUser = catalogManager.getUserManager().login("user", PASSWORD).getToken(); + catalogManager.getUserManager().create("user", "User Name", "mail@ebi.ac.uk", TestParamConstants.PASSWORD, "", null, Account.AccountType.FULL, null); + sessionIdUser = catalogManager.getUserManager().login("user", TestParamConstants.PASSWORD).getToken(); project = catalogManager.getProjectManager().create("1000G", "Project about some genomes", "", "Homo sapiens", null, "GRCh38", INCLUDE_RESULT, sessionIdUser).first(); study = catalogManager.getStudyManager().create(project.getId(), "phase1", null, "Phase 1", "Done", null, null, null, null, INCLUDE_RESULT, sessionIdUser).first(); diff --git a/opencga-catalog/src/test/java/org/opencb/opencga/templates/TemplateManagerTest.java b/opencga-catalog/src/test/java/org/opencb/opencga/templates/TemplateManagerTest.java index 865e6715cd4..c212acaf1b3 100644 --- a/opencga-catalog/src/test/java/org/opencb/opencga/templates/TemplateManagerTest.java +++ b/opencga-catalog/src/test/java/org/opencb/opencga/templates/TemplateManagerTest.java @@ -7,6 +7,7 @@ import org.junit.Rule; import org.junit.Test; import org.opencb.commons.datastore.core.QueryOptions; +import org.opencb.opencga.TestParamConstants; import org.opencb.opencga.catalog.managers.CatalogManager; import org.opencb.opencga.catalog.managers.CatalogManagerExternalResource; import org.opencb.opencga.catalog.templates.TemplateManager; @@ -36,15 +37,15 @@ public void after() { @Test public void test() throws Exception { CatalogManager catalogManager = catalogManagerResource.getCatalogManager(); - String adminToken = catalogManager.getUserManager().loginAsAdmin("admin").getToken(); + String adminToken = catalogManager.getUserManager().loginAsAdmin(TestParamConstants.ADMIN_PASSWORD).getToken(); catalogManager.getUserManager().create(new User().setId("user1").setName("User 1").setAccount(new Account().setType(Account.AccountType.FULL)), - "password", adminToken); - catalogManager.getUserManager().create(new User().setId("user2").setName("User 2").setAccount(new Account().setType(Account.AccountType.GUEST)), "password", adminToken); - catalogManager.getUserManager().create(new User().setId("user3").setName("User 3").setAccount(new Account().setType(Account.AccountType.GUEST)), "password", adminToken); - catalogManager.getUserManager().create(new User().setId("user4").setName("User 4").setAccount(new Account().setType(Account.AccountType.GUEST)), "password", adminToken); + TestParamConstants.PASSWORD, adminToken); + catalogManager.getUserManager().create(new User().setId("user2").setName("User 2").setAccount(new Account().setType(Account.AccountType.GUEST)), TestParamConstants.PASSWORD, adminToken); + catalogManager.getUserManager().create(new User().setId("user3").setName("User 3").setAccount(new Account().setType(Account.AccountType.GUEST)), TestParamConstants.PASSWORD, adminToken); + catalogManager.getUserManager().create(new User().setId("user4").setName("User 4").setAccount(new Account().setType(Account.AccountType.GUEST)), TestParamConstants.PASSWORD, adminToken); - String token = catalogManager.getUserManager().login("user1", "password").getToken(); + String token = catalogManager.getUserManager().login("user1", TestParamConstants.PASSWORD).getToken(); catalogManager.getProjectManager().create("project", "Project", "", "name", "common", "GRCh38", QueryOptions.empty(), token); catalogManager.getStudyManager().create("project", new Study().setId("study"), QueryOptions.empty(), token); @@ -63,15 +64,15 @@ public void test() throws Exception { @Test public void test_yaml() throws Exception { CatalogManager catalogManager = catalogManagerResource.getCatalogManager(); - String adminToken = catalogManager.getUserManager().loginAsAdmin("admin").getToken(); + String adminToken = catalogManager.getUserManager().loginAsAdmin(TestParamConstants.ADMIN_PASSWORD).getToken(); catalogManager.getUserManager().create(new User().setId("user1").setName("User 1").setAccount(new Account().setType(Account.AccountType.FULL)), - "password", adminToken); - catalogManager.getUserManager().create(new User().setId("user2").setName("User 2").setAccount(new Account().setType(Account.AccountType.GUEST)), "password", adminToken); - catalogManager.getUserManager().create(new User().setId("user3").setName("User 3").setAccount(new Account().setType(Account.AccountType.GUEST)), "password", adminToken); - catalogManager.getUserManager().create(new User().setId("user4").setName("User 4").setAccount(new Account().setType(Account.AccountType.GUEST)), "password", adminToken); + TestParamConstants.PASSWORD, adminToken); + catalogManager.getUserManager().create(new User().setId("user2").setName("User 2").setAccount(new Account().setType(Account.AccountType.GUEST)), TestParamConstants.PASSWORD, adminToken); + catalogManager.getUserManager().create(new User().setId("user3").setName("User 3").setAccount(new Account().setType(Account.AccountType.GUEST)), TestParamConstants.PASSWORD, adminToken); + catalogManager.getUserManager().create(new User().setId("user4").setName("User 4").setAccount(new Account().setType(Account.AccountType.GUEST)), TestParamConstants.PASSWORD, adminToken); - String token = catalogManager.getUserManager().login("user1", "password").getToken(); + String token = catalogManager.getUserManager().login("user1", TestParamConstants.PASSWORD).getToken(); catalogManager.getProjectManager().create("project", "Project", "", "name", "common", "GRCh38", QueryOptions.empty(), token); catalogManager.getStudyManager().create("project", new Study().setId("study"), QueryOptions.empty(), token); diff --git a/opencga-client/pom.xml b/opencga-client/pom.xml index 8712d69125a..b75261299a7 100644 --- a/opencga-client/pom.xml +++ b/opencga-client/pom.xml @@ -22,7 +22,7 @@ org.opencb.opencga opencga - 2.2.0-rc2 + 2.2.0 ../pom.xml diff --git a/opencga-client/src/main/R/R/Admin-methods.R b/opencga-client/src/main/R/R/Admin-methods.R index 8191f45dbbf..ee2f1226a21 100644 --- a/opencga-client/src/main/R/R/Admin-methods.R +++ b/opencga-client/src/main/R/R/Admin-methods.R @@ -2,7 +2,7 @@ # WARNING: AUTOGENERATED CODE # # This code was generated by a tool. -# Autogenerated on: 2022-03-15 19:08:35 +# Autogenerated on: 2022-03-25 00:37:46 # # Manual changes to this file may cause unexpected behavior in your application. # Manual changes to this file will be overwritten if the code is regenerated. diff --git a/opencga-client/src/main/R/R/Alignment-methods.R b/opencga-client/src/main/R/R/Alignment-methods.R index 4846cc8f6cc..ab71db63db8 100644 --- a/opencga-client/src/main/R/R/Alignment-methods.R +++ b/opencga-client/src/main/R/R/Alignment-methods.R @@ -2,7 +2,7 @@ # WARNING: AUTOGENERATED CODE # # This code was generated by a tool. -# Autogenerated on: 2022-03-15 19:08:35 +# Autogenerated on: 2022-03-25 00:37:46 # # Manual changes to this file may cause unexpected behavior in your application. # Manual changes to this file will be overwritten if the code is regenerated. @@ -22,7 +22,7 @@ #' | -- | :-- | --: | #' | runBwa | /{apiVersion}/analysis/alignment/bwa/run | study, jobId, jobDependsOn, jobDescription, jobTags, body[*] | #' | runCoverageIndex | /{apiVersion}/analysis/alignment/coverage/index/run | study, jobId, jobDependsOn, jobDescription, jobTags, body[*] | -#' | runQcGeneCoverageStats | /{apiVersion}/analysis/alignment/coverage/qc/geneCoverageStats/run | study, jobId, jobDependsOn, jobDescription, jobTags, body[*] | +#' | coverageQcGeneCoverageStatsRun | /{apiVersion}/analysis/alignment/coverage/qc/geneCoverageStats/run | study, jobId, jobDependsOn, jobDescription, jobTags, body[*] | #' | queryCoverage | /{apiVersion}/analysis/alignment/coverage/query | file[*], study, region, gene, offset, onlyExons, range, windowSize, splitResults | #' | ratioCoverage | /{apiVersion}/analysis/alignment/coverage/ratio | file1[*], file2[*], study, skipLog2, region, gene, offset, onlyExons, windowSize, splitResults | #' | statsCoverage | /{apiVersion}/analysis/alignment/coverage/stats | file[*], gene[*], study, threshold | @@ -74,7 +74,7 @@ setMethod("alignmentClient", "OpencgaR", function(OpencgaR, endpointName, params #' @param jobDescription Job description. #' @param jobTags Job tags. #' @param data Gene coverage stats parameters for a given BAM file and a list of genes. - runQcGeneCoverageStats=fetchOpenCGA(object=OpencgaR, category="analysis", categoryId=NULL, + coverageQcGeneCoverageStatsRun=fetchOpenCGA(object=OpencgaR, category="analysis", categoryId=NULL, subcategory="alignment/coverage/qc/geneCoverageStats", subcategoryId=NULL, action="run", params=params, httpMethod="POST", as.queryParam=NULL, ...), diff --git a/opencga-client/src/main/R/R/AllGenerics.R b/opencga-client/src/main/R/R/AllGenerics.R index fa3966ba04b..08a081161c7 100644 --- a/opencga-client/src/main/R/R/AllGenerics.R +++ b/opencga-client/src/main/R/R/AllGenerics.R @@ -1,6 +1,6 @@ # ############################################################################## ## UserClient -setGeneric("userClient", function(OpencgaR, filterId, user, users, endpointName, params=NULL, ...) +setGeneric("userClient", function(OpencgaR, users, user, filterId, endpointName, params=NULL, ...) standardGeneric("userClient")) # ############################################################################## @@ -10,12 +10,12 @@ setGeneric("projectClient", function(OpencgaR, projects, project, endpointName, # ############################################################################## ## StudyClient -setGeneric("studyClient", function(OpencgaR, group, study, templateId, members, studies, variableSet, endpointName, params=NULL, ...) +setGeneric("studyClient", function(OpencgaR, study, templateId, studies, group, variableSet, members, endpointName, params=NULL, ...) standardGeneric("studyClient")) # ############################################################################## ## FileClient -setGeneric("fileClient", function(OpencgaR, files, file, members, folder, annotationSet, endpointName, params=NULL, ...) +setGeneric("fileClient", function(OpencgaR, files, annotationSet, file, folder, members, endpointName, params=NULL, ...) standardGeneric("fileClient")) # ############################################################################## @@ -25,27 +25,27 @@ setGeneric("jobClient", function(OpencgaR, jobs, job, members, endpointName, par # ############################################################################## ## SampleClient -setGeneric("sampleClient", function(OpencgaR, samples, sample, members, annotationSet, endpointName, params=NULL, ...) +setGeneric("sampleClient", function(OpencgaR, sample, samples, annotationSet, members, endpointName, params=NULL, ...) standardGeneric("sampleClient")) # ############################################################################## ## IndividualClient -setGeneric("individualClient", function(OpencgaR, individual, individuals, members, annotationSet, endpointName, params=NULL, ...) +setGeneric("individualClient", function(OpencgaR, individuals, individual, annotationSet, members, endpointName, params=NULL, ...) standardGeneric("individualClient")) # ############################################################################## ## FamilyClient -setGeneric("familyClient", function(OpencgaR, annotationSet, members, families, family, endpointName, params=NULL, ...) +setGeneric("familyClient", function(OpencgaR, family, annotationSet, families, members, endpointName, params=NULL, ...) standardGeneric("familyClient")) # ############################################################################## ## CohortClient -setGeneric("cohortClient", function(OpencgaR, cohort, cohorts, members, annotationSet, endpointName, params=NULL, ...) +setGeneric("cohortClient", function(OpencgaR, cohorts, cohort, annotationSet, members, endpointName, params=NULL, ...) standardGeneric("cohortClient")) # ############################################################################## ## PanelClient -setGeneric("panelClient", function(OpencgaR, members, panels, endpointName, params=NULL, ...) +setGeneric("panelClient", function(OpencgaR, panels, members, endpointName, params=NULL, ...) standardGeneric("panelClient")) # ############################################################################## @@ -60,7 +60,7 @@ setGeneric("variantClient", function(OpencgaR, endpointName, params=NULL, ...) # ############################################################################## ## ClinicalClient -setGeneric("clinicalClient", function(OpencgaR, clinicalAnalyses, interpretations, members, interpretation, clinicalAnalysis, endpointName, params=NULL, ...) +setGeneric("clinicalClient", function(OpencgaR, interpretations, clinicalAnalysis, clinicalAnalyses, interpretation, members, endpointName, params=NULL, ...) standardGeneric("clinicalClient")) # ############################################################################## @@ -75,7 +75,7 @@ setGeneric("metaClient", function(OpencgaR, endpointName, params=NULL, ...) # ############################################################################## ## GA4GHClient -setGeneric("ga4ghClient", function(OpencgaR, file, study, endpointName, params=NULL, ...) +setGeneric("ga4ghClient", function(OpencgaR, study, file, endpointName, params=NULL, ...) standardGeneric("ga4ghClient")) # ############################################################################## diff --git a/opencga-client/src/main/R/R/Clinical-methods.R b/opencga-client/src/main/R/R/Clinical-methods.R index abb031d60db..be30d075985 100644 --- a/opencga-client/src/main/R/R/Clinical-methods.R +++ b/opencga-client/src/main/R/R/Clinical-methods.R @@ -2,7 +2,7 @@ # WARNING: AUTOGENERATED CODE # # This code was generated by a tool. -# Autogenerated on: 2022-03-15 19:08:35 +# Autogenerated on: 2022-03-25 00:37:46 # # Manual changes to this file may cause unexpected behavior in your application. # Manual changes to this file will be overwritten if the code is regenerated. @@ -58,7 +58,7 @@ #' [*]: Required parameter #' @export -setMethod("clinicalClient", "OpencgaR", function(OpencgaR, clinicalAnalyses, interpretations, members, interpretation, clinicalAnalysis, endpointName, params=NULL, ...) { +setMethod("clinicalClient", "OpencgaR", function(OpencgaR, interpretations, clinicalAnalysis, clinicalAnalyses, interpretation, members, endpointName, params=NULL, ...) { switch(endpointName, #' @section Endpoint /{apiVersion}/analysis/clinical/acl/{members}/update: diff --git a/opencga-client/src/main/R/R/Cohort-methods.R b/opencga-client/src/main/R/R/Cohort-methods.R index dec6bd4cfdf..f82962e1307 100644 --- a/opencga-client/src/main/R/R/Cohort-methods.R +++ b/opencga-client/src/main/R/R/Cohort-methods.R @@ -2,7 +2,7 @@ # WARNING: AUTOGENERATED CODE # # This code was generated by a tool. -# Autogenerated on: 2022-03-15 19:08:35 +# Autogenerated on: 2022-03-25 00:37:46 # # Manual changes to this file may cause unexpected behavior in your application. # Manual changes to this file will be overwritten if the code is regenerated. @@ -39,7 +39,7 @@ #' [*]: Required parameter #' @export -setMethod("cohortClient", "OpencgaR", function(OpencgaR, cohort, cohorts, members, annotationSet, endpointName, params=NULL, ...) { +setMethod("cohortClient", "OpencgaR", function(OpencgaR, cohorts, cohort, annotationSet, members, endpointName, params=NULL, ...) { switch(endpointName, #' @section Endpoint /{apiVersion}/cohorts/acl/{members}/update: diff --git a/opencga-client/src/main/R/R/Family-methods.R b/opencga-client/src/main/R/R/Family-methods.R index 974f2f9016f..bcdae772d4d 100644 --- a/opencga-client/src/main/R/R/Family-methods.R +++ b/opencga-client/src/main/R/R/Family-methods.R @@ -2,7 +2,7 @@ # WARNING: AUTOGENERATED CODE # # This code was generated by a tool. -# Autogenerated on: 2022-03-15 19:08:35 +# Autogenerated on: 2022-03-25 00:37:46 # # Manual changes to this file may cause unexpected behavior in your application. # Manual changes to this file will be overwritten if the code is regenerated. @@ -38,7 +38,7 @@ #' [*]: Required parameter #' @export -setMethod("familyClient", "OpencgaR", function(OpencgaR, annotationSet, members, families, family, endpointName, params=NULL, ...) { +setMethod("familyClient", "OpencgaR", function(OpencgaR, family, annotationSet, families, members, endpointName, params=NULL, ...) { switch(endpointName, #' @section Endpoint /{apiVersion}/families/acl/{members}/update: diff --git a/opencga-client/src/main/R/R/File-methods.R b/opencga-client/src/main/R/R/File-methods.R index 829f8bacc06..461a9979b59 100644 --- a/opencga-client/src/main/R/R/File-methods.R +++ b/opencga-client/src/main/R/R/File-methods.R @@ -2,7 +2,7 @@ # WARNING: AUTOGENERATED CODE # # This code was generated by a tool. -# Autogenerated on: 2022-03-15 19:08:35 +# Autogenerated on: 2022-03-25 00:37:46 # # Manual changes to this file may cause unexpected behavior in your application. # Manual changes to this file will be overwritten if the code is regenerated. @@ -54,7 +54,7 @@ #' [*]: Required parameter #' @export -setMethod("fileClient", "OpencgaR", function(OpencgaR, files, file, members, folder, annotationSet, endpointName, params=NULL, ...) { +setMethod("fileClient", "OpencgaR", function(OpencgaR, files, annotationSet, file, folder, members, endpointName, params=NULL, ...) { switch(endpointName, #' @section Endpoint /{apiVersion}/files/acl/{members}/update: diff --git a/opencga-client/src/main/R/R/GA4GH-methods.R b/opencga-client/src/main/R/R/GA4GH-methods.R index 8ba16724eb8..707aa858d08 100644 --- a/opencga-client/src/main/R/R/GA4GH-methods.R +++ b/opencga-client/src/main/R/R/GA4GH-methods.R @@ -2,7 +2,7 @@ # WARNING: AUTOGENERATED CODE # # This code was generated by a tool. -# Autogenerated on: 2022-03-15 19:08:35 +# Autogenerated on: 2022-03-25 00:37:46 # # Manual changes to this file may cause unexpected behavior in your application. # Manual changes to this file will be overwritten if the code is regenerated. @@ -31,7 +31,7 @@ #' [*]: Required parameter #' @export -setMethod("ga4ghClient", "OpencgaR", function(OpencgaR, file, study, endpointName, params=NULL, ...) { +setMethod("ga4ghClient", "OpencgaR", function(OpencgaR, study, file, endpointName, params=NULL, ...) { switch(endpointName, #' @section Endpoint /{apiVersion}/ga4gh/reads/search: diff --git a/opencga-client/src/main/R/R/Individual-methods.R b/opencga-client/src/main/R/R/Individual-methods.R index 59c95197280..7c95364ec24 100644 --- a/opencga-client/src/main/R/R/Individual-methods.R +++ b/opencga-client/src/main/R/R/Individual-methods.R @@ -2,7 +2,7 @@ # WARNING: AUTOGENERATED CODE # # This code was generated by a tool. -# Autogenerated on: 2022-03-15 19:08:35 +# Autogenerated on: 2022-03-25 00:37:46 # # Manual changes to this file may cause unexpected behavior in your application. # Manual changes to this file will be overwritten if the code is regenerated. @@ -39,7 +39,7 @@ #' [*]: Required parameter #' @export -setMethod("individualClient", "OpencgaR", function(OpencgaR, individual, individuals, members, annotationSet, endpointName, params=NULL, ...) { +setMethod("individualClient", "OpencgaR", function(OpencgaR, individuals, individual, annotationSet, members, endpointName, params=NULL, ...) { switch(endpointName, #' @section Endpoint /{apiVersion}/individuals/acl/{members}/update: diff --git a/opencga-client/src/main/R/R/Job-methods.R b/opencga-client/src/main/R/R/Job-methods.R index fb1b80602d7..57db5eefe8a 100644 --- a/opencga-client/src/main/R/R/Job-methods.R +++ b/opencga-client/src/main/R/R/Job-methods.R @@ -2,7 +2,7 @@ # WARNING: AUTOGENERATED CODE # # This code was generated by a tool. -# Autogenerated on: 2022-03-15 19:08:35 +# Autogenerated on: 2022-03-25 00:37:46 # # Manual changes to this file may cause unexpected behavior in your application. # Manual changes to this file will be overwritten if the code is regenerated. diff --git a/opencga-client/src/main/R/R/Meta-methods.R b/opencga-client/src/main/R/R/Meta-methods.R index 48fefa8ea3d..c05d22e0a57 100644 --- a/opencga-client/src/main/R/R/Meta-methods.R +++ b/opencga-client/src/main/R/R/Meta-methods.R @@ -2,7 +2,7 @@ # WARNING: AUTOGENERATED CODE # # This code was generated by a tool. -# Autogenerated on: 2022-03-15 19:08:35 +# Autogenerated on: 2022-03-25 00:37:46 # # Manual changes to this file may cause unexpected behavior in your application. # Manual changes to this file will be overwritten if the code is regenerated. diff --git a/opencga-client/src/main/R/R/Operation-methods.R b/opencga-client/src/main/R/R/Operation-methods.R index a0b11c93ffc..6e55851734b 100644 --- a/opencga-client/src/main/R/R/Operation-methods.R +++ b/opencga-client/src/main/R/R/Operation-methods.R @@ -2,7 +2,7 @@ # WARNING: AUTOGENERATED CODE # # This code was generated by a tool. -# Autogenerated on: 2022-03-15 19:08:35 +# Autogenerated on: 2022-03-25 00:37:46 # # Manual changes to this file may cause unexpected behavior in your application. # Manual changes to this file will be overwritten if the code is regenerated. @@ -36,7 +36,7 @@ #' | synchronizeVariantMetadata | /{apiVersion}/operation/variant/metadata/synchronize | jobId, jobDescription, jobDependsOn, jobTags, study, body | #' | deleteVariantSample | /{apiVersion}/operation/variant/sample/delete | jobId, jobDescription, jobDependsOn, jobTags, study, body | #' | indexVariantSample | /{apiVersion}/operation/variant/sample/index | jobId, jobDescription, jobDependsOn, jobTags, study, body | -#' | configureSampleIndex | /{apiVersion}/operation/variant/sample/index/configure | study, skipRebuild, body | +#' | variantSampleIndexConfigure | /{apiVersion}/operation/variant/sample/index/configure | study, skipRebuild, body | #' | deleteVariantScore | /{apiVersion}/operation/variant/score/delete | jobId, jobDescription, jobDependsOn, jobTags, study, name, resume, force | #' | indexVariantScore | /{apiVersion}/operation/variant/score/index | jobId, jobDescription, jobDependsOn, jobTags, study, body | #' | secondaryIndexVariant | /{apiVersion}/operation/variant/secondaryIndex | jobId, jobDescription, jobDependsOn, jobTags, project, study, body | @@ -241,7 +241,7 @@ setMethod("operationClient", "OpencgaR", function(OpencgaR, endpointName, params #' @param study Study [[user@]project:]study where study and project can be either the ID or UUID. #' @param skipRebuild Skip sample index re-build. #' @param data New SampleIndexConfiguration. - configureSampleIndex=fetchOpenCGA(object=OpencgaR, category="operation", categoryId=NULL, + variantSampleIndexConfigure=fetchOpenCGA(object=OpencgaR, category="operation", categoryId=NULL, subcategory="variant/sample/index", subcategoryId=NULL, action="configure", params=params, httpMethod="POST", as.queryParam=NULL, ...), diff --git a/opencga-client/src/main/R/R/Panel-methods.R b/opencga-client/src/main/R/R/Panel-methods.R index 9103025a206..748d428caa9 100644 --- a/opencga-client/src/main/R/R/Panel-methods.R +++ b/opencga-client/src/main/R/R/Panel-methods.R @@ -2,7 +2,7 @@ # WARNING: AUTOGENERATED CODE # # This code was generated by a tool. -# Autogenerated on: 2022-03-15 19:08:35 +# Autogenerated on: 2022-03-25 00:37:46 # # Manual changes to this file may cause unexpected behavior in your application. # Manual changes to this file will be overwritten if the code is regenerated. @@ -35,7 +35,7 @@ #' [*]: Required parameter #' @export -setMethod("panelClient", "OpencgaR", function(OpencgaR, members, panels, endpointName, params=NULL, ...) { +setMethod("panelClient", "OpencgaR", function(OpencgaR, panels, members, endpointName, params=NULL, ...) { switch(endpointName, #' @section Endpoint /{apiVersion}/panels/acl/{members}/update: diff --git a/opencga-client/src/main/R/R/Project-methods.R b/opencga-client/src/main/R/R/Project-methods.R index beb8e2c1f40..aea037ee183 100644 --- a/opencga-client/src/main/R/R/Project-methods.R +++ b/opencga-client/src/main/R/R/Project-methods.R @@ -2,7 +2,7 @@ # WARNING: AUTOGENERATED CODE # # This code was generated by a tool. -# Autogenerated on: 2022-03-15 19:08:35 +# Autogenerated on: 2022-03-25 00:37:46 # # Manual changes to this file may cause unexpected behavior in your application. # Manual changes to this file will be overwritten if the code is regenerated. diff --git a/opencga-client/src/main/R/R/Sample-methods.R b/opencga-client/src/main/R/R/Sample-methods.R index 9612429690a..04a2a919428 100644 --- a/opencga-client/src/main/R/R/Sample-methods.R +++ b/opencga-client/src/main/R/R/Sample-methods.R @@ -2,7 +2,7 @@ # WARNING: AUTOGENERATED CODE # # This code was generated by a tool. -# Autogenerated on: 2022-03-15 19:08:35 +# Autogenerated on: 2022-03-25 00:37:46 # # Manual changes to this file may cause unexpected behavior in your application. # Manual changes to this file will be overwritten if the code is regenerated. @@ -39,7 +39,7 @@ #' [*]: Required parameter #' @export -setMethod("sampleClient", "OpencgaR", function(OpencgaR, samples, sample, members, annotationSet, endpointName, params=NULL, ...) { +setMethod("sampleClient", "OpencgaR", function(OpencgaR, sample, samples, annotationSet, members, endpointName, params=NULL, ...) { switch(endpointName, #' @section Endpoint /{apiVersion}/samples/acl/{members}/update: diff --git a/opencga-client/src/main/R/R/Study-methods.R b/opencga-client/src/main/R/R/Study-methods.R index 8486bbaea66..4a460d54395 100644 --- a/opencga-client/src/main/R/R/Study-methods.R +++ b/opencga-client/src/main/R/R/Study-methods.R @@ -2,7 +2,7 @@ # WARNING: AUTOGENERATED CODE # # This code was generated by a tool. -# Autogenerated on: 2022-03-15 19:08:35 +# Autogenerated on: 2022-03-25 00:37:46 # # Manual changes to this file may cause unexpected behavior in your application. # Manual changes to this file will be overwritten if the code is regenerated. @@ -46,7 +46,7 @@ #' [*]: Required parameter #' @export -setMethod("studyClient", "OpencgaR", function(OpencgaR, group, study, templateId, members, studies, variableSet, endpointName, params=NULL, ...) { +setMethod("studyClient", "OpencgaR", function(OpencgaR, study, templateId, studies, group, variableSet, members, endpointName, params=NULL, ...) { switch(endpointName, #' @section Endpoint /{apiVersion}/studies/acl/{members}/update: diff --git a/opencga-client/src/main/R/R/User-methods.R b/opencga-client/src/main/R/R/User-methods.R index 6693790fd08..1ff9ff2bf12 100644 --- a/opencga-client/src/main/R/R/User-methods.R +++ b/opencga-client/src/main/R/R/User-methods.R @@ -2,7 +2,7 @@ # WARNING: AUTOGENERATED CODE # # This code was generated by a tool. -# Autogenerated on: 2022-03-15 19:08:35 +# Autogenerated on: 2022-03-25 00:37:46 # # Manual changes to this file may cause unexpected behavior in your application. # Manual changes to this file will be overwritten if the code is regenerated. @@ -39,7 +39,7 @@ #' [*]: Required parameter #' @export -setMethod("userClient", "OpencgaR", function(OpencgaR, filterId, user, users, endpointName, params=NULL, ...) { +setMethod("userClient", "OpencgaR", function(OpencgaR, users, user, filterId, endpointName, params=NULL, ...) { switch(endpointName, #' @section Endpoint /{apiVersion}/users/create: diff --git a/opencga-client/src/main/R/R/Variant-methods.R b/opencga-client/src/main/R/R/Variant-methods.R index f2f769ca48f..c2edb4f0e68 100644 --- a/opencga-client/src/main/R/R/Variant-methods.R +++ b/opencga-client/src/main/R/R/Variant-methods.R @@ -2,7 +2,7 @@ # WARNING: AUTOGENERATED CODE # # This code was generated by a tool. -# Autogenerated on: 2022-03-15 19:08:35 +# Autogenerated on: 2022-03-25 00:37:46 # # Manual changes to this file may cause unexpected behavior in your application. # Manual changes to this file will be overwritten if the code is regenerated. diff --git a/opencga-client/src/main/java/org/opencb/opencga/client/config/ClientConfiguration.java b/opencga-client/src/main/java/org/opencb/opencga/client/config/ClientConfiguration.java index cda69ed3b70..322176c0c8d 100644 --- a/opencga-client/src/main/java/org/opencb/opencga/client/config/ClientConfiguration.java +++ b/opencga-client/src/main/java/org/opencb/opencga/client/config/ClientConfiguration.java @@ -16,6 +16,8 @@ package org.opencb.opencga.client.config; +import com.fasterxml.jackson.core.JsonFactory; +import com.fasterxml.jackson.databind.DeserializationFeature; import com.fasterxml.jackson.databind.ObjectMapper; import com.fasterxml.jackson.dataformat.yaml.YAMLFactory; import org.apache.commons.collections4.CollectionUtils; @@ -57,16 +59,13 @@ public static ClientConfiguration load(InputStream configurationInputStream) thr public static ClientConfiguration load(InputStream configurationInputStream, String format) throws IOException { ClientConfiguration clientConfiguration; - ObjectMapper objectMapper; switch (format.toUpperCase()) { case "JSON": - objectMapper = new ObjectMapper(); - clientConfiguration = objectMapper.readValue(configurationInputStream, ClientConfiguration.class); + clientConfiguration = readClientConfiguration(new JsonFactory(), configurationInputStream); break; case "YML": case "YAML": - objectMapper = new ObjectMapper(new YAMLFactory()); - clientConfiguration = objectMapper.readValue(configurationInputStream, ClientConfiguration.class); + clientConfiguration = readClientConfiguration(new YAMLFactory(), configurationInputStream); break; default: logger.warn("Not valid client configuration format '{}'", format); @@ -95,6 +94,13 @@ public static ClientConfiguration load(InputStream configurationInputStream, Str return clientConfiguration; } + private static ClientConfiguration readClientConfiguration(JsonFactory jf, InputStream configurationInputStream) + throws IOException { + ObjectMapper objectMapper = new ObjectMapper(jf); + objectMapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false); + return objectMapper.readValue(configurationInputStream, ClientConfiguration.class); + } + private static void parseEnvironmentVariables(ClientConfiguration configuration) { Map envVariables = System.getenv(); for (String variable : envVariables.keySet()) { diff --git a/opencga-client/src/main/java/org/opencb/opencga/client/rest/AbstractParentClient.java b/opencga-client/src/main/java/org/opencb/opencga/client/rest/AbstractParentClient.java index aaaece6f17e..b9df98c5e2c 100644 --- a/opencga-client/src/main/java/org/opencb/opencga/client/rest/AbstractParentClient.java +++ b/opencga-client/src/main/java/org/opencb/opencga/client/rest/AbstractParentClient.java @@ -588,6 +588,20 @@ private void checkErrors(RestResponse restResponse, Response.StatusType s if (!Response.Status.Family.SUCCESSFUL.equals(status.getFamily())) { String message = "Unsuccessful HTTP status " + status.getFamily() + ":" + status.getStatusCode() + " '" + status.getReasonPhrase() + "'"; + if (restResponse != null && CollectionUtils.isNotEmpty(restResponse.getResponses())) { + OpenCGAResult result = restResponse.getResponses().get(0); + if (CollectionUtils.isNotEmpty(result.getEvents())) { + List msgs = new ArrayList<>(); + for (Event event : result.getEvents()) { + if (Event.Type.ERROR.equals(event.getType())) { + msgs.add(event.getMessage()); + } + } + if (!msgs.isEmpty()) { + message += " [" + StringUtils.join(msgs, ",") + "]"; + } + } + } if (throwExceptionOnError) { throw new ClientException(message); diff --git a/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/AdminClient.java b/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/AdminClient.java index b9205c24077..e8318b029c8 100644 --- a/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/AdminClient.java +++ b/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/AdminClient.java @@ -35,7 +35,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 19:08:35 +* Autogenerated on: 2022-03-25 00:37:46 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -44,7 +44,7 @@ /** * This class contains methods for the Admin webservices. - * Client version: 2.2.0-rc2-SNAPSHOT [9160ac7368cfd4011dc8764856956dd793c77c96] + * Client version: 2.2.0-SNAPSHOT [dfe705195791e3286a61838762d29ee845a5c875] * PATH: admin */ public class AdminClient extends AbstractParentClient { diff --git a/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/AlignmentClient.java b/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/AlignmentClient.java index e9d1c52bdcf..38ba605f91e 100644 --- a/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/AlignmentClient.java +++ b/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/AlignmentClient.java @@ -40,7 +40,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 19:08:35 +* Autogenerated on: 2022-03-25 00:37:46 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -49,7 +49,7 @@ /** * This class contains methods for the Alignment webservices. - * Client version: 2.2.0-rc2-SNAPSHOT [9160ac7368cfd4011dc8764856956dd793c77c96] + * Client version: 2.2.0-SNAPSHOT [dfe705195791e3286a61838762d29ee845a5c875] * PATH: analysis/alignment */ public class AlignmentClient extends AbstractParentClient { @@ -106,7 +106,8 @@ public RestResponse runCoverageIndex(CoverageIndexParams data, ObjectMap pa * @return a RestResponse object. * @throws ClientException ClientException if there is any server error. */ - public RestResponse runQcGeneCoverageStats(AlignmentGeneCoverageStatsParams data, ObjectMap params) throws ClientException { + public RestResponse coverageQcGeneCoverageStatsRun(AlignmentGeneCoverageStatsParams data, ObjectMap params) + throws ClientException { params = params != null ? params : new ObjectMap(); params.put("body", data); return execute("analysis", null, "alignment/coverage/qc/geneCoverageStats", null, "run", params, POST, Job.class); diff --git a/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/ClinicalClient.java b/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/ClinicalClient.java index 3cba5bf6868..eab055b6d35 100644 --- a/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/ClinicalClient.java +++ b/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/ClinicalClient.java @@ -49,7 +49,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 19:08:35 +* Autogenerated on: 2022-03-25 00:37:46 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -58,7 +58,7 @@ /** * This class contains methods for the Clinical webservices. - * Client version: 2.2.0-rc2-SNAPSHOT [9160ac7368cfd4011dc8764856956dd793c77c96] + * Client version: 2.2.0-SNAPSHOT [dfe705195791e3286a61838762d29ee845a5c875] * PATH: analysis/clinical */ public class ClinicalClient extends AbstractParentClient { diff --git a/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/CohortClient.java b/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/CohortClient.java index f98fd75d2be..8d2ddec6511 100644 --- a/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/CohortClient.java +++ b/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/CohortClient.java @@ -36,7 +36,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 19:08:35 +* Autogenerated on: 2022-03-25 00:37:46 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -45,7 +45,7 @@ /** * This class contains methods for the Cohort webservices. - * Client version: 2.2.0-rc2-SNAPSHOT [9160ac7368cfd4011dc8764856956dd793c77c96] + * Client version: 2.2.0-SNAPSHOT [dfe705195791e3286a61838762d29ee845a5c875] * PATH: cohorts */ public class CohortClient extends AbstractParentClient { diff --git a/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/DiseasePanelClient.java b/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/DiseasePanelClient.java index 7ea721acb9e..efba87e4df8 100644 --- a/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/DiseasePanelClient.java +++ b/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/DiseasePanelClient.java @@ -32,7 +32,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 19:08:35 +* Autogenerated on: 2022-03-25 00:37:46 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -41,7 +41,7 @@ /** * This class contains methods for the DiseasePanel webservices. - * Client version: 2.2.0-rc2-SNAPSHOT [9160ac7368cfd4011dc8764856956dd793c77c96] + * Client version: 2.2.0-SNAPSHOT [dfe705195791e3286a61838762d29ee845a5c875] * PATH: panels */ public class DiseasePanelClient extends AbstractParentClient { diff --git a/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/FamilyClient.java b/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/FamilyClient.java index e049832de0e..8507bc1136d 100644 --- a/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/FamilyClient.java +++ b/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/FamilyClient.java @@ -35,7 +35,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 19:08:35 +* Autogenerated on: 2022-03-25 00:37:46 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -44,7 +44,7 @@ /** * This class contains methods for the Family webservices. - * Client version: 2.2.0-rc2-SNAPSHOT [9160ac7368cfd4011dc8764856956dd793c77c96] + * Client version: 2.2.0-SNAPSHOT [dfe705195791e3286a61838762d29ee845a5c875] * PATH: families */ public class FamilyClient extends AbstractParentClient { diff --git a/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/FileClient.java b/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/FileClient.java index 2188aa62466..020bf11da89 100644 --- a/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/FileClient.java +++ b/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/FileClient.java @@ -42,7 +42,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 19:08:35 +* Autogenerated on: 2022-03-25 00:37:46 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -51,7 +51,7 @@ /** * This class contains methods for the File webservices. - * Client version: 2.2.0-rc2-SNAPSHOT [9160ac7368cfd4011dc8764856956dd793c77c96] + * Client version: 2.2.0-SNAPSHOT [dfe705195791e3286a61838762d29ee845a5c875] * PATH: files */ public class FileClient extends AbstractParentClient { diff --git a/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/GA4GHClient.java b/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/GA4GHClient.java index ef6173a1e3f..0ee23a6631e 100644 --- a/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/GA4GHClient.java +++ b/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/GA4GHClient.java @@ -27,7 +27,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 19:08:35 +* Autogenerated on: 2022-03-25 00:37:46 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -36,7 +36,7 @@ /** * This class contains methods for the GA4GH webservices. - * Client version: 2.2.0-rc2-SNAPSHOT [9160ac7368cfd4011dc8764856956dd793c77c96] + * Client version: 2.2.0-SNAPSHOT [dfe705195791e3286a61838762d29ee845a5c875] * PATH: ga4gh */ public class GA4GHClient extends AbstractParentClient { diff --git a/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/IndividualClient.java b/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/IndividualClient.java index a5923b54c83..a5c8050085a 100644 --- a/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/IndividualClient.java +++ b/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/IndividualClient.java @@ -35,7 +35,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 19:08:35 +* Autogenerated on: 2022-03-25 00:37:46 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -44,7 +44,7 @@ /** * This class contains methods for the Individual webservices. - * Client version: 2.2.0-rc2-SNAPSHOT [9160ac7368cfd4011dc8764856956dd793c77c96] + * Client version: 2.2.0-SNAPSHOT [dfe705195791e3286a61838762d29ee845a5c875] * PATH: individuals */ public class IndividualClient extends AbstractParentClient { diff --git a/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/JobClient.java b/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/JobClient.java index 8abd147dffd..6b5962643c6 100644 --- a/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/JobClient.java +++ b/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/JobClient.java @@ -36,7 +36,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 19:08:35 +* Autogenerated on: 2022-03-25 00:37:46 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -45,7 +45,7 @@ /** * This class contains methods for the Job webservices. - * Client version: 2.2.0-rc2-SNAPSHOT [9160ac7368cfd4011dc8764856956dd793c77c96] + * Client version: 2.2.0-SNAPSHOT [dfe705195791e3286a61838762d29ee845a5c875] * PATH: jobs */ public class JobClient extends AbstractParentClient { diff --git a/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/MetaClient.java b/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/MetaClient.java index 5a9c77185d7..ad82dbfb52e 100644 --- a/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/MetaClient.java +++ b/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/MetaClient.java @@ -28,7 +28,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 19:08:35 +* Autogenerated on: 2022-03-25 00:37:46 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -37,7 +37,7 @@ /** * This class contains methods for the Meta webservices. - * Client version: 2.2.0-rc2-SNAPSHOT [9160ac7368cfd4011dc8764856956dd793c77c96] + * Client version: 2.2.0-SNAPSHOT [dfe705195791e3286a61838762d29ee845a5c875] * PATH: meta */ public class MetaClient extends AbstractParentClient { diff --git a/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/ProjectClient.java b/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/ProjectClient.java index f625ab4a512..23cb156d1bc 100644 --- a/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/ProjectClient.java +++ b/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/ProjectClient.java @@ -32,7 +32,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 19:08:35 +* Autogenerated on: 2022-03-25 00:37:46 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -41,7 +41,7 @@ /** * This class contains methods for the Project webservices. - * Client version: 2.2.0-rc2-SNAPSHOT [9160ac7368cfd4011dc8764856956dd793c77c96] + * Client version: 2.2.0-SNAPSHOT [dfe705195791e3286a61838762d29ee845a5c875] * PATH: projects */ public class ProjectClient extends AbstractParentClient { diff --git a/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/SampleClient.java b/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/SampleClient.java index aa1d19527da..6147ce3b2b0 100644 --- a/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/SampleClient.java +++ b/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/SampleClient.java @@ -35,7 +35,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 19:08:35 +* Autogenerated on: 2022-03-25 00:37:46 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -44,7 +44,7 @@ /** * This class contains methods for the Sample webservices. - * Client version: 2.2.0-rc2-SNAPSHOT [9160ac7368cfd4011dc8764856956dd793c77c96] + * Client version: 2.2.0-SNAPSHOT [dfe705195791e3286a61838762d29ee845a5c875] * PATH: samples */ public class SampleClient extends AbstractParentClient { diff --git a/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/StudyClient.java b/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/StudyClient.java index 1ebafc11cae..d0364c31eb3 100644 --- a/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/StudyClient.java +++ b/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/StudyClient.java @@ -43,7 +43,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 19:08:35 +* Autogenerated on: 2022-03-25 00:37:46 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -52,7 +52,7 @@ /** * This class contains methods for the Study webservices. - * Client version: 2.2.0-rc2-SNAPSHOT [9160ac7368cfd4011dc8764856956dd793c77c96] + * Client version: 2.2.0-SNAPSHOT [dfe705195791e3286a61838762d29ee845a5c875] * PATH: studies */ public class StudyClient extends AbstractParentClient { diff --git a/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/UserClient.java b/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/UserClient.java index e1d1d3f880c..7470894b451 100644 --- a/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/UserClient.java +++ b/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/UserClient.java @@ -37,7 +37,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 19:08:35 +* Autogenerated on: 2022-03-25 00:37:46 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -46,7 +46,7 @@ /** * This class contains methods for the User webservices. - * Client version: 2.2.0-rc2-SNAPSHOT [9160ac7368cfd4011dc8764856956dd793c77c96] + * Client version: 2.2.0-SNAPSHOT [dfe705195791e3286a61838762d29ee845a5c875] * PATH: users */ public class UserClient extends AbstractParentClient { diff --git a/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/VariantClient.java b/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/VariantClient.java index 4ca86286950..6ebc4ca03d0 100644 --- a/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/VariantClient.java +++ b/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/VariantClient.java @@ -60,7 +60,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 19:08:35 +* Autogenerated on: 2022-03-25 00:37:46 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -69,7 +69,7 @@ /** * This class contains methods for the Variant webservices. - * Client version: 2.2.0-rc2-SNAPSHOT [9160ac7368cfd4011dc8764856956dd793c77c96] + * Client version: 2.2.0-SNAPSHOT [dfe705195791e3286a61838762d29ee845a5c875] * PATH: analysis/variant */ public class VariantClient extends AbstractParentClient { diff --git a/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/VariantOperationClient.java b/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/VariantOperationClient.java index c556e8c51f0..73a4ea954d8 100644 --- a/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/VariantOperationClient.java +++ b/opencga-client/src/main/java/org/opencb/opencga/client/rest/clients/VariantOperationClient.java @@ -48,7 +48,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. -* Autogenerated on: 2022-03-15 19:08:35 +* Autogenerated on: 2022-03-25 00:37:46 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -57,7 +57,7 @@ /** * This class contains methods for the VariantOperation webservices. - * Client version: 2.2.0-rc2-SNAPSHOT [9160ac7368cfd4011dc8764856956dd793c77c96] + * Client version: 2.2.0-SNAPSHOT [dfe705195791e3286a61838762d29ee845a5c875] * PATH: operation */ public class VariantOperationClient extends AbstractParentClient { @@ -358,7 +358,7 @@ public RestResponse indexVariantSample(VariantSampleIndexParams data, Objec * @return a RestResponse object. * @throws ClientException ClientException if there is any server error. */ - public RestResponse configureSampleIndex(SampleIndexConfiguration data, ObjectMap params) throws ClientException { + public RestResponse variantSampleIndexConfigure(SampleIndexConfiguration data, ObjectMap params) throws ClientException { params = params != null ? params : new ObjectMap(); params.put("body", data); return execute("operation", null, "variant/sample/index", null, "configure", params, POST, Job.class); diff --git a/opencga-client/src/main/javascript/Admin.js b/opencga-client/src/main/javascript/Admin.js index 7e7aa02ade1..35a3ebee7e7 100644 --- a/opencga-client/src/main/javascript/Admin.js +++ b/opencga-client/src/main/javascript/Admin.js @@ -12,7 +12,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. - * Autogenerated on: 2022-03-15 19:08:35 + * Autogenerated on: 2022-03-25 00:37:46 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. diff --git a/opencga-client/src/main/javascript/Alignment.js b/opencga-client/src/main/javascript/Alignment.js index 2b48c8f8a26..ce10a6c1f21 100644 --- a/opencga-client/src/main/javascript/Alignment.js +++ b/opencga-client/src/main/javascript/Alignment.js @@ -12,7 +12,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. - * Autogenerated on: 2022-03-15 19:08:35 + * Autogenerated on: 2022-03-25 00:37:46 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -73,7 +73,7 @@ export default class Alignment extends OpenCGAParentClass { * @param {String} [params.jobTags] - Job tags. * @returns {Promise} Promise object in the form of RestResponse instance. */ - runQcGeneCoverageStats(data, params) { + coverageQcGeneCoverageStatsRun(data, params) { return this._post("analysis", null, "alignment/coverage/qc/geneCoverageStats", null, "run", data, params); } diff --git a/opencga-client/src/main/javascript/Clinical.js b/opencga-client/src/main/javascript/Clinical.js index 4396575e33f..9531175dff4 100644 --- a/opencga-client/src/main/javascript/Clinical.js +++ b/opencga-client/src/main/javascript/Clinical.js @@ -12,7 +12,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. - * Autogenerated on: 2022-03-15 19:08:35 + * Autogenerated on: 2022-03-25 00:37:46 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. diff --git a/opencga-client/src/main/javascript/Cohort.js b/opencga-client/src/main/javascript/Cohort.js index 54ee2ec29b0..159b71efc46 100644 --- a/opencga-client/src/main/javascript/Cohort.js +++ b/opencga-client/src/main/javascript/Cohort.js @@ -12,7 +12,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. - * Autogenerated on: 2022-03-15 19:08:35 + * Autogenerated on: 2022-03-25 00:37:46 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. diff --git a/opencga-client/src/main/javascript/DiseasePanel.js b/opencga-client/src/main/javascript/DiseasePanel.js index dbc27855a09..913575eb54f 100644 --- a/opencga-client/src/main/javascript/DiseasePanel.js +++ b/opencga-client/src/main/javascript/DiseasePanel.js @@ -12,7 +12,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. - * Autogenerated on: 2022-03-15 19:08:35 + * Autogenerated on: 2022-03-25 00:37:46 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. diff --git a/opencga-client/src/main/javascript/Family.js b/opencga-client/src/main/javascript/Family.js index 476d850c019..c2c66caa278 100644 --- a/opencga-client/src/main/javascript/Family.js +++ b/opencga-client/src/main/javascript/Family.js @@ -12,7 +12,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. - * Autogenerated on: 2022-03-15 19:08:35 + * Autogenerated on: 2022-03-25 00:37:46 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. diff --git a/opencga-client/src/main/javascript/File.js b/opencga-client/src/main/javascript/File.js index 6cb37081ad5..2c1196bdd09 100644 --- a/opencga-client/src/main/javascript/File.js +++ b/opencga-client/src/main/javascript/File.js @@ -12,7 +12,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. - * Autogenerated on: 2022-03-15 19:08:35 + * Autogenerated on: 2022-03-25 00:37:46 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. diff --git a/opencga-client/src/main/javascript/GA4GH.js b/opencga-client/src/main/javascript/GA4GH.js index d3fb342d856..0a04044c3e1 100644 --- a/opencga-client/src/main/javascript/GA4GH.js +++ b/opencga-client/src/main/javascript/GA4GH.js @@ -12,7 +12,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. - * Autogenerated on: 2022-03-15 19:08:35 + * Autogenerated on: 2022-03-25 00:37:46 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. diff --git a/opencga-client/src/main/javascript/Individual.js b/opencga-client/src/main/javascript/Individual.js index 78418676436..11b4982e002 100644 --- a/opencga-client/src/main/javascript/Individual.js +++ b/opencga-client/src/main/javascript/Individual.js @@ -12,7 +12,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. - * Autogenerated on: 2022-03-15 19:08:35 + * Autogenerated on: 2022-03-25 00:37:46 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. diff --git a/opencga-client/src/main/javascript/Job.js b/opencga-client/src/main/javascript/Job.js index 3f6d9705094..9c6bb14dc26 100644 --- a/opencga-client/src/main/javascript/Job.js +++ b/opencga-client/src/main/javascript/Job.js @@ -12,7 +12,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. - * Autogenerated on: 2022-03-15 19:08:35 + * Autogenerated on: 2022-03-25 00:37:46 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. diff --git a/opencga-client/src/main/javascript/Meta.js b/opencga-client/src/main/javascript/Meta.js index c6d5b646dbf..63cb1d4250f 100644 --- a/opencga-client/src/main/javascript/Meta.js +++ b/opencga-client/src/main/javascript/Meta.js @@ -12,7 +12,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. - * Autogenerated on: 2022-03-15 19:08:35 + * Autogenerated on: 2022-03-25 00:37:46 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. diff --git a/opencga-client/src/main/javascript/Project.js b/opencga-client/src/main/javascript/Project.js index d344cb2b530..d200dded0ac 100644 --- a/opencga-client/src/main/javascript/Project.js +++ b/opencga-client/src/main/javascript/Project.js @@ -12,7 +12,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. - * Autogenerated on: 2022-03-15 19:08:35 + * Autogenerated on: 2022-03-25 00:37:46 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. diff --git a/opencga-client/src/main/javascript/Sample.js b/opencga-client/src/main/javascript/Sample.js index fe2d7d6475c..cc54e52b5d4 100644 --- a/opencga-client/src/main/javascript/Sample.js +++ b/opencga-client/src/main/javascript/Sample.js @@ -12,7 +12,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. - * Autogenerated on: 2022-03-15 19:08:35 + * Autogenerated on: 2022-03-25 00:37:46 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. diff --git a/opencga-client/src/main/javascript/Study.js b/opencga-client/src/main/javascript/Study.js index ffdfcc286bb..f0fd616ab55 100644 --- a/opencga-client/src/main/javascript/Study.js +++ b/opencga-client/src/main/javascript/Study.js @@ -12,7 +12,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. - * Autogenerated on: 2022-03-15 19:08:35 + * Autogenerated on: 2022-03-25 00:37:46 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. diff --git a/opencga-client/src/main/javascript/User.js b/opencga-client/src/main/javascript/User.js index 19a0ca9e311..39016bde26f 100644 --- a/opencga-client/src/main/javascript/User.js +++ b/opencga-client/src/main/javascript/User.js @@ -12,7 +12,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. - * Autogenerated on: 2022-03-15 19:08:35 + * Autogenerated on: 2022-03-25 00:37:46 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. diff --git a/opencga-client/src/main/javascript/Variant.js b/opencga-client/src/main/javascript/Variant.js index 849db45f246..4801e6d7375 100644 --- a/opencga-client/src/main/javascript/Variant.js +++ b/opencga-client/src/main/javascript/Variant.js @@ -12,7 +12,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. - * Autogenerated on: 2022-03-15 19:08:35 + * Autogenerated on: 2022-03-25 00:37:46 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. diff --git a/opencga-client/src/main/javascript/VariantOperation.js b/opencga-client/src/main/javascript/VariantOperation.js index e4fb4e809b9..cf58a74d678 100644 --- a/opencga-client/src/main/javascript/VariantOperation.js +++ b/opencga-client/src/main/javascript/VariantOperation.js @@ -12,7 +12,7 @@ * WARNING: AUTOGENERATED CODE * * This code was generated by a tool. - * Autogenerated on: 2022-03-15 19:08:35 + * Autogenerated on: 2022-03-25 00:37:46 * * Manual changes to this file may cause unexpected behavior in your application. * Manual changes to this file will be overwritten if the code is regenerated. @@ -272,7 +272,7 @@ export default class VariantOperation extends OpenCGAParentClass { * @param {Boolean} [params.skipRebuild] - Skip sample index re-build. * @returns {Promise} Promise object in the form of RestResponse instance. */ - configureSampleIndex(data, params) { + variantSampleIndexConfigure(data, params) { return this._post("operation", null, "variant/sample/index", null, "configure", data, params); } diff --git a/opencga-client/src/main/python/pyopencga/rest_clients/admin_client.py b/opencga-client/src/main/python/pyopencga/rest_clients/admin_client.py index 3c6381d3ca1..ec6057b5dfe 100644 --- a/opencga-client/src/main/python/pyopencga/rest_clients/admin_client.py +++ b/opencga-client/src/main/python/pyopencga/rest_clients/admin_client.py @@ -2,7 +2,7 @@ WARNING: AUTOGENERATED CODE This code was generated by a tool. - Autogenerated on: 2022-03-15 19:08:35 + Autogenerated on: 2022-03-25 00:37:46 Manual changes to this file may cause unexpected behavior in your application. Manual changes to this file will be overwritten if the code is regenerated. @@ -14,7 +14,7 @@ class Admin(_ParentRestClient): """ This class contains methods for the 'Admin' webservices - Client version: 2.2.0-rc2-SNAPSHOT [9160ac7368cfd4011dc8764856956dd793c77c96] + Client version: 2.2.0-SNAPSHOT [dfe705195791e3286a61838762d29ee845a5c875] PATH: /{apiVersion}/admin """ diff --git a/opencga-client/src/main/python/pyopencga/rest_clients/alignment_client.py b/opencga-client/src/main/python/pyopencga/rest_clients/alignment_client.py index 1aa2e3b6c1c..2d22deaf544 100644 --- a/opencga-client/src/main/python/pyopencga/rest_clients/alignment_client.py +++ b/opencga-client/src/main/python/pyopencga/rest_clients/alignment_client.py @@ -2,7 +2,7 @@ WARNING: AUTOGENERATED CODE This code was generated by a tool. - Autogenerated on: 2022-03-15 19:08:35 + Autogenerated on: 2022-03-25 00:37:46 Manual changes to this file may cause unexpected behavior in your application. Manual changes to this file will be overwritten if the code is regenerated. @@ -14,7 +14,7 @@ class Alignment(_ParentRestClient): """ This class contains methods for the 'Analysis - Alignment' webservices - Client version: 2.2.0-rc2-SNAPSHOT [9160ac7368cfd4011dc8764856956dd793c77c96] + Client version: 2.2.0-SNAPSHOT [dfe705195791e3286a61838762d29ee845a5c875] PATH: /{apiVersion}/analysis/alignment """ @@ -56,7 +56,7 @@ def run_coverage_index(self, data=None, **options): return self._post(category='analysis', resource='run', subcategory='alignment/coverage/index', data=data, **options) - def run_qc_gene_coverage_stats(self, data=None, **options): + def coverage_qc_gene_coverage_stats_run(self, data=None, **options): """ Compute gene coverage stats for a given alignment file and a list of genes. diff --git a/opencga-client/src/main/python/pyopencga/rest_clients/clinical_client.py b/opencga-client/src/main/python/pyopencga/rest_clients/clinical_client.py index 7abce19cef5..b74a9a8ef1a 100644 --- a/opencga-client/src/main/python/pyopencga/rest_clients/clinical_client.py +++ b/opencga-client/src/main/python/pyopencga/rest_clients/clinical_client.py @@ -2,7 +2,7 @@ WARNING: AUTOGENERATED CODE This code was generated by a tool. - Autogenerated on: 2022-03-15 19:08:35 + Autogenerated on: 2022-03-25 00:37:46 Manual changes to this file may cause unexpected behavior in your application. Manual changes to this file will be overwritten if the code is regenerated. @@ -14,7 +14,7 @@ class Clinical(_ParentRestClient): """ This class contains methods for the 'Analysis - Clinical' webservices - Client version: 2.2.0-rc2-SNAPSHOT [9160ac7368cfd4011dc8764856956dd793c77c96] + Client version: 2.2.0-SNAPSHOT [dfe705195791e3286a61838762d29ee845a5c875] PATH: /{apiVersion}/analysis/clinical """ diff --git a/opencga-client/src/main/python/pyopencga/rest_clients/cohort_client.py b/opencga-client/src/main/python/pyopencga/rest_clients/cohort_client.py index 18353d95e82..c62d2b7ad76 100644 --- a/opencga-client/src/main/python/pyopencga/rest_clients/cohort_client.py +++ b/opencga-client/src/main/python/pyopencga/rest_clients/cohort_client.py @@ -2,7 +2,7 @@ WARNING: AUTOGENERATED CODE This code was generated by a tool. - Autogenerated on: 2022-03-15 19:08:35 + Autogenerated on: 2022-03-25 00:37:46 Manual changes to this file may cause unexpected behavior in your application. Manual changes to this file will be overwritten if the code is regenerated. @@ -14,7 +14,7 @@ class Cohort(_ParentRestClient): """ This class contains methods for the 'Cohorts' webservices - Client version: 2.2.0-rc2-SNAPSHOT [9160ac7368cfd4011dc8764856956dd793c77c96] + Client version: 2.2.0-SNAPSHOT [dfe705195791e3286a61838762d29ee845a5c875] PATH: /{apiVersion}/cohorts """ diff --git a/opencga-client/src/main/python/pyopencga/rest_clients/disease_panel_client.py b/opencga-client/src/main/python/pyopencga/rest_clients/disease_panel_client.py index c6dffd8e2e6..0229dd12a47 100644 --- a/opencga-client/src/main/python/pyopencga/rest_clients/disease_panel_client.py +++ b/opencga-client/src/main/python/pyopencga/rest_clients/disease_panel_client.py @@ -2,7 +2,7 @@ WARNING: AUTOGENERATED CODE This code was generated by a tool. - Autogenerated on: 2022-03-15 19:08:35 + Autogenerated on: 2022-03-25 00:37:46 Manual changes to this file may cause unexpected behavior in your application. Manual changes to this file will be overwritten if the code is regenerated. @@ -14,7 +14,7 @@ class DiseasePanel(_ParentRestClient): """ This class contains methods for the 'Disease Panels' webservices - Client version: 2.2.0-rc2-SNAPSHOT [9160ac7368cfd4011dc8764856956dd793c77c96] + Client version: 2.2.0-SNAPSHOT [dfe705195791e3286a61838762d29ee845a5c875] PATH: /{apiVersion}/panels """ diff --git a/opencga-client/src/main/python/pyopencga/rest_clients/family_client.py b/opencga-client/src/main/python/pyopencga/rest_clients/family_client.py index 6f41c6f72fb..9cdae2a2545 100644 --- a/opencga-client/src/main/python/pyopencga/rest_clients/family_client.py +++ b/opencga-client/src/main/python/pyopencga/rest_clients/family_client.py @@ -2,7 +2,7 @@ WARNING: AUTOGENERATED CODE This code was generated by a tool. - Autogenerated on: 2022-03-15 19:08:35 + Autogenerated on: 2022-03-25 00:37:46 Manual changes to this file may cause unexpected behavior in your application. Manual changes to this file will be overwritten if the code is regenerated. @@ -14,7 +14,7 @@ class Family(_ParentRestClient): """ This class contains methods for the 'Families' webservices - Client version: 2.2.0-rc2-SNAPSHOT [9160ac7368cfd4011dc8764856956dd793c77c96] + Client version: 2.2.0-SNAPSHOT [dfe705195791e3286a61838762d29ee845a5c875] PATH: /{apiVersion}/families """ diff --git a/opencga-client/src/main/python/pyopencga/rest_clients/file_client.py b/opencga-client/src/main/python/pyopencga/rest_clients/file_client.py index cc2e83c9aab..1002dff1ac7 100644 --- a/opencga-client/src/main/python/pyopencga/rest_clients/file_client.py +++ b/opencga-client/src/main/python/pyopencga/rest_clients/file_client.py @@ -2,7 +2,7 @@ WARNING: AUTOGENERATED CODE This code was generated by a tool. - Autogenerated on: 2022-03-15 19:08:35 + Autogenerated on: 2022-03-25 00:37:46 Manual changes to this file may cause unexpected behavior in your application. Manual changes to this file will be overwritten if the code is regenerated. @@ -14,7 +14,7 @@ class File(_ParentRestClient): """ This class contains methods for the 'Files' webservices - Client version: 2.2.0-rc2-SNAPSHOT [9160ac7368cfd4011dc8764856956dd793c77c96] + Client version: 2.2.0-SNAPSHOT [dfe705195791e3286a61838762d29ee845a5c875] PATH: /{apiVersion}/files """ diff --git a/opencga-client/src/main/python/pyopencga/rest_clients/ga4gh_client.py b/opencga-client/src/main/python/pyopencga/rest_clients/ga4gh_client.py index 2bad88615ca..eae0ef110c4 100644 --- a/opencga-client/src/main/python/pyopencga/rest_clients/ga4gh_client.py +++ b/opencga-client/src/main/python/pyopencga/rest_clients/ga4gh_client.py @@ -2,7 +2,7 @@ WARNING: AUTOGENERATED CODE This code was generated by a tool. - Autogenerated on: 2022-03-15 19:08:35 + Autogenerated on: 2022-03-25 00:37:46 Manual changes to this file may cause unexpected behavior in your application. Manual changes to this file will be overwritten if the code is regenerated. @@ -14,7 +14,7 @@ class GA4GH(_ParentRestClient): """ This class contains methods for the 'GA4GH' webservices - Client version: 2.2.0-rc2-SNAPSHOT [9160ac7368cfd4011dc8764856956dd793c77c96] + Client version: 2.2.0-SNAPSHOT [dfe705195791e3286a61838762d29ee845a5c875] PATH: /{apiVersion}/ga4gh """ diff --git a/opencga-client/src/main/python/pyopencga/rest_clients/individual_client.py b/opencga-client/src/main/python/pyopencga/rest_clients/individual_client.py index 69a1d44649f..848231c6f5f 100644 --- a/opencga-client/src/main/python/pyopencga/rest_clients/individual_client.py +++ b/opencga-client/src/main/python/pyopencga/rest_clients/individual_client.py @@ -2,7 +2,7 @@ WARNING: AUTOGENERATED CODE This code was generated by a tool. - Autogenerated on: 2022-03-15 19:08:35 + Autogenerated on: 2022-03-25 00:37:46 Manual changes to this file may cause unexpected behavior in your application. Manual changes to this file will be overwritten if the code is regenerated. @@ -14,7 +14,7 @@ class Individual(_ParentRestClient): """ This class contains methods for the 'Individuals' webservices - Client version: 2.2.0-rc2-SNAPSHOT [9160ac7368cfd4011dc8764856956dd793c77c96] + Client version: 2.2.0-SNAPSHOT [dfe705195791e3286a61838762d29ee845a5c875] PATH: /{apiVersion}/individuals """ diff --git a/opencga-client/src/main/python/pyopencga/rest_clients/job_client.py b/opencga-client/src/main/python/pyopencga/rest_clients/job_client.py index c3856a7575c..f14f62f2039 100644 --- a/opencga-client/src/main/python/pyopencga/rest_clients/job_client.py +++ b/opencga-client/src/main/python/pyopencga/rest_clients/job_client.py @@ -2,7 +2,7 @@ WARNING: AUTOGENERATED CODE This code was generated by a tool. - Autogenerated on: 2022-03-15 19:08:35 + Autogenerated on: 2022-03-25 00:37:46 Manual changes to this file may cause unexpected behavior in your application. Manual changes to this file will be overwritten if the code is regenerated. @@ -14,7 +14,7 @@ class Job(_ParentRestClient): """ This class contains methods for the 'Jobs' webservices - Client version: 2.2.0-rc2-SNAPSHOT [9160ac7368cfd4011dc8764856956dd793c77c96] + Client version: 2.2.0-SNAPSHOT [dfe705195791e3286a61838762d29ee845a5c875] PATH: /{apiVersion}/jobs """ diff --git a/opencga-client/src/main/python/pyopencga/rest_clients/meta_client.py b/opencga-client/src/main/python/pyopencga/rest_clients/meta_client.py index 1b24f05f13a..bbf5ed4ea59 100644 --- a/opencga-client/src/main/python/pyopencga/rest_clients/meta_client.py +++ b/opencga-client/src/main/python/pyopencga/rest_clients/meta_client.py @@ -2,7 +2,7 @@ WARNING: AUTOGENERATED CODE This code was generated by a tool. - Autogenerated on: 2022-03-15 19:08:35 + Autogenerated on: 2022-03-25 00:37:46 Manual changes to this file may cause unexpected behavior in your application. Manual changes to this file will be overwritten if the code is regenerated. @@ -14,7 +14,7 @@ class Meta(_ParentRestClient): """ This class contains methods for the 'Meta' webservices - Client version: 2.2.0-rc2-SNAPSHOT [9160ac7368cfd4011dc8764856956dd793c77c96] + Client version: 2.2.0-SNAPSHOT [dfe705195791e3286a61838762d29ee845a5c875] PATH: /{apiVersion}/meta """ diff --git a/opencga-client/src/main/python/pyopencga/rest_clients/project_client.py b/opencga-client/src/main/python/pyopencga/rest_clients/project_client.py index 6dc473317dc..f066c04be44 100644 --- a/opencga-client/src/main/python/pyopencga/rest_clients/project_client.py +++ b/opencga-client/src/main/python/pyopencga/rest_clients/project_client.py @@ -2,7 +2,7 @@ WARNING: AUTOGENERATED CODE This code was generated by a tool. - Autogenerated on: 2022-03-15 19:08:35 + Autogenerated on: 2022-03-25 00:37:46 Manual changes to this file may cause unexpected behavior in your application. Manual changes to this file will be overwritten if the code is regenerated. @@ -14,7 +14,7 @@ class Project(_ParentRestClient): """ This class contains methods for the 'Projects' webservices - Client version: 2.2.0-rc2-SNAPSHOT [9160ac7368cfd4011dc8764856956dd793c77c96] + Client version: 2.2.0-SNAPSHOT [dfe705195791e3286a61838762d29ee845a5c875] PATH: /{apiVersion}/projects """ diff --git a/opencga-client/src/main/python/pyopencga/rest_clients/sample_client.py b/opencga-client/src/main/python/pyopencga/rest_clients/sample_client.py index 2f12d49cabc..87f340dfff6 100644 --- a/opencga-client/src/main/python/pyopencga/rest_clients/sample_client.py +++ b/opencga-client/src/main/python/pyopencga/rest_clients/sample_client.py @@ -2,7 +2,7 @@ WARNING: AUTOGENERATED CODE This code was generated by a tool. - Autogenerated on: 2022-03-15 19:08:35 + Autogenerated on: 2022-03-25 00:37:46 Manual changes to this file may cause unexpected behavior in your application. Manual changes to this file will be overwritten if the code is regenerated. @@ -14,7 +14,7 @@ class Sample(_ParentRestClient): """ This class contains methods for the 'Samples' webservices - Client version: 2.2.0-rc2-SNAPSHOT [9160ac7368cfd4011dc8764856956dd793c77c96] + Client version: 2.2.0-SNAPSHOT [dfe705195791e3286a61838762d29ee845a5c875] PATH: /{apiVersion}/samples """ diff --git a/opencga-client/src/main/python/pyopencga/rest_clients/study_client.py b/opencga-client/src/main/python/pyopencga/rest_clients/study_client.py index f4f40faaa87..804e931883b 100644 --- a/opencga-client/src/main/python/pyopencga/rest_clients/study_client.py +++ b/opencga-client/src/main/python/pyopencga/rest_clients/study_client.py @@ -2,7 +2,7 @@ WARNING: AUTOGENERATED CODE This code was generated by a tool. - Autogenerated on: 2022-03-15 19:08:35 + Autogenerated on: 2022-03-25 00:37:46 Manual changes to this file may cause unexpected behavior in your application. Manual changes to this file will be overwritten if the code is regenerated. @@ -14,7 +14,7 @@ class Study(_ParentRestClient): """ This class contains methods for the 'Studies' webservices - Client version: 2.2.0-rc2-SNAPSHOT [9160ac7368cfd4011dc8764856956dd793c77c96] + Client version: 2.2.0-SNAPSHOT [dfe705195791e3286a61838762d29ee845a5c875] PATH: /{apiVersion}/studies """ diff --git a/opencga-client/src/main/python/pyopencga/rest_clients/user_client.py b/opencga-client/src/main/python/pyopencga/rest_clients/user_client.py index 65c9470964c..4a9585e8692 100644 --- a/opencga-client/src/main/python/pyopencga/rest_clients/user_client.py +++ b/opencga-client/src/main/python/pyopencga/rest_clients/user_client.py @@ -2,7 +2,7 @@ WARNING: AUTOGENERATED CODE This code was generated by a tool. - Autogenerated on: 2022-03-15 19:08:35 + Autogenerated on: 2022-03-25 00:37:46 Manual changes to this file may cause unexpected behavior in your application. Manual changes to this file will be overwritten if the code is regenerated. @@ -14,7 +14,7 @@ class User(_ParentRestClient): """ This class contains methods for the 'Users' webservices - Client version: 2.2.0-rc2-SNAPSHOT [9160ac7368cfd4011dc8764856956dd793c77c96] + Client version: 2.2.0-SNAPSHOT [dfe705195791e3286a61838762d29ee845a5c875] PATH: /{apiVersion}/users """ diff --git a/opencga-client/src/main/python/pyopencga/rest_clients/variant_client.py b/opencga-client/src/main/python/pyopencga/rest_clients/variant_client.py index f02dd9cf0a6..67802bbd1a0 100644 --- a/opencga-client/src/main/python/pyopencga/rest_clients/variant_client.py +++ b/opencga-client/src/main/python/pyopencga/rest_clients/variant_client.py @@ -2,7 +2,7 @@ WARNING: AUTOGENERATED CODE This code was generated by a tool. - Autogenerated on: 2022-03-15 19:08:35 + Autogenerated on: 2022-03-25 00:37:46 Manual changes to this file may cause unexpected behavior in your application. Manual changes to this file will be overwritten if the code is regenerated. @@ -14,7 +14,7 @@ class Variant(_ParentRestClient): """ This class contains methods for the 'Analysis - Variant' webservices - Client version: 2.2.0-rc2-SNAPSHOT [9160ac7368cfd4011dc8764856956dd793c77c96] + Client version: 2.2.0-SNAPSHOT [dfe705195791e3286a61838762d29ee845a5c875] PATH: /{apiVersion}/analysis/variant """ diff --git a/opencga-client/src/main/python/pyopencga/rest_clients/variant_operation_client.py b/opencga-client/src/main/python/pyopencga/rest_clients/variant_operation_client.py index 4a543ab1652..e8474fb05a9 100644 --- a/opencga-client/src/main/python/pyopencga/rest_clients/variant_operation_client.py +++ b/opencga-client/src/main/python/pyopencga/rest_clients/variant_operation_client.py @@ -2,7 +2,7 @@ WARNING: AUTOGENERATED CODE This code was generated by a tool. - Autogenerated on: 2022-03-15 19:08:35 + Autogenerated on: 2022-03-25 00:37:46 Manual changes to this file may cause unexpected behavior in your application. Manual changes to this file will be overwritten if the code is regenerated. @@ -14,7 +14,7 @@ class VariantOperation(_ParentRestClient): """ This class contains methods for the 'Operations - Variant Storage' webservices - Client version: 2.2.0-rc2-SNAPSHOT [9160ac7368cfd4011dc8764856956dd793c77c96] + Client version: 2.2.0-SNAPSHOT [dfe705195791e3286a61838762d29ee845a5c875] PATH: /{apiVersion}/operation """ @@ -308,7 +308,7 @@ def index_variant_sample(self, data=None, **options): return self._post(category='operation', resource='index', subcategory='variant/sample', data=data, **options) - def configure_sample_index(self, data=None, **options): + def variant_sample_index_configure(self, data=None, **options): """ Update SampleIndex configuration. PATH: /{apiVersion}/operation/variant/sample/index/configure diff --git a/opencga-client/src/main/resources/client-configuration.yml b/opencga-client/src/main/resources/client-configuration.yml index d156bc9834f..758e3045e71 100644 --- a/opencga-client/src/main/resources/client-configuration.yml +++ b/opencga-client/src/main/resources/client-configuration.yml @@ -13,7 +13,7 @@ rest: url: "https://ws.opencb.org/opencga-test" - name: "opencga-demo1" url: "http://localhost:9191/opencga" - defaultHostIndex: 0 + defaultHostIndex: 1 tokenAutoRefresh: true tlsAllowInvalidCertificates: false query: diff --git a/opencga-clinical/pom.xml b/opencga-clinical/pom.xml index 62f0dcc72d6..384891c31ea 100644 --- a/opencga-clinical/pom.xml +++ b/opencga-clinical/pom.xml @@ -5,7 +5,7 @@ org.opencb.opencga opencga - 2.2.0-rc2 + 2.2.0 ../pom.xml 4.0.0 diff --git a/opencga-core/pom.xml b/opencga-core/pom.xml index 116539c0847..2d48278ee6f 100644 --- a/opencga-core/pom.xml +++ b/opencga-core/pom.xml @@ -22,7 +22,7 @@ org.opencb.opencga opencga - 2.2.0-rc2 + 2.2.0 ../pom.xml diff --git a/opencga-core/src/main/java/org/opencb/opencga/core/models/operations/variant/VariantFamilyIndexParams.java b/opencga-core/src/main/java/org/opencb/opencga/core/models/operations/variant/VariantFamilyIndexParams.java index da1cd4c7e71..0a2c8d3b8b5 100644 --- a/opencga-core/src/main/java/org/opencb/opencga/core/models/operations/variant/VariantFamilyIndexParams.java +++ b/opencga-core/src/main/java/org/opencb/opencga/core/models/operations/variant/VariantFamilyIndexParams.java @@ -25,14 +25,16 @@ public class VariantFamilyIndexParams extends ToolParams { public static final String DESCRIPTION = "Variant family index params."; private List family; private boolean overwrite; + private boolean updateIndex; private boolean skipIncompleteFamilies; public VariantFamilyIndexParams() { } - public VariantFamilyIndexParams(List family, boolean overwrite, boolean skipIncompleteFamilies) { + public VariantFamilyIndexParams(List family, boolean overwrite, boolean updateIndex, boolean skipIncompleteFamilies) { this.family = family; this.overwrite = overwrite; + this.updateIndex = updateIndex; this.skipIncompleteFamilies = skipIncompleteFamilies; } @@ -45,6 +47,15 @@ public VariantFamilyIndexParams setFamily(List family) { return this; } + public boolean isUpdateIndex() { + return updateIndex; + } + + public VariantFamilyIndexParams setUpdateIndex(boolean updateIndex) { + this.updateIndex = updateIndex; + return this; + } + public boolean isOverwrite() { return overwrite; } diff --git a/opencga-core/src/main/java/org/opencb/opencga/core/models/operations/variant/VariantStatsExportParams.java b/opencga-core/src/main/java/org/opencb/opencga/core/models/operations/variant/VariantStatsExportParams.java index 15fa42f9f64..dc412b115b2 100644 --- a/opencga-core/src/main/java/org/opencb/opencga/core/models/operations/variant/VariantStatsExportParams.java +++ b/opencga-core/src/main/java/org/opencb/opencga/core/models/operations/variant/VariantStatsExportParams.java @@ -25,18 +25,18 @@ public class VariantStatsExportParams extends ToolParams { public VariantStatsExportParams() { } - public VariantStatsExportParams(List cohorts, String output, String region, String gene, String outputFormat) { + public VariantStatsExportParams(List cohorts, String output, String region, String gene, String outputFileFormat) { this.cohorts = cohorts; this.output = output; this.region = region; this.gene = gene; - this.outputFormat = outputFormat; + this.outputFileFormat = outputFileFormat; } private List cohorts; private String output; private String region; private String gene; - private String outputFormat; + private String outputFileFormat; public List getCohorts() { return cohorts; @@ -74,12 +74,12 @@ public VariantStatsExportParams setGene(String gene) { return this; } - public String getOutputFormat() { - return outputFormat; + public String getOutputFileFormat() { + return outputFileFormat; } - public VariantStatsExportParams setOutputFormat(String outputFormat) { - this.outputFormat = outputFormat; + public VariantStatsExportParams setOutputFileFormat(String outputFileFormat) { + this.outputFileFormat = outputFileFormat; return this; } } diff --git a/opencga-core/src/main/java/org/opencb/opencga/core/models/variant/GwasAnalysisParams.java b/opencga-core/src/main/java/org/opencb/opencga/core/models/variant/GwasAnalysisParams.java index 6a648c29078..0fa1db8e12a 100644 --- a/opencga-core/src/main/java/org/opencb/opencga/core/models/variant/GwasAnalysisParams.java +++ b/opencga-core/src/main/java/org/opencb/opencga/core/models/variant/GwasAnalysisParams.java @@ -24,7 +24,7 @@ public class GwasAnalysisParams extends ToolParams { public static final String DESCRIPTION = "Gwas analysis params"; private String phenotype; - private boolean index; + private Boolean index; private String indexScoreId; private GwasConfiguration.Method method; private GwasConfiguration.FisherMode fisherMode; @@ -67,11 +67,11 @@ public GwasAnalysisParams setPhenotype(String phenotype) { return this; } - public boolean isIndex() { + public Boolean isIndex() { return index; } - public GwasAnalysisParams setIndex(boolean index) { + public GwasAnalysisParams setIndex(Boolean index) { this.index = index; return this; } diff --git a/opencga-core/src/main/java/org/opencb/opencga/core/models/variant/VariantExportParams.java b/opencga-core/src/main/java/org/opencb/opencga/core/models/variant/VariantExportParams.java index 3936737b4b1..0fdde4e22aa 100644 --- a/opencga-core/src/main/java/org/opencb/opencga/core/models/variant/VariantExportParams.java +++ b/opencga-core/src/main/java/org/opencb/opencga/core/models/variant/VariantExportParams.java @@ -22,7 +22,7 @@ public class VariantExportParams extends VariantQueryParams { public static final String DESCRIPTION = "Variant export params"; private String outdir; private String outputFileName; - private String outputFormat; + private String outputFileFormat; //<<<<< CAMBIAR private String variantsFile; private String include; private String exclude; @@ -34,12 +34,12 @@ public class VariantExportParams extends VariantQueryParams { public VariantExportParams() { } - public VariantExportParams(Query query, String outdir, String outputFileName, String outputFormat, + public VariantExportParams(Query query, String outdir, String outputFileName, String outputFileFormat, String variantsFile) { super(query); this.outdir = outdir; this.outputFileName = outputFileName; - this.outputFormat = outputFormat; + this.outputFileFormat = outputFileFormat; this.variantsFile = variantsFile; } @@ -61,12 +61,12 @@ public VariantExportParams setOutputFileName(String outputFileName) { return this; } - public String getOutputFormat() { - return outputFormat; + public String getOutputFileFormat() { + return outputFileFormat; } - public VariantExportParams setOutputFormat(String outputFormat) { - this.outputFormat = outputFormat; + public VariantExportParams setOutputFileFormat(String outputFileFormat) { + this.outputFileFormat = outputFileFormat; return this; } diff --git a/opencga-core/src/main/resources/configuration.yml b/opencga-core/src/main/resources/configuration.yml index 93a9841e81b..4244e009a47 100644 --- a/opencga-core/src/main/resources/configuration.yml +++ b/opencga-core/src/main/resources/configuration.yml @@ -159,6 +159,7 @@ analysis: k8s.imagePullPolicy: "IfNotPresent" # k8s.imagePullSecrets: # name : "dockerhub-secrets-name" + k8s.ttlSecondsAfterFinished: 3600 # s k8s.namespace: "default" # FOR Cluster Autoscaler and dedicated jobs agent pool: k8s.requests: diff --git a/opencga-core/src/main/resources/doc/json/Sample.json b/opencga-core/src/main/resources/doc/json/Sample.json deleted file mode 100644 index 79904a1bc2f..00000000000 --- a/opencga-core/src/main/resources/doc/json/Sample.json +++ /dev/null @@ -1,78 +0,0 @@ -{ - id: "ISDBM322015", - uuid: "eba13afe-0172-0004-0001-d4c92fd95e0a", - individualId: "ISDBM322015", - fileIds: [ - "data:quartet.variants.annotated.vcf.gz", - "SonsAlignedBamFile.bam" - ], - annotationSets: [], - description: "", - somatic: false, - qualityControl: { - fileIds: [], - comments: [], - alignmentMetrics: [ - { - bamFileId: SonsAlignedBamFile.bam, - fastQc: {13 items}, - samtoolsFlagstats: {14 items}, - geneCoverageStats: [2 items] - } - ], - variantMetrics: { - variantStats: [1 item], - signatures: [], - vcfFileIds: [] - } - }, - release: 1, - version: 5, - creationDate: "20200625131831", - modificationDate: "20200709003738", - phenotypes: [ - { - id: "HP:0000545", - name: "Myopia", - source: "HPO" - } - ], - status: { - name: "", - description: "", - date: "" - }, - internal: { - status: { - name: "READY", - date: "20200625131831", - description: "" - } - }, - attributes: { - OPENCGA_INDIVIDUAL: { - id: "ISDBM322015", - name: "ISDBM322015", - uuid: "eba13738-0172-0006-0001-283471b7ae69", - father: {4 items}, - mother: {4 items}, - location: {}, - qualityControl: {4 items}, - sex: "MALE", - karyotypicSex: "XY", - ethnicity: "", - population: {}, - release: 1, - version: 6, - creationDate: "20200625131830", - modificationDate: "20201027004616", - lifeStatus: "ALIVE", - phenotypes: [2 items], - disorders: [1 item], - parentalConsanguinity: false, - status: {3 items}, - internal: {1 item}, - attributes: {} - } - } -} diff --git a/opencga-core/src/main/resources/log4j2.xml b/opencga-core/src/main/resources/log4j2.xml index 96a6c1dcdaa..1611325528f 100644 --- a/opencga-core/src/main/resources/log4j2.xml +++ b/opencga-core/src/main/resources/log4j2.xml @@ -1,15 +1,42 @@ + + ${sys:user.home}/.opencga/logs + + + + [%-5level] %d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %c{1} - %msg%n + + + + + + + + + + + + + + - - + + + + + + + diff --git a/opencga-master/pom.xml b/opencga-master/pom.xml index 685b858ad95..593fccdd629 100644 --- a/opencga-master/pom.xml +++ b/opencga-master/pom.xml @@ -22,7 +22,7 @@ opencga org.opencb.opencga - 2.2.0-rc2 + 2.2.0 ../pom.xml diff --git a/opencga-master/src/main/java/org/opencb/opencga/master/monitor/executors/K8SExecutor.java b/opencga-master/src/main/java/org/opencb/opencga/master/monitor/executors/K8SExecutor.java index c2349ec4389..1c3e43bec60 100644 --- a/opencga-master/src/main/java/org/opencb/opencga/master/monitor/executors/K8SExecutor.java +++ b/opencga-master/src/main/java/org/opencb/opencga/master/monitor/executors/K8SExecutor.java @@ -48,6 +48,7 @@ public class K8SExecutor implements BatchExecutor { public static final String K8S_IMAGE_NAME = "k8s.imageName"; public static final String K8S_IMAGE_PULL_POLICY = "k8s.imagePullPolicy"; public static final String K8S_IMAGE_PULL_SECRETS = "k8s.imagePullSecrets"; + public static final String K8S_TTL_SECONDS_AFTER_FINISHED = "k8s.ttlSecondsAfterFinished"; public static final String K8S_DIND_IMAGE_NAME = "k8s.dind.imageName"; public static final String K8S_REQUESTS = "k8s.requests"; public static final String K8S_LIMITS = "k8s.limits"; @@ -102,6 +103,7 @@ public class K8SExecutor implements BatchExecutor { private final Watch jobsWatcher; private String imagePullPolicy; private List imagePullSecrets; + private int ttlSecondsAfterFinished; public K8SExecutor(Configuration configuration) { Execution execution = configuration.getAnalysis().getExecution(); @@ -121,6 +123,7 @@ public K8SExecutor(Configuration configuration) { this.kubernetesClient = new DefaultKubernetesClient(k8sConfig).inNamespace(namespace); this.imagePullPolicy = execution.getOptions().getString(K8S_IMAGE_PULL_POLICY, "IfNotPresent"); this.imagePullSecrets = buildLocalObjectReference(execution.getOptions().get(K8S_IMAGE_PULL_SECRETS)); + this.ttlSecondsAfterFinished = execution.getOptions().getInt(K8S_TTL_SECONDS_AFTER_FINISHED, 3600); nodeSelector = getMap(execution, K8S_NODE_SELECTOR); if (execution.getOptions().containsKey(K8S_SECURITY_CONTEXT)) { securityContext = buildObject(execution.getOptions().get(K8S_SECURITY_CONTEXT), SecurityContext.class); @@ -243,7 +246,7 @@ public void execute(String jobId, String queue, String commandLine, Path stdout, .withLabels(Collections.singletonMap("opencga", "job")) .build()) .withSpec(new JobSpecBuilder() - .withTtlSecondsAfterFinished(30) + .withTtlSecondsAfterFinished(ttlSecondsAfterFinished) .withBackoffLimit(0) // specify the number of retries before considering a Job as failed .withTemplate(new PodTemplateSpecBuilder() .withMetadata(new ObjectMetaBuilder() diff --git a/opencga-server/pom.xml b/opencga-server/pom.xml index 7ac7e9fdab2..85b6468a8ef 100644 --- a/opencga-server/pom.xml +++ b/opencga-server/pom.xml @@ -22,7 +22,7 @@ org.opencb.opencga opencga - 2.2.0-rc2 + 2.2.0 ../pom.xml diff --git a/opencga-server/src/main/java/org/opencb/opencga/server/generator/ClientsGenerator.java b/opencga-server/src/main/java/org/opencb/opencga/server/generator/ClientsGenerator.java index 97b799361e8..8ef8e0ca464 100644 --- a/opencga-server/src/main/java/org/opencb/opencga/server/generator/ClientsGenerator.java +++ b/opencga-server/src/main/java/org/opencb/opencga/server/generator/ClientsGenerator.java @@ -96,7 +96,7 @@ private static void generateLibrary(String language, String restFilePath, String BufferedReader input = new BufferedReader(new InputStreamReader(p.getInputStream())); String line; while ((line = input.readLine()) != null) { - logger.info("{} library generator: {}", language, line); + logger.debug("{} library generator: {}", language, line); } p.waitFor(); input.close(); diff --git a/opencga-server/src/main/java/org/opencb/opencga/server/generator/RestApiParser.java b/opencga-server/src/main/java/org/opencb/opencga/server/generator/RestApiParser.java index 5d9d81d01b5..ebe8c9ac5b9 100644 --- a/opencga-server/src/main/java/org/opencb/opencga/server/generator/RestApiParser.java +++ b/opencga-server/src/main/java/org/opencb/opencga/server/generator/RestApiParser.java @@ -195,7 +195,7 @@ private RestCategory getCategory(Class clazz) { && !Modifier.isStatic(modifiers) && !Modifier.isFinal(modifiers)) { RestParameter innerParam = getParameter(declaredField.getName(), variablePrefix, declaredField, - declaredField.getType().getName(), declaredField.getName()); + declaredField.getType().getName(), ""); if (innerParam.isList()) { innerParam.setGenericType(declaredField.getGenericType().getTypeName()); } else { diff --git a/opencga-server/src/main/java/org/opencb/opencga/server/generator/models/RestEndpoint.java b/opencga-server/src/main/java/org/opencb/opencga/server/generator/models/RestEndpoint.java index bf0ce4d954b..05bdde7bae0 100644 --- a/opencga-server/src/main/java/org/opencb/opencga/server/generator/models/RestEndpoint.java +++ b/opencga-server/src/main/java/org/opencb/opencga/server/generator/models/RestEndpoint.java @@ -40,10 +40,17 @@ public boolean hasQueryParams() { return false; } - public String getBodyParamsObject() { + public String getBodyClassName() { for (RestParameter restParameter : getParameters()) { - if (restParameter.getData() != null) { - return restParameter.getTypeClass().substring(restParameter.getTypeClass().lastIndexOf('.') + 1).replaceAll(";", "").trim(); + if (restParameter.getName().equals("body")) { + String className = restParameter.getTypeClass() + .substring(restParameter.getTypeClass().lastIndexOf('.') + 1).replaceAll(";", "").trim(); + + // We cannot instantiate an interface, we always use ObjectMap in these cases. + if (className.equalsIgnoreCase("Map")) { + className = "ObjectMap"; + } + return className; } } return ""; diff --git a/opencga-server/src/main/java/org/opencb/opencga/server/generator/writers/ParentClientRestApiWriter.java b/opencga-server/src/main/java/org/opencb/opencga/server/generator/writers/ParentClientRestApiWriter.java index c5f9713f34b..73e0daa7241 100644 --- a/opencga-server/src/main/java/org/opencb/opencga/server/generator/writers/ParentClientRestApiWriter.java +++ b/opencga-server/src/main/java/org/opencb/opencga/server/generator/writers/ParentClientRestApiWriter.java @@ -190,7 +190,8 @@ public String getMethodName(RestCategory restCategory, RestEndpoint restEndpoint } } else if (items.length == 4) { if (notAny(items)) { - methodName = items[3] + "_" + items[1] + "_" + items[2]; + methodName = items[0] + "_" + items[1] + "_" + items[2] + "_" + items[3]; +// methodName = items[3] + "_" + items[1] + "_" + items[2]; } else if (items[0].contains("}") && items[2].contains("}") && (!items[1].contains("}")) && (!items[3].contains("}"))) { methodName = items[3] + "_" + items[1]; } diff --git a/opencga-server/src/main/java/org/opencb/opencga/server/generator/writers/cli/AutoCompleteWriter.java b/opencga-server/src/main/java/org/opencb/opencga/server/generator/writers/cli/AutoCompleteWriter.java index 29ce13b7c62..0e2745b2b8e 100644 --- a/opencga-server/src/main/java/org/opencb/opencga/server/generator/writers/cli/AutoCompleteWriter.java +++ b/opencga-server/src/main/java/org/opencb/opencga/server/generator/writers/cli/AutoCompleteWriter.java @@ -107,9 +107,8 @@ protected String getClassHeader(String key) { sb.append(" private List " + getCategoryCommandName(restCategory, config) + "List = asList( "); for (RestEndpoint restEndpoint : restCategory.getEndpoints()) { String commandName = getMethodName(restCategory, restEndpoint).replaceAll("_", "-"); - if ((!"POST".equals(restEndpoint.getMethod()) || restEndpoint.hasPrimitiveBodyParams(config, commandName)) && restEndpoint.hasParameters()) { + if ("POST".equals(restEndpoint.getMethod()) || restEndpoint.hasParameters()) { if (config.isAvailableCommand(commandName)) { - sb.append("\"" + commandName + "\","); } } diff --git a/opencga-server/src/main/java/org/opencb/opencga/server/generator/writers/cli/ExecutorsCliRestApiWriter.java b/opencga-server/src/main/java/org/opencb/opencga/server/generator/writers/cli/ExecutorsCliRestApiWriter.java index 17d225925ab..9097baac37b 100644 --- a/opencga-server/src/main/java/org/opencb/opencga/server/generator/writers/cli/ExecutorsCliRestApiWriter.java +++ b/opencga-server/src/main/java/org/opencb/opencga/server/generator/writers/cli/ExecutorsCliRestApiWriter.java @@ -78,7 +78,6 @@ protected String getClassImports(String key) { } } if (bodyParam.getType().equals("enum")) { - // sb.append("enum " + bodyParam.getTypeClass()); imports.add(getEnumImport(bodyParam.getTypeClass().replaceAll("\\$", "\\."))); } } @@ -174,10 +173,8 @@ private String methodExecute(RestCategory restCategory, CategoryConfig config) { sb.append(" RestResponse queryResponse = null;\n\n"); sb.append(" switch (subCommandString) {\n"); for (RestEndpoint restEndpoint : restCategory.getEndpoints()) { - //If it is post, it must have parameters in the body, - // therefore it must be different from post or have some primitive value in the body so that we can generate the method String commandName = getMethodName(restCategory, restEndpoint).replaceAll("_", "-"); - if ((!"POST".equals(restEndpoint.getMethod()) || restEndpoint.hasPrimitiveBodyParams(config, commandName)) && restEndpoint.hasParameters()) { + if ("POST".equals(restEndpoint.getMethod()) || restEndpoint.hasParameters()) { if (config.isAvailableCommand(commandName)) { sb.append(" case \"" + reverseCommandName(commandName) + "\":\n"); sb.append(" queryResponse = " + getAsCamelCase(commandName) + "();\n"); @@ -216,10 +213,8 @@ protected String getClassMethods(String key) { CategoryConfig config = availableCategoryConfigs.get(key); sb.append(methodExecute(restCategory, config)); for (RestEndpoint restEndpoint : restCategory.getEndpoints()) { - //If it is post, it must have parameters in the body, - // therefore it must be different from post or have some primitive value in the body so that we can generate the method String commandName = getMethodName(restCategory, restEndpoint).replaceAll("_", "-"); - if ((!"POST".equals(restEndpoint.getMethod()) || restEndpoint.hasPrimitiveBodyParams(config, commandName)) && restEndpoint.hasParameters()) { + if ("POST".equals(restEndpoint.getMethod()) || restEndpoint.hasParameters()) { if (config.isAvailableCommand(commandName)) { sb.append("\n"); sb.append(" " + (config.isExecutorExtendedCommand(commandName) ? "protected" : @@ -252,8 +247,10 @@ private String getReturn(RestCategory restCategory, RestEndpoint restEndpoint, C + getAsCamelCase(commandName) + "("; res += restEndpoint.getPathParams(); res += restEndpoint.getMandatoryQueryParams(config, commandName); - if (restEndpoint.hasPrimitiveBodyParams(config, commandName)) { - res += getAsVariableName(restEndpoint.getBodyParamsObject()) + ", "; + + String bodyClassName = restEndpoint.getBodyClassName(); + if (StringUtils.isNotEmpty(bodyClassName)) { + res += getAsVariableName(bodyClassName) + ", "; } if (restEndpoint.hasQueryParams()) { res += "queryParams"; @@ -267,63 +264,70 @@ private String getReturn(RestCategory restCategory, RestEndpoint restEndpoint, C private String getBodyParams(RestEndpoint restEndpoint, CategoryConfig config, String commandName) { StringBuilder sb = new StringBuilder(); - if (restEndpoint.hasPrimitiveBodyParams(config, commandName)) { - String bodyParamsObject = restEndpoint.getBodyParamsObject(); + String bodyClassName = restEndpoint.getBodyClassName(); + if (StringUtils.isNotEmpty(bodyClassName)) { for (RestParameter restParameter : restEndpoint.getParameters()) { if (restParameter.getData() != null && !restParameter.getData().isEmpty()) { sb.append(generateBeans(restParameter.getData())); for (RestParameter bparameter : restParameter.getData()) { if (bparameter.getType().equals("enum")) { - sb.append(" " + getEnumName(bparameter.getTypeClass()) + " " + normaliceNames(getAsCamelCase(bparameter.getName() + "Param")) + " = " - + getEnumName(bparameter.getTypeClass()) + ".valueOf(commandOptions." + normaliceNames(getAsCamelCase(bparameter.getName())) + ");\n"); + sb.append(" " + getEnumName(bparameter.getTypeClass()) + " " + normalizeNames(getAsCamelCase(bparameter.getName() + "Param")) + " = null;"); + sb.append("\n if (commandOptions." + normalizeNames(getAsCamelCase(bparameter.getName())) + " != null) {\n "); + sb.append(" " + normalizeNames(getAsCamelCase(bparameter.getName() + "Param")) + " = " + + getEnumName(bparameter.getTypeClass()) + ".valueOf(commandOptions." + normalizeNames(getAsCamelCase(bparameter.getName())) + ");\n"); + sb.append("\n } \n"); // sb.append(getSwitchEnum(bparameter, normaliceNames(getAsCamelCase(bparameter.getName() + "Param")), getEnumName(bparameter.getTypeClass()))); - } } } - } - sb.append("\n " + bodyParamsObject + " " + getAsVariableName(bodyParamsObject) + " = new " + bodyParamsObject + "();"); - sb.append("\n if (commandOptions.jsonViewTemplate) {"); + sb.append("\n " + bodyClassName + " " + getAsVariableName(bodyClassName) + " = new " + bodyClassName + "();"); + sb.append("\n if (commandOptions.jsonDataModel) {"); sb.append("\n RestResponse<" + getValidResponseNames(restEndpoint.getResponse()) + "> res = new RestResponse<>();"); sb.append("\n res.setType(QueryType.VOID);"); - sb.append("\n PrintUtils.println(getObjectAsJSON(" + getAsVariableName(bodyParamsObject) + "));"); + sb.append("\n PrintUtils.println(getObjectAsJSON(" + getAsVariableName(bodyClassName) + "));"); sb.append("\n return res;"); sb.append("\n } else if (commandOptions.jsonFile != null) {"); sb.append("\n ObjectMapper objectMapper = new ObjectMapper();"); - sb.append("\n objectMapper.writeValue(new java.io.File(commandOptions.jsonFile), " + getAsVariableName(bodyParamsObject) + ");"); + sb.append("\n objectMapper.writeValue(new java.io.File(commandOptions.jsonFile), " + getAsVariableName(bodyClassName) + ");"); sb.append("\n } "); if (hasParameters(restEndpoint.getParameters(), commandName, config)) { sb.append(" else {"); - sb.append("\n ((" + bodyParamsObject + ")" + getAsVariableName(bodyParamsObject) + ")"); + if (hasParametersNoBoolean(restEndpoint.getParameters(), commandName, config)) { + sb.append("\n ((" + bodyClassName + ")" + getAsVariableName(bodyClassName) + ")"); + } Set variables = new HashSet<>(); for (RestParameter restParameter : restEndpoint.getParameters()) { if (restParameter.getData() != null && !restParameter.getData().isEmpty()) { + List booleanParams = new ArrayList<>(); for (RestParameter bodyParam : restParameter.getData()) { if (config.isAvailableSubCommand(bodyParam.getName(), commandName)) { - if (!bodyParam.isComplex() && !bodyParam.isInnerParam()) { - //sometimes the name of the parameter has the prefix "body" so as not to coincide with another parameter + if (bodyParam.getType().toLowerCase().contains("boolean") && !bodyParam.isComplex() && !bodyParam.isInnerParam()) { + // boolean params must check null values + booleanParams.add(bodyParam); + } else if (!bodyParam.isComplex() && !bodyParam.isInnerParam()) { + // sometimes the name of the parameter has the prefix "body" so as not to coincide with another parameter // with the same name, but the setter does not have this prefix, so it must be removed sb.append("\n .set" + getAsClassName(bodyParam.getName().replaceAll("body_", "")) + "(commandOptions." - + normaliceNames(getAsCamelCase(bodyParam.getName())) + ")"); + + normalizeNames(getAsCamelCase(bodyParam.getName())) + ")"); } else if (bodyParam.isStringList()) { sb.append("\n .set" + getAsClassName(bodyParam.getName().replaceAll("body_", "")) + "(splitWithTrim(commandOptions." - + normaliceNames(getAsCamelCase(bodyParam.getName())) + "))"); - + + normalizeNames(getAsCamelCase(bodyParam.getName())) + "))"); } if (bodyParam.getType().equals("enum")) { if (reverseCommandName(commandName).contains("create")) { sb.append("\n .set" + getAsClassName(bodyParam.getName().replaceAll("body_", "")) + "(" - + normaliceNames(getAsCamelCase(bodyParam.getName() + "Param")) + ")"); + + normalizeNames(getAsCamelCase(bodyParam.getName() + "Param")) + ")"); } } } - //If the parameter is InnerParam (It means it's a field of inner bean) need to add to the variables Set + + // If the parameter is InnerParam (It means it's a field of inner bean) need to add to the variables Set // for no duplicate set action of Bean (Parent) if (bodyParam.isInnerParam() && !bodyParam.isCollection()) { if (!variables.contains(bodyParam.getParentParamName())) { @@ -332,24 +336,70 @@ private String getBodyParams(RestEndpoint restEndpoint, CategoryConfig config, S + ")"); variables.add(bodyParam.getParentParamName()); } + } + } + sb.append(";\n"); + sb.append(appendBooleanParams(booleanParams, "((" + bodyClassName + ")" + getAsVariableName(bodyClassName) + ")")); + sb.append("\n }\n"); + } + } + } + } + return sb.toString(); + } + private StringBuilder appendBooleanParams(List booleanParams, String variableName) { + StringBuilder sb = new StringBuilder(); + for (RestParameter bodyParam : booleanParams) { + sb.append("\n if (commandOptions." + normalizeNames(getAsCamelCase(bodyParam.getName())) + " != null){"); + sb.append("\n " + variableName + ".set" + getAsClassName(bodyParam.getName().replaceAll("body_", "")) + + "(commandOptions." + normalizeNames(getAsCamelCase(bodyParam.getName())) + ");"); + sb.append("\n }\n"); + + } + return sb; + } + + private boolean hasParametersNoBoolean(List parameters, String commandName, CategoryConfig config) { + boolean res = false; + Set variables = new HashSet<>(); + for (RestParameter restParameter : parameters) { + if (restParameter.getData() != null && !restParameter.getData().isEmpty()) { + for (RestParameter bodyParam : restParameter.getData()) { + if (config.isAvailableSubCommand(bodyParam.getName(), commandName)) { + if (!bodyParam.isComplex() && !bodyParam.isInnerParam() && !bodyParam.getType().toLowerCase().contains("boolean")) { + //sometimes the name of the parameter has the prefix "body" so as not to coincide with another parameter + // with the same name, but the setter does not have this prefix, so it must be removed + return true; + } else { + if (bodyParam.isStringList() && !bodyParam.getType().toLowerCase().contains("boolean")) { + return true; + } + } + if (bodyParam.getType().equals("enum")) { + if (reverseCommandName(commandName).contains("create")) { + return true; } } } + // If the parameter is InnerParam (It means it's a field of inner bean) need to add to the variables Set + // for no duplicate set action of Bean (Parent) + if (bodyParam.isInnerParam() && !bodyParam.isCollection()) { + if (!variables.contains(bodyParam.getParentParamName())) { + variables.add(bodyParam.getParentParamName()); + return true; + } + } } - sb.append(";\n"); - sb.append("\n }\n"); } } - - return sb.toString(); + return res; } private boolean hasParameters(List parameters, String commandName, CategoryConfig config) { - boolean res = false; Set variables = new HashSet<>(); for (RestParameter restParameter : parameters) { @@ -357,7 +407,7 @@ private boolean hasParameters(List parameters, String commandName for (RestParameter bodyParam : restParameter.getData()) { if (config.isAvailableSubCommand(bodyParam.getName(), commandName)) { if (!bodyParam.isComplex() && !bodyParam.isInnerParam()) { - //sometimes the name of the parameter has the prefix "body" so as not to coincide with another parameter + // sometimes the name of the parameter has the prefix "body" so as not to coincide with another parameter // with the same name, but the setter does not have this prefix, so it must be removed return true; } else if (bodyParam.isStringList()) { @@ -370,19 +420,17 @@ private boolean hasParameters(List parameters, String commandName } } } - //If the parameter is InnerParam (It means it's a field of inner bean) need to add to the variables Set + + // If the parameter is InnerParam (It means it's a field of inner bean) need to add to the variables Set // for no duplicate set action of Bean (Parent) if (bodyParam.isInnerParam() && !bodyParam.isCollection()) { if (!variables.contains(bodyParam.getParentParamName())) { variables.add(bodyParam.getParentParamName()); return true; } - - } } } - } return res; } @@ -392,21 +440,18 @@ private String getSwitchEnum(RestParameter parameter, String normaliceNames, Str final String allowedValues = parameter.getAllowedValues(); String[] values = allowedValues.split(" "); - sb.append(" switch (commandOptions." + normaliceNames(getAsCamelCase(parameter.getName())) + ") {\n"); + sb.append(" switch (commandOptions." + normalizeNames(getAsCamelCase(parameter.getName())) + ") {\n"); for (String s : values) { sb.append(" case \"" + s + "\":\n"); sb.append(" " + normaliceNames + "=" + enumName + "." + s + ";\n"); sb.append(" break;\n"); - } sb.append(" }\n"); - return sb.toString(); } private String getEnumName(String typeClass) { return typeClass.replace(";", "").substring(typeClass.lastIndexOf(".") + 1).replace("$", "."); - } private String generateBeans(List restParameters) { @@ -419,17 +464,14 @@ private String generateBeans(List restParameters) { } } - for (String nameBean : beans) { - - sb.append("\n " + CommandLineUtils.getClassName(nameBean) + " " + CommandLineUtils.getAsVariableName(CommandLineUtils.getClassName(nameBean)) + "= new " + CommandLineUtils.getClassName(nameBean) + "();\n"); for (RestParameter restParameter : restParameters) { if (restParameter.getGenericType() != null && restParameter.getGenericType().equals(nameBean)) { sb.append(" invokeSetter(" + CommandLineUtils.getAsVariableName(CommandLineUtils.getClassName(nameBean)) + ", \"" + restParameter.getName() + - "\", commandOptions." + normaliceNames(getAsCamelCase(restParameter.getParentParamName() + " " + restParameter.getName())) + ")" + + "\", commandOptions." + normalizeNames(getAsCamelCase(restParameter.getParentParamName() + " " + restParameter.getName())) + ")" + ";\n"); } } @@ -438,27 +480,35 @@ private String generateBeans(List restParameters) { } private String getQueryParams(RestEndpoint restEndpoint, CategoryConfig config, String commandName) { - String res = "\n ObjectMap queryParams = new ObjectMap();\n"; + StringBuilder res = new StringBuilder("\n ObjectMap queryParams = new ObjectMap();\n"); boolean enc = false; boolean studyPresent = false; for (RestParameter restParameter : restEndpoint.getParameters()) { if (config.isAvailableSubCommand(restParameter.getName(), commandName)) { if ("query".equals(restParameter.getParam()) && !restParameter.isRequired() && restParameter.isAvailableType()) { enc = true; - if (normaliceNames(restParameter.getName()).equals("study")) { + if (normalizeNames(restParameter.getName()).equals("study")) { studyPresent = true; } if (StringUtils.isNotEmpty(restParameter.getType()) && "string".equalsIgnoreCase(restParameter.getType())) { - res += " queryParams.putIfNotEmpty(\"" + normaliceNames(restParameter.getName()) + "\", commandOptions." - + normaliceNames(restParameter.getName()) + ");\n"; + res.append(" queryParams.putIfNotEmpty(\"") + .append(normalizeNames(restParameter.getName())) + .append("\", commandOptions.") + .append(normalizeNames(restParameter.getName())) + .append(");\n"); } else { if (restParameter.isStringList()) { - res += " queryParams.putIfNotNull(\"" + normaliceNames(restParameter.getName()) + "\", CommandLineUtils" + - ".getListValues(commandOptions." - + normaliceNames(restParameter.getName()) + "));\n"; + res.append(" queryParams.putIfNotNull(\"") + .append(normalizeNames(restParameter.getName())) + .append("\", CommandLineUtils").append(".getListValues(commandOptions.") + .append(normalizeNames(restParameter.getName())) + .append("));\n"); } else { - res += " queryParams.putIfNotNull(\"" + normaliceNames(restParameter.getName()) + "\", commandOptions." - + normaliceNames(restParameter.getName()) + ");\n"; + res.append(" queryParams.putIfNotNull(\"") + .append(normalizeNames(restParameter.getName())) + .append("\", commandOptions.") + .append(normalizeNames(restParameter.getName())) + .append(");\n"); } } } @@ -466,9 +516,9 @@ private String getQueryParams(RestEndpoint restEndpoint, CategoryConfig config, } if (enc) { if (studyPresent) { - res += " if(queryParams.get(\"study\")==null && OpencgaMain.isShellMode()){\n"; - res += " queryParams.putIfNotEmpty(\"study\", sessionManager.getSession().getCurrentStudy());\n"; - res += " }\n"; + res.append(" if (queryParams.get(\"study\") == null && OpencgaMain.isShellMode()) {\n"); + res.append(" queryParams.putIfNotEmpty(\"study\", sessionManager.getSession().getCurrentStudy());\n"); + res.append(" }\n"); } return res + "\n"; } @@ -489,7 +539,7 @@ private String getValidResponseNames(String responseClass) { return responseClass; } - private String normaliceNames(String name) { + private String normalizeNames(String name) { name = getAsCamelCase(name, "\\."); if (invalidNames.containsKey(name)) { name = invalidNames.get(name); diff --git a/opencga-server/src/main/java/org/opencb/opencga/server/generator/writers/cli/OptionsCliRestApiWriter.java b/opencga-server/src/main/java/org/opencb/opencga/server/generator/writers/cli/OptionsCliRestApiWriter.java index fc3f8abaee3..302545e4828 100644 --- a/opencga-server/src/main/java/org/opencb/opencga/server/generator/writers/cli/OptionsCliRestApiWriter.java +++ b/opencga-server/src/main/java/org/opencb/opencga/server/generator/writers/cli/OptionsCliRestApiWriter.java @@ -102,10 +102,8 @@ protected String getClassHeader(String key) { sb.append("\n"); } for (RestEndpoint restEndpoint : restCategory.getEndpoints()) { - //If it is post, it must have parameters in the body, - // therefore it must be different from post or have some primitive value in the body so that we can generate the method String commandName = getMethodName(restCategory, restEndpoint).replaceAll("_", "-"); - if ((!"POST".equals(restEndpoint.getMethod()) || restEndpoint.hasPrimitiveBodyParams(config, commandName)) && restEndpoint.hasParameters()) { + if ("POST".equals(restEndpoint.getMethod()) || restEndpoint.hasParameters()) { if (config.isAvailableCommand(commandName)) { sb.append(" public " + getAsClassName(getAsCamelCase(getMethodName(restCategory, restEndpoint))) + "CommandOptions " + getAsVariableName(getAsCamelCase(getMethodName(restCategory, restEndpoint))) + "CommandOptions;\n"); @@ -125,12 +123,9 @@ protected String getClassHeader(String key) { sb.append(" this.commonCommandOptions = commonCommandOptions;\n"); } for (RestEndpoint restEndpoint : restCategory.getEndpoints()) { - //If it is post, it must have parameters in the body, - // therefore it must be different from post or have some primitive value in the body so that we can generate the method String commandName = getMethodName(restCategory, restEndpoint).replaceAll("_", "-"); - if ((!"POST".equals(restEndpoint.getMethod()) || restEndpoint.hasPrimitiveBodyParams(config, commandName)) && restEndpoint.hasParameters()) { + if ("POST".equals(restEndpoint.getMethod()) || restEndpoint.hasParameters()) { if (config.isAvailableCommand(commandName)) { - sb.append(" this." + getAsVariableName(getAsCamelCase(getMethodName(restCategory, restEndpoint))) + "CommandOptions = " + "new " + getAsClassName(getAsCamelCase(getMethodName(restCategory, restEndpoint))) + "CommandOptions();\n"); @@ -145,15 +140,12 @@ protected String getClassHeader(String key) { @Override protected String getClassMethods(String key) { - RestCategory restCategory = availableCategories.get(key); CategoryConfig config = availableCategoryConfigs.get(key); StringBuilder sb = new StringBuilder(); for (RestEndpoint restEndpoint : restCategory.getEndpoints()) { - //If it is post, it must have parameters in the body, - // therefore it must be different from post or have some primitive value in the body so that we can generate the method String commandName = getMethodName(restCategory, restEndpoint).replaceAll("_", "-"); - if ((!"POST".equals(restEndpoint.getMethod()) || restEndpoint.hasPrimitiveBodyParams(config, commandName)) && restEndpoint.hasParameters()) { + if ("POST".equals(restEndpoint.getMethod()) || restEndpoint.hasParameters()) { if (config.isAvailableCommand(commandName) && !config.isExtendedOptionCommand(commandName)) { sb.append(" @Parameters(commandNames = {\"" + reverseCommandName(commandName) + "\"}, commandDescription =\"" + restEndpoint.getDescription().replaceAll("\"", "'") + "\")\n"); @@ -167,30 +159,28 @@ protected String getClassMethods(String key) { sb.append(" @Parameter(names = {\"--json-file\"}, description = \"File with the body data in JSON format. Note, that using this parameter will ignore all the other parameters.\", required = false, arity = 1)\n"); sb.append(" public String jsonFile;\n"); sb.append(" \n"); - sb.append(" @Parameter(names = {\"--json-view-template\"}, description = \"Show example of file structure for body data.\", help = true, arity = 0)\n"); - sb.append(" public Boolean jsonViewTemplate= false;\n"); + sb.append(" @Parameter(names = {\"--json-data-model\"}, description = \"Show example of file structure for body data.\", help = true, arity = 0)\n"); + sb.append(" public Boolean jsonDataModel = false;\n"); sb.append(" \n"); } - Set variable_names = new HashSet<>(); - for (RestParameter restParameter : restEndpoint.getParameters()) { if (config.isAvailableSubCommand(restParameter.getName(), commandName)) { - if (!"body".equals(normaliceNames(restParameter.getName()))) { - if (restParameter.isAvailableType() && !variable_names.contains(normaliceNames(getAsCamelCase(restParameter.getName())))) { + if (!"body".equals(normalizeNames(restParameter.getName()))) { + if (restParameter.isAvailableType() && !variable_names.contains(normalizeNames(getAsCamelCase(restParameter.getName())))) { sb.append(" @Parameter(names = {" + getShortCuts(restParameter, config) + "}, description = " + "\"" + restParameter.getDescription().replaceAll("\"", "'") + "\", required = " + restParameter.isRequired() + ", arity = 1)\n"); sb.append(" public " + getValidValue(restParameter.getType()) + " " + getVariableName(restParameter) + ";" + " " + "\n"); sb.append(" \n"); - variable_names.add(normaliceNames(getAsCamelCase(restParameter.getName()))); + variable_names.add(normalizeNames(getAsCamelCase(restParameter.getName()))); } } else { - if (restParameter.getData() != null) + if (restParameter.getData() != null) { for (RestParameter bodyRestParameter : restParameter.getData()) { - if (config.isAvailableSubCommand(bodyRestParameter.getName(), commandName) && bodyRestParameter.isAvailableType() && !variable_names.contains(normaliceNames(getAsCamelCase(restParameter.getName())))) { + if (config.isAvailableSubCommand(bodyRestParameter.getName(), commandName) && bodyRestParameter.isAvailableType() && !variable_names.contains(normalizeNames(getAsCamelCase(restParameter.getName())))) { sb.append(" @Parameter(names = {" + getShortCuts(bodyRestParameter, config) + "}, " + "description" + " = \"" + bodyRestParameter.getDescription().replaceAll("\"", "'") + "\", required = " @@ -200,7 +190,7 @@ protected String getClassMethods(String key) { sb.append(" public " + getValidValue(bodyRestParameter.getType()) + " " + getVariableName(bodyRestParameter) + ";\n"); sb.append(" \n"); - variable_names.add(normaliceNames(getAsCamelCase(bodyRestParameter.getName()))); + variable_names.add(normalizeNames(getAsCamelCase(bodyRestParameter.getName()))); } else if (bodyRestParameter.getType().equals("enum")) { sb.append(" @Parameter(names = {" + getShortCuts(bodyRestParameter, config) + "}, " + @@ -214,6 +204,7 @@ protected String getClassMethods(String key) { sb.append(" \n"); } } + } } } } @@ -228,9 +219,9 @@ protected String getClassMethods(String key) { private String getVariableName(RestParameter restParameter) { String res = ""; if (restParameter.isInnerParam()) { - res = normaliceNames(getAsCamelCase(restParameter.getParentParamName() + " " + restParameter.getName())); + res = normalizeNames(getAsCamelCase(restParameter.getParentParamName() + " " + restParameter.getName())); } else { - res = normaliceNames(getAsCamelCase(restParameter.getName())); + res = normalizeNames(getAsCamelCase(restParameter.getName())); } return res; } @@ -239,7 +230,7 @@ private boolean isMandatory(String commandName, String normaliseNames) { return "create".equals(commandName) && "id".equals(normaliseNames); } - private String normaliceNames(String name) { + private String normalizeNames(String name) { name = getAsCamelCase(name, "\\."); if (invalidNames.containsKey(name)) { name = invalidNames.get(name); @@ -274,7 +265,7 @@ public String getStringShortcuts(String parameter, CategoryConfig categoryConfig } } - //category shortcuts + // category shortcuts if (categoryConfig.getShortcuts() != null) { for (Shortcut sc : categoryConfig.getShortcuts()) { if (parameter.equals(sc.getName()) && !scut.contains(sc.getShortcut())) { diff --git a/opencga-server/src/main/java/org/opencb/opencga/server/generator/writers/cli/ParserCliRestApiWriter.java b/opencga-server/src/main/java/org/opencb/opencga/server/generator/writers/cli/ParserCliRestApiWriter.java index 920d967a83c..42a6aea50f8 100644 --- a/opencga-server/src/main/java/org/opencb/opencga/server/generator/writers/cli/ParserCliRestApiWriter.java +++ b/opencga-server/src/main/java/org/opencb/opencga/server/generator/writers/cli/ParserCliRestApiWriter.java @@ -105,10 +105,11 @@ protected String getClassHeader(String key) { + "\", " + getAsVariableName(restCategory.getName()) + "CommandOptions);\n"); sb.append(" JCommander " + getAsVariableName(restCategory.getName()) + "SubCommands = jCommander.getCommands().get(\"" + getCategoryCommandName(restCategory, config) + "\");\n"); - for (RestEndpoint restEndpoint : restCategory.getEndpoints()) { + for (RestEndpoint restEndpoint : restCategory.getEndpoints()) { String commandName = getMethodName(restCategory, restEndpoint).replaceAll("_", "-"); - if ((!"POST".equals(restEndpoint.getMethod()) || restEndpoint.hasPrimitiveBodyParams(config, commandName)) && restEndpoint.hasParameters()) { +// if ((!"POST".equals(restEndpoint.getMethod()) || restEndpoint.hasPrimitiveBodyParams(config, commandName)) && restEndpoint.hasParameters()) { + if ("POST".equals(restEndpoint.getMethod()) || restEndpoint.hasParameters()) { if (config.isAvailableCommand(commandName)) { sb.append(" " + getAsVariableName(restCategory.getName()) + "SubCommands.addCommand(\"" + reverseCommandName(commandName) + "\", " + getAsVariableName(restCategory.getName()) + "CommandOptions." + getAsCamelCase(commandName) + @@ -147,7 +148,6 @@ public void write() { @Override protected String getClassMethods(String key) { StringBuilder sb = new StringBuilder(); - for (RestCategory restCategory : availableCategories.values()) { sb.append(" \n"); sb.append(" public " + getAsClassName(restCategory.getName()) + "CommandOptions get" + getAsClassName(restCategory.getName()) diff --git a/opencga-server/src/main/java/org/opencb/opencga/server/rest/OpenCGAWSServer.java b/opencga-server/src/main/java/org/opencb/opencga/server/rest/OpenCGAWSServer.java index a962da30224..50b3a0885f5 100644 --- a/opencga-server/src/main/java/org/opencb/opencga/server/rest/OpenCGAWSServer.java +++ b/opencga-server/src/main/java/org/opencb/opencga/server/rest/OpenCGAWSServer.java @@ -20,7 +20,6 @@ import com.fasterxml.jackson.databind.ObjectMapper; import com.fasterxml.jackson.databind.ObjectWriter; import com.google.common.base.Splitter; -import org.opencb.opencga.core.tools.annotations.ApiParam; import org.apache.avro.generic.GenericRecord; import org.apache.commons.collections4.CollectionUtils; import org.apache.commons.lang3.NotImplementedException; @@ -57,6 +56,7 @@ import org.opencb.opencga.core.response.OpenCGAResult; import org.opencb.opencga.core.response.RestResponse; import org.opencb.opencga.core.tools.ToolParams; +import org.opencb.opencga.core.tools.annotations.ApiParam; import org.opencb.opencga.server.WebServiceException; import org.opencb.opencga.server.rest.analysis.ClinicalWebService; import org.opencb.opencga.storage.core.StorageEngineFactory; @@ -393,10 +393,6 @@ private void parseParams() throws VersionException { throw new ParamException.PathParamException(new Throwable("Version not valid: '" + apiVersion + "'"), "apiVersion", "v2"); } } - uriInfo.getPathParameters().forEach((key, value) -> { - logger.warn(key); - logger.warn(value.toString()); - }); // Check apiVersion parameter, must be: v1, v2, ... If 'latest' then is converted to appropriate apiVersion. if (apiVersion.equalsIgnoreCase("latest")) { @@ -665,7 +661,11 @@ private Response.Status getResponseStatus(List> list) { if (CollectionUtils.isNotEmpty(openCGAResult.getEvents())) { for (Event event : openCGAResult.getEvents()) { if (event.getType().equals(Event.Type.ERROR)) { - return Response.Status.BAD_REQUEST; + if (event.getMessage().contains("denied")) { + return Response.Status.UNAUTHORIZED; + } else { + return Response.Status.BAD_REQUEST; + } } } } @@ -781,7 +781,7 @@ private void verifyHeaders(HttpHeaders httpHeaders) { } protected List getIdListOrEmpty(String id) throws WebServiceException { - return id == null ? Collections.emptyList() : getIdList(id, true); + return StringUtils.isEmpty(id) ? Collections.emptyList() : getIdList(id, true); } protected List getIdList(String id) throws WebServiceException { diff --git a/opencga-server/src/main/resources/cli-config.yaml b/opencga-server/src/main/resources/cli-config.yaml index 3762afe3aeb..1e7ef104331 100644 --- a/opencga-server/src/main/resources/cli-config.yaml +++ b/opencga-server/src/main/resources/cli-config.yaml @@ -66,8 +66,9 @@ apiConfig: - name: run-templates executorExtended: True optionExtended: True - addedMethods: - - templates-upload + - name: upload-templates + executorExtended: True + optionExtended: True ignore: False key: Study - name: files @@ -75,8 +76,6 @@ apiConfig: key: File executorExtended: True optionExtended: True - addedMethods: - - upload commands: - name: upload executorExtended: True @@ -92,6 +91,14 @@ apiConfig: - name: jobs ignore: False key: Job + executorExtended: True + optionExtended: True + commands: + - name: top + executorExtended: True + optionExtended: True + addedMethods: + - log - name: samples ignore: False key: Sample diff --git a/opencga-server/src/test/java/org/opencb/opencga/server/generator/RestApiParserTest.java b/opencga-server/src/test/java/org/opencb/opencga/server/generator/RestApiParserTest.java index 25fb6cac48b..d80d6da1061 100644 --- a/opencga-server/src/test/java/org/opencb/opencga/server/generator/RestApiParserTest.java +++ b/opencga-server/src/test/java/org/opencb/opencga/server/generator/RestApiParserTest.java @@ -22,6 +22,7 @@ import org.opencb.opencga.server.generator.models.RestEndpoint; import org.opencb.opencga.server.rest.SampleWSServer; import org.opencb.opencga.server.rest.analysis.ClinicalWebService; +import org.opencb.opencga.server.rest.operations.VariantOperationWebService; import java.io.IOException; import java.nio.file.Paths; @@ -42,11 +43,11 @@ public static void beforeClass() throws Exception { @Test public void parse() { - RestApi parse = restApiParser.parse(ClinicalWebService.class); + RestApi parse = restApiParser.parse(VariantOperationWebService.class); List create = parse.getCategories().get(0).getEndpoints().stream() - .filter(endpoint -> endpoint.getPath().contains("create")) + .filter(endpoint -> endpoint.getPath().contains("configure")) .collect(Collectors.toList()); - System.out.println("create = " + create); + System.out.println("configure = " + create); } @Test diff --git a/opencga-server/src/test/java/org/opencb/opencga/server/rest/FileWSServerTest.java b/opencga-server/src/test/java/org/opencb/opencga/server/rest/FileWSServerTest.java index 5c63aa1163f..7734d880fe7 100644 --- a/opencga-server/src/test/java/org/opencb/opencga/server/rest/FileWSServerTest.java +++ b/opencga-server/src/test/java/org/opencb/opencga/server/rest/FileWSServerTest.java @@ -22,7 +22,11 @@ import org.junit.*; import org.junit.rules.ExpectedException; import org.opencb.biodata.models.variant.Variant; -import org.opencb.commons.datastore.core.*; +import org.opencb.commons.datastore.core.ObjectMap; +import org.opencb.commons.datastore.core.Query; +import org.opencb.commons.datastore.core.QueryOptions; +import org.opencb.commons.datastore.core.QueryResponse; +import org.opencb.opencga.TestParamConstants; import org.opencb.opencga.catalog.db.api.FileDBAdaptor; import org.opencb.opencga.catalog.exceptions.CatalogException; import org.opencb.opencga.catalog.managers.CatalogManagerTest; @@ -83,7 +87,7 @@ static public void shutdownServer() throws Exception { @Before public void init() throws Exception { webTarget = serverTestUtils.getWebTarget(); - sessionId = OpenCGAWSServer.catalogManager.getUserManager().login("user", CatalogManagerTest.PASSWORD).getToken(); + sessionId = OpenCGAWSServer.catalogManager.getUserManager().login("user", TestParamConstants.PASSWORD).getToken(); if (ROOT_DIR.toFile().exists()) { IOUtils.deleteDirectory(ROOT_DIR); diff --git a/opencga-server/src/test/java/org/opencb/opencga/server/rest/IndividualWSServerTest.java b/opencga-server/src/test/java/org/opencb/opencga/server/rest/IndividualWSServerTest.java index 6af0f35743e..e75f31bf250 100644 --- a/opencga-server/src/test/java/org/opencb/opencga/server/rest/IndividualWSServerTest.java +++ b/opencga-server/src/test/java/org/opencb/opencga/server/rest/IndividualWSServerTest.java @@ -26,13 +26,13 @@ import org.opencb.commons.datastore.core.ObjectMap; import org.opencb.commons.datastore.core.QueryOptions; import org.opencb.commons.datastore.core.QueryResponse; +import org.opencb.opencga.TestParamConstants; import org.opencb.opencga.catalog.exceptions.CatalogException; -import org.opencb.opencga.catalog.managers.CatalogManagerTest; +import org.opencb.opencga.core.models.common.GenericRecordAvroJsonMixin; import org.opencb.opencga.core.models.individual.Individual; import org.opencb.opencga.core.models.individual.IndividualReferenceParam; import org.opencb.opencga.core.models.individual.IndividualUpdateParams; import org.opencb.opencga.core.models.sample.Sample; -import org.opencb.opencga.core.models.common.GenericRecordAvroJsonMixin; import org.opencb.opencga.storage.core.variant.io.json.mixin.GenotypeJsonMixin; import org.opencb.opencga.storage.core.variant.io.json.mixin.VariantStatsJsonMixin; @@ -88,7 +88,7 @@ static public void shutdownServer() throws Exception { @Before public void init() throws Exception { webTarget = serverTestUtils.getWebTarget(); - sessionId = OpenCGAWSServer.catalogManager.getUserManager().login("user", CatalogManagerTest.PASSWORD).getToken(); + sessionId = OpenCGAWSServer.catalogManager.getUserManager().login("user", TestParamConstants.PASSWORD).getToken(); in1 = OpenCGAWSServer.catalogManager.getIndividualManager().create(studyId, new Individual().setId("in1"), null, sessionId).first().getUid(); in2 = OpenCGAWSServer.catalogManager.getIndividualManager().create(studyId, new Individual().setId("in2"), null, sessionId).first() diff --git a/opencga-server/src/test/java/org/opencb/opencga/server/rest/JobWSServerTest.java b/opencga-server/src/test/java/org/opencb/opencga/server/rest/JobWSServerTest.java index 8e86c879f96..cb8ba2151dc 100644 --- a/opencga-server/src/test/java/org/opencb/opencga/server/rest/JobWSServerTest.java +++ b/opencga-server/src/test/java/org/opencb/opencga/server/rest/JobWSServerTest.java @@ -18,7 +18,7 @@ import org.junit.*; import org.junit.rules.ExpectedException; -import org.opencb.opencga.catalog.managers.CatalogManagerTest; +import org.opencb.opencga.TestParamConstants; import javax.ws.rs.client.WebTarget; @@ -51,7 +51,7 @@ static public void shutdownServer() throws Exception { public void init() throws Exception { // serverTestUtils.setUp(); webTarget = serverTestUtils.getWebTarget(); - sessionId = OpenCGAWSServer.catalogManager.getUserManager().login("user", CatalogManagerTest.PASSWORD).getToken(); + sessionId = OpenCGAWSServer.catalogManager.getUserManager().login("user", TestParamConstants.PASSWORD).getToken(); } @After diff --git a/opencga-server/src/test/java/org/opencb/opencga/server/rest/SampleWSServerTest.java b/opencga-server/src/test/java/org/opencb/opencga/server/rest/SampleWSServerTest.java index d4236d11346..a567fc7c373 100644 --- a/opencga-server/src/test/java/org/opencb/opencga/server/rest/SampleWSServerTest.java +++ b/opencga-server/src/test/java/org/opencb/opencga/server/rest/SampleWSServerTest.java @@ -19,8 +19,8 @@ import org.junit.*; import org.junit.rules.ExpectedException; import org.opencb.commons.datastore.core.QueryResult; +import org.opencb.opencga.TestParamConstants; import org.opencb.opencga.catalog.db.api.SampleDBAdaptor; -import org.opencb.opencga.catalog.managers.CatalogManagerTest; import org.opencb.opencga.core.models.cohort.Cohort; import org.opencb.opencga.core.models.common.Enums; import org.opencb.opencga.core.models.individual.Individual; @@ -62,7 +62,7 @@ static public void shutdownServer() throws Exception { public void init() throws Exception { // serverTestUtils.setUp(); webTarget = serverTestUtils.getWebTarget(); - sessionId = OpenCGAWSServer.catalogManager.getUserManager().login("user", CatalogManagerTest.PASSWORD).getToken(); + sessionId = OpenCGAWSServer.catalogManager.getUserManager().login("user", TestParamConstants.PASSWORD).getToken(); in1 = OpenCGAWSServer.catalogManager.getIndividualManager().create(studyId, new Individual().setId("in1"), null, sessionId).first().getUid(); s1 = OpenCGAWSServer.catalogManager.getSampleManager().create(studyId, new Sample().setId("s1"), null, sessionId).first().getUid(); diff --git a/opencga-storage/opencga-storage-app/pom.xml b/opencga-storage/opencga-storage-app/pom.xml index b93936b545d..8fcb0cfc446 100644 --- a/opencga-storage/opencga-storage-app/pom.xml +++ b/opencga-storage/opencga-storage-app/pom.xml @@ -22,7 +22,7 @@ org.opencb.opencga opencga-storage - 2.2.0-rc2 + 2.2.0 ../pom.xml diff --git a/opencga-storage/opencga-storage-benchmark/pom.xml b/opencga-storage/opencga-storage-benchmark/pom.xml index 42045eee3cd..da96825475f 100644 --- a/opencga-storage/opencga-storage-benchmark/pom.xml +++ b/opencga-storage/opencga-storage-benchmark/pom.xml @@ -21,7 +21,7 @@ opencga-storage org.opencb.opencga - 2.2.0-rc2 + 2.2.0 4.0.0 diff --git a/opencga-storage/opencga-storage-core/pom.xml b/opencga-storage/opencga-storage-core/pom.xml index b279ec24d26..77f29996f81 100644 --- a/opencga-storage/opencga-storage-core/pom.xml +++ b/opencga-storage/opencga-storage-core/pom.xml @@ -25,7 +25,7 @@ org.opencb.opencga opencga-storage - 2.2.0-rc2 + 2.2.0 ../pom.xml diff --git a/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/metadata/VariantStorageMetadataManager.java b/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/metadata/VariantStorageMetadataManager.java index 8aede6c989c..287ab7dd00d 100644 --- a/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/metadata/VariantStorageMetadataManager.java +++ b/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/metadata/VariantStorageMetadataManager.java @@ -208,20 +208,33 @@ public StudyMetadata createStudy(String studyName) throws StorageEngineException } public StudyMetadata.SampleIndexConfigurationVersioned addSampleIndexConfiguration( - String study, SampleIndexConfiguration configuration) throws StorageEngineException { + String study, SampleIndexConfiguration configuration, boolean staging) throws StorageEngineException { Integer idOrNull = getStudyIdOrNull(study); + int studyId; if (idOrNull == null) { - createStudy(study); + studyId = createStudy(study).getId(); + } else { + studyId = idOrNull; } - return updateStudyMetadata(study, studyMetadata -> { + StudyMetadata.SampleIndexConfigurationVersioned.Status status = staging + ? StudyMetadata.SampleIndexConfigurationVersioned.Status.STAGING + : StudyMetadata.SampleIndexConfigurationVersioned.Status.ACTIVE; + return updateStudyMetadata(studyId, studyMetadata -> { List configurations = studyMetadata.getSampleIndexConfigurations(); if (configurations == null || configurations.isEmpty()) { - configurations = new ArrayList<>(1); + configurations = new ArrayList<>(2); + configurations.add(new StudyMetadata.SampleIndexConfigurationVersioned( + SampleIndexConfiguration.defaultConfiguration(), + StudyMetadata.DEFAULT_SAMPLE_INDEX_VERSION, + Date.from(Instant.now()), StudyMetadata.SampleIndexConfigurationVersioned.Status.ACTIVE)); studyMetadata.setSampleIndexConfigurations(configurations); } int version = studyMetadata.getSampleIndexConfigurationLatest().getVersion() + 1; - configurations.add(new StudyMetadata.SampleIndexConfigurationVersioned(configuration, version, Date.from(Instant.now()))); - return studyMetadata; + configurations.add(new StudyMetadata.SampleIndexConfigurationVersioned( + configuration, + version, + Date.from(Instant.now()), + status)); }).getSampleIndexConfigurationLatest(); } @@ -411,8 +424,8 @@ public Integer getStudyId(Object studyObj, boolean skipNegated, Map) studyObj).getStudyId(); - } else if (studyObj instanceof Integer) { - studyId = ((Integer) studyObj); + } else if (studyObj instanceof Number) { + studyId = ((Number) studyObj).intValue(); } else { String studyName = studyObj.toString(); if (isNegated(studyName)) { //Skip negated studies @@ -756,7 +769,9 @@ public void addIndexedFiles(int studyId, List fileIds) throws StorageEn } for (Integer sample : samples) { if (!isSampleIndexed(studyId, sample)) { - updateSampleMetadata(studyId, sample, sampleMetadata -> sampleMetadata.setIndexStatus(TaskMetadata.Status.READY)); + updateSampleMetadata(studyId, sample, sampleMetadata -> { + sampleMetadata.setIndexStatus(TaskMetadata.Status.READY); + }); } } @@ -793,7 +808,6 @@ public void removeIndexedFiles(int studyId, Collection fileIds) throws if (!indexed) { sampleMetadata.setIndexStatus(TaskMetadata.Status.NONE); } - return sampleMetadata; }); } fileDBAdaptor.removeIndexedFiles(studyId, fileIds); @@ -812,13 +826,13 @@ public void unsecureUpdateSampleMetadata(int studyId, SampleMetadata sample) { sampleDBAdaptor.updateSampleMetadata(studyId, sample, null); } - public SampleMetadata updateSampleMetadata(int studyId, int sampleId, UpdateFunction update) + public SampleMetadata updateSampleMetadata(int studyId, int sampleId, UpdateConsumer consumer) throws E, StorageEngineException { getSampleName(studyId, sampleId); // Check sample exists Lock lock = sampleDBAdaptor.lock(studyId, sampleId, lockDuration, lockTimeout); try { SampleMetadata sample = getSampleMetadata(studyId, sampleId); - sample = update.update(sample); + sample = consumer.toFunction().update(sample); lock.checkLocked(); unsecureUpdateSampleMetadata(studyId, sample); return sample; @@ -827,6 +841,10 @@ public SampleMetadata updateSampleMetadata(int studyId, in } } + public Iterable sampleMetadataIterable(int studyId) { + return () -> sampleMetadataIterator(studyId); + } + public Iterator sampleMetadataIterator(int studyId) { return sampleDBAdaptor.sampleMetadataIterator(studyId); } @@ -1116,9 +1134,13 @@ private CohortMetadata updateCohortSamples(int studyId, String cohortName, Colle for (Integer sampleId : sampleIds) { Integer finalCohortId = cohortId; if (secondaryIndexCohort) { - updateSampleMetadata(studyId, sampleId, sampleMetadata -> sampleMetadata.addSecondaryIndexCohort(finalCohortId)); + updateSampleMetadata(studyId, sampleId, sampleMetadata -> { + sampleMetadata.addSecondaryIndexCohort(finalCohortId); + }); } else { - updateSampleMetadata(studyId, sampleId, sampleMetadata -> sampleMetadata.addCohort(finalCohortId)); + updateSampleMetadata(studyId, sampleId, sampleMetadata -> { + sampleMetadata.addCohort(finalCohortId); + }); } } @@ -1133,12 +1155,10 @@ private CohortMetadata updateCohortSamples(int studyId, String cohortName, Colle if (secondaryIndexCohort) { updateSampleMetadata(studyId, sampleFromCohort, sampleMetadata -> { sampleMetadata.getSecondaryIndexCohorts().remove(finalCohortId); - return sampleMetadata; }); } else { updateSampleMetadata(studyId, sampleFromCohort, sampleMetadata -> { sampleMetadata.getCohorts().remove(finalCohortId); - return sampleMetadata; }); } } @@ -1484,7 +1504,6 @@ protected Integer registerSample(int studyId, Integer fileId, String sample) thr if (!sampleMetadata.getFiles().contains(fileId)) { updateSampleMetadata(studyId, sampleId, s -> { s.getFiles().add(fileId); - return s; }); } } diff --git a/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/metadata/models/SampleMetadata.java b/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/metadata/models/SampleMetadata.java index 106b0ebcb28..ac2b5c7d8c6 100644 --- a/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/metadata/models/SampleMetadata.java +++ b/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/metadata/models/SampleMetadata.java @@ -3,6 +3,7 @@ import com.fasterxml.jackson.annotation.JsonIgnore; import org.apache.commons.lang3.builder.ToStringBuilder; import org.opencb.biodata.models.variant.metadata.SampleVariantStats; +import org.opencb.commons.datastore.core.ObjectMap; import org.opencb.opencga.storage.core.variant.VariantStorageEngine; import java.util.ArrayList; @@ -31,14 +32,13 @@ public class SampleMetadata extends StudyResourceMetadata { private SampleVariantStats stats; - private static final String SAMPLE_INDEX_STATUS = "sampleIndexGenotypes"; - private static final String SAMPLE_INDEX_VERSION = "sampleIndexGenotypesVersion"; - private static final String SAMPLE_INDEX_ANNOTATION_STATUS = "sampleIndexAnnotation"; - private static final String SAMPLE_INDEX_ANNOTATION_VERSION = "sampleIndexAnnotationVersion"; - - @Deprecated // Deprecated to avoid confusion with actual "SAMPLE_INDEX_STATUS" - private static final String SAMPLE_INDEX_ANNOTATION_STATUS_OLD = "sampleIndex"; - + private static final String SAMPLE_INDEX_STATUS_PREFIX = "sampleIndexGenotypes_"; + private static final String SAMPLE_INDEX_VERSIONS = "sampleIndexGenotypesReadyVersions"; + private static final String SAMPLE_INDEX_ANNOTATION_STATUS_PREFIX = "sampleIndexAnnotation_"; + private static final String SAMPLE_INDEX_ANNOTATION_VERSIONS = "sampleIndexAnnotationReadyVersions"; + private static final String FAMILY_INDEX_STATUS_PREFIX = "familyIndex_"; + private static final String FAMILY_INDEX_VERSIONS = "familyIndexReadyVersions"; + private static final String FAMILY_INDEX_DEFINED = "familyIndexDefined"; public SampleMetadata() { files = new ArrayList<>(1); @@ -165,54 +165,47 @@ public SampleMetadata setAnnotationStatus(TaskMetadata.Status annotationStatus) } @JsonIgnore - public TaskMetadata.Status getSampleIndexAnnotationStatus(int latestSampleIndexVersion) { - TaskMetadata.Status status = this.getStatus(SAMPLE_INDEX_ANNOTATION_STATUS, null); - if (status == null) { - // The status name was renamed. In case of missing value (null), check for the deprecated value. - status = this.getStatus(SAMPLE_INDEX_ANNOTATION_STATUS_OLD); - } - if (status == TaskMetadata.Status.READY) { - int actualSampleIndexVersion = this.getAttributes().getInt(SAMPLE_INDEX_ANNOTATION_VERSION, 1); - if (actualSampleIndexVersion != latestSampleIndexVersion) { -// logger.debug("Sample index annotation version outdated. Actual : " + actualSampleIndexVersion -// + " , expected : " + latestSampleIndexVersion); - status = TaskMetadata.Status.NONE; - } - } - return status; + public TaskMetadata.Status getSampleIndexStatus(int sampleIndexVersion) { + return this.getStatus(SAMPLE_INDEX_STATUS_PREFIX + sampleIndexVersion); } @JsonIgnore - public SampleMetadata setSampleIndexAnnotationStatus(TaskMetadata.Status status, int version) { - // Remove deprecated value. - this.getStatus().remove(SAMPLE_INDEX_ANNOTATION_STATUS_OLD); - this.setStatus(SAMPLE_INDEX_ANNOTATION_STATUS, status); - this.getAttributes().put(SAMPLE_INDEX_ANNOTATION_VERSION, version); - return this; + public List getSampleIndexVersions() { + return this.getAttributes().getAsIntegerList(SAMPLE_INDEX_VERSIONS); } @JsonIgnore - public TaskMetadata.Status getSampleIndexStatus(int latestSampleIndexVersion) { - TaskMetadata.Status status = this.getStatus(SAMPLE_INDEX_STATUS, null); - if (status == null) { - // This is a new status. In case of missing value (null), assume it's READY - status = TaskMetadata.Status.READY; - } - if (status == TaskMetadata.Status.READY) { - int actualSampleIndexVersion = this.getAttributes().getInt(SAMPLE_INDEX_VERSION, 1); - if (actualSampleIndexVersion != latestSampleIndexVersion) { -// logger.debug("Sample index version outdated. Actual : " + actualSampleIndexVersion -// + " , expected : " + latestSampleIndexVersion); - status = TaskMetadata.Status.NONE; - } - } - return status; + public Integer getSampleIndexVersion() { + return getLatestVersion(SAMPLE_INDEX_VERSIONS); } @JsonIgnore public SampleMetadata setSampleIndexStatus(TaskMetadata.Status status, int version) { - this.getAttributes().put(SAMPLE_INDEX_VERSION, version); - return this.setStatus(SAMPLE_INDEX_STATUS, status); + registerVersion(SAMPLE_INDEX_VERSIONS, status, version); + this.setStatus(SAMPLE_INDEX_STATUS_PREFIX + version, status); + return this; + } + + @JsonIgnore + public TaskMetadata.Status getSampleIndexAnnotationStatus(int sampleIndexVersion) { + return this.getStatus(SAMPLE_INDEX_ANNOTATION_STATUS_PREFIX + sampleIndexVersion); + } + + @JsonIgnore + public List getSampleIndexAnnotationVersions() { + return this.getAttributes().getAsIntegerList(SAMPLE_INDEX_ANNOTATION_VERSIONS); + } + + @JsonIgnore + public Integer getSampleIndexAnnotationVersion() { + return getLatestVersion(SAMPLE_INDEX_ANNOTATION_VERSIONS); + } + + @JsonIgnore + public SampleMetadata setSampleIndexAnnotationStatus(TaskMetadata.Status status, int version) { + registerVersion(SAMPLE_INDEX_ANNOTATION_VERSIONS, status, version); + this.setStatus(SAMPLE_INDEX_ANNOTATION_STATUS_PREFIX + version, status); + return this; } @JsonIgnore @@ -225,14 +218,71 @@ public TaskMetadata.Status getMendelianErrorStatus() { return getStatus("mendelian_error"); } + /** + * @return if the family index was enabled on this sample. It might not be READY. + */ + @JsonIgnore + public boolean isFamilyIndexDefined() { + return getAttributes().getBoolean(FAMILY_INDEX_DEFINED, false); + } + + @JsonIgnore + public void setFamilyIndexDefined(boolean defined) { + getAttributes().put(FAMILY_INDEX_DEFINED, defined); + } + + @JsonIgnore + public SampleMetadata setFamilyIndexStatus(TaskMetadata.Status status, int version) { + if (status == TaskMetadata.Status.READY) { + // If any family index is ready, set "family index defined" + setFamilyIndexDefined(true); + } + registerVersion(FAMILY_INDEX_VERSIONS, status, version); + return setStatus(FAMILY_INDEX_STATUS_PREFIX + version, status); + } + @JsonIgnore - public SampleMetadata setFamilyIndexStatus(TaskMetadata.Status familyIndexStatus) { - return setStatus("family_index", familyIndexStatus); + public List getFamilyIndexVersions() { + return getAttributes().getAsIntegerList(FAMILY_INDEX_VERSIONS); } @JsonIgnore - public TaskMetadata.Status getFamilyIndexStatus() { - return getStatus("family_index"); + public Integer getFamilyIndexVersion() { + return getLatestVersion(FAMILY_INDEX_VERSIONS); + } + + @JsonIgnore + public TaskMetadata.Status getFamilyIndexStatus(int version) { + return getStatus(FAMILY_INDEX_STATUS_PREFIX + version); + } + + private void registerVersion(String versionsKey, TaskMetadata.Status status, int version) { + ObjectMap attributes = this.getAttributes(); + List versions = attributes.getAsIntegerList(versionsKey); + if (status == TaskMetadata.Status.READY) { + if (!versions.contains(version)) { + versions = new ArrayList<>(versions); + versions.add(version); + versions.sort(Integer::compareTo); + attributes.put(versionsKey, versions); + } + } else { + if (versions.contains(version)) { + versions = new ArrayList<>(versions); + versions.remove(version); + versions.sort(Integer::compareTo); + attributes.put(versionsKey, versions); + } + } + } + + private Integer getLatestVersion(String key) { + List versions = this.getAttributes().getAsIntegerList(key); + if (versions.isEmpty()) { + return null; + } else { + return versions.get(versions.size() - 1); + } } @Override @@ -244,7 +294,8 @@ public String toString() { .append("status", getStatus()) .append("files", files) .append("stats", stats) -// .append("cohorts", cohorts) + .append("cohorts", cohorts) + .append("attributes", getAttributes().toJson()) .toString(); } } diff --git a/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/metadata/models/StudyMetadata.java b/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/metadata/models/StudyMetadata.java index 90a3841aca1..8968fb5f10a 100644 --- a/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/metadata/models/StudyMetadata.java +++ b/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/metadata/models/StudyMetadata.java @@ -24,13 +24,14 @@ public class StudyMetadata { public static final String UNKNOWN_HEADER_ATTRIBUTE = "."; + public static final int DEFAULT_SAMPLE_INDEX_VERSION = 1; private int id; private String name; private Aggregation aggregation; private Long timeStamp; private VariantFileHeader variantHeader; private List variantScores; - private List sampleIndexConfigurations; + private List sampleIndexConfigurations = Collections.emptyList(); private ObjectMap attributes; @@ -135,13 +136,23 @@ public StudyMetadata setVariantScores(List variantScores) } public SampleIndexConfigurationVersioned getSampleIndexConfigurationLatest() { + return getSampleIndexConfigurationLatest(true); + } + + public SampleIndexConfigurationVersioned getSampleIndexConfigurationLatest(boolean includeStagingSchemas) { if (sampleIndexConfigurations == null || sampleIndexConfigurations.isEmpty()) { - return new SampleIndexConfigurationVersioned(SampleIndexConfiguration.defaultConfiguration(), 1, Date.from(Instant.now())); + return new SampleIndexConfigurationVersioned(SampleIndexConfiguration.defaultConfiguration(), + DEFAULT_SAMPLE_INDEX_VERSION, Date.from(Instant.now()), SampleIndexConfigurationVersioned.Status.ACTIVE); } else { - SampleIndexConfigurationVersioned conf = sampleIndexConfigurations.get(0); + SampleIndexConfigurationVersioned conf = null; for (SampleIndexConfigurationVersioned thisConf : sampleIndexConfigurations) { - if (thisConf.getVersion() > conf.getVersion()) { - conf = thisConf; + if (thisConf.getStatus() == SampleIndexConfigurationVersioned.Status.ACTIVE + || includeStagingSchemas && thisConf.getStatus() == SampleIndexConfigurationVersioned.Status.STAGING) { + if (conf == null) { + conf = thisConf; + } else if (thisConf.getVersion() > conf.getVersion()) { + conf = thisConf; + } } } return conf; @@ -152,6 +163,24 @@ public List getSampleIndexConfigurations() { return sampleIndexConfigurations; } + public SampleIndexConfigurationVersioned getSampleIndexConfiguration(int version) { + if (sampleIndexConfigurations == null || sampleIndexConfigurations.isEmpty()) { + if (version == DEFAULT_SAMPLE_INDEX_VERSION) { + return new SampleIndexConfigurationVersioned(SampleIndexConfiguration.defaultConfiguration(), + DEFAULT_SAMPLE_INDEX_VERSION, Date.from(Instant.now()), SampleIndexConfigurationVersioned.Status.ACTIVE); + } else { + return null; + } + } else { + for (SampleIndexConfigurationVersioned v : sampleIndexConfigurations) { + if (v.getVersion() == version) { + return v; + } + } + return null; + } + } + public StudyMetadata setSampleIndexConfigurations(List sampleIndexConfigurations) { this.sampleIndexConfigurations = sampleIndexConfigurations; return this; @@ -224,16 +253,24 @@ public static class SampleIndexConfigurationVersioned { private SampleIndexConfiguration configuration; private int version; private Date date; + private Status status; // private int numSamples; + public enum Status { + STAGING, // Index being built. Not ready. Not to be used. + ACTIVE, // Index ready to be used (if present) + DEPRECATED, // Index marked to be removed. + REMOVED // Index no longer exists. + } public SampleIndexConfigurationVersioned() { } - public SampleIndexConfigurationVersioned(SampleIndexConfiguration configuration, int version, Date date) { + public SampleIndexConfigurationVersioned(SampleIndexConfiguration configuration, int version, Date date, Status status) { this.configuration = configuration; this.version = version; this.date = date; + this.status = status; } public SampleIndexConfiguration getConfiguration() { @@ -263,7 +300,16 @@ public SampleIndexConfigurationVersioned setDate(Date date) { return this; } -// public int getNumSamples() { + public Status getStatus() { + return status; + } + + public SampleIndexConfigurationVersioned setStatus(Status status) { + this.status = status; + return this; + } + + // public int getNumSamples() { // return numSamples; // } // diff --git a/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/metadata/models/StudyResourceMetadata.java b/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/metadata/models/StudyResourceMetadata.java index 3c0ac800d12..01dfb82d2f7 100644 --- a/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/metadata/models/StudyResourceMetadata.java +++ b/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/metadata/models/StudyResourceMetadata.java @@ -98,6 +98,11 @@ public T setAttributes(ObjectMap attributes) { return getThis(); } + public T putAttribute(String key, Object value) { + getAttributes().put(key, value); + return getThis(); + } + @SuppressWarnings("unchecked") public T getThis() { return (T) this; diff --git a/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/variant/VariantStorageEngine.java b/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/variant/VariantStorageEngine.java index c59e94a7561..323ea94088a 100644 --- a/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/variant/VariantStorageEngine.java +++ b/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/variant/VariantStorageEngine.java @@ -568,6 +568,39 @@ public DataResult> familyIndex(String study, List> tri throw new UnsupportedOperationException("Unsupported familyIndex"); } + public DataResult> familyIndexUpdate(String study, ObjectMap options) throws StorageEngineException { + StudyMetadata studyMetadata = getMetadataManager().getStudyMetadata(study); + int studyId = studyMetadata.getId(); + int version = studyMetadata.getSampleIndexConfigurationLatest().getVersion(); + List> trios = new LinkedList<>(); + for (SampleMetadata sampleMetadata : getMetadataManager().sampleMetadataIterable(studyId)) { + if (sampleMetadata.isFamilyIndexDefined()) { + if (sampleMetadata.getFamilyIndexStatus(version) != TaskMetadata.Status.READY) { + // This sample's family index needs to be updated + String father; + if (sampleMetadata.getFather() == null) { + father = "-"; + } else { + father = getMetadataManager().getSampleName(studyId, sampleMetadata.getFather()); + } + String mother; + if (sampleMetadata.getMother() == null) { + mother = "-"; + } else { + mother = getMetadataManager().getSampleName(studyId, sampleMetadata.getMother()); + } + trios.add(Arrays.asList(father, mother, sampleMetadata.getName())); + } + } + } + if (trios.isEmpty()) { + logger.info("Nothing to do!"); + return new DataResult>().setEvents(Collections.singletonList(new Event(Event.Type.INFO, "Nothing to do"))); + } else { + return familyIndex(study, trios, options); + } + } + /** * Provide a new VariantStatisticsManager for creating and loading statistics. * @@ -878,12 +911,19 @@ protected void postRemoveFiles(String study, List fileIds, List { s.setIndexStatus(TaskMetadata.Status.NONE); + for (Integer v : s.getSampleIndexVersions()) { + s.setSampleIndexStatus(TaskMetadata.Status.NONE, v); + } + for (Integer v : s.getSampleIndexAnnotationVersions()) { + s.setSampleIndexAnnotationStatus(TaskMetadata.Status.NONE, v); + } + for (Integer v : s.getFamilyIndexVersions()) { + s.setFamilyIndexStatus(TaskMetadata.Status.NONE, v); + } s.setAnnotationStatus(TaskMetadata.Status.NONE); - s.setFamilyIndexStatus(TaskMetadata.Status.NONE); s.setMendelianErrorStatus(TaskMetadata.Status.NONE); s.setFiles(Collections.emptyList()); s.setCohorts(Collections.emptySet()); - return s; }); } diff --git a/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/variant/adaptors/VariantQueryException.java b/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/variant/adaptors/VariantQueryException.java index 9cfe31b354c..9fa8cff4188 100644 --- a/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/variant/adaptors/VariantQueryException.java +++ b/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/variant/adaptors/VariantQueryException.java @@ -189,9 +189,10 @@ private static VariantQueryException missingStudyFor0(String resource, String va + (availableStudies == null || availableStudies.isEmpty() ? "" : " Available studies: " + availableStudies)); } -// public static VariantQueryException missingStudy() { -// -// } + public static VariantQueryException missingStudy(Collection availableStudies) { + return new VariantQueryException("Missing study param. Please, specify one study." + + (availableStudies == null || availableStudies.isEmpty() ? "" : " Available studies: " + availableStudies)); + } public static VariantQueryException sampleNotFound(Object sample, Object study) { return new VariantQueryException("Sample '" + sample + "' not found in study '" + study + "'"); diff --git a/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/variant/annotation/DefaultVariantAnnotationManager.java b/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/variant/annotation/DefaultVariantAnnotationManager.java index e412c3d511f..3c768eda7aa 100644 --- a/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/variant/annotation/DefaultVariantAnnotationManager.java +++ b/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/variant/annotation/DefaultVariantAnnotationManager.java @@ -508,7 +508,6 @@ protected void postAnnotate(Query query, boolean doCreate, boolean doLoad, Objec for (Integer sampleId : sampleIds) { metadataManager.updateSampleMetadata(studyId, sampleId, sampleMetadata -> { sampleMetadata.setAnnotationStatus(TaskMetadata.Status.READY); - return sampleMetadata; }); } } diff --git a/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/variant/annotation/VariantAnnotationManager.java b/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/variant/annotation/VariantAnnotationManager.java index 92f8967e53e..8e009ab9a3f 100644 --- a/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/variant/annotation/VariantAnnotationManager.java +++ b/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/variant/annotation/VariantAnnotationManager.java @@ -17,6 +17,7 @@ package org.opencb.opencga.storage.core.variant.annotation; import org.apache.commons.collections4.CollectionUtils; +import org.apache.commons.lang3.StringUtils; import org.opencb.commons.datastore.core.ObjectMap; import org.opencb.commons.datastore.core.Query; import org.opencb.opencga.storage.core.exceptions.StorageEngineException; @@ -93,8 +94,10 @@ protected final VariantAnnotationMetadata checkCurrentAnnotation(ProjectMetadata // Check using same annotator and same source version VariantAnnotatorProgram currentAnnotator = current.getAnnotator(); if (currentAnnotator != null && !currentAnnotator.equals(newAnnotator)) { + String currentVersion = removePatchFromVersion(currentAnnotator.getVersion()); + String newVersion = removePatchFromVersion(newAnnotator.getVersion()); if (!currentAnnotator.getName().equals(newAnnotator.getName()) - || !currentAnnotator.getVersion().equals(newAnnotator.getVersion())) { + || !currentVersion.equals(newVersion)) { String msg = "Using a different annotator! " + "Existing annotation calculated with " + currentAnnotator.toString() + ", attempting to annotate with " + newAnnotator.toString(); @@ -104,7 +107,7 @@ protected final VariantAnnotationMetadata checkCurrentAnnotation(ProjectMetadata throw new VariantAnnotatorException(msg); } } else if (!currentAnnotator.getCommit().equals(newAnnotator.getCommit())) { - String msg = "Using a different commit for annotating variants. " + String msg = "Using a different patch version for annotating variants. " + "Existing annotation calculated with " + currentAnnotator.toString() + ", attempting to annotate with " + newAnnotator.toString(); if (overwrite) { @@ -132,6 +135,14 @@ protected final VariantAnnotationMetadata checkCurrentAnnotation(ProjectMetadata return current; } + private static String removePatchFromVersion(String version) { + String[] split = StringUtils.split(version, '.'); + if (split.length <= 1) { + return version; + } + return split[0] + "." + split[1]; + } + private boolean sameSourceVersion(List newSourceVersion, List currentSourceVersion) { if (currentSourceVersion.size() != newSourceVersion.size()) { return false; diff --git a/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/variant/query/VariantQueryParser.java b/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/variant/query/VariantQueryParser.java index 38df6d1a012..762de68ffe4 100644 --- a/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/variant/query/VariantQueryParser.java +++ b/opencga-storage/opencga-storage-core/src/main/java/org/opencb/opencga/storage/core/variant/query/VariantQueryParser.java @@ -595,7 +595,7 @@ protected void preProcessStudyParams(Query query, QueryOptions options) { throw VariantQueryException.sampleNotFound(sample, defaultStudy.getName()); } SampleMetadata sampleMetadata = metadataManager.getSampleMetadata(defaultStudy.getId(), sampleId); - if (!TaskMetadata.Status.READY.equals(sampleMetadata.getMendelianErrorStatus())) { + if (TaskMetadata.Status.READY != sampleMetadata.getMendelianErrorStatus()) { throw VariantQueryException.malformedParam(param, "Sample \"" + sampleMetadata.getName() + "\" does not have the Mendelian Errors precomputed yet"); } diff --git a/opencga-storage/opencga-storage-core/src/test/java/org/opencb/opencga/storage/core/variant/VariantStorageEngineSVTest.java b/opencga-storage/opencga-storage-core/src/test/java/org/opencb/opencga/storage/core/variant/VariantStorageEngineSVTest.java index ea317b35231..b959f34956b 100644 --- a/opencga-storage/opencga-storage-core/src/test/java/org/opencb/opencga/storage/core/variant/VariantStorageEngineSVTest.java +++ b/opencga-storage/opencga-storage-core/src/test/java/org/opencb/opencga/storage/core/variant/VariantStorageEngineSVTest.java @@ -73,7 +73,7 @@ public void checkAllAnnotated() throws Exception { @Test public void checkCount() throws Exception { - int expected = 24 + 7; + int expected = 24 + 7 + 1; int count = variantStorageEngine.getDBAdaptor().count().first().intValue(); assertEquals(expected, count); } diff --git a/opencga-storage/opencga-storage-core/src/test/java/org/opencb/opencga/storage/core/variant/adaptors/VariantDBAdaptorMultiFileTest.java b/opencga-storage/opencga-storage-core/src/test/java/org/opencb/opencga/storage/core/variant/adaptors/VariantDBAdaptorMultiFileTest.java index 123f70c0e8c..e24805b7403 100644 --- a/opencga-storage/opencga-storage-core/src/test/java/org/opencb/opencga/storage/core/variant/adaptors/VariantDBAdaptorMultiFileTest.java +++ b/opencga-storage/opencga-storage-core/src/test/java/org/opencb/opencga/storage/core/variant/adaptors/VariantDBAdaptorMultiFileTest.java @@ -273,6 +273,7 @@ public void testGetByStudies() throws Exception { .append(INCLUDE_SAMPLE.key(), ALL); queryResult = query(query, options); VariantQueryResult allVariants = dbAdaptor.get(new Query() + .append(INCLUDE_SAMPLE.key(), ALL) .append(VariantQueryParam.INCLUDE_STUDY.key(), study1), options); assertThat(queryResult, everyResult(allVariants, withStudy(study1))); diff --git a/opencga-storage/opencga-storage-core/src/test/java/org/opencb/opencga/storage/core/variant/dummy/DummyVariantStorageEngine.java b/opencga-storage/opencga-storage-core/src/test/java/org/opencb/opencga/storage/core/variant/dummy/DummyVariantStorageEngine.java index 4ce46333fa3..7ac22b0d316 100644 --- a/opencga-storage/opencga-storage-core/src/test/java/org/opencb/opencga/storage/core/variant/dummy/DummyVariantStorageEngine.java +++ b/opencga-storage/opencga-storage-core/src/test/java/org/opencb/opencga/storage/core/variant/dummy/DummyVariantStorageEngine.java @@ -23,6 +23,7 @@ import org.opencb.opencga.storage.core.exceptions.StorageEngineException; import org.opencb.opencga.storage.core.metadata.StudyConfiguration; import org.opencb.opencga.storage.core.metadata.VariantStorageMetadataManager; +import org.opencb.opencga.storage.core.metadata.models.StudyMetadata; import org.opencb.opencga.storage.core.metadata.models.TaskMetadata; import org.opencb.opencga.storage.core.variant.VariantStorageEngine; import org.opencb.opencga.storage.core.variant.adaptors.VariantDBAdaptor; @@ -78,20 +79,20 @@ public DummyVariantStoragePipeline newStoragePipeline(boolean connected) throws public DataResult> familyIndex(String study, List> trios, ObjectMap options) throws StorageEngineException { logger.info("Running family index!"); VariantStorageMetadataManager metadataManager = getMetadataManager(); - int studyId = metadataManager.getStudyId(study); + StudyMetadata studyMetadata = metadataManager.getStudyMetadata(study); + int studyId = studyMetadata.getId(); for (int i = 0; i < trios.size(); i += 3) { Integer father = metadataManager.getSampleId(studyId, trios.get(i)); Integer mother = metadataManager.getSampleId(studyId, trios.get(i + 1)); Integer child = metadataManager.getSampleId(studyId, trios.get(i + 2)); metadataManager.updateSampleMetadata(studyId, child, sampleMetadata -> { - sampleMetadata.setFamilyIndexStatus(TaskMetadata.Status.READY); + sampleMetadata.setFamilyIndexStatus(TaskMetadata.Status.READY, studyMetadata.getSampleIndexConfigurationLatest().getVersion()); if (father != null && father > 0) { sampleMetadata.setFather(father); } if (mother != null && mother > 0) { sampleMetadata.setMother(mother); } - return sampleMetadata; }); } return new DataResult>().setResults(trios); diff --git a/opencga-storage/opencga-storage-core/src/test/java/org/opencb/opencga/storage/core/variant/query/VariantQueryParserTest.java b/opencga-storage/opencga-storage-core/src/test/java/org/opencb/opencga/storage/core/variant/query/VariantQueryParserTest.java index 8a2eb762c8c..d9a7065c4ec 100644 --- a/opencga-storage/opencga-storage-core/src/test/java/org/opencb/opencga/storage/core/variant/query/VariantQueryParserTest.java +++ b/opencga-storage/opencga-storage-core/src/test/java/org/opencb/opencga/storage/core/variant/query/VariantQueryParserTest.java @@ -58,7 +58,6 @@ public void setUp() throws Exception { int file2 = metadataManager.registerFile(study1.getId(), "file2.vcf", Arrays.asList("sample1")); metadataManager.updateSampleMetadata(study1.getId(), metadataManager.getSampleId(study1.getId(), "sample1"), sm -> { sm.setSplitData(VariantStorageEngine.SplitData.MULTI); - return sm; }); metadataManager.addIndexedFiles(study1.getId(), Collections.singletonList(file2)); } diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/pom.xml b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/pom.xml index c8bc981e2fe..1f7173ad2fe 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/pom.xml +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/pom.xml @@ -23,7 +23,7 @@ org.opencb.opencga opencga-storage-hadoop - 2.2.0-rc2 + 2.2.0 ../pom.xml diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/app/SampleIndexMain.java b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/app/SampleIndexMain.java index ea686c5090d..ab680ebf59c 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/app/SampleIndexMain.java +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/app/SampleIndexMain.java @@ -168,7 +168,7 @@ private void rawQuery(SampleIndexDBAdaptor dbAdaptor, ObjectMap argsMap) throws private void detailedQuery(SampleIndexDBAdaptor dbAdaptor, ObjectMap argsMap) throws Exception { SampleIndexQuery sampleIndexQuery = dbAdaptor.parseSampleIndexQuery(new Query(argsMap)); CloseableIterator iterator = dbAdaptor.rawIterator(sampleIndexQuery); - SampleIndexSchema schema = dbAdaptor.getSchema(sampleIndexQuery.getStudy()); + SampleIndexSchema schema = sampleIndexQuery.getSchema(); if (argsMap.getBoolean("quiet", false)) { print(Iterators.size(iterator)); } else { diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/HadoopLocalLoadVariantStoragePipeline.java b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/HadoopLocalLoadVariantStoragePipeline.java index 8fe1e0db294..c742267539f 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/HadoopLocalLoadVariantStoragePipeline.java +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/HadoopLocalLoadVariantStoragePipeline.java @@ -123,10 +123,10 @@ protected void preLoadRegisterAndValidateFile(int studyId, VariantFileMetadata v if (sampleMetadata.isIndexed()) { alreadyIndexedSamples.add(sample); if (sampleMetadata.isAnnotated() - || !loadSampleIndex && SampleIndexDBAdaptor.getSampleIndexStatus(sampleMetadata, version) == Status.READY - || SampleIndexDBAdaptor.getSampleIndexAnnotationStatus(sampleMetadata, version) == Status.READY - || sampleMetadata.getFamilyIndexStatus() == Status.READY - || sampleMetadata.getMendelianErrorStatus() == Status.READY) { + || !loadSampleIndex && sampleMetadata.getSampleIndexStatus(version) == Status.READY + || sampleMetadata.getSampleIndexAnnotationStatus(version) == Status.READY + || sampleMetadata.getFamilyIndexStatus(version) == Status.READY + || sampleMetadata.isFamilyIndexDefined()) { processedSamples.add(sampleMetadata.getId()); } } @@ -146,13 +146,18 @@ protected void preLoadRegisterAndValidateFile(int studyId, VariantFileMetadata v for (Integer sampleId : processedSamples) { getMetadataManager().updateSampleMetadata(studyId, sampleId, sampleMetadata -> { if (!loadSampleIndex) { - SampleIndexDBAdaptor.setSampleIndexStatus(sampleMetadata, Status.NONE, 0); + for (Integer v : sampleMetadata.getSampleIndexVersions()) { + sampleMetadata.setSampleIndexStatus(Status.NONE, v); + } + } + for (Integer v : sampleMetadata.getSampleIndexAnnotationVersions()) { + sampleMetadata.setSampleIndexAnnotationStatus(Status.NONE, v); + } + for (Integer v : sampleMetadata.getFamilyIndexVersions()) { + sampleMetadata.setFamilyIndexStatus(Status.NONE, v); } sampleMetadata.setAnnotationStatus(Status.NONE); - SampleIndexDBAdaptor.setSampleIndexAnnotationStatus(sampleMetadata, Status.NONE, 0); - sampleMetadata.setFamilyIndexStatus(Status.NONE); sampleMetadata.setMendelianErrorStatus(Status.NONE); - return sampleMetadata; }); } } @@ -160,8 +165,9 @@ protected void preLoadRegisterAndValidateFile(int studyId, VariantFileMetadata v if (splitData != null) { // Register loadSplitData for (Integer sampleId : samplesWithoutSplitData) { - getMetadataManager().updateSampleMetadata(studyId, sampleId, - sampleMetadata -> sampleMetadata.setSplitData(splitData)); + getMetadataManager().updateSampleMetadata(studyId, sampleId, sampleMetadata -> { + sampleMetadata.setSplitData(splitData); + }); } } } @@ -497,10 +503,9 @@ public URI postLoad(URI input, URI output) throws StorageEngineException { for (Integer sampleId : metadataManager.getSampleIdsFromFileId(getStudyId(), getFileId())) { // Worth to check first to avoid too many updates in scenarios like 1000G SampleMetadata sampleMetadata = metadataManager.getSampleMetadata(getStudyId(), sampleId); - if (SampleIndexDBAdaptor.getSampleIndexStatus(sampleMetadata, sampleIndexVersion) != Status.READY) { - metadataManager.updateSampleMetadata(getStudyId(), sampleId, s -> { - return SampleIndexDBAdaptor.setSampleIndexStatus(s, Status.READY, sampleIndexVersion); - }); + if (sampleMetadata.getSampleIndexStatus(sampleIndexVersion) != Status.READY) { + metadataManager.updateSampleMetadata(getStudyId(), sampleId, + s -> s.setSampleIndexStatus(Status.READY, sampleIndexVersion)); } } } @@ -537,7 +542,7 @@ private SampleIndexDBLoader newSampleIndexDBLoader(List sampleIds) thro getMetadataManager(), getStudyId(), getFileId(), sampleIds, VariantStorageEngine.SplitData.from(getOptions()), - getOptions()); + getOptions(), sampleIndexDbAdaptor.getSchemaLatest(getStudyId())); return sampleIndexDBLoader; } diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/HadoopVariantStorageEngine.java b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/HadoopVariantStorageEngine.java index de5f07cc95a..aa1f74ba66f 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/HadoopVariantStorageEngine.java +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/HadoopVariantStorageEngine.java @@ -388,15 +388,16 @@ public void deleteStats(String study, Collection cohorts, ObjectMap para @Override public void sampleIndex(String study, List samples, ObjectMap options) throws StorageEngineException { options = getMergedOptions(options); - new SampleIndexLoader(getSampleIndexDBAdaptor(), getMRExecutor()) - .buildSampleIndex(study, samples, options); + System.out.println("options.toJson() = " + options.toJson()); + new SampleIndexLoader(getSampleIndexDBAdaptor(), study, getMRExecutor()) + .buildSampleIndex(samples, options); } @Override public void sampleIndexAnnotate(String study, List samples, ObjectMap options) throws StorageEngineException { options = getMergedOptions(options); - new SampleIndexAnnotationLoader(hBaseManager, getTableNameGenerator(), getMetadataManager(), getMRExecutor()) + new SampleIndexAnnotationLoader(getSampleIndexDBAdaptor(), getMRExecutor()) .updateSampleAnnotation(study, samples, options); } @@ -749,7 +750,7 @@ private void remove(String study, List files, List samples, URI String archiveTable = getArchiveTableName(studyId); String variantsTable = getVariantTableName(); - String sampleIndexTable = getSampleIndexDBAdaptor().getSampleIndexTableName(studyId); + String sampleIndexTable = getSampleIndexDBAdaptor().getSampleIndexTableNameLatest(studyId); long startTime = System.currentTimeMillis(); @@ -874,9 +875,15 @@ private void remove(String study, List files, List samples, URI for (String sample : samplesToRebuildIndex) { int sampleId = getMetadataManager().getSampleIdOrFail(studyId, sample); getMetadataManager().updateSampleMetadata(studyId, sampleId, sampleMetadata -> { - SampleIndexDBAdaptor.setSampleIndexStatus(sampleMetadata, TaskMetadata.Status.ERROR, 0); - SampleIndexDBAdaptor.setSampleIndexAnnotationStatus(sampleMetadata, TaskMetadata.Status.ERROR, 0); - return sampleMetadata; + for (int v : sampleMetadata.getSampleIndexVersions()) { + sampleMetadata.setSampleIndexStatus(TaskMetadata.Status.ERROR, v); + } + for (int v : sampleMetadata.getSampleIndexAnnotationVersions()) { + sampleMetadata.setSampleIndexAnnotationStatus(TaskMetadata.Status.ERROR, v); + } + for (int v : sampleMetadata.getFamilyIndexVersions()) { + sampleMetadata.setFamilyIndexStatus(TaskMetadata.Status.ERROR, v); + } }); } sampleIndex(study, samplesToRebuildIndex, options); diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/SampleIndexVariantAggregationExecutor.java b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/SampleIndexVariantAggregationExecutor.java index de00552efd8..c27c2294e79 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/SampleIndexVariantAggregationExecutor.java +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/SampleIndexVariantAggregationExecutor.java @@ -12,12 +12,10 @@ import org.opencb.opencga.core.response.VariantQueryResult; import org.opencb.opencga.storage.core.io.bit.BitBuffer; import org.opencb.opencga.storage.core.metadata.VariantStorageMetadataManager; -import org.opencb.opencga.storage.core.metadata.models.StudyMetadata; import org.opencb.opencga.storage.core.utils.iterators.CloseableIterator; import org.opencb.opencga.storage.core.variant.adaptors.VariantField; import org.opencb.opencga.storage.core.variant.adaptors.VariantQueryException; import org.opencb.opencga.storage.core.variant.adaptors.VariantQueryParam; -import org.opencb.opencga.storage.core.variant.query.VariantQueryParser; import org.opencb.opencga.storage.core.variant.query.VariantQueryUtils; import org.opencb.opencga.storage.core.variant.query.executors.VariantAggregationExecutor; import org.opencb.opencga.storage.core.variant.query.executors.accumulators.*; @@ -110,7 +108,10 @@ protected boolean canUseThisExecutor(Query query, QueryOptions options, String f protected VariantQueryResult aggregation(Query query, QueryOptions options, String facet) throws Exception { StopWatch stopWatch = StopWatch.createStarted(); boolean filterTranscript = options.getBoolean("filterTranscript", false); - List> accumulators = createAccumulators(query, facet, filterTranscript); + SampleIndexQuery sampleIndexQuery = sampleIndexDBAdaptor.parseSampleIndexQuery(new Query(query)); + SampleIndexSchema schema = sampleIndexQuery.getSchema(); + + List> accumulators = createAccumulators(schema, query, facet, filterTranscript); List fields = new ArrayList<>(); logger.info("Filter transcript = {}", filterTranscript); @@ -147,19 +148,19 @@ protected VariantQueryResult aggregation(Query query, QueryOptions o } } - private List> createAccumulators(Query query, String facet, boolean filterTranscript) { + private List> createAccumulators( + SampleIndexSchema schema, Query query, String facet, boolean filterTranscript) { List> list = new ArrayList<>(); for (String f : facet.split(FACET_SEPARATOR)) { - list.add(createAccumulator(query, f, filterTranscript)); + list.add(createAccumulator(schema, query, f, filterTranscript)); } return list; } - private FacetFieldAccumulator createAccumulator(Query query, String facet, boolean filterTranscript) { + private FacetFieldAccumulator createAccumulator( + SampleIndexSchema schema, Query query, String facet, boolean filterTranscript) { String[] split = facet.split(NESTED_FACET_SEPARATOR); FacetFieldAccumulator accumulator = null; - StudyMetadata defaultStudy = VariantQueryParser.getDefaultStudy(query, metadataManager); - SampleIndexSchema schema = sampleIndexDBAdaptor.getSchema(defaultStudy.getId()); Set ctFilter = new HashSet<>(VariantQueryUtils .parseConsequenceTypes(query.getAsStringList(VariantQueryParam.ANNOT_CONSEQUENCE_TYPE.key()))); diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/annotation/mr/SampleIndexAnnotationLoaderDriver.java b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/annotation/mr/SampleIndexAnnotationLoaderDriver.java index d39749bef1d..41826fd4abe 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/annotation/mr/SampleIndexAnnotationLoaderDriver.java +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/annotation/mr/SampleIndexAnnotationLoaderDriver.java @@ -9,7 +9,6 @@ import org.opencb.biodata.models.core.Region; import org.opencb.commons.datastore.core.ObjectMap; import org.opencb.opencga.core.config.storage.SampleIndexConfiguration; -import org.opencb.opencga.storage.core.exceptions.StorageEngineException; import org.opencb.opencga.storage.core.metadata.VariantStorageMetadataManager; import org.opencb.opencga.storage.core.metadata.models.SampleMetadata; import org.opencb.opencga.storage.core.metadata.models.StudyMetadata; @@ -21,7 +20,6 @@ import org.opencb.opencga.storage.hadoop.variant.adaptors.phoenix.PhoenixHelper; import org.opencb.opencga.storage.hadoop.variant.adaptors.phoenix.VariantPhoenixSchema; import org.opencb.opencga.storage.hadoop.variant.converters.HBaseToVariantConverter; -import org.opencb.opencga.storage.hadoop.variant.index.sample.SampleIndexAnnotationLoader; import org.opencb.opencga.storage.hadoop.variant.index.sample.SampleIndexSchema; import org.opencb.opencga.storage.hadoop.variant.mr.VariantAlignedInputFormat; import org.opencb.opencga.storage.hadoop.variant.mr.VariantMapReduceUtil; @@ -40,6 +38,7 @@ public class SampleIndexAnnotationLoaderDriver extends AbstractVariantsTableDriv private static final Logger LOGGER = LoggerFactory.getLogger(SampleIndexAnnotationLoaderDriver.class); public static final String OUTPUT = "output-table"; + public static final String SAMPLE_INDEX_VERSION = "sample-index-version"; public static final String SAMPLES = "samples"; public static final String SAMPLE_IDS = "sampleIds"; @@ -59,6 +58,7 @@ protected Map getParams() { Map params = new HashMap<>(); params.put("--" + SAMPLES, ""); params.put("--" + SAMPLE_IDS, ""); + params.put("--" + SAMPLE_INDEX_VERSION, ""); params.put("--" + OUTPUT, ""); params.put("--" + VariantQueryParam.REGION.key(), ""); return params; @@ -152,12 +152,12 @@ protected Job setupJob(Job job, String archiveTable, String variantTable) throws VariantMapReduceUtil.setNoneReduce(job); + sampleIndexVersion = Integer.parseInt(getParam(SAMPLE_INDEX_VERSION)); StudyMetadata.SampleIndexConfigurationVersioned versioned = getMetadataManager() .getStudyMetadata(getStudyId()) - .getSampleIndexConfigurationLatest(); + .getSampleIndexConfiguration(sampleIndexVersion); SampleIndexConfiguration configuration = versioned.getConfiguration(); - sampleIndexVersion = versioned.getVersion(); - VariantMapReduceUtil.setSampleIndexConfiguration(job, configuration); + VariantMapReduceUtil.setSampleIndexConfiguration(job, configuration, sampleIndexVersion); return job; } @@ -167,14 +167,6 @@ protected String getJobOperationName() { return "sample_index_annotation_loader"; } - @Override - protected void postExecution(boolean succeed) throws IOException, StorageEngineException { - super.postExecution(succeed); - if (succeed && StringUtils.isEmpty(region)) { - SampleIndexAnnotationLoader.postAnnotationLoad(getStudyId(), sampleIds, getMetadataManager(), sampleIndexVersion); - } - } - public static void main(String[] args) throws Exception { try { System.exit(new SampleIndexAnnotationLoaderDriver().privateMain(args, null)); diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/annotation/mr/SampleIndexAnnotationLoaderMapper.java b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/annotation/mr/SampleIndexAnnotationLoaderMapper.java index dad4178b181..28c53627d75 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/annotation/mr/SampleIndexAnnotationLoaderMapper.java +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/annotation/mr/SampleIndexAnnotationLoaderMapper.java @@ -64,7 +64,7 @@ protected void setup(Context context) throws IOException, InterruptedException { for (int i = 0; i < annotationIndices.length; i++) { annotationIndices[i] = new HashMap<>(); } - schema = new SampleIndexSchema(VariantMapReduceUtil.getSampleIndexConfiguration(context.getConfiguration())); + schema = VariantMapReduceUtil.getSampleIndexSchema(context.getConfiguration()); converter = new AnnotationIndexConverter(schema); } diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/family/FamilyIndexDriver.java b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/family/FamilyIndexDriver.java index 1f0e31ffbfb..36f17f5f3a1 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/family/FamilyIndexDriver.java +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/family/FamilyIndexDriver.java @@ -49,6 +49,7 @@ public class FamilyIndexDriver extends AbstractVariantsTableDriver { public static final String TRIOS_COHORT_DELETE = "triosCohortDelete"; public static final String OVERWRITE = "overwrite"; public static final String OUTPUT = "output-table"; + public static final String SAMPLE_INDEX_VERSION = "sample-index-version"; private static final String TRIOS_LIST = "FamilyIndexDriver.trios_list"; // Samples where at least one parent is not in its file @@ -59,6 +60,7 @@ public class FamilyIndexDriver extends AbstractVariantsTableDriver { private boolean partial; private String region; private String sampleIndexTableName; + private int sampleIndexVersion; @Override protected Class getMapperClass() { @@ -70,6 +72,7 @@ protected Map getParams() { Map params = new HashMap<>(); params.put("--" + TRIOS, "(father,mother,child;)**"); params.put("--" + VariantQueryParam.REGION.key(), ""); + params.put("--" + SAMPLE_INDEX_VERSION, ""); params.put("--" + OVERWRITE, ""); return params; } @@ -84,6 +87,7 @@ protected void parseAndValidateParameters() throws IOException { throw new IllegalArgumentIOException("Missing output table"); } sampleIndexTableName = getParam(OUTPUT); + sampleIndexVersion = Integer.parseInt(getParam(SAMPLE_INDEX_VERSION)); boolean overwrite = Boolean.parseBoolean(getParam(OVERWRITE)); String triosCohort = getParam(TRIOS_COHORT); @@ -111,7 +115,7 @@ protected void parseAndValidateParameters() throws IOException { throw new IllegalArgumentException("Found trio with " + trioList.size() + " members, instead of 3: " + trioList); } SampleMetadata sampleMetadata = metadataManager.getSampleMetadata(getStudyId(), trioList.get(2)); - if (!overwrite && sampleMetadata.getMendelianErrorStatus().equals(TaskMetadata.Status.READY)) { + if (!overwrite && sampleMetadata.getFamilyIndexStatus(sampleIndexVersion) == TaskMetadata.Status.READY) { LOGGER.info("Skip sample " + sampleMetadata.getName() + ". Already precomputed!"); } else { sampleIds.addAll(trioList); @@ -248,14 +252,14 @@ protected void postExecution(boolean succeed) throws IOException, StorageEngineE Integer child = sampleIds.get(i + 2); metadataManager.updateSampleMetadata(getStudyId(), child, sampleMetadata -> { sampleMetadata.setMendelianErrorStatus(TaskMetadata.Status.READY); - sampleMetadata.setFamilyIndexStatus(TaskMetadata.Status.READY); + sampleMetadata.setFamilyIndexStatus(TaskMetadata.Status.READY, sampleIndexVersion); + sampleMetadata.setFamilyIndexDefined(true); if (father > 0) { sampleMetadata.setFather(father); } if (mother > 0) { sampleMetadata.setMother(mother); } - return sampleMetadata; }); } } diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/family/FamilyIndexLoader.java b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/family/FamilyIndexLoader.java index 1c49e9250aa..6333227febf 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/family/FamilyIndexLoader.java +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/family/FamilyIndexLoader.java @@ -8,6 +8,7 @@ import org.opencb.opencga.storage.core.metadata.models.CohortMetadata; import org.opencb.opencga.storage.core.metadata.models.SampleMetadata; import org.opencb.opencga.storage.core.metadata.models.TaskMetadata; +import org.opencb.opencga.storage.core.utils.BatchUtils; import org.opencb.opencga.storage.hadoop.variant.HadoopVariantStorageOptions; import org.opencb.opencga.storage.hadoop.variant.adaptors.VariantHadoopDBAdaptor; import org.opencb.opencga.storage.hadoop.variant.executors.MRExecutor; @@ -48,6 +49,9 @@ public DataResult> load(String study, List> trios, Obj throw new StorageEngineException("Undefined family trios"); } int studyId = metadataManager.getStudyId(study); + int version = sampleIndexDBAdaptor.getSchemaFactory().getSampleIndexConfigurationLatest(studyId, true).getVersion(); + options.put(FamilyIndexDriver.SAMPLE_INDEX_VERSION, version); + options.put(FamilyIndexDriver.OUTPUT, sampleIndexDBAdaptor.getSampleIndexTableName(studyId, version)); Iterator> iterator = trios.iterator(); while (iterator.hasNext()) { List trioIds = new ArrayList<>(3); @@ -68,7 +72,7 @@ public DataResult> load(String study, List> trios, Obj throw new IllegalArgumentException("Found trio with " + trioIds.size() + " members, instead of 3: " + trioIds); } SampleMetadata sampleMetadata = metadataManager.getSampleMetadata(studyId, trioIds.get(2)); - if (!overwrite && sampleMetadata.getMendelianErrorStatus().equals(TaskMetadata.Status.READY)) { + if (!overwrite && sampleMetadata.getFamilyIndexStatus(version) == TaskMetadata.Status.READY) { String msg = "Skip sample " + sampleMetadata.getName() + ". Already precomputed!"; logger.info(msg); dr.getEvents().add(new Event(Event.Type.INFO, msg)); @@ -82,12 +86,12 @@ public DataResult> load(String study, List> trios, Obj || motherDefined && !motherId.equals(sampleMetadata.getMother())) { metadataManager.updateSampleMetadata(studyId, sampleMetadata.getId(), s -> { if (fatherDefined) { - sampleMetadata.setFather(fatherId); + s.setFather(fatherId); } if (motherDefined) { - sampleMetadata.setMother(motherId); + s.setMother(motherId); } - return sampleMetadata; + s.setFamilyIndexDefined(true); }); } } @@ -99,22 +103,23 @@ public DataResult> load(String study, List> trios, Obj int batchSize = options.getInt(HadoopVariantStorageOptions.SAMPLE_INDEX_FAMILY_MAX_TRIOS_PER_MR.key(), HadoopVariantStorageOptions.SAMPLE_INDEX_FAMILY_MAX_TRIOS_PER_MR.defaultValue()); - List>> batches = splitLists(trios, batchSize); + List>> batches = BatchUtils.splitBatches(trios, batchSize); if (batches.size() == 1) { - run(study, trios, options, studyId); + runBatch(study, trios, options, studyId); } else { logger.warn("Unable to run family index in one single MapReduce operation."); logger.info("Split in {} jobs of {} samples each.", batches, batches.get(0).size()); for (int i = 0; i < batches.size(); i++) { List> batch = batches.get(i); logger.info("Running MapReduce {}/{} over {} trios", i + 1, batches, batch.size()); - run(study, batch, options, studyId); + runBatch(study, batch, options, studyId); } } + postIndex(studyId, version); return dr; } - private void run(String study, List> trios, ObjectMap options, int studyId) throws StorageEngineException { + private void runBatch(String study, List> trios, ObjectMap options, int studyId) throws StorageEngineException { if (trios.size() < 500) { options.put(FamilyIndexDriver.TRIOS, trios.stream().map(trio -> String.join(",", trio)).collect(Collectors.joining(";"))); } else { @@ -124,7 +129,6 @@ private void run(String study, List> trios, ObjectMap options, int options.put(FamilyIndexDriver.TRIOS_COHORT, cohortMetadata.getName()); options.put(FamilyIndexDriver.TRIOS_COHORT_DELETE, true); } - options.put(FamilyIndexDriver.OUTPUT, sampleIndexDBAdaptor.getSampleIndexTableName(studyId)); mrExecutor.run(FamilyIndexDriver.class, FamilyIndexDriver.buildArgs( tableNameGenerator.getArchiveTableName(studyId), @@ -133,15 +137,9 @@ private void run(String study, List> trios, ObjectMap options, int "Precompute mendelian errors for " + (trios.size() == 1 ? "trio " + trios.get(0) : trios.size() + " trios")); } - private static List> splitLists(List list, int maxBatchSize) { - int batchSize = maxBatchSize; - int batches = (int) Math.round(Math.ceil(list.size() / ((float) batchSize))); - batchSize = (int) Math.round(Math.ceil(list.size() / ((float) batches))); - List> parts = new ArrayList<>(batches); - for (int i = 0; i < batches; i++) { - parts.add(list.subList(i * batchSize, Math.min((i + 1) * batchSize, list.size()))); - } - return parts; + public void postIndex(int studyId, int version) + throws StorageEngineException { + sampleIndexDBAdaptor.updateSampleIndexSchemaStatus(studyId, version); } } diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexAnnotationLoader.java b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexAnnotationLoader.java index ac0d86cbebd..557b86c25a1 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexAnnotationLoader.java +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexAnnotationLoader.java @@ -1,11 +1,5 @@ package org.opencb.opencga.storage.hadoop.variant.index.sample; -import org.apache.commons.lang3.tuple.Pair; -import org.apache.hadoop.hbase.TableName; -import org.apache.hadoop.hbase.client.BufferedMutator; -import org.apache.hadoop.hbase.client.Put; -import org.opencb.biodata.models.core.Region; -import org.opencb.biodata.models.variant.Variant; import org.opencb.commons.datastore.core.ObjectMap; import org.opencb.opencga.storage.core.exceptions.StorageEngineException; import org.opencb.opencga.storage.core.metadata.VariantStorageMetadataManager; @@ -13,18 +7,15 @@ import org.opencb.opencga.storage.core.variant.adaptors.VariantQueryException; import org.opencb.opencga.storage.core.variant.query.VariantQueryUtils; import org.opencb.opencga.storage.hadoop.utils.HBaseManager; -import org.opencb.opencga.storage.hadoop.variant.GenomeHelper; import org.opencb.opencga.storage.hadoop.variant.executors.MRExecutor; -import org.opencb.opencga.storage.hadoop.variant.index.annotation.AnnotationIndexEntry; -import org.opencb.opencga.storage.hadoop.variant.index.annotation.AnnotationIndexPutBuilder; import org.opencb.opencga.storage.hadoop.variant.index.annotation.mr.SampleIndexAnnotationLoaderDriver; import org.opencb.opencga.storage.hadoop.variant.utils.HBaseVariantTableNameGenerator; import org.slf4j.Logger; import org.slf4j.LoggerFactory; -import java.io.IOException; -import java.util.*; -import java.util.function.Function; +import java.util.ArrayList; +import java.util.LinkedList; +import java.util.List; import static org.opencb.opencga.core.api.ParamConstants.OVERWRITE; import static org.opencb.opencga.storage.core.metadata.models.TaskMetadata.Status; @@ -35,22 +26,25 @@ */ public class SampleIndexAnnotationLoader { - private final HBaseManager hBaseManager; private final HBaseVariantTableNameGenerator tableNameGenerator; private final MRExecutor mrExecutor; private final SampleIndexDBAdaptor sampleDBAdaptor; - private final byte[] family; private final VariantStorageMetadataManager metadataManager; - private Logger logger = LoggerFactory.getLogger(SampleIndexAnnotationLoader.class); + private final Logger logger = LoggerFactory.getLogger(SampleIndexAnnotationLoader.class); public SampleIndexAnnotationLoader(HBaseManager hBaseManager, HBaseVariantTableNameGenerator tableNameGenerator, VariantStorageMetadataManager metadataManager, MRExecutor mrExecutor) { - this.hBaseManager = hBaseManager; this.tableNameGenerator = tableNameGenerator; this.mrExecutor = mrExecutor; this.metadataManager = metadataManager; this.sampleDBAdaptor = new SampleIndexDBAdaptor(hBaseManager, tableNameGenerator, this.metadataManager); - family = GenomeHelper.COLUMN_FAMILY_BYTES; + } + + public SampleIndexAnnotationLoader(SampleIndexDBAdaptor sampleDBAdaptor, MRExecutor mrExecutor) { + this.mrExecutor = mrExecutor; + this.sampleDBAdaptor = sampleDBAdaptor; + this.metadataManager = sampleDBAdaptor.getMetadataManager(); + this.tableNameGenerator = sampleDBAdaptor.getTableNameGenerator(); } public void updateSampleAnnotation(String study, List samples, ObjectMap options) @@ -80,14 +74,14 @@ public void updateSampleAnnotation(int studyId, List samples, ObjectMap public void updateSampleAnnotation(int studyId, List samples, ObjectMap options, boolean overwrite) throws StorageEngineException { - int sampleIndexVersion = metadataManager.getStudyMetadata(studyId).getSampleIndexConfigurationLatest().getVersion(); + int sampleIndexVersion = sampleDBAdaptor.getSchemaLatest(studyId).getVersion(); List finalSamplesList = new ArrayList<>(samples.size()); List nonAnnotated = new LinkedList<>(); List alreadyAnnotated = new LinkedList<>(); for (Integer sampleId : samples) { SampleMetadata sampleMetadata = metadataManager.getSampleMetadata(studyId, sampleId); if (sampleMetadata.isAnnotated()) { - if (SampleIndexDBAdaptor.getSampleIndexAnnotationStatus(sampleMetadata, sampleIndexVersion).equals(Status.READY) + if (sampleMetadata.getSampleIndexAnnotationStatus(sampleIndexVersion).equals(Status.READY) && !overwrite) { // SamplesIndex already annotated alreadyAnnotated.add(sampleMetadata.getName()); @@ -115,6 +109,9 @@ public void updateSampleAnnotation(int studyId, List samples, ObjectMap logger.info("Skip sample index annotation. Nothing to do!"); return; } + + sampleDBAdaptor.createTableIfNeeded(studyId, sampleIndexVersion, options); + if (finalSamplesList.size() < 20) { logger.info("Run sample index annotation on samples " + finalSamplesList); } else { @@ -135,227 +132,38 @@ public void updateSampleAnnotation(int studyId, List samples, ObjectMap for (int i = 0; i < batches; i++) { List subSet = finalSamplesList.subList(i * batchSize, Math.min((i + 1) * batchSize, finalSamplesList.size())); logger.info("Running MapReduce {}/{} over {} samples", i + 1, batches, subSet.size()); - updateSampleAnnotationBatchMapreduce(studyId, subSet, options); + updateSampleAnnotationBatchMapreduce(studyId, subSet, sampleIndexVersion, options); } } else { - updateSampleAnnotationBatchMapreduce(studyId, finalSamplesList, options); + updateSampleAnnotationBatchMapreduce(studyId, finalSamplesList, sampleIndexVersion, options); } + + postAnnotationLoad(studyId, sampleIndexVersion); } - private void updateSampleAnnotationBatchMapreduce(int studyId, List samples, ObjectMap options) + private void updateSampleAnnotationBatchMapreduce(int studyId, List samples, int sampleIndexVersion, ObjectMap options) throws StorageEngineException { - options.put(SampleIndexAnnotationLoaderDriver.OUTPUT, sampleDBAdaptor.getSampleIndexTableName(studyId)); + options.put(SampleIndexAnnotationLoaderDriver.OUTPUT, sampleDBAdaptor.getSampleIndexTableNameLatest(studyId)); + options.put(SampleIndexAnnotationLoaderDriver.SAMPLE_INDEX_VERSION, sampleIndexVersion); mrExecutor.run(SampleIndexAnnotationLoaderDriver.class, SampleIndexAnnotationLoaderDriver.buildArgs( tableNameGenerator.getArchiveTableName(studyId), tableNameGenerator.getVariantTableName(), studyId, samples, options), "Annotate sample index for " + (samples.size() < 10 ? "samples " + samples : samples.size() + " samples")); - } - - -// private void updateSampleAnnotationBatchMultiThread(int studyId, List samples) throws IOException, StorageEngineException { -// logger.info("Update sample index annotation of " + samples.size() + " samples"); -// -// String sampleIndexTableName = tableNameGenerator.getSampleIndexTableName(studyId); -// -// ProgressLogger progressLogger = new ProgressLogger("Sample index annotation updated variants"); -// -// ParallelTaskRunner, Put> ptr = new ParallelTaskRunner<>( -// new DataReader>() { -// -// private Iterator> iterator = annotationIndexDBAdaptor.iterator(); -// private int initialCapacity = 200000; -// private Pair nextPair = null; -// -// private String chromosome = ""; -// private int start = -1; -// private int end = -1; -// -// @Override -// public List> read(int n) { -// List> annotationMasks = new ArrayList<>(initialCapacity); -// -// // Read next batch -// if (nextPair == null && iterator.hasNext()) { -// nextPair = iterator.next(); -// } -// if (nextPair != null) { -// annotationMasks.add(nextPair); -// Variant firstVariant = nextPair.getKey(); -// chromosome = firstVariant.getChromosome(); -// start = firstVariant.getStart() - (firstVariant.getStart() % SampleIndexSchema.BATCH_SIZE); -// end = start + SampleIndexSchema.BATCH_SIZE; -// nextPair = null; -// } -// while (iterator.hasNext()) { -// Pair pair = iterator.next(); -// Variant variant = pair.getKey(); -// if (variant.getChromosome().equals(chromosome) && variant.getStart() > start && variant.getStart() < end) { -// annotationMasks.add(pair); -// } else { -//// logger.info("Variant " + variant -//// + "(" + variant.getChromosome() + ":" + variant.getStart() + "-" + variant.getEnd() + ")" -//// + " not in batch " + chromosome + ":" + start + "-" + end); -// nextPair = pair; -// break; -// } -// } -// -// return annotationMasks; -// } -// }, -// annotationMasks -> { -// // Ensure is sorted as expected -// annotationMasks.sort(Comparator.comparing(Pair::getKey, -// SampleIndexSchema.INTRA_CHROMOSOME_VARIANT_COMPARATOR)); -// -// Variant firstVariant = annotationMasks.get(0).getKey(); -// String chromosome = firstVariant.getChromosome(); -// int start = firstVariant.getStart() - (firstVariant.getStart() % SampleIndexSchema.BATCH_SIZE); -// int end = start + SampleIndexSchema.BATCH_SIZE; -// -// progressLogger.increment(annotationMasks.size(), () -> "Up to batch " + chromosome + ":" + start + "-" + end); -// List puts = new ArrayList<>(samples.size()); -// -// for (Integer sampleId : samples) { -// Map> map = sampleDBAdaptor.queryByGt(studyId, sampleId, chromosome, start); -// Put put = annotate(chromosome, start, sampleId, map, annotationMasks); -// if (!put.isEmpty()) { -// puts.add(put); -// } -//// else logger.warn("Empty put for sample " + sampleId + " -> " + chromosome + ":" + start + ":" + end); -// } -// -// return puts; -// }, -// new HBaseDataWriter<>(hBaseManager, sampleIndexTableName), -// ParallelTaskRunner.Config.builder().setNumTasks(8).setCapacity(2).build() -// ); -// -// try { -// ptr.run(); -// } catch (ExecutionException e) { -// throw new StorageEngineException("Error", e); -// } -// -// postAnnotationLoad(studyId, samples); -// } - - public void updateSampleAnnotationMultiSampleIterator(int studyId, List samples, - Function>> annotationIndexReader) - throws IOException, StorageEngineException { - int version = sampleDBAdaptor.getSampleIndexConfiguration(studyId).getVersion(); - String sampleIndexTableName = sampleDBAdaptor.getSampleIndexTableName(studyId); - Map>>> sampleIterators = new HashMap<>(samples.size()); - SampleIndexSchema schema = sampleDBAdaptor.getSchema(studyId); - - for (Integer sample : samples) { - sampleIterators.put(sample, sampleDBAdaptor.iteratorByGt(studyId, sample)); - } - - BufferedMutator mutator = hBaseManager.getConnection().getBufferedMutator(TableName.valueOf(sampleIndexTableName)); - - String chromosome = ""; - int start = -1; - int end = -1; - List> annotationEntries = null; - do { - for (Map.Entry>>> sampleIteratorPair : sampleIterators.entrySet()) { - Iterator>> sampleIterator = sampleIteratorPair.getValue(); - Integer sampleId = sampleIteratorPair.getKey(); - if (sampleIterator.hasNext()) { - Map> next = sampleIterator.next(); - - Variant firstVariant = next.values().iterator().next().get(0); - if (annotationEntries == null - || !chromosome.equals(firstVariant.getChromosome()) - || firstVariant.getStart() < start - || firstVariant.getStart() > end) { - chromosome = firstVariant.getChromosome(); - start = firstVariant.getStart() - firstVariant.getStart() % SampleIndexSchema.BATCH_SIZE; - end = start + SampleIndexSchema.BATCH_SIZE; - // FIXME -// annotationEntries = annotationIndexDBAdaptor.get(chromosome, start, end); - annotationEntries = annotationIndexReader.apply(new Region(chromosome, start, end)); - } - - Put put = annotate(chromosome, start, sampleId, next, annotationEntries, schema); - mutator.mutate(put); - } - } - - // Remove exhausted iterators - sampleIterators.entrySet().removeIf(e -> !e.getValue().hasNext()); - } while (!sampleIterators.isEmpty()); - - mutator.close(); - - postAnnotationLoad(studyId, samples, version); - } - - private Put annotate(String chromosome, int start, Integer sampleId, - Map> sampleIndex, List> annotationMasks, - SampleIndexSchema schema) { - byte[] rk = SampleIndexSchema.toRowKey(sampleId, chromosome, start); - Put put = new Put(rk); - - for (Map.Entry> entry : sampleIndex.entrySet()) { - String gt = entry.getKey(); - List variantsToAnnotate = entry.getValue(); - if (!SampleIndexSchema.isAnnotatedGenotype(gt)) { - continue; - } - - ListIterator> iterator = annotationMasks.listIterator(); - AnnotationIndexPutBuilder builder = new AnnotationIndexPutBuilder(schema, variantsToAnnotate.size()); - int missingVariants = 0; - // Assume both lists are ordered, and "variantsToAnnotate" is fully contained in "annotationMasks" - for (Variant variantToAnnotate : variantsToAnnotate) { - boolean restarted = false; - while (iterator.hasNext()) { - Pair annotationPair = iterator.next(); - if (annotationPair.getKey().sameGenomicVariant(variantToAnnotate)) { - builder.add(annotationPair.getRight()); - break; - } else if (annotationPair.getKey().getStart() > variantToAnnotate.getStart()) { - if (!restarted) { - logger.warn("Missing variant to annotate " + variantToAnnotate + " RESTART ITERATOR"); - while (iterator.hasPrevious()) { - iterator.previous(); - } - restarted = true; - } else { - logger.error("Missing variant to annotate " + variantToAnnotate); - builder.add(AnnotationIndexEntry.empty(schema)); - missingVariants++; - break; - } - } - } - } - if (missingVariants > 0) { - // TODO: What if a variant is not annotated? - String msg = "Error annotating batch. " + missingVariants + " missing variants"; - logger.error(msg); -// throw new IllegalStateException(msg); - } - builder.buildAndReset(put, gt, family); - } - return put; + postAnnotationBatchLoad(studyId, samples, sampleIndexVersion); } - private void postAnnotationLoad(int studyId, List samples, int version) throws StorageEngineException { - postAnnotationLoad(studyId, samples, metadataManager, version); - } - - public static void postAnnotationLoad(int studyId, List samples, VariantStorageMetadataManager metadataManager, int version) + public void postAnnotationBatchLoad(int studyId, List samples, int version) throws StorageEngineException { for (Integer sampleId : samples) { metadataManager.updateSampleMetadata(studyId, sampleId, sampleMetadata -> { - return SampleIndexDBAdaptor.setSampleIndexAnnotationStatus(sampleMetadata, Status.READY, version); + sampleMetadata.setSampleIndexAnnotationStatus(Status.READY, version); }); } } - + public void postAnnotationLoad(int studyId, int version) + throws StorageEngineException { + sampleDBAdaptor.updateSampleIndexSchemaStatus(studyId, version); + } } diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexDBAdaptor.java b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexDBAdaptor.java index 196b66f250b..8ee2937f922 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexDBAdaptor.java +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexDBAdaptor.java @@ -2,17 +2,18 @@ import com.google.common.collect.Iterators; import org.apache.commons.collections4.CollectionUtils; -import org.apache.commons.lang3.StringUtils; import org.apache.hadoop.hbase.client.Get; import org.apache.hadoop.hbase.client.Result; import org.apache.hadoop.hbase.client.ResultScanner; import org.apache.hadoop.hbase.client.Scan; +import org.apache.hadoop.hbase.io.compress.Compression; import org.apache.hadoop.hbase.util.Bytes; import org.opencb.biodata.models.core.Region; import org.opencb.biodata.models.variant.Variant; +import org.opencb.commons.datastore.core.ObjectMap; import org.opencb.commons.datastore.core.Query; import org.opencb.commons.datastore.core.QueryOptions; -import org.opencb.opencga.core.config.storage.SampleIndexConfiguration; +import org.opencb.opencga.storage.core.exceptions.StorageEngineException; import org.opencb.opencga.storage.core.metadata.VariantStorageMetadataManager; import org.opencb.opencga.storage.core.metadata.models.SampleMetadata; import org.opencb.opencga.storage.core.metadata.models.StudyMetadata; @@ -27,11 +28,11 @@ import org.opencb.opencga.storage.core.variant.adaptors.iterators.IntersectMultiVariantKeyIterator; import org.opencb.opencga.storage.core.variant.adaptors.iterators.UnionMultiVariantKeyIterator; import org.opencb.opencga.storage.core.variant.adaptors.iterators.VariantDBIterator; -import org.opencb.opencga.storage.core.variant.query.VariantQueryParser; import org.opencb.opencga.storage.core.variant.query.VariantQueryUtils; import org.opencb.opencga.storage.core.variant.query.VariantQueryUtils.QueryOperation; import org.opencb.opencga.storage.hadoop.utils.HBaseManager; import org.opencb.opencga.storage.hadoop.variant.GenomeHelper; +import org.opencb.opencga.storage.hadoop.variant.HadoopVariantStorageOptions; import org.opencb.opencga.storage.hadoop.variant.index.IndexUtils; import org.opencb.opencga.storage.hadoop.variant.index.core.filters.IndexFieldFilter; import org.opencb.opencga.storage.hadoop.variant.index.query.*; @@ -40,6 +41,7 @@ import org.slf4j.LoggerFactory; import java.io.IOException; +import java.io.UncheckedIOException; import java.util.*; import java.util.stream.Collectors; @@ -56,6 +58,7 @@ public class SampleIndexDBAdaptor implements VariantIterable { private final HBaseManager hBaseManager; private final HBaseVariantTableNameGenerator tableNameGenerator; private final VariantStorageMetadataManager metadataManager; + private final SampleIndexSchemaFactory schemaFactory; private static byte[] family = GenomeHelper.COLUMN_FAMILY_BYTES; private static Logger logger = LoggerFactory.getLogger(SampleIndexDBAdaptor.class); @@ -64,22 +67,7 @@ public SampleIndexDBAdaptor(HBaseManager hBaseManager, HBaseVariantTableNameGene this.hBaseManager = hBaseManager; this.tableNameGenerator = tableNameGenerator; this.metadataManager = metadataManager; - } - - public static TaskMetadata.Status getSampleIndexAnnotationStatus(SampleMetadata sampleMetadata, int latestSampleIndexVersion) { - return sampleMetadata.getSampleIndexAnnotationStatus(latestSampleIndexVersion); - } - - public static SampleMetadata setSampleIndexAnnotationStatus(SampleMetadata sampleMetadata, TaskMetadata.Status status, int version) { - return sampleMetadata.setSampleIndexAnnotationStatus(status, version); - } - - public static TaskMetadata.Status getSampleIndexStatus(SampleMetadata sampleMetadata, int latestSampleIndexVersion) { - return sampleMetadata.getSampleIndexStatus(latestSampleIndexVersion); - } - - public static SampleMetadata setSampleIndexStatus(SampleMetadata sampleMetadata, TaskMetadata.Status status, int version) { - return sampleMetadata.setSampleIndexStatus(status, version); + this.schemaFactory = new SampleIndexSchemaFactory(metadataManager); } @Override @@ -97,7 +85,7 @@ public VariantDBIterator iterator(SampleIndexQuery query, QueryOptions options) if (samples.isEmpty()) { throw new VariantQueryException("At least one sample expected to query SampleIndex!"); } - SampleIndexSchema schema = getSchema(query.getStudy()); + SampleIndexSchema schema = query.getSchema(); QueryOperation operation = query.getQueryOperation(); if (samples.size() == 1) { @@ -171,7 +159,7 @@ protected VariantDBIterator applyLimitSkip(VariantDBIterator iterator, QueryOpti * @return SingleSampleIndexVariantDBIterator */ private SingleSampleIndexVariantDBIterator internalIterator(SingleSampleIndexQuery query, SampleIndexSchema schema) { - String tableName = getSampleIndexTableName(toStudyId(query.getStudy())); + String tableName = getSampleIndexTableName(query); try { return hBaseManager.act(tableName, table -> { @@ -183,7 +171,7 @@ private SingleSampleIndexVariantDBIterator internalIterator(SingleSampleIndexQue } private RawSingleSampleIndexVariantDBIterator rawInternalIterator(SingleSampleIndexQuery query, SampleIndexSchema schema) { - String tableName = getSampleIndexTableName(toStudyId(query.getStudy())); + String tableName = getSampleIndexTableName(query); try { return hBaseManager.act(tableName, table -> { @@ -194,8 +182,16 @@ private RawSingleSampleIndexVariantDBIterator rawInternalIterator(SingleSampleIn } } - public String getSampleIndexTableName(int studyId) { - int version = getSampleIndexConfiguration(studyId).getVersion(); + public String getSampleIndexTableName(SampleIndexQuery query) { + return tableNameGenerator.getSampleIndexTableName(toStudyId(query.getStudy()), query.getSchema().getVersion()); + } + + public String getSampleIndexTableNameLatest(int studyId) { + int version = schemaFactory.getSampleIndexConfigurationLatest(studyId, true).getVersion(); + return tableNameGenerator.getSampleIndexTableName(studyId, version); + } + + public String getSampleIndexTableName(int studyId, int version) { return tableNameGenerator.getSampleIndexTableName(studyId, version); } @@ -209,19 +205,20 @@ public VariantStorageMetadataManager getMetadataManager() { protected Map> queryByGt(int study, int sample, String chromosome, int position) throws IOException { - Result result = queryByGtInternal(study, sample, chromosome, position); - return newConverter(getSchema(study)).convertToMap(result); + SampleIndexSchema schema = schemaFactory.getSchema(study, sample, false); + Result result = queryByGtInternal(study, sample, chromosome, position, schema.getVersion()); + return newConverter(schema).convertToMap(result); } protected SampleIndexEntryPutBuilder queryByGtBuilder(int study, int sample, String chromosome, int position, SampleIndexSchema schema) throws IOException { - Result result = queryByGtInternal(study, sample, chromosome, position); + Result result = queryByGtInternal(study, sample, chromosome, position, schema.getVersion()); return new SampleIndexEntryPutBuilder(sample, chromosome, position, schema, newConverter(schema).convertToMapSampleVariantIndex(result)); } - private Result queryByGtInternal(int study, int sample, String chromosome, int position) throws IOException { - String tableName = getSampleIndexTableName(study); + private Result queryByGtInternal(int study, int sample, String chromosome, int position, int version) throws IOException { + String tableName = getSampleIndexTableName(study, version); return hBaseManager.act(tableName, table -> { Get get = new Get(SampleIndexSchema.toRowKey(sample, chromosome, position)); get.addFamily(family); @@ -230,8 +227,9 @@ private Result queryByGtInternal(int study, int sample, String chromosome, int p } public Iterator>> iteratorByGt(int study, int sample) throws IOException { - String tableName = getSampleIndexTableName(study); - HBaseToSampleIndexConverter converter = newConverter(getSchema(study)); + SampleIndexSchema schema = schemaFactory.getSchema(study, sample, false); + String tableName = getSampleIndexTableName(study, schema.getVersion()); + HBaseToSampleIndexConverter converter = newConverter(schema); return hBaseManager.act(tableName, table -> { Scan scan = new Scan(); scan.setRowPrefixFilter(SampleIndexSchema.toRowKey(sample)); @@ -250,8 +248,8 @@ public CloseableIterator rawIterator(int study, int sample) th } public CloseableIterator rawIterator(int study, int sample, Region region) throws IOException { - String tableName = getSampleIndexTableName(study); - + SampleIndexSchema schema = schemaFactory.getSchema(study, sample, false); + String tableName = getSampleIndexTableName(study, schema.getVersion()); return hBaseManager.act(tableName, table -> { Scan scan = new Scan(); if (region != null) { @@ -261,7 +259,8 @@ public CloseableIterator rawIterator(int study, int sample, Re } else { scan.setRowPrefixFilter(SampleIndexSchema.toRowKey(sample)); } - HBaseToSampleIndexConverter converter = newConverter(getSchema(study)); + + HBaseToSampleIndexConverter converter = newConverter(schema); ResultScanner scanner = table.getScanner(scan); Iterator resultIterator = scanner.iterator(); return CloseableIterator.wrap(Iterators.transform(resultIterator, converter::convert), scanner); @@ -277,7 +276,8 @@ public CloseableIterator rawIterator(int study, int sample, Re */ public RawSingleSampleIndexVariantDBIterator rawIterator(String study, String sample) { Query query = new Query(VariantQueryParam.STUDY.key(), study).append(VariantQueryParam.SAMPLE.key(), sample); - return rawInternalIterator(parseSampleIndexQuery(query).forSample(sample), getSchema(study)); + SingleSampleIndexQuery sampleIndexQuery = parseSampleIndexQuery(query).forSample(sample); + return rawInternalIterator(sampleIndexQuery, sampleIndexQuery.getSchema()); } public CloseableIterator rawIterator(Query query) throws IOException { @@ -290,7 +290,7 @@ public CloseableIterator rawIterator(SampleIndexQuery q if (samples.isEmpty()) { throw new VariantQueryException("At least one sample expected to query SampleIndex!"); } - SampleIndexSchema schema = getSchema(query.getStudy()); + SampleIndexSchema schema = query.getSchema(); QueryOperation operation = query.getQueryOperation(); if (samples.size() == 1) { @@ -349,7 +349,7 @@ public boolean isFastCount(SampleIndexQuery query) { } public long count(List regions, String study, String sample, List gts) { - return count(newParser(toStudyId(study)).parse(regions, study, sample, gts)); + return count(newParser().parse(regions, study, sample, gts)); } public long count(SampleIndexQuery query) { @@ -370,9 +370,10 @@ private long count(SingleSampleIndexQuery query) { locusQueries = query.getLocusQueries(); } - String tableName = getSampleIndexTableName(toStudyId(query.getStudy())); + int studyId = toStudyId(query.getStudy()); + String tableName = getSampleIndexTableName(studyId, query.getSchema().getVersion()); - HBaseToSampleIndexConverter converter = newConverter(getSchema(query.getStudy())); + HBaseToSampleIndexConverter converter = newConverter(query.getSchema()); try { return hBaseManager.act(tableName, table -> { long count = 0; @@ -439,28 +440,22 @@ protected HBaseToSampleIndexConverter newConverter(SampleIndexSchema schema) { return new HBaseToSampleIndexConverter(schema); } - public SampleIndexSchema getSchema(String study) { - int studyId = toStudyId(study); - return getSchema(studyId); - } - - public SampleIndexSchema getSchema(int studyId) { - SampleIndexConfiguration configuration = getSampleIndexConfiguration(studyId).getConfiguration(); - return new SampleIndexSchema(configuration); + public SampleIndexSchemaFactory getSchemaFactory() { + return schemaFactory; } - public StudyMetadata.SampleIndexConfigurationVersioned getSampleIndexConfiguration(int studyId) { - return metadataManager.getStudyMetadata(studyId).getSampleIndexConfigurationLatest(); + public SampleIndexSchema getSchemaLatest(Object study) { + return schemaFactory.getSchemaLatest(toStudyId(study)); } - protected int toStudyId(String study) { + protected int toStudyId(Object study) { int studyId; - if (StringUtils.isEmpty(study)) { + if (study == null || study instanceof String && ((String) study).isEmpty()) { Map studies = metadataManager.getStudies(null); if (studies.size() == 1) { studyId = studies.values().iterator().next(); } else { - throw VariantQueryException.studyNotFound(study, studies.keySet()); + throw VariantQueryException.missingStudy(studies.keySet()); } } else { studyId = metadataManager.getStudyId(study); @@ -536,18 +531,11 @@ && matchesWithBatch(locusQuery.getRegions().get(0)))) { } public SampleIndexQuery parseSampleIndexQuery(Query query) { - StudyMetadata defaultStudy = VariantQueryParser.getDefaultStudy(query, metadataManager); - int studyId; - if (defaultStudy == null) { - studyId = toStudyId(null); - } else { - studyId = defaultStudy.getId(); - } - return newParser(studyId).parse(query); + return newParser().parse(query); } - protected SampleIndexQueryParser newParser(int studyId) { - return new SampleIndexQueryParser(metadataManager, getSchema(studyId)); + protected SampleIndexQueryParser newParser() { + return new SampleIndexQueryParser(metadataManager, schemaFactory); } public Scan parse(SingleSampleIndexQuery query, LocusQuery regions) { @@ -636,7 +624,7 @@ private Scan parse(SingleSampleIndexQuery query, LocusQuery locusQuery, boolean scan.setCaching(hBaseManager.getConf().getInt("hbase.client.scanner.caching", 100)); logger.info("---------"); - logger.info("Sample = \"" + query.getSample() + "\""); + logger.info("Sample = \"" + query.getSample() + "\" , schema version = " + query.getSchema().getVersion()); printScan(scan); printQuery(locusQuery); printQuery(query); @@ -770,4 +758,128 @@ private int toSampleId(int studyId, String sample) { return metadataManager.getSampleId(studyId, sample); } + public boolean createTableIfNeeded(int studyId, int version, ObjectMap options) { + String sampleIndexTable = getSampleIndexTableName(studyId, version); + int files = options.getInt( + HadoopVariantStorageOptions.EXPECTED_FILES_NUMBER.key(), + HadoopVariantStorageOptions.EXPECTED_FILES_NUMBER.defaultValue()); + int samples = options.getInt( + HadoopVariantStorageOptions.EXPECTED_SAMPLES_NUMBER.key(), + files); + int preSplitSize = options.getInt( + HadoopVariantStorageOptions.SAMPLE_INDEX_TABLE_PRESPLIT_SIZE.key(), + HadoopVariantStorageOptions.SAMPLE_INDEX_TABLE_PRESPLIT_SIZE.defaultValue()); + + int splits = samples / preSplitSize; + ArrayList preSplits = new ArrayList<>(splits); + for (int i = 0; i < splits; i++) { + preSplits.add(SampleIndexSchema.toRowKey(i * preSplitSize)); + } + + Compression.Algorithm compression = Compression.getCompressionAlgorithmByName( + options.getString( + HadoopVariantStorageOptions.SAMPLE_INDEX_TABLE_COMPRESSION.key(), + HadoopVariantStorageOptions.SAMPLE_INDEX_TABLE_COMPRESSION.defaultValue())); + + try { + return hBaseManager.createTableIfNeeded(sampleIndexTable, GenomeHelper.COLUMN_FAMILY_BYTES, preSplits, compression); + } catch (IOException e) { + throw new UncheckedIOException(e); + } + } + + public void updateSampleIndexSchemaStatus(int studyId, int version) throws StorageEngineException { + StudyMetadata studyMetadata = metadataManager.getStudyMetadata(studyId); + if (studyMetadata.getSampleIndexConfiguration(version).getStatus() + != StudyMetadata.SampleIndexConfigurationVersioned.Status.STAGING) { + // Update only if status is STAGING + return; + } + boolean allSamplesWithThisVersion = isAllSamplesWithThisVersion(studyId, version); + if (allSamplesWithThisVersion) { + metadataManager.updateStudyMetadata(studyId, sm -> { + sm.getSampleIndexConfiguration(version) + .setStatus(StudyMetadata.SampleIndexConfigurationVersioned.Status.ACTIVE); + }); + } else { + logger.info("Not all samples had the sample index version {} on GENOTYPES and ANNOTATION", version); + } + } + + public void deprecateSampleIndexSchemas(int studyId) throws StorageEngineException { + StudyMetadata studyMetadata = metadataManager.getStudyMetadata(studyId); + Set deprecatables = new LinkedHashSet<>(); + for (StudyMetadata.SampleIndexConfigurationVersioned sampleIndexConfiguration : studyMetadata.getSampleIndexConfigurations()) { + if (sampleIndexConfiguration.getStatus() == StudyMetadata.SampleIndexConfigurationVersioned.Status.ACTIVE + || sampleIndexConfiguration.getStatus() == StudyMetadata.SampleIndexConfigurationVersioned.Status.STAGING) { + deprecatables.add(sampleIndexConfiguration.getVersion()); + } + } + StudyMetadata.SampleIndexConfigurationVersioned latest = studyMetadata.getSampleIndexConfigurationLatest(false); + if (latest == null) { + logger.info("No active sample index available for study '{}'", studyMetadata.getName()); + return; + } + int latestVersion = latest.getVersion(); + deprecatables.remove(latestVersion); + if (deprecatables.isEmpty()) { + logger.info("No sample indexes to deprecate for study '{}'", studyMetadata.getName()); + return; + } + + int totalSamples = 0; + int samplesNotInLatest = 0; + for (SampleMetadata sampleMetadata : metadataManager.sampleMetadataIterable(studyId)) { + // Only check indexed samples + if (sampleMetadata.getIndexStatus() == TaskMetadata.Status.READY) { + totalSamples++; + if (!sampleWithThisVersion(sampleMetadata, latestVersion)) { + samplesNotInLatest++; + } + } + } + if (samplesNotInLatest > 0) { + logger.warn("Unable to deprecate sample indexes. {} samples out of {} not in the latest sample index for study '{}'", + samplesNotInLatest, totalSamples, studyMetadata.getName()); + return; + } + logger.info("Deprecate sample indexes {} for study '{}'", deprecatables, studyMetadata.getName()); + metadataManager.updateStudyMetadata(studyId, sm -> { + for (Integer version : deprecatables) { + sm.getSampleIndexConfiguration(version).setStatus(StudyMetadata.SampleIndexConfigurationVersioned.Status.DEPRECATED); + } + }); + } + + private boolean isAllSamplesWithThisVersion(int studyId, int version) { + boolean allSamplesWithThisVersion = true; + for (SampleMetadata sampleMetadata : metadataManager.sampleMetadataIterable(studyId)) { + // Only check indexed samples + if (sampleMetadata.getIndexStatus() == TaskMetadata.Status.READY) { + allSamplesWithThisVersion = sampleWithThisVersion(sampleMetadata, version); + if (!allSamplesWithThisVersion) { + break; + } + } + } + return allSamplesWithThisVersion; + } + + private boolean sampleWithThisVersion(SampleMetadata sampleMetadata, int version) { + // Check SampleIndex + SampleIndexAnnotation + FamilyIndex (if needed) + if (sampleMetadata.getSampleIndexStatus(version) != TaskMetadata.Status.READY) { + return false; + } + if (sampleMetadata.getSampleIndexAnnotationStatus(version) != TaskMetadata.Status.READY) { + return false; + } + if (sampleMetadata.isFamilyIndexDefined()) { + if (sampleMetadata.getFamilyIndexStatus(version) != TaskMetadata.Status.READY) { + return false; + } + } + return true; + } + + } diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexDBLoader.java b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexDBLoader.java index 4259fc93565..daec9851343 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexDBLoader.java +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexDBLoader.java @@ -11,7 +11,6 @@ import org.opencb.opencga.core.common.YesNoAuto; import org.opencb.opencga.storage.core.io.bit.BitBuffer; import org.opencb.opencga.storage.core.metadata.VariantStorageMetadataManager; -import org.opencb.opencga.storage.core.metadata.models.StudyMetadata; import org.opencb.opencga.storage.core.variant.VariantStorageEngine.SplitData; import org.opencb.opencga.storage.core.variant.adaptors.GenotypeClass; import org.opencb.opencga.storage.hadoop.utils.AbstractHBaseDataWriter; @@ -23,7 +22,8 @@ import java.util.*; import static org.opencb.opencga.storage.core.variant.VariantStorageOptions.INCLUDE_GENOTYPE; -import static org.opencb.opencga.storage.hadoop.variant.index.sample.SampleIndexSchema.*; +import static org.opencb.opencga.storage.hadoop.variant.index.sample.SampleIndexSchema.getChunkStart; +import static org.opencb.opencga.storage.hadoop.variant.index.sample.SampleIndexSchema.validGenotype; /** * Created on 14/05/18. @@ -46,13 +46,12 @@ public class SampleIndexDBLoader extends AbstractHBaseDataWriter sampleIds, - SplitData splitData, ObjectMap options) { - super(hBaseManager, dbAdaptor.getSampleIndexTableName(studyId)); + SplitData splitData, ObjectMap options, SampleIndexSchema schema) { + super(hBaseManager, dbAdaptor.getSampleIndexTableName(studyId, schema.getVersion())); this.studyId = studyId; this.sampleIds = sampleIds; family = GenomeHelper.COLUMN_FAMILY_BYTES; @@ -88,9 +87,8 @@ public SampleIndexDBLoader(SampleIndexDBAdaptor dbAdaptor, HBaseManager hBaseMan } includeGenotype = YesNoAuto.parse(options, INCLUDE_GENOTYPE.key()).yesOrAuto(); this.dbAdaptor = dbAdaptor; - sampleIndexConfiguration = dbAdaptor.getSampleIndexConfiguration(studyId); - schema = dbAdaptor.getSchema(studyId); - variantFileIndexConverter = new VariantFileIndexConverter(schema); + this.schema = schema; + variantFileIndexConverter = new VariantFileIndexConverter(this.schema); } private class Chunk implements Iterable { @@ -186,7 +184,7 @@ public String toString() { @Override public boolean open() { super.open(); - SampleIndexSchema.createTableIfNeeded(tableName, hBaseManager, options); + dbAdaptor.createTableIfNeeded(studyId, schema.getVersion(), options); return true; } @@ -273,6 +271,6 @@ public HashSet getLoadedGenotypes() { } public int getSampleIndexVersion() { - return sampleIndexConfiguration.getVersion(); + return schema.getVersion(); } } diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexDriver.java b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexDriver.java index 143f2ad7924..66b9bbf970b 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexDriver.java +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexDriver.java @@ -18,8 +18,7 @@ import org.apache.hadoop.mapreduce.MRJobConfig; import org.opencb.biodata.models.core.Region; import org.opencb.biodata.models.variant.Variant; -import org.opencb.commons.datastore.core.ObjectMap; -import org.opencb.opencga.storage.core.exceptions.StorageEngineException; +import org.opencb.opencga.core.config.storage.SampleIndexConfiguration; import org.opencb.opencga.storage.core.io.bit.BitBuffer; import org.opencb.opencga.storage.core.metadata.VariantStorageMetadataManager; import org.opencb.opencga.storage.core.metadata.models.SampleMetadata; @@ -82,6 +81,7 @@ public class SampleIndexDriver extends AbstractVariantsTableDriver { private static final String FIXED_ATTRIBUTES = "SampleIndexDriver.fixedAttributes"; private static final String FIXED_SAMPLE_DATA_KEYS = "SampleIndexDriver.fixedSampleDataKeys"; private static final String PARTIAL_SCAN = "SampleIndexDriver.partial_scan"; + public static final String SAMPLE_INDEX_VERSION = "sample-index-version"; private int study; private String outputTable; @@ -116,6 +116,7 @@ protected Map getParams() { params.put("--" + SAMPLE_IDS, ""); params.put("--" + VariantStorageOptions.STUDY.key(), ""); params.put("--" + OUTPUT, ""); + params.put("--" + SAMPLE_INDEX_VERSION, ""); params.put("--" + SECONDARY_ONLY, ""); // params.put("--" + MAIN_ONLY, ""); params.put("--" + VariantQueryParam.REGION.key(), ""); @@ -331,20 +332,13 @@ protected Job setupJob(Job job, String archiveTable, String table) throws IOExce } StudyMetadata studyMetadata = getMetadataManager().getStudyMetadata(getStudyId()); - VariantMapReduceUtil.setSampleIndexConfiguration(job, studyMetadata.getSampleIndexConfigurationLatest().getConfiguration()); + int sampleIndexVersion = Integer.parseInt(getParam(SAMPLE_INDEX_VERSION)); + SampleIndexConfiguration configuration = studyMetadata.getSampleIndexConfiguration(sampleIndexVersion).getConfiguration(); + VariantMapReduceUtil.setSampleIndexConfiguration(job, configuration, sampleIndexVersion); return job; } - @Override - protected void preExecution() throws IOException, StorageEngineException { - super.preExecution(); - - ObjectMap options = new ObjectMap(); - options.putAll(getParams()); - SampleIndexSchema.createTableIfNeeded(outputTable, getHBaseManager(), options); - } - public static void main(String[] args) throws Exception { try { System.exit(new SampleIndexDriver().privateMain(args, null)); @@ -373,7 +367,7 @@ public static class SampleIndexerMapper extends VariantTableSampleIndexOrderMapp @Override protected void setup(Context context) throws IOException, InterruptedException { hasGenotype = context.getConfiguration().getBoolean(HAS_GENOTYPE, true); - schema = new SampleIndexSchema(VariantMapReduceUtil.getSampleIndexConfiguration(context.getConfiguration())); + schema = VariantMapReduceUtil.getSampleIndexSchema(context.getConfiguration()); fileIndexConverter = new VariantFileIndexConverter(schema); int[] sampleIds = context.getConfiguration().getInts(SAMPLES); diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexLoader.java b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexLoader.java index cb79073a7e9..3778b4d047a 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexLoader.java +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexLoader.java @@ -26,23 +26,27 @@ public class SampleIndexLoader { private final SampleIndexDBAdaptor sampleIndexDBAdaptor; private final HBaseVariantTableNameGenerator tableNameGenerator; + private final String study; private final MRExecutor mrExecutor; private final VariantStorageMetadataManager metadataManager; + private final SampleIndexSchema schema; private Logger logger = LoggerFactory.getLogger(SampleIndexLoader.class); - public SampleIndexLoader(SampleIndexDBAdaptor sampleIndexDBAdaptor, MRExecutor mrExecutor) { + public SampleIndexLoader(SampleIndexDBAdaptor sampleIndexDBAdaptor, String study, MRExecutor mrExecutor) { this.sampleIndexDBAdaptor = sampleIndexDBAdaptor; + this.study = study; this.mrExecutor = mrExecutor; this.metadataManager = sampleIndexDBAdaptor.getMetadataManager(); this.tableNameGenerator = sampleIndexDBAdaptor.getTableNameGenerator(); + schema = sampleIndexDBAdaptor.getSchemaLatest(study); } - public void buildSampleIndex(String study, List samples, ObjectMap options) + public void buildSampleIndex(List samples, ObjectMap options) throws StorageEngineException { - buildSampleIndex(study, samples, options, options.getBoolean(OVERWRITE, false)); + buildSampleIndex(samples, options, options.getBoolean(OVERWRITE, false)); } - public void buildSampleIndex(String study, List samples, ObjectMap options, boolean overwrite) + public void buildSampleIndex(List samples, ObjectMap options, boolean overwrite) throws StorageEngineException { int studyId = metadataManager.getStudyId(study); List sampleIds; @@ -58,12 +62,12 @@ public void buildSampleIndex(String study, List samples, ObjectMap optio sampleIds.add(sampleId); } } - int version = metadataManager.getStudyMetadata(studyId).getSampleIndexConfigurationLatest().getVersion(); + int version = schema.getVersion(); List finalSamplesList = new ArrayList<>(samples.size()); List alreadyIndexed = new LinkedList<>(); for (Integer sampleId : sampleIds) { SampleMetadata sampleMetadata = metadataManager.getSampleMetadata(studyId, sampleId); - if (overwrite || SampleIndexDBAdaptor.getSampleIndexStatus(sampleMetadata, version) != TaskMetadata.Status.READY) { + if (overwrite || sampleMetadata.getSampleIndexStatus(version) != TaskMetadata.Status.READY) { finalSamplesList.add(sampleId); } else { // SamplesIndex already annotated @@ -79,6 +83,9 @@ public void buildSampleIndex(String study, List samples, ObjectMap optio logger.info("Skip sample index build. Nothing to do!"); return; } + + sampleIndexDBAdaptor.createTableIfNeeded(studyId, schema.getVersion(), options); + if (finalSamplesList.size() < 20) { logger.info("Run sample index build on samples " + finalSamplesList); } else { @@ -99,18 +106,19 @@ public void buildSampleIndex(String study, List samples, ObjectMap optio for (int i = 0; i < batches; i++) { List subSet = finalSamplesList.subList(i * batchSize, Math.min((i + 1) * batchSize, finalSamplesList.size())); logger.info("Running MapReduce {}/{} over {} samples", i + 1, batches, subSet.size()); - buildSampleIndexBatchMapreduce(studyId, subSet, options, version); + buildSampleIndexBatchMapreduce(studyId, subSet, options); } } else { - buildSampleIndexBatchMapreduce(studyId, finalSamplesList, options, version); + buildSampleIndexBatchMapreduce(studyId, finalSamplesList, options); } } - private void buildSampleIndexBatchMapreduce(int studyId, List samples, ObjectMap options, int version) + private void buildSampleIndexBatchMapreduce(int studyId, List samples, ObjectMap options) throws StorageEngineException { options = new ObjectMap(options); options.put(SampleIndexDriver.SAMPLE_IDS, samples); - options.put(SampleIndexDriver.OUTPUT, sampleIndexDBAdaptor.getSampleIndexTableName(studyId)); + options.put(SampleIndexDriver.OUTPUT, sampleIndexDBAdaptor.getSampleIndexTableName(studyId, schema.getVersion())); + options.put(SampleIndexDriver.SAMPLE_INDEX_VERSION, schema.getVersion()); mrExecutor.run(SampleIndexDriver.class, SampleIndexDriver.buildArgs( @@ -120,13 +128,13 @@ private void buildSampleIndexBatchMapreduce(int studyId, List samples, null, options), "Build sample index for " + (samples.size() < 10 ? "samples " + samples : samples.size() + " samples")); - postSampleIndexBuild(studyId, samples, version); + postSampleIndexBuild(studyId, samples); } - private void postSampleIndexBuild(int studyId, List samples, int version) throws StorageEngineException { + private void postSampleIndexBuild(int studyId, List samples) throws StorageEngineException { for (Integer sampleId : samples) { metadataManager.updateSampleMetadata(studyId, sampleId, sampleMetadata -> { - return SampleIndexDBAdaptor.setSampleIndexStatus(sampleMetadata, TaskMetadata.Status.READY, version); + sampleMetadata.setSampleIndexStatus(TaskMetadata.Status.READY, schema.getVersion()); }); } } diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexQueryParser.java b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexQueryParser.java index b471b50aea4..22f1eba45bf 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexQueryParser.java +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexQueryParser.java @@ -29,7 +29,6 @@ import org.opencb.opencga.storage.hadoop.variant.index.core.filters.RangeIndexFieldFilter; import org.opencb.opencga.storage.hadoop.variant.index.family.GenotypeCodec; import org.opencb.opencga.storage.hadoop.variant.index.query.*; -import org.opencb.opencga.storage.hadoop.variant.index.query.LocusQuery; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -48,12 +47,17 @@ */ public class SampleIndexQueryParser { private static Logger logger = LoggerFactory.getLogger(SampleIndexQueryParser.class); - private final SampleIndexSchema schema; private final VariantStorageMetadataManager metadataManager; + private final SampleIndexSchemaFactory schemaFactory; - public SampleIndexQueryParser(VariantStorageMetadataManager metadataManager, SampleIndexSchema schema) { - this.schema = schema; + public SampleIndexQueryParser(VariantStorageMetadataManager metadataManager) { this.metadataManager = metadataManager; + this.schemaFactory = new SampleIndexSchemaFactory(metadataManager); + } + + public SampleIndexQueryParser(VariantStorageMetadataManager metadataManager, SampleIndexSchemaFactory schemaFactory) { + this.metadataManager = metadataManager; + this.schemaFactory = schemaFactory; } /** @@ -146,8 +150,10 @@ public SampleIndexQuery parse(Query query) { boolean partialIndex = false; QueryOperation queryOperation; + // Unprocessed input genotype query + Map> sampleGenotypeInputQuery = new HashMap<>(); // Map from all samples to query to its list of genotypes. - Map> samplesMap = new HashMap<>(); + Map> sampleGenotypeQuery = new HashMap<>(); // Samples that are returning data from more than one file Set multiFileSamples = new HashSet<>(); // Samples that are querying @@ -167,71 +173,32 @@ public SampleIndexQuery parse(Query query) { boolean onlyDeNovo = false; boolean partialGtIndex = false; + Map sampleMetadatas = new HashMap<>(); + + // Extract sample and genotypes to filter if (isValidParam(query, GENOTYPE)) { // Get samples with non negated genotypes - Map> map = new HashMap<>(); - Map allSamples = new HashMap<>(); queryOperation = parseGenotypeFilter(query.getString(GENOTYPE.key()), map); - // Extract parents from each sample - Map> gtMap = new HashMap<>(); - Map> parentsMap = new HashMap<>(); for (Map.Entry> entry : map.entrySet()) { Object sample = entry.getKey(); Integer sampleId = metadataManager.getSampleId(studyId, sample); SampleMetadata sampleMetadata = metadataManager.getSampleMetadata(studyId, sampleId); - allSamples.put(sampleMetadata.getName(), sampleMetadata); + String sampleName = sampleMetadata.getName(); + sampleMetadatas.put(sampleName, sampleMetadata); - List gts = GenotypeClass.filter(entry.getValue(), allGenotypes); - if (gts.stream().allMatch(SampleIndexSchema::validGenotype)) { - if (sampleMetadata.getFamilyIndexStatus() == TaskMetadata.Status.READY) { - String fatherName = null; - if (sampleMetadata.getFather() != null) { - fatherName = metadataManager.getSampleName(studyId, sampleMetadata.getFather()); - } - String motherName = null; - if (sampleMetadata.getMother() != null) { - motherName = metadataManager.getSampleName(studyId, sampleMetadata.getMother()); - } - if (fatherName != null || motherName != null) { - parentsMap.put(sampleMetadata.getName(), Arrays.asList(fatherName, motherName)); - } - } - } else { - negatedSamples.add(sampleMetadata.getName()); - } - if (VariantStorageEngine.SplitData.MULTI.equals(sampleMetadata.getSplitData())) { - multiFileSamples.add(sampleMetadata.getName()); + if (VariantStorageEngine.SplitData.MULTI == sampleMetadata.getSplitData()) { + multiFileSamples.add(sampleName); } - gtMap.put(sampleMetadata.getName(), gts); - } - - // Determine which samples are parents, and which are children - Set childrenSet = findChildren(gtMap, queryOperation, parentsMap); - Set parentsSet = new HashSet<>(); - for (String child : childrenSet) { - // may add null values - parentsSet.addAll(parentsMap.get(child)); - } - - for (Map.Entry> entry : gtMap.entrySet()) { - String sampleName = entry.getKey(); - if (queryOperation != QueryOperation.OR && parentsSet.contains(sampleName) && !childrenSet.contains(sampleName)) { - // We can skip parents, as their genotype filter will be tested in the child - // Discard parents that are not children of another sample - // Parents filter can only be used when intersecting (AND) with child - logger.debug("Discard parent {}", sampleName); - parentsInQuery.add(sampleName); - - // Remove from negatedSamples (if present) - negatedSamples.remove(sampleName); - - continue; + List gts = GenotypeClass.filter(entry.getValue(), allGenotypes); + if (!gts.stream().allMatch(SampleIndexSchema::validGenotype)) { + negatedSamples.add(sampleName); } + if (hasNegatedGenotypeFilter(queryOperation, entry.getValue())) { // Discard samples with negated genotypes negatedGenotypesSamples.add(sampleName); @@ -239,71 +206,9 @@ public SampleIndexQuery parse(Query query) { logger.info("Set partialGtIndex to true. Prev value: {}", partialGtIndex); partialGtIndex = true; } else { - samplesMap.put(sampleName, entry.getValue()); - if (queryOperation != QueryOperation.OR && childrenSet.contains(sampleName)) { - // Parents filter can only be used when intersecting (AND) with child - List parents = parentsMap.get(sampleName); - String father = parents.get(0); - String mother = parents.get(1); - - if (father != null) { - Integer fatherId = metadataManager.getSampleId(studyId, father); - boolean includeDiscrepancies = VariantStorageEngine.SplitData.MULTI - .equals(metadataManager.getLoadSplitData(studyId, fatherId)); - List fatherFiles = allSamples.get(father).getFiles(); - List sampleFiles = allSamples.get(sampleName).getFiles(); - boolean parentInSeparatedFile = - fatherFiles.size() != sampleFiles.size() || !fatherFiles.containsAll(sampleFiles); - boolean[] filter = buildParentGtFilter(gtMap.get(father), includeDiscrepancies, parentInSeparatedFile); - if (!isFullyCoveredParentFilter(filter)) { - logger.debug("FATHER - Set partialGtIndex to true. Prev value: {}", partialGtIndex); - partialGtIndex = true; - } -// logger.info("Father={}, includeDiscrepancies={}, fatherFiles={}, sampleFiles={}, " -// + "parentInSeparatedFile={}, fullyCoveredFilter={}", -// father, includeDiscrepancies, fatherFiles, sampleFiles, -// parentInSeparatedFile, isFullyCoveredParentFilter(filter)); - fatherFilterMap.put(sampleName, filter); - } - if (mother != null) { - Integer motherId = metadataManager.getSampleId(studyId, mother); - boolean includeDiscrepancies = VariantStorageEngine.SplitData.MULTI - .equals(metadataManager.getLoadSplitData(studyId, motherId)); - List motherFiles = allSamples.get(mother).getFiles(); - List sampleFiles = allSamples.get(sampleName).getFiles(); - boolean parentInSeparatedFile = - motherFiles.size() != sampleFiles.size() || !motherFiles.containsAll(sampleFiles); - boolean[] filter = buildParentGtFilter(gtMap.get(mother), includeDiscrepancies, parentInSeparatedFile); - if (!isFullyCoveredParentFilter(filter)) { - logger.debug("MOTHER - Set partialGtIndex to true. Prev value: {}", partialGtIndex); - partialGtIndex = true; - } -// logger.info("Mother={}, includeDiscrepancies={}, motherFiles={}, sampleFiles={}, " -// + "parentInSeparatedFile={}, fullyCoveredFilter={}", -// mother, includeDiscrepancies, motherFiles, sampleFiles, -// parentInSeparatedFile, isFullyCoveredParentFilter(filter)); - motherFilterMap.put(sampleName, filter); - } - } - } - } - // If not all genotypes are valid, query is not covered - if (!negatedSamples.isEmpty()) { - logger.debug("NEG_SAMPLES - Set partialGtIndex to true. Prev value: {}", partialGtIndex); - partialGtIndex = true; - } - - for (String negatedSample : negatedSamples) { - List negatedGenotypes = new ArrayList<>(validGenotypes); - negatedGenotypes.removeAll(samplesMap.get(negatedSample)); - samplesMap.put(negatedSample, negatedGenotypes); - } - - if (!partialGtIndex) { - // Do not remove genotypes list if FORMAT is present. - if (!isValidParam(query, SAMPLE_DATA)) { - query.remove(GENOTYPE.key()); + sampleGenotypeQuery.put(sampleName, entry.getValue()); } + sampleGenotypeInputQuery.put(sampleName, gts); } } else if (isValidParam(query, SAMPLE)) { // Filter by all non negated samples @@ -312,7 +217,8 @@ public SampleIndexQuery parse(Query query) { List samples = VariantQueryUtils.splitValue(samplesStr, queryOperation); for (String s : samples) { if (!isNegated(s)) { - samplesMap.put(s, mainGenotypes); + sampleGenotypeQuery.put(s, mainGenotypes); + sampleGenotypeInputQuery.put(s, mainGenotypes); } } @@ -327,7 +233,7 @@ public SampleIndexQuery parse(Query query) { queryOperation = mendelianError.getOperation(); for (String s : mendelianErrorSet) { // Return any genotype - samplesMap.put(s, mainGenotypes); + sampleGenotypeQuery.put(s, mainGenotypes); } query.remove(SAMPLE_MENDELIAN_ERROR.key()); // Reading any MendelianError could return variants from GT=0/0, which is not annotated in the SampleIndex, @@ -340,7 +246,7 @@ public SampleIndexQuery parse(Query query) { queryOperation = sampleDeNovo.getOperation(); for (String s : mendelianErrorSet) { // Return any genotype - samplesMap.put(s, mainGenotypes); + sampleGenotypeQuery.put(s, mainGenotypes); } query.remove(SAMPLE_DE_NOVO.key()); //} else if (isValidParam(query, FILE)) { @@ -348,19 +254,157 @@ public SampleIndexQuery parse(Query query) { } else { throw new IllegalStateException("Unable to query SamplesIndex"); } + boolean requireFamilyIndex = !mendelianErrorSet.isEmpty(); + SampleIndexSchema schema = schemaFactory.getSchema(studyId, sampleGenotypeQuery.keySet(), false, requireFamilyIndex); + + + if (queryOperation == QueryOperation.AND && mendelianErrorSet.isEmpty()) { + // Parents filter can only be used when intersecting (AND) with child + // And if not filtering by mendelian error + Map> parentsMap = new HashMap<>(); + + // Get parents "tree" from valid samples filter + for (String sampleName : sampleGenotypeQuery.keySet()) { + if (!negatedSamples.contains(sampleName)) { + SampleMetadata sampleMetadata = getSampleMetadata(sampleMetadatas, sampleName, studyId); + + if (sampleMetadata.getFamilyIndexStatus(schema.getVersion()) == TaskMetadata.Status.READY) { + String fatherName = null; + if (sampleMetadata.getFather() != null) { + fatherName = metadataManager.getSampleName(studyId, sampleMetadata.getFather()); + } + String motherName = null; + if (sampleMetadata.getMother() != null) { + motherName = metadataManager.getSampleName(studyId, sampleMetadata.getMother()); + } + if (fatherName != null || motherName != null) { + parentsMap.put(sampleMetadata.getName(), Arrays.asList(fatherName, motherName)); + } + } + } + } + + // Determine which samples are parents, and which are children + Set childrenSet = findChildren(sampleGenotypeQuery, parentsMap); + Set parentsSet = findParents(childrenSet, parentsMap); + + // Copy list of samples, so we can modify the map internally + for (String sampleName : new ArrayList<>(sampleGenotypeQuery.keySet())) { + if (parentsSet.contains(sampleName) && !childrenSet.contains(sampleName)) { + parentsInQuery.add(sampleName); + // We can skip parents, as their genotype filter will be tested in the child + // Discard parents that are not children of another sample + // Parents filter can only be used when intersecting (AND) with child + boolean discardParent = true; + + // We should avoid discarding parents if there were other sampleData filters covered by the parent sample index + // Samples of this map will determine which set of sampleIndexes should be read + if (isValidParam(query, SAMPLE_DATA)) { + ParsedQuery>> sampleDataParsedQuery = parseSampleData(query); + KeyValues> sampleDataFilter + = sampleDataParsedQuery.getValue(k -> k.getKey().equals(sampleName)); + if (sampleDataFilter != null) { + // SampleData filter exists for this sample! + // Check if ANY filter is covered by the index + for (KeyOpValue entry : sampleDataFilter.getValues()) { + if (schema.getFileIndex().getField(IndexFieldConfiguration.Source.SAMPLE, entry.getKey()) != null) { + // This key is covered by the sample index. Do not discard this parent! + discardParent = false; + break; + } + } + } + } + + if (discardParent) { + logger.debug("Discard parent {}", sampleName); + // Remove from negatedSamples (if present) + negatedSamples.remove(sampleName); + sampleGenotypeQuery.remove(sampleName); + } + } else if (childrenSet.contains(sampleName)) { + // Parents filter can only be used when intersecting (AND) with child + List parents = parentsMap.get(sampleName); + String father = parents.get(0); + String mother = parents.get(1); + + if (father != null) { + Integer fatherId = metadataManager.getSampleId(studyId, father); + boolean includeDiscrepancies = VariantStorageEngine.SplitData.MULTI + == metadataManager.getLoadSplitData(studyId, fatherId); + List fatherFiles = sampleMetadatas.get(father).getFiles(); + List sampleFiles = sampleMetadatas.get(sampleName).getFiles(); + boolean parentInSeparatedFile = + fatherFiles.size() != sampleFiles.size() || !fatherFiles.containsAll(sampleFiles); + boolean[] filter = buildParentGtFilter( + sampleGenotypeInputQuery.get(father), + includeDiscrepancies, + parentInSeparatedFile); + if (!isFullyCoveredParentFilter(filter)) { + logger.debug("FATHER - Set partialGtIndex to true. Prev value: {}", partialGtIndex); + partialGtIndex = true; + } +// logger.info("Father={}, includeDiscrepancies={}, fatherFiles={}, sampleFiles={}, " +// + "parentInSeparatedFile={}, fullyCoveredFilter={}", +// father, includeDiscrepancies, fatherFiles, sampleFiles, +// parentInSeparatedFile, isFullyCoveredParentFilter(filter)); + fatherFilterMap.put(sampleName, filter); + } + if (mother != null) { + Integer motherId = metadataManager.getSampleId(studyId, mother); + boolean includeDiscrepancies = VariantStorageEngine.SplitData.MULTI + .equals(metadataManager.getLoadSplitData(studyId, motherId)); + List motherFiles = sampleMetadatas.get(mother).getFiles(); + List sampleFiles = sampleMetadatas.get(sampleName).getFiles(); + boolean parentInSeparatedFile = + motherFiles.size() != sampleFiles.size() || !motherFiles.containsAll(sampleFiles); + boolean[] filter = buildParentGtFilter( + sampleGenotypeInputQuery.get(mother), + includeDiscrepancies, + parentInSeparatedFile); + if (!isFullyCoveredParentFilter(filter)) { + logger.debug("MOTHER - Set partialGtIndex to true. Prev value: {}", partialGtIndex); + partialGtIndex = true; + } +// logger.info("Mother={}, includeDiscrepancies={}, motherFiles={}, sampleFiles={}, " +// + "parentInSeparatedFile={}, fullyCoveredFilter={}", +// mother, includeDiscrepancies, motherFiles, sampleFiles, +// parentInSeparatedFile, isFullyCoveredParentFilter(filter)); + motherFilterMap.put(sampleName, filter); + } + } + } + } + + // If not all genotypes are valid, query is not covered + if (!negatedSamples.isEmpty()) { + logger.debug("NEG_SAMPLES - Set partialGtIndex to true. Prev value: {}", partialGtIndex); + partialGtIndex = true; + } + for (String negatedSample : negatedSamples) { + List negatedGenotypes = new ArrayList<>(validGenotypes); + negatedGenotypes.removeAll(sampleGenotypeQuery.get(negatedSample)); + sampleGenotypeQuery.put(negatedSample, negatedGenotypes); + } + if (!partialGtIndex) { + if (isValidParam(query, GENOTYPE)) { + // Do not remove genotypes list if FORMAT is present. + if (!isValidParam(query, SAMPLE_DATA)) { + query.remove(GENOTYPE.key()); + } + } + } boolean partialFilesIndex = false; if (!negatedGenotypesSamples.isEmpty() || !parentsInQuery.isEmpty()) { - Set sampleFiles = new HashSet<>(samplesMap.size()); - for (String sample : samplesMap.keySet()) { - Integer sampleId = metadataManager.getSampleId(studyId, sample); - sampleFiles.addAll(metadataManager.getSampleMetadata(studyId, sampleId).getFiles()); + Set sampleFiles = new HashSet<>(sampleGenotypeQuery.size()); + for (String sample : sampleGenotypeQuery.keySet()) { + sampleFiles.addAll(getSampleMetadata(sampleMetadatas, sample, studyId).getFiles()); } // If the file of any other sample is not between the files of the samples in the query, mark as partial for (String sample : negatedGenotypesSamples) { - Integer sampleId = metadataManager.getSampleId(studyId, sample); - for (Integer file : metadataManager.getSampleMetadata(studyId, sampleId).getFiles()) { + for (Integer file : getSampleMetadata(sampleMetadatas, sample, studyId).getFiles()) { if (!sampleFiles.contains(file)) { partialFilesIndex = true; break; @@ -369,8 +413,7 @@ public SampleIndexQuery parse(Query query) { } // If the file of any parent is not between the files of the samples in the query, mark as partial for (String sample : parentsInQuery) { - Integer sampleId = metadataManager.getSampleId(studyId, sample); - for (Integer file : metadataManager.getSampleMetadata(studyId, sampleId).getFiles()) { + for (Integer file : getSampleMetadata(sampleMetadatas, sample, studyId).getFiles()) { if (!sampleFiles.contains(file)) { partialFilesIndex = true; break; @@ -380,16 +423,15 @@ public SampleIndexQuery parse(Query query) { } Map> fileIndexMap - = parseSampleSpecificQuery(query, studyId, queryOperation, samplesMap, multiFileSamples, partialGtIndex, partialFilesIndex); + = parseSampleSpecificQuery(schema, query, studyId, queryOperation, sampleGenotypeQuery, multiFileSamples, + partialGtIndex, partialFilesIndex); - int sampleIndexVersion = defaultStudy.getSampleIndexConfigurationLatest().getVersion(); + // It might happen that some sampleIndexes are not fully annotated. boolean allSamplesAnnotated = true; if (negatedGenotypesSamples.isEmpty()) { - for (String sample : samplesMap.keySet()) { - Integer sampleId = metadataManager.getSampleId(studyId, sample); - SampleMetadata sampleMetadata = metadataManager.getSampleMetadata(studyId, sampleId); - if (!SampleIndexDBAdaptor.getSampleIndexAnnotationStatus(sampleMetadata, sampleIndexVersion) - .equals(TaskMetadata.Status.READY)) { + for (String sample : sampleGenotypeQuery.keySet()) { + SampleMetadata sampleMetadata = getSampleMetadata(sampleMetadatas, sample, studyId); + if (sampleMetadata.getSampleIndexAnnotationStatus(schema.getVersion()) != TaskMetadata.Status.READY) { allSamplesAnnotated = false; break; } @@ -398,8 +440,9 @@ public SampleIndexQuery parse(Query query) { allSamplesAnnotated = false; } + boolean completeIndex = allSamplesAnnotated && !partialIndex; - SampleAnnotationIndexQuery annotationIndexQuery = parseAnnotationIndexQuery(query, completeIndex); + SampleAnnotationIndexQuery annotationIndexQuery = parseAnnotationIndexQuery(schema, query, completeIndex); Set variantTypes = null; if (isValidParam(query, TYPE)) { List typesStr = query.getAsStringList(VariantQueryParam.TYPE.key()); @@ -469,11 +512,31 @@ public SampleIndexQuery parse(Query query) { Collection regionGroups = buildLocusQueries(regions, variants); - return new SampleIndexQuery(schema, regionGroups, variantTypes, study, samplesMap, multiFileSamples, negatedSamples, + return new SampleIndexQuery(schema, regionGroups, variantTypes, study, sampleGenotypeQuery, multiFileSamples, negatedSamples, fatherFilterMap, motherFilterMap, fileIndexMap, annotationIndexQuery, mendelianErrorSet, onlyDeNovo, queryOperation); } + private Set findParents(Set childrenSet, Map> parentsMap) { + Set parentsSet = new HashSet<>(); + for (String child : childrenSet) { + // may add null values + parentsSet.addAll(parentsMap.get(child)); + } + return parentsSet; + } + + private SampleMetadata getSampleMetadata(Map sampleMetadatas, String sampleName, int studyId) { + SampleMetadata sampleMetadata = sampleMetadatas.get(sampleName); + if (sampleMetadata == null) { + Integer sampleId = metadataManager.getSampleId(studyId, sampleName); + sampleMetadata = metadataManager.getSampleMetadata(studyId, sampleId); + sampleMetadatas.put(sampleName, sampleMetadata); + } + return sampleMetadata; + } + + /** * Merge and group regions by chunk start. * Concurrent groups of regions are merged into one single group. @@ -543,27 +606,25 @@ protected static boolean hasNegatedGenotypeFilter(QueryOperation queryOperation, * * i.e. sample with non negated genotype filter and parents in the query * - * @param gtMap Genotype filter map - * @param queryOperation Query operation + * @param samplesMap Genotype filter map * @param parentsMap Parents map * @return Set with all children from the query */ - protected Set findChildren(Map> gtMap, QueryOperation queryOperation, - Map> parentsMap) { + protected Set findChildren(Map> samplesMap, Map> parentsMap) { Set childrenSet = new HashSet<>(parentsMap.size()); for (Map.Entry> entry : parentsMap.entrySet()) { String child = entry.getKey(); List parents = entry.getValue(); - if (hasNegatedGenotypeFilter(queryOperation, gtMap.get(child))) { - // Discard children with negated iterators - continue; - } +// if (hasNegatedGenotypeFilter(queryOperation, gtMap.get(child))) { +// // Discard children with negated iterators +// continue; +// } // Remove parents not in query for (int i = 0; i < parents.size(); i++) { String parent = parents.get(i); - if (!gtMap.containsKey(parent)) { + if (!samplesMap.containsKey(parent)) { parents.set(i, null); } } @@ -609,10 +670,9 @@ public static boolean isFullyCoveredParentFilter(boolean[] filter) { - private Map> parseSampleSpecificQuery(Query query, int studyId, QueryOperation queryOperation, - Map> samplesMap, - Set multiFileSamples, - boolean partialGtIndex, boolean partialFilesIndex) { + private Map> parseSampleSpecificQuery( + SampleIndexSchema schema, Query query, int studyId, QueryOperation queryOperation, + Map> samplesMap, Set multiFileSamples, boolean partialGtIndex, boolean partialFilesIndex) { // 1. Split query -- getQueriesPerSample // 2. Parse Files Query // 2.1. Parse File query @@ -624,8 +684,8 @@ private Map> parseSampleSpecificQuery(Query Map queriesPerSample = getQueriesPerSample(studyId, query, queryOperation, samplesMap.keySet(), nonCoveredQuery); for (String sample : samplesMap.keySet()) { Query sampleQuery = queriesPerSample.get(sample); - Values fileIndexQuery = - parseFilesQuery(sampleQuery, studyId, sample, multiFileSamples.contains(sample), partialGtIndex, partialFilesIndex); + Values fileIndexQuery = parseFilesQuery( + schema, sampleQuery, studyId, sample, multiFileSamples.contains(sample), partialGtIndex, partialFilesIndex); fileIndexMap.put(sample, fileIndexQuery); } @@ -792,9 +852,16 @@ private Map getQueriesPerSample(int studyId, Query query, QueryOp } - protected Values parseFilesQuery(Query query, int studyId, String sample, + protected Values parseFilesQuery(SampleIndexSchema schema, Query query, int studyId, String sample, boolean multiFileSample, boolean partialGtIndex, boolean partialFilesIndex) { - return parseFilesQuery(query, sample, multiFileSample, partialGtIndex, partialFilesIndex, s -> getFilesFromSample(studyId, s)); + return parseFilesQuery( + schema, + query, + sample, + multiFileSample, + partialGtIndex, + partialFilesIndex, + s -> getFilesFromSample(studyId, s)); } private List getFilesFromSample(int studyId, String s) { @@ -807,7 +874,7 @@ private List getFilesFromSample(int studyId, String s) { return fileNames; } - protected Values parseFilesQuery(Query query, String sample, boolean multiFileSample, + protected Values parseFilesQuery(SampleIndexSchema schema, Query query, String sample, boolean multiFileSample, boolean partialGtIndex, boolean partialFilesIndex, Function> filesFromSample) { ParsedQuery>> fileDataParsedQuery = parseFileData(query); @@ -850,7 +917,8 @@ protected Values parseFilesQuery(Query query, String sampl } else { subQuery.put(FILE_DATA.key(), fileDataQuery.toQuery()); } - fileIndexQueries.add(parseFileQuery(subQuery, sample, multiFileSample, partialGtIndex, partialFilesIndex, filesFromSample)); + fileIndexQueries.add(parseFileQuery(schema, subQuery, sample, multiFileSample, + partialGtIndex, partialFilesIndex, filesFromSample)); if (isValidParam(subQuery, FILE_DATA)) { // This subquery did not remove the fileData, so it's not fully covered. Can't remove the fileData filter. fileDataCovered = false; @@ -888,11 +956,11 @@ protected Values parseFilesQuery(Query query, String sampl return new Values<>(filesOperation, fileIndexQueries); } else { return new Values<>(null, Collections.singletonList( - parseFileQuery(query, sample, multiFileSample, partialGtIndex, partialFilesIndex, filesFromSample))); + parseFileQuery(schema, query, sample, multiFileSample, partialGtIndex, partialFilesIndex, filesFromSample))); } } - protected SampleFileIndexQuery parseFileQuery(Query query, String sample, boolean multiFileSample, + protected SampleFileIndexQuery parseFileQuery(SampleIndexSchema schema, Query query, String sample, boolean multiFileSample, boolean partialGtIndex, boolean partialFilesIndex, Function> filesFromSample) { @@ -1089,19 +1157,20 @@ private boolean hasCopyNumberLossFilter(List types) { return types.contains(VariantType.COPY_NUMBER_LOSS.name()) && !types.contains(VariantType.COPY_NUMBER.name()); } - protected SampleAnnotationIndexQuery parseAnnotationIndexQuery(Query query) { - return parseAnnotationIndexQuery(query, false); + protected SampleAnnotationIndexQuery parseAnnotationIndexQuery(SampleIndexSchema schema, Query query) { + return parseAnnotationIndexQuery(schema, query, false); } /** * Builds the SampleAnnotationIndexQuery given a VariantQuery. * + * @param schema Sample index schema for the set of samples in the query * @param query Input VariantQuery. If the index is complete, covered filters could be removed from here. * @param completeIndex Indicates if the annotation index is complete for the samples in the query. * Otherwise, the index can only be used as a hint, and should be completed with further filtering. * @return SampleAnnotationIndexQuery */ - protected SampleAnnotationIndexQuery parseAnnotationIndexQuery(Query query, boolean completeIndex) { + protected SampleAnnotationIndexQuery parseAnnotationIndexQuery(SampleIndexSchema schema, Query query, boolean completeIndex) { byte annotationIndex = 0; IndexFieldFilter biotypeFilter = schema.getBiotypeIndex().getField().noOpFilter(); IndexFieldFilter consequenceTypeFilter = schema.getCtIndex().getField().noOpFilter(); diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexSchema.java b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexSchema.java index ed0970b36fa..be24989a7f4 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexSchema.java +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexSchema.java @@ -2,25 +2,19 @@ import org.apache.hadoop.hbase.Cell; import org.apache.hadoop.hbase.CellUtil; -import org.apache.hadoop.hbase.io.compress.Compression; import org.apache.hadoop.hbase.util.Bytes; import org.apache.phoenix.schema.types.PInteger; import org.apache.phoenix.schema.types.PVarchar; import org.opencb.biodata.models.core.Region; import org.opencb.biodata.models.variant.Variant; import org.opencb.biodata.models.variant.avro.VariantAvro; -import org.opencb.commons.datastore.core.ObjectMap; import org.opencb.commons.utils.ListUtils; import org.opencb.opencga.core.config.storage.SampleIndexConfiguration; import org.opencb.opencga.core.models.variant.VariantAnnotationConstants; +import org.opencb.opencga.storage.core.metadata.models.StudyMetadata; import org.opencb.opencga.storage.core.variant.adaptors.GenotypeClass; -import org.opencb.opencga.storage.hadoop.utils.HBaseManager; -import org.opencb.opencga.storage.hadoop.variant.GenomeHelper; -import org.opencb.opencga.storage.hadoop.variant.HadoopVariantStorageOptions; import org.opencb.opencga.storage.hadoop.variant.index.annotation.*; -import java.io.IOException; -import java.io.UncheckedIOException; import java.util.*; import static org.apache.hadoop.hbase.util.Bytes.SIZEOF_INT; @@ -124,6 +118,7 @@ public final class SampleIndexSchema { static final String ANNOTATION_CLINICAL_PREFIX = META_PREFIX + "CL_"; static final byte[] ANNOTATION_CLINICAL_PREFIX_BYTES = Bytes.toBytes(ANNOTATION_CLINICAL_PREFIX); + private final int version; private final SampleIndexConfiguration configuration; private final FileIndexSchema fileIndex; private final PopulationFrequencyIndexSchema popFreqIndex; @@ -134,7 +129,8 @@ public final class SampleIndexSchema { private final ClinicalIndexSchema clinicalIndexSchema; // private final AnnotationSummaryIndexSchema annotationSummaryIndexSchema; - public SampleIndexSchema(SampleIndexConfiguration configuration) { + public SampleIndexSchema(SampleIndexConfiguration configuration, int version) { + this.version = version; this.configuration = configuration; fileIndex = new FileIndexSchema(configuration.getFileIndexConfiguration()); // annotationSummaryIndexSchema = new AnnotationSummaryIndexSchema(); @@ -157,7 +153,11 @@ public SampleIndexSchema(SampleIndexConfiguration configuration) { */ public static SampleIndexSchema defaultSampleIndexSchema() { SampleIndexConfiguration sampleIndexConfiguration = SampleIndexConfiguration.defaultConfiguration(); - return new SampleIndexSchema(sampleIndexConfiguration); + return new SampleIndexSchema(sampleIndexConfiguration, StudyMetadata.DEFAULT_SAMPLE_INDEX_VERSION); + } + + public int getVersion() { + return version; } public SampleIndexConfiguration getConfiguration() { @@ -351,36 +351,6 @@ public static byte[] toParentsGTColumn(String genotype) { return Bytes.toBytes(PARENTS_PREFIX + genotype); } - public static boolean createTableIfNeeded(String sampleIndexTable, HBaseManager hBaseManager, ObjectMap options) { - - int files = options.getInt( - HadoopVariantStorageOptions.EXPECTED_FILES_NUMBER.key(), - HadoopVariantStorageOptions.EXPECTED_FILES_NUMBER.defaultValue()); - int samples = options.getInt( - HadoopVariantStorageOptions.EXPECTED_SAMPLES_NUMBER.key(), - files); - int preSplitSize = options.getInt( - HadoopVariantStorageOptions.SAMPLE_INDEX_TABLE_PRESPLIT_SIZE.key(), - HadoopVariantStorageOptions.SAMPLE_INDEX_TABLE_PRESPLIT_SIZE.defaultValue()); - - int splits = samples / preSplitSize; - ArrayList preSplits = new ArrayList<>(splits); - for (int i = 0; i < splits; i++) { - preSplits.add(toRowKey(i * preSplitSize)); - } - - Compression.Algorithm compression = Compression.getCompressionAlgorithmByName( - options.getString( - HadoopVariantStorageOptions.SAMPLE_INDEX_TABLE_COMPRESSION.key(), - HadoopVariantStorageOptions.SAMPLE_INDEX_TABLE_COMPRESSION.defaultValue())); - - try { - return hBaseManager.createTableIfNeeded(sampleIndexTable, GenomeHelper.COLUMN_FAMILY_BYTES, preSplits, compression); - } catch (IOException e) { - throw new UncheckedIOException(e); - } - } - public static boolean isAnnotatedGenotype(String gt) { return GenotypeClass.MAIN_ALT.test(gt); } diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexSchemaFactory.java b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexSchemaFactory.java new file mode 100644 index 00000000000..b733aa2c27b --- /dev/null +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexSchemaFactory.java @@ -0,0 +1,160 @@ +package org.opencb.opencga.storage.hadoop.variant.index.sample; + +import org.apache.commons.collections4.CollectionUtils; +import org.opencb.opencga.core.config.storage.SampleIndexConfiguration; +import org.opencb.opencga.storage.core.metadata.VariantStorageMetadataManager; +import org.opencb.opencga.storage.core.metadata.models.SampleMetadata; +import org.opencb.opencga.storage.core.metadata.models.StudyMetadata; +import org.opencb.opencga.storage.core.variant.adaptors.VariantQueryException; + +import java.util.ArrayList; +import java.util.Collection; +import java.util.Collections; +import java.util.List; + +public class SampleIndexSchemaFactory { + private final VariantStorageMetadataManager metadataManager; + + public SampleIndexSchemaFactory(VariantStorageMetadataManager metadataManager) { + this.metadataManager = metadataManager; + } + + public SampleIndexSchema getSchema(int studyId, int sampleId) { + return getSchema(studyId, sampleId, true); + } + + public SampleIndexSchema getSchema(int studyId, int sampleId, boolean requireAnnotation) { + StudyMetadata studyMetadata = metadataManager.getStudyMetadata(studyId); + Collection versions = getSampleIndexConfigurationVersions(studyId, sampleId, true, false); + removeStagingVersions(studyMetadata, versions); + if (versions.isEmpty() && !requireAnnotation) { + versions = getSampleIndexConfigurationVersions(studyId, sampleId, false, false); + } + removeStagingVersions(studyMetadata, versions); + if (versions.isEmpty()) { + throw sampleIndexNotFound(Collections.singletonList(metadataManager.getSampleName(studyId, sampleId))); + } + int version = versions.stream().mapToInt(i -> i).max().getAsInt(); + SampleIndexConfiguration sampleIndexConfiguration = studyMetadata.getSampleIndexConfiguration(version).getConfiguration(); + + if (sampleIndexConfiguration == null) { + throw new VariantQueryException("Unable to use sample index version " + version + " required to query sample " + + metadataManager.getSampleName(studyId, sampleId)); + } + return new SampleIndexSchema(sampleIndexConfiguration, version); + } + + public SampleIndexSchema getSchema(int studyId, Collection samples, boolean requireAnnotation, boolean requireFamilyIndex) { + if (samples.isEmpty()) { + throw new IllegalArgumentException("Missing samples"); + } + StudyMetadata studyMetadata = metadataManager.getStudyMetadata(studyId); + int version = getSampleIndexConfigurationVersion(studyId, samples, requireAnnotation, requireFamilyIndex, studyMetadata); + SampleIndexConfiguration sampleIndexConfiguration = studyMetadata.getSampleIndexConfiguration(version).getConfiguration(); + + if (sampleIndexConfiguration == null) { + throw new VariantQueryException("Unable to use sample index version " + version + " required to query samples " + samples); + } + return new SampleIndexSchema(sampleIndexConfiguration, version); + } + + public Collection getSampleIndexConfigurationVersions(int studyId, Collection samples) { + return getSampleIndexConfigurationVersions(studyId, samples, true, false); + } + + public Collection getSampleIndexConfigurationVersions(int studyId, Collection samples, + boolean withAnnotation, + boolean withFamilyIndex) { + List> allVersions = new ArrayList<>(samples.size()); + for (Object sample : samples) { + allVersions.add(getSampleIndexConfigurationVersions(studyId, sample, withAnnotation, withFamilyIndex)); + } + Collection intersection = allVersions.get(0); + for (int i = 1; i < allVersions.size(); i++) { + intersection = CollectionUtils.intersection(intersection, allVersions.get(i)); + } + return intersection; + } + + private Collection getSampleIndexConfigurationVersions(int studyId, Object sample, boolean withAnnotation, + boolean withFamilyIndex) { + int sampleId = metadataManager.getSampleIdOrFail(studyId, sample); + SampleMetadata sampleMetadata = metadataManager.getSampleMetadata(studyId, sampleId); + Collection versions = sampleMetadata.getSampleIndexVersions(); + if (withAnnotation) { + versions = CollectionUtils.intersection( + versions, + sampleMetadata.getSampleIndexAnnotationVersions()); + } + if (withFamilyIndex) { + versions = CollectionUtils.intersection( + versions, + sampleMetadata.getFamilyIndexVersions()); + } + return versions; + } + + public int getSampleIndexConfigurationVersion(int studyId, Collection samples, boolean requireAnnotation) { + return getSampleIndexConfigurationVersion( + studyId, samples, requireAnnotation, false, metadataManager.getStudyMetadata(studyId)); + } + + private int getSampleIndexConfigurationVersion(int studyId, Collection samples, boolean requireAnnotation, + boolean requireFamilyIndex, StudyMetadata studyMetadata) { + Collection validVersions = getSampleIndexConfigurationVersions(studyId, samples, true, requireFamilyIndex); + removeStagingVersions(studyMetadata, validVersions); + if (validVersions.isEmpty() && !requireAnnotation) { + validVersions = getSampleIndexConfigurationVersions(studyId, samples, false, requireFamilyIndex); + } + removeStagingVersions(studyMetadata, validVersions); + if (validVersions.isEmpty()) { + throw sampleIndexNotFound(samples); + } + return validVersions.stream().mapToInt(i -> i).max().getAsInt(); + } + + private void removeStagingVersions(StudyMetadata studyMetadata, Collection validVersions) { + if (studyMetadata.getSampleIndexConfigurations() == null || validVersions.isEmpty()) { + return; + } + for (StudyMetadata.SampleIndexConfigurationVersioned v : studyMetadata.getSampleIndexConfigurations()) { + if (v.getStatus() == StudyMetadata.SampleIndexConfigurationVersioned.Status.STAGING) { + validVersions.remove(v.getVersion()); + } + } + } + + private VariantQueryException sampleIndexNotFound(Collection samples) { + if (samples.size() == 1) { + throw new VariantQueryException("Not found valid sample index for sample '" + samples.iterator().next() + "'"); + } else { + throw new VariantQueryException("Not found valid sample index for samples " + samples); + } + } + + /** + * Get the latest schema available, including staging schemas. + * @param studyId studyId + * @return Latest schema available + */ + public SampleIndexSchema getSchemaLatest(int studyId) { + return getSchemaLatest(studyId, true); + } + + /** + * Get the latest schema available. + * @param studyId studyId + * @param includeStagingSchemas Include schemas with status + * {@link org.opencb.opencga.storage.core.metadata.models.StudyMetadata.SampleIndexConfigurationVersioned.Status#STAGING}. + * @return Latest schema available + */ + public SampleIndexSchema getSchemaLatest(int studyId, boolean includeStagingSchemas) { + StudyMetadata.SampleIndexConfigurationVersioned latest = getSampleIndexConfigurationLatest(studyId, includeStagingSchemas); + return new SampleIndexSchema(latest.getConfiguration(), latest.getVersion()); + } + + public StudyMetadata.SampleIndexConfigurationVersioned getSampleIndexConfigurationLatest(int studyId, boolean includeStagingSchemas) { + return metadataManager.getStudyMetadata(studyId).getSampleIndexConfigurationLatest(includeStagingSchemas); + } + +} diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/migration/v2_0_0/HadoopVariantStorage200MigrationToolExecutor.java b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/migration/v2_0_0/HadoopVariantStorage200MigrationToolExecutor.java index deee956d940..f9bb6222f76 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/migration/v2_0_0/HadoopVariantStorage200MigrationToolExecutor.java +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/migration/v2_0_0/HadoopVariantStorage200MigrationToolExecutor.java @@ -11,11 +11,11 @@ import org.opencb.opencga.core.tools.migration.v2_0_0.VariantStorage200MigrationToolParams; import org.opencb.opencga.storage.core.metadata.VariantStorageMetadataManager; import org.opencb.opencga.storage.core.metadata.models.ProjectMetadata; +import org.opencb.opencga.storage.core.metadata.models.TaskMetadata; import org.opencb.opencga.storage.core.variant.adaptors.VariantQueryParam; import org.opencb.opencga.storage.hadoop.variant.HadoopVariantStorageEngine; import org.opencb.opencga.storage.hadoop.variant.analysis.HadoopVariantStorageToolExecutor; import org.opencb.opencga.storage.hadoop.variant.annotation.pending.AnnotationPendingVariantsManager; -import org.opencb.opencga.storage.hadoop.variant.index.sample.SampleIndexDBAdaptor; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -76,16 +76,22 @@ protected void run() throws Exception { Integer studyId = entry.getValue(); metadataManager.sampleMetadataIterator(studyId).forEachRemaining(sampleMetadata -> { if (sampleMetadata.isIndexed() - && (SampleIndexDBAdaptor.getSampleIndexStatus(sampleMetadata, 1).equals(Status.READY) - || SampleIndexDBAdaptor.getSampleIndexAnnotationStatus(sampleMetadata, 1).equals(Status.READY))) { + && (sampleMetadata.getSampleIndexStatus(1).equals(Status.READY) + || sampleMetadata.getSampleIndexAnnotationStatus(1).equals(Status.READY))) { samplesToModify.add(sampleMetadata.getId()); } }); for (Integer sampleId : samplesToModify) { metadataManager.updateSampleMetadata(studyId, sampleId, sampleMetadata -> { - SampleIndexDBAdaptor.setSampleIndexStatus(sampleMetadata, Status.NONE, 0); - SampleIndexDBAdaptor.setSampleIndexAnnotationStatus(sampleMetadata, Status.NONE, 0); - return sampleMetadata; + for (int v : sampleMetadata.getSampleIndexVersions()) { + sampleMetadata.setSampleIndexStatus(TaskMetadata.Status.ERROR, v); + } + for (int v : sampleMetadata.getSampleIndexAnnotationVersions()) { + sampleMetadata.setSampleIndexAnnotationStatus(TaskMetadata.Status.ERROR, v); + } + for (int v : sampleMetadata.getSampleIndexAnnotationVersions()) { + sampleMetadata.setFamilyIndexStatus(TaskMetadata.Status.ERROR, v); + } }); } } diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/mr/VariantMapReduceUtil.java b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/mr/VariantMapReduceUtil.java index f0de4b74406..623deec75da 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/mr/VariantMapReduceUtil.java +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/main/java/org/opencb/opencga/storage/hadoop/variant/mr/VariantMapReduceUtil.java @@ -42,6 +42,7 @@ import org.opencb.opencga.storage.hadoop.variant.index.query.SampleIndexQuery; import org.opencb.opencga.storage.hadoop.variant.index.sample.SampleIndexDBAdaptor; import org.opencb.opencga.storage.hadoop.variant.index.sample.SampleIndexQueryParser; +import org.opencb.opencga.storage.hadoop.variant.index.sample.SampleIndexSchema; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -165,7 +166,9 @@ public static void initVariantMapperJob(Job job, Class Object geneRegions = query.get(VariantQueryUtils.ANNOT_GENE_REGIONS.key()); // Remove extra fields from the query SampleIndexQuery sampleIndexQuery = new SampleIndexDBAdaptor(null, null, metadataManager).parseSampleIndexQuery(query); - setSampleIndexConfiguration(job, sampleIndexQuery.getSchema().getConfiguration()); + setSampleIndexConfiguration(job, + sampleIndexQuery.getSchema().getConfiguration(), + sampleIndexQuery.getSchema().getVersion()); // Preserve regions and gene_regions query.put(VariantQueryParam.REGION.key(), regions); @@ -274,7 +277,9 @@ public static void initVariantRowMapperJob(Job job, Class // Update all but missingCount! } - - return sampleMetadata; }); if (write) { diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/HadoopVariantStorageEngineSVTest.java b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/HadoopVariantStorageEngineSVTest.java index 2f4379ae406..3517246b20e 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/HadoopVariantStorageEngineSVTest.java +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/HadoopVariantStorageEngineSVTest.java @@ -6,9 +6,11 @@ import org.opencb.biodata.models.variant.Variant; import org.opencb.commons.datastore.core.Query; import org.opencb.commons.datastore.core.QueryOptions; +import org.opencb.opencga.core.api.ParamConstants; import org.opencb.opencga.core.response.VariantQueryResult; import org.opencb.opencga.storage.core.variant.VariantStorageEngineSVTest; import org.opencb.opencga.storage.core.variant.adaptors.GenotypeClass; +import org.opencb.opencga.storage.core.variant.adaptors.VariantQuery; import org.opencb.opencga.storage.core.variant.adaptors.VariantQueryParam; import org.opencb.opencga.storage.hadoop.variant.adaptors.VariantHadoopDBAdaptor; @@ -34,7 +36,7 @@ protected void loadFiles() throws Exception { @Test public void checkSampleIndex() throws Exception { - for (Variant variant : variantStorageEngine) { + for (Variant variant : variantStorageEngine.iterable(new VariantQuery().includeSample(ParamConstants.ALL), new QueryOptions())) { Set samplesInVariant = new HashSet<>(); for (String sample : metadataManager.getIndexedSamplesMap(studyMetadata.getId()).keySet()) { QueryOptions options = new QueryOptions(VariantHadoopDBAdaptor.NATIVE, false); diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/HadoopVariantStorageEngineSplitDataTest.java b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/HadoopVariantStorageEngineSplitDataTest.java index 2a5f575f316..e901235bfa2 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/HadoopVariantStorageEngineSplitDataTest.java +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/HadoopVariantStorageEngineSplitDataTest.java @@ -70,7 +70,7 @@ public void testMultiChromosomeSplitData() throws Exception { SampleMetadata sampleMetadata = mm.getSampleMetadata(studyId, mm.getSampleId(studyId, sample)); assertEquals(TaskMetadata.Status.READY, sampleMetadata.getIndexStatus()); assertEquals(TaskMetadata.Status.NONE, sampleMetadata.getAnnotationStatus()); - assertEquals(TaskMetadata.Status.NONE, SampleIndexDBAdaptor.getSampleIndexAnnotationStatus(sampleMetadata, 1)); + assertEquals(TaskMetadata.Status.NONE, sampleMetadata.getSampleIndexAnnotationStatus(1)); } variantStorageEngine.annotate(new Query(), new QueryOptions(DefaultVariantAnnotationManager.OUT_DIR, outputUri)); @@ -78,7 +78,7 @@ public void testMultiChromosomeSplitData() throws Exception { SampleMetadata sampleMetadata = mm.getSampleMetadata(studyId, mm.getSampleId(studyId, sample)); assertEquals(TaskMetadata.Status.READY, sampleMetadata.getIndexStatus()); assertEquals(TaskMetadata.Status.READY, sampleMetadata.getAnnotationStatus()); - assertEquals(TaskMetadata.Status.READY, SampleIndexDBAdaptor.getSampleIndexAnnotationStatus(sampleMetadata, 1)); + assertEquals(TaskMetadata.Status.READY, sampleMetadata.getSampleIndexAnnotationStatus(1)); } variantStorageEngine.getOptions().put(VariantStorageOptions.STUDY.key(), STUDY_NAME); @@ -90,14 +90,14 @@ public void testMultiChromosomeSplitData() throws Exception { SampleMetadata sampleMetadata = mm.getSampleMetadata(studyId, mm.getSampleId(studyId, sample)); assertEquals(TaskMetadata.Status.READY, sampleMetadata.getIndexStatus()); assertEquals(TaskMetadata.Status.NONE, sampleMetadata.getAnnotationStatus()); - assertEquals(TaskMetadata.Status.NONE, SampleIndexDBAdaptor.getSampleIndexAnnotationStatus(sampleMetadata, 1)); + assertEquals(TaskMetadata.Status.NONE, sampleMetadata.getSampleIndexAnnotationStatus(1)); } variantStorageEngine.annotate(new Query(), new QueryOptions(DefaultVariantAnnotationManager.OUT_DIR, outputUri)); for (String sample : SAMPLES) { SampleMetadata sampleMetadata = mm.getSampleMetadata(studyId, mm.getSampleId(studyId, sample)); assertEquals(TaskMetadata.Status.READY, sampleMetadata.getIndexStatus()); assertEquals(TaskMetadata.Status.READY, sampleMetadata.getAnnotationStatus()); - assertEquals(TaskMetadata.Status.READY, SampleIndexDBAdaptor.getSampleIndexAnnotationStatus(sampleMetadata, 1)); + assertEquals(TaskMetadata.Status.READY, sampleMetadata.getSampleIndexAnnotationStatus(1)); } @@ -311,7 +311,7 @@ public void testLoadByRegion() throws Exception { SampleMetadata sampleMetadata = mm.getSampleMetadata(studyId_split, mm.getSampleId(studyId_split, sample)); assertEquals(TaskMetadata.Status.READY, sampleMetadata.getIndexStatus()); assertEquals(TaskMetadata.Status.NONE, sampleMetadata.getAnnotationStatus()); - assertEquals(TaskMetadata.Status.NONE, SampleIndexDBAdaptor.getSampleIndexAnnotationStatus(sampleMetadata, 1)); + assertEquals(TaskMetadata.Status.NONE, sampleMetadata.getSampleIndexAnnotationStatus(1)); } variantStorageEngine.getOptions().put(VariantStorageOptions.LOAD_SPLIT_DATA.key(), null); @@ -324,10 +324,10 @@ public void testLoadByRegion() throws Exception { SampleMetadata sampleMetadata = mm.getSampleMetadata(studyId_normal, mm.getSampleId(studyId_normal, sample)); assertEquals(TaskMetadata.Status.READY, sampleMetadata.getIndexStatus()); assertEquals(TaskMetadata.Status.NONE, sampleMetadata.getAnnotationStatus()); - assertEquals(TaskMetadata.Status.NONE, SampleIndexDBAdaptor.getSampleIndexAnnotationStatus(sampleMetadata, 1)); + assertEquals(TaskMetadata.Status.NONE, sampleMetadata.getSampleIndexAnnotationStatus(1)); } - checkVariantsTable(studyId_split, studyId_normal, new Query(), new QueryOptions(QueryOptions.EXCLUDE, VariantField.STUDIES_FILES)); + checkVariantsTable(studyId_split, studyId_normal, new VariantQuery().includeSample(ParamConstants.ALL), new QueryOptions(QueryOptions.EXCLUDE, VariantField.STUDIES_FILES)); checkSampleIndex(studyId_split, studyId_normal); } diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/VariantHadoopMultiSampleTest.java b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/VariantHadoopMultiSampleTest.java index 9e1eff1db7e..997b6aa2587 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/VariantHadoopMultiSampleTest.java +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/VariantHadoopMultiSampleTest.java @@ -36,6 +36,7 @@ import org.opencb.commons.datastore.core.ObjectMap; import org.opencb.commons.datastore.core.Query; import org.opencb.commons.datastore.core.QueryOptions; +import org.opencb.opencga.core.api.ParamConstants; import org.opencb.opencga.core.models.operations.variant.VariantAggregateFamilyParams; import org.opencb.opencga.core.models.operations.variant.VariantAggregateParams; import org.opencb.opencga.storage.core.StoragePipelineResult; @@ -48,6 +49,7 @@ import org.opencb.opencga.storage.core.variant.VariantStorageEngine; import org.opencb.opencga.storage.core.variant.VariantStorageOptions; import org.opencb.opencga.storage.core.variant.adaptors.GenotypeClass; +import org.opencb.opencga.storage.core.variant.adaptors.VariantQuery; import org.opencb.opencga.storage.core.variant.query.VariantQueryUtils; import org.opencb.opencga.storage.hadoop.variant.adaptors.VariantHadoopDBAdaptor; import org.opencb.opencga.storage.hadoop.variant.archive.ArchiveRowKeyFactory; @@ -464,7 +466,7 @@ public void checkLoadedFilesS1S2(StudyMetadata studyMetadata, VariantHadoopDBAda System.out.println("studyMetadata = " + studyMetadata); Map variants = new HashMap<>(); - for (Variant variant : dbAdaptor) { + for (Variant variant : dbAdaptor.iterable(new VariantQuery().includeSample(ParamConstants.ALL), new QueryOptions())) { String v = variant.toString(); assertFalse(variants.containsKey(v)); variants.put(v, variant); @@ -615,7 +617,7 @@ public void testMultiSampleFile() throws Exception { printVariants(studyMetadata, dbAdaptor, newOutputUri()); int numHomRef = 0; - for (Variant variant : dbAdaptor) { + for (Variant variant : dbAdaptor.iterable(new VariantQuery().includeSample(ParamConstants.ALL), new QueryOptions())) { StudyEntry study = variant.getStudies().get(0); for (String s : study.getSamplesName()) { String gt = study.getSampleData(s, "GT"); diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/VariantHbaseTestUtils.java b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/VariantHbaseTestUtils.java index 49001a5fed3..baafd3ae17d 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/VariantHbaseTestUtils.java +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/VariantHbaseTestUtils.java @@ -415,11 +415,18 @@ private static void printVcf(StudyMetadata studyMetadata, VariantHadoopDBAdaptor public static void printSampleIndexTable(VariantHadoopDBAdaptor dbAdaptor, Path outDir) throws IOException { for (Integer studyId : dbAdaptor.getMetadataManager().getStudies(null).values()) { - int version = dbAdaptor.getMetadataManager().getStudyMetadata(studyId).getSampleIndexConfigurationLatest().getVersion(); - String sampleGtTableName = dbAdaptor.getTableNameGenerator().getSampleIndexTableName(studyId, version); - if (dbAdaptor.getHBaseManager().tableExists(sampleGtTableName)) { - printSampleIndexTable(dbAdaptor, outDir, studyId, sampleGtTableName); - printSampleIndexTable2(dbAdaptor, outDir, studyId, sampleGtTableName); + StudyMetadata studyMetadata = dbAdaptor.getMetadataManager().getStudyMetadata(studyId); + List confs = studyMetadata.getSampleIndexConfigurations(); + if (confs == null) { + confs = Collections.singletonList(studyMetadata.getSampleIndexConfigurationLatest()); + } + for (StudyMetadata.SampleIndexConfigurationVersioned c : confs) { + int version = c.getVersion(); + String sampleGtTableName = dbAdaptor.getTableNameGenerator().getSampleIndexTableName(studyId, version); + if (dbAdaptor.getHBaseManager().tableExists(sampleGtTableName)) { + printSampleIndexTable(dbAdaptor, outDir, studyId, sampleGtTableName); + printSampleIndexTable2(dbAdaptor, outDir, studyId, sampleGtTableName); + } } } } @@ -434,7 +441,7 @@ public static void printSampleIndexTable2(VariantHadoopDBAdaptor dbAdaptor, Path FileOutputStream fos = new FileOutputStream(fileName.toFile()); PrintStream out = new PrintStream(fos) ) { SampleIndexDBAdaptor sampleIndexDBAdaptor = new SampleIndexDBAdaptor(dbAdaptor.getHBaseManager(), dbAdaptor.getTableNameGenerator(), dbAdaptor.getMetadataManager()); - SampleIndexSchema schema = sampleIndexDBAdaptor.getSchema(studyId); + SampleIndexSchema schema = sampleIndexDBAdaptor.getSchemaLatest(studyId); for (Integer sampleId : dbAdaptor.getMetadataManager().getIndexedSamples(studyId)) { String sampleName = dbAdaptor.getMetadataManager().getSampleName(studyId, sampleId); RawSingleSampleIndexVariantDBIterator it = sampleIndexDBAdaptor.rawIterator(dbAdaptor.getMetadataManager().getStudyName(studyId), sampleName); @@ -462,7 +469,7 @@ public static void printSampleIndexTable(VariantHadoopDBAdaptor dbAdaptor, Path try ( FileOutputStream fos = new FileOutputStream(fileName.toFile()); PrintStream out = new PrintStream(fos) ) { - SampleIndexSchema schema = new SampleIndexSchema(dbAdaptor.getMetadataManager().getStudyMetadata(studyId).getSampleIndexConfigurationLatest().getConfiguration()); + SampleIndexSchema schema = new SampleIndexSchemaFactory(dbAdaptor.getMetadataManager()).getSchemaLatest(studyId); dbAdaptor.getHBaseManager().act(sampleGtTableName, table -> { table.getScanner(new Scan()).iterator().forEachRemaining(result -> { diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/index/annotation/AnnotationIndexConverterTest.java b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/index/annotation/AnnotationIndexConverterTest.java index eb2eaf62933..51f227d714b 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/index/annotation/AnnotationIndexConverterTest.java +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/index/annotation/AnnotationIndexConverterTest.java @@ -8,6 +8,7 @@ import org.opencb.biodata.models.variant.avro.VariantAnnotation; import org.opencb.opencga.core.config.storage.SampleIndexConfiguration; import org.opencb.opencga.storage.core.io.bit.BitBuffer; +import org.opencb.opencga.storage.core.metadata.models.StudyMetadata; import org.opencb.opencga.storage.hadoop.variant.index.sample.SampleIndexSchema; import java.util.Arrays; @@ -43,7 +44,7 @@ public void setUp() throws Exception { configuration.getAnnotationIndexConfiguration().getPopulationFrequency().setThresholds(new double[]{0.001, 0.005, 0.01}); populations.stream().map(SampleIndexConfiguration.Population::new).forEach(configuration::addPopulation); - schema = new SampleIndexSchema(configuration); + schema = new SampleIndexSchema(configuration, StudyMetadata.DEFAULT_SAMPLE_INDEX_VERSION); converter = new AnnotationIndexConverter(schema); } @@ -173,7 +174,7 @@ public void testDuplicatedPopulations() { List populations = Arrays.asList("1kG_phase3:ALL", "GNOMAD_GENOMES:ALL", "1kG_phase3:ALL"); SampleIndexConfiguration configuration = new SampleIndexConfiguration(); populations.stream().map(SampleIndexConfiguration.Population::new).forEach(configuration::addPopulation); - new AnnotationIndexConverter(new SampleIndexSchema(configuration)); + new AnnotationIndexConverter(new SampleIndexSchema(configuration, StudyMetadata.DEFAULT_SAMPLE_INDEX_VERSION)); } @Test diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/index/core/CategoricalIndexFieldTest.java b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/index/core/CategoricalIndexFieldTest.java index 9356a8e0410..3e90d12ce5e 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/index/core/CategoricalIndexFieldTest.java +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/index/core/CategoricalIndexFieldTest.java @@ -38,7 +38,7 @@ public void testLength() { @Test public void testEncodeDecode() { - SampleIndexSchema indexSchema = new SampleIndexSchema(SampleIndexConfiguration.defaultConfiguration()); + SampleIndexSchema indexSchema = SampleIndexSchema.defaultSampleIndexSchema(); CategoricalMultiValuedIndexField field = (CategoricalMultiValuedIndexField) indexSchema.getCtIndex().getField(); List expected = Arrays.asList("synonymous_variant", "missense_variant"); diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/index/family/FamilyIndexTest.java b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/index/family/FamilyIndexTest.java index 6e392b81477..07cbc4bc02c 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/index/family/FamilyIndexTest.java +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/index/family/FamilyIndexTest.java @@ -24,7 +24,6 @@ import org.opencb.opencga.storage.hadoop.variant.index.query.SampleIndexQuery; import org.opencb.opencga.storage.hadoop.variant.index.sample.AbstractSampleIndexEntryFilter; import org.opencb.opencga.storage.hadoop.variant.index.sample.SampleIndexQueryParser; -import org.opencb.opencga.storage.hadoop.variant.index.sample.SampleIndexSchema; import java.net.URI; import java.util.*; @@ -77,7 +76,7 @@ public void before() throws Exception { VariantHbaseTestUtils.printVariants(getVariantStorageEngine().getDBAdaptor(), newOutputUri(getTestName().getMethodName())); loaded = true; } - sampleIndexQueryParser = new SampleIndexQueryParser(metadataManager, SampleIndexSchema.defaultSampleIndexSchema()); + sampleIndexQueryParser = new SampleIndexQueryParser(metadataManager); } @Test diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexEntryFilterTest.java b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexEntryFilterTest.java index f7d0b27ce9d..0d8e2c3dc4a 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexEntryFilterTest.java +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexEntryFilterTest.java @@ -11,6 +11,7 @@ import org.opencb.opencga.core.config.storage.SampleIndexConfiguration; import org.opencb.opencga.core.config.storage.SampleIndexConfiguration.Population; import org.opencb.opencga.storage.core.metadata.VariantStorageMetadataManager; +import org.opencb.opencga.storage.core.metadata.models.StudyMetadata; import org.opencb.opencga.storage.core.variant.adaptors.VariantQueryParam; import org.opencb.opencga.storage.core.variant.dummy.DummyVariantStorageMetadataDBAdaptorFactory; import org.opencb.opencga.storage.core.variant.query.OpValue; @@ -45,7 +46,8 @@ public void setUp() throws Exception { .addPopulation(new Population("s2", "ALL")) .addPopulation(new Population("s3", "ALL")) .addPopulation(new Population("s4", "ALL")) - .addPopulation(new Population("s5", "ALL"))); + .addPopulation(new Population("s5", "ALL")), + StudyMetadata.DEFAULT_SAMPLE_INDEX_VERSION); } @Test @@ -244,8 +246,9 @@ private RangeIndexFieldFilter buildPopulationFrequencyQuery(String study, String private SingleSampleIndexQuery getSingleSampleIndexQuery(Query query) { VariantStorageMetadataManager metadataManager = new VariantStorageMetadataManager(new DummyVariantStorageMetadataDBAdaptorFactory()); - SampleIndexQueryParser parser = new SampleIndexQueryParser(metadataManager, SampleIndexSchema.defaultSampleIndexSchema()); - SampleAnnotationIndexQuery annotQuery = parser.parseAnnotationIndexQuery(query); + SampleIndexSchema schema = SampleIndexSchema.defaultSampleIndexSchema(); + SampleIndexQueryParser parser = new SampleIndexQueryParser(metadataManager); + SampleAnnotationIndexQuery annotQuery = parser.parseAnnotationIndexQuery(schema, query); return getSingleSampleIndexQuery(annotQuery); } diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexQueryParserTest.java b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexQueryParserTest.java index f56ca2388ca..0d2732c3535 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexQueryParserTest.java +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexQueryParserTest.java @@ -16,6 +16,8 @@ import org.opencb.opencga.core.config.storage.SampleIndexConfiguration; import org.opencb.opencga.core.models.variant.VariantAnnotationConstants; import org.opencb.opencga.storage.core.metadata.VariantStorageMetadataManager; +import org.opencb.opencga.storage.core.metadata.models.SampleMetadata; +import org.opencb.opencga.storage.core.metadata.models.StudyMetadata; import org.opencb.opencga.storage.core.metadata.models.TaskMetadata; import org.opencb.opencga.storage.core.variant.VariantStorageEngine; import org.opencb.opencga.storage.core.variant.VariantStorageOptions; @@ -60,6 +62,7 @@ public class SampleIndexQueryParserTest { private double[] qualThresholds; private double[] dpThresholds; private SampleIndexConfiguration configuration; + private SampleIndexSchema schema; @Before public void setUp() throws Exception { @@ -69,14 +72,14 @@ public void setUp() throws Exception { .addPopulation(new SampleIndexConfiguration.Population("s3", "ALL")) .addPopulation(new SampleIndexConfiguration.Population("s4", "ALL")); - SampleIndexSchema schema = new SampleIndexSchema(configuration); + schema = new SampleIndexSchema(configuration, StudyMetadata.DEFAULT_SAMPLE_INDEX_VERSION); fileIndex = schema.getFileIndex(); qualThresholds = fileIndex.getCustomField(IndexFieldConfiguration.Source.FILE, StudyEntry.QUAL).getConfiguration().getThresholds(); dpThresholds = fileIndex.getCustomField(IndexFieldConfiguration.Source.SAMPLE, VCFConstants.DEPTH_KEY).getConfiguration().getThresholds(); DummyVariantStorageMetadataDBAdaptorFactory.clear(); mm = new VariantStorageMetadataManager(new DummyVariantStorageMetadataDBAdaptorFactory()); - sampleIndexQueryParser = new SampleIndexQueryParser(mm, schema); + sampleIndexQueryParser = new SampleIndexQueryParser(mm); studyId = mm.createStudy("study").getId(); mm.addIndexedFiles(studyId, Arrays.asList(mm.registerFile(studyId, "F1", Arrays.asList("S1", "S2", "S3")))); @@ -84,10 +87,11 @@ public void setUp() throws Exception { mm.addIndexedFiles(studyId, Arrays.asList(mm.registerFile(studyId, "fam2_child", Arrays.asList("fam2_child")))); mm.addIndexedFiles(studyId, Arrays.asList(mm.registerFile(studyId, "fam2_father", Arrays.asList("fam2_father")))); mm.addIndexedFiles(studyId, Arrays.asList(mm.registerFile(studyId, "fam2_mother", Arrays.asList("fam2_mother")))); + int version = mm.getStudyMetadata(studyId).getSampleIndexConfigurationLatest().getVersion(); for (String family : Arrays.asList("fam1", "fam2")) { mm.updateSampleMetadata(studyId, mm.getSampleId(studyId, family + "_child"), sampleMetadata -> sampleMetadata - .setFamilyIndexStatus(TaskMetadata.Status.READY) + .setFamilyIndexStatus(TaskMetadata.Status.READY, version) .setFather(mm.getSampleId(studyId, family + "_father")) .setMother(mm.getSampleId(studyId, family + "_mother"))); } @@ -99,7 +103,6 @@ public void setUp() throws Exception { for (int i = 1; i <= 4; i++) { mm.updateSampleMetadata(studyId, mm.getSampleIdOrFail(studyId, "MULTI_S" + i), s -> { s.setSplitData(VariantStorageEngine.SplitData.MULTI); - return s; }); } @@ -108,6 +111,15 @@ public void setUp() throws Exception { studyMetadata.getAttributes().put(VariantStorageOptions.LOADED_GENOTYPES.key(), "./.,0/0,0/1,1/1"); return studyMetadata; }); + + Iterator it = mm.sampleMetadataIterator(studyId); + while (it.hasNext()) { + SampleMetadata sm = it.next(); + mm.updateSampleMetadata(studyId, sm.getId(), sampleMetadata -> { + sampleMetadata.setSampleIndexStatus(TaskMetadata.Status.READY, version) + .setSampleIndexAnnotationStatus(TaskMetadata.Status.READY, version); + }); + } } private SampleIndexQuery parse(final Query query) { @@ -122,27 +134,27 @@ private SampleFileIndexQuery parseFileQuery(Query query, String sample, Function } private SampleFileIndexQuery parseFileQuery(Query query, String sample, Function> filesFromSample, boolean multiFileSample) { - return sampleIndexQueryParser.parseFileQuery(query, sample, multiFileSample, false, false, filesFromSample); + return sampleIndexQueryParser.parseFileQuery(schema, query, sample, multiFileSample, false, false, filesFromSample); } private Values parseFilesQuery(Query query, String sample, Function> filesFromSample, boolean multiFileSample) { - return sampleIndexQueryParser.parseFilesQuery(query, sample, multiFileSample, false, false, filesFromSample); + return sampleIndexQueryParser.parseFilesQuery(schema, query, sample, multiFileSample, false, false, filesFromSample); } private byte parseAnnotationMask(Query query) { - return sampleIndexQueryParser.parseAnnotationIndexQuery(query).getAnnotationIndexMask(); + return sampleIndexQueryParser.parseAnnotationIndexQuery(schema, query).getAnnotationIndexMask(); } private byte parseAnnotationMask(Query query, boolean completeIndex) { - return sampleIndexQueryParser.parseAnnotationIndexQuery(query, completeIndex).getAnnotationIndexMask(); + return sampleIndexQueryParser.parseAnnotationIndexQuery(schema, query, completeIndex).getAnnotationIndexMask(); } private SampleAnnotationIndexQuery parseAnnotationIndexQuery(Query query) { - return sampleIndexQueryParser.parseAnnotationIndexQuery(query); + return sampleIndexQueryParser.parseAnnotationIndexQuery(schema, query); } private SampleAnnotationIndexQuery parseAnnotationIndexQuery(Query query, boolean completeIndex) { - return sampleIndexQueryParser.parseAnnotationIndexQuery(query, completeIndex); + return sampleIndexQueryParser.parseAnnotationIndexQuery(schema, query, completeIndex); } @Test @@ -966,22 +978,22 @@ public void parseFileDataTest() { Values fileQuery; query = new Query(FILE_DATA.key(), "F1:FILTER=PASS"); - fileQuery = sampleIndexQueryParser.parseFilesQuery(query, "S1", true, false, false, n -> Arrays.asList("F1", "F2")); + fileQuery = sampleIndexQueryParser.parseFilesQuery(schema, query, "S1", true, false, false, n -> Arrays.asList("F1", "F2")); assertEquals(1, fileQuery.size()); assertFalse(VariantQueryUtils.isValidParam(query, FILE_DATA)); query = new Query(FILE_DATA.key(), "F1:FILTER=PASS"); - fileQuery = sampleIndexQueryParser.parseFilesQuery(query, "S1", true, false, false, n -> Arrays.asList("F1", "F2")); + fileQuery = sampleIndexQueryParser.parseFilesQuery(schema, query, "S1", true, false, false, n -> Arrays.asList("F1", "F2")); assertEquals(1, fileQuery.size()); assertFalse(VariantQueryUtils.isValidParam(query, FILE_DATA)); query = new Query(FILE_DATA.key(), "F1:FILTER=PASS" + OR + "F2:FILTER=PASS"); - fileQuery = sampleIndexQueryParser.parseFilesQuery(query, "S1", true, false, false, n -> Arrays.asList("F1", "F2")); + fileQuery = sampleIndexQueryParser.parseFilesQuery(schema, query, "S1", true, false, false, n -> Arrays.asList("F1", "F2")); assertEquals(2, fileQuery.size()); assertFalse(VariantQueryUtils.isValidParam(query, FILE_DATA)); query = new Query(FILE_DATA.key(), "F1:FILTER=PASS" + OR + "F3:FILTER=PASS"); - fileQuery = sampleIndexQueryParser.parseFilesQuery(query, "S1", true, false, false, n -> Arrays.asList("F1", "F2")); + fileQuery = sampleIndexQueryParser.parseFilesQuery(schema, query, "S1", true, false, false, n -> Arrays.asList("F1", "F2")); assertEquals(1, fileQuery.size()); assertTrue(VariantQueryUtils.isValidParam(query, FILE_DATA)); } @@ -1177,14 +1189,55 @@ public void parseFamilyQuery_dp() { query = new Query() .append(SAMPLE.key(), "fam1_child;fam1_father;fam1_mother") - .append(SAMPLE_DATA.key(), "fam1_father:DP>15;fam1_child:DP>=15;fam1_mother:DP>15"); + .append(SAMPLE_DATA.key(), "fam1_father:DP>=15;fam1_child:DP>=15;fam1_mother:DP>=15"); indexQuery = parse(query); - assertEquals(Collections.singleton("fam1_child"), indexQuery.getSamplesMap().keySet()); + // Father and mother not discarded + assertEquals(new HashSet<>(Arrays.asList("fam1_child", "fam1_father", "fam1_mother")), indexQuery.getSamplesMap().keySet()); + // Still using parent's filter assertEquals(1, indexQuery.getFatherFilterMap().size()); assertTrue(indexQuery.getSampleFileIndexQuery("fam1_child").get(0).getFilter(IndexFieldConfiguration.Source.SAMPLE, "DP").isExactFilter()); - assertEquals("fam1_father:DP>15;fam1_mother:DP>15", query.getString(SAMPLE_DATA.key())); + assertTrue(indexQuery.getSampleFileIndexQuery("fam1_father").get(0).getFilter(IndexFieldConfiguration.Source.SAMPLE, "DP").isExactFilter()); + assertTrue(indexQuery.getSampleFileIndexQuery("fam1_mother").get(0).getFilter(IndexFieldConfiguration.Source.SAMPLE, "DP").isExactFilter()); + assertEquals("", query.getString(SAMPLE_DATA.key())); + } + + @Test + public void parseFamilyQuery_dp_partial() { + Query query; + SampleIndexQuery indexQuery; + query = new Query() + .append(SAMPLE.key(), "fam1_child;fam1_father;fam1_mother") + .append(SAMPLE_DATA.key(), "fam1_father:DP>=15;fam1_child:DP>=15"); + indexQuery = parse(query); + + // Father and mother not discarded + assertEquals(new HashSet<>(Arrays.asList("fam1_child", "fam1_father")), indexQuery.getSamplesMap().keySet()); + // Still using parent's filter + assertEquals(1, indexQuery.getFatherFilterMap().size()); + assertTrue(indexQuery.getSampleFileIndexQuery("fam1_child").get(0).getFilter(IndexFieldConfiguration.Source.SAMPLE, "DP").isExactFilter()); + assertTrue(indexQuery.getSampleFileIndexQuery("fam1_father").get(0).getFilter(IndexFieldConfiguration.Source.SAMPLE, "DP").isExactFilter()); + assertEquals("", query.getString(SAMPLE_DATA.key())); + } + + @Test + public void parseFamilyQuery_dp_partial_no_exact() { + Query query; + SampleIndexQuery indexQuery; + + query = new Query() + .append(SAMPLE.key(), "fam1_child;fam1_father;fam1_mother") + .append(SAMPLE_DATA.key(), "fam1_father:DP>18;fam1_child:DP>=15"); + indexQuery = parse(query); + + // Father and mother not discarded + assertEquals(new HashSet<>(Arrays.asList("fam1_child", "fam1_father")), indexQuery.getSamplesMap().keySet()); + // Still using parent's filter + assertEquals(1, indexQuery.getFatherFilterMap().size()); + assertTrue(indexQuery.getSampleFileIndexQuery("fam1_child").get(0).getFilter(IndexFieldConfiguration.Source.SAMPLE, "DP").isExactFilter()); + assertFalse(indexQuery.getSampleFileIndexQuery("fam1_father").get(0).getFilter(IndexFieldConfiguration.Source.SAMPLE, "DP").isExactFilter()); + assertEquals("fam1_father:DP>18", query.getString(SAMPLE_DATA.key())); } @Test diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexSchemaFactoryTest.java b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexSchemaFactoryTest.java new file mode 100644 index 00000000000..eeb1d0d4b97 --- /dev/null +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexSchemaFactoryTest.java @@ -0,0 +1,82 @@ +package org.opencb.opencga.storage.hadoop.variant.index.sample; + +import org.junit.Assert; +import org.junit.Before; +import org.junit.Test; +import org.opencb.opencga.storage.core.metadata.VariantStorageMetadataManager; +import org.opencb.opencga.storage.core.metadata.models.StudyMetadata; +import org.opencb.opencga.storage.core.metadata.models.TaskMetadata; +import org.opencb.opencga.storage.core.variant.dummy.DummyVariantStorageMetadataDBAdaptorFactory; + +import java.util.Arrays; + +public class SampleIndexSchemaFactoryTest { + + private SampleIndexSchemaFactory schemaFactory; + private StudyMetadata study1; + + @Before + public void setUp() throws Exception { + VariantStorageMetadataManager metadataManager = new VariantStorageMetadataManager(new DummyVariantStorageMetadataDBAdaptorFactory(true)); + schemaFactory = new SampleIndexSchemaFactory(metadataManager); + + study1 = metadataManager.createStudy("study1"); + int fileId1 = metadataManager.registerFile(study1.getId(), "file1.vcf", Arrays.asList("S0", "S1", "S2", "S3", "S4")); + +// metadataManager.updateSampleMetadata(study1.getId(), metadataManager.getSampleId(study1.getId(), "S0"), sm -> { +// // Sample index never built +// }); + + metadataManager.updateSampleMetadata(study1.getId(), metadataManager.getSampleId(study1.getId(), "S1"), sm -> { + sm.setSampleIndexStatus(TaskMetadata.Status.READY, 1); + sm.setSampleIndexStatus(TaskMetadata.Status.READY, 2); + sm.setSampleIndexAnnotationStatus(TaskMetadata.Status.READY, 1); + sm.setSampleIndexAnnotationStatus(TaskMetadata.Status.READY, 2); + }); + + metadataManager.updateSampleMetadata(study1.getId(), metadataManager.getSampleId(study1.getId(), "S2"), sm -> { + sm.setSampleIndexStatus(TaskMetadata.Status.READY, 1); + sm.setSampleIndexStatus(TaskMetadata.Status.READY, 2); + sm.setSampleIndexAnnotationStatus(TaskMetadata.Status.READY, 1); + // Annotation not yet built for version 2! + }); + + metadataManager.updateSampleMetadata(study1.getId(), metadataManager.getSampleId(study1.getId(), "S3"), sm -> { + sm.setSampleIndexStatus(TaskMetadata.Status.READY, 1); + sm.setSampleIndexStatus(TaskMetadata.Status.READY, 3); + sm.setSampleIndexAnnotationStatus(TaskMetadata.Status.READY, 1); + sm.setSampleIndexAnnotationStatus(TaskMetadata.Status.READY, 3); + // Version 2 index never built + }); + + metadataManager.updateSampleMetadata(study1.getId(), metadataManager.getSampleId(study1.getId(), "S4"), sm -> { + sm.setSampleIndexStatus(TaskMetadata.Status.READY, 1); + sm.setSampleIndexStatus(TaskMetadata.Status.READY, 2); + sm.setSampleIndexStatus(TaskMetadata.Status.READY, 3); + sm.setSampleIndexStatus(TaskMetadata.Status.READY, 4); + sm.setSampleIndexAnnotationStatus(TaskMetadata.Status.READY, 1); + sm.setSampleIndexAnnotationStatus(TaskMetadata.Status.READY, 2); + sm.setSampleIndexAnnotationStatus(TaskMetadata.Status.READY, 3); + sm.setSampleIndexAnnotationStatus(TaskMetadata.Status.READY, 4); + }); + } + + @Test + public void testGetVersion() { + Assert.assertEquals(Arrays.asList(), schemaFactory.getSampleIndexConfigurationVersions(study1.getId(), Arrays.asList("S0"))); + Assert.assertEquals(Arrays.asList(1, 2), schemaFactory.getSampleIndexConfigurationVersions(study1.getId(), Arrays.asList("S1"))); + Assert.assertEquals(Arrays.asList(1), schemaFactory.getSampleIndexConfigurationVersions(study1.getId(), Arrays.asList("S2"))); + Assert.assertEquals(Arrays.asList(1, 3), schemaFactory.getSampleIndexConfigurationVersions(study1.getId(), Arrays.asList("S3"))); + Assert.assertEquals(Arrays.asList(1, 2, 3, 4), schemaFactory.getSampleIndexConfigurationVersions(study1.getId(), Arrays.asList("S4"))); + + Assert.assertEquals(Arrays.asList(1), schemaFactory.getSampleIndexConfigurationVersions(study1.getId(), Arrays.asList("S1", "S2"))); + Assert.assertEquals(Arrays.asList(1), schemaFactory.getSampleIndexConfigurationVersions(study1.getId(), Arrays.asList("S2", "S3"))); + Assert.assertEquals(Arrays.asList(1), schemaFactory.getSampleIndexConfigurationVersions(study1.getId(), Arrays.asList("S1", "S3"))); + Assert.assertEquals(Arrays.asList(1), schemaFactory.getSampleIndexConfigurationVersions(study1.getId(), Arrays.asList("S1", "S2", "S3"))); + Assert.assertEquals(Arrays.asList(), schemaFactory.getSampleIndexConfigurationVersions(study1.getId(), Arrays.asList("S1", "S2", "S3", "S0"))); + + Assert.assertEquals(Arrays.asList(1, 2), schemaFactory.getSampleIndexConfigurationVersions(study1.getId(), Arrays.asList("S1", "S4"))); + Assert.assertEquals(Arrays.asList(1), schemaFactory.getSampleIndexConfigurationVersions(study1.getId(), Arrays.asList("S2", "S4"))); + Assert.assertEquals(Arrays.asList(1, 3), schemaFactory.getSampleIndexConfigurationVersions(study1.getId(), Arrays.asList("S3", "S4"))); + } +} \ No newline at end of file diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexTest.java b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexTest.java index 6dfedc69b49..d951c9ccaa5 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexTest.java +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/index/sample/SampleIndexTest.java @@ -28,6 +28,7 @@ import org.opencb.opencga.core.response.VariantQueryResult; import org.opencb.opencga.storage.core.exceptions.StorageEngineException; import org.opencb.opencga.storage.core.metadata.models.SampleMetadata; +import org.opencb.opencga.storage.core.metadata.models.StudyMetadata; import org.opencb.opencga.storage.core.variant.VariantStorageBaseTest; import org.opencb.opencga.storage.core.variant.VariantStorageEngine; import org.opencb.opencga.storage.core.variant.VariantStorageOptions; @@ -73,6 +74,7 @@ public class SampleIndexTest extends VariantStorageBaseTest implements HadoopVar public static ExternalResource externalResource = new HadoopExternalResource(); private VariantHadoopDBAdaptor dbAdaptor; + private SampleIndexDBAdaptor sampleIndexDBAdaptor; private static boolean loaded = false; public static final String STUDY_NAME_3 = "study_3"; private static final List studies = Arrays.asList(STUDY_NAME, STUDY_NAME_2, STUDY_NAME_3); @@ -98,6 +100,7 @@ public class SampleIndexTest extends VariantStorageBaseTest implements HadoopVar @Before public void before() throws Exception { dbAdaptor = getVariantStorageEngine().getDBAdaptor(); + sampleIndexDBAdaptor = ((HadoopVariantStorageEngine) variantStorageEngine).getSampleIndexDBAdaptor(); if (!loaded) { load(); loaded = true; @@ -106,7 +109,7 @@ public void before() throws Exception { public void load() throws Exception { clearDB(DB_NAME); - + StudyMetadata.SampleIndexConfigurationVersioned versioned; HadoopVariantStorageEngine engine = getVariantStorageEngine(); // Study 1 - single file @@ -124,19 +127,26 @@ public void load() throws Exception { .append(VariantStorageOptions.STATS_CALCULATE.key(), false) .append(VariantStorageOptions.LOAD_SPLIT_DATA.key(), VariantStorageEngine.SplitData.MULTI); - int version = metadataManager.addSampleIndexConfiguration(STUDY_NAME_2, SampleIndexConfiguration.defaultConfiguration() - .addFileIndexField(new IndexFieldConfiguration(IndexFieldConfiguration.Source.SAMPLE, "DS", new double[]{0, 1, 2}))).getVersion(); - System.out.println("version = " + version); + versioned = metadataManager.addSampleIndexConfiguration(STUDY_NAME_2, SampleIndexConfiguration.defaultConfiguration() + .addFileIndexField(new IndexFieldConfiguration(IndexFieldConfiguration.Source.SAMPLE, "DS", new double[]{0, 1, 2})), true); + assertEquals(2, versioned.getVersion()); + assertEquals(StudyMetadata.SampleIndexConfigurationVersioned.Status.STAGING, versioned.getStatus()); runETL(engine, getResourceUri("by_chr/chr22_1-1.variant-test-file.vcf.gz"), outputUri, params, true, true, true); runETL(engine, getResourceUri("by_chr/chr22_1-2.variant-test-file.vcf.gz"), outputUri, params, true, true, true); runETL(engine, getResourceUri("by_chr/chr22_1-2-DUP.variant-test-file.vcf.gz"), outputUri, params, true, true, true); engine.familyIndex(STUDY_NAME_2, trios, new ObjectMap()); + versioned = metadataManager.getStudyMetadata(STUDY_NAME_2).getSampleIndexConfiguration(versioned.getVersion()); + assertEquals(2, versioned.getVersion()); + // Not annotated + assertEquals(StudyMetadata.SampleIndexConfigurationVersioned.Status.STAGING, versioned.getStatus()); + + // Study 3 - platinum metadataManager.addSampleIndexConfiguration(STUDY_NAME_3, SampleIndexConfiguration.defaultConfiguration() .addFileIndexField(new IndexFieldConfiguration(IndexFieldConfiguration.Source.FILE, "culprit", - IndexFieldConfiguration.Type.CATEGORICAL, "DP", "FS", "MQ", "QD").setNullable(true))); + IndexFieldConfiguration.Type.CATEGORICAL, "DP", "FS", "MQ", "QD").setNullable(true)), true); params = new ObjectMap() .append(VariantStorageOptions.STUDY.key(), STUDY_NAME_3) @@ -145,9 +155,52 @@ public void load() throws Exception { runETL(engine, getPlatinumFile(0), outputUri, params, true, true, true); runETL(engine, getPlatinumFile(1), outputUri, params, true, true, true); + versioned = metadataManager.getStudyMetadata(STUDY_NAME_3).getSampleIndexConfiguration(versioned.getVersion()); + assertEquals(2, versioned.getVersion()); + // Not annotated + assertEquals(StudyMetadata.SampleIndexConfigurationVersioned.Status.STAGING, versioned.getStatus()); + + + // ---------------- Annotate this.variantStorageEngine.annotate(new Query(), new QueryOptions(DefaultVariantAnnotationManager.OUT_DIR, outputUri)); engine.familyIndex(STUDY_NAME_3, triosPlatinum, new ObjectMap()); + // Study 1 - extra sample index configuration, not staging, only one sample in that configuration + + SampleIndexConfiguration configuration = engine.getMetadataManager().getStudyMetadata(STUDY_NAME).getSampleIndexConfigurationLatest().getConfiguration(); + // Don't modify the configuration. + versioned = engine.getMetadataManager().addSampleIndexConfiguration(STUDY_NAME, configuration, true); + assertEquals(2, versioned.getVersion()); + assertEquals(StudyMetadata.SampleIndexConfigurationVersioned.Status.STAGING, versioned.getStatus()); + + engine.sampleIndex(STUDY_NAME, Collections.singletonList("NA19660"), new ObjectMap()); + engine.sampleIndexAnnotate(STUDY_NAME, Collections.singletonList("NA19660"), new ObjectMap()); + + versioned = engine.getMetadataManager().getStudyMetadata(STUDY_NAME).getSampleIndexConfigurationLatest(false); + assertEquals(1, versioned.getVersion()); + assertEquals(StudyMetadata.SampleIndexConfigurationVersioned.Status.ACTIVE, versioned.getStatus()); + + versioned = engine.getMetadataManager().getStudyMetadata(STUDY_NAME).getSampleIndexConfigurationLatest(true); + assertEquals(2, versioned.getVersion()); + assertEquals(StudyMetadata.SampleIndexConfigurationVersioned.Status.STAGING, versioned.getStatus()); + + engine.getMetadataManager().updateStudyMetadata(STUDY_NAME, sm -> { + sm.getSampleIndexConfigurationLatest(true).setStatus(StudyMetadata.SampleIndexConfigurationVersioned.Status.ACTIVE); + }); + versioned = engine.getMetadataManager().getStudyMetadata(STUDY_NAME).getSampleIndexConfigurationLatest(false); + assertEquals(2, versioned.getVersion()); + assertEquals(StudyMetadata.SampleIndexConfigurationVersioned.Status.ACTIVE, versioned.getStatus()); + + // Study 2 - Latest should be active + versioned = metadataManager.getStudyMetadata(STUDY_NAME_2).getSampleIndexConfiguration(versioned.getVersion()); + assertEquals(2, versioned.getVersion()); + assertEquals(StudyMetadata.SampleIndexConfigurationVersioned.Status.ACTIVE, versioned.getStatus()); + + // Study 3 - Latest should be active + versioned = metadataManager.getStudyMetadata(STUDY_NAME_3).getSampleIndexConfiguration(versioned.getVersion()); + assertEquals(2, versioned.getVersion()); + assertEquals(StudyMetadata.SampleIndexConfigurationVersioned.Status.ACTIVE, versioned.getStatus()); + VariantHbaseTestUtils.printVariants(dbAdaptor, newOutputUri()); } @@ -159,7 +212,7 @@ public void checkLoadedData() throws Exception { Iterator it = variantStorageEngine.getMetadataManager().sampleMetadataIterator(studyId); while (it.hasNext()) { SampleMetadata sample = it.next(); - Iterator indexIt = variantStorageEngine.getSampleIndexDBAdaptor().rawIterator(studyId, sample.getId()); + Iterator indexIt = sampleIndexDBAdaptor.rawIterator(studyId, sample.getId()); while (indexIt.hasNext()) { SampleIndexEntry record = indexIt.next(); @@ -197,7 +250,10 @@ public void checkLoadedData() throws Exception { public void regenerateSampleIndex() throws Exception { for (String study : studies) { int studyId = dbAdaptor.getMetadataManager().getStudyId(study); - int version = dbAdaptor.getMetadataManager().getStudyMetadata(studyId).getSampleIndexConfigurationLatest().getVersion(); + // Get the version with ALL samples indexed + // This is a special case for STUDY, that has a sample index version with missing samples + int version = sampleIndexDBAdaptor.getSchemaFactory() + .getSchema(studyId, dbAdaptor.getMetadataManager().getIndexedSamplesMap(studyId).keySet(), true, false).getVersion(); String orig = dbAdaptor.getTableNameGenerator().getSampleIndexTableName(studyId, version); String copy = orig + "_copy"; @@ -205,6 +261,7 @@ public void regenerateSampleIndex() throws Exception { Compression.Algorithm.NONE); ObjectMap options = new ObjectMap() + .append(SampleIndexDriver.SAMPLE_INDEX_VERSION, version) .append(SampleIndexDriver.OUTPUT, copy) .append(SampleIndexDriver.SAMPLES, "all"); new TestMRExecutor().run(SampleIndexDriver.class, SampleIndexDriver.buildArgs( @@ -534,7 +591,6 @@ public SampleIndexQuery testQueryIndex(Query testQuery, Query query) throws Exce // Query SampleIndex System.out.println("#Query SampleIndex"); - SampleIndexDBAdaptor sampleIndexDBAdaptor = ((HadoopVariantStorageEngine) variantStorageEngine).getSampleIndexDBAdaptor(); Query sampleIndexVariantQuery = variantStorageEngine.preProcessQuery(query, new QueryOptions()); SampleIndexQuery indexQuery = sampleIndexDBAdaptor.parseSampleIndexQuery(sampleIndexVariantQuery); // int onlyIndex = (int) ((HadoopVariantStorageEngine) variantStorageEngine).getSampleIndexDBAdaptor() @@ -684,7 +740,6 @@ public void testCount() throws StorageEngineException { Query query = new Query(baseQuery) .append(VariantQueryParam.STUDY.key(), study) .append(GENOTYPE.key(), sampleName + ":1|0,0|1,1|1"); - SampleIndexDBAdaptor sampleIndexDBAdaptor = ((HadoopVariantStorageEngine) variantStorageEngine).getSampleIndexDBAdaptor(); long actualCount = sampleIndexDBAdaptor.count(sampleIndexDBAdaptor.parseSampleIndexQuery(new Query(query))); System.out.println("---"); @@ -711,7 +766,6 @@ public void testCount() throws StorageEngineException { @Test public void testAggregationCorrectnessFilterTranscript() throws Exception { - SampleIndexDBAdaptor sampleIndexDBAdaptor = ((HadoopVariantStorageEngine) variantStorageEngine).getSampleIndexDBAdaptor(); SampleIndexVariantAggregationExecutor executor = new SampleIndexVariantAggregationExecutor(metadataManager, sampleIndexDBAdaptor); String ct = "missense_variant"; @@ -753,8 +807,7 @@ public void testAggregationCorrectnessFilterTranscript() throws Exception { @Test public void testAggregationCorrectnessCt() throws Exception { - SampleIndexDBAdaptor sampleIndexDBAdaptor = ((HadoopVariantStorageEngine) variantStorageEngine).getSampleIndexDBAdaptor(); - SampleIndexSchema schema = sampleIndexDBAdaptor.getSchema(STUDY_NAME_3); + SampleIndexSchema schema = sampleIndexDBAdaptor.getSchemaLatest(STUDY_NAME_3); CategoricalMultiValuedIndexField field = schema.getCtIndex().getField(); IndexFieldConfiguration ctConf = field.getConfiguration(); @@ -785,7 +838,6 @@ public void testAggregationCorrectnessRegulatoryRegionVariant() throws Exception } private void testAggregationCorrectness(String ct) throws Exception { - SampleIndexDBAdaptor sampleIndexDBAdaptor = ((HadoopVariantStorageEngine) variantStorageEngine).getSampleIndexDBAdaptor(); SampleIndexVariantAggregationExecutor executor = new SampleIndexVariantAggregationExecutor(metadataManager, sampleIndexDBAdaptor); Query query = new Query(STUDY.key(), STUDY_NAME_3) @@ -816,7 +868,6 @@ private void testAggregationCorrectness(String ct) throws Exception { @Test public void testAggregation() throws Exception { - SampleIndexDBAdaptor sampleIndexDBAdaptor = ((HadoopVariantStorageEngine) variantStorageEngine).getSampleIndexDBAdaptor(); SampleIndexVariantAggregationExecutor executor = new SampleIndexVariantAggregationExecutor(metadataManager, sampleIndexDBAdaptor); testAggregation(executor, "qual", STUDY_NAME_3, "NA12877"); diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/mr/VariantMapReduceUtilTest.java b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/mr/VariantMapReduceUtilTest.java index 64864358501..933abfa26b7 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/mr/VariantMapReduceUtilTest.java +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/mr/VariantMapReduceUtilTest.java @@ -14,10 +14,13 @@ public class VariantMapReduceUtilTest { public void testSerializeConfiguration() throws IOException { Job job = Job.getInstance(); SampleIndexConfiguration configuration = SampleIndexConfiguration.defaultConfiguration(); - VariantMapReduceUtil.setSampleIndexConfiguration(job, configuration); + int version = 156; + VariantMapReduceUtil.setSampleIndexConfiguration(job, configuration, version); SampleIndexConfiguration actualConfiguration = VariantMapReduceUtil.getSampleIndexConfiguration(job.getConfiguration()); + int actualVersion = VariantMapReduceUtil.getSampleIndexConfigurationVersion(job.getConfiguration()); // System.out.println("configuration = " + configuration); assertEquals(configuration, actualConfiguration); + assertEquals(version, actualVersion); } } \ No newline at end of file diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/stats/SampleVariantStatsTest.java b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/stats/SampleVariantStatsTest.java index 598ea7497cb..7ecc4777547 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/stats/SampleVariantStatsTest.java +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-core/src/test/java/org/opencb/opencga/storage/hadoop/variant/stats/SampleVariantStatsTest.java @@ -60,7 +60,6 @@ public void before() throws Exception { mm.updateSampleMetadata(studyId, mm.getSampleId(studyId, child), sampleMetadata -> { sampleMetadata.setFather(mm.getSampleId(studyId, father)); sampleMetadata.setMother(mm.getSampleId(studyId, mother)); - return sampleMetadata; }); engine.annotate(new Query(), new ObjectMap()); diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-deps/opencga-storage-hadoop-deps-cdh5.13/pom.xml b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-deps/opencga-storage-hadoop-deps-cdh5.13/pom.xml index 1288b0e70b6..ad0a12c39f5 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-deps/opencga-storage-hadoop-deps-cdh5.13/pom.xml +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-deps/opencga-storage-hadoop-deps-cdh5.13/pom.xml @@ -22,7 +22,7 @@ org.opencb.opencga opencga-storage-hadoop-deps - 2.2.0-rc2 + 2.2.0 ../pom.xml diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-deps/opencga-storage-hadoop-deps-emr5.31/pom.xml b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-deps/opencga-storage-hadoop-deps-emr5.31/pom.xml index 3f725584646..6ce278d18f0 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-deps/opencga-storage-hadoop-deps-emr5.31/pom.xml +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-deps/opencga-storage-hadoop-deps-emr5.31/pom.xml @@ -22,7 +22,7 @@ org.opencb.opencga opencga-storage-hadoop-deps - 2.2.0-rc2 + 2.2.0 ../pom.xml diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-deps/opencga-storage-hadoop-deps-emr5.8/pom.xml b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-deps/opencga-storage-hadoop-deps-emr5.8/pom.xml index 7fe4bd49d01..e973698aa3b 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-deps/opencga-storage-hadoop-deps-emr5.8/pom.xml +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-deps/opencga-storage-hadoop-deps-emr5.8/pom.xml @@ -22,7 +22,7 @@ org.opencb.opencga opencga-storage-hadoop-deps - 2.2.0-rc2 + 2.2.0 ../pom.xml diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-deps/opencga-storage-hadoop-deps-emr6.1/pom.xml b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-deps/opencga-storage-hadoop-deps-emr6.1/pom.xml index a96a214ee6b..107060cff8e 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-deps/opencga-storage-hadoop-deps-emr6.1/pom.xml +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-deps/opencga-storage-hadoop-deps-emr6.1/pom.xml @@ -22,7 +22,7 @@ org.opencb.opencga opencga-storage-hadoop-deps - 2.2.0-rc2 + 2.2.0 ../pom.xml diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-deps/opencga-storage-hadoop-deps-hdp2.5/pom.xml b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-deps/opencga-storage-hadoop-deps-hdp2.5/pom.xml index 4a739530d95..2f89c731748 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-deps/opencga-storage-hadoop-deps-hdp2.5/pom.xml +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-deps/opencga-storage-hadoop-deps-hdp2.5/pom.xml @@ -22,7 +22,7 @@ org.opencb.opencga opencga-storage-hadoop-deps - 2.2.0-rc2 + 2.2.0 ../pom.xml diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-deps/opencga-storage-hadoop-deps-hdp2.6/pom.xml b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-deps/opencga-storage-hadoop-deps-hdp2.6/pom.xml index de61771d956..f90ef6c0610 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-deps/opencga-storage-hadoop-deps-hdp2.6/pom.xml +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-deps/opencga-storage-hadoop-deps-hdp2.6/pom.xml @@ -22,7 +22,7 @@ org.opencb.opencga opencga-storage-hadoop-deps - 2.2.0-rc2 + 2.2.0 ../pom.xml diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-deps/opencga-storage-hadoop-deps-hdp3.1/pom.xml b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-deps/opencga-storage-hadoop-deps-hdp3.1/pom.xml index 855133aad32..e5b4dfa1ba5 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-deps/opencga-storage-hadoop-deps-hdp3.1/pom.xml +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-deps/opencga-storage-hadoop-deps-hdp3.1/pom.xml @@ -22,7 +22,7 @@ org.opencb.opencga opencga-storage-hadoop-deps - 2.2.0-rc2 + 2.2.0 ../pom.xml diff --git a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-deps/pom.xml b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-deps/pom.xml index 643f377e968..7684cb435b7 100644 --- a/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-deps/pom.xml +++ b/opencga-storage/opencga-storage-hadoop/opencga-storage-hadoop-deps/pom.xml @@ -50,23 +50,23 @@ org.opencb.opencga opencga-storage-hadoop - 2.2.0-rc2 + 2.2.0 ../pom.xml - opencga-storage-hadoop-deps-hdp2.6 opencga-storage-hadoop-deps-hdp3.1 + opencga-storage-hadoop-deps-emr6.1 + - opencga-storage-hadoop-deps-emr6.1 opencga-storage-hadoop-deps - 2.2.0-rc2 + 2.2.0 pom diff --git a/opencga-storage/opencga-storage-hadoop/pom.xml b/opencga-storage/opencga-storage-hadoop/pom.xml index b066f831c38..5398ca5d146 100644 --- a/opencga-storage/opencga-storage-hadoop/pom.xml +++ b/opencga-storage/opencga-storage-hadoop/pom.xml @@ -30,12 +30,12 @@ org.opencb.opencga opencga-storage - 2.2.0-rc2 + 2.2.0 ../pom.xml opencga-storage-hadoop - 2.2.0-rc2 + 2.2.0 pom @@ -52,23 +52,33 @@ - hdp2.6 + hdp3.1 true - hdp2.6 + hdp3.1 opencga-storage-hadoop-deps-${opencga-storage-hadoop-deps.id} - hdp3.1 + emr6.1 - hdp3.1 + emr6.1 opencga-storage-hadoop-deps-${opencga-storage-hadoop-deps.id} + + + + + + + + + + @@ -82,20 +92,13 @@ - - emr5.31 - - emr5.31 - opencga-storage-hadoop-deps-${opencga-storage-hadoop-deps.id} - - - - emr6.1 - - emr6.1 - opencga-storage-hadoop-deps-${opencga-storage-hadoop-deps.id} - - + + + + + + + diff --git a/opencga-storage/opencga-storage-mongodb/pom.xml b/opencga-storage/opencga-storage-mongodb/pom.xml index 6a7655c1291..102f7e0c4d1 100644 --- a/opencga-storage/opencga-storage-mongodb/pom.xml +++ b/opencga-storage/opencga-storage-mongodb/pom.xml @@ -22,7 +22,7 @@ org.opencb.opencga opencga-storage - 2.2.0-rc2 + 2.2.0 ../pom.xml diff --git a/opencga-storage/opencga-storage-mongodb/src/main/java/org/opencb/opencga/storage/mongodb/variant/MongoDBVariantStorageEngine.java b/opencga-storage/opencga-storage-mongodb/src/main/java/org/opencb/opencga/storage/mongodb/variant/MongoDBVariantStorageEngine.java index cc31bdf0f9d..c93c6592f8e 100644 --- a/opencga-storage/opencga-storage-mongodb/src/main/java/org/opencb/opencga/storage/mongodb/variant/MongoDBVariantStorageEngine.java +++ b/opencga-storage/opencga-storage-mongodb/src/main/java/org/opencb/opencga/storage/mongodb/variant/MongoDBVariantStorageEngine.java @@ -130,14 +130,13 @@ public DataResult> familyIndex(String study, List> tri Integer mother = metadataManager.getSampleId(studyId, trio.get(1)); Integer child = metadataManager.getSampleId(studyId, trio.get(2)); metadataManager.updateSampleMetadata(studyId, child, sampleMetadata -> { - sampleMetadata.setFamilyIndexStatus(TaskMetadata.Status.READY); + sampleMetadata.setFamilyIndexStatus(TaskMetadata.Status.READY, 1); if (father != null && father > 0) { sampleMetadata.setFather(father); } if (mother != null && mother > 0) { sampleMetadata.setMother(mother); } - return sampleMetadata; }); } return new DataResult>().setResults(trios); diff --git a/opencga-storage/opencga-storage-server/pom.xml b/opencga-storage/opencga-storage-server/pom.xml index f14d577e4c7..f237b1ef494 100644 --- a/opencga-storage/opencga-storage-server/pom.xml +++ b/opencga-storage/opencga-storage-server/pom.xml @@ -22,7 +22,7 @@ org.opencb.opencga opencga-storage - 2.2.0-rc2 + 2.2.0 ../pom.xml diff --git a/opencga-storage/pom.xml b/opencga-storage/pom.xml index ab06f6cf764..4a48c849f71 100644 --- a/opencga-storage/pom.xml +++ b/opencga-storage/pom.xml @@ -22,13 +22,13 @@ org.opencb.opencga opencga - 2.2.0-rc2 + 2.2.0 ../pom.xml opencga-storage - 2.2.0-rc2 + 2.2.0 pom diff --git a/opencga-test/pom.xml b/opencga-test/pom.xml index 3193bb71f66..e4384debd95 100644 --- a/opencga-test/pom.xml +++ b/opencga-test/pom.xml @@ -24,7 +24,7 @@ org.opencb.opencga opencga-test - 2.2.0-rc2 + 2.2.0 pom diff --git a/pom.xml b/pom.xml index f4a16496982..5c859be80ab 100644 --- a/pom.xml +++ b/pom.xml @@ -22,7 +22,7 @@ org.opencb.opencga opencga - 2.2.0-rc2 + 2.2.0 pom OpenCGA @@ -43,12 +43,12 @@ - 2.2.0-rc2 + 2.2.0 ${opencga.version} 2.2.0-dev 5.0.2 - 2.2.1 - 4.2.0 + 2.2.2 + 4.2.1 2.2.0-dev 0.2.0 @@ -113,9 +113,9 @@ joaquintarraga@gmail.com - wbari - Wasim - wasimbari@gmail.com + juanfeSanahuja + Juanfe + juanfesanahuja@gmail.com