Add descriptions.tsv to the schema (#1707)

* Add descriptions.tsv to schema.

* Fix up column names.

* Keep working on schema.

* Fix mistakes and add template.

* Update src/derivatives/common-data-types.md

Co-authored-by: Chris Markiewicz <[email protected]>

* Make description column generic.

* Remove duplicate file definition.

descriptions is a suffix, rather than a file.

---------

Co-authored-by: Chris Markiewicz <[email protected]>

src/derivatives/common-data-types.md

@@ -256,24 +256,33 @@ static volume, a `RepetitionTime` property would no longer be relevant).
 
 ## descriptions.tsv
 
+Template:
+
+```Text
+[sub-<label>/]
+    [ses-<label>/]
+        [sub-<label>_][ses-<label>_]descriptions.tsv
+        [sub-<label>_][ses-<label>_]descriptions.json
+```
+
+Optional: Yes
+
 To keep a record of processing steps applied to the data, a `descriptions.tsv` file MAY be used.
-The `descriptions.tsv` file MUST contain at least the following two columns:
+The `descriptions.tsv` file consists of one row for each unique `desc-<label>`
+entity used in the dataset and a set of REQUIRED and OPTIONAL columns:
 
--   `desc_id`
--   `description`
+<!-- This block generates a columns table.
+The definitions of these fields can be found in
+  src/schema/rules/tabular_data/*.yaml
+and a guide for using macros can be found at
+ https://github.com/bids-standard/bids-specification/blob/master/macros_doc.md
+-->
+{{ MACROS___make_columns_table("derivatives.common_derivatives.Descriptions") }}
 
 This file MAY be located at the root of the derivative dataset,
 or at the subject or session level
 ([Inheritance Principle](../common-principles.md#the-inheritance-principle)).
 
-The `desc_id` column contains the labels used with the [`desc entity`](../appendices/entities.md#desc),
-within the particular nesting that the `descriptions.tsv` file is placed.
-For example, if the `descriptions.tsv` file is placed at the root of the derivative dataset,
-its `desc_id` column SHOULD contain all labels of the [`desc entity`](../appendices/entities.md#desc)
-used across the entire derivative dataset.
-
-The `description` column contains human-readable descriptions of the processing steps.
-
 The use of `descriptions.tsv` files together with the [`desc entity`](../appendices/entities.md#desc)
 are helpful to document how files are generated, even if their use may not be sufficient
 to provide full computational reproducibility.

src/schema/objects/columns.yaml

@@ -110,6 +110,21 @@ derived_from:
     for example a slice of tissue (`sample-02`) derived from a block of tissue (`sample-01`).
   type: string
   pattern: ^sample-[0-9a-zA-Z]+$
+desc_id:
+  name: desc_id
+  display_name: Description Label
+  description: |
+    A `desc-<label>` entity present in the derivatives dataset.
+
+    The `desc_id` column contains the labels used with the
+    [`desc` entity](SPEC_ROOT/appendices/entities.md#desc),
+    within the particular nesting that the `descriptions.tsv` file is placed.
+
+    For example, if the `descriptions.tsv` file is placed at the root of the derivative dataset,
+    its `desc_id` column SHOULD contain all labels of the `desc` entity)
+    used across the entire derivative dataset.
+  type: string
+  pattern: ^desc-[0-9a-zA-Z]+$
 description:
   name: description
   display_name: Description
@@ -122,6 +137,12 @@ description__optode:
   description: |
     Free-form text description of the optode, or other information of interest.
   type: string
+description__entities:
+  name: description
+  display_name: Description
+  description: |
+    Free-form text description of the entity's label (defined in `<entity>_id` column).
+  type: string
 dimension:
   name: dimension
   display_name: Dimension

src/schema/objects/suffixes.yaml

@@ -549,6 +549,18 @@ defacemask:
   description: |
     A binary mask that was used to remove facial features from an anatomical MRI
     image.
+descriptions:
+  value: descriptions
+  display_name: Description Entity Definitions
+  description: |
+    A TSV file describing labels found for the `desc` entity in a Derivatives dataset.
+
+    This file MAY be located at the root of the derivative dataset,
+    or at the subject or session level.
+
+    The use of `descriptions.tsv` files together with the desc entity are helpful to
+    document how files are generated,
+    even if their use may not be sufficient to provide full computational reproducibility.
 dseg:
   value: dseg
   display_name: Discrete Segmentation

src/schema/rules/files/deriv/tables.yaml

@@ -0,0 +1,11 @@
+---
+descriptions:
+  level: optional
+  suffixes:
+    - descriptions
+  extensions:
+    - .tsv
+    - .json
+  entities:
+    subject: optional
+    session: optional

src/schema/rules/tabular_data/derivatives/common_derivatives.yaml

@@ -10,3 +10,19 @@ SegmentationLookup:
     color: optional
     mapping: optional
   index_columns: [index]
+
+Descriptions:
+  selectors:
+    - suffix == "descriptions"
+    - extension == ".tsv"
+  initial_columns:
+    - desc_id
+    - description__entities
+  columns:
+    desc_id: required
+    description__entities:
+      level: required
+      description_addendum: |
+        The corresponding label column is `desc_id`.
+  index_columns: [desc_id]
+  additional_columns: allowed