Standardize static vocabulary JSON files between tools

[alyssadai]

The n-API and annotation tool currently use slightly different formats for JSON files containing all terms from an external vocabulary/sub-vocab, e.g. SNOMED assessments or SNOMED disorders.

e.g., for diagnoses,

API:

  {
    "sctid": "10007009",
    "preferred_name": "Coffin-Siris syndrome"
  },

Annotation tool:

   {
      "identifier":"snomed:10007009",
      "label":"Coffin-Siris syndrome"
   },

We should pick one format and be consistent with it across tools, if not across all vocabularies, then at least across the same vocabulary.

Considerations:

• The code to generate SNOMED vocabulary files from Athena currently produces the second format, i.e. where Neurobagel-specific prefixes are baked into the terms
• The annotation tool vocab file for SNOMED disorders (second format) additionally has the NCIT healthy control term in it (if we use the same file for the API, we may want to remove it from vocab endpoints to avoid confusion?)
• The first format does not assume anything about vocabulary namespace prefixes specific to Neurobagel, and instead relies on the code that uses the JSON files to attach those as needed
• The downside of hardcoding namespace prefixes (which are arbitrary shorthands decided by us - they do not end up in any of the actual graphs) is that if we ever decide to change them, we have to also change all the vocab files

[alyssadai]

@neurobagel/dev, please take a look at the issue description (including the considerations) and comment your preference - the key decision is whether to include the namespace prefixes in the JSON.

[rmanaem]

I prefer the Annotation tool version i.e., keeping the namespace prefix in the JSON.

[surchs]

Me too, I prefer the annotation tool version. I think having the prefix in there is a helpful sanity check if and when we switch vocabularies. It's likely a bit of extra works for some future (and past) vocabularies that only provide the term IDs, but I think that's worth the effort.

[alyssadai]

Okay, sounds good. To clarify, assuming we include prefixes, I think the two scenarios in which we would need to change the JSON file are:

1. If we decide to rename a vocabulary prefix itself (e.g. ncit used to be called purl by our tools, even though the actual healthy control term URL we’ve used hasn’t changed)
2. if we change the vocabulary itself (but we would need to update the JSON anyways since the terms would change