docs: add tutorials

docs/index.md

@@ -41,6 +41,8 @@ creation of new workflow jobs for sites to claim and execute.
 
 ## Site Federation
 
+Below is a figure that illustrates the federated nature of site interaction with the runtime.
+
 <figure markdown>
   ![NMDC Runtime Site Federation](img/nmdc-runtime-federation.png)
   <figcaption>NMDC Runtime Site Federation</figcaption>

docs/tutorial-auth.md

@@ -0,0 +1,86 @@
+# Tutorial - User/Site Login and Authentication
+
+## Log in as a User
+
+Open <https://api.dev.microbiomedata.org/docs> and click on the `Authorize` button near the top of
+the page:
+
+![auth button](img/auth-button.png)
+
+In the modal dialog, enter your given `username` and `password` in the first form, and click
+`Authorize`:
+
+![auth-username](img/auth-username.png)
+
+Once authorized, hit `Close` to get back to the interactive API documentation:
+
+![auth-close-modal](img/auth-close-modal.png)
+
+## Create a User
+
+!!! info 
+    You need to be already logged in to create a new user.
+    Also, only users `dehays`, `dwinston`, and `scanon` can create new users at this time.
+
+Go to [POST /users](https://api.dev.microbiomedata.org/docs#/users/create_user_users_post), and
+click the `Try it out` button. In the request body, the only required fields are `username` and
+`password`. If you know the `id`s of any sites you would like the new user to administer, enter
+those as an array value for the `site_admin` field.
+
+Share a new user's password securely with them. For example, a free instance of the open-source
+[snappass](https://github.com/pinterest/snappass) web app is hosted by `dwinston` at
+<https://snappass-polyneme.herokuapp.com/>. This will generate a one-time link that you can email to
+the new user (you cannot send the link via Slack because Slack by default opens links in order to
+display a preview of the page content).
+
+## Create a Site Client
+
+If you administer one or more sites, you can generate credentials for a site client that will act on
+behalf of the site. This is used for managing certain API resources -- rather than a person being
+responsible for a resource, a site is, and users that administer the site can come and go.
+
+Once logged in, you can use [GET
+/users/me](https://api.dev.microbiomedata.org/docs#/users/read_users_me_users_me__get) to see the
+`id`s of sites you can administer. Example response:
+
+```json
+{
+  "username": "dwinston",
+  "site_admin": [
+    "dwinston-laptop",
+    "nmdc-runtime-useradmin",
+    "dwinston-cloud"
+  ]
+}
+```
+
+You can create your own sites via [POST
+/sites](https://api.dev.microbiomedata.org/docs#/sites/create_site_sites_post). If the `id` you
+request already exists, you will get an error response. Do not worry about setting `capability_ids`;
+those can be set later.
+
+Once you have identified a site for which to generate credentials, use [POST
+/sites/{site_id}:generateCredentials](https://api.dev.microbiomedata.org/docs#/sites/generate_credentials_for_site_client_sites__site_id__generateCredentials_post)
+to do so. The response will look something like
+
+```json
+{
+  "client_id": "sys0***",
+  "client_secret": "***"
+}
+```
+
+Save this information like you saved your username and password.
+
+## Log in as a Site
+
+Click on the `Authorize` button near the top of <https://api.dev.microbiomedata.org/docs>. You may
+need to `Logout` first. Scroll all the way down to the second form, the one with only two fields,
+`client_id` and `client_secret`:
+
+![auth-client-id](img/auth-client-id.png)
+
+Note that the form is for the `clientCredentials` flow, whereas the first form is for the `password`
+flow. In this clientCredentials form, enter your site client credentials and `Authorize`. You are
+now logged in as a site client.
+

docs/tutorial-create-user.md

@@ -0,0 +1,2 @@
+# Tutorial - Create a User
+

docs/tutorial-json.md

@@ -0,0 +1,176 @@
+# Tutorial - Validate JSON and Fetch JSON
+
+Let's dive in and get acquainted with the NMDC Runtime API.
+
+## Validate JSON
+
+Already? Yes. Let's do this. Here is a tiny
+[nmdc:Database](https://microbiomedata.github.io/nmdc-schema/Database/) JSON object:
+
+```json
+{"biosample_set": [{"id": 42}]}
+```
+
+This represents a set of [nmdc:Biosample](https://microbiomedata.github.io/nmdc-schema/Biosample/)
+objects. There is just one, with an `id` of `42`.
+
+Let's validate it. Go to the [POST
+/metadata/json:validate](https://api.dev.microbiomedata.org/docs#/metadata/validate_json_metadata_json_validate_post)
+endpoint at <https://api.dev.microbiomedata.org/docs> and click "Try it out":
+
+![Try it Out](img/validate-json-try-it-out.png)
+
+Now, copy the above JSON object, paste it into the `Request body` field, and hit `Execute`:
+
+![Paste Body and Execute](img/validate-json-copy-paste-execute.png)
+
+This gives us a response where the result is "errors". Looks like a biosample `id` needs to be a
+string value, and we are missing required properties. We also get a display of a `curl` command
+to reproduce the request on the command line:
+
+![Validation Response](img/validate-json-response.png)
+
+Let's see what a "valid" response looks like. The [GET
+/nmdcschema/{collection_name}/{doc_id}](https://api.dev.microbiomedata.org/docs#/metadata/get_from_collection_by_id_nmdcschema__collection_name___doc_id__get)
+endpoint allows us to get the NMDC-schema-validated JSON object for one of the NMDC metadata
+collections:
+
+![Get one valid](img/validate-json-get-one-valid.png)
+
+For example,
+[https://api.dev.microbiomedata.org/nmdcschema/biosample_set/gold:Gb0115217](https://api.dev.microbiomedata.org/nmdcschema/biosample_set/gold:Gb0115217)
+is
+
+```json
+{
+  "location": "groundwater-surface water interaction zone in Washington, USA",
+  "env_medium": {
+    "has_raw_value": "ENVO:01000017"
+  },
+  "depth2": {
+    "has_raw_value": "1.0",
+    "has_numeric_value": 1,
+    "has_unit": "meter"
+  },
+  "env_broad_scale": {
+    "has_raw_value": "ENVO:01000253"
+  },
+  "alternative_identifiers": [
+    "img.taxon:3300042741"
+  ],
+  "ecosystem": "Engineered",
+  "ecosystem_category": "Artificial ecosystem",
+  "id": "gold:Gb0115217",
+  "env_local_scale": {
+    "has_raw_value": "ENVO:01000621"
+  },
+  "community": "microbial communities",
+  "mod_date": "2021-06-17",
+  "ecosystem_subtype": "Unclassified",
+  "INSDC_biosample_identifiers": [
+    "biosample:SAMN06343863"
+  ],
+  "description": "Sterilized sand packs were incubated back in the ground and collected at time point T2.",
+  "collection_date": {
+    "has_raw_value": "2014-09-23"
+  },
+  "ecosystem_type": "Sand microcosm",
+  "sample_collection_site": "sand microcosm",
+  "name": "Sand microcosm microbial communities from a hyporheic zone in Columbia River, Washington, USA - GW-RW T2_23-Sept-14",
+  "lat_lon": {
+    "has_raw_value": "46.37228379 -119.2717467",
+    "latitude": 46.37228379,
+    "longitude": -119.2717467
+  },
+  "specific_ecosystem": "Unclassified",
+  "identifier": "GW-RW T2_23-Sept-14",
+  "GOLD_sample_identifiers": [
+    "gold:Gb0115217"
+  ],
+  "add_date": "2015-05-28",
+  "habitat": "sand microcosm",
+  "type": "nmdc:Biosample",
+  "depth": {
+    "has_raw_value": "0.5",
+    "has_numeric_value": 0.5,
+    "has_unit": "meter"
+  },
+  "part_of": [
+    "gold:Gs0114663"
+  ],
+  "ncbi_taxonomy_name": "sediment metagenome",
+  "geo_loc_name": {
+    "has_raw_value": "USA: Columbia River, Washington"
+  }
+}
+```
+
+Now, copying and paste the above into the request body for `POST /metadata/json:validate`. Remember,
+the body needs to be a nmdc:Database object, in this case with a single member of the biosample_set
+collection, so copy and paste the `{"biosample_set": [` and `]}` parts to book-end the document
+JSON:
+
+```json
+{"biosample_set": [
+"PASTE_JSON_DOCUMENT_HERE"
+]}
+```
+
+Now, when you execute the request, the response body will be
+
+```json
+{
+  "result": "All Okay!"
+}
+```
+
+Hooray!
+
+## Get a List of NMDC-Schema-Compliant Documents
+
+The [GET
+/nmdcschema/{collection_name}](https://api.dev.microbiomedata.org/docs#/metadata/list_from_collection_nmdcschema__collection_name__get)
+endpoint allows you to get a filtered list of documents from one of the NMDC Schema collections:
+
+![List from collections](img/list-from-collection.png)
+
+The `collection_name` must be one defined for a
+[nmdc:Database](https://microbiomedata.github.io/nmdc-schema/Database/), in the form expected by the
+JSON Schema,
+[nmdc.schema.json](https://github.com/microbiomedata/nmdc-schema/blob/69fd1ee91afac1a943b2cc9bfbfdecd0e2cdd089/jsonschema/nmdc.schema.json#L987).
+This typically means that any spaces in the name should be entered as underscores (`_`) instead.
+
+The `filter`, if provided, is a JSON document in the form of the
+[MongoDB Query Language](https://docs.mongodb.com/manual/tutorial/query-documents/). For example,
+the filter `{"part_of": "gold:Gs0114663"}` on collection_name `biosample_set` will list biosamples
+that are part of the `gold:Gs0114663` study:
+
+![List from collection, with filter](img/list-from-collection-filter.png)
+
+When I execute that query, I use the default `max_page_size` of 20, meaning at most 20 documents are
+returned at a time. A much larger `max_page_size` is fine for programs/scripts, but can make your
+web browser less responsive when using the interactive documentation.
+
+The response body for [our
+request](https://api.dev.microbiomedata.org/nmdcschema/biosample_set?filter=%7B%22part_of%22%3A%20%22gold%3AGs0114663%22%7D&max_page_size=20)
+has two fields, `resources` and `next_page_token`:
+
+```json
+{
+  "resources": [
+    ...
+  ],
+  "next_page_token": "nmdc:sys0s8f846"
+}
+
+```
+
+`resources` is a list of documents. `next_page_token` is a value you can plug into a subsequent
+request as the `page_token` parameter:
+
+![List from collection, with filter and page token](img/list-from-collection-page-token.png)
+
+This will return the next page of results. You do need to keep the other request parameters the
+same. In this way, you can page through and retrieve all documents that match a given filter (or no
+filter) for a given collection. Page tokens are ephemeral: once you use one in a request, it is
+removed from the system's memory.

docs/tutorial-metadata-in.md

@@ -0,0 +1,3 @@
+# Tutorial - Submit Metadata as JSON
+
+Requires site credentials.

mkdocs.yml

@@ -1,5 +1,17 @@
 site_name: nmdc-runtime docs
 site_url: https://microbiomedata.github.io/nmdc-runtime/
+nav:
+  - index.md
+  - Jobs:
+    - jobs/emsl-translation-etl.md
+    - jobs/gold-translation-etl.md
+    - jobs/jgi-translation-etl.md
+  - tutorial-json.md
+  - tutorial-auth.md
+  #- tutorial-metadata-in.md
+  - contributing.md
+  - admin.md
+
 theme:
   name: material
   palette:
@@ -14,14 +26,6 @@ theme:
         icon: material/toggle-switch
         name: Switch to light mode
 
-nav:
-  - index.md
-  - Jobs:
-    - jobs/emsl-translation-etl.md
-    - jobs/gold-translation-etl.md
-    - jobs/jgi-translation-etl.md
-  - contributing.md
-  - admin.md
 markdown_extensions:
   - admonition
   - attr_list

tmp.json

@@ -0,0 +1,61 @@
+{
+  "id": "gold:Gb0115217",
+  "location": "groundwater-surface water interaction zone in Washington, USA",
+  "env_medium": {
+    "has_raw_value": "ENVO:01000017"
+  },
+  "depth2": {
+    "has_raw_value": "1.0",
+    "has_numeric_value": 1,
+    "has_unit": "meter"
+  },
+  "env_broad_scale": {
+    "has_raw_value": "ENVO:01000253"
+  },
+  "alternative_identifiers": [
+    "img.taxon:3300042741"
+  ],
+  "ecosystem": "Engineered",
+  "ecosystem_category": "Artificial ecosystem",
+  "env_local_scale": {
+    "has_raw_value": "ENVO:01000621"
+  },
+  "community": "microbial communities",
+  "mod_date": "2021-06-17",
+  "ecosystem_subtype": "Unclassified",
+  "INSDC_biosample_identifiers": [
+    "biosample:SAMN06343863"
+  ],
+  "description": "Sterilized sand packs were incubated back in the ground and collected at time point T2.",
+  "collection_date": {
+    "has_raw_value": "2014-09-23"
+  },
+  "ecosystem_type": "Sand microcosm",
+  "sample_collection_site": "sand microcosm",
+  "name": "Sand microcosm microbial communities from a hyporheic zone in Columbia River, Washington, USA - GW-RW T2_23-Sept-14",
+  "lat_lon": {
+    "has_raw_value": "46.37228379 -119.2717467",
+    "latitude": 46.37228379,
+    "longitude": -119.2717467
+  },
+  "specific_ecosystem": "Unclassified",
+  "identifier": "GW-RW T2_23-Sept-14",
+  "GOLD_sample_identifiers": [
+    "gold:Gb0115217"
+  ],
+  "add_date": "2015-05-28",
+  "habitat": "sand microcosm",
+  "type": "nmdc:Biosample",
+  "depth": {
+    "has_raw_value": "0.5",
+    "has_numeric_value": 0.5,
+    "has_unit": "meter"
+  },
+  "part_of": [
+    "gold:Gs0114663"
+  ],
+  "ncbi_taxonomy_name": "sediment metagenome",
+  "geo_loc_name": {
+    "has_raw_value": "USA: Columbia River, Washington"
+  }
+}