From e16246f472e4d74fbade3d4085d8a41c93b87808 Mon Sep 17 00:00:00 2001 From: Ryan Brush Date: Wed, 28 Feb 2024 10:11:24 -0800 Subject: [PATCH] Reorganize pages as discussed during working meetings --- ...tructureDefinition-ViewDefinition-intro.md | 6 +- ...tructureDefinition-ViewDefinition-notes.md | 2 +- input/pagecontent/contributing.md | 22 +++++ ...ssing_model.md => implementer_guidance.md} | 12 ++- input/pagecontent/index.md | 91 +++++++++++++------ input/pagecontent/layers.md | 63 ------------- input/pagecontent/purpose.md | 4 +- sushi-config.yaml | 9 +- 8 files changed, 106 insertions(+), 103 deletions(-) create mode 100644 input/pagecontent/contributing.md rename input/pagecontent/{processing_model.md => implementer_guidance.md} (93%) delete mode 100644 input/pagecontent/layers.md diff --git a/input/pagecontent/StructureDefinition-ViewDefinition-intro.md b/input/pagecontent/StructureDefinition-ViewDefinition-intro.md index cc719e7..7148fe0 100644 --- a/input/pagecontent/StructureDefinition-ViewDefinition-intro.md +++ b/input/pagecontent/StructureDefinition-ViewDefinition-intro.md @@ -1,4 +1,4 @@ -A *view definition* is the central piece of the [View Layer](layers.html#the-view-layer) and represents a tabular projection of FHIR resources with the columns and filtering criteria defined by [FHIRPath](https://hl7.org/fhirpath/) expressions. The ViewDefinition logical model is described below. +A *view definition* is the central piece of the [View Layer](index.html#system-layers) and represents a tabular projection of FHIR resources with the columns and filtering criteria defined by [FHIRPath](https://hl7.org/fhirpath/) expressions. The ViewDefinition logical model is described below. ## Key ViewDefinition Elements The key elements of the ViewDefinition are: @@ -27,4 +27,6 @@ The ViewDefinition may include several other fields with `name`, `status`, and o ## Implementation A system-specific *view runner* implementation can execute a *view definition* and -return the results as a table that can easily be used in the user's tech stack. See the [layers](layers.html) page for details. +return the results as a table that can easily be used in the user's tech stack. See the +[system layers](index.html#system-layers) for details. + diff --git a/input/pagecontent/StructureDefinition-ViewDefinition-notes.md b/input/pagecontent/StructureDefinition-ViewDefinition-notes.md index b9e5f79..2e710da 100644 --- a/input/pagecontent/StructureDefinition-ViewDefinition-notes.md +++ b/input/pagecontent/StructureDefinition-ViewDefinition-notes.md @@ -488,7 +488,7 @@ This is an example of a constant used in the `where` constraint of a view: ## Processing Algorithm (Model) -See [Processing Algorithm](processing_model.html) for a description of how to +See [Processing Algorithm](implementer_guidance.html#processing-model) for a description of how to process a FHIR resource as input for a `ViewDefinition`. Implementations do not need to follow this algorithm directly, but their outputs should be consistent with what this model produces. diff --git a/input/pagecontent/contributing.md b/input/pagecontent/contributing.md new file mode 100644 index 0000000..a21b484 --- /dev/null +++ b/input/pagecontent/contributing.md @@ -0,0 +1,22 @@ +Contributors and early users are welcome! Here are some places to start: + +* Contribute to [github discussion](https://github.com/FHIR/sql-on-fhir-v2/discussions) +* Join us on [Weekly Meetings](https://us02web.zoom.us/meeting/register/tZApd-CgqzIiGdI163Q23yc6wihcfswAWBmO) +* Ask any questions in [FHIR chat](https://chat.fhir.org/#narrow/stream/179219-analytics-on-FHIR) + +### Credits +* Nikolai Ryzhikov @niquola (Health Samurai) +* Dan Gottlieb @gotdan (Central Square Solutions) +* Vadim Peretokin @vadi2 (Philips) +* Marat Surmashev @aitem (Health Samurai) +* Ryan Brush @rbrush (Google) +* Brian Kaney @bkaney (Vermonster) +* Josh Mandel @jmandel (Microsoft) +* John Grimes @johngrimes (CSIRO) +* FHIR Community - https://chat.fhir.org/ + +Work is sponsored and supported by: +* [Health Samurai](https://www.health-samurai.io/) +* [FHIR Foundation](https://fhir.org/) +* [Health Dev Hub](https://www.healthdevhub.com/) +* Interested in helping to support this work - contact us! \ No newline at end of file diff --git a/input/pagecontent/processing_model.md b/input/pagecontent/implementer_guidance.md similarity index 93% rename from input/pagecontent/processing_model.md rename to input/pagecontent/implementer_guidance.md index 2d1e086..8e5601c 100644 --- a/input/pagecontent/processing_model.md +++ b/input/pagecontent/implementer_guidance.md @@ -1,9 +1,19 @@ +This spec defines the ViewDefinition model and logical system layers, but intentionally +leaves the details of those layers to implementations. However, implementations themselves +are encouraged to share common approaches when helpful. + +### Implementation Patterns + +TODO: add links to implementation patterns (like Spark, JSON, etc) as these efforts materialize. + +### Processing Model + The following description provides an algorithm for how to process a FHIR resource as input for a `ViewDefinition`. Implementations do not need to follow this algorithm directly, but their outputs should be consistent with what this model produces. -### Validate Columns (entry point) +#### Validate Columns (entry point) **Purpose**: This step ensures that a ViewDefinition's columns are valid, by setting up a recursive call. diff --git a/input/pagecontent/index.md b/input/pagecontent/index.md index 9fc2ab7..0a7f695 100644 --- a/input/pagecontent/index.md +++ b/input/pagecontent/index.md @@ -61,39 +61,72 @@ in your database of choice, using the view name as the table name: Such tabular views can be created for any FHIR resource, with [more examples here](artifacts.html#example-example-instances). See the -[View Definition page](StructureDefinition-ViewDefinition.html) for details, and the -[System Layers](layers.html) page for how views fit into a larger analytic ecosystem. +[View Definition page](StructureDefinition-ViewDefinition.html) for the full definition +of the above structure. -### Contributing -Contributors and early users are welcome! Here are some places to start: +### System Layers -* Contribute to [github discussion](https://github.com/FHIR/sql-on-fhir-v2/discussions) -* Join us on [Weekly Meetings](https://us02web.zoom.us/meeting/register/tZApd-CgqzIiGdI163Q23yc6wihcfswAWBmO) -* Ask any questions in [FHIR chat](https://chat.fhir.org/#narrow/stream/179219-analytics-on-FHIR) +The [View Definition](StructureDefinition-ViewDefinition.html) is the central element of this +spec, but in practice it is really one layer of an overall system. The layers are: -### How to read this guide -* **[Home](index.html)**: Intro to the project -* Specification: - * **[Purpose](purpose.html)**: Backround and purpose of this project - * **[System Layers](layers.html)**: Conceptual layers in this specification - * **[View Definition](StructureDefinition-ViewDefinition.html)**: A tabular projection of a FHIR resource +- the *Data Layer* +- the *View Layer* +- and the *Analytics Layer*. + +High-level diagram of layers + +**Figure 1: High-level diagram of layers** + +### The Data Layer +The *Data Layer* is a set of lossless representations that collectively enable FHIR +to be used with a wide variety of different query technologies. It may +optionally be persisted and annotated to make it or implementations of the view +layer more efficient, but no specific Data Layer structure will be required by +this specification. + +Implementations are encouraged but not required to further annotate the FHIR +resources to help View layer implementations run efficient queries. This +primarily applies when the underlying FHIR resources are stored in databases +that the View layer will query. + +### The View Layer +The *View Layer* defines portable, tabular views of FHIR data that can more easily +be consumed by a wide variety of analytic tools. The use of these tools is +described in *Analytics Layer* section. Our goal here is simply to get +the needed FHIR data in a form that matches user needs and common analytic +patterns. + +The View Layer itself has two key components: + +* *View Definitions*, allowing users to define flattened views of FHIR data that +are portable between systems. +* *View Runners* are system-specific tools or libraries that apply view definitions to +the underlying data layer, optionally making use of annotations to optimize performance. + +See the [View Definition documentation](StructureDefinition-ViewDefinition.html) for details and examples; +these are the central piece of this specification. + +View Runners will be specific to the data +layer they use. Each data layer may have one or more corresponding view +runners, but a given View Definition can be run by many runners over many +data layers. + +Example view runners may include: + +* A runner that creates a virtual, tabular view in an analytic database +* A runner that queries FHIR JSON directly and creates a table in a web application +* A runner that loads data directly into a notebook or other data analysis tool + +### The Analytics Layer + +Finally, users must be able to easily leverage the above views with the analytic +tools of their choice. This spec purposefully does not define what these are, +but common use cases may be SQL queries by consuming applications, dataframe-based +data science tools in Python or R, or integration with business intelligence tools. ### License FHIR® is the registered trademark of HL7 and is used with the permission of HL7. Use of the FHIR trademark does not constitute endorsement of the contents of this repository by HL7, nor affirmation that this data is conformant to the various applicable standards -### Credits -* Nikolai Ryzhikov @niquola (Health Samurai) -* Dan Gottlieb @gotdan (Central Square Solutions) -* Vadim Peretokin @vadi2 (Philips) -* Marat Surmashev @aitem (Health Samurai) -* Ryan Brush @rbrush (Google) -* Brian Kaney @bkaney (Vermonster) -* Josh Mandel @jmandel (Microsoft) -* John Grimes @johngrimes (CSIRO) -* FHIR Community - https://chat.fhir.org/ - -Work is sponsored and supported by: -* [Health Samurai](https://www.health-samurai.io/) -* [FHIR Foundation](https://fhir.org/) -* [Health Dev Hub](https://www.healthdevhub.com/) -* Interested in helping to support this work - contact us! +--- + +**[Next: Purpose](purpose.html)** diff --git a/input/pagecontent/layers.md b/input/pagecontent/layers.md deleted file mode 100644 index 2bc1424..0000000 --- a/input/pagecontent/layers.md +++ /dev/null @@ -1,63 +0,0 @@ - -The system consists of three logical layers as shown in the diagram below: -- the *Data Layer* -- the *View Layer* -- and the *Analytics Layer*. - -This specification focuses primarily upon the View layer. The Data and Analytics layers are optional, -and are provided as general patterns to assist with implementation. - -High-level diagram of layers - -**Figure 1: High-level diagram of layers** - -### The Data Layer -The *Data Layer* is a set of lossless representations that collectively enable FHIR -to be used with a wide variety of different query technologies. It may -optionally be persisted and annotated to make it or implementations of the view -layer more efficient, but no specific Data Layer structure will be required by -this specification. - -Implementations are encouraged but not required to further annotate the FHIR -resources to help View layer implementations run efficient queries. This -primarily applies when the underlying FHIR resources are stored in databases -that the View layer will query. - -### The View Layer -The *View Layer* defines portable, tabular views of FHIR data that can more easily -be consumed by a wide variety of analytic tools. The use of these tools is -described in *Analytics Layer* section. Our goal here is simply to get -the needed FHIR data in a form that matches user needs and common analytic -patterns. - -The View Layer itself has two key components: - -* *View Definitions*, allowing users to define flattened views of FHIR data that -are portable between systems. -* *View Runners* are system-specific tools or libraries that apply view definitions to -the underlying data layer, optionally making use of annotations to optimize performance. - -See the [View Definition documentation](StructureDefinition-ViewDefinition.html) for details and examples; -these are the central piece of this specification. - -View Runners will be specific to the data -layer they use. Each data layer may have one or more corresponding view -runners, but a given View Definition can be run by many runners over many -data layers. - -Example view runners may include: - -* A runner that creates a virtual, tabular view in an analytic database -* A runner that queries FHIR JSON directly and creates a table in a web application -* A runner that loads data directly into a notebook or other data analysis tool - -### The Analytics Layer - -Finally, users must be able to easily leverage the above views with the analytic -tools of their choice. This spec purposefully does not define what these are, -but common use cases may be SQL queries by consuming applications, dataframe-based -data science tools in Python or R, or integration with business intelligence tools. - ---- - -**[Next: View Definitions](StructureDefinition-ViewDefinition.html)** diff --git a/input/pagecontent/purpose.md b/input/pagecontent/purpose.md index 9a0fa4c..2e1ebe6 100644 --- a/input/pagecontent/purpose.md +++ b/input/pagecontent/purpose.md @@ -1,4 +1,4 @@ -This document proposes an approach to make large-scale analysis of FHIR data +This specification proposes an approach to make large-scale analysis of FHIR data accessible to a larger audience and portable between systems. The central goal of this project is to make FHIR data work well with the best available analytic tools, regardless of the technology stack. @@ -101,4 +101,4 @@ that is, to use the ETL or ELT patterns. --- -**[Next: System Layers](layers.html)** +**[Next: View Definitions](StructureDefinition-ViewDefinition.html)** diff --git a/sushi-config.yaml b/sushi-config.yaml index 7dedff9..970fbcc 100644 --- a/sushi-config.yaml +++ b/sushi-config.yaml @@ -70,12 +70,11 @@ parameters: # │ property below. │ # ╰────────────────────────────────────────────────────────────────────────────────────────────────╯ menu: - Home: index.html + Overview: index.html Purpose: purpose.html - Spec: - System Layers: layers.html - View Definition: StructureDefinition-ViewDefinition.html - Processing Model: processing_model.html + View Definition: StructureDefinition-ViewDefinition.html + Implementer Guidance: implementer_guidance.html + Contributing: contributing.html # ╭───────────────────────────Less Common Implementation Guide Properties──────────────────────────╮ # │ Uncomment the properties below to configure additional properties on the ImplementationGuide │