JSON based node description for KHR_interactivity

[javagl]

The pull request at #2293 adds a specification for KHR_interactivity in form of an AsciiDoc document. There have been considerations to additionally have a representation of the node definitions in a machine-processable form. In order to keep that PR focussed on the specification itself, I'll open this issue for further discussion about that topic.

Starting with disclaimers:

• What I'm posting here is NOT a normative reference! The ground truth can be found in the linked PR.
• What I'm posting here is not even a "proposal". Maybe not even a suggestion. Rather some random output from some experiments.

If we decide to define some form of JSON-based representation, then the question of how a possible JSON-based representation and the actual specification document could be aligned (when one of them changes) still has to be decided. This includes the question of what will be the normative source of truth, given that the normative-ness can only be achieved with the latexmath-based descriptions and the additional text that comes with the tables that define the actual node structures.

---

Representations

I'm summarizing a few, first thoughts about possible representations (and the full definitions and outputs are attached below). The representations have slightly different structures, which all could make sense for one case or another. Other representations may make more sense. This is intended for discussion, and ... maybe someone will find some of that useful.

Basic representation

One basic representation is that of a plain JSON hierarchy where the propery names reflect the section names, like
Nodes -> Math Nodes -> Constant -> E
The innermost elements could then be the descriptions of the actual nodes. An example:

{
  "name" : "Nodes",
  "children" : [ {
    "name" : "Math Nodes",
    "children" : [ {
      "name" : "Constants",
      "nodes" : [ {
        "title" : "E",
        "name" : "math/e",
        "description" : "Euler's number",
        "outputValueSockets" : [ {
          "type" : "float",
          "name" : "value",
          "description" : "2.718281828459045"
        } ]
      }, 

In this representation, the types are still written in the form that they have in the specification document. For example, with types like float{2|3|4} or floatN:

      {
        "title" : "Length",
        "name" : "math/length",
        "description" : "Vector length",
        "inputValueSockets" : [ {
          "type" : "float{2|3|4}",
          "name" : "a",
          "description" : "Vector"
        } ],
        "outputValueSockets" : [ {
          "type" : "float",
          "name" : "value",
          "description" : "Length of stem:[a], e.g., stem:[sqrt(a_x^2 + a_y^2)] for `float2`; see the description for details"
        } ]
      }

Spread representation

Similar to the above, but with with types like float{2|3|4} or floatN being "instantiated" accordingly - so each node that existed for float{2|3|4} now exists for float2, float3, float4. For example, for math/length:

      "name" : "Vector Nodes",
      "nodes" : [ {
        "title" : "Length",
        "name" : "math/length",
        "description" : "Vector length",
        "inputValueSockets" : [ {
          "type" : "float2",
          "name" : "a",
          "description" : "Vector"
        } ],
        "outputValueSockets" : [ {
          "type" : "float",
          "name" : "value",
          "description" : "Length of stem:[a], e.g., stem:[sqrt(a_x^2 + a_y^2)] for `float2`; see the description for details"
        } ]
      }, {
        "title" : "Length",
        "name" : "math/length",
        "description" : "Vector length",
        "inputValueSockets" : [ {
          "type" : "float3",
          "name" : "a",
          "description" : "Vector"
        } ],
        "outputValueSockets" : [ {
          "type" : "float",
          "name" : "value",
          "description" : "Length of stem:[a], e.g., stem:[sqrt(a_x^2 + a_y^2)] for `float2`; see the description for details"
        } ]
      }, {
        "title" : "Length",
        "name" : "math/length",
        "description" : "Vector length",
        "inputValueSockets" : [ {
          "type" : "float4",
          "name" : "a",
          "description" : "Vector"
        } ],
        "outputValueSockets" : [ {
          "type" : "float",
          "name" : "value",
          "description" : "Length of stem:[a], e.g., stem:[sqrt(a_x^2 + a_y^2)] for `float2`; see the description for details"
        } ]
      }

Spread with subsections

Similar to the above, but with all type instantations being in an own subsection. I.e. the instantations of the math/length nodes are now in a subection called math/length:

      "children" : [ {
        "name" : "math/length",
        "nodes" : [ {
          "title" : "Length (float2)",
          "name" : "math/length",
          "description" : "Vector length",
          "inputValueSockets" : [ {
            "type" : "float2",
            "name" : "a",
            "description" : "Vector"
          } ],
          "outputValueSockets" : [ {
            "type" : "float",
            "name" : "value",
            "description" : "Length of stem:[a], e.g., stem:[sqrt(a_x^2 + a_y^2)] for `float2`; see the description for details"
          } ]
        }, {
          "title" : "Length (float3)",
          "name" : "math/length",
          "description" : "Vector length",
          "inputValueSockets" : [ {
            "type" : "float3",
            "name" : "a",
            "description" : "Vector"
          } ],
          "outputValueSockets" : [ {
            "type" : "float",
            "name" : "value",
            "description" : "Length of stem:[a], e.g., stem:[sqrt(a_x^2 + a_y^2)] for `float2`; see the description for details"
          } ]
        }, {
          "title" : "Length (float4)",
          "name" : "math/length",
          "description" : "Vector length",
          "inputValueSockets" : [ {
            "type" : "float4",
            "name" : "a",
            "description" : "Vector"
          } ],
          "outputValueSockets" : [ {
            "type" : "float",
            "name" : "value",
            "description" : "Length of stem:[a], e.g., stem:[sqrt(a_x^2 + a_y^2)] for `float2`; see the description for details"
          } ]
        } ]
      }

---

Many other representations are possibe. Maybe the exact instantiation or section structure is also something that could or should be decided by the consumer of that JSON data. In that case, we should opt for the most generic and versatile representation (i.e. the first one, from which all others can be derived).

---

Current output

The actual sets of node definitions in these three representations is attached here:

leztew-output-2025-02-19.zip

---

Asides

This output is generated with some experimental project at https://github.com/javagl/leztew which I just switched to "public", because ... why not. Originally, this had a name like KhrInteractivityProcessing, and was just an experiment for processing AsciiDoc to begin with. However, one reason for trying out different structures was that I wanted to play a bit with feeding this data into some Flow processing library that I created a few years ago:

The leftmost column reflects the structure of the JSON representation (spread with subsections, as it can be seen in the math/length node).

(The other colums are just filtered views on the nodes that can provide an input or consume and output of the currently selected node - many degrees of freedom for how to make something like this accessible...)

[javagl]

Updated output with the matCompose and matDecompose nodes from c30626a :

leztew-output-2025-02-27.zip