From 1044a7ee278e2eb365a8fe2068a8a26b7c7673a4 Mon Sep 17 00:00:00 2001 From: Auto Gen Date: Fri, 16 Feb 2024 07:29:06 -0800 Subject: [PATCH 1/3] Add Mqtt extension --- DTDL/v3/DTDL.Extensions.md | 1 + DTDL/v3/DTDL.mqtt.v1.md | 198 ++++++++++++++++++ DTDL/v3/DTDL.v3.md | 1 + ...DTDL.FeatureExtension.mqtt.v1.context.json | 23 ++ ...TDL.FeatureExtension.mqtt.v1.Elements.json | 74 +++++++ ...DL.FeatureExtension.mqtt.v1.RDF-SHACL.json | 144 +++++++++++++ 6 files changed, 441 insertions(+) create mode 100644 DTDL/v3/DTDL.mqtt.v1.md create mode 100644 DTDL/v3/context/DTDL.FeatureExtension.mqtt.v1.context.json create mode 100644 DTDL/v3/metamodel/DTDL.FeatureExtension.mqtt.v1.Elements.json create mode 100644 DTDL/v3/metamodel/DTDL.FeatureExtension.mqtt.v1.RDF-SHACL.json diff --git a/DTDL/v3/DTDL.Extensions.md b/DTDL/v3/DTDL.Extensions.md index 4815fbd0..2cd224e6 100644 --- a/DTDL/v3/DTDL.Extensions.md +++ b/DTDL/v3/DTDL.Extensions.md @@ -94,6 +94,7 @@ The chart below lists the feature extensions that are currently available. | [Historization v1](./DTDL.historization.v1.md) | Record the historical sequence of values of a Property or Telemetry and the times at which values change. | [3](./DTDL.v3.md) | | [Annotation v1](./DTDL.annotation.v1.md) | Add custom metadata to a Property or a Telemetry. | [3](./DTDL.v3.md) | | [Overriding v1](./DTDL.overriding.v1.md) | Override a model property with an instance value. | [3](./DTDL.v3.md) | +| [MQTT v1](./DTDL.mqtt.v1.md) | Specify Interface properties to facilitate communication via the MQTT pub/sub protocol. | [3](./DTDL.v3.md) | ## Service and tool support of language extensions diff --git a/DTDL/v3/DTDL.mqtt.v1.md b/DTDL/v3/DTDL.mqtt.v1.md new file mode 100644 index 00000000..08364078 --- /dev/null +++ b/DTDL/v3/DTDL.mqtt.v1.md @@ -0,0 +1,198 @@ +# Mqtt extension + +**Version 1** + +**Usable in DTDL version 3** + +The DTDL Mqtt extension enables a model to specify properties for an Interface that are relevant to communication via the MQTT pub/sub protocol. +If a service supports the Mqtt extension, it recognizes and understands the Mqtt, Idempotent, and Indexed adjunct types and their defined properties if the Mqtt context is specified. + +## Mqtt context + +The context specifier for version 1 of the Mqtt extension is "dtmi:dtdl:extension:mqtt;1". + +## Mqtt adjunct type + +The Mqtt adjunct type can co-type an Interface in DTDL version 3. +The chart below lists the additional properties that may be part of an element that is co-typed Mqtt. + +| Property | Required | Data type | Description | +| --- | --- | --- | --- | +| `commandTopic` | optional | *string* | MQTT topic pattern on which a Command request is published. | +| `payloadFormat` | required | [PayloadFormat](#payloadformat) | The format to use when serializing an instance to an MQTT payload. | +| `telemetryTopic` | optional | *string* | MQTT topic pattern on which a Telemetry or a collection of Telemetries is published. | + +When an Interface in a model is co-typed Mqtt, values of the above properties indicate the serialization format and MQTT topic pattern used for any instance of Interface contents when conveyed via an MQTT message. + +See the documentation on [Topic Structure](https://github.com/microsoft/mqtt-patterns/blob/main/docs/specs/topic-structure.md) for the appropriate format of MQTT topic pattern strings. + +## Idempotent adjunct type + +The Idempotent adjunct type can co-type a Command in DTDL version 3. +There are no additional properties conferred on an element that has co-type Idempotent. + +When a Command in a model is co-typed Idempotent, a service that implements the Command is permitted to execute the Command multiple times for a single invocation of the Command. +In the absence of an Idempotent co-type, a service must ensure that each Command invocation results in no more than one execution of the Command, despite possible duplication of Command requests due to failure-induced or delay-induced retries. + +See the documentation on [idempotency and cacheability](https://github.com/microsoft/mqtt-patterns/blob/main/docs/specs/cache.md#idempotency-and-cacheability). + +## Cacheable adjunct type + +The Cacheable adjunct type can co-type a Command in DTDL version 3. + +The chart below lists the additional properties that may be part of an element that is co-typed Cacheable. + +| Property | Required | Data type | Description | +| --- | --- | --- | --- | +| `ttl` | required | *duration* | Maximum duration for which a response to a Command instance may be reused as a response to other Command instances. | + +When a Command in a model is co-typed Cacheable, a service that implements the Command is permitted to store the response value for a Command instance and subsequently to reuse the stored value as a response to another Command instance, as long as the Command request values for the two Command instances are identical, and as long as the specfied `ttl` (time to live) duration is not exceeded. + +See the documentation on [idempotency and cacheability](https://github.com/microsoft/mqtt-patterns/blob/main/docs/cache.md#idempotency-and-cacheability). + +## Indexed adjunct type + +The Indexed adjunct type can co-type an EnumValue, a Field, or a Telemetry in DTDL version 3. + +The chart below lists the additional properties that may be part of an element that is co-typed Indexed. + +| Property | Required | Data type | Limits | Description | +| --- | --- | --- | --- | --- | +| `index` | required | *integer* | must be >= 1; must be unique across all EnumValues, Fields, or Telemetries that are values of the same property | Index number to uniquely identify the serialized element within its parent container. | + +Some serialization formats require index values for string EnumValues, Fields, and Telemetries. +The following section indicates which `payloadFormat` values necessitate using the Index adjunct type. +If an element is co-typed Indexed but the specified `payloadFormat` does not use indexes, the `index` property values are ignored. + +## PayloadFormat + +The following table lists the designators that are allowed as values for the `payloadFormat` property of an element that is co-typed Mqtt. + +| Format designator | Serialization format | Indexing | +| --- | --- | --- | +| `avro` | [Apache AVRO](https://avro.apache.org/docs/) data serialization format | ignored | +| `cbor` | RFC 8949 Concise Binary Object Representation ([CBOR](https://cbor.io/)) data format | expected | +| `json` | ECMA-404 JavaScript Object Notation ([JSON](https://www.json.org/json-en.html)) data interchange syntax | ignored | +| `proto2` | Google [Protocol Buffers](https://protobuf.dev/) data interchange format, [version 2](https://protobuf.dev/programming-guides/proto2/) | expected | +| `proto3` | Google [Protocol Buffers](https://protobuf.dev/) data interchange format, [version 3](https://protobuf.dev/programming-guides/proto3/) | expected | +| `raw` | unserialized raw bytes | ignored | + +Some implementations may not support all of the above payload formats, and/or they may restrict which formats can be used for which types of contents. + +## Mqtt examples + +The following example shows an Interface with four `contents` elements, two Telemetries and two Commands. +The Interface is co-typed Mqtt, and the `payloadFormat` is specified as JSON. + +The "getSpeed" Command is co-typed Idempotent, so a single invocation of the Command might result in multiple executions due to message duplication in the network. +By contranst, the "setColor" Command is not co-typed Idempotent, so a single invocation must result in a single execution, despite any message duplication. + +```json +{ + "@context": [ + "dtmi:dtdl:context;3", + "dtmi:dtdl:extension:mqtt;1" + ], + "@id": "dtmi:example:TestVehicle;1", + "@type": [ "Interface", "Mqtt" ], + "telemetryTopic": "vehicles/{modelId}/{senderId}/telemetry", + "commandTopic": "vehicles/{executorId}/command/{commandName}", + "payloadFormat": "json", + "contents": [ + { + "@type": "Telemetry", + "name": "distance", + "schema": "double", + "description": "The total distance from the origin." + }, + { + "@type": "Telemetry", + "name": "color", + "schema": "string", + "description": "The color currently being applied." + }, + { + "@type": [ "Command", "Idempotent" ], + "name": "getSpeed", + "response": { + "name": "mph", + "schema": "integer" + } + }, + { + "@type": "Command", + "name": "setColor", + "request": { + "name": "newColor", + "schema": "string" + }, + "response": { + "name": "oldColor", + "schema": "string" + } + } + ] +} +``` + +The following example is almost identical to the previous one, except the `payloadFormat` is specified as proto3 instead of JSON. +Because protobuf requires indexed fields, each Telemetry element is co-typed Indexed, and it specifies a unique (within the Interface) positive integer value for the `index` property. + +```json +{ + "@context": [ + "dtmi:dtdl:context;3", + "dtmi:dtdl:extension:mqtt;1" + ], + "@id": "dtmi:example:TestVehicle;1", + "@type": [ "Interface", "Mqtt" ], + "telemetryTopic": "vehicles/{modelId}/{senderId}/telemetry", + "commandTopic": "vehicles/{executorId}/command/{commandName}", + "payloadFormat": "proto3", + "contents": [ + { + "@type": [ "Telemetry", "Indexed" ], + "name": "distance", + "schema": "double", + "description": "The total distance from the origin.", + "index": 1 + }, + { + "@type": [ "Telemetry", "Indexed" ], + "name": "color", + "schema": "string", + "description": "The color currently being applied.", + "index": 2 + }, + { + "@type": [ "Command", "Idempotent" ], + "name": "getSpeed", + "response": { + "name": "mph", + "schema": "integer" + } + }, + { + "@type": "Command", + "name": "setColor", + "request": { + "name": "newColor", + "schema": "string" + }, + "response": { + "name": "oldColor", + "schema": "string" + } + } + ] +} +``` + +## Feature versions + +The chart below lists the versions of the Annotation extension that are currently available. + +| Extension | Context | DTDL versions | +| --- | --- | --- | +| [MQTT v1](./DTDL.mqtt.v1.md) | dtmi:dtdl:extension:mqtt;1 | [3](./DTDL.v3.md) | + diff --git a/DTDL/v3/DTDL.v3.md b/DTDL/v3/DTDL.v3.md index ccfb6769..9496dcf2 100644 --- a/DTDL/v3/DTDL.v3.md +++ b/DTDL/v3/DTDL.v3.md @@ -990,6 +990,7 @@ The chart below lists the language extensions that are currently available for u | [Historization v1](./DTDL.historization.v1.md) | feature | Record the historical sequence of values of a Property or Telemetry and the times at which values change. | | [Annotation v1](./DTDL.annotation.v1.md) | feature | Add custom metadata to a Property or a Telemetry. | | [Overriding v1](./DTDL.overriding.v1.md) | feature | Override a model property with an instance value. | +| [MQTT v1](./DTDL.mqtt.v1.md) | feature | Specify Interface properties to facilitate communication via the MQTT pub/sub protocol. | ## Changes from Version 2 diff --git a/DTDL/v3/context/DTDL.FeatureExtension.mqtt.v1.context.json b/DTDL/v3/context/DTDL.FeatureExtension.mqtt.v1.context.json new file mode 100644 index 00000000..bd99c918 --- /dev/null +++ b/DTDL/v3/context/DTDL.FeatureExtension.mqtt.v1.context.json @@ -0,0 +1,23 @@ +{ + "Mqtt": { "@id": "dtmi:dtdl:extension:mqtt:v1:Mqtt" }, + "Idempotent": { "@id": "dtmi:dtdl:extension:mqtt:v1:Idempotent" }, + "Cacheable": { "@id": "dtmi:dtdl:extension:mqtt:v1:Cacheable" }, + "Indexed": { "@id": "dtmi:dtdl:extension:mqtt:v1:Indexed" }, + + "PayloadFormat": { "@id": "dtmi:dtdl:extension:mqtt:v1:PayloadFormat" }, + + "telemetryTopic": { "@id": "dtmi:dtdl:extension:mqtt:v1:Mqtt:telemetryTopic" }, + "commandTopic": { "@id": "dtmi:dtdl:extension:mqtt:v1:Mqtt:commandTopic" }, + "payloadFormat": { "@id": "dtmi:dtdl:extension:mqtt:v1:Mqtt:payloadFormat" }, + + "ttl": { "@id": "dtmi:dtdl:extension:mqtt:v1:Cacheable:ttl" }, + + "index": { "@id": "dtmi:dtdl:extension:mqtt:v1:Indexed:index" }, + + "avro": { "@id": "dtmi:dtdl:extension:mqtt:v1:payloadFormat:avro" }, + "cbor": { "@id": "dtmi:dtdl:extension:mqtt:v1:payloadFormat:cbor" }, + "json": { "@id": "dtmi:dtdl:extension:mqtt:v1:payloadFormat:json" }, + "proto2": { "@id": "dtmi:dtdl:extension:mqtt:v1:payloadFormat:proto2" }, + "proto3": { "@id": "dtmi:dtdl:extension:mqtt:v1:payloadFormat:proto3" }, + "raw": { "@id": "dtmi:dtdl:extension:mqtt:v1:payloadFormat:raw" } +} diff --git a/DTDL/v3/metamodel/DTDL.FeatureExtension.mqtt.v1.Elements.json b/DTDL/v3/metamodel/DTDL.FeatureExtension.mqtt.v1.Elements.json new file mode 100644 index 00000000..3b37c783 --- /dev/null +++ b/DTDL/v3/metamodel/DTDL.FeatureExtension.mqtt.v1.Elements.json @@ -0,0 +1,74 @@ +[ + { + "@context": [ + "dtmi:dtdl:context;3", + "dtmi:dtdl:extension:mqtt;1" + ], + "@id": "dtmi:dtdl:extension:mqtt:v1:payloadFormat:avro", + "@type": "PayloadFormat", + "displayName": "AVRO", + "description": { + "en-US": "[Apache AVRO](https://avro.apache.org/docs/) data serialization format" + } + }, + { + "@context": [ + "dtmi:dtdl:context;3", + "dtmi:dtdl:extension:mqtt;1" + ], + "@id": "dtmi:dtdl:extension:mqtt:v1:payloadFormat:cbor", + "@type": "PayloadFormat", + "displayName": "CBOR", + "description": { + "en-US": "RFC 8949 Concise Binary Object Representation ([CBOR](https://cbor.io/)) data format" + } + }, + { + "@context": [ + "dtmi:dtdl:context;3", + "dtmi:dtdl:extension:mqtt;1" + ], + "@id": "dtmi:dtdl:extension:mqtt:v1:payloadFormat:json", + "@type": "PayloadFormat", + "displayName": "JSON", + "description": { + "en-US": "ECMA-404 JavaScript Object Notation ([JSON](https://www.json.org/json-en.html)) data interchange syntax" + } + }, + { + "@context": [ + "dtmi:dtdl:context;3", + "dtmi:dtdl:extension:mqtt;1" + ], + "@id": "dtmi:dtdl:extension:mqtt:v1:payloadFormat:proto2", + "@type": "PayloadFormat", + "displayName": "Protobuf v2", + "description": { + "en-US": "Google [Protocol Buffers](https://protobuf.dev/) data interchange format, [version 2](https://protobuf.dev/programming-guides/proto2/)" + } + }, + { + "@context": [ + "dtmi:dtdl:context;3", + "dtmi:dtdl:extension:mqtt;1" + ], + "@id": "dtmi:dtdl:extension:mqtt:v1:payloadFormat:proto3", + "@type": "PayloadFormat", + "displayName": "Protobuf v3", + "description": { + "en-US": "Google [Protocol Buffers](https://protobuf.dev/) data interchange format, [version 3](https://protobuf.dev/programming-guides/proto3/)" + } + }, + { + "@context": [ + "dtmi:dtdl:context;3", + "dtmi:dtdl:extension:mqtt;1" + ], + "@id": "dtmi:dtdl:extension:mqtt:v1:payloadFormat:raw", + "@type": "PayloadFormat", + "displayName": "raw", + "description": { + "en-US": "unserialized raw bytes" + } + } +] diff --git a/DTDL/v3/metamodel/DTDL.FeatureExtension.mqtt.v1.RDF-SHACL.json b/DTDL/v3/metamodel/DTDL.FeatureExtension.mqtt.v1.RDF-SHACL.json new file mode 100644 index 00000000..cd4279b2 --- /dev/null +++ b/DTDL/v3/metamodel/DTDL.FeatureExtension.mqtt.v1.RDF-SHACL.json @@ -0,0 +1,144 @@ +{ + "@context": [ + "dtmi:dtdl:context;3", + "dtmi:dtdl:extension:mqtt;1" + ], + "@graph": [ + { + "@id": "dtmi:dtdl:extension:mqtt:v1:documentation", + "@type": [ "dtmm:Documentation" ], + "dtmm:description": { + "en-US": "Specify Interface properties to facilitate communication via the MQTT pub/sub protocol." + } + }, + { + "@id": "dtmi:dtdl:extension:mqtt:v1:Mqtt", + "@type": [ "rdfs:Class", "sh:NodeShape" ], + "rdfs:subClassOf": "AdjunctType", + "sh:or": [ + { "sh:class": "Interface" } + ], + "dtmm:property": [ + { + "@id": "dtmi:dtdl:extension:mqtt:v1:Mqtt:telemetryTopic", + "@type": "rdf:Property", + "rdfs:domain": "Mqtt" + }, + { + "@id": "dtmi:dtdl:extension:mqtt:v1:Mqtt:commandTopic", + "@type": "rdf:Property", + "rdfs:domain": "Mqtt" + }, + { + "@id": "dtmi:dtdl:extension:mqtt:v1:Mqtt:payloadFormat", + "@type": "rdf:Property", + "rdfs:domain": "Mqtt" + } + ], + "sh:property": [ + { + "sh:path": "telemetryTopic", + "sh:maxCount": 1, + "sh:datatype": "xsd:string", + "sh:nodeKind": "sh:Literal", + "dtmm:description": { + "en-US": "MQTT topic pattern on which a Telemetry or a collection of Telemetries is published." + } + }, + { + "sh:path": "commandTopic", + "sh:maxCount": 1, + "sh:datatype": "xsd:string", + "sh:nodeKind": "sh:Literal", + "dtmm:description": { + "en-US": "MQTT topic pattern on which a Command request is published." + } + }, + { + "sh:path": "payloadFormat", + "sh:minCount": 1, + "sh:maxCount": 1, + "sh:class": "PayloadFormat", + "sh:nodeKind": "sh:IRI", + "dtmm:description": { + "en-US": "The format to use when serializing an instance to an MQTT payload." + } + } + ] + }, + { + "@id": "dtmi:dtdl:extension:mqtt:v1:PayloadFormat", + "@type": [ "rdfs:Class", "sh:NodeShape" ], + "rdfs:subClassOf": "LatentType" + }, + { + "@id": "dtmi:dtdl:extension:mqtt:v1:Idempotent", + "@type": [ "rdfs:Class", "sh:NodeShape" ], + "rdfs:subClassOf": "AdjunctType", + "sh:or": [ + { "sh:class": "Command" } + ] + }, + { + "@id": "dtmi:dtdl:extension:mqtt:v1:Cacheable", + "@type": [ "rdfs:Class", "sh:NodeShape" ], + "rdfs:subClassOf": "AdjunctType", + "sh:or": [ + { "sh:class": "Command" } + ], + "sh:and": [ + { "sh:class": "Idempotent" } + ], + "dtmm:property": [ + { + "@id": "dtmi:dtdl:extension:mqtt:v1:Cacheable:ttl", + "@type": "rdf:Property", + "rdfs:domain": "Cacheable" + } + ], + "sh:property": [ + { + "sh:path": "ttl", + "sh:minCount": 1, + "sh:maxCount": 1, + "sh:datatype": "xsd:duration", + "sh:nodeKind": "sh:Literal", + "dtmm:description": { + "en-US": "Maximum duration for which a response to a Command instance may be reused as a response to other Command instances." + } + } + ] + }, + { + "@id": "dtmi:dtdl:extension:mqtt:v1:Indexed", + "@type": [ "rdfs:Class", "sh:NodeShape" ], + "rdfs:subClassOf": "AdjunctType", + "sh:or": [ + { "sh:class": "Telemetry" }, + { "sh:class": "Field" }, + { "sh:class": "EnumValue" } + ], + "dtmm:property": [ + { + "@id": "dtmi:dtdl:extension:mqtt:v1:Indexed:index", + "@type": "rdf:Property", + "rdfs:domain": "Indexed" + } + ], + "sh:property": [ + { + "sh:path": "index", + "sh:minCount": 1, + "sh:maxCount": 1, + "sh:datatype": "xsd:integer", + "sh:nodeKind": "sh:Literal", + "sh:minInclusive": 1, + "dtmm:uniqueValue": true, + "dtmm:description": { + "en-US": "Index number to uniquely identify the serialized element within its parent container." + } + } + ] + } + ] +} From 363a366c729fedafb7385340990d58903aa0fadb Mon Sep 17 00:00:00 2001 From: Auto Gen Date: Wed, 21 Feb 2024 14:01:05 -0800 Subject: [PATCH 2/3] update per review of MQTT extension --- DTDL/v3/DTDL.mqtt.v1.md | 105 ++++-------------- ...DTDL.FeatureExtension.mqtt.v1.context.json | 11 +- ...TDL.FeatureExtension.mqtt.v1.Elements.json | 74 ------------ ...DL.FeatureExtension.mqtt.v1.RDF-SHACL.json | 17 +-- 4 files changed, 32 insertions(+), 175 deletions(-) delete mode 100644 DTDL/v3/metamodel/DTDL.FeatureExtension.mqtt.v1.Elements.json diff --git a/DTDL/v3/DTDL.mqtt.v1.md b/DTDL/v3/DTDL.mqtt.v1.md index 08364078..1219ce8e 100644 --- a/DTDL/v3/DTDL.mqtt.v1.md +++ b/DTDL/v3/DTDL.mqtt.v1.md @@ -5,7 +5,7 @@ **Usable in DTDL version 3** The DTDL Mqtt extension enables a model to specify properties for an Interface that are relevant to communication via the MQTT pub/sub protocol. -If a service supports the Mqtt extension, it recognizes and understands the Mqtt, Idempotent, and Indexed adjunct types and their defined properties if the Mqtt context is specified. +If a service supports the Mqtt extension, it recognizes and understands the Mqtt, Idempotent, Cacheable, and Indexed adjunct types and their defined properties if the Mqtt context is specified. ## Mqtt context @@ -16,15 +16,27 @@ The context specifier for version 1 of the Mqtt extension is "dtmi:dtdl:extensio The Mqtt adjunct type can co-type an Interface in DTDL version 3. The chart below lists the additional properties that may be part of an element that is co-typed Mqtt. -| Property | Required | Data type | Description | -| --- | --- | --- | --- | -| `commandTopic` | optional | *string* | MQTT topic pattern on which a Command request is published. | -| `payloadFormat` | required | [PayloadFormat](#payloadformat) | The format to use when serializing an instance to an MQTT payload. | -| `telemetryTopic` | optional | *string* | MQTT topic pattern on which a Telemetry or a collection of Telemetries is published. | +| Property | Required | Data type | Limits | Description | +| --- | --- | --- | --- | --- | +| `commandTopic` | optional | *string* | slash-separated sequence of character-restricted labels and/or brace-enclosed tokens | MQTT topic pattern on which a Command request is published. | +| `payloadFormat` | required | *string* | | The format to use when serializing an instance to an MQTT payload. | +| `telemetryTopic` | optional | *string* | slash-separated sequence of character-restricted labels and/or brace-enclosed tokens | MQTT topic pattern on which a Telemetry or a collection of Telemetries is published. | When an Interface in a model is co-typed Mqtt, values of the above properties indicate the serialization format and MQTT topic pattern used for any instance of Interface contents when conveyed via an MQTT message. -See the documentation on [Topic Structure](https://github.com/microsoft/mqtt-patterns/blob/main/docs/specs/topic-structure.md) for the appropriate format of MQTT topic pattern strings. +### Topic Pattern + +MQTT topic pattern strings are restricted as follows. + +* A topic pattern is a sequence of labels separated by `/` +* Each label is one of: + * A string of printable ASCII characters not including space, `"`, `+`, `#`, `{`, `}`, or `/`, to be copied literally into the MQTT topic. + * A string that indicates a replaceable token, to be replaced by an implementation-dependent value: + * The string starts with `{`, ends with `}`, and contains an implementation-defined token + * The token is a non-empty sequence of ASCII letters, optionally preceded by a prefix + * The prefix is a non-empty sequence of ASCII letters followed by a `:` + +* The first label must not start with `$` ## Idempotent adjunct type @@ -34,8 +46,6 @@ There are no additional properties conferred on an element that has co-type Idem When a Command in a model is co-typed Idempotent, a service that implements the Command is permitted to execute the Command multiple times for a single invocation of the Command. In the absence of an Idempotent co-type, a service must ensure that each Command invocation results in no more than one execution of the Command, despite possible duplication of Command requests due to failure-induced or delay-induced retries. -See the documentation on [idempotency and cacheability](https://github.com/microsoft/mqtt-patterns/blob/main/docs/specs/cache.md#idempotency-and-cacheability). - ## Cacheable adjunct type The Cacheable adjunct type can co-type a Command in DTDL version 3. @@ -48,8 +58,6 @@ The chart below lists the additional properties that may be part of an element t When a Command in a model is co-typed Cacheable, a service that implements the Command is permitted to store the response value for a Command instance and subsequently to reuse the stored value as a response to another Command instance, as long as the Command request values for the two Command instances are identical, and as long as the specfied `ttl` (time to live) duration is not exceeded. -See the documentation on [idempotency and cacheability](https://github.com/microsoft/mqtt-patterns/blob/main/docs/cache.md#idempotency-and-cacheability). - ## Indexed adjunct type The Indexed adjunct type can co-type an EnumValue, a Field, or a Telemetry in DTDL version 3. @@ -61,29 +69,11 @@ The chart below lists the additional properties that may be part of an element t | `index` | required | *integer* | must be >= 1; must be unique across all EnumValues, Fields, or Telemetries that are values of the same property | Index number to uniquely identify the serialized element within its parent container. | Some serialization formats require index values for string EnumValues, Fields, and Telemetries. -The following section indicates which `payloadFormat` values necessitate using the Index adjunct type. -If an element is co-typed Indexed but the specified `payloadFormat` does not use indexes, the `index` property values are ignored. - -## PayloadFormat - -The following table lists the designators that are allowed as values for the `payloadFormat` property of an element that is co-typed Mqtt. - -| Format designator | Serialization format | Indexing | -| --- | --- | --- | -| `avro` | [Apache AVRO](https://avro.apache.org/docs/) data serialization format | ignored | -| `cbor` | RFC 8949 Concise Binary Object Representation ([CBOR](https://cbor.io/)) data format | expected | -| `json` | ECMA-404 JavaScript Object Notation ([JSON](https://www.json.org/json-en.html)) data interchange syntax | ignored | -| `proto2` | Google [Protocol Buffers](https://protobuf.dev/) data interchange format, [version 2](https://protobuf.dev/programming-guides/proto2/) | expected | -| `proto3` | Google [Protocol Buffers](https://protobuf.dev/) data interchange format, [version 3](https://protobuf.dev/programming-guides/proto3/) | expected | -| `raw` | unserialized raw bytes | ignored | - -Some implementations may not support all of the above payload formats, and/or they may restrict which formats can be used for which types of contents. +Although index values can be generated automatically, the Indexed adjunct type is available for setting explicit index values when needed for cross-version compatibility or interoperation across different implementations. ## Mqtt examples The following example shows an Interface with four `contents` elements, two Telemetries and two Commands. -The Interface is co-typed Mqtt, and the `payloadFormat` is specified as JSON. - The "getSpeed" Command is co-typed Idempotent, so a single invocation of the Command might result in multiple executions due to message duplication in the network. By contranst, the "setColor" Command is not co-typed Idempotent, so a single invocation must result in a single execution, despite any message duplication. @@ -97,7 +87,7 @@ By contranst, the "setColor" Command is not co-typed Idempotent, so a single inv "@type": [ "Interface", "Mqtt" ], "telemetryTopic": "vehicles/{modelId}/{senderId}/telemetry", "commandTopic": "vehicles/{executorId}/command/{commandName}", - "payloadFormat": "json", + "payloadFormat": "Avro/1.11.0", "contents": [ { "@type": "Telemetry", @@ -135,59 +125,6 @@ By contranst, the "setColor" Command is not co-typed Idempotent, so a single inv } ``` -The following example is almost identical to the previous one, except the `payloadFormat` is specified as proto3 instead of JSON. -Because protobuf requires indexed fields, each Telemetry element is co-typed Indexed, and it specifies a unique (within the Interface) positive integer value for the `index` property. - -```json -{ - "@context": [ - "dtmi:dtdl:context;3", - "dtmi:dtdl:extension:mqtt;1" - ], - "@id": "dtmi:example:TestVehicle;1", - "@type": [ "Interface", "Mqtt" ], - "telemetryTopic": "vehicles/{modelId}/{senderId}/telemetry", - "commandTopic": "vehicles/{executorId}/command/{commandName}", - "payloadFormat": "proto3", - "contents": [ - { - "@type": [ "Telemetry", "Indexed" ], - "name": "distance", - "schema": "double", - "description": "The total distance from the origin.", - "index": 1 - }, - { - "@type": [ "Telemetry", "Indexed" ], - "name": "color", - "schema": "string", - "description": "The color currently being applied.", - "index": 2 - }, - { - "@type": [ "Command", "Idempotent" ], - "name": "getSpeed", - "response": { - "name": "mph", - "schema": "integer" - } - }, - { - "@type": "Command", - "name": "setColor", - "request": { - "name": "newColor", - "schema": "string" - }, - "response": { - "name": "oldColor", - "schema": "string" - } - } - ] -} -``` - ## Feature versions The chart below lists the versions of the Annotation extension that are currently available. diff --git a/DTDL/v3/context/DTDL.FeatureExtension.mqtt.v1.context.json b/DTDL/v3/context/DTDL.FeatureExtension.mqtt.v1.context.json index bd99c918..1911d381 100644 --- a/DTDL/v3/context/DTDL.FeatureExtension.mqtt.v1.context.json +++ b/DTDL/v3/context/DTDL.FeatureExtension.mqtt.v1.context.json @@ -4,20 +4,11 @@ "Cacheable": { "@id": "dtmi:dtdl:extension:mqtt:v1:Cacheable" }, "Indexed": { "@id": "dtmi:dtdl:extension:mqtt:v1:Indexed" }, - "PayloadFormat": { "@id": "dtmi:dtdl:extension:mqtt:v1:PayloadFormat" }, - "telemetryTopic": { "@id": "dtmi:dtdl:extension:mqtt:v1:Mqtt:telemetryTopic" }, "commandTopic": { "@id": "dtmi:dtdl:extension:mqtt:v1:Mqtt:commandTopic" }, "payloadFormat": { "@id": "dtmi:dtdl:extension:mqtt:v1:Mqtt:payloadFormat" }, "ttl": { "@id": "dtmi:dtdl:extension:mqtt:v1:Cacheable:ttl" }, - "index": { "@id": "dtmi:dtdl:extension:mqtt:v1:Indexed:index" }, - - "avro": { "@id": "dtmi:dtdl:extension:mqtt:v1:payloadFormat:avro" }, - "cbor": { "@id": "dtmi:dtdl:extension:mqtt:v1:payloadFormat:cbor" }, - "json": { "@id": "dtmi:dtdl:extension:mqtt:v1:payloadFormat:json" }, - "proto2": { "@id": "dtmi:dtdl:extension:mqtt:v1:payloadFormat:proto2" }, - "proto3": { "@id": "dtmi:dtdl:extension:mqtt:v1:payloadFormat:proto3" }, - "raw": { "@id": "dtmi:dtdl:extension:mqtt:v1:payloadFormat:raw" } + "index": { "@id": "dtmi:dtdl:extension:mqtt:v1:Indexed:index" } } diff --git a/DTDL/v3/metamodel/DTDL.FeatureExtension.mqtt.v1.Elements.json b/DTDL/v3/metamodel/DTDL.FeatureExtension.mqtt.v1.Elements.json deleted file mode 100644 index 3b37c783..00000000 --- a/DTDL/v3/metamodel/DTDL.FeatureExtension.mqtt.v1.Elements.json +++ /dev/null @@ -1,74 +0,0 @@ -[ - { - "@context": [ - "dtmi:dtdl:context;3", - "dtmi:dtdl:extension:mqtt;1" - ], - "@id": "dtmi:dtdl:extension:mqtt:v1:payloadFormat:avro", - "@type": "PayloadFormat", - "displayName": "AVRO", - "description": { - "en-US": "[Apache AVRO](https://avro.apache.org/docs/) data serialization format" - } - }, - { - "@context": [ - "dtmi:dtdl:context;3", - "dtmi:dtdl:extension:mqtt;1" - ], - "@id": "dtmi:dtdl:extension:mqtt:v1:payloadFormat:cbor", - "@type": "PayloadFormat", - "displayName": "CBOR", - "description": { - "en-US": "RFC 8949 Concise Binary Object Representation ([CBOR](https://cbor.io/)) data format" - } - }, - { - "@context": [ - "dtmi:dtdl:context;3", - "dtmi:dtdl:extension:mqtt;1" - ], - "@id": "dtmi:dtdl:extension:mqtt:v1:payloadFormat:json", - "@type": "PayloadFormat", - "displayName": "JSON", - "description": { - "en-US": "ECMA-404 JavaScript Object Notation ([JSON](https://www.json.org/json-en.html)) data interchange syntax" - } - }, - { - "@context": [ - "dtmi:dtdl:context;3", - "dtmi:dtdl:extension:mqtt;1" - ], - "@id": "dtmi:dtdl:extension:mqtt:v1:payloadFormat:proto2", - "@type": "PayloadFormat", - "displayName": "Protobuf v2", - "description": { - "en-US": "Google [Protocol Buffers](https://protobuf.dev/) data interchange format, [version 2](https://protobuf.dev/programming-guides/proto2/)" - } - }, - { - "@context": [ - "dtmi:dtdl:context;3", - "dtmi:dtdl:extension:mqtt;1" - ], - "@id": "dtmi:dtdl:extension:mqtt:v1:payloadFormat:proto3", - "@type": "PayloadFormat", - "displayName": "Protobuf v3", - "description": { - "en-US": "Google [Protocol Buffers](https://protobuf.dev/) data interchange format, [version 3](https://protobuf.dev/programming-guides/proto3/)" - } - }, - { - "@context": [ - "dtmi:dtdl:context;3", - "dtmi:dtdl:extension:mqtt;1" - ], - "@id": "dtmi:dtdl:extension:mqtt:v1:payloadFormat:raw", - "@type": "PayloadFormat", - "displayName": "raw", - "description": { - "en-US": "unserialized raw bytes" - } - } -] diff --git a/DTDL/v3/metamodel/DTDL.FeatureExtension.mqtt.v1.RDF-SHACL.json b/DTDL/v3/metamodel/DTDL.FeatureExtension.mqtt.v1.RDF-SHACL.json index cd4279b2..4b892ce0 100644 --- a/DTDL/v3/metamodel/DTDL.FeatureExtension.mqtt.v1.RDF-SHACL.json +++ b/DTDL/v3/metamodel/DTDL.FeatureExtension.mqtt.v1.RDF-SHACL.json @@ -41,6 +41,10 @@ "sh:maxCount": 1, "sh:datatype": "xsd:string", "sh:nodeKind": "sh:Literal", + "sh:pattern": "^(?:(?:[!%-*,-.0-z|~][!$-*,-.0-z|~]*)|(?:{(?:[A-Za-z]+:)?[A-Za-z]+}))(?:\\/(?:(?:[!$-*,-.0-z|~]+)|(?:{(?:[A-Za-z]+:)?[A-Za-z]+})))*$", + "dtmm:patternDesc": { + "en-US": "slash-separated sequence of character-restricted labels and/or brace-enclosed tokens" + }, "dtmm:description": { "en-US": "MQTT topic pattern on which a Telemetry or a collection of Telemetries is published." } @@ -50,6 +54,10 @@ "sh:maxCount": 1, "sh:datatype": "xsd:string", "sh:nodeKind": "sh:Literal", + "sh:pattern": "^(?:(?:[!%-*,-.0-z|~][!$-*,-.0-z|~]*)|(?:{(?:[A-Za-z]+:)?[A-Za-z]+}))(?:\\/(?:(?:[!$-*,-.0-z|~]+)|(?:{(?:[A-Za-z]+:)?[A-Za-z]+})))*$", + "dtmm:patternDesc": { + "en-US": "slash-separated sequence of character-restricted labels and/or brace-enclosed tokens" + }, "dtmm:description": { "en-US": "MQTT topic pattern on which a Command request is published." } @@ -58,19 +66,14 @@ "sh:path": "payloadFormat", "sh:minCount": 1, "sh:maxCount": 1, - "sh:class": "PayloadFormat", - "sh:nodeKind": "sh:IRI", + "sh:datatype": "xsd:string", + "sh:nodeKind": "sh:Literal", "dtmm:description": { "en-US": "The format to use when serializing an instance to an MQTT payload." } } ] }, - { - "@id": "dtmi:dtdl:extension:mqtt:v1:PayloadFormat", - "@type": [ "rdfs:Class", "sh:NodeShape" ], - "rdfs:subClassOf": "LatentType" - }, { "@id": "dtmi:dtdl:extension:mqtt:v1:Idempotent", "@type": [ "rdfs:Class", "sh:NodeShape" ], From 4e078a25bbe3919154b455de334c42f9e14dac5f Mon Sep 17 00:00:00 2001 From: Auto Gen Date: Thu, 22 Feb 2024 08:59:29 -0800 Subject: [PATCH 3/3] enhance MQTT doc examples --- DTDL/v3/DTDL.mqtt.v1.md | 65 ++++++++++++++++++++++++++++++++++++++++- 1 file changed, 64 insertions(+), 1 deletion(-) diff --git a/DTDL/v3/DTDL.mqtt.v1.md b/DTDL/v3/DTDL.mqtt.v1.md index 1219ce8e..f99c288d 100644 --- a/DTDL/v3/DTDL.mqtt.v1.md +++ b/DTDL/v3/DTDL.mqtt.v1.md @@ -24,6 +24,9 @@ The chart below lists the additional properties that may be part of an element t When an Interface in a model is co-typed Mqtt, values of the above properties indicate the serialization format and MQTT topic pattern used for any instance of Interface contents when conveyed via an MQTT message. +Note that the co-types on an Interface, and the additional properties added by these co-types, are not imported by an Interface that `extends` the Interface. +Therefore, the properties above apply only to the Interface to which they are directly applied, not to any extending Interfaces. + ### Topic Pattern MQTT topic pattern strings are restricted as follows. @@ -76,6 +79,7 @@ Although index values can be generated automatically, the Indexed adjunct type i The following example shows an Interface with four `contents` elements, two Telemetries and two Commands. The "getSpeed" Command is co-typed Idempotent, so a single invocation of the Command might result in multiple executions due to message duplication in the network. By contranst, the "setColor" Command is not co-typed Idempotent, so a single invocation must result in a single execution, despite any message duplication. +The "getSpeed" Command is also co-typed Cacheable and has a "ttl" property with value "PT15S", which is a duration of 15 seconds expressed in [ISO 8601 Duration](https://en.wikipedia.org/wiki/ISO_8601#Durations) format. ```json { @@ -102,12 +106,71 @@ By contranst, the "setColor" Command is not co-typed Idempotent, so a single inv "description": "The color currently being applied." }, { - "@type": [ "Command", "Idempotent" ], + "@type": [ "Command", "Idempotent", "Cacheable" ], "name": "getSpeed", "response": { "name": "mph", "schema": "integer" + }, + "ttl": "PT15S" + }, + { + "@type": "Command", + "name": "setColor", + "request": { + "name": "newColor", + "schema": "string" + }, + "response": { + "name": "oldColor", + "schema": "string" } + } + ] +} +``` + +The following example is identical to the previous one except for two changes. +First, the `payloadFormat` is specified as "Protobuf/3" instead of "Avro/1.11.0". +Second, each Telemetry element is co-typed Indexed, and it specifies a unique (within the Interface) positive integer value for the `index` property. + +The Protobuf format uses field indices instead of names in its on-wire representation. +These indices can be generated automatically, but the example illustrates how the Indexed adjunct type can be used to set explicit index values if desired. + +```json +{ + "@context": [ + "dtmi:dtdl:context;3", + "dtmi:dtdl:extension:mqtt;1" + ], + "@id": "dtmi:example:TestVehicle;1", + "@type": [ "Interface", "Mqtt" ], + "telemetryTopic": "vehicles/{modelId}/{senderId}/telemetry", + "commandTopic": "vehicles/{executorId}/command/{commandName}", + "payloadFormat": "Protobuf/3", + "contents": [ + { + "@type": [ "Telemetry", "Indexed" ], + "name": "distance", + "schema": "double", + "description": "The total distance from the origin.", + "index": 3 + }, + { + "@type": [ "Telemetry", "Indexed" ], + "name": "color", + "schema": "string", + "description": "The color currently being applied.", + "index": 2 + }, + { + "@type": [ "Command", "Idempotent", "Cacheable" ], + "name": "getSpeed", + "response": { + "name": "mph", + "schema": "integer" + }, + "ttl": "PT15S" }, { "@type": "Command",