Error in user YAML: (<unknown>): did not find expected key while parsing a block mapping at line 1 column 1
---
theme: dracula
style: |
section {
background-color: #ddd;
color-background-code: #ccc;
color-background-paginate: rgba(128, 128, 128, 0.05);
color-foreground: #345;
color-highlight: #99c;
color-highlight-hover: #aaf;
color-highlight-heading: #99c;
color-header: #bbb;
color-header-shadow: transparent;
}
paginate: true
title: CoAP & LwM2m 101
description: An introduction into CoAP and how it correlates to our LwM2m
author: Raffael Wojtas
keywords: coap,lwm2m
---
Answer questions like:
- What is coap?
- How do observations work?
- What is a piggybacked message?
- Blockwise transfer in action
- see wireshark 🦈
- CoAP overview
- CoAP messaging model
- CoAP Observations
- CoAP Blockwise transfers 🦈
- LwM2m overview
- LwM2m operations
- LwM2m operations in action
<style scoped> pre { font-size: 0.8em; } </style>CoAP, or Constrained Application Protocol, is a specialized web transfer protocol designed for resource-constrained devices and networks in the context of the Internet of Things (IoT).
+----------------------+
| Application |
+----------------------+
+----------------------+ \
| Requests/Responses | |
|----------------------| | CoAP
| Messages | |
+----------------------+ /
+----------------------+
| UDP |
+----------------------+
- Simple & easy to understand
- HTTP convertable
- Constrained devices oriented
- A full redesign of HTTP with new features, not just a compressed / ‘dumped down’ version
- Binary protocol for different content types
- Runs on top of UDP (as well as TCP, SMS, ...)
- Native support for push notifications (with best effort semantics) in addition to traditional request / response
- Other features: discovery, multicast, HTTP mapping, DTLS security binding
- Requests and responses, mirroring the structure of HTTP
- Requests include methods such as GET, POST, PUT, and DELETE defining the action to be performed on a resource
- Responses include status codes
- URI structure:
coap-URI = "coap:" "//" host [ ":" port ] path-abempty [ "?" query ]
- adjusted to constraind, power limited devices (it's binary)
- reduction of header overheader by using UDP
- built-in support for observing resources
- both sides are client and server
<style scoped> p { font-size: 0.6em; } </style>
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Ver| T | TKL | Code | Message ID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Token (if any, TKL bytes) ...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Options (if any) ...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|1 1 1 1 1 1 1 1| Payload (if any) ...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
T: Indicates if this message is of type Confirmable (0), Non-confirmable (1), Acknowledgement (2), or Reset (3). The semantics of these message types are defined in Section 4.
- Confirmable
- Non-confirmable
- Acknowledgement
- Reset message
- indicates that a specific message could not be processed
- i.e. empty confirmable
- Piggybacked Response
- Separate Response
- Empty
Client Server
| |
| CON [0x7d34] |
+----------------->|
| |
| ACK [0x7d34] |
|<-----------------+
| |
Figure 2: Reliable Message Transmission
Client Server
| |
| NON [0x01a0] |
+----------------->|
| |
Figure 3: Unreliable Message Transmission
<style scoped> pre { font-size: 1em; } p { font-size: 0.5em; } </style>
Client Server Client Server
| | | |
| CON [0xbc90] | | CON [0xbc91] |
| GET /temperature | | GET /temperature |
| (Token 0x71) | | (Token 0x72) |
+----------------->| +----------------->|
| | | |
| ACK [0xbc90] | | ACK [0xbc91] |
| 2.05 Content | | 4.04 Not Found |
| (Token 0x71) | | (Token 0x72) |
| "22.5 C" | | "Not found" |
|<-----------------+ |<-----------------+
| | | |
Figure 4: Two GET Requests with Piggybacked Responses
Piggybacked: In the most basic case, the response is carried directly in the Acknowledgement message that acknowledges the request (which requires that the request was carried in a Confirmable message). This is called a "Piggybacked Response".
<style scoped> pre { font-size: 0.7em; } </style>
Client Server
| |
| CON [0x7a10] |
| GET /temperature |
| (Token 0x73) |
+----------------->|
| |
| ACK [0x7a10] |
|<-----------------+
| |
... Time Passes ...
| |
| CON [0x23bb] |
| 2.05 Content |
| (Token 0x73) |
| "22.5 C" |
|<-----------------+
| |
| ACK [0x23bb] |
+----------------->|
| |
Figure 5: A GET Request with a Separate Response
Client Server
| |
| NON [0x7a11] |
| GET /temperature |
| (Token 0x74) |
+----------------->|
| |
| NON [0x23bc] |
| 2.05 Content |
| (Token 0x74) |
| "22.5 C" |
|<-----------------+
| |
Figure 6: A Request and a Response Carried in Non-confirmable
Messages
<style scoped> pre { font-size: 0.7em; } </style>
+-----+---+---+---+---+----------------+--------+--------+----------+
| No. | C | U | N | R | Name | Format | Length | Default |
+-----+---+---+---+---+----------------+--------+--------+----------+
| 1 | x | | | x | If-Match | opaque | 0-8 | (none) |
| 3 | x | x | - | | Uri-Host | string | 1-255 | (see |
| | | | | | | | | below) |
| 4 | | | | x | ETag | opaque | 1-8 | (none) |
| 5 | x | | | | If-None-Match | empty | 0 | (none) |
| 7 | x | x | - | | Uri-Port | uint | 0-2 | (see |
| | | | | | | | | below) |
| 8 | | | | x | Location-Path | string | 0-255 | (none) |
| 11 | x | x | - | x | Uri-Path | string | 0-255 | (none) |
| 12 | | | | | Content-Format | uint | 0-2 | (none) |
| 14 | | x | - | | Max-Age | uint | 0-4 | 60 |
| 15 | x | x | - | x | Uri-Query | string | 0-255 | (none) |
| 17 | x | | | | Accept | uint | 0-2 | (none) |
| 20 | | | | x | Location-Query | string | 0-255 | (none) |
| 35 | x | x | - | | Proxy-Uri | string | 1-1034 | (none) |
| 39 | x | x | - | | Proxy-Scheme | string | 1-255 | (none) |
| 60 | | | x | | Size1 | uint | 0-4 | (none) |
+-----+---+---+---+---+----------------+--------+--------+----------+
C=Critical, U=Unsafe, N=NoCacheKey, R=Repeatable
Table 1: Options
- is an extension RFC 7641
simple protocol extension for CoAP that enables CoAP clients to "observe" resources, i.e., to retrieve a representation of a resource and keep this representation updated by the server over a period of time [...] The protocol does not offer explicit means for setting up triggers or thresholds;
+-----+---+---+---+---+---------+--------+--------+---------+
| No. | C | U | N | R | Name | Format | Length | Default |
+-----+---+---+---+---+---------+--------+--------+---------+
| 6 | | x | - | | Observe | uint | 0-3 B | (none) |
+-----+---+---+---+---+---------+--------+--------+---------+
C=Critical, U=Unsafe, N=No-Cache-Key, R=Repeatable
Table 2: The Observe Option
- The only difference between a notification and a normal response is the presence of the Observe Option.
- the server sends a notification with an appropriate response code (such as 4.04 Not Found) and removes the client's entry from the list of observers of the resource.
- can be confirmable or non-confirmable
- An acknowledgement message signals to the server that the client is alive and interested in receiving further notifications; if the server does not receive an acknowledgement in reply to a confirmable notification, it will assume that the client is no longer interested and will eventually remove the associated entry from the list of observers
- If a client does not recognize the token in a confirmable notification, it must not acknowledge the message and should reject it with a Reset message; otherwise, the client MUST acknowledge the message as usual. In the case of a non-confirmable notification, rejecting the message with a Reset message is OPTIONAL.
- To cancel an observation the client sends on the next message a Reset message
- To de-register clients sets observation flag to 1
- on registration the server returns the current representation of the resource and adds client to list of observers
- consists of client endpoint + token
<style scoped> pre { font-size: 0.7em; } </style>
Observed CLIENT SERVER Actual
t State | | State
____________ | | ____________
1 | |
2 unknown | | 18.5 Cel
3 +----->| Header: GET 0x41011633
4 | GET | Token: 0x4a
5 | | Uri-Path: temperature
6 | | Observe: 0 (register)
7 | |
8 | |
9 ____________ |<-----+ Header: 2.05 0x61451633
10 | 2.05 | Token: 0x4a
11 18.5 Cel | | Observe: 9
12 | | Max-Age: 15
13 | | Payload: "18.5 Cel"
14 | |
15 | | ____________
16 ____________ |<-----+ Header: 2.05 0x51457b50
17 | 2.05 | 19.2 Cel Token: 0x4a
18 19.2 Cel | | Observe: 16
29 | | Max-Age: 15
20 | | Payload: "19.2 Cel"
21 | |
<style scoped> pre { font-size: 0.65em; } </style>
t Observed CLIENT SERVER Actual
____________ | | ____________
22 | |
23 19.2 Cel | | 19.2 Cel
24 | | ____________
25 | X----+ Header: 2.05 0x51457b51
26 | 2.05 | 19.7 Cel Token: 0x4a
27 | | Observe: 25
28 | | Max-Age: 15
29 | | Payload: "19.7 Cel"
30 | |
31 ____________ | |
32 | |
33 19.2 Cel | |
34 (stale) | |
35 | |
36 | |
37 | |
38 +----->| Header: GET 0x41011634
39 | GET | Token: 0xb2
40 | | Uri-Path: temperature
41 | | Observe: 0 (register)
42 | |
43 | |
44 ____________ |<-----+ Header: 2.05 0x61451634
45 | 2.05 | Token: 0xb2
46 19.7 Cel | | Observe: 44
47 | | Max-Age: 15
48 | | ETag: 0x78797a7a79
49 | | Payload: "19.7 Cel"
50 | |
CoAP is nice for small values, but what about big files, i.e. FOTA?
<style scoped> p { font-size: 0.75em; } </style>
TLDR: you want to avoid splitting on UDP or IP level
- Adds a pair of Block options
- Transfers larger than what can be accommodated in onstrained network link layer packets can be performed in smaller blocks
- no hard-to-manage conversation state is created at the adaptation layer or IP layer for fragmentation
- the transfer of each block is acknowledged, enabling individual retransmission if required
- both sides discuss block sizes:
Size1, Size2, Block1, Block2
<style scoped> pre { font-size: 0.75em; } </style>
Download example - Block2
CLIENT SERVER
| |
| CON [MID=1234], GET, /status, 2:0/0/64 ------> |
| |
| <------ ACK [MID=1234], 2.05 Content, 2:0/1/64 |
| |
| CON [MID=1235], GET, /status, 2:1/0/64 ------> |
| |
| <------ ACK [MID=1235], 2.05 Content, 2:1/1/64 |
: :
: ... :
: :
| CON [MID=1238], GET, /status, 2:4/0/64 ------> |
| |
| <------ ACK [MID=1238], 2.05 Content, 2:4/1/64 |
| |
| CON [MID=1239], GET, /status, 2:5/0/64 ------> |
| |
| <------ ACK [MID=1239], 2.05 Content, 2:5/0/64 |
<style scoped> pre { font-size: 0.75em; } </style>
Upload example - Block1
CLIENT SERVER
| |
| CON [MID=1234], PUT, /options, 1:0/1/128 ------> |
| |
| <------ ACK [MID=1234], 2.31 Continue, 1:0/1/32 |
| |
| CON [MID=1235], PUT, /options, 1:4/1/32 ------> |
| |
| <------ ACK [MID=1235], 2.31 Continue, 1:4/1/32 |
| |
| CON [MID=1236], PUT, /options, 1:5/1/32 ------> |
| |
| <------ ACK [MID=1235], 2.31 Continue, 1:5/1/32 |
| |
| CON [MID=1237], PUT, /options, 1:6/0/32 ------> |
| |
| <------ ACK [MID=1236], 2.04 Changed, 1:6/0/32 |
- Wireshark 🦈
Why? Because we have a deduplicator in our code
See backup slides 🤓
- CoAP stands out as a robust and efficient protocol tailored for IoT scenarios.
- Its lightweight design, support for push notifications, and blockwise transfer make it well-suited for resource-constrained devices.
- CoAP's reliable messaging model sets a solid foundation for IoT communication.
Technical Spec 1.1 Transport Spec 1.1
Lightweight Machine-to-Machine (LwM2M) is an IoT protocol designed for efficient and secure device management, building on the CoAP protocol. It provides a standardized way for managing communication between IoT devices and servers, offering features such as remote device configuration, firmware updates, and real-time data reporting. LwM2M focuses on simplicity, low overhead, and secure interactions, making it well-suited for resource-constrained devices in diverse IoT deployments.
- they serve different purposes in the IoT ecosystem
- LwM2M is a higher-level protocol that builds on CoAP and is specifically designed for device management in IoT deployments
- LwM2M provides a standardized framework and data model for managing devices and their resources
- LwM2M extends CoAP with additional features and semantics tailored for device management, such as the ability to perform remote firmware updates, execute commands on devices, and manage device registration and security
LwM2M can be seen as an application-layer protocol that leverages CoAP for communication while adding higher-level semantics for managing IoT devices
- Bootstrap Interface ❌
- Client Registration interface
- Device Mangement and Service Enablement interface
- Information Reporting interface
- Device Initialization
- Security Credential Provisioning
- Configuration Management
- Registration, Update and Deregistration
- Lifetime Management
- Endpoint and Object Discovery
- Remote Configuration
- Firmware Updates
- Resource Monitoring and Control
- Observation and Notification
- Periodic Reporting
- Event Reporting
Follows a resource model based on
- Client has different objects
- Can have an object multiple times
- Objects have different resources
- Resource may have multiple values
/{Object Id}/{Instance Id}/{Resource Id}/{Multiple-Instance Resources Id}
- specified in a separate document
- may have different versions
- must be set in the registration
A resource may
- contain a value (readable and/or writeable)
- can be addressed by a LwM2m Server to trigger an action (executable)
This goes along with the operations (read, write, execute)
-
Registration
-
Update
-
Deregistration
-
Write Attribute
-
(Composite-) Observation
-
Notify
-
Send
-
Execution
-
For all mappings to CoAP see:
Table: 6.4.4.-1andTable: 6.4.5.-1here
- Clients boots and wants to communicate with the LwM2m Server
- Note: On CoAP level both parties act as server and client
CoAP POST on /rd?lwm2m={Version}&ep={ICCID}<={number}with object model in payload- Server responses with
CoAP 2.01and option:Location-Pathfor a device specifc endpoint
| Parameter | Required | Value | Notes |
|---|---|---|---|
| Endpoint Client Name | No | See Section 7.3. Identifiers | |
| Lifetime | Yes | Indicates the expected lifetime of the registration for this LwM2M client. This value MUST be the same as the value held in the Resource named "Lifetime" of the corresponding instance of Server Object (ID #1): /1/x/1. | |
| LwM2M Version | Yes | Indicates the version of the LwM2M Enabler that the LwM2M Client supports. The LwM2M version number reported MUST correspond to the approved version number of this document. | |
| Binding Mode | No | U | Indicates the supported binding modes in the LwM2M Client. This value SHOULD be the same as the value in the “Supported Binding and Modes” resource in the Device Object (/3/0/16). The valid values of the parameter are listed in Section 6.2.1.2. Behaviour with Current Transport Binding and Modes |
| Queue Mode | No | Indicates whether Queue Mode is supported. The Queue Mode is useful when the LwM2M Device is not reachable by the LwM2M Server at all times and it helps the LwM2M Client to reduce power consumption by sleeping longer. | |
| SMS Number | No | The value of this parameter is the MSISDN or External Identifier where the LwM2M Client can be reached for use with the SMS binding or when both the LwM2M Client and the LwM2M Server support the SMS registration update trigger mechanism defined in [LwM2M-TRANSPORT]. | |
| Objects and Object Instances | Yes | The list of Objects supported and Object Instances available on the LwM2M Client (Security Object ID:0, and OSCORE Object ID:21, if present, MUST NOT be part of this list. |
- LwM2M Client updates its registration information
- Update operation can be initiated by the LwM2M Server via an "Execute" operation on the "Registration Update Trigger
- The object list must be transmitted again
CoAP POST /{location}?lt={Lifetime}&b={binding}
Use cases:
- Lifetime update
- Objects changed
- LwM2m Client goes offline
CoAP DELETE /{location}
- Wireshark 🦈
- Change a value of a resource
CoAP PUT or POST on /{Object ID}/{Object Instance ID}/{Resource ID}/{Resource Instance ID} or /{Object ID}/{Object Instance ID}with payload- Time Syncronization
- Device Configuration: How often to send data (assettracker specific)
- Initiate some action on the client
- Can only be done on resources
- Example: Execute an update
CoAP POST on /{Object ID}/{Object Instance ID}/{Resource ID}(optional: content, format: text/plain or none)
- Configure Observations
CoAP PUT {Object ID}/{Object Instance ID}/{Resource ID}/{Resource Instance ID}?pmin={minimum period}&pmax={maximum period}>={greater than}<={less than}&st={step}&epmin={minimum evaluation period}&epmax={maximum evaluation period}
- Write-Attributes
/3/0/9?pmin=1means the Battery Level value will be notified to the Server with a minimum interval of 1sec; - Write-Attributes
/3/0/9?gt=45&st=10means the Battery Level will be notified to the Server when:- old value is 45 and new value is 50 due to gt condition
- old value is 38 and new value is 49 due to both gt and step conditions
CoAP GET /{Object ID}/{Object Instance ID}/{Resource ID}/{Resource Instance ID}with Observe option setCoAP FETCHwith Observer option set and URI paths in body- To cancel an observation send a
Resetmessage on aNotifymessage - Wireshark 🦈
- An asynchronous message containing the CoAP token
- Can be confirmable of non-confirmable
- See
Figure: 6.4.5.-1here for a flow example
- Clients sends data without any configuration from server side
CoAP POST on /dpwith data in payload (format: CBOR or JSON)
The LwM2M Server MAY use the "Mute Send" Resource in the LwM2M Server Object (Resource ID 23) to enable or disable the use of the "Send" operation. The reported LwM2M Object Instances (potentially different Object types) MUST have been registered by the LwM2M Client to the LwM2M Server.
- Wireshark 🦈
Overview:
- In CoAP, message deduplication is crucial for handling duplicate Confirmable or Non-confirmable messages.
- Ensures reliable communication and prevents unintended processing of duplicate messages.
Confirmable Messages:
- Challenge: Confirmable messages may be received multiple times (lost acknowledgments, timeouts).
- Action: Recipient should acknowledge each duplicate Confirmable message using the same Acknowledgement or Reset message.
- Processing: Recipient should process any request or response in the message only once.
- Relaxation: Allowed for idempotent requests, where duplicate processing is less costly than tracking previous responses.
Examples of Relaxation:
- A server may relax the requirement to answer all retransmissions of an idempotent request (GET, PUT, DELETE) to avoid maintaining state for Message IDs.
- A constrained server might relax this requirement for certain non-idempotent requests based on favorable trade-offs in application semantics.
Non-confirmable Messages:
- Challenge: Non-confirmable messages may be received multiple times within NON_LIFETIME.
- Action: Recipient SHOULD silently ignore any duplicated Non-confirmable message.
- Processing: Recipient SHOULD process any request or response in the message only once.
- Relaxation: General rule MAY be relaxed based on specific message semantics.
Conclusion:
- Message deduplication ensures reliable and efficient handling of CoAP messages, with flexibility for relaxation in certain scenarios, especially for idempotent requests or based on application-specific trade-offs.
- Is set dureing registration
- When the LwM2M Client is not online, the LwM2M Server does not immediately send downlink requests, but instead waits until the LwM2M Client is online
- Due to the congestion control approach used by CoAP the LwM2M Server has to wait for a response to a request before sending out the next request from the queue since [CoAP] limits the number of simultaneous outstanding interactions to 1.
- For more details see here under 6.5 Queue Mode Operation