Data
Augmenta OSC Protocol v2.0
Note : All coordinates are (otherwise specified) normalized [0-1] with a top left origin.
When an object enters, is updated or leaves the scene
/object/enter arg0 arg1 ... argN
/object/update arg0 arg1 ... argN
/object/leave arg0 arg1 ... argN
where args are :
0: frame (int) // Scene frame number
1: id (int) // id ex : 42th object to enter stage has id=42
2: oid (int) // Ordered id ex : if 3 objects on stage, 43th object can have oid=2
3: age (float) // Alive time (in s)
4: centroid.x (float 0:1) // Position projected to the ground (normalized)
5: centroid.y (float 0:1)
6: velocity.x (float) // Speed and direction vector (in unit.s-1) (normalized)
7: velocity.y (float)
8: orientation (float 0:360) // With respect to horizontal axis right (0° = (1,0)), rotate counterclockwise
Estimation of the object orientation from its rotation and velocity
9: boundingRect.x (float 0:1) // Bounding box center coord (normalized)
10: boundingRect.y (float 0:1)
11: boundingRect.width (float 0:1) // Bounding box width (normalized)
12: boundingRect.height (float 0:1)
13: boundingRect.rotation (float 0:360)// With respect to horizontal axis right counterclockwise
14: height (float) // Height of the object (in m) (absolute)
/object/enter/extra arg0 arg1 ... argN
/object/update/extra arg0 arg1 ... argN
/object/leave/extra arg0 arg1 ... argN
0: frame (int) // Scene frame number
1: id (int) // id ex : 42th object to enter scene has pid=42
2: oid (int) // Ordered id ex : if 3 object on stage, 43th object might have oid=2
3: highest.x (float 0:1) // Highest point placement (normalized)
4: highest.y (float 0:1)
5: distance (float) // Sensor distance (in m)
6: reflectivity (float) // Reflection value from Lidar sensor
Scene information
/scene arg0 arg1 ... argN
0: frame (int) // Scene frame number
1: objectCount (int) // Number of objects
2: scene.width (float) // Size (in m)
3: scene.height (float)
Fusion software specific information
/fusion arg0 arg1 ... argN
0: videoOut.offset.x (float) // Offset from scene top left (in m)
1: videoOut.offset.y (float)
2: videoOut.width (float) // Size (in m)
3: videoOut.height (float)
4: videoOut.widthInPixels (int) // Resolution (in pixels)
5: videoOut.heightInPixels (int)
Legacy OSC protocols can be found here
Standard TUIO 1.1 protocol plus /scene and /fusion messages from the OSC protocol.
Note : TUIO Protocol does not give access to the complete data collection from the Augmenta protocol. As the TUIO protocol doesn't support sending both the bounding box and the centroid position and orientation, only the bounding box information are sent with the blob descriptors mode. In the case of cursor and object descriptors mode, the centroid information is sent.
Here is the correspondence table between Augmenta data and TUIO data:
| TUIO | Augmenta (cursor and object TUIO mode) | Augmenta (blob TUIO mode) |
|---|---|---|
| s | id | id |
| i | oid | oid |
| x, y, z | centroid.x, centroid.y, height/20 | boundingRect.x, boundingRect.y, height/20 |
| a, b, c | orientation, 0, 0 | boundingRect.rotation, 0, 0 |
| w, h, d | - | boundingRect.width, boundingRect.height, height/10 |
| f, v | - | w x h, w x h x d |
| X, Y, Z | velocity.x, velocity.y, 0 | velocity.x, velocity.y, 0 |
| A, B, C | 0, 0, 0 | 0, 0, 0 |
| m | 0 | 0 |
| r | 0 | 0 |
Reminder of TUIO 1.1 semantic types:
| TUIO Data | Description | Type |
|---|---|---|
| s | Session ID (temporary objcet ID) | int32 |
| i | Class ID (e.g. marker ID) | int32 |
| x, y, z | Position |
float32 [0,1] |
| a, b, c | Angle |
float32 [0,2PI] |
| w, h, d | Dimension |
float32 [0,1] |
| f, v | Area, Volume |
float32 [0,1] |
| X, Y, Y | Velocity vector (motion speed and direction) | float32 |
| A, B, C | Rotation velocity vector (rotation speed and direction) | float32 |
| m | Motion acceleration | float32 |
| r | Rotation acceleration | float32 |
Presets are implemented in fusion to help you choose the best TUIO format:
| Presets | Supported descriptors | Supported dimensions | Default | Description |
|---|---|---|---|---|
| None | Cursor, Blob, Object | 2D, 2.5D, 3D | - | - |
| Minimal | Cursor | 2D | 2D Cursor | - |
| Best | Blob | 2.5D, 3D | 2.5D Blob | Best fit for Augmenta's data |
| Notch | Cursor, Object, Blob | 2D | 2D Blob | Supported formats in Notch |
| Touch Designer | Cursor, Object, Blob | 2D, 2.5D, 3D | 3D Blob | Supported formats in Touch Designer |
Note : By default, OSC and TUIO are sent on the same port for Touch Designer's preset as you can receive both OSC messages and TUIO messages on the same connection. For Notch's preset they are sent on separate ports.
Augmenta JSON Protocol v2.0
Note : All coordinates are (otherwise specified) normalized [0-1] with a top left origin.
When an object enters the scene
{"object":
{ "enter": {
"frame": frame,
"id": id,
"oid": oid,
"age": age,
"centroid": {
"x": centroid.x,
"y": centroid.y
},
"velocity": {
"x": velocity.x,
"y": velocity.y
},
"orientation": orientation,
"boundingRect": {
"x": boundingRect.x,
"y": boundinRect.y,
"width": boundingRect.width,
"height": boundingRect.height,
"rotation": boundingRect.rotation
},
"height": height,
"extra": {
"frame": extra.frame,
"id": extra.id,
"oid": extra.oid,
"highest": {
"x": extra.highest.x,
"y": extra.highest.y
},
"distance": extra.distance,
"reflectivity": extra.reflectivity
}
}
}
When an object is updated (same format than enters)
{"object":
{ "update": {
...
}
}
}
When an object leaves the scene
{"object":
{ "leave": {
...
}
}
}
Scene information
{"scene": {
"frame": frame,
"objectCount": objectCount,
"scene": {
"width": scene.width,
"height": scene.height
}
}
Fusion software specific information
{"fusion":
"textureOffset": {
"x": textureOffset.x,
"y": textureOffset.y
},
"textureBounds": {
"x": textureBounds.x,
"y": textureBounds.y
},
"targetOutSize": {
"x": targetOutSize.x,
"y": targetOutSize.y
}
}
Augmenta OSC Protocol v3.0
Each frame is sent as an udp osc bundle
- i : int
- f : float (if not specified, type is float)
- t : osc timestamp format
Note : This is the current internal coordinate system of Augmenta, the origin and axis are freely tunable when sending data.
- Origin : Top-left of detected plane
- Axis : left-handed, Z up
- System : metric
// TODO : to discuss about in and out management
When an object enters, is updated or leaves the scene
/in/objN/... // first frame
/up/objN/... // next frames
/out/objN/... // last frame before exit
where N is the object id (also called slot id) // Insert page : More about the slot ID algorithm
where ... is
/frame i // frame id (resets at MAX_INT32)
/uid i // unique ID ex : 42th object to enter stage has uid=41 (resets at MAX_INT32)
/pos/xyz x y z // centroid position 3d coordinates
/height f // height, altitude of highest point (alongside normal axis from detected plane)
/velocity x y z // velocity vector coordinates
/speed f // velocity norm in m/s
/direction x y z // vector coordinates of current direction (equivalent to velocity while moving)
/box/dim x y z w h l // x y z : bounding box center coordinates, w h l : length alongside x y z axis
/box/rot r // rotation angle in deg (around normal axis from plane, 0° at top view horizontal axis, counterclockwise)
# Optional args
/confidence f // % of confidence [0-1]
/pos/x x // centroid X coordinate
/pos/y y // centroid Y coordinate
/pos/z z // centroid Z coordinate
Scene information
/scene/frame i // frame id (resets at MAX_INT32)
/scene/objCount i // number of objects
/scene/pos x y z // scene top left 3d coordinate
/scene/size w h // scene size
/scene/video/pos x y z // video top left 3d coordinate
/scene/video/size w h // video surface size (in m)
/scene/video/res i i // resolution in pixels (square)
/scene/objects ... lists of obj ? // osc type ? // TODO : TO DISCUSS
# Optional args
/scene/timestamp t // timestamp
/scene/maxHeight z // scene max height for normalized value