# Protocol

Internet of Things Message Protocol (IOTMP)

## Introduction

The **Internet of Things Message Protocol (IoTMP)** is a lightweight messaging protocol specifically designed to address the most common challenges in IoT communication. Unlike generic protocols such as **MQTT, CoAP, or AMQP**, IoTMP focuses on **simplicity, efficiency, and versatility**, enabling devices to easily handle key usage patterns: sensing, actuation, data streaming, and interoperability with external services.

IoTMP is optimized for **resource-constrained devices** (sensors, microcontrollers, gateways) and for **large-scale deployments** that demand security, reliability, and scalability.

### Key Features

* **Lightweight and efficient**\
  Built for low-power environments and embedded devices, with a minimal footprint and straightforward integration into existing projects.
* **Client–Server model**
  * **Clients** expose resources (e.g., sensors, actuators, or data streams).
  * The **Server** centralizes communication and makes client resources available via **REST APIs** and **WebSockets**.
* **Native interoperability**\
  Clients are not limited to sending data—they can also consume server-side resources:
  * Retrieve configurations.
  * Interact with third-party APIs.
  * Subscribe to events and alarms.
  * Trigger remote actions.
* **Flexible communication patterns**
  * **Request/Response** for simple interactions.
  * **Streaming** for continuous, real-time data delivery.
* **Scalability**\
  Capable of managing **thousands of devices simultaneously** and processing **high data volumes**, making it suitable for industrial and smart city scenarios.
* **Built-in security**\
  Secure communication through **username/password authentication** combined with **SSL/TLS encryption**.
* **Multi-language support**\
  Reference implementations are available for **Arduino, C++, and more comming.** 

### Use Cases

* **Industrial automation**: sensor monitoring, actuator control, and real-time metric streaming.
* **Smart homes**: remote control of appliances, seamless integration with cloud services, and mobile applications.
* **Energy management**: consumption monitoring, efficiency alerts, and remote device management.
* **Custom IoT solutions**: flexible enough to integrate with any external API or service.

### Conclusion

IoTMP is not “just another IoT protocol.” It has been designed with a **practical mindset**, addressing the real-world needs of connected devices and management platforms. Its **simplicity, security, and scalability** make it a strong alternative to traditional protocols, delivering a more complete experience for both developers and integrators.


# Message Structure

IOTMP Message Structure

IOTMP communication is based on the exchange of structured messages between clients and servers. Each IOT **message** is composed of two main parts:

1. **Header** – mandatory, contains metadata about the message type and the size of its payload.
2. **Body (Payload)** – optional, contains the actual data being transmitted.

The general structure is:

|                                     Message Structure                                    |
| :--------------------------------------------------------------------------------------: |
| [Message Header](/iotmp/message-structure/message-header), present in all IOTMP messages |
|   [Message Body](/iotmp/message-structure/message-body), present in some IOTMP messages  |


# Message Header

IOTMP Message Header

The **header** is the mandatory first part of every IOTMP message. It provides the essential metadata required to parse the message:

* **Message Type** – identifies the purpose of the message.
* **Message Size** – defines the length of the message body in bytes (excluding the header).

The header is encoded using **varint fields**, resulting in a **minimum size of 2 bytes**.

<table><thead><tr><th width="179">Field</th><th width="111">Type</th><th width="134">Mandatory</th><th>Description</th></tr></thead><tbody><tr><td><a href="#message-type"><strong>Message Type</strong></a></td><td><a href="/iotmp/definitions#varint">varint</a></td><td>Yes</td><td>Specifies the <a href="/iotmp/message-structure/message-header#message-types">Message Type</a>. </td></tr><tr><td><a href="#message-size"><strong>Message Size</strong></a></td><td><a href="/iotmp/definitions#varint">varint</a></td><td>Yes</td><td>Specifies the <a href="/iotmp/message-structure/message-header#message-size">Message Size</a>, without taking into account the header size.</td></tr></tbody></table>

## Message Type

The **Message Type** field identifies the purpose of the IOTMP message. It is always present in the header and is encoded as a **varint**.

Message types define the control flow of the protocol, covering actions such as connection management, resource execution, error handling, and data streaming.

The protocol reserves specific numeric values for core operations, while leaving others available for future extensions. This ensures both **compatibility** and **extensibility** as new functionality is introduced.

The currently defined message types are:

<table><thead><tr><th width="229">Message Type</th><th width="130.33333333333331">Value</th><th>Description</th></tr></thead><tbody><tr><td><strong>Reserved</strong></td><td>0x00</td><td>Reserved field, not used</td></tr><tr><td><a href="/iotmp/messages/ok"><strong>Ok</strong></a></td><td>0x01</td><td>Acknowledges success</td></tr><tr><td><a href="/iotmp/messages/error"><strong>Error</strong></a></td><td>0x02</td><td>Indicates an error occurred</td></tr><tr><td><a href="/iotmp/messages/connect"><strong>Connect</strong></a></td><td>0x03</td><td>Initiates a connection and specifies its parameters</td></tr><tr><td><a href="/iotmp/messages/disconnect"><strong>Disconnect</strong></a></td><td>0x04</td><td>Terminates the current connection</td></tr><tr><td><a href="/iotmp/messages/keep-alive"><strong>Keep Alive</strong></a></td><td>0x05</td><td>Keep the connection open</td></tr><tr><td><a href="/iotmp/messages/run"><strong>Run Resource</strong></a></td><td>0x06</td><td>Execute a specific resource on the client or server</td></tr><tr><td><a href="/iotmp/messages/describe"><strong>Describe Resources</strong></a></td><td>0x07</td><td>Request a description of available resources</td></tr><tr><td><a href="/iotmp/messages/streams/start-stream"><strong>Start Stream</strong></a></td><td>0x08</td><td>Starts a stream on the specified resource</td></tr><tr><td><a href="/iotmp/messages/streams/stop-stream"><strong>Stop Stream</strong></a></td><td>0x09</td><td>Stops an active stream</td></tr><tr><td><a href="/iotmp/messages/streams/stream-event"><strong>Stream Data</strong></a></td><td>0x0A</td><td>Transmits data for an active stream</td></tr></tbody></table>

## Message Size

The **Message Size** field specifies the length of the message body, expressed in **bytes**. It does **not** include the size of the header itself.

* Encoded as a **varint**.
* Always present in the message header.
* A value of **0** indicates that the message has no body (only a header).

This field enables the receiver to determine exactly how many bytes to read for the payload and when a message is complete.


# Message Body

IOTMP Message Body

The **message body** (payload) contains the actual data transmitted between client and server. It directly follows the header and its size is defined by the **Message Size** field.

The body is composed of a sequence of **key–value pairs**, which provide a flexible and extensible way to encode data.

### Key–Value Pairs

Each message body consists of zero or more **key–value pairs**, concatenated into a byte stream.

* **Keys** identify the field and how its value is encoded.
* **Values** hold the actual data.
* Parsers must be able to **skip unrecognized keys**, ensuring forward compatibility: new fields can be added without breaking older implementations.

<table><thead><tr><th width="96.33333333333331">Field</th><th width="76">Type</th><th>Description</th></tr></thead><tbody><tr><td><a href="#key"><strong>Key</strong></a></td><td><a href="/iotmp/definitions#varint">varint</a></td><td>Encodes both the field identifier and the wire type.</td></tr><tr><td><a href="#value"><strong>Value</strong></a></td><td><a href="/iotmp/definitions#any">any</a></td><td>Contains the actual payload of the field.</td></tr></tbody></table>

### Key

A **Key** is encoded as a **varint** that combines two elements:

1. **Field Identifier** – specifies the meaning of the field (e.g., stream ID, parameter, payload).
2. **Wire Type** – specifies how the value is encoded on the wire.

The binary layout of a key is:

```
[continuation bit][field identifier (4 bits)][wire type (3 bits)]
```

* **4-bit Field Identifier** → up to 16 unique fields per message (extendable with multi-byte varints).
* **3-bit Wire Type** → up to 8 encoding formats.
* **Continuation bit** → allows extending the field identifier beyond 4 bits.

**Example:**\
A key with wire type `0` (Varint) and field identifier `1` is encoded as:

```
[0][0001][000]
```

This compact scheme ensures efficient encoding while allowing future extensibility.

#### Wire Type

The **Wire Type** specifies how the value associated with a key is encoded on the wire. It is encoded in the **3 least significant bits** of the key varint.

Currently, IOTMP defines the following wire types:

<table><thead><tr><th width="185.33333333333331">Type</th><th width="103.41796875">Value</th><th>Used For</th></tr></thead><tbody><tr><td><strong>Varint</strong></td><td>0x00</td><td>Positive integers encoded as varint. Commonly used for protocol identifiers (e.g., stream ID).</td></tr><tr><td><strong>PSON</strong></td><td>0x01</td><td>Structured data encoded in PSON. Supports streaming decoding, allowing the payload to be read immediately after the key.</td></tr><tr><td>Reserved</td><td>0x02 - 0x06</td><td>Reserved for future extensions.</td></tr></tbody></table>

This design allows IOTMP to support both **lightweight numeric values** (varint) and **complex data structures** (PSON), while keeping room for additional encodings in the future.

#### Field Identifier

The **Field Identifier** specifies the role of the field within a message.

* Encoded in the **4 bits** of the key (plus additional bytes if extended).
* Determines the semantic meaning of the value (e.g., stream identifier, parameter, payload).
* Each message definition specifies its own set of valid field identifiers.

This allows different message types to reuse the same encoding scheme while defining their own unique fields.

### Value

The **Value** represents the actual payload associated with a key.

* Its encoding is determined by the **Wire Type**.
* Can be a simple integer (varint) or a structured object (PSON).
* Parsers must be able to skip values they do not recognize, preserving forward compatibility.

Values are placed in the message body **immediately after their key**, enabling compact and efficient encoding.


# Messages


# Ok

Message to indicate success on the previous request.

An Ok Message can be sent both by clients or servers to indicate success on a previous request identified with its [Stream Identifier](/iotmp/definitions#stream-identifier).

## Header

<table><thead><tr><th width="191">Field</th><th width="116.33333333333331">Value</th><th>Description</th></tr></thead><tbody><tr><td><strong>Message Type</strong></td><td>0x01</td><td>Ok</td></tr><tr><td><strong>Message Size</strong></td><td><a href="/iotmp/definitions#varint">varint</a></td><td>Remaining Message Length</td></tr></tbody></table>

## Body

<table><thead><tr><th width="163">Name</th><th width="85">Field</th><th width="93">Type<select><option value="faaeb5d050da4a2a90335311051c9a1f" label="varint" color="blue"></option><option value="fe8503ea020645ffb7798d63f1250324" label="any" color="blue"></option></select></th><th width="117" data-type="checkbox">Mandatory</th><th>Description</th></tr></thead><tbody><tr><td><strong>Stream Id</strong></td><td>0x01</td><td><span data-option="faaeb5d050da4a2a90335311051c9a1f">varint</span></td><td>true</td><td><a href="/iotmp/definitions#stream-identifier">Stream identifier</a> of the source request.</td></tr><tr><td><strong>Parameters</strong></td><td>0x02</td><td><span data-option="fe8503ea020645ffb7798d63f1250324">any</span></td><td>false</td><td>Optional parameters associated to the response, like a response code, payload description, etc.</td></tr><tr><td><strong>Payload</strong></td><td>0x03</td><td><span data-option="fe8503ea020645ffb7798d63f1250324">any</span></td><td>false</td><td>Optional payload associated to the response</td></tr></tbody></table>


# Error

Message to indicate an error on the previous request.

An [Error ](/iotmp/messages/error)Message can be sent both by clients or servers to indicate an error on a previous request identified with its [Stream Identifier](/iotmp/definitions#stream-identifier).

## Header

<table><thead><tr><th width="180.33333333333331">Field</th><th width="119">Value</th><th>Description</th></tr></thead><tbody><tr><td><strong>Message Type</strong></td><td>0x02</td><td>Error</td></tr><tr><td><strong>Message Size</strong></td><td><a href="/iotmp/definitions#varint">varint</a></td><td>Remaining Message Length</td></tr></tbody></table>

## Body

<table><thead><tr><th width="156">Name</th><th width="120">Field</th><th width="92">Type<select><option value="b4a3a828357a4352a2e594e477411a79" label="varint" color="blue"></option><option value="a5439d31a66a452599cbeb1bf6534973" label="any" color="blue"></option></select></th><th width="125" data-type="checkbox">Mandatory</th><th>Description</th></tr></thead><tbody><tr><td><strong>Stream Id</strong></td><td>0x01</td><td><span data-option="b4a3a828357a4352a2e594e477411a79">varint</span></td><td>true</td><td><a href="/iotmp/definitions#stream-identifier">Stream identifier</a> of the source request.</td></tr><tr><td><strong>Parameters</strong></td><td>0x02</td><td><span data-option="a5439d31a66a452599cbeb1bf6534973">any</span></td><td>false</td><td>Optional parameters associated to the response, like a response code, payload description, etc.</td></tr><tr><td><strong>Payload</strong></td><td>0x03</td><td><span data-option="a5439d31a66a452599cbeb1bf6534973">any</span></td><td>false</td><td>Optional payload associated to the response</td></tr></tbody></table>


# Connect

Message to authenticate and negotiate connection parameters.

## Request

A [Connect ](/iotmp/messages/connect)message MUST be sent from the client to the server to authenticate and negotiate the connection parameters. This message MUST be the first one after establishing the connection. If any other message or data is received, the server MUST close the connection immediately.

### Header

<table><thead><tr><th width="187.33333333333331">Field</th><th width="113">Value</th><th>Description</th></tr></thead><tbody><tr><td><strong>Message Type</strong></td><td>0x03</td><td>Connect</td></tr><tr><td><strong>Message Size</strong></td><td><a href="/iotmp/definitions#varint">varint</a></td><td>Remaining Message Length</td></tr></tbody></table>

### Body

<table><thead><tr><th width="154">Name</th><th width="80">Field</th><th width="87">Type<select><option value="ea193d2ddd7f48eca68a517c3f5feb01" label="any" color="blue"></option><option value="373c47abb07d42ea817dd87cd33c01bf" label="varint" color="blue"></option></select></th><th width="120" data-type="checkbox">Mandatory</th><th>Description</th></tr></thead><tbody><tr><td><strong>Stream Id</strong></td><td>0x01</td><td><span data-option="373c47abb07d42ea817dd87cd33c01bf">varint</span></td><td>true</td><td>Connect <a href="/iotmp/definitions#stream-identifier">Stream identifier</a>.</td></tr><tr><td><strong>Parameters</strong></td><td>0x02</td><td><span data-option="ea193d2ddd7f48eca68a517c3f5feb01">any</span></td><td>false</td><td><del>Connect parameters, that can be used to identify the authentication type.</del></td></tr><tr><td><strong>Payload</strong></td><td>0x03</td><td><span data-option="ea193d2ddd7f48eca68a517c3f5feb01">any</span></td><td>true</td><td>Authentication payload to log-in with the server.</td></tr></tbody></table>

#### Parameters

<table><thead><tr><th width="96.26171875">Key</th><th width="100.109375">Type</th><th width="108.53125">Default</th><th>Description</th></tr></thead><tbody><tr><td>"pv"</td><td>number</td><td>0</td><td><p><strong>Protocol Version</strong></p><p>0: Legacy PSON encoding</p><p>1:  New PSON encoding (more efficient)</p></td></tr><tr><td>"ka"</td><td>number</td><td>60</td><td><p><strong>Keep-Alive Interval</strong></p><p>Heartbeat interval in seconds (max: 1800)</p><p>Server applies 15% margin for timeout detection</p></td></tr><tr><td>"at"</td><td>number</td><td>0</td><td><p><strong>Authentication Type</strong></p><p>0: Credentials (username/device/password)</p><p>Future: </p><p>1 (Auto-provision), </p><p>2 (Token), etc.</p></td></tr><tr><td>"ct"</td><td>string</td><td>-</td><td><p><strong>Client Type (future)</strong></p><p>Platform identifier: "arduino", "esp32", "raspberry", etc.</p></td></tr><tr><td>"fw"</td><td>string</td><td>-</td><td><p><strong>Firmware Version (future)</strong></p><p>Client firmware version for compatibility checks</p></td></tr></tbody></table>

Example PARAMETERS

<pre><code>{
    "pv": 1, // Use new PSON protocol
<strong>    "ka": 120, // 2-minute keep-alive
</strong>    "at": 0 // Credentials authentication
}
</code></pre>

#### Payload

The PAYLOAD field format depends on the authentication type ("at" parameter).

Authentication Type 0: Credentials (Default)

When "at": 0 or not specified, PAYLOAD must be a JSON array with exactly 3 elements:

\["username", "device\_id", "password"]

Credentials

| Index | Type   | Description           |
| ----- | ------ | --------------------- |
| 0     | string | Account username      |
| 1     | string | Device identifier     |
| 2     | string | Device password/token |

## Response


# Disconnect

Message used to cleanly disconnect a current connection.

## Request

A `Disconnect`message can be sent anytime to close an ongoing connection. It can be initiated both by the server or the client.

Once sent or received, it is expected that the connection is closed.&#x20;

### Header

<table><thead><tr><th width="171">Field</th><th width="83.33333333333331">Value</th><th>Description</th></tr></thead><tbody><tr><td><strong>Message Type</strong></td><td>0x04</td><td>Disconnect</td></tr><tr><td><strong>Message Size</strong></td><td><a href="/iotmp/definitions#varint">varint</a></td><td>Remaining Message Length</td></tr></tbody></table>

### Body

<table><thead><tr><th width="165">Name</th><th width="98">Field</th><th width="100">Type<select><option value="4249dc80adf740f9bf2c571bbd56f37b" label="varint" color="blue"></option><option value="74aff7f486bd46f5a916d0a0d79a24cb" label="any" color="blue"></option></select></th><th width="120" data-type="checkbox">Mandatory</th><th>Description</th></tr></thead><tbody><tr><td><strong>Stream Id</strong></td><td>0x01</td><td><span data-option="4249dc80adf740f9bf2c571bbd56f37b">varint</span></td><td>false</td><td>Disconnect <a href="/iotmp/definitions#stream-identifier">Stream Identifier</a>. Not required if acknowledgement is not expected from the other side.</td></tr><tr><td><strong>Parameters</strong></td><td>0x02</td><td><span data-option="74aff7f486bd46f5a916d0a0d79a24cb">any</span></td><td>false</td><td>Disconnect reason.</td></tr><tr><td><strong>Payload</strong></td><td>0x03</td><td><span data-option="74aff7f486bd46f5a916d0a0d79a24cb">any</span></td><td>false</td><td>Additional disconnect information.</td></tr></tbody></table>

## Response


# Keep Alive

Message to keep the connection alive

`Keep Alive` messages must be sent by clients at the predefined time interval. If the client fails to send the keep alive at the specified rate, the server will consider the client as disconnected and will close the on-going connection. The server MUST send back a `Keep Alive` message to the client as soon as received. Clients not receiving the keep alive response at the specified interval must consider the connection as broken, and re-initiate the connection mechanism (if required).

## Header

<table><thead><tr><th width="183.33333333333331">Field</th><th width="116">Value</th><th>Description</th></tr></thead><tbody><tr><td><strong>Message Type</strong></td><td>0x05</td><td>Keep Alive</td></tr><tr><td><strong>Message Size</strong></td><td>0x00</td><td>No body</td></tr></tbody></table>

## Body

The Keep Alive message does not contain any body.


# Run Resource

Message to run a resource in a device or in the broker.

This message enables running remote [resources ](/iotmp/definitions#resources)defined both in client and server. Resources are defined dynamically by clients and servers. As described in the [resources ](/iotmp/definitions#resources)definition, a resource is basically like a function that can be executed like in RPC (Remote Procedure Call). As a "function" it can receive inputs, and provide an output.&#x20;

A **device** can define multiple resources as required by its use case, i.e., read humidity, adjust temperature, turn on a light, etc.  Typical resources for an IoT device are functions for reading device states, sensor values, adjust parameters, or actuating over digital pins to mention a few.

A **server** or broker can also define multiple resources that can be executed by their clients. Such resources can vary depending on the use case, and normally involves complex task that are not worth to be executed in the device. Typical resources defined by a server can be actions like writing to a database, calling an external endpoint for sending communications, managing device configuration, etc.

## Request

### Header

<table><thead><tr><th width="168">Field</th><th width="82.33333333333331">Value</th><th>Description</th></tr></thead><tbody><tr><td><strong>Message Type</strong></td><td>0x06</td><td>Run</td></tr><tr><td><strong>Message Size</strong></td><td><a href="/iotmp/definitions#varint">varint</a></td><td>Remaining Message Length</td></tr></tbody></table>

### Body

<table><thead><tr><th width="139">Name</th><th width="83">Field</th><th width="97">Type<select><option value="d8b0253b7149426c8cf0ed66d9fdf64f" label="any" color="blue"></option><option value="52a3abba3ca146318f76e894227193db" label="varint" color="blue"></option></select></th><th width="115" data-type="checkbox">Mandatory</th><th>Description</th></tr></thead><tbody><tr><td><strong>Stream Id</strong></td><td>0x01</td><td><span data-option="52a3abba3ca146318f76e894227193db">varint</span></td><td>false</td><td><a href="/iotmp/definitions#stream-identifier">Stream identifier</a>. If set, the target endpoint should answer to the run message. If not, it should not send back any response.</td></tr><tr><td><strong>Parameters</strong></td><td>0x02</td><td><span data-option="d8b0253b7149426c8cf0ed66d9fdf64f">any</span></td><td>false</td><td><a href="/iotmp/definitions#resource-parameters">Resource parameters</a> to be supplied while executing  the resource.</td></tr><tr><td><strong>Payload</strong></td><td>0x03</td><td><span data-option="d8b0253b7149426c8cf0ed66d9fdf64f">any</span></td><td>false</td><td>The payload sent to the target resource.</td></tr><tr><td><strong>Resource</strong> </td><td>0x04</td><td><span data-option="d8b0253b7149426c8cf0ed66d9fdf64f">any</span></td><td>true</td><td><a href="/iotmp/definitions#resource-identifier">Resource identifier</a>.</td></tr></tbody></table>

## Responses

### OK

The resource executed correctly.

### Error

An error occurred while executing the resource.


# Describe Resources

Message to discover available resources in endpoints

This message enables discovering remote [resources ](/iotmp/definitions#resources)defined both in client and server. Resources can be defined dynamically by clients and servers. As described in the [resources ](/iotmp/definitions#resources)definition, a resource is basically like a function that can be executed like in RPC (Remote Procedure Call). As a "function" it can receive parameters, inputs, and provide an output.&#x20;

This message allows discovering such resources, like its names, function type, if they support streaming, required parameters, etc.

## Request

### Header

<table><thead><tr><th width="168.33333333333331">Field</th><th width="93">Value</th><th>Description</th></tr></thead><tbody><tr><td><strong>Message Type</strong></td><td>0x07</td><td>Describe Resources</td></tr><tr><td><strong>Message Size</strong></td><td><a href="/iotmp/definitions#varint">varint</a></td><td>Remaining Message Length</td></tr></tbody></table>

### Body

<table><thead><tr><th width="163">Name</th><th width="84">Field</th><th width="100">Type<select><option value="0c77f58c89ed4decabfda2706eab0ed3" label="any" color="blue"></option><option value="6e677f5f03024d73956cdc02b81f1ef0" label="varint" color="blue"></option></select></th><th width="115" data-type="checkbox">Mandatory</th><th>Description</th></tr></thead><tbody><tr><td><strong>Stream Id</strong></td><td>0x01</td><td><span data-option="6e677f5f03024d73956cdc02b81f1ef0">varint</span></td><td>true</td><td><a href="/iotmp/definitions#stream-identifier">Stream identifier</a>.</td></tr><tr><td><strong>Parameters</strong></td><td>0x02</td><td><span data-option="0c77f58c89ed4decabfda2706eab0ed3">any</span></td><td>false</td><td>Params</td></tr><tr><td><strong>Resource</strong> </td><td>0x04</td><td><span data-option="0c77f58c89ed4decabfda2706eab0ed3">any</span></td><td>false</td><td><a href="/iotmp/definitions#resource-definition">Resource identifier</a>. It should be a string or a resource identifier. If not sent, the target endpoint should report all available resources.</td></tr></tbody></table>

{% hint style="info" %}
If not resource is specified, the target endpoint should list all available resources.
{% endhint %}

## Response (All Resources)

If no resource is specified while sending the Describe Resources message, the target endpoint should list all available resources,

### Ok

If the request succeed, the remote endpoint should return an [Ok Message](/iotmp/messages/ok) with the following payload:

```javascript
{
  "relay": {
    "fn": 2
    "id": 0
  },
  "temperature": {
    "fn": 3
    "id": 1
  },
  "reset" : {
    "fn": 1,
    "st": false,
    "id": 2
}
```

In such document each key represents the resource name

<table><thead><tr><th width="166">Describe Field</th><th width="82">Key</th><th width="173">Value</th><th width="116" data-type="checkbox">Mandatory</th><th>Description</th></tr></thead><tbody><tr><td>Function Type</td><td><strong>fn</strong></td><td><p><strong>1</strong> : function without parameters</p><p><strong>2</strong> : function with input</p><p><strong>3</strong>: function with output</p><p><strong>4</strong>: function with input and output</p></td><td>true</td><td>Provides information about the function type, i.e., if it requires and input, provides an output, none, or both of them.</td></tr><tr><td>Parameters</td><td><strong>pr</strong></td><td><p><strong>true</strong>: requires parameters</p><p><strong>false</strong>: no parameters required</p></td><td>false</td><td>Determines if the resource requires parameters to be executed. By default, if not specified, the resource does not require parameters to be executed.</td></tr><tr><td>Stream</td><td><strong>st</strong></td><td><p><strong>true</strong>: support Stream</p><p><strong>false</strong>: no stream support</p></td><td>false</td><td>Determines if the resource can be used with the Streams functionality. By default, if not specified, the stream support is enabled.</td></tr><tr><td>Resource identifier</td><td><strong>id</strong></td><td>varint</td><td>false</td><td>An alternative numeric resource identifier, instead of a name, that can be used to call this resource. </td></tr></tbody></table>

### Error

The resources cannot be described, i.e., no support for describing resources.

## Response (Single Resource)

If a resource is specified while sending the Describe Resources message, the target endpoint should reply with the resource details.

### Ok

If the request succeed, the remote endpoint should return an [Ok Message](/iotmp/messages/ok) with the following payload:

```json
{
  "in": true
}
```

```json
{
  "out": 22.33
}  
```

```json
{
  "in": 0
  "out": 25
}
```

### Error

The resource cannot be described, i.e, it does not exists.


# Resource Streams

The [Run ](/iotmp/messages/run)message is perfectly suited for request/response paradigms. Those messages allows to send a request and receive a response. This is quite useful for sending a request to a device, i.e., to turn on a light, or to query the current power consumption. However, this approach is not optimized if it is necessary a long-lived monitoring over a resource. For example, monitoring every few seconds the temperature and humidity. In this case, a constant polling to the device is not efficient.

To solve this problem, there are other protocols like MQTT where the devices just transmit the information periodically to the broker, and then this information is stored or shown in dashboards. However, this approach is not optimal, as it is assuming that the transferred information will be used somewhere. It does not take into account if the information really requires to be transmitted, i.e., there is an user viewing the dashboard, or there is a data sink storing the information. In large projects, for sure there are several topics and information that is going to nowhere.

This is where the [Streams ](/iotmp/definitions#stream)comes into play. It enables an endpoint, i.e., the server, to subscribe to a device resource. This way, the server proactively subscribes to device resources when necessary. The subscription can be periodical, or just triggered as required (i.e., an event detected by the device). In this approach, the target resource is not streamed by default, unless there is a source requesting a streaming, i.e., a user just opened a dashboard or a mobile application, or the server defined a data sink to store the information. It adds extra complexity at the server, but optimizes how the resources are streamed, saving bandwidth and scaling easily under some circumstances.

TODO add diagram

TODO add description about devices opening streams over server

TODO devices opening streams over other devices ?&#x20;

## Messages

Streams functionality has 3 different messages to control the streams:

{% content-ref url="/pages/-MDAxCl5lyr7gGq3PeZe" %}
[Start Stream](/iotmp/messages/streams/start-stream)
{% endcontent-ref %}

{% content-ref url="/pages/-MDBNMt9enoxy3VN05y7" %}
[Stop Stream](/iotmp/messages/streams/stop-stream)
{% endcontent-ref %}

{% content-ref url="/pages/-MDBNcut5E75iQLPPWkd" %}
[Stream Data](/iotmp/messages/streams/stream-event)
{% endcontent-ref %}

## Use Cases

TBD


# Start Stream

Initiates a long-lived stream for transmitting data associated to a resource.

A `Start Stream` method is a way of opening a communication channel between the server and the client. It can be opened by both sides, and each `Stream` will have its own Stream Identifier.

## Stream Identifiers

In all `IOTMP` protocol messages, there is a `Stream Id` field that allows identifying a request and its response over the wire. Once the request is completed (with an [`Ok` ](/iotmp/messages/ok)or [`Error`](/iotmp/messages/error)), the `Stream Id`can be discarded by both sides.&#x20;

With the `Start Stream` there is also a `Stream Id`, but it is kept until the Stream channel is closed, i.e., one side request a [`Stop Stream`](/iotmp/messages/streams/stop-stream). So, all the messages sent over an established stream with [`Stream Data`](/iotmp/messages/streams/stream-event), will include the `Stream Id` established in the `Start Stream` request. This allows to save bandwidth, as it is not required to send the resource identifier or stream configuration over and over again.

## Request

### Header

<table><thead><tr><th width="165">Field</th><th width="76.33333333333331">Value</th><th>Description</th></tr></thead><tbody><tr><td><strong>Message Type</strong></td><td>0x08</td><td>Start Stream</td></tr><tr><td><strong>Message Size</strong></td><td><a href="/iotmp/definitions#varint">varint</a></td><td>Remaining Message Length</td></tr></tbody></table>

### Body

<table><thead><tr><th width="147">Name</th><th width="76">Field</th><th width="89">Type<select><option value="3d49da89b71b499a80437b39847316bb" label="any" color="blue"></option><option value="c8a9c9a9a79041a7b6d8b6641035a863" label="varint" color="blue"></option></select></th><th width="119" data-type="checkbox">Mandatory</th><th>Description</th></tr></thead><tbody><tr><td><strong>Stream Id</strong></td><td>0x01</td><td><span data-option="c8a9c9a9a79041a7b6d8b6641035a863">varint</span></td><td>true</td><td>Stream identifier to be used within the stream life-time.</td></tr><tr><td><strong>Parameters</strong></td><td>0x02</td><td><span data-option="3d49da89b71b499a80437b39847316bb">any</span></td><td>false</td><td>Start stream parameters.</td></tr><tr><td><strong>Payload</strong></td><td>0x03</td><td><span data-option="3d49da89b71b499a80437b39847316bb">any</span></td><td>false</td><td>Start stream payload.</td></tr><tr><td><strong>Resource</strong> </td><td>0x04</td><td><span data-option="3d49da89b71b499a80437b39847316bb">any</span></td><td>true</td><td>A string with the resource name or a numeric value with the resource identifier.</td></tr></tbody></table>

## Response

<table><thead><tr><th width="116">Message</th><th>Description</th></tr></thead><tbody><tr><td><a href="/iotmp/messages/ok"><strong>Ok</strong></a></td><td>Should answer with an OK if it is able to stream the requested resource with the specified parameters. It must contain the Stream Id specified in this message.</td></tr><tr><td><a href="/iotmp/messages/error"><strong>Error</strong></a></td><td>Should answer with an Error if the target resource cannot be streamed both if the resource does not exists or the provided parameters are unacceptable. It must contain the Stream Id specified in this message.</td></tr></tbody></table>

## Thinger.io

### Server Streams

In the Thinger.io implementation, clients can start streams to publish/subscribe to MQTT topics, subscribe to server events, or other device resources. These streams can be configured over parameters.&#x20;

#### MQTT Topics

A device can open a Stream to both publish and subscribe. The topic is specified in the resource field of the message, and the parameters should be configured as the following.&#x20;

```javascript
{
  "type" : "mqtt",
  "scope" : ["publish", "subscribe"]
}
```

#### Server Events

```javascript
{ 
  "type" : "event"
}
```

#### Device Resource

```javascript
{ 
  "type" : "device",
  "id" : "device_id",
  "interval": 5
}
```

### Resource definition

A resource to be used in a stream should be defined in the same way to a normal resource that can be executed over the [Run ](/iotmp/messages/run)message, i.e., a resource defined with the [Thinger.io Arduino Library](https://github.com/thinger-io/Arduino-Library) is defined as the following:

```cpp
void setup(){
  thing["heading"] >> [](pson& out){     
    out = getHeading();
  };
}
```

Internally, the client should keep information about each resource state, i.e., the associated stream identifier (if any), and the latest stream timestamp.

{% hint style="info" %}
From the perspective of a developer creating a firmware, streams should be completely transparent to the resource definition.
{% endhint %}

### Streaming with a Sampling interval

When a `Start Stream` message is received with a sampling interval, the client should look for the requested resource, and then adjust the `Stream Identifier` and the expected timestamp. In each loop, it should check all the enabled streams, and then transmit those which are required according to its timestamp and sampling interval. If a following `Start Stream` is received for the same resource, it must overwrite it, that is, update its interval, or change the Stream Identifier if necessary (depending if the message comes only with Stream Id or Stream Id and Resource).

### Streaming Events

To send events over a stream, it is usually required to detect the event or the change. It is completely dependent on the use case, so, the firmware must be prepared to detect and transmit the information when it is required. In the following example, the same `heading` resource defined before is transmitted if it is detected an angle difference greater than one degree. To to this, the [Thinger.io Arduino Library](https://github.com/thinger-io/Arduino-Library) defines a method to stream a given resource. This method should take into account if this resource Stream is enabled, that is, it contains a `Stream Identifier`, otherwise the message must not be sent (as there is no anyone listening for it).

```cpp
void setup(){
  thing["heading"] >> [](pson& out){     
    out = getHeading();
  };
}

float previousHeading = 0;
void loop() {
  thing.handle();
  float currentHeading = getHeading();
  if(abs(currentHeading-previousHeading)>=1.0f){
    thing.stream("heading");
    previousHeading=currentHeading;
  }
}
```

![Example ESP8266 running IOTMP for sending heading when there is a subtle difference.](https://2903141702-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MD0ZpP6y0GVFrIV2ASe%2F-MDBkrqb09rAs1PxlTDZ%2F-MDBltrzrCjpEm9UcnT1%2Fimage.png?alt=media\&token=7e51e89f-af38-4a98-bd11-a4fcb3e4c6cb)

## Server Implementation notes

### HTTP/REST API Integration

As described before, a typical scenario where a `Start Stream` is required is when a new `Websocket`or `Server Sent Event` is opened over the server to start listening over a resource, for example, to feed a real-time dashboard. In [Thinger.io](https://Thinger.io) implementation, a device resource endpoint is defined as follow:

```
https://.../v3/users/alvarolb/devices/smart_irrigation/resources/temp
```

In such implementation it is possible to open a `Websocket` over the resource, so the endpoint is now:

```
wss://.../v3/users/alvarolb/devices/smart_irrigation/resources/temp
```

Opening such `Websocket` will listen for events on the `temp` resource, that is, changes that are detected by device and transmitted to the `Websocket`. But it is possible to open also the resource as following:

```
wss://.../v3/users/alvarolb/devices/smart_irrigation/resources/temp?interval=5
```

In such case, the `Websocket` will start receiving the information every 5 seconds. Moreover, it is possible to open an endpoint without defining a resource:

```
wss://.../v3/users/alvarolb/devices/smart_irrigation/resources
```

And then transmit over the `Websocket` different frames with expected resources and their intervals:

```javascript
{"resource": "temp", "interval": 5, "enabled" : true}
{"resource": "humidity", "interval": 5, "enabled" : true}
```

### MQTT Integration

~~TBD~~

~~devices/smart\_irrigation/resources/temp?interval=5~~

### Different streaming intervals

There can be conflicts if different sources requires different sampling intervals for the same resource. For example, a data storage configured to write a resource every minute, and a dashboard requiring real-time data from from the same resource every 10 seconds. There are different workarounds here that can be implemented in the server, as the client should remain as simple as possible:

* Forbid multiple`Start Stream` over a resource with different sampling intervals, and force the client to define different resources for each data-sink/interval, i.e., one for the data storage, and another for the dashboard, so there are not conflicts as the resource are different.
* Allow multiple `Start Stream` over a resource with different sampling interval, but ignore newer sampling intervals.
* Allow multiple `Start Stream` and dynamically compute the Greatest Common Divisor (GCD) between stream requests, reconfigure the device stream interval, and provide data to each source at the specified interval. This is the most complex approach, and is the one implemented in [Thinger.io](https://thinger.io). In this case, if the dashboard A requires data every 6 seconds, and dashboard B every 9 seconds, the server computes the GCD of both sources (3) and configures the device to transmit every 3 seconds.  The dashboard A will get data every 6 seconds, and B every 9. If one source is added or removed, the process is repeated to accommodate to the new situation.

### Sampling intervals and events subscriptions&#x20;

There is another use case where a data sink may require to receive a resource data periodically, i.e., to store it, and other data sink require only events, i.e., raised events by the device. ~~TBD~~


# Stop Stream

Stops a long-lived stream previously started

## Request

A stop stream request is usually sent by the server to stop receiving data from a connected client, i.e., when an existing stream has been closed.

### Header

<table><thead><tr><th width="186.33333333333331">Field</th><th width="105">Value</th><th>Description</th></tr></thead><tbody><tr><td><strong>Message Type</strong></td><td>0x09</td><td>Start Stream</td></tr><tr><td><strong>Message Size</strong></td><td><a href="/iotmp/definitions#varint">varint</a></td><td>Remaining Message Length</td></tr></tbody></table>

### Body

<table><thead><tr><th width="153">Field</th><th width="112">Identifier</th><th width="109">Type</th><th width="120">Mandatory</th><th>Value</th></tr></thead><tbody><tr><td><strong>Stream Id</strong></td><td>0x01</td><td><a href="/iotmp/definitions#varint">varint</a></td><td>Yes</td><td>Stream identifier to stop.</td></tr><tr><td><strong>Parameters</strong></td><td>0x02</td><td><a href="/iotmp/definitions#any">any</a></td><td>No</td><td></td></tr><tr><td><strong>Payload</strong></td><td>0x03</td><td><a href="/iotmp/definitions#any">any</a></td><td>No</td><td></td></tr></tbody></table>

## Response

| Message                            | Description                                                          |
| ---------------------------------- | -------------------------------------------------------------------- |
| [**Ok**](/iotmp/messages/ok)       | Should answer with an OK if it is able to stop the requested stream. |
| [**Error**](/iotmp/messages/error) | Should answer with an error if the target stream is not enabled.     |


# Stream Data

Message used to represent information transmitted over a stream.

## Message

### Header

| Field            | Value                               | Description              |
| ---------------- | ----------------------------------- | ------------------------ |
| **Message Type** | 0x0A                                | Stream Data              |
| **Message Size** | [varint](/iotmp/definitions#varint) | Remaining Message Length |

### Body

| Field          | Identifier | Type                                | Mandatory | Value                                                                                                                     |
| -------------- | ---------- | ----------------------------------- | --------- | ------------------------------------------------------------------------------------------------------------------------- |
| **Stream Id**  | 0x01       | [varint](/iotmp/definitions#varint) | Yes       | Stream identifier.                                                                                                        |
| **Parameters** | 0x02       | [any](/iotmp/definitions#any)       | No        | User-defined parameters associated to the information that can be used for signaling the payload, provide a context, etc. |
| **Payload**    | 0x03       | [any](/iotmp/definitions#any)       | No        | Current stream payload.                                                                                                   |

## Thinger.io

In thinger.io, clients streaming resources establish the following parameters to help the server identifying the source of information while sending data.

### Parameters

| Name                | Value | Description                                               |
| ------------------- | ----- | --------------------------------------------------------- |
| **Resource Input**  | 0x01  | Stream data represents the input of the stream resource.  |
| **Resource Output** | 0x02  | Stream data represents the output of the stream resource. |


# Definitions

## Data Types

### Varint

To understand IOTMP, you first need to understand *varints*. Varints are a method of serializing integers using one or more bytes. Smaller numbers take a smaller number of bytes.

Each byte in a varint, except the last byte, has the *most significant bit* (msb) set – this indicates that there are further bytes to come. The lower 7 bits of each byte are used to store the two's complement representation of the number in groups of 7 bits, **least significant group first**.

So, for example, here is the number 1 – it's a single byte, so the msb is not set:

```
0000 0001
```

And here is 300 – this is a bit more complicated:

```
1010 1100 0000 0010
```

How do you figure out that this is 300? First you drop the msb from each byte, as this is just there to tell us whether we've reached the end of the number (as you can see, it's set in the first byte as there is more than one byte in the varint):

```
 1010 1100 0000 0010
→ 010 1100  000 0010
```

You reverse the two groups of 7 bits because, as you remember, varints store numbers with the least significant group first. Then you concatenate them to get your final value:

```
000 0010  010 1100
→  000 0010 ++ 010 1100
→  100101100
→  256 + 32 + 8 + 4 = 300
```

### Any

In a message field value, `Any`, means that it can store anything. It can be encoded with the default [wire-types](/iotmp/message-structure/message-body) supported by IOTMP, or also encoded with the mechanism negotiated in the [Connect](/iotmp/messages/connect) message.

## Stream

A "stream" is an independent, bidirectional sequence of messages exchanged between the client and server within an IOTMP connection. Streams have several important characteristics:

* A single IOTMP connection can contain multiple concurrently open streams, with either endpoint interleaving frames from multiple streams.
* Streams can be established and used unilaterally or shared by either the client or server.
* Streams can be closed by either endpoint, or closed automatically depending on the message type.
* The order in which frames are sent on a stream is significant. Recipients process frames in the order they are received.
* Streams are identified by an integer. Stream identifiers are assigned to streams by the endpoint initiating the stream.

### Stream Identifier

## Resources

Recurs

### Field Identifier

### Resource Parameters


# Features

Thinger.io is the main promoter of the IOTMP protocol, as the Thinger.io platform has been designed with IOTMP as its core. This way, it offers a set of funcionasteis over remote IoT devices that are not possible with other IoT protocols. Some of the features are

<table data-view="cards"><thead><tr><th></th><th></th><th></th></tr></thead><tbody><tr><td></td><td></td><td><strong>Remote Filesystem</strong></td></tr><tr><td></td><td></td><td><strong>Remote Terminal</strong></td></tr><tr><td></td><td></td><td><strong>Remote Proxy</strong></td></tr><tr><td><strong>Device Configuration</strong></td><td></td><td></td></tr><tr><td><strong>MQTT Interoperability</strong></td><td></td><td></td></tr><tr><td><strong>Server Events</strong></td><td></td><td></td></tr><tr><td>Automatic REST API</td><td></td><td></td></tr><tr><td></td><td>Device Configuration</td><td></td></tr></tbody></table>


# REST API


# Remote Desktop


# Remote Proxies


# Remote Webservices


# Remote Terminal


# Remote Filesystem


# MQTT Interoperability


# Implemenation


# Messages


# Connect

Message to authenticate and negotiate connection parameters.

## Request

In the Thinger.io server implementation, there are the following default values and parameters:

<table><thead><tr><th width="204">Field</th><th width="112">Type<select><option value="2e69e44032ab4c89805b1ffe1685475a" label="varint" color="blue"></option><option value="b75cdae3e1134c06a833b7d7ddd4bbeb" label="any" color="blue"></option></select></th><th width="105" data-type="checkbox">Required</th><th>Value</th></tr></thead><tbody><tr><td><strong>Stream Id</strong><br>(0x01)</td><td><span data-option="2e69e44032ab4c89805b1ffe1685475a">varint</span></td><td>true</td><td>Connect stream identifier</td></tr><tr><td><p><strong>Parameters</strong> </p><p>(0x02)</p></td><td><span data-option="2e69e44032ab4c89805b1ffe1685475a">varint</span></td><td>false</td><td><p><strong>1</strong>: Payload contains an array with username, device, and credentials</p><p></p><p><strong>2</strong>: Auth Info contains a string with a token.</p><p></p><p><strong>Default: 1 (Username/Password Credentials)</strong></p></td></tr><tr><td><strong>Payload</strong><br><strong>(0x03)</strong></td><td><span data-option="b75cdae3e1134c06a833b7d7ddd4bbeb">any</span></td><td>true</td><td><p><strong>Parameter with value 1:</strong></p><p>Auth Info contains an array with ["username", "device", "credential"].</p><p></p><p><strong>Parameter with value 2:</strong><br>Auth Info contains a string with a token.</p></td></tr><tr><td><strong>Keep Alive Interval</strong><br><strong>(0x04)</strong></td><td><span data-option="2e69e44032ab4c89805b1ffe1685475a">varint</span></td><td>false</td><td><p>Number of expected seconds between keep alives.</p><p></p><p><strong>Default: 60 seconds</strong></p></td></tr><tr><td><strong>Payload Encoding</strong><br><strong>(0x05)</strong></td><td><span data-option="b75cdae3e1134c06a833b7d7ddd4bbeb">any</span></td><td>false</td><td><p>0x00: Reserved  </p><p>0x01: PSON</p><p>0x02: JSON</p><p>0x03: MessagePack</p><p>0x04: BSON</p><p>0x05: CBOR</p><p>0x06: UBJSON</p><p><strong>Default: 0x01 (PSON)</strong></p></td></tr></tbody></table>

## Response

### OK

<table data-header-hidden><thead><tr><th width="163.33333333333331">Field</th><th>Type</th><th>Description</th></tr></thead><tbody><tr><td>Field</td><td>Type</td><td>Description</td></tr><tr><td>OK</td><td><p>Parameters: None</p><p>Payload: None</p></td><td>The server agree the Connect </td></tr><tr><td></td><td></td><td></td></tr><tr><td></td><td></td><td></td></tr><tr><td></td><td></td><td></td></tr></tbody></table>

### Error

<table><thead><tr><th>Error</th><th width="141" align="center">Parameter</th><th width="157">Payload</th><th>Description</th></tr></thead><tbody><tr><td><strong>Unknown</strong></td><td align="center">N/A</td><td>N/A</td><td>The server refuses the connection.</td></tr><tr><td><strong>Redirect</strong></td><td align="center">1</td><td>Target server redirect</td><td><p>The server requires the device to connect to another host, i.e., for load balancing, route to a nearest host, etc.</p><p><strong>Examples:</strong></p><p>iotmp://newserver.io</p><p>iotmps://newserver.io</p></td></tr><tr><td><strong>Bad Credentials</strong></td><td align="center">2</td><td></td><td>The provided credentials are not valid in the server.</td></tr><tr><td><strong>Invalid Keep Alive</strong></td><td align="center">3</td><td></td><td>The negotiated keep-alive is incorrect.</td></tr><tr><td><strong>Bad Encoding</strong></td><td align="center">4</td><td></td><td>The negotiated encoding is not supported by the server.</td></tr></tbody></table>


# SDKs


# Linux IOTMP


# Arduino IOTMP


# NodeJS IOTMP


# Web IOTMP


# Python SDK

Looking for help!