ThingsBoard

MQTT IoT

Synopsis

Creates a ThingsBoard target that publishes records as telemetry through the ThingsBoard gateway MQTT API. One connection sends the data of many devices. ThingsBoard creates a device that does not exist yet.

The target signs in with the access token of a gateway device in ThingsBoard (see Before You Start).

Schema

- name: <string>
  description: <string>
  type: thingsboard
  pipelines: <pipeline[]>
  status: <boolean>
  properties:
    endpoint: <string>
    port: <integer>
    access_token: <string>
    client_id: <string>
    protocol_version: <string>
    keep_alive: <integer>
    clean_session: <boolean>
    auto_reconnect: <boolean>
    device_name: <string>
    device_name_missing_field: <string>
    device_name_fallback: <string>
    device_name_default_value: <string>
    device_profile: <string>
    fields: <string[]>
    qos: <integer>
    batch_size: <integer>
    max_payload_size_kb: <integer>
    timeout: <integer>
    max_messages_per_second: <integer>
    field_format: <string>
    max_retries: <integer>
    retry_delay: <integer>
    disable_retries: <boolean>
    tls:
      status: <boolean>
      verify: <boolean>
      ca_name: <string>
      cert_name: <string>
      key_name: <string>
      passphrase: <string>
      server_name: <string>
      min_tls_version: <string>
      max_tls_version: <string>
    debug:
      status: <boolean>
      dont_send_logs: <boolean>

Configuration

The following fields are used to define the target:

Field	Required	Default	Description
`name`	Y		Target name
`description`	N	-	Optional description
`type`	Y		Must be `thingsboard`
`pipelines`	N	-	Optional post-processor pipelines
`status`	N	`true`	Enable/disable the target

Connection

Field	Required	Default	Description
`endpoint`	Y	-	The host name or IP address of the ThingsBoard server. Write the address alone, with no scheme, port or path. In the console: Address
`port`	N	`1883`, or `8883` with TLS on	The MQTT port of the server. In the console: Port
`access_token`	Y	-	The access token of a gateway device in ThingsBoard (see Before You Start). The target signs in with it as the MQTT user name, and sends no password. Plain text, an environment variable or a secret. In the console: Access token
`client_id`	N	Generated	The MQTT client ID. In the console: Client ID
`protocol_version`	N	`"3.1.1"`	`"3.1.1"` or `"5"`, written as text in quotes. In the console: Protocol version
`keep_alive`	N	`60`	How often the target pings the server while the connection is idle, in seconds. In the console: Keep-alive interval
`clean_session`	N	`true`	As for the MQTT target. In the console: Clean session
`auto_reconnect`	N	`true`	Connects again after a lost connection. Not in the console.

Device Settings

Field	Required	Default	Description
`device_name`	N	The name of the device that sent the record	The ThingsBoard device that each record goes to. A template with record fields in braces, such as `{host.name}` (see How Device Names Work). In the console: Device name
`device_name_missing_field`	N	`drop`	What happens to a record that lacks a field the device name uses: `drop`, `fallback` or `default_value`. In the console: Missing device name field (Drop the record, Use a fallback device name, Use a default value), shown once the device name holds a field in braces
`device_name_fallback`	Y, with `fallback`	-	The device that such a record goes to. It cannot hold a field in braces. In the console: Fallback device name
`device_name_default_value`	N	`unknown`	The text that takes the place of a missing field, with `default_value`. In the console: Default value
`device_profile`	N	-	The device profile that ThingsBoard gives a device it creates (see How Device Profiles Work). Empty leaves ThingsBoard's default profile. In the console: Device profile
`fields`	N	All fields	The fields to send. An entry also takes every field inside it (see What Each Message Holds). In the console: Fields to send

ThingsBoard Settings

Field	Required	Default	Description
`qos`	N	`1`	`0` or `1`. ThingsBoard does not take QoS 2. In the console: Quality of service (At most once (0), At least once (1))
`batch_size`	N	`1000`	The most records in one batch. In the console: Batch size
`max_payload_size_kb`	N	`64`	The largest message the target sends, in KB. 64 KB is the default limit of ThingsBoard's MQTT transport. Raise it only when the server takes larger messages. `1` or more. In the console: Maximum message size
`timeout`	N	`30`	The seconds that the connection, and each publish, may take. In the console: Timeout
`max_messages_per_second`	N	No limit	The most messages the target publishes in one second, all its workers together. `0` or no value means no limit. In the console: Maximum messages per second
`field_format`	N	-	The format each record is converted to before the target publishes it. See applicable Normalization section. In the console: Field format
`max_retries`	N	`3`	How often the target tries a failed message again. In the console: Maximum retries
`retry_delay`	N	`1`	The longest wait before the first retry, in seconds. In the console: Retry delay
`disable_retries`	N	`false`	`true` makes one attempt for each message, with no retry and no wait. The director still sends a failed batch again later, as it does for any failed send. In the console: Disable retries

The target sets no last will, so the MQTT target's will settings have no effect here.

TLS

Field	Required	Default	Description
`tls.status`	N	`false`	Turns TLS on. The server must offer MQTT over TLS. In the console: the TLS switch
`tls.verify`	N	`true`	Checks the server's certificate. In the console: Certificate verification, with TLS on
`tls.ca_name`	N	-	The CA that issued the server's certificate, for a certificate that a public CA did not sign. Empty uses the system trust store. In the console: CA certificate, with TLS on
`tls.cert_name`, `tls.key_name`	N	-	A client certificate and its private key, for a server that asks for one. Give both or neither. Not in the console.
`tls.passphrase`, `tls.server_name`, `tls.min_tls_version`, `tls.max_tls_version`	N	-	As for the MQTT target. Not in the console.

The target signs in with the access token, so it needs no client certificate to sign in.

Scheduling

See Scheduling and Pool Behavior for interval and cron fields shared by all targets.

Debug Options

Field	Required	Default	Description
`debug.status`	N	`false`	Enable debug logging
`debug.dont_send_logs`	N	`false`	Process logs but don't send to target (testing)

Details

Before You Start

In ThingsBoard, add a device with the gateway option turned on, and copy its access token. The target signs in with that token. The token of a device that is not a gateway does not work.
The target sends to the server's MQTT transport, on port 1883 by default. With TLS on, the server must offer MQTT over TLS, by default on port 8883.

How Device Names Work

Without device_name, each record goes to the ThingsBoard device that has the name of the device that sent the record. A record from a device with no name is dropped.
With device_name, each {field.path} in the template takes the value of that field in the record. The value is used as it is, so it can hold a /.
A value is missing when the record does not have the field, or when the field holds null, an empty text, an object or a list. Then device_name_missing_field decides: drop drops the record, fallback sends it to device_name_fallback, and default_value puts device_name_default_value where the value would go.
A record whose device name comes out empty is dropped.
ThingsBoard creates each device the first time it sees its name.

What Each Message Holds

Each record becomes one reading of its device: the time of the record, in milliseconds, and the fields of the record as its values.
A field inside an object is sent under its path, with dots between the names, such as host.name. Lists, text, numbers and true/false values are sent as they are. A field that is null is left out, and so is @timestamp, because the reading carries the record's time already.
A record that is not a JSON object is sent as one value, message, that holds its text.
fields limits what is sent. An entry takes the field of that name and every field inside it: cpu takes cpu.user and cpu.idle, but not cpu_total. A record that has no value left to send is dropped.
Large numbers keep every digit.
One message holds the readings of many devices. The target starts a new message when the next reading would take it over max_payload_size_kb. A record that is larger than that on its own is dropped.

How Device Profiles Work

With device_profile, the target tells ThingsBoard the profile of each device before it sends that device's first reading. ThingsBoard uses the profile when it creates the device. A device that exists already keeps its own profile.
Without device_profile, ThingsBoard creates new devices with its default profile.

Sending Again Adds No Copies

When a message fails, the target sends the whole batch again. ThingsBoard keeps one value for each device, key and time, so a message that arrives twice adds no copies.
The same rule applies to different records. When two records go to the same device in the same millisecond, ThingsBoard keeps only the last value of each key they share. So the target suits telemetry, which has one reading per device and moment. Log lines from a source that stamps whole seconds, sent to one device, overwrite each other's shared keys.

Dropped Records

A dropped record counts as dropped in the target's statistics, and the rest of the batch is still sent. With debug.status on, the log says how many records were dropped and why.

Examples

Basic

The simplest form: each record goes to the device it came from:

targets:
  - name: thingsboard
    type: thingsboard
    properties:
      endpoint: "thingsboard.example.com"
      access_token: "$secret{thingsboard-gateway-token}"

Device Names from a Record Field, over TLS

Device names from a record field, a device profile and a few fields, over TLS:

targets:
  - name: thingsboard_sensors
    type: thingsboard
    properties:
      endpoint: "thingsboard.example.com"
      access_token: "$secret{thingsboard-gateway-token}"
      device_name: "{sensor.id}"
      device_name_missing_field: fallback
      device_name_fallback: "unknown-sensor"
      device_profile: "sensor-profile"
      fields:
        - temperature
        - humidity
        - battery
      tls:
        status: true
        ca_name: "$path{certs/thingsboard-ca.crt}"

Larger Messages

A server whose MQTT transport takes messages of up to 256 KB:

targets:
  - name: thingsboard_large
    type: thingsboard
    properties:
      endpoint: "10.0.0.30"
      access_token: "${THINGSBOARD_ACCESS_TOKEN}"
      max_payload_size_kb: 256
      qos: 0

Troubleshooting

Configuration the target refuses

Cause: the director refuses a configuration it cannot use. The message names the setting:

endpoint is required for thingsboard target: use the host name or the IP address of the ThingsBoard MQTT server, and endpoint must be a host name or an IP address alone: leave out the scheme, the port and the path
access_token is required for thingsboard target: use the access token of the gateway device
port must be between 1 and 65535
qos must be 0 or 1: ThingsBoard does not take QoS 2
protocol_version must be "3.1.1" or "5"
max_messages_per_second must be 0 or more
max_payload_size_kb must be 1 or more
both tls.cert_name and tls.key_name must be provided together or omitted together
device_name_fallback is required when device_name_missing_field is "fallback", device_name_fallback "…" is used as it is, so it cannot hold a placeholder, and device_name_missing_field must be "drop", "fallback" or "default_value", not "…"
device_name is not valid: …, followed by what is wrong with the template: a brace without its partner, an empty placeholder {}, or a NUL character.

Fix: correct the setting the message names.

"failed to announce the device"

Cause: ThingsBoard did not take the message that tells it the profile of a device. The target sends that message only with device_profile set.

Fix: check that the access token belongs to a gateway device, and read the rest of the message for the reason. The director sends the batch again.

Messages in the debug log

With debug.status on, a batch that dropped records logs Dropped N of M records in this batch:, with the reasons:

had no device name: see How Device Names Work.
had no value to send: fields left nothing to send, or every value was null.
were larger than the … bytes that max_payload_size_kb allows: raise max_payload_size_kb if the server takes larger messages.

Synopsis​

Schema​

Configuration​

Connection​

Device Settings​

ThingsBoard Settings​

TLS​

Scheduling​

Debug Options​

Details​

Before You Start​

How Device Names Work​

What Each Message Holds​

How Device Profiles Work​

Sending Again Adds No Copies​

Dropped Records​

Examples​

Basic​

Device Names from a Record Field, over TLS​

Larger Messages​

Troubleshooting​

Configuration the target refuses​

"failed to announce the device"​

Messages in the debug log​