Getting a sensor to publish to an MQTT broker takes a few lines of code. Getting a whole house of sensors, switches and scripts to talk over MQTT without stale values, lost commands and devices that look online when they are not takes a little more thought. Four features decide most of it: topic structure, Quality of Service (QoS), retained messages and the Last Will. This guide explains each one with references to the MQTT 3.1.1 specification and the Mosquitto manual, and shows how to test them with mosquitto_pub and mosquitto_sub.

Topics: a hierarchy you design yourself

An MQTT topic is a string split into levels with /, for example home/livingroom/temperature. The Mosquitto manual points out that there is no need to create a topic beforehand: publishing to it is enough. That freedom is also the main risk, because nothing stops one script from writing to home/LivingRoom/temp and another from reading home/livingroom/temperature. Topic names are case sensitive, so these are two different topics.

A few rules that save trouble later:

  • Go from general to specific: home/<room>/<device>/<value> or <site>/<device>/<value>. This makes wildcards useful.
  • Use lowercase and no spaces. It avoids case mistakes and quoting problems in shell commands.
  • Do not start topics with /. A leading slash creates an empty first level. The Mosquitto manual notes that zero-length levels are valid but lead to "slightly non-obvious behaviour".
  • Do not use topics beginning with $. The specification reserves them for the broker. $SYS/ is the common prefix for broker statistics, and a subscription to # does not receive anything from topics that begin with $.
  • Separate state from commands. For example, a relay publishes its state on home/garage/door/state and listens for commands on home/garage/door/set. Zigbee2MQTT, for example, uses a /set suffix for commands.

Wildcards

Subscriptions can use two wildcards:

  • + matches exactly one level: home/+/temperature receives home/kitchen/temperature and home/bedroom/temperature.
  • # matches all remaining levels and must be the last character: home/# receives everything under home.

Wildcards work only in subscriptions. You always publish to a specific topic.

QoS: how hard delivery is attempted

MQTT has three QoS levels. The Mosquitto manual summarises them like this:

  • QoS 0: the message is delivered at most once, with no confirmation.
  • QoS 1: the message is delivered at least once, with confirmation required. Duplicates are possible.
  • QoS 2: the message is delivered exactly once, using a four-step handshake.

Higher levels are more reliable but add latency and traffic. An important detail: QoS applies separately to each hop. The publisher chooses the QoS towards the broker, and each subscriber chooses the maximum QoS it wants to receive. If a sensor publishes at QoS 2 and a dashboard subscribes at QoS 0, the dashboard gets the message at QoS 0.

Practical choices:

  • Frequent sensor readings: QoS 0. If one reading is lost, the next one arrives shortly.
  • Commands and events that must not be lost (open the valve, doorbell pressed): QoS 1, and make the receiver tolerate duplicates. Sending "ON" twice to a relay does no harm; toggling twice does. Prefer absolute commands such as ON and OFF over TOGGLE.
  • QoS 2 is rarely needed in home automation, and some small client libraries do not support it, so check yours before relying on it.

QoS does not help when a client is offline, unless it uses a persistent session. According to the Mosquitto manual, a client that connects with "clean session" set to false keeps its subscriptions after it disconnects, and QoS 1 and 2 messages are stored for it until it reconnects. Many clients connect with a clean session by default; mosquitto_sub, for example, needs the -c option to disable it.

Retained messages: the last known value

A retained message is kept by the broker after it has been delivered. When a new client subscribes to a matching topic, it receives the retained message immediately. The Mosquitto manual calls this a "last known good" mechanism: without it, a dashboard that starts up has to wait until a sensor that reports every 15 minutes publishes again.

Use retain for state: temperature, the position of a switch, whether a device is online. Do not use it for events or commands. A retained ON on a command topic is delivered again to the device every time it reconnects, which can switch something on long after anyone asked for it.

To remove a retained message, publish an empty, retained message to the same topic. Mosquitto's -n option sends a zero-length payload:

mosquitto_pub -h BROKER -t 'home/garage/door/set' -r -n

When you inspect a busy broker, mosquitto_sub -R hides retained (stale) messages, and --retained-only shows only them.

Last Will: telling everyone a device has gone

A client can give the broker a will when it connects: a topic, a payload, a QoS and a retain flag. The broker publishes the will if the connection ends without a proper DISCONNECT packet. The MQTT 3.1.1 specification lists cases such as a network failure, a client that does not communicate within its keep-alive time, or a client that closes the connection without sending DISCONNECT. If the client disconnects cleanly, the will is discarded.

The keep-alive time matters here. The specification says the broker must close the connection if it receives nothing from the client within one and a half times the keep-alive period. With a 60-second keep-alive, an unplugged device is therefore reported offline within about 90 seconds.

A common pattern for device availability is:

  1. On connect, set a will: topic home/garage/door/status, payload offline, retained.
  2. Right after connecting, publish online to the same topic, retained.
  3. Before a planned shutdown, publish offline yourself, because a clean disconnect does not trigger the will.

Any subscriber, including one that starts later, then sees the current status thanks to the retain flag. Zigbee2MQTT's zigbee2mqtt/bridge/state topic follows the same idea: according to its documentation, a retained online is published on startup and offline right before the bridge stops.

Try it with the Mosquitto command-line tools

Open a terminal and watch everything under home:

mosquitto_sub -h BROKER -u USER -P 'PASSWORD' -t 'home/#' -v

-v prints the topic together with the payload. In a second terminal, simulate a device with a will:

mosquitto_sub -h BROKER -u USER -P 'PASSWORD' -t 'home/test/set' -k 10 \
  --will-topic 'home/test/status' --will-payload 'offline' --will-retain

Publish its online status:

mosquitto_pub -h BROKER -u USER -P 'PASSWORD' -t 'home/test/status' -m 'online' -r

Now end the simulated device without a DISCONNECT packet, for example with kill -9 on its process. The operating system closes the connection, the broker notices that it closed without DISCONNECT, and the first terminal shows home/test/status offline. Subscribe again from a third terminal and you get offline immediately, because the will was retained.

The keep-alive set with -k 10 matters when the connection disappears silently, for example when a device loses power or Wi-Fi and nothing closes the connection. Then the broker waits up to one and a half times the keep-alive period, here about 15 seconds, before it publishes the will.

If you stop the device with Ctrl+C instead, mosquitto_sub disconnects cleanly and no will is sent. That is the expected behaviour, and the reason for step 3 above.

A short checklist

  • Topics: lowercase, no leading /, no $, general to specific, state and commands separated.
  • QoS 0 for frequent readings, QoS 1 for commands and important events, receivers tolerant of duplicates.
  • Retain state and availability, never commands or one-off events.
  • Every long-running device sets a retained offline will and publishes a retained online after connecting.
  • Choose a keep-alive that matches how quickly you need to know a device has gone.