Retained messages is used to store the “Last Good Message” on a topic. Usually, if an MQTT client subscribes to a topic on a broker it will not recieve any of the messages published on it before subscription. If a client publishes a message to a topic with the retain flag set to True then the broker will save that message as the “Last Good Message” on that topic. This message will be received by any client who subscribes to that topic.
This is a part of a series explaining the different concepts of MQTT. If you are new to MQTT, please read the Fundaments of MQTT first.
How MQTT Retained Messages work
Lets take 4 MQTT clients, 3 that subscribe and the 4th that publishes with retained message flag. You can follow these flow of events in order:
- Client 1 publishes the message “Online” to a topic “device1/status” with retain flag set to True.
- Broker stores that message on the topic “device1/status” as the Last Good Message.
- Client 2 subscribes to the topic “device1/status”. As soon as it subscribes it will receive the message “Online”.
- Client 1 publishes the message “Offline” to the same topic with retain flag set to True.The broker replaces the message “Online” with “Offline” as the Last Good Message. Client 2 receives the message as its already subscribed.
- Client 3 subscribes to the same topic and receives the message “Offline” as soon as it subscribes.
- Client 1 publishes the message “Online” to the same topic but this time with retain flag set to False. Client 2 & 3 recieves the message as its already subscribed.
- Client 4 subscribes to the topic. This time client 4 will receive the message “Offline” as it was the last message sent with retain flag set to True. It will not recieve the message “Online” as the broker did not store it.
I hope this flow of events helps you understand Retained Messages in MQTT.
How does QoS affect Retained Messages?
Short answer: It doesn’t. A client can publish or subscribe with any QoS level, the retained message will only get printed if the message is published with retain flag set to true.
If a client publishes with QoS 1 or 2 and a client has subscibed with QoS 1 or 2, then all the messages will be saved including the retained messages
How to delete a retained message?
To delete a retained message from a topic, a client must either replace it with another message with retain flag set to True OR a client must publish a blank message with retain flag set to True on that topic. This is the only way to delete a retained message.
When to use Retained messages?
Use retained message to update the status of a device on a topic. For example:
“device1/status” can be Offline/Online
or
“kitchenlights/status” can be “On/Off”.
Retained messages can be combined with the Last Will & Testament feature in MQTT. See more about that here:
MQTT Last Will And Testament (Explained with Example)
Example Using Paho-MQTT Python Client
This example assumes you know the basics of Paho-MQTT. If not see this post:
MQTT Python With Paho-MQTT Client (Step-by-Step Guide With Examples)
To publish a retained message on Paho simply set the retain flag to True while publishing:publish(topic, payload=None, qos=0, retain=True)
To test this make sure the client subscribes AFTER the retained message is published.
Hi Maulin,
Thanks for the explanation.
I am trying the same but everything works fine on single node but when moved to a cluster the retain messages are not same.
It seems they are dependent on nodes (on which i connect, the last retained message on that node)
Can you help me in resolving the issue.
Can you explain your architecture in more detail? Where is your broker running?
The retained message is stored on the broker, not on the node.
Thanks for the reply. We have a three node rabbitmq cluster with queue mirroring enabled for all queues. We have one microservices that pushes a message (status) on a topic (with retained flag) with specific routing key. Now their may be no subscriber at the time when publisher publishes. Queues are only created when a subscriber subscribes. Now when the subscriber(e.g mobile client) subcribes it should get the last published messages with retain flag. But during our testing we found that retain mesasge are not same for all node, its the last published message (with retained flag) for that node.… Read more »
The broker only stores the last message published with retain flag on a topic. If you send another message with retain flag to the same topic the previous message will get replaces.
Can you tell me more about what you are trying to achieve? You mentioned queues so I am a bit confused.
As Tushar explained , We have 3 node rabbitmq cluster and we are able to publish the ratined messages on broker. But the problem is the consumer devices could directly connect with any of the cluster nodes and in that case they should get the retained messages if topic is same. But what happening is , only the node on which retained message saved would serve the consumer with correct one. The other 2 nodes are not returning the correct message as retained message do not get replicated over the cluster nodes. Is there any solution for the same ?… Read more »
It is mentioned in the rabbitMQT docs that retained messages are node-local in the cluster. You can possibly have a daemon running in the background somewhere that works like a trigger and is subscribed to all topics on all 3 nodes. When a message with the retain flag is published on one node, this daemon can publish this message to the same topic on other nodes. Alternatively, you can use rabbitmq’s disk store configuration to store retained messages on the disk. You can run rsync on all 3 nodes to keep the disk in sync. These should work theoretically. I… Read more »
Hi, Maulin.
Thank you for the explanation and example.
It seems that the last sentence of Step 5 (“The broker replaces the message “Online” with “Offline” as the Last Good Message.”) really belongs in Step 4. Is it correct that the broker would have replaced the Last Good Message as soon as Client 1 published it (Step 4), not after Client 3 subscribes (Step 5)?
Thanks for the suggestion! Fixed it 🙂