MQTT Persistent Sessions & Message Queuing Explained

MQTT does not behave like a classical message queue, but under specific conditions it can queue messages for clients. That capability is one of the protocol’s most important reliability features for devices on unreliable networks, and it depends entirely on a concept called the persistent session. Understanding when a session is persistent, what the broker stores in it, and what survives across a disconnect is the difference between an MQTT deployment that loses data when devices drop offline and one that resumes seamlessly when they reconnect.

This article explains what a persistent session is, what the broker and the client each store, how the clean session flag controls everything, how the session present flag in the CONNACK tells a reconnecting client what to expect, and how MQTT 5 refines this model into Clean Start plus a session expiry interval. It is a technical reference; related mechanisms such as Quality of Service, the connection handshake, and Last Will and Testament have their own dedicated articles in this category.

Sessions at a glance

Property	Persistent session	Clean session
Flag in CONNECT	cleanSession = false (MQTT 3) / cleanStart = false + nonzero session expiry (MQTT 5)	cleanSession = true (MQTT 3) / cleanStart = true (MQTT 5)
Subscriptions kept across disconnect	Yes	No
QoS 1 and 2 messages queued while offline	Yes	No
In-flight QoS 1 and 2 state preserved	Yes	No
Session present flag in CONNACK on reconnect	True if prior session exists	Always false
Identified by	ClientId	ClientId

The problem persistent sessions solve

To receive messages from a broker, a client connects and creates subscriptions to the topics it cares about. If the connection between the client and the broker is interrupted during a non-persistent session, those subscriptions are gone, and the client must subscribe again when it reconnects.

For a constrained device with limited resources, re-subscribing every time the connection blips is a real burden, in both bytes on the wire and round trips before the device is operational again. A device on a weak cellular link might disconnect and reconnect many times in an hour; if each reconnect costs several SUBSCRIBE messages and a wait for the SUBACKs before any data flows, the protocol’s lightweight design has been undone by the network conditions it was built for.

Persistent sessions exist to fix this. When a client requests a persistent session at connection time, the broker stores everything relevant to that client. On reconnect with the same ClientId, all of that stored state is restored automatically, so the client picks up where it left off without re-subscribing and without losing the messages that arrived in its absence.

What the broker stores in a persistent session

While the session is active, even when the client is offline, the broker keeps a specific set of state for that client. When the client reconnects, this state is immediately available again. The persistent session stores:

• The existence of the session itself, even if the client has no subscriptions.
• All of the client’s subscriptions, with their requested QoS levels.
• All messages in a QoS 1 or 2 flow that the client has not yet confirmed.
• All new QoS 1 or 2 messages that the client missed while offline, for topics it had subscribed to.
• All QoS 2 messages received from the client that are not yet fully acknowledged.

The result is that a client reconnecting to a persistent session finds its subscriptions intact, any in-progress QoS handshakes ready to resume, and a queue of QoS 1 and 2 messages waiting to be delivered. The broker is doing the bookkeeping that the client would otherwise have to redo, every time.

Note what is not in this list. The client’s last will message is part of the connection, not the session, and is re-registered on each connect. Retained messages are stored against topics, not against any one client’s session. And QoS 0 messages that arrive while the client is offline are typically not queued for it, though the specification permits brokers to do so optionally, so a client should not rely on receiving QoS 0 messages it missed.

How the clean session flag controls everything

In MQTT 3.1.1, the entire choice between persistent and non-persistent behavior is made through one boolean field in the CONNECT message: the clean session flag. The way it works is direct.

When the client connects with clean session = true, it explicitly does not want a persistent session. The broker discards any existing session state for that ClientId from a previous connection and creates no new persistent state. If this client disconnects for any reason, all subscriptions and queued messages tied to it are lost. This is the right choice for a client that has no need to survive a reconnection, for example a tool publishing a one-off message or a service whose state lives elsewhere.

When the client connects with clean session = false, it asks for a persistent session. The broker preserves any state it already has for that ClientId, or, if none exists, creates a fresh persistent session. From that point on, everything described in the previous section is stored across disconnections until the next time the client connects with clean session = true.

Two consequences of this design are worth being explicit about. First, the flag is checked on every connect, not just the first one. A client that wants to continue a session must keep using clean session = false; a single connect with clean session = true will wipe out the persistent state for that ClientId. Second, the choice is the client’s, not the broker’s. The broker honors what the client asks for; a deployment where session policy is centrally enforced has to do that enforcement at a layer above the protocol, for example by rejecting connections that ask for the wrong setting.

The session present flag on reconnect

A reconnecting client needs to know whether its session is still alive on the broker, because that determines whether it has to re-subscribe and whether it can expect queued messages to arrive. Since MQTT 3.1.1, the broker reports this in the CONNACK using the session present flag.

The flag’s value depends on what the client asked for and what the broker had stored:

• When a client connects with clean session = true, the session present flag is always false, because by definition there is no carried-over session.
• When a client connects with clean session = false, the flag is true if the broker has stored session information for that ClientId, and false if it does not.

This is the protocol-level signal that lets a reconnecting client decide what to do next. A true value tells it the broker still holds its subscriptions and any queued state, so it can resume without re-subscribing. A false value tells it the broker has nothing for it, either because there was no prior persistent session or because it was lost, and it must subscribe again from scratch.

This is the kind of feedback that disciplined client implementations actually act on. Always reading the session present flag, rather than assuming the broker still has your subscriptions, prevents silent subscription drift, where a client thinks it is subscribed to something that the broker has long since forgotten about.

A persistent session across a disconnect, step by step

It helps to trace what actually happens to a session across the moments when it matters: connecting, disconnecting, and reconnecting. Suppose a sensor connects with a persistent session, drops offline for a while, and comes back.

1. First connect. The sensor sends a CONNECT with ClientId sensor-42 and clean session = false. The broker has no record of sensor-42, so it creates a fresh persistent session for that ClientId and replies with a CONNACK whose session present flag is false. The sensor sees the false flag and knows it must subscribe from scratch.
2. Subscribe. The sensor sends a SUBSCRIBE for commands/sensor-42/+ at QoS 1 and receives the SUBACK. The broker stores this subscription as part of the persistent session.
3. Normal operation. The sensor publishes its readings, the broker delivers any matching command messages, and life is uneventful. The session’s state is just the subscriptions and any transient in-flight QoS state.
4. Connection drops. The sensor loses its cellular link without sending a DISCONNECT. The broker eventually detects the loss, either when the TCP layer reports it or via the keep alive timeout. It closes the connection. Crucially, it does not discard the persistent session: it still has sensor-42‘s subscription and is still entitled to receive matching messages.
5. Messages arrive while offline. A controller publishes a message to commands/sensor-42/reboot at QoS 1. The broker matches this against sensor-42‘s stored subscription, sees the client is offline, and queues the message in the session. A few more commands arrive over the next hour; they queue up too. A status update is published to the same topic at QoS 0; the broker typically does not queue this, so the sensor will not see it on reconnect.
6. Reconnect. The sensor’s link returns. It sends a CONNECT with ClientId sensor-42 and clean session = false. The broker finds the stored session for that ClientId and replies with a CONNACK whose session present flag is true. The sensor sees the true flag and knows its subscription is still active; it does not re-subscribe.
7. Delivery resumes. Almost immediately, the broker delivers the queued QoS 1 command messages, one by one, completing the at-least-once handshakes for each. Any QoS 2 messages that had been mid-handshake when the connection dropped resume from where they were, using the stored transaction state. Once the queue is drained, the sensor is back in normal operation.
8. An accidental clean. Now imagine the sensor’s firmware is updated and a bug causes it to reconnect with clean session = true. The CONNACK reports session present = false (as it always does for a clean session). The broker discards the prior persistent session, including any subscription state. The sensor would now have to re-subscribe to receive new commands, and any messages queued for it that had not yet been delivered are lost in the wipe.

This walkthrough shows the essence of the session model in action: the same ClientId plus a non-clean session is what re-attaches a returning client to its server-side state; the session present flag is the protocol’s way of confirming it; and a single careless connect with clean session = true is enough to discard the state the rest of the design was protecting.

Persistent session on the client side

A subtlety that is easy to miss: persistent sessions are not only a broker responsibility. The MQTT specification places matching obligations on the client. When a client requests the broker to hold session data, the client is itself responsible for storing certain state across disconnections, specifically:

• All messages in a QoS 1 or 2 flow that have not yet been confirmed by the broker.
• All QoS 2 messages received from the broker that are not yet completely acknowledged.

In other words, both sides keep the parts of the QoS handshake state they would otherwise lose, so the handshake can be resumed across a reconnect. A client that asks for a persistent session but does not preserve its own state on disconnect can confuse the broker and undermine the very guarantees the session is supposed to provide. Library implementations generally handle this correctly, but if you are implementing a client or working at the protocol level, it is your responsibility to get the client-side persistence right.

What changes in MQTT 5: Clean Start and session expiry

MQTT 5 keeps the persistent-session model but renames and refines the controls, addressing a real shortcoming of the MQTT 3 design. The shortcoming was that a persistent session in MQTT 3.1.1 lived indefinitely on the broker until something explicitly wiped it. A device that connected once with clean session = false and then never came back left its session lying around forever, consuming broker resources.

MQTT 5 splits the old single flag into two clearer controls:

• Clean Start, a boolean in the CONNECT, which says whether to start fresh on this connect by discarding any existing session.
• Session Expiry Interval, a separate field in seconds, which says how long after this client disconnects the broker should keep the session before discarding it on its own.

Together these give finer control than the old clean session flag did. To get the equivalent of MQTT 3.1.1’s clean session = true, set Clean Start = true and Session Expiry Interval = 0; the broker starts fresh and removes the session as soon as the connection closes. To get the equivalent of MQTT 3.1.1’s clean session = false, set Clean Start = false and Session Expiry Interval to a value greater than zero, or to the maximum 4,294,967,295 seconds (just over 136 years) for “keep it indefinitely.” Between those endpoints, you can ask the broker to keep a session for, say, an hour after disconnect, which is something MQTT 3 simply could not express.

The session expiry interval is what fixes the indefinite-storage problem. After a client disconnects, the broker counts down from the interval, and if the client has not reconnected by the time it elapses, the broker discards the session automatically. Each broker may also impose its own server-side maximum on the interval, which is useful when the operator of the broker does not control the configuration of every client. The session and message expiry article in this category covers the new mechanism in more detail.

When to use a persistent session versus a clean one

The choice between persistent and clean is a matter of what the client needs after a disconnect, and it has clear right answers in common cases.

Choose a persistent session when:

• The client must receive every matching message on a topic, even messages that arrive while it is offline. The broker queues QoS 1 and 2 messages for it and delivers them on reconnect.
• The client has limited resources and you want the broker to remember its subscriptions, so reconnection is fast and cheap.
• The client must resume in-flight QoS 1 and 2 publishes across a reconnect, preserving the at-least-once or exactly-once guarantees end to end across the disconnection.

Choose a clean session when:

• The client only publishes messages and never subscribes. There is no subscription state to preserve, and you do not want the broker holding stale state for a tool or one-off process.
• The client does not need to receive messages it missed while offline. Live-only consumers, for example a debugging viewer or a service whose state is rebuilt from a database elsewhere, can usually use clean sessions.

A practical hybrid worth knowing: it is common to use clean sessions for short-lived clients (tools, web dashboards) and persistent sessions for long-lived field devices that need delivery guarantees across their unreliable links. The decision is per-client and per-purpose, not a system-wide setting.

Common gotchas with persistent sessions

A handful of mistakes recur in deployments that rely on persistent sessions. Each follows directly from how the model works, but each is easy to make if you are not watching for it.

ClientId instability. The session is keyed entirely on the ClientId. If a client generates a different ClientId each time it connects, every connection is effectively a fresh client from the broker’s point of view, and persistent sessions never get reused. Subscriptions accumulate as orphans, queued messages never get delivered, and broker resources grow without bound. A persistent client must use a stable, unique ClientId across reconnects. This is one of the most common silent failures in MQTT deployments, because everything appears to work; nothing is broken, just nothing is being reused.

Two clients sharing a ClientId. The flip side of the above: when two physically different clients use the same ClientId, the broker cannot tell them apart, and exactly one of them is “the” client at any moment. When the second one connects, the broker performs a take-over, closing the first one’s connection so the second can attach to the session. The two then fight, each repeatedly kicking the other off. If you ever see a client mysteriously and constantly being disconnected and reconnecting, a ClientId clash is one of the first things to suspect.

Accidentally cleaning the session. As the walkthrough above showed, a single connect with clean session = true wipes the persistent session. This can happen as a result of a configuration mistake, a code path that uses the wrong default, or a deliberate “reset” that did more than intended. If your design depends on session persistence, audit every connect path to confirm it uses clean session = false (or in MQTT 5, clean start = false with a nonzero session expiry).

Assuming retained messages live in the session. Retained messages are stored against topics, independently of any client’s session. A persistent session does not “hold” a client’s retained messages, and clearing a session does not clear retained messages on its topics. The two features are often used together but they are not the same storage.

Treating QoS 0 as queued. Because QoS 0 messages are typically not queued for offline clients, a deployment that relies on QoS 0 for important commands will silently lose them when devices drop offline. If a message must survive a disconnect, send it at QoS 1 or 2, and ensure the receiving client uses a persistent session and subscribes at the same QoS level or higher.

Not reading the session present flag. A client that assumes its subscriptions survived, without reading the session present flag, will silently fail when the broker has in fact lost the session, for instance because of a server-side cleanup, a deliberate purge, or a previous accidental clean. Reading the flag and re-subscribing on a false value is the simple, correct behavior, and it costs nothing on the happy path.

Forgetting the client-side obligation. As covered above, persistent sessions are not just a broker responsibility. A client that asks for persistence but does not preserve its own in-flight QoS state across restarts can undermine the guarantees the session is meant to provide.

How long does the broker store a session?

The natural question once you have a persistent session is how long the broker keeps it. The short answer in MQTT 3.1.1 is “until the client comes back online and receives the queued messages.” The harder question is what happens if the client does not come back for a long time, or at all.

The MQTT 3.1.1 specification does not bound this. In practice, the broker’s operating environment is what limits storage, and it is the broker implementation that decides what to do when storage is tight. Mature brokers provide configuration to bound the lifetime of sessions and the size of queues, and tools to purge sessions that have grown stale. Without that, a deployment with many short-lived clients that never return can accumulate persistent sessions indefinitely.

A related question that the protocol leaves to the implementation is whether sessions survive a broker restart. Some brokers hold sessions in memory only, so a process restart discards all session state for offline clients regardless of any expiry setting; others persist sessions to disk, so they survive restarts. Brokers commonly let you configure this, and the choice has significant operational consequences: in-memory storage is faster but means a broker restart looks identical to a clean session for every offline client, while disk persistence preserves state at a cost of slower writes and more I/O. The right setting depends on how strictly you need delivery guarantees to hold across broker restarts, not just across client disconnections.

Beyond session lifetime, brokers typically also impose operational limits on what they will store per session: maximum queue depth (how many messages can wait for one offline client), maximum total memory or disk used for queues, and expiry on individual queued messages. These limits are essential, because without them a single client that is offline for a long time on a high-traffic topic can cause the broker to fill its storage. When a limit is reached, broker behavior again varies: some drop the oldest queued messages to make room, some drop the newest, some refuse to queue further messages. Persistent sessions are a powerful feature, but in production they need bounding policies behind them; understand what your broker does when its limits are hit before relying on the guarantee in anger.

MQTT 5 directly addresses session lifetime with the Session Expiry Interval. By setting an explicit expiry on the session at connect time, you give the broker permission, and indeed an obligation, to discard the session automatically if the client does not return within the configured window. MQTT 5 also adds a per-message expiry: queued messages can expire before the client reconnects if their Message Expiry Interval elapses, so a slow-returning client may find that the older items in its queue have been pruned even though the session is still alive. Both features are covered in the session and message expiry article. For new deployments, MQTT 5 is the cleaner answer; for MQTT 3.1.1 deployments, lifetime management has to be configured at the broker, not negotiated in the protocol.

How persistent sessions connect to the rest of MQTT

Persistent sessions sit at the intersection of several other MQTT features and shape how each behaves:

• Quality of Service: QoS 1 and 2 messages are queued for offline clients only if the client has a persistent session, so the offline delivery guarantee of those QoS levels is conditional on the session model.
• The connection handshake: the clean session flag is part of the CONNECT, and the session present flag is part of the CONNACK, as covered in the connection establishment article.
• Last Will and Testament: the will is part of the connection, not the session, and is re-registered on every CONNECT.
• Retained messages: retained messages are stored against topics rather than client sessions, so they survive independently of any single client’s session state.
• MQTT 5 session and message expiry: the new Session Expiry Interval and Message Expiry Interval refine how long sessions and queued messages live.

Getting the session model right is a foundational decision for any MQTT deployment that runs on unreliable networks, because everything else about delivery reliability is contingent on it.

Frequently asked questions