Python MQTT: Complete Guide to paho-mqtt 2.x

If you write Python and you need to move small messages between devices or services, MQTT is usually the right fit. It is a publish/subscribe protocol: a client sends a message to a topic, a broker holds the connections, and every client subscribed to that topic gets a copy. Publishers and subscribers never talk to each other directly. They don’t even have to be online at the same time.

This guide covers the working parts you actually touch in code: connecting, subscribing, publishing, quality of service, retained messages, last will, TLS, reconnection, JSON and binary payloads, WebSockets, and the errors that will slow you down. The reference library is the Eclipse Paho MQTT Python client (the paho-mqtt package), which speaks MQTT 5.0, 3.1.1, and 3.1.

One thing to know before you start: paho-mqtt 2.0 changed the client API, and code written for the 1.x line no longer runs as-is. A lot of the examples you’ll find online still target 1.x. This guide uses the current 2.x library throughout, and there’s a section near the end covering exactly what changed, so older snippets are quick to fix.

Which Python MQTT library should you use?

There is more than one option, and they split cleanly by how your program is shaped.

The honest default: use paho-mqtt for scripts, workers, and anything blocking. Use aiomqtt when the rest of your app is already async. Everything below uses paho-mqtt first, then shows the aiomqtt equivalent.

Install it

bash

pip install paho-mqtt

That pulls 2.x. It needs Python 3.7 or newer. If you are stuck on Python 2 or 3.6, you’re stuck on paho-mqtt 1.x too — and none of the 2.x code here will run.

You also need a broker to talk to. For local work, [Mosquitto] is the usual pick. For a quick throwaway test, the public broker test.mosquitto.org on port 1883 works, but never send anything you care about to a public broker. There’s a broker comparison later if you’re choosing one for real.

Your first client

Here is the smallest useful program: connect, subscribe to one topic, and print whatever arrives.

python

import paho.mqtt.client as mqtt

def on_connect(client, userdata, flags, reason_code, properties):
    print("Connected:", reason_code)
    client.subscribe("sensors/room1/temp", qos=1)

def on_message(client, userdata, msg):
    print(f"{msg.topic} -> {msg.payload.decode()}")

client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2)
client.on_connect = on_connect
client.on_message = on_message

client.connect("broker.example.com", 1883, keepalive=60)
client.loop_forever()

Two lines carry most of the weight.

mqtt.Client(mqtt.CallbackAPIVersion.VERSION2) creates the client. That first argument is mandatory in paho-mqtt 2.x. Leave it out and the library raises an error. VERSION2 is the modern callback style and the one to use for new code.

client.loop_forever() hands control to the network loop. It blocks, handles socket traffic, and fires your callbacks. More on the loop options later — the choice of loop shapes your whole program.

Notice the pattern: you don’t call subscribe() in a straight line after connect(). You put it inside on_connect, so it runs after the broker confirms the connection. If the client ever drops and reconnects, on_connect fires again and your subscriptions come back automatically.

Connecting: the parameters that matter

python

client.connect(host="broker.example.com", port=1883, keepalive=60)

• host / port. Plain MQTT is port 1883. TLS-encrypted MQTT is port 8883. Those are conventions, not laws, but almost every broker follows them.
• keepalive. Seconds. If the client has nothing to send, it pings the broker at this interval so both sides know the link is alive. If the broker hears nothing for 1.5× this value, it considers the client dead and can fire its last will (below).
• client id. Set it with mqtt.Client(..., client_id="pi01"). Every client on a broker needs a unique id. Leave it blank and the broker assigns a random one — fine for throwaway subscribers, bad for anything using persistent sessions, because the session is keyed to the id.

Username and password

python

client.username_pw_set("device01", "s3cret")

Call it before connect(). On its own this sends credentials in the clear, so it only makes sense over TLS.

Clean session vs persistent session

When a client id reconnects, does the broker remember it?

• Clean session (default). The broker forgets everything on disconnect. Fresh start every time.
• Persistent session. The broker keeps your subscriptions and queues missed QoS 1 and 2 messages while you were gone. You get them on reconnect.

Persistent sessions only help if your client id is stable — a random id gets a new empty session every time. In MQTT 3.1.1 you turn it on with clean_session=False:

python

client = mqtt.Client(
    mqtt.CallbackAPIVersion.VERSION2,
    client_id="gateway-01",     # must stay the same across restarts
    clean_session=False,
)
client.connect("broker.example.com", 1883)

MQTT 5.0 reworked the idea into clean_start plus a session expiry interval, so you can say “keep my session for one hour after I drop”:

python

from paho.mqtt.properties import Properties
from paho.mqtt.packettypes import PacketTypes

client = mqtt.Client(
    mqtt.CallbackAPIVersion.VERSION2,
    client_id="gateway-01",
    protocol=mqtt.MQTTv5,
)
props = Properties(PacketTypes.CONNECT)
props.SessionExpiryInterval = 3600     # seconds
client.connect("broker.example.com", 1883, clean_start=False, properties=props)

Designing your topic hierarchy

Topics are free-form strings, so nothing stops you from naming everything data or sensor1. Don’t. Flat, vague topics make wildcards useless and subscriptions painful. Topics are slash-separated levels, meant to read like a path — general on the left, specific on the right.

# Good — hierarchical, specific
site1/area2/device7/status
site1/area2/device7/temperature
site1/area2/device7/humidity

# Bad — flat, ambiguous
temperature
status
sensor1

With the hierarchical version you can subscribe to site1/area2/device7/# for one device, site1/+/+/temperature for every temperature reading on the site, or site1/# for everything. With the flat version you’re stuck subscribing to each topic by name.

A few rules that save pain later: keep it consistent (same depth and order everywhere), don’t start a level with $ (those are reserved for broker system topics like $SYS), avoid leading slashes and spaces, and put the changing part (device id, room) in its own level so wildcards can target it. Design the tree once, up front — reworking topic names across a live fleet is miserable.

Subscribing and topic wildcards

python

client.subscribe("sensors/room1/temp", qos=1)

You can subscribe to many topics at once by passing a list of (topic, qos) tuples. Two wildcards let one subscription cover many topics:

• + matches exactly one level. sensors/+/temp catches sensors/room1/temp and sensors/room2/temp, but not sensors/room1/floor2/temp.
• # matches every level below, and must be last. sensors/# catches everything under sensors.

Wildcards are for subscribing. You never publish to a wildcard — a published message goes to exactly one concrete topic.

Because a wildcard subscription can match several topics, check msg.topic inside on_message to see which one actually arrived.

Publishing

python

info = client.publish(
    topic="sensors/room1/temp",
    payload="22.5",
    qos=1,
    retain=False,
)
info.wait_for_publish()      # optional: block until the broker confirms

• payload is bytes on the wire. Pass a string or bytes. MQTT does not care what’s inside; the payload is opaque to the broker.
• qos sets the delivery guarantee (next section).
• retain keeps this message on the broker as the “last known value” for the topic.

publish() returns an MQTTMessageInfo object. For QoS 1 and 2 you can call .wait_for_publish() to block until the broker acknowledges, or check .is_published(). For QoS 0 there’s nothing to confirm — it’s fire and forget.

Sending JSON

Most real payloads are structured, and JSON is the common choice. Serialize before publishing, parse on receipt.

python

import json

payload = json.dumps({"temperature": 22.4, "humidity": 45})
client.publish("site1/area2/device7/env", payload, qos=1)

python

def on_message(client, userdata, msg):
    try:
        data = json.loads(msg.payload)     # msg.payload is bytes; json.loads accepts bytes
    except json.JSONDecodeError:
        print("Bad payload:", msg.payload)
        return
    print(data["temperature"], data["humidity"])

Always wrap json.loads in a try/except. A malformed or empty message will otherwise crash your on_message — and on a shared broker you don’t control every publisher.

Sending binary payloads

JSON is readable but heavy. When you’re pushing a lot of small readings, or working close to a device that speaks raw bytes, pack the payload with struct. It’s compact and exact.

python

import struct

# ">" big-endian, "f" float (4 bytes), "H" unsigned short (2 bytes) -> 6 bytes total
payload = struct.pack(">fH", 22.4, 45)
client.publish("site1/area2/device7/env", payload, qos=1)

python

def on_message(client, userdata, msg):
    temperature, humidity = struct.unpack(">fH", msg.payload)
    print(temperature, humidity)

The format string is a contract: publisher and subscriber must agree on the byte order (> big-endian, < little-endian) and the field layout, or you’ll unpack garbage. Use bytearray if you need a mutable buffer to build up before sending. Binary is unbeatable for size, but you lose self-description — nobody can read the payload without the format string, so document it.

Quality of Service (QoS), in plain terms

QoS is the delivery promise for a single message. Three levels:

Pick by what the message is. A temperature reading every second? QoS 0 — if one drops, another is right behind it. A command like “unlock the door”? QoS 1 or 2 — it has to land. A billing event? QoS 2 — a duplicate would be wrong.

One catch worth remembering: the QoS a subscriber receives is the lower of the publish QoS and the subscribe QoS. Publish at 2 but subscribe at 0, and that subscriber gets it at 0.

Retained messages

Normally, if you subscribe to a topic after a message was published, you missed it — the broker already forwarded it and moved on. A retained message changes that.

python

client.publish("site1/area2/device7/status", "online", qos=1, retain=True)

The broker keeps this one message. Any client that subscribes later immediately gets it as its first message. Exactly one retained message is kept per topic — a new retained publish overwrites the old one.

This is how you handle state instead of events. “Current temperature” or “device online/offline” should be retained, so a client that just connected knows the situation without waiting for the next update.

To clear a retained message, publish an empty payload to the topic with retain=True.

Last will and testament

A client can hand the broker a message to publish on its behalf if it drops unexpectedly — a crash, a dead network, a yanked cable. That’s the last will.

python

client.will_set(
    topic="site1/area2/device7/status",
    payload="offline",
    qos=1,
    retain=True,
)
client.connect("broker.example.com", 1883)

Call will_set() before connect() — the will travels inside the CONNECT packet. If the client disconnects cleanly (it calls disconnect()), the broker throws the will away. If the client vanishes and the keepalive expires, the broker publishes it.

Pair it with a retained “online” message the client sends itself on connect, and you get a clean presence signal: a monitor subscribing to the status topic always sees the current state, whether the device is up or gone.

The network loop: blocking, threaded, or forever

paho-mqtt does nothing until its loop runs. The loop reads and writes the socket and fires callbacks. You have three ways to run it.

loop_forever() — blocks the calling thread and runs until you disconnect. Right when the client is your whole program.

python

client.connect("broker.example.com", 1883)
client.loop_forever()

loop_start() / loop_stop() — runs the loop on a background thread. Your main thread keeps going and can publish whenever it likes. Right when MQTT is one part of a larger program.

python

client.connect("broker.example.com", 1883)
client.loop_start()
try:
    while True:
        client.publish("site1/area2/device7/temp", read_sensor())
        time.sleep(1)
finally:
    client.loop_stop()
    client.disconnect()

loop() — runs one iteration and returns. You call it yourself inside your own loop when you want to control the timing. Lower level; you’ll reach for it less often than the two above.

If you’re publishing from your main code while also wanting messages in, use loop_start(). Trying to publish while loop_forever() blocks the same thread just deadlocks you.

Staying connected: reconnection

A client that runs unattended for months will lose its link at some point — Wi-Fi drops, the broker restarts, a switch reboots. The good news: paho-mqtt 2.x reconnects on its own. With reconnect_on_failure left at its default (True), both loop_forever() and the background loop automatically try to reconnect after a drop. You do not need to catch an error and call connect() again for that case — and wrapping loop_forever() in a try/except mostly does nothing, because it doesn’t raise on a lost connection, it retries.

What you do control is the backoff between attempts:

python

client.reconnect_delay_set(min_delay=1, max_delay=120)

That gives exponential backoff: the first retry waits ~1 second, and the wait doubles each failure up to a 120-second ceiling. It keeps a flapping network from hammering the broker.

The one real gap is the first connect(). If the broker is down when your program starts, connect() raises before any loop runs, so the auto-reconnect never gets a chance. That’s where an explicit retry loop earns its place — and it’s also the pattern to use with loop_start():

python

import time

client.reconnect_delay_set(min_delay=1, max_delay=120)

while True:
    try:
        client.connect("broker.example.com", 1883, keepalive=60)
        break
    except (ConnectionRefusedError, OSError) as e:
        print("Broker not ready, retrying in 5s:", e)
        time.sleep(5)

client.loop_forever()     # handles every later drop on its own

Put your subscribe() calls in on_connect, not after connect(). Then every automatic reconnect restores your subscriptions for free.

Encrypting the connection with TLS

Everything so far is plaintext on port 1883 — fine on a trusted local network, not fine across the internet. TLS fixes that. Configure it before connecting, and switch to port 8883.

python

client.tls_set(ca_certs="ca.crt")          # verify the broker
client.connect("broker.example.com", 8883)

If the broker uses a certificate from a public authority (many cloud brokers do), you can call client.tls_set() with no arguments and let the system trust store handle it.

For mutual TLS — where the broker also verifies each client with its own certificate — pass the client cert and key too:

python

client.tls_set(
    ca_certs="ca.crt",
    certfile="client.crt",
    keyfile="client.key",
)
client.connect("broker.example.com", 8883)

A note carried over from 1.x: if you ever need tls_insecure_set() for testing against a self-signed cert, in 2.x it must be called after tls_set(), not before. And don’t ship that to production.

MQTT over WebSockets

Some networks only allow standard web ports, and browsers can’t open raw TCP at all — so MQTT can ride over WebSockets instead. paho supports it with one constructor argument.

python

client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2, transport="websockets")
client.ws_set_options(path="/mqtt")
client.connect("broker.example.com", 8083)      # plain ws

For encrypted wss://, add TLS the same way as above and use the broker’s secure WebSocket port:

python

client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2, transport="websockets")
client.ws_set_options(path="/mqtt")
client.tls_set(ca_certs="ca.crt")
client.connect("broker.example.com", 8084)      # wss

WebSocket ports and paths vary by broker — EMQX uses 8083 (ws) and 8084 (wss) with path /mqtt; others differ. Check your broker’s config. Everything else — QoS, retain, wildcards, will — behaves exactly as it does over TCP. Only the transport changes.

The async version: aiomqtt

If your app is built on asyncio, callbacks feel wrong. aiomqtt wraps paho in an async interface — no callbacks, no manual loop. You await calls and iterate over incoming messages.

python

import asyncio
import aiomqtt

async def main():
    async with aiomqtt.Client("broker.example.com") as client:
        await client.subscribe("sensors/#")
        async for message in client.messages:
            print(message.topic.value, message.payload.decode())

asyncio.run(main())

Publishing is just as flat:

python

async with aiomqtt.Client("broker.example.com") as client:
    await client.publish("sensors/room1/temp", payload="22.5", qos=1)

The async with block handles connect and clean disconnect for you. Errors surface as aiomqtt.MqttError exceptions instead of reason codes you have to inspect, which fits normal Python try/except reconnect loops. Under the hood it’s still paho, so the protocol behavior — QoS, retain, wildcards, will — is identical. Only the shape of your code changes.

Using MQTT 5.0 from Python

MQTT 5.0 adds features 3.1.1 doesn’t have: user properties, per-message expiry, reason codes on more packet types, topic aliases, request/response, and shared subscriptions. paho-mqtt supports it — you opt in at client creation.

python

import paho.mqtt.client as mqtt
from paho.mqtt.properties import Properties
from paho.mqtt.packettypes import PacketTypes

client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2, protocol=mqtt.MQTTv5)

props = Properties(PacketTypes.PUBLISH)
props.MessageExpiryInterval = 60      # broker drops it after 60s if undelivered
props.UserProperty = ("source", "sensor-fleet")

client.connect("broker.example.com", 1883, clean_start=mqtt.MQTT_CLEAN_START_FIRST_ONLY)
client.publish("sensors/room1/temp", "22.5", qos=1, properties=props)

Two differences from 3.1.1 to keep straight. MQTT 5 replaces clean_session with clean_start and a session expiry interval, set at connect. And reason codes carry more detail — a QoS 1 publish can come back with an ack that explains why the broker refused, instead of a silent drop. The VERSION2 callback style exists partly to expose those cleanly.

Your broker must support MQTT 5 for any of this to work. Mosquitto, EMQX, and HiveMQ all do. If the broker only speaks 3.1.1, connecting with MQTTv5 fails.

Request/response pattern

Pub/sub is one-way by design, but sometimes you need an answer — “give me the current config.” MQTT 5 makes this a first-class feature with two properties: a response topic (where to reply) and correlation data (a token that ties the reply to the request).

python

# Requester
props = Properties(PacketTypes.PUBLISH)
props.ResponseTopic = "reply/req-42"
props.CorrelationData = b"req-42"
client.subscribe("reply/req-42", qos=1)
client.publish("request/get-config", "config?", qos=1, properties=props)

The responder reads msg.properties.ResponseTopic and msg.properties.CorrelationData off the incoming message, does the work, and publishes the answer back to that response topic with the same correlation data. The requester matches the token to know which reply belongs to which request.

On MQTT 3.1.1 you build the same thing by hand: publish to a request/... topic, include your own correlation id in the JSON payload, and tell the responder to reply on a reply/... topic you’re subscribed to. More wiring, same idea.

Shared subscriptions (load balancing)

Normally every subscriber to a topic gets every message. A shared subscription splits the load: a group of subscribers shares one logical subscription, and the broker hands each message to only one member of the group. Prefix the topic with $share/<group>/:

python

client.subscribe("$share/workers/site1/area2/#", qos=1)

Run five workers that all subscribe to $share/workers/... and the broker spreads messages across them — horizontal scaling with no extra code. It’s an MQTT 5 feature, though many brokers (Mosquitto, EMQX) support it. Without $share, all five would each process every message.

Sparkplug B

If you’re comparing MQTT options, you’ll run into Sparkplug B. It isn’t a competing protocol. It’s an application specification on top of MQTT that standardizes the topic namespace (spBv1.0/...), the payload format (Google Protocol Buffers), and state management via birth/death certificates, so different vendors’ systems interoperate instead of each inventing their own topic tree. You still use a normal MQTT client and broker underneath. If you need it, see our dedicated [Sparkplug B] article — it’s out of scope here.

Brokers: which one

The client library is only half the picture; you also pick a broker. A quick orientation:

Mosquitto covers most people’s needs and is the right place to start. Reach for the others when you hit real scale, need clustering, or want a vendor with a support contract. All of them speak standard MQTT, so your Python code doesn’t change when you switch.

What changed in paho-mqtt 2.x

Old paho-mqtt code often fails on 2.x — not because the logic is wrong, but because the API moved. If you pulled a snippet from an older guide and it errors out, this is usually why.

1. The client constructor now requires a callback API version.

python

# Old (1.x) — breaks in 2.x
client = mqtt.Client()

# New (2.x)
client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2)

Missing this raises “Unsupported callback API version.” If you just want an old script running with no other edits, pass VERSION1 instead of VERSION2 — the 1.x callback signatures still work under it. It’s deprecated, but it buys you time.

2. Callback signatures gained arguments.

python

# Old
def on_connect(client, userdata, flags, rc): ...

# New (VERSION2)
def on_connect(client, userdata, flags, reason_code, properties): ...

rc (an int) became reason_code (a ReasonCode object). flags is now an object — use flags.session_present instead of flags["session present"]. on_subscribe and on_publish also gained a properties argument.

3. reason_code == 0 still means success, but other numeric comparisons don’t map 1:1. If your old error handling checked rc == 5 or similar, switch to string comparisons like reason_code == "Not authorized".

4. Dropped Python versions. 2.x needs Python 3.7+. Old Python means old paho.

5. A few removals. ReasonCodes was renamed ReasonCode (the old name still works but is deprecated). The max_packets argument on loop() and friends is gone. tls_insecure_set() moved to after tls_set().

If you can’t or won’t migrate a large codebase right now, CallbackAPIVersion.VERSION1 is the escape hatch. For anything new, write VERSION2.

Common errors and what they mean

These are the messages you’ll actually hit, with the usual cause and fix.

Unsupported callback API version paho-mqtt 2.x, and you created the client without the version argument. Add mqtt.CallbackAPIVersion.VERSION2 as the first argument to Client().

Connection Refused: Not authorized (reason code 135 / return code 5) The broker rejected you at the protocol level. The credentials are missing, or the user isn’t allowed on that topic by the broker’s ACL. Check username_pw_set() and the broker’s access rules.

Connection Refused: bad username or password (return code 4) Wrong user or password. Different from “not authorized” — here the identity itself failed, before any ACL check.

ConnectionRefusedError / Connection refused (OS-level) Nothing is listening on that host and port. The broker isn’t running, or you’re on the wrong port (1883 vs 8883), or a firewall is dropping it.

[Errno 104] Connection reset by peer The broker accepted the socket, then closed it. Common causes: connecting plaintext to a TLS-only port, an MQTT-version mismatch, or a duplicate client id (another client with the same id kicked you off — see the checklist).

BrokenPipeError / Broken pipe You tried to write to a socket the broker already closed. Usually a symptom of a drop you didn’t handle. Let the auto-reconnect deal with it, and don’t publish from a thread while the connection is down.

ssl.SSLCertVerificationError: CERTIFICATE_VERIFY_FAILED TLS couldn’t verify the broker’s certificate. Wrong or missing CA file in tls_set(ca_certs=...), an expired cert, or the hostname you connected to doesn’t match the cert. Fix the CA/hostname — do not reach for tls_insecure_set(True) outside a lab.

[Errno 113] No route to host A networking/routing problem, not MQTT. The host is unreachable — bad IP, wrong subnet, VPN down, or a firewall. Ping it first.

TimeoutError / connect hangs The broker address resolves but never answers. Firewall silently dropping packets, wrong port, or the broker is overloaded. Set a sane keepalive and confirm reachability outside Python.

Troubleshooting checklist

When a client won’t connect or messages don’t arrive, walk this list top to bottom:

1. Broker reachable? ping the host; telnet host 1883 (or nc) to confirm the port answers.
2. Right port? 1883 plaintext, 8883 TLS, 8083/8084 WebSockets. Mixing them fails.
3. TLS matched? Plaintext to a TLS port (or the reverse) gets reset. CA file correct?
4. Credentials? Username/password set before connect(), and correct.
5. Firewall? Local and network firewalls, plus cloud security groups, allow the port.
6. Topic typo? Publisher and subscriber topics match exactly — topics are case-sensitive.
7. Wildcards right? + is one level, # is multi-level and must be last. No wildcards when publishing.
8. QoS mismatch? Subscriber gets the lower of publish and subscribe QoS.
9. Retained expectation? Expecting an immediate first message means the publisher set retain=True.
10. Duplicate client id? Two clients with the same id keep kicking each other off. Every client needs a unique id.
11. Subscribe placed in on_connect? If it’s after connect() instead, a reconnect silently drops your subscriptions.
12. Broker logs. When all else looks right, the broker log usually names the real reason in one line.

Quick reference