MQTT 5 Correlation Data Explained: Request/Response Pattern

MQTT was designed for publish/subscribe — fire-and-forget messaging from many publishers to many subscribers. For two decades, that’s what MQTT did. If you wanted classic request/response (“send a query, get back exactly one answer for that query”), you had to use HTTP, AMQP, or roll your own correlation scheme on top of MQTT topics.

MQTT 5 added native request/response support through two new PUBLISH properties: Response Topic (Property 0x08) tells the responder where to send the answer, and Correlation Data (Property 0x09) lets the requester match incoming responses back to the original request. Together they enable RPC-style communication over MQTT — without abandoning the pub/sub model that makes MQTT valuable in the first place.

This article explains how Correlation Data works, the request/response pattern it enables, common correlation strategies (UUIDs, sequence numbers, structured binary), and the subtle race conditions and timeout patterns engineers need to handle correctly.

For broader MQTT 5 context, see our MQTT 5 New Features Overview and MQTT 5 User Properties.

What Correlation Data is in one paragraph

Correlation Data is an opaque binary identifier that an MQTT 5 requester includes in a PUBLISH packet (via Property 0x09) so that responses can be matched back to the original request. It works alongside Response Topic (Property 0x08), which tells the responder where to publish its response. The MQTT Server does not interpret Correlation Data — it’s binary bytes meaningful only to the requester. The Server’s job is to forward Correlation Data unchanged to subscribers receiving the PUBLISH. The responder reads Correlation Data from the incoming request, processes the request, and includes the same Correlation Data in its response PUBLISH on the Response Topic. The requester (subscribed to that Response Topic) then matches the incoming response back to its outstanding request by comparing Correlation Data. This enables RPC-style request/response patterns over MQTT’s pub/sub foundation — supporting UUIDs, sequence numbers, or arbitrary structured binary as correlation identifiers, up to 65,535 bytes per request.

Why MQTT 5 added request/response support

MQTT 3.1.1 had no native concept of request/response. If you wanted RPC over MQTT 3.1.1, you had to invent your own conventions:

The DIY correlation pain in MQTT 3.1.1

Common workaround in MQTT 3.1.1:
  Encode correlation ID into the topic structure:
    "service/calculate/req/client-007/req-id-12345"
  
  Service responds to:
    "service/calculate/resp/client-007/req-id-12345"

Problems with this approach:

1. Topic explosion — every request needed its own response topic, creating millions of short-lived topic subscriptions
2. Authorization complexity — clients needed wildcard subscribe permissions covering arbitrary response topic patterns
3. No protocol enforcement — every team invented their own convention, none portable
4. Subscription overhead — subscribing to a unique response topic per request was expensive
5. Payload-level correlation — JSON IDs in payloads required parsing every message just to correlate

What MQTT 5 fixed

The MQTT 5 design moves correlation into protocol-level properties — out of the topic structure, out of the payload. The result:

• No topic explosion — clients subscribe to one stable response topic
• Server-side authorization — straightforward topic-based permissions
• Standardized correlation — every MQTT 5 client/Server library handles it the same way
• Cheap correlation — properties are part of the packet header parsing, not payload
• Payload format independence — correlation works whether payload is JSON, Protobuf, MessagePack, raw bytes

Request/response over MQTT became operationally practical with these two properties.

The two properties

Request/response in MQTT 5 uses two complementary PUBLISH properties:

Response Topic (Property 0x08)

The Response Topic tells the responder where to publish the response. It must be a concrete topic name — no wildcards (+, #, $share/). The requester is expected to be subscribed to (or about to subscribe to) this topic.

A common pattern: clients establish a long-lived response topic when they connect, then reuse it for all their requests:

Client "device-007" subscribes to "device-007/responses" once.
Every request includes Response Topic: "device-007/responses"
Every response arrives on "device-007/responses"

The Client Identifier in the response topic ensures responses go only to this client.

Correlation Data (Property 0x09)

Correlation Data is arbitrary binary bytes chosen by the requester to identify which request a response belongs to. The MQTT Server never inspects Correlation Data — it just passes the bytes through unchanged when forwarding PUBLISH messages to subscribers.

What you put in Correlation Data is your choice: UUIDs, sequence numbers, hash digests, encoded request metadata, anything. The only requirement is that the requester can match a response’s Correlation Data back to the corresponding outstanding request.

Are both properties required?

No — they’re independent. You can use Response Topic without Correlation Data (one request at a time, no matching needed). You can use Correlation Data without Response Topic (correlating fire-and-forget messages for tracing purposes). But for proper request/response with multiple outstanding requests, you need both:

The complete request/response flow

A full request/response cycle in MQTT 5:

Step 1: Requester subscribes to its response topic

Before sending any request, the requester subscribes to where responses will arrive:

Client A → Server: SUBSCRIBE
  Packet Identifier: 1
  Topic Filter: "client-a/responses"
  QoS: 1

Wait for SUBACK before publishing requests — see the Race conditions section.

Step 2: Requester publishes the request

Client A → Server: PUBLISH
  Topic: "service/calculator/compute"
  QoS: 1
  Packet Identifier: 42
  Properties:
    Response Topic: "client-a/responses"
    Correlation Data: <16 bytes UUID>
  Payload: {"operation": "add", "a": 5, "b": 7}

Server → Client A: PUBACK (Packet Identifier: 42)

Step 3: Service receives and processes the request

Server → Service: PUBLISH
  Topic: "service/calculator/compute"
  Properties:
    Response Topic: "client-a/responses"     ← forwarded unchanged
    Correlation Data: <same 16 bytes UUID>   ← forwarded unchanged
  Payload: {"operation": "add", "a": 5, "b": 7}

[Service computes: 5 + 7 = 12]

Step 4: Service publishes the response

Service → Server: PUBLISH
  Topic: "client-a/responses"              ← from Response Topic in request
  QoS: 1
  Properties:
    Correlation Data: <same 16 bytes UUID> ← echoed from request
  Payload: {"result": 12, "status": "ok"}

Step 5: Requester receives the response

Server → Client A: PUBLISH
  Topic: "client-a/responses"
  Properties:
    Correlation Data: <16 bytes UUID>
  Payload: {"result": 12, "status": "ok"}

[Client A matches UUID to outstanding request 42, completes the future/callback]

The complete cycle. Multiple requests can be in flight simultaneously — each has unique Correlation Data, so responses are matched correctly even if they arrive out of order.

Common correlation patterns

What you put in Correlation Data depends on your needs:

Pattern 1: UUID

The simplest and most common pattern. Each request gets a fresh UUID:

Correlation Data = bytes of UUID v4
Length = 16 bytes

Example: 0xb4 0x42 0x6f 0x3e 0xaa 0xfd 0x4d 0xc8 0x9c 0x6e 0xa7 0xb2 0x5e 0xfc 0xc0 0x91

Advantages: globally unique, no coordination needed, easy to generate Disadvantages: 16 bytes per request, no internal structure

Pattern 2: Sequence number

For high-throughput requesters needing minimal overhead:

Correlation Data = monotonically increasing UINT64
Length = 8 bytes

Example: 0x00 0x00 0x00 0x00 0x00 0x00 0x04 0x21  (sequence 1057)

Advantages: smaller (8 bytes), easy to look up in array indexed by sequence Disadvantages: must reset on restart, conflicts if multiple processes share sequence space

Pattern 3: Structured binary

For complex correlation with multiple identifying fields:

Correlation Data = packed binary structure:
  [request_type (1 byte)][tenant_id (4 bytes)]
  [request_id (8 bytes)][timestamp (8 bytes)]
Length = 21 bytes

Example: 0x02 0x00 0x00 0x12 0x34 ...
         type=2, tenant=4660, request_id=...

Advantages: carries metadata in correlation itself (multi-tenant routing, type info) Disadvantages: requires consistent encoding/decoding logic on both sides

Pattern 4: Cryptographic correlation

For security-sensitive deployments:

Correlation Data = HMAC-SHA256(secret_key, request_data)
Length = 32 bytes

Example: 0xa7 0x84 0xe5 ... (32 bytes)

Advantages: response correlation can be cryptographically verified Disadvantages: 32 bytes, requires shared secret management

Pattern 5: Application-specific compact identifiers

For specific protocols layered on MQTT:

Correlation Data = [device_class (1 byte)][stream_id (3 bytes)][packet_num (4 bytes)]
Length = 8 bytes

The choice depends on your specific use case. Most teams start with UUIDs (simple, works) and only optimize if they have specific bandwidth or matching efficiency requirements.

Race conditions and subscription ordering

A critical pitfall: the requester must be subscribed to the Response Topic BEFORE publishing the request, otherwise responses arriving before the subscription completes are lost.

The race

WRONG ORDER:
1. Client publishes request with Response Topic = "client-a/responses"
2. Server processes request immediately, service responds
3. Server tries to deliver response to "client-a/responses"
4. Nobody is subscribed → response dropped
5. Client subscribes to "client-a/responses"
6. Client waits forever for response that never arrives

The fix

CORRECT ORDER:
1. Client subscribes to "client-a/responses"
2. Client waits for SUBACK confirming subscription is active
3. Client publishes request with Response Topic = "client-a/responses"
4. Server processes request
5. Service responds on the topic
6. Client receives response, matches via Correlation Data

Pattern: subscribe once at connection startup

For long-lived clients, the cleanest pattern is subscribe once and reuse:

python

def on_connect():
    # Subscribe to response topic ONCE at startup
    client.subscribe("device-007/responses", qos=1)
    # Wait for SUBACK before doing anything else
    await suback_received

def send_request(operation, params):
    correlation_id = uuid.uuid4().bytes
    pending_requests[correlation_id] = create_future()
    
    client.publish(
        topic="service/api/" + operation,
        payload=encode(params),
        response_topic="device-007/responses",
        correlation_data=correlation_id,
        qos=1
    )
    
    return await pending_requests[correlation_id]  # waits for response

def on_message(packet):
    if packet.topic == "device-007/responses":
        cd = packet.properties.correlation_data
        if cd in pending_requests:
            pending_requests[cd].set_result(packet.payload)
            del pending_requests[cd]

This pattern subscribes once, then handles thousands of concurrent requests through Correlation Data matching.

What about per-request response topics?

Some designs use a unique response topic per request:

Response Topic = "client-a/responses/<request_uuid>"

This avoids needing Correlation Data (the topic itself identifies the request) but creates topic explosion at the Server. Not recommended for high-throughput systems — use stable response topics with Correlation Data instead.

The responder’s responsibility

The MQTT Server doesn’t automatically generate responses or correlate requests — that’s the responder’s job. A well-implemented responder must:

Required behavior

1. Read Response Topic from the incoming PUBLISH
2. Read Correlation Data from the incoming PUBLISH
3. Process the request
4. Construct a response PUBLISH:
   • Topic = the Response Topic from the request
   • Correlation Data = the same Correlation Data from the request
   • Payload = response data

Handling requests without Response Topic

If an incoming PUBLISH has Response Topic absent, the responder should treat it as a fire-and-forget message — process it but don’t send a response. Some patterns combine request/response and fire-and-forget on the same service topic; the presence or absence of Response Topic distinguishes them.

Handling failed processing

When the responder can’t process the request (validation error, authorization failure, internal error), it should still send a response on the Response Topic with appropriate error indication. The requester is waiting for SOME response — silent failure leaves the requester hanging until timeout.

Common practice: include status in the response payload:

json

{
  "status": "error",
  "error_code": "INVALID_INPUT",
  "message": "Field 'amount' must be positive"
}

Echoing Correlation Data carefully

The responder must copy the Correlation Data bytes exactly — don’t reformat, don’t truncate, don’t normalize. Treat it as opaque binary. Even if it looks like a UUID string, treat it as the byte sequence it is.

Bugs in Correlation Data echoing cause the most common request/response failure: the requester sees a response but can’t match it to any outstanding request.

Timeout handling

Network failures, slow responders, lost messages — request/response over MQTT must handle the case where no response arrives.

Pattern: client-side timeout

python

def send_request_with_timeout(operation, params, timeout=30):
    future = send_request(operation, params)
    try:
        return await asyncio.wait_for(future, timeout)
    except asyncio.TimeoutError:
        cleanup_pending_request(future.correlation_id)
        raise RequestTimeoutError("No response in 30 seconds")

The requester waits up to a timeout, then gives up. The pending request entry is cleaned up so memory doesn’t leak.

Handling late responses

If a response arrives after the timeout has elapsed:

1. The pending request entry has been removed
2. The on_message handler finds no match for the Correlation Data
3. The response is silently discarded (already gave up on this request)

This is the correct behavior — late responses are stale. Logging them at DEBUG level is useful for monitoring.

QoS choice for request/response

Most request/response patterns use QoS 1 — acceptable to occasionally retry, but losing messages is bad. QoS 2 is appropriate for financial transactions or anywhere duplicates are unacceptable but adds latency.

Idempotency for QoS 1

If using QoS 1 and the requester retries (because it timed out), the request may be delivered twice. If the operation is non-idempotent (e.g., “transfer $100”), this is dangerous.

The fix: include an idempotency key — often the same Correlation Data — and have the responder detect duplicates:

Responder logic:
  if seen_correlation_data(cd):
    return cached_response(cd)  // same response as first time
  else:
    process_request()
    cache_response(cd, response)
    return response

Many request/response systems use Correlation Data as the idempotency key naturally.

Service discovery via Request Response Information

MQTT 5 also defines a service discovery mechanism that complements Correlation Data:

Request Response Information (Property 0x19)

Client tells the Server during CONNECT that it would like response topic information:

Client → Server: CONNECT
  Properties:
    Request Response Information: 1 (boolean: true)

Response Information (Property 0x1A)

Server responds in CONNACK with a topic prefix the client should use for its response topics:

Server → Client: CONNACK
  Properties:
    Response Information: "responses/client-007"

The client then uses topics under this prefix for its response topics. This lets the Server centrally manage response topic conventions (authorization, routing, cleanup) without each client inventing its own scheme.

When to use service discovery

This mechanism is useful in:

• Multi-tenant systems where the Server enforces tenant-specific response topic prefixes
• Centrally-managed deployments where response topic conventions are policy
• Auto-configuration scenarios where clients shouldn’t hardcode topic names

For simpler deployments, clients can just use predictable response topics like <client_id>/responses without Request Response Information.

Comparison with HTTP request/response

MQTT 5’s request/response is more flexible than HTTP for asynchronous and event-driven scenarios — you can have thousands of in-flight requests on one connection, the Server can push notifications, and there’s no concept of “the response is delayed too long” forcing a connection-level timeout.

It’s less suitable than HTTP for synchronous web-style request/response where every request needs exactly one response within a request-scoped time budget. The MQTT model is request → “…eventually, an event matching this correlation will arrive…” which suits IoT and event-driven systems better than transactional REST APIs.

Use cases

RPC-style service calls

Classic request/response usage. A client invokes a service, gets back a result:

Client → Service: PUBLISH "service/calculator/add", {"a": 5, "b": 7}
Service → Client: PUBLISH on response topic, {"result": 12}

Configuration retrieval

A device fetches its configuration from a centralized config service:

Device → Config Service: PUBLISH "config/get/v1", {"device_id": "sensor-007"}
Config Service → Device: PUBLISH on response topic, <config blob>

Health check / liveness probes

A monitor probes services:

Monitor → Service: PUBLISH "service/healthcheck", {} 
Service → Monitor: PUBLISH on response topic, {"status": "ok", "uptime_s": 12345}

Distributed tracing

Even without traditional request/response, Correlation Data carries trace IDs:

Service A → Service B: PUBLISH ... Correlation Data = trace_id
Service B → Service C: PUBLISH ... Correlation Data = same trace_id
[Logs collected with trace_id allow reconstruction of the full request path]

Multi-step workflows

Long-running workflows track step correlation:

Step 1: Initiate order with Correlation Data = order_uuid
Step 2: Payment processing with Correlation Data = order_uuid
Step 3: Shipping notification with Correlation Data = order_uuid
[All steps trace back to the same order via Correlation Data]

Common errors and troubleshooting

Frequently asked questions

What is MQTT 5 Correlation Data?

Correlation Data is an opaque binary identifier (Property 0x09) that MQTT 5 publishers include in PUBLISH packets to enable request/response patterns. The requester puts arbitrary bytes (typically a UUID or sequence number) in Correlation Data; the responder echoes the same bytes in its response; the requester matches the response back to the original request by comparing Correlation Data values. The MQTT Server doesn’t interpret Correlation Data — it just forwards the bytes unchanged to subscribers.

What is the difference between Response Topic and Correlation Data?

Response Topic (Property 0x08) and Correlation Data (Property 0x09) work together for request/response. Response Topic is a UTF-8 string telling the responder where to publish its response. Correlation Data is binary bytes identifying which request the response belongs to. You need both for proper RPC: Response Topic to route the response, Correlation Data to match it. Response Topic without Correlation Data only works if you have one in-flight request at a time.

Does the MQTT Server interpret Correlation Data?

No. The MQTT Server treats Correlation Data as opaque binary bytes — it does not parse, validate, or interpret the content. The Server’s only job is to forward Correlation Data unchanged when relaying PUBLISH packets to subscribers. What goes in Correlation Data is meaningful only to the requester and responder; the Server is neutral about format and meaning.

What can I put in MQTT 5 Correlation Data?

Anything within the 65,535-byte limit. Common choices: UUIDs (16 bytes), monotonically increasing sequence numbers (8 bytes), HMAC digests (32 bytes), structured binary with multiple fields (tenant ID + request type + timestamp), or application-specific compact identifiers. Most implementations start with UUIDs for simplicity. The only requirement is that the requester can match a response’s Correlation Data back to the corresponding outstanding request.

Do I need to use Correlation Data if I’m using Response Topic?

If you only ever have one in-flight request at a time, no — the response on the Response Topic must be for that request. But for any system with multiple concurrent requests (common in any production system), yes — without Correlation Data you cannot distinguish which response belongs to which request. Best practice: always use both Response Topic AND Correlation Data for request/response.

What QoS level should I use for MQTT 5 request/response?

QoS 1 is typically the right choice. It guarantees at-least-once delivery (acceptable to occasionally retry) without QoS 2’s overhead. QoS 0 risks losing requests or responses, breaking the request/response pattern. QoS 2 is appropriate when duplicates would be harmful (financial transactions, critical state changes) but adds latency through its 4-message handshake. When using QoS 1, design responders to be idempotent — Correlation Data can serve as the idempotency key.

How does the responder handle a request without Response Topic?

If a PUBLISH arrives without Response Topic, the responder should treat it as a fire-and-forget message — process the request without sending a response. This pattern lets the same service topic handle both request/response and fire-and-forget messages. Always respond to messages WITH Response Topic; never respond to messages without it.

What is the maximum size of Correlation Data?

The maximum is 65,535 bytes — the limit imposed by the Binary Data encoding in MQTT 5 (a 2-byte length prefix). In practice, Correlation Data is almost always much smaller — 8 to 32 bytes is typical. Large Correlation Data wastes bandwidth on every message in the request/response pair. Use the smallest size that meets your correlation needs.

What happens if the responder doesn’t echo Correlation Data correctly?

The requester can’t match the response to any outstanding request. The response is processed by the requester’s on_message handler, but no matching request future or callback is found, so the response is silently discarded (often logged at DEBUG level). The requester eventually times out waiting for what it thinks is a missing response. This is the most common request/response failure mode — always ensure the responder copies Correlation Data exactly without any normalization or reformatting.

Should I subscribe to a unique response topic per request?

No — this causes topic explosion at the Server (thousands of short-lived subscriptions) and is operationally expensive. The recommended pattern is subscribe once to a stable response topic (e.g., <client_id>/responses) at connection startup, then use Correlation Data to match responses. This pattern scales to thousands of concurrent requests on a single subscription.

How do I handle timeouts in MQTT 5 request/response?

Implement client-side timeouts: track outstanding requests with their Correlation Data and timestamp, and remove entries after a timeout elapses. When a late response arrives, the on_message handler finds no matching entry and silently discards the response. Common timeout values: 5-30 seconds for interactive RPC, longer for batch operations. Without timeouts, pending request maps grow unboundedly on network failures.

Where does Service Discovery fit in MQTT 5 request/response?

MQTT 5 defines Request Response Information (Property 0x19, in CONNECT) and Response Information (Property 0x1A, in CONNACK) for Server-side service discovery. The client requests response topic information; the Server returns a topic prefix the client should use. This is useful for multi-tenant systems and centrally-managed deployments where response topic conventions are policy. For simpler deployments, clients can use predictable response topics without these properties.

Can I use Correlation Data without request/response patterns?

Yes. Correlation Data is technically just an opaque binary tag attached to PUBLISH messages. It can be used for distributed tracing (tag every message in a workflow with the same trace_id), debugging (correlate log entries to specific messages), or any application-specific purpose. Without Response Topic, there’s no “response” expected — Correlation Data just provides per-message context that the Server forwards unchanged.