A message queue allows work to be accepted and completed later, at a rate consumers can sustain. Introducing one into a design requires explaining what is stored, when the work is considered accepted, and how an operation left partly complete is recovered.
In an interview, the broker’s name matters less than that contract. The queue must address a concrete need: decoupling execution times, absorbing a bounded spike, or coordinating background processing.
What a queue solves
Decoupling reduces the temporal dependency between producer and consumer. The producer can persist work while a consumer is stopped, within the system’s availability, capacity, and retention limits. Schema and semantic dependencies remain: both must interpret the message compatibly.
Asynchrony allows tasks such as generating reports or preparing notifications to be moved out of a request’s path. The response can communicate that a request was accepted while its result is queried later. The work and its errors still exist; they now need observable state and a recovery policy.
The buffer absorbs bursts when arrivals temporarily exceed processing. Its size and duration are finite. The queue buys time to process the excess and adds waiting time to the final result.
Reliability depends on persistence, replication, acknowledgements, and recovery. Expiring messages and uncertain publication results also matter. A queue helps preserve pending work under its contract; it does not by itself guarantee no losses or a single execution of effects.
A processing contract
To generate a report, the API can record a jobID, pending state, and an event in an outbox within the same local transaction. It then responds with HTTP 202 Accepted and an identifier for querying status. That response confirms durable acceptance of the request, which is still awaiting publication or processing.
A relay publishes the work, and a worker generates the report. The result must be recorded idempotently for that jobID, with states that distinguish pending, running, completed, and failed. If there are effects outside the database, their recovery is also part of the design.
--- config: look: handDrawn --- flowchart TD client["Client"] --> api["API<br/>Accept request"] api --> database[("Local transaction<br/>Pending job + outbox")] database -->|"Request persisted"| accepted["HTTP 202<br/>Query status by jobID"] accepted --> client database --> relay["Outbox relay"] relay --> queue[("Queue<br/>Finite buffer and retention")] queue --> workers["Workers<br/>Idempotent processing"] workers -->|"Result recorded"| result["Job completed<br/>Report available"] workers -->|"Recovery exhausted"| dlq[("DLQ<br/>Diagnosis and recovery")] workers -.->|"Limit admission as waiting time grows"| api
The figure shows the workflow and a capacity control signal. Publication and consumption acknowledgements must complete it according to the broker used. The transactional outbox prevents acceptance of the local change from being separated from recording the pending event, but publication may still repeat.
Capacity and backpressure
If the arrival rate persistently exceeds the useful completion rate, backlog and the age of pending work grow. Relevant capacity includes the effects that must be completed, not just the number of messages a worker can read.
With approximately constant rates, during an interval when arrivals exceed processing:
backlogIncrease ≈ (arrivalsPerSecond - completionsPerSecond) * durationFor example, a burst of 3,000 messages per second for 60 seconds, with capacity to complete 2,000 per second, adds approximately 60,000 pending messages. If arrivals then fall to 1,000 per second and capacity remains at 2,000, the excess drains in about 60 seconds. These are calculation assumptions; retries and tasks with different costs can change that effective capacity.
Backpressure limits how much work enters or remains in execution when capacity is under pressure. It can slow production, bound concurrent requests, or reject new operations with a response that communicates the admission condition. The policy must consider tolerated waiting time and fairness between clients.
Autoscaling helps while there is parallelism and capacity in dependencies. Adding workers against a saturated database can make the problem worse. Concurrency and in-flight message limits protect those dependencies, while queue age and lag help decide when to scale or limit admission.
Partitions, keys, and ordering
In Kafka, a topic can be divided into partitions, and a conventional consumer group distributes those partitions among consumers. Each partition is assigned to one consumer in the group at a time. If there are more consumers than partitions, some may remain unassigned; this limits useful parallelism in that model, but does not prohibit starting more consumers or describe every message queue.
The partition key determines where related messages are placed under the chosen strategy. An account_id can keep an account’s records in the same partition as long as the partitioning configuration preserves that assignment. If one account concentrates a large share of traffic, it can also create a hot partition: preserving that sequence and distributing its load requires resolving an explicit trade-off.
Log ordering is per partition. To preserve the order of effects, the consumer must process the sequence compatibly and handle retries, rebalances, and stale operations. A handler that launches concurrent tasks can finish the second event before the first even if it read them in order.
The key does not establish which message should be published first either. The producer must preserve the sequence relevant to the business; the broker preserves the order in which records are appended to the partition.
For example, an account with an initial balance of $0 has two pending operations to apply, with no overdrafts allowed. Both messages use account_id = 42:
| Required sequence | Pending operation | Balance after applying it |
|---|---|---|
| 1 | Deposit $100. | $100. |
| 2 | Withdraw $50. | $50. |
If the deposit is published first and the consumer applies the operations in that order, the final balance is $50. Applying the withdrawal before the deposit may cause an insufficient-funds rejection. Directing both messages to the same partition allows their sequence to be preserved, but still requires the producer and consumer to maintain that order.
A sequence per account assigned by the service that controls the operations allows it to detect that operation 2 has a pending predecessor. The consumer must wait or reconcile that case before applying it, and deduplicate each operation using a stable identity. The balance change and its conditions are validated atomically in the authoritative system. This example concerns pending requests; if the messages describe already confirmed transactions, the consumer reconstructs a projection of those transactions.
The scope of the business rule also matters. Keeping one account’s events in order does not make a transfer between two accounts atomic. That operation requires coordination that preserves its invariants.
In other models, scaling and ordering are expressed through competing workers, multiple queues, or message groups. The one-consumer-per-partition rule, within the same group, must be applied to the model that actually provides it.
Delivery guarantees and recovery
Delivery guarantees describe what can happen to a message during transmission and recovery. Interpreting them requires establishing the scope: publication, delivery to the consumer, or the committed processing result. Receiving a message and completing its business effect are different facts.
| Model | Contract within the defined scope | Consequence for the application |
|---|---|---|
| At least once | One or more deliveries under the agreed recovery conditions. | Must tolerate duplicates and coordinate idempotent effects. |
| At most once | Zero or one delivery; the work may not arrive. | Must accept loss and avoid repetitions within that scope. |
| Exactly once | A single committed observable result per record within a defined boundary. | Requires coordinating state, progress, and effects covered by the guarantee. |
At-least-once delivery requires acknowledgements, retries, and storage compatible with the defined failures and time limits. Consumers must assume that a message may be processed again. The central case occurs when the effect completes and the acknowledgement is lost: RabbitMQ may redeliver a message without an ack; in Kafka, a group may reread records whose progress was not committed.
Acknowledging before completing the effect can leave work undone after a failure. The at-most-once model may fit some metrics or telemetry when that loss is explicitly accepted. A fire-and-forget send describes an interaction without waiting for acknowledgement; by itself, it does not define all the broker’s guarantees. In Kafka, committing an offset does not delete the record: the group may advance without completing the work even though the data remains available for replay.
An idempotent consumer can repeat the same operation without adding another business effect. A stable identity allows it to be recognized. If the effect is in a database, the deduplication record and local change must be coordinated atomically, for example in the same transaction. An external charge or publication also needs a strategy for checking uncertain results and repeating the operation under the conditions and validity period of its idempotency contract.
The exactly-once guarantee does not necessarily mean executing the handler only once. There may be several attempts and one committed result. Kafka allows produced records and consumed offsets to be coordinated transactionally, with read_committed readers that see committed results. That boundary does not automatically include a write to an external database or a charge; those effects need their own coordination.
When recovering work is a requirement, at least once with idempotent effects is often a solid option to defend in an interview. It allows an explanation of how a lost acknowledgement is recovered and how repeating the effect is prevented. The choice must follow the requirements for tolerated loss, recovery, and the scope of effects.
Delivery leases and visibility timeouts also require attention. An expired deadline may allow another attempt while the previous worker is still executing. Sizing or renewing that deadline helps reduce accidental concurrency; correctness still depends on validating state and coordinating repeated effects.
DLQ and failing messages
A poison message causes repeated failures because of its content or how the consumer processes it. It may have a malformed payload, an incompatible schema, or trigger a handler defect. Retrying it indefinitely consumes capacity and can halt progress. An infrastructure outage, by contrast, can affect valid messages and requires diagnosing its cause.
Temporary errors can be recovered through bounded retries, increasing delays, and jitter. The policy defines an attempt limit, clarifies whether it includes the initial attempt, and preserves its history during recovery. When that budget is exhausted, the message is routed to a Dead Letter Queue (DLQ). A permanent error, such as an invalid payload that cannot be processed, can be routed directly without repeating attempts that would produce the same failure.
The DLQ isolates the message so independent work can continue and allows the cause to be inspected. It must preserve identity, original content, error, and attempt history, along with alerts and a recovery procedure. That data helps distinguish a message failure from a consumer problem.
Transfer to that destination must preserve the delivery contract. When publishing to the DLQ and acknowledging the original are separate steps, the destination is confirmed before the original is considered resolved; a failure between those actions can still duplicate the transfer. In Kafka, the DLQ is usually another topic and needs explicit publication and offset advancement logic, which can be coordinated through a Kafka transaction within its scope.
An administrator or an AI-assisted tool can classify failures and propose corrections based on that data. Before replay, the cause and correction are validated, traceability is preserved, and the rate of reentry is controlled. Attempts of the same operation retain their identity so they remain idempotent. If the correction changes the meaning of an already confirmed fact, it must be represented as a new operation linked to the original.
Separating a failed message allows independent work to continue. If later events depend on that message, skipping it can break the business sequence. In that case, the affected sequence must be stopped or reconciliation defined before continuing. The DLQ preserves the problem for resolution; it does not automatically complete the original operation.
Choosing the model and technology
The choice must start with whether directed tasks are being distributed or a stream is being retained for independent readers. Routing, retention, recovery, ordering, and load are then specified.
| Technology | Model that may be useful | Decisions that remain |
|---|---|---|
| Kafka | Partitioned log, independent groups, and replay of retained data. | Keys, partitions, offsets, retention, and processing order. |
| SQS | Managed queues for asynchronous work. | Standard or FIFO mode, visibility, duplicates, and ordering per group where applicable. |
| RabbitMQ | Routing through exchanges and task queues; Streams offers another model. | Queue type, acknowledgements, concurrency, and recovery of failed messages. |
A managed service reduces part of infrastructure operations. The application contract still needs capacity limits, idempotency, and failure handling. No technology replaces those decisions.
What to defend in an interview
The design must allow a request to be followed from acceptance through completion, and then the same path to be repeated with a failure introduced. Decisions worth justifying include:
- Where work is persisted before responding and how its result is queried.
- How much backlog a burst produces and how much waiting time is acceptable.
- Which operations must preserve order and how their load is distributed.
- What happens if an effect completes and its acknowledgement is lost.
- When admission is limited, a message is retried, or it is routed to the DLQ.
Useful metrics are the age of pending work, queue depth, in-flight messages, lag by group or partition, end-to-end latency, and duplicate effects. Messages awaiting recovery and stopped consumers must also be observed: an empty queue can coexist with work that ended up in a DLQ and remains unresolved.