Backpressure: when the producer is 3× faster, how does the queue not overflow?
TL;DR
Backpressure lets the receiving side tell the sender to slow down. A buffer has a high watermark that triggers PAUSE and a low watermark that triggers RESUME. The queue never overflows and the whole system runs at the consumer’s speed.
A producer generates data three times faster than the consumer. What happens to the queue in between? Left alone, it grows until memory runs out (OOM) or latency grows without bound. Backpressure is the mechanism by which the receiving side tells the sender to "slow down". A common implementation uses two thresholds: a high watermark that signals PAUSE and a low watermark that signals RESUME.
The watermark mechanism (8-slot buffer)
| Point | What happens |
|---|---|
| Hits the high watermark at 6/8 | A PAUSE signal goes back to the producer. |
| Peaks at 7/8 | 2 in-flight messages still arrive; setting the threshold below capacity is exactly to absorb them. |
| Falls to the low watermark at 2/8 | A RESUME signal; the producer sends again. |
A producer 3× faster doesn't raise throughput: the system always runs at the consumer's 1 msg/s.
Why two thresholds?
- With a single threshold, the producer would keep stopping and starting around that exact point (flapping).
- The gap between the two thresholds creates hysteresis, so state changes are stable.
- The high threshold sits below capacity to leave room for messages already on the way.
Strategies when the queue is full
| Strategy | Behaviour | Fits when |
|---|---|---|
| Block the producer (backpressure) | The producer waits until there's room | Data must not be lost |
| Drop/sample | Discard old or new messages | Real-time data where loss is acceptable (telemetry, location) |
| Reject (429) | Return an error so the client retries later | Public APIs, load control at the edge |
| Scale consumers | Add processing resources | Load regularly increases |
Built into many tools
- Node.js streams:
highWaterMarkand thedrainevent. - Reactive Streams, RxJS: requesting a desired number of items.
- TCP flow control: the receive window.
- Kafka: monitoring consumer lag to see whether consumers keep up.
How to apply it
- Don't use unbounded queues in production code; it's the fastest route to OOM.
- Choose a full-queue strategy based on the nature of the data: block, drop or reject.
- Fix the root cause: if the producer is always faster, scale consumers; backpressure only keeps the system from collapsing.
- Monitor: queue depth and wait time are early indicators of overload.
Sources & references
- Backpressuring in Streams — Node.js