Backpressure

Backpressure is how a stream prevents a fast producer from overwhelming a slow consumer. Without it, unbounded queuing or dropped elements are the only alternatives.

How Datum handles it

Datum's linear fused path is pull-based at the implementation level: the sink drives the loop by pulling elements from the chain one at a time. No element is produced until the downstream consumer is ready to receive it.

The pull protocol means:

• A Source never emits an element until asked.
• A slow Sink (e.g. one that writes to disk or performs an actor ask) automatically slows the whole chain — the source stops being pulled.
• No intermediate buffer is required for a fully fused linear chain.

Bounded operators

Some operators introduce explicit bounds:

Operator	Behavior
.buffer(n, strategy)	Decouple upstream/downstream with a bounded queue; strategy determines what happens when full (DropHead, DropTail, DropBuffer, Fail, Backpressure)
.conflate(f)	Merge excess upstream elements when the downstream is slow
.expand(f)	Produce extra elements from each input when the downstream is faster
.throttle(n, per, burst, mode)	Rate-limit to n elements per per duration with a burst allowance; mode is ThrottleMode::Shaping (slow elements down) or ThrottleMode::Enforcing (fail on excess)

The default strategy when no .buffer(...) is present is strict backpressure: the source is not asked for more until the downstream has finished with the previous element.

The GraphDSL layer

In the GraphDSL layer, backpressure flows through typed ports (Inlet/Outlet). The push/pull protocol is explicit: a stage calls pull(inlet) to demand an element and push(outlet, elem) to emit one. The fused executor connects these ports and enforces that no stage pushes without a pending demand from downstream.

Junctions like Merge and Balance implement their own demand-forwarding logic. For example, Balance forwards demand to whichever output has capacity, and Merge forwards demand upstream when any of its inputs have capacity.

Relation to Akka Streams

Datum's backpressure model mirrors Akka Streams' demand-driven protocol. In Akka, the underlying mechanism is Reactive Streams (Publisher/Subscriber with request(n) signals). Datum does not implement the Reactive Streams interfaces — its backpressure is internal and not exposed as a cross-library interop surface.

Practical consequences

For most linear stream use-cases, backpressure is automatic and invisible: use the operators you need, and the chain self-regulates. You only need to think about it when:

• You are bridging Datum with an external system that can produce faster than you consume (use .buffer(n, OverflowStrategy::Backpressure) or .throttle(...)).
• You are building a custom GraphStage and need to decide when to pull from upstream.
• You observe growing memory usage, which usually signals an unbounded queue upstream of a slow consumer.