Datum

Appearance

Design Principles ​

Datum mirrors Akka/Pekko Streams Typed in shape and behavior, but uses Rust-native error handling, Tokio-first async, and a split graph-validation model: compile-time checks for the static typed subset, runtime validation for dynamic graph completeness and erased interop. This page summarizes the rules behind the API and links to the deeper concept pages.

Blueprints first ​

Building a Source, Flow, Sink, or graph constructs an immutable blueprint. Construction has no side effects: it does not spawn actors, open files, start timers, or pull elements.

Execution starts at materialization: run, run_with, materialize, or the GraphDSL run helpers. Materialization returns handles such as StreamCompletion<T> immediately, while stream completion is observed through those handles.

rust
use datum::{Keep, Sink, Source};

let (_source_mat, completion) = Source::from_iter([1_u64, 2, 3])
    .to_mat(Sink::collect(), Keep::both)
    .run()
    .unwrap();

assert_eq!(completion.wait().unwrap(), vec![1, 2, 3]);

See Blueprint vs. Run and Materialization for the full model.

Demand and backpressure ​

Datum streams are demand-driven. A downstream stage requests work, upstream emits only when there is demand, and bounded buffers define the amount of permitted run-ahead. That applies to linear flows, GraphDSL junctions, actor interop, StreamRefs, queues, and IO adapters.

See Backpressure and Execution Model.

Graph correctness ​

Static Rust graph construction uses typed ports and TypedGraphBuilder open-port tokens: Outlet<T> -> Inlet<U> mismatches fail to compile, and double-wiring a token is a Rust move error.

Compile-time types still describe external shapes such as FlowShape<In, Out>, SourceShape<Out>, and SinkShape<In>. Full shape completeness, cycles, dynamic arity, method-based wire, and explicit erased interop through connect_any remain runtime-validated on the mutable GraphBuilder.

Rust-native API choices ​

Datum uses Rust names where the Scala name does not translate directly. For example, Akka's collect maps to filter_map in Datum because that is the Rust iterator vocabulary.

Failures flow through Result<T, StreamError>, exposed as StreamResult<T>. Ordinary operator failures should return errors, not panic.

rust
use datum::{Source, StreamError, StreamResult};

let result: StreamResult<Vec<u64>> = Source::from_iter([4_u64, 0, 2])
    .try_map(|item| {
        8_u64
            .checked_div(item)
            .ok_or_else(|| StreamError::Failed("zero divisor".to_owned()))
    })
    .run_collect();

assert_eq!(result, Err(StreamError::Failed("zero divisor".to_owned())));

Tokio first ​

Tokio is the async foundation. Ractor runs on Tokio, and map_async, actor ask, async sources, async sinks, timers, and IO adapters integrate with Tokio. The pure synchronous fused path can still run without dispatching per element onto Tokio.

Unsafe code is forbidden ​

The core crate declares #![forbid(unsafe_code)]. Performance work is done with typed fast paths, bounded queues, actor handoff tuning, and fused execution, not by adding unsafe escape hatches.

Reactive Streams interop is not a goal ​

Datum mirrors the Akka Streams programming model, but it does not implement Akka's Reactive Streams publisher/subscriber bridge APIs. Rust-native Source, Flow, Sink, GraphDSL, actor interop, and IO adapters are the supported boundaries.

Next steps ​