Guides

Task-oriented guides for common Datum use cases. Start here after the core concepts when you need to apply a specific part of the API.

Core Library

These guides cover datum-core: stream construction, graph wiring, concurrency primitives, backpressure, error handling, testing, and actor interop.

Guide	What it covers
Modularity And Composition	reusable flows, materialized-value composition, BidiFlow, partial graphs, attributes
Working with Graphs	GraphDSL, junctions, partial graphs, and reusable graph shapes
Custom Stream Processing	GraphStage, GraphStageLogic, handlers, async callbacks, logging, monitoring
Buffers & Rate	Backpressure strategies, buffers, throttle, conflate, and expand
Error Handling	recover, supervision deciders, restart sources and flows
Futures Interop	map_async, map_async_unordered, map_async_partitioned, Tokio dispatch
Pipelining And Parallelism	bounded future concurrency and async-boundary fused-region isolation
Context Propagation	SourceWithContext, FlowWithContext, offset/trace context, actor ask context
Actors Interop	ActorFlow::ask, ActorSource, ActorSink, ActorPubSub, Ractor integration
Concurrency Primitives	Signal, Subscription, Channel, Topic, overflow policies, and performance targets
Signal & Subscription	the state cell in depth: coalesced vs lossless feeds, get() semantics, pitfalls, integration
Channel	bounded MPSC handoff in depth: send/try_send, graceful close, vs SourceQueue/MergeHub
Topic	pub-sub broadcast in depth: overflow-policy selection, ordering, vs BroadcastHub
Dynamic Streams	KillSwitch, MergeHub, BroadcastHub, PartitionHub
Streaming IO	File IO, framing, compression, TCP, IO adapters (as_input_stream/as_output_stream)
Adding A Satellite	Naming, metadata, CI, docs, benches, and publish ordering for new datum-* crates
Testing Streams	TestSource, TestSink, and the TestKit probes
Substreams	group_by, split_when/split_after, flat_map_concat/flat_map_merge

Platform Guides

Section	What it covers
Persistence	event sourcing, durable state, storage backends, CQRS queries, managed projections, and schema evolution
Python	datum-stream package, Python blueprints, Arrow UDFs, and Datum Connect
Networking	datum-net TLS, UDP, QUIC carriers and local or remote StreamRefs
Cluster	datum-cluster membership plus datum-cluster-sharding entity routing
Operations	datum-agent, DCP, and the datum CLI

Kafka, PostgreSQL CDC, and SQL live in the top-level Integrations section.