Persistence backends

The three storage contracts are byte-oriented: Journal, SnapshotStore, and DurableStateStore. A backend may implement all three, while a behavior receives only the trait objects it needs. Serialization and domain adaptation remain in the behavior layer.

Features

protobuf is enabled by default. Database backends and sharding integration are opt-in.

[dependencies]
datum-persistence = { version = "0.11.3", features = ["postgres"] }

For embedded RocksDB, select rocksdb instead; for Cassandra/ScyllaDB, cassandra. Features can be combined; sharding is covered in the sharded entity recipe.

Memory backends and the testkit bar

MemoryJournal, MemorySnapshotStore, and MemoryDurableStateStore are conformant reference implementations. They are ideal for behavior tests and also implement the currently shipped in-memory query capabilities. They are process-local and not production durability.

The public testkit makes the backend contract executable. Run the relevant functions against a new, isolated backend instance:

use datum_persistence::{
    MemoryDurableStateStore, MemoryJournal, MemorySnapshotStore,
    assert_durable_state_store_conformance, assert_journal_conformance,
    assert_snapshot_store_conformance,
};

assert_journal_conformance(&MemoryJournal::new()).await.unwrap();
assert_snapshot_store_conformance(&MemorySnapshotStore::new())
    .await
    .unwrap();
assert_durable_state_store_conformance(&MemoryDurableStateStore::new())
    .await
    .unwrap();

The query conformance functions live under datum_persistence::testkit. FailingJournal wraps an in-memory journal and can reject or fail the next append to test the rejection/failure boundary and persist-failure recovery policies.

PostgreSQL

Enable the postgres feature. PostgresPersistence uses SQLx with Tokio and rustls-ring and implements the journal, snapshot, and durable-state stores. Construction is side-effect-free; connect or initialize creates the pool and executes idempotent schema DDL.

use std::sync::Arc;

use datum_persistence::{
    EventSourcedRuntime, PostgresDeletePolicy, PostgresPersistence,
    PostgresPersistenceConfig,
};

let mut config = PostgresPersistenceConfig::new(
    std::env::var("DATABASE_URL").expect("DATABASE_URL must be set"),
);
config.schema = "datum_persistence".into();
config.max_connections = 16;
config.delete_policy = PostgresDeletePolicy::Soft;

let backend = Arc::new(PostgresPersistence::connect(config).await.unwrap());
let runtime = EventSourcedRuntime::new(backend.clone())
    .with_snapshot_store(backend.clone());

PostgresPersistenceConfig exposes:

• Pool sizing and lifetime: max_connections, min_connections, acquire_timeout, idle_timeout, and max_lifetime.
• Layout: schema and PostgresTableNames for event_journal, snapshot, durable_state, and journal_head.
• Journal behavior: delete_policy and replay_page_size.
• Live-query polling settings: PostgresPollingConfig { min_interval, max_interval, batch_size }. Empty polls double the delay up to the maximum; any emitted row resets it.

Use PostgresPersistenceConfig::for_pool() with PostgresPersistence::from_pool when the application owns an existing sqlx::PgPool. pool() returns the connected pool after initialization, and config() returns the immutable effective configuration.

Inspect or manage the schema

schema_ddl() returns the exact idempotent DDL that initialization executes, including tables, indexes, and consistency checks. Creating a blueprint validates configuration without connecting.

use datum_persistence::{PostgresPersistence, PostgresPersistenceConfig};

let config = PostgresPersistenceConfig::new("postgres://datum:secret@localhost/datum");
let backend = PostgresPersistence::new(config).unwrap();
let ddl = backend.schema_ddl();
assert!(ddl.contains("CREATE TABLE IF NOT EXISTS"));

Datum's live conformance tests target PostgreSQL 18 and are gated by DATABASE_URL. When it is unset, they skip cleanly. Each test creates a unique schema and drops it afterward, so the configured database user needs create/drop-schema privileges in the test database.

PostgresPersistence also implements all four query capability traits. Current queries run to a fixed materialization boundary; live queries poll adaptively. Tag and durable-state checkpoints use Offset::Timestamp, and resuming the exact timestamp/sequence pair is inclusive, so projections must tolerate duplicate delivery at that position.

Soft versus hard journal delete

PostgresDeletePolicy::Soft is the default: rows are marked deleted but remain physically present. This preserves operational evidence and makes accidental policy changes less destructive. PostgresDeletePolicy::Hard physically removes the selected journal rows. Both modes preserve the journal_head high watermark, so the next append cannot reuse deleted sequence numbers.

Deletion is inclusive in both modes. Choose hard delete only when your retention/compliance model requires physical removal and your backups and replicas follow the same policy.

RocksDB

Enable the rocksdb feature. RocksDbBackend is an embedded combined journal, snapshot store, and durable-state store. Opening is synchronous because it establishes the database before returning a usable handle.

use std::sync::Arc;

use datum_persistence::{
    EventSourcedRuntime, RocksDbBackend, RocksDbDurability, RocksDbOptions,
};

let backend = Arc::new(
    RocksDbBackend::open_with_options(
        "./data/persistence",
        RocksDbOptions {
            command_capacity: 512,
            durability: RocksDbDurability::Sync,
        },
    )
    .unwrap(),
);
let runtime = EventSourcedRuntime::new(backend.clone())
    .with_snapshot_store(backend.clone());

RocksDbOptions::default() uses a bounded command capacity of 256 and RocksDbDurability::Sync. RocksDbBackend::open(path) is the shorthand for those defaults.

Durability modes

Mode	WAL	Acknowledgement
Sync	Enabled	Fsync before acknowledging every mutation. This is the default.
Relaxed	Enabled	Acknowledge before fsync. A process, OS, or power failure may lose recently acknowledged writes.

Use Relaxed only for explicitly rebuildable data or labeled benchmarks. It changes the durability contract, not just throughput.

Single-owner thread model

One long-lived thread owns the rocksdb::DB. Async callers communicate through the bounded Tokio command channel and receive one-shot replies; operations do not call spawn_blocking and do not block Tokio executor threads. Entity concurrency therefore does not create per-entity storage threads. RocksDB may still use its own bounded flush/compaction workers.

Dropping the last backend handle closes the command channel and joins the owner thread, allowing the same path to be reopened deterministically. Keep the backend in an Arc and share it among behavior runtimes.

RocksDbBackend implements all four query capabilities through the same owner channel. Live cursors sleep on a watch notification instead of polling; tag and durable-state checkpoints use the durable per-database Offset::Sequence counter and resume strictly after the supplied value.

Cassandra / ScyllaDB

The cassandra feature provides CassandraPersistence over the pure-Rust async scylla CQL driver — no C driver bindings, no blocking seam. It speaks to Apache Cassandra and ScyllaDB.

use datum_persistence::{CassandraPersistence, CassandraPersistenceConfig};

let mut config = CassandraPersistenceConfig::new(["127.0.0.1:9042"]);
config.keyspace = "orders".into();
let backend = CassandraPersistence::connect(config).await?;

connect creates the keyspace (SimpleStrategy, replication factor configurable, default 1 for development) and its tables, and prepares all statements; construction itself performs no IO.

Storage model and caveats:

• The journal partitions events as ((persistence_id, partition_nr), sequence_nr) with a configurable partition_size (default 100 000 events). One AtomicWrite becomes one single-partition unlogged batch — atomic and isolated within the partition. An atomic write that would cross a partition boundary is rejected (the unsupported-atomic-batch rejection from the failure taxonomy), mirroring akka-persistence-cassandra's behavior.
• The high watermark and deleted_to live in a separate head table updated after the event batch; cross-table atomicity is deliberately not claimed (Akka parity — document your recovery expectations accordingly).
• Durable state uses lightweight transactions: revision 1 inserts with IF NOT EXISTS, later revisions compare-and-set with IF revision = ?, and a failed [applied] maps to the RevisionConflict error. This is a Datum extension — Akka ships no Cassandra durable-state plugin. journal_head transitions use the same LWT mechanism by default (CassandraWriteMode::Lwt); an opt-in CassandraWriteMode::SingleWriter replaces that head write with a plain statement to remove the Paxos round on every append, relying on the existing reconcile-on-read orphan repair for crash safety. SingleWriter requires the same single-writer-per-entity discipline the runtime already provides in-process and is not safe with concurrent cross-process writers to the same persistence id — see the CassandraWriteMode API docs for the full rationale.
• Queries: per-persistence-id (Current and Live via adaptive polling) and Current persistence_ids are supported. events_by_tag is served from a dedicated tag_events index table maintained in the same unlogged batch as the event insert; DurableStateChangesQuery is served from a durable_state_changes table holding one coalesced row per id per tag. Because each index table is a different Cassandra partition from its source table, neither write is atomic with it — a documented eventual-consistency window, mirroring akka-persistence-cassandra's own tag_views table. Live id discovery (persistence_ids(QueryMode::Live)) remains unsupported.

Conformance tests are gated on CASSANDRA_NODE (for example 127.0.0.1:9042) and skip cleanly when unset; each test creates and drops an isolated keyspace, and the full suite runs under both CassandraWriteMode variants.

Backend selection

Need	Backend
Fast deterministic tests and query examples	Memory backends
Shared durable storage, external operations, and SQL observability	PostgreSQL
Embedded local durability with a bounded single-owner execution seam	RocksDB
Wide-column, multi-node write scaling with tunable consistency	Cassandra / ScyllaDB

Backend query capability availability is called out separately in the read-side guide.