Appearance
Benchmarks
Datum benchmark rows are recorded in the repo before they become claims. The comparison harnesses run same-session A/B where the row needs it, use warmed JVM baselines for Akka/Pekko/FS2/ZIO rows, start fresh brokers for Kafka-backed rows, and report process CPU beside wall time. Rows that move data also carry correctness counters, committed-offset checks, row counts, or deterministic checksums.
This page copies numbers from roadmap/benchmarks/*.md. It does not recompute ratios or replace the raw records. Losses stay in the public summary: a row with better wall time and worse CPU, a backend that misses another Datum backend, or a Python binding gate miss is still listed.
Akka/Pekko Family
Labels: Pekko Connectors Kafka 1.1.0 with Apache Kafka 4.2.0, 2026-07-08, 1,000,000 records, 256-byte payloads, 16 partitions; Kafka is native-only as of v0.11.0, so any rdkafka rows here are historical pre-removal comparisons, not selectable Datum backends; Akka Streams 2.9.0-M2, RT5 forced-remote StreamRefs, 2026-06-23; graph rows are v0.4.0, 2026-06-11, Akka Streams 2.9.0-M2.
| Area | Scenario | Datum row | Comparison row | Loss or caveat |
|---|---|---|---|---|
| Remote StreamRefs | streamref_remote_tcp_1024 | 1,951 us/op; CPU 3,149 us/op; alloc 133,585 B/op; RSS 63,872 KiB | Akka forced remote: 18,612 us/op; CPU 46,293 us/op; alloc 4,765,533 B/op; RSS 757,312 KiB | Plaintext TCP only for the fair baseline; QUIC rows are labeled separately. |
| Kafka consume | consume_committable_1M, native-only client | wall p50/p99 2,299.984/2,392.061 ms; CPU 1,720 ms; RSS 327,376 KiB | Pekko wall p50/p99 2,329.851/2,864.320 ms; CPU 2,400 ms; historical pre-removal rdkafka wall p50/p99 2,104.547/2,208.612 ms | Native beats Pekko wall/CPU but misses the removed rdkafka comparison row on the public group path. |
| Kafka latency | e2e_latency_1M, native-only client | wall p50/p99 3,186.930/3,256.021 ms; latency p50/p99 1.788/6.777 ms; CPU 5,120 ms | Pekko wall p50/p99 4,722.438/5,322.990 ms; latency p50/p99 4.773/9.151 ms; CPU 14,500 ms | Historical rdkafka comparison row has wall p50/p99 3,805.536/3,964.183 ms but latency p50/p99 137.438/363.036 ms. |
| Kafka rebalance | rebalance_disruption_1M, native-only client | wall p50/p99 1,982.586/4,976.025 ms; CPU 1,610 ms; pause 1.455 ms | Pekko wall p50/p99 34,714.154/34,844.183 ms; CPU 3,390 ms; pause 1.642 ms | Native p99 is 368.144 ms above the historical rdkafka p99 4,607.881 ms. |
| Graph execution | opt-cycles no-regression rows | junction_merge_latest_10k: 891 us vs Akka 1,985 us, 2.23x; junction_concat_10k: 3.09 us vs Akka 932 us, 302x | Akka same-host JMH | Graph build is a parity range, 0.94-1.68x, not an across-the-board execution win. |
| Cyclic graph | cycle_merge_preferred_feedback_10k | 69.7 us; CPU 60.0 us; alloc 154,296 B/op | Akka 2,034 us; speedup 29.2x | Only the output-first MergePreferred(1) -> Broadcast(2) feedback shape uses this typed kernel. |
SQL Engines
Labels: Arroyo 0.15.0, Timeplus Proton v3.0.26, Apache Kafka 4.2.0, 2026-07-09 native-ingest remeasure for q0/q1/q2/q4/q5/q7, 10M-row Nexmark subset. q3/q8 are from the 2026-07-08 WP-PROTON three-engine run because the native-ingest remeasure did not rerun those queries. rdkafka ingest rows are historical pre-v0.11.0 comparisons; the Kafka path is native-only in current Datum.
All comparable ok rows in the source records had matching row counts and deterministic checksums. Cells are wall p50 ms / CPU ms.
| Query | Datum native-ingest | Arroyo | Proton | Historical Datum rdkafka-ingest | Result |
|---|---|---|---|---|---|
| q0 | 17,754.696 / 21,870.000 | 62,545.325 / 31,240.000 | 10,677.278 / 73,640.000 | 65,538.062 / 37,000.000 | Proton wall p50 is lower than Datum native; Proton CPU is 73,640.000 ms. |
| q1 | 15,590.800 / 19,810.000 | 61,596.819 / 29,330.000 | 10,496.049 / 72,530.000 | 65,235.932 / 36,830.000 | Proton wall p50 is lower than Datum native; Proton CPU is 72,530.000 ms. |
| q2 | 5,631.439 / 7,580.000 | 53,229.552 / 13,490.000 | 4,439.755 / 58,510.000 | 49,242.645 / 19,210.000 | Proton wall p50 is lower than Datum native; Proton CPU is 58,510.000 ms. |
| q3 | not measured in native-ingest remeasure | 1,783.127 / 510.000 | 1,517.328 / 700.000 | 1,324.456 / 370.000 | The four-way native-ingest claim is not made for q3. |
| q4 | 6,725.423 / 9,280.000 | 65,225.071 / 14,260.000 | 77,864.577 / 19,310.000 | 54,544.894 / 23,210.000 | Datum native has the lower wall p50 and CPU in this row. |
| q5 | 32,712.214 / 36,010.000 | 71,702.537 / 25,410.000 | 85,192.947 / 26,630.000 | 78,298.063 / 46,490.000 | Datum native has the lower wall p50; Arroyo has the lower CPU row at 25,410.000 ms. |
| q7 | 6,525.646 / 8,880.000 | 64,132.055 / 13,710.000 | 76,736.694 / 17,320.000 | 60,325.958 / 22,620.000 | Datum native has the lower wall p50 and CPU in this row. |
| q8 | not measured in native-ingest remeasure | 1,787.792 / 1,240.000 | 1,714.744 / 730.000 | 1,636.046 / 830.000 | The 2026-07-08 historical Datum rdkafka row has lower p50/p99 wall; Proton has lower CPU. |
Concurrency Primitives
Labels: M9 v0.9.0 record, captured 2026-07-02; JVM baselines used OpenJDK 26.0.1. Competitors are FS2 Channel/Topic/SignallingRef, ZIO Queue/Hub/SubscriptionRef, and Akka MergeHub/BroadcastHub. Cells are wall mean us/op / CPU us/op.
| Scenario | Datum | Best competitor target | Result | Caveat |
|---|---|---|---|---|
channel_mpsc_send_1024x1 | 159 / 250 | ZIO Queue 272 / 600 | pass | Run 1 missed; Run 2 is the shipped row. |
channel_mpsc_send_1024x1024 | 38,953 / 70,000 | Akka MergeHub 293,274 / 1,435,369 | pass | High-concurrency JVM rows are sparse and reported as captured. |
topic_fanout_1024x1024 | 61,891 / 70,000 | ZIO Hub 1,430,782 / 11,034,842 | pass | Round 1 missed several wall targets; Round 2 is the shipped row. |
topic_overflow_dropping_s64 | 2,207 / 2,667 | ZIO Hub.dropping 3,168 / 13,913 | pass | FS2 Topic and Akka BroadcastHub have no direct sliding/dropping topic policy. |
signal_get_r64 | 462 / 7,000 | ZIO SubscriptionRef 1,657 / 43,048 | pass | Round 1 measured Arc refcount contention; Round 4 is the shipped row. |
signal_propagation_1024x1024 | 6,254 / 53,333 | FS2 wall 522,147; ZIO CPU 7,768,200 | pass | FS2 is coalesced final-value observation; ZIO is lossless. |
subscription_lossless_1024x256 | 4,575 / 8,000 | ZIO SubscriptionRef.changes 197,036 / 1,867,783 | pass | FS2 comparison is the closest Topic-fed state shape, not native lossless state. |
CDC
Labels: Debezium PostgreSQL connector 3.6.0.Final, PostgreSQL 17.10, Kafka 4.3.1. Debezium target captured 2026-07-03T18:29:42Z; Datum row captured 2026-07-04T09:27:26Z. Workload: 100 tx/s for 60s, one PostgreSQL table, pgoutput, snapshot.mode=no_data.
| Metric | Debezium PostgreSQL connector | Datum datum-cdc pgoutput v1 | Result or scope |
|---|---|---|---|
| Correctness and throughput | 6,000 observed events; 98.13 events/s | 6,000 observed events; 100.02 events/s | Both rows pass: 0 duplicate, 0 missing, 0 mismatched, per-key ordered. |
| Latency | p50 519.43 ms; p99 969.19 ms | p50 0.14 ms; p99 4.91 ms | Narrow single-table mixed insert/update/delete workload only. |
| Consumer CPU and RSS | 9.77 core-s; 0.1545 avg cores; 1,600.76 MiB peak RSS | 6.69 core-s; 0.1080 avg cores; 9.08 MiB peak RSS | The row is not a snapshot or multi-table claim. |
| Slot lag | retained WAL 1,648,096 bytes; confirmed lag 873,072 bytes | retained WAL 844,288 bytes; confirmed lag 4,816 bytes | Same slot/publication shape; one local PostgreSQL instance. |
Python Bindings
Labels: CPython 3.14.6, rustc 1.97.0 for PY-1/PY-2/PY-3. PY-2 used PyArrow 24.0.0; PY-3 used PyArrow 25.0.0. The lead-refined PY-2 compute qualification is dated 2026-07-09.
| Area | Scenario | Baseline | Python or Connect row | Binding cost |
|---|---|---|---|---|
| Plan build | PY-0 10,000,000-element linear pipeline | rust_direct 650.916 ms wall; 650.000 ms CPU | python_built 661.324 ms wall; 661.386 ms CPU | +2.082 ms wall delta per iteration; no per-element Python execution. |
| Graph build | PY-1 broadcast_zip, 10,000,000 elements | Rust 14,306.623 ms wall; 14,310.000 ms CPU | Python-built 14,662.667 ms wall; 14,655.099 ms CPU | +71.209 ms wall delta per iteration. |
| Graph build | PY-1 balance_merge, 10,000,000 elements | Rust 8,219.938 ms wall; 8,230.000 ms CPU | Python-built 8,370.402 ms wall; 8,369.391 ms CPU | +30.093 ms wall delta per iteration. |
| In-process UDF | PY-2 compute, 1,048,576-byte batches | Rust compute 2,456.554 ms wall; 2,450.000 ms CPU | UDF compute 3,411.441 ms wall; 3,400.000 ms CPU | Miss: 38.9% wall and 38.8% CPU overhead; crossing constant 25.688 us/batch. |
| In-process UDF | PY-2 compute, 16,777,216-byte batches | Rust compute 5,400.868 ms wall; 5,400.000 ms CPU | UDF compute 2,368.794 ms wall; 2,370.000 ms CPU | 16 MiB compute row clears the refined gate; crossing constant 27.649 us/batch. |
| Connect submit | PY-3a no-input plan submit | in-process p50/p99 8.470/23.496 us | loopback TCP p50/p99 44.936/119.573 us | +36.466 us p50; +96.077 us p99. |
| Worker scaling | PY-3b CPU-bound pure-Python UDF, 1 MiB batches | 1 worker 3,113.105 ms wall; 3,160.000 ms server+worker CPU | 4 workers 832.890 ms wall; 3,200.000 ms server+worker CPU | 3.74x speedup vs 1 worker. |
| Worker shm | PY-5 identity UDF, 16 MiB batches, four workers | pipe 1,310.636 ms wall; 2,040 ms server+worker CPU; 1,038,540 KiB RSS | shm 1,134.384 ms wall; 1,310 ms server+worker CPU; 679,780 KiB RSS | 1.155x wall; remains opt-in because both eight-worker wall rows regressed. |
What We Do Not Claim
These rows are single-host records. They do not claim cloud-managed service performance, multi-host cluster throughput, tuned Akka scheduler variants, ZIO Kafka, PySpark, PyFlink, or systems not listed in the benchmark records. ZIO Kafka is pending, not inferred from ZIO Queue/Hub rows.
Raw records and harnesses live in the repo under roadmap/benchmarks/, baselines/, crates/datum-core/benches/, crates/datum-py/benchmarks/evidence/, and the benchmark-specific target/... paths named in those records.