Datum

Appearance

CLI ​

datum is the command-line client for a datum-agent daemon. It speaks DCP (the Datum Control Protocol) and gives you the job control surface — list, inspect, start, drain, stop, restart — plus live lifecycle-event and stream-metric feeds. It is the same typed DCP client the TUI and cluster tooling use, wrapped in a clap CLI.

sh
cargo install datum-cli   # installs the `datum` binary

Everything the CLI does goes through a running datum-agent; start one first (see the datum-agent guide). The CLI connects to the loopback TCP dev listener by default.

Connecting ​

There are two transports, matching the two DCP listeners:

  • Loopback TCP (local dev) — the default. No flags needed; the CLI connects to 127.0.0.1:9555. Point it elsewhere with --addr host:port. The agent refuses to bind a plaintext listener to a non-loopback address, so this stays a local-only convenience.
  • QUIC + mTLS (remote) — pass all three of --tls-ca, --tls-cert, --tls-key (PEM or DER). This is the default for anything non-local.
sh
# Local dev — loopback TCP.
datum ps

# A non-default local port.
datum --addr 127.0.0.1:9600 ps

# Remote agent over QUIC + mTLS.
datum --addr agent.internal:9555 \
  --tls-ca   ca.pem \
  --tls-cert client.pem \
  --tls-key  client-key.pem \
  ps

Passing some-but-not-all of the TLS flags is a usage error: "QUIC+mTLS mode needs --tls-ca, --tls-cert, and --tls-key; pass all three or omit all three for loopback TCP."

Global options ​

These apply to every subcommand:

FlagMeaning
--addr <ADDR>DCP address. Default 127.0.0.1:9555.
--tls-ca <PATH>CA certificate for QUIC+mTLS mode.
--tls-cert <PATH>Client certificate for QUIC+mTLS mode.
--tls-key <PATH>Client private key for QUIC+mTLS mode.
--jsonEmit machine-readable JSON instead of human tables.
$ datum --help
Manage Datum jobs through the Datum Control Protocol

Usage: datum [OPTIONS] <COMMAND>

Commands:
  ps       List jobs
  nodes    List cluster nodes
  status   Show one job
  start    Start a registered factory
  submit   Submit a job through cluster placement
  drain    Gracefully drain a job
  stop     Stop a job immediately
  restart  Restart a job
  events   Tail lifecycle events
  metrics  Show stream metrics for one job
  help     Print this message or the help of the given subcommand(s)

nodes and submit, and the --cluster flag on ps/status/drain/stop, apply when the agent is a cluster node — see Cluster subcommands.

Subcommands ​

datum ps — list jobs ​

$ datum ps
JOB           ID  STATE    DESIRED  GEN  STARTS  RESTARTS  ACTIVE
------------  --  -------  -------  ---  ------  --------  ------
ingest        1   Running  Running  2    1       1         4
daily-rollup  2   Drained  Stopped  1    1       0         -

With no jobs, datum ps prints no jobs. ACTIVE is the count of active stream instances (a dash when the job reports none).

datum status <job> — inspect one job ​

$ datum status ingest
FIELD                 VALUE
--------------------  -----------------------
job                   ingest
id                    1
state                 Running
desired               Running
generation            2
starts                1
restarts              1
last_start_ms         1751622013422
last_exit_ms          1751621948217
last_exit_reason      restart policy: backoff
backoff_remaining_ms  -
drain_remaining_ms    -
drain_supported       true
active_streams        4

datum start <factory> [--name <name>] [--param k=v ...] ​

Jobs start from registered factories, not arbitrary blueprints — the daemon registers named JobSpec factories at startup and you start instances by factory name (closures don't cross the wire; see the datum-agent guide). --name gives the instance a name (defaults to the factory name); repeat --param key=value to pass parameters to the factory.

$ datum start ticker --name ingest --param shard=0
started 'ingest' from factory 'ticker' — state Running, generation 1

A malformed --param fails before connecting, with exit code 2:

$ datum start ticker --param bad-param
invalid --param 'bad-param'; use --param key=value.

datum drain <job> — graceful drain ​

Drain flips the job's kill switch: intake stops, in-flight elements finish, then the job settles. The CLI polls until the job reaches a terminal drain state:

$ datum drain ingest
drained 'ingest' — in-flight streams completed

If the job hasn't settled within the poll window, the CLI reports the current state and points you at datum status <job> to keep watching. Jobs whose factory doesn't wire in a kill switch report drain_supported = false and must be stopped instead.

datum stop <job> — stop immediately ​

$ datum stop ingest
stopped 'ingest' — state Stopped

datum restart <job> — restart ​

Rematerializes the whole graph and advances the generation:

$ datum restart ingest
restarted 'ingest' — state Running, generation 2

datum events [--follow] — lifecycle feed ​

Without --follow, the CLI collects the events available now and prints a table (or no events observed — run \datum events --follow` to keep waiting`):

$ datum events
SEQ  TIME_MS        JOB     ID  GEN  KIND       DETAIL
---  -------------  ------  --  ---  ---------  -----------------------
1    1751622013001  ingest  1   1    Submitted  -
2    1751622013010  ingest  1   1    Started    -
3    1751622048217  ingest  1   2    Restarted  restart policy: backoff

With --follow, it streams events one per line until interrupted:

$ datum events --follow
1 Submitted 'ingest' generation 1
2 Started 'ingest' generation 1
3 Restarted 'ingest' generation 2

Events carry monotonic sequence numbers, so a client can detect a gap and resync with ps/status.

datum metrics <job> [--follow] — stream metrics ​

Instrumentation samples for one job's streams (opt-in per-job counters from the factory). Element counts are grouped for readability:

$ datum metrics ingest
TIME_MS        STREAM    ID  STATE    ELEMENTS   RESTARTS  UPTIME_MS
-------------  --------  --  -------  ---------  --------  ---------
1751622061044  ingest:2  7   Running  1,204,338  1         47622

--follow renders a fresh table per metric tick. If the job has no metrics yet, the CLI prints no metrics for '<job>' yet.

Cluster subcommands ​

When the agent you connect to is a cluster node (a ClusterAgent), these commands read and drive the whole cluster over DCP. They fan out to peer node sessions with bounded partial-result semantics: an unreachable node degrades to a row in a trailing NODE / ERROR table instead of hanging the command.

datum nodes — membership & session view ​

$ datum nodes
NODE    MEMBER  UNREACHABLE  SESSION    AGENT_ADDR       ROLES
------  ------  -----------  ---------  ---------------  -------------
node-0  Up      false        connected  127.0.0.1:9555   agent,orders
node-1  Up      false        connected  127.0.0.1:9556   agent,worker

MEMBER is the member state; UNREACHABLE is the orthogonal reachability flag; SESSION is this node's DCP session state to that peer (connected, connecting, backing_off:…, closed). With no nodes it prints no cluster nodes.

datum ps --cluster — jobs across the cluster ​

$ datum ps --cluster
NODE    ADDRESS         JOB         ID  STATE    DESIRED  GEN  STARTS  RESTARTS  CLUSTER  PLACED_ON  PGEN
------  --------------  ----------  --  -------  -------  ---  ------  --------  -------  ---------  ----
node-0  127.0.0.1:9555  placed-job  3   Running  Running  1    1       0         yes      node-1     1
node-1  127.0.0.1:9556  remote-job  1   Running  Running  1    1       0         -        -          -

CLUSTER marks jobs the coordinator placed; PLACED_ON is the assigned node and PGEN the placement generation. With no jobs anywhere it prints no cluster jobs.

datum submit --cluster — place a job through the coordinator ​

Cluster jobs use the same registered-factory model as start (closures don't cross the wire). submit hands the factory + parameters to the placement coordinator, which chooses a node:

$ datum submit --cluster --factory ingest --name ingest-eu --param shard=0
submitted cluster job 'ingest-eu' from factory 'ingest' to node 'node-1' — state Running, placement generation 1

Placement flags:

FlagEffect
(none)LeastJobs — the eligible node with the fewest cluster jobs, tie-broken by node id
--role <r>constrain eligibility to members advertising role r
--node <id>pin the job to a specific node

datum submit requires --cluster; without it the CLI fails with a usage error (exit 2) and points you at datum start for local jobs.

--cluster on status / drain / stop ​

status, drain, and stop accept --cluster <job> to target a cluster-placed job by name, routing to whichever node currently runs it rather than the connected node's local registry:

$ datum status --cluster ingest-eu
$ datum drain  --cluster ingest-eu
$ datum stop   --cluster ingest-eu

See the datum-cluster guide for placement strategies and re-placement semantics.

--json for scripting ​

Add --json (globally) for machine-readable output. ps/status/start/ stop/drain/restart print a single pretty JSON document; the --follow feeds print one compact JSON object per line (newline-delimited JSON).

$ datum --json status ingest
{
  "job": {
    "name": "ingest",
    "job_id": 1,
    "state": "Running",
    "desired_state": "Running",
    "generation": 2,
    "starts_total": 1,
    "restarts_total": 1,
    "last_start_at_ms": 1751622013422,
    "last_exit_at_ms": 1751621948217,
    "last_exit_reason": "restart policy: backoff",
    "backoff_remaining_ms": null,
    "drain_remaining_ms": null,
    "drain_supported": true,
    "active_streams": 4
  }
}

ps wraps the array in {"jobs":[…]} ({"jobs":[]} when empty); action commands wrap in {"action","message","job"}. Follow lines look like:

$ datum --json events --follow
{"event":{"sequence":2,"timestamp_ms":1751622013010,"name":"ingest","job_id":1,"generation":1,"kind":"Started","detail":""}}

Fields map straight onto the DCP wire types, so jq and friends work as expected — datum --json ps | jq '.jobs[] | select(.state=="Running").name'.

Exit codes ​

CodeMeaningExamples
0Success.any command that completed
1The agent rejected the request.unknown job/factory, name conflict, unauthorized, protocol mismatch
2Usage error or could not connect.bad flags/args, malformed --param, agent unreachable

A rejected request (exit 1) prints the agent's reason plus a targeted fix:

$ datum status missing
datum-agent rejected the request: job "missing" not found
Check the job or factory name with `datum ps`.

A connection failure (exit 2) tells you how to reach an agent:

$ datum ps
could not connect to datum-agent at 127.0.0.1:9555: connection refused
start datum-agent or pass --addr host:port; loopback TCP defaults to 127.0.0.1:9555

These codes are stable enough to branch on in scripts: 0 = done, 1 = the agent said no (don't retry blindly), 2 = fix your invocation or start/point-to an agent.

See also ​

  • Agent — the daemon the CLI talks to: jobs, drain, restart, DCP.
  • Membership — membership, downing, and job placement behind nodes, ps --cluster, and submit --cluster.