ADR-099
Durable-lite run resume — the trace IS the checkpoint
accepted · 2026-07-05 · L2 L4 · cites 3
0ADR-099 — Durable-lite run resume · the trace IS the checkpoint #
- Status: Accepted (2026-07-05) — implemented by the engine
feat/resume-implarc:nika-runtimeresume keys (JCS + blake3) + the--resumeskip fold + the pause rider ·nika-cli run --resume/--from/--answer·EventKind::{TaskCacheHit, WorkflowPaused}· exit code4(paused) · the 3 conformance fixtures as engine e2e tests (crates/nika-cli/tests/resume_e2e.rs). - Surface: CLI + trace/event vocabulary ONLY. Zero envelope change ·
zero new YAML · the 4-verbs and
nika: v1locks are untouched. - Home: the CANONICAL prose lives in the
nika-specrepo'sadr/(language-surface decisions — the relative spec links below resolve THERE); this copy registers the number in the shared Nika ADR series (the enginedocs/adr/registry is the series' index).
Context #
Durability is the largest capability gap the spec names about itself.
08-out-of-scope holds four interlocking
deferrals: §Persistence defers workflow checkpointing/resumption (with a
checkpoint:-block sketch), §Idempotency defers step dedup "together
with the durable-execution waypoint", H1 states plainly that crash =
re-run from the top, and H5 admits the human gate's pause-state is
live ("the run keeps a process while blocked · durable pauses arrive
with H1"). Across the 2026 workflow-runner landscape, crash-resume and
the durable human gate are the two most-asked-for capabilities this
class of tool gets measured on — Nextflow's -resume (content-addressed
task skip) is that ecosystem's single most-loved feature, and it taxes
authors with nothing.
Meanwhile the checkpoint artifact already exists. A conformant Runtime
engine MUST emit the workflow event stream
(07 §Runtime-4):
task.completed records carry per-task outcome, and the reference
engine journals every run as an NDJSON trace (crates/nika-event/src/kind.rs
serializes the kinds snake_case: task_completed · and pre-reserves
checkpoint_written, "a durable run state snapshot"). What is missing
is not a new runtime — it is a reader.
The trap to avoid is equally documented: Temporal-class durable execution buys resume by imposing replay-determinism constraints on workflow authors (code must be deterministic · side effects must be wrapped · histories version with the code). That tax is the single most hated property of the durable-execution class, and this spec's whole posture (audit-before-run · no hidden magic · the engine owns the how) forbids exporting it to authors.
Decision #
`nika run wf.nika.yaml --resume <trace>` re-executes a workflow, skipping every task whose identity matches a completed record in the given trace. The trace is the run's own NDJSON journal — no second store, no new artifact, no daemon.
1 · The skip rule — content-addressed · two hashes · both MUST match #
A task is skippable iff the trace holds a task.completed record whose
- task-definition hash — a canonical serialization of the task's
behavior-bearing fields (the verb body ·
with:·output:·retry:/on_error:/on_finally:·when:·for_each:) — matches the task as now written, AND - resolved-input hash — the values its
${{ }}references resolved to at execution time (upstream outputs ·vars·with·env) — matches the values the resumed run resolves.
Edit a prompt → that task re-runs. Change a var an early task consumes →
it re-runs, and the mismatch cascades through resolved-input hashes
exactly as far as the data actually flows — untouched siblings still
skip. for_each tasks record per-iteration (the item participates in
the input hash), so a partially-completed fan-out resumes at the
iteration level.
Secrets participate by declared reference identity (name · declared
source path), never by value — masking is normative
(01 §envelope MUSTs) and a trace MUST NOT
carry secret-derived material, including value hashes (a hash of a
low-entropy secret is an oracle). Stated limit: a rotated secret does
NOT invalidate the cache — force the affected node with --from.
2 · The skip is VISIBLE — `task.cache_hit` · never silent #
Each skipped task emits a `task.cache_hit` event (additive to the
07 §Runtime-4
event set · reference-engine trace kind task_cache_hit) carrying
task_id + a reference to the matched trace record. The task's output
is rehydrated from the record; downstream tasks observe status: success and the bound output exactly as if it had run live. The task
state enum stays CLOSED (03 §task states
· no new state): the cache/live distinction rides the event stream and
the run report, never the ${{ }} surface. Anti-hidden-magic is the
design line — a resumed run's console and trace say cache_hit per
skipped task, loudly.
3 · `--from <task_id>` — the manual override #
Forces re-execution from a node: the named task and its transitive
downstream re-run even on hash match (tasks upstream of it may still
cache-hit). An unknown task id is refused pre-run (the same contract as
nika run --var unknown-key refusal). This is the lever for "the world
changed in a way the hashes cannot see" (rotated secret · external
state · an infer: output you want re-rolled).
4 · Non-deterministic tasks replay their recorded output #
An infer:/agent: task that hash-matches replays the recorded output
— that is the point (crash-resume without re-spending tokens).
Determinism is never the author's problem: there are no replay rules,
no "workflow versioning", no side-effect wrappers. A task that does not
hash-match simply re-runs live, side effects included (see Non-goals).
Rider — the durable human gate (`nika:prompt` pause) #
Today's normative non-interactive contract
(stdlib §nika:prompt): with default:
the engine MUST answer with it; without, it MUST fail
NIKA-BUILTIN-PROMPT-001 — never hang. The rider occupies exactly
that failure branch, changing nothing else:
- Under a non-interactive surface (
--json·nika serve), a blockingnika:promptwith no usabledefault:journals a `workflow.paused` event (trace kindworkflow_paused) carrying the prompt payload (task_id·mode·message·choices· secret-masked like every journal surface) and exits cleanly with run statepausedinstead of failing. - `--resume <trace>` re-arms the prompt: completed upstream work cache-hits, the run continues AT the prompt — interactively it asks; resumed non-interactively with an answer supplied (CLI arg · serve webhook payload — delivery ergonomics are the engine's) it binds it; resumed non-interactively with no answer it pauses again (idempotent · a paused trace can be resumed any number of times).
- This resolves H5's noted limitation ("pause-state is live · durable
pauses arrive with H1") with zero new syntax — "ONE construct · a
tool under
invoke:· forever" stays exactly as locked.
The one spec-surface delta this rider needs: the run-report/event
vocabulary gains `paused` as an additive workflow-state value
(05 §workflow-level semantics)
plus the two events above. Like pending/running, paused exists in
run reports and events, never inside ${{ }}.
Non-goals #
- No author-facing determinism constraints (Temporal-style replay
rules). Side-effectful tasks re-run whenever their hashes mismatch —
durability is the engine's problem, never the author's. An author who
needs a side effect to be exactly-once under resume marks the node
with
--fromdiscipline or makes the tool idempotent via its own args (today's H1 guidance · unchanged). - Idempotency keys stay deferred — the NEXT deferral in this ladder (08 §idempotency). Resume shrinks the re-run window (completed work never re-fires) but a crash mid-side-effect can still double-fire on re-run; at-least-once dedup remains the full durable-execution waypoint's job.
- No envelope/syntax change. The
checkpoint:-block sketch in 08 §Persistence REMAINS deferred exactly as written: a workflow never declares persistence. This ADR is CLI flags + trace/event vocabulary. - No daemon.
--resumeis a reader of a file. No supervisor · no run-store service · no background process · no wake-on-event (durable timers ride a later design). - Not general memoization.
task.cache_hitfires only under--resume; a freshnika runexecutes every task. The H17cache_key:field stays deferred on its own terms.
Consequences #
Positive #
- Crash-resume · re-run-from-a-node · no-token-re-spend land behind ONE flag, riding an artifact every Runtime-conformant engine already emits.
- The durable human gate closes the biggest practitioner ask with zero new syntax and zero change to answered-prompt behavior (the pause path occupies only what is a hard error today — strictly additive).
- Sovereignty intact: the trace is a local NDJSON file · grep-able · portable · no vendor run store, no cloud dependency.
- The determinism tax is structurally refused, not just avoided: there is no replay engine to feed, so no replay rules can ever leak to authors.
Negative #
- The
task.completedrecord fields resume reads (definition hash · input hash · output) become a compatibility surface: they MUST evolve additively, and an engine facing a trace it cannot read MUST say so and re-run fully — never guess a partial skip. - Secrets-by-name hashing means a rotated secret silently cache-hits;
the limit is documented and
--fromis the override, but it is a real sharp edge to teach. pausedis one more workflow-state value every run-report consumer must render.
Neutral #
- Resume conformance is behavioral-tier — it lands with the runtime fixtures, post-announce (07 §suite status).
- Another engine MAY keep a different trace file layout: the conformance contract binds the EVENTS and the skip/pause semantics, not the bytes of the journal.
Conformance sketch — 3 fixtures · `tests/runtime/resume/` #
Behavioral-tier shape per
conformance/tests/runtime/README.md
(input.nika.yaml + run.json + expected-run.json · mock/echo) ·
resume fixtures add the two-phase invocation (run → interrupt → resume)
to the runner protocol:
- (a) `001-kill-midrun-resume-completes-remainder` — first
invocation is interrupted after task
acompletes; the resumed run MUST cache-hita(events_include: ["task.cache_hit:a"]), execute only the remainder live, and endworkflow_state: successwith outputs identical to an uninterrupted run. - (b) `002-input-change-rehashes-and-reruns` — same trace, one
varsvalue changed inrun.json: the consuming task's resolved-input hash mismatches → it re-runs (notask.cache_hitfor it) · an untouched sibling branch still cache-hits. - (c) `003-paused-prompt-rearms` — a non-interactive run hits a
default-less
nika:prompt→ exitsworkflow_state: paused, trace carriesworkflow.pausedwith the prompt payload; resumed with an answer supplied → upstream cache-hits, the prompt binds, the run completes; resumed without one → it pauses again (idempotent).
Evidence / Affected surfaces #
- `spec/08-out-of-scope.md` — §Persistence checkpointing (the deferral this ADR lifts at the durable-lite tier) · §Idempotency (the next deferral) · H1 · H5 · H17 (amended this arc).
- `stdlib/builtins-v0.1.md` §
nika:prompt— theNIKA-BUILTIN-PROMPT-001branch the rider occupies. - `spec/07-conformance.md` §Runtime-4 —
the event vocabulary
task.cache_hit/workflow.pausedextend additively. - Reference engine
crates/nika-event/src/kind.rs— snake_case trace kinds (task_completed) + the pre-reservedcheckpoint_written("durable run state snapshot"): the seam was planted before the design. - External poles · Nextflow
-resume(the adopted shape · content-addressed task skip · zero author tax) · Temporal workflow determinism constraints (the rejected shape · the tax this ADR structurally refuses).
Alternatives considered #
Alt A — full durable-execution runtime (Temporal-class) #
Event-sourced replay: re-execute workflow code against recorded histories, requiring deterministic authorship, wrapped side effects, and code-version pinning. Rejected — it exports the runtime's problem to every author forever, contradicts the no-hidden-magic spine, and buys nothing this design does not already deliver for a finite DAG whose journal carries every task boundary.
Alt B — sqlite checkpoint store #
A dedicated checkpoints.db beside the trace (the shape the old 08
sketch reserved). Rejected for v1 — a second stateful artifact to
create, locate, version, and explain, duplicating data the NDJSON trace
already journals. One artifact · plain text · portable. An engine MAY
later keep an internal index (sqlite or otherwise) as a pure
optimization — invisible to this contract.
Alt C — do nothing #
Rejected — crash-resume and the durable human gate are the top two practitioner asks of this tool class, and H5's live-pause limitation is flagged in the spec's own text. Waiting for the full durable-execution waypoint gates the 90% capability on the 10% problem (idempotent delivery) that finite single-run DAGs mostly do not have.
Related #
- `spec/08-out-of-scope.md` §Persistence · §Idempotency · H1 · H5 · H17
- `spec/07-conformance.md` §Runtime conformance · §suite status
- `spec/03-dag.md` §task states (closed enum ·
unchanged) · `spec/05-errors.md`
§workflow-level semantics (
pausedadditive) - `stdlib/builtins-v0.1.md` §
nika:prompt
Notes #
- Registration · DONE (this file · the
feat/resume-implarc) — ADR-099 is registered in the enginedocs/adr/index (the series registry). - Status flip · DONE ·
proposed → acceptedwith the engine implementation; the behavioral fixtures of the Conformance sketch ship ascrates/nika-cli/tests/resume_e2e.rs. (The spec-repo copy's frontmatter flip is a spec-repo commit — tracked there.) - Revisit trigger · if implementation shows secrets-by-name hashing is too blunt (rotated-secret staleness bites in practice), a keyed commitment (HMAC under an engine-local key · never a bare hash) re-opens HERE first — the trace-carries-no-secret-material invariant is not negotiable.
read at v0.107.0 · the decision record ships with the engine