[{"k":"h","depth":1,"text":"07 · Conformance","id":"07--conformance"},{"k":"quote","inline":[{"k":"text","text":"An engine MAY claim « v0.1-compliant » if it passes the conformance\ntest suite at "},{"k":"link","text":"`../conformance/tests/`","href":"../conformance/"},{"k":"text","text":". This document\ndefines the conformance levels and what each requires."}]},{"k":"hr"},{"k":"h","depth":2,"text":"Notation","id":"notation"},{"k":"p","inline":[{"k":"text","text":"The key words "},{"k":"strong","text":"MUST"},{"k":"text","text":", "},{"k":"strong","text":"MUST NOT"},{"k":"text","text":", "},{"k":"strong","text":"REQUIRED"},{"k":"text","text":", "},{"k":"strong","text":"SHALL"},{"k":"text","text":", "},{"k":"strong","text":"SHALL\nNOT"},{"k":"text","text":", "},{"k":"strong","text":"SHOULD"},{"k":"text","text":", "},{"k":"strong","text":"SHOULD NOT"},{"k":"text","text":", "},{"k":"strong","text":"MAY"},{"k":"text","text":", and "},{"k":"strong","text":"OPTIONAL"},{"k":"text","text":" in this\nspecification are to be interpreted as described in "},{"k":"link","text":"BCP 14","href":"https://www.rfc-editor.org/info/bcp14"},{"k":"text","text":"\n("},{"k":"link","text":"RFC 2119","href":"https://www.rfc-editor.org/rfc/rfc2119"},{"k":"text","text":" + "},{"k":"link","text":"RFC 8174","href":"https://www.rfc-editor.org/rfc/rfc8174"},{"k":"text","text":")\nwhen, and only when, they appear in all capitals."}]},{"k":"hr"},{"k":"h","depth":2,"text":"Conformance levels","id":"conformance-levels"},{"k":"p","inline":[{"k":"text","text":"Three nested levels · increasing scope ·"}]},{"k":"list","ordered":false,"items":[[{"k":"text","text":"Level · What it covers · Use case"}],[{"k":"text","text":"**Core** · Parse + validate · DAG semantics · variable resolution · error structure · Linters · spec editors · static analyzers"}],[{"k":"text","text":"**Runtime** · Core + verb execution · Working engine (with own provider/tool impls)"}],[{"k":"text","text":"**Stdlib v0.1** · Runtime + the 17 providers + <!-- canon:extract_modes -->9<!-- /canon --> extract modes + 28 builtins · Full reference-impl-equivalent engine"}]]},{"k":"p","inline":[{"k":"text","text":"A higher level "},{"k":"strong","text":"includes"},{"k":"text","text":" the lower levels."}]},{"k":"hr"},{"k":"h","depth":2,"text":"Level 1 · Core conformance","id":"level-1--core-conformance"},{"k":"p","inline":[{"k":"text","text":"An engine claims « Core v0.1-compliant » if it ·"}]},{"k":"list","ordered":true,"items":[[{"k":"strong","text":"Parses"},{"k":"text","text":" any valid v0.1 workflow YAML correctly"},{"k":"text","text":"- Accepts exactly `nika: v1` · a `workflow:` object carrying a kebab-case `id` · rejects any other `nika:` value (a scalar `workflow:` is the dead W1 form · `NIKA-PARSE-020`)\n- Validates the `workflow` identifier kebab-case\n- Validates typed `inputs` (type + required) · validates `config` / `const` / `secrets` shape\n- Recognizes the 4 verbs (`infer` · `exec` · `invoke` · `agent`)\n- Rejects unknown top-level fields with a clear error OR ignores with warning (engine's choice · documented behavior)\n"}],[{"k":"strong","text":"Computes DAG topology"},{"k":"text","text":" correctly"},{"k":"text","text":"- Derives E_d from `with:` bindings (role per referenced field) and E_c from `after:` entries · G_p = E_d ∪ E_c ([03 §four graphs](./03-dag.md#the-four-graphs-normative))\n- Detects cycles in G_p · rejects with `NIKA-DAG-001`\n- Detects a `with:`/`after:` reference to an undeclared task · rejects with `NIKA-DAG-002`\n- Rejects an `after:` predicate outside the closed set · `NIKA-DAG-005`\n- Rejects `depends_on:` (dead form) · `NIKA-PARSE-024` · and a `tasks.*` reference outside the boundary · `NIKA-VAR-021`\n- Detects an `on_error.recover` reference to a task downstream of the declaring task · rejects with `NIKA-DAG-004` (the await would deadlock · [05](./05-errors.md#recover-reference-resolution-normative))\n- Computes topological waves for parallel execution\n"}],[{"k":"strong","text":"Resolves variable references"},{"k":"text","text":" correctly (static · reference-resolution · NOT runtime evaluation)"},{"k":"text","text":"- `\u0024{{ inputs.x }}` · `\u0024{{ const.x }}` resolve to declared envelope `inputs:` / `const:` entries\n- `\u0024{{ with.x }}` resolves to a declared task `with:` key\n- `\u0024{{ tasks.X.field }}` resolves to a declared upstream task + a valid field name\n- `\u0024{{ config.X }}` · `\u0024{{ secrets.X }}` resolve to declared namespaces\n- `when:` and `for_each:` expressions are valid **CEL** (the v0.1 subset · see 03-dag) and their references **resolve to known namespaces**: Core parses but does NOT *evaluate* them (no execution = no `tasks.X.status` to compare against · that is Runtime's job)\n- `output:` bindings are valid **jq** expressions (the one data language · see 04-variables) · `\u0024{{ }}` never appears inside a binding\n- Reports undefined references with `NIKA-VAR-001` · static expression violations with `NIKA-VAR-005` (the deep-static layer · CEL subset parse · jq compile · `when:` boolean shape)\n"}],[{"k":"strong","text":"Produces typed errors"},{"k":"text","text":" matching the v0.1 spec"},{"k":"text","text":"- `code` follows `NIKA-<NAMESPACE>-<NNN>` format\n- `category` is one of the closed enum values\n- `transient` correctly set\n"}],[{"k":"strong","text":"Passes"},{"k":"text","text":" all tests in "},{"k":"code","text":"conformance/tests/core/"}]]},{"k":"p","inline":[{"k":"text","text":"A Core-compliant engine does NOT execute verbs and does NOT evaluate "},{"k":"code","text":"when:"},{"k":"text","text":" / "},{"k":"code","text":"for_each:"},{"k":"text","text":" over runtime state. It parses · validates · builds the DAG · resolves variable "},{"k":"em","text":"references"},{"k":"text","text":" (syntax + namespace validity) · produces typed errors. Runtime evaluation of conditions and iteration is Level 2."}]},{"k":"h","depth":3,"text":"`nika check` · the static pre-flight (the audit-before-it-runs surface)","id":"nika-check--the-static-pre-flight-the-audit-before-it-runs-surface"},{"k":"p","inline":[{"k":"text","text":"Because the language is "},{"k":"strong","text":"statically analyzable by construction"},{"k":"text","text":" (the DAG\nis acyclic, "},{"k":"code","text":"for_each"},{"k":"text","text":" is bounded, CEL is non-Turing, and effects are\ndeclared), a conformant engine can answer « what will this workflow do, cost,\nand touch? » with "},{"k":"strong","text":"zero API calls and zero tokens spent"},{"k":"text","text":". "},{"k":"code","text":"nika check"},{"k":"text","text":" is\nthe canonical CLI surface for that (Core conformance + the four static\nguarantees below) ·"}]},{"k":"list","ordered":false,"items":[[{"k":"text","text":"Guarantee · What it reports · zero execution · Backed by"}],[{"k":"text","text":"**Plan** · the wave topology · which tasks run in parallel · the critical path · DAG waves (Core §2)"}],[{"k":"text","text":"**Cost ceiling** · the worst-case spend · `Σ (max_tokens × provider price)` across `infer:`/`agent:` tasks · before one token is spent · the `nika:inspect view: cost` model, run statically"}],[{"k":"text","text":"**Secret leak** · every `secrets.X` that flows into an `exec` capture or a tool whose output is bound (the masking boundary · [04 §secrets](./04-variables.md)) · reference graph"}],[{"k":"text","text":"**Capability escape** · any effect outside a declared `permits:` block: a write outside `fs.write`, a fetch to an unlisted host, an `exec` under `exec: false`, an unlisted tool · `permits:` ([01](./01-envelope.md))"}],[{"k":"text","text":"**Provider parity** · (`--providers`) that the workflow uses zero provider-specific fields → the same `schema:` runs identically on all 17 providers (incl. the 5 local) · the closed verb-field set"}]]},{"k":"p","inline":[{"k":"text","text":"This is the property no other AI workflow runner gives: "},{"k":"strong","text":"GitHub Actions,\nTemporal, and LangGraph tell you nothing (and charge you nothing back)\nuntil you run."},{"k":"text","text":" A Nika workflow is auditable for cost, capabilities,\nsecrets, and portability "},{"k":"em","text":"as a static fact about the file"},{"k":"text","text":". "},{"k":"code","text":"nika check"},{"k":"text","text":" is\nan engine CLI surface (not a separate conformance level: it composes Core\nvalidation with the cost/secret/permits/parity reports); the guarantees it\nsurfaces ARE normative (they derive from Core conformance + the "},{"k":"code","text":"permits:"},{"k":"text","text":"\nand "},{"k":"code","text":"secrets:"},{"k":"text","text":" MUSTs), the CLI ergonomics around them are the engine's."}]},{"k":"h","depth":3,"text":"Editor tooling · the canonical JSON Schema","id":"editor-tooling--the-canonical-json-schema"},{"k":"p","inline":[{"k":"text","text":"The spec ships a canonical "},{"k":"strong","text":"JSON Schema"},{"k":"text","text":" at\n"},{"k":"link","text":"`schemas/workflow.schema.json`","href":"../schemas/"},{"k":"text","text":" describing the envelope +\ntask shape + verb argument structures. It is the machine-readable companion to\nthis prose spec (kept in sync · the prose is normative on conflict)."}]},{"k":"p","inline":[{"k":"text","text":"Editors (VS Code · Zed · JetBrains · Neovim) pick it up via the standard\n"},{"k":"code","text":"yaml.schemas"},{"k":"text","text":" association (or a "},{"k":"code","text":"# yaml-language-server: \u0024schema=…"},{"k":"text","text":" modeline)\nto give "},{"k":"strong","text":"autocomplete + inline validation"},{"k":"text","text":" as you type, the same DX as\nGitHub Actions and Docker Compose. A working modeline today:"}]},{"k":"code","lang":"yaml","text":"# yaml-language-server: \u0024schema=https://raw.githubusercontent.com/supernovae-st/nika-spec/main/schemas/workflow.schema.json\nnika: v1"},{"k":"p","inline":[{"k":"text","text":"(The short "},{"k":"code","text":"https://nika.sh/spec/v1/workflow.schema.json"},{"k":"text","text":" form goes live\nwith the site launch · both will resolve to the same schema.) This is also what makes a Nika file\npleasant (and trap-free) for an AI to author: the schema constrains the shape\nbefore the engine ever runs. CEL expressions and jq expressions inside string\nfields are validated by the engine (Core level), not the JSON Schema."}]},{"k":"p","inline":[{"k":"strong","text":"Use case"},{"k":"text","text":" · linters · spec editors · LSP server intelligence · static analyzers."}]},{"k":"hr"},{"k":"h","depth":2,"text":"Level 2 · Runtime conformance","id":"level-2--runtime-conformance"},{"k":"p","inline":[{"k":"text","text":"An engine claims « Runtime v0.1-compliant » if it satisfies Core conformance PLUS ·"}]},{"k":"list","ordered":true,"items":[[{"k":"strong","text":"Executes the 4 verbs"},{"k":"text","text":" with correct semantics"},{"k":"text","text":"- `infer:` calls a configured provider · returns response\n- `exec:` runs the command in a shell · honors timeout + security\n- `invoke:` resolves and calls the tool · returns response\n- `agent:` runs the multi-turn loop · honors max_turns + tools whitelist\n"}],[{"k":"strong","text":"Honors task fields"},{"k":"text","text":" correctly"},{"k":"text","text":"- GATE-v2 admission · every incoming edge's producer settles inside that edge's pass-set, else `cancelled` (dead-path · [03 §gate algebra](./03-dag.md#the-gate-algebra-v2-normative))\n- `when` skips when false (POST-gate · local namespaces)\n- `timeout` (Go-duration string · `\"30s\"` · `\"5m\"` etc.) hard-kills on timeout\n- `retry` strategy honored on transient errors\n- `on_error` recovery honored on terminal errors\n- `with` scope injected into task body\n- `output` binding via jq\n"}],[{"k":"strong","text":"Implements security policies"},{"k":"text","text":"- `exec:` blocklist (engine SHOULD ship a sane default · MAY allow override)\n- `fetch:` SSRF defense for private IP ranges\n- `invoke:` capability checks (engine's choice of capability model)\n"}],[{"k":"strong","text":"Emits workflow events"},{"k":"text","text":"- `task.started` · `task.completed` · `task.failed` · `task.skipped`\n- `workflow.started` · `workflow.completed` · `workflow.failed`\n- Event payload includes `task_id` · timestamp · status · duration\n"}],[{"k":"strong","text":"Passes"},{"k":"text","text":" all tests in "},{"k":"code","text":"conformance/tests/runtime/"}]]},{"k":"p","inline":[{"k":"text","text":"Runtime-compliant engines may bring "},{"k":"strong","text":"their own"},{"k":"text","text":" provider implementations · tool implementations · MCP server registries. They are not required to ship the canonical stdlib."}]},{"k":"p","inline":[{"k":"strong","text":"Use case"},{"k":"text","text":" · custom engines for specialized environments (embedded · WASM · custom LLM gateway · etc.)."}]},{"k":"hr"},{"k":"h","depth":2,"text":"Level 3 · Stdlib v0.1 conformance","id":"level-3--stdlib-v01-conformance"},{"k":"p","inline":[{"k":"text","text":"An engine claims « Stdlib v0.1-compliant » if it satisfies Runtime conformance PLUS ·"}]},{"k":"list","ordered":true,"items":[[{"k":"strong","text":"Ships all 17 canonical providers"},{"k":"text","text":" (per "},{"k":"link","text":"stdlib/providers-v0.1.md","href":"../stdlib/providers-v0.1.md"},{"k":"text","text":")"}],[{"k":"strong","text":"Ships all 9 canonical extract modes"},{"k":"text","text":" (per "},{"k":"link","text":"stdlib/extract-modes-v0.1.md","href":"../stdlib/extract-modes-v0.1.md"},{"k":"text","text":")"}],[{"k":"strong","text":"Ships at least all 28 canonical builtins"},{"k":"text","text":" (core 6 + file 5 + data 9 + network 2 + introspection 2 + media 4 · the remaining deferred media builtins are optional · the byte-determinism clauses of "},{"k":"code","text":"nika:image_fx"},{"k":"text","text":" and "},{"k":"code","text":"nika:chart"},{"k":"text","text":" are part of the bar — an engine that cannot honor them is not conformant, no waiver)"}],[{"k":"strong","text":"Passes"},{"k":"text","text":" all tests in "},{"k":"code","text":"conformance/tests/stdlib/"}]]},{"k":"p","inline":[{"k":"text","text":"A Stdlib-compliant engine is functionally equivalent to the reference implementation for any workflow that uses only the canonical stdlib elements."}]},{"k":"p","inline":[{"k":"strong","text":"Use case"},{"k":"text","text":" · the default level for production engines."}]},{"k":"hr"},{"k":"h","depth":2,"text":"Suite status · v0.1 (honest)","id":"suite-status--v01-honest"},{"k":"p","inline":[{"k":"text","text":"What is populated TODAY vs what lands with the reference engine ·"}]},{"k":"list","ordered":false,"items":[[{"k":"text","text":"Layer · Status · What it proves"}],[{"k":"text","text":"**Core fixtures** (`tests/core/`) · ✅ populated · runner-executable · parse + validate + DAG + variables + errors · the full Level-1 static contract"}],[{"k":"text","text":"**Deep-static fixtures** (`tests/deep/`) · ✅ populated · runner-executable · the expression layer the schema cannot see · the normative CEL EBNF parsed for real · jq compile · duration grammar · schema-meta · `when:` shape · binding purity"}],[{"k":"text","text":"**Stdlib static surface** (`tests/stdlib/`) · ✅ populated · runner-executable · the stdlib **names + shapes** layer · provider prefixes · the closed `nika:*` builtin set · extract modes · checkable with zero execution (lists derive from [`canon.yaml`](../canon.yaml))"}],[{"k":"text","text":"**Examples as conformance inputs** (`examples/`) · ✅ executed by the runner `all` gate · every shipped example MUST validate at the full static level"}],[{"k":"text","text":"**Runtime behavioral fixtures** (`tests/runtime/`) · ⏳ **post-announce** · verb execution · task fields · events · they require an executing engine · they land with the reference engine's vertical slice (1.0.0)"}],[{"k":"text","text":"**Stdlib behavioral fixtures** (`tests/stdlib/` · execution half) · ⏳ **post-announce** · provider/builtin/extract-mode *behavior* under the `mock` provider + HTTP mocks"}]]},{"k":"p","inline":[{"k":"text","text":"Run the static gate yourself · "},{"k":"code","text":"python conformance/runner.py all"},{"k":"text","text":": the\nrunner output is the live count (counts in prose drift · the suite is the\nsource). A « Core v0.1-compliant » claim is FULLY testable today. « Runtime »\nand « Stdlib v0.1 » claims are testable on their static halves today · their\nbehavioral halves when the behavioral fixtures publish."}]},{"k":"hr"},{"k":"h","depth":2,"text":"Conformance test structure","id":"conformance-test-structure"},{"k":"code","lang":"text","text":"conformance/\n├── tests/\n│   ├── core/                  # Level-1 fixtures · parse · validate · DAG · variables · errors\n│   ├── deep/                  # deep-static layer · CEL subset parse · jq compile ·\n│   │                          # durations · schema-meta · when shape · binding purity\n│   ├── lints/                 # advisory-lint fixtures\n│   ├── runtime/               # verb execution · task fields · events (behavioral half)\n│   └── stdlib/                # provider/extract/builtin canonical surface\n│\n├── runner.py                  # the static oracle · `all` is the CI gate\n├── *_core.py                  # per-domain reference evaluators (type · decision ·\n│                              # gateway · outcome · composition · yaml-profile ·\n│                              # proof · projection) + their `*_selftest.py` sweeps\n├── yaml-profile/              # R11 profile fixtures (valid/ + invalid/)\n├── type-corpus/               # the generated type corpus (gen-type-corpus.py)\n├── values/                    # the four-authority family (C2 · R3a · valid/ + invalid/)\n├── types/                     # io-declaration predicate vocabulary (C2 · R3b · valid/ + invalid/)\n├── gates/                     # the after: predicate vocabulary (C2 · R5 · valid/ + invalid/)\n└── runner-protocol.md         # how to run the suite against any engine"},{"k":"p","inline":[{"k":"strong","text":"The directory is the source"},{"k":"text","text":" — this sketch is a map, not the inventory\n("},{"k":"code","text":"python3 conformance/runner.py all"},{"k":"text","text":" prints the live fixture counts).\nEach fixture is a pair · "},{"k":"code","text":"input.yaml"},{"k":"text","text":" (the workflow to feed) + "},{"k":"code","text":"expected.json"},{"k":"text","text":" (the expected output or error structure)."}]},{"k":"p","inline":[{"k":"text","text":"For tests that require executing against real LLMs / networks · the suite uses the "},{"k":"code","text":"mock"},{"k":"text","text":" provider and HTTP mocks to keep tests deterministic."}]},{"k":"hr"},{"k":"h","depth":2,"text":"Runner protocol","id":"runner-protocol"},{"k":"p","inline":[{"k":"text","text":"A conformance runner ·"}]},{"k":"list","ordered":true,"items":[[{"k":"text","text":"Reads each "},{"k":"code","text":"input.yaml"}],[{"k":"text","text":"Pipes it to the engine being tested ("},{"k":"code","text":"engine run --input -"},{"k":"text","text":")"}],[{"k":"text","text":"Captures the engine's output (stdout JSON · structured)"}],[{"k":"text","text":"Compares against "},{"k":"code","text":"expected.json"}],[{"k":"text","text":"Reports pass/fail per test · final summary"}]]},{"k":"p","inline":[{"k":"text","text":"See "},{"k":"code","text":"conformance/runner-protocol.md"},{"k":"text","text":" for the exact JSON wire format."}]},{"k":"hr"},{"k":"h","depth":2,"text":"Claiming conformance","id":"claiming-conformance"},{"k":"p","inline":[{"k":"text","text":"The public claim string is "},{"k":"strong","text":"« Nika v1 Conformant — <Level> (spec <commit>) »"},{"k":"text","text":"\n· Level ∈ "},{"k":"code","text":"Core"},{"k":"text","text":" · "},{"k":"code","text":"Runtime"},{"k":"text","text":" · "},{"k":"code","text":"Stdlib v0.1"},{"k":"text","text":" · the spec commit is the pin\nthe suite ran against. One form everywhere — badges · READMEs · listings ·\nrelease notes. (In normative sentences, « v0.1-compliant » names the level\n"},{"k":"em","text":"bar"},{"k":"text","text":" an engine satisfies; the string above is the one public "},{"k":"em","text":"claim"},{"k":"text","text":".)"}]},{"k":"p","inline":[{"k":"text","text":"To make the claim · an engine ·"}]},{"k":"list","ordered":true,"items":[[{"k":"text","text":"MUST pass the conformance suite at the claimed level"}],[{"k":"text","text":"MUST name the level and the spec commit in the claim (the format above)"}],[{"k":"text","text":"MAY open a PR on "},{"k":"link","text":"supernovae-st/nika-spec","href":"https://github.com/supernovae-st/nika-spec"},{"k":"text","text":" to be listed in "},{"k":"code","text":"CONFORMANT_IMPLEMENTATIONS.md"}]]},{"k":"p","inline":[{"k":"text","text":"The claim is earned by passing the suite, never by declaration\n("},{"k":"link","text":"NEP-0000 §Relationship to conformance","href":"../governance/nep-0000-the-nep-process.md"},{"k":"text","text":").\nThe reference implementation "},{"k":"link","text":"supernovae-st/nika","href":"https://github.com/supernovae-st/nika"},{"k":"text","text":" targets Stdlib v0.1 conformance."}]},{"k":"hr"},{"k":"h","depth":2,"text":"Versioning","id":"versioning"},{"k":"p","inline":[{"k":"text","text":"A conformance claim is "},{"k":"strong","text":"specific to a spec version"},{"k":"text","text":". As the stdlib evolves to v0.2 · engines re-claim conformance against the new suite. The Core conformance level is stable forever within v1 of the language."}]},{"k":"hr"},{"k":"p","inline":[{"k":"text","text":"🦋 "},{"k":"em","text":"Next · [08 · Out of scope](./08-out-of-scope.md)"}]}]

the spec · 07

Conformance

An engine MAY claim « v0.1-compliant » if it passes the conformance test suite at [`../conformance/tests/`](../conformance/).

07 · Conformance #

An engine MAY claim « v0.1-compliant » if it passes the conformance test suite at `../conformance/tests/`. This document defines the conformance levels and what each requires.

Notation #

The key words MUST, MUST NOT, REQUIRED, SHALL, SHALL NOT, SHOULD, SHOULD NOT, MAY, and OPTIONAL in this specification are to be interpreted as described in BCP 14 (RFC 2119 + RFC 8174) when, and only when, they appear in all capitals.

Conformance levels #

Three nested levels · increasing scope ·

Level · What it covers · Use case
**Core** · Parse + validate · DAG semantics · variable resolution · error structure · Linters · spec editors · static analyzers
**Runtime** · Core + verb execution · Working engine (with own provider/tool impls)
**Stdlib v0.1** · Runtime + the 17 providers + 9 extract modes + 28 builtins · Full reference-impl-equivalent engine

A higher level includes the lower levels.

Level 1 · Core conformance #

An engine claims « Core v0.1-compliant » if it ·

Parses any valid v0.1 workflow YAML correctly- Accepts exactly `nika: v1` · a `workflow:` object carrying a kebab-case `id` · rejects any other `nika:` value (a scalar `workflow:` is the dead W1 form · `NIKA-PARSE-020`) - Validates the `workflow` identifier kebab-case - Validates typed `inputs` (type + required) · validates `config` / `const` / `secrets` shape - Recognizes the 4 verbs (`infer` · `exec` · `invoke` · `agent`) - Rejects unknown top-level fields with a clear error OR ignores with warning (engine's choice · documented behavior)
Computes DAG topology correctly- Derives E_d from `with:` bindings (role per referenced field) and E_c from `after:` entries · G_p = E_d ∪ E_c ([03 §four graphs](./03-dag.md#the-four-graphs-normative)) - Detects cycles in G_p · rejects with `NIKA-DAG-001` - Detects a `with:`/`after:` reference to an undeclared task · rejects with `NIKA-DAG-002` - Rejects an `after:` predicate outside the closed set · `NIKA-DAG-005` - Rejects `depends_on:` (dead form) · `NIKA-PARSE-024` · and a `tasks.*` reference outside the boundary · `NIKA-VAR-021` - Detects an `on_error.recover` reference to a task downstream of the declaring task · rejects with `NIKA-DAG-004` (the await would deadlock · [05](./05-errors.md#recover-reference-resolution-normative)) - Computes topological waves for parallel execution
Resolves variable references correctly (static · reference-resolution · NOT runtime evaluation)- `${{ inputs.x }}` · `${{ const.x }}` resolve to declared envelope `inputs:` / `const:` entries - `${{ with.x }}` resolves to a declared task `with:` key - `${{ tasks.X.field }}` resolves to a declared upstream task + a valid field name - `${{ config.X }}` · `${{ secrets.X }}` resolve to declared namespaces - `when:` and `for_each:` expressions are valid **CEL** (the v0.1 subset · see 03-dag) and their references **resolve to known namespaces**: Core parses but does NOT *evaluate* them (no execution = no `tasks.X.status` to compare against · that is Runtime's job) - `output:` bindings are valid **jq** expressions (the one data language · see 04-variables) · `${{ }}` never appears inside a binding - Reports undefined references with `NIKA-VAR-001` · static expression violations with `NIKA-VAR-005` (the deep-static layer · CEL subset parse · jq compile · `when:` boolean shape)
Produces typed errors matching the v0.1 spec- `code` follows `NIKA-<NAMESPACE>-<NNN>` format - `category` is one of the closed enum values - `transient` correctly set
Passes all tests in conformance/tests/core/

A Core-compliant engine does NOT execute verbs and does NOT evaluate when: / for_each: over runtime state. It parses · validates · builds the DAG · resolves variable references (syntax + namespace validity) · produces typed errors. Runtime evaluation of conditions and iteration is Level 2.

`nika check` · the static pre-flight (the audit-before-it-runs surface) #

Because the language is statically analyzable by construction (the DAG is acyclic, for_each is bounded, CEL is non-Turing, and effects are declared), a conformant engine can answer « what will this workflow do, cost, and touch? » with zero API calls and zero tokens spent. nika check is the canonical CLI surface for that (Core conformance + the four static guarantees below) ·

Guarantee · What it reports · zero execution · Backed by
**Plan** · the wave topology · which tasks run in parallel · the critical path · DAG waves (Core §2)
**Cost ceiling** · the worst-case spend · `Σ (max_tokens × provider price)` across `infer:`/`agent:` tasks · before one token is spent · the `nika:inspect view: cost` model, run statically
**Secret leak** · every `secrets.X` that flows into an `exec` capture or a tool whose output is bound (the masking boundary · [04 §secrets](./04-variables.md)) · reference graph
**Capability escape** · any effect outside a declared `permits:` block: a write outside `fs.write`, a fetch to an unlisted host, an `exec` under `exec: false`, an unlisted tool · `permits:` ([01](./01-envelope.md))
**Provider parity** · (`--providers`) that the workflow uses zero provider-specific fields → the same `schema:` runs identically on all 17 providers (incl. the 5 local) · the closed verb-field set

This is the property no other AI workflow runner gives: GitHub Actions, Temporal, and LangGraph tell you nothing (and charge you nothing back) until you run. A Nika workflow is auditable for cost, capabilities, secrets, and portability as a static fact about the file. nika check is an engine CLI surface (not a separate conformance level: it composes Core validation with the cost/secret/permits/parity reports); the guarantees it surfaces ARE normative (they derive from Core conformance + the permits: and secrets: MUSTs), the CLI ergonomics around them are the engine's.

Editor tooling · the canonical JSON Schema #

The spec ships a canonical JSON Schema at `schemas/workflow.schema.json` describing the envelope + task shape + verb argument structures. It is the machine-readable companion to this prose spec (kept in sync · the prose is normative on conflict).

Editors (VS Code · Zed · JetBrains · Neovim) pick it up via the standard yaml.schemas association (or a # yaml-language-server: $schema=… modeline) to give autocomplete + inline validation as you type, the same DX as GitHub Actions and Docker Compose. A working modeline today:

yaml

1# yaml-language-server: $schema=https://raw.githubusercontent.com/supernovae-st/nika-spec/main/schemas/workflow.schema.json2nika: v1

(The short https://nika.sh/spec/v1/workflow.schema.json form goes live with the site launch · both will resolve to the same schema.) This is also what makes a Nika file pleasant (and trap-free) for an AI to author: the schema constrains the shape before the engine ever runs. CEL expressions and jq expressions inside string fields are validated by the engine (Core level), not the JSON Schema.

Use case · linters · spec editors · LSP server intelligence · static analyzers.

Level 2 · Runtime conformance #

An engine claims « Runtime v0.1-compliant » if it satisfies Core conformance PLUS ·

Executes the 4 verbs with correct semantics- `infer:` calls a configured provider · returns response - `exec:` runs the command in a shell · honors timeout + security - `invoke:` resolves and calls the tool · returns response - `agent:` runs the multi-turn loop · honors max_turns + tools whitelist
Honors task fields correctly- GATE-v2 admission · every incoming edge's producer settles inside that edge's pass-set, else `cancelled` (dead-path · [03 §gate algebra](./03-dag.md#the-gate-algebra-v2-normative)) - `when` skips when false (POST-gate · local namespaces) - `timeout` (Go-duration string · `"30s"` · `"5m"` etc.) hard-kills on timeout - `retry` strategy honored on transient errors - `on_error` recovery honored on terminal errors - `with` scope injected into task body - `output` binding via jq
Implements security policies- `exec:` blocklist (engine SHOULD ship a sane default · MAY allow override) - `fetch:` SSRF defense for private IP ranges - `invoke:` capability checks (engine's choice of capability model)
Emits workflow events- `task.started` · `task.completed` · `task.failed` · `task.skipped` - `workflow.started` · `workflow.completed` · `workflow.failed` - Event payload includes `task_id` · timestamp · status · duration
Passes all tests in conformance/tests/runtime/

Runtime-compliant engines may bring their own provider implementations · tool implementations · MCP server registries. They are not required to ship the canonical stdlib.

Use case · custom engines for specialized environments (embedded · WASM · custom LLM gateway · etc.).

Level 3 · Stdlib v0.1 conformance #

An engine claims « Stdlib v0.1-compliant » if it satisfies Runtime conformance PLUS ·

Ships all 17 canonical providers (per stdlib/providers-v0.1.md)
Ships all 9 canonical extract modes (per stdlib/extract-modes-v0.1.md)
Ships at least all 28 canonical builtins (core 6 + file 5 + data 9 + network 2 + introspection 2 + media 4 · the remaining deferred media builtins are optional · the byte-determinism clauses of nika:image_fx and nika:chart are part of the bar — an engine that cannot honor them is not conformant, no waiver)
Passes all tests in conformance/tests/stdlib/

A Stdlib-compliant engine is functionally equivalent to the reference implementation for any workflow that uses only the canonical stdlib elements.

Use case · the default level for production engines.

Suite status · v0.1 (honest) #

What is populated TODAY vs what lands with the reference engine ·

Layer · Status · What it proves
**Core fixtures** (`tests/core/`) · ✅ populated · runner-executable · parse + validate + DAG + variables + errors · the full Level-1 static contract
**Deep-static fixtures** (`tests/deep/`) · ✅ populated · runner-executable · the expression layer the schema cannot see · the normative CEL EBNF parsed for real · jq compile · duration grammar · schema-meta · `when:` shape · binding purity
**Stdlib static surface** (`tests/stdlib/`) · ✅ populated · runner-executable · the stdlib **names + shapes** layer · provider prefixes · the closed `nika:*` builtin set · extract modes · checkable with zero execution (lists derive from [`canon.yaml`](../canon.yaml))
**Examples as conformance inputs** (`examples/`) · ✅ executed by the runner `all` gate · every shipped example MUST validate at the full static level
**Runtime behavioral fixtures** (`tests/runtime/`) · ⏳ **post-announce** · verb execution · task fields · events · they require an executing engine · they land with the reference engine's vertical slice (1.0.0)
**Stdlib behavioral fixtures** (`tests/stdlib/` · execution half) · ⏳ **post-announce** · provider/builtin/extract-mode *behavior* under the `mock` provider + HTTP mocks

Run the static gate yourself · python conformance/runner.py all: the runner output is the live count (counts in prose drift · the suite is the source). A « Core v0.1-compliant » claim is FULLY testable today. « Runtime » and « Stdlib v0.1 » claims are testable on their static halves today · their behavioral halves when the behavioral fixtures publish.

Conformance test structure #

text

1conformance/2├── tests/3│   ├── core/                  # Level-1 fixtures · parse · validate · DAG · variables · errors4│   ├── deep/                  # deep-static layer · CEL subset parse · jq compile ·5│   │                          # durations · schema-meta · when shape · binding purity6│   ├── lints/                 # advisory-lint fixtures7│   ├── runtime/               # verb execution · task fields · events (behavioral half)8│   └── stdlib/                # provider/extract/builtin canonical surface9│10├── runner.py                  # the static oracle · `all` is the CI gate11├── *_core.py                  # per-domain reference evaluators (type · decision ·12│                              # gateway · outcome · composition · yaml-profile ·13│                              # proof · projection) + their `*_selftest.py` sweeps14├── yaml-profile/              # R11 profile fixtures (valid/ + invalid/)15├── type-corpus/               # the generated type corpus (gen-type-corpus.py)16├── values/                    # the four-authority family (C2 · R3a · valid/ + invalid/)17├── types/                     # io-declaration predicate vocabulary (C2 · R3b · valid/ + invalid/)18├── gates/                     # the after: predicate vocabulary (C2 · R5 · valid/ + invalid/)19└── runner-protocol.md         # how to run the suite against any engine

The directory is the source — this sketch is a map, not the inventory (python3 conformance/runner.py all prints the live fixture counts). Each fixture is a pair · input.yaml (the workflow to feed) + expected.json (the expected output or error structure).

For tests that require executing against real LLMs / networks · the suite uses the mock provider and HTTP mocks to keep tests deterministic.

Runner protocol #

A conformance runner ·

Reads each input.yaml
Pipes it to the engine being tested (engine run --input -)
Captures the engine's output (stdout JSON · structured)
Compares against expected.json
Reports pass/fail per test · final summary

See conformance/runner-protocol.md for the exact JSON wire format.

Claiming conformance #

The public claim string is « Nika v1 Conformant — <Level> (spec <commit>) » · Level ∈ Core · Runtime · Stdlib v0.1 · the spec commit is the pin the suite ran against. One form everywhere — badges · READMEs · listings · release notes. (In normative sentences, « v0.1-compliant » names the level bar an engine satisfies; the string above is the one public claim.)

To make the claim · an engine ·

MUST pass the conformance suite at the claimed level
MUST name the level and the spec commit in the claim (the format above)
MAY open a PR on supernovae-st/nika-spec to be listed in CONFORMANT_IMPLEMENTATIONS.md

The claim is earned by passing the suite, never by declaration (NEP-0000 §Relationship to conformance). The reference implementation supernovae-st/nika targets Stdlib v0.1 conformance.

Versioning #

A conformance claim is specific to a spec version. As the stdlib evolves to v0.2 · engines re-claim conformance against the new suite. The Core conformance level is stable forever within v1 of the language.

🦋 Next · [08 · Out of scope](./08-out-of-scope.md)

nika-spec@6ac29351a · 07-conformance.md · sha256 bed30ec054def660… · the pack upstream