AGPL-3.0-or-later · forever.

the workflow library

etl-quarantine

zero model · bad rows quarantine, good rows aggregate. This file carries a real recorded trace — the home page replays it, beat for beat, from the run record.

  • 7tasksthe plan below walks them
  • 5wavesrun together when they can
  • invokeverbthe whole grammar it speaks
  • recordeda real runthe trace is the proof

the plan

derived from the file, never drawn by hand
empty_batchinvokerawinvoke
rowsinvoke
checkinvoke
goodinvokequarantineinvoke
reportinvoke

the file, whole

etl-quarantine.nika.yaml
etl-quarantine.nika.yaml
nika: v1workflow:  id: etl-quarantine# zero model · a schema gate splits the batch# the file IS the blast radiuspermits:  fs: { read: [ ./data/incoming/* ], write: [ ./data/* ] }  tools: [ "nika:read", "nika:convert", "nika:validate", "nika:jq", "nika:write" ]tasks:  # the deterministic fallback if parsing dies  empty_batch:    invoke: { tool: "nika:jq", args: { input: [], expression: "." } }  raw:    invoke: { tool: "nika:read", args: { path: ./data/incoming/orders.csv } }  rows:    with:      raw: ${{ tasks.raw.output }}    invoke:      tool: "nika:convert"      args: { input: "${{ with.raw }}", from: csv, to: json, has_header: true }    # malformed CSV → empty batch · the run survives    on_error:      recover: ${{ tasks.empty_batch.output }}  check:    with:      rows: ${{ tasks.rows.output }}    invoke:      tool: "nika:validate"      args:        data: "${{ with.rows }}"        format: json        schema:          type: array          items:            type: object            required: [ order_id, amount, currency ]            properties:              order_id: { type: string }              amount: { type: string }              currency: { type: string, enum: [ EUR, USD, GBP ] }  good:    with:      rows: ${{ tasks.rows.output }}      valid: ${{ tasks.check.output.valid }}    when: ${{ with.valid == true }}    invoke:      tool: "nika:jq"      args:        input: "${{ with.rows }}"        expression: 'group_by(.currency) | map({currency: .[0].currency, orders: length, total: (map(.amount | tonumber) | add)})'  quarantine:    with:      valid: ${{ tasks.check.output.valid }}      errors: ${{ tasks.check.output.errors }}    when: ${{ with.valid == false }}    invoke:      tool: "nika:write"      args:        path: ./data/quarantine/rejected.json        content: "${{ with.errors }}"        create_dirs: true  report:    with:      totals: ${{ tasks.good.output }}   # value edge · passes when good is skipped (reads null)    when: ${{ with.totals != null && size(with.totals) > 0 }}    invoke:      tool: "nika:write"      args:        path: ./data/daily-totals.json        content: "${{ with.totals }}"        create_dirs: trueoutputs:  totals: "${{ tasks.good.output }}"

lines 2527: on_error: a bad batch degrades, the run survives. Source: the served copy · byte-pinned to the recorded trace by the honesty suite · open it in the playground →

Watch the recorded one replay on the home page, walk the showcase gallery, or install and run the file yourself.