the provider catalog

vLLM

{"meta":{"title":"vLLM · high-throughput Nika workflow batches","angle":"The serving engine for throughput: PagedAttention and continuous batching turn one GPU into a fleet, behind the standard wire shape.","pick":"when the batch is the unit of work and tokens per second is the bill.","note":"Open source (Apache-2.0) · a PyTorch Foundation project, born at UC Berkeley.","setup":[{"text":"Install","code":"uv pip install vllm"},{"text":"Serve the model the file names","code":"vllm serve Qwen/Qwen3-8B"}]},"usage":{"id":"vllm","yaml":"nika: v1\nworkflow:\n  id: throughput-batch\n  description: \"one vLLM server, one batch — continuous batching earns its keep under fan-out\"\n\ntasks:\n  caption:\n    for_each: [\"checkout\", \"search\", \"onboarding\"]\n    max_parallel: 3\n    infer:\n      model: vllm/Qwen/Qwen3-8B\n      max_tokens: 60\n      prompt: |\n        One-line release note for the improved ${{ item }} flow.\n\noutputs:\n  notes: ${{ tasks.caption.output }}\n","file":"vllm.nika.yaml"},"audit":{"clean":true,"models_resolve":true,"tasks":1,"waves":1,"cost_ceiling_usd":null,"cost_floor_usd":0,"pricing":[{"model":"vllm/Qwen/Qwen3-8B","input_per_million":null,"output_per_million":null}],"pricing_as_of":"2026-07-07"}}

[{"slug":"qwen-qwen3-8b","id":"Qwen/Qwen3-8B","aliases":["qwen"]}]

The serving engine for throughput: PagedAttention and continuous batching turn one GPU into a fleet, behind the standard wire shape. One provider/model line moves a workflow here or away again: the same file runs on every provider the catalog names, and swapping is a one-word diff. Machines read the catalog; the binary answers nika catalog.

1pinned modelaliases pin exact upstream ids
localfamilythe sovereign default: your hardware, no key
no keynothing to leakthe sovereign path
openai-chatwire dialectthe adapter speaks it

the wire

local · the sovereign default: your hardware, no key

A workflow names this provider in the model line (model: vllm/Qwen/Qwen3-8B), and the engine speaks openai-chat on the wire. No key exists on this path: nothing to rotate, nothing to leak.

default: Qwen/Qwen3-8B
dialect: openai-chat
tags: local · open-source

the models

1 pin

Aliases are what the file may write; the engine pins each one to an exact upstream id, so a run stays reproducible after a vendor rotates names.

qwenQwen/Qwen3-8Bctx 128k · vision

in the released catalog

1 room

The wire models this provider serves in the released binary's catalog · each one is a room with its seats, exact prices and measured energy.

Qwen/Qwen3-8Bas qwen

the whole register: /catalog/models · what the released binary knows, vendored at the engine pin.

in a real file

a complete workflow · shown whole

vllm.nika.yaml

1nika: v12workflow:3  id: throughput-batch4  description: "one vLLM server, one batch — continuous batching earns its keep under fan-out"56tasks:7  caption:8    for_each: ["checkout", "search", "onboarding"]9    max_parallel: 310    ◇infer:11      model: vllm/Qwen/Qwen3-8B12      max_tokens: 6013      prompt: |14        One-line release note for the improved ${{ item }} flow.1516outputs:17  notes: ${{ tasks.caption.output }}18

check-green against the released engine · 1 task · 1 wave · no closed ceiling: an unpriced model is unpriced, never free · the drift gate re-proves this copy on every test run · open it in the playground →

get set up

1Install uv pip install vllm
2Serve the model the file names vllm serve Qwen/Qwen3-8B

Pick it for when the batch is the unit of work and tokens per second is the bill. Open source (Apache-2.0) · a PyTorch Foundation project, born at UC Berkeley. Then prove the seat: nika doctor names every provider this binary can reach.

swap it

4 more local seats

Moving off vLLM is editing one line. The rest of the file (tasks, permits, outputs, the trace) does not change. The register orders the catalog local first; beyond it the engine embeds 21 more OpenAI-compatible endpoints.

model: vllm/Qwen/Qwen3-8B model: mistral/mistral-large-latestthe whole migration · nothing else moves

cross-references

the words that meet it

[]

where it lives

Try it in the playground, or install and go local first. Read the spec →