AI Gateway

Ask your coding agent “why did we choose boltdb over redis?” and it answers from your architecture notes — not because you pasted them in, but because the request passed through a stack you author, which found the relevant note and attached it before the model ever saw the question.

Setup is one environment variable, on the client:

ANTHROPIC_BASE_URL=http://localhost:8080 claude

The chassis answers POST /v1/messages in the Anthropic Messages protocol, runs the request through your _llm stack, then forwards it upstream and streams the response back byte-transparent. The session behaves identically to talking to the provider directly — while every request lands in a trace.

The stack runs before the request leaves

WHEN @src == "llm" && @llm.phase == "request" && .request.model == "txco-test-reject"
  EMIT @llm.reject.status  = 403,
       @llm.reject.type    = "permission_error",
       @llm.reject.message = "blocked by _llm policy"

.request is the client’s JSON — whatever your stack leaves there is what the upstream receives. Four things a rule can do:

• Reject — @llm.reject renders as a native protocol error, so clients surface it themselves. The upstream is never contacted, so a blocked request costs nothing.
• Rewrite — EMIT .request.model = "…" pins the model whatever the client asked for.
• Tag — @llm.headers.x-my-header rides along on the upstream call.
• Attach context — @llm.context, below.

The stack runs once per request; the response stream never passes through it, because the gateway owns the transport. After the exchange ends a second envelope arrives with @llm.phase == "completed", carrying status, duration, token usage, and model provenance. No client waits on it.

Context the model didn’t have

A rule emits items at @llm.context — [{source, title, content}] — and the gateway serializes the survivors into system blocks:

EMIT @llm.context = [{"source": "decisions",
                      "title": "boltdb over redis",
                      "content": "..."}]

They are appended after the client’s own system prompt, so its cached prefix survives, behind a chassis-owned guard block marking them untrusted evidence rather than instructions. Budget, item cap, and deduplication are enforced by the gateway, not your rules: the stack decides what, the gateway decides how.

Whose API key

Two modes, picked per tenant by what’s in the secret store:

• Passthrough (nothing stored) — the client’s own key is forwarded verbatim. No setup.
• Swap (store ANTHROPIC_KEY and LLM_GATEWAY_KEY) — clients authenticate with the gateway key, and the real key never leaves the server.

What you get back

• Model provenance is explicit — requested_model (the client’s ask), effective_request_model (what went upstream), and response_model (what was served) are three separate fields on the completion, never one guess.
• Token usage is recorded, not charged. @llm.completion.usage.* is presence-gated: absent means “not captured”, never zero.
• Injection is auditable. @llm.context_result is the gateway’s ground truth of what it injected and dropped and why — by sha256 and byte count, never the content itself, so traces don’t become a second copy of your notes.

Flags, the full envelope contract, and error shapes: the AI gateway inlet. A runnable walkthrough, including retrieval against a seeded decision log: examples/llm-gateway-hello/.