Trace logs

The chassis records per-request artifacts (the inbound envelope, every op execution, the final response, plus a timeline of routing events) to durable storage. Developers browse the result to see exactly what happened during a request without rerunning it or turning on debug logging.

Modes

--trace-mode controls how much detail is written:

Mode	What gets written
off (default)	Nothing. Zero-cost no-op.
summary	Request header + timeline + per-step meta.json (timings, sizes, status). No payload bytes.
full	Everything, including handler in/out bodies per step.

--trace-dir is the root (default ./data/trace). --trace-mode=full --trace-dir=/var/log/txco/traces is a typical production setting when you actually want payloads. In full mode, bodies are capped per step by --trace-body-cap-bytes (default 65536).

--trace-async=true buffers writes through a worker goroutine so the request path never blocks on disk I/O. Recommended in production.

File layout

Per request, under <trace_dir>/requests/<rid>/:

in.json            inlet's initial envelope + chassis metadata (rid, src, tenant, stack)
out.json           chassis's final response, after all merges
timeline.jsonl     line-per-event log (request.start, step.start, step.end, request.end)
steps/
  0000-boot/       one folder per fired op, zero-padded scope so the directory sorts in scope order
    op.json        the rule's stored definition (txcl, exec, etc.)
    in.json        envelope handed to the handler
    out.json       handler's raw response
    meta.json      timing, sizes, status, transport

Two ops at the same scope produce sibling step folders with the same NNNN- prefix — that’s the trace’s signal for “ran in parallel.”

In summary mode the per-step op.json / in.json / out.json are omitted; meta.json and the timeline still record what ran and how long it took.

Browsing traces — txco trace

txco trace                  # list recent traces with rid, tenant, src, route, duration
txco trace <rid>            # one-row-per-step table for that request
txco trace last             # alias for the most recent rid
txco trace <rid> --verbose  # include payloads (full mode only)

Other flags: --step, --json, --plain, --grep, --watch.

The admin UI’s Traces tab shows the same data with a step-by-step explorer.

Keeping sensitive data out of traces

Trace logs persist whatever the rules touched, byte-for-byte. For production deployments that handle user data — auth tokens in headers, raw mail bodies, classifier outputs containing PII — two reserved WITH keys on any rule scrub the trace artifacts at write time, without touching runtime data:

WHEN @web.req.url.path = "/checkout"
WITH redact = "_txc.web.req.body, _txc.web.req.headers.authorization"
WITH omit   = "_txc.lmtp.msg.attachments"
EXEC "op://CHECKOUT"

Keyword	Effect on the trace JSON
redact = "a.b, c.d"	Replaces each path’s value with the string "[REDACTED]". Field stays; value is masked.
omit = "x.y"	Deletes the path entirely. Field vanishes from the trace JSON.

• The rule’s own WHEN / SELECT / EXEC still see the full envelope. Only what hits durable storage is affected.
• Paths are exact gjson dot-paths.
• A path listed in both redact and omit is resolved by omit wins (the more aggressive choice).
• Hints are scoped per (tenant, stack): a redact declared in acme/support does NOT bleed into acme/billing even though both are tenant acme. If a request enters acme/support and an EXEC jumps to acme/billing, both stacks’ hints apply to the trace from that point on (union semantic).
• Hints are static — only literal string values are honored. WITH redact = @some.path (runtime resolution) is silently skipped, since the trace sink must know the paths at write time.
• The chassis collects hints at startup and rebuilds the registry on every dbcache reload (so txco apply picks up new hint declarations without a restart).
• Application order is omits first, then redacts. Applying omits first lets _txc.lmtp.msg vanish without wasting a sentinel write on _txc.lmtp.msg.headers.authorization.

Where it runs

The redaction layer lives below the async-sink boundary, so in async mode the work happens on the trace worker goroutine — off the request hot path. Per-write cost is one gjson.Get existence check plus an sjson.Set or sjson.Delete per declared path; sub-millisecond for typical envelopes.

In sync mode (--trace-async=false) redaction runs on the request goroutine just before the disk write. Still cheap; the disk fsync that follows is the dominant cost.

What’s NOT redacted

• Runtime data is never touched. The rule’s matchers, the op’s input, the merged response all see real values.
• Timeline events (timeline.jsonl) are not currently filtered. Avoid stamping sensitive fields directly into timeline event payloads.
• Operational logs (zap stdout) are not affected — apply log redaction at the logger if needed.
• The DLQ preserves the original envelope by design (forensics).