All docs

Docs · Reference

TCP

line-delimited JSON

The TCP head (--tcp-listen-addrs, default :5050) is the raw socket channel: one JSON object per line in, one response line out, connection held open. A listener can speak another protocol instead — see Protocol handlers.

Wire protocol

  • In: newline-delimited messages, up to 10 MB per line (an over-limit line is consumed and dropped; the connection stays open). Each line becomes one event.
  • Out: by default, the merged envelope as one JSON line (_-prefixed fields stripped). A rule can write raw bytes instead via @tcp.res.write (base64-decoded on the way out), or close the connection with @tcp.res.action = "close" (after any write).
  • On accept, a connect event fires first (before any line). Accept is implicit: the connection stays open unless the run was denied, failed, or set @tcp.res.action = "close". A @tcp.res.write on the connect run is the greeting; the envelope itself is never echoed there. Then the read loop: line → message event → response, repeated until idle timeout or close.
  • Admission denials are written as a "<status> <reason>" line, then the connection closes. A draining or full node answers a new connection with 503 draining / 503 too many connections and closes before any connect run.

The verdict subtree follows the same pattern as the other protocol heads: @tcp.* holds facts the chassis observed (read-only), @tcp.res.* is what the stack decides.

Envelope fields

FieldMeaning
@tcp.listenerListener name (--tcp-listen-addrs=webhooks=:5050,iot=:5051; bare addresses are default)
@tcp.local.{ip,port}The address the client connected to — route on raw port without any ingress config
@tcp.remote.portThe client’s port (@client.ip is its address)
@client.ipClient address — behind a trusted edge, the real client’s, not the edge’s
@client.bodyThe incoming line, base64-encoded (absent on the connect event)
@tcp.res.writeVerdict: base64 bytes to write to the socket
@tcp.res.actionVerdict: "close" hangs up after any write
@tcp.inletThe inlet a hostname routes this listener’s connections into: _tcp, or _NAME under ;handler=NAME
@tcp.hostThe connection’s canonical hostname (TLS SNI, lower-cased, port and trailing dot dropped) — the routing fact detect-tenant reads
@tcp.tls.enabledWhether the client’s connection is TLS
@tcp.tls.{sni,alpn,version}What the handshake observed (sni is the raw name; host above is its canonical form)

The @tcp.host / @tcp.tls.* facts read the same whether the chassis terminated TLS itself (;tls) or a trusted edge did and reported it (;proxy=, below). A stack never needs to know which.

Routing

A connection is routed once, on the connect event, through the same _sys/boot path as every other source (txco://detect-tenant):

  1. @tcp.host (the SNI — ours, or the one a trusted edge reported) is matched against the tenant’s verified hostnames — the same gate a certificate needs, whatever --require-hostname-verification says for HTTP — and the connection enters that hostname’s stack’s _tcp inlet: hostname → stack shop → shop/_tcp. (On a listener with a protocol handler it is that protocol’s inlet instead: shop/_echo.)
  2. Otherwise the listener name is matched against ingress.tcp.listeners in the routing YAML — the single-destination case, where the operator names the stack outright.
  3. Otherwise the connection is closed before any line is read. An unknown hostname never lands in a default tenant.

The connection is then pinned: every later event on it carries the route the connect run chose, so it lands in the same stack without being routed again. Pinning does not outlive the opt-in, though — before each event the head asks the router whether that route still stands, and if it no longer does (the inlet deactivated, the hostname revoked or re-bound) the connection closes without running the event. Admission (a suspended tenant, a rate limit) still applies to every event.

Opting in: the _tcp inlet

TCP is opt-in per stack. A verified hostname that serves a stack over HTTP is not a TCP endpoint until that stack has an active _tcp inlet — a nested stack beside its other channels, like _mail:

OPS/shop/0100_…            ← the stack's HTTP rules: never see a TCP event
OPS/shop/_tcp/0100_GREET/greet.txcl
# OPS/shop/_tcp/0100_GREET/greet.txcl — the connect run (no line yet)
WHEN @client.body == ""
  EMIT @tcp.res.write = "aGVsbG8K"
txco push shop/_tcp        # a nested inlet is its own stack: push it by name

No _tcp inlet → step 1 misses and the connection is closed without a single rule running, so opening a TCP port never exposes the stacks that did not ask for it. txco deactivate shop/_tcp closes the door again: new connections are refused, and open ones close at their next event. The inlet shares its parent’s hostname, and everything in it is @src == "tcp" by construction — no source guards needed.

TLS

--tcp-listen-addrs "irc=:6697;tls,raw=:5050"

;tls terminates TLS on that listener with the bundled cert manager (the same certificates the web and IMAPS heads serve; needs --web-tls-addr or --imap-tls-addrs with the dns personality to exist). The handshake’s SNI becomes @tcp.host. For local development ;self-signed mints a dev certificate (loopback + *.local.thanks.computer, written beside the data dir) instead — never for a public deployment.

openssl s_client -connect 127.0.0.1:6697 -servername irc.moo.local.thanks.computer

Behind a TLS-terminating edge

--tcp-listen-addrs "edge=:16697;proxy=172.16.0.0/12|fdaa::/8"

When an edge proxy terminates TLS in front of the chassis, the chassis never sees the handshake — so the edge says what it saw. The contract is one thing: a PROXY protocol v2 header ahead of the stream, carrying

Header partBecomes
source / destination address@client.ip, @tcp.remote.port, @tcp.local.{ip,port}
PP2_TYPE_AUTHORITY (the SNI)@tcp.tls.sni, and canonicalised, @tcp.host
PP2_TYPE_ALPN@tcp.tls.alpn
PP2_TYPE_SSL (+ SSL_VERSION)@tcp.tls.enabled, @tcp.tls.version

;proxy= lists the networks the edge connects from (|-separated CIDRs or IPs — never spaces or commas inside an entry: both separate entries, on the flag and in TXCO_TCP_LISTEN_ADDRS). The listener is an edge-only door: the header is required from those networks, and a connection from anywhere else — or one that skips or stalls the header — is closed before any event runs. A header an outsider sends is never parsed. Keep the port off the public internet anyway; the CIDR list is the trust boundary, not a substitute for one.

;proxy= does not combine with ;tls: a listener either observes the handshake or trusts an edge that did. Any proxy that writes this header works (HAProxy’s send-proxy-v2-ssl + proxy-v2-options authority does); for Caddy, whose layer4 proxy sends no TLVs, the caddy/proxytlv module adds them.

Protocol handlers

--tcp-listen-addrs "raw=:5050,echo=:7007;tls;handler=echo"

Everything above — accept, caps, TLS or the edge header, the connect run, routing, drain — is the head’s, whatever is spoken afterwards. What happens to the bytes after a connection is accepted belongs to the listener’s handler. ;handler=NAME picks it; without the option a listener speaks line, the protocol this page describes.

HandlerSpeaksInletEvents
line (default)one event per newline-terminated line<stack>/_tcp@src == "tcp"
echoraw bytes written straight back; hangs up after 500<stack>/_echoone @src == "echo" when the connection ends

The handler’s name does three jobs, so that a port’s protocol is the operator’s choice and speaking it is each stack’s:

  • it is the inlet a hostname routes into — _NAME (_tcp for line). A stack opts into a protocol, not into “TCP”: shop/_tcp does not make shop an echo endpoint;
  • it is the @src of the events the handler emits, with the handler’s facts under @NAME.*. The connection’s @tcp.* facts and @client.ip ride along on every one of them;
  • an unknown name refuses to boot — a typo never falls back to line.

The connect run is the same on every listener: a @src == "tcp" event in the handler’s inlet, answered on @tcp.res.*. So any protocol can be greeted or refused in txcl before the handler reads a byte.

echo is the worked example — useful on its own for checking a listener, an edge and a hostname end to end:

# OPS/shop/_echo/0100_GREET/greet.txcl — the connect run
WHEN @src == "tcp"
  EMIT @tcp.res.write = "ZWNobyByZWFkeQo="

# OPS/shop/_echo/0200_CLOSED/closed.txcl — after the connection ended
WHEN @src == "echo"
  EMIT .echoed = @echo.bytes       # @echo.reason: limit | eof | idle | error
txco push shop/_echo
openssl s_client -quiet -connect 127.0.0.1:7007 -servername shop.local.thanks.computer

Writing a handler

A handler is Go, compiled into the chassis, registered at init:

func init() { tcp.RegisterHandler("irc", func(pu *processor.Unit) tcp.Handler { return &ircHandler{} }) }

func (h *ircHandler) ServeConn(ctx context.Context, rc tcp.RoutedConn) error {
    // rc.Conn is the decrypted stream; rc.Tenant / rc.Stack / rc.Host are trusted.
    res, err := rc.Emit(ctx, tcp.Event{Body: line, Facts: map[string]any{"cmd": "PRIVMSG"}})
    if err != nil {
        return err // rerouted, denied (*tcp.DeniedError), timed out, shutting down: hang up
    }
    // read the stack's verdict from res.Payload — @irc.res.*
    return nil
}
  • RoutedConn carries only what the head observed or the router decided. A handler never learns a tenant from an envelope, or from the client.
  • Emit is the only way to run an event. It stamps the source, the connection facts and the pinned route itself — after the handler’s facts, so a handler cannot displace them — re-checks the route, and returns the run’s result. Any error ends the connection.
  • One handler value serves all of a listener’s connections concurrently. A panic in ServeConn costs that connection, not the chassis.
  • The stack’s verdict for a new source needs its own author-writable subtree (@irc.res.*), allow-listed alongside tcp.res when the handler lands. Names another head owns (mail, http, cron, …) are refused at registration.
  • Work a handler does itself, outside any run, is charged with rc.AddFuel(n) (see Fuel); the connect and every byte through rc.Conn are charged for it already.
  • There is no per-connection budget: each event is bounded like any other run, a connection’s lifetime total is metered but not capped.

Fuel

Every run on a connection — the connect run, each line, each handler event — is an ordinary metered run with its own usage line. What a connection costs between runs has no run to be counted in, so the head meters it and carries it onto the connection’s next run, as that run’s starting _txc.fuel_used. It lands on the same usage event, attributed to the same tenant, as everything else the run cost.

ChargedFuelLands on
Accepting the connection (--tcp-conn-fuel)100the connect run
Bytes in and out, per MiB (exact; fractions carried)100the next event
Handler work (rc.AddFuel)what the handler saysthe next event
  • A connection that is never routed is nobody’s: its connect fee reaches no tenant.
  • At most half of --max-fuel-per-request is carried onto one run, so a connection that moved a lot between events never makes its next run arrive exhausted; the rest follows on later runs.
  • Time is not charged. An idle socket does no work; what it holds is capacity, which --tcp-max-conns and --tcp-max-conns-per-tenant bound.
  • The close log line carries the lifetime figures — bytes_in, bytes_out, conn_fuel, and conn_fuel_unbilled: whatever accrued after the last run (a final response’s bytes, a handler that never emits) and so reached no usage event.

Timeouts and limits

FlagDefaultGuards
--tcp-connect-resp-timeout3sThe connect event’s rule response
--tcp-resp-timeout10sEach message event’s rule response (and the write back)
--tcp-max-idle-timeout5sSilence between lines (echo: between reads)
--tcp-max-conns0 (unlimited)Open connections on this node, all listeners
--tcp-max-conns-per-tenant0 (unlimited)Open connections per tenant, counted once the connect run has routed
--tcp-conn-fuel100Fuel charged per accepted connection (Fuel); 0 = none
--tcp-handshake-timeout5sThe TLS handshake on a ;tls listener, or the PROXY header on a ;proxy= one
--tcp-drain-timeout5sHow long shutdown waits for open connections to unwind after closing them

Edit this page · View as markdown