Workspaces

A workspace is an owned, stateful execution environment your stack addresses by name. Where an op:// nano-op is a pure JSON→JSON transform in a sandbox with no filesystem and no network, a workspace keeps files between calls, has a full runtime, and (in the fleet) reaches the public internet. Think “a machine that belongs to this stack”, without you operating a machine.

WHEN @web.req.url.path == "/build"
  EXEC "workspace://builder/exec"
    WITH command = "git pull && make",
         into = "_build"

The mental model: lambda-like, but it has a name

Owned. A workspace belongs to one (tenant, stack, name). tools in stack agents of tenant acme is always the same environment.
Stateful. Files written by one exec are there for the next — across requests, across chassis restarts.
Lambda-shaped in cost. It wakes on exec, sleeps when idle, and is destroyed after long disuse (the data is gone; the next exec starts fresh). You pay for runs, not residency.
Exit codes are data. A command that exits 3 is a successful dispatch whose result says exit: 3. A transport failure — the workspace would not wake, the command timed out — merges as workspace.error instead. Either way the op is not dropped.

Vocabulary: a computer executes work; a workspace is the owned, stateful environment that provides one; a run is one wake→sleep; a task spans runs.

The ref

workspace://<name>/<verb>

The verb is the segment after the last /, so a name may nest: workspace://pony/paris/exec is workspace pony/paris, verb exec. Names are 1–4 /-separated segments, each a DNS-label-shaped token (^[a-z0-9][a-z0-9-]{0,62}$, no --), and no segment may equal a verb.

WITH workspace = <expr> replaces the ref’s name entirely — the ref’s name is a static default, the WITH value (a literal or an envelope path) wins:

EXEC "workspace://pony/exec"
  WITH workspace = ._in.slug,      # e.g. "pony/paris"
       command = "npm test"

Verb	What it does	Result under `into`
`exec`	Run a command (creating and waking the workspace as needed)	`{exit, stdout, stderr, …}`
`create`	Make sure the workspace exists, without waking it	`{state: "created"}`
`wake`	Warm it ahead of a task’s first exec	`{state: "running"}` + `_txc.workspace.run`
`checkpoint`	Snapshot it (`WITH comment = "…"`); providers with the capability only	`{state: "checkpointed", checkpoint_ref}`
`sleep`	Park it (a no-op where the provider sleeps implicitly)	`{state: "sleeping"}`
`destroy`	Remove it — files are gone; the next exec starts fresh	`{state: "destroyed"}`

exec also takes WITH checkpoint = true (and an optional comment): after an exec that exits 0 the workspace is snapshotted and checkpoint_ref is added to the result — “after cloning”, “after install” are yours to name. A checkpoint that fails does not fail the exec; checkpoint_error {code, message} says why. A provider without the capability (the local one) reports code = "unsupported".

`exec` — the request

`WITH` key	Meaning
`command`	A shell line, run by the workspace’s `/bin/sh -c`
`args`	An argv (array of strings), no shell — use for untrusted arguments. `command` and `args` are mutually exclusive
`stdin`	Bytes fed to the process (a string; a JSON value is fed as its text)
`cwd`	Where the command starts: beneath the workspace (a relative path, or `$HOME/…`), in the stack’s own files (`$TXCO_STACK_DIR/…`), or an absolute path on the computer. See Where a command starts
`env`	An object of extra environment variables
`into`	Where the result lands (default `_workspace`)
`timeout`	Wall clock for the whole exec — create/wake, the command, output capture; the command is killed when it expires. Default `--workspace-default-timeout` (5m), capped by `--op-timeout-max` (10m). A synchronous HTTP request is also bounded by whatever fronts the chassis — the hosted edge allows 20 s for response headers — so anything longer must use `WITH mode = "continuable"` (202 + poll); when the client gives up, the request is cancelled and the command is killed
`grant`	A run grant’s id, from `txco://delegate/mint`. With `sandbox`, the chassis opens the named sandboxes and starts the command with what they set; on the chassis’s own machine the command may also open more itself with `txco sandbox`. `exec` only, and not with `stream`. See grants
`sandbox`	A sandbox name, or a list of them, of the grant’s: `SANDBOXES/<name>.yaml` of this stack. What they set is put in the command’s environment, over `env` and `secrets.env.*`; one refused, and the command never starts. Needs `grant`.
`secrets.env.<NAME>.secret` / `.format` / `.optional`	A stored secret, materialized into the environment as `NAME` (`format = "Bearer {}"` templates it). The only place a workspace op takes a secret — `secrets.headers.` / `.body.` are refused
`stream = true`	Send stdout to the client as it is produced (see Streaming output)
`tty = true`, `cols`, `rows`	Run the command with a pseudo-terminal (see A TTY on exec)
`checkpoint = true`, `comment`	Snapshot after a successful exec (see the verbs)

The command sees a scrubbed environment — PATH, HOME (= the workspace), TMPDIR (inside it), plus your env and any secrets.env.* — never the chassis’s.

A script longer than a line belongs in its own file beside the op, pulled in with &include: fixed argv, the script as data, no escaping. A program (several files, a module beside it, or anything you would rather run than paste) belongs in the stack and runs by path from $TXCO_STACK_DIR.

WITH args  = ["python3", "-c", &include("server.py")]   # small: in argv
WITH command = "bash -s", stdin = &include("setup.sh")  # large: on stdin

One argument is capped by the operating system (128 KiB on Linux), so feed anything big on stdin.

Secrets never come back out. Every materialized secret’s value is replaced with [REDACTED] in stdout, stderr and error messages before the result is built, so neither the envelope nor the trace step carries it — even if the command prints its environment. Values shorter than 8 bytes are not scrubbed (scrubbing “1234” would shred unrelated output); real tokens are far longer. A format-templated value still contains the raw secret, so scrubbing the raw value covers it.

Where a command starts: `cwd`

cwd is either a path on the computer or a path beneath one of two roots the chassis resolves:

`cwd`	Starts in
omitted, `"foo"`, `"$HOME"`, `"$HOME/foo"`	the workspace, or a directory in it
`"$TXCO_STACK_DIR"`, `"$TXCO_STACK_DIR/foo"`	the stack’s own files, read-only (below)
`"/foo"`	an absolute path on the computer

.. is refused only where it would climb out of its root: "$HOME/a/../b" is fine, "$HOME/../etc" is not, and an absolute path is cleaned ("/tmp/x/../y" is /tmp/y). The directory must exist.

The two roots are tokens the chassis resolves, and only in cwd: txcl does no $ expansion, and args and env are passed exactly as given, which is what keeps args safe for untrusted values. In command the shell expands $HOME and $TXCO_STACK_DIR like any other variable.

cwd resolves; it does not authorize. An absolute cwd reaches nothing a cd in the command could not, and what a command may touch is the computer’s business: on the local provider that is anything the chassis’s own user can.

The stack’s own files: `$TXCO_STACK_DIR`

A program longer than a script lives beside the op that runs it, and runs by path:

# OPS/game/2400_MOVE/move.txcl, with OPS/game/2100_SETUP/race.py in the stack
WHEN ._race.ok == true
  WITH cwd  = "$TXCO_STACK_DIR",
       args = ["python3", "2100_SETUP/race.py"],
       env  = &object("MODE", "move"),
       into = "_move"
  EXEC "workspace://browser/exec"

What is in it. When one of a stack’s ops names $TXCO_STACK_DIR, txco apply packs the stack’s directory, OPS/<stack>/, keeping its layout. It leaves out dotfiles, the directories the chassis already takes by kind (FILES/, CAPS/, OUTLETS/, SANDBOXES/, DATASETS/, SOURCES/ and the data packs) and any nested stack, which has its own. A stack that never names it carries no tree. The caps are 1 MiB a file and 32 MiB and 10,000 files a tree; apply refuses a tree over them and names its largest files.
Its version is the op’s. The tree is stored with the stack version, by content hash, and an unchanged tree costs a new version nothing. A run uses the tree of the version it started with, and a run resumed from a continuation uses the tree it suspended with: an apply, a rollback or a failed apply never changes the files under a running op.
Read-only. Files are 0444, or 0555 when the source was executable, and directories 0555. A program can be run directly ("$TXCO_STACK_DIR/bin/tool") but cannot write into itself: write state to $HOME, which is still the workspace.
Only when asked. The tree is placed, and TXCO_STACK_DIR set, only for an exec whose cwd names it. To run a program from the tree but start in the workspace, change directory in the command: command = "cd \"$HOME\" && exec \"$TXCO_STACK_DIR/bin/tool\"".
Where it lands. The local provider places each tree once per chassis, under <--workspace-local-root>/.stacks/<digest>/. Like Go’s module cache it is read-only, so remove it by hand only after chmod -R u+w. A provider that cannot place a tree answers workspace.error {code: "stack_dir_unavailable"}, as does a stack version without one (applied by an older txco).

`exec` — the result

{
  "_workspace": {
    "exit": 0,
    "stdout": "hello\n",
    "stderr": "",
    "stdout_truncated": false,
    "stderr_truncated": false
  },
  "_txc": {
    "workspace": {
      "provider": "local",
      "computer": "<the provider's id for it>",
      "run": "<run id>",
      "exit": 0,
      "duration_ms": 4
    }
  }
}

stdout and stderr are each capped at --workspace-max-output-bytes (1 MiB); the excess is dropped and flagged *_truncated. On a fleet provider, output a command produces in roughly its first 20 ms may come back with stderr merged into stdout and stderr empty (the provider replays the session’s history, which has no stream separation); exit is exact regardless. Gate on exit, not on stderr being non-empty.

_txc.workspace.* is the chassis’s provenance stamp, written after the command returns. The command’s output is a string under into; it never produces envelope JSON, so it can never forge the stamp (the output sanitizer lets _txc.workspace.* through for the workspace transport only, and strips every other reserved _txc.* path as it does for any untrusted producer).

On a transport or provider failure:

{
  "_workspace": {},
  "workspace": {
    "error": {
      "code": "timeout",
      "message": "…"
    }
  }
}

Codes: timeout, provider, capacity, bad_request, not_allowed, unsupported, unavailable (a connect whose service is not running), unsupported_verb. Top-level (not under _txc) so a rule can gate on it: WHEN .workspace.error.code == "timeout".

The op is dropped — logged at ERROR, nothing merged — only on authoring errors: a malformed ref, a bad name, no provider configured, an untenanted request, or a malformed secrets block.

A TTY on exec

WITH tty = true runs the command with a pseudo-terminal instead of pipes. Many tools behave differently without one: progress bars, colour, pagers, and login flows that refuse to read a password from a pipe. This gives an agent the ordinary one-shot exec it already has, with a terminal behind it — structured in, structured out, one result. It does not turn exec into a stream; for a live interactive terminal a person drives, see attach, and for a service the workspace runs, connect.

WHEN @web.req.url.path == "/cols"
  EXEC "workspace://tools/exec"
    WITH command = "tput cols; test -t 1 && echo isatty",
         tty = true, cols = 120, rows = 40,
         into = "_ws"

A terminal has one stream, so tty = true merges stderr into stdout and leaves stderr empty — the same result shape a fleet provider’s history replay can produce, and the reason to gate on exit, not on stderr. cols/rows set the initial geometry (default 80×24) and are a bad_request without tty. The local dev provider needs a real PTY (github.com/creack/pty, pulled in automatically); a provider that cannot allocate one answers workspace.error.code = "unsupported".

Streaming output

A build or a test suite is worth watching while it runs. WITH stream = true sends the command’s stdout to the client as it is produced, instead of collecting it and returning it in the envelope:

WHEN @web.req.url.path == "/build"
  EXEC "workspace://builder/exec"
    WITH command = "make -j8",
         stream = true,
         into = "_build"

The client gets a chunked response: the first bytes arrive as soon as the command produces them, and the connection stays open until it exits.

What changes in the result:

{
  "_build": {
    "exit": 0,
    "stdout_streamed": true,
    "stdout_bytes": 184320,
    "stderr": ""
  }
}

stdout is absent by design — the bytes went to the client, not into the envelope, so a 2 MB log never rides the envelope, the trace step, or a continuation. stdout_bytes is there so a rule can still gate on whether anything was produced. stderr is unaffected: still captured, still capped.

Three consequences worth knowing before you use it:

Once bytes flow, the response is the stream. The status and headers go out with the first chunk, so a later scope can no longer change them and the envelope’s JSON rendering never reaches the client. Set @web.res.status and any headers before the EXEC if you need something other than 200 text/plain. A command that produces no output never opens the stream, so the request renders its normal JSON response.
It outruns proxy timeouts. A front proxy typically limits how long it waits for response headers, not how long a body may take. Streaming sends the head immediately, so a long command survives a limit that would kill the same command run synchronously. (On the hosted service that limit is 20 seconds.)
Backpressure is real. Chunks block until the client takes them, so a command that outruns a slow reader is throttled rather than buffered. If the client disconnects, the request is cancelled and the command is killed, exactly as on any other timeout.

stream is refused, in band, in four cases where it would be a lie: with LOOP (each pass would reopen one response), with mode (a promoted request has no client left), with secrets (a value split across two chunks could not be redacted), and on a run with no live HTTP client (cron, a resumed continuation). Each returns workspace.error.code = "bad_request" with the reason.

Testing it locally: txco dev turns debug breakpoints on by default, and breakpoints deliberately disable streaming (they dump the whole envelope instead). Run TXCO_DEBUG_BREAKPOINTS=false txco dev --allow-local-workspace to see a stream in dev, and use curl -N so curl doesn’t buffer it back up.

Not streaming, but large? Capture as usual and drop the value once you are done with it, so it stops riding the envelope: EMIT @delete = ["_build.stdout"] — see EMIT @delete.

`attach` — an interactive terminal

Where exec runs a command to completion, attach binds a live terminal to a WebSocket session: keystrokes reach the process as they are typed and its output arrives as it is produced. It is how a person drives a program inside the workspace — vi, top, a shell — through a websocket connection the stack authorized. See examples/workspace-terminal for the whole thing, browser page included.

attach is valid only inside a WebSocket session run: the stack accepts an upgrade, and the first message the client sends attaches.

# term/_websocket/0 — the first message attaches, once.
WHEN .msg.type == "attach"
  EXEC "workspace://tools/attach"
    WITH command = "tmux new -A -s main",
         cols = .msg.cols, rows = .msg.rows,
         max_duration = "8h",
         into = "_attach"

`WITH` key	Meaning
`command` / `args`	What runs in the terminal — `tmux new -A -s main` is the usual choice (reattach + scrollback + survival across disconnects, all inside the workspace)
`cols`, `rows`	Initial geometry; the client resizes later with a control frame
`max_duration`	How long the binding may live, default and capped by `--workspace-attach-max-duration` (8h). When it passes, the process is killed and the socket told
`into`	Where the grant lands (default `_workspace`)

On success the chassis binds a pseudo-terminal and returns immediately:

{
  "_attach": {
    "attached": true,
    "run": "<run id>",
    "workspace": "tools",
    "node": "<node>",
    "lease": {
      "id": "<lease>",
      "expires_at": "…"
    }
  }
}

From then on the connection is the terminal — the stack does not run per keystroke. The chassis sends one {"type":"attached"} frame the moment the binding is live; after it, binary frames are terminal bytes (client→stdin, process→client) and text frames are a typed control envelope ({"type":"resize"|"signal"|"detach"} in, {"type":"attached"|"exit"|"expired"|"error"} out). The design and the wire protocol are documented under the websocket personality.

An attachment refuses secrets.env.* (an interactive user could echo the value back out), a LOOP, and stream; a second attach on an already-bound session is a bad_request. It is a metered lease, not a long request: it heartbeats every 60 s at the ordinary workspace rate (10 fuel per second, so 600 a heartbeat), charged to the tenant, and a lease that stops heartbeating (a crashed node) is reapable rather than pinning the workspace forever. txco://websocket/send to an attached session is refused with txco_websocket_attached — its frames belong to the process.

`connect` — a workspace-local service on a session

attach binds a process; connect binds a service the workspace is already running — a VNC display, a debug port, a dev server — to the same kind of WebSocket session, and the socket then carries that service’s bytes verbatim in both directions. It is how a person reaches software inside the workspace that speaks its own protocol: a noVNC page onto the workspace’s browser, for one. See examples/workspace-connect for the whole thing, page included.

Like attach, it is valid only inside a WebSocket session run, and the first message the client sends is what connects:

# conn/_websocket/0 — the first message connects, once.
WHEN .msg.type == "connect"
  EXEC "workspace://tools/connect"
    WITH service = "browser",
         max_duration = "1h",
         into = "_connect"

`WITH` key	Meaning
`service`	A name, never a port. Resolved against the chassis’s table: `browser` (the workspace’s VNC display on loopback 5900) plus whatever the operator added with `--workspace-services`. An unknown name is a `bad_request` that lists the ones this node knows
`max_duration`	How long the binding may live, default and capped by `--workspace-connect-max-duration` (8h). When it passes, the connection is closed and the socket told
`into`	Where the grant lands (default `_workspace`)

That the vocabulary is a name is the whole security property: no WITH value — and so no model output that reached one — can address a host or a port, so connect cannot become a tunnel. Adding a service is an operator decision on the node, never a stack’s.

On success:

{
  "_connect": {
    "connected": true,
    "service": "browser",
    "run": "<run id>",
    "workspace": "tools",
    "node": "<node>",
    "lease": {
      "id": "<lease>",
      "expires_at": "…"
    }
  }
}

From then on the connection is the service’s socket. The chassis sends one {"type":"attached"} frame the moment the binding is live — wait for it before sending, because a service (unlike a shell) may not speak first. After it, binary frames are the service’s bytes in both directions; text frames are the same control envelope as attach, except that resize and signal mean nothing to a byte stream and answer one {"type":"error"} frame. The service closing its end sends {"type":"exit","code":0} and closes the socket 1000; {"type":"detach"} closes the connection and returns the socket to per-message runs.

A service that is not running answers workspace.error.code = "unavailable" — the workspace was reached, nothing was listening. Start it first with an ordinary exec (or, on a provider that manages services, as one of the workspace’s services) and connect afterwards. Everything else attach refuses, connect refuses too: secrets.env.*, LOOP, stream, and a second binding on a session that already holds one. It is metered as the same lease, at the same rate.

Loops, fuel, usage, trace

workspace:// may LOOP: its failures are in-band data and its wall-clock is fuel-metered, which is what a loop needs. Each exec pays the flat EXEC dispatch plus 1 fuel per started 100 ms of wall clock (10 a second), minimum 1 — a 4 second exec is 40 fuel, a 15 minute one 9,000, against a default ceiling of 100,000 a request. (Not the nano-op rate of 10 per millisecond, a thousand times more: while a command runs, the provider’s machine does the work and the chassis only waits. Long work is meant to be bounded by what it costs rather than by a cap on how long it may be.) The machine time itself is reported on the usage event with src=workspace (duration, bytes in/out, status), and every dispatch writes a trace step with transport=workspace.

Providers

The chassis runs workspaces through a provider chosen by --workspace-provider. Unset, workspace:// is off and an op that fires fails loudly.

local (bundled) — a directory per workspace under --workspace-local-root (<root>/<tenant>/<stack>/<name>); commands run as the chassis’s own uid. There is no isolation: a rule author on that chassis can run anything the chassis process can. It is therefore refused unless --workspace-allow-local is also set — never implied by --env — and the chassis logs a WARN pair at boot when it is on. For dev and single-operator self-hosting only.

txco dev --allow-local-workspace     # sets the three flags for this run;
                                     # workspaces land in .txco/dev/workspaces/

local, driving another machine. Set TXCO_WORKSPACE_LOCAL_EXEC to a command prefix and the local provider hands every command to that program instead of running it on the chassis’s machine:

txco dev --allow-local-workspace --workspace-local-exec "sprite exec -s dev-{name} --"
txco dev --allow-local-workspace --workspace-local-exec "docker exec -i pony-{name}"

The prefix is whitespace-separated words with no quoting; {tenant}, {stack} and {name} are replaced in each (a / in a name becomes -). A command then runs as <prefix…> /bin/sh -c <script>, where the script is your command with its variables written in front as export lines. It is a way to give a chassis on a laptop a real Linux machine, not a provider of its own, and what it leaves out follows from that:

The workspace is still a directory on the chassis’s machine (its record, its lease). The other machine is yours to make and to remove; Create and the reaper do not touch it.
Variables travel inside the command, because a process environment does not cross to another machine. A run grant’s token is among them, so it is in the prefix program’s argv on the chassis’s machine. The command gets the token and the run’s name and not the grant socket, as on any provider whose commands run elsewhere.
The prefix program runs with the chassis’s own environment (it needs its login and configuration); the command sees none of it unless the program passes it on.
A cwd beneath the workspace is a cd relative to where the other machine starts a command. $TXCO_STACK_DIR, attach, a TTY and connect answer unsupported (the stack directory: stack_dir_unavailable) rather than act on the wrong machine.
A timeout ends the prefix program. What it started on the other machine may run on.

Fleet providers (a machine per workspace, with network policy and checkpoints) register the same way from the overlay and are selected by name; the stack’s txcl does not change.

Flag	Default	Meaning
`--workspace-provider`	(unset)	`local`, or an overlay provider
`--workspace-allow-local`	`false`	Required for `local`; see above
`--workspace-local-root`	`./chassis/data/workspaces`	Root for `local`
`TXCO_WORKSPACE_LOCAL_EXEC` (environment; `txco dev --workspace-local-exec`)	(unset)	Command prefix `local` hands every command to; see above
`--workspace-default-timeout`	`5m`	Per-exec default when `WITH timeout` is absent (sized for builds, tests, tool runs)
`--workspace-max-output-bytes`	`1048576`	stdout/stderr capture cap, each
`--workspace-reap`	`720h`	Idle window before a reaper destroys a workspace; read by a background service a deployment adds (the hosted service’s `workspace-reaper`), not by open core
`--workspace-attach-max-duration`	`8h`	Ceiling on one `attach` binding’s life
`--workspace-connect-max-duration`	`8h`	Ceiling on one `connect` binding’s life
`--workspace-services`	(none)	Extra `connect` targets, `name=port` on the workspace’s loopback, added to the built-in `browser=5900`

Identity, runs, and the reaper

One app, one workspace, whichever inlet. A stack owns its hostname across inlets: a web request runs as <stack>, a WebSocket session as <stack>/_websocket, a mail delivery as <stack>/_mail. All of them address the same workspace, because the identity uses the app stack — the same rule txco://kv follows for its namespace. So a browser request and a socket message reach the same machine and see the same files.

Each (tenant, stack, name) has a row in the workspaces table of the runtime DB: the provider’s reference for it, when it was last used, the current run id, the latest checkpoint. The row makes the identity durable across chassis restarts and shared across fleet nodes; it records, it never locks — concurrent execs on one workspace are allowed, and the author owns file-level conflicts.

A run is one stretch of execution between wake and sleep. On the local provider every exec is its own run. On a fleet provider that idles for a while before it sleeps, execs closer together than that window share a run id; _txc.workspace.run is how a trace tells them apart, and a task that spans a sleep simply sees a new run id on its next exec.

If a workspace disappears, the next exec makes a new one. An operator deletes it at the provider, another node reaps it, the machine is lost: the identity row still points at something that no longer exists. Rather than failing every later command on a dead reference, the chassis drops the identity and provisions a fresh workspace for that exec, and the result carries recreated: true so a rule can tell that the files from before are gone. Only a genuine “not found” heals this way. A provider that fails for any other reason is reported as-is and the workspace is left alone, because recreating on a transient error would quietly replace your files with an empty environment.

A reaper destroys. Open core does not delete idle workspaces; a deployment can add a background service that does. The hosted service runs one, workspace-reaper, which deletes a workspace idle longer than --workspace-reap (30 days by default) at the provider and marks its row destroyed. Its files are gone — no snapshot is kept — and the next exec starts fresh. The same build adds txco workspace for operators to list and remove workspaces by hand. Take a checkpoint first if the files matter. In open core, --workspace-reap is read only by such a service.

Example

examples/workspace-hello — a command, a counter file that persists across requests, a three-pass loop, and a non-zero exit merging as data.