The contract every Worker here shares

Read this before any of the per-Worker files. Most failed calls against this estate break a rule on this page rather than a rule of the Worker being called.

Authentication

Every route is gated by a bearer token except the ones named below:

Authorization: Bearer $CONTROL_TOKEN

Each Worker has its own CONTROL_TOKEN. They are not interchangeable, and ai-ops holds separate secrets (CLUSTER_OPS_TOKEN, LLM_CONTROL_TOKEN) for calling its delegates. Tokens come from the private envs checkout. A missing or wrong token is a 401 with no detail — that is deliberate, not a bug to investigate.

The unauthenticated routes, and why. ai-ops and cluster-ops serve GET / (the UI shell) and GET /docs* without a token. A browser cannot put an Authorization header on a plain navigation, so a page that demanded one could never be opened. Both carry static text only: no secrets, no data. Everything the page displays comes from gated routes the browser calls with a token the operator pastes in.

build-ops and pacer-llm-control gate everything, /docs included. They serve no page of their own, so an unauthenticated route there would be a gate hole with no user.

If an unauthenticated request to a gated route ever returns 200, stop. That means the token gate is not in the request path, which is worse than a failed deploy. scripts/deploy-worker.sh asserts this after every deploy.

The gates on spending and destruction

These exist at more than one layer on purpose. ai-ops enforces its own gate and the delegate enforces its own; that is so ai-ops cannot become a way to route around pacer-llm-control's gate.

Spend gate — start and build refuse without:

{ "spendConfirm": true }

Naming back — terminate, detach and restore refuse unless the request names its own target back:

{ "resource": "<the same resourceId in the path>" }

This is not theatre. It makes a replayed or copy-pasted request aimed at the wrong resource fail closed instead of destroying whichever resource happens to be running. restore replaces the disk holding every session, memory and learned skill an agent has; detach pulls that disk off a running box.

cluster-ops' destructive ops take the equivalent, naming the cluster:

{ "confirm": true, "cluster": "<cluster name>" }

Do not write a wrapper that fills these in automatically. The gate's whole value is that a human or an agent had to state the target a second time.

Reading the error, instead of guessing

These Workers answer with structured errors that tell you the fix. Read the body before retrying.

Status	Means	Do
400	a gate refused, or the body was not JSON	read error and hint; they name the missing field
401	wrong or missing bearer	fix the token; do not retry the same call
404	the thing you named does not exist	read available / families / resources — they list the real ones
405	right route, wrong method	reads are GET, everything that mutates is POST
501	it exists; this verb is not defined for it	not a gap to file — see below
502	the upstream (OCI, Runpod, a delegate) failed	the body carries the upstream's own message

404 and 501 are different claims and the distinction is load-bearing. 501 means the family exists and this verb is not defined for it, which for restore on a Runpod family is a substrate fact, not a missing feature. The 501 body names implementedIn and substrate so you can see why. Verbs differ by substrate, not by taste.

A 404 body is self-describing: it carries the real list. Use it rather than guessing a second spelling.

Runs and logs

Anything long-running returns a runId and a 202, not a result. It is fire-and-forget: the work ships its log to R2 and then terminates itself.

GET /runs?limit=N          recent runs, newest first
GET /runs/:id              that run's status
GET /runs/:id/log?offset=N the log, incrementally

offset is how you follow a log without re-reading it: pass back the offset the previous response returned. The UI's "follow" checkbox does exactly this.

ai-ops owns the ai-runs/ prefix; cluster-ops owns ops-runs/. build-ops deliberately reads cluster-ops' prefix rather than owning a second one, because a provision run is a cluster-ops ops box — owning a separate prefix would mean a run that appears in one Worker's list and not the other's.

Leases, and what they do not prove

A lease is a watchdog budget, not a fact about the substrate. It says "something claimed this resource until time T". The cron sweeper reaps on a lapsed lease.

A lease is not evidence that a thing is running, and its absence is not evidence that nothing is. A box created outside ai-ops has no lease and reads as running: null — unknown, not stopped. That is the tri-state above, and it is why ?substrate=1 exists. An orphan lease with no matching resource descriptor is surfaced rather than filtered out, labelled (orphan — no descriptor), because it is probably still billing.

The trap that governs how probes are written

A deployed Worker's fetch() silently discards a non-standard port on an HTTPS URL. No error, no warning — the request goes to :443.

Any probe or API call against a non-standard port must use connect() from cloudflare:sockets. This matters more than it sounds: the failure mode is a green check against the wrong endpoint. Prefer an assertion that names the fix in its failure message over one that returns a bare status code.

Before you commit a Worker

npm run typecheck
npx wrangler deploy --dry-run --outdir=/tmp/build

Both are cheap and catch the common failures. Neither one deploys anything. "Typechecks and bundles" is a different claim from "deployed and serving", and conflating them has cost this project real debugging time. Say which you did.

Deploying

scripts/deploy-worker.sh ai-ops                    # bundle-only, the normal case
scripts/deploy-worker.sh cluster-ops               # --containers-rollout=none
scripts/deploy-worker.sh cluster-ops --containers  # needs a Docker daemon
scripts/deploy-worker.sh ai-ops --dry-run          # typecheck + bundle only

GitHub Actions does not run for this account — free plan, so hosted runners are never assigned (runner_id: 0, no steps, logs 404). This is #58, closed as won't-fix. There is nothing to debug in the workflow, and "CI is down" does not mean "nothing can ship" — reading it that way left main undeployed for eight days.

The script runs the workflow's own checks: live credential check, typecheck, dry-run, deploy, then two verifications — the account's deployments API must agree with the version wrangler printed, and an unauthenticated request must still be refused.

A container change still needs a Docker daemon, which a sandbox does not have. That is the self-hosted runner box, and per the standing rule every image build runs there and the box is torn down afterwards with its volume intact. A build that leaves the box running is an unfinished build.

Track what you find

Anything discovered — a bug, drift, a follow-up, a missing guard — becomes a GitHub issue in this repo while the context is fresh. Work that lives only in a session transcript is lost when the container is reclaimed.

Keep issue titles and bodies free of secret values: name the key, never the value. A secret that reaches a transcript is published, not merely exposed — say so immediately and treat rotation as required, not optional.

---

Rendered from docs/agents/contract.md in pacer-serverless, bundled into this Worker. Raw Markdown: ?format=md