pacer cluster-ops docs

← console
indexcontractai-opscluster-opspacer-llm-controlbuild-opsui

build-ops — the CI runner box

DEPLOYED and serving at https://build-ops.testtemplate.org.

Be careful with older notes about this Worker: README.md, CLAUDE.md and pacer-agent-system-doctrine.md § 7 have all said "not deployed; has never served a request", and this file said it too until 2026-09-17. It is wrong. The Worker answers on its custom domain and enforces its own token gate — recorded as #50 on 2026-09-10 and confirmed again on 2026-09-17. When it first went live, and by what route, is still not established.

What is verified, and what is not. status, backups and terminate were exercised live on 2026-09-17, after #80 corrected how this Worker addresses the box. provision and start have not been run — they spend money, and nothing has needed a runner box since. So this Worker reports correctly; whether it can successfully build a box end to end is untested.

cluster-ops' runner op remains the path ADR 0038's own notes point at, and is the one that has actually provisioned boxes — see cluster-ops.md § The runner op.

ADR 0038 gives this Worker the lease, run log, watchdog and UI for the ephemeral OCI instance that runs Docker builds outside the cluster and hosts the self-hosted runners. It holds no cloud-provider client: it delegates its OCI calls to cluster-ops, exactly as ai-ops does.

Everything is gated, /docs included — it serves no page of its own yet, so an unauthenticated route added "for the UI" before then would be a gate hole with no user.

Routes

GET  /version
GET  /status?arch=x86|arm
GET  /backups?arch=x86|arm
POST /provision    {"arch": "...", "mode": "...", "confirm": true}
POST /start        {"spendConfirm": true, "arch": "..."}
POST /pause        {"arch": "..."}
POST /terminate    {"resource": "runner", "arch": "..."}
POST /backup       {"arch": "..."}
POST /restore      {"resource": "runner", "backupId": "...", "arch": "..."}
GET  /runs?limit=N
GET  /runs/:id
GET  /runs/:id/log?offset=N

arch is on every lifecycle route, and it is not optional detail. The two boxes have separate instances, separate volumes and separate backup lists, so a call that omits it acts on x86 and never touches the ARM box. x86 or arm; anything else is a 400 naming the valid set.

This Worker serves no page. GET / is a 404 — the UI ADR 0038 gives it was never built, and the estate dashboard at https://ai-ops.testtemplate.org/dashboard is this box's screen.

Two names for one box, and why both are right

This Worker calls the box build/runner — its own namespace, where its lease and run log live. cluster-ops knows the same box as runner/oci (or runner/oci-arm), because that is the key its name overrides carry:

runner/oci      ->  instance pacer-runner-oci      volume pacer-runner-oci-data
runner/oci-arm  ->  instance pacer-runner-oci-arm  volume pacer-runner-oci-arm-data

/status reports the translation as ociResource, so a future mismatch is visible in the response rather than only in a volume name nobody recognises.

This was broken until 2026-09-17 (#80). Every volume call went to /oci/resources/build/runner, which derives build-runner-data — a name nothing has ever created — so /status?arch=arm reported the volume absent while pacer-runner-oci-arm-data existed with 50 GB of real state. Had provision run, it would have created a second empty volume while the TLS certificate and runner registration sat on the first.

Two name-backs, and they are different strings. You name the box back as runner; this Worker translates that to oci / oci-arm for cluster-ops, whose gate compares the name-back against its own path segment. Forwarding your word verbatim would fail that gate and read as your typo.

It reads cluster-ops' run prefix, deliberately

/runs* here reads the ops-runs/ prefix that cluster-ops owns, not a prefix of its own. That is correct rather than sloppy: a provision run is literally a cluster-ops ops box, launched by cluster-ops and reporting where every other ops box reports. Owning a second prefix would produce a run that appears in one Worker's list and not the other's.

The standing rule this Worker exists to serve

Anything that builds or pushes a Docker image runs on the self-hosted OCI runner box, and the box is torn down afterwards with its volume intact.

build / push an image   ->  the self-hosted runner box      DEFAULT
GitHub-hosted runners   ->  an option, chosen deliberately   NOT the default

This is a default, not a prohibition. ubuntu-latest remains available and is sometimes right — but choosing it is a decision to state in the PR or the log, not something to fall into because a workflow already said runs-on: ubuntu-latest. A runs-on: line already sitting in a workflow is not a decision anyone made recently.

Provision → build → push → destroy, volume intact. A build that leaves the box running is an unfinished build. The volume is never deleted: it carries the TLS certificate and the runner registration, which make the next provision cheap.

What is still unsettled

Stated here so a session does not discover them mid-build and assume it broke something:

Two related corrections worth carrying, because both were wrong in the obvious direction before being measured:

pacer-runner-oci was terminated 2026-09-11 at the owner's instruction to stop billing. The next build provisions a fresh one — that is the intended lifecycle, not an error state.

Rendered from docs/agents/build-ops.md in pacer-serverless, bundled into this Worker. Raw Markdown: ?format=md