pacer cluster-ops docs

← console

Agent instructions for the pacer-serverless control surface

Audience: an agent (or a person) about to drive these Workers. One file per part of the surface. Read contract.md first — it holds the rules that are true of every Worker here, and most mistakes are violations of it rather than of any one Worker's API.

These files are served by the Workers themselves at /docs, from these exact bytes — shared/docs/ imports them into the bundle. The page in the UI and the file in the repo cannot drift, because there is only one copy.

PartFileWhat it drives
Cross-cutting contractcontract.mdauth, gates, errors, run logs, the port trap
ai-opsai-ops.mdfamilies, resources, verbs, the stack, leases
cluster-opscluster-ops.mdthe OCI path — cluster ops, the starter box, OCI resources
pacer-llm-controlpacer-llm-control.mdthe Runpod path — pod specs, GPU availability
build-opsbuild-ops.mdthe CI runner box (ADR 0038)
The browser UIui.mdthe estate dashboard, the two older consoles, and what they will not do for you

Driving this by hand? The estate dashboard at https://ai-ops.testtemplate.org/dashboard covers every resource below in one page with one token — the cluster and its 13 ops, all 6 AI resources, the stack, the starter box, the CI runner box, and both run shelves. See ui.md.

The one-paragraph version

There are four Workers. ai-ops is the front door: it owns leases, run logs, the watchdog, the UI and the family registry, and holds no cloud-provider client of its own. Every family it exposes delegates to the Worker that owns its substrate — cluster-ops for OCI, pacer-llm-control for Runpod. If you are starting or stopping something, you almost certainly want ai-ops. Reach for a substrate Worker directly only when you need something ai-ops does not expose, and know that doing so bypasses the lease and the run log.

Pick your entry point

start / stop / inspect an AI resource   ->  ai-ops      /families/...
"what is running right now, really"     ->  ai-ops      /resources?substrate=1
bring the whole stack up or down        ->  ai-ops      /stacks/pacer/up|down
cluster rebuild, teardown, diagnose     ->  cluster-ops /ops/<op>
a dev box (starter)                     ->  cluster-ops /starter
raw OCI instance/volume for a resource  ->  cluster-ops /oci/resources/...
GPU capacity before you spend           ->  ai-ops      /gpu/availability
build a Docker image                    ->  build-ops   (not deployed — see its file)

Three things that are true and are usually got wrong

  1. running is tri-state, not a boolean. true / false / null, where null means nobody asked the substrate. Treating null as false is how a live, billing box gets reported as "nothing is running" and a destroy request is reported as a no-op. Pass ?substrate=1 when the answer has to be right, and read the confidence and why fields that come with it.
  1. A green typecheck is not a deployment. npm run typecheck and wrangler deploy --dry-run prove the bundle builds. They prove nothing about what is serving. Say which one you did.
  1. GitHub Actions does not run for this account. Workers ship with scripts/deploy-worker.sh, not through CI. That is the normal path here, not a workaround — see contract.md § Deploying.

Where the authority lives

This directory explains how to use the surface. It does not decide architecture. Layout, port wire-shapes, and which Worker may call which provider are settled in the repo README.md, CLAUDE.md, and the ADRs in pacer-docs. Where this directory and one of those disagree, those win and this file is the bug — say so rather than coding to it.


Rendered from docs/agents/README.md in pacer-serverless, bundled into this Worker. Raw Markdown: ?format=md