pacer cluster-ops docs

← console
indexcontractai-opscluster-opspacer-llm-controlbuild-opsui

pacer-llm-control — the Runpod path

Deployed at https://pacer-llm-control.testtemplate.org.

Its name says "llm"; its job is Runpod — the LLM pod and pacer-media's image and video pods (ADR 0041). There is exactly one Runpod client in this estate and it is here.

Everything is gated, /docs included. This Worker serves no page of its own.

Read contract.md first for auth, gates and error shapes.

Routes

GET  /ready                          can a pod be started at all
GET  /resources                      the three specs, substrate-confirmed
GET  /pods                           live pods
GET  /gpu/availability?resource=<id> capacity, before you spend
POST /resources/:id/up               {"spendConfirm": true}
GET  /resources/:id/status
POST /resources/:id/down
POST /up                             alias for resource `vllm`
GET  /status?id=<workflow-id>        alias for resource `vllm`
POST /down                           alias for resource `vllm`

The bare /up, /status and /down are kept as aliases for vllm because ai-ops' llm.ts and stack.ts call them, and breaking the running stack to add media is not an improvement. Prefer the /resources/:id/* forms in new code.

The spec registry

src/specs.ts holds three pod specs:

idGPUServes
vllmNVIDIA A100-SXM4-80GBGLM-4.5-Air
media-imageNVIDIA L40SvLLM-Omni, text-to-image
media-videoNVIDIA A100-SXM4-80GBvLLM-Omni, LTX-2.3 text/image-to-video

Adding a resource is a registry entry, not a second Runpod client.

Two hard rules about this file, both from ADR 0041:

  1. No provider discriminant. The moment an entry carries provider: "oci" | "runpod", the registry has become the forbidden dispatch table. It is data for one provider's client.
  2. It is never read from outside this Worker. ai-ops does not import it.

Proposing either is a ddd/modeling escalation, not a refactor.

The module asserts at load time that no pod name is a prefix of another's. That is not fussiness: /down deletes by pod name, and a prefix collision would make one resource's teardown delete another's pod.

/down is per resource

ai-ops' llm.ts documented its own /down as coarse and correct "only while the family has one member". A second Runpod resource exists now, so the per-pod delete it named as the fix is what landed.

/down returns deleted, failed, and volumeKept: true. It deletes pods one at a time and reports which ones failed — a batch that half-succeeds silently leaves you paying for pods you believe are gone. A non-empty failed comes back as 502.

The volume is always kept. It carries the model weights and the TLS certificate.

Volumes, and why each resource has its own

A Runpod network volume is data-centre-pinned and mounts read-write to exactly one pod. Two models that must run concurrently therefore cannot share one. So each media resource has its own volume, hostname and certificate — that is a substrate constraint, not a design preference.

The volume attaches only in the data centre it was created in. The LLM volume pacer-llm-weights (tggdlsm22u, 70 GB) is in US-KS-2, and wrangler.jsonc carries the real NETWORK_VOLUME_ID / DATA_CENTER_ID.

Weights live at /workspace/model, the TLS cert at /workspace/certs. Both outlive the pod, so a cycle reuses them — which matters because it avoids spending one of Let's Encrypt's 5 duplicate certificates per host per week.

Certificates and addresses — be exact

The loose version of this claim is false, and it has been written down wrongly more than once:

A certificate makes the origin verifiable.

LiteLLM makes the address steady.

Runpod reassigns a pod's external TCP port on every pod, and a DNS record cannot carry a port. So nothing about a certificate stabilises an address. What a cert buys is a direct route under our own name with verification on. What makes an address steady is LiteLLM — point-litellm.sh / register-litellm.sh rewrite one field.

TLS-from-volume is ON for every Runpod resource, llm included. It is one pattern, not a media exception:

Media only makes the cap bite on every request rather than on a cold one.

Spend, measured

The pinned profile is an A100-SXM4-80GB at $1.59/hr in US-KS-2, billed per second. Cold start has measured 149s–642s, and once 62 minutes.

Ignore the $4.41 H100 figure quoted throughout the older docs: the volume is in US-KS-2 and that data centre does not carry an H100 80GB HBM3 at all.

Check capacity before you spend — GET /gpu/availability?resource=<id>. Note that Runpod's capacity data lives in its GraphQL API (api.runpod.io), not the REST one (rest.runpod.io). Both hosts are forbidden from ai-ops, which is why the forbidden-call rule names them both.

Known limits

Rendered from docs/agents/pacer-llm-control.md in pacer-serverless, bundled into this Worker. Raw Markdown: ?format=md