Runbooks
Diagnose and fix the readiness, indexing, and delivery problems platform operators run into most on a DALP instance.
Every runbook in this family opens from a symptom you can observe directly: kubectl output, a Console screen, or a delivery log, and ends with what actually fixes it. None of the three asks you to diagnose the cause before you start; each narrows to it through checks that name a signal and the value that means healthy. Start from what you are seeing, not from a guess.
A DALP instanceA DALP Instance is one self-contained copy of the platform, deployed inside a boundary you choose and connected to ledger networks you approve.Read the concept runs three backend roles: one serves Console and API traffic, one indexes chains into Ledger DataLedger Data lets an integration answer a holder list, a transfer history, or a compliance check with one query, instead of decoding chain events itself.Read the concept, and one executes durable workflows, including webhook delivery. The indexer writes each webhook event as part of the same transaction that advances its indexed state, and the runner reads that queue on its own cadence to deliver it, so a stalled indexer starves webhook deliveries too. All three answer the same /readyz endpoint with the same status words, and all three write into one PostgreSQL database, so a symptom on one surface sometimes traces back to another. Each runbook names the moment its own checks point somewhere else, and links straight there.
The indexer publishes a servable schema that the Platform API's readiness waits on, which is why a readiness symptom can turn out to be an indexing fault. The runner, the Workflow Engine role, owns durable execution and delivers webhooks from a queue the indexer writes in the same transaction as its indexed state, so a stalled indexer starves deliveries too. All three write down into one PostgreSQL database, so a database problem can surface as more than one symptom at once.
Match the symptom to a runbook
| What you observe | Open | Why this one |
|---|---|---|
kubectl get pods shows a pod Running but not 1/1 Ready, or Console and API calls fail with connection errors or 503s while every pod shows Running. | Platform API not ready | The pod is out of rotation; check the readiness of the other replicas. |
Console, the API, or reports stop reflecting new on-chain activity while the pods themselves report Ready. | Ledger indexing stalled | The pod is healthy; only the chain data behind it has stopped advancing. |
| A registered webhook endpoint stops receiving events, alone or while sibling endpoints on the same instance keep working, or its status in Console flips to Disabled on its own. | Webhook deliveries failing | Start with that endpoint's delivery history and breaker state. |
| Every registered webhook endpoint on the instance goes quiet at the same time, not just one. | Ledger indexing stalled | The indexer writes each webhook event in the same transaction that advances its indexed state, so a stalled indexer can stop new chain events across endpoints. Also inspect the shared delivery runner. |
An indexer pod answers /healthz and /readyz with 200 while its process runs. A non-reorg chain ingest failure logs one Chain <verdict>; exiting: <failure> error in category didx.host, with chain_id and outcome annotations, then exits with status 1. Kubernetes restarts the pod. Operators read pod restarts and that log line. On a Platform API pod, readiness still reports whether its servable schema is bound, and its Actions point to diagnosing the indexer pod.
Open a runbook
When nothing here matches
If the pod is Ready, Ledger Data is current, and webhook deliveries are flowing, but you are still looking at a failure none of these three name, capture the evidence before you touch anything. Take the pod's logs and kubectl describe pod output, the delivery history, or whatever error your own integration returned. Every runbook in this family says the same thing about its own domain: a restart or a resend without evidence trades a diagnosis for a guess, and the wedge or the failure reason you needed does not survive it. Escalate with what you captured; each runbook's own closing step names the specific fields the platform team needs, and the same principle holds even when none of the three fits.
Backup and disaster recovery
What a DALP instance backs up inside the cluster, what your managed dependencies must back up themselves, and how to restore or plan a failover.
Platform API not ready
A Platform API, indexer, or Workflow Engine pod runs but is not Ready: how to read the pod's own diagnosis, which waits are normal, and when a restart helps.