DALP chain indexer: event indexing and read-model consistency

The Ledger Index reads EVM logs, decodes DALP events, and writes queryable read models. It explains how checkpoints, finality, reorg handling, and reindexing affect the state shown by APIs and dashboards.

The Ledger Index turns supported EVM events into the read models that DALP APIs, dashboards, and operational screens consume. It runs as a cluster-owned face of the merged DALP backend, with one fenced ingest owner for each enabled chain.

Ownership covers the part of the chain where ownership is required. The block range near the chain tip can still reorganise, so exactly one owner writes it and that owner holds the only coherent view of the tip. Settled history cannot reorganise, so it needs no tip view and no owner: the chain owner divides it into leased ranges, and every backend replica claims and processes those ranges in parallel. Historical catch-up is therefore work for the whole backend pool, while live following stays with a single owner per chain.

During normal operation, two facts can differ: a transaction may already exist on-chain while the platform has not yet processed the events from that block. Use this page to understand how the indexer produces indexed state, how it catches up, and what consistency guarantees you can expect. The page does not document a public API contract or replace your deployment runbooks.

The migration job owns the public application schema. Versioned indexer deployment schemas are reconciled by the Ledger Index through the ordinary transaction-pooled database service. A transaction-scoped advisory lock makes that reconciliation single-flight across backend replicas; waiters reread the committed deployment state instead of repeating DDL. Deployment repositories use immutable schema-qualified Drizzle definitions rather than connection-local search_path state, so no pooled-connection swap can ever redirect indexed writes.

Numeric values cross the Ledger Index through the same boundary discipline as events and identifiers. Chain payloads and database rows are decoded with Effect Schema into the opaque @core/units quantity, rate, duration, and quantization types before a handler performs arithmetic. A token quantity is bound to its chain, address, and decimal witness; an FX or price edge carries an exact rational factor; and elapsed measurements use monotonic Duration values. Handlers may persist canonical decimal strings or exact integer columns, but they do not expose the underlying scalar for intermediate arithmetic. API and chain-facing schemas encode and decode these values again at the boundary, so a malformed, mixed-denomination, or lossy conversion is rejected before it can mutate an indexed read model. This is the invariant that keeps a replay deterministic when a value is larger than JavaScript's safe integer range.

The Ledger Index is released with the Platform API and Workflow Engine as one versioned backend unit; it is not deployed as a separately versioned indexer. Every replica therefore has the same event handlers, workflow coordination, and read implementations. The DAPI package that implements a read also owns its boundary schema, while the server aggregate composes those package-owned groups for transport. There is no separately versioned contract package or generated contract mirror that can drift from the query implementation. Cluster fencing still selects one active ingest owner per chain, while readiness prevents any replica from serving until its index deployment and database expectations agree with the release.

Those faces are compiled together as the same Bun executable for each supported Linux architecture. The Ledger Index is not a sidecar or a runtime-installed plugin: changing its handlers changes the DAPI image digest and rolls the whole backend boundary. On startup, the executable completes index deployment binding before Bun binds the HTTP listener, so an image cannot serve an older read model while its new indexer face is still initializing.

Indexing flow

Rendering diagram...

What the indexer owns

The table below describes each area the indexer controls, how it currently behaves, and what that means for the read state you observe.

Area	Current behaviour	Consistency effect
Chain progress	A live watcher checks the chain head and calls the sync path when the head advances. WebSocket block subscriptions can trigger the same sync path, with polling as the steady fallback.	API and UI reads follow the latest processed checkpoint, not the raw chain head.
Contract discovery	Genesis and event handlers register known DALP contracts in PostgreSQL. An activation fence stays closed while the new contract is registered, its initial coverage is reconciled, and every forward range that started before discovery commits. Backfill and cascade work then advance the drained watermark.	A parent range may advance the reached watermark, but consumers cannot observe DRAINED past its block while discovery can still add work below that height.
Indexing scope	A platform ledger holds one row for each announced system on each chain, with a flag that states whether the Ledger Index may follow that system. The system announcement handler records a system it may not follow, in the same database transaction that would otherwise register it, and writes no index rows for it. The watched contract set, the backfill claim, the stalled-work sweep, and the historical catch-up completion check read that flag on every pass.	Read models hold data only for the systems the platform tracks. The contracts under an untracked system stay registered and keep their pending state, and nothing under them reaches a read model until the system is tracked again. A flag change takes effect on the next range, with no restart and no reindex.
Event ordering	The log fetch path address-filters `eth_getLogs`, chunks block ranges, decodes ABI events, and sorts results by block number and log index before handlers run.	Replaying the same block range produces the same handler order.
Event attribution	Each event row records the transaction sender, emitting contract, and address-type event arguments in `involved[]`, keeping both the raw address and its identity-resolved account (smart wallet or identity to owner account).	An activity query filtered by an address returns rows whose `involved[]` visibly contains that address, whether the caller filters by the raw or the resolved form.
Checkpointing	The sync path stores two per-chain progress axes in PostgreSQL. REACHED records contiguous primary range progress. DRAINED records the highest contiguous block whose primary ranges, discovery activation, backfill, and cascade work have all committed. A replacement chain owner resumes from the persisted axes after downtime.	A range or activation still in flight pins DRAINED at the gap even if REACHED and later ranges have advanced. Read-after-write barriers use DRAINED, never REACHED.
Ownership claim	A chain's owner claims a database owner epoch once when its singleton starts, under the same retry ladder the ingest loop uses, which keeps retrying a transient fault at up to a minute between attempts and gives up only on a failure the database reports as permanent. A loop restart inside one ownership tenure deliberately does not re-claim.	A transient database fault at claim time retries instead of leaving the chain unowned until the next shard movement, and a restart cannot fence out the pod's own in-flight writes.
Network read routing	Genesis discovery, native balance refresh, and normal indexing use the configured network and RPC policy. Control-plane saved RPC upstream pools overlay that baseline so indexing follows the same endpoints operators configured after deployment.	Chain selection and failure behaviour stay consistent, and background reads share the configured limit for each RPC upstream.
Native balance refresh	The collector refreshes chain-scoped native balances from the configured network. Reorg recovery rejects a result that started against stale chain state.	A network read that overlaps a reorganisation cannot overwrite repaired balance state or discard replacement work.
Genesis reconciliation	A background pass re-checks that the contracts a chain declares on-chain are registered in the index, and self-heals stale or incomplete registration. The Forwarder the Directory reports now is the one that wins: a pass activates it and retires the one it replaced together, and it retires the replaced Forwarder at or above the newest block the index has reached. Directory field updates and Account Factory module register, remove, and set-default writes advance monotonic generation watermarks on the Directory. A tip-pinned genesis snapshot whose observation is older than either watermark is refused before the chain read and again at write time, so a pin cannot resurrect a module or Directory instance removed after that height. A deploy height the chain endpoint cannot answer is left to a later pass instead of being recorded unproven. A release that changes these rules raises the indexer version, so existing deployments run genesis again. It starts at a one-second cadence and backs off toward one sweep a minute only while consecutive passes find nothing to do, resetting to the fast cadence the moment a pass registers something.	A newly deployed Directory still converges in seconds, because its contracts are set one transaction at a time and each pass that finds work keeps the fast cadence. A missed Forwarder rotation repairs to the live address without reactivating a stale one, and events the replaced Forwarder emitted while it was still current stay indexed. A tip-pinned module or instance snapshot cannot undo a later removal. A deploy height is never recorded from a lookup the endpoint could not complete, so a later pass against a full-history endpoint can still find the exact block. An idle chain stops paying for the check. Its chain reads share the same admission control as ordinary indexing, so reconciliation cannot crowd out forward sync on a rate-limited endpoint.
Historical catch-up	The chain owner divides settled history into leased ranges below the finalized watermark. Every backend replica claims ranges, processes them through the same decoding and write path the owner uses, and records completion. Each claim carries the current chain owner epoch; a worker that omits or holds a stale epoch receives no lease. A lease that expires because a replica stopped is offered to another. The owner processes only blocks above that boundary until the claims drain.	Catch-up throughput scales with backend replicas rather than being limited to the one replica that owns the chain, so a reindex or a cold start finishes faster on a larger deployment. A range indexed by a claim is identical to one indexed by the owner. A superseded owner cannot keep leasing history after handoff.
Finality tracking	The watcher maintains a finalized-block watermark. PoS chains can use the RPC `finalized` tag. Private or test chains use configured confirmation depth when network configuration allows that signal.	Pruning and reorg cleanup are tied to the finalized watermark rather than only to the latest observed head.
Reorg recovery	The sync path checks stored block hashes for reorganisations, rolls back affected indexed state, and reprocesses the canonical chain. Directory field and Account Factory module generation watermarks that sat above the fork are clamped to the fork height; the Directory row itself stays so replay can advance those watermarks again from canonical history.	Recent indexed data can be corrected when the chain reorganises. Finalized data is the safer point for cleanup. A fork cannot permanently lock genesis out of seeding modules or Directory fields that belong below the fork.
Derived value history	Value movements are recorded as the events arrive, one row per source event, and the history view aggregates those recorded rows by day. Nothing is recomputed from the full price and event history at request time.	A history request costs what the requested date range holds rather than what the deployment has ever recorded, so response time stops growing with platform age.
Reindexing	One cross-replica owner creates and reconciles a new deployment schema while the old schema continues serving reads. When the new deployment catches up across chains, public views swap under the same fenced lifecycle.	Long reindex work does not multiply across backend replicas or remove the currently served read model while the replacement is being built.
Usage aggregation	One Effect Cluster singleton aggregates deployment-wide usage metrics across every system. The owner resolves the current serving deployment on each pass.	Scaling backend replicas does not multiply identical aggregate queries or metric rows, and an ownership handoff continues on another replica.
Wallet classification	One Effect Cluster singleton re-derives operator and bundler wallets for the whole deployment on a slow cadence, reading each organisation's configured wallet against the systems already indexed on the chains this deployment runs. It is not tied to chain ownership, and every write is an idempotent upsert. Each pass is traced, and it logs only when its outcome differs from the pass before it.	A wallet whose system was not yet indexed, for example during a reindex backfill, is classified by a later pass instead of waiting for a process restart, so it reaches the operator wallet view and the native balance queue on its own. Scaling replicas does not multiply the pass, and a deployment in a steady state produces no recurring log volume from it.
Webhook emission	Handlers that index an identity registration, a role change, or a token movement also write the matching webhook event in the same database transaction as the indexed rows. Endpoints are matched by system, so an event reaches only the endpoints registered for the system that produced it. A reorg that removes the source block retracts the event with a `.retracted` successor. Historical catch-up below the finalized watermark indexes its ranges without queueing notifications; forward processing, including restart catch-up and canonical reorg repair, stays notification-eligible.	A delivered event and the indexed row it describes are always consistent: you cannot receive an event for a movement the index does not hold, and you cannot hold the row without the event having been queued for every subscribed endpoint. A subscription registered before the token's system is indexed receives nothing until that system is in the index.

Indexing scope

The Ledger Index does not index every system a chain announces. A platform ledger, stored outside the index deployment schema, states for each chain and system address whether indexing is allowed. The ledger sits outside that schema on purpose: a reindex rebuilds the index deployment, and the choice of which systems to index has to survive it.

The flow is short:

The system factory announces a new system on the chain.
The announcement handler reads the ledger row for that chain and system address, in the same database transaction it would use to register the system.
A tracked system is registered exactly as before. For a system that is not tracked, the handler records the announcement facts a later activation needs and writes no index row.
Every later pass over that chain applies the same flag. A contract under an untracked system is never watched, never claimed for backfill, and never swept into a failed state.
Before each pass over a chain, the indexer applies the pending changes on that chain's ledger. This step is what turns a flag into work.

Stopping indexation needs no work, because the flag alone keeps the contracts out of every pass. Starting it does. A system that was announced while it was untracked has no index entry, so the activation pass rebuilds that entry from the facts the skipped announcement recorded. A system that already has one is sent back through backfill from its own start block, so its full history is indexed and not only the blocks that follow the change. Both cases wake the backfill workers at once, which is why a re-enabled system starts to catch up on a quiet chain instead of waiting for the next event. The pass is safe to repeat: each change is applied once, and a repeated request on a system that is already catching up leaves that work alone.

The invariant is that a system with no ledger row counts as untracked. The gate is closed by default, so a system the platform does not know cannot reach the read models by accident. One protected exception keeps the reindex lifecycle safe: while a chain has no ledger row at all, the historical catch-up completion check keeps its earlier unfiltered behaviour, because reporting catch-up complete against an empty ledger would promote an index deployment whose backfill is still running.

Platform operators own the flag. The operating consequence is that scope is a per-system decision with chain-wide reach: an untracked system's contracts stay registered and keep their pending state, the data indexed before the change stays in place, and a change of the flag applies from the next processed range with no restart and no reindex.

How reads become visible

Indexed state is current to the indexer's DRAINED checkpoint.

A transaction can be mined on-chain before it appears in DALP read surfaces. Visibility can lag while the indexer catches up, drains contract backfill, or builds a reindex deployment.

The indexer exposes two progress axes:

REACHED means the primary range processor has covered the height contiguously.
DRAINED means no range, contract activation, backfill, or cascade at or below the height can still publish another read-model row.

The distinction is a producer contract, not a consumer grace period. The transaction processor does not publish COMPLETED until DRAINED has reached the transaction block. A consumer that receives COMPLETED performs one read. If that read is missing, the producer graph or watermark is broken; retrying the projection, sleeping, or adding a read-back polling loop would hide the defect.

Readiness propagation is event-driven. Each process seeds progress once from persisted watermarks, catches up once after a listener reconnect, and then advances from PostgreSQL notifications. A queue barrier seeds the target once and subscribes to the DRAINED condition under one bounded timeout. It does not periodically reseed or reconcile a supposedly terminal transaction.

When reasoning about state, treat on-chain transaction finality and indexed read visibility as related but separate facts:

The EVM transaction decides whether the on-chain operation succeeded.
The indexer observes supported events and writes the DALP read model.
The activation fence prevents DRAINED from advancing while newly discovered contract history or pre-discovery forward ranges can still publish rows.
The transaction reaches COMPLETED only after DRAINED covers its block.
APIs, dashboards, and review screens perform their indexed read once.
Monitoring shows whether the indexer is caught up, lagging, backfilling, or recovering.

That distinction matters for support and operations. Before transaction completion, check whether REACHED and DRAINED cover the transaction block, whether the contract is registered, and whether activation or backfill work remains. After COMPLETED, a missing row is not ordinary lag: inspect the activation fence, outstanding range claims, cascade work, and progress-listener notifications as an invariant failure.

Recorded history versus recomputed history

Some read models are not a direct copy of an event. A value history is derived: it answers what a system was worth on a given day, which no single event states. The indexer records these derived movements when it processes the event that causes them, rather than reconstructing them from the full price and event history each time a chart is opened.

Each chain's ingest owner folds those recorded movements into the holder and system history read models with a drain whose pace is deployment configuration: indexer.derivedMeasures.intervalMs (default one second, bounded 100 ms to 60 s) sets how often a pass runs and rowCeiling (default 500, bounded 1 to 10,000) how many rows one pass may process. Both are scheduling knobs only: they change how often and how much the drain works, never what it writes, so tuning them trades history freshness against database load without affecting the recorded values.

That choice moves a boundary, and the boundary is worth understanding because it changes what you should expect from the platform.

Property	What it means for you
Cost	A history request is bounded by the date range you asked for. It does not grow as the deployment accumulates more price observations, so a chart that was fast in the first month stays fast in the third year.
Replay safety	Each recorded movement is keyed to the single chain event that produced it, and a repeat of that event is ignored rather than recorded again. Catch-up and retries therefore cannot inflate a total.
Reorg behaviour	Recorded movements are removed with the rest of the affected block range when the chain reorganises, then rewritten as the canonical blocks are reprocessed.
Availability	History exists only for the period the indexer has processed under the current recording behaviour. It is not reconstructed on demand.

The last row is the one that matters during an upgrade. When a release changes how a derived history is recorded, the new records begin empty on an existing deployment and are populated by a reindex. Until that reindex completes, a history read has no stored movements for earlier periods.

Interest converted at a conversion boundary

When a convertible instrument combines conversion with continuous interest accrual, the interest a holder earns can leave in two different ways: paid out as cash, or converted into target tokens alongside the principal. Both are recorded history. The contract emits the amount it settled and the moment it stopped the holder's accrual, and the indexer stores those figures rather than re-deriving them from balances and rates.

That matters for what an issuer sees. Interest routed into a conversion is not still-claimable, and a holder whose accrual has been closed does not keep accruing. An indexer that recomputed those figures from the accrual model alone would show converted interest as outstanding and a closed position as still earning. Because the values are recorded, the read models stay correct across a rate change, an emergency pause, or a partial conversion that leaves the rest of the position accruing.

The same events also record which conversion feature and escrow address an interest feature is wired to. That wiring is what tells the platform which principal has stopped earning; without it the read model cannot distinguish converted principal from principal that is still working.

Both balance-based yield features record that wiring the same way. A fixed treasury yield feature that has been moved to its second implementation emits the same authorization and exclusion events, and the Ledger Index projects them onto the fixed treasury feature row (which feature may settle interest into a conversion) and onto the holder row (whether that holder — a Lock conversion escrow in practice — has been excluded from the yield budget). Fixed treasury yield settles per period from historical balances, so an exclusion applies to the current and later periods and leaves completed periods as they were; the read model carries the current flag and the chain position of the newest change so a late-delivered event cannot flip it back. Rows for features still on the first implementation, or not yet wired, show no authorized consumer, which is what the rollout runbook keys on.

Adopting this behaviour is release-required: the events are new, so the conversion-related figures are correct for newly processed blocks and incomplete for earlier ones until the release's rebuild drains.

Failure and recovery model

The table below shows what the indexer does under each failure condition and what you should check as an operator.

Condition	What happens	Operator check
Chain ingest owner unavailable	Another healthy backend replica can acquire the chain and resume from the stored checkpoint.	Check current ownership, the per-chain processed block, and block lag.
RPC interruption	The watcher keeps its loop alive and retries on later ticks. Finality advancement can pause when the configured finality signal is unavailable.	Check RPC health, latest chain head, finality lag, and indexer errors.
Newly discovered contract	The discovered contract is registered, then queued for backfill and cascade processing.	Check contract registration and whether backfill work remains for that contract.
Chain reorganisation	The indexer rolls back affected rows within the supported reorg window and reprocesses canonical blocks.	Check reorg metrics and whether the finalized watermark has advanced.
Reindex requested	A building deployment schema is created. The serving schema continues to back current reads until the replacement is ready to swap.	Check deployment registry state and pending backfill progress.
Catch-up database failure	The supervised chain job stops and restarts from stored progress. A failed watermark read or update, work-list read, or range write does not become an empty catch-up pass.	Check database availability, chain ownership, indexer persistence errors, and pending backfill.
Usage metrics owner unavailable	Another healthy backend replica acquires the singleton. The next interval reads the current serving deployment rather than retaining a stale schema-bound connection.	Check singleton ownership and the latest usage-metrics timestamp.
Operator or bundler wallet missing from the operator wallet view	The wallet reconciler re-derives the classification on its next pass. A wallet stays absent only while its system is not yet indexed on an active chain, or while the chain head read for that chain is failing.	Confirm the reconciler is still running by its pass span on the trace rather than by its logs, which stay quiet while nothing changes. Then check that the organisation's system is registered in the index for a chain this deployment runs, and check RPC health for that chain.
Database contention between concurrent writers	The database aborts one of the competing transactions to break the lock cycle. The indexer re-attempts the whole block transaction a bounded number of times, so contention does not surface as a handler failure and does not spend a contract's backfill attempts.	Check whether retries are recurring rather than isolated. Sustained contention is a capacity signal, not an indexing defect.
Event batch too large for one write	An event carrying a very large set of derived rows, such as a long schedule, is written in bounded portions instead of one oversized statement. Some derived sets also have a documented upper bound, and a warning names both the scheduled and the indexed count when that bound applies.	Check indexer warnings for truncation notices. A truncated set means the read model deliberately holds fewer rows than the contract declares.

Monitoring and troubleshooting signals

Use Blockchain monitoring for live chain and indexer health. It surfaces the signals you need: block lag, block age, finality lag, sync failures, handler errors, pending backfill, and reindex deployment state. The backend also emits rate-limited structured progress logs after successful index commits, including chain, stage, processed and head blocks, lag, and batch or throughput fields when available. A caught-up transition is logged immediately even when the periodic interval has not elapsed.

Before requesting a reindex, run the indexer troubleshooting checks first. Those checks help you distinguish normal read-model lag from RPC failure, stale contract discovery, reorg recovery, or active reindex work. Running them avoids triggering a reindex when one is not needed.

Observability provides metrics, logs, and traces when the observability chart is enabled. Use it to diagnose why the indexer is lagging.

How many Directory generations a chain serves

Every deployed contract on a chain is reachable through a Directory contract, and a deployment names exactly one Directory address per chain. A chain can hold more than one Directory generation at the same time, for example after a contract redeployment, and the index records each one it was pointed at.

The indexer reports how many generations each chain currently serves. The healthy value is exactly 1. A sustained 2 means a move to a Directory at a new address left the previous generation in place beside it. That is an alertable condition, because lookups that are not scoped to the configured address can then return either generation.

One elected owner in the deployment reads this count from committed database state on a fixed interval, rather than each writer publishing it when a Directory row changes. Directory rows change only at deploy and re-point time, so a value published on write goes missing after a restart and can be overwritten by a slower writer that finished later. Reading on an interval keeps the reported value equal to what the database holds, and a single owner means a pod that stops owning a chain cannot keep reporting an old value.

The count covers one deployment's own database. A second deployment that uses the same chain keeps its own records and does not affect this number.

Two reasons a reindex happens

A reindex is a controlled rebuild of indexed read models. There are two distinct reasons to run one, and confusing them wastes an operator's time.

Reason	Who decides	How you know
Release-required	The release. A change to how events are interpreted or recorded raises the indexer version, and the rebuild is part of adopting the release.	The release notes state it. Affected read models are correct for newly processed blocks and incomplete for earlier ones until the rebuild drains.
Operator-requested	You, after investigation shows the read model itself needs rebuilding.	The checks below have ruled out chain execution, checkpoint lag, contract registration, and RPC health as the explanation.

A release-required reindex needs planning rather than triage: schedule it, and treat the read models it rebuilds as incomplete rather than authoritative until it finishes. An operator-requested reindex needs the opposite discipline, because it is the expensive answer to a question that usually has a cheaper one.

Before requesting a reindex

Reserve an operator-requested reindex for cases where the read model itself needs rebuilding, not as a first response to a missing dashboard row. Run the four checks below first.

Check the live signals prior to requesting one:

Confirm the transaction succeeded on the EVM network and emitted an event that DALP supports.
Confirm the indexer's processed checkpoint has reached that transaction block.
Check whether the contract is registered for indexing and backfill work is complete.
Check RPC health and chain-head freshness when checkpoints stop moving.

The reindex endpoint accepts a chain ID and acknowledges that the request was accepted. A conflict means another reindex is already in progress. Operators should keep watching deployment state, pending backfill, and block lag until the rebuilt schema becomes the served read model.

What the indexer covers

The Ledger Index does not change EVM finality, provide an external proof of reserve, or guarantee an RPC provider's availability. It turns supported chain events into DALP read models. Use those models and related signals to understand freshness, lag, reorg recovery, and reindex progress.

By default, the Ledger Index covers only contract types and events registered through the platform's contract discovery and handler registry. External contracts require separate registration.

For evidence from external bridges, non-EVM networks, custody-provider records, or off-chain reserves, use a separate integration or evidence source.

Ledger Index