Upgrades and migrations

A dalp Helm upgrade updates the platform release and runs its database migration Job. With the default pre-upgrade hook, the Job must succeed before Helm updates application Deployments. Database changes are separate from the application rollout and do not reverse with helm rollback. Check migration compatibility, recovery points, and any required index rebuild before the upgrade.

What runs, and whenCopy Anchor Link

The dapp subchart renders a Kubernetes Job named <release>-migrations whenever migrator.enabled is true, which is the shipped default; set migrator.enabled: false and no migration runs at all, which means your application pods start against whatever schema the database already has. That escape hatch exists for a pipeline that runs migrations through its own tooling before calling helm upgrade; on every other path, disabling it removes the one check that stands between a schema change and a pod that expects it.

migrator.hookType decides when the Job runs, and the choice is not cosmetic. The default, pre, runs it as a pre-install,pre-upgrade hook at hook weight -5, so it completes before the application workload manifests are updated; choose this whenever Postgres is external or already running, which is the common case for a production instance. Setting hookType: post instead runs the Job as post-install,post-upgrade at weight 0, after the release's other resources; choose this only when the same release also stands up the Postgres the migration targets, so the database has to exist first. That ordering has a cost: under post, Helm applies and waits on the release's workload manifests before the Job runs, so new pods can start before the migration finishes. Readiness checks may hold them out of traffic, and a wait for readiness can prevent a post hook from running if those checks require the new schema. The hard gate the next section describes holds only for the pre hook. A third value, none, renders the Job with no hook annotations at all: it becomes an ordinary resource in the release manifest rather than a Helm-managed hook. Helm's before-hook-creation cleanup and the pre/post gate described below no longer apply to it, so running migrations on that path is a pipeline you own outside Helm's default upgrade flow.

Before running its migration command, the Job's init containers wait for the Postgres endpoint to accept connections and, when you enable Conjur, fetch a Conjur access token. That wait draws from the same clock the migration itself runs on: activeDeadlineSeconds bounds the Job from the moment it starts, init containers included. The Postgres wait's own timeout defaults to 120 seconds, so a slow database can burn a fifth of the default budget before the migration command runs at all. The migration container itself runs with a default backoffLimit of 3 and an activeDeadlineSeconds of 600 (10 minutes) total.

Its database connection is not the one the application containers use: a dedicated migration Secret gives it a direct, non-pooled DATABASE_URL, bypassing the connection pooler even when pooling is enabled for the application. A transaction-mode pooler would break the session-scoped advisory lock the migration takes. Everything else about how it reads configuration matches the application containers: a mounted config.yml, an envFrom secret, and, when you configure Conjur secrets for the migrator or inherit them from the runtime dapp configuration, a summon-wrapped entrypoint. migrator.env can also carry TXSIGNER_DATABASE_URL; set it and this same Job also runs the wallet-migration script alongside any pending schema migration. It can carry the encryption-rotation keys too (ENCRYPTION_MASTER_PASSWORD, ENCRYPTION_PREVIOUS_MASTER_PASSWORD) for a local, non-Conjur rotation run. Values and secrets covers the two-step rotation procedure and when the old key can be removed safely.

With the default pre hook type, the migration Job is a hard gate: your running Deployments are not touched until the Job succeeds. The previous pods remain in place, but migration locks, data changes, or incompatible schema changes can still affect their requests. A pre-upgrade hook does not guarantee uninterrupted service. A separate obligation, an index rebuild, is not a gate in the same way. Some releases require one; check the release notes for your target version. A rebuild runs behind an already-healthy, already-serving deployment, not before it, and the read models it populates are incomplete for earlier periods until it finishes draining. Treat affected historical figures as provisional during the rebuild. Open Indexing detail from Platform status and wait for the affected chain to return from Reindexing to Healthy. Helm release completion does not prove that the rebuild finished.

What a failed migration leaves behindCopy Anchor Link

The Job's hook delete policy defaults to before-hook-creation,hook-succeeded, deliberately without hook-failed. A hook-failed policy would make Helm delete the failed Job and its pod immediately, which removes the pod-local evidence before you inspect it; centralized logs may still retain the error. With the default policy, Helm itself leaves a failed migration Job and its pod in the cluster for you to inspect (kubectl logs job/<release>-migrations, kubectl describe job/<release>-migrations) until the next upgrade attempt, at which point before-hook-creation deletes the previous Job to make room for the new one.

Helm's hook-delete policy is not the only thing that can remove that Job. The Job template also sets a Kubernetes-native ttlSecondsAfterFinished, defaulting to 86400 seconds (24 hours), which the cluster's own TTL controller uses to delete the Job, on success or on failure, independently of anything Helm decides. That clock can still be shorter than "until the next upgrade attempt" if your process upgrades less than once a day: by default, you have 24 hours after a migration Job finishes to read its logs before the cluster removes it on its own. If your operating process needs longer than that to notice a failed upgrade and pull logs, raise migrator.ttlSecondsAfterFinished before you rely on the failed Job still being there. Do not assume the two settings move together: raising migrator.hookDeletePolicy retention without also raising ttlSecondsAfterFinished leaves the shorter Kubernetes-native window as the real limit.

Before you upgradeCopy Anchor Link

• Read the release notes for the version you are upgrading to. A migration Job runs unconditionally on every helm upgrade, whether or not that release changed the schema, so check specifically for a required index rebuild, which needs its own place in the change window.
• Confirm the Postgres endpoint the new release will use is reachable and has enough headroom for the migration to complete inside the 10 minute default deadline; raise migrator.timeout for a migration you expect to run longer. The migration connects direct to Postgres rather than through the pooler, so that headroom comes from your server's own max_connections, not from the pooler's fixed pools. Values and secrets covers how to compare those pools against your server's ceiling.
• If your cluster enforces NetworkPolicy, confirm the migration Job's egress rule permits the Postgres port and any external Postgres host you use. That policy ships at an earlier hook weight than the Job specifically so it is in place before the Job needs it. Get it wrong and it can deadlock the very upgrade meant to fix it: the pre-upgrade Job can no longer reach Postgres, the hook fails, and the manifests carrying the fix never apply.
• Confirm migrator.ttlSecondsAfterFinished gives your on-call process enough time to pull logs from a failed Job before the cluster deletes it; 86400 seconds (24 hours) is the default, not a guarantee that matches your response time.
• A migration Job failure blocks the rest of a pre hook upgrade: the release does not proceed to update your running Deployments until the Job succeeds or you intervene. That block runs on a different clock than the --timeout you pass to helm upgrade itself; see Helm charts for how to size the release-wide wait.

After you upgradeCopy Anchor Link

All charts in a release share one version: Helm charts covers why mixing subchart versions inside one release is not supported. After the upgrade, confirm the deployed state actually matches what you intended, rather than trusting that a completed release proves it:

• helm list reports the release's current chart version; confirm it matches the version you upgraded to.
• helm history <release> shows which revision landed, so you can compare it against the one you expected.
• Confirm the migration Job itself succeeded rather than assuming it did because the release reported success.
• If the release requires an index rebuild, open Data freshness on the Console's Platform status screen, then open Indexing detail. Confirm that the affected chain has moved from Reindexing to Healthy. Release completion does not prove rebuild completion; see What runs, and when.

Rolling back after a bad upgradeCopy Anchor Link

A Helm rollback (helm rollback <release> <revision>) returns your Deployments, Services, and other templated resources to a prior release's rendered manifests. It does not run a migration Job to reverse a schema change, and it does not restore database rows a completed migration already altered. Nothing in the chart defines a rollback hook, so a helm rollback triggers no Job at all, forward or reverse. Rolling back the release moves your running pods back to code that expects the older schema. If the failed or completed migration already changed the schema in a way the older application code cannot read, a Helm rollback alone reintroduces a version mismatch between your code and your data, rather than resolving one.

The same holds for an index rebuild. It runs as an indexer process behind the deployment, not as a Helm hook, so a rollback has no lever over it. Rolling back does not pause or reverse a rebuild already in progress, and it does not restore the projections a rebuild has already rewritten.

Retry safety depends on the failed migration and the state it left behind. Treat a failed migration, and any rollback decision that follows it, as a stop-and-inspect event: read the Job's logs, confirm what state the schema is actually in, and involve whoever owns your database change process before retrying or rolling back. Do not retry a failed migration Job on the assumption that a backoffLimit retry is always safe to repeat.

Frequently asked questionsCopy Anchor Link