Running AccelerUp/Operations/Backups and recovery

Backups and recovery

What must be backed up, what can be rebuilt, and how to restore.

What matters

Data	Backed up	Why
Postgres	Yes — everything depends on it	All state: users, services, alerts, incidents, audit, delivery records, metric history
Redis	No	Cache only. Rebuilds itself
Uploaded files	Yes	Source maps, attachments
Configuration	In the secret store	Environment variables, not on disk

There is one thing to back up: Postgres.

What can be rebuilt

Metric history that came from an external source — Cloudflare, ELK, the warehouse — can be re-fetched, but only within the source's own retention.

Warning

Some of it cannot. Cloudflare's load-balancing dataset is kept about three days on this plan, which is exactly why AccelerUp captures the region→pool split into its own table. Lose that table and the history past three days is gone for good — no re-fetch can bring it back. See Load balancing.

Taking a backup

scripts/backup-automation.sh runs the scheduled backup. Manually:

bash

pg_dump --format=custom --no-owner --no-acl \
        --dbname="$DATABASE_URL" \
        --file="accelerup-$(date -u +%Y%m%dT%H%M%SZ).dump"

Custom format so pg_restore can do partial restores and parallel loads.

Restoring

bash

# 1. Stop the application so nothing writes during the restore.
docker service scale accelerup_backend=0

# 2. Restore.
pg_restore --clean --if-exists --no-owner --no-acl \
           --dbname="$DATABASE_URL" --jobs=4 \
           accelerup-20260915T081402Z.dump

# 3. Start. Migrations run and bring the schema to the current version.
docker service scale accelerup_backend=2

Important

Starting the application against a newer dump than the deployed code is not supported. Restore, then deploy the version that produced the dump, then upgrade forward.

Test the restore

A backup nobody has restored is a hypothesis.

Restore into a scratch database quarterly and check:

row counts on alerts, services, notification_delivery_log
that the application starts against it and migrations report no pending work
that a dashboard renders

Retention of the backups themselves

Keep at least one backup older than your longest detection window. Data corruption that goes unnoticed for a week is not recoverable from a six-day retention.

What a restore loses

Everything between the dump and the failure. For a nightly dump that is up to 24 hours of alert history, delivery records and audit entries.

If that window is too wide, enable continuous archiving (WAL) — the schema and the application do not need to change for it.

Where this behaviour lives: scripts/backup-automation.sh. If the code and this page disagree, the code is right — please fix the page.

Part of Running AccelerUp — Operating the platform itself: architecture, deploys, backups.