Backup Restore Guide
What We Back Up
| Component | Method | Cadence | Retention |
|---|---|---|---|
| Object store | Snapshot | daily | 5 y |
| Time-series | Snapshot | hourly | 30 d |
| Checkpoint inventory | Per-shard checksum | continuous | 90 d |
| RBAC / policy | YAML + GitOps | per change | 5 y |
Tools
Object Store (S3-compatible)
mc alias set local http://minio:9000 denpex denpex-secret
mc mirror --watch local denpex-backup
Time-series + graph snapshots are stored under s3://<bucket>/tenants/<tenant_id>/<type>/.
Audit Log
Backed up as an append-only file:
cp -a /var/log/denpex/audit /backup/audit-$(date +%Y%m%d)
RBAC / Policy
Stored in GitOps. Each change is a commit; the Git history is the recovery path.
Restore
Restore From Object Store Snapshot
mc mirror denpex-backup/tenants/acme local/tenants/acme \
--overwrite --remove
denpex-control-plane reload --tenant acme
Restore Audit Log
denpex-control-plane audit load --from /backup/audit
denpex-control-plane audit-verify --since forever
Restore CRDs
kubectl apply -f /backup/crds.yaml
kubectl apply -f /backup/cluster.yaml
RPO / RTO
| Tier | RPO | RTO |
|---|---|---|
| Team | 1 h | 4 h |
| Scale | 15 min | 1 h |
| Growth | 5 min | 30 min |
| Air-gap | configurable | configurable |
Validation After Restore
denpex-control-plane selftest --scenario self-test/baseline.json
Air-Gapped Backup
Backups land on a local NFS share or RDMA-mounted BeeGFS volume. They are not replicated to AWS by default in air-gapped mode.