Backup module
Layer 11. Lifecycle. Lifecycle
SHC provides comprehensive backup and restore capabilities using Restic for deduplication and remote storage.
Overview
Section titled “Overview”- Backs up DATA mounts (persistent volumes)
- Supports all Restic backends (S3, B2, Azure, GCS, SFTP, etc.)
- Optional converters for application-level exports (e.g., pg_dump)
- Restore jobs to import converter outputs
- Mount API for browsing/editing backups without full extraction
- Wrap import for ingesting external files as artifacts
Backup Archive Format (.bak)
Section titled “Backup Archive Format (.bak)”AR archive structure for single-stack backups:
```
{id}.bak (AR archive)
├── manifest.json # Metadata
├── index.json # File tree with byte offsets (enables seekable reads)
└── backup.tar # All backup data (uncompressed for seekability)
├── .shc/
│ └── manifest.json
├── app/
├── deployment/
├── secrets.vlt
└── data/
├──
Seekable Index Format
Section titled “Seekable Index Format”The index.json enables O(1) random access to any file in the archive:
```json { “version”: 1, “compression”: “none”, “entries”: { “/app/main.py”: { “offset”: 1024, “size”: 2048, “mode”: 33188, “mtime”: 1704067200, “type”: “file” } } } ```
When reading files via the mount API, the backend seeks directly to ar_offset + tar_offset + user_offset instead of extracting the entire archive.
Backup Group Format (.bkg)
Section titled “Backup Group Format (.bkg)”AR archive structure for multi-stack backup groups:
``` {name}.bkg (AR archive) ├── manifests.json # GroupManifest with all backup metadata └── backups.tar # Tar archive containing all .bak files ├── stack1_abc12345.bak ├── stack2_abc12345.bak └── stack3_abc12345.bak ```
GroupManifest structure:
version: Format version (currently “1”)consistency_id: Shared ID linking related backupscreated_at: Group creation timestampstacks: List of stack names includedmanifests: Array of individual BackupManifest objects
Use cases:
- Export/Import: Portable multi-stack backup transfer between environments
- Clone/Move: Multi-stack operations bundle backups into .bkg
- Disaster Recovery: Group related stacks for coordinated restoration
Restic Integration
Section titled “Restic Integration”- Auto-init repositories on first use
- Deduplication via Restic chunking
- Incremental backups
- Sync-on-write cache for fast listing
Converters
Section titled “Converters”Transform DATA mounts during backup:
```yaml converters: pgdata: image: postgres:15-alpine command: pg_dump -U postgres -F custom -f /output/db.dump volumes: - pgdata:/var/lib/postgresql/data:ro - “@output:/output” ```
Restore Jobs
Section titled “Restore Jobs”Process converter outputs during restore:
```yaml jobs: restore: - name: import_database image: postgres:15-alpine command: pg_restore -U postgres -d mydb /input/db.dump volumes: - @pgdata:/input:ro - pgdata:/var/lib/postgresql/data ```
Restore vs Recover
Section titled “Restore vs Recover”| Aspect | restore | recover |
|---|---|---|
| Target | Existing stack | Creates new stacks |
| Scope | Single stack | Multiple stacks |
| Use Case | In-place recovery | Disaster recovery |
Consistency Groups
Section titled “Consistency Groups”Multi-stack backups share a consistency_id for coordinated recovery.
Retention
Section titled “Retention”GFS (Grandfather-Father-Son) retention policy:
- Daily, weekly, monthly, yearly retention
- Configured via config files
- Auto-apply after scheduled backups
CLI-to-API Architecture
Section titled “CLI-to-API Architecture”All backup CLI commands communicate with the server via Unix socket API. This provides:
- Single source of truth: business logic lives only in the server
- Consistency: same behavior whether running locally or remotely
- Progress streaming: real-time SSE updates for long-running operations
Command Flow
Section titled “Command Flow”sequenceDiagram participant CLI as CLI Command participant Sock as Socket API participant Srv as Server Service CLI->>Srv: POST /api/method/shc.backup.create_async (async, returns op_id) CLI->>Srv: GET /api/method/shc.operation.stream/{op_id} Srv-->>CLI: event: step {"progress": 25, "message": "..."} Srv-->>CLI: event: step {"progress": 60, "message": "..."} Srv-->>CLI: event: complete {"backup_id": "abc123"}API Endpoints
Section titled “API Endpoints”The async slice of the backup surface (full inventory: generated route reference):
| Endpoint | Method | Description |
|---|---|---|
/api/method/shc.backup.create_async | POST | Start backup with SSE progress |
/api/method/shc.backup.restore_async | POST | Start restore with SSE progress |
/api/method/shc.backup.recover_async | POST | Start recovery with SSE progress |
/api/method/shc.backup.export/{backup_id} | GET | Stream backup file download |
/api/method/shc.backup.import | POST | Upload backup file (multipart) |
/api/resource/Backup | GET | List backups (resource API) |
/api/resource/Backup/{backup_id} | DELETE | Delete a backup (resource API) |
/api/method/shc.operation.stream/{operation_id} | GET | SSE progress stream |
/api/method/shc.operation.state/{operation_id} | GET | Snapshot of current operation state |
Progress Events
Section titled “Progress Events”SSE events emitted during operations:
| Event | Data Fields |
|---|---|
step | step, progress, message |
complete | Operation result data |
error | error_code, error_message |
Mount API
Section titled “Mount API”Browse and optionally edit backup contents without full extraction.
Architecture
Section titled “Architecture”flowchart TB MI["Mount Interface"] RB["Restic Backend"] LB["Local .bak Backend"] HB["Remote HTTP Backend"] COW["Copy-on-Write Layer (for --writable)"] FUSE["Local FUSE Mount"] API["HTTP API Endpoints"] MI --> RB MI --> LB MI --> HB RB --> COW LB --> COW HB --> COW COW --> FUSE COW --> APIThe mount interface is the shc mount <source> <mountpoint> [--writable] command.
Mount Backends
Section titled “Mount Backends”| Backend | Source | Seekable | Use Case |
|---|---|---|---|
| BakBackend | .bak file | Yes (with index) | Local archive browsing |
| BkgBackend | .bkg file | Yes | Multi-stack archive browsing |
| ResticBackend | Restic repo | No | Remote repository browsing |
HTTP API Endpoints
Section titled “HTTP API Endpoints”The mount-session routes (shc.mount.*) are registered by the
backup/mount subpackage; the full inventory lives in the generated
route reference
and per-endpoint notes in the API reference.
Session Lifecycle
Section titled “Session Lifecycle”Sessions use a sliding TTL based on last_access (default 1 hour). Each
request to a session endpoint refreshes the timer, so actively-used FUSE
mounts stay alive indefinitely. A background task sweeps idle sessions every
60 seconds. Explicit DELETE always wins.
Security
Section titled “Security”- Authentication: endpoints require
stack:read,stack:operate,stack:manageorplatform:managevia the sameRequirePermissions(...)middleware as the rest of the API. UDS callers bypass via filesystem perms; HTTP callers sendAuthorization: Bearer <jwt>(orBasic). - Path traversal: every backend rejects paths containing
..segments in_normalize_path(). The overlay additionally verifies the resolved filesystem path stays inside the overlay root viais_relative_to(), defense in depth even if a future refactor breaks the normalization step.
Mounting a backup
Section titled “Mounting a backup”Mount a backup as a read-only FUSE tree with shc backup mount. There is
no separate mount binary and no mount-specific credentials: the mount is
daemon-resident: the daemon supervises a restic mount subprocess on
its own host, so it survives the CLI exiting and is released explicitly
with shc backup umount. The CLI reaches the daemon over its unix socket
(filesystem-gated) or over HTTP with the ordinary bearer/basic flow.
# Mount a backup by id. Omit the mountpoint to let the daemon allocate one# under its state dir (the resolved path is printed in the response).shc backup mount --id <backup-id> /mnt/backup
# Release the mount, by mountpoint path or by mount id.shc backup umount /mnt/backup--writable is accepted for Python-CLI compatibility but ignored: restic
mounts are read-only. The daemon serves the same tree over its HTTP API
(POST /api/method/shc.backup.mount / shc.backup.umount,
router_mounts.go), so a UI can list, stat, and read backup contents
through the daemon rather than a local kernel mount.
Copy-on-Write Overlay
Section titled “Copy-on-Write Overlay”For --writable mounts, changes are staged in an overlay directory:
- Read: Check overlay first, fall through to base
- Write: Copy-up from base then write to overlay
- Delete: Creates tombstone in overlay
- Symlink / chmod: Created or copied-up into overlay; mode changes persist across the session
- Commit: Merges overlay with base → creates new backup
- Discard: Deletes overlay directory
Wrap Import
Section titled “Wrap Import”Import external files/directories as SHC artifacts without running an export.
# Wrap a file as an artifactshc import <deployment> /path/to/realm.json --wrap keycloak/realms
# Wrap a directoryshc import <deployment> /path/to/realms/ --wrap keycloak/realms
# Keep the generated artifactshc import <deployment> /path/to/dump.sql --wrap postgres/dump --keep-artifact ./backup.expWrap Spec Format
Section titled “Wrap Spec Format”--wrap <app>/<import-name>
app: Target application name (must match exporter definition)import-name: Name of the import operation (from app’s exporter schema)
Wrapped Artifact Metadata
Section titled “Wrapped Artifact Metadata”Wrapped artifacts include additional metadata fields:
{ "id": "wrap-abc123", "exporter": "realms", "stack": "keycloak", "timestamp": "2024-01-15T12:00:00Z", "source": "wrapped", "wrapped_from": "/path/to/realm.json", "wrapped_by": "admin", "wrapped_at": "2024-01-15T12:00:00Z"}- Parse
--wrap <app>/<import-name> - Create staging directory with input files
- Generate synthetic
.expartifact withsource: wrapped - Import the artifact normally
- Clean up (unless
--keep-artifact)