Back up and restore your app
This page backs an app up and restores it. A backup copies a deployment’s
volumes into the cluster’s backup repository.
backup.yaml is the app’s say in it: which volumes matter, which never go in,
and the jobs that turn a volume into what is kept and back again.
The cluster has a key for its repository since shc init but no repository
yet. Make it a directory on home01’s disk. It is the cluster’s repository, so
the set names --scope global: with prod selected, a set without it writes
a row for prod alone.
$ shc config set backup.repo /var/backups/shc/repo --scope global...Typed alone, shc backup lists its verbs, and the next two pages use most of
them.
$ shc backup...Give notebook its backup.yaml on home01.
include: - notesconverters: notes: image: busybox:1.36 entrypoint: ["sh", "-c"] command: | set -e test -s /data/log cp /data/log /output/log echo "notes: $(wc -l < /output/log) lines" volumes: - notes:/data - "@output:/output"jobs: restore: - name: place-notes image: busybox:1.36 entrypoint: ["sh", "-c"] command: | set -e if [ "$(stat -c %d:%i /backup)" = "$(stat -c %d:%i /data)" ] || [ ! -s /backup/log ]; then echo "place-notes: no converter copy staged" exit 0 fi cp /backup/log /data/log chown 65534:65534 /data/log echo "place-notes: $(wc -l < /data/log) lines back" volumes: - "@notes:/backup" - notes:/datahook_artifacts: []recreate_on_restore: truererender_on_restore: falseWhat each part says:
includelists what a backup copies, and a move carries:notes.htmlis nginx’s two pages, made again from the image, andentrypointstarts empty, so neither is listed. They’re scratch: never backed up, and recreated empty wherever the replica lands.- A converter is keyed by the volume it claims, and that volume has to be
under
include, because a backup that doesn’t run the converter copies the volume itself. At backup time it runs with the volume mounted and writes what is kept to@output; the archive holds that instead of the raw volume.test -sfails it on an empty log, and a failed converter leaves the raw bytes in the archive instead. jobs.restoreruns at restore time.@notesis the converter’s copy as the archive brought it back. When no copy was staged,@notesis the live volume itself, which is why the job compares the two directories before it writes.hook_artifactslists files apre_backuphook writes into a volume for the backup to carry. notebook’s hooks write none.recreate_on_restorebrings the deployment back with new containers rather than restarting the old ones.rerender_on_restorere-renders the deployment from the restored vault before it starts and beforepost_restore. notebook leaves it false. Set it when the restored data already holds the source deployment’s credentials.
shc update reads the directory again.
$ shc update notebook...Back it up and check what the archive holds.
$ shc backup create notebook --name first...$ shc backup verify first -s notebook...$ shc backup verify first -s notebook --deep...$ shc backup ls -s notebook --status completed --limit 5 --offset 0...verify reads the archive’s manifest; --deep streams every byte of it back
out of the repository. The list keeps what the last verify found: its
VERIFIED column reads deep for first, and stays out of the table while
no listed backup was verified. A backup can also be browsed: mount makes it a
read-only tree on home01 until umount, in a directory that already exists.
$ mkdir -p ~/learn/snapshot$ shc backup mount ~/learn/snapshot --id first -s notebook...$ ls -1R ~/learn/snapshot...$ shc backup umount ~/learn/snapshot...Empty the log and restore it. A backup’s name works wherever its id does;
-s names the deployment it goes back into.
$ shc exec -s notebook -S notes -- sh -c ': > /srv/notes/log'$ shc backup restore first -s notebook --yes...$ shc exec -s notebook -S notes -- wc -l /srv/notes/log...A restore first takes a safety backup of what it is about to replace, and it stops the deployment for the replay. Its flags change that, narrow what comes back and say how files already there are treated. Take a second backup to restore from, then restore it three ways.
$ shc backup create notebook --name second...$ shc backup restore second -s notebook --paths @notes --strategy merge --keep-running --no-rollback --yes...$ shc backup restore second -s notebook --no-jobs --force --no-rollback --yes...$ shc backup restore --before 0m -s notebook --no-rollback --yes...--paths @notes brings back the converter’s copy alone, --strategy merge keeps files
the archive does not hold, and --keep-running replays without stopping
notebook. Under --paths the deployment’s secrets, port and host rows and
config overrides stay as they are; name vault_secrets, port_mappings,
host_mappings or deployment_config to bring one back with the volume. --no-rollback skips the safety backup. --no-jobs skips
place-notes, so the log comes back as the raw bytes the archive kept beside
the converter’s copy, if it kept any, and --force restores over the running
deployment. --before 0m takes the newest backup, second.
A restore brings data back. shc update --rollback is the way back for the
spec: it puts the deployment on the last spec that finished its rollout and
differs from the one it runs, never a failed attempt, or on the kept spec
--rollback=<digest> names, and leaves the volumes as they are. Its plan
names both specs, says when the one it goes back to last finished, and says
the data is not rolled back.
$ shc update notebook --rollback --plan$ shc update notebook --rollback=9f8e7d6c --yesThe backup side has its own flags: what to take and how long to wait for the
freeze. --fail gives up rather than stop the deployment when the freeze
takes too long.
$ shc backup create notebook --name tuned --exclude '*.tmp' --timeout 30 --fail...--paths takes some of the volumes only, named as restore --paths names
them.
$ shc backup create notebook --name notes-only --paths notes...A connected deployment is backed up with what it connects to:
--all-connected takes its whole integration graph and --with-dependants
adds every deployment that depends on it, each set under one consistency id.
notebook connects to nothing, so each set is notebook alone.
$ shc backup create notebook --all-connected --name connected...$ shc backup create notebook --with-dependants --name dependants...A restore by --id widens the same way: --with-connected restores every
backup of the target’s consistency group and --with-dependants the target
with its dependants’. Each member goes back into the deployment it was taken
from, so a group restore takes no -s.
$ shc backup restore --id connected --with-connected --no-rollback --yes...$ shc backup restore --id dependants --with-dependants --no-rollback --yes...A backup to hand over
Section titled “A backup to hand over”--file writes a one-off .bak to a path you name instead. It is not in the
repository, not encrypted and never pruned; it is yours. --secrets-password
seals the deployment’s secrets in it under a password, so a cluster whose
vault password differs can open it.
$ mkdir -p ~/learn/handover$ shc backup create notebook --file ~/learn/handover/notebook.bak --secrets-password P@ssw0rd...$ shc backup inspect ~/learn/handover/notebook.bak...$ shc backup restore --file ~/learn/handover/notebook.bak -s notebook --secrets-password P@ssw0rd --secrets-password-file ~/learn/vault.pw --yes...--secrets-password and --secrets-password-file are candidates, tried after
the cluster’s own vault password, and either may be given more than once.
import makes a .bak an ordinary backup in the repository; importing it
twice leaves one. export writes a backup from the repository out as a
.bak. --from-store sweeps a whole directory of them, and without
--apply it only reports.
$ shc backup import ~/learn/handover/notebook.bak...$ shc backup import ~/learn/handover/notebook.bak...$ shc backup export -s notebook --latest --file ~/learn/handover/latest.bak...$ shc backup import --from-store ~/learn/handover...$ shc backup import --from-store ~/learn/handover --apply...group bundles several .bak into one .bkg, decrypted: every member’s
data in the clear in one file, readable only by root.
$ shc backup group -b ~/learn/handover/notebook.bak -b ~/learn/handover/latest.bak --out ~/learn/handover/pair.bkg...--backup takes repository backups by name or id instead of files, and --dir names
the directory the .bkg is written to under a name of its own.
$ shc backup ls -s notebook --limit 1...$ shc backup group --backup first,second --dir ~/learn/handover...inspect reads a group as it reads a .bak: the group, then one row per
member.
$ shc backup inspect ~/learn/handover/pair.bkg...A new deployment can start from a backup. Install the app under a new name
and restore the archive into it: -s names the deployment the data goes
into, which need not be the one it was taken from.
$ shc install ~/learn/notebook -s notecopy...$ shc backup restore --file ~/learn/handover/notebook.bak -s notecopy --secrets-password P@ssw0rd --no-rollback --yes...$ shc exec -s notecopy -S notes -- wc -l /srv/notes/log...$ shc uninstall notecopy --no-backup --yes...uninstall takes a safety backup first; now that a repository is set,
--no-backup is right only for a copy like this one.
install --from-backup does both in one command, with the same
--secrets-password flags. It waits for the replay as backup restore
does and hands the deployment over once the data is in.
$ shc install ~/learn/notebook -s notecopy --from-backup ~/learn/handover/notebook.bak --secrets-password P@ssw0rd --secrets-password-file ~/learn/vault.pw...$ shc exec -s notecopy -S notes -- wc -l /srv/notes/log...$ shc uninstall notecopy --no-backup --yes...Declarations this app does not need
Section titled “Declarations this app does not need”A deployment backed up while it is stopped is captured as it lies on disk, a crash image. An app whose data cannot come back from one says so, and SHC then refuses that backup and never moves the deployment off a failed node. keyholder has nothing to lose, so let it refuse to show the refusal.
crash_restore: false$ shc update keyholder...$ shc stop keyholder...$ shc backup create keyholder[keyholder] Failed 1/2✔ validate the request✘ back up 1sshc: [X409487BKPCRS] deployment keyholder cannot restore a crash image; start it and back it up again$ shc start keyholder...Four more keys belong to apps unlike notebook. x-shc-scale.backup in the
compose file is how many replicas a service keeps while a backup copies its
volumes, 0 for one whose files are only whole when nothing writes them; the
service comes back to its count once the copy is done. at_rest names a
mount captured after the stack is thawed again, for bytes a pre-backup job
finished writing before the freeze. scope.external names data a hook
reaches outside the deployment, such as a bucket, with the reason, so the
backup check does not flag the crossing. export gives an app a native
format of its own for shc backup export --native: the recipe names the unit
directory, the member names and where each member’s bytes come from, the
engine’s dump, a tree or one file. Frappe’s bench is the one the catalogue
ships. Write them on an app that is validated and never installed. --restore-from-backup on an
update restores durable volumes from the latest backup when a rule change
moves a deployment off a node that died with them.
name: benchlikeversion: "1.0.0"title: Benchlikedescription: The backup keys of apps unlike notebook, validated and never installedcategory: demorequires: shc: "0.1.22"license: id: MITservices: bench: image: nginx:1.27-alpine expose: - "80" volumes: - sites:/srv/sites - dumps:/srv/dumps x-shc-scale: backup: 0volumes: sites: dumps:include: - sites - dumpsat_rest: mounts: - dumpsscope: external: - subject: the bucket named in vars.bucket reason: the converter only reads it, with aws s3 sync from s3:// to a local directoryexport: format: bench mount: sites plug: mysql unit: label: site marker: site_config.json exclude: [assets] timestamp: "%Y%m%d_%H%M%S" prefix: "{timestamp}-{unit_slug}" members: - name: "{prefix}-database.sql.gz" from: dump - name: "{prefix}-files.tar" from: tree path: "{unit}/public/files" - name: "{prefix}-site_config_backup.json" from: file path: "{unit}/site_config.json"$ shc validate ~/learn/benchlike...$ shc backup export -i 20261008T154816Z_a9022e670c7c3a73 --native --engine-backup 20261008T154900Z_b1c2d3e4f5a6b7c8 --database site1 --file ~/learn/handover$ shc backup export -i 20261008T154816Z_a9022e670c7c3a73 --with-connected --file ~/learn/handover$ shc update notebook --where node=home02 --restore-from-backup