Skip to content
SHC Docs

Restore onto a new cluster

Copy the key first. The archive in your bucket is encrypted, and the key that opens it lives in the cluster’s vault and nowhere else. Delete the cluster without copying that key off it and nobody can ever open the archive, you included. Step 2 is the copy. Don’t delete anything until you’ve done it.

This page takes an app and its database from one cluster to a new one, with only a bucket surviving in between. You don’t need SSH on either cluster.

That bucket is the only off-server copy this page needs. A second copy (shc backup mirror) stays optional.

  • shc 0.1.56 or newer, and no older than the cluster. Get it or update it with Reach a portal cluster from your laptop, then check which shc you’re running:

    Terminal window
    $ command -v shc
    $ shc version

    The path should be the folder that page puts it in, and the version the one https://downloads.selfhostedcloud.org/shc/dev/RELEASE names (shc/RELEASE on the production portal), or newer. An older shc from an earlier install still signs in, then sends requests the cluster refuses, such as X400976TNTNON from shc backup sync.

  • The shc command line, signed in to the cluster:

    Terminal window
    $ shc auth login --url https://<cluster domain>

    You sign in through the portal. You have to be the cluster’s registrant, an owner or an admin, which gives you platform:manage on it. An organisation member or viewer is refused, and every step below needs that role.

Follow Set a backup destination through its step 3. Skip it if shc config get --scope global backup.repo already names your bucket. Keep hz.json: the new cluster needs it in step 6.

Two keys, kept together, somewhere that isn’t the cluster.

The first opens the repository. The cluster made it for you:

Terminal window
$ shc vault get backup/password > backup.key
$ chmod 600 backup.key

The second opens the app’s own secrets inside the backup. You make it:

Terminal window
$ openssl rand -base64 32 > secrets.pw
$ chmod 600 secrets.pw

Without it the app’s secrets are sealed under the cluster’s vault password, which the portal doesn’t show you.

--all-connected takes the app and its database together, so they restore to the same moment. The app is paused while it’s read, so do this when nothing is writing. <env> is default unless you installed the app under another name.

Terminal window
$ shc backup create <app> -t <tenant> -e <env> --all-connected \
--secrets-password-file secrets.pw

For an app with no database of its own, leave --all-connected off.

If your shc doesn’t know --secrets-password-file, update it and the cluster first. An older cluster loses the password on --all-connected and still reports success, and that backup won’t open anywhere else.

A schedule can’t name a password, so a scheduled backup is sealed under the vault password and won’t open on a new cluster. Take this one by hand.

Terminal window
$ shc backup ls -s <app> -t <tenant> -e <env> -o json
$ shc backup verify <id> -t <tenant> --deep

Check the size against what you put in the app. A backup can complete and hold less than you expected. Then look at the bucket in the Hetzner console: it should hold objects, and its size should have grown by about that much.

Keep the deployment names and the backup ids. You’ll want them in step 7.

Delete it in the portal. The bucket stays, because nothing in the delete touches it. Check that it’s still there in the Hetzner console before you go on.

6. Build the new cluster and give it the keys

Section titled “6. Build the new cluster and give it the keys”

Build it, sign in with shc auth login as above, and make the tenant again if your app wasn’t in master and shc tenant ls doesn’t list it:

Terminal window
$ shc tenant add <tenant> --no-admin

Connect and grant the bucket again, with hz.json: step 2 of Set a backup destination. Then hand over the repository key and point backups at the bucket, in this order:

Terminal window
$ shc vault set --scope global old/backup-key --value-file backup.key
$ shc config set --scope global backup.password 'ref+vault://shc/system/old/backup-key'
$ shc config set --scope global backup.repo @hzbak/<bucket>

All three are global, because the orphan scan in step 7 reads a repository with the key set at the same scope as the repository.

This cluster’s own backups now go to the same bucket under the old key. That’s fine. Its retention only removes what it wrote itself.

The cluster’s Console does the same from Manage > Backups: press Backup destination, pick the connection, type the bucket, paste backup.key into Repository key and secrets.pw into Secrets password, and press Save. It stores both keys in the vault and sets the three keys globally.

The new cluster has no record of your backup, so it shows up as an orphan:

Terminal window
$ shc backup orphan scan -t master
$ shc backup orphan ls --all-tenants

Each row has an ID, the deployment it came from under STACK and its age. foreign-estate under REASON means another cluster wrote it. TENANT says master, because the repository belongs to the cluster. SOURCE TENANT is where the app was. locked: secrets-key under STATE is expected here: step 8 brings that key.

Name every id that belongs together. Look at the plan first:

Terminal window
$ shc recover -b <database id> -b <app id> -t <tenant> -e <env> \
--secrets-password-file secrets.pw --dry-run

Then run it:

Terminal window
$ shc recover -b <database id> -b <app id> -t <tenant> -e <env> \
--secrets-password-file secrets.pw --yes

If you pick two backups that weren’t taken together, it’s refused: recovery inputs have different consistency ids.

The app comes back under the hostname it had on the old cluster. See Give the app its address.

In the Console, the backup is on the Orphans tab of Manage > Backups. Open it and use Recover, which fills in its id.

For one deployment on its own, this also works:

Terminal window
$ shc backup restore --id <id> --as <name> --into <tenant> -t master -e <env> \
--secrets-password-file secrets.pw
Terminal window
$ shc ls -t <tenant>

Both deployments should say running. Open the app, sign in with the password it had before, and read back a file you put there. Compare it to the original by its hash, not by looking at it.

The app answers on the hostname it had before. Point that name’s DNS at the new cluster. To bring it up under another name instead, add --override hosts=<new host> to the shc recover above, or --override <deployment>:hosts=<new host> to rename one deployment only.

You seeIt meansDo this
X400486BKPNRPNo backup.repo is setStep 1
X400976TNTNON though you passed -tYour shc is older than the cluster and dropped the tenantUpdate it, see What you need
X200962CLIOLDThe same, said before anything is refusedUpdate it, see What you need
X403910CONGRTThe connection isn’t granted to this tenantThe grants on Set a backup destination
X403925CONBKTThe grant doesn’t name this bucketThe grants on Set a backup destination
X503956BKPIRF from orphan scanA repository didn’t open, usually a wrong backup.passwordCheck backup.key is the file from step 2, then step 6
X409948BKPOLK, needs backup.passwordThe repository’s key is wrong or missingStep 6
X409948BKPOLK, needs backup.secrets_passwordNo secrets password is set or givenPass --secrets-password-file secrets.pw, the file from step 2
X409958BKPSNONo password you gave opens the app’s secretsCheck secrets.pw is the file from step 2. A backup taken without --secrets-password-file opens only on the cluster that made it

If you’d rather not pass the file each time, store it on the new cluster:

Terminal window
$ shc vault set --scope global old/secrets-key --value-file secrets.pw
$ shc config set --scope global backup.secrets_password 'ref+vault://shc/system/old/secrets-key'