Block storage by default
A deployment’s durable volumes belong on block storage whenever the estate
has a provider that offers it. backup.yaml decides which volumes are
durable: the ones it lists under include or keep.
The ruling
Section titled “The ruling”“why are you not using block storage for everything? are we not in a hetzner cloud? shouldn’t deployments use block storage when it’s available (possibly with the exception of ephemeral or non important volumes). we know which ones are important and which ones don’t matter based on what is / isn’t in the backup.yaml and if nothing is specified in the backup.yaml we consider them all important. important as in if moving a deployment you have to move the volumes instead of simply allowing brand new volumes. for example a cache or temporary files wouldn’t need to move that volume to a new node when shc relocates a deployment.”
Owner ruling, 2026-08-26 (SHC-417)
The ruling of 2026-10-18
(backup.yaml lists what a backup copies)
replaced the last clause. A backup.yaml says what’s important by listing
it, an empty one says nothing is, and a package with named volumes and no
file doesn’t install.
The decision procedure
Section titled “The decision procedure”Per DATA mount, in this order. The order is the policy; nothing here is a tiebreak.
| # | Condition | Result |
|---|---|---|
| 1 | the package declares x-shc-volumes for the mount | block: the author already decided |
| 2 | the package’s backup.yaml lists the mount under neither include nor keep | local: scratch, the only exemption |
| 3 | the deployment’s own record says the mount is block-backed | block: replayed, every render |
| 3b | the deployment’s own record says its first render held the mount back | local: replayed, not pending |
| 4 | the estate has no block provider (or --storage names another) | local, with the reason recorded |
| 5 | the deployment already exists | local + pending: flagged, never switched |
| 6 | the mount directory already holds bytes | local + pending: flagged, never switched |
| 7 | the binder holds the derivation back (below) | local, with the reason recorded |
| 8 | otherwise | block |
Rule 2 is the whole exemption. The file earns it for every mount it leaves
out, which is why install refuses a package with named volumes and no
backup.yaml (X400601BKPUND): an omission has to be written down. A package
that reaches the binder with no file anyway falls through with every mount
durable.
The first volume on an estate is a decision
Section titled “The first volume on an estate is a decision”Rule 7 exists because “a provider exists” is not the same as “somebody
chose block storage”. With storage.volumes.provider unset, an estate’s
sole volume-capable connection is auto-selected, and on a Hetzner estate
that is usually the connection that also serves DNS. Without rule 7 a CI
deploy could buy the estate’s first paid volumes, one per durable mount,
without anyone having asked, on servers that hold at most 16 volumes each.
So a derived binding is held back, and the mount stays on the node disk, when either of these is true:
- Nobody asked by name and nothing is in use. The provider was only
auto-selected, the install did not pass
--storage <provider>, and no volume on the estate is attached. A row a failed bind left behind does not count. Passing--storage <provider>or settingstorage.volumes.providerarms derivation, and once one volume is attached it stays armed. - The provider says no machine it can see is the node. The Hetzner
provider resolves an SHC node to a server by the id the node’s own
metadata service reports (this daemon’s node only), then a server named
after the node or its hostname, then the node’s recorded public address,
then its recorded internal address. A server in another location than the
one the connection creates volumes in is never a match, since the volume
could not attach to it, and an internal-address match must also hold the
node’s recorded public address, so a node with none recorded is never
matched by its private address: private ranges repeat in every project,
so on their own they name no machine. A node none of those match
(
X404346VOLHSV) stays local instead of failing the install (SHC-933: nothing requires a server to be named after its SHC node).
Only that answer holds back. A hold-back is permanent, because rule 5 never switches a live mount, so a question that could not be answered today (the Hetzner API timed out or refused, the resolve stalled, the volume store could not be read) fails the install instead, before any volume or row exists. The next attempt asks again.
The render that holds back says so once:
X200336VOLLOC deployment acme/wiki/prod keeps durable mounts data, uploads on the node disk instead of new block storage, for example data is durable, but this estate has no block volume in use and hcloud was only auto-selected (the first one waits for --storage hcloud or storage.volumes.provider), so it stays on the node diskNothing is owed: the mounts were empty and stay where every install before
this policy put them. The sidecar records the hold-back, so every later
update keeps those mounts local (rule 3b) without saying X200336VOLLOC
again and without recording them as pending migration, and shc doctor
reports no debt for them. Moving one onto block storage later is a
deliberate migration, like any other local mount. An authored x-shc-volumes entry and a deployment’s
recorded block mount are never held back. They reach the binder, which
resolves the machine before it provisions anything, so a node it cannot find
fails the install with X404346VOLHSV and no volume is created.
Saying a volume is scratch
Section titled “Saying a volume is scratch”Leave it out of backup.yaml. The file is one declaration, and the backup,
move and delete paths all read it. A mount under include or keep is
durable, and every other mount is scratch:
include: - pgdata # a backup copies these - repo-dumpskeep: # Repositories. The pre_backup hook packs them into repo-dumps. - repos# lsp-cache isn't listed. It's a cache: a move doesn't carry it, a delete# doesn't warn about it.keep is persistent data another path protects. gitlab is the worked
example: it keeps gitlab-data because gitaly trees are hostile to a raw
copy, and its pre_backup hook packs them into gitlab-backups, which it
includes. Those bytes matter as much as any, so a move carries them and block
storage holds them, but no backup reads them raw.
There’s no second place to say this. x-shc-volumes refuses a
durability: or disposable: key so a second spelling can’t exist to
disagree with the lists.
Which storage answers which question
Section titled “Which storage answers which question”The two are different axes and they are routinely confused.
| Object storage (S3) | Block storage | |
|---|---|---|
| What it replaces | the app’s own storage backend | the disk underneath everything else |
| When it applies | only where upstream supports it natively | every durable mount |
| Nextcloud | yes: native objectstore primary storage | not needed for user files |
| Immich | no: no native S3 backend | yes |
| How it is chosen | the package ships an s3 plug | derived from backup.yaml |
Object storage is only ever used where the application implements it itself. SHC does not, and will not, mount a bucket as a filesystem: no rclone, no JuiceFS, no s3fs. A POSIX shim over an object store is a different set of consistency and latency guarantees wearing a filesystem’s clothes, and every failure it produces looks like a filesystem bug.
So: object storage where the app speaks it, block storage for the rest.
See App buckets and the s3 plug for
the object-storage half.
An existing deployment is flagged, not switched
Section titled “An existing deployment is flagged, not switched”This is the half that matters most, and it is the reason arming the default is safe on a running estate.
Repointing a mount source moves no bytes. Mounting a fresh block device
over <deployDir>/mounts/nextcloud-data does not migrate 5 GiB of data; it
hides it, and the app comes up on an empty filesystem, running and healthy and
blank. So the policy derives a binding only on a deployment’s first render
whose mount directory provably holds nothing. Every other durable mount on a
providered estate is recorded pending and reported once per render:
X200334VOLPEND deployment corp/nextcloud/prod has durable mounts nextcloud-data still on the node disk while this estate offers block storage, migrate them deliberatelyMigrating those bytes is a deliberate act. It is not a side effect of an update, and no render will ever do it for you.
Where the decision is written down
Section titled “Where the decision is written down”Each render records, per mount, what the bytes sit on and what the app said
about them, in the deployment’s state.json:
"volume_backings": [ {"mount": "appdata", "backing": "block", "durability": "durable"}, {"mount": "appcache", "backing": "local", "durability": "scratch"}, {"mount": "legacy", "backing": "local", "durability": "durable", "pending": true}]It is persisted rather than re-derived for the same reason the node-local
volume guard is. The compose file is rebuilt from the app package on every
render, so without the record a first install’s derivation would be dropped by
the very next shc update: the mount would fall back to a plain host bind at
the same path, correct for exactly as long as the device happened to stay
mounted, and silently writing to the node’s own disk the moment it did not.
Turning it off
Section titled “Turning it off”- Per deployment:
shc install --storage local(or any backend other than the estate’s volume provider). The decision is recorded as the operator’s, not as a missing provider. From CI, setSHC_STORAGE(cicdtemplates/shc-base.yml), which passes--storageon a first install.shc deploy --storageon a stack that already exists changes nothing: the update keeps the storage its install chose and warnsX200421DPFSTO. - Per estate: do not configure a volume-capable connection, or leave more
than one configured without setting
storage.volumes.provider, an ambiguous estate declines rather than guessing which cloud a deployment’s data should live in.
An estate with no block provider installs exactly as it did before this policy existed. That is asserted as a byte comparison of the rendered compose, not as a claim.
What it costs
Section titled “What it costs”Block volumes are billed per volume, with a provider minimum: 10 GiB on Hetzner, about €0.57/month each. The mechanism provisions one volume per durable mount, so a deployment with eight small durable mounts buys eight minimum-size volumes, not one. On an estate with many small mounts that arithmetic is worth doing before arming a provider.
Source of truth
Section titled “Source of truth”| What | Where (ce) |
|---|---|
| the policy | modules/app/volume_durability.go |
| the durability declaration | modules/app/backup_list.go, modules/app/backup_selection.go |
| the compose extension + parser | core/volume/binding.go |
| the binder (provision → attach → mkfs → mount) | daemon/volume_binder.go |
| provider selection | daemon/volume_binder.go, storage.volumes.provider |
Tests: tests/_ce/modules/app/volume_durability_test.go in this repo.
See also
Section titled “See also”- Storage providers: local / NFS / CephFS setup
- Enable backups:
backup.yaml, the same declaration read for capture