Core concepts
Read this once. Everything else (the CLI, app definitions, flow docs) assumes this vocabulary.
The composite identity: tenant × stack × environment
Section titled “The composite identity: tenant × stack × environment”The most important thing to understand about SHC is that a running app instance is identified by a tuple, not a name:
(tenant, stack, environment) → one deploymentThis tuple is SHC’s universal primary key. File paths, vault scopes, DNS names, audit events, backup archives: all of them derive from it. See architecture/core/deployment-identity for the full rationale; here’s what you need to know day-to-day:
- Tenant: the isolation boundary. Different tenants never see
each other’s data, secrets, stacks, or events. Nothing picks one
for you: name it with
-t,$SHC_TENANTorshc ctx -t, or the command is refused withX400976TNTNON. Tenants says which one you have. - Stack: a named instance of an app. You can install the same
app multiple times with different stack names. Name it for its JOB,
not its software, and the word for its job is the first label of
the hostname it will answer on, so a Nextcloud at
cloud.corp.example.comis the stackcloud. Package names stay product-named; it is the stack name that follows DNS. SHC warns (X200967NAMDNS) when they disagree at install, rename or route change, and theapp.deployment-namingdiagnosis (shc doctor run --category app) finds the ones that already drifted. See Naming a deployment. - Environment: a variant of one stack (e.g.
prod,staging,dev). Fully isolated data, config, secrets. Default:default.
The examples on this page select the tenant default once with
shc ctx -t, and every command after it runs in that tenant. On a
cluster you installed yourself, first shc tenant add default.
shc tenant ls # the tenants you haveshc ctx -t default # select it onceshc install nextcloud # (default, nextcloud, default)shc install nextcloud --stack cloud # (default, cloud, default)shc install nextcloud -s cloud -e prod # (default, cloud, prod)shc install nextcloud -s cloud -e staging # (default, cloud, staging)Those four installs are four completely separate deployments, with separate databases, separate vault entries, separate DNS names.
Cluster, nodes, runtimes
Section titled “Cluster, nodes, runtimes”Cluster
Section titled “Cluster”A single-node SHC keeps its control-plane database as a plain
SQLite file. The moment a second node joins, the database is
promoted into the raft engine (an external rqlited process
the daemon supervises) and replicated across the cluster. The
promotion is automatic; there is no manual migration step.
- Single-node: plain SQLite, all services run on one host
- Multi-node: 3 or 5 raft voters over mTLS + mesh0, Raft consensus, horizontal scale-out of stacks
A node is one host running the SHC daemon.
shc node ls # list nodesshc node ls -v # verbose, full IDsshc install postgres --where 'node=<node>' # hold a stack to one nodeNodes have attributes that affect scheduling: their runtime
(docker, podman, incus, apple), their architecture (amd64/arm64),
their linked storage providers, and whether they’re currently
draining. See
operations/cluster-setup.
Node selectors and service selectors
Section titled “Node selectors and service selectors”Node selectors and service selectors pick things by label, in one grammar, and they are separate kinds. A node selector picks nodes: it’s what --where, --prefer and an expose’s @( ) read, written inline. A service selector picks replicas of its own deployment and gives them a name and a VIP. The app declares it under x-shc-service-selectors, it has no CLI, and it reads service facts only (Groups).
Naming a node selector is optional. A name stands for one, and @<name> works wherever a node selector is written.
shc node selector set edge 'tier=edge&runtime=docker' --scope clustershc node ls --where '@edge&cloud!=hetzner'A name is defined for the whole cluster, or for one tenant. A tenant sees the cluster’s names and its own, and cannot define a name the cluster already has, so a name means one thing wherever it is read. The default scope is the current tenant (-t). shc node selector ls lists the names you can see, shc node selector show prints the nodes a name matches now, and shc node selector rm removes one. The old shc selector is refused with a hint naming shc node selector.
Near or apart from other replicas
Section titled “Near or apart from other replicas”@( ) places a deployment on the nodes of the replicas it names, apart from those nodes, or in their location. A replica carries tenant, environment, stack, app, service, ordinal, group, member, node and location. service and ordinal are set on a service replica, group and member on a group member, and node and location from the node it runs on (location at every level of that node’s path). A service’s compose labels: become every replica’s labels, and its live labels add what the replica says it is now, such as role=primary. SHC refuses its own keys there. A label only matters through a node selector or service selector someone wrote.
shc install myapp --where 'node=@(app=postgres&stack=db)'shc install myapp --where 'node!=@(service=web)'shc install myapp --where 'location=@(stack=db)'A tenant’s rules see only that tenant’s replicas, a cluster constraint sees every tenant’s, and a deployment never counts its own. The answer is read when the deployment is placed, so it does not follow an anchor that moves later. A node=@( ) that picks no replica leaves no node, and the where is refused.
Labels
Section titled “Labels”A deployment carries labels. SHC derives tenant, environment, stack and app from the deployment. An operator adds the rest with --label key=value on shc install and shc update. Naming the same key again replaces the value. key! removes that key. ! clears every operator label. Keys you do not name stay. A deployfile labels: map is that same set, and applying the file replaces the operator labels with what the file says. shc clone copies the source deployment’s operator labels.
Every replica of the deployment carries those operator labels, so node=@(tier=db) is the nodes of the replicas whose deployment is labelled tier=db. When the app author and the operator name the same key, the operator’s value wins.
A volume carries volume (its name), storage (the store it lives on, when that store is known), the owning deployment’s tenant, environment, stack and app, and that deployment’s operator labels. There is no command that labels a volume.
These keys are SHC’s. SHC derives them and an operator cannot write them: tenant, environment, stack, app, service, ordinal, group, member, node, location, and any key that starts with shc.. A label value is plain text. A label changes nothing by itself. shc ls --where and shc volume ls --where keep the rows whose labels match. @( ) is placement, so those listings refuse it. shc status prints a deployment’s derived labels and its operator labels.
Live labels
Section titled “Live labels”A label that changes, such as which replica is primary, is a live label the app declares. SHC runs its command inside the container and reads the first line. The app does not call SHC.
services: db: image: postgres x-shc-live-labels: role: test: ["CMD", "cat", "/var/run/role"] interval: 30s timeout: 5stest uses the healthcheck shape: ["CMD", ...] or ["CMD-SHELL", "..."]. interval defaults to 30s and cannot be below 5s. timeout defaults to 5s and must be shorter than the interval.
The value is the first line, trimmed. It may be at most 63 bytes, and only letters, digits, ., _ and -. A command that exits 0 with an empty line, or with a value outside those rules, means the label is absent. A command that times out, cannot start or exits nonzero did not answer, and the label keeps its value. A new value is taken once two answers in a row give it, so a value that flaps moves nothing. SHC stores a row only when the value changes.
SHC does not act on the value by itself. A node selector’s @( ) reads it the same way it reads service or stack, and an app can declare service selectors, names that go to the replicas it picks by it (Groups):
shc install myapp --where 'node=@(role=primary&stack=pg)'When the app’s compose labels, a live label and the operator name the same key, the operator’s value wins, then the live label’s. shc status shows the labels last read for each replica. A reserved key (tenant, environment, stack, app, service, ordinal, group, member, node, location) or a key under shc. or traefik. is refused.
Container runtimes
Section titled “Container runtimes”| Runtime | Engine |
|---|---|
docker | Docker |
podman | Podman Compose |
incus | incus-compose |
apple | Apple container CLI on macOS |
An empty runtime selects docker. Each node runs one runtime. SHC places
an install within its fences and --where node selector; --prefer ranks eligible
nodes. Both flags can name one service as <service>:<node selector>. See
runtime support.
shc install, shc update and shc uninstall require shcd. An unreachable
daemon returns X503738DMNREQ without changing the deployment.
The trilemma: what SHC gives up
Section titled “The trilemma: what SHC gives up”Three properties pull against each other in any self-hosted platform:
- High availability for stateful workloads: a node dies, the database keeps serving, and no human is in the loop.
- Small-team operability: one person can hold the whole system in their head, and a bad night is diagnosable without a specialist.
- Reliability with no drift: what the platform reports is what is actually running, and it stays true across upgrades, restores and reboots.
Pick two. SHC picks operability and reliability, and that choice is permanent. It is not a roadmap item waiting for engineering time.
What it costs is the first one. SHC does not automatically fail a stateful workload over to another node. Automatic stateful failover means running consensus over the data: replicated volumes, quorum storage, split-brain protection, and a single-writer story for every storage driver. Each of those is a component that can be quietly wrong for months and is only discovered on the night it decides an outcome. That is drift, and drift is the thing this platform refuses to buy: a system that lies to you about its own state is worse than one that is honestly unavailable.
Placement and durable data
Section titled “Placement and durable data”A replica with durable data on its node’s disk stays pinned to that node. If the node is lost, it waits for the node to return. See the resource model.
The route out of a lost node
Section titled “The route out of a lost node”Because the platform will not move stateful work for you, the exits are explicit verbs and they are meant to be practised before you need them:
| Situation | Verb |
|---|---|
| Planned: taking a node out of service | shc node drain <node> |
| The node is gone, its volumes survive | shc update <stack> --where '<key>=<value>' naming labels another node has |
| The node and its data are both gone | shc recover from a backup archive |
The update treats the node as dead only when it is registered unreachable and a fresh probe fails, because the failure it prevents (force-detaching storage out from under a still-live writer) is exactly the split-brain that automatic failover has to solve and this platform declined to.
Apps, stacks, deployments
Section titled “Apps, stacks, deployments”Three similar-sounding words, three different things. Keep them straight:
-
App: a template. Lives on disk under
apps/<name>/. Never runs anything. Just definition files (app.yaml,compose.yaml, plug/socket files). Shipped with SHC or pulled from a repo. -
Stack: the installation record of an app. Created by
shc install <app> --stack <name>. One stack = one row in the control-plane database’sstackstable. The stack carries the install-time variables, which node it’s assigned to, which runtime, … -
Deployment: the running state of a stack in a specific environment. One stack + one environment = one deployment. This is the
(tenant, stack, environment)tuple above. Backups, snapshots, compose state all key off the deployment.
A stack with environments default, staging, and prod is one
stack row but three deployments.
A catalog app can carry more than one published version: shc install <app> <version> (or the equivalent <app>@<version> one-token form, see
shc install)
pins one, a bare version resolves to its newest packaged revision, and
the registry keeps only the newest 10 packaged versions per app,
pruning older ones automatically.
Tenants
Section titled “Tenants”Tenants are isolation boundaries:
- Separate database records scoped to the tenant
- Separate vault scopes (so
ref+vault://db_password?scope=tenanton tenant A resolves differently from the same reference on tenant B) - Tenant network policy on the
shcfabric - Separate audit trails
- API permissions scoped by tenant (see security.md)
Tenant permissions and vault scopes separate each tenant’s data. On Linux,
the shc fabric applies container network policy on shc0, while mesh0 carries
the node plane. This is organizational separation; see the
security model and
network fabric.
shc init creates none of your tenants, only SHC’s own master. A
cluster made through the portal already has one named default. Run
shc tenant ls to see which you have. If it lists default, use that
one. If it lists only master, add one with shc tenant add <name>,
which is free. A second tenant needs the Enterprise tenancy entitlement,
and without it shc tenant add is refused with X403029LICENT.
Nothing selects a tenant for you, and -t never creates one. Name it
with -t, $SHC_TENANT or shc ctx -t, or shc install is refused
with X400976TNTNON; a name the cluster does not have is refused with
X404973TNTNXS. To select a tenant and work in it:
shc tenant lsshc ctx -t defaultshc install nextcloud --stack cloudThe cross-tenant --all-tenants flag on a few admin commands is
guarded by an explicit admin.tenants permission; see
security.md.
Integrations: plugs & sockets
Section titled “Integrations: plugs & sockets”Apps connect to each other via sockets (providers) and plugs
(consumers). Install a consumer with -i provider and SHC runs a
multi-phase job pipeline to provision the connection (create a
database, grant a user, issue an API key, …).
shc install wordpress --stack blog -i postgres# ^^^^^^^^ provider stackThis is SHC’s killer feature: the orchestration isn’t limited to “start these containers in the right order,” it can actually provision the things apps need from each other.
Full model: plugs-and-sockets.
A related concept, vApps, lets one stack host many sub-resources (many databases in one Postgres, many realms in one Keycloak). See vapps.
Jobs, hooks, crons
Section titled “Jobs, hooks, crons”When SHC needs to run something (a DB migration, a backup converter, a cache flush, an integration provisioning step), it uses a job.
A job is defined in <name>.job.yaml inside an app directory. Its
context selects where it runs: in an already-running service
(service), in a new container that runs once (container), as an
HTTP call (http), as a GraphQL mutation (graphql), as a pipeline,
or as a Playwright browser script.
A job that names after: or before: is a start job, and it is
how an app runs its run-once work (a migration, a config render, an
admin seed). It runs once per deployment at the install and at each
update, after the services in after: read ready, and the services in
before: start only once it ended 0. A failed start job fails the
install, which rolls back. A container start job that names
service: and no image: runs as that service’s own definition, so
a migration names its server and its command and nothing else. See
Lifecycle hooks.
A container job may set where: and prefer:, both selectors.
where: ordinal=1 runs that job once, in ordinal 1 (of service:
when the job names one). Several matches run in the lowest ordinal.
prefer: ranks the replicas it matches first and never drops the
rest. When nothing matches, the run is refused and the refusal lists
each replica’s labels.
Three things trigger jobs:
| Trigger | Fires when | Docs |
|---|---|---|
| Hooks | Lifecycle events (pre_install, post_install, pre_backup, …) | lifecycle-hooks |
| Crons | Cron schedule defined in app.yaml | architecture/modules/schedule |
| Integration phases | discovery, integration, post_deployment on connect; unintegration on teardown, for every edge on uninstall and for the dropped edge on -i <plug>! | plugs-and-sockets: integration phases |
Run history is persisted as rows in the control-plane database;
see shc job history.
A repo is a source of apps. SHC ships with a shipped repo of
~60 apps installed with the package, and the daemon catalogs them
automatically. You can register additional repos pointing at:
- A git repository (
https://github.com/you/shc-apps) - A local bare git remote (
git+file:///srv/mirrors/apps.git) - An OCI image (not yet)
Repos are remote-only. To install an app
from a local directory, install it directly
(shc install ./path/to/app) instead of registering the directory.
shc repo lsshc repo add my-apps https://github.com/acme/shc-appsshc install acme-internal-tool --repo my-appsSee app source resolution for the full search order.
Storage & vault
Section titled “Storage & vault”Two orthogonal systems you’ll touch as an operator:
-
Storage providers: where stack volumes and backup data live. Types:
local,nfs,cephfs. See operations/storage-providers. -
Vault: where secrets live. Referenced from YAML via
ref+vault://key?scope=.... Backend providers include a built-in store encrypted in the control-plane database plus 10 external options. See operations/vault.
Events & audit
Section titled “Events & audit”Two streams of observability data, both sharing the same event infrastructure:
-
Events: lifecycle signals (
A200100STKINSwhen a stack installs,V500320STKERRwhen it errors). Subscribe via Server-Sent Events throughshc eventsor the HTTP API. See flow: event lifecycle. -
Audit trail: who did what, when. Every vault read, every permission-checked API call, every tenant admin action.
shc events --audit. See architecture/modules/audit.
Both streams can be pushed to an external collector (default: OpenObserve); see operations/observability.
Quick reference: what owns what
Section titled “Quick reference: what owns what”tenant├── stacks (one app install → one stack)│ ├── deployments (one stack × environment → one deployment)│ │ ├── compose services (containers running the app)│ │ ├── volumes (data on storage provider)│ │ ├── vault entries (secrets at deployment scope)│ │ ├── integrations (edges to other stacks' sockets)│ │ └── vdeployments (sub-resources: realms, DBs, buckets, …)│ └── stack-scoped state (hooks, crons, variables, …)├── tenant-scoped vault entries (?scope=tenant)├── audit trail└── events
cluster├── nodes (one host = one node)├── control-plane database (SQLite; raft-replicated via rqlited in cluster mode)├── mesh0 (WireGuard node plane, UDP 4244; kernel on Linux, utun on macOS)├── storage providers (local / nfs / cephfs)└── vault (providers: local + 10 external)Where to go next
Section titled “Where to go next”- I want to run my first stack: Quick start
- I want to understand the install pipeline in depth: Architecture: install flow
- I want to stand up a multi-node cluster: Multi-node cluster setup
- I want to write my own app: App definition
- I want to know exactly how tenant isolation works: Security