Skip to content
SHC Docs

Node module

Layer 2. Core. Core The module is the node record and voter slots, not the container runtime.

SHC supports highly available multi-node deployments using distributed consensus for fault tolerance.

Every node runs the full shc server process. There is no separate agent.

A Linux node can carry mesh, fabric and voter roles. A macOS node carries mesh only, using userspace WireGuard on a utun. The voter target is the largest odd number no greater than the eligible control-plane members, capped at five; other members are non-voters. Sources: CE core/hostplat and modules/cluster/voters.go.

ModeDatabaseOverheadUse Case
Single-nodeSQLiteZeroDefault installation
Multi-noderqlite (raft-replicated SQLite)~50MB RAMHA clusters

Conversion from SQLite to rqlite happens automatically on first shc node join.

The node-plane services bind and dial mesh0 addresses. Their mTLS certificates are separate from the WireGuard keys that authenticate mesh0.

PortProtocolPurpose
4001HTTPrqlite API
4002Raftrqlite consensus
4003HTTPshc internal API

The listener is subsystems/impl/internal_api.go. A node-to-node call is a named internal RPC function (node.health, vault.unlock, certs.rotate, storage.nfs_enable_client, …) that its owning module registers with modules/internal_rpc, all served by the one RPC route (SHC-1207). The generated internal RPC reference lists every function with its arguments and result. Live streams keep their own WebSocket routes. Enrolment is POST /api/join/node.enroll on the public API through Traefik on HTTPS 443, authenticated by a join token. tests/unit/internalroutes pins the internal route set: a new route under /internal/ fails it.

EndpointMethodPurpose
/internal/rpc/{function_name}POSTA node-to-node call, by function name (see communication patterns)
/internal/transfer/{id}GETRaw bytes of a staged RPC transfer handle; only internal_rpc pulls it
/internal/containers/{id}/logs/ws, /internal/stacks/{id}/logs/ws, /internal/ws/logsWebSocketLog follow
/internal/containers/{id}/exec, /internal/ws/execWebSocketInteractive exec
/internal/ws/forwardWebSocketPort-forward. Also the one /internal/ route the unix socket serves, for the local CLI

Leader-aware subsystems perform their leader work only on the rqlite leader. Node-local work, including lease renewal and the node agent, runs on every node. A non-voter receives replicated state without voting. Sources: CE subsystems/register.go and modules/cluster/voters.go.

Terminal window
# On first node - generate join token
shc node token
# On second node - join the cluster
shc node join <token>
# Check cluster status
shc node status
Terminal window
# List cluster nodes
shc node ls
# Drain workloads before maintenance
shc node drain node-2
# Gracefully leave cluster
shc node leave
# Remove unreachable node
shc node forget node-2
# Force leader election (emergency recovery)
shc cluster repair --force

A node runs docker, podman, incus or apple. The daemon owns deployment operations. The node-agent subsystem runs assigned work through the runtime executor. A replica runs on the node it is bound to, and that node’s runner starts the project the leader finished for it (SHC-1426); see Runtime support.

shcd watchdog runs outside the daemon and stops containers labelled shc.single_writer after the node lease expires. It covers all four engines. Sources: CE modules/watchdog and subsystems/impl/node_agent.go.

ProviderSharedQuotasSnapshotsUse Case
Local FSNoZFS/XFS/BtrfsZFS/BtrfsSingle-node
NFSYesNoNoSimple shared
CephFSYesxattrNativeProduction HA

Node leases record liveness. The suffrage reconciler adjusts voter roles to the control-plane target, one member at a time. A down member keeps its vote until explicit removal; quorum loss requires operator recovery. Sources: CE subsystems/impl/node_lease.go, modules/cluster/voters.go and suffrage.go.

ScenarioResponse
Quorum LostWrites rejected, reads continue, 503 error
Leader FailureAutomatic re-election (~seconds)
Split-brainMinority partition becomes read-only