Skip to content
SHC Docs

Layer 4. Primitives

Layer 4. Primitives. Layer page.

Primitives are mechanisms. They have no owner and no HUD page. Clusterfs, the node lease, labels and constraints, and pressure sit here. An owner and a HUD page is an object.

The materialiser in shcd is the only writer of a node’s clusterfs. Each section has its own directory, <clusterfs>/<section>/, with its own rev-<n> revisions and its own current link. A build renders every section into a staging directory inside the section’s directory. A section whose content changed is fsynced, renamed to its next revision, and its current swapped by rename; one that did not change keeps its revision. The root REVISION is written last and marks the build complete (ce modules/clusterfs/materialiser.go). A reader of a section sees one revision or the next.

A container binds one section, never the whole tree: /etc/shc/clusterfs/traefik for the system Traefik, /etc/shc/clusterfs/collector for the collector, /etc/shc/clusterfs/auth for forward-auth. It reads through the section’s current/, which resolves inside the bind, so it sees each swap on its next open and sees no other section. Each section stamps FRESH with the revision and digest a clean build proved and when.

The auth section holds what forward-auth decides from: realms.json (each realm’s issuer and client, and each host the cluster routes with its realm, deployment, cookie domain and gating), platform_admins.json, revocations.json, scoped_clients.json and ca-bundle.pem (ce modules/app/auth_section.go). The declared platform admins come from one row the leader writes as user ids, so every node renders the same file. Each realm’s forward-auth client secrets live in a secret section on the RAM root, auth-secrets, one file per realm, bound at /etc/shc/auth-secrets. A revocation, a change to the declared admins and a stamp of a new client secret each move the revision, so every node carries it at its next build.

forward-auth decides from those files and calls shcd for no verdict. It verifies the token against the realm’s keys, applies the revocations and the declared admins, and decides with ce core/gatedecide, the package shcd’s own gate handler uses. A gate handle is bound to its session with one call to shcd at login and one per renewal, so the launcher on a gated host still knows who is looking, and a logout proves its revocation with the session’s own token as the bearer. The Traefik plugin’s realm lookups, for the gate and the launcher, read realms.json and auth-secrets too. Until a node has built the section, forward-auth and the plugin ask shcd as a v0.1.21 node did, so an upgrade in any order serves on. A section that is there but whose proof is older than five minutes, or names another digest, is stale: every gated request is the coded 503 stale_replica, and shc doctor’s system.auth-section names the node.

forward-auth answers its health check 503 until its node has built the section, so Traefik sends it nothing before it can decide. shcd’s realm lookup routes, realm-for-host and realm, answer only while a system Traefik replica still runs a render that binds no auth section: the leader lists those replicas into one kv row on every pass, and once the list is empty both routes answer 410 X410059RLMGON. The list never latches, so a node back from being offline with an old replica opens them again until it is rolled.

The licence directory /etc/shc/license is the license section, flat: each file it serves is named at the top of its directory as a link through current/, so the system Traefik and Keycloak read <bind>/license.lic as they always did. Every node renders it from the store: the licence rows, the estate domain, the DNS record the leader looked up, and the ingress intents the apps menu lists (ce modules/app/license_section.go, shc modules/license/section.go). A build starts from what the section served, so a row that cannot be read keeps the file it fed. Secrets sections (certs) live on the RAM root beside the run directory, never on this disk.

A spec stored on v0.1.21 still binds the whole tree, or a package file through the root current. For those containers the root current names a legacy/ directory of links, one per section to that section’s current, so they keep reading live sections until they are re-rendered. A render that binds the bare /etc/shc/clusterfs is refused, so no new one appears. After each pass over its replicas a node writes to the store what on it still reads through the view, and every node’s build decides from all of those reports: it keeps the view while any node reports one, and makes it again if one comes back. Once every node reports none, the build reads the reports again from the leader and removes legacy/, the root current and the whole-tree revisions v0.1.21 left. A node with no report yet, an older build or one that has not finished its first pass, leaves each view as it is. A node the cluster cannot reach counts by the last report it wrote, and a node that cannot reach the leader keeps its own view. A container that reads through the view again once it is gone, a rollback to a spec stored on v0.1.21 say, finds no current until its node’s next pass reports it and the next build makes the view again. shc doctor names each container still reading through it (pods.clusterfs-whole-tree), with the shc update that moves it to its own section.

A bind of a path below current/ holds the directory it resolved to when the container started, and a revision is unlinked three builds after it stops being current. So what a container binds, an app’s package files, sits in the section’s store beside its revisions, and the section links to it from its subtree (ce modules/pods/files_section.go). An entry stays while any revision of the section on disk links into it (SHC-1648).

While this node leads, the node lease writes its transitions and a coalesced renewal (ce modules/nodelease/steward.go).

Placement is node labels and the selectors that match them: --where and --prefer, named node selectors and constraints. shc node label puts a label on a node; shc label, shc node selector and shc constraint read and store the rest (ce modules/fence/commands/label.go, selector.go, constraint.go). shc fence is retired; each of its verbs refuses with X400251FNCGON and names its replacement (fence.go). The placement term is location.

Automatic moves freeze while any pressure signal is red, and thaw only after every signal has stayed calm (ce core/placement/freeze.go, SHC-1208). The scheduler reports replicas waiting on the engine as pressure (ce modules/scheduler/pressure.go).

The mechanisms are this page. Where the materialiser sits in a deploy is render once. A relocate that waits on the freeze is relocate.