Skip to content
SHC Docs

The apple runtime (design)

Status: open design (SHC-1200). The executor is built in ce and linked into every shcd (ce daemon/imports.go), and a node runs it only on macOS: the gate is executor.RunsOn (ce core/executor/executor.go, SHC-1211). A Mac node also brings up mesh0 and shc0 on utun devices, guards its node plane with pf and runs replica projects (SHC-1287, SHC-1623, SHC-1642, SHC-1557), which sections 5.2 and 7.2 once placed after the cutover. A restore that names --runtime apple lands on a node that runs the apple executor (SHC-3155). The healthcheck runner still lives in the executor although modules/nodeagent now exists. Clay ruled on 2026-10-01 that apple is part of the architecture now, beside docker, podman and incus (memory: shc-runtimes-docker-podman-incus). This note is the contract for ce core/executor/apple and the port of shcd to a macOS node. It does not edit the simplification proposal; the deltas it owes that document are in section 9.

Written against apple/container 1.5.0 (2026-09-29): its docs and Sources/ at that tag, read 2026-10-01. Every flag below was checked there. Upstream issue and PR states were read on 2026-10-01.

1. What apple/container is, as it bears on an executor

Section titled “1. What apple/container is, as it bears on an executor”

Each container is its own Linux micro-VM under the Virtualization framework, Apple Silicon only, macOS 26 or newer. The CLI is container; its state lives in an apiserver that launchd supervises, rooted at an app root (--app-root, CONTAINER_APP_ROOT).

What exists (1.5.0):

  • container run -d --name <id> -l k=v --env-file <path> --network <name>[,mac=..] -p [ip:]host:ctr[/proto] -v src:dst[:ro] --mount type=bind|volume|tmpfs,...[,size=,mode=][,readonly] --tmpfs <path[:opts]> --dns <ip> --dns-search <d> --dns-option <o> -u <user> -w <dir> --read-only --shm-size --init --platform --cpus <n> --memory <size> --entrypoint <cmd> -i -t <image> [args]. Also --cap-add, --cap-drop and --ulimit, applied inside the container’s own VM, and --runtime <handler>, which picks the runtime helper the host apiserver launches.
  • container network create [--subnet <cidr>] [--subnet-v6 <cidr>] [--internal] [--label k=v] <name>. Networks are isolated from each other, every network carries IPv4 and IPv6 (a v6 subnet is allocated when none is given), and the host reaches every container address. container network list --format json prints each network’s configuration (mode nat or hostOnly, ipv4Subnet, ipv6Subnet, labels).
  • container volume create [--label k=v] <name>: an ext4 image, mounted as a block device into the VM.
  • container list --all --format json, container exec [-i] [-u] [-w] [--env-file] <id> <cmd...> (answers the process’s exit code), container stop [-s SIG] -t <seconds> <ids>, container delete [--force] <ids>, container start <id>.
  • container run fetches a missing image itself.
  • Label keys must match docker’s grammar: lowercase, digits, dots and hyphens. Container names match [a-zA-Z0-9][a-zA-Z0-9_.-]+, at most 63; network names are lowercase.

What does not exist:

  • No compose. The executor translates the projection itself.
  • No event stream (container events is not a command).
  • No healthcheck (#1918), no restart policy (PR #1258, tracking #2142), no exit code: RuntimeStatus is running, stopped, stopping or unknown. The executor records the exit itself (section 2.7).
  • No static address (#282; PR #751 “Specify ip in --network” and PR #1740 “sticky IPs” are open).
  • No --hostname, --add-host, --sysctl, --privileged, no host or shared network namespace, no device passthrough, no annotations.
  • Headless start (#2008): container system start bootstraps the apiserver plist into gui/<uid> or user/<uid> and discards launchctl’s failure, so as root outside a login session it fails with a misleading XPC error. The reported workaround is launchctl bootstrap system <plist> and a second container system start. PR #2045 only surfaces the bootstrap failure; bootstrapping into the system domain as root outside Aqua is PR #2058. Both are open.

core/executor/apple implements executor.Executor (simplification §4.7). It registers under the runtime name apple through executor.Register, so executor.New("apple") answers once the package is linked; the node agent, the interface’s only caller, blank-imports it.

2.1 Up: a projection becomes network, volume and run calls

Section titled “2.1 Up: a projection becomes network, volume and run calls”

The projection is the compose file the leader rendered for this node: one entry per replica named <service>-r<ordinal>, restart: "no" stamped. Up first runs the refusal (section 6.1), then converges:

  1. Every network the file declares is created when absent: container network create --subnet <ipam subnet> [--subnet-v6 ..] [--internal] --label com.docker.compose.network=<key> [--label com.docker.compose.project=<project>] <name>. A network without a name: is <project>_<key> and carries the project label; one with a name: is shared with other deployments on the node, carries no project label and is never removed by Down. The implicit default is created when a service names it or declares no networks. An existing network is reused only if it carries the declared subnets and mode (internal is hostOnly, otherwise nat); anything else fails Up with X400775APLCMP before a container starts, because reusing it would break the pin.
  2. Every top-level volume is created when absent: container volume create --label com.docker.compose.project=<project> --label com.docker.compose.volume=<key> <project>_<key>, so Down can remove by label. A volume is exclusive to one entry: an apple volume is an ext4 image that one VM mounts at a time, so a projection where two entries mount the same volume is refused, and so is a volume with its own name:, which another deployment could mount. 32 of the 107 catalog apps share a volume between services today; on apple such a volume has to reach the projection as a host directory bind (virtiofs is shareable across VMs). That is the render’s change to make, and until it does, placement keeps those apps off Mac nodes. Each container records the volumes it mounts in shc.volumes, and before Up starts any entry it removes the orphans and every container that holds a volume another entry now mounts, so a volume renamed or moved between entries is never attached to two VMs.
  3. Entries start in depends_on order. Each container is named <project>-<entry> and carries com.docker.compose.project, com.docker.compose.service, the user’s labels, the healthcheck as shc.health.* labels, the stop grace and signal as shc.stop.* labels, and shc.spec.digest, a sha256 of everything the engine is handed. A compose label with any of those keys is refused: it would move a container, network or volume into another project’s ownership. Each start or recreate Up (or the agent’s Start) makes is a new generation of the container, separate from its digest (a restart: true recreate keeps the digest). The generation begins before the old container is touched: its word is discarded, its runner is stopped, and no runner is admitted for that container until the new one has started and a listing begun after that has seen it, so no probe of the old container, and no row describing it, can count for the new generation.
  4. Up is idempotent by that digest. A running container with the same digest is left alone. A stopped one is container started, except a one-shot (shc.lifecycle=one-shot) with the same digest, which has run for this revision and is left alone (section 3.2). A changed one is container delete --forced and run again. A container with the project label the projection no longer declares is removed (compose’s --remove-orphans).
  5. Before an entry starts, each condition: service_healthy dependency must report healthy: Up runs that dependency’s healthcheck through container exec with the runner of section 3.1 until it is healthy, and an unhealthy dependency fails Up before the dependant starts, as docker compose up does. Only a word about the dependency’s current generation counts, so a dependency Up just recreated is probed again. An entry whose restart: true dependency Up started in this pass is recreated. A required: false dependency the projection does not carry is skipped, and one that turns unhealthy releases its dependant, as compose does.
  6. Environment never rides an argv (SHC-1176): environment and env_file (one path, a list, or {path, required, format} mappings) merge into a mode-0600 file in a mode-0700 private directory, handed as --env-file and deleted after the call. Env files are read with compose’s dotenv rules (export, single quotes literal, double quotes with \n \t \\ \" \$, unquoted trimmed and cut at #) or verbatim with format: raw. Inline environment values and every label (on a service, a network or a volume) are read by the same rule: a value that would interpolate a variable is refused by name before anything runs, since the projection is already rendered, and $$ is a literal $, as docker hands it over. A bare KEY with no value is refused; the daemon’s environment is never copied in. A bare NAME line in an env file, which compose reads as “unset”, is refused too: the CLI can set a variable but never remove one the image declares. A value with a line break or NUL cannot ride an env file and is refused before any engine call.

Every refusal Up can hit is also what apple.Refuse answers: it runs the key classification and the whole translation (everything but reading env files), so placement refuses an empty entrypoint or command (apple cannot clear an image’s), a mistyped size or an undeclared volume, instead of Up failing on the node.

Memory is rounded up to whole mebibytes (--memory 512M), CPUs up to whole vCPUs (--cpus 1 for cpus: "0.5"); a VM takes nothing finer.

Ruling 2026-10-01: ports: and expose: in an app’s compose only declare the container ports a service listens on. The executor publishes a port only from a long-form entry with mode: host and published:, which is what the render writes for --port or a load-balancer connection: -p [host_ip:]published:target[/protocol]. Any other ports: entry in a projection (a short form, or a long form without mode: host and published) means the render left an app’s own port in, which docker would publish; apple refuses it by name rather than ignore it. An app’s own ports reach the projection as expose:.

Down needs no compose file. It lists the project’s containers by label, stops them grouped by stop signal and grace (container stop [-s SIG] -t <grace> ids, grace from shc.stop.grace, default 10 s, DownOpts.Timeout overriding every grace), deletes them, deletes the project-labelled networks and, with Volumes, the project-labelled volumes.

Ps lists by label. State is apple’s own word. For a container that is not running, ExitCode and FinishedAt are what the exec helper recorded when its first process ended (section 2.7), and ExitUnknown (-1) and no time where it left no record, because the CLI reports neither; nothing above the executor may read a stopped apple container with no record as “exited 0”. Health is "" for a container without a healthcheck or one not running, otherwise the runner’s last word, starting until the runner has probed.

Exec is container exec [-i] [-u user] [-w dir] [--env-file path] <id> /.shc-bin/exec-env --exec-keep-env --cap-shed CAP_NET_ADMIN --cap-shed CAP_SYS_ADMIN -- cmd..., env through the same private file. The helper stands in front of the command only to give up the capability every run adds (section 2.8) and the one a named service’s run adds (section 2.9), and then becomes the command with the environment the CLI gave it.

2.4 Events: polling, deaths, and the healthcheck runner

Section titled “2.4 Events: polling, deaths, and the healthcheck runner”

Events polls container list --all --format json every Options.PollInterval (default 2 s). A container seen running that is no longer running is an EventDie, and so is a container first seen stopped after the subscription began: it started and stopped between two polls. A container is followed run by run: one that reads another startedDate or another spec digest than it did is another run, whatever was seen of the one before, so a run that began and ended between two polls of a container already seen stopped is a death too, and a run first seen stopping is followed to its end. A run replaced while it was up sends no death: its container is up again under the same name. A death carries the exit code and the time the helper recorded, 137 for a container a stop of the executor is killing at that moment, and ExitUnknown without either. Every running container that carries shc.health.* labels gets a runner (section 3.1) whose transitions are EventHealth with docker’s words, counting start_period from the container’s own startedDate. A container recreated under the same name (a new digest) or restarted by Up or Start (a new generation) gets a new runner from starting, admitted only from a listing begun after the replacement finished. A runner is stopped, and its last report finished or dropped, before its container’s death is sent, so no health word follows a death. One window remains: a word the old runner validated just before a replacement can still reach the channel after it, always followed by the new runner’s starting, which a consumer takes as the container’s state; the dependency gate never reads events, only the generation-scoped word.

The channel closes when a listing fails or the context ends; the caller subscribes again and re-reads Ps. Polling can still miss a container that died and was recreated between two polls, so the node agent does not rely on events alone: on every resubscribe, and on a periodic resync, it reconciles the desired replicas against Ps (missing, stopped, wrong digest) and acts on the difference.

The CLI cannot pin an address, so on an apple node the subnet is the pin, and the address is observed. The leader reserves each network’s subnet out of the node’s grant (ruling 2026-10-01: node-local deployment networks from 10.64.0.0/10, the fabric from 10.128.0.0/10 with a bridge per tenant per node) and writes it as the network’s ipam subnet. apple assigns each container’s address inside it; the list row reports it, and the node agent carries it into the node report (the one field executor.Container needs for it is added by the agent’s spine, not here).

What needs only the subnet is rendered from the reservation: routes and the peer’s AllowedIPs. What needs the exact address is rendered from node reports: the resolver’s zone, Traefik’s backends, and the policy compiler’s per-service rules (section 5.4), because a tenant’s replicas share one subnet and “port 5432 on that subnet” would open the port on every replica, including services that never declared it. When an observed address changes (a recreate), the node report changes, and the resolver, Traefik and the pf anchor are re-rendered, coalesced to at most one revision a second as the health ruling requires. That window fails closed (section 5.4): an address has no permission until a revision names it, and a retired address loses its rules and its pf states before a new container may start on that network, so a reused address never inherits another service’s permissions. The pin ledger row for a replica on an apple node is filled by the node, not chosen by the leader. ipv4_address on a service is refused by name so a render that still stamps it fails loudly.

An executor that cannot run every compose key implements executor.Refuser. executor.Refusals(runtimes, project, compose) asks each runtime present in the snapshot and returns, per refusing runtime, the refusal’s text. The demand builder puts it on placement.ReplicaDemand.Refused, and Place narrows every node of that runtime out with a runtime reason that names the keys. A cap_add replica therefore lands on a docker node, or pends with “service app-r0 cap_add: …” against each Mac, instead of failing at Up. Up runs the same refusal again before touching the engine.

2.7 Exit status: the exec helper is every container’s entrypoint

Section titled “2.7 Exit status: the exec helper is every container’s entrypoint”

The CLI reports no exit code and no finish time, and that kept jobs, restart: on-failure and service_completed_successfully off Mac nodes. The executor records both itself (SHC-1300), with the static exec helper the incus runtime uses (core/executor/incus/execenv).

  • The helper. A linux/arm64 build ships in the Mac install at lib/shc/exec-env/linux-arm64/exec-env under the install prefix (a container is a Linux micro-VM, so the build is linux/arm64 whichever Mac it is). Its directory is shared read-only into every container at /.shc-bin (--mount type=bind,...,readonly; apple shares directories, not single files) and the helper is the container’s --entrypoint. A Mac whose install lacks it starts nothing. The mount sits directly under / and not under /.shc because every exec runs this path while it still holds a capability the helper is there to take away (section 2.8): a root process in a container can rename /.shc and put another directory in its place, while a mount point cannot be renamed, unmounted, written or covered from inside (measured on apple 1.5.0).
  • The argv. The CLI can replace an image’s entrypoint but cannot put a program in front of it, so the executor reads what the image declares (container image inspect, the variant for the entry’s platform, linux/arm64 without one, and that variant exactly, its variant word included: linux/arm64/v9 is not the v8 the node holds, and the one equivalence is OCI’s own, arm64 with no variant is v8. An image the node does not hold, or holds for another platform only, is pulled with --platform first, and one with no such variant is refused) and hands the helper the argv docker would run: the compose entrypoint and command, each the image’s where the compose file gives none, and the image’s command dropped where the compose file gives an entrypoint. With a compose entrypoint the image is not read at all. The run is container run ... --entrypoint /.shc-bin/exec-env --cap-add CAP_NET_ADMIN <image> --supervise --keep-env --cap-drop CAP_NET_ADMIN -- <argv> (the capability and its drop are section 2.8’s). No compose sits between the projection and the engine here, so the executor does compose’s part itself: an entrypoint or command written as one string is split exactly as compose splits it (executor.SplitWords, compose’s own parser kept rule for rule), $$ is a literal $, and a word that would interpolate a variable is refused.
  • The environment is the runtime’s own: the image’s ENV with the --env-file over it. --keep-env tells the helper to hand its child exactly the environment it was started with and to read no staged file (on incus it builds docker’s environment from one, because Incus gives a first process its own).
  • What the helper does: starts the argv as its child, passes every signal it gets on to it, reaps whatever dies under it (it is process 1, so orphans are its to reap), waits, writes <exit code> <finish time> to /.shc/status/exit through a rename, and exits with that code: the command’s own, 128 plus the signal for one a signal ended, 126 or 127 for one it could not start. container stop -s <signal> signals the container’s first process, which is the helper, so the entry’s stop_signal and its whole grace reach the app. It keeps two signals for itself: SIGURG, which its own runtime uses, and SIGPIPE, which its own writes raise when an attach client has hung up (docker’s signal proxy leaves both out), so stop_signal: SIGURG and SIGPIPE are refused (SHC-3428). A container the CLI reads as stopping is still inside its grace: its death is sent once it has stopped.
  • The record’s home is a directory of the container’s own on the Mac, <state>/apple/status/<container>, shared read-write at /.shc/status. It is mode 0777 under a 0700 parent: a shared directory carries the Mac’s owner into the VM and the entry’s user is whoever the image or the compose file names, while nothing else on the Mac can reach it. macOS has no cheap way to cap a shared directory’s size, so unlike incus’s page of RAM this one is bounded only by what the executor reads from it: one regular file, opened without following a link, no further than a record is long. The executor empties the directory before every start it makes (a run, the start of a stopped container, the agent’s Start), the helper clears the record again as it starts, and a record whose file was written before the container’s last start does not read: that is the file’s own time on the Mac, not the time in the record, which is the VM’s clock. The directory goes with the container. A compose mount at or under /.shc or /.shc-bin is refused.
  • The last start. The runtime lists a container’s startedDate only until its API service next restarts: it reads its containers back as stopped with no start date (ContainersService loads them with startedDate: nil, and its exit handler never clears one), which is every container after the Mac reboots. The record is on disk and outlives that, so the executor keeps each container’s last start itself, in <state>/apple/status/.started/<container> (the 0700 parent, no container’s name): every start it makes, noted before it makes it, and any later one a listing shows. A record is read against the later of the listed start and the kept one. With neither, no record is read: whether one is this run’s cannot be told, and that is an unknown exit. So a job that finished before a reboot still reads its exit after it, and a dependent that waits on it starts. What is left open: a container started by hand past the executor, whose helper was killed before it cleared the earlier record, on a runtime whose API service restarted before the executor listed that start, reads the earlier run’s exit.
  • No record is an unknown exit, never a clean one: a helper that was killed writes none. A container stop kills what outlives its grace, so after one the executor writes 137 for a container that left no record, as docker reads a killed container.
  • Process 1: the trade-off, stated. The app was process 1 in its container and is now the helper’s child (with init: true it already was not). A signal it does not handle therefore takes its default action instead of being ignored as process 1’s are, which is docker’s --init behaviour and usually what an app wants; an app that inspects process 1 or relies on being it changes. There is no opt-out key. An image whose entrypoint insists on being process 1 (s6-overlay v3, which every linuxserver image is built on) exits at once under the helper; the executor watches a started container for a second, in Up and in the agent’s Start, and one that ended with a failure and “can only run as pid 1” in container logs is refused by name: X400788APLPID, “insists on being process 1 … it runs only on docker or podman”. Catalog apps known to need process 1 get an ordinary constraint keeping them off incus and apple once selectors land.
  • Not proven on a real Mac (nothing is installed on the build Mac; the stubbed CLI and a live test that skips are the proof here): the shape of container image inspect’s JSON (read as variants[].platform with its os, architecture and variant, and variants[].config.config.Entrypoint / Cmd), that a shared directory is writable by a non-root entry user and that a file written through it carries the Mac’s clock, not the VM’s, as its time, that a stopped container still lists its startedDate, that container logs carries the first process’s stderr, and what the helper’s exit does to the VM. Section 7.1 lists the checks.

2.8 TX checksum offload is off in every container

Section titled “2.8 TX checksum offload is off in every container”

A service VIP on a Mac is translated by pf (section 5.4): a dial is redirected to the replica’s pin and, on the replica’s own network, given the network’s host as its source. The reply comes back through both states. pf patches the TCP checksum field for each rewritten address as if the field held a whole checksum. A container’s virtio device offloads TX checksums, so the reply leaves the replica with only the pseudo-header sum in that field, nothing completes it on the way, and the client drops what arrives (InCsumErrors). Measured on a real Mac with tcpdump -vv inside both replicas (SHC-1989): the field received is exactly pf’s patch of the pseudo-header sum, and with the offload off in the guests every VIP and every name answers. A dial by pin is bridged or routed with no rewrite and was never affected.

So the executor turns the offload off itself, in every container, whoever its entry runs as:

  • Every container run adds CAP_NET_ADMIN (beside the helper’s own flags), unless the entry’s cap_add already holds it or ALL.
  • Right after every start it makes (a run, the start of a stopped container in Up, the agent’s Start) the executor runs the helper as root in the container: container exec -u 0 <name> /.shc-bin/exec-env --tx-checksum-off. The helper turns TX checksum offload off on every interface but loopback (SIOCETHTOOL, ETHTOOL_STXCSUM) and reads it back. The entry’s own user may be anyone, and a user holds no capability in an apple container, which is why this is the executor’s exec and not the first process’s work. A container that has already ended is no failure of its start; one still running with its offload on is, with the helper’s words.
  • The capability is taken back. As the container’s first process the helper drops it from its bounding, ambient, permitted, effective and inheritable sets before it starts the entry (--cap-drop). The CLI gives every exec whatever the run added and has no flag to hold one back, so every Exec runs through the helper (section 2.3), which drops it the same way unless the helper that supervises the entry is root and still has it in its bounding set: the entry that declared it. That helper is found in /proc by its argv, process 1 or the child of the runtime’s init under init: true; only its bounding set is trusted, which no process can grow.
  • A container run before this lacks the capability, so its spec digest differs and Up recreates it.

What this leaves, stated:

  • An entry whose first process is not root holds no capability, and cannot shrink its bounding set (that takes CAP_SETPCAP). There CAP_NET_ADMIN stays reachable through a set-uid binary in the image, as CAP_NET_RAW already is in every container.
  • An exec that starts in the instant before the supervising helper has dropped the capability keeps it.
  • TCP segmentation offload goes off with the checksum offload, so a bulk send from a Mac replica costs more CPU than it did.

2.9 A declared hostname is set with a capability the helper gives up

Section titled “2.9 A declared hostname is set with a capability the helper gives up”

The CLI has no --hostname, so the helper names the VM before the entry’s process: sethostname(2) for hostname, setdomainname(2) for domainname. Both need CAP_SYS_ADMIN, and Apple’s VM gives a container’s first process none unasked: on a real Mac a service with hostname: exited 125 in a loop, “the hostname cannot be set: operation not permitted” (SHC-1964). Measured on apple 1.5.0: root holds a capability the run adds in its permitted, effective and bounding sets, a user holds nothing whatever the run adds, and an exec as root holds what the run added.

  • The run of a service that declares hostname or domainname adds CAP_SYS_ADMIN beside the helper’s own flags, unless the service’s cap_add already holds it or ALL. A service that declares neither is run without it.
  • The helper drops it once the VM is named, the way it drops section 2.8’s capability (--cap-drop), so the entry’s process holds what the service asked for and nothing more. Every Exec sheds it too (--cap-shed), unless the entry holds it.
  • Root only. A first process that is not root cannot name the VM, so the service is refused by name, X400774APLKEY: a compose user before placement, an image’s user at start. cap_add and extra_hosts are refused the same way for the same reason.
  • A named container run before this never started. Its spec digest differs, so Up runs it afresh.

What this leaves is what section 2.8 leaves for its capability: an exec that starts in the instant before the supervising helper has dropped CAP_SYS_ADMIN keeps it, and under init: true Apple’s own init keeps it. The third case there does not arise here, since a first process that is not root is refused.

3. What the node agent does that the runtime does not

Section titled “3. What the node agent does that the runtime does not”

Ruling: health is defined once in compose, run by the runtime, read by the node agent. apple has no runtime health, so the agent runs it.

core/executor/healthcheck is runtime-neutral. A Spec is parsed from the compose healthcheck: against its closed key set (test, interval, timeout, start_period, start_interval, retries, disable; any other key is refused). CMD, CMD-SHELL and string forms are read; disable: true and [NONE] mean none. Zero values read as docker’s defaults: interval 30 s, timeout 30 s, start_interval 5 s, retries 3. The spec rides the container as labels, so a runner that finds a running container after a daemon restart knows what to probe without the compose file.

The Runner follows moby’s daemon/health.go: the next probe is due one interval after the previous one finished, start_interval instead while the check is starting inside start_period; a failure whose probe began inside start_period does not count while starting; retries consecutive failures after that mean unhealthy; one success means healthy at any time. The apple executor’s Events owns the runners today. modules/nodeagent exists on next, and the runner is still to move there, unchanged.

Gap, stated: docker runs an image’s own HEALTHCHECK when compose declares none, and a compose healthcheck: without test inherits the image’s test. apple exposes neither to the executor today, so a compose healthcheck without test is refused, and a service with no compose healthcheck is ready when running even if its image declares one. Follow-up: read the image config through container image inspect and run it.

restart: "no" is stamped by the render. The agent restarts a replica on EventDie or on a reconcile that finds it stopped. The exit code is the one the exec helper recorded (section 2.7), so a one-shot (shc.lifecycle=one-shot) that finished is told from one that failed, and restart: on-failure from a clean exit, as on docker. Up itself still leaves a stopped one-shot whose digest is current alone, whatever it exited with: rerunning a job is the agent’s decision, and a new revision (a new digest) recreates and reruns it. A stopped long-running container is a death.

Up gates starts the way docker compose up does (section 2.1, step 5): service_started orders, service_healthy waits for a healthy probe, restart: true recreates. service_completed_successfully waits until the dependency has stopped and takes only an exit of 0 from its record; one that failed or left no record fails the up, unless the dependency is required: false. A dependency on another node is not the executor’s: the projection only carries this node’s entries, and the agent’s pre-run check (cross-node depends_on targets are ready) gates the start.

Ruling: a fabric bridge per tenant per node. On apple the mapping is one vmnet network per tenant per node, the tenant’s slice of the node’s grant as its subnet, every deployment of that tenant on the node attached to it. vmnet isolates networks from each other, which gives “cross-tenant blocked” between networks. Port scoping inside a tenant’s network is pf’s job (section 5.4) and is unproven: whether VM-to-VM traffic on one vmnet bridge passes a pf-filtered hook at all is the open question. The render decides the mapping by what it declares in networks:; the executor creates what is declared.

The system tier (Traefik and forward-auth, the OpenObserve collector, the per-node resolver) must reach every tenant’s network. The host reaches container addresses, so a system-tier process on the host does; a system-tier container on its own vmnet network reaches another network only through host routing, which is unproven. Proposed, for a ruling: on a Mac node the system tier runs as host processes supervised by shcd, not as containers, until host routing between vmnet networks is proven.

shcd runs as a system launch daemon: /Library/LaunchDaemons/com.selfhostedcloud.shcd.plist, RunAtLoad, KeepAlive, root, state under /var/lib/shc as on Linux, logs to /var/log/shc. shc node install writes the plist and runs launchctl bootstrap system on it; launchctl kickstart -k system/com.selfhostedcloud.shcd is the restart.

The apiserver has one root-owned app root, /var/lib/shc-apple, and runs in the system domain only. The app root sits beside SHC’s state home, never inside it: shc init on a node with no identity, and shc init --force, remove /var/lib/shc whole, and an app root under it loses its kernel, images and network plists to the first genesis (SHC-1859). The node never starts a per-user instance: a gui/<uid> instance would answer the user’s own Mach lookups first and shadow the system one. The sequence (section 8 spells out the commands):

  1. container system start --app-root /var/lib/shc-apple as root writes /var/lib/shc-apple/apiserver/apiserver.plist with CONTAINER_APP_ROOT set, then its own bootstrap fails (#2008).
  2. That plist is installed as /Library/LaunchDaemons/ com.apple.container.apiserver.plist, root:wheel 0644, and bootstrapped with launchctl bootstrap system from there. A plist bootstrapped from anywhere else survives logout but is not loaded at boot; one in /Library/LaunchDaemons is.
  3. container system start --app-root /var/lib/shc-apple --enable-kernel-install runs again; its lookup reaches the system instance, and it installs the init filesystem and the kernel.
  4. Verification reads the identity, not just liveness: launchctl print system/com.apple.container.apiserver shows it running with CONTAINER_APP_ROOT=/var/lib/shc-apple, and no gui/<uid>/com.apple.container.apiserver is loaded.

shcd does not set CONTAINER_APP_ROOT: the CLI reaches the system-domain apiserver over its Mach service, and the app root is the one the apiserver’s plist names. shcd checks container system status on start, and refuses work until it answers. When #2058 lands, container system start bootstraps the system domain itself, which removes the failing first bootstrap; installing the plist under /Library/LaunchDaemons stays, since #2058 does not install anything that starts at boot.

Ruling (shc-mesh-two-wireguard-tunnels): the darwin shcd embeds wireguard-go in-process, behind the same control code as the Linux netlink applier. On darwin wireguard-go only accepts utun or utunN as an interface name, so mesh0 and shc0 are the two tunnels’ roles; each maps to the utunN the kernel allocates when shcd opens it, and every route and pf rule is rendered against the allocated name, re-read on each start. The ruling that the two-tunnel mesh lands after the cutover, with a Mac node a single-node cluster until then, was overtaken: the mesh is on next and a Mac node joins over it, as built below. Section 7.1 is still its single-node acceptance.

As built (SHC-1287, ce modules/fabric): UserspaceApplier is the same fabric.Applier the netlink applier is, over the darwin host. The peer diff and the order of an apply are shared with Linux; what differs is where the writes land.

  • The address is set point to point to itself, which is how a utun takes one. macOS gives such an interface no route for its prefix, so the applier adds the pool’s route onto the utun itself. An address with no peer then dies in the device, as on Linux, instead of leaving on the default route.
  • Routes are written on a PF_ROUTE socket with the utun as gateway. A per-peer MTU is a locked route MTU; macOS rewrites an unlocked one to the interface’s MTU whenever that changes.
  • macOS compares a packet with the interface MTU and reads a route’s MTU only for TCP’s segment size (xnu ip_output). So the utun runs at the smallest peer MTU, where Linux runs the device at the largest and lets each route cap its own peer. A Mac node sends every peer packets sized for its narrowest path.
  • A route is the applier’s when the applier added it, at the MTU it added it. The utun and its routes cannot outlive shcd, so that memory is the whole set; a route an operator puts on the utun is left alone.
  • macOS routes a packet for the node’s own mesh address into the utun. shcd hands it straight back, so the node reaches its own listeners (rqlited, the internal API) on its mesh address.
  • Nothing finds the device by interface name. mesh0 is found by the address inside the mesh range, or by asking the applier.
  • The tunnels live inside shcd. When shcd exits, for any reason, the kernel removes the utun and every route on it; nothing is left to clean up. On start shcd brings mesh0 back from its cache before the store answers, and peers handshake again within seconds.

5.3 Routing from a utun into a vmnet network: UNPROVEN, the multi-node gate

Section titled “5.3 Routing from a utun into a vmnet network: UNPROVEN, the multi-node gate”

A peer’s packet for a replica arrives on the shc0 utun with a destination inside a vmnet subnet. macOS must forward it: net.inet.ip.forwarding=1 and a route for the subnet onto the vmnet bridge interface. Whether vmnet’s NAT network accepts forwarded traffic from a utun, whether the reply routes back through the utun rather than the NAT, and whether --internal (host-only) networks behave differently, has not been measured. Nothing multi-node works on apple without it (section 7.2). If it fails, the fallback is a per-node userspace proxy on the host for inbound fabric traffic, which this design does not want.

5.4 pf as the policy compiler’s second output

Section titled “5.4 pf as the policy compiler’s second output”

ce modules/fabric/policy compiles the tenant policy to an nftables ruleset. On darwin the same model compiles to a pf ruleset loaded into one anchor:

  • /etc/pf.conf carries anchor "com.shc" after Apple’s own anchor "com.apple/*"; shc node install adds it and loads the main ruleset with pfctl -f /etc/pf.conf, and shcd re-checks the line on every start, since a macOS update can rewrite /etc/pf.conf.
  • shcd enables pf with pfctl -E and holds the reference token it prints for its lifetime (pfctl -X <token> on stop), so it neither depends on nor disturbs anyone else’s enable.
  • Each revision is one file in the clusterfs fabric section, loaded with pfctl -a com.shc -f <file> (the rules go directly into com.shc, so the main ruleset’s anchor "com.shc" evaluates them).
  • The rules are the nftables model’s accept classes, by observed address (section 2.5): within a tenant, a replica’s declared ports only; cross-tenant blocked except the system tier; the node plane (mesh, rqlite, internal API) unreachable from any vmnet interface. The anchor is default-deny for vmnet traffic, so an address no revision names has no permission.
  • Retiring an address is ordered: when a replica leaves, the revision without it is loaded and its states are killed (pfctl -k <addr> and pfctl -k 0.0.0.0/0 -k <addr>) before the agent starts another container on that network, because loading new rules does not end states already established.
  • A revision that removes an allowance between addresses that both stay (a port no longer declared, a policy change) kills the states between those two hosts (pfctl -k <src> -k <dst>) once it is loaded. macOS pfctl kills by host pair, not by port, so connections the new revision still allows are cut too and re-establish.
  • IPv6: every vmnet network carries it, and node reports carry IPv4 only today, so the anchor blocks all IPv6 to and from vmnet interfaces, link-local to the node included, until reports and policy are dual-stack.

Unproven, and not promised until measured: that pf sees and filters traffic between two VMs on the same vmnet bridge. Isolation between vmnet networks alone would make a cross-tenant test pass without pf, so the proof (section 7.1) uses same-tenant allowed and denied ports, and reads the pf rule counters (pfctl -a com.shc -vsr) to show pf, not vmnet, made the decision. Over IPv6 the anchor blocks everything, so there the proof is that nothing at all gets through.

The node’s CA goes into the system keychain: security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain <ca.crt>, run by shc node install, so the host’s own clients trust estate hostnames. Containers get the resolver as --dns <ip> with --dns-search the stack and environment zones, as on every runtime. The resolver listens on a host address every vmnet network routes to; which address (each network’s gateway, or a host alias) is proven in section 7.1.

An apple named volume is an ext4 image under the app root, mounted as a block device inside one VM; the host cannot read it, and mounting the image in a second VM while the first has it mounted is unsafe concurrent access to one filesystem. So a capture needs exclusive access. Volumes are exclusive to one entry (section 2.1), so that entry’s replica is the only holder:

  • Default: stop the replica, run the restic helper (container run --rm -v <vol>:/data <restic image> ...) against the volume, start the replica. The whole sequence runs under a maintenance hold on the replica that the node agent honours: no restart on its death, no reconcile and no Up for it, until the helper has exited and the replica is started again. The backup window is a stop, like the stopped-backup path elsewhere.
  • To prove, for a capture without a stop: freeze the filesystem inside the owning VM (container exec <id> fsfreeze -f <mount>), APFS-clone the image file (cp -c), thaw, and run the helper against the clone.

Bind mounts of host directories work (virtiofs) and are what the clusterfs section and app files use; the host reads them directly.

  • Replicated volumes: no LINSTOR or DRBD; out of the first cut anyway (ruling 2026-10-01). Local volumes and restic. A restore that names --runtime apple lands only on a node that runs the apple executor, and is refused elsewhere (X400955RTMDRV). A replica on a local volume stays pinned to its node when that node is lost, as on Linux.
  • GPU: nothing passes a GPU into the VM.
  • Anything needing the Linux host kernel: the refused keys below, kernel WireGuard (wireguard-go instead, slower), nftables (pf instead), host networking, device passthrough, cgroup tuning.
  • Exit codes and events: a death is learned by polling, without a code.
  • An image’s own HEALTHCHECK (section 3.1).
  • amd64 images run under Rosetta; only arm64 Macs on macOS 26 or newer are nodes at all.
  • Vote in raft. A Mac node joins as a non-voter (ruling 2026-10-02, pending Clay’s confirmation on SHC-1287): its tunnel dies with every shcd restart, so a Mac voter would leave raft on each daemon restart or upgrade. Promoting one is refused with a coded error.
  • Keep its tunnels across a daemon restart. mesh0 and shc0 are down from shcd’s exit until it has rebuilt them from the cache, so the node is off the node plane and its replicas lose fabric reach for those seconds. A Linux node’s kernel devices stay up through a restart.

apple.Refuse(project, compose) answers X400774APLKEY naming every refused path with its reason, and then runs the translation, whose refusals are X400775APLCMP and X400772EXEENV. Placement reaches it through executor.Refusals (section 2.6), and Up runs it before touching the engine. A path is in exactly one class; a path in none is refused as unknown, so a new compose key is honoured on purpose or not at all. Checked against compose-spec schema/compose-spec.json at commit 914ec15d1fa498969c0df5c1d672306db3256089: every service, network, volume, mount, port, deploy and top-level key it defines is in one class below.

Honoured by the executor: image, command (not empty), entrypoint (not empty), environment, env_file (path, list, {path, required, format: raw}), labels (not the reserved keys), ports (only long form with mode: host and published; keys target, published, host_ip, protocol, mode, name, app_protocol), expose, volumes (short form with ro/rw; long form type bind, volume or tmpfs with source, target, read_only, tmpfs.size, tmpfs.mode written in octal), tmpfs, dns, dns_search, dns_opt, user, working_dir, read_only, shm_size, init, platform, stdin_open, tty, cpus, mem_limit, deploy.resources.limits.cpus, deploy.resources.limits.memory, networks (list or map; per attachment mac_address), mac_address, stop_grace_period, stop_signal, healthcheck (section 3.1), depends_on (condition service_started or service_healthy, required, restart as a boolean), pull_policy always, missing or if_not_present. Networks: name, ipam.config[].subnet (one IPv4 and one IPv6), ipam.driver: default, internal, labels, driver: bridge, attachable: true, enable_ipv4: true, enable_ipv6: true. Volumes: labels, driver: local. Top level: services, networks, volumes, name, version, and x-* everywhere.

Owned elsewhere: of the deploy paths the render’s closed set lets through, every one but the two honoured limits above, namely deploy.replicas (placement), deploy.mode: replicated (the render), deploy.endpoint_mode: dnsrr (the resolver), deploy.restart_policy.condition (the node agent), deploy.update_config and deploy.rollback_config (the agent’s rollout), and deploy.resources.reservations.cpus and .memory (placement). Also restart (the node agent; the render stamps “no”), attach and profiles (mean nothing on a node). A deploy path neither honoured above nor listed here is refused below.

Refused because the CLI has no such capability:

pathwhy
sysctls, privileged, devices, device_cgroup_rules, security_opt, userns_mode, cgroup, cgroup_parent, pid, ipc, uts, oom_kill_disable, oom_score_adj, pids_limit, cpu_period, cpu_quota, cpu_shares, cpu_count, cpu_percent, cpu_rt_period, cpu_rt_runtime, cpuset, mem_reservation, mem_swappiness, memswap_limit, blkio_config, storage_opt, isolation, gpus, group_add, deploy.resources.limits.pids, .reservations.devices, .reservations.generic_resourcesa host-kernel or device setting the CLI has no flag for
deploy.labels, deploy.placement, deploy.mode other than replicated, deploy.endpoint_mode other than dnsrr, deploy.restart_policy keys but condition, any other deploy keythe render refuses them; a projection entry is one replica on one node
network_mode (any value)a container cannot share another’s network namespace or the host’s; a deployment is its network
hostname, domainname on a service whose first process is not rootthe helper names the VM, and only root holds the capability that takes (section 2.9)
extra_hosts, links, external_links, attachment aliasesno --add-host, no alias; the resolver answers names
attachment ipv4_address, ipv6_address, link_local_ips, gw_priority; ipam.config[].ip_range, .gateway, .aux_addressesapple assigns addresses inside the declared subnet; the subnet is the pin
attachment interface_name, priority, driver_opts; network driver_opts, ipam.options, any other drivervmnet takes none of them
network attachable: false, enable_ipv4: false, enable_ipv6: falseevery apple network is attachable and carries both families
volumes_from; network and volume externala VM cannot mount another VM’s volumes; networks are isolated, a shared one is declared by name in both
volume driver_opts, any driver but local; mount bind.propagation, bind.create_host_path, bind.recursive, bind.selinux, volume.nocopy, volume.subpath, volume.labels, image, consistency, any type but bind, volume, tmpfs; short-form options but ro/rwvirtiofs and ext4 images have none of them; the render creates host paths
volume name; a volume two entries mountan ext4 image takes one VM at a time (section 2.1)
ports entries but long form mode: host with publishedthe render leaves only those in a projection (section 2.2)
loggingapple keeps container logs itself; there is no log driver
annotationsthe CLI has no annotation flag
post_start, pre_stop, pre_startlifecycle hooks and init containers; the executor runs none
models, provider (and top-level models)need Docker’s model runner or provider plugins
credential_speca Windows container setting
developcompose watch is a development loop
use_api_socketno container reaches an engine socket
pull_policy: neverrun fetches a missing image and cannot be stopped from doing so
pull_policy refresh, daily, weekly, every_<d>; pull_refresh_afterapple has no pull schedule
command: [] or "" without an entrypoint, entrypoint: [] or ""an entry with no command or entrypoint of its own runs its image’s; clearing one is not built
an entrypoint or command word that would interpolate a variablethe projection is already rendered; $$ is a literal $
a volume or tmpfs at or under /.shc or /.shc-binthe executor shares the exec helper and the entry’s exit record there (section 2.7)
stop_signal: SIGURG or SIGPIPEthe helper keeps these two for itself (section 2.7)
an image with no variant for the entry’s platform (linux/arm64 without one)another platform’s entrypoint is not this one’s (section 2.7)
an image whose entrypoint insists on being process 1 (s6-overlay v3, every linuxserver image)the helper that records the exit holds that place; refused at start, X400788APLPID (section 2.7)
an environment value with a line break or NULan env file carries one line per variable
a bare NAME line in an env filecompose reads it as unset, and the CLI cannot remove an image’s variable
service_healthy on a service with no healthcheck; a required dependency the projection does not carrynothing to wait for; a dependency on another node is the agent’s pre-run check
healthcheck without test; any key outside its setsection 3.1
env_file format other than raw; an env file, inline environment or label value that interpolatesthe projection is already rendered
labels com.docker.compose.project, .service, .network, .volume, shc.spec.digest, shc.stop.*, shc.health.*, shc.volumesthe executor writes them

Refused by SHC’s choice, not for lack of a flag:

pathwhy
cap_add, cap_drop, ulimitsthe CLI has --cap-add, --cap-drop and --ulimit, applied inside the container’s own VM rather than on a shared host kernel. SHC-1200’s acceptance criteria refuse them; honouring each is a one-line flag if Clay rules so
runtimenames a docker OCI runtime; apple’s --runtime takes its own handler, and the node’s runtime is placement’s
pull_policy: build, build, container_name, configs, secrets, extends, label_file, scale; top-level configs, secrets, include, jobsthe render’s: they never reach a projection

7.1 A single-node Mac (the acceptance before any mesh)

Section titled “7.1 A single-node Mac (the acceptance before any mesh)”
  1. The bootstrap script of section 8 on Clay’s Mac, then the live harness: a catalog app (whoami) and an nginx replica with a healthcheck come up through the executor, the probe reports healthy through container exec, the host reaches whoami at its observed address inside the declared subnet, and Down removes everything. This closes acceptance criterion 5.
  2. The apiserver bootstrapped into the system domain answers container system status in a non-login session and after a reboot, with no user instance loaded.
  3. The resolver, as a host process, answers from inside a container on a tenant network, at the address chosen in section 5.5.
  4. The system tier as host processes (section 4) reaches every tenant network: Traefik routes to a replica, the collector scrapes one.
  5. pf (section 5.4): the anchor loads; on one tenant network a declared port is reachable and an undeclared one is not, with the com.shc rule counters moving; a second tenant’s network and the node plane are unreachable from a container. That is over IPv4; over IPv6 every destination is unreachable from a container (another replica, another tenant, the node plane, the node’s link-local address), as the anchor blocks it all. Then address reuse: remove a replica holding an established connection, confirm the connection is cut and its state gone, and start a different service that is given the same address; it must hold none of the old service’s permissions.
  6. The healthcheck runner’s container exec cost at a 5 s interval with twenty replicas. 6a. Exit status (section 2.7), with the linux/arm64 helper from the Mac archive in place. SHC_CE_TEST_REAL_APPLE=1 runs three live tests in tests/_ce/core/executor/hosttest: TestAppleIsAHost (the conformance run, with exit codes), TestAppleRecordsHowEachEntryEnded (exits 3, 5 as a non-root user, 7 from a two-line command, 0 from a job a dependent waits on, 42 on the entry’s own stop signal, 137 for one killed at the end of its grace, read through Ps, Labelled and Events) and TestAppleRefusesAnImageThatInsistsOnBeingProcessOne (a linuxserver image fails its start with X400788APLPID). Then by hand: what container image inspect alpine:3.22 prints (the executor reads variants[].platform and variants[].config.config.Entrypoint/Cmd), which process is 1 inside a container started with and without init: true, that the status directory shared from the Mac’s disk is writable by a non-root entry user, and that the VM stops when the helper exits.
  7. The volume image path under /var/lib/shc-apple, the stop-and-capture restic backup, then the freeze-and-clone capture.
  1. mesh0 on a real utun (SHC-1287). One command, as root, on a Mac that routes nothing overlapping 10.192.0.0/11; it refuses to start otherwise and removes everything it made: tests/_ce/modules/fabric/realmac_macos_test.go, built as the user and run with SHC_MESH_REAL_MAC=1. It proves, against a WireGuard peer in the same process over loopback: the utun is up at the smallest peer MTU with the mesh address; the pool’s route and each peer’s /32 are on it; a peer route’s MTU survives a change of the interface MTU; a matching apply writes nothing; the Mac’s kernel reaches the peer with the mesh address as source and nothing larger than the interface MTU arrives; the peer reaches a socket on the Mac; the Mac reaches its own mesh address; Remove takes the interface and its routes; and a process that dies without removing the device leaves nothing behind.
  2. A Mac node against Linux nodes: it enrols, takes its mesh address, joins as a non-voter and answers internal RPC over mesh0; shc cluster mesh diagnose on a Linux node shows a fresh handshake with it. Restart shcd on the Mac and measure the gap until the handshake is fresh again.
  3. Published ports on a Mac node (SHC-1364): the steps are in that ticket’s design note, and need a container network to forward to.
  4. Routing from a utun into a vmnet network and back (section 5.3), for a nat network and an internal one.

8. Install and bootstrap on a Mac (Clay’s step)

Section titled “8. Install and bootstrap on a Mac (Clay’s step)”

Installing apple/container needs an administrator, so the agent does not do it. Both steps are fail-stop POSIX sh scripts: save each and run it with sh. The bootstrap is for 1.5.0 and the #2008 workaround, with one root-owned app root and no per-user instance; it installs the kernel and verifies as root from an ordinary shell, proves the instance by running a disposable container, and can be rerun: an already loaded system service is reused only when its LaunchDaemon plist is in place, root:wheel 0644 and identical to the one under the app root, its app root is exactly this one, and the loaded program is exactly the plist’s. The app root is /var/lib/shc-apple, outside the state home that shc init removes, so the bootstrap may run before or after the node’s genesis.

No step runs under launchctl bsexec 1. On macOS 27.0 a process started that way has no launchd manager (launchctl managername prints “Could not get manager name.” and exits 153), so container system start fails, container system status reports the running service as not registered, and container run fails with “XPC connection error: Connection invalid”. A root process in an ordinary shell reaches the system-domain apiserver’s Mach service, and the apiserver refuses any other uid (“unauthorized request”). container system status asks the caller’s own launchd domain, so from a shell it is not a check of the system instance; launchctl print system/com.apple.container.apiserver is. Measured on a real Mac on 2026-10-07 (SHC-1200).

cat > ~/shc1200-bootstrap.sh <<'SCRIPT'
#!/bin/sh
set -eu
label=com.apple.container.apiserver
root=/var/lib/shc-apple
uid=$(id -u)
daemon=/Library/LaunchDaemons/$label.plist
field() { sudo launchctl print "system/$label" | sed -n "s/^[[:space:]]*$1 =>* *//p" | head -n 1; }
# 1. install the signed package (Apple Silicon, macOS 26+)
curl -fsSLo /tmp/container-1.5.0-installer-signed.pkg \
https://github.com/apple/container/releases/download/1.5.0/container-1.5.0-installer-signed.pkg
sudo installer -pkg /tmp/container-1.5.0-installer-signed.pkg -target /
bin=$(command -v container)
# 2. no per-user instance may exist anywhere; it would shadow the system one
container system stop >/dev/null 2>&1 || true
for d in "gui/$uid" "user/$uid" gui/0 user/0; do
if sudo launchctl print "$d/$label" >/dev/null 2>&1; then
echo "STOP: $label is loaded in $d; sudo launchctl bootout $d/$label, then rerun" >&2
exit 1
fi
done
sudo mkdir -p "$root"
sudo chmod 700 "$root"
# 3. the system-domain service: reuse one already loaded with this app root,
# else write the plist (its own bootstrap fails, #2008), clear whatever that
# registered, install it as a LaunchDaemon and bootstrap the system domain
if sudo launchctl print "system/$label" >/dev/null 2>&1; then
stale="sudo launchctl bootout system/$label, then rerun"
sudo cmp -s "$daemon" "$root/apiserver/apiserver.plist" ||
{ echo "STOP: $daemon is missing or differs from the app root's plist; $stale" >&2; exit 1; }
[ "$(stat -f '%Su:%Sg %Lp' "$daemon")" = "root:wheel 644" ] ||
{ echo "STOP: $daemon is not root:wheel 0644; $stale" >&2; exit 1; }
[ "$(field CONTAINER_APP_ROOT)" = "$root" ] ||
{ echo "STOP: system/$label runs with another app root; $stale" >&2; exit 1; }
prog=$(sudo plutil -extract ProgramArguments.0 raw "$daemon")
[ "$(field program)" = "$prog" ] ||
{ echo "STOP: system/$label does not run $prog; $stale" >&2; exit 1; }
echo "reusing system/$label"
else
sudo "$bin" system start --app-root "$root" --disable-kernel-install --timeout 5 || true
for d in "gui/$uid" "user/$uid" gui/0 user/0; do sudo launchctl bootout "$d/$label" 2>/dev/null || true; done
sudo install -m 644 -o root -g wheel "$root/apiserver/apiserver.plist" "$daemon"
sudo launchctl bootstrap system "$daemon"
fi
# 4. install the kernel through the system instance (never `system start` again:
# from a shell it registers a second copy in user/0)
sudo "$bin" system kernel set --recommended
# 5. verify: system domain, this app root, no user instance, and a container runs headless
sudo launchctl print "system/$label" | grep -E 'state = running' >/dev/null
[ "$(field CONTAINER_APP_ROOT)" = "$root" ]
sudo cmp -s "$daemon" "$root/apiserver/apiserver.plist"
[ "$(stat -f '%Su:%Sg %Lp' "$daemon")" = "root:wheel 644" ]
for d in "gui/$uid" "user/$uid" gui/0 user/0; do
if sudo launchctl print "$d/$label" >/dev/null 2>&1; then echo "SHADOWED by $d" >&2; exit 1; fi
done
sudo "$bin" list --all >/dev/null
sudo "$bin" run --rm docker.io/library/alpine:3.20 /bin/true
echo "BOOTSTRAP OK"
SCRIPT
sh ~/shc1200-bootstrap.sh

The live proof runs the build-tagged harness tests/_ce/core/executor/apple/live_test.go as root in the same system bootstrap context, through a wrapper the script writes for go test -exec, from worktrees at the ce and shc merge commits named on SHC-1200. It fails unless the log holds both --- PASS: TestLiveAppleRuntime and make exit 0; a missing PASS line means the test did not run.

cat > ~/shc1200-live.sh <<'SCRIPT'
#!/bin/sh
set -eu
ce_ref=$1
shc_ref=$2
base=$HOME/Projects/bitspur/selfhosted-cloud
log=$HOME/shc1200-live.log
wrap=$HOME/shc1200-exec.sh
git -C "$base/ce" fetch corp
git -C "$base/shc" fetch corp
tree() {
want=$(git -C "$1" rev-parse --verify "$3^{commit}")
if [ -d "$2" ]; then
repo=$(cd "$1" && cd "$(git rev-parse --git-common-dir)" && pwd -P)
have=$(cd "$2" && cd "$(git rev-parse --git-common-dir)" && pwd -P)
self=$(cd "$2" && pwd -P)
top=$(cd "$(git -C "$2" rev-parse --show-toplevel)" && pwd -P)
head=$(git -C "$2" rev-parse HEAD)
dirty=$(git -C "$2" status --porcelain)
if [ "$have" != "$repo" ] || [ "$top" != "$self" ] || [ "$head" != "$want" ] || [ -n "$dirty" ]; then
echo "STOP: $2 exists but is not a clean checkout of $3 from $1" >&2
exit 1
fi
else
git -C "$1" worktree add --detach "$2" "$want"
fi
}
tree "$base/ce" "$HOME/wt/SHC-1200-live-ce" "$ce_ref"
tree "$base/shc" "$HOME/wt/SHC-1200-live-shc" "$shc_ref"
test -f "$HOME/wt/SHC-1200-live-ce/core/executor/apple/apple.go"
test -f "$HOME/wt/SHC-1200-live-shc/tests/_ce/core/executor/apple/live_test.go"
printf '#!/bin/sh\nexec sudo /usr/bin/env PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin CONTAINER_APP_ROOT=/var/lib/shc-apple "$@"\n' > "$wrap"
chmod 755 "$wrap"
cd "$HOME/wt/SHC-1200-live-shc"
sudo -v
rc=0
SHC_CE_DIR=$HOME/wt/SHC-1200-live-ce SHC_CE_STORE_BASELINE_REF=corp/next \
CE_PKGS=./core/executor/apple CE_RUN='^TestLiveAppleRuntime$' \
CE_GOFLAGS="-tags=applelive -exec=$wrap -v -timeout=20m" make test/ce > "$log" 2>&1 || rc=$?
echo "make exit $rc" >> "$log"
if [ "$rc" -ne 0 ] || ! grep -q -- '--- PASS: TestLiveAppleRuntime' "$log"; then
echo "PROOF FAILED, see $log" >&2
exit 1
fi
echo "PROOF OK: $log"
SCRIPT
sh ~/shc1200-live.sh <ce merge sha> <shc merge sha>

~/shc1200-live.log goes on SHC-1200: every container view, the Ps rows, the health events and whoami’s answer are in it.

9. Deltas owed to the simplification proposal

Section titled “9. Deltas owed to the simplification proposal”

For the SHC-1198 owner of 2026-09-30-new-line-simplification.md:

  • §3.1 “Runtime becomes a node fact (a node runs Docker or Podman)”: a node runs docker, podman, incus or apple.
  • §4.7 “One interface, two implementations”: four; executor.New consults a registry so a runtime in its own package can answer, and an executor may implement executor.Refuser.
  • §4.7 Container.ExitCode: apple reports what the exec helper recorded, and ExitUnknown only where it left no record (section 2.7).
  • §4.1 placement: ReplicaDemand.Refused keeps a replica off a runtime that refuses its compose keys, with the refusal as the node’s reason.
  • §3.8 restarts: on apple the agent reconciles desired replicas against Ps rather than trusting polled events alone (sections 2.4 and 3.2); a finished one-shot is told from a crash by the exit the helper recorded.
  • §3.10 compose keys: on apple a volume is exclusive to one entry; the render projects a volume several services share as a host bind there.
  • §5 feature parity: “healthcheck” is agent-run on apple; an image’s own HEALTHCHECK is a gap there until the follow-up in section 3.1.