Skip to content
SHC Docs

Release lines and backports

SHC has two lines (SHC-1093, decided on SHC-1092). release/v0.0 in shc and ce is the old line. The live estates take debs, providers and the catalog only from it. main is the new line and versions as 0.1. One .gitlab-ci.yml serves both, and SHC_LINE in its variables: block says which line a tree is.

PipelineLineWrites
push to release/v0.0, tag v0.0.*oldgeneric/*/0.0.0-latest, selfhostedcloud.org/apps/index.yaml and apps/infra/index.yaml, the Pulp copies at downloads.selfhostedcloud.org/shc-apps, every image’s :latest. The site sync writes only the catalog, the provider mirrors and the frozen modules, never the docs.
push to main with SHC_LINE: '0.1', tag v0.1.*newgeneric/*/0.1.0-latest, selfhostedcloud.org/apps/0.1/, every image’s :0.1.0-latest, snapshots versioned 0.1.0-SNAPSHOT-<sha>, and the docs. The site sync holds back the old line’s paths. No Pulp, no deploy trigger, and no app packages or catalog yet: generic/<app> is one namespace, and the old line’s prune and index would read new-line revisions.
push to main with SHC_LINE: '0.0'oldeverything, as before the split. Only valid until release/v0.0 exists.

publish/line-guard runs before publish/host-guard on every publishing pipeline and refuses one whose resolved names cross lines. It also refuses main while main still says SHC_LINE: '0.0' and release/v0.0 exists, so a forgotten flip stops publishing instead of overwriting the estates’ channels. tests/unit/ciline checks the same rules statically.

  1. Fix it on main first when the code still exists there, through the usual lane/SHC-<n>-<slug> MR.

  2. After it merges, cherry-pick it onto the old line on a lane named for the same ticket with a -v0.0 suffix, and open the MR against release/v0.0:

    Terminal window
    git fetch corp
    git worktree add ~/wt/SHC-<n>-<slug>-v0.0 -b lane/SHC-<n>-<slug>-v0.0 corp/release/v0.0
    cd ~/wt/SHC-<n>-<slug>-v0.0
    git cherry-pick -x <main-sha>...
    git push -o merge_request.create -o merge_request.target=release/v0.0 corp lane/SHC-<n>-<slug>-v0.0
  3. Code the new line deleted is fixed on release/v0.0 directly, with the same lane naming and target. Say in the MR that there is nothing to fix on main.

  4. A ce fix goes the same way in ce. On an shc -v0.0 lane, make ce/latest pins the head of ce’s release/v0.0, never ce main.

Never merge release/v0.0 into main or the reverse. The two lines share history only up to the cut.

The CI needs no edit at the cut. The cut is: protect the branch, push it in ce and shc, move the daily deploy schedule to it, seed the new line’s builder tags, and merge the one-line flip of SHC_LINE on main. The exact commands for release/v0.0 are on SHC-1093.

ce’s schedule 26 is the release gate: it runs every suite on ce next’s tip against the shc next commit that pins it. The lane that lands a merge train plays it the moment it declares next green, on the landed pair:

Terminal window
curl -fsS -X POST -H "PRIVATE-TOKEN: $GITLAB_TOKEN" \
https://git.corp.bitspur.com/api/v4/projects/10/pipeline_schedules/26/play

The release lane’s R1 then only confirms that pipeline ran on the final pair, and plays the schedule again if anything landed after it. Started at R1, the gate waited for the release lane to get there: 55 minutes between the landing and the gate for v0.1.18 (SHC-3369).

Past guard, build and lint, the gate runs the Go suites (next/test/shc, next/test/unit, next/test/ce), make test/scripts’s bats suites (next/test/scripts) and the image check. R1g stops the cut on any job that is not green.

Every release must run the app suites after R7 reads PUBLISHED and before any estate rolls to it. The release lane runs them by hand until the new line is done, then the Mac takes them back as a CI runner (FZ-7289). A not ok row stops the roll until a fix release or main’s written ruling on that row.

Use the release lane’s existing cell kit to create one docker node and one incus node, named shc-test-* and labelled. Record their ids in a ledger at create and reap them by those recorded ids when the suites end.

Install the released deb from Based’s apt repository, never a lane deb, with Recommends so bats comes with it. A --no-install-recommends node has no bats and every suite dies at once with X500010BATSNF. The kit’s prep initialises each node as a single node; then enable the container broker. The suites’ direct rows run in job containers and refuse X403985CBROFF without that opt-in.

Run the suites on both nodes from a checkout of the release tag. The deb ships only the built-in postgres app under /usr/lib/shc/apps. Set ver to the released version and node to the ledger’s node label; keep a TAP file per suite and node:

Terminal window
apt-get install --install-recommends "shc=1:$ver"
shc config set container_broker.enabled true
shc test run /usr/lib/shc/apps/postgres >"postgres-$node.tap"
shc test run apps/standalone/mail >"mail-$node.tap"
shc test run apps/flagship/mailcow >"mailcow-$node.tap"
shc test run apps/flagship/netbird >"netbird-$node.tap"
LANG=en_US.UTF-8 shc test run apps/integrated/dependency-track >"dependency-track-$node.tap"

Attach every TAP file, the deb version from dpkg-query -W shc, and the node ids to the release card. Jellyfin needs a GPU node to deploy, so its suite is outside this set: its rows skip on these nodes, and a skip proves nothing. These runs prove FZ-6662, FZ-6344, FZ-6410, FZ-6442, FZ-6417 and FZ-6399 at every release.

An estate’s roll MR moves the provider with the deb

Section titled “An estate’s roll MR moves the provider with the deb”

Each estate rolls to a release in one ops MR. That MR moves four things to the release together:

  • the deb, local.version in the root’s shc.tf;
  • module core;
  • the selfhosted-cloud/shc provider pin in the root’s versions.tf;
  • that provider’s entry in the root’s .terraform.lock.hcl.

A provider older than the shcd it talks to can misread an answer whose shape changed, and then every plan of the root fails refresh. Ogden’s v0.1.22 roll (bitspur/ops!554) moved the deb and module core but held the provider at 0.1.21. Provider 0.1.21 reads shc -o json backup schedule ls as {rows, refused}, shcd 0.1.22 prints a bare array, and every untargeted plan failed on the root’s six backup schedules until bitspur/ops!563 moved the pin (SHC-3667). A change a release holds back doesn’t hold the provider back with it: the tag builds its own provider, and the roll takes that one.

Lock the provider’s entry alone. Delete the provider "registry.opentofu.org/selfhosted-cloud/shc" block from the root’s .terraform.lock.hcl, then run this from the ops checkout (Ogden’s platform root here):

Terminal window
export TF_CLI_CONFIG_FILE=$PWD/gorillaservers/ogden/platform/registry.tfrc
tofu -chdir=gorillaservers/ogden/platform providers lock \
-net-mirror=https://selfhostedcloud.org/terraform/0.1/ \
-platform=darwin_arm64 -platform=linux_amd64 \
registry.opentofu.org/selfhosted-cloud/shc

The lock’s diff is that one entry: the new version and its hashes. With the old entry still in place, providers lock refuses the new version, and tofu init -upgrade gets past that only by adding hashes to every other provider’s entry too.