Skip to content
SHC Docs

Origin repos (foreign catalogs) + migrations to native apps (2026-07-17)

Archived, shipped. Now described in shc repo add, shc adapter migrate and Migration recipes.

Origin repos (foreign catalogs) + migrations to native apps (2026-07-17)

Section titled “Origin repos (foreign catalogs) + migrations to native apps (2026-07-17)”

Status: DESIGN. Part of the platform-compat program, faithful capture of docs/handoff/2026-07-16-hel1-reprovision-and-platform-compat.md (§C3, §C4, §C6, §D, §E), no new design. Siblings: 2026-07-17-origin-ingest-gate.md (the gate + pipeline every sync runs), 2026-07-17-manifest-adapters.md (the per-origin adapters + terminology), 2026-07-17-app-metadata-schema.md (the canonical fields + compat: bag), 2026-07-17-notes-templating.md (post-deploy notes).

Terminology (origin, adapter, adapted app, origin repo, import, migration) is the collision-checked set defined in 2026-07-17-manifest-adapters.md.

Two gaps beyond single-app adaptation:

  1. Whole catalogs. Adapting apps one at a time doesn’t scale to ecosystems of hundreds of packages. A whole foreign catalog should register as an shc repo and serve its adaptable subset like any other repo, with honest, visible coverage.
  2. Adapted is not the destination. An adapted app runs with a dumb backup, volume snapshots that are crash-consistent but conform to NOTHING (no exporter format, no per-app backup contract), second-class for export/import. Users need a path from an adapted app to the official native SHC app(s), data included, and people running a real Umbrel/CasaOS/Runtipi/Cloudron box need a way onto SHC at all.

Register a whole foreign catalog as an shc repo:

shc repo add --origin umbrel https://github.com/getumbrel/umbrel-apps

(same for casaos / runtipi / coolify / cloudron-packages). Semantics:

  • Sync = pin + transpile + gate. Each sync pins the upstream to a git sha, runs the ingest gate + ManifestAdapter transpile per app (the 7-stage pipeline in 2026-07-17-origin-ingest-gate.md), and serves the ACCEPTED subset as a normal SHC repo (index.yaml + packages). Rejected apps stay listed with their reject reason (shc repo apps --rejected), honest coverage, no silent 40%.
  • Re-sync is the rot-prevention CI. New upstream sha → re-pin, re-transpile, DIFF the normalized output; changed packages surface as app updates. Conformance fixtures per origin live with the adapter (steady-state requirements in 2026-07-17-manifest-adapters.md).
  • Foreign metadata rides the compat: passthrough bag (2026-07-17-app-metadata-schema.md).
  • Licensing diligence applies per catalog before serving it beyond private use, a legal decision, not engineering (the Umbrel platform is PolyForm Noncommercial; catalog + gallery screenshots need diligence, given SHC is commercially deployed).

Installing from an origin repo is a plain shc install; the enrichment core, tenancy, ingress, and backup machinery cannot tell the package is adapted.

Migration paths: adapted app → NATIVE SHC app(s), via the backup system

Section titled “Migration paths: adapted app → NATIVE SHC app(s), via the backup system”

THE KEY INSIGHT: an adapted app running on SHC has a dumb backup: volume snapshots that are crash-consistent but conform to NOTHING (no exporter format, no per-app backup contract). Migration’s whole job is to transform that dumb backup into conforming SHC backup(s) / a backup GROUP that fit the native SHC app(s) being migrated to, because then the EXISTING restore machinery (restore = deploy-from-backup + data-replay) does the rest. Migration is a data transpile whose output is exactly what restore already consumes. No new deploy path.

A *.migration.yaml (shipped in the catalog next to apps, versioned, testable) declares:

name: runtipi-erpnext
source:
platform: runtipi # umbrel | casaos | runtipi | cloudron | coolify
app: erpnext
match:
version: '^(1[45])\.(\d+)\.(\d+)$' # REGEX w/ capture groups, one file
# covers every tipi ERPNext version
targets: # 1..N native SHC apps (splits are normal:
- app: frappe # bundled-DB foreign app -> app + db app)
version: '${1}' # capture substitution; target version is
# DERIVED from source version…
transform:
job: migrate-erpnext.job.yaml # runs with source volumes mounted RO
produces: exporter # emits the frappe exporter-format export
- app: mariadb
version:
map: # …or an explicit mapping table for
'14.*': '10.6' # irregular version relationships
'15.*': '10.8'
transform:
from_volume: db-data
produces: dump # logical dump extracted from the snapshot
group: consistency # one backup GROUP -> all-or-nothing restore

Load-bearing properties:

  • Version mapping is regex/range-based, not per-version files (the ERPNext requirement). Capture substitution for regular mappings, map: tables for irregular ones. And it is NOT cosmetic: the target app version must match the DATA format (the same-DB-major restore constraint; frappe major ↔ site schema); the mapping selects the restore-compatible target.
  • 1→N splits produce one conforming backup per target + a consistency group; restore of the group brings up the native apps together (the group-backup + #91 manifest-inventory machinery already exists and validates the output).
  • Transforms run as jobs with the source volumes mounted read-only (never mutate the source), producing whatever the target’s backup contract expects (exporter-format export, logical dump, file tree). Dry-run = produce + validate the group without cutover.
  • Cutover: stop foreign stack → final dumb backup → transform → restore targets → move hosts → foreign stack kept (protected) until the operator confirms, then destroyed.
  • Layout + matching: migrations are defined by the app they migrate INTO: apps/<targetApp>/migrations/<origin>/<sourceApp>.migration.yaml (e.g. apps/postgres/migrations/runtipi/linkwarden.migration.yaml), with the transform *.job.yaml files beside the recipe. The loader DISCOVERS the owning app by globbing the catalog for the (origin, source-app) key; two apps claiming the same source is an error. A multi-target recipe lives under its PRIMARY target (the app the source natively is); auxiliary datastore targets ride in the same file. New apps add migration paths as a drop-in under their own dir (no shc source edit) and recipes ship automatically because app dirs are packaged recursively. (NOT the repo-root migrations/, which is the Atlas SQL schema catalogue.) Match key = (origin, app, version-matcher). Same target app from different origins = separate files (the data layouts differ). NO match (or unmappable version) → the adapted app simply cannot be migrated yet and keeps running as-is; shc app info on an adapted app shows migratable to: <apps> when a path exists. Adapted apps are first-class migration sources regardless of how they arrived (fresh install from an origin repo, or shc origin import off a real box), identical dumb-backup shape.

Off-box IMPORT: migrating from a real origin box to SHC

Section titled “Off-box IMPORT: migrating from a real origin box to SHC”

For someone actually RUNNING umbrel/casaos/tipi/cloudron on their own box, the journey is two hops, both terminating in the existing restore machinery:

Hop 1: shc origin import --origin runtipi ssh://root@box [app]

  1. The importer reads the box’s app registry + the INSTALLED manifest version (known layouts: umbrel ~/umbrel/app-data/<id>, casaos /DATA/AppData/<id>, tipi app-data/<id> + user-config; CLOUDRON: consume its native per-app backup tarball directly, no SSH).
  2. The adapter transpiles that exact manifest version → the adapted SHC package.
  3. The importer quiesces the origin app, pulls its data, packages it as a DUMB BACKUP of the adapted app.
  4. shc restore deploys the adapted app from that backup.

Import = adapter (package) + importer (data) + existing restore.

Hop 2, later, optional: run the migration.yaml above (adapted app → conforming backup group → restore as official native SHC app(s)). The migration cannot tell whether the adapted app arrived by fresh install or by import, same input either way. Users may stop at hop 1 indefinitely (the adapted app already gains SSO-gating, per-app backups, LE ingress).

  • Sync pins the upstream to a git sha; nothing floats. Rejects are recorded with reasons and stay visible (shc repo apps --rejected).
  • The accepted set is served as a normal SHC repo; install from it is a plain shc install.
  • Re-sync = re-pin + re-transpile + diff the normalized output; conformance fixtures per origin live with the adapter.
  • Migration output = conforming backup(s) + a consistency group, consumed by the EXISTING restore machinery, no new deploy path.
  • Version matching is regex/map-based (never per-version files) and selects the restore-compatible target version.
  • Transforms mount source volumes read-only; dry-run produces + validates the group without cutover.
  • Cutover keeps the foreign stack (protected) until the operator confirms.
  • Migrations are app-defined (apps/<targetApp>/migrations/<origin>/<sourceApp>.migration.yaml, shipped with the target app); same source app from different origins = separate files.
  • Unmigratable adapted apps keep running as-is; shc app info surfaces migratable to: when a path exists.
  • Import = adapter + importer + existing restore; Cloudron’s importer consumes its native per-app backup tarball (no SSH). Hop 2 is optional and may never happen.
  • Flag spelling: the draft uses both shc repo add --type <origin> and --origin <origin>; the collision-checked terminology settles on origin as the noun; the flag name follows it here, to be confirmed at implementation. Likewise the example’s source.platform: key predates the terminology ruling.
  • Licensing diligence per catalog (legal, not engineering) before an origin repo is served beyond private use.
  • Transform-job contract details (what validates a produces: output against the target’s backup contract) ride the existing backup/restore machinery, not re-specified here.