Skip to content
SHC Docs

App metrics and the trust score (2026-08-18)

Status: open design. No trust score is implemented, and no metric row is written. This answers the owner’s question of 2026-08-18 in his own framing: “I need you to figure out what all data points we should be collecting”, with OpenRouter’s model pages and Snyk’s npm metrics as the stated reference. It carries the ranked list of data points, what each costs, what is measurable today, what is not, and what is refused.

★ This is the “scoring lane” that scripts/gen-app-capabilities-md.go was built to feed. Makefile:629 says of that generator: “-format json is the artifact the scoring lane consumes. This program emits no score, no weight and no total: the two axes are INPUTS to a model that lives elsewhere.” This document is that model. §2 is written against its JSON.

Companions: 2026-08-14-marketplace-architecture.md (the store this renders in) · 2026-08-16-app-landing-pages.md (the page these metrics fill) · ../app-development/app-capabilities.md and ../app-development/app-features.md (the two capability axes, shipped 2026-08-18 in !292) · ../architecture/sso-redirect-coverage.md and ../architecture/vendor-resource-floors.md (two fitness registers that already exist and are the model for the rest).

Every count below was measured on this checkout at 7cdaba2da on 2026-08-18. Denominators are 90 store-facing packages: apps/<bucket>/<app>/, the same set gen-app-capabilities-md.go counts; the six top-level built-ins (branding, keycloak, license, openobserve, postgres, traefik) are excluded, as are the 10 apps/tests/ fixtures. Method is stated per number. Anything not measured is marked INFERENCE.


0. Three corrections to the working brief, before anything is built on it

Section titled “0. Three corrections to the working brief, before anything is built on it”

★ AN INSTALL COUNTER ALREADY EXISTS. The brief that opened this lane said “NO INSTALL COUNTER EXISTS ANYWHERE”. That is stale. App Listing.install_cluster_count and install_count_basis are on the doctype (frappe/apps/marketplace/marketplace/marketplace/doctype/app_listing/app_listing.json:34), recomputed by an hourly bench sweep (marketplace/hooks.py:15, marketplace/community/api.py:739), counting DISTINCT REGISTERED CLUSTERS from two bases: settled Install Dispatch rows and Cluster.app_inventory heartbeats. A k-anonymity floor of 5 is applied at write time (community/installs.py:110) because the row is Guest-readable, and the label is fixed in code as “installs from registered clusters” (community/installs.py:112). The reader never returns 0 (features/marketplace/src/listing/community.ts:45). It is better privacy-engineered than anything this document would have proposed; nothing here reopens it.

★ THE SHC RUNNER IS BACK. The brief said shc CI was dead with ten consecutive runner_system_failure. Measured against the GitLab API today: pipeline 2768223652 (schedule, main, 07:07Z) ran four jobs green: marketplace/seed-dry, apps/verify-pins, catalog/freshness, shc:generate. Push pipelines on main are still red, but on script_failure and job_execution_timeout in apps/index, images and test/baked, real failures in real scripts, not a missing docker bridge. Nothing in this document is blocked on the runner, and one thing in it (§4) now has a working home it did not have this morning.

★ THE CAPABILITY AXES LANDED THREE HOURS AGO. !292 (c552e3f82, merged 7cdaba2da at 10:07 local) added a closed 16-key features: block to app.yaml, a derived SHC-integration axis, make catalog/capabilities, and 479 lines of generated page. It changes this document’s shape rather than adding to it; see §2.0. Any earlier draft of this lane that predates it is describing a catalogue that no longer exists.


1. ★ THE DIFFERENTIATOR, AND IT SURVIVED THE ANALYSIS

Section titled “1. ★ THE DIFFERENTIATOR, AND IT SURVIVED THE ANALYSIS”

OpenRouter can tell you a model’s latency because OpenRouter calls the model. Snyk can tell you an npm package’s CVEs because npm is one registry at total coverage. Neither can tell you whether the thing works when you run it, because neither runs it.

SHC is the only party obliged to make these apps work together, on someone else’s hardware, and keep them working. That obligation leaves a residue (files, verdicts, registers) that SHC already owns and nobody else can compute:

what we can saywho else can say itwhere it already lives
this app’s backup was restored and the contents read backnobodyshc-backup-verify.service, weekly PROVEN/DISPROVEN/SKIPPED/UNVERIFIABLE/ORPHANED
SSO actually redirects at this pin, or here is the author check it failsnobodysso-redirect-coverage.md:26-28: 24 wired / 6 pending / 10 structurally excluded
the vendor asks for 4 GB, we ship a 2 GB cap, and here is the vendor’s pagenobodyvendor-resource-floors.md, hand-researched, source per row, four BREACH→fixed rows
every image in this stack still pulls, checked todaynobodycmd/verify-image-pins (2,732 lines) on the daily schedule, computed and thrown away
this app’s whole stack passes an end-to-end suitenobody89 of 90 packages ship tests/tests.bats
what this app plugs into, what plugs into it, and where WE have not caught upnobody214 plugs, 62 sockets, 77 add-ons, and !292’s public 8-gap / 3-substitution / 2-inert lists

The refinement worth stating plainly: “we run them” sounds like it points at runtime telemetry from customer estates. It does not, and it must not; that is the one thing the privacy line forbids, and it is also the weakest of these signals. Every row above is a packaging artifact: deterministic, un-gameable by a vendor, zero privacy cost, and 60–80% already written. Fleet telemetry ranks last here not out of caution but because it is worth less.

If only one signal beyond the star rating ever ships, ship integration depth: “SSO wired · Backup declared · metrics exported · 7 plugs · 3 sockets · 43 add-ons”. That is not a fact about Nextcloud. It is a fact about how well SHC packaged Nextcloud, which is the actual product being sold.


2. THE DATA POINTS, RANKED BY VALUE PER COST

Section titled “2. THE DATA POINTS, RANKED BY VALUE PER COST”

Five privacy classes, extending the vocabulary community/installs.py already established:

  • A: Catalogue. Read from files in this repo. No network, no estate, no person.
  • B: Lab. Measured on our own machines. Our data about our own runs.
  • C: Receipt. A transaction the customer initiated with us; they pressed Install on our storefront and we carried it.
  • D: Volunteered. A person chose to publish it (a review, a comment).
  • E: Refused. Never collected. §7.

2.0: ★ Rung 0: the two capability axes, shipped today, already Class A

Section titled “2.0: ★ Rung 0: the two capability axes, shipped today, already Class A”

!292 did the hardest part of this lane before the lane started, and it did it in the shape this document needs. Measured from docs/app-development/app-capabilities.md:

count
packages90
with a features: block60
with none (every value unknown)30
feature keys in the closed vocabulary16
…with an SHC integration surface (derivable)9
…with no SHC surface at all (declared or nothing)7
GAPS: app has it, SHC wires nothing8
SUBSTITUTIONS: app lacks it, SHC supplies it anyway3
INERT INTEGRATIONS: SHC ships a plug the app cannot consume2

Four properties of that design are binding constraints on any score built on top of it, and each is already argued in tree:

  1. The two axes are kept apart on purpose. “Does it do SSO” is not the same question as “does the SHC package ship an oidc plug”. Reading the second as the first “punishes a vendor for our backlog” (c552e3f82).
  2. ★ A gap is never a mark against the app. app-capabilities.md:20. The 8-gap list is SHC’s to-do list; it must reduce our Operability sub-score and never the app’s.
  3. unknown is a value, not an absence, and it is the DEFAULT, because on the day it shipped, 30 of 90 packages declared nothing and “a schema where silence reads as failure marks the whole catalogue down for existing”. This is exactly the evidence-class doctrine §5 needs, already shipped.
  4. Derived beats declared wherever the filesystem can answer, because “a declared answer to a question the filesystem already answers is an answer that rots”. Everything in Rung 1 below is derived for that reason.

The inert-integration finding is the strongest single argument for building this lane at all. blockscout and monica each ship an oidc.plug.yaml against an application measured to have no SSO entry of any kind: the plug mints a Keycloak client nobody reads. Neither axis finds that alone. A trust model is not a decoration on top of two lists; crossing the lists is where the defects are.

Rung 1: Catalogue facts · Class A · available TODAY · 90/90 coverage

Section titled “Rung 1: Catalogue facts · Class A · available TODAY · 90/90 coverage”

Zero network. Zero telemetry. One importer extension plus the capabilities JSON. Every number in this table was produced by reading these files, which is the proof they are readable.

iddata pointmeasured nowsourcewhy a shopper cares
PKG-01pin discipline: images pinned to a digest vs floating on a tag258 of 321 refs pinned (80.4%); 232 unique, 175 pinned; 72 of 90 packages fully pinned; 18 have ≥1 floating; 1 empty refapps/**/vars.yaml images:what you install next month is what we tested this month; a tag can be re-pushed, a digest cannot
PKG-02health-check coverage80 of 90 composes carry ≥1 healthcheck:apps/**/compose.yamlwhether the platform can tell “started” from “working”
PKG-03backup declared, and what it captures70 of 90apps/**/backup.yamlthe single most-asked self-hosting question
PKG-04SSO wired by SHC (derived, ≠ “app supports SSO”)41 oidc.plug.yaml + 3 saml.plug.yaml; the capability page counts 50 wired incl. 3 gate substitutionsapps/**/*.plug.yamlone login across the estate, or 41
PKG-05upgrade path: migrations and lifecycle jobs5 packages carry migrations/; 107 *.job.yamlapps/**whether an upgrade is a button or a weekend
PKG-06package test suite present89 of 90apps/**/tests/tests.batssomebody wrote an end-to-end test for this whole stack
PKG-07★ integration depth214 plugs, 62 sockets, 77 add-ons; metrics exported by 37, import/export by 70apps/** + capabilities JSONthe differentiator (§1)
PKG-08privilege posture8 packages declare privileged_waiversapps/**this container asks for more than the others
PKG-09declared footprint vs published vendor flooralready a sourced tablevendor-resource-floors.mdwill it fit on the box I have
PKG-10licence, and whether we verified it68 of 90 say unverified, only 22 verifiedapp.yaml license.idcan I ship this at work
PKG-11declared runtime / architecture support4 of 90 declare requires.runtimes; 1 of 90 requires.architecturesapp.yamlalmost entirely absent; Rung 2 measures it instead of asking
PKG-12image count as attack surfacemedian 2, max 21 (flagship/openreplay)vars.yamla 21-container stack is a 21-container problem
PKG-13packaging cadence: dated writes to version:90/90 have ≥1; 55/90 have ≥2git log -L /^version:/,+1:our clock, and it must be labelled as ours, §5
PKG-14upstream capability (16 keys, tri-state + unknown)60 of 90 declaredapp.yaml features:the only axis that answers “can the APP do it”
PKG-15★ the SHC gap list8 gaps, 3 substitutions, 2 inertcapabilities JSONour debt, in public; §5 makes it cost us, not the app

★ Ship the uncomfortable rows with the flattering ones. PKG-01’s 63 floating refs, PKG-10’s 68 unverified licences, PKG-15’s gap list, and the existing apps/VERSION_IMAGE_DIVERGENCE and apps/FRESHNESS_UNVERIFIABLE registers are findings against SHC, not against the apps. A store that publishes its own defect registers is believed about everything else, and every one of those numbers shrinks once it is visible. !292 already did this with the gap list; extend the habit rather than inventing it.

Rung 2: Registry facts · Class A · available TODAY · one anonymous request per digest

Section titled “Rung 2: Registry facts · Class A · available TODAY · one anonymous request per digest”

cmd/verify-image-pins already enumerates every ref in every package, authenticates anonymously against docker.io / ghcr.io / quay.io / Harbor / Gitea, handles rate limits and declared registry outages, and sends the manifest-index Accept header. Lift collect / parseRef / token into an internal package and add a sibling command: ~14 KB of network per ref.

iddata pointsourcewhy a shopper cares
IMG-01★ still pullable, checked today: the verdict apps/verify-pins already computes daily and discardsmanifest HEADthe honest SHC analogue of OpenRouter’s uptime chart. greenbone’s registry garbage-collects superseded manifests within hours; a freshly-installed stack has gone unpullable the same evening
IMG-02architectures actually publishedmanifest indexflagship/ivatar/app.yaml:136-140 hand-documents amd64-only in a prose comment. One request answers it for all 232 unique refs, and closes PKG-11’s 1-of-90 hole by measurement
IMG-03compressed download size, layer countmanifest”how much am I about to pull”
IMG-04★ image build dateconfig blob createdthe freshness signal that replaces the NULL released_on with no backfill and no new collection. Answers the owner’s requirement 5 today. (scripts/import_catalog.py:1654 leaves released_on to the bench default; it is NULL on all 90)
IMG-05runs as root / declared USERconfig bloba real security signal, actionable, 14 KB
IMG-06base distroconfig blobreport factually; let SEC-01 carry the consequence (§6)

Rung 3: Install receipts · Class C · half available TODAY, unread

Section titled “Rung 3: Install receipts · Class C · half available TODAY, unread”

Install Dispatch already records state ∈ {queued, delivered, running, done, failed, refused}, an error code, a detail, and three timestamps: issued_on, delivered_on, settled_on (frappe/apps/portal/portal/portal/doctype/install_dispatch/install_dispatch.json). The marketplace reads only settled rows. Every failure and every refusal, with its reason and its timing, is already on disk and unread. Highest value per hour in this document, and it collects nothing new.

iddata pointavailablewhy a shopper cares
INST-01install success rate: state mix per apptoday, from existing rows”94% of store installs of this app succeeded” is a claim no competitor can make
INST-02failure reason mix: the code histogramtodaytells us which packages to fix; tells a shopper what usually goes wrong
INST-03time to running: issued→delivered→settledtoday”typically running in four minutes”
INST-04installs from registered clustersLIVE (§0)adoption. Never a sort key, §5

Rung 4: Our own lab · Class B · computed TODAY, stored nowhere

Section titled “Rung 4: Our own lab · Class B · computed TODAY, stored nowhere”
iddata pointstate
LAB-01backup restore PROVEN / DISPROVENthe weekly sweep runs and emits a per-deployment verdict; nothing stores it per app
LAB-02SSO actually redirectssso-redirect-coverage.md is a dated, researched matrix, in prose. It needs a machine-readable companion beside it, not a rewrite
LAB-03vendor floor breachessame shape, same need (vendor-resource-floors.md)
LAB-04end-to-end smoke verdict per app89 suites exist; nothing records pass/fail per package per release
LAB-05upgrade success across a version pairnot built

★ LAB-01, LAB-02 and LAB-04 are the flagship metrics of this store, and none of them touches a customer. Two of the three are already researched and already dated; they are invisible to shoppers only because they live in prose.

Rung 5: The scanner · Class A · needs the service (§4)

Section titled “Rung 5: The scanner · Class A · needs the service (§4)”
iddata pointnote
SEC-01fixable CVE counts by severity, per digest★ score on fixable, never raw. Measured: nextcloud:34.0.2-apache carries 36 CRITICAL with 0 fixable; gitlab/gitlab-ce:18.11.7 carries 20 CRITICAL with 20 fixable. A raw count ranks those identically and is therefore worthless
SEC-02SBOM component countthe anti-opacity floor. busybox (shipped by 16+ packages) makes Trivy exit 0 with "Results": null; a naive consumer renders a green tick. Below a component floor the verdict is unknown, never “0 findings”
SEC-03ecosystem / language mixfalls out of the scan for free; reported, never scored (§6)
SEC-04exposure trend over timethe owner’s requirement 4. ~0.3 s per image per night from a cached SBOM, zero registry traffic

Rung 6: Community · Class D · needs the bench migration first

Section titled “Rung 6: Community · Class D · needs the bench migration first”

App Review / App Comment / App Comment Vote / App Content Report are merged, with Wilson lower-bound ranking (community/ranking.py:95, 95%, no time decay). The anti-gaming lever exists and is unused: community/api.py:674 _verification_for proves a reviewer’s own registered cluster runs the app, so each fake vote costs a cluster registration plus an install. Show two numbers side by side, always: all reviews, and verified-install reviews, and rank on the second.

The owner’s ruling that a star requires a review is already the shipped shape: ratings live inside the comment system, not as a one-click widget.


3. WHAT IS AVAILABLE WHEN: the plain answer

Section titled “3. WHAT IS AVAILABLE WHEN: the plain answer”
availabilitydata points
Today, no new collection, no networkPKG-01…PKG-15 (15 metrics, 90/90 coverage, two of them shipped this morning), INST-01…INST-03 (already on disk), LAB-01…LAB-04 (already computed, not stored)
Today, one anonymous registry request per digestIMG-01…IMG-06
Needs the install-reporting infrastructure to growINST-04’s coverage. The counter is live; it renders nowhere until a portal_dispatch_key is minted. Not blocked on design
Needs the scanner serviceSEC-01…SEC-04
Needs the community doctypes migrated onto the live benchCOM-01…COM-03. ★ Merged in git, absent in production: a field query for review_count against api.marketplace.selfhostedcloud.org returns DataError: Field not permitted in query (measured read-only 2026-08-18). The review system cannot receive its first review until this ships
Off-limits, refused§7

4. THE SCANNER: Trivy produces; our own packaged apps optionally consume

Section titled “4. THE SCANNER: Trivy produces; our own packaged apps optionally consume”

Trivy is the authority the storefront renders. One static binary, no daemon; --image-src remote scans straight from a registry with no Docker involved; emits CycloneDX and SPDX; covers OS and language ecosystems in one pass. integrated/dependency-track (5.0.4, already packaged) and enterprise/defectdojo (3.2, already packaged) are optional deep surfaces off the critical path; they ingest what Trivy produces. Dependency-Track cannot be the scanner: it has no container image analyzer. Keeping them off the critical path is what stops a DT outage blanking the shop window; that is dogfooding without making the shop depend on the merchandise.

Four tiers, because the costs differ by ~40,000×:

tierwhatwhencost
0inventory: reuse cmd/verify-image-pins collect() verbatimon apps/** changefree
1config facts: manifest + config blob (IMG-02…IMG-06)on apps/** change14 KB per ref; ~42 s for the catalogue at concurrency 8
2SBOM, generated once per digest and kept forever (a digest is immutable)when a new digest appears~8.6 s/image cold; 40–80 GB one-time transfer; ~192 MB stored
3vulnerability match from the cached SBOMnightly, every stored digest~0.3 s/image, zero registry network, ~12 s for the catalogue

Tier 3 is the time series. The expensive part is digest-keyed and paid once; the part that changes daily is the match against a moving vuln DB. A digest whose counts move while the package did not change is exactly the trend the owner asked for, and it costs seconds.

★ WHERE IT RUNS: the finding that changed today. The repo already has the right lane. .gitlab-ci.yml:1951 runs catalog/freshness and :1851 apps/verify-pins on a daily schedule described in that file’s own header as “the CHECKS-ON-THE-WORLD only”, and that schedule pipeline was green at 07:07Z today. It is not a CI guard: it gates no merge and reds no MR, so it satisfies the owner’s no-guards ruling exactly, and it is already the home of two probes of precisely this shape. make catalog/scan belongs beside them. That is strictly better than a cron on the owner’s laptop: it exists, it has credentials, and it already reports.

★ unknown must be a first-class value, and the failure it prevents is real. Two reproductions from the prototype: busybox exits 0 with "Results": null, and a live ghcr.io blob failure mid-run would have overwritten a real immich result (29 CRITICAL / 111 HIGH) with “0 vulnerabilities” had the state guard not held. Therefore state ∈ {scanned, empty, unresolved, failed}; counts are NULL, never 0, for every state but scanned; coverage is a published field on every rollup; any derived grade degrades to a literal unknown rather than a number. A consumer that forgets to branch gets a blank or a type error, never a green tick it did not earn. This is the same rule !292 already applies to features:, one layer down.

One cheap thing first: repin the 63 floating refs. They are 20% of all refs, one is an empty string, and greenbone alone accounts for 19 (seven with no tag at all, i.e. :latest), huly 14, dify 9, penpot 4. Every scan result attached to a floating ref decays silently, and apps/VERSION_IMAGE_DIVERGENCE already records that greenbone’s registry garbage-collects superseded manifests within hours. apps/verify-pins already probes them, so the work is bounded.


Ship a ledger with a denominator and four named sub-scores. Do not ship a composite number yet. Do not leave it undecided.

One doctype, App Fitness Check: (app, app_version, check, verdict, value, source, measured_at). Every number on every page is a view over rows, and clicking a number opens the row that produced it. verdict uses the vocabulary this repo has already converged on four times independently:

  • cmd/verify-image-pins → PULLABLE / UNPULLABLE / UNVERIFIED / UNREACHABLE, where UNVERIFIED is explicitly never a pass
  • the backup sweep → PROVEN / DISPROVEN / SKIPPED / UNVERIFIABLE / ORPHANED
  • apps/FRESHNESS_UNVERIFIABLE → structurally unmeasurable subjects, named with a reason, itemised in every run’s output, suppressing the failure but never the report
  • !292’s features: → supported / paid / absent / unknown as the default

The doctrine underneath all four, in FRESHNESS_UNVERIFIABLE’s own words: “a probe which cannot see is not a green.” The ledger is that doctrine given a schema.

source is mandatory and visible: probe:verify-image-pins@2026-08-18, derive:catalog-capabilities@7cdaba2da, or manual:clay@2026-08-18. Hand-baked inputs are allowed and labelled; the precedent is already in tree, since vendor-resource-floors.md is entirely hand-researched, cites a vendor doc per row, and nobody mistakes it for a probe. The rule is that a hand-entered input must never silently count as measured.

Four sub-scores, each with its own coverage

Section titled “Four sub-scores, each with its own coverage”
sub-scoreshopper-facing questioninputscoverage today
Operabilityhow well did SHC package this?PKG-02…PKG-08, PKG-15, LAB-01, LAB-0490/90, measurable now
Securitywhat am I exposing?SEC-01, SEC-02, PKG-01 pin gate, PKG-08 waivers, IMG-050/90, needs §4
Freshnessis anyone still working on this?IMG-04 image build date, PKG-01 pin lag, PKG-13 packaging cadencepartial, IMG-04 needs Rung 2
Communitywhat do operators say?COM-01, COM-020/167 listings, needs the bench migration

Capability (PKG-14) is a FILTER, not a score. !292 was explicit that a gap is never a mark against the app; folding the 16 keys into a number would do exactly that by the back door, because 30 of 90 packages are entirely unknown and any weighting of unknown is an invention. Render the chips, let shoppers filter, and keep them out of the arithmetic.

Each sub-score carries one of three evidence classes, and this is how absence and confidence are expressed:

  • Measured: every input present → number and bar.
  • Partial: number computed over the present inputs only, with the missing ones named on the page. A missing input never scores as average and never as zero; it removes itself and says so.
  • Insufficient: no number at all. An empty hatched bar and the reason.

Print the denominator beside every score: “7 of 9 checks measured.” An app with a lower score and full coverage should outrank one with a higher score and three blanks, and only a printed denominator makes that legible.

The single scoring rule that keeps this honest, and it is already ruled in app-capabilities.md:20: when the app supports a capability and SHC wires nothing, Operability falls and the app’s own chips are untouched. A SUBSTITUTION (SHC supplies what the app lacks, the forward-auth gate in front of uptime-kuma, s3manager, agent-canvas) raises Operability. An INERT INTEGRATION (blockscout, monica: an oidc plug against an app with no SSO entry) is a package defect and must reduce Operability harder than a plain gap, because a gap is honest absence and an inert plug is a claim that does not hold.

Why no composite yet: the argument is measured, not aesthetic

Section titled “Why no composite yet: the argument is measured, not aesthetic”

Coverage of the inputs a composite would need, today: Operability 90/90, pin hygiene 90/90, capability declared 60/90, CVE 0/90, upstream cadence 0/90, ratings 0/167, installs rendering nowhere, licence verified 22/90. A weighted sum written today resolves arithmetically to Operability while printing itself under a heading that says trust. His own cited reference (OpenRouter) ships no composite: it ships measurements with units. Snyk’s composite is defensible because Snyk’s inputs come from one registry at near-total coverage. On coverage we are OpenRouter, not Snyk.

Meanwhile the one strip cell !264 asked for carries Operability, not a fake composite. features/marketplace/src/listing/ListingFacts.tsx on !264’s branch (design/marketplace-product-page; the comment is not on main) says in its own header that whoever merges the two branches “owes this cell a decision rather than a rebase”. Measured at 390×844 the four-cell case already wraps to two rows, so it is a one-cell budget. Spend it on the number that is real.

The composite revisit gate: three testable conditions

Section titled “The composite revisit gate: three testable conditions”

Publish a composite the day all three hold, and not before:

  1. the scanner covers ≥95% of digest-pinned refs with results <14 days old;
  2. ≥30% of app listings carry ≥5 published reviews;
  3. release recency is populated for ≥80% of apps, from IMG-04 or a real released_on backfill.

When they hold, Snyk’s shape becomes defensible because the coverage that justifies it finally exists. Until then a composite is a costume.

Name it Vendor Note. It may suppress a listing from a shelf, attach a warning, or force a sub-score to Insufficient. It may never raise a computed score. The asymmetry is the safeguard: the owner is the best source alive for “this upgrade path eats data” and structurally not a trustworthy source for “better than its measurements say”, which is indistinguishable from a thumb on the scale, including to him. Mandatory fields: a closed flag vocabulary (upgrade-hazard, upstream-abandoned, not-production-ready, restore-verified, licence-changed), set_by, set_on, expires_on (max 12 months; an unrenewed note stops rendering), and a one-sentence reason that prints in the ledger. A closed vocabulary, exactly as features: is closed, for exactly the same reason.

Operability may sort the catalogue. It is measured at 90/90, and absence there is genuine absence: a package that declares no backup is a package with no backup, which is a measurement, not a blank.

Installs may never sort anything. The existing Trending/Popular refusal (features/marketplace/src/catalog/types.ts:140-158) stands verbatim, and install_cluster_count does not overturn it: it counts only registered clusters, is withheld below a privacy floor so most listings have no value at all to sort by, and “sorting a shelf by it would put the apps we happen to have registrations for at the top and call that popularity.” Adoption is a fact, not a quality judgement.


His own instinct was right: “maybe that just comes down to CVEs”.

  1. Already priced in. Memory-safety shows up as fewer CVEs at the digest. Counting the language as well counts the same evidence twice and calls it independent.
  2. Not measurable from what we ship. The catalogue ships containers, and !292’s closed 16-key vocabulary, chosen because every key earned its place by varying across the 90 packages, contains no language key. It would be hand-entered: an opinion dressed as a measurement.
  3. It penalises the catalogue’s own spine on a correlation (nextcloud PHP, midPoint Java, frappe Python) while saying nothing about those pinned digests.
  4. It puts “written in PHP, therefore less trustworthy” on a page selling that vendor’s software.

Report the base image factually (distro, size, root or non-root (IMG-05, IMG-06)) because a shopper can act on that. Let SEC-01 carry the consequence.


7. ★ WHAT WE REFUSE TO COLLECT, AND WHY IT IS WRITTEN DOWN

Section titled “7. ★ WHAT WE REFUSE TO COLLECT, AND WHY IT IS WRITTEN DOWN”

The per-version install histogram is FORBIDDEN. AppEntry.Version already rides the heartbeat into Cluster.app_inventory (modules/portal/wire/wire.go:113). Nothing today breaks the install count down by version, and nothing today stops someone doing it; it is one GROUP BY away. Joined to a published CVE posture it stops being a metric and becomes a sized, prioritised target list: here are the estates running the version with the unpatched RCE. The refusal belongs in code, beside min_display_count in community/installs.py, in that file’s own voice, so the next person to reach for it reads the argument before writing the query.

Also refused:

  • Any usage signal. Not a launch, a request, a session, a page view, a login, a user count, or a byte of what anybody did with the app. The owner named this line himself: “we can’t really collect number of users that people use when they install it. So I have to be careful.”
  • Real memory from customer estates: deferred, probably permanently. It is the only proposal that would widen the heartbeat wire, and the wire’s narrowness is a security property: AppEntry has no field that could carry a usage fact, so nobody can add one quietly. Declared footprint (PKG-09, Class A) plus OOM diagnosis on our own estates (Class B) answer the shopper’s question without touching anyone’s data.
  • Anything derived from an unregistered estate. There is nothing to derive: it has no Cluster row and sends no heartbeat. It is not counted as zero; it is not counted.

★ A privacy hole that must close before installs feed any surface. portal.enabled defaults off, good; not being counted is the default and the majority path. But a registered estate currently cannot decline the inventory: daemon/wiring/portal_wire.go:147 wires Apps: localAppInventory unconditionally. Today it is all-or-nothing. Do not let a trust score start depending on install data before that toggle exists; a score that needs the number is a reason to keep collecting it.


8. THE PRIVACY POSTURE: one paragraph, written for a customer to read

Section titled “8. THE PRIVACY POSTURE: one paragraph, written for a customer to read”

Almost everything on this page is measured from the app’s own package files and its public container images. We read the files we wrote, and we ask public registries public questions; that requires nothing from you and nothing from your servers. A few figures come from our own test estates (whether a backup restored, whether single sign-on redirected) and those are our machines, not yours. The install count is different, and we say so wherever it appears: it counts registered clusters, from estates that chose to connect to our portal, and it is shown only above a floor of five, so no single estate can be picked out of it. We never collect usage. Not a login, not a session, not a page view, not a user count; there is no field in the channel that could carry one. If you never register, you are not counted as zero; you are simply not counted, and everything on this page still works.


Each rung is days, not weeks, and each ships a real number before the next starts.

  1. Publish the catalogue facts (PKG-01…PKG-15). One importer extension plus make catalog/capabilities -format json; no network, 90/90 coverage. This alone gives a real body to the listings that currently have none. Ship the defect registers with it: the 63 floating refs, the 68 unverified licences, VERSION_IMAGE_DIVERGENCE, the 8-gap list, and SSO-01’s ten structural exclusions.
  2. Read the install outcomes already on disk (INST-01…INST-03). Highest value per hour here; collects nothing new. Needs the frappe/apps/** subtree republish ceremony (scripts/publish-frappe-apps.sh, guarded by tests/unit/ci/frappe_apps_subtree_test.go), or a guard reds main.
  3. Land the fitness ledger (App Fitness Check) now that four real producers exist to prove the shape. Not first; the shape is only credible once something fills it.
  4. Machine-read the two curated registers (LAB-02, LAB-03). Two of the three most differentiating facts in the store, already researched, invisible only because they are prose.
  5. Repin the 63 floating refs. Bounded work; makes everything downstream honest.
  6. Add the registry probe (IMG-01…IMG-06) by lifting collect / parseRef / token out of cmd/verify-image-pins. Brings IMG-04, the freshness signal that unblocks the third gate condition in §5.
  7. Build the scan service (§4) as make catalog/scan, on the existing daily schedule beside catalog/freshness.
  8. Migrate the community doctypes onto the live bench. A prerequisite, not a nicety; the review system cannot receive its first review until it lands.
  9. Ship the portal inventory opt-out before installs feed any surface.
  10. Store the per-app smoke verdict (LAB-04), then restore proofs (LAB-01), then upstream cadence last.

★ One thing to be careful about: install_cluster_count already works and is well designed. Do not extend it. The temptation to add a version breakdown, a trend line or a growth rate is exactly the pressure §7 exists to resist.


  1. Four named sub-scores with printed coverage, and NO composite until the §5 gate opens. Accept, or overrule. If overruled, the composite must still print its four components beside it; that half is non-negotiable and Snyk agrees.
  2. A gap costs SHC, an inert plug costs it more, a substitution earns credit, and none of the three touches the app’s own chips. This is the scoring rule that makes !292’s two axes safe to build on. Confirm.
  3. Capability (features:) is a filter, never a score input. 30 of 90 packages are entirely unknown; any weighting of unknown is an invention.
  4. The one strip cell in !264 carries Operability, not a composite. !264’s own header asks whoever merges main into it to decide this rather than rebase past it. At 390×844 the four-cell case already wraps, a one-cell budget.
  5. Vendor Note is veto-only (may suppress, warn, or force Insufficient; may never raise a score). Accept, or name a separately-labelled boost path, but never a silent one.
  6. The per-version install histogram is forbidden in code, beside min_display_count. Confirm.
  7. Operability may sort the catalogue; installs may never sort anything. Rule explicitly before anything ranks.
  8. Repin the 63 floating refs, or ship a per-listing “as of digest X, observed <date>” caveat instead? Recommend repinning.
  9. The scanner runs on the existing daily schedule, catalog/freshness’s lane, a check-on-the-world, not a merge guard, and not on a laptop cron. Confirm.
  10. Grade bands on any exposure metric are a product call, not a measurement. Nothing ships publicly until you set them.
  11. 68 of 90 packages declare license: {id: unverified}. Not a scoring input either way, but “unverified” on three quarters of a store’s pages is its own trust problem. Is that a backfill project?
  12. Two cells !264 refused for lack of data (“Available architectures” and “Download size”) become available the day the Rung-2 probe lands. Reopen them then, or leave them closed?