Quota model: caps versus derivation
The question this page answers: when shc quota status prints
no quota limits, does that mean containers have no limits?
No. Two different mechanisms share the word “quota”, and only one of them is configured by an operator. Confusing the two produces the conclusion that “the quota work is inert” and the recommendation to “turn on resource limits”, both wrong. The limits are on. What is absent is the ceiling.
The distinction in one paragraph
Section titled “The distinction in one paragraph”| (a) Per-service derivation | (b) Scope-level caps | |
|---|---|---|
| What it is | the CPU and memory figures written into the rendered compose, which become the container’s cgroup limits | the ceilings a tenant’s total consumption is checked against |
| Key namespace | deploy.cpu_quota, deploy.memory_quota, deploy.limit_ratio, deploy.reservation_ratio | quota.cpu, quota.memory, quota.disk, quota.max_* |
| Set with | shc config set deploy.<key> (or --cpu-quota / --memory-quota at install) | shc quota set --scope <tier> |
| Needs configuration? | No. With nothing set, the number the app author wrote in the compose IS the quota | Yes. With nothing set, every axis is unlimited and nothing is ever refused |
| Reported by | the stack status wire (a resources block per service, carrying cpu_quota_millicores, cpu_quota_source, the derived limit and the derived reservation), the service row, and the x-shc-quota marker in the render | shc quota status |
| State on both estates | live: every container carries derived limits | absent: no cap row at any scope |
no quota limits is the cap side reporting that it found no cap rows.
It says nothing at all about the derivation side.
(a) The derivation chain
Section titled “(a) The derivation chain”The ladder
Section titled “The ladder”Every service gets one quota per axis (CPU, memory). The quota is
the input; the compose limits and reservations the container runs
under are both derived from it. The resolution order for one axis of one
service, highest tier first
(ce modules/app/resource_quota.go:560,
resolveQuotaAxis):
| Tier | Source | QuotaSource |
|---|---|---|
| 1 | the explicit --cpu-quota <svc>= / --memory-quota <svc>= install flag | override |
| 2 | scoped config deploy.cpu_quota / deploy.memory_quota | config |
| 3 | a prior resolution replayed off this document’s own x-shc-quota marker | (whichever tier originally won) |
| 4 | the app’s authored deploy.resources.limits | limit |
| 5 | the app’s authored deploy.resources.reservations | reservation |
| 6 | uncapped: no figure is written and the service charges nothing | "" |
Tier 4 is the one that matters for reading the estates. No estate
sets tiers 1 or 2, so for essentially every service in the catalogue the
authored limits figure is the quota. The owner’s framing is exact:
apps already carry effective quotas because the numbers authored in
their compose files are the quota when nothing is set.
Tier 3 is not a new precedence level: it replays whichever tier already
won, so the transform is a fixpoint on provenance as well as on the
figure. Without it, re-running over its own output reported every
non-limit tier as limit, which lies in the one direction that
matters: telling an operator the app asked for a cap the app never
asked for.
From the quota to the two derived figures
Section titled “From the quota to the two derived figures”quota (the ONE number, settable at any scope) ├─ limit = quota × deploy.limit_ratio (default 1.0, bounds [1, ∞)) └─ reservation = quota × deploy.reservation_ratio (default 0.25, bounds (0, 1])Both defaults are Go constants, not yaml values:
| Knob | Default | Constant |
|---|---|---|
deploy.limit_ratio | 1.0 | defaultLimitRatio in ce modules/app/resource_quota.go:174 |
deploy.reservation_ratio | 0.25 | defaultReservationRatio in ce modules/app/resource_reservations.go:102 |
deploy.cpu_reservation_ratio | falls back to whatever reservation_ratio resolves to | resolveAxisRatio in resource_quota.go:283 |
deploy.memory_reservation_ratio | same | same |
What null in default.yaml means
Section titled “What null in default.yaml means”default.yaml:509 reads reservation_ratio: null,
and limit_ratio: null (line 541), cpu_reservation_ratio /
memory_reservation_ratio: null (528-529), cpu_quota /
memory_quota: null (564-565). null is not “the default is
nothing”. It means the key is declared here and read from somewhere
else:
default.yamlis the koanf startup surface. It is loaded from the file at boot.- These six keys are read through the dqlite-scoped cascade
(
configmod.Service.Get, reached viaScopedConfigValue,ce modules/app/dns_target.go:667), which consults theconfigtable and never the yaml file.
So a number written into the yaml would be a second source of truth that
no reader consults, and an operator who edited it would see nothing
change. The line exists only so shc config set deploy.limit_ratio 1.5
is not flagged as an unknown key. The effective default is the Go
constant, and the guards
TestRatioKeyIsDeclaredAndNotDoubleDefaulted /
TestQuotaKeysAreDeclaredAndNotDoubleDefaulted pin both halves.
Because these keys live in the dqlite cascade, they resolve through all
five scope tiers: deployment > stack > environment > tenant >
global, most specific first
(ce core/scope Rank; the walk is
Service.lookupPrecedenceRows, modules/config/service.go:496). This
is the same cascade dns.ttl and acme.default use.
The ratios do not compound
Section titled “The ratios do not compound”Both ratios multiply the quota. Neither multiplies the other’s
output. ApplyResourceQuotas
(ce modules/app/resource_quota.go:360)
writes:
limits["cpus"] = scaledCPULimit(cpuQuota, pol.LimitRatio) // line 449cpuReservation, _ := effectiveCPUReservation(cpuQuota, authored, pol.CPURatio) // line 424Both calls take cpuQuota. The proof is
TestLimitRatio_DerivationTable
(tests/_ce/modules/app/resource_limit_ratio_test.go:134), row “original
recovery: 1.5 over an authored limit”:
| authored limit | quota | limit_ratio | rendered limit | reservation_ratio | rendered reservation |
|---|---|---|---|---|---|
2.0 cpu | 2000m | 1.5 | 3000m | 0.25 | 500m |
500m is 0.25 × 2000 (the quota), not 0.25 × 3000 (the derived
limit), which would be 750m. A companion mutation anchor,
TestLimitRatio_ScalesTheQuotaNotTheAuthoredLimit (line 193), pins the
other operand: with a config-tier quota of 1.0 over an authored limit
of 2.0, limit_ratio 1.5 must render 1.5, not 3.0.
Two more properties worth knowing
Section titled “Two more properties worth knowing”- Derive only what is absent. A service that authored both a limit
and a reservation keeps both; neither is recomputed. A reservation is
derived only for the keys the service does not already declare
(
effectiveCPUReservation/effectiveMemoryReservation,resource_quota.go:519and:539). The one exception: an authored reservation above the resolved quota is lowered to it, becauselimitsunderreservationsis a compose docker refuses outright, and it is reported asX200947QTACLM. - Derived reservations have floors. 10 millicores
(
minReservationMillicores,resource_reservations.go:195) and 4 MiB (minReservationBytes,:222). The CPU floor stays above the engine’s minimum NanoCPUs (resource_reservations.go). - An out-of-bounds
limit_ratiois clamped, not honoured, and it says so. Anything below 1, NaN, infinite or unreadable resolves to 1.0 and the refused raw value is reported asX200948QTALRT(ResolveLimitRatio,resource_quota.go:299). A reservation ratio outside(0, 1]silently falls back to 0.25.
Where the derivation runs
Section titled “Where the derivation runs”One call site, inside the install/update render:
ce modules/app/install_runner.go:1015 quotaPolicy := ResolveQuotaPolicy(ctx, quotaScope)ce modules/app/install_runner.go:1025 rendered, quotas, err = ApplyResourceQuotas(rendered, opts.ResourceOverrides, quotaPolicy)ce modules/app/install_runner.go:1068 chargeRenderQuotas(...) // the cap check, see (b)It does not branch on the node’s runtime.
(b) The cap side
Section titled “(b) The cap side”The keys, and who writes them
Section titled “The keys, and who writes them”shc quota set writes one config row per axis under the quota.
prefix (cli/quota_client.go:33,
quotaConfigKeyPrefix). The daemon reads exactly those keys back
through the same five-level cascade
(daemon/wiring/quota_limits.go:17-25):
quota.cpu, quota.memory, quota.disk, quota.max_deployments,
quota.max_environments, quota.max_apps, quota.max_vdeployments.
Note the namespaces do not overlap. deploy.memory_quota is a
derivation input; quota.memory is a ceiling. Setting one does
nothing to the other.
Unlimited by default, and what that means mechanically
Section titled “Unlimited by default, and what that means mechanically”configLimitsResolver.Limits
(daemon/wiring/quota_limits.go:66)
leaves an axis’s QuotaLimits pointer nil when no row exists at any
scope. Service.Check evaluates an axis only when its pointer is
non-nil (modules/quota/service.go:419), so an unconfigured scope is a
complete no-op: installs are never blocked, never slowed. A
present-but-zero row is a real cap (a non-nil pointer to 0): the
operator explicitly setting “zero allowed”.
Ceilings are checked at each position of EnforcementWalk
(modules/quota/service.go:524): the leaf, the environment, the tenant.
A stack-scoped delta must clear all three.
What shc quota status reports
Section titled “What shc quota status reports”Service.Status (modules/quota/service.go:593) attaches the
Quota envelope only when limits.HasAny() is true, that is, only when
at least one of the seven cap pointers is non-nil (modules/quota/types.go:44).
The CLI maps that to HasQuota (cli/quota_client.go:225), and with it
false, renderStatusText (modules/quota/commands/status.go:62) calls
render.EmptyList(w, mode, "quota limits"), which prints
"no " + pluralNoun (ce core/render/voice.go:82).
So the string no quota limits means precisely: no quota.* cap row
was found at any scope on the walk. It is not a statement about
containers, about deploy.*, or about whether the derivation ran. The
usage rows are still printed underneath it, which is why the output
looks like a table with an empty limit column rather than an error.
What gets charged against a cap
Section titled “What gets charged against a cap”The quota: one number per axis per service, times that service’s
replica count, summed over the deployment. Never the derived
reservation, never the derived limit
(ce modules/app/quota_render_charge.go). Charging the reservation
instead would discount every deployment by 1/ratio: at 0.25 a tenant
capped at 4 cores would bring up 16 cores of limits. Charging the quota
makes overcommit exactly limit_ratio, one visible number.
Two charge points, because one cannot do both jobs:
- counts at the install gate, before the QUEUED intent row
(
appQuotaEnforcer.Enforce,daemon/wiring/quota_limits.go:193); CPU and memory totals do not exist yet, because there is no rendered compose. - CPU and memory at
RecordRender(:304), inside the render and before the deployment directory is written, so an over-ceiling verdict is a pre-deploy refusal. The figure handed in is an absolute total and the delta applied istotal − what the leaf already holds, so an update, a converge or a retry lands a zero delta instead of billing the deployment again.
Measured state of both estates
Section titled “Measured state of both estates”Measured 2026-08-04 by direct read: rqlite SELECT over the config
and usage tables, docker inspect over every running container, the
rendered composes under /var/lib/shc/deployments/, and
shc quota status. Nothing was mutated.
Both estates are identical on every point below.
| Measurement | promanager (204.168.129.34) | bitspur (37.27.215.210) |
|---|---|---|
shc version / deb | 0.0.16 (shc 1:0.0.16, binaries dated 2026-08-03) | 0.0.16 (same) |
quota.* config rows | 0 | 0 |
deploy.* config rows | 0 | 0 |
shc quota status -t corp | no quota limits + usage rows | no quota limits + usage rows |
usage.cpu_millicores / memory_bytes | 0 on every one of 39 rows | 0 on every one of 28 rows |
usage.deployment_count / app_count | populated (corp 8/8, master 4/4) | populated (corp 8/8, master 4/4) |
usage.disk_bytes | 0 on every row, though disk_measured_at is current | 0 across 26 scanned rows |
rendered composes carrying x-shc-quota | 0 of 12 | 0 of 12 |
rendered composes carrying reservations: | 0 of 12 | 0 of 12 |
HostConfig.MemoryReservation | 0 on every container | 0 on every container |
HostConfig.Memory (hard limit) | set on every container | set on every container |
HostConfig.NanoCpus (hard limit) | set on every container | set on every container |
Where the measured numbers come from
Section titled “Where the measured numbers come from”Every Postgres runs at exactly 1073741824 bytes and every pgclient at
67108864. Those are the authored compose figures of the artifacts
these deployments were installed from, applied by docker compose
directly. Corroborated three ways:
- The rendered compose on disk says so verbatim.
master/postgrescarrieslimits: {cpus: "1.0", memory: 1G}and nothing else: noreservations, nox-shc-quota. 1Gis the figurece modules/postgres/compose.yamlcarried at the pin these estates were built from. That authored limit has since been raised to 4G at cemain(it was demonstrated to break: a singleselect json_agg(...)over a 755 MB table OOM-killed the backend and dropped every connection from every app).64M/0.1cpu is likewise the pre-raise pgclient figure; the catalogue at shcmainnow authors512M/0.5for the same sidecar.
The oldest render on either estate is 2026-07-27 and the newest is 2026-08-01 18:29. Both predate the 0.0.16 deb (2026-08-03 09:54), so nothing has passed through any version of the enricher since the current binaries were installed.
So: do containers have limits? Yes.
Section titled “So: do containers have limits? Yes.”They have them because deploy.resources.limits is honoured by
docker compose.
Measured on docker 29.4.1 / compose v5.1.3: compose sets both NanoCpus
and Memory from deploy.resources.limits, and an over-allocating
container is OOM-killed (exit 137, OOMKilled=true). What compose
ignores is deploy.placement, deploy.labels, deploy.mode,
deploy.endpoint_mode and deploy.update_config, none of which the
catalogue uses.
This path depends on no quota configuration whatsoever. At the
default limit_ratio of 1.0 the derivation is by-value a no-op over an
authored limit anyway: it re-states the identical scalar. That is the
property the whole-catalogue blast-radius proof asserts service by
service (catalogue_corpus_test.go).
The applied ceiling, per tenant
Section titled “The applied ceiling, per tenant”Summed from docker inspect on each leader (both estates identical):
| Tenant | Memory limits | CPU limits |
|---|---|---|
corp | 14.99 GiB | 13.95 cores |
master | 5.69 GiB | 4.25 cores |
| total | 20.68 GiB | 18.20 cores |
| node capacity (compose01) | 30 GiB | 16 cores |
What is NOT working or configured
Section titled “What is NOT working or configured”Four distinct things, deliberately separated because they have different fixes.
1. No caps exist: the only thing an operator has to set
Section titled “1. No caps exist: the only thing an operator has to set”Zero quota.* rows on either estate. This is the whole content of
no quota limits. Nothing is broken; the ceiling was never set.
Everything below the ceiling is live.
2. On the estates today, the CPU and memory books are empty
Section titled “2. On the estates today, the CPU and memory books are empty”usage.cpu_millicores and usage.memory_bytes are 0 on every row,
because the install-time CPU/memory charge landed in v0.0.17 and
both estates run 0.0.16. At 0.0.16 the count axes are charged and
the resource axes are not, so a CPU or memory cap set on these estates
today would compare a real ceiling against a permanently-zero
numerator and never refuse.
Rolling either estate to v0.0.17 or later fixes the code half. The
books only become correct for existing deployments once each is
re-rendered (installed or updated) under the newer binary, because
RecordRender fires inside the render. It converges rather than adds,
so a redeploy is safe to repeat.
3. Neither estate has ever run the derivation
Section titled “3. Neither estate has ever run the derivation”No rendered compose carries x-shc-quota or a reservations block, and
MemoryReservation is 0 everywhere. Two consequences:
- Nothing is recorded about provenance: the
servicerow’s quota columns do not even exist in the estates’ schema (theirservicetable hascpu_request_millicores,cpu_limit_millicores,memory_request_bytes,memory_limit_bytes, all zero). - The memory soft limit (
MemoryReservation, the kernel’s reclaim bias) is unset. That is the runtime effect the reservation half has on compose.
4. Two node-fit checks are defined and unwired
Section titled “4. Two node-fit checks are defined and unwired”EnforceNodeCapacity (modules/quota/enforcement.go) refuses a
reservation that does not fit one node’s free CPU or memory.
EnforceLargestReplicaFitsNode refuses when no listed node can hold one
replica. Both have tests and no production caller. Their codes are
X403010NODCPU, X403011NODMEM and X403013SWMREP. An unknown axis is
skipped. The node table carries memory_bytes (NULL when unknown),
written on the same schedule as cpu_cores.
Separately, the disk axis measures but reports 0: disk_measured_at is
current on both estates while disk_bytes is 0 on every scanned row.
What tonight’s atomicity fix protects, and whether it can protect anything now
Section titled “What tonight’s atomicity fix protects, and whether it can protect anything now”4f696c843 (on main, not in v0.0.17, so on neither estate) routes
every usage mutation through ce core/usagecharge so that the
insert-if-absent, the delta-relative charge and the post-charge
invariant assertion are one batch: one sqlite transaction, one raft
entry. It closes two defects, and they need separating:
- The check/charge race: 8 concurrent 500m charges under a 2000m
cap admitted 8 instead of 4. This half needs a cap to protect
anything. With no
quota.*row,chargeLevelsattaches no caps at all (modules/quota/charge.go:89: ceilings are read only for theEnforcementWalkpositions, and only whenenforceis true), so nothing can be refused and there is no race to lose. - The lost-update on the roll-up: concurrent
UpdateUsagecalls dropped each other’s increments out of the tenant row. This half protects the ledger regardless of caps. The file says so in as many words: withenforcefalse, “no ceilings, so nothing can be refused and the only thing the atomic unit buys is that no update is lost.” The count axes are charged on both estates today, so this is the half that is already earning its keep: it keepsdeployment_count/app_counthonest under concurrent installs.
So the honest statement is: the fix is a prerequisite for enforcement being trustworthy, and independently a correctness fix for the counters that already move.
Enabling enforcement: what to set, and what it would then do
Section titled “Enabling enforcement: what to set, and what it would then do”Do not do this on either estate without an owner ruling on the numbers; this section documents the mechanism, not a recommendation.
# A tenant ceiling. Writes quota.cpu and quota.memory rows at tenant scope.shc quota set --scope tenant -t corp --cpu 16 --memory 24g
# Count ceilings at the same scope.shc quota set --scope tenant -t corp --max-deployments 20 --max-apps 20
# Read it back.shc quota status -t corp # the `limit` / `available` / `used_pct` # columns appear once any cap existsOnce at least one cap row exists at a scope on the enforcement walk:
shc quota statusstops printingno quota limitsand renders the per-axis table withlimit,availableandused_pct.- Count axes refuse immediately. They are charged at the install
gate on every version including 0.0.16, so
max_deployments/max_apps/max_environments/max_vdeploymentstake effect with nothing else changed, and an install over the ceiling comes back asX403004DEPEXC/X403006APPEXC/X403005ENVEXC/X403007VDPEXC. - CPU and memory axes need v0.0.17 or later to have a non-zero
numerator (see What is NOT working item 2). After that, an install
or update whose per-service quota totals would breach the ceiling is
refused before the deployment directory is written, as
X403001CPUEXC/X403002MEMEXC: nothing created, intent row deleted, count charge released. - The disk axis will not refuse until the measurement stops reporting 0.
What setting a cap does not do: it does not change any container’s
limits. Those come from the derivation, which was already running (or,
on these estates, from the authored compose directly). To change what a
container gets, set deploy.cpu_quota / deploy.memory_quota at a
scope, or deploy.limit_ratio to buy burst headroom above the
accounting charge.
The ordering trap
Section titled “The ordering trap”deploy.cpu_quota set at a scope outranks what the app author
wrote (tier 2 beats tier 4). That is deliberate: it is how an
operator caps a tenant’s workloads without editing apps, and it is
why the resolved tier is persisted on the service row and logged at
install as X200946QTASRC: a cap nobody typed at the deployment must
never be silent. Per-service granularity is the --cpu-quota svc= flag,
not the scoped key.
Risk in the current state
Section titled “Risk in the current state”Yes, and it is bounded by the apps rather than by policy.
- No cap means a tenant can install without bound. Nothing in the install path refuses on resource grounds: the cap check is a no-op with nil limits, and the three physical capacity gates are unwired. A tenant with install rights can bring up deployments until the node runs out of memory, and the first symptom is an OOM kill inside somebody else’s container.
- The only thing holding the line today is what app authors wrote. Both estates already carry 20.68 GiB of hard memory limits on a 30 GiB node and 18.20 cores of CPU limits on 16 cores. Memory limits do not reserve, so this is not itself overcommit of allocated memory, but it does mean the sum of what the running set is permitted to touch already exceeds the box. Nothing detects or reports that.
- A cap set today on CPU or memory would be a false comfort. On
0.0.16 the numerator is structurally zero, so
used_pctwould read 0.0% and no install would ever be refused, whileshc quota statusdisplayed a limit and made the estate look governed. This is the failure mode worth naming: a cap that renders is not a cap that holds. - Uncapped services charge nothing even after a roll. A service
that resolves no quota on either axis is absent from the resolved map
and contributes zero to the render charge
(
renderQuotaTotals,ce modules/app/quota_render_charge.go). A cap therefore bounds only the services that carry a figure: an app with no authoreddeploy.resourcesblock, and no scopeddeploy.cpu_quotaabove it, sits outside the accounting entirely. Settingdeploy.cpu_quota/deploy.memory_quotaat global or tenant scope is what closes that hole, because it gives every service in scope a tier-2 quota.
Related
Section titled “Related”- Quota module architecture: the module’s internals, parse grammar, disk measurement, reconciliation crons
shc quotaCLI: flags forset/unset/status/scan- Quota API: the four
/api/method/shc.quota.*routes - Config keys: the koanf startup
surface. Note that the six
deploy.*keys on this page are declared asnullthere and read from the dqlite cascade instead - Error code reference:
X403001–X403013,X200945–X200948