Skip to content
SHC Docs

Published ports

New line, in development. This page describes SHC as it is on the dev branch, not the released product. See The new line.

Verified against shc d1cc800882 and the ce pin it carries, 0ca02485c9. Paths are ce unless they say shc. The flag grammar is also CLI symbols.

--expose is the one flag that reaches a deployment from outside. It is on shc install and shc update (modules/app/commands/install.go). The parser is modules/app/exposeflag/parse.go ParseExpose. Repeat the flag, or comma-join values. It is a string slice, so a comma splits.

<outside>[+<option>...][@<where>][=<service>[:<container port>]]

<outside> is a port, a hostname with an optional path, or a hostname with an outside port. A value that is only digits is a port, from 1 to 65535. A hostname is a DNS name: not only digits, and not an IP address. IPv6 is refused.

No outside port means HTTP through the system Traefik. SHC writes ingress-<tenant>_<env>_<stack>.yaml into Traefik’s dynamic directory (modules/app/inject_traefik.go WriteIngressRouteFile). That file is not a row in the published-port ledger. The bytes arrive on the system Traefik’s listeners, below.

Leave the target out to use the app default: the service marked x-shc-primary, else the first service in file order, with its default port. An explicit target selects a service or a service selector, a container port (=:8080), or both and must resolve to one ingress candidate. The first = outside parentheses separates that target. The path belongs to the hostname before the target and may contain + and @.

A service selector over group=x takes the ports of its group’s members. Any other service selector has no ports of its own, so the target names one, app.example.com=web:8080, and some service of the app must declare it (modules/app/surface_named.go serviceSelectorPortTarget). The route dials the service selector’s VIP, so it reaches only the replicas that match now. A target that is neither a service nor a service selector of the app is refused with X400498EXPTGT, which lists both.

Terminal window
shc install wiki --expose wiki.example.com
shc install wiki --expose wiki.example.com/docs=web:8080

A route with no target is the app’s front door, and SHC reads the front door again from the app’s definition at every install and every update (modules/app/host_spec.go MergeHostSpecs). Move x-shc-primary to another service and a plain shc update wiki moves wiki.example.com to it. The same update serves each path the definition declares with x-shc-path under that hostname, and withdraws a path the definition dropped. A definition with no x-shc-primary resolves as it would on a fresh install: the first service in file order, or the same refusal.

A route with a target is the operator’s pin. No update moves it, and it takes no path the app declares. Exposing a hostname again changes its kind: --expose wiki.example.com=web:8080 pins it, and a bare --expose wiki.example.com makes it follow the front door.

shc ingress prints which each route is under TARGET, front door or pinned. The store holds it in host_mapping.follows.

A route’s hostname may be *.<domain>: one whole leading * label over a domain of at least two labels. It answers every name exactly one label under the domain and never the domain itself or a name two labels down, the depth a wildcard certificate and a wildcard DNS record cover. GitLab Pages serves every namespace’s site through one:

Terminal window
shc update gitlab --expose '*.pages.git.example.com=gitlab:8090'

Its routers match with Traefik’s HostRegexp and rank below every router of a named host, so docs.pages.git.example.com routed by name wins over the wildcard (modules/app/wildcard_route.go). With an ACME CA configured the certificate is one DNS-01 order for the wildcard name, since HTTP-01 cannot validate one; with no DNS connection that can solve DNS-01 for the zone, the route is left out of the file and X201018ACMWDN names the connection owning the zone. With no CA it is served an SHC leaf for the wildcard name. The zone gets the wildcard record (DNS provider sync).

A wildcard route is no origin. It is never a service’s host() or one of its hosts_all(), and never a Keycloak redirect host, though * sorts first among the routes. Any other place for a * is refused with X400497EXPWLD.

An outside port is a published port: the deployment opens a service’s port outside the fabric (core/publish/publish.go). With no target, it uses the app’s default service and port (modules/app/port_mappings.go ResolveDefaultPortTargets). With a named service but no container port, the container port is the outside port. A container port without a service (8080=:80) selects the one service that declares that port.

Terminal window
shc install postgres --expose 5432
shc install gitea --expose 8022=gitea:22

The first publishes Postgres’s default service and port on outside port 5432. The second publishes container port 22 on outside port 8022.

The target may be a service selector of the app. --expose 5432=postgres, with postgres: role=primary:1, stores postgres as the published port’s service and changes nothing in the compose. The listener forwards to the service selector’s VIP, whose backends are the replicas that match when the node’s VIP table is built (modules/vip/service_selector.go routeServiceSelectors), so after a failover the port reaches the new primary with no update run. The container port is the target’s :port or the outside port, and some service of the app must declare it (X400984PRTNOP). A service selector answers only at its VIP, so on a cluster with no service VIPs the publish is refused with X400499EXPVIP. The fabric policy opens the port on the service selector’s replicas that declare it (modules/fabric/policyinput.go serviceSelectorPublishedEntries). A target that is neither a service nor a service selector is refused with X400498EXPTGT.

+udp is UDP. +tcp is accepted and is the default, TCP. +spread sends the connection to any ready replica, and picks at random when several are ready (modules/vip/compile/compile.go listens and target). Without it, a listening node prefers a ready replica of its own, then one under the deepest shared location level, then any ready replica if no level is shared (choose). Levels compare whole path segments; an empty location shares no level. +local answers only on nodes that hold a ready replica and never forwards to another node, so the app sees the client’s address. With no @, a +local port listens on the nodes that hold a replica of its service and follows the replica when it moves. +local and +spread together are refused with X400496EXPLSP. +private takes a plain publish on each listening node’s private underlay address only, never on a public one, so a port meant for the estate’s LAN does not answer the internet; a node with no private address does not listen for it. A connection publish already listens on the node’s private address and its edge decides who reaches it, so +private on one is refused with X400212PUBPRV.

Terminal window
shc install ./dns --expose 1053+udp=dns:53
shc install postgres --expose 5432+spread
shc install mailcow --expose '25+local=postfix-mailcow:25' --expose '3478+udp+local=coturn:3478'
shc install wazuh --expose 1514+private=wazuh-manager:1514 --expose 514+udp+private=wazuh-manager:514

+proxy-protocol is no option: it is refused like any other unknown one, X400492EXPOPT. Whether a port reads a PROXY v2 header is the app’s declaration (A port that reads a PROXY v2 header). The UDP example assumes a local app with a dns service that declares port 53; it uses outside port 1053 because SHC reserves 53 for its own resolver.

A hostname and an outside port are an L4 listener plus a DNS name. A path on that form is refused. An IPv4 literal is refused. No hostname means no DNS record.

Terminal window
shc install postgres --expose db.example.com:5432=postgres

The container port is 5432, the same rule as a bare port. DNS is below.

@(<selector>) listens on the nodes whose labels match. A value that contains / is a location path, stored as location=<path>. A name with no slash stays a connection when that connection exists, or when locations cannot be named. It becomes location=<name> when a node is in that location and no connection has the name (modules/app/listen_place.go ResolveListenPlaces). A name that is both is refused. A location path is refused the same way when a connection has that name.

Terminal window
shc install postgres --expose '5432@(edge=true)=postgres'
shc install postgres --expose 5432@eu/hel=postgres
shc install postgres --expose 5432@hz-lb=postgres

The third is a connection named hz-lb when that connection exists, and a location when it does not and a node is there.

Options and placement precede the target: 36094+udp@(node=home01)=echo:9999. The = inside @(...) belongs to the node selector.

No @ uses the cluster key publish.listen (core/publish ListenKey, registered in modules/app/register.go). Unset, and the empty selector, mean every node. Install and update refuse a selector that does not resolve (X400138PUBSEL, modules/app/listen_overlap.go). A valid selector that matches no node is allowed and has no listener. During a label edit, a selector that cannot resolve contributes no listeners to the overlap check (LabelOverlapGuard calls Listeners, which drops that error).

An incus connection publishes on a public address of the Incus host that owns it (modules/proxy/providers/incusforward). It is for an Incus estate with no load balancer in front. The connection carries two more creds, neither required at connect:

Terminal window
shc connection connect --type incus --name ogden ... \
--cred network=incusbr0 \
--cred public_addresses=incus01=203.0.113.67-203.0.113.70

network is the managed bridge the SHC nodes have a NIC on. public_addresses lists the addresses the connection may hand out, comma or space separated, each <member>=<address> or <member>=<first>-<last>. The member is the cluster member that owns the address. A server that is not clustered writes the addresses alone.

A deployment holds one address of the connection. It takes the first that is free and whose member runs a node, or the one named after the connection:

Terminal window
shc install forgejo --expose 22@ogden=forgejo:22
shc install forgejo --expose 22@ogden:203.0.113.68=forgejo:22
shc install forgejo --expose git.example.com:22@ogden:203.0.113.68=forgejo:22

Every mapping of the deployment on that connection shares the address, so naming two is refused (X400994PRTONE). An address the connection does not list is refused at install (X400975PRXADR), and so is one another deployment holds (X409976PRXHLD). With every address held, a deployment that names none stays unpublished and the reconcile warns X409977PRXFUL. An address after a connection of any other type, or after a location, is refused (X400993PRTADR).

The leader’s proxy reconcile reads the address back from the forward and records it on the deployment. shc ls and shc status show it in an ADDRESS column, and -o json carries it as proxied_addresses. A load-balancer connection records its VIP the same way.

The publish is one Incus network forward at that address, on the member that owns it. Incus keeps a bridge forward per member and refuses a target outside the bridge’s subnet, so the forward targets the SHC node that is an instance on that member, at the mapping’s node port. That node carries the connection to the replica wherever it runs, so a replica that moves changes nothing at Incus. The forward’s user.shc.deployment key names the deployment. A forward SHC did not make keeps its address.

The forward translates and nothing more. The host must already answer for the address on its uplink. The connection’s certificate must be allowed to edit the bridge’s forwards: a bridge of the default project is closed to a certificate restricted to another project, and Incus then answers 403.

A network that is not a bridge is refused (X400978PRXNET). So is an address with no member on a clustered bridge, where Incus would make the forward on whichever member answered. A member that runs no node of the cluster leaves its addresses unused.

Uninstall deletes the forward, and so does an update that removes the deployment’s last mapping on the connection. The address is then free. A release that fails is reported at the end of the uninstall (X500995PRXREL) and the forward stays until it is deleted at Incus.

On shc update, a trailing ! removes one entry by its outside part. A removal does not name a service or a container port. For a port, the name is the outside port, the protocol, and a connection when there is one. 514! removes TCP. 53+udp! removes UDP. Listen, spread and the hostname are not part of that name, so db.example.com:5432! and 5432! remove the same mapping.

Terminal window
shc update wiki --expose 'wiki.example.com!'
shc update dns --expose '1053+udp!'
shc update wiki --expose '!'

A bare ! removes every route and every port the same command does not name. The entries beside it are then the whole set, in the order given, and the first HTTP route is the canonical origin (modules/app/commands/update.go, modules/app/host_spec.go). Alone, the deployment has no route and no published port.

A removal names an entry the operator exposed. A path the app declares under a bare hostname is not one: --expose 'wiki.example.com/push!' matches nothing and is reported X200970RMVNOP. That path goes when its hostname is removed or pinned, or when the definition drops it.

shc install refuses a trailing ! and a bare !. A fresh install has no persisted route or port to remove (X400002VALINP, modules/app/commands/update.go errInstallRemoval).

Listen rules are on when the fabric mode is shc and the fabric range does not overlap the service VIP range (modules/app/install_runner.go). A fabric CIDR that does not parse counts as overlapping (modules/fabric/types.go OverlapsVIPRange).

On that cluster, ResolveAndInjectPorts takes every ports: entry out and declares the container port under expose:. An --expose of an outside port does the same. The rendered compose has no ports: entry for it. The test is shc tests/_ce/modules/app/published_render_test.go TestAPublishedPortIsAListenRuleNotAComposePort.

A host publish the app carried, which no plain --expose covers, warns X200141PUBTPL. The warning names the --expose that would publish it.

Off that cluster, a plain --expose renders a long-form ports: entry with mode: host. A connection publish binds that entry to the placement node’s private address. On the listening cluster a connection publish is a listen rule as well, and that bind is not called (shc tests/_ce/modules/app/published_render_test.go TestAConnectionPublishUsesAListenRuleWithVIPs).

Before either of those, modules/app/swarm_vocabulary.go ports turns an app author’s ports: into expose:. A host side on that path warns X200941SWMXLT. A canonical system stack keeps its ports: through that step (translateSwarmVocabulary). The listen-rules render then takes them out.

A load balancer in TCP mode opens its own connection to the node, so an app behind it sees the balancer’s address for every client. A mail server’s SPF, RBL and postscreen checks would all judge the balancer. A package that wants the client’s own address declares a listener that reads a PROXY v2 header first, on the container port itself (SHC-3444):

services:
postfix-mailcow:
ports:
- target: 10025
protocol: tcp
app_protocol: proxy-v2

ResolveAndInjectPorts marks every mapping that reaches such a port (CLIPortMapping.ProxyProtocol, persisted as proxy_protocol), from the package on every render, so a package that drops the declaration unmarks it. The entry leaves the render, and the port is not declared under expose:, so no runtime sees it and no peer of the tenant can reach it to write a header of its own. It is reached through its publish alone, and that publish listens only on nodes holding a ready replica (publishedPortsOf sets Local). A node that forwarded would hand it a flow wearing the node’s fabric address.

Because the pin does not declare the port, the published-port ledger says it reads the header (proxy_protocol on the row), and BuildPolicyInput keeps it as a published-only port (policy.PublishedPort.ProxyProtocol) instead of leaving it out with X200376FABQRT. The compiler takes such a port without a declaration. Its only accepts are the publication ones: the listener’s marked flow on its node, or one a listener on another node forwards. No tenant rule names it. The test is shc tests/_ce/modules/fabric/published_proxy_v2_3444_test.go.

The forward to it carries ProxyProtocol (subsystems/impl/proxy_reconcile.go forwardsOf), and the pfsense provider writes send-proxy-v2 port <node port> on its server lines, as it does for the system Traefik. postfix’s submission smtpd dies on the PROXY v2 LOCAL header a health check carries (SHC-3712). HAProxy sends that header on every check of a send-proxy-v2 line whose check names no port and no addr, check-send-proxy or not (Ogden 0.1.25 flapped 465 and 587 with send-proxy-v2 alone). So the provider names the server’s own node port as its check port, which makes the check a plain TCP connect. Only a haproxy connection that declares proxy_protocol=v2, or any haproxy connection once proxy.proxy_protocol_trusted_ips is set, sends the header. Any other connection is refused X400986PRXNOH, and a plain publish X400986PRXPLN.

The node’s listener for such a port admits only the sources trusted to write the header (SHC-3482). postfix keeps no list of its own, and a connection publish listens on each node’s private address, so without it any host that reached that address could write a header naming any client, one in mynetworks included, and relay mail. The vip table puts the sources on the published port (compile.Published.Sources), read through vip.SetProxySources, which the daemon wires to app.ProxyProtocolTrustedSources: the same list the app gets below. Only IPv4 sources reach it, each once as its masked prefix. The node’s rule matches ip saddr on them. On a Mac only a trusted source is redirected and a local one still reaches the user-space listener, which writes no header to an app’s port. With no trusted source the port has no listener, so its load balancer pool is empty, and shc doctor (vip.published) says the port reads a PROXY v2 header and trusts no source. A host can also route the pin range through the node and skip the listener. The fabric policy’s entry line drops a flow to such a port that no listener took, whatever it entered on (modules/fabric/policy proxyPortDrops), and the verifier holds those drops to exactly one per published-only PROXY v2 port. The tests are shc tests/_ce/modules/vip/compile/proxy_sources_3482_test.go, tests/_ce/modules/vip/proxy_sources_3482_test.go, tests/_ce/modules/vip/proxy_sources_kernel_3482_linux_test.go, tests/_ce/subsystems/impl/proxy_listen_pool_sources_3482_test.go and tests/_ce/modules/fabric/policy/proxy_port_drop_3482_test.go.

The template namespace hands the app the sources a header may come from: {{ shc.proxy_protocol_trusted_ips | join(' ') }}. It is proxy.proxy_protocol_trusted_ips and every edge’s trusted address, less every network a tenant’s container sits in and every node’s own address, the list the system Traefik trusts less the Mac forwarders, which write a header for Traefik alone. An edge set that does not read warns X200986PRXTRU and leaves the list empty, so the app trusts no header. The tests are shc tests/_ce/modules/app/proxy_v2_port_3444_test.go and tests/_ce/subsystems/impl/proxy_reconcile_proxy_v2_port_3444_test.go.

Every listener that reads a PROXY v2 header trusts one list: the system Traefik’s entrypoints, a node’s listener for a port that reads one, and the app behind that port. It starts from proxy.proxy_protocol_trusted_ips and every edge’s trusted address. A tenant’s container could write any client into a header, so the list never holds an address its flow can wear at any of those listeners (FZ-10142, modules/app/proxy_edge.go withoutProxyUntrusted):

Taken outWhy a container wears it
the block’s deployment-network quarter and fabric half, the posture’s fabric, every block the cluster addedits own address on its own node
every node’s underlay, public and mesh addressits node’s address once the node’s NAT has had it, at a peer node or a public address

An entry inside one of them is dropped, an exact address too. A wider entry is cut around it, and what remains is trusted in its place. Nothing warns: no edge sits in any of them. An entry none of them touches stays as written, so a list with nothing to cut renders byte for byte as before. A Mac’s forwarder is trusted by the system Traefik after the cut, from its node’s mesh address alone.

When the posture or the node table doesn’t read, nothing is trusted without the cut. The system Traefik render is refused X500981PRXUNT and keeps its last render, the node’s listener table keeps its last build, and a template is handed an empty list under X200986PRXTRU. The tests are shc tests/_ce/modules/app/proxy_trust_untrusted_10142_test.go and tests/unit/modules/app/traefik_proxy_untrusted_10142_test.go.

The system ingress publishes vars.ports.http and vars.ports.https. Those are 80 and 443 in modules/traefik/vars.yaml, written as mode: host in modules/traefik/compose.yaml. On a cluster that listens itself, withIngressListens turns each one into a plain mapping: service traefik, listen selector node=@(service=traefik), and Local set. A node listens only when it holds a ready replica of that service. The render declares expose: ["80", "443"] and writes no ports:. The test is shc tests/_ce/modules/app/template_ports_test.go TestTheSystemIngressListensOnTheNodesHoldingItsReplicas.

A plain publish of that same deployment, including one an operator typed without +spread, is local-only too. A mapping the deployment already has for the service, container port and protocol wins over the app’s own publish. An ssh entrypoint in that compose, when vars.ssh.port is set, is the same kind of rule. Another deployment of the traefik app does not get these rules.

The edge owns 80/tcp and 443/tcp (core/nodeports/ports.go Read). The canonical system ingress may publish them. Any other deployment is refused X400992PRTOWN.

Each deployment’s L4 publishes are JSON on its replicated row, column published_ports (core/publish Encode, modules/app/published_ports_intent.go publishedPortsOf). None is an empty column. The listeners, the fabric policy, and the overlap check read it (modules/fabric/published.go ReadPublished). On a first install the recorder writes nothing until the deployment row exists. The stack service then writes the column (PublishedPortsOut, install_runner.go RecordPublishedPortRows). A failed write fails the operation (X500139PUBREC).

A plain publish is stored on its outside port. A connection publish is stored on its node port: a free port in 30000-32767, or the outside port when the backend does not remap (port_mappings.go ResolveAndInjectPorts). The row is omitted when a plain sibling already publishes that node port for the same service, container port and protocol. The ledger stores no hostname on a connection publish. The manifest still has it, and DNS reads the manifest.

Two deployments cannot take one port and protocol on one node. Install, update, and a label edit that would do that are refused. When a clash is already in the table, the older deployment keeps the port (core/publish Listeners).

The host-plane policy reads a different list: node ports from the manifest of running, unstopped deployments recorded on that node (subsystems/impl/fabric_host_ports.go fabricPublishedPorts). That list is for a runtime bind. The listeners on every node come from the ledger.

On Linux the rule is nftables, table shc_vip, chain published, rendered by modules/vip/compile/compile.go render. It is tcp or udp, and it DNATs to a replica pin on the container port. A published port whose service has no live VIP is left out of the table. See modules/vip/section.go published.

A published port answers on the node’s own IPv4 addresses, including 127.0.0.0/8. TCP and UDP from a node process take the output jump to published. The fabric and mesh ranges stay excluded by ownExcluded in modules/vip/compile/compile.go. Packets arriving on the fabric or mesh device take no published listener.

modules/fabric/route_localnet_linux.go sets net.ipv4.conf.<device>.route_localnet=1 on shc0 and on the host end of every pin’s veth on the node (shr and the pin’s address in hex), the system tier’s pins included. subsystems/impl/fabric_reconcile.go converges it each fabric sweep. It never sets all or default. A pin’s veth exists only while its container runs, and the sweep passes over a pin whose veth is absent. Any other failure, a missing shc0 among them, warns with X200405FABLOP and is retried.

Before enabling any device, fabric installs the shc_fabric_loopback nft table. Its filter prerouting chain at priority -150 drops any packet with a source or destination in 127.0.0.0/8 arriving outside lo. This keeps containers and fabric peers away from the node’s loopback services. Legitimate replies still carry their masqueraded destination at that priority. Conntrack restores the loopback destination at dstnat. The guard stays when VIP listeners are removed or the daemon stops, because the per-device switches stay too.

On Linux, TCP also answers on [::1]:<port> through NewLoopbackForwarder in modules/vip/portforward.go. It forwards to 127.0.0.1:<port> and that connection takes the IPv4 DNAT rule. It sends the original IPv6 source in a PROXY v2 header when TargetReadsProxy confirms that the target trusts the masqueraded source. Other targets receive the TCP stream directly. The Linux VIP kernel’s Ensure and Drop start and stop the forwarder through subsystems/impl/service_vip.go. A [::1] listener that cannot be bound is logged under X200286VIPLOP and does not hold the node’s VIP table. UDP on ::1 is not answered. Private-address listeners keep their private binding and get no loopback forwarder.

+spread picks among every ready replica. Without it the order is local, then the deepest location level the node shares with any ready replica, comparing whole path segments, then any if no level is shared. An empty location shares no level. +local, which the system ingress also sets on its own publishes, drops the rule unless a local replica is ready (listens), so a node without one has no listener for the port. A +local publish that names no @ listens on node=@(service=<service>) (modules/app/port_mappings.go replicaListen), which DNS, load balancers and the one-port-per-node claim all read.

A client sent to a replica on the listening node keeps its own address. One sent across the fabric to a replica on another node wears the listening node’s fabric address, the only source a peer admits from that node (render, postrouting rule shc publish: a client forwarded to a replica on another node). An app that reads the client’s address, such as a mail server’s RBL and SPF checks or a STUN server, publishes with +local so no connection crosses (SHC-2605). Behind a load balancer, the app declares a port that reads a PROXY v2 header instead (A port that reads a PROXY v2 header).

A connection publish whose owner is a proxy connection, or whose owner is not recorded, matches only the node’s private address. See modules/vip/section.go ConnectionPrivate. A proxy connection’s load balancer targets that address.

On a Mac the same listeners are pf rdr rules from modules/vip/compile/pf_ports.go CompilePFPorts. A client on the Mac’s LAN or any other network is redirected and keeps its address, and the anchor passes the flow both in and out by its tag SHC_PUB, since pf judges the packet again on the bridge it leaves by. pf does not redirect a local source: the Mac’s own addresses, its container networks, and the networks of its own bridgeN interfaces (PFKernel.Bridges: vmnet VMs, Internet Sharing, Thunderbolt Bridge), since a VM behind the Mac’s NAT hands pf an offloaded checksum that its rewrite breaks (SHC-2660). For local TCP, the process forwarder sends a PROXY v2 header and forwards only to the system Traefik when its dial source is in the deployed trust list; it resets other TCP connections. It binds 0.0.0.0 and does not answer ::1 or UDP. See modules/vip/portforward.go TargetReadsProxy and subsystems/impl/service_vip.go publishedReadsProxy.

Fabric policy admits the container port on the service’s pin, so the forwarded client can reach the replica. See modules/fabric/policyinput.go. A pin that does not declare that container port is left out, and the apply warns X200376FABQRT.

A routed hostname resolves inside the cluster through the node resolver (modules/resolver, see private names). The node, the system tier’s containers and every other address are answered the ready nodes of the app’s location at the address their published ports take. Those ports are taken on a node’s own IPv4 addresses outside the fabric and mesh ranges, so the answer is never a mesh address. A tenant’s container is answered the system Traefik service’s VIP.

AskerAnswer
a tenant’s containerthe system Traefik service’s VIP while the node routes one. Without one, the nodes’ public addresses, and nothing when they have none
the node itself, a system tier container, any other addressthe nodes’ underlay addresses

A tenant’s container that dials the VIP reaches a system Traefik replica wearing its own fabric address. The fabric admits every tenant’s container to the ingress pins on the ports the ingress publishes, for a connection no published port’s listener took. See modules/fabric/policy compile.go, the ingress client accepts. So a cluster whose nodes hold no public address serves its routed hostnames to its tenants’ containers.

The route’s LAN allowlist reads the container’s tenant from that address. A LAN-only route answers a container of its own tenant and gives a container of any other tenant 403 Forbidden (Traefik’s ipAllowList). A shared provider’s LAN-only route answers every tenant’s containers. A route published to the internet answers them all.

The VIP is answered only where it serves what a node’s address serves: the system Traefik publishes each TCP port on the node at the container’s own port, and the node’s kernel routes the VIP. Otherwise the container is answered the nodes’ public addresses.

A tenant’s container that dials a node’s address reaches a published port at a public address, and at one private address: its own node’s. The fabric’s accept for that flow matches an original destination outside the private, shared, loopback and link-local ranges (modules/fabric/policy TenantReachesPublishedAt), or that node’s private address (policy.Node.Private). At a public address the flow wears the address it dialled, so a LAN-only route gives it 403 Forbidden as it gives any outside client.

At its own node’s private address the flow never wears that address. To a replica on its node, or to the ingress on another node, it keeps its own fabric address, so a LAN-only route reads its tenant as on the VIP: it answers a container of its own tenant and gives any other tenant’s 403 Forbidden. To any other replica on another node it wears the node’s fabric address, as a public dial does. Its gateway and an engine’s bridge stay dropped, and no node’s private address is answered for a name, since a name’s answer spans every node.

This door is cross-tenant. A private publication names no tenant, so any tenant’s container on a node reaches another tenant’s +private port at that node’s private address. A connection publish and a x-shc-reach: published port are the same. The dial that asked for this door is one: a VPN router in one tenant dialling a cluster gate another tenant publishes.

A port that reads a PROXY v2 header is the exception at the container’s own node’s private address. The fabric writes no accept for it there, whichever node holds the replica (tenantPublicationAccepts, held by verifyPublication), whatever proxy.proxy_protocol_trusted_ips holds: the flow carries a fabric address there, and a list that covers the fabric would have the app believe its header. A dial at a peer node’s private address leaves through the node’s NAT and wears the node’s address, and one at a public listener keeps a fabric address. Neither is a trusted source (Who may write a PROXY v2 header).

A routed hostname answers on every port its deployment publishes for it, inside the cluster as outside. When a load balancer carries some of them, the name is an open host: a container of its tenant, one of its own deployment too, and the node are answered a VIP of its own. Its web ports go to the ingress, and each load-balanced port goes to the container port the balancer sends it to, on another number too. A PROXY v2 port goes to the service’s plain port of the same number, since no peer can write the header. A port that goes to no service logs X200289VIPLBP.

A LAN-only route answers the node and the system tier’s containers when the node holds a ready system Traefik replica: their connection keeps a fabric address of that node. The LAN allowlist holds the host’s own networks outside the fabric’s range. Of the fabric it holds only the pins of the system tier’s tenant and of the route’s own tenant, each a /32, on every node, so no other tenant’s fabric address is a LAN source of the route. A shared provider’s route holds the whole fabric range. See modules/app/lan_allowlist.go DeriveLanCIDRs. A node that dials another node’s underlay address arrives from its own underlay address, which the allowlist holds when it is private and not when it is public.

shc install and shc update wait until every replica is ready and every routing node carries it. Then the waiter asks each node that listens for a published TCP port to dial that port on its own address, through internal RPC vip.published_probe (subsystems/rollout_pass.go unanswered, modules/vip/published_probe.go). Only the routing nodes the wait counts are asked, so a node with no ready lease holds no install. A Local port is asked only of a node with a ready replica of its service, and a private one only at the node’s private address.

A dial from the node itself never meets the fabric policy’s forward chains, where a client sent on to a replica on another node needs the publication accepts. So the probe names the clusterfs revision the routing wait read, and a node answers only once its applied fabric policy carries it (modules/fabric/policy_applied.go). Every node’s fabric sweep runs as soon as a published port is recorded, since the fabric section carries each deployment’s published ports (modules/fabric/ledger_section.go), rather than at its next minute tick. A node off the shc fabric writes no publication accepts and answers on the dial alone.

A port that reads a PROXY v2 header listens only for the sources trusted to write one, so a dial from the node matches no rule and is refused by design (A port that reads a PROXY v2 header). The node does not wear an edge’s address to get past it. It checks what the edge relies on instead (FZ-9878, modules/vip/published_probe.go probeScoped): its kernel holds the port’s rule at the revision the waiter names, the rule sends to a ready replica, and each replica the rule sends to answers on its container port, dialled from the node. The node reads its rules from its last converge that loaded them (KernelState.Listen). A port that admits any source is dialled on the node’s address as before. The test is shc tests/_ce/modules/vip/published_probe_scoped_9878_test.go.

The command returns once every port answers. A port still silent after 30 seconds is named in the result with its node and what the dial or the fabric policy found, under X200261PUBPND, and the command still exits 0. UDP ports are not asked. A Mac answers that it cannot dial its own published ports, since pf does not redirect a local source, so its ports are not named.

A host route (--expose <host>=<service>:<port>) has no port to dial. When one fronts the deployment, every routing node is asked only whether its applied fabric policy carries that revision: a request through any node’s Traefik needs that node’s ingress accepts and the replica node’s routed accepts, and no dial from a node meets either. The fabric section carries each host route’s backend as well, so recording a route runs every node’s sweep at once. A node still behind after the same 30 seconds is named with how far its policy has got, under X200262ROUPND, and the command still exits 0.

shc status prints a Published ports section for the deployments in that request (modules/health/published.go filterPublished). No rows, and the section is omitted. One line, from modules/health/commands/status_published.go renderPublishedPorts:

<deployment> <service> <port>/<protocol> listen=<selector>[ +spread] nodes: <node> <traffic>, ...

<selector> is the port’s own listen selector, else the cluster key publish.listen, else every node when that key is unset (modules/vip/published_view.go PublishedView). The JSON field listen carries the port’s own selector only, so an empty listen beside a selector says the selector came from the default. <traffic> is local, crossing, unready or refused. No listening node prints none. The same rows are the JSON field published. The line does not name the hostname.

<node> is the node’s name as shc node ls shows it, or its id when it has none, and the nodes go in name order (modules/health/published.go namePublishedNodes). The JSON field published carries the same names.

When a refresh has been in progress for 10 seconds and an earlier snapshot exists, the heading names that snapshot’s age (modules/health/published.go publishedAge).

shc doctor check vip.published reports <n> published ports, all listening when every port has a node whose traffic is local or crossing, nothing is withheld, and the table has no clash. Otherwise it warns. The hint is to check the selectors and the ready replicas, and to resolve an owned port or an overlap (publishedDiagnosis).

Check app.ingress-publish-posture, on a cluster that listens itself, reports system ingress has local-only listen rules for 80/tcp and 443/tcp, or warns that the ledger lacks them (ingressListenPosture). On any other cluster it reads the live ingress container’s host bindings.

The record is on the tenant’s DNS connection, the same managed-DNS plane an HTTP host uses (modules/app/l4_dns.go CreateL4DNSRecords). One record per distinct hostname. An L4 host does not follow the system Traefik (modules/app/dns_target.go ResolveL4DNSTargets). With no DNS service wired, this function writes no record.

--public sends exposure internet (modules/app/commands/install.go). With no flag, ResolveExposure uses the app’s exposure: and otherwise lan (modules/app/inject_traefik.go). A bare shc update sends no exposure and leaves the stored value (modules/app/commands/update.go). Exposure internet uses the internet ladder. Any other exposure uses the LAN ladder.

A connection publish points the record at that connection’s load-balancer endpoints. On an incus connection that is the address the deployment holds. Endpoints that are not ready warn X200983DNSVIP and defer. On internet exposure a private or loopback endpoint is dropped, and nothing left defers the record and warns X200980DNSDEF.

A plain publish points the record at the nodes that listen. Internet exposure uses each node’s public address, drops a private or loopback address, and defers with X200980DNSDEF when none is left. Any other exposure uses dns.lan_address (LanRecordAddress), then dns.public_address (NodePublicAddress), then each listening node’s underlay address. Nothing left defers the record, and only internet exposure warns X200980DNSDEF.

A deployment whose uninstall is releasing its records warns X200961DNSRTD and writes nothing (CreateL4DNSRecords).

Once targets are ready, a DNS write error fails the deploy for a newly bound hostname (X500982DNSPUB); an empty write result also fails when the zone is managed. The deployment stays up. A hostname that was already published only warns (X200982DNSPUB). A deferred hostname skips the write and this failure check, even on its first deploy (CreateL4DNSRecords).

Each line is the code’s template.

CodeLine
X400490EXPFMT--expose {entry} is not usable: {reason}
X400492EXPOPT--expose {entry} names unknown option +{option}
X400493EXPATR--expose {entry} puts @ on a Traefik route, which has no meaning yet; drop the @ or write an L4 port
X400495EXPPTH--expose {entry} puts a path on an L4 port, and a port has no path
X400496EXPLSP--expose {entry} sets +local and +spread, but +local never forwards to another node and +spread sends to any node's replica; keep one
X400497EXPWLD--expose {entry} names the wildcard host {host}, and a wildcard host is one whole leading * label over a domain of at least two labels, like *.pages.example.com
X400002VALINPinvalid input {detail}
X400226FLGOLD{flag} is gone, write {now}
X400936HSTNOPno ingress candidate matches selectors {selectors}, write --expose <hostname>=<service>:<container port>
X400937HSTAMBmultiple ingress candidates match {candidates}, write --expose <hostname>=<service>:<container port>
X400984PRTNOPno service exposes container port {container_port}{service_selector}, cannot resolve the port's target
X400498EXPTGT--expose {entry} targets {target}, which is neither a service nor a service selector of this app, its services are {services} and its service selectors are {service_selectors}
X400499EXPVIP--expose {entry} publishes service selector {service_selector}, which answers only at its service VIP, and this cluster runs no service VIPs
X400985PRTAMBmultiple services expose container port {container_port}, {services}, name the service in --expose <port>=<service>:<container port>
X400992PRTOWNport {port}/{protocol} is {service}'s on this node; publish another port
X409136PUBOVR{deployment} cannot listen on {port}/{protocol} at {nodes}: {holder} already does
X409137PUBAMB--expose @{name} is both a connection and a location; rename one of them
X400138PUBSELlisten selector {selector} does not resolve: {reason}
X409142PUBHST{hostname} is already served by {holder}
X400151PRTHSThostname {hostname} is shared by ports {ports} whose targets differ, give each port its own hostname or one shared listen
X400986PRTCONconnection {conn} of type {type} has no proxy capability, port {mapping} needs a proxy-capable connection
X400987PRTUDPproxy connection {conn} backend {impl} is tcp-only, port {mapping} cannot forward udp
X400988PRTDUPport {external} on shared-vip connection {conn} is already claimed by deployment {stack} environment {environment}
X400989PRTALOno free node port in {min}-{max} for port {mapping}
X400990PRTFIXconnection {conn} backend forwards without port remap, port {mapping} needs node port {external} which is already published
X400993PRTADRconnection {conn} has no addresses to choose from, write the port without :{address}
X400986PRXPLNport {mapping} reaches {service}:{container}, which declares app_protocol: proxy-v2 and reads a PROXY v2 header, so only a load balancer that sends one can publish it; publish it through a connection with proxy_protocol=v2
X400986PRXNOHport {mapping} reaches {service}:{container}, which declares app_protocol: proxy-v2 and reads a PROXY v2 header, and connection {conn} sends none; set proxy_protocol=v2 and edge_addresses on {conn}
X400994PRTONEports on connection {conn} name both {first} and {second}, a deployment holds one address of a connection
X400975PRXADRconnection {conn} does not hold address {address}, it holds {addresses}
X409976PRXHLDaddress {address} of connection {conn} is held by {holder}
X500139PUBRECpublished ports of {tenant}/{stack} not recorded{error}, listener state may be stale, run the update again
X500982DNSPUBno managed dns record was published for newly bound host {hostname}{error}, the deployment is up but that hostname does not resolve, re-run the deploy or check the dns connection owning its zone

X400002VALINP on this flag is a trailing ! on install. X400226FLGOLD is --host, --route, --port or --hosts-exact.

Owned ports, from Read: sshd’s 22 and any port its config adds, shcd’s 4001, 4002, 4003 and 8000, WireGuard 4243 (shc0) and 4244 (mesh0) over UDP, 53 where DNS is on, and the edge’s 80 and 443. A configured daemon port, WireGuard listen, DNS port or edge list replaces that fallback.

These warn. They do not refuse.

CodeLine
X200141PUBTPL{stack}: service {service} publishes {port} in its own compose, which a cluster that listens itself does not do, publish it with --expose {expose}
X200986PRXTRUthe sources a PROXY v2 port may trust did not read, so the render trusts none{error}
X200376FABQRTfabric pin {service} of {deployment} left out of the policy, {reason}
X200941SWMXLTapp {app} speaks Swarm, translated: {detail}
X200961DNSRTDmanaged dns write for {tenant}/{stack}/{environment} suppressed, its uninstall is releasing those records
X200980DNSDEFexposure=internet but the system ingress has no public endpoint for stack {stack}, dns record deferred until a traefik-node public address via shc node update --public-address or dns.public_address exists
X200983DNSVIPsystem ingress connection {conn} has no ready vip yet, dns for stack {stack} deferred until it provisions
X200982DNSPUBmanaged dns record for host {hostname} was not republished{error}, whatever the last successful pass wrote is still what resolves

ResolveL4DNSTargets emits X200980DNSDEF and X200983DNSVIP for an L4 hostname. The templates still say system ingress.

Load balancers target the ready nodes whose compiled listener for that port has a ready backend (subsystems/impl/proxy_reconcile.go CompiledListenerPools, wired by daemon/services_seams_proxy_listen.go). The target is the node port; a proxy connection uses the node’s private address. This is built.