CLI flag spec
For the Operator-facing workflow per command, see the CLI section.
Every erun flag, by command. Type, default, validation, and where the resolved value persists. Operator-facing pages show only the flags an Operator types day-to-day; this page is the complete contract.
Common flags inherited from the root command apply to every subcommand:
| Flag | Type | Default | Effect |
|---|---|---|---|
--output | enum text | json | text | Output mode. text is the human-readable stream. json emits a single structured result object on stdout for orchestrators (the human stream is suppressed or sent to stderr). See Structured output. |
--dry-run | bool | false | Resolve and print the trace without performing side effects. Implies trace verbosity. |
-v / --verbose | bool | false | Stream external tool output (helm --debug, kubectl --v=4, …). |
-vv | bool | false | -v plus per-command trace: lines for every action + decision. |
--time | bool | false | Print elapsed wall time at the end. |
--help / -h | bool | false | Print command help and exit 0. |
Structured output (--output json)
--output json is the orchestration handoff. Every command accepts it; each emits the typed result documented under its section below. The result for erun build is the one orchestrators most depend on:
{
"version": "1.0.81-snapshot-20260616120000", // the minted version — the content identity
"baseVersion": "1.0.81", // the bare VERSION base, before the snapshot suffix
"images": [ // every image built/tagged at this version
{ "image": "ghcr.io/sophium/erun-devops", "tag": "1.0.81-snapshot-20260616120000",
"arches": ["linux/amd64", "linux/arm64"], "status": "built" }
]
}
An orchestrator (the desktop app, a script, an Agent over MCP) runs erun build --output json, captures version, then threads that exact value into erun push --version <version> and erun deploy --version <version>. This is why push/deploy require an explicit version and the convenience switches (build --deploy / build --release) are reserved for an Operator at the terminal — programmatic callers compose the primitives and pass the version themselves.
erun init
Common flags
See erun init — --tenant, --environment, --kubernetes-context, --container-registry, --runtime-image, --set-default-tenant, -y / --yes.
Advanced flags
| Flag | Type | Default | Validation | Persists to |
|---|---|---|---|---|
--project-root <path> | string (absolute path) | <cwd>'s git repo root (git rev-parse --show-toplevel) | Must be an existing directory; must contain a .git/ directory or .git file. | The new env's EnvConfig.localRepoPath (every env type records it; #549). |
--type <type> | enum (local-agent, remote-agent, runtime) | unset. A new env then resolves to local-agent (or remote-agent when --remote is given); an existing env keeps EnvConfig.type. | Must be one of the three values. Conflicts with a --remote whose value disagrees. | EnvConfig.type. On an existing env this is a retype, permitted between any two types in either direction. |
--remote | bool | false | Conflicts with a --type whose value disagrees (e.g. --type=local-agent --remote). | Deprecated alias for --type=remote-agent: sets EnvConfig.type = remote-agent. Init then writes the in-pod bootstrap marker. |
--no-git | bool | false | Only meaningful with --remote / --type=remote-agent. | Skips the in-pod git clone step. |
--version <version> | string (semver) | A new env takes the CLI's built-in ERUN_VERSION; an existing env keeps EnvConfig.runtimeversion (the built-in fills in only when the env records none). | Must satisfy ^[0-9]+\.[0-9]+\.[0-9]+(-[A-Za-z0-9.-]+)?$. | EnvConfig.runtimeversion. The transport's own version is a fallback, not a request — an init about something else never repins a running env; move a version with erun deploy --version. |
--runtime-image <ref> | string | unset — deploy then defaults the image from the chart (a <tenant>-devops umbrella → its own image; the shared erun-devops chart → no override). | A full OCI reference (registry path and/or tag present) is used verbatim; a bare name resolves to <registry>/<name>:<runtime version> at deploy time. | EnvConfig.runtimeimage; applied as imageOverrides.erun-devops on every published-chart deploy. |
--runtime-registry <host> | string (registry host, optional org path) | unset — the runtime chart search then resolves ERun's artifacts from the env's deploy-marked registry, widening to the runtime image's registry when the chart is not there. | Recorded verbatim (trimmed); no scheme, no charts/ suffix. | EnvConfig.runtimeregistry, which the chart search and the in-pod RUNTIME_REGISTRY projection both honour first. Init's own runtime deploy sees it in the same run, so it is also the recovery path for an env that cannot complete a deploy. It is the only writer that replaces the field: a deploy records the registry its chart search resolved at, but only when the field is empty or already agrees — a value set here survives a deploy that resolved elsewhere, which traces deploy: the env's runtime registry <recorded> stands; the runtime chart resolved from <resolved> instead (…) rather than overwriting it. |
--bootstrap | bool | false | — | Deprecated, ignored. Prints a deprecation warning; init no longer scaffolds a <tenant>-devops/ module — envs deploy the published erun-devops chart. |
--runtime-cpu <value> | Kubernetes quantity | A new env takes 4; an existing env keeps EnvConfig.runtimepod.cpu. | Must match the Kubernetes Quantity grammar (m, plain integer, decimal). | EnvConfig.runtimepod.cpu. Supplied alone it merges — naming only the CPU leaves the recorded memory where it was. |
--runtime-memory <value> | Kubernetes quantity | A new env takes 8916Mi; an existing env keeps EnvConfig.runtimepod.memory. | Must match the Kubernetes Quantity grammar (Ki, Mi, Gi, …). | EnvConfig.runtimepod.memory. Merges with --runtime-cpu the same way. |
--codecommit-ssh-key-id <id> | string (APKA… shape) | unset | Must start with APKA; must be a valid IAM key id (length 21). | Stored in the in-pod bootstrap marker (bootstrap.yaml → codecommitSshKeyId). |
--confirm-environment | bool | false | — | Equivalent to -y for the env-overwrite confirmation only. |
--platform-account | bool | false | — | Makes the env a cluster platform account: EnvConfig.platformaccount = true, which threads --set platformAccount=true at deploy so the runtime chart binds the env's ServiceAccount to the built-in cluster-admin (a <release>-platform ClusterRoleBinding). Lets in-pod platform Terraform (the cluster edge) and component installs manage cluster-scoped resources. The first deploy that adds the binding must run from an admin-capable context (the API server's escalation check). |
--components <a,b,…> | list of strings | unset — nothing changes. | Any string; not validated against a chart universe at init time (that check happens at deploy time). | EnvConfig.deploy.components, the saved deploy selection — the same field erun deploy --components overrides per run but never persists. An explicit empty value (--components ''), distinct from omitting the flag, clears a saved selection and returns the env to its repo k8s.deployments plan; there is no other command that resets it. |
--erun-registry | bool | false | Conflicts with --container-registry and with --cluster-registry — pick one. | Seeds the env's registry list with a single static entry, registry.erunpaas.com/<tenant> (eruncommon.HostedRegistryReference), marked build+deploy — the same shape --container-registry produces, just pointed at erun's hosted registry instead of an operator-named host. (Planned.) end to end: the flag, BootstrapInitParams.ErunRegistry, and the resolved registry entry all work today, but registry.erunpaas.com itself is not yet a live, reachable registry (see Container registries · Hosted registry) — the DNS, TLS certificate, and the platform's zot deployment are not cut over. The env's deploy registry then holds only this project's images, so pair it with --runtime-registry ghcr.io/sophium (see the row above) to keep resolving erun's own runtime chart. |
Re-initializing an existing environment
When the env config already exists, init reconciles it against the invocation instead of creating it. Two questions are answered separately, and conflating them is what made settings vanish before: what type is this invocation asking for, and which settings did it supply.
A setting is applied because it was supplied, not because of the type the invocation resolved to. Every field below describes the runtime pod, which an env of any type has, so erun init <tenant> <env> --image-pull-secret X lands on a remote-agent or runtime env without restating --type.
| Input | Supplied | Omitted |
|---|---|---|
--version | Sets EnvConfig.runtimeversion. Trace: init: runtime version set to <v> (or … already <v>). | Keeps it. Trace: init: runtime version not given; keeping <v>. The transport's built-in version fills in only when the env records none, tracing init: env records no runtime version; adopting <v>. |
--runtime-image | Sets EnvConfig.runtimeimage — tagless, even when the flag value carries a tag (a trailing :<tag>/digest is stripped before persisting). Deploy resolution already pins a tagless reference to the env's own runtime version on every deploy; persisting the tag is what leaves a stale pin behind after the env's version moves on, so init never records one. | Keeps it. Trace: init: runtime image not given; keeping <ref>. |
--runtime-registry | Sets EnvConfig.runtimeregistry. | Keeps it. |
--image-pull-secret | Replaces EnvConfig.imagepullsecrets with the trimmed, de-duplicated list. | Keeps the recorded list. |
--runtime-cpu / --runtime-memory | Merges onto EnvConfig.runtimepod: the limit named is set, the other is kept. Trace: init: runtime pod resources set to cpu=<c> memory=<m>. | Keeps both. Trace: init: runtime pod resources not given; keeping cpu=<c> memory=<m>. |
--type / --remote | Retypes the env — see below. | Never retypes. Trace: init: --type not given; keeping env type "<t>". |
--components | Replaces EnvConfig.deploy.components outright, including with an empty list when the value is explicitly empty (--components '') — that clears a saved selection and returns deploy to the repo plan. Trace: init: deploy components set to <a,b,…> (or … (cleared — deploy now follows the repo k8s.deployments plan)). | Keeps the recorded selection. Trace: init: deploy components not given; keeping <a,b,…> (silent when there was none). |
The --type default is the asymmetry that matters: a new env with no --type resolves to local-agent, but that fallback is a default, not a request, so an existing env is not moved by it.
Retyping. A named --type that differs from EnvConfig.type changes it, in either direction and between any two of the three types — including runtime → remote-agent, which is what makes a runtime env orchestratable by the desktop. Trace: init: env type "<from>" -> "<to>" (or init: env type already "<t>" when they match). The rest of the run then does the work the named type implies: retyping to remote-agent or runtime runs the same runtime deploy and in-pod checkout a fresh --type=<t> init would.
Retyping to local-agent is the one case that needs more than the field: a local-agent worktree is hostPath-mounted, and the path a remote env carries in EnvConfig.localRepoPath names an in-pod directory. The retype re-resolves the host project root (--project-root, else the cwd's git root) and records it (init: local repo path set to <path>); when neither answers, it fails with LOCAL_AGENT_RETYPE_NEEDS_REPO_PATH and writes nothing.
Settings are reconciled before init's own runtime deploy, so that deploy carries them: a re-init that adds --image-pull-secret deploys with the secret, and one that omits --runtime-cpu deploys at the env's recorded limits rather than the defaults.
An env created by this same run skips the reconcile entirely — it was written from these params moments ago, so there is nothing to reconcile and no trace lines are emitted.
Side effects
erun init writes these files in this order:
~/.config/erun/<tenant>/tenant.yaml(creating~/.config/erun/<tenant>/if missing).~/.config/erun/<tenant>/<env>/config.yaml.<projectroot>/.erun/config.yaml. Existing values are preserved; new defaults are merged.- Helm-installs the runtime chart into the namespace
<tenant>-<environment>— the repo-local chart when the project has one, otherwise the publishedoci://<registry>/charts/erun-devopschart pinned to the runtime version (seeerun deploy). - With
--remote: writes the in-pod marker at/home/erun/.erun/<tenant>/<env>/bootstrap.yaml.
erun init lifecycle algorithm
- Parse flags; resolve effective tenant + env (see Configuration · Resolution order).
- Validate
--kubernetes-contextagainst~/.kube/config. On miss, abort with the available context list. - Resolve
--project-root(defaults togit rev-parse --show-toplevel). On miss, abort withnot in a git repository. - If the tenant/env already exists, prompt unless
-y/--confirm-environment. Aborting onnis the safe default. - With
--remote/--type=remote-agent(and--type=runtime): resolve a ghcr.io credential from the machine runninginititself (a docker config entry, a gh session, orGH_TOKEN/GITHUB_TOKEN) for every registry the env is configured to build to or deploy from. When one resolves, mint (or refresh) akubernetes.io/dockerconfigjsonSecret named<tenant>-devops-registry-credentialviakubectl apply -f -and persist its name toEnvConfig.registrycredentialsecretname, so step 6's chart install mounts it. Resolves to nothing (no error) when the host itself has no credential to give. - Resolve the runtime chart — repo-local when the project carries one, the published
oci://<registry>/charts/erun-devopsotherwise — andhelm upgrade --installit into<tenant>-<environment>, threadingregistryCredentialSecretNamewhen step 5 minted one. - With
--remote/--type=remote-agent(and--type=runtime): verify the pod can authenticate to any ghcr.io registry it is configured to build to or deploy from — a docker config entry, a gh session, orGH_TOKEN/GITHUB_TOKEN, checked directly in the pod. The Secret step 5 minted is what usually makes this resolve on a freshly created environment; abort if none resolves regardless — the pod is left deployed (init is safe to re-run once authenticated). - With
--remote: open SSH and write the in-pod bootstrap marker. - Update default-tenant pointer if
--set-default-tenant. - Exit
0.
Error codes
| Code | Cause | Exit code |
|---|---|---|
NOT_IN_GIT_REPO | --project-root unset and cwd is not in a git repo. | 1 |
LOCAL_AGENT_RETYPE_NEEDS_REPO_PATH | --type=local-agent on an existing env, with no --project-root and no git repo at the cwd, so there is no host path to mount as the worktree. Nothing is written. Message: cannot change <tenant>/<env> to type local-agent: it needs a host repo path to mount — run init from the project directory or pass --project-root. | 1 |
KUBE_CONTEXT_MISSING | --kubernetes-context is not present in ~/.kube/config. | 1 |
HELM_INSTALL_FAILED | Runtime chart install failed; the per-user config is written but the in-pod marker is not. | 2 |
REGISTRY_UNREACHABLE | --container-registry is set but DNS/network failed. (Warning, not abort.) | 0 (with warning) |
REGISTRY_CREDENTIAL_MISSING | The pod init just deployed has no ghcr.io credential for a registry it is configured to build to or deploy from (no docker config entry, no gh session, no GH_TOKEN/GITHUB_TOKEN), and the machine running init had none to provision either. The pod is left deployed; authenticate it (erun open) and re-run erun init to confirm. | 1 |
erun open
Common flags
--tenant, --environment, --no-shell, --deploy, --vscode, --intellij, --reconnect.
Advanced flags
| Flag | Type | Default | Validation | Persists to |
|---|---|---|---|---|
--deploy | bool | false | Operator-convenience switch. | None. |
--reconnect | bool | false | Declares the run a machine-initiated reattach rather than an Operator open. Suppresses everything in open that starts something: the cloud-context start in step 3, and both halves of the wake in step 5 (the EnvConfig.stopped clear and the scale-to-one). A stopped runtime aborts with RUNTIME_STOPPED. Composes with every other flag. | None — and, by design, prevents the EnvConfig.stopped write a plain open performs. |
--no-alias-prompt | bool | false | Only meaningful with --no-shell. | None (interactive choice only). |
--version <version> | string (semver) | EnvConfig.runtimeversion or the CLI built-in. | Same as erun init --version. Implies --deploy (pinning a version is only meaningful if it rolls out). | EnvConfig.runtimeversion for this run only (not persisted). |
--runtime-image <ref> | string | EnvConfig.runtimeimage (unset → the published image). | Same reference rules as erun init --runtime-image. Applies only to envs deploying the published chart (rides in as imageOverrides.erun-devops); envs with a repo-local chart ignore it. Implies --deploy. | Run-only override (not persisted). |
erun open is a pure primitive: it verifies the runtime is already deployed, best-effort port-forwards SSH/MCP/API for laptop-side tooling, and attaches a shell to the in-pod session. By default it does not build, push, mint a version, or deploy — there is no build branch on env type, and no helm upgrade. The retired --snapshot/--no-snapshot pair has no replacement flag. Rolling out a version is the caller's job: the desktop app composes build → push → deploy around the open, threading the version it captured from build --output json. --deploy is the operator-convenience switch that deploys before opening (builds-here envs build → push → deploy; runtime/remote envs install the recorded/--version published chart by reference); a --version/--runtime-image override implies it. Programmatic callers never use --deploy — they compose the primitives themselves.
erun open lifecycle algorithm
-
Parse flags; resolve effective tenant + env.
--version/--runtime-imageset the effective deploy flag. -
Load
EnvConfig(Kubernetes context, container registry, runtime version, type). By defaultopenneither builds, pushes, nor deploys; it expects the runtime to already exist. -
If
EnvConfig.cloudprovideraliasis set, look up the cloud context. Ifstopped, send the provider-specific start command. Poll the cluster API every5suntil reachable or 5 minutes elapse (then abortCLUSTER_UNREACHABLE).--reconnectskips this step entirely — the same reasonerun stopskips it: starting the machine an Operator (or an idle policy) just stopped is not a decision a reattach gets to make. The reconnect then fails against the unreachable cluster, which is the honest outcome. For an already-running context the step is a no-op either way. -
If
--deploy(or an implied deploy): deploy the runtime first — a builds-here env composes build → push → deploy; a runtime/remote env runshelm upgrade --install <env>-runtime <chart>into<tenant>-<env>for the recorded/--versionpublished chart by reference (requires a resolvable version, elseRUNTIME_VERSION_REQUIRED). Default (no--deploy): a trace line records thatopenis a pure primitive that is not deploying, thenopenverifies the runtime deployment exists in<tenant>-<env>and aborts withRUNTIME_NOT_DEPLOYEDif it does not. This deployment-presence check is the authoritative "is the runtime up" signal — reachability is not inferred from a later port-forward timeout, so a forward that merely can't bind is never misreported as an undeployed environment. -
Wake the runtime if it is stopped. Read the runtime Deployment's
spec.replicas. If it is0,kubectl scale deployment/<tenant>-devops --replicas=1, thenkubectl wait --for=condition=Available --timeout 2m0s. This runs before any port-forward becausekubectl port-forward deployment/…cannot attach to zero replicas. A Deployment already asking for>= 1replica gets no scale call. A Deployment that is absent, or a replica count that cannot be read, is treated as running and the open proceeds — the wake must never be more fragile than the open it precedes. Independently, ifEnvConfig.stoppedis set it is cleared before step 4, so a--deployrun rendersreplicas: 1instead of re-applying the recorded stop.--reconnectinverts this step. Waking is what an Operator opening the environment means; it is not what a supervisor re-establishing a dropped session means, and the two are indistinguishable from insideopenunless the caller says which it is. A--reconnectrun therefore makes no scale call and noEnvConfig.stoppedclear at all: a stopped Deployment aborts withRUNTIME_STOPPEDand a running one proceeds through the rest of the algorithm unchanged. This is what makes a stop durable against reconnects —erun stopdrops every attached session, so a reconnect that woke would undo the stop that caused it, erasing the recorded intent on the way, on a loop for as long as anything is attached. -
Refresh the host AWS credentials if
EnvConfig.cloudprovideraliasnames an AWS alias — the same writeerun cloud refreshperforms. This runs after the wake because the credentials are streamed into the running pod: a stopped environment has nothing to write to. Best-effort: a failure (usually a lapsed SSO session) is traced as a warning andopencontinues, so the environment keeps whatever credentials it already had and the session still opens. -
Wait for the runtime pod's SSH server to be reachable on the in-pod port (
EnvConfig.sshd.port, default22). Readiness probe is a TCP connect + banner-line read, retried every2swith a60scap. -
Establish local port-forwards (best-effort).
erun openstarts a detachedkubectl port-forwardper channel (MCP, SSH, API) and records each at<UserConfigDir>/erun/portforward/{mcp,sshd,api}/<tenant>/<env>.jsonwith{tenant, environment, kubernetesContext, namespace, localPort, logPath, processId}— see Networking spec · Port-forward state files. These forwards back laptop-side tooling only (the desktop app's panels,erun api,erun mcp); the shell/AI session itself runs in-pod viakubectl execand does not use them, so a forward that cannot bind is logged as a warning and skipped rather than abortingopen. -
Attach a terminal (default), print kubectl/cwd switching commands (
--no-shell), or launch the IDE (--vscode/--intellij). -
Exit
0when the terminal exits.
Error codes
| Code | Cause | Exit code |
|---|---|---|
TENANT_NOT_CONFIGURED | Resolved tenant has no ~/.config/erun/<tenant>/tenant.yaml. | 1 |
KUBE_CONTEXT_MISSING | EnvConfig.kubernetescontext is absent from ~/.kube/config. | 1 |
CLUSTER_UNREACHABLE | Cluster API does not respond after 5 minutes. | 2 |
CLOUD_START_FAILED | Cloud-provider start command returned an error or the context entered a terminal failure state. | 2 |
RUNTIME_VERSION_REQUIRED | --deploy for an env with no local chart and no resolvable runtime version (none recorded, none passed via --version). Run erun deploy --version <v> or persist a version. | 1 |
HELM_UPGRADE_FAILED | helm upgrade --install returned non-zero (only on the --deploy path). The release is in helm's failure state; consult helm history. | 2 |
RUNTIME_NOT_DEPLOYED | Pure open (no --deploy) but the runtime deployment is absent in <tenant>-<env>. Detected before the port-forwards so the message is actionable: run erun deploy or erun open --deploy. | 1 |
RUNTIME_STOPPED | --reconnect against a Deployment scaled to zero. Deliberate: a reattach does not start an environment. Nothing is scaled and EnvConfig.stopped is left set; the message names the plain erun open <tenant> <env> that starts it. | 1 |
SSH_READY_TIMEOUT | The runtime is deployed but its SSH server did not become reachable within the 60s readiness window. (A genuinely undeployed runtime is caught earlier as RUNTIME_NOT_DEPLOYED.) | 2 |
IDE_LAUNCHER_MISSING | --vscode / --intellij requested but the launcher binary isn't on PATH. Falls back to printing SSH details; exit 0. | 0 |
erun build
erun build is the version-minting primitive: it builds the images and stamps the version that push/deploy later consume. By default it mints a snapshot (<base>-snapshot-<UTC-timestamp>); --release, an explicit --version, or a version carried by the build directory pins the bare version instead. build never decides snapshot-vs-stable from the environment type.
Common flags
--deploy, --release, --force, --dry-run, --output.
--jobs/-j sets how many images build at once: 0 (default) resolves a conservative degree from the host, 1 is strictly sequential, N is explicit. ERUN_BUILD_JOBS sets the same value by environment, and is the deterministic seam for tests — pin it rather than inheriting the runner's core count.
Scheduling honours the FROM graph: independent images share a wave, and an image that FROMs a sibling waits for it. With more than one worker the wave plan is emitted as a trace line before any build, followed by every image's decision lines in dependency order, then the builds themselves with each image's output buffered and flushed in wave order — so output is deterministic at any degree and the dry-run contract is unaffected. At --jobs 1 the decision lines stay interleaved with each image's own output, exactly as before. push, release, and build --deploy are always sequential. A FROM cycle fails with an error naming the images rather than deadlocking.
--deploy and --release are operator-convenience switches that compose downstream primitives (--deploy → push + deploy; --release → the release flow). Programmatic callers do not use them: they run erun build --output json, capture version, and call push/deploy themselves. See Structured output.
Advanced flags
| Flag | Type | Default | Validation | Notes |
|---|---|---|---|---|
--no-incremental | bool | false | — | Disables the fingerprint cache. Every Docker context rebuilds. |
--version <version> | string (semver) | Resolved per Build path resolution · VERSION walking. | Same as erun init --version. Conflicts with --release (which resolves the version itself). | Pins a bare version for this build instead of minting a snapshot. |
--output json result
erun build --output json prints the structured output object: {version, baseVersion, images}. version is the minted content identity an orchestrator threads into push/deploy.
erun build lifecycle algorithm
- Parse flags; resolve effective tenant + env. Refuse with
BUILD_AGAINST_RUNTIME_ENVif env type isruntime(a runtime env has no source to build). - Resolve project root, build scope, Dockerfile, build context, VERSION per Build path resolution.
- Mint the version. Default: append the snapshot suffix
-snapshot-<UTC-timestamp>to the resolved base. With--release/--version/ a build-dir version: use the bare version. Compute the per-image content fingerprint. - For each resolved image:
a. If fingerprint matches the registry copy and
--no-incremental/--forceis not set: promote the registry copy locally; skip the build. b. Otherwise: invokedocker buildx build --platform linux/amd64,linux/arm64 -t <registry>/<image>:<version> -f <Dockerfile> <context>with the resolved--build-argset. c. Tag the result<registry>/<image>:fp-<fingerprint>-<arch>for each architecture. d.ERUN_VERSIONfor a base built by this same run. Images are ordered so aFROM <registry>/<base>:${ERUN_VERSION}wrapper builds after its base. A build that does not push tags only per-arch (…:<version>-<arch>) — the arch-less…:<version>is a manifest listpushmints in the registry, so it exists neither locally nor remotely for an unpublished version. So when the wrapper's base is one of this run's own unpublished images, itsERUN_VERSIONbuild arg is<version>-<arch>for the architecture being built, resolving the base from the local daemon at the matching arch (an arch-less local tag would be last-arch-wins, and therefore single-arch). This is what letserun build --version <v>validate a whole release locally, dependent images included, before any git ref moves. A local snapshot base follows the same rule at<base>-snapshot-<arch>. A--releasewrapper keeps the plain<version>: its base is pushed earlier in the same run, so it resolves from the published multi-arch manifest. No plain-<version>local tag is ever created, so a local build can never be mistaken for a published manifest or pushed in place of one. - Emit the minted version (and, with
--output json, the structured result). - If
--deploy: compose push + deploy at the minted version (operator-convenience shortcut). If--release: run theerun releaseflow, which builds and reusespushto publish the release-tagged variants and chart, verifies they resolve, and only then moves the git refs. Programmatic callers skip both and orchestrate the primitives themselves. - Exit
0.
Multi-arch verification
Before any docker build call, the binary inspects docker buildx ls for builders advertising linux/amd64 and linux/arm64. If either platform is missing, abort with BINFMT_MISSING and print:
binfmt for <arch> not installed. Run:
docker run --privileged --rm tonistiigi/binfmt --install all
Error codes
| Code | Cause | Exit code |
|---|---|---|
BUILD_AGAINST_RUNTIME_ENV | erun build called against an env where EnvConfig.type == "runtime". | 1 |
NO_PROJECT_ROOT | cwd is not inside a project root. | 1 |
NO_BUILDABLE_CONTEXT | Walked up from cwd and found no <tenant>-devops/docker/<image>/ directory. | 1 |
BINFMT_MISSING | Local docker daemon cannot produce one of the target platforms. | 2 |
BUILD_FAILED | docker buildx build returned non-zero. | 2 |
erun push
Version (required, unless --build)
| Flag | Type | Required | Notes |
|---|---|---|---|
--version <version> | string (semver, snapshot or bare) | Yes, unless --build is set | The version to publish (the same flag deploy uses, for consistency across commands). push does not mint a version — it builds each image from source at this version (promoting unchanged images from the fingerprint cache), pushes the per-arch tags, assembles the multi-arch manifest list, then publishes each component's helm chart. Missing (and no --build) → NO_VERSION (exit 1). |
--build | bool (default false) | — | Operator-only convenience switch (CLI top-level erun push only). Builds the current source first — the same pure build erun build runs, minting a snapshot version — then pushes that exact minted version. Equivalent to erun build && erun push --version <minted>. Mutually exclusive with --version (the version is whatever build mints); passing both → exit 1, push --build builds and pushes the version it mints; do not also pass --version. --force propagates to the build step. Not exposed over MCP: the push tool keeps version required, because programmatic callers compose build → push themselves and thread the minted version (see Command primitives). |
What push publishes
For the supplied --version, push always builds each image from its source context (never a prebuilt bare tag), pushes per-arch tags, assembles the manifest list, then publishes every Helm chart discovered under the project's k8s/* directories — a directory scan, not a lookup keyed to same-named images, so image-less charts (a tenant's own frs-backend-api, frs-powerdns, … wrappers) publish too. For each: helm dependency build (umbrella charts that vendor published subcharts) + helm package + helm push to oci://<registry>/charts at --version, verified with a helm pull round-trip. Chart publishing is decoupled from the image push: a version-pinned base (erun-powerdns, erun-backend-postgres, erun-zitadel, erun-zitadel-login) keeps its image at the upstream pin and is not re-pushed at --version, but its chart still publishes at --version so platform deploys resolve it. erun build packages the same charts locally (validate + --output json) without publishing. Charts publish under /charts, separate from the same-named image repo so a chart never collides with its image at the same ref. There is no environment-type branch. erun release reuses this step for all its publishing.
Chart verification retry semantics
The helm pull round-trip reads back an artifact push itself just wrote, so a registry that has not finished propagating the new tag is a race, not a verdict — GHCR in particular mints the pull token before the tag is listed and answers the first fetch 403: denied. Verification therefore retries up to 4 attempts with a linear backoff of 500ms, 1s, 1.5s (≈3s worst case) when the failed read's output matches a transient class, case-insensitive:
| Class | Matched substrings |
|---|---|
| Authorization / propagation | 401, 403, 404, denied, unauthorized, not found, manifest unknown |
| Transport | timeout, timed out, temporary failure, connection reset, connection refused, eof, no such host, tls handshake |
| Server-side | service unavailable, too many requests, 500 , 502, 503 |
Any other failure is treated as final and fails on the first attempt. A read that never succeeds still fails the push after the last attempt — the retry bounds a race, it does not swallow a persistent failure.
Common flags
--force, --dry-run, --output.
Upfront registry-credential check
Before building anything, erun push (and erun release, which reuses push's publish stage) checks whether any credential resolves for a ghcr.io registry it would push to — a docker config entry, a gh session, or GH_TOKEN/GITHUB_TOKEN. GHCR never accepts an anonymous push, so no credential at all is a certain failure, not an ambiguous one; refusing here turns a multi-arch build spent for nothing into an immediate, actionable error naming the missing credential and the gh auth login/docker login commands to fix it.
Authentication retry semantics
When docker push returns one of these registry-side error strings, erun push retries automatically:
| Registry response (substring match, case-insensitive) | Retry |
|---|---|
unauthorized | Re-runs docker login <registry> interactively (TTY required). |
denied | Same. |
insufficient_scope | Same. |
does not match expected scopes (GHCR-specific) | Invokes gh auth refresh -s write:packages,read:packages and retries. Requires an interactive browser login (see gating below). |
permission_denied (GHCR-specific) | Same as above. |
If no TTY is attached, the generic docker login retry skips the login prompt and surfaces the original error.
The GHCR scope refresh has a stricter gate: it drives gh's interactive browser device-code flow, so erun push never launches it when there is no browser or no operator at the prompt. It is skipped when either:
- the process runs inside the chart-injected runtime pod (
ERUN_TENANTandERUN_ENVIRONMENTset) — headless, no browser, even though the desktop terminal is a PTY-backed pod shell; or stdinis not an interactive terminal (MCP, CI, pipes).
When the refresh is skipped, erun push does not hang on a device-code prompt. It fails with an actionable error naming the missing write:packages scope and the exact commands to run from a host shell with a browser:
gh auth refresh -h github.com -u <owner> -s write:packages,read:packages
gh auth token -u <owner> -h github.com | docker login ghcr.io -u <owner> --password-stdin
Error codes
| Code | Cause | Exit code |
|---|---|---|
NO_VERSION | No <version> argument. push publishes a specific version; it does not mint one. | 1 |
NO_BUILDABLE_CONTEXT | No <tenant>-devops/docker/<image>/ build context found to build the version from. | 1 |
REGISTRY_CREDENTIAL_MISSING | No credential resolves for a ghcr.io registry to push to at all (no docker config entry, no gh session, no GH_TOKEN/GITHUB_TOKEN). Refused before any build. | 1 |
REGISTRY_AUTH_FAILED | All retry attempts failed (or no TTY for the interactive login). | 2 |
MANIFEST_LIST_ASSEMBLY_FAILED | Per-arch tags pushed but docker manifest create failed. | 2 |
CHART_PUSH_FAILED | Images pushed but a chart's helm push, or its helm pull verification after every retry, failed — the version is not yet deployable. Charts publish one at a time, so the error names the split explicitly (published: / failed: / not attempted:) and states the recovery: re-run erun push --version <version>, which republishes idempotently. | 2 |
erun deploy
erun deploy is a pure consume primitive: it helm-installs an already-published version by reference. It never builds, pushes, or publishes — a version is required input, not something it mints.
Common flags
--version, --runtime-image, --runtime-chart, --current, --components, --force, --rollout-timeout, --mcp-auth-public-key, --no-mcp-auth, --dry-run, --output. Subcommand: erun deploy <component>.
Version selection — --version / --current (required)
erun deploy and erun upgrade are consume operations: the resolved version names a content identity to install, not a label to stamp on a fresh build. A version is required — exactly one of:
| Flag | Type | Meaning |
|---|---|---|
--version <v> | string (semver, snapshot or bare) | Install version <v> by reference. |
--current | bool | Install the env's persisted runtime version (EnvConfig.runtimeversion) — redeploy what it already runs. Errors if the env has no recorded version yet. |
Passing neither is an error (NO_VERSION): deploy requires a version — pass --version <v> or --current; exit 1; nothing runs. deploy does not build, so there is no fallback to "produce a version from the working tree" — that path no longer exists. The desktop app and other orchestrators always pass --version, threading the value captured from erun build --output json.
Once the version is resolved, deploy:
- Resolves no docker builds and pushes nothing (so it can never overwrite the published version). It runs
helm upgrade --installpinned to the version. - Verifies each image the chart references at the version exists. For every
image:ref the chart resolves atAppVersion == <v>: registry-less refs (the app images, which take their registry from--set containerRegistry) are qualified with the deploy registry; refs pinned at a different version (infra/base images such as dind and binfmt) are skipped. The check isdocker manifest inspect(then a localdocker image inspectfallback); in--dry-runit is traced and the network call is skipped. A registry error that is not a definitive "absent" does not block the deploy. - Errors during resolution, before
helm upgrade, when an image at the version is absent both locally and in the registry:deploy --version <v>: image <ref> is not present locally or in the registry; deploy installs an existing version and does not build it — run erun build/push to create it first.
The dry-run trace names the decision per spec: deploy: version <v> pinned; installing the published image, no local build. Every env installs by reference from the published oci://…/charts/erun-devops chart (or the repo-local chart when the project carries one).
Subchart value forwarding for wrapped umbrellas
A tenant that publishes its own artifacts ships umbrella charts — the runtime <tenant>-devops and each <tenant>-<component> — that wrap the canonical erun-<base> chart as a subchart (dependency name erun-<base>, no alias; the erun-build-env / erun-blueprint-platform pattern). helm does not pass top-level --set values into subchart scope, so a by-reference deploy of such a chart would leave the wrapped subchart's {{ required }} tenant/environment unset (tenant is required at render). Deploy closes that gap for any chart it installs by reference whose name is tenant-prefixed (not the canonical erun-<base>):
- Re-scopes the threaded
--sets under the subchart keyerun-<base>(--set-string erun-backend-api.tenant=<t>, …), so every value erun resolves at deploy time —tenant/environment, ports, cloud context, MCP auth,imageOverrides, registry — reaches the wrapped subchart exactly as it would a chart installed directly. A canonicalerun-<base>chart installed directly (theerunproduct tenant, or an explicitly selectederun-*chart) is not re-scoped — its top-level--sets already reach it. - Applies the chart's bundled
values.<env>.yaml. Before the rollout, deploy runshelm pull <ref> --version <v> --untar --untardir <tmp>and adds-f <tmp>/<chart>/values.<env>.yaml, forwarding the tenant's own authored per-env subchart values (pod-shape:extraContainers/extraVolumes/extraEnv/extraRules, and any overrides authored under the subchart key). This is the by-reference analogue of a worktree deploy's localvalues.<env>.yaml. The file is-f'd before any config-dir overlay (~/.config/erun/<tenant>/<env>/values.yaml), and the re-scoped--sets win over both — so erun-resolved values are authoritative and a key authored in the bundled file that erun also threads (e.g.api.oidcAllowedIssuers) is owned by erun, not the file.
The dry-run trace shows the helm pull … --untar line before the helm upgrade line; the temp dir is removed after the rollout. Local (worktree) deploys are unchanged: a local runtime umbrella re-scopes via its Chart.yaml erun-devops dependency and -fs its worktree values.<env>.yaml; a local component umbrella -fs its worktree values.<env>.yaml (which is why authoring the nested subchart values there is still required for the worktree path).
Runtime chart search order
An env with no repo-local runtime chart and no stated chart (--runtime-chart / EnvConfig.runtimechart) resolves the chart by probing coordinates in order. Let R be the chart registry — EnvConfig.runtimeregistry when set, else the runtime image's registry for a --cluster-registry env, else the env's deploy-marked registry, else the project's configured registry, else ghcr.io/sophium — and P the platform registry, the registry prefix of EnvConfig.runtimeimage when it names one, else ghcr.io/sophium.
| # | Coordinate | Probed when | Trace on a miss |
|---|---|---|---|
| 1 | oci://R/charts/<tenant>-devops | RuntimeReleaseName(tenant) != erun-devops (skipped for the erun product tenant) | deploy: runtime chart <tenant>-devops <v> not found in R (the tenant's own umbrella) |
| 2 | oci://R/charts/erun-devops | always | deploy: runtime chart erun-devops <v> not found in R (the shared platform chart) |
| 3 | oci://P/charts/erun-devops | P != "" and P != R | deploy: runtime chart erun-devops <v> not found in P (the shared platform chart in the runtime image's registry | in erun's own registry) |
Each probe is the same authenticated registry read push writes with; the first coordinate that publishes <v> installs, tracing deploy: runtime chart <chart> <v> found in <registry> (<reason>). Rung 3 exists because ERun publishes charts/erun-devops only beside the runtime image it releases: an env whose deploy registry is its own ECR (or the in-cluster erun-registry) has the platform chart at no version there, and a search that stopped at rung 2 left it undeployable at every version.
When no coordinate is confirmed published at <v>, the search refuses rather than install rung 2 unconfirmed: charts/erun-devops is versioned on ERun's own release line, so pairing it with another project's version is a coordinate that can never exist, and installing it anyway was the failure mode this refusal replaced. Each probe answer is one of two kinds, and the refusal distinguishes them rather than treating a registry it couldn't read as a "no": confirmed absent (the registry answered and the version was not in the chart's published tags) or could not determine (the read itself failed — an unreadable or unauthenticated registry, a network error). The search traces each rung's answer as it goes (deploy: runtime chart <chart> <v> not found in <registry> (<reason>) for a confirmed miss, deploy: runtime chart <chart> <v> could not be confirmed in <registry> (<reason>): <error> for an inconclusive one), then a final deploy: no runtime chart candidate confirmed at <v>; refusing to guess before returning the error, so the dry-run trace shows the stopping decision even though the deploy never reaches a helm command. The error message enumerates every coordinate probed and its answer, plus the three ways out: erun init --runtime-registry <host> to record where ERun's artifacts live, erun push --version <v> from the project that owns a <tenant>-devops umbrella, or --runtime-chart / EnvConfig.runtimechart to name the chart outright — the last of which also lets a deploy proceed on an env whose search cannot itself confirm a coordinate, since a --runtime-chart/EnvConfig.runtimechart value supersedes the search's answer entirely.
The registry the search resolved at — not R, where it started — is what a successful deploy (or open) memoizes as EnvConfig.runtimeregistry, so the next search short-circuits there. The write is fill-or-confirm only, and resolution traces which of the two applies whenever the resolved registry differs from R:
EnvConfig.runtimeregistry before | Deploy records | Trace |
|---|---|---|
| empty | the registry the chart resolved at | deploy: recording runtime registry <resolved>, where the runtime chart resolved, rather than R, where the search started |
set (so R = it) and the chart resolved at R | the same value (no change) | none — nothing decided |
set (so R = it) and the chart resolved at P | the value already there, unchanged | deploy: the env's runtime registry <recorded> stands; the runtime chart resolved from <resolved> instead (`erun init <tenant> <env> --runtime-registry <resolved>` changes it) |
A deploy that did not search — a repo-local chart, a chart stated via --runtime-chart / EnvConfig.runtimechart, or a component-only rollout — records the registry it pulled the runtime image from, which is the provenance --current re-addresses.
Runtime image override — --runtime-image
--runtime-image <ref> installs the runtime running the given image via the canonical published erun-devops chart (the image rides in as imageOverrides.erun-devops), pinned to --version — even when the env carries a repo-local <tenant>-devops chart, which the override deliberately bypasses. This lets an operator bootstrap an environment on the canonical ERun base image (or any external image) before the env's own <tenant>-devops image has been built and pushed, then switch to the tenant image once it exists. It mirrors erun open --runtime-image but on the pure deploy primitive (it does not imply anything beyond the deploy).
| Flag | Type | Default | Validation | Persists to |
|---|---|---|---|---|
--runtime-image <ref> | string (OCI image ref) | unset → the env's resolved runtime image (repo-local chart, EnvConfig.runtimeimage, or the tenant-umbrella default). | Same reference rules as erun init --runtime-image: a full reference (registry path and/or tag present) is used verbatim; a bare/tagless reference is qualified against the env's registry and pinned to <version> (never :latest). | EnvConfig.runtimeimage, so a later open/redeploy addresses the same image. |
The dry-run trace names the decision: deploy: bypassing the repo-local runtime chart for the runtime image override <ref>; using published chart <chart> version <v>, followed by deploy: runtime image override <ref>:<v> (imageOverrides.erun-devops). The desktop runtime dialog threads this flag automatically when the operator picks the ERun-base entry in the version picker (#697); picking the env's own <tenant>-devops image deploys the env's own chart with no override.
Default runtime image for a tenant umbrella
With no --runtime-image and no EnvConfig.runtimeimage, deploy resolves imageOverrides.erun-devops from the published chart it is installing:
- Deploying the tenant's own
charts/<tenant>-devopsumbrella (rung 1 of the runtime chart search) → deploy defaults the image to the umbrella's own<registry>/<tenant>-devops:<version>.erun pushpublishes the umbrella and its<tenant>-devopsimage together on the tenant version line, so the chart's identity names the image; building and pushing it is sufficient. Trace:deploy: defaulting runtime image to the <tenant>-devops chart's own image <ref> (imageOverrides.erun-devops), re-scoped into the deploy as--set-string erun-devops.imageOverrides.erun-devops=<ref>. - Deploying the shared
charts/erun-devopschart (no tenant umbrella published) → no override is set; the chart's own default image runs. An image-only build env therefore still points at its image throughruntimeimage.
An explicit runtimeimage (or --runtime-image) always wins over this default — so a tenant that publishes its own umbrella can still pin a different image (e.g. a hotfix build) by setting the field, which traces the runtime image override line above rather than the defaulting line. This default is why a <tenant>-devops umbrella deploy runs the tenant's own image without any runtimeimage, instead of silently falling back to a stock erun-devops:<tenant-version> the tenant line never published (which would ImagePullBackOff).
Runtime chart override -- --runtime-chart
--runtime-chart <ref> names the runtime chart as its own deploy coordinate instead of deriving it from --version. ERun has four coordinates in play -- chart repository, chart version, image repository, image version -- and --version normally collapses all four, which is correct whenever erun push published the chart and image as a pair. It is wrong the moment they ship on different release lines: a project whose <tenant>-devops image is versioned on the project's own line (9.9.9-snapshot-<ts>) has no chart at that version and never will, so the published-chart lookup resolves nothing and the deploy fails FetchReference … not found. With this flag the operator states the chart (repository, and optionally version) while --version keeps stamping the env's runtime version and tagging the image.
| Flag | Type | Default | Validation | Persists to |
|---|---|---|---|---|
--runtime-chart <ref> | string (OCI chart ref, optional :<version> suffix) | unset → the runtime chart search at --version. | An oci:// scheme is added when absent. The version is split from the last path segment only, so a registry port (registry.example:5000/charts/erun-devops) is not read as a version. No version → the chart resolves at --version. | Nothing — run-only, deliberately not persisted, so an env's recorded state never implies a chart it was not deployed with. |
The override applies to the runtime release only; component charts continue to resolve at --version. It composes with --runtime-image and with EnvConfig.runtimeimage, which is how each artifact ends up on its own line — the dry-run then carries both decisions:
deploy: runtime image override registry.example/acme/team-devops:9.9.9-snapshot-20260101010101 (imageOverrides.erun-devops)
deploy: runtime chart override oci://ghcr.io/sophium/charts/erun-devops version 1.2.3
helm upgrade --install … oci://ghcr.io/sophium/charts/erun-devops --version 1.2.3 …
--components value set and selection precedence
erun deploy is opt-in only: it deploys exactly the resolved selection and nothing else — no chart deploys "by default" beyond that selection. Validation of --components depends on the deploy path:
- Local path (a repo with source): the selectable universe is every chart directory discovered under
<tenant>-devops/k8s/plus the runtime aliases<tenant>-devopsanderun-devops(the runtime resolves to a repo-local<tenant>-devops/erun-devopschart, or the publishederun-devopschart when neither exists — the same dual-lookuperun openuses). A name matching no discovered chart and no runtime alias is rejected before any deploy runs withunknown deploy component "<name>"; valid components for this environment are: <sorted universe>, so typos and stale saved entries surface immediately. - Sourceless path (a remote/runtime env, installing by reference from the registry): there are no local charts to validate against, and a tenant may publish its own component charts beyond the fixed platform set, so the selection is trusted — any name installs
oci://<registry>/charts/<name> --version <v>. A name whose chart was never published at the version surfaces at deploy time asMISSING_CHART_IN_REGISTRY(an actionable "that version has no published chart" error), not an up-front rejection.
The selection resolves by precedence — the first non-empty tier wins entirely; tiers do not merge:
--components <a,b,…>— the explicit one-shot selection for this run.EnvConfig.deploy.components— the environment's saved per-machine default (erun init --components <a,b,…>, or the desktop app's Runtime-tab checklist; see Configuration ·deploy.components).ProjectConfig.environments.<env>.k8s.deployments[]— the repo deployment plan.- Empty (none of the above name anything) → the runtime chart alone, which bootstraps or heals the environment.
A chart deploys iff its component name is in the resolved selection. The runtime deploys only when the selection names a runtime alias, or when the selection is empty (tier 4) — an explicit selection that omits the runtime deploys the named components without it. erun-powerdns is the platform's authoritative DNS singleton; it runs the gpgsql backend against erun-backend-postgres, so sequence it after postgres in the plan. erun-zitadel is the platform's hosted IdP singleton, sequenced after postgres for the same reason (its own zitadel database on the shared instance); it renders one pod carrying both Zitadel core and the separate Login V2 container, a Service, and one Ingress routing /ui/v2/login to login and everything else to core, and it refuses to render without zitadel.masterkeySecretName naming an existing Secret and an auth host resolvable from the platform: block. The dry-run trace names the tier: deploy: component selection source <tier>; deploying the runtime chart alone (empty selection) or deploy: component selection source <tier>; components <a, b, …>.
Tiers never merge, so a saved tier-2 selection permanently shadows a richer tier-3 plan — the same divergence a --components flag run once and never cleared can leave behind. Whenever the resolved source is tier 2 and the repo plan (tier 3) names components the saved set omits, a second trace line says so at normal verbosity: deploy: saved components shadow the repo plan; plan also names <a, b, …>. erun init --components '' (an explicit empty value, not an omitted flag) clears the saved selection outright and returns the environment to the plan; nothing else exists today that resets it.
MCP-auth stickiness and the downgrade guard
erun deploy renders the chart from scratch (no helm --reuse-values), so any mcpAuth.* value it does not set falls back to the chart default — off. Because the edge's raw tool executes commands in the pod, an omission is a privilege downgrade, not a cosmetic one. The setting is therefore resolved from the environment, with an explicit opt-out:
| Input | Resolution |
|---|---|
--mcp-auth-public-key <path> (MCP deploy mcp_auth_public_key input) | Trust that key. The path is persisted to EnvConfig.mcpauthpublickeypath at the point the deploy applies the key — after the <release>-mcp-auth Secret apply, before the helm upgrade — so a rollout that fails afterwards still leaves the environment naming the key its release trusts. Trace: deploy: mcp auth: recording the public key <path> on <tenant>/<environment>, emitted only when the recorded value would change. |
Neither flag, EnvConfig.mcpauthpublickeypath set | Rethread the recorded key. Trace: deploy: mcp auth: rethreading the env's recorded public key <path>. |
--no-mcp-auth (MCP deploy no_mcp_auth input) | Resolve no authentication and clear mcpauthpublickeypath — the clear is written after the unauthenticated release has rolled out, since only then has the edge actually stopped trusting the key. Trace: deploy: mcp auth disabled by request; …. |
There is one signing mechanism — a file://-issued key — used by two callers: the desktop passes its own key via --mcp-auth-public-key, and a hosted environment's server-side deploy Job passes the backend's own MCP-signing public key (mcptoken.Signer) the same way, automatically, so the console's minted tokens verify with no Operator action. Both write the same mcpAuth.* chart values; only the key's origin differs.
Downgrade guard. When the resolved plan has authentication off and --no-mcp-auth was not given, deploy reads the live release's values (helm get values <release> -o json) and fails at resolution if mcpAuth.enabled is true — the case of an environment that enabled authentication before the key was recorded. Error code MCP_AUTH_DOWNGRADE_REFUSED; the message names what the release trusts, resolved from the same read plus the release's own Secret:
-
mcpAuth.issueris an OIDC (https://) issuer — only possible on a legacy or hand-configured release, since erun has no supported way to write one — the message names that issuer and says so, pointing at--mcp-auth-public-keyto switch the release onto the key-based path instead. Trace:deploy: mcp auth: release <release> authenticates against the OIDC issuer <issuer>; no local key is involved. -
Otherwise the release trusts a desktop key, so deploy reads it out of
mcpAuth.secretName(defaulting to<release>-mcp-auth) withkubectl get secret <name> -o jsonpath={.data.desktopid\.pub}and compares it byte-for-byte with this host's desktop identity public key (<user config dir>/ERun/desktopid.pub):- Match → the message names that path, and its
sha256, as the key to pass to--mcp-auth-public-key— the match is also what says re-supplying it keeps the edge's existing trust rather than rotating it. Trace:deploy: mcp auth: release <release> trusts this host's desktop identity key <path>. - No match → the message names the Secret and the key's
sha256, and says it is not this host's desktop identity key. - Secret unreadable → the message names the Secret alone. Trace:
deploy: mcp auth: secret <name> could not be read.
The
--no-mcp-authopt-out is named in every case. The read runs only on that path (an authenticated deploy pays nothing) and a release that cannot be read imposes no constraint, so an unreachable cluster never blocks a deploy.ERUN_MCP_AUTH_LIVE_PROBE_OVERRIDEis the integration-suite seam that answers the read without a cluster; it is a test seam, not a production knob. - Match → the message names that path, and its
The guard is scoped to explicit deploy requests (erun deploy, erun upgrade, erun publish, open --deploy, build --deploy). erun open's heal-redeploy rethreads a recorded key but is never blocked by the guard — it must still be able to hand over a shell.
In-pod guard for local-agent environments
The erun config store inside a runtime pod is a projection of the ERUN_* env vars the chart injects (see erun doctor --sync-config), not the host config that defines the environment. For a local-agent environment that projection is missing everything that shapes the env — the hostPath worktree, the local port range, the pod resource limits, and the runtime registry — so an in-pod resolve silently substitutes defaults and the rollout reshapes the environment (and cuts the MCP channel that issued it).
erun deploy refuses that combination at resolution: error code IN_POD_LOCAL_AGENT_RUNTIME_DEPLOY, naming the host command to run instead. The guard fires only when all of the following hold, so nothing else regresses:
- The environment's
typeislocal-agent. - The process is inside an erun runtime pod, identified by the chart-set
ERUN_TENANT+ERUN_ENVIRONMENTpair, and they name this environment. (A kubeconfig or anin-clustercontext is not the signal — both exist off-pod.) - The resolved specs include the runtime chart (
<tenant>-devops). A component-only selection is unaffected.
A remote-agent environment owns its worktree inside the pod, so it keeps deploying itself in-pod.
Deploy-plan resolution
The deploy plan comes from ProjectConfig.environments.<env>.k8s.deployments[] (see Configuration · environments.<env>.k8s.deployments[]). It serves two roles: selection tier 3 above — its charts are the deploy selection when no --components and no saved deploy.components apply — and the ordering source: steps deploy in listed order, and a list within a step deploys in parallel. When the field is absent, erun deploy falls back to ordering by chart dependency declarations; on a tie, alphabetical by component name.
Skip-helm semantics
erun deploy skips the helm upgrade --install for a component when:
- The resolved version equals the version the env already runs (no rollout would change anything), and
- The chart directory (
<tenant>-devops/k8s/<component>/) has no diff against the last successful deploy's snapshot, and --forceis not set.
The skip emits result: skipped (no change) in the trace. deploy never pushes, so there is no docker push to skip.
Rollout wait and pod monitoring
erun deploy runs helm upgrade --install --wait --wait-for-jobs --timeout <t> and, concurrently, polls the release's pods to fail fast on a real failure while staying patient through a slow image pull.
Timeout resolution (<t>), highest precedence first:
| Source | Value |
|---|---|
--rollout-timeout <dur> flag (MCP deploy timeout input) | Per-deploy override. Go duration (e.g. 8m, 90s); a malformed or non-positive value aborts before any rollout: invalid rollout timeout "<v>", exit 1. |
EnvConfig.deploy.timeout | Per-environment default (see Configuration · EnvConfig). A malformed value aborts at spec resolution: invalid environment deploy timeout "<v>", exit 1. |
| Built-in default | 5m0s. |
The resolved value is the helm --timeout argument (visible in the dry-run helm command line) and the waiting for helm rollout (timeout <t>)... real-run line. It is the upper bound on how long a still-progressing rollout waits; it does not apply to the rollback path (helm rollback --wait --timeout 2m0s) or the shell-launch wait, which carry their own fixed timeout.
Pod monitor. While helm waits, erun polls kubectl get pods -o json for the release (filtered by the meta.helm.sh/release-name annotation) every 2s (ERUN_DEPLOY_POD_WATCH_INTERVAL, default 2s, floor 100ms). It classifies each init + main container state and decides between keep waiting and abort early:
- Keep waiting (image still pulling). A container in
ImagePullBackOfforErrImagePullwhose message is not a permanent rejection is treated as a pull in progress — a large image on a slow or rate-limited registry legitimately cyclesPulling → ErrImagePull → ImagePullBackOff → retry. The watcher does not abort; it keeps waiting up to<t>and printspod <p>: <c> Pulling image (<reason>)status lines so the wait is visible.helm --timeoutis the only bound on this case. - Abort early (real failure). When a container reaches a state that will not recover, the watcher sends
SIGINT(thenSIGKILLafter 2s) to the helm process and the deploy fails immediately withdeploy failed early: pod <p> container <c> <reason>: <message>rather than waiting out<t>. The terminal states are:InvalidImageName,CreateContainerConfigError,CreateContainerError,RunContainerError,ContainerCannotRun— config/runtime errors that no wait fixes.CrashLoopBackOffoncerestartCount ≥ 2(a single transient init crash is tolerated). The last terminated message is surfaced.- A permanent image-pull rejection: an
ImagePullBackOff/ErrImagePullmessage containingmanifest unknown,not found,repository does not exist,pull access denied,unauthorized,authentication required,forbidden,denied,invalid reference format, orno such image— a missing tag, absent repository, or bad credentials that retrying will never resolve. (Transient causes — timeouts, DNS blips, connection resets, TLS handshake failures — are deliberately not treated as permanent, so a briefly unreachable registry keeps waiting.)
The monitor is best-effort: a transient kubectl get pods error is ignored (it never aborts a deploy helm would otherwise drive to success). In --dry-run the watcher action is traced (deploy: watching pods in <ns> …) and the kubectl get pods -o json command is shown, but no polling happens.
Immutable-selector recovery
A Kubernetes Deployment.spec.selector is immutable: helm cannot patch a release whose installed selector differs from the chart's rendered selector, and aborts the upgrade with Deployment.apps "<name>" is invalid: spec.selector: … field is immutable. This happens when an environment was first installed under a chart that rendered a different selector than the one now being applied (e.g. a pre-cutover per-tenant chart that labelled pods app: <release> versus a chart that hardcoded app: erun-devops, or vice-versa).
erun deploy detects this specific failure and recovers automatically, in erun-common so both CLI and MCP flows get it:
- It parses the offending Deployment name from helm's error and deletes only that Deployment (
kubectl delete deployment <name> --namespace <ns> [--context <ctx>] --ignore-not-found). The release's PVCs (<release>-home,<release>-docker,<release>-worktree), ServiceAccount, and RBAC are separate objects and are not touched, so build cache and/home/erunsurvive. - It retries the
helm upgrade --installonce. With the Deployment gone, helm creates it fresh with the new selector.
The recovery is bounded to a single retry (the delete removes the conflict, so the retry cannot hit the same error) and fires only for an immutable spec.selector change — an unrelated immutable-field error is not caught and never triggers a delete. It runs only in real execution, not --dry-run (the conflict is a helm side-effect failure, not a pre-action decision). The trace names the decision on the audit channel: deploy: Deployment <name> selector is immutable and changed; deleting it (PVCs preserved) and retrying the upgrade; the literal kubectl delete is logged at -vv. If the retried upgrade fails for any other reason, that error surfaces as HELM_UPGRADE_FAILED.
Error codes
| Code | Cause | Exit code |
|---|---|---|
NO_VERSION | Neither --version nor --current given. deploy does not mint a version, so there is nothing to install. | 1 |
NO_CURRENT_VERSION | --current given but the env has no recorded runtime version yet. Deploy a specific --version once to seed it. | 1 |
CLUSTER_UNREACHABLE | Same as erun open. | 2 |
MISSING_IMAGE_IN_REGISTRY | A chart references <registry>/<component>:<version> that does not exist (and was never built/pushed). | 1 |
RUNTIME_CHART_NOT_CONFIRMED | The runtime chart search confirmed no coordinate published at the requested version — refused before any helm command runs. The message names each coordinate probed and whether it was confirmed absent or could not be read; see that section for the full contract. | 1 |
MISSING_CHART_IN_REGISTRY | A chart resolution did confirm a coordinate (a component chart, always trusted on the sourceless path; or a runtime chart the search resolved) but the helm pull for it failed anyway — a tag evicted between the probe and the pull, or a genuinely unpublished component chart version. The message names each coordinate: record where ERun's artifacts live (erun init --runtime-registry), push the version from the project that owns the chart, or name the chart with --runtime-chart. For a component chart, push the version first — push publishes image and chart together. | 1 |
HELM_UPGRADE_FAILED | A step in the plan failed (or helm's own --timeout elapsed while the rollout was still not ready); later steps are not executed. | 2 |
ROLLOUT_CONTAINER_FAILED | The pod monitor observed a terminal container failure (crash loop, config/runtime error, or a permanent image-pull rejection) and aborted the rollout early instead of waiting out the timeout. The message names the pod, container, and reason. | 2 |
INVALID_ROLLOUT_TIMEOUT | --rollout-timeout or EnvConfig.deploy.timeout is not a positive Go duration. Nothing runs. | 1 |
MCP_AUTH_DOWNGRADE_REFUSED | The live release has mcpAuth.enabled=true but the resolved plan has no authentication, and --no-mcp-auth was not given. Nothing runs. See MCP-auth stickiness. | 1 |
IN_POD_LOCAL_AGENT_RUNTIME_DEPLOY | A local-agent environment's runtime chart was deployed from inside that environment's own runtime pod, where the config store is not authoritative. Nothing runs. See In-pod guard. | 1 |
erun doctor
Flags
| Flag | Type | Default | Effect |
|---|---|---|---|
--dry-run | bool | false | Run the inspection; print the recovery plan; do not execute it. |
-y | bool | false | Auto-approve every offered recovery action. |
--clear-pending-helm | bool | false | Run the clear-pending-helm recovery without prompting (see Deploy recovery actions). |
--rollback | bool | false | Run the rollback recovery without prompting (see Deploy recovery actions). |
--sync-config | bool | false | In-pod only. Reconcile the on-disk env config with the helm-injected ERUN_* env vars (injected env wins): rewrite the projected keys (type, kubernetescontext, cloudprovideralias, managedcloud, cloud provider/context blocks, idle, runtimeregistry, containerregistries, disablebuildscript), preserving every unprojected key. Reports per-key drift as missing / wrong / legacy; under --dry-run the file writes are traced but not performed. Short-circuits the remote-init flow. |
--restore-env-config-from-backup | string | "" | Restore the target environment's config.yaml from a dated backup (YYYY-MM-DD) or an absolute path, before any tenant/env work so a corrupted env config can be recovered first. Requires explicit <tenant> <environment> args. Under --dry-run the copy is traced but not performed. Errors: missing explicit tenant+environment → --restore-env-config-from-backup needs an explicit tenant and environment (exit 1); no matching backup → no env config backup matches "<date>" for <tenant>/<env> (exit 1). The MCP doctor tool exposes the same operation as the restoreEnvConfigFromBackup input. |
--repair-workspace-sync | bool | false | For a remote-agent env with sshd.workspacesync.enabled, diagnose and repair the host mirror's SSH provisioning without a helm redeploy: resolve/persist the SSH public key, write the local ~/.ssh/config alias, install the pod's authorized_keys through the runtime container, and ensure the SSH port-forward. When SSH still can't reach the pod afterwards, it names erun sshd init as the remaining step (the redeploy this repair deliberately skips). Under --dry-run every action is traced and nothing runs; when it is the only action requested, doctor stops after it (no deploy diagnosis or prune prompts). Host-side provisioning, so it is CLI-only — the MCP doctor tool does not expose it, mirroring erun sshd init. |
Check catalogue
Each check returns one of ok, missing, error (parse failure, permission denied), or skip (not applicable in this context).
Local-host checks (run when ERUN_REPO_REMOTE is not true)
| Check id | What it inspects | Recovery if missing |
|---|---|---|
config.tenant | ~/.config/erun/<tenant>/tenant.yaml exists and parses. | Suggests erun init <tenant>. |
config.environment | ~/.config/erun/<tenant>/<env>/config.yaml exists and parses. | Suggests erun init <tenant> <env>. |
config.project | <projectroot>/.erun/config.yaml exists. | Suggests erun init. |
cluster.kube_context | EnvConfig.kubernetescontext is in ~/.kube/config. | Lists available contexts. |
cluster.runtime_pod | A pod matching the runtime-chart's labels is Running in <tenant>-<env>. | Suggests erun open. |
workspace.project_root | <projectroot> exists and is a git repo. | No automatic recovery. |
In-pod checks (run when ERUN_REPO_REMOTE=true)
| Check id | What it inspects | Recovery if missing |
|---|---|---|
bootstrap.marker | /home/erun/.erun/<tenant>/<env>/bootstrap.yaml exists and parses. | Suggests re-running erun init --remote from the host. |
workspace.project_root | The in-pod project root exists. | Offers to git clone from the marker's recorded remote. |
workspace.git_checkout | The checkout's HEAD is on the marker's recorded branch. | Offers to git checkout the branch. |
ssh.keypair | ~/.ssh/id_ed25519 and .pub exist. | Offers ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519 -N ''. |
ssh.codecommit_key | When the marker recorded a CodeCommit host: ~/.ssh/id_rsa (RSA, not ed25519) is registered with the IAM user. | Offers to generate and upload via aws iam upload-ssh-public-key. |
Deploy recovery actions
After the read-only deploy diagnosis (helm release status + runtime pods), doctor can run two recovery actions that mutate the live release. They are alternative fixes for different failure modes, not additive steps — clearing a pending lock leaves the release at its last deployed revision, so a rollback run straight after would step back a further revision. --clear-pending-helm and --rollback are therefore mutually exclusive; passing both aborts with --clear-pending-helm and --rollback are alternative recoveries; pass only one (exit 1, nothing runs).
Gating: each action runs non-interactively with its flag. With no flag, doctor inspects the helm status and prompts for the single recommended action — pending-install/pending-upgrade/pending-rollback → clear pending; a present-but-unhealthy release (failed, superseded, …) → rollback; a healthy (status: deployed), missing (not found), or unreadable release → no destructive prompt at all. It never offers both at once. Under --dry-run the exact command is traced and nothing runs.
| Action | Flag | Command run | Use when |
|---|---|---|---|
| Clear pending helm release | --clear-pending-helm | kubectl [--context <ctx>] --namespace <ns> delete secrets,configmaps -l 'owner=helm,name=<release>,status in (pending-install,pending-upgrade,pending-rollback)' --ignore-not-found | A deploy died mid-upgrade and left the release locked in a pending state, so the next erun deploy refuses to start. |
| Roll back to last successful revision | --rollback | helm rollback <release> --namespace <ns> [--kube-context <ctx>] --wait --timeout 2m0s | The current revision is bad or never converged and a previous revision was healthy. |
<release> is the runtime release name for the tenant; <ns> and <ctx> are the resolved env namespace and kube-context. To rebuild and roll out fresh images instead of recovering the existing release, re-run erun deploy --force — the desktop's failed-deploy card surfaces that as its Force rebuild & redeploy button.
Both actions are also exposed on the MCP doctor tool via the clearPendingHelm and rollback boolean inputs.
Exit codes
| Code | Meaning |
|---|---|
0 | All checks ok, or every missing check was recovered. |
1 | At least one check missing and recovery declined (or --dry-run). |
2 | At least one check error (parse failure, permission denied). Inspect the trace to find which. |
erun observe
Reports an environment's Kubernetes state, read-only: every underlying call is kubectl [--context <ctx>] --namespace <ns> get <resource> [name] -o json, never anything that mutates. Same operation as the MCP observe tool (see MCP overview § observe).
Flags
| Flag | Type | Default | Effect |
|---|---|---|---|
--tenant <t> | string | current scope | Target tenant. |
--environment <e> | string | current scope | Target environment; requires --tenant. |
--secret <name>=<key> | string, repeatable | none | Check Secret <name> for key <key>'s presence. Malformed (missing =, empty name, or empty key) aborts with --secret must be name=key, got "<value>" (exit 1) before any kubectl call. |
Resolution and output shape
Resolves tenant/environment/namespace the same way every other typed command does (ResolveOpen), then issues, in order: get pods, get resourcequota, get limitrange, get ingress, get certificates.cert-manager.io, then one get secret <name> per --secret check. --output json emits:
{
"tenant": "myapp", "environment": "prod", "namespace": "myapp-prod",
"pods": [ { "name": "web-0", "phase": "Running", "ready": true, "restartCount": 0, "reason": "" } ],
"resourceQuotas": [ { "name": "erun-quota", "hard": { "limits.cpu": "4" }, "used": { "limits.cpu": "1" } } ],
"limitRanges": [ { "name": "erun-limits", "limits": [
{ "type": "Container", "max": {}, "min": {}, "default": { "cpu": "1" }, "defaultRequest": { "cpu": "100m" } }
] } ],
"ingresses": [ { "name": "web", "hosts": ["prod.example.com"],
"tls": [ { "hosts": ["prod.example.com"], "secretName": "web-tls" } ] } ],
"certificates": [ { "name": "wildcard", "ready": false, "reason": "Issuing", "message": "…",
"secretName": "wildcard-tls", "dnsNames": ["*.prod.example.com"], "orders": [ /* see below */ ] } ],
"secrets": [ { "name": "db-credentials", "key": "password", "exists": true, "hasKey": true, "error": "" } ]
}
reason on a pod is the container's waiting/terminated reason if present, else the PodScheduled=False reason (a pod never admitted to a node has no container status to read a reason from), else the Ready=False condition's reason. secrets is omitted entirely when no --secret was given.
The Certificate → CertificateRequest → Order → Challenge walk
certificates[].orders is populated only when that Certificate's status.conditions[type=Ready] is not True. The walk, run once against a fresh listing of each resource kind in the namespace (not once per certificate):
- List
certificaterequests.cert-manager.io; filter to the ones labelledcert-manager.io/certificate-name=<certificate>; take the one with the latestmetadata.creationTimestamp(a Certificate can be reissued, leaving stale requests behind — only the newest one's chain is live). None matching →ordersis empty. - List
orders.acme.cert-manager.io; keep the ones whoseownerReferencesinclude{kind: CertificateRequest, name: <request from step 1>}. - For each such Order, list
challenges.acme.cert-manager.ioand keep the ones owned (ownerReferences) by that Order. - Each reported order carries
state/reasonfromstatus; each challenge carriestype/dnsNamefromspecandstate/reasonfromstatus—reasonis the field that explains a stuck issuance (e.g. a webhook solver's RBAC denial), which is otherwise three separatekubectl getcalls away.
A cluster with no cert-manager CRDs installed (kubectl reports "the server doesn't have a resource type" / "no matches for kind") reports certificates: [] rather than erroring — a cluster simply has no certificates to walk.
Secret presence checks
Each --secret <name>=<key> becomes one kubectl get secret <name> -o json, read only for its key names (data/stringData), never a value:
| Outcome | exists | hasKey | error |
|---|---|---|---|
| Secret and key both present. | true | true | "" |
| Secret present, key absent. | true | false | "" |
| Secret not found. | false | false | "" |
| Any other failure (e.g. RBAC denial reading the Secret). | false | false | the kubectl error, so a permission problem is never reported indistinguishably from "does not exist" |
Error behaviour
| Failure | Behaviour |
|---|---|
| Tenant/environment can't be resolved. | Errors before any kubectl call. |
--secret isn't name=key. | Errors before any kubectl call, naming the malformed value. |
get pods / resourcequota / limitrange / ingress fails (namespace or cluster unreachable). | Errors naming the failed call; nothing is reported. |
get certificates.cert-manager.io fails because the CRD isn't installed. | certificates: []; not an error. |
get certificates.cert-manager.io fails for another reason. | Errors naming the failed call. |
erun usage
Reports an environment's live CPU, memory, and disk usage, read from the runtime container's own cgroup v2 accounting and a statfs of its workspace mount. Same operation as the MCP usage tool (see MCP overview § usage).
No metrics-server is required. Unlike kubectl top (which reports error: Metrics API not available on any cluster without the metrics-server add-on — every local orbstack/k3s-style cluster included), the underlying kubectl execs a fixed diagnostic script into the runtime pod's erun-devops container and reads /sys/fs/cgroup and df directly. Nothing here can mutate the cluster.
Flags
| Flag | Type | Default | Effect |
|---|---|---|---|
--tenant <t> | string | current scope | Target tenant. |
--environment <e> | string | current scope | Target environment; requires --tenant. |
--interval <seconds> | float | 1 | CPU sample window, clamped to [0.1, 30]. cpu.stat's usage_usec is read, the window elapses, then it is read again, so utilisation is a rate over the interval rather than a meaningless cumulative counter. |
Resolution and output shape
Resolves tenant/environment/namespace the same way every other typed command does (ResolveOpen), then runs one kubectl exec -c erun-devops deployment/<tenant>-devops -- /bin/sh -lc <script> against the resolved namespace. --output json emits:
{
"tenant": "myapp", "environment": "prod",
"cpu": { "quotaCores": 1, "utilizationPercent": 12.4, "intervalSeconds": 1 },
"memory": { "currentBytes": 413589504, "peakBytes": 1027301376, "limitBytes": 2147483648, "percentOfLimit": 19.3, "oomKills": 0 },
"disk": [ { "mount": "/home/erun", "totalBytes": 202991730688, "usedBytes": 101495865344, "percentUsed": 50.0 } ],
"warnings": []
}
cpu.quotaCores is cpu.max's quota ÷ period; memory.percentOfLimit is memory.current ÷ memory.max; disk[].percentUsed is df's used ÷ total for the watched mount (the runtime chart's HOME, /home/erun, is the only mount watched today). warnings is omitted (empty) unless a threshold below is crossed.
Unavailability, not failure
Every field group reports its own unavailability rather than failing the whole call — cgroup v1, an unlimited limit, and a file the exec script could not read are all normal on some clusters, not errors:
| Condition | Reported as |
|---|---|
/sys/fs/cgroup is not cgroup2fs (cgroup v1, or absent). | cpu.unavailable and memory.unavailable name the reason; every other CPU/memory field stays zero. |
cpu.max's quota is max (unlimited) or the file could not be read. | cpu.unavailable names the reason — there is no quota to measure utilisation against. |
memory.max is max (unlimited). | memory.unlimited: true; memory.limitBytes/percentOfLimit stay zero rather than a fabricated percentage. |
memory.current could not be read. | memory.unavailable names the reason. |
df reported nothing for the watched mount. | that entry's disk[].unavailable names the reason. |
memory.oomKills comes from memory.events' oom_kill counter — a real kill count, not a guess made after the fact.
Named warning thresholds
A reading nobody acts on is decoration, so warnings fires a plain-language entry (not a code) when:
| Threshold | Reasoning |
|---|---|
memory.percentOfLimit ≥ 85%. | A container this close to its limit is one build step away from an OOM kill. |
memory.peak ÷ memory.limitBytes ≥ 95%. | memory.peak is a high-water mark, so a near-limit peak matters even after current usage drops back down. |
any disk[].percentUsed ≥ 90%. | Disk fills silently — no kernel counter tracks "close calls" the way memory.peak does for RAM — so the warning threshold sits ahead of the failure rather than reacting to it. |
memory.oomKills > 0. | Always reported: a kill already happened. |
Error behaviour
| Failure | Behaviour |
|---|---|
| Tenant/environment can't be resolved. | Errors before any kubectl call. |
| The namespace, deployment, or cluster is unreachable. | Errors naming the failed kubectl exec. |
erun outputs
erun outputs lists and downloads files an agent produced in an environment's runtime pod outputs directory ($ERUN_OUTPUTS_DIR, default /home/erun/.erun/outputs). Both subcommands resolve the pod from tenant/environment scope and read it over kubectl exec; the MCP outputs_list/outputs_download tools cover the same operations for in-pod callers (which read the filesystem directly).
erun outputs list
| Flag | Type | Default | Effect |
|---|---|---|---|
--tenant <t> | string | current scope | Target tenant. |
--environment <e> | string | current scope | Target environment; requires --tenant. |
--path <dir> | absolute path | $ERUN_OUTPUTS_DIR → /home/erun/.erun/outputs | Pod directory to list. Must be absolute and free of ... |
--limit <n> | int | 0 (all) | Cap on entries returned, newest-first. |
Lists one directory one level deep over kubectl exec … find <dir> -maxdepth 1, sorted newest-first by mtime. A missing directory yields an empty result, not an error. --output json emits {dir, entries:[{name,path,size,modTime,isDir}], total, truncated}.
erun outputs download
| Flag | Type | Default | Effect |
|---|---|---|---|
<name> (arg) | string | required | Entry to download, a single path segment under the directory. A name with directory components is reduced to its base segment; ./../empty are rejected. |
--tenant / --environment / --path | — | — | As for list. |
--dest <local-path> | path | current directory | Local file or directory to write to. (--dest, not --output, which is the global mode flag.) |
--force | bool | false | Overwrite an existing local destination. |
A file streams as base64; a folder streams as a tar.gz archive (saved as <name>.tar.gz). The payload is SHA-256'd and capped at 100 MB (MaxRuntimeOutputBytes) — a larger file errors before transfer. --output json emits {name, dest, size, sha256, isArchive, archiveFormat}. Both subcommands support --dry-run (traces the kubectl exec argv + script and the planned destination; no I/O).
erun inputs
erun inputs upload is the inverse of erun outputs download: it streams a file from this host into an environment's runtime pod over kubectl exec -i (stdin), never through argv or a base64 blob in a tool argument. It has no in-pod MCP counterpart — the edge runs in the pod and has no path back to the operator's filesystem — but an MCP-connected orchestrator reaches the same transfer through the inputs_upload local tool erun mcp proxy serves (see MCP overview § Host-served).
erun inputs upload
| Flag | Type | Default | Effect |
|---|---|---|---|
<local-path> (arg) | path | required | File on this host to upload; must exist and not be a directory. |
<remote-path> (arg) | absolute path | required, never defaulted | Full destination inside the pod, including the file name. Must be absolute and free of ... Not defaulted deliberately: a transfer can never silently land somewhere a background process (e.g. the workspace-sync mirror) reconciles away. |
--tenant <t> | string | current scope | Target tenant. |
--environment <e> | string | current scope | Target environment; requires --tenant. |
The remote script creates the destination directory if missing, writes to a same-directory temp file, and renames into place — so a killed transfer never leaves a partial file visible at the final path — then reports the written size and SHA-256. The command errors if that checksum (or size) disagrees with what was sent. --output json emits {remotePath, bytes, sha256}. --dry-run traces the kubectl exec argv and the upload script without sending anything (the local file must still exist to resolve its size).
erun cloud refresh
erun cloud refresh TENANT ENVIRONMENT re-injects the operator's short-lived AWS credentials into an environment's runtime pod. It is the non-leaking counterpart to the cloud_inject_aws_credentials MCP tool: the credential values are never inputs, so an Agent or a script can call it without writing a secret into a transcript.
| Argument / flag | Type | Default | Effect |
|---|---|---|---|
TENANT (arg) | string | required | Target tenant. No default-scope fallback — the target is always explicit. |
ENVIRONMENT (arg) | string | required | Target environment. |
--dry-run | bool | false | Resolve the plan, trace the deployment wait, the kubectl exec argv, and the write script, and exit without exporting credentials or touching the pod. |
--output | text | json | text | Global mode flag. |
Algorithm:
- Resolve the environment. Read
EnvConfig.cloudprovideralias; abortno AWS cloud provider aliaswhen empty. - Resolve the alias in the root config; abort when it is not configured, or when its
provideris notaws. - Resolve the AWS region (managed cloud context → kubeconfig context name → the alias's
ssoregion→ the region in an ECR registry host). An unresolved region is traced as<unresolved>and simply omitted from the written profile; it is not an error. kubectl wait --for=condition=Available deployment/<tenant>-devops(2 minute timeout).- Trace the
kubectl exec -i … /bin/sh -lc <script>argv and the script body. Under--dry-run, stop here. aws configure export-credentials --format process --profile <alias profile>to mint the credentials. A failure nameserun cloud login --alias <alias>, the usual cause being a lapsed SSO session.- Render the
[erun-host]profile block (access key, secret, session token, resolved region, and anx_erun_expirationmarkerdoctorreads back) and stream it to the pod on the exec's stdin. The script drops any existing[erun-host]section before appending, so a repeat refresh overwrites in place; every other profile in~/.aws/credentialsis preserved, and the file is left0600.
The credential material never appears in an argument, a trace line, or a golden file. The write script is a constant — it carries the profile name, not the values — so tracing it in full is safe.
| Failure | Behaviour |
|---|---|
| Environment carries no AWS alias. | Exit 1 before any cluster call; names erun cloud set <tenant> <env> --alias <alias>. |
| Alias not configured in the root config. | Exit 1, cloud provider alias … is not configured. |
| Alias is a Cloudflare alias. | Exit 1, host credential refresh applies to AWS aliases only. |
| Credential export fails (expired SSO). | Exit 1; the error names erun cloud login --alias <alias>. The pod's existing profile is untouched. |
| Runtime deployment not Available. | Exit 1 at the wait; nothing is written. |
erun open runs the same refresh for any environment with an AWS alias, after its deployment-presence check and after the wake that follows it (the credentials are written into the running pod, so there is nothing to write to until it is up). There it is best-effort: a failure is traced as a warning and the session still opens, because a lapsed SSO session degrades the environment but is not a reason to withhold the shell. erun deploy deliberately does not refresh — it is a pure primitive driven by orchestrators and erun release, often with no operator present and against environments nobody is about to use; the credentials file lives on the home PVC and survives the pod replacement a deploy causes, so a deploy invalidates nothing that a refresh would fix.
erun release
erun release orchestrates build → push → git-tag, in that order: it stamps the version and creates the commit and a local tag, builds the release-tagged images, reuses erun push to publish the multi-arch image manifest and the runtime chart at the release version, re-resolves each published manifest, and only then pushes the tag and branches. It has no chart-publishing step of its own. The ordering is the contract — a release that exits 0 means the announced version is deployable, and one that cannot publish fails with nothing public. See Release version policy for the version-pattern rules and the publishing contract; the erun release flag set is just --dry-run and --output, and is documented on the Operator page.
erun stop
erun stop scales an environment's runtime Deployment to zero, returning the runtime container's
resource limits and its unlimited dind sidecar's real consumption to the node. It is the
counterpart to erun open, which is the only thing that starts an environment again. There is
deliberately no MCP stop tool: the env's MCP edge runs inside the runtime container, so
stopping over MCP would kill the caller mid-call. Lifecycle is host-side, as it always has been for
deploy and open.
Flags
| Flag | Type | Default | Effect |
|---|---|---|---|
--tenant <name> | string | current scope | Target tenant. |
--environment <name> | string | current scope | Target environment. |
--dry-run | bool | false | Trace every action and decision; perform no side effect. |
--output json | string | text | Emit the structured result on stdout (see below). |
erun stop lifecycle algorithm
- Resolve the target the same way
erun opendoes (positional args, then--tenant/--environment, then the current scope). A missing Kubernetes context aborts. - No cloud-context preflight. Unlike every other cluster-touching command,
stopnever starts a stopped cloud context to reach the cluster. - Read the runtime Deployment's
spec.replicas/status.readyReplicas. An absent Deployment aborts withRUNTIME_NOT_DEPLOYED. - If the Deployment is already at
0replicas, skip to step 8 — steps 5–7 are the "this run actually reclaims capacity" path. - List the attached desktop sessions.
kubectl exec deployment/<tenant>-devopsruns the same in-pod session heartbeat probe the desktop app polls, and the ids it reports are traced and returned asendedSessions. They live in the pod, so the stop ends them; naming them makes that a stated consequence rather than tabs mysteriously going dark. An unreadable probe is traced and the stop continues — it is reporting, not a precondition. kubectl scale deployment/<tenant>-devops --replicas=0.- Confirm the stop took effect. Re-read
spec.replicas. Anything other than0aborts withSTOP_NOT_APPLIEDbefore the config write, soEnvConfig.stoppednever claims a stop the cluster did not keep and the command never reports success for a stop that did not happen. Skipped under--dry-run, which traces the check instead. - If
EnvConfig.stoppedis not alreadytrue, set it. This is the durable half: a bare scale patch is drift that the nexthelm upgradereverts, sodeployrenders the chart'sstoppedvalue from this field and reconcilesreplicasdeclaratively. - Emit
==> Stopped <tenant>/<env>and exit0.
Durability and the interaction with deploy
| Sequence | Result |
|---|---|
stop → open | The environment starts. open clears EnvConfig.stopped and scales the Deployment back to 1. |
stop → open --reconnect | The environment stays stopped. The reconnect aborts with RUNTIME_STOPPED; nothing is scaled and EnvConfig.stopped stays set. This is the sequence the desktop app produces on its own — a stop drops every attached session and each tab respawns open — so it is what makes a stop hold for an environment somebody has open. |
stop → deploy | The environment stays stopped. deploy threads --set stopped=true, so the chart renders replicas: 0. deploy installs a version; it does not decide whether the environment should be running. |
stop → deploy → open | The environment starts, on the version deploy installed. |
Every automatic stop inherits the same protection, because it is open that refuses rather than stop that defends. An idle-stop that scales an environment down records the same EnvConfig.stopped and drops the same sessions, and the reconnects it triggers decline for the same reason. The idle stop that ships today stops the whole cloud context rather than one Deployment, and it is covered by the same flag one layer up: --reconnect skips open's cloud-context start, so a reattach cannot restart the machine an idle policy just stopped.
Making deploy a wake would have been unreliable in one specific way: deploy skips the helm call
when the released version already matches, so a wake-on-deploy would fire or not depending on
whether anything changed. Reconciling the recorded intent instead is consistent whether the helm
call runs or is skipped.