---
sidebar_position: 15
title: "CLI Internals"
description: "How hermes_cli is shaped: slash dispatch, config loaders, the skin engine, the transactional update pipeline, and process-identity rules"
---

# CLI Internals

Companion to `hermes_cli/AGENTS.md` (the rules) — this page holds the longer explanations.

## Update pipeline

User-facing behaviour (receipts, `--plan`, snapshot modes) is in
[Updating](../getting-started/updating.md); `hermes_cli/AGENTS.md` carries the short form of the rules.

Fleet-update campaign #91277 (Aug 2026). A PR that weakens a stage must answer for the failure class
it guards. `plan → snapshot → apply → restart-per-kind → verify → report`

- **Plan** (`update_inventory.py`, `hermes update --plan`): read-only inventory — install kind, all
  profiles, every live gateway with supervisor + running code version. Deployment kinds are
  first-class: `git` updates in place; `docker`/`nix`/`apt` are NOT in-place-updatable and the
  updater reports the correct external command instead of fighting the deployment model.
- **Snapshot** (`backup.py`): pre-update quick snapshot for EVERY profile (the code swap + fleet
  restart touch all of them), each into its own `state-snapshots/`, identical file set, 1 GiB
  per-file cap, keep=1. **Never add a partial/tiered snapshot set** — mixed coverage creates
  torn-restore states across schema generations. Quick snapshots are FILE-LOSS RECOVERY (the
  per-profile cron-jobs safety net restores from them), NOT code-rollback insurance; `--backup`
  full mode owns rollback.
- **Apply**: git pull, or the Windows ZIP fallback — which fires ONLY when git itself failed
  (`_should_zip_fallback_on_update_error`, argv-classified; a dependency-install failure must never
  trigger a tree-clobbering re-download), REFUSES a dirty working tree (`-uall` + a pre-swap TOCTOU
  re-check — but classifies a `!!` line by whether the swap would destroy it: an ignored path under a
  root entry the ZIP does not ship (`.bytecode-fingerprint`, `.hermes-bootstrap-complete`,
  `hermes_agent.egg-info/`; tracked root entries stand in for the ZIP set before the download, the
  re-check gets the real one), a nested `__pycache__`/`node_modules`, or a `_ZIP_PRESERVED_NESTED`
  output is admitted; other ignored files under shipped dirs still block), and grafts the live nested
  build outputs (`_ZIP_PRESERVED_NESTED`: `apps/desktop/{release,dist,node_modules,build}`,
  `hermes_cli/web_dist`, `ui-tui/{dist,node_modules,packages/hermes-ink/dist}`, `web/node_modules`,
  `scripts/whatsapp-bridge/node_modules`) into the staged swap by hardlink (the GitHub source ZIP has
  none of them; without the graft the swap deletes them). Post-swap, the Desktop
  rebuild decision also trusts the build stamp under HERMES_HOME, so an install that already lost
  its artifacts in an earlier update is rebuilt instead of "forgotten" (#90495).
- **Restart-per-kind**: systemd and launchd restarts are FLEET-WIDE within the updating install (every
  `hermes-gateway*` unit / `ai.hermes.gateway*` LaunchAgent whose home is the updating root or one of its
  `profiles/<name>`), drain-first (SIGUSR1), with per-unit/per-label failure isolation. Restarting only the
  invoking profile's service leaves siblings on stale `sys.modules` until they crash — the largest dupe-PR
  cluster in the repo's history came from that bug. The fleet is bounded by HOME, not by namespace:
  `hermes_cli/update_fleet_scope.py` judges every unit/label/process by the home it actually runs on
  (live environ, unit `Environment=`, plist `HERMES_HOME`), and a runtime of another `HERMES_HOME` on the
  same account — a sibling install, the real `hermes-gateway.service` seen from a scratch home — is named and
  left alone, never restarted (#93349).
- **Verify**: gateways stamp `code_sha`/`code_version` into `gateway_state.json` on every
  runtime-status write (`gateway/status.py`); the updater compares each live gateway against the
  fresh checkout and prints a fleet version matrix. After the commit point (the tree moved) nothing
  fails `hermes update` (exit 0 unless the code was rolled back): a provably-stale or stopped
  gateway, a failed build/launcher/config migration/bytecode sweep/channel adoption/Windows resume
  each print `⚠`, land in the receipt as `outcome: "success"` with `followups: [{step, reason}]`,
  and keep their own obligation armed (`source-completion-pending` for the tail, the fleet-restart
  obligation for gateways) so the next launch or `hermes update` — including "Already up to date" —
  retries them. The owed-fleet-restart CLI startup warning is what keeps a mixed-version fleet from
  ever looking healthy. A completion process that could not be started (temporary directory,
  request file, spawn) or died without a correlated result (OOM, SIGKILL) is the same kind of
  debt: an owed `completion` follow-up with the source-update tail re-armed, exit 0. Exit 2
  (refused/concurrent) and exit 1 (nothing committed / rolled back) keep their meaning; the one
  committed run that exits 1 is an autostash whose restore conflicted and stays parked: the
  receipt is `partial` with a `user_action`, since nothing retries what only the user can re-apply
  (#122557).
- **Report**: every run writes a machine-readable receipt to the ROOT home's
  `logs/update_receipts/` (never a sticky profile's; `update.log` likewise) — `latest.json` pointer;
  steps, skips WITH reasons, restart outcome, plan, fleet snapshot, `followups`. The receipt is on
  disk as `outcome: "running"` from the start and refreshed at each stage boundary; the next
  update marks a `running` record whose processes are gone `interrupted` and says so. Readers
  (`hermes logs update`, debug bundle, dashboard status, pm receipts) use the root home too.
  A dependency-sync failure in the completion bootstrap is a `dependencies` follow-up (A6), and
  Ctrl-C after the commit point finalizes `interrupted` (exit 130), never `failed`.
  Before a source swap, the parent captures plan/snapshots/receipt and its Windows pause token.
  `update_completion.py` runs new-code PM preparation with site initialization disabled, then
  selected-Python builds, maintenance, scans/restarts and verification. Git/current/ZIP share
  this owner; never reload or purge modules to continue in the old interpreter. The parent keeps
  the lock, waits, and accepts only a correlated terminal result. `cmd_update` still finalizes
  early failures and missing/killed-child outcomes; PM refusal data survives the handoff.
  See `website/docs/developer-guide/source-update-completion.md`. A begun-but-unwritten receipt is a bug.
- **Nothing runs pulled code in the pre-pull interpreter.** This tree finishes updates through
  `update_completion.run_completion` / `_update_takeover.py`.
  `hermes_cli/update_handoff.py` and `hermes_cli/update_serve_obligations.py` are the
  FROZEN COMPAT SURFACE for releases that lazily import those module
  names from the NEW tree after the checkout swap.
  Keep their public names importable and behavior-preserving. They call into
  `_old_updater.stop_for_relaunch` → `_run_child`. Removing a name
  bricks every release mid-update. see `tests/compat/old_updater_surface.json`.

Process-scan coordination between updater, serve/dashboard, and gateway is being replaced by a
gateway-owned control socket (#92091); scans are the fallback layer for old/crashed processes — read
#92091 before adding any heuristic. Process identity rules (never argv substrings; canonical
matchers; parser-derived flag sets; never blanket-exclude gateway ancestors, #87594): root
`AGENTS.md` and `website/docs/developer-guide/cli-internals.md`.

The systemd blunt-restart fallback waits for the unit's `TimeoutStopUSec` plus
`TimeoutStartUSec`, with 15 seconds of client-side slack. It reads the target unit
in the same manager scope as the restart; both the initial attempt and retry use
this budget, including the catch-up restart after an interrupted update.
A start after a graceful drain uses only the start budget plus slack.
A missing, unparseable, or infinite phase limit falls back to 90 seconds
for that phase, keeping unattended updates bounded. Timing out the `systemctl`
client does **not** cancel the manager's transaction. Custom multi-command stop
chains or `EXTEND_TIMEOUT_USEC` can still outlast this estimate; a real timeout
remains an incomplete restart, and successful commands still require the existing
service-health and fleet-version verification. Raw numeric `*USec` values are
microseconds, while formatted values use systemd's fixed units, including days,
weeks, months and years. The combined timeout is capped below the native signed
32-bit millisecond poll limit (with rounding headroom), so exceptionally long
unit limits cannot overflow subprocess polling. Zero/unknown/infinite phase
limits use the bounded fallback. This does not change active-turn drain settings.

## Nous sign-in

Every Nous sign-in is the plain device-code login. Sign-in completion is one function,
`settle_after_upgrade`, called once per completion: it moves a config on the welcome route to the
account's host and the tier's recommended default (`models.recommended_nous_default_model`, shared
with `GET /api/model/recommended-default`).

The shared flow, states, and copy live in `anon_sign_in.py`; CLI rendering lives in
`anon_sign_in_cli.py`. `anon_auth.py` keeps identity and settlement, and re-exports the sign-in
API. The flow resolves identity and persistence collaborators through `anon_auth` at call time to
preserve module-attribute monkeypatch seams.

The sign-in itself is one composition: `anon_auth.run_sign_in()` yields `SignInState`s (`Code`,
`Waiting`, `Completed`, `Cancelled`, `TimedOut`, `Failed`, `AlreadySignedIn`). It requests a device
code, polls the token endpoint on one deadline (an expired code is `TimedOut`), saves the account,
runs `settle_after_upgrade` exactly once, and never lets a persist or settle failure escape as an
exception: it becomes `Failed`. It works with or without a free tier on disk. `Completed.email`
comes from the access token's `email` claim; `Completed.handoff` carries the guest handoff's result
lines (what moved, what to reconnect) and `Completed.handoff_status` the latest record as fields
(`guest_handoff.wire_status`: the public view plus those `lines`), for a surface that renders the
result itself. Every state carries its own `.copy` (the chat form, which
never contains a raw exception, a URL or a `hermes` verb) and `.copy_terminal`, so no caller maps a
reason to a string. `cancelled()` is checked at least once a second inside the token poll and once more
before the save; a cancel always wins and nothing is saved. TLS verification comes from
`_resolve_verify`, so `HERMES_CA_BUNDLE` and friends apply. `scope` is entered only around the
precondition and persist blocks, never across a `yield` or a network wait, because
`run_in_executor` does not carry contextvars. `replace_account=True` lets a signed-in install sign in
again (the desktop's "connect another account"); otherwise a signed-in install gets
`AlreadySignedIn` before any network. `upgrade_guest` (`hermes auth upgrade`), the CLI and TUI
`/login`, the gateway `/login` and the desktop start route are renderers over it; a surface that
needs the cancel check and the save to be atomic passes `persist_guard`. `hermes model` and
`hermes auth add nous` keep `_login_nous` (it carries `--portal-url`, `--ca-bundle`, `--insecure`
and the shared-store import); that is the same plain device login, and it prints the handoff result
the same way. Every sign-in that saves an account (`run_sign_in`, `_login_nous`, `hermes auth add
nous`, the billing step-up, the tool-setup login) calls `settle_after_upgrade` itself once its locks
are released; the handoff runner never settles.

### Guest handoff after a sign-in

When any writer saves a Nous account over a free-tier identity, `hermes_cli/guest_handoff.py` hands
that identity's connected apps to the account: `POST /api/anonymous/claim {"token": "<anon_...>"}`
with the account's own Bearer token. Nothing asks the user first; the result is shown afterwards
(`hermes auth status nous`; the desktop reads it from the sign-in poll's `handoff_status` and, while a
move is still retrying, from the `GET /api/providers/oauth/nous/handoff?since=&wait=` long-poll, which
answers once the record's `updated_at` changes). Two halves:

- `note_replacement` runs at the two lowest write seams (`auth._store_provider_state` and
  `auth_nous._write_shared_nous_state`) under the caller's auth-store locks and only writes a pending
  record. Records live in `guest_handoff.json` beside the shared Nous store, keyed by a hash of the
  `anon_` credential, so profiles sharing one free-tier identity share one record and one claim.
- Only a sign-in changes which identity the shared store holds. Every `_write_shared_nous_state`
  caller declares `renewal`: a token refresh (`renewal=True`) never replaces another identity there
  (identity = the guest's `anon_` credential, or the account's JWT `sub`), so a refresh of one
  profile's account never replaces a guest another profile minted. That refresh is kept beside the
  held identity under `renewal`, where sibling profiles on the same account read their next
  single-use refresh token.
- `drain` makes one attempt per due record of this profile and returns what each did. It sends a
  request, so it runs only outside the auth-store locks: every sign-in surface calls it right after
  its save (`run_sign_in` after its `persist_guard` is released), and so do a free-tier reconcile,
  `hermes auth status` and `hermes auth upgrade`. It only moves apps; config completion
  (`settle_after_upgrade`) stays with the sign-in that saved the account.
- A move a transient error left retrying is finished by a retry worker that every long-lived Nous
  process starts (`start_nous_auth_keepalive` starts it, guest or not); each tick drains the
  process's own profile.
- Each attempt is counted and the record leased (not due for about 30 s) in one locked write before
  the request leaves, so a lost answer still counts and two runs never send the same record.

Answers: `claimed` records what moved (and refreshes the account token only when
`cohort_carried`; an answer without `connectors` reads as unknown, never "nothing moved");
`unknown_token` is Gone, read as "probably moved" after an earlier attempt (a lost success);
`already_retired` is Gone too. Only codes no later attempt can change are final
(`client_not_allowed`, `account_locked`, `invalid_request`). Every other answer, 401 and unknown
codes included, and every transport failure retries after 1 minute, 10 minutes, 1 hour and 6 hours
(Retry-After honoured), then the record is Failed. A Claimed record is never reopened, and user copy
never shows a server code; app slugs show as display names (`tools.tool_labels.app_title`).

## Process identity: never infer it from argv substrings

The bug class behind ~10 fleet-update issues (#90778, #87594, #78089, #76129, #91964, ...):
classifying a process by `"serve" in cmdline` or similar. `kanban --preserve-cache` contains
"serve"; a flag VALUE can equal a subcommand (`-m dashboard serve`); truncated cmdlines hide the real
subcommand. Rules:

- Use the canonical matchers: `gateway.status.looks_like_gateway_command_line` (gateway run),
  `hermes_cli.update_cmd._hermes_holder_subcommand` (top-level subcommand of any Hermes argv). Never
  hand-roll token scans.
- Flag sets must be DERIVED from the parser (`_holder_value_flags()` introspects
  `build_top_level_parser()`), never hand-written lists — they drift.
- Never blanket-exclude ancestors from process scans: when `/update` runs as the gateway's child, a
  gateway ancestor must stay visible to the pause machinery (#87594). Exclude interactive ancestry,
  carve out gateway-shaped ancestors.
- Match on FULL cmdlines; truncate only at display time (#78089).
- Before adding any new scan heuristic, read #92091 — the gateway control socket replaces scans as
  the primary coordination mechanism; scans are the fallback layer for old/crashed processes.

## Skin engine — what skins customize

| Element | Skin key | Used by |
|---|---|---|
| Banner panel border / title / section headers / dim / body | `colors.banner_border`, `banner_title`, `banner_accent`, `banner_dim`, `banner_text` | `banner.py` |
| Response box border | `colors.response_border` | `cli.py` |
| Spinner faces (waiting / thinking) | `spinner.waiting_faces`, `spinner.thinking_faces` | `display.py` |
| Spinner verbs / wings (optional) | `spinner.thinking_verbs`, `spinner.wings` | `display.py` |
| Tool output prefix / per-tool emojis | `tool_prefix`, `tool_emojis` | `display.py` → `get_tool_emoji()` |
| Agent name / welcome / response label / prompt symbol | `branding.agent_name`, `welcome`, `response_label`, `prompt_symbol` | `banner.py`, `cli.py` |

Built-in skins (`_BUILTIN_SKINS` in `hermes_cli/skin_engine.py`): `default` (classic gold/kawaii),
`ares` (crimson/bronze with custom spinner wings), `mono` (grayscale), `slate` (cool blue). Add a
built-in as a dict entry `{"name", "description", "colors", "spinner", "branding", "tool_prefix"}`.
User skins are `~/.hermes/skins/<name>.yaml` with the same keys, activated with `/skin <name>` or
`display.skin: <name>`; the full YAML template is in the
[Skins & Themes](../user-guide/features/skins.md) user guide.

## Profiles: multi-instance support

Hermes supports profiles — fully isolated instances, each with its own `HERMES_HOME` (config, API
keys, memory, sessions, skills, gateway). For single-profile commands (`hermes -p x <cmd>`),
`_apply_profile_override()` in `hermes_cli/main.py` sets `HERMES_HOME` before any module imports, so
every `get_hermes_home()` reference scopes to the active profile. The multiplex gateway and the
Desktop/dashboard `serve` backend serve several profiles from one process instead: the active
profile is a contextvar override bound per activity, `os.environ["HERMES_HOME"]` stays the launch
profile's, and a module-level constant derived from the home freezes to that launch profile (see
[Gateway Internals § Multiplexed profiles](./gateway-internals.md#multiplexed-profiles)). Profile
operations are HOME-anchored (`_get_profiles_root()` returns
`Path.home() / ".hermes" / "profiles"`, not `get_hermes_home() / "profiles"`) so
`hermes -p coder profile list` sees all profiles regardless of which one is active — intentional.
Profile-safe coding rules are in the root `AGENTS.md`; multiplex secret-scope rules in
`gateway/AGENTS.md`.