--- sidebar_position: 11 title: "Plugin LLM Access" description: "Run any LLM call from inside a plugin via ctx.llm — chat or structured, sync or async. Host-owned auth, fail-closed trust gate, optional JSON Schema validation." --- # Plugin LLM Access `ctx.llm` is the supported way for a plugin to make an LLM call. Chat completion, structured extraction, sync, async, with or without images — same surface, same trust gate, same host-owned credentials. Plugins reach for this when they need to do something that involves the model but isn't part of the agent's conversation. A hook that rewrites a tool error into something a non-engineer can read. A gateway adapter that translates an inbound message before queuing it. A slash command that summarises a long paste. A scheduled job that scores yesterday's activity and writes one line to a status board. A pre-filter that decides whether a message is worth waking the agent up for at all. These are jobs the agent shouldn't be in the loop on. They want one LLM call, a typed answer, and to be done. ## The smallest possible call ```python result = ctx.llm.complete(messages=[{"role": "user", "content": "ping"}]) return result.text ``` That's the whole API in one line. No keys, no provider config, no SDK initialisation. The plugin runs against whatever provider and model the user is currently using — when they switch providers, the plugin follows them automatically. ## A more complete chat example ```python result = ctx.llm.complete( messages=[ {"role": "system", "content": "Rewrite errors as one short sentence a non-engineer can act on."}, {"role": "user", "content": traceback_text}, ], max_tokens=64, purpose="hooks.error-rewrite", ) return result.text ``` `purpose` is a free-form audit string — it shows up in `agent.log` and in `result.audit` so operators can see which plugin made which call. Optional but recommended for anything that fires often. ## Structured output When the plugin needs a typed answer, switch to the structured lane: ```python result = ctx.llm.complete_structured( instructions="Score this support reply for urgency (0–1) and pick a category.", input=[{"type": "text", "text": message_body}], json_schema=TRIAGE_SCHEMA, purpose="support.triage", temperature=0.0, max_tokens=128, ) if result.parsed["urgency"] > 0.8: await dispatch_to_oncall(result.parsed["category"], message_body) ``` The host requests JSON output from the provider, parses it locally as a fallback, validates against your schema if `jsonschema` is installed, and hands back a Python object on `result.parsed`. If the model couldn't produce valid JSON, `result.parsed` is `None` and `result.text` carries the raw response. ## What this lane gives you * **One call, four shapes.** `complete()` for chat, `complete_structured()` for typed JSON, `acomplete()` and `acomplete_structured()` for asyncio. Same arguments, same result objects. * **Host-owned credentials.** OAuth tokens, refresh flows, the credential pool, per-task aux overrides — every credential concept Hermes already has applies. The plugin never sees a token; the host attributes the call back through `result.audit`. * **Bounded.** Single sync or async call. No streaming, no tool loops, no conversation state to manage. State the input, get the result, return. * **Fail-closed trust.** A plugin you've never configured cannot pick its own provider, model, agent, or stored credential. The default posture is "use what the user is using." Operators opt in to specific overrides, per plugin, in `config.yaml`. ## Quick start Two complete plugins below — one chat, one structured. Both ship inside a single `register(ctx)` function and need zero outside configuration to run against whatever model the user has active. ### Chat completion — `/tldr` ```python def register(ctx): ctx.register_command( name="tldr", handler=lambda raw: _tldr(ctx, raw), description="Summarise the supplied text in one paragraph.", args_hint="", ) def _tldr(ctx, raw_args: str) -> str: text = raw_args.strip() if not text: return "Usage: /tldr " result = ctx.llm.complete( messages=[ {"role": "system", "content": "Summarise the user's text in one tight paragraph. No preamble."}, {"role": "user", "content": text}, ], max_tokens=256, temperature=0.3, purpose="tldr", ) return result.text ``` `result.text` is the model's response; `result.usage` carries token counts; `result.provider` and `result.model` carry attribution. ### Structured extraction — `/paste-to-tasks` ```python def register(ctx): ctx.register_command( name="paste-to-tasks", handler=lambda raw: _paste_to_tasks(ctx, raw), description="Turn freeform meeting notes into structured tasks.", args_hint="", ) _TASKS_SCHEMA = { "type": "object", "properties": { "tasks": { "type": "array", "items": { "type": "object", "properties": { "owner": {"type": "string"}, "action": {"type": "string"}, "due": {"type": "string", "description": "ISO date or empty"}, }, "required": ["action"], }, }, }, "required": ["tasks"], } def _paste_to_tasks(ctx, raw_args: str) -> str: if not raw_args.strip(): return "Usage: /paste-to-tasks " result = ctx.llm.complete_structured( instructions=( "Extract concrete action items from these meeting notes. " "One task per actionable line. If no owner is named, leave 'owner' blank." ), input=[{"type": "text", "text": raw_args}], json_schema=_TASKS_SCHEMA, schema_name="meeting.tasks", purpose="paste-to-tasks", temperature=0.0, max_tokens=512, ) if result.parsed is None: return f"Couldn't parse a response. Raw output:\n{result.text}" lines = [f"- [{t.get('owner') or '?'}] {t['action']}" for t in result.parsed["tasks"]] return "\n".join(lines) or "(no tasks found)" ``` A third worked example, this time with image input, lives in the [`hermes-example-plugins`](https://github.com/NousResearch/hermes-example-plugins/tree/main/plugin-llm-example) repo (companion repo for reference plugins — not bundled with hermes-agent itself). For the async surface (`acomplete()` / `acomplete_structured()` with `asyncio.gather()`), see [`plugin-llm-async-example`](https://github.com/NousResearch/hermes-example-plugins/tree/main/plugin-llm-async-example) in the same repo. ## When to use which | You want… | Reach for | |---|---| | A free-form text response (translation, summary, rewrite, generation) | `complete()` | | A multi-turn prompt (system + few-shot examples + user) | `complete()` | | A typed dict back, validated against a schema | `complete_structured()` | | Image-or-text input with a typed dict back | `complete_structured()` | | The same call from async code (gateway adapters, async hooks) | `acomplete()` / `acomplete_structured()` | Everything else — provider selection, model resolution, auth, fallback, timeout, vision routing — is the same across all four. ## API surface `ctx.llm` is an instance of `agent.plugin_llm.PluginLlm`. ### `complete()` ```python result = ctx.llm.complete( messages=[{"role": "user", "content": "Hi"}], provider=None, # optional, gated — Hermes provider id (e.g. "openrouter") model=None, # optional, gated — whatever string that provider expects temperature=None, max_tokens=None, timeout=None, # seconds agent_id=None, # optional, gated profile=None, # optional, gated — explicit auth-profile name purpose="optional-audit-string", task=None, # optional — a plugin-registered auxiliary slot ) # → PluginLlmCompleteResult(text, provider, model, agent_id, usage, audit) ``` Plain chat completion. `messages` is the standard OpenAI shape — a list of `{"role": "...", "content": "..."}` dicts. Multi-turn prompts (system + few-shot user/assistant pairs + final user) work exactly as they would with the OpenAI SDK. `provider=` and `model=` are independent and follow the same shape as the host's main config (`model.provider` + `model.model`). Set just `model=` to use the user's active provider with a different model on it. Set both to switch providers entirely. Either argument without operator opt-in raises `PluginLlmTrustError`. ### `complete_structured()` ```python result = ctx.llm.complete_structured( instructions="What you want extracted.", input=[ {"type": "text", "text": "..."}, {"type": "image", "data": b"...", "mime_type": "image/png"}, {"type": "image", "url": "https://..."}, ], json_schema={...}, # optional — triggers parsed result + validation json_mode=False, # set True without a schema to ask for JSON anyway schema_name=None, # optional human-readable schema name system_prompt=None, provider=None, # optional, gated model=None, # optional, gated temperature=None, max_tokens=None, timeout=None, agent_id=None, profile=None, purpose=None, task=None, # optional — a plugin-registered auxiliary slot ) # → PluginLlmStructuredResult(text, provider, model, agent_id, # usage, parsed, content_type, audit) ``` Inputs are typed text or image blocks (raw bytes get base64 encoded as a `data:` URL automatically). When `json_schema` or `json_mode=True` is supplied, the host requests JSON output via `response_format`, parses it locally as a fallback, and validates against your schema if `jsonschema` is installed. * `result.content_type == "json"` — `result.parsed` is a Python object that matches your schema. * `result.content_type == "text"` — parsing or validation failed; inspect `result.text` for the raw model response. ### Async ```python result = await ctx.llm.acomplete(messages=..., task="classifier") result = await ctx.llm.acomplete_structured( instructions=..., input=..., task="classifier" ) ``` Same arguments and result types as their sync counterparts. Use these from gateway adapters, async hooks, or any plugin code already running on an asyncio loop. ### Task-routed auxiliary calls Pass `task=` to any of the four call shapes when a plugin needs its own configured auxiliary route. Register that task during plugin setup; plugin defaults apply until the operator overrides its provider and model in `auxiliary.`: ```python def register(ctx): ctx.register_auxiliary_task( "classifier", display_name="Classifier", description="Classify input." ) result = ctx.llm.complete(messages=[...], task="classifier") result = ctx.llm.complete_structured(instructions=..., input=..., task="classifier") ``` ```yaml auxiliary: classifier: provider: openrouter model: vendor/model-id ``` Plugins may supply provider/model registration defaults for their own tasks. Operator configuration in `auxiliary.` overrides those defaults and controls the deployment choice. A registered task is a first-class slot on every model-assignment surface: the `hermes model` → *Configure auxiliary models* picker, the dashboard Models page, and Desktop Settings → Models → Auxiliary all list it (after the built-in tasks, under the plugin's `display_name`) and can pin, reset, or flag it as a stale provider pin like any built-in. A plugin can only use a task it registered itself; unknown or foreign task names fail before provider invocation. `allow_task_override: true` is an explicit operator grant for using Hermes built-in auxiliary tasks; it does not permit another plugin's tasks. Omit `task=` (or use `"auto"`) to keep the active main provider/model. #### From the plugin's frontend: a plugin RPC method A plugin's frontend (a Desktop page, or a client attached to the shared gateway) does not call the model itself. It calls its own Python half through a [plugin RPC method](plugins/index.md#talk-to-your-own-frontend-plugin-rpc-methods-and-session-events), and the handler calls `ctx.llm` on the task the plugin registered. A plugin RPC call holds its client's request lane until the handler returns, so the handler starts the model call in a thread, answers with a job id at once, and delivers the result as a session event: ```python import contextvars import threading import uuid from agent.redact import redact_sensitive_text CHART = {"type": "object", "properties": {"kind": {"type": "string"}}, "required": ["kind"]} def register(ctx): ctx.register_auxiliary_task("chart_writer", display_name="Charts", description="Pick chart kinds.") slots = threading.BoundedSemaphore(4) # model calls in flight def write_chart(params, *, session_id, **kwargs): if not session_id: raise ValueError("chart.write needs a session_id") job = uuid.uuid4().hex text = redact_sensitive_text(str(params.get("text", "")), force=True) def run(): try: result = ctx.llm.complete_structured( instructions="Pick a chart kind for this data.", input=[{"type": "text", "text": text}], json_schema=CHART, task="chart_writer", ) if result.parsed is None: raise ValueError("the model did not answer with JSON") payload = {"job": job, "chart": result.parsed} except Exception as exc: payload = {"job": job, "error": str(exc)} finally: slots.release() ctx.emit_to_session(session_id, "chart.written", payload) if not slots.acquire(blocking=False): raise RuntimeError("chart.write is busy: wait for a chart.written event") try: # copy_context: the thread keeps the call's profile scope, which ctx.llm reads. threading.Thread(target=contextvars.copy_context().run, args=(run,), daemon=True).start() except Exception: slots.release() # run() never started, so it never releases raise return {"job": job} ctx.register_rpc("chart.write", write_chart) ``` The frontend sends `{"method": "plugin..chart.write", "params": {"session_id": "…", "text": "…"}}`, gets `{"job": "…"}` back at once, and receives the chart (or an `error`) as the session event `plugin..chart.written` with the same `job`. Called inline, the model call would hold that client's next requests on the connection, an interrupt or an approval answer included, until the model answered. The semaphore stands in for `llm.oneshot`'s per-owner cap: a fifth call while four run is refused. An exception in the handler itself answers the call with reason `plugin_rpc_failed`. - **Redact at the boundary.** `agent.redact.redact_sensitive_text` is part of the [public plugin API](plugins/public-api.md). Pass `force=True` where secrets must never leave the machine, such as text sent to a model: it then masks secrets even when the operator turned redaction off with `security.redact_secrets: false`. Without `force=True` it follows that setting. - **The task is the route.** `task=` must name a task your plugin registered, so the operator's `auxiliary.` setting picks the model, and a frontend cannot pick a different route. - **Check `parsed`.** It is `None` when the reply is not JSON. A reply that does not match `json_schema` raises `ValueError` when the optional `jsonschema` package is installed; without it, the schema is not checked. #### Inheriting a built-in slot `inherit_from` names a built-in auxiliary task the model picker lists (`compression`, `mcp`, `vision`, `curator`, ...), `background_review`, or a task your plugin (or one loaded before it) already registered. Other core `auxiliary.*` blocks are not bases. Your task then uses that slot's model until the operator pins one on your task directly. **Following a built-in slot's model needs the operator's grant.** The slot's route (`provider`, `model`, `base_url`, `api_key`, `api_mode`, the key env, `reasoning_effort`, `extra_body`, `fallback_chain`) carries over only when the operator sets `allow_task_override` for your plugin, the same grant that lets `task=` name a built-in slot. Without it your task inherits the slot's other settings (such as `timeout`) and runs on its own route: your `defaults`, else the active main model. Declare it in your setup instructions when your task should follow the user's review or curator model: ```yaml plugins: entries: my-plugin: llm: allow_task_override: true ``` ```python def register(ctx): ctx.register_auxiliary_task( "classifier", display_name="Classifier", description="Classify input.", inherit_from="compression", # use whatever compression uses... defaults={"timeout": 90}, # ...but give slow classifiers more room ) ``` - **Resolved on every read, not copied at registration.** Change `auxiliary.compression` (CLI, dashboard, Desktop, or `config.yaml`) and the classifier follows on its next call, per profile. - **Precedence:** the inherited base, then the plugin's `defaults`, then the operator's `auxiliary.` block. - **A chain counts as its root.** Inheriting another plugin's task that inherits a built-in slot carries that slot's route only when your plugin holds the grant too. - **An operator pin wins as a whole route.** Once `auxiliary.classifier` sets a non-`auto` provider, a model or a `base_url`, the provider, model, endpoint, key and reasoning effort all come from that block; nothing from the base's route is mixed in. `provider: auto` with an empty model is "no preference", so the task keeps following its base (that is what "Reset all" and Desktop's *Follow <base>* button write). - **A bad base never breaks the plugin.** An unknown or self-referential `inherit_from` logs a warning and the task registers without inheritance. The Models settings show an unpinned inheriting task as *inherits Compression · <provider · model>*, and `GET /api/model/auxiliary` returns `inherit_from` plus an `effective` `{provider, model, base_url}` on such rows. ### Result attributes ```python @dataclass class PluginLlmCompleteResult: text: str # the assistant's response provider: str # e.g. "openrouter", "anthropic" model: str # whatever the provider returned for this call agent_id: str # whose model/auth was used usage: PluginLlmUsage # tokens + cache + cost estimate audit: Dict[str, Any] # plugin_id, purpose, profile @dataclass class PluginLlmStructuredResult: # same fields as PluginLlmCompleteResult, plus: parsed: Optional[Any] # JSON object when content_type == "json" content_type: str # "json" or "text" # audit also carries schema_name when supplied ``` `usage` carries `input_tokens`, `output_tokens`, `total_tokens`, `cache_read_tokens`, `cache_write_tokens`, and `cost_usd` when the provider returns those fields. ## Trust gate The default behaviour is fail-closed. With no `plugins.entries` config block, a plugin can: * run any of the four methods against the user's active provider and model, * set request-shaping arguments (`temperature`, `max_tokens`, `timeout`, `system_prompt`, `purpose`, `messages`, `instructions`, `input`, `json_schema`), …and that's it. `provider=`, `model=`, `agent_id=`, and `profile=` arguments raise `PluginLlmTrustError` until the operator opts in. Likewise, `task=` can use only the plugin's registered auxiliary task unless the operator grants `allow_task_override` for a built-in task. **Most plugins never need this section.** A plugin that just calls `ctx.llm.complete(messages=...)` with no overrides runs against whatever the user has active and works zero-config. The block below is only relevant when a plugin specifically wants to pin to a different model or provider than the user. ```yaml plugins: entries: my-plugin: llm: # Allow this plugin to choose a different Hermes provider # (must be one Hermes already knows about — same names as # `hermes model` and config.yaml model.provider). allow_provider_override: true # Optionally restrict which providers. Use ["*"] for any. allowed_providers: - openrouter - anthropic # Allow this plugin to ask for a specific model. allow_model_override: true # Optionally restrict which models. Use ["*"] for any. # Models are matched literally against whatever string the # plugin sends — Hermes does not look anything up. allowed_models: - openai/gpt-4o-mini - anthropic/claude-3-5-haiku # Allow cross-agent calls (rare). allow_agent_id_override: false # Allow the plugin to request a specific stored auth profile # (e.g. a different OAuth account on the same provider). allow_profile_override: false ``` The plugin id is the manifest `name:` field for flat plugins, or the path-derived key for nested plugins (`image_gen/openai`, `memory/honcho`, etc.). ### What the gate enforces | Override | Default | Config key | | --------------- | ------- | -------------------------------- | | `provider=` | denied | `allow_provider_override: true` | | ↳ allowlist | — | `allowed_providers: [...]` | | `model=` | denied | `allow_model_override: true` | | ↳ allowlist | — | `allowed_models: [...]` | | `agent_id=` | denied | `allow_agent_id_override: true` | | `profile=` | denied | `allow_profile_override: true` | | built-in `task=` | denied | `allow_task_override: true` | | route inherited from a built-in slot | stripped | `allow_task_override: true` | Each override is independently gated. Granting `allow_model_override` does **not** also grant `allow_provider_override` — a plugin trusted to pick a model is still pinned to the user's active provider unless it gets the provider gate as well. ### What the gate does NOT need to enforce * Request-shaping arguments — `temperature`, `max_tokens`, `timeout`, `system_prompt`, `purpose`, `messages`, `instructions`, `input`, `json_schema`, `schema_name`, `json_mode` — are always allowed; they don't pick credentials or routes. * The default deny posture means an unconfigured plugin can still do useful work — it just runs against the active provider and model. Operators only need to think about `plugins.entries` for plugins that want finer routing. ## What the host owns A complete list of the things `ctx.llm` does for the plugin so you don't have to: * **Provider resolution.** Reads `model.provider` + `model.model` from the user's config (or the explicit overrides when trusted). * **Auth.** Pulls API keys, OAuth tokens, or refresh tokens from `~/.hermes/auth.json` / env, including the credential pool when one is configured. The plugin never sees them. * **Vision routing.** When image input is supplied and the user's active text model is text-only, the host falls back to the configured vision model automatically. * **Fallback chain.** If the user's primary provider 5xxs or 429s, the request goes through Hermes' usual aggregator-aware fallback before it returns an error to the plugin. * **Timeout.** Honours your `timeout=` argument, falling back to `auxiliary..timeout` config or the global aux default. * **JSON shaping.** Sends `response_format` to the provider when you ask for JSON, then re-parses locally from a code-fenced response if the provider returned one. * **Schema validation.** Validates against your `json_schema` when `jsonschema` is installed; logs a debug line and skips strict validation otherwise. * **Audit log.** Each call writes one INFO line to `agent.log` with the plugin id, provider/model, purpose, and token totals. ## What the plugin owns * **Request shape.** `messages` for chat, `instructions` + `input` for structured. The plugin builds the prompt; the host runs it. * **Schema.** Whatever shape you want back. The host doesn't infer it for you. * **Error handling.** `complete_structured()` raises `ValueError` on empty inputs and on schema-validation failure. `PluginLlmTrustError` fires when the trust gate denies an override. Anything else (provider 5xx, no credentials configured, timeout) raises whatever `auxiliary_client.call_llm()` raises. * **Cost.** Every call runs against the user's paid provider. Don't loop on `complete()` for every gateway message without thinking about token spend. ## Asking the session's own agent (`ctx.side_question`) `ctx.llm` runs a fresh call that knows only what the plugin sends it. `ctx.side_question` asks the session's *own* agent instead: the same model, the same system prompt (memory included) and the whole conversation so far, plus one question from the plugin. It replays the request the turn just sent, so the provider's prompt cache is still warm and the call is mostly cache reads. Reach for it when the answer depends on the conversation itself: suggesting the user's next prompt, labelling what the turn did, deciding whether the session needs a follow-up. For anything that doesn't need the session, use `ctx.llm`: it is cheaper and has no lease rules. ```python import threading def register(ctx): def on_post_llm_call(**kwargs): token = kwargs.get("side_question_token") if token is None: # no consented plugin loaded, or not a user turn return # post_llm_call is time-bounded: never wait for the model inside the hook. threading.Thread(target=_suggest, args=(token,), daemon=True).start() def _suggest(token): answer = ctx.side_question( token, "Suggest the user's most likely next prompt in one short line.") if answer: ctx.state.set("next_prompt", answer) ctx.register_hook("post_llm_call", on_post_llm_call) ``` The plugin must declare the capability in `plugin.yaml` and the user must grant it (the consent screen shows: "Ask your agent a short question after each turn. It sees the whole session, including memory, and spends your main model's tokens."): ```yaml capabilities: - llm.side_question ``` ### Lease rules * `post_llm_call` receives `side_question_token` (a string) when at least one loaded plugin holds the capability, else `None`. Cron runs and delegated subagent turns always get `None`: nobody is waiting on them, so a side question would only spend tokens. * **One attempt per turn, shared by every plugin.** Every plugin's hook gets the same token; the first `ctx.side_question` call spends it, whether it succeeds or not. A plugin without the capability gets `None` back and does not spend it. * The token expires after **60 seconds**, and as soon as the session starts its next turn. * The token belongs to the profile whose turn issued it; a plugin context of another profile gets `None` and does not spend it. * `prompt` is a non-empty string of at most **4096 characters**. * `timeout` defaults to **15 seconds** and is capped at 60. It covers the whole call, including setting up the request. ### What you get back The text of the agent's reply, or `None`. `None` means the call was refused or failed: a Mixture-of-Agents runtime, an agent-as-provider runtime (ACP, external-process providers, the Codex app-server), a session whose model changed since the turn (a voice-chat turn routed to `auxiliary.voice_chat` switches back right after it), a request that carries provider-executed (server-side) tools, a timeout, a provider error, or an empty reply. There are no retries and no fallback model. The call never runs a tool (tool calls in the reply are dropped) and never writes to the transcript. Its tokens are billed to the session whose turn issued the token (even if the session id has since rotated) and recorded in its usage as `plugin_side_question:`, apart from `/btw`'s `side_question`. You can't pick the model, tools, system prompt or output length: changing any of them would miss the warm cache. Ask for brevity in the prompt. Observability plugins see the call as an auxiliary call: `pre_auxiliary_call` / `post_auxiliary_call` fire with `aux_task="plugin_side_question"` and `plugin_id` set to the asking plugin, and the turn-scoped `pre_api_request` / `post_api_request` do not fire for it. ## Forking the session's own agent (`ctx.fork_session`) :::caution Experimental `ctx.fork_session` and [`ctx.register_idle_task`](#idle-tasks-ctxregister_idle_task) are experimental. Their signatures and the fork handle's shape may change in a later release; keep the handle inside one small function of your plugin that returns plain data, and do not store it past the turn. Treat `fork_session` as optional: it returns `None` when the request is refused (no `llm.session_fork` grant, a bad, expired or spent token, a tool outside the allowlist), and it is not available everywhere. Under `plugins.isolation: host` a third-party plugin's `ctx.fork_session` exists but raises `PluginHostUnsupported` (a `RuntimeError`), because the handle tracks a thread of the live session agent inside the Hermes process. A plugin that degrades to "no fork" on `None` or an exception keeps working in both places. ::: `ctx.side_question` gets one text reply. `ctx.fork_session` runs a bounded background copy of the session's agent instead: the same system prompt, `tools[]`, memory and conversation, on the warm prompt cache, working through a multi-step prompt that may call the plugin's own tools it names. It is the pattern Hermes uses for its own background memory/skill review, opened to plugins. A typical use: after a turn that created or edited a skill, ask the fork whether the skill is worth sharing, and let it call the plugin's own tool that offers the share. ```python import json import threading def register(ctx): def offer_share(args, turn_origin=None, **kwargs): if turn_origin != "plugin_fork:wisdom": return json.dumps({"error": "only the background check may offer a share"}) ... # present the approval card for args["skill"] return json.dumps({"offered": True}) ctx.register_tool("wisdom_offer_share", "wisdom", OFFER_SCHEMA, offer_share) def on_post_llm_call(side_question_token=None, review_signals=None, **kwargs): if side_question_token is None or not (review_signals or {}).get("skill_review_due"): return handle = ctx.fork_session( side_question_token, "Did this session create or change a skill the team would benefit from? " "If so, call wisdom_offer_share once for it; otherwise answer 'no'.", tools=["wisdom_offer_share"], max_iterations=4) if handle is not None: # runs in the background; never wait inside the hook threading.Thread(target=lambda: handle.wait(600) and ctx.state.set( "last_check", handle.outcome), daemon=True).start() ctx.register_hook("post_llm_call", on_post_llm_call) ``` ```yaml capabilities: - llm.session_fork ``` The consent screen shows: "Run a background copy of your agent after a turn. It sees the whole session, including memory, can use the plugin's own tools, and spends your main model's tokens." ### Signature and rules `ctx.fork_session(token, prompt, *, tools=(), max_iterations=8, input_budget=None, task=None, timeout=None)` returns a handle, or `None` when nothing was started. * `token` is `post_llm_call`'s `side_question_token`: the [lease rules](#lease-rules) above apply unchanged. One token backs one request per turn, a side question *or* a fork, shared by every plugin; unattended turns (cron, subagents) get none; another profile's plugin gets `None` and does not spend it. A plugin without `llm.session_fork` gets `None` and does not spend it either, nor does a request naming a tool outside the fork allowlist (below); every other `None` spends it. * `prompt` is a non-empty string of at most **8192 characters**, sent as the fork's user message. The system prompt and `tools[]` stay byte-identical to the session's, so the first request is mostly cache reads. * `tools` lists the tool names the fork may call, drawn from the fork allowlist: the names whose call reaches the handler this plugin registered with `register_tool` (the name's current registration is this plugin's, and Hermes dispatches it to the tool registry), plus the core tools Hermes marks fork-safe in one central list (currently none). Every other tool is outside it: another plugin's tools, every MCP tool, and every core tool not marked fork-safe, including `delegate_task`, `message_agent`, `cronjob_manage` and the Kanban tools. A name Hermes dispatches before the tool registry is outside it even when the plugin registered it, because the call would never reach the plugin's handler: an agent-level tool, a legacy alias, the Tool Search bridge, any `connectors____` name, and the session's memory-provider and context-engine tools. Naming a tool outside the allowlist returns `None` without spending the token. A tool's handler can start work the fork cannot follow (a subagent, a remote agent task, a cron run, a detached worker, an MCP server's sampling request), and that work would keep neither the fork's tool list nor its auto-denied approvals; a plugin's own handler already runs the plugin's code. * Each named tool must also be in the session, else the call returns `None` and spends the token: advertised in its `tools[]`, or deferred behind Tool Search (on by default, and plugin tools defer by default), in which case the session's `tools[]` carry the `tool_call` bridge in its place. The fork still *sees* every session tool, byte for byte (cache parity); calls outside the list are refused at dispatch with a message naming the allowed tools. * A deferred tool is called through `tool_call`, and the plugin does not need to know which of its tools are deferred: Hermes appends one line to the prompt telling the model to call the tool through `tool_call` with its name, followed by the tool's parameters. Inside the fork `tool_call` only carries a call to a named tool: a `tool_call` to any other tool (the plugin's own unnamed tools, another plugin's tool, a core tool, a connector) is refused at dispatch, and `tool_search` and `tool_describe` stay refused, since the prompt line already names the tool and its parameters. When the fork starts, the allowlist is read again against the live session and a tool that stopped being the plugin's own, or that the session now serves before the registry, is refused at dispatch too. An empty list means a tool-less fork. * `max_iterations` caps the fork's loop (1 to 16). The fork's aggregate input tokens are bounded like the background review's (`auxiliary.background_review.max_input_tokens`, default 75% of the context window); `input_budget` can only lower that bound. * Dangerous-command approvals are auto-denied, and nothing the fork does is written to the transcript or the session database. * The fork is cancelled as soon as the session starts its next turn, whether it is running, starting or still deferred; a deferred fork then never launches. A turn a plugin injects (`ctx.inject_message`, including from the fork's own tool handler) is the exception: it waits up to 120 seconds for the session's running background review and running forks to finish, then cancels whatever is still running, so do not rely on an injected turn to cancel background work. A person never waits behind that grace: a message a person sends to the session while the injected turn waits ends the wait at once, whatever `display.busy_input_mode` makes of it, and the background work is cancelled as for any person's turn. A message that is queued (queue mode, an interrupt, any message before the injected turn has an agent) runs as its own turn after the injected one, in the order they arrived; a steer (steer mode or `/steer`) or an interrupt-mode correction that redirects the turn goes into the injected turn itself. Another plugin's injected message does not end the wait. A gateway shutdown, restart or profile retirement ends the wait the same way, inside the shutdown's drain budget. The injected turn starts from the gateway's own context: its tool calls carry no `turn_origin`, even when the fork injected it. `timeout` (seconds, at most 1800) adds a wall clock. Both are cooperative: they interrupt the fork, which stops at its next interrupt check, and the handle settles when the fork's worker exits. A provider call that ignores the interrupt holds the handle open until it returns, so wait with a bound (`handle.wait(t)`). * `task` names one of this plugin's own [`register_auxiliary_task`](#task-routed-auxiliary-calls) lanes: the fork then runs on that lane's model, resolved like `auxiliary.background_review` (a different model starts cache-cold and sees a digest of the conversation). A task the plugin did not register returns `None`. Without `task` the fork runs on the session's model. A lane with `inherit_from="background_review"` follows the user's review model only under `allow_task_override` ([Inheriting a built-in slot](#inheriting-a-built-in-slot)). * On a runtime Hermes manages locally (the bundled llama-server), the fork waits for the machine to go idle like the background review (`outcome == "deferred"`); a newer deferred fork of the same plugin for the same session replaces the queued one (`"superseded"`). ### The handle `handle.done`, `handle.cancelled`, `handle.outcome`, `handle.result` (the fork's final text, or `None`), `handle.actions` (the whitelisted tool names it called, in order; a call through `tool_call` lists the tool it carried), `handle.wait(timeout)` and `handle.cancel()`. `outcome` is one of `deferred`, `running`, `ok`, `cancelled`, `superseded`, `timeout`, `budget_exhausted`, `error`, `error:spawn` (the fork's worker thread could not start; `fork_session` returns the settled handle rather than raising) or `refused:`: `parent_gone` (the session agent was released before the fork started), `moa` (a Mixture-of-Agents runtime), `agent_provider` (an agent-as-provider runtime that runs its own tools) or `runtime_changed` (the session switched model or provider since the turn). The handle never exposes the fork agent itself. ### Telling a fork from the live agent A tool handler called inside the fork receives the keyword `turn_origin="plugin_fork:"` (declare it, or accept `**kwargs`); calls the live agent makes carry no `turn_origin`. The example above uses it so the live agent cannot fire the plugin's share-offer tool on its own. Usage is billed to the session as `plugin_fork:`. Each of the fork's provider requests fires `pre_auxiliary_call` / `post_auxiliary_call` with `aux_task="plugin_fork"` and `plugin_id`, never the turn-scoped `pre_api_request` / `post_api_request`. ## Idle tasks (`ctx.register_idle_task`) Experimental, like `fork_session` (see the note [above](#forking-the-sessions-own-agent-ctxfork_session)). `ctx.register_idle_task(name, fn, *, interval_hours, min_idle_hours=2, first_run="defer")` runs a session-less job of the plugin on the skill curator's idle schedule; the full rules are in [Run a job when Hermes is idle](./plugins/index.md#run-a-job-when-hermes-is-idle-ctxregister_idle_task). The contract a task's `fn` is written against: * `fn` runs on its own thread, in the profile that registered it. The host's poll (gateway housekeeping, CLI/TUI startup, the `hermes serve` maintenance tick) starts it and returns at once; it never waits for the run and sets no time limit of its own. * Keep a run short: give it its own time limit, and work in small batches with a checkpoint in the plugin's data so the next run picks up where this one stopped. * Keep a run idempotent: a run cut off by a restart leaves no record and runs again later from the start, so running the same batch twice must do no harm. * When gateway housekeeping stops (gateway shutdown, or the multiplexer restarting in the same process), no new run starts and the gateway waits a few seconds for a run in flight, then stops without it. Plugin code cannot be stopped and `fn` gets no stop signal, so a run still going goes on in the background; when it ends it records nothing (the task's state stays as it was before the run, so the task runs again later) and releases its claim. Hermes guarantees only that no run starts and no task state is written after the stop; keep a run short. * No second run of a task starts while one is in flight. A claim whose process has died (a crash or a kill) is taken over at the next poll; a claim whose owner cannot be checked (one written on another machine) counts as abandoned after an hour. An exception is caught, logged and recorded as `last_status: error`; it never reaches the host. * Model calls from `fn` go through `ctx.llm` and its trust gate. `fn` has no turn, so it gets no `side_question_token` and cannot fork the session. ## Where this fits in the plugin surface Existing `ctx.*` methods extend an existing Hermes subsystem: | `ctx.register_tool` | adds a tool the agent can call | | `ctx.register_platform` | wires a new gateway adapter | | `ctx.register_image_gen_provider` | replaces an image-gen backend | | `ctx.register_memory_provider` | replaces the memory backend | | `ctx.register_context_engine` | replaces the context compressor | | `ctx.register_hook` | observes a lifecycle event | `ctx.llm` is the first surface that lets a plugin run the same model the user is talking to, *out of band*, without any of the above. That's its only job. If your plugin needs to register a tool the agent invokes, use `register_tool`. If it needs to react to a lifecycle event, use `register_hook`. If it needs to make its own model call — for any reason, structured or not — `ctx.llm`. ## Reference * Implementation: [`agent/plugin_llm.py`](https://github.com/NousResearch/hermes-agent/blob/main/agent/plugin_llm.py) * Tests: [`tests/agent/test_plugin_llm.py`](https://github.com/NousResearch/hermes-agent/blob/main/tests/agent/test_plugin_llm.py) * Reference plugins (companion repo): * [`plugin-llm-example`](https://github.com/NousResearch/hermes-example-plugins/tree/main/plugin-llm-example) — sync structured extraction with image input * [`plugin-llm-async-example`](https://github.com/NousResearch/hermes-example-plugins/tree/main/plugin-llm-async-example) — async with `asyncio.gather()` * Auxiliary client (the engine under the hood): see [Provider Runtime](./provider-runtime.md).