From d5b3c781329f957ebf124b7582b4222144cfc684 Mon Sep 17 00:00:00 2001 From: can1357 Date: Wed, 17 Jun 2026 04:37:39 +0200 Subject: [PATCH] chore: update docs --- docs/ERRATA-GPT5-HARMONY.md | 8 +- docs/advisor-watchdog.md | 6 +- docs/ai-schema-normalize.md | 9 +- docs/approval-mode.md | 2 +- docs/bash-tool-runtime.md | 44 +- docs/blob-artifact-architecture.md | 6 +- docs/collab.md | 2 +- docs/compaction.md | 2 +- docs/config-usage.md | 3 +- docs/context-files.md | 4 +- docs/custom-tools.md | 2 +- docs/environment-variables.md | 3 + docs/extension-loading.md | 24 +- docs/extensions.md | 2 + docs/fs-scan-cache-architecture.md | 7 +- docs/handoff-generation-pipeline.md | 6 +- docs/hooks.md | 2 +- docs/install-id.md | 2 +- docs/mcp-protocol-transports.md | 2 +- docs/mcp-server-tool-authoring.md | 2 +- docs/memory.md | 2 +- docs/mnemosyne-memory-backend.md | 7 +- docs/models.md | 13 +- docs/natives-addon-loader-runtime.md | 2 +- docs/natives-architecture.md | 4 +- docs/natives-binding-contract.md | 6 +- docs/natives-build-release-debugging.md | 4 +- docs/natives-media-system-utils.md | 2 +- docs/natives-rust-task-cancellation.md | 7 +- docs/natives-text-search-pipeline.md | 6 +- docs/non-compaction-retry-policy.md | 15 +- docs/plugin-manager-installer-plumbing.md | 14 +- docs/porting-from-pi-mono.md | 2 +- docs/provider-streaming-internals.md | 6 +- docs/providers.md | 5 +- docs/python-repl.md | 4 +- docs/resolve-tool-runtime.md | 2 +- docs/rpc.md | 2 +- docs/rulebook-matching-pipeline.md | 13 +- docs/secrets.md | 2 +- docs/session-switching-and-recent-listing.md | 12 +- docs/session-tree-plan.md | 21 +- docs/session.md | 49 +- docs/settings.md | 2 +- docs/skills.md | 8 +- docs/skills/authoring-extensions.md | 6 +- docs/skills/authoring-hooks.md | 2 +- .../examples/mini-marketplace/README.md | 2 +- docs/slash-command-internals.md | 3 +- docs/task-agent-discovery.md | 2 +- docs/theme.md | 2 +- docs/toolconv/anthropic.md | 6 +- docs/toolconv/deepseek.md | 27 + docs/toolconv/gemma.md | 53 +- docs/toolconv/glm-4.5.md | 4 +- docs/tools/ask.md | 4 +- docs/tools/ast-edit.md | 8 +- docs/tools/ast-grep.md | 19 +- docs/tools/bash.md | 12 +- docs/tools/browser.md | 29 +- docs/tools/checkpoint.md | 2 +- docs/tools/debug.md | 6 +- docs/tools/edit.md | 21 +- docs/tools/eval.md | 28 +- docs/tools/github.md | 10 +- docs/tools/inspect_image.md | 2 +- docs/tools/lsp.md | 2 +- docs/tools/read.md | 6 +- docs/tools/retain.md | 2 +- docs/tools/rewind.md | 10 +- docs/tools/search.md | 4 +- docs/tools/search_tool_bm25.md | 2 +- docs/tools/ssh.md | 2 +- docs/tools/task.md | 5 +- docs/tools/todo.md | 12 +- docs/tools/web_search.md | 7 +- docs/tools/write.md | 3 +- docs/tree.md | 2 +- docs/tui-core-renderer.md | 17 +- docs/tui-runtime-internals.md | 7 +- docs/tui.md | 1 + packages/coding-agent/DEVELOPMENT.md | 1321 ++--------------- 82 files changed, 549 insertions(+), 1450 deletions(-) diff --git a/docs/ERRATA-GPT5-HARMONY.md b/docs/ERRATA-GPT5-HARMONY.md index 78f886678..5173740d2 100644 --- a/docs/ERRATA-GPT5-HARMONY.md +++ b/docs/ERRATA-GPT5-HARMONY.md @@ -140,10 +140,10 @@ reproduction (§7.3), independent of the prompt's natural language. The `edit` tool exists in two variants in the corpus: -| Variant | Calls | Recovery | -| ------------------------------------ | ----: | ---------------------------------------------------------------------------------------------------------------------------------------------------- | -| Patch-DSL (`§PATH`/anchor/`«»≔` ops) | 27 | **Recoverable** by op-truncation (§3.3) | -| JSON-schema (`{path,edits:[…]}`) | 11 | **Not recoverable** — contamination is escaped _inside_ JSON strings, parser accepts it cleanly, content would be written verbatim into source files | +| Variant | Calls | Recovery | +| -------------------------------------------------- | ----: | ---------------------------------------------------------------------------------------------------------------------------------------------------- | +| Patch-DSL (`[PATH#TAG]`/anchor/`SWAP DEL INS` ops) | 27 | **Recoverable** by op-truncation (§3.3) | +| JSON-schema (`{path,edits:[…]}`) | 11 | **Not recoverable** — contamination is escaped _inside_ JSON strings, parser accepts it cleanly, content would be written verbatim into source files | For Patch-DSL leaks specifically: diff --git a/docs/advisor-watchdog.md b/docs/advisor-watchdog.md index f8faa69bc..9d19b49e1 100644 --- a/docs/advisor-watchdog.md +++ b/docs/advisor-watchdog.md @@ -44,14 +44,14 @@ Slash commands: | `/advisor on` | Enable the setting and start the runtime when an advisor model is assigned. | | `/advisor off` | Disable the setting and stop the runtime. | | `/advisor status` | Show active model, context usage, token usage, and cost. | -| `/advisor dump` | Print the advisor's compact transcript. | -| `/advisor dump raw` | Print the advisor's full dump with system prompt, tools, thinking, and calls. | +| `/advisor dump` | Copy the advisor's compact transcript to the clipboard. | +| `/advisor dump raw` | Copy the advisor's full dump (system prompt, tools, thinking, and calls) to the clipboard. | If `advisor.enabled` is true but no `modelRoles.advisor` value resolves to an available model, status reports that the setting is enabled but no advisor model is assigned. ## What the advisor sees -At each primary turn end, `AdvisorRuntime` receives only the new transcript delta since the last advisor update. Deltas are rendered with `formatSessionHistoryMarkdown(..., { includeThinking: true })`, so the advisor can review assistant reasoning as well as user-visible text, tool calls, and tool results. +At each primary turn end, `AdvisorRuntime` receives only the new transcript delta since the last advisor update. Deltas are rendered with `formatSessionHistoryMarkdown(..., { includeThinking: true, includeToolIntent: true, watchedRoles: true })`, so the advisor can review assistant reasoning as well as user-visible text, tool calls, and tool results. Advisor messages already injected into the primary transcript are filtered out before the next delta is rendered. This prevents the advisor from recursively reviewing its own advice. diff --git a/docs/ai-schema-normalize.md b/docs/ai-schema-normalize.md index dfb4762df..cd97f3c2c 100644 --- a/docs/ai-schema-normalize.md +++ b/docs/ai-schema-normalize.md @@ -20,8 +20,8 @@ All exports live under `@oh-my-pi/pi-ai/utils/schema`: - `normalizeSchemaForMCP(value)` — MCP inputSchemas before they enter the custom-tool registry. `tool-bridge.ts` runs every MCP `inputSchema` through this dispatcher. -- `normalizeSchemaForOpenAIResponses(schema)` (alias - `sanitizeSchemaForOpenAIResponses`) — rewrites `oneOf` → `anyOf` for the +- `sanitizeSchemaForOpenAIResponses(schema)` (alias + `normalizeSchemaForOpenAIResponses`) — rewrites `oneOf` → `anyOf` for the Responses family. - `sanitizeSchemaForStrictMode(schema)` and `enforceStrictSchema(schema)` / `tryEnforceStrictSchema(schema)` — the @@ -77,7 +77,8 @@ pinned by the dispatcher. Each node: marker on Google, plain `type: "T"` on CCA). 5. Collapses object-only / same-type combiners, optionally lossy-collapses mixed-type combiners (CCA only), and runs the residual-combiner fixpoint. -6. Validates against AJV 2020 when `validateAndFallback` is set (CCA path) +6. Validates with the in-house structural validator (`isValidJsonSchema` + from `meta-validator.ts`) when `validateAndFallback` is set (CCA path) and emits the per-tool fallback `{ "type": "object", "properties": {} }` on residual incompatibility — `type` array, `type: "null"`, `nullable` key, or any remaining `anyOf`/`oneOf`/`allOf`. @@ -139,7 +140,7 @@ so callers MUST emit `strict: true` only when enforcement actually succeeded. `resolveProviderModels` in `packages/catalog/src/model-manager.ts` and `readModelCache`/`writeModelCache` in `packages/catalog/src/model-cache.ts` cooperate via a `static_fingerprint` column on the `model_cache` SQLite -table (current cache schema version 5). +table (current cache schema version 6). - `fingerprintStatic(staticModels)` hashes the static catalog slice (`Bun.hash(JSON.stringify(models))` in base36) and memoizes the result diff --git a/docs/approval-mode.md b/docs/approval-mode.md index 97c59d3a0..85167f526 100644 --- a/docs/approval-mode.md +++ b/docs/approval-mode.md @@ -8,7 +8,7 @@ Tool approval has two independent inputs: - `exec`: executes code, shells out, drives a browser, spawns agents, or performs similarly broad actions. 2. **User policy** — `tools.approval.: allow | deny | prompt` overrides the mode for that tool unless a non-yolo safety override forces a prompt. -Tools without an `approval` declaration are treated as `exec`. This is the safe default for MCP and unknown custom tools. +Tools without an `approval` declaration are treated as `exec`. This is the safe default for unknown custom tools. MCP server tools declare `write`. ## Modes diff --git a/docs/bash-tool-runtime.md b/docs/bash-tool-runtime.md index 8adcb007c..468967ba0 100644 --- a/docs/bash-tool-runtime.md +++ b/docs/bash-tool-runtime.md @@ -33,7 +33,7 @@ There are no structured `head` or `tail` tool parameters in the current schema. ## 2) Optional interception (blocked-command path) -If `bashInterceptor.enabled` is true, `BashTool` loads rules from settings and runs `checkBashInterception()` against the normalized command. +If `bashInterceptor.enabled` is true, `BashTool` loads rules from settings (`getBashInterceptorRules()`) and runs `checkBashInterception()` against the command — checking both the original and the cwd-normalized form (after a leading `cd … &&` is extracted) when they differ. Interception behavior: @@ -75,13 +75,15 @@ Before execution, the tool allocates an artifact path/id (best-effort) for trunc ## 5) PTY vs non-PTY execution selection -`BashTool` chooses PTY execution only when all are true: +PTY eligibility is decided by `canUseInteractiveBashPty(pty, ctx)` (`src/tools/bash-pty-selection.ts`); the local PTY overlay runs only when all are true: - tool input `pty === true` - `PI_NO_PTY !== "1"` - tool context has UI (`ctx.hasUI === true` and `ctx.ui` set) -Otherwise it uses non-interactive `executeBash()`. +If `pty` is requested but unavailable, the call falls back to non-PTY and appends a `pty requested but unavailable …` notice. + +Before the local PTY/non-PTY choice, a foreground (`async: false`) call can route to a managed background job (auto-backgrounding; see below) or — when the session's client advertises a terminal capability (`clientBridge.capabilities.terminal` + `createTerminal`, with `pty` false) — to a **client-bridge editor terminal** that runs the command remotely (streaming `terminalId` updates, killing on timeout, mapping a signal kill to exit code `137`). Otherwise it uses non-interactive `executeBash()`. That means print mode and non-UI RPC/tool contexts always use non-PTY. @@ -120,6 +122,14 @@ If `prefix` is configured, command becomes: ``` +The per-command child environment is built by `buildNonInteractiveEnv()` (`src/exec/non-interactive-env.ts`), which layers non-interactive hardening defaults **under** the caller's `env` overrides: + +- pagers disabled (`PAGER=cat`, `GIT_PAGER=cat`, … and `LESS=FRX`), +- editor prompts disabled (`GIT_EDITOR=true`, `EDITOR=true`, `VISUAL=true`), +- terminal/credential prompts reduced (`TERM=dumb`, `GIT_TERMINAL_PROMPT=0`, `SSH_ASKPASS=/usr/bin/false`, `NO_COLOR=1`, `CI=1`), +- package-manager/tooling automation flags for non-interactive behavior (npm/pnpm/yarn/pip/cargo/terraform/gh, …), +- on Windows, UTF-8 locale/codepage defaults are added when absent. + ## Streaming and cancellation `Shell.run()` streams chunks to `OutputSink` and optional `onChunk` callback. @@ -143,12 +153,7 @@ Behavior highlights: - `esc` while running kills the PTY session, - terminal resize propagates to PTY (`session.resize(cols, rows)`). -Environment hardening defaults are injected for unattended runs: - -- pagers disabled (`PAGER=cat`, `GIT_PAGER=cat`, etc.), -- editor prompts disabled (`GIT_EDITOR=true`, `EDITOR=true`, ...), -- terminal/auth prompts reduced (`GIT_TERMINAL_PROMPT=0`, `SSH_ASKPASS=/usr/bin/false`, `CI=1`), -- package-manager/tool automation flags for non-interactive behavior. +Unlike the non-PTY engine, the interactive PTY path does **not** apply the non-interactive hardening. It inherits the user's environment and sets a real `TERM=xterm-256color` (applied as an override on the Rust side) so editors, pagers, and TUIs behave like a normal terminal. PTY output is normalized (`CRLF`/`CR` to `LF`, `sanitizeText`) and written into `OutputSink`, including artifact spill support. @@ -160,11 +165,14 @@ Both PTY and non-PTY paths use `OutputSink`. ## OutputSink semantics -- keeps an in-memory UTF-8-safe tail buffer (`DEFAULT_MAX_BYTES`, currently 50KB), +The bash executor builds the sink with `headBytes` and `maxColumns` from settings (`resolveOutputSinkHeadBytes` / `resolveOutputMaxColumns`). + +- keeps a UTF-8-safe rolling **tail** window (`spillThreshold`, `DEFAULT_MAX_BYTES`, currently 50KB); on overflow it trims to the tail (UTF-8 boundary safe) and marks `truncated`, +- when `headBytes > 0` (`tools.artifactHeadBytes`, default 20KB) it also retains a **head** window and elides the middle, splicing an elision marker between head and tail in `dump()`, +- per-line column cap: when `maxColumns > 0` (`tools.outputMaxColumns`, default 768 bytes) over-wide lines are ellipsis-truncated at write time and the rest of the line is dropped, - tracks total bytes/lines seen, -- if artifact path exists and output overflows (or file already active), writes full stream to artifact file, -- when memory threshold overflows, trims in-memory buffer to tail (UTF-8 boundary safe), -- marks `truncated` when overflow/file spill occurs. +- mirrors the **raw, uncapped** stream to the artifact file when output overflows, a column cap dropped bytes, or the file is already active, +- marks `truncated` on tail overflow, middle elision, column-cap drops, or file spill. `dump()` returns: @@ -172,11 +180,13 @@ Both PTY and non-PTY paths use `OutputSink`. - `truncated`, - `totalLines/totalBytes`, - `outputLines/outputBytes`, +- `elidedBytes/elidedLines` when the middle was elided, +- `columnDroppedBytes/columnTruncatedLines` when the per-line cap fired, - `artifactId` if artifact file was active. ### Long-output caveat -Runtime truncation is byte-threshold based in `OutputSink` (50KB default). It does not enforce a hard 2000-line cap in this code path. +Runtime truncation is byte-threshold based in `OutputSink` (50KB tail window by default, plus an optional head window for middle elision). It does not enforce a hard line-count cap in this code path. ### Shell output minimizer @@ -213,7 +223,7 @@ Success payload structure: - `shownRange`, - `artifactId` when available. -Because built-in tools are wrapped with `wrapToolWithMetaNotice()`, truncation notice text is appended to final text content automatically (for example: `Full: artifact://`). +Because built-in tools are wrapped with `wrapToolWithMetaNotice()`, truncation notice text is appended to final text content automatically (for example: `Read artifact:// for full output`). ## Rendering paths @@ -264,10 +274,12 @@ This component is wired by `CommandController.handleBashCommand()` and fed from ## Implementation files - [`src/tools/bash.ts`](../packages/coding-agent/src/tools/bash.ts) — tool entrypoint, input handling/interception, async and PTY/non-PTY selection, result/error mapping, bash tool renderer. +- [`src/tools/bash-pty-selection.ts`](../packages/coding-agent/src/tools/bash-pty-selection.ts) — `canUseInteractiveBashPty` predicate for choosing the local PTY overlay. - [`src/tools/bash-command-fixup.ts`](../packages/coding-agent/src/tools/bash-command-fixup.ts) — native-backed conservative cleanup for trailing `head`/`tail` pipes and redundant `2>&1`. - [`src/tools/bash-interceptor.ts`](../packages/coding-agent/src/tools/bash-interceptor.ts) — interceptor rule matching and blocked-command messages. - [`src/exec/bash-executor.ts`](../packages/coding-agent/src/exec/bash-executor.ts) — non-PTY executor, shell session reuse, cancellation wiring, output sink integration. -- [`src/tools/bash-interactive.ts`](../packages/coding-agent/src/tools/bash-interactive.ts) — PTY runtime, overlay UI, input normalization, non-interactive env defaults. +- [`src/exec/non-interactive-env.ts`](../packages/coding-agent/src/exec/non-interactive-env.ts) — non-interactive child-process env defaults (`buildNonInteractiveEnv`) used by the non-PTY executor. +- [`src/tools/bash-interactive.ts`](../packages/coding-agent/src/tools/bash-interactive.ts) — PTY runtime, overlay UI, input normalization, and interactive `TERM` setup. - [`src/session/streaming-output.ts`](../packages/coding-agent/src/session/streaming-output.ts) — `OutputSink`, `TailBuffer`, truncation/artifact spill, and summary metadata. - [`src/tools/output-meta.ts`](../packages/coding-agent/src/tools/output-meta.ts) — truncation metadata shape + notice injection wrapper. - [`src/session/agent-session.ts`](../packages/coding-agent/src/session/agent-session.ts) — session-level `executeBash`, message recording, abort lifecycle. diff --git a/docs/blob-artifact-architecture.md b/docs/blob-artifact-architecture.md index 516a2dc00..edf696b18 100644 --- a/docs/blob-artifact-architecture.md +++ b/docs/blob-artifact-architecture.md @@ -83,7 +83,7 @@ Non-persistent sessions without an adopted manager can store `saveArtifact(...)` ### 1) Session entry persistence rewrite path -Before session entries are written (`#rewriteFile` / incremental persist), `SessionManager` calls `prepareEntryForPersistence()` / `prepareEntryForPersistenceSync()` through the truncation pipeline. +Before a session entry is written — incremental append (`#appendToSessionFile`) or a full-file rewrite (`#rewriteSynchronously` / `#rewriteAtomically`) — `SessionManager` serializes it through `#lineFor()`, which runs `prepareEntryForPersistence()` over the truncation pipeline. Key behaviors: @@ -234,7 +234,9 @@ The two systems intersect only indirectly: both reduce session JSONL bloat, but - [`src/session/blob-store.ts`](../packages/coding-agent/src/session/blob-store.ts) — blob reference format, hashing, put/get, externalize/resolve helpers. - [`src/session/artifacts.ts`](../packages/coding-agent/src/session/artifacts.ts) — session artifact directory model and numeric artifact ID/path allocation. - [`src/session/streaming-output.ts`](../packages/coding-agent/src/session/streaming-output.ts) — `OutputSink` truncation/spill-to-file behavior and summary metadata. -- [`src/session/session-manager.ts`](../packages/coding-agent/src/session/session-manager.ts) — persistence transforms, blob rehydration on load, session fork/move interactions. +- [`src/session/session-manager.ts`](../packages/coding-agent/src/session/session-manager.ts) — `BlobStore`/`ArtifactManager` construction, persistence-transform and blob-rehydration call sites, session fork/move interactions. +- [`src/session/session-persistence.ts`](../packages/coding-agent/src/session/session-persistence.ts) — `prepareEntryForPersistence()`: large-string truncation, transient-field stripping, and synchronous image-blob externalization. +- [`src/session/session-loader.ts`](../packages/coding-agent/src/session/session-loader.ts) — `resolveBlobRefsInEntries()`: blob-ref rehydration to base64 / data URLs on load. - [`src/session/agent-session.ts`](../packages/coding-agent/src/session/agent-session.ts) — artifact directory copy during interactive fork. - [`src/internal-urls/artifact-protocol.ts`](../packages/coding-agent/src/internal-urls/artifact-protocol.ts) — `artifact://` resolver. - [`src/internal-urls/agent-protocol.ts`](../packages/coding-agent/src/internal-urls/agent-protocol.ts) — `agent://` resolver + JSON extraction. diff --git a/docs/collab.md b/docs/collab.md index 5f199625a..e86a709a3 100644 --- a/docs/collab.md +++ b/docs/collab.md @@ -77,7 +77,7 @@ Guests with a full link can: Guests with a view-only link can read everything live — back-transcript, streaming text, tool cards, subagent transcripts — but the host rejects prompting, interrupting, and agent control from them. -Everything that mutates the host session or machine is host-only: `/model`, `/compact`, `/resume`, `/branch`, bash (`!`), python (`$`), skills, etc. Guests keep a small local allowlist (`/dump`, `/export`, `/copy`, `/help`, `/hotkeys`, `/theme`, `/settings`, `/leave`, `/collab`, `/exit`). +Everything that mutates the host session or machine is host-only: `/model`, `/compact`, `/resume`, `/branch`, bash (`!`), python (`$`), skills, etc. Guests keep a small local allowlist (`/dump`, `/export`, `/copy`, `/help`, `/hotkeys`, `/theme`, `/settings`, `/leave`, `/collab`, `/exit`, `/quit`). Known v1 limit for guests: a turn already streaming when you join becomes visible from its next message boundary. diff --git a/docs/compaction.md b/docs/compaction.md index b745aa0ca..84a0cb0c5 100644 --- a/docs/compaction.md +++ b/docs/compaction.md @@ -132,7 +132,7 @@ The automatic paths are intentionally different: `compaction.strategy: "snapcompact"` replaces the LLM summarization call with a local, deterministic archival pass (`compact` from `@oh-my-pi/snapcompact`): -- The discarded history is serialized, whitespace-collapsed, and printed onto model-aware PNG frames (frame width fixed per shape; frame height hugs the rows actually printed) using bundled public-domain pixel fonts. The shape — and frame size — resolve from the **model id** when the model line was measured: Claude reads X.org `6x12` glyphs with dimmed stopwords (`6x12-dim`; high-res lines — Opus 4.7+, Fable, Mythos — get 1932px frames under Anthropic's 4,784 visual-token cap, older lines stay at 1568px), Gemini reads two word-wrapped columns of `8x13` glyphs with sentence-hue ink and dimmed stopwords (`doc-8on16-sent-dim` at 2048px — Gemini 3.x bills a fixed 1,120-token budget per image at any pixel size), GPT/Kimi/GLM read `8x13` glyphs on a 16px pitch (`8on16-bw` at 1568px — patch billing is area-proportional, and kimi's processor downscales past 1792px). A Claude routed through Vertex or OpenRouter keeps its Claude shape. Unmeasured models fall back to their wire API family (Anthropic-family/unknown → `6x12-dim`, Google → `doc-8on16-sent-dim`, OpenAI-compatible → `8on16-bw`); billing (per-family patch/budget formulas, OpenAI's `detail: "original"` hint) always follows the API carrying the request, computed for the resolved frame size. The `snapcompact.shape` setting (default `auto`) forces one of the research-eval variants instead: square grids (`8x8r`/`8x8u`/`6x6u`/`5x8` × sentence-hue/black ink) or the per-model eval winners (`6x12-dim`, `8x13-bw`, `8on16-bw`, and the two-column word-wrapped `doc-8on16-bw`/`-sent`/`-sent-dim`, where `dim` prints stopwords in gray). A forced variant keeps its geometry but is re-priced for the target provider's image billing. The same setting governs inline system-prompt/tool-result imaging (`snapcompact.systemPrompt`, `snapcompact.toolResults`). +- The discarded history is serialized, whitespace-collapsed, and printed onto model-aware PNG frames (frame width fixed per shape; frame height hugs the rows actually printed) using bundled public-domain pixel fonts. The shape — and frame size — resolve from the **model id** when the model line was measured: Claude reads X.org `8x13` glyphs on an 11px advance (extra letter-spacing, black ink — `11on16-bw`; high-res lines — Opus 4.7+, Fable, Mythos — get 1932px frames under Anthropic's 4,784 visual-token cap, older lines stay at 1568px), Gemini reads `8x13` glyphs on a 22px pitch (extra leading, black ink — `8on22-bw` at 2048px, since Gemini 3.x bills a fixed 1,120-token budget per image at any pixel size), GPT/Codex read the same `8on22-bw` shape at 1568px (patch billing is area-proportional, so larger frames cannot improve chars per token), and Kimi/GLM read `8x13` glyphs on a 16px pitch (`8on16-bw` at 1568px — kimi's processor downscales past 1792px). A Claude routed through Vertex or OpenRouter keeps its Claude shape. Unmeasured models fall back to their wire API family (Anthropic-family/unknown → `11on16-bw`, Google → `8on22-bw`, OpenAI-compatible → `8on22-bw`); billing (per-family patch/budget formulas, OpenAI's `detail: "original"` hint) always follows the API carrying the request, computed for the resolved frame size. The `snapcompact.shape` setting (default `auto`) forces one of the research-eval variants instead: square grids (`8x8r`/`8x8u`/`6x6u`/`5x8` × sentence-hue/black ink) or the per-model eval winners (`6x12-dim`, `8x13-bw`, `8on16-bw`, `8on22-bw`, `11on16-bw`, and the two-column word-wrapped `doc-8on16-bw`/`-sent`/`-sent-dim`, where `dim` prints stopwords in gray). A forced variant keeps its geometry but is re-priced for the target provider's image billing. The same setting governs inline system-prompt/tool-result imaging (`snapcompact.systemPrompt`, `snapcompact.toolResults`). - Serialization keeps the archive conversation-dense: tool results are truncated head+tail (default 2,000 chars at a 0.6 head ratio), tool-call argument values are capped per value (500) and per call (2,000), and tool output is printed in dim gray ink so conversation reads louder than tool noise. All budgets and the dimming are configurable via `SerializeOptions` (`toolResultMaxChars`, `toolArgMaxChars`, `toolCallMaxChars`, `truncateHeadRatio`, `dimToolResults`). - Frames persist under `CompactionEntry.preserveData.snapcompact` and are re-attached to the `compactionSummary` message as image blocks on every context rebuild; the entry's `summary` is a deterministic reading guide (grid geometry, role tags, truncation notes) plus the usual file-operation lists. - Later compactions carry earlier frames forward. The frame budget is provider-aware (`providerFrameBudget`): the per-provider image cap clamped to 8 (`MAX_FRAMES`) — OpenRouter hard-caps requests at 8 images and silently drops the excess, unknown providers get a safe floor of 5. Beyond the budget the archive fades from the middle out: the earliest frame (session head — the original request, or the filmed summary of older history) is pinned, and the oldest *unpinned* frames are evicted. Pages of the *current* compaction that no longer fit are never rendered or dropped — the newest unframed slice survives verbatim as a text tail on the summary (`Archive.textTail`, capped at two frame capacities with middle elision) and is folded back into frames by the next compaction. If the previous compaction was text-based, its summary is printed at the head of the frame archive as `[Summary of earlier history]`. diff --git a/docs/config-usage.md b/docs/config-usage.md index e4269b695..2744a827d 100644 --- a/docs/config-usage.md +++ b/docs/config-usage.md @@ -7,6 +7,7 @@ This document describes how the coding-agent resolves configuration today: which Primary implementation: - `packages/coding-agent/src/config.ts` +- `packages/coding-agent/src/config/config-file.ts` (re-exported from `config.ts`) - `packages/coding-agent/src/config/settings.ts` - `packages/coding-agent/src/config/settings-schema.ts` - `packages/coding-agent/src/discovery/builtin.ts` @@ -116,7 +117,7 @@ Use this when project config should be inherited from ancestor directories (mono --- -## 3) File config wrapper (`ConfigFile` in `src/config.ts`) +## 3) File config wrapper (`ConfigFile` in `src/config/config-file.ts`, re-exported from `src/config.ts`) `ConfigFile` is the schema-validated loader for single config files. diff --git a/docs/context-files.md b/docs/context-files.md index 814734763..776b81cff 100644 --- a/docs/context-files.md +++ b/docs/context-files.md @@ -63,7 +63,7 @@ Put broad, durable project background in `AGENTS.md`. Reserve `RULES.md` for sho | `codex` | `.codex/AGENTS.md` | User | User file `~/.codex/AGENTS.md` only. Project-level Codex context comes from a standalone `AGENTS.md` via the `agents-md` provider, not from `/.codex/AGENTS.md`. | | `gemini` | `.gemini/GEMINI.md` | User + project | User file `~/.gemini/GEMINI.md`; project file `/.gemini/GEMINI.md` only (no ancestor walk-up). | | `opencode` | `.config/opencode/AGENTS.md` | User | User file `~/.config/opencode/AGENTS.md` only. | -| `github` | `.github/copilot-instructions.md` | Project | Project-only GitHub Copilot instructions from `/.github/copilot-instructions.md`. | +| `github` | `.github/copilot-instructions.md` | User + project | Project file `/.github/copilot-instructions.md` only (no ancestor walk-up), plus a user-global `~/.copilot/copilot-instructions.md` (relocate with `COPILOT_HOME`) and an `AGENTS.md` from each `COPILOT_CUSTOM_INSTRUCTIONS_DIRS` entry. | | `agents` | `.agent/AGENTS.md`, `.agents/AGENTS.md` | User + project | User files from `~/.agent/` and `~/.agents/`; project files discovered while walking up from the current directory to the repository root. | | `agents-md` | `AGENTS.md` | Project | Standalone (non-config-directory) `AGENTS.md` files, discovered by walking up from the current directory to the repository root (or home when no repo root is known). Files whose parent directory name starts with `.` are ignored — those belong to a config-directory provider instead. | | `github` | `.github/instructions/**/*.instructions.md` | Project rules | GitHub Copilot / VS Code instruction files become rules. `applyTo: '*'` or `applyTo: '**'` is injected as always-apply context; other `applyTo` globs are listed in the rulebook with `description` and are readable as `rule://`. | @@ -225,7 +225,7 @@ At one user scope or project depth, the higher-priority provider shadows the oth ### User context disappeared -Only one user-level context file survives, and `~/.omp/agent/AGENTS.md` has the highest priority. If it exists, it shadows user-level `~/.claude/CLAUDE.md`, `~/.codex/AGENTS.md`, `~/.gemini/GEMINI.md`, `~/.config/opencode/AGENTS.md`, and `~/.agent`/`~/.agents` files. Consolidate user guidance into the native file or remove the native one if you prefer another tool's file. +Only one user-level context file survives, and `~/.omp/agent/AGENTS.md` has the highest priority. If it exists, it shadows user-level `~/.claude/CLAUDE.md`, `~/.codex/AGENTS.md`, `~/.gemini/GEMINI.md`, `~/.config/opencode/AGENTS.md`, `~/.copilot/copilot-instructions.md`, and `~/.agent`/`~/.agents` files. Consolidate user guidance into the native file or remove the native one if you prefer another tool's file. ### A `RULES.md` file is ignored diff --git a/docs/custom-tools.md b/docs/custom-tools.md index 775be04a8..9f74137fd 100644 --- a/docs/custom-tools.md +++ b/docs/custom-tools.md @@ -143,7 +143,7 @@ execute(toolCallId, params, onUpdate, ctx, signal); - `params` is statically typed from your Zod/TypeBox schema via `Static`. - Runtime argument validation happens before execution in the agent loop. - `onUpdate` emits partial results for UI streaming. -- `ctx` includes `sessionManager`, `modelRegistry`, current `model`, `isIdle()`, `hasQueuedMessages()`, `abort()`, and optional `settings` / `autoApprove`. +- `ctx` includes `sessionManager`, `modelRegistry`, current `model`, `isIdle()`, `hasQueuedMessages()`, `abort()`, and optional `settings`, `fetch`, and `autoApprove`. - `signal` carries cancellation. `CustomToolAdapter` bridges this to the agent tool interface and forwards calls in the correct argument order. diff --git a/docs/environment-variables.md b/docs/environment-variables.md index 328a7e2ae..252f3bddd 100644 --- a/docs/environment-variables.md +++ b/docs/environment-variables.md @@ -55,6 +55,9 @@ These are consumed via `getEnvApiKey()` (`packages/ai/src/stream.ts`) unless not | `OLLAMA_API_KEY` | Ollama auth (optional) | Using `ollama` provider with authenticated hosts | Local Ollama usually runs without auth; any non-empty token works when a key is required | | `LLAMA_CPP_API_KEY` | llama.cpp auth (optional) | Using `llama.cpp` provider with authenticated hosts | Local llama.cpp usually runs without auth; any non-empty token works when a key is configured | | `XIAOMI_API_KEY` | Xiaomi MiMo auth | Using `xiaomi` provider | | +| `XIAOMI_TOKEN_PLAN_AMS_API_KEY` | Xiaomi MiMo Token Plan auth (AMS) | Using `xiaomi-token-plan-ams` provider | | +| `XIAOMI_TOKEN_PLAN_CN_API_KEY` | Xiaomi MiMo Token Plan auth (CN) | Using `xiaomi-token-plan-cn` provider | | +| `XIAOMI_TOKEN_PLAN_SGP_API_KEY` | Xiaomi MiMo Token Plan auth (SGP) | Using `xiaomi-token-plan-sgp` provider | | | `MOONSHOT_API_KEY` | Moonshot auth | Using `moonshot` provider | | | `XAI_API_KEY` | xAI auth | Using xAI models or as fallback for `xai-oauth` | | | `XAI_OAUTH_TOKEN` | xAI OAuth/SuperGrok auth | Using `xai-oauth` provider | Takes precedence over `XAI_API_KEY` for `xai-oauth` | diff --git a/docs/extension-loading.md b/docs/extension-loading.md index 5cfe94ee0..3ed7eb08d 100644 --- a/docs/extension-loading.md +++ b/docs/extension-loading.md @@ -41,13 +41,19 @@ Notes: - Native auto-discovery is currently `.omp` based. - Legacy `.pi` is still accepted in package manifests (`pi.extensions`) and project override lookup, but `.pi/extensions` is not a native root here. -### 2) Installed plugin extension entries +### 2) Discovered JS/TS hook factories -After native auto-discovery, `discoverAndLoadExtensions()` appends extension entry points from enabled installed plugins via `getAllPluginExtensionPaths(cwd)`. +After native auto-discovery, `discoverAndLoadExtensions()` also appends JS/TS hook factories from the `hook` capability — any hook whose entry path is a `.ts`/`.js` file — so they load through the same module pipeline. + +Hook-capability loading already applies its own hook-specific disabled ids, so these paths are not additionally filtered by `disabledExtensions` extension-module names. + +### 3) Installed plugin extension entries + +After hook discovery, `discoverAndLoadExtensions()` appends extension entry points from enabled installed plugins via `getAllPluginExtensionPaths(cwd)`. Plugin extension entries come from package `omp.extensions` / `pi.extensions` manifests, including enabled feature entries. -### 3) Explicitly configured paths +### 4) Explicitly configured paths After plugin extension entries, configured paths are appended and resolved. @@ -163,8 +169,9 @@ Rules and constraints: Order: 1. Native auto-discovered modules -2. Installed plugin extension entries -3. Explicit configured paths (in provided order) +2. Discovered JS/TS hook factories +3. Installed plugin extension entries +4. Explicit configured paths (in provided order) In `sdk.ts`, configured order is: @@ -183,10 +190,11 @@ Implication: if the same module path is both auto-discovered and explicitly conf ## Module import and factory contract -Each candidate path is loaded with dynamic import: +Each candidate path is loaded via `loadLegacyPiModule()` (`src/extensibility/plugins/legacy-pi-compat.ts`): -- `await import(resolvedPath)` -- factory is `module.default ?? module` +- the entry's realpath is resolved, then dynamically imported with an `?mtime` cache-buster so edited source reloads +- a scoped Bun `onLoad` hook rewrites legacy pi-package specifiers (`@mariozechner/*`, `@earendil-works/*`) and bare `@sinclair/typebox` onto the host-bundled copies before evaluation +- factory is selected by `getExtensionFactory(module)`: the module itself if it is a function, otherwise `module.default` - factory must be a function (`ExtensionFactory`) If export is not a function, that path fails with a structured error and loading continues. diff --git a/docs/extensions.md b/docs/extensions.md index 047304f4b..a28a4a5a1 100644 --- a/docs/extensions.md +++ b/docs/extensions.md @@ -157,6 +157,7 @@ Handlers and tool `execute` receive `ctx` with: - `isIdle()`, `hasPendingMessages()`, `abort()` - `shutdown()` - `getSystemPrompt()` +- `memory` (optional structured memory runtime — status/search/save across the configured backend) ### Model selection (`ctx.models`) @@ -224,6 +225,7 @@ Cancelable pre-events: - `tool_call` (pre-exec, may block) - `tool_result` (post-exec, may patch content/details/isError) - `tool_execution_start` / `tool_execution_update` / `tool_execution_end` (observability) +- `tool_approval_requested` / `tool_approval_resolved` (observability; emitted by `wrapper.ts` only when a tool requires approval and an approval handler is registered) `tool_result` is middleware-style: handlers run in extension order and each sees prior modifications. diff --git a/docs/fs-scan-cache-architecture.md b/docs/fs-scan-cache-architecture.md index 57ae42d68..f036af8c1 100644 --- a/docs/fs-scan-cache-architecture.md +++ b/docs/fs-scan-cache-architecture.md @@ -19,6 +19,7 @@ Primary goals: - `crates/pi-natives/src/glob.rs` - `crates/pi-natives/src/fd.rs` (`fuzzyFind`) - `crates/pi-natives/src/grep.rs` (cached directory mode only) + - `crates/pi-natives/src/ast.rs` (`astGrep`/`astEdit` file discovery; always cached) - JS binding/export: - `packages/natives/native/index.d.ts` (`invalidateFsScanCache`) - `packages/natives/native/index.js` @@ -97,16 +98,18 @@ Current consumers: - `glob`: rechecks when filtered matches are empty and scan age exceeds threshold - `fuzzyFind` (`fd.rs`): rechecks only when query is non-empty and scored matches are empty - `grep`: rechecks when cached directory candidate file list is empty +- `astGrep`/`astEdit` (`ast.rs`): recheck when the candidate file list is empty ## Consumer defaults and cache usage -Cache is opt-in on exposed scan/search APIs (`cache?: boolean`, default `false`). +Cache is opt-in on `glob`/`fuzzyFind`/`grep` (`cache?: boolean`, default `false`). `astGrep`/`astEdit` file discovery always uses the cache (there is no opt-in flag). Current defaults in native APIs: - `glob`: `hidden=false`, `gitignore=true`, `cache=false`; `node_modules` is included only when `includeNodeModules=true` or the pattern mentions `node_modules`; full detail is used only when `sortByMtime=true` - `fuzzyFind`: `hidden=false`, `gitignore=true`, `cache=false`, `node_modules` is skipped, `follow_links=true`, minimal detail - `grep`: `hidden=true`, `gitignore=true`, `cache=false`; cached directory mode skips `node_modules` unless the glob mentions `node_modules`; minimal detail +- `astGrep`/`astEdit` (file discovery): `hidden=true`, `gitignore=true`, always cached; `node_modules` is skipped unless the glob mentions `node_modules`; `follow_links=false`; minimal detail Current callers: @@ -181,5 +184,5 @@ When introducing cache use in a new scanner/search path: - Cache scope is process-local in-memory (`DashMap`), not persisted across process restarts. - Cache stores scan entries, not final tool results. -- `glob`/`fuzzyFind`/cached `grep` share scan entries only when key dimensions (`root`, `hidden`, `gitignore`, `skip_node_modules`, `detail`) match. +- `glob`/`fuzzyFind`/cached `grep`/`astGrep` share scan entries only when key dimensions (`root`, `hidden`, `gitignore`, `skip_node_modules`, `detail`) match. - `.git` is always excluded at scan collection time regardless of caller options. diff --git a/docs/handoff-generation-pipeline.md b/docs/handoff-generation-pipeline.md index 3ed79675b..c240a6b11 100644 --- a/docs/handoff-generation-pipeline.md +++ b/docs/handoff-generation-pipeline.md @@ -115,7 +115,7 @@ If text was generated and not aborted: 4. Reset in-memory agent state (`agent.reset()`). 5. Rebind `agent.sessionId` to the new session id. 6. Rekey/reset Hindsight and Mnemopi memory session tracking for the new session. -7. Clear queued context arrays (`#steeringMessages`, `#followUpMessages`, `#pendingNextTurnMessages`) and any scheduled hidden next-turn generation. +7. Clear the queued next-turn context array (`#pendingNextTurnMessages`) and the scheduled hidden next-turn generation (`#scheduledHiddenNextTurnGeneration`). The agent's steering and follow-up queues are already cleared by `agent.reset()` in step 4. 8. Reset todo reminder counter. ### 5) Handoff-context injection @@ -191,7 +191,7 @@ Auto-triggered handoffs can additionally write a timestamped `handoff-*.md` arti - On exception: - if message is `"Handoff cancelled"` or error name is `AbortError`: `showError("Handoff cancelled")` - otherwise: `showError("Handoff failed: ")` -- Stops the loader, restores the previous Escape handler, and requests render at end. +- Stops the loader, clears the status container, and requests render at end. Manual `/handoff` no longer streams the generated document into chat. A cancellable loader remains visible while the oneshot request runs, and the chat is rebuilt after generation completes. @@ -208,7 +208,7 @@ When this abort path is used, the abort signal is passed to `completeSimple(...) ### Interactive `/handoff` path -The command controller installs a temporary Escape handler for `/handoff` while the loader is visible. Pressing Escape calls `session.abortHandoff()`, which aborts the `completeSimple(...)` request through `#handoffAbortController`. +`InputController`'s global `editor.onEscape` handler dispatches on live session state instead of swapping handlers: while `isGeneratingHandoff` is true, pressing Escape calls `session.abortHandoff()`, which aborts the `completeSimple(...)` request through `#handoffAbortController`. ## Aborted vs failed handoff diff --git a/docs/hooks.md b/docs/hooks.md index d51247e89..934b1e973 100644 --- a/docs/hooks.md +++ b/docs/hooks.md @@ -96,7 +96,7 @@ Hook events are strongly typed in `types.ts`. ### Agent/context events - `context` → can return `{ messages?: Message[] }` -- `before_agent_start` → can return `{ message?: { customType; content; display; details } }` +- `before_agent_start` → can return `{ message?: { customType; content; display; details; attribution } }` - `agent_start` - `agent_end` - `turn_start` diff --git a/docs/install-id.md b/docs/install-id.md index 445757570..69fec02c5 100644 --- a/docs/install-id.md +++ b/docs/install-id.md @@ -15,7 +15,7 @@ Generated IDs are lowercase RFC 4122 UUIDs. Existing persisted values are accept ## Storage -- Path: `/install-id` — i.e. `~/.omp/install-id` by default, respecting `PI_CONFIG_DIR` via `getConfigRootDir()`. +- Path: `/install-id` — i.e. `~/.omp/install-id` by default, respecting `PI_CONFIG_DIR`. Resolved against the base config root (`getBaseConfigRoot()`) regardless of the active profile, so every profile on a host shares one install ID (install identity is per-install, not per-profile). - Format: a single UUID line (trailing `\n`). - Permissions: file is created with mode `0o600`. - Lifecycle: independent of `~/.omp/agent/`. Wiping agent state (sessions, settings, DB) does NOT regenerate the install ID; only deleting the `install-id` file itself does. diff --git a/docs/mcp-protocol-transports.md b/docs/mcp-protocol-transports.md index eb402635b..9aff6ee94 100644 --- a/docs/mcp-protocol-transports.md +++ b/docs/mcp-protocol-transports.md @@ -114,7 +114,7 @@ Server-initiated notifications are surfaced through transport `onNotification`; - `close()`: - `#handleClose()`: mark disconnected, reject all pending requests (`Transport closed`), emit `onClose` - kill subprocess - - await read loop shutdown + - detach read loop without awaiting (it can hang indefinitely) If read loop exits unexpectedly, `finally` triggers `#handleClose()` which performs the same pending-request rejection and close callback. diff --git a/docs/mcp-server-tool-authoring.md b/docs/mcp-server-tool-authoring.md index 2d262a2a9..8a2093309 100644 --- a/docs/mcp-server-tool-authoring.md +++ b/docs/mcp-server-tool-authoring.md @@ -35,7 +35,7 @@ Config sources (.omp/.claude/.cursor/.vscode/mcp.json, mcp.json, etc.) - non-empty - max 100 chars -- only `[a-zA-Z0-9_.-]` +- only `[a-zA-Z0-9_.:-]` (colon allows namespaced plugin server names, e.g. `cloudflare:cloudflare-api`) ### Transport pitfalls diff --git a/docs/memory.md b/docs/memory.md index 558e51d07..8f1192ba1 100644 --- a/docs/memory.md +++ b/docs/memory.md @@ -65,7 +65,7 @@ Memory extraction and consolidation behavior is driven by static prompt files in | `stage_one_input.md` | User-turn template wrapping session content | `{{thread_id}}`, `{{response_items_json}}` | | `consolidation_system.md`| System prompt for cross-session consolidation | — | | `consolidation.md` | User-turn prompt for cross-session consolidation | `{{raw_memories}}`, `{{rollout_summaries}}` | -| `read-path.md` | Memory guidance injected into live sessions | `{{memory_summary}}` | +| `read-path.md` | Memory guidance injected into live sessions | `{{memory_summary}}`, `{{learned}}` | ### Model selection diff --git a/docs/mnemosyne-memory-backend.md b/docs/mnemosyne-memory-backend.md index 347beb643..e725ca939 100644 --- a/docs/mnemosyne-memory-backend.md +++ b/docs/mnemosyne-memory-backend.md @@ -34,7 +34,7 @@ Recalled memory is background context, not instructions. Current user messages a | ------------------------------- | ---------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `memory.backend` | `off` | Set to `mnemopi` to enable this backend. | | `mnemopi.dbPath` | agent memories dir | Optional SQLite database path. | -| `mnemopi.bank` | unset | Optional shared bank base name passed to `Mnemopi`; the coding-agent wrapper scopes from this base according to `mnemopi.scoping`. Unset → shared bank `default`; per-project modes derive a project bank from the project root name plus a stable hash. | +| `mnemopi.bank` | unset | Optional shared bank base name passed to `Mnemopi`; the coding-agent wrapper scopes from this base according to `mnemopi.scoping`. Unset → shared bank `default`; per-project modes derive a project bank from the working-directory basename plus a stable hash of its absolute path. | | `mnemopi.scoping` | `per-project` | Memory visibility mode: `global` = one shared bank, `per-project` = isolated project memory, `per-project-tagged` = project-local writes plus global recall visibility. | | `mnemopi.autoRecall` | `true` | Recall memory on the first turn of a session. | | `mnemopi.autoRetain` | `true` | Retain completed turns automatically. | @@ -47,7 +47,8 @@ Recalled memory is background context, not instructions. Current user messages a | `mnemopi.injectionTokenLimit` | `5000` | Approximate token budget for memory prompt injection. | | `mnemopi.debug` | `false` | Enable debug logging for backend failures. | | `mnemopi.noEmbeddings` | `false` | Pass `noEmbeddings` to `Mnemopi` and force FTS-only recall. | -| `mnemopi.embeddingModel` | env/default | Embedding model passed to `Mnemopi`. | +| `mnemopi.embeddingVariant` | `en` | Local embedding model variant: `en` = `BAAI/bge-base-en-v1.5` (768d), `multilingual` = `intfloat/multilingual-e5-large` (1024d). `mnemopi.embeddingModel`/`MNEMOPI_EMBEDDING_MODEL` override it; changing it rebuilds stored embeddings on the next writable start. | +| `mnemopi.embeddingModel` | variant default | Explicit embedding model id; overrides `mnemopi.embeddingVariant`. Precedence: this setting > `MNEMOPI_EMBEDDING_MODEL` env > variant default. | | `mnemopi.embeddingApiUrl` | env/default | OpenAI-compatible embedding endpoint passed to `Mnemopi`. | | `mnemopi.embeddingApiKey` | env/default | Embedding API key passed to `Mnemopi`. | | `mnemopi.llmMode` | `smol` | `smol` uses the configured pi-ai smol model, `remote` uses the settings below, and `none` disables LLM calls. | @@ -60,7 +61,7 @@ Recalled memory is background context, not instructions. Current user messages a The coding-agent wrapper applies scoping on top of the underlying `Mnemopi` package: - `global` uses one shared bank for recall and writes. -- `per-project` writes to and recalls from a bank derived from the current git repository root (or cwd) plus a stable hash. +- `per-project` writes to and recalls from a bank derived from the current working directory alone — its basename plus a stable hash of its absolute path, independent of the surrounding git layout. - `per-project-tagged` writes to the project-local bank and recalls from both the project-local bank and the shared global bank, with duplicate recall results merged. The combined project-plus-global behavior lives in the wrapper. The `@oh-my-pi/pi-mnemopi` package itself still exposes banks and constructor options directly, including `bank` for selecting a bank name. Project-local banks other than the shared bank are stored as sibling bank databases managed by Mnemopi's `BankManager`. diff --git a/docs/models.md b/docs/models.md index 2286c3a47..e195310c4 100644 --- a/docs/models.md +++ b/docs/models.md @@ -9,7 +9,7 @@ Primary implementation files: - `src/config/model-registry.ts` — loads built-in + custom models, provider overrides, runtime discovery, auth integration - `src/config/model-resolver.ts` — parses model patterns and selects initial/smol/slow models - `src/config/settings-schema.ts` — model-related settings (`modelRoles`, provider transport preferences) -- `src/session/auth-storage.ts` — API key + OAuth resolution order +- `src/session/auth-storage.ts` — re-exports `AuthStorage` from `@oh-my-pi/pi-ai` (`packages/ai/src/auth-storage.ts`); API key + OAuth resolution order - `packages/catalog/src/models.ts` and `packages/catalog/src/types.ts` — built-in providers/models (`getBundledModels` / `getBundledProviders`) and `Model`/`compat` types ## Config file location and legacy behavior @@ -98,6 +98,7 @@ providers: - `azure-openai-responses` - `anthropic-messages` - `google-generative-ai` +- `google-gemini-cli` - `google-vertex` ### Allowed auth/discovery values @@ -122,6 +123,7 @@ Must define at least one of: - `baseUrl` - `apiKey` +- `auth: none` - `headers` - `compat` - `disableStrictTools` @@ -155,7 +157,7 @@ Successful command outputs are cached for the process lifetime so the command is ModelRegistry pipeline (on refresh): -1. Load built-in providers/models from `@oh-my-pi/pi-ai`. +1. Load built-in providers/models from `@oh-my-pi/pi-catalog` (`getBundledProviders` / `getBundledModels`). 2. Load `models.yml` custom config. 3. Apply provider overrides (`baseUrl`, `headers`, `disableStrictTools`) to built-in models. 4. Apply `modelOverrides` (per provider + model id). @@ -167,7 +169,7 @@ ModelRegistry pipeline (on refresh): ### Provider-model cache and static fingerprint Cached per-provider model lists are persisted in the model-cache SQLite -database (current schema version 5) with a `static_fingerprint` column that +database (current schema version 6) with a `static_fingerprint` column that hashes the static catalog slice merged into the row. When `resolveProviderModels` skips the network fetch and the fingerprint of the in-memory static catalog matches the cached one, the cached rows are returned verbatim — @@ -245,7 +247,7 @@ Provider defaults vs per-model overrides: - Provider `headers` are baseline. - Model `headers` override provider header keys. -- `modelOverrides` can override model metadata (`name`, `reasoning`, `thinking`, `input`, `cost`, `premiumMultiplier`, `contextWindow`, `maxTokens`, `headers`, `compat`, `contextPromotionTarget`). +- `modelOverrides` can override model metadata (`name`, `reasoning`, `thinking`, `input`, `supportsTools`, `cost`, `premiumMultiplier`, `contextWindow`, `maxTokens`, `omitMaxOutputTokens`, `headers`, `compat`, `contextPromotionTarget`). - `compat` is deep-merged for nested routing blocks (`openRouterRouting`, `vercelGatewayRouting`, `extraBody`). ## Runtime discovery integration @@ -537,6 +539,7 @@ Request shaping: - `supportsUsageInStreaming` — send `stream_options: { include_usage: true }` to receive token usage on streaming responses. Default: `true`. - `maxTokensField` — `"max_completion_tokens"` or `"max_tokens"`. Default: auto. - `supportsToolChoice` — emit the `tool_choice` parameter when the caller forces a specific tool. Default: `true`. Set `false` for endpoints that 400 on `tool_choice` (e.g. DeepSeek when reasoning is on). +- `supportsForcedToolChoice` — accept a forced `tool_choice` that requires a specific tool. Default: `true`. When `false`, a forced selector is downgraded to `auto` so the tool stays available for endpoints that reject forced tool calls (e.g. some thinking-required OpenAI-compatible models). - `disableReasoningOnForcedToolChoice` — drop `reasoning_effort` / OpenRouter `reasoning` whenever `tool_choice` forces a call. Default: auto (Kimi/Anthropic-fronted endpoints). - `disableReasoningOnToolChoice` — drop reasoning fields whenever any `tool_choice` is sent. Default: auto (DeepSeek reasoning models). - `alwaysSendMaxTokens` — always send a max-token field when the caller did not provide one. Default: auto (Kimi-family models derive TPM limits from `max_tokens`). @@ -576,7 +579,7 @@ Provider-level `compat` is the baseline; per-model `compat` is deep-merged on to ### Anthropic compatibility (`anthropic-messages`) -For `anthropic-messages` models the runtime uses a separate `AnthropicCompat` shape (`packages/catalog/src/types.ts`). The `models.yml` schema exposes the strict-tools opt-out as a top-level provider field (see below) plus two Anthropic-side flags in the same `compat` slot — `requiresToolResultId` (non-standard `id` alias on `tool_result` blocks for Z.AI-style proxies) and `replayUnsignedThinking` (replay unsigned thinking blocks as native thinking instead of demoting them to text); the remaining Anthropic-side knobs (`disableAdaptiveThinking`, `supportsEagerToolInputStreaming`, `supportsLongCacheRetention`, `supportsMidConversationSystem`, `supportsForcedToolChoice`, `supportsSamplingParams`) are set by built-in catalog metadata and are not user-configurable from `models.yml`. +For `anthropic-messages` models the runtime uses a separate `AnthropicCompat` shape (`packages/catalog/src/types.ts`). The `models.yml` schema exposes the strict-tools opt-out as a top-level provider field (see below) plus two Anthropic-side flags in the same `compat` slot — `requiresToolResultId` (non-standard `id` alias on `tool_result` blocks for Z.AI-style proxies) and `replayUnsignedThinking` (replay unsigned thinking blocks as native thinking instead of demoting them to text); the remaining Anthropic-side knobs (`disableAdaptiveThinking`, `supportsEagerToolInputStreaming`, `supportsLongCacheRetention`, `supportsMidConversationSystem`, `supportsForcedToolChoice`, `supportsSamplingParams`, `escapeBuiltinToolNames`) are set by built-in catalog metadata and are not user-configurable from `models.yml`. ### Strict tool schemas (`disableStrictTools`) diff --git a/docs/natives-addon-loader-runtime.md b/docs/natives-addon-loader-runtime.md index 2e4ee100f..ce2451c3f 100644 --- a/docs/natives-addon-loader-runtime.md +++ b/docs/natives-addon-loader-runtime.md @@ -20,7 +20,7 @@ The loader is intentionally narrow: - On Windows `node_modules` installs, stage addon files into the versioned cache to avoid locked-DLL update failures. - Attempt candidates in deterministic order and return the first addon that `require(...)` loads and validates. -For install and compiled-binary paths, the loader verifies a release sentinel export named from `package.json#version` (for example `__piNativesV15_7_2`). Workspace-dev loads skip this validation so a local checkout can rebuild after a pull. The loader does not validate the full export surface; stale same-version or incomplete binaries still surface as missing members or native errors at use sites. +For install and compiled-binary paths, the loader verifies a release sentinel export named from `package.json#version` (for example `__piNativesV16_0_3`). Workspace-dev loads skip this validation so a local checkout can rebuild after a pull. The loader does not validate the full export surface; stale same-version or incomplete binaries still surface as missing members or native errors at use sites. ## Runtime inputs and derived state diff --git a/docs/natives-architecture.md b/docs/natives-architecture.md index 681671740..1121277ec 100644 --- a/docs/natives-architecture.md +++ b/docs/natives-architecture.md @@ -38,7 +38,7 @@ Current capability groups in the generated API include: ## Loader layer -`packages/natives/native/index.js` owns runtime addon selection and optional embedded extraction. +`packages/natives/native/index.js` is the package entrypoint; it calls `loadNative()` from `loader-state.js`, which owns runtime addon selection and optional embedded extraction. ### Candidate resolution model @@ -166,6 +166,6 @@ N-API exports are generated from Rust `#[napi]` functions/classes/objects/enums. - **Platform leaf package**: Per-platform npm package `@oh-my-pi/pi-natives-` that carries one platform's prebuilt `.node`. The core depends on every leaf via `optionalDependencies`; the package manager installs only the host-matching one (`os`/`cpu`). - **Variant**: x64 CPU-specific build flavor (`modern` AVX2, `baseline` fallback). - **Generated binding declaration**: `native/index.d.ts` emitted by napi-rs during `build-native.ts`. -- **Version sentinel**: Rust export named from the package version (for example `__piNativesV15_7_2`) that lets the loader reject a `.node` from a different release. +- **Version sentinel**: Rust export named from the package version (for example `__piNativesV16_0_3`) that lets the loader reject a `.node` from a different release. - **Compiled binary mode**: Runtime mode where the CLI is bundled and native addons are resolved from embedded/cache paths before package-local paths. - **Embedded addon**: Build artifact metadata and archive reference generated into `native/embedded-addon.js` so compiled binaries can extract matching `.node` payloads. diff --git a/docs/natives-binding-contract.md b/docs/natives-binding-contract.md index bc608b660..e15718851 100644 --- a/docs/natives-binding-contract.md +++ b/docs/natives-binding-contract.md @@ -62,9 +62,10 @@ Consumers in `packages/coding-agent` and `packages/tui` import directly from `@o | Fuzzy path search | `fuzzyFind(options)` | `fd.rs` | `Promise` | | Glob/workspace | `glob(options, onMatch?)`, `listWorkspace(options)` | `glob.rs`, `workspace.rs` | `Promise<...>` | | Glob cache | `invalidateFsScanCache(path?)` | `fs_cache.rs` | `void` | -| AST/block/summary | `astGrep(options)`, `astEdit(options)`, `blockRangeAt(options)`, `summarizeCode(options)` | `ast.rs`, `block.rs`, `summary.rs` | mixed | +| AST/block/summary | `astGrep(options)`, `astMatch(options)`, `astEdit(options)`, `blockRangeAt(options)`, `enclosingBlockBoundaries(options)`, `summarizeCode(options)` | `ast.rs`, `block.rs`, `summary.rs` | mixed | | Shell | `executeShell(options, onChunk?)` | `shell.rs` | `Promise` | | Shell | `new Shell(options?)`, `shell.run(...)`, `shell.abort()` | `shell.rs` | class / promises | +| Shell | `applyBashFixups(command)` | `shell.rs` | `BashFixupResult` | | PTY | `new PtySession()`, `start/write/resize/kill` | `pty.rs` | class / promises | | Process | `Process.fromPid/fromPath`, `status/children/killTree/terminate/waitForExit` | `ps.rs` | class / mixed | | Keys | `parseKey`, `matchesKey`, Kitty/legacy helpers | `keys.rs` | sync | @@ -72,6 +73,7 @@ Consumers in `packages/coding-agent` and `packages/tui` import directly from `@o | Highlight | `highlightCode`, `supportsLanguage`, `getSupportedLanguages` | `highlight.rs` | sync | | HTML | `htmlToMarkdown(html, options?)` | `html.rs` | `Promise` | | SIXEL | `encodeSixel` | `sixel.rs` | sync | +| Snapcompact | `renderSnapcompactPng(text, options)` | `snapcompact.rs` | sync | | Clipboard | `copyToClipboard`, `readImageFromClipboard` | `clipboard.rs` | sync / promise | | Tokens | `countTokens(input, encoding?)` | `tokens.rs` | sync | | System/isolation | `detectMacOSAppearance`, `MacAppearanceObserver`, `MacOSPowerAssertion`, `getWorkProfile`, `iso*` helpers | `appearance.rs`, `power.rs`, `prof.rs`, `iso.rs` | mixed | @@ -80,7 +82,7 @@ Consumers in `packages/coding-agent` and `packages/tui` import directly from `@o The contract preserves Rust/N-API call style: -- **Promise-returning exports** for worker-thread or async runtime work (`grep`, `glob`, `fuzzyFind`, `astGrep`, `astEdit`, `htmlToMarkdown`, shell/PTY runs, `isoStart`/`isoStop`/`isoDiff`, clipboard image read, workspace scan). +- **Promise-returning exports** for worker-thread or async runtime work (`grep`, `glob`, `fuzzyFind`, `astGrep`, `astMatch`, `astEdit`, `htmlToMarkdown`, shell/PTY runs, `isoStart`/`isoStop`/`isoDiff`, clipboard image read, workspace scan). - **Synchronous exports** for deterministic in-memory transforms/parsers or direct system calls (`search`, `hasMatch`, highlighting, text utilities, token counting, process construction/status, `copyToClipboard`, `encodeSixel`, isolation probe/resolve helpers). - **Constructor exports** for stateful runtime objects (`Shell`, `PtySession`, `Process`, macOS observer/power handles). diff --git a/docs/natives-build-release-debugging.md b/docs/natives-build-release-debugging.md index 471c596e8..cf8f547f4 100644 --- a/docs/natives-build-release-debugging.md +++ b/docs/natives-build-release-debugging.md @@ -158,7 +158,7 @@ In compiled mode (`PI_COMPILED`, Bun embedded URL markers, or populated embedded - package/executable directories. 4. First successfully loaded addon with the expected version sentinel is returned. -This is why packaging + runtime loader expectations must align: filenames, platform tags, CPU variants, and embedded manifest version must match what `native/index.js` probes. +This is why packaging + runtime loader expectations must align: filenames, platform tags, CPU variants, and embedded manifest version must match what `native/loader-state.js` probes. ## JS API ↔ Rust export mapping (build sanity subset) @@ -185,7 +185,7 @@ Generated declarations currently include exports from these Rust modules: - Install failure: explicit message; Windows includes locked-file hint. - Generated binding install failure: explicit source/destination message. -## Runtime loader failures (`native/index.js`) +## Runtime loader failures (`native/loader-state.js`) - Unsupported platform tag: throws with supported platform list after probing fails. - No candidate could load: throws with full candidate error list and mode-specific remediation hints. diff --git a/docs/natives-media-system-utils.md b/docs/natives-media-system-utils.md index a351a8105..e1604863f 100644 --- a/docs/natives-media-system-utils.md +++ b/docs/natives-media-system-utils.md @@ -25,7 +25,7 @@ There is no native `PhotonImage` class, `image.rs`, or ProjFS overlay helper mod | `copyToClipboard(text)` | `copy_to_clipboard` | `clipboard.rs` | | `readImageFromClipboard()` | `read_image_from_clipboard` | `clipboard.rs` | | `countTokens(input, encoding?)` | `count_tokens` | `tokens.rs` | -| `detectMacOSAppearance()` | `detect_mac_os_appearance` | `appearance.rs` | +| `detectMacOSAppearance()` | `detect_macos_appearance` | `appearance.rs` | | `MacAppearanceObserver.start(cb)` | `MacAppearanceObserver::start` | `appearance.rs` | | `MacOSPowerAssertion.start(options?)` | `MacOSPowerAssertion::start` | `power.rs` | | `getWorkProfile(lastSeconds)` | `get_work_profile` | `prof.rs` | diff --git a/docs/natives-rust-task-cancellation.md b/docs/natives-rust-task-cancellation.md index e69712470..0a210c534 100644 --- a/docs/natives-rust-task-cancellation.md +++ b/docs/natives-rust-task-cancellation.md @@ -9,6 +9,7 @@ This document describes how `crates/pi-natives` schedules native work and how ca - `crates/pi-natives/src/glob.rs` - `crates/pi-natives/src/fd.rs` - `crates/pi-natives/src/ast.rs` +- `crates/pi-natives/src/workspace.rs` - `crates/pi-natives/src/shell.rs` - `crates/pi-natives/src/pty.rs` - `crates/pi-natives/src/html.rs` @@ -37,7 +38,7 @@ This document describes how `crates/pi-natives` schedules native work and how ca - `CancelToken::new(timeout_ms, signal)` combines an optional deadline and optional JS `AbortSignal` converted from `Unknown`. - `CancelToken::heartbeat()` is cooperative cancellation for blocking loops. - `CancelToken::wait()` asynchronously waits for signal or timeout. - - `CancelToken::emplace_abort_token()` creates an abortable flag when `AbortSignal`, `Shell.abort()`, or an internal bridge needs one. + - `CancelToken::emplace_abort_token()` lazily installs the shared abort flag (when the token has none) and returns an `AbortToken`; `CancelToken::new` uses it to bridge a JS `AbortSignal` to `AbortReason::Signal`. - `AbortToken::abort(reason)` lets external code request abort. ## `blocking` vs `future`: execution model and selection @@ -77,7 +78,8 @@ Behavior: | `grep(options, onMatch?)` | `grep` | `task::blocking("grep", ct, ...)` | `CancelToken::new(options.timeoutMs, options.signal)` + heartbeat checks | | `glob(options, onMatch?)` | `glob` | `task::blocking("glob", ct, ...)` | `CancelToken::new(...)` + heartbeat checks | | `fuzzyFind(options)` | `fuzzy_find` | `task::blocking("fuzzy_find", ct, ...)` | `CancelToken::new(...)` + heartbeat checks | -| `astGrep(options)` / `astEdit(options)` | ast exports | blocking worker path | timeout/signal fields are accepted by options and checked cooperatively in worker loops | +| `astGrep(options)` / `astMatch(options)` / `astEdit(options)` | ast exports | blocking worker path | timeout/signal fields are accepted by options and checked cooperatively in worker loops | +| `listWorkspace(options)` | `list_workspace` | `task::blocking("listWorkspace", ct, ...)` | `CancelToken::new(options.timeoutMs, options.signal)` + heartbeat checks | | `Shell#run(options, onChunk?)` | `Shell::run` | `task::future(env, "shell.run", ...)` | JS `CancelToken` is converted into `pi_shell::cancel::CancelToken`; shell races it against command completion and descendant cleanup | | `executeShell(options, onChunk?)` | `execute_shell` | `task::future(env, "shell.execute", ...)` | same cancel race and 2s graceful window | | `PtySession#start(options, onChunk?)` | `PtySession::start` | `task::future(env, "pty.start", ...)` + inner `spawn_blocking` | `CancelToken` checked in sync PTY loop via `heartbeat()` | @@ -128,6 +130,7 @@ Observed patterns: - `fd` scoring checks scanned candidates. - `grep` checks before/during expensive search and passes tokens into shared scan/cache helpers. - `run_pty_sync` checks every loop tick with a maximum 16ms wait cadence. +- `listWorkspace` checks before the parallel walk and per directory visit during traversal. Practical rule: no loop over external-size input should exceed a short bounded interval without a heartbeat. diff --git a/docs/natives-text-search-pipeline.md b/docs/natives-text-search-pipeline.md index ec0b3b760..e59804128 100644 --- a/docs/natives-text-search-pipeline.md +++ b/docs/natives-text-search-pipeline.md @@ -32,6 +32,7 @@ Terminology follows `docs/natives-architecture.md`: | `glob(options, onMatch?)` | `glob` | `glob.rs` | | `invalidateFsScanCache(path?)` | `invalidateFsScanCache` | `fs_cache.rs` | | `astGrep(options)` | `astGrep` | `ast.rs` | +| `astMatch(options)` | `astMatch` | `ast.rs` | | `astEdit(options)` | `astEdit` | `ast.rs` | | `wrapTextWithAnsi(text, width, tabWidth)` | `wrapTextWithAnsi` | `text.rs` | | `truncateToWidth(text, maxWidth, ellipsis, pad, tabWidth)` | `truncateToWidth` | `text.rs` | @@ -100,6 +101,7 @@ Terminology follows `docs/natives-architecture.md`: - Invalid repetition-like braces are escaped (`{`/`}` -> `\{`/`\}`) when they cannot form `{N}`, `{N,}`, `{N,M}`. - This prevents common literal-template fragments (for example `${platform}`) from failing as malformed repetition. +- After brace sanitization, a compile error reporting an unclosed/unopened group triggers one retry with unescaped parentheses escaped, so literal snippets like `fetchAnthropicProvider(` still search instead of erroring. - Remaining invalid regex syntax still returns a regex error. ## 2) File discovery (`glob`) and fuzzy path search (`fuzzyFind`) @@ -144,11 +146,12 @@ Terminology follows `docs/natives-architecture.md`: - auto-prefixes simple recursive patterns with `**/` when `recursive=true`, - auto-closes unbalanced `{...` alternation groups before compile. -## 3) AST search/edit (`astGrep`, `astEdit`) +## 3) AST search/match/edit (`astGrep`, `astMatch`, `astEdit`) `ast.rs` exposes syntax-aware code search and rewrite operations. - `astGrep(options)` returns matches with byte/line/column coordinates and optional metavariable bindings. +- `astMatch(options)` runs the same patterns against an in-memory `source` string instead of files; `lang` is required (there is no path to infer it from), and the result keeps matches, `totalMatches`, `limitReached`, and parse errors but omits the file-count fields. - `astEdit(options)` returns replacement changes, per-file counts, searched/touched file counts, parse errors, and whether edits were applied. - `dryRun` defaults to true for edit options in the generated documentation. - Options include language override, path/glob/selector, strictness, limits, parse-error policy, `signal`, and `timeoutMs`. @@ -248,6 +251,7 @@ Text functions generally return deterministic transformed output; errors are lim | `text` module functions | No | No | ANSI/width utilities only | | `highlight` module functions | No | No | syntax + ANSI coloring only | | `countTokens` | No | No | tokenization only | +| `astMatch` | No | No | in-memory syntax-aware match (no disk) | | `astGrep` / `astEdit` | Yes | No | syntax-aware file search/edit | | `glob` | Yes | Optional | directory scans + glob filtering | | `fuzzyFind` | Yes | Optional | directory scans + fuzzy scoring | diff --git a/docs/non-compaction-retry-policy.md b/docs/non-compaction-retry-policy.md index 8e405b720..31511261c 100644 --- a/docs/non-compaction-retry-policy.md +++ b/docs/non-compaction-retry-policy.md @@ -9,6 +9,7 @@ It explicitly excludes context-overflow recovery via auto-compaction. Overflow i - [`../src/session/agent-session.ts`](../packages/coding-agent/src/session/agent-session.ts) - [`../src/config/settings-schema.ts`](../packages/coding-agent/src/config/settings-schema.ts) - [`../src/modes/controllers/event-controller.ts`](../packages/coding-agent/src/modes/controllers/event-controller.ts) +- [`../src/modes/controllers/input-controller.ts`](../packages/coding-agent/src/modes/controllers/input-controller.ts) - [`../src/modes/rpc/rpc-mode.ts`](../packages/coding-agent/src/modes/rpc/rpc-mode.ts) - [`../src/modes/rpc/rpc-client.ts`](../packages/coding-agent/src/modes/rpc/rpc-client.ts) - [`../src/modes/rpc/rpc-types.ts`](../packages/coding-agent/src/modes/rpc/rpc-types.ts) @@ -37,6 +38,8 @@ So: overload/rate/server/network-style failures use this retry policy; context-w - the error is a stale OpenAI Responses replay failure (`Item with id '…' not found`, or an invalid/expired/not-found `previous_response`) - `errorMessage` matches transient transport/envelope patterns or `isUsageLimitError(...)` +The stale-replay and transient/usage-limit branches additionally require that the stream was **not** interrupted after already emitting observable output. `#streamInterruptedAfterObservableOutput(...)` treats a `STREAM_INTERRUPTED_AFTER_CONTENT` stop detail — or any tool call, non-empty text, thinking, or redacted-thinking block — as non-retryable, so a partially produced turn is not silently replayed. Classifier refusals are checked first and bypass this exclusion. + Current retryable inputs are regex/string-classified: - transient transport/envelope failures, including Anthropic stream-envelope failures before `message_start` @@ -49,6 +52,8 @@ Current retryable inputs are regex/string-classified: Transport classification is regex text matching, not typed provider error codes; classifier refusals are the exception, detected from the typed `stopDetails` field. +Beyond `#isRetryableError(...)`, a narrower trigger feeds the same retry engine: `#isRetryableReasonlessAbort(...)` routes a content-less `aborted` stop carrying the generic abort sentinel (`GENERIC_ABORT_SENTINEL`) — only when no user, dispose, or streaming-edit-guard abort is in progress — into `#handleRetryableError(message, { allowModelFallback: false })`, i.e. retried without model fallback. + ## Retry lifecycle and state transitions Session state used by retry: @@ -133,12 +138,14 @@ If abort hits while sleeping, catch path emits: ### TUI interaction -On `auto_retry_start`, EventController: +On `auto_retry_start`, EventController (`#handleAutoRetryStart`): -- swaps `Esc` handler to `session.abortRetry()` -- renders loader text: `Retrying (attempt/maxAttempts) in Ns… (esc to cancel)` +- stops the working loader and clears the status container +- renders a `retryLoader` with text: `Retrying (attempt/maxAttempts) in Ns… (esc to cancel)` -On `auto_retry_end`, it restores prior `Esc` handler and clears loader state. +`Esc` cancellation dispatches on live session state rather than a swapped handler: the input controller checks `viewSession.isRetrying` and calls `viewSession.abortRetry()` (alongside its compaction/handoff abort checks). + +On `auto_retry_end` (`#handleAutoRetryEnd`), it stops and clears the `retryLoader` and status container. ## Streaming and prompt completion behavior diff --git a/docs/plugin-manager-installer-plumbing.md b/docs/plugin-manager-installer-plumbing.md index b87651d94..ad696bba8 100644 --- a/docs/plugin-manager-installer-plumbing.md +++ b/docs/plugin-manager-installer-plumbing.md @@ -108,7 +108,8 @@ Malformed `package.json` JSON is a hard failure at read time; malformed manifest - `[a,b]`: validates each feature exists in manifest features map - `[]`: empty feature list - bare spec: `null` (use defaults policy later in loader) -7. Upsert lockfile runtime state: `{ version, enabledFeatures, enabled: true }`. +7. Validate declared extension entries (`#validateInstalledExtensions`): each manifest `extensions` entry must resolve on disk and import to a factory function. On failure, roll back the install — restore the previous `plugins/package.json`, remove the freshly installed package, and restore any prior version from a backup taken before `bun install` — then abort. +8. Upsert lockfile runtime state: `{ version, enabledFeatures, enabled: true }`. ### Update semantics @@ -220,9 +221,9 @@ No cross-process locking or merge strategy exists; concurrent writers can overwr Active manager path enforces package-name validation: -- npm specs: regex for scoped/unscoped package specs (optionally with version) -- shell metacharacter denylist: `;`, `&`, `|`, backtick, `$`, `(`, `)`, `{`, `}`, `<`, `>`, `\`, newline, CR, tab (`[`/`]` are allowed for feature brackets) -- git specs: `validateGitSpec` (permits `:`, `/`, `#`, `+`, `.`, `-`, `_`) instead of the npm regex +- npm specs: a package-name regex (`VALID_PACKAGE_NAME`) for scoped/unscoped specs, optionally with version. +- npm shell-metacharacter denylist: `;`, `&`, `|`, backtick, `$`, `(`, `)`, `{`, `}`, `[`, `]`, `<`, `>`, `\` — applied after `parsePluginSpec` strips the feature brackets, so a normal `pkg[feat]` spec never reaches it. +- git specs: `validateGitSpec` rejects only the shared `SHELL_METACHARS` set (`;`, `&`, `|`, backtick, `$`, `(`, `)`, `{`, `}`, `<`, `>`, `\`, newline, CR, tab) instead of the npm regex, so `:`, `/`, `#`, `+`, `.`, `-`, `_`, `~`, `@` are permitted. This limits command-injection risk when invoking `bun install/uninstall`. @@ -248,7 +249,8 @@ The plugin manager is not transactional. | Operation stage | Failure behavior | Rollback | | -------------------------------------------------------- | -------------------------- | ----------------------------------------------------------------------------- | | `bun install` fails | install aborts with stderr | N/A (no state writes yet) | -| Install succeeds, then manifest/feature validation fails | command fails | No uninstall rollback; dependency may remain in `node_modules`/`package.json` | +| Install succeeds, then feature validation fails | command fails | No uninstall rollback; dependency may remain in `node_modules`/`package.json` | +| Install succeeds, then extension validation fails | command fails | Rolls back: restores `package.json`, removes installed package, restores prior version from backup | | Install succeeds, then lockfile write fails | command fails | No rollback of installed package | | `bun uninstall` succeeds, lockfile write fails | command fails | Package removed, stale runtime state may remain | | `link` removes old target then symlink creation fails | command fails | No restoration of previous link/dir | @@ -266,7 +268,7 @@ Operationally, `doctor --fix` can repair some drift (`bun install`, orphaned con ## Mode differences and precedence -- `--dry-run` (install): returns synthetic install result, no filesystem/network/state writes. +- `--dry-run` (install): returns a synthetic install result with no `bun install`, no network, and no lockfile/runtime-state writes (it still ensures the plugins `package.json` skeleton exists). - `--json`: output formatting only, no behavior change. - Project overrides always take precedence over global lockfile for feature/settings view. - Effective enablement is `runtimeEnabled && !projectDisabled`. diff --git a/docs/porting-from-pi-mono.md b/docs/porting-from-pi-mono.md index a7ac8dbab..4119d043e 100644 --- a/docs/porting-from-pi-mono.md +++ b/docs/porting-from-pi-mono.md @@ -306,7 +306,7 @@ Our fork has architectural decisions that differ from upstream. **Do not port th | ------------------------------------------- | --------------------------------------------------------- | --------------------------------------------------------------------- | | `FooterDataProvider` class | `StatusLineComponent` | Simpler, integrated status line | | `ctx.ui.setHeader()` / `ctx.ui.setFooter()` | No-op stubs in current extension contexts | Not currently wired to replace the TUI status/header UI | -| `ctx.ui.setEditorComponent()` | No-op stubs in current extension contexts | Custom editor replacement is not currently wired | +| `ctx.ui.setEditorComponent()` | Wired in interactive mode; no-op stubs in ACP/RPC/headless contexts | Custom editor replacement works in the interactive TUI; non-TUI runtimes keep stubs | | `InteractiveModeOptions` options object | Positional constructor args (options type still exported) | Keep constructor signature; update the type when upstream adds fields | ### Component Naming diff --git a/docs/provider-streaming-internals.md b/docs/provider-streaming-internals.md index 17ee6f169..7659030c0 100644 --- a/docs/provider-streaming-internals.md +++ b/docs/provider-streaming-internals.md @@ -131,15 +131,15 @@ If provider stream throws or signals failure, each provider wrapper catches and - `stopReason = "aborted"` when abort signal is set - otherwise `stopReason = "error"` -- `errorMessage = formatErrorMessageWithRetryAfter(error)` +- `errorMessage = finalizeErrorMessage(error, rawRequestDump)` (`packages/ai/src/utils/http-inspector.ts`), which wraps `formatErrorMessageWithRetryAfter()` and appends any captured HTTP-error body / raw-request dump (the `cursor` wrapper calls `formatErrorMessageWithRetryAfter()` directly) ## Malformed chunk / SSE parse failure behavior -The OpenAI Completions/Responses paths delegate chunk/SSE framing to the `openai` SDK stream. Anthropic uses the in-repo `AnthropicMessagesClient` (`packages/ai/src/providers/anthropic-client.ts`); the Google paths and the Codex SSE fallback read SSE via `readSseJson()` directly, and websocket Codex frames are normalized through the same event handler. +The OpenAI Completions/Responses paths use the in-repo HTTP+SSE transport `postOpenAIStream()` (`packages/ai/src/utils/openai-http.ts`), which decodes frames with `readSseJson()` and replaced the `openai` SDK client. Anthropic uses the in-repo `AnthropicMessagesClient` (`packages/ai/src/providers/anthropic-client.ts`); the Google paths and the Codex SSE fallback read SSE via `readSseJson()` directly, and websocket Codex frames are normalized through the same event handler. Observed behavior in current implementation: -- malformed SDK stream parsing surfaces as an exception or stream `error` event +- malformed SSE framing or chunk JSON surfaces as an exception or stream `error` event - malformed Codex SSE JSON/framing throws from the local SSE reader - provider wrapper converts failures into unified terminal `error` events - no provider-specific resume/retry inside the stream function itself, except Codex websocket-to-SSE transport fallback before replay-unsafe output is emitted diff --git a/docs/providers.md b/docs/providers.md index 85c67c5ea..d37ea6081 100644 --- a/docs/providers.md +++ b/docs/providers.md @@ -90,7 +90,7 @@ Each provider has one or more environment variables that supply a key when no st | `xai-oauth` | `XAI_OAUTH_TOKEN`, then `XAI_API_KEY` | | `github-copilot` | `COPILOT_GITHUB_TOKEN` | | `cursor` | `CURSOR_ACCESS_TOKEN` | -| `azure-openai-responses` | `AZURE_OPENAI_API_KEY` | +| `azure` | `AZURE_OPENAI_API_KEY` | | `amazon-bedrock` | `AWS_PROFILE`, or `AWS_ACCESS_KEY_ID` + `AWS_SECRET_ACCESS_KEY`, or an ECS/IRSA credential chain | ### Additional hosted providers @@ -104,7 +104,6 @@ Each provider has one or more environment variables that supply a key when no st | `nvidia` | `NVIDIA_API_KEY` | | `huggingface` | `HUGGINGFACE_HUB_TOKEN`, then `HF_TOKEN` | | `moonshot` | `MOONSHOT_API_KEY` | -| `kimi-code` | `KIMI_API_KEY` | | `nanogpt` | `NANO_GPT_API_KEY` | | `venice` | `VENICE_API_KEY` | | `vercel-ai-gateway` | `AI_GATEWAY_API_KEY` (also `VERCEL_AI_GATEWAY_API_KEY` for catalog discovery) | @@ -132,7 +131,7 @@ Each provider has one or more environment variables that supply a key when no st | `lm-studio` | `LM_STUDIO_API_KEY` (optional; keyless by default) | | `llama.cpp` | `LLAMA_CPP_API_KEY` (only when the server requires auth) | -OAuth-backed providers such as `anthropic`, `github-copilot`, `cursor`, `ollama-cloud`, `qwen-portal`, `xai-oauth`, `wafer-pass`, `wafer-serverless`, `google-gemini-cli`, and `google-antigravity` are normally reached through `/login` rather than an environment variable. See [Environment variables](./environment-variables.md) for search-tool and configuration variables not listed here. +OAuth-backed providers such as `anthropic`, `github-copilot`, `cursor`, `ollama-cloud`, `qwen-portal`, `kimi-code`, `xai-oauth`, `wafer-pass`, `wafer-serverless`, `google-gemini-cli`, and `google-antigravity` are normally reached through `/login` rather than an environment variable. See [Environment variables](./environment-variables.md) for search-tool and configuration variables not listed here. ### `.env` discovery and precedence diff --git a/docs/python-repl.md b/docs/python-repl.md index 5ce950b09..67582309b 100644 --- a/docs/python-repl.md +++ b/docs/python-repl.md @@ -160,7 +160,7 @@ The runner additionally receives `PYTHONUNBUFFERED=1` and `PYTHONIOENCODING=utf- If Python preflight fails and `eval.js` is enabled, `eval` remains available for `js` cells; `py` cells fail with a Python-backend availability error. -Python prelude helpers include `agent(prompt, *, agent_type="task", model=None, label=None, schema=None)`. It synchronously calls the host bridge, runs one subagent through the task executor, and returns the final text. When `schema` is supplied, the helper parses the subagent's JSON output and returns the object. +Python prelude helpers include `agent(prompt, *, agent_type="task", model=None, label=None, schema=None, return_handle=False)`. It synchronously calls the host bridge, runs one subagent through the task executor, and returns the final text. When `schema` is supplied, the helper parses the subagent's JSON output and returns the object. When `return_handle=True`, it instead returns a DAG node dict (`{"text", "output", "handle", "id", "agent"}`) whose `handle` is the spawned agent's recoverable `agent://` URI (the parsed object lands under `"data"` when `schema` is also set), so a downstream `pipeline`/`parallel` stage can reference the transcript by handle instead of re-inlining it. ## Execution flow and cancellation/timeout @@ -218,7 +218,7 @@ Output is streamed through `OutputSink` and may be persisted to artifact storage ### Renderer behavior -- Tool renderer (`eval.ts`): +- Tool renderer (`eval-render.ts`, re-exported from `eval.ts`): - shows code-cell blocks with per-cell status - collapsed preview defaults to 10 lines - supports expanded mode for all output retained in the tool result diff --git a/docs/resolve-tool-runtime.md b/docs/resolve-tool-runtime.md index 1a3f1c423..298dc9953 100644 --- a/docs/resolve-tool-runtime.md +++ b/docs/resolve-tool-runtime.md @@ -47,7 +47,7 @@ Multiple pending previews therefore follow the active tool-choice queue ordering - `sourceToolName` (`ast_edit`) - `apply(reason: string, extra?: Record)` callback that reruns AST edit with `dryRun: false` -`resolve(action="apply", reason="...")` passes `reason` into this callback. `ast_edit` currently ignores `extra`. +`resolve(action="apply", reason="...")` passes both `reason` and `extra` into this callback, but `ast_edit`'s apply ignores both — its parameter is `_reason`, and the rerun is independent of `reason`/`extra`. ## Custom tools: `pushPendingAction` diff --git a/docs/rpc.md b/docs/rpc.md index ed65e1c68..16131c629 100644 --- a/docs/rpc.md +++ b/docs/rpc.md @@ -23,7 +23,7 @@ Behavior notes: - `@file` CLI arguments are rejected in RPC mode. - RPC mode disables automatic session title generation by default to avoid an extra model call. -- RPC mode resets workflow-altering `todo.*`, `task.*`, `memory.backend`/`memories.enabled`, `async.*`, and `bash.autoBackground.*` settings to their built-in defaults instead of inheriting user overrides. +- RPC mode resets workflow-altering `todo.*`, `task.*`, `memory.backend`/`memories.enabled`, `advisor.*`, `async.*`, and `bash.autoBackground.*` settings to their built-in defaults instead of inheriting user overrides. - The process reads stdin as JSONL (`readJsonl(Bun.stdin.stream())`). - At startup it writes `{ "type": "ready" }` before processing commands. - When stdin closes, pending host-tool calls and host-URI requests are rejected and the process exits with code `0`. diff --git a/docs/rulebook-matching-pipeline.md b/docs/rulebook-matching-pipeline.md index a20326bc6..3bd66fae5 100644 --- a/docs/rulebook-matching-pipeline.md +++ b/docs/rulebook-matching-pipeline.md @@ -3,7 +3,7 @@ This document describes how coding-agent discovers rules from supported config formats, normalizes them into a single `Rule` shape, resolves precedence conflicts, and splits the result into: - **Rulebook rules** (available to the model via system prompt + `rule://` URLs) -- **TTSR rules** (time-travel stream interruption rules) +- **TTSR rules** (Time Traveling Stream Rules) It reflects the current implementation, including partial semantics and metadata that is parsed but not enforced. @@ -15,6 +15,7 @@ It reflects the current implementation, including partial semantics and metadata - [`packages/coding-agent/src/discovery/index.ts`](../packages/coding-agent/src/discovery/index.ts) - [`packages/coding-agent/src/discovery/helpers.ts`](../packages/coding-agent/src/discovery/helpers.ts) - [`packages/coding-agent/src/discovery/builtin.ts`](../packages/coding-agent/src/discovery/builtin.ts) +- [`packages/coding-agent/src/discovery/omp-plugins.ts`](../packages/coding-agent/src/discovery/omp-plugins.ts) - [`packages/coding-agent/src/discovery/builtin-defaults.ts`](../packages/coding-agent/src/discovery/builtin-defaults.ts) - [`packages/coding-agent/src/discovery/agents.ts`](../packages/coding-agent/src/discovery/agents.ts) - [`packages/coding-agent/src/discovery/cursor.ts`](../packages/coding-agent/src/discovery/cursor.ts) @@ -75,7 +76,7 @@ Normalization: - `name` = filename without `.md`/`.mdc` - frontmatter parsed via `parseFrontmatter` - `content` = body (frontmatter stripped) -- `globs`, `alwaysApply`, `description`, `condition`/legacy `ttsr_trigger`, `scope`, and `interruptMode` are parsed by `buildRuleFromMarkdown` +- `globs`, `alwaysApply`, `description`, `condition`/legacy `ttsr_trigger`, `astCondition`, `scope`, and `interruptMode` are parsed by `buildRuleFromMarkdown` - top-level `RULES.md` is synthesized as rule name `RULES` and forced to `alwaysApply: true` Important caveat: `condition` values that look like file globs are converted into `tool:edit(...)` / `tool:write(...)` scope shorthands with catch-all condition `.*`. @@ -87,7 +88,7 @@ Loads from both `.agent` and `.agents` directories: - project: walk upward from `cwd` to repo root, loading `/.agent/rules/*.{md,mdc}` and `/.agents/rules/*.{md,mdc}` - user: `~/.agent/rules/*.{md,mdc}` and `~/.agents/rules/*.{md,mdc}` -Normalization uses the shared `buildRuleFromMarkdown` path: filename-derived name, stripped frontmatter body, and parsed `globs`, `alwaysApply`, `description`, `condition`/legacy `ttsr_trigger`, `scope`, and `interruptMode`. +Normalization uses the shared `buildRuleFromMarkdown` path: filename-derived name, stripped frontmatter body, and parsed `globs`, `alwaysApply`, `description`, `condition`/legacy `ttsr_trigger`, `astCondition`, `scope`, and `interruptMode`. ### Cursor provider (`cursor.ts`) @@ -101,7 +102,7 @@ Normalization (`transformMDCRule`): - `description`: kept only if string - `alwaysApply`: normalized to a boolean — `true` only when frontmatter has `alwaysApply: true` (anything else becomes `false`) - `globs`: accepts array (string elements only) or single string -- `condition`/legacy `ttsr_trigger`, `scope`, and `interruptMode` are parsed by shared rule helpers +- `condition`/legacy `ttsr_trigger`, `astCondition`, `scope`, and `interruptMode` are parsed by shared rule helpers - `name` from filename without extension ### Windsurf provider (`windsurf.ts`) @@ -114,7 +115,7 @@ Loads from: Normalization: - `globs`: array-of-string or single string -- `alwaysApply`, `description`, `condition`/legacy `ttsr_trigger`, `scope`, and `interruptMode` parsed by shared rule helpers +- `alwaysApply`, `description`, `condition`/legacy `ttsr_trigger`, `astCondition`, `scope`, and `interruptMode` parsed by shared rule helpers - `name` is fixed to `global_rules` for the user global file and derived from filename for project rules ### Cline provider (`cline.ts`) @@ -127,7 +128,7 @@ Searches upward from `cwd` for nearest `.clinerules`: Normalization: - `globs`: array-of-string or single string -- `alwaysApply`, `description`, `condition`/legacy `ttsr_trigger`, `scope`, and `interruptMode` parsed by shared rule helpers +- `alwaysApply`, `description`, `condition`/legacy `ttsr_trigger`, `astCondition`, `scope`, and `interruptMode` parsed by shared rule helpers - `name` is fixed to `clinerules` for a `.clinerules` file and derived from filename for `.clinerules/*.md` ## 3. Frontmatter parsing behavior and ambiguity diff --git a/docs/secrets.md b/docs/secrets.md index aaa4c20e5..506269577 100644 --- a/docs/secrets.md +++ b/docs/secrets.md @@ -98,7 +98,7 @@ Regex entries always scan globally (the `g` flag is enforced automatically). The ## Interaction with env var detection -Environment variables are collected first, then file-defined entries are appended. File entries can cover secrets that don't live in env vars (config files, hardcoded values, etc.). If the same plain value appears in both env and file entries, the env entry's obfuscate-mode mapping is used first. +Environment variables are collected first, then file-defined entries are appended. File entries can cover secrets that don't live in env vars (config files, hardcoded values, etc.). Env and file entries are not deduplicated against each other, so a plain value present in both is registered twice; both placeholders restore to the same secret, so deobfuscation is unaffected. ## Key files diff --git a/docs/session-switching-and-recent-listing.md b/docs/session-switching-and-recent-listing.md index 21f38e1cd..24987c6c7 100644 --- a/docs/session-switching-and-recent-listing.md +++ b/docs/session-switching-and-recent-listing.md @@ -7,6 +7,8 @@ It focuses on current implementation behavior, including fallback paths and cave ## Implementation files - [`../src/session/session-manager.ts`](../packages/coding-agent/src/session/session-manager.ts) +- [`../src/session/session-listing.ts`](../packages/coding-agent/src/session/session-listing.ts) +- [`../src/session/session-paths.ts`](../packages/coding-agent/src/session/session-paths.ts) - [`../src/session/agent-session.ts`](../packages/coding-agent/src/session/agent-session.ts) - [`../src/cli/session-picker.ts`](../packages/coding-agent/src/cli/session-picker.ts) - [`../src/modes/components/session-selector.ts`](../packages/coding-agent/src/modes/components/session-selector.ts) @@ -33,7 +35,7 @@ There are two different listing pipelines: 1. `getRecentSessions(sessionDir, limit)` (welcome/summary view) - Reads only a 4KB prefix (`readTextSlices(..., 4096, 0)[0]`) from each file. - Parses header + earliest user text preview. - - Returns lightweight `RecentSessionInfo` with lazy `name` and `timeAgo` getters. + - Returns lightweight `RecentSessionInfo` (`path`, `name`, `timeAgo`); `name` and `timeAgo` are computed eagerly (`sessionDisplayName` / `formatTimeAgo`), not lazy getters. - Sorts by file `mtime` descending. 2. `SessionManager.list(...)` / `SessionManager.listAll()` (resume pickers and ID matching) @@ -46,9 +48,9 @@ There are two different listing pipelines: For recent summaries (`RecentSessionInfo`): -- display name preference: `header.title` -> first user prompt -> `header.id` -> filename -- name is truncated to 40 chars for compact displays -- control characters/newlines are stripped/sanitized from title-derived names +- display name preference (`sessionDisplayName`): `title` -> first user message -> an `Untitled ·