Why:
RpcFrameEncoder serializes normal frames once for protocol routing and again
for output, while v2 snapshots serialize message payloads separately.
Changes:
- Feed the existing JSON into frame-size enforcement.
- Build v2 message snapshots from the serialized frame.
Evidence:
- End-to-end paired medians improved 10.72% for v1 and 15.72% for v2
across 25 pairs, with identical output hashes.
Refs #8118
- Implemented in-house, zero-dependency utility modules in `pi-utils` covering DOM manipulation, markdown parsing, templating, browser automation helpers, and terminal buffers.
- Migrated packages across the repository to consume the new internal utilities and `omptype` schema validators instead of external dependencies.
- Removed multiple external runtime and development dependencies including Zod, Marked, LRU cache, Turndown, and Puppeteer browser packages.
AgentSession#setModelWithProviderSessionReset is the single choke point
every model mutation runs through (explicit /model, prewalk hand-offs,
retry-fallback, model cycling). It previously changed agent.state.model
silently — no session event told subscribers (ACP, RPC, TUI) that the
active model moved.
Emit a new model_changed AgentSessionEvent from that choke point
whenever the model actually changes, and wire it into every consumer
that must exhaustively handle AgentSessionEvent: the TUI event
controller (invalidates the status line, same as thinking_level_changed)
and the RPC client's forwarded-event allowlist.
(cherry picked from commit f76325de2c7821dd7046ddb67546577c3575a263)
Moved RPC dialog request lifecycle into a reusable helper that emits a cancel frame targeting the original request before settling an aborted local promise.
Added coverage for remote confirmation cancellation and pending-request cleanup.
Two fixes on top of the paged transport:
- Near-limit v2 framing no longer materializes the full base64 transport:
chunk lines are generated lazily from a single serialization, the 64 MiB
reassembly ceiling is enforced via Buffer.byteLength before any
full-payload allocation, and RPC stdout writes drain with backpressure
one physical line at a time. Peak RSS for a 63 MiB response drops
~686 MB -> ~521 MB; a rejected 80 MiB response drops ~507 MB -> ~259 MB
(parity with the v1 path).
- get_messages_page errors now carry a machine-readable code
(session_busy | stale_cursor). Both bundled clients' high-level
getMessages() drains discard partial pages and fall back to the legacy
snapshot on either code — previously a cursor invalidated by a
background mutation (e.g. an appended bash message) threw instead of
falling back. Direct page calls remain strict.
Three read paths raced background model discovery on cold start:
1. `get_available_models` RPC (rpc-mode.ts) read the registry
synchronously and returned a partial catalog containing only
statically-bundled models.
2. `set_model` RPC (rpc-mode.ts) read the registry synchronously and
rejected discovery-backed selectors with "Model not found".
3. `--model <provider>/<pattern>` CLI flag deferred retry (sdk.ts:2078)
resolved synchronously after extension registration, before
discovery-backed providers had populated `#models`.
Paths 1 and 2 are fixed by exposing the existing in-flight background
refresh promise (`#backgroundRefresh`, already tracked and cleared by
`refreshInBackground`) via a new public
`ModelRegistry.awaitBackgroundRefresh()` method, and awaiting it at each
RPC read site. No-op when no refresh is in flight (warm sessions
unaffected).
Path 3 mirrors the cold-cache race fix already applied to the
default-role fallback on this branch (issues #6114, #6162, sdk.ts:2343):
when a deferred pattern is unresolved and any discoverable provider is
registered, run a cache-aware `refresh("online-if-uncached")` pass
before the retry. Reuses the existing discovery machinery rather than
introducing a new ordering dependency.
The `omp models` CLI never had this bug because it awaits
`modelRegistry.refresh()` directly before listing.
Behavioral characteristics:
- **Warm-session fast path preserved**: when no refresh is in flight
(`#backgroundRefresh === undefined`), `await undefined` resolves in a
microtask. No regression for sessions that don't need discovery or
have already settled.
- **Failure isolation preserved**: `refreshInBackground()` already
swallows discovery errors via `.catch(...)`, so `awaitBackgroundRefresh()`
resolves even when discovery fails — callers then read whatever models
made it into `#models` (built-in + cached). No new failure modes.
- **Scoped**: doesn't change `refreshInBackground()` semantics. Adds a
new read-only awaiter with minimal API surface. Reuses the
well-established `refresh("online-if-uncached")` pattern for the
deferred retry path.
Reproduction (get_available_models RPC, with any discovery-backed
provider configured in `~/.omp/agent/models.yaml`):
cd ~
{
sleep 1
printf '%s\n' '{"id":"m1","type":"get_available_models"}'
sleep 5
} | timeout 15 omp --mode rpc-ui --approval-mode yolo 2>/dev/null \
| grep '"id":"m1"' | jq '.data.models | {count: length, providers: ([.[].provider]|unique)}'
Before: discovery-backed provider absent from the response on cold start.
After: discovery-backed provider present.
Reproduction (--model CLI flag, same config):
omp --mode rpc-ui --model <discovery-provider>/<model-id> --approval-mode yolo
Before: exits 1 with "Model \"<discovery-provider>/<model-id>\" not found".
After: starts rpc-ui session with the requested model selected.
Claimed Bun's singleton stdin reader before loading extensions and passed the owned stream into RPC and RPC-UI mode.
Added process-level regressions for both modes with a startup extension that attempts to lock stdin.
Fixes#5898
Native /login and /logout (plus setup-wizard sign-in and RPC login)
refreshed model discovery with the default all-provider
online-if-uncached strategy, which reused a fresh authoritative cache row
and never re-ran fetchDynamicModels with the just-persisted credential,
so newly authenticated models stayed unavailable in-session and stale
endpoint data survived a relogin. Each auth-completion path now awaits a
provider-scoped refreshProvider(providerId, "online").
Also fixed writeModelCache serializing credential-bearing request headers
(Authorization, X-Api-Key, api-key, cookie, proxy-authorization) into the
plaintext models.db; they are now stripped before persistence and
re-derived on load from AuthStorage / provider config.
Fixes#5780
Resolved against the newer RpcInputDispatcher loop: kept serial dispatch
and shutdown coordination, replaced only the readJsonl generator with
line-based reads and a per-line parse-error response frame.
Extension/SDK/RPC registerTool defaulted an omitted loadMode to
"discoverable". A UI-only re-register of an essential built-in
(read/write/bash/edit/glob) then became discoverable and, with tools.xdev
on, was unmounted from the top-level schema. read/write dropping also
broke the xd:// transport (read xd://, write xd://<tool>), leaving the
model with no callable coding essentials.
- Add defaultLoadModeForToolName: omitted loadMode resolves to "essential"
for known essential built-in names, "discoverable" otherwise.
- Apply it at all four adapter boundaries (extension wrapper, custom-tools
wrapper, sdk customToolToDefinition, rpc normalizeHostToolDefinitions).
- Transport invariant: read/write never mount under xdev regardless of
loadMode (they carry the transport).
- Regression test covering the demotion, transport invariant, and a drift
guard tying the essential-name set to the tool classes.
Fixes#5764
Print-mode assistant-error/aborted exit, RPC pi.shutdown() and stdin-EOF
shutdowns, and the extension command-context shutdown() called
process.exit() before (or racing) session.dispose(), skipping the bounded
browser reaper (releaseTabsForOwner) installed in dispose(). An OMP-owned
Chromium could survive the parent and reparent to PID 1.
Route all four graceful paths through the idempotent, promise-memoized
session.dispose() and await it before the final exit. The RPC
performShutdown no longer emits session_shutdown directly (dispose() emits
it), avoiding a double emit.
Fixes#5643
- Added the `xd://` virtual device protocol (`internal-urls/xd-protocol.ts`, `tools/xdev.ts`): tools declaring `loadMode: "discoverable"` are unmounted from the request tools array and driven via `read xd://` (list/docs+schema) and `write xd://<tool>` (execute), gated by the `tools.xdev` setting (default on) and inlined into the system prompt.
- Merged the `irc`, `job`, and `launch` tools into a single `hub` tool (`tools/hub/`, `async/job-manager.ts`): messaging keeps `send`/`inbox`/`list`, job control maps to `wait`/`cancel`/`jobs`, process supervision keeps `start`/`logs`/`stop`/`restart`/`describe` with `ps`, and the unified `wait` races background jobs against peer messages; SDK `IrcTool`/`JobTool`/`LaunchTool` are replaced by `HubTool`.
- Removed the hidden `resolve` tool in favor of the `xd://resolve`/`xd://reject`/`xd://propose` resolution devices, auto-including `write` whenever a deferrable tool or plan mode is present.
- Removed the BM25 tool-discovery system: the `search_tool_bm25` tool, the `tool-discovery` module, the `tools.discoveryMode`/`mcp.discoveryMode`/`mcp.discoveryDefaultServers`/`tools.essentialOverride` settings, per-tool MCP selection, and the `mcp_tool_selection` message type.
- Unified tool presentation on `ToolLoadMode` (`essential`|`discoverable`), replacing the custom-tool `xdev?: boolean` opt-out; custom, extension, MCP, RPC host, image-generation, and TTS tools now default to `discoverable`, and added a `satisfies` predicate to `SoftToolRequirement`.
- Removed the standalone `ssh` command tool and `ssh/ssh-executor` (the `ssh://` read/write/search protocol stays), and made `--tools` address hidden built-ins.
- Updated collab-web to render `xd://` dispatches and `hub` op families, dropped the `search_tool_bm25`/`ssh`/`report-finding` renderers, refreshed tool docs and prompts, and migrated the affected tests and changelogs.
RPC mode iterated readJsonl(Bun.stdin.stream()), where JSON parsing runs
inside the generator. A parse error thrown from the generator escaped the
frame loop's try/catch (which only wrapped dispatch), unwound runRpcMode,
and exited the process on any non-JSON stdin line.
Read raw lines via readLines and JSON.parse each inside the existing
try/catch so a malformed line emits a Failed to parse command error frame
and the loop keeps running. Shared readJsonl stays strict for session-file
reads.
Fixes#5194
- Closed RPC host tool bridge before draining queued commands after stdin EOF.
- Rejected active host tool calls and future queued host tool calls with the disconnect error instead of emitting new host_tool_call frames.
- Added regression coverage for dispatcher drain with active and queued host tool requests.
Fixes#5153
- Added a serialized RPC input dispatcher so control-plane frames can resolve dialogs while ordinary commands remain ordered.
- Made pending extension UI requests fail closed on RPC disconnect so EOF drains active and queued commands.
- Covered extension UI response overtaking, queue ordering, queue recovery, and EOF dialog rejection in rpc input tests.
Fixes#5153
Extensions calling ctx.ui.addAutocompleteProvider (e.g. @ff-labs/pi-fff)
crashed at load with 'TypeError: ... is not a function' because omp's
ExtensionAPI.ui omitted pi's autocomplete-provider API; the throw also
aborted the rest of a try/catch-guarded session_start init.
ExtensionUIContext now declares addAutocompleteProvider(factory).
Interactive mode stacks each factory on the built-in editor provider in
registration order, re-applies the stack on every slash-command refresh,
and skips throwing/malformed factories; RPC, ACP, and headless contexts
accept the factory as a no-op, matching upstream pi's RPC behavior.
Fixes#4919
Extension sendUserMessage() without deliverAs fell through to prompt(),
which throws AgentBusyError during an active stream; the message was
dropped and surfaced as 'Extension sendUserMessage failed'. Route the
omitted-deliverAs path through prompt() with streamingBehavior 'steer'
so streaming queues a steer with normal prompt-flow side effects
(keyword notices, advisor auto-resume reset) and idle still starts a
turn.
ACP skill-command prompts now pass streamingBehavior 'steer'; the RPC
skill fast-path honors the prompt command's streamingBehavior field
(default steer) like the plain-prompt path already did. Documented the
extension-facing delivery semantics.
Synthesized from PR #4942 (prompt-flow steer routing, docs, tests) and
PR #4922 (RPC streamingBehavior threading, steer regression test);
dropped PR #4942's unrelated workflow-notice.md ellipsis churn.
Fixes#4923
Co-authored-by: roboomp <omp@can.ac>
Co-authored-by: metaphorics <metaphorics@users.noreply.github.com>
Codex review flagged two P2s the reporter (@DylanBohlender) confirmed:
1. OAuthCallbackFlow#handleCallback checked LAUNCH_PATH BEFORE the
`pathname !== this.callbackPath` guard, so an OMP config that pinned
the provider callback at `/launch` (via `oauth.callbackPath` or a
matching `oauth.redirectUri`) had the launch route eat its
`/launch?code=...&state=...` redirect and 302 it back to the
authorization URL instead of resolving the callback. Reorder so
`callbackPath` resolution wins the collision, and suppress `launchUrl`
in that case (also when `redirectUri`'s pathname resolves to `/launch`
even without an explicit `callbackPath` override) so UIs never
advertise a self-redirecting copy target.
2. RpcClient.login's `open_url` listener called
`onOpenUrl(req.url, req.instructions)` and dropped `launchUrl`, so SDK
hosts built on the public helper couldn't surface the truncation-safe
copy target. Extend the callback signature to
`(url, instructions?, launchUrl?)` and forward the field — backward
compatible for existing 1-2 arg consumers.
Regression tests in packages/ai/test/callback-server-launch-route.test.ts:
- callbackPath = /launch: launchUrl is undefined AND a
`/launch?code=...&state=...` request resolves via the callback template
(200/HTML), never 302s to the authorize URL.
- redirectUri pathname = /launch (callbackPath default): launchUrl still
suppressed defensively by the parsed-pathname guard so the base class
never advertises a colliding launch route even when callers skip the
MCPOAuthFlow path-derivation.
Two independent defects broke /mcp reauth against S256-only providers on
Windows boxes whose PATH no longer references System32:
1. openPath spawned bare rundll32 and swallowed the
`Executable not found in $PATH` throw with a bare `catch {}`, so the MCP
controller's outer try/catch was dead and the transcript unconditionally
claimed "Opening browser automatically...".
2. TUI#prepareLine silently truncates any composed row wider than the
viewport. MCPAuthorizationLinkPrompt rendered `Copy URL: <full URL>` as a
single ~271-column line whose trailing parameter is
code_challenge_method=S256. On the reporter's 270-col terminal the cut
landed inside that parameter, dropping the method while keeping
code_challenge — which RFC 7636 §4.3 treats as plain PKCE, which Linear
correctly rejects with "The plain PKCE method is not allowed. Use S256
instead."
OAuthCallbackFlow now hosts a `GET /launch` route on the same loopback
callback server it already runs; the route 302-redirects to the pending
authorization URL and is advertised as `OAuthAuthInfo.launchUrl` — a
~30-char copy target no viewport can meaningfully truncate. The MCP OAuth
fallback, /login, setup wizard, auth-broker CLI, and login-dialog all
prefer the launch URL for the visible copy target, keep the full URL in
the OSC 8 hyperlink for click-through, and the MCP flow additionally
stages the copy target on the clipboard via OSC 52 (same pattern the
setup wizard uses).
openPath now resolves rundll32.exe through %SystemRoot%\System32 (with a
C:\Windows fallback when SystemRoot is unset) and logs both synchronous
spawn throws and non-zero exits via the shared logger, so silent
misconfigurations show up in ~/.omp/logs/omp.*.log. The dead try/catch
around openPath in the MCP controller is removed.
Fixes#4418
- Prevent premature failure during process startup by waiting for stderr to drain before throwing exit errors.
- Resolve race conditions between stdout closure, process exit, and readiness timeouts by using a local child process reference.
- Ensure proper cleanup of abandoned processes during startup failures to prevent leaks.
- Introduced `RpcShutdownCoordinator` to track background tasks and manage deferred shutdowns safely.
- Guaranteed all background bash task response frames are fully written before the process exits.
- Re-checked shutdown requests automatically as each tracked background task settles.
- Latched the shutdown sequence to prevent concurrent execution from duplicate triggers.
- Updated `mock-rpc-agent` to consume stdin via an async iterator to match standard behavior.
- Added comprehensive unit tests in `rpc-input-frame.test.ts` covering background task coordination.
The RPC transport did not treat cancellation as an independent, resettable
control plane, so two lifecycle contract violations shared a root cause:
A. Server-side: the stdin loop in `rpc-mode.ts` awaited each commands
`handleCommand` before pulling the next frame, so `abort_bash` queued
behind the `bash` it must cancel. Extracted `dispatchRpcInputFrame` and
dispatch `bash` in the background: the input loop keeps reading, so
`abort_bash` (or any other command) can preempt an in-flight shell.
Response correlation still rides `command.id`; ordering across
concurrent commands is documented as not guaranteed.
B. Client-side: `RpcClient.stop()` aborted the shared `#abortController`
but never replaced it, so a subsequent `start()` handed a pre-aborted
signal to `readJsonl` and the stdout reader exited immediately with a
spurious "Agent process exited before ready" while the spawned child
leaked in `#process`. Mint a fresh `AbortController` inside `start()`
and clean up the child on any post-spawn failure.
Adds:
- `dispatchRpcInputFrame` unit tests covering the concurrent bash + abort
ordering, serial dispatch of other commands, and background error
reporting.
- `RpcClient` lifecycle tests covering start->stop->start on the same
instance (via a mock fixture agent) and retry after a failed start.
- Documents the bash concurrency contract in `docs/rpc.md`.
Fixes#4079
The mid-prompt slash skill autocomplete added in #3654 replaced the
entire editor draft with /skill:<name> on accept so the dispatcher
(which only matched leading /skill:) would still fire. That wiped
every keystroke the user had typed before reaching for the skill.
Insert the /skill:<name> token at the cursor in the TUI editor —
replacing only the partial /sk slash token, leaving prose before and
after intact — and extend the skill-command parser so a /skill:<name>
token surrounded by whitespace is recognized as an invocation too,
with the surrounding prose threaded through to the skill as args.
The parser change is shared across all three dispatch sites
(interactive TUI, ACP, RPC) via a new parseSkillInvocation helper
in extensibility/skills, so the three Map<string,string> /
session.skills lookups stay aligned on the same parse.
Fixes#3913
User-invoked skills (typed /skill:, steered, follow-up, interrupted/
resumed via compaction, ACP, RPC) only appended a bare "Skill: <path>"
line, so the model neither learned the user had invoked that specific
skill nor where the skill directory was. Relative paths in skill bodies
(scripts/, templates/) could not be resolved.
Route all user-invoked paths through a self-identifying, baseDir-aware
prompt template; keep hidden autoload skills on the minimal non-user
format. Interactive skillCommands now carries the loaded Skill object
instead of a bare path so baseDir flows through without reconstruction.
The invocation kind defaults to "user" to keep buildSkillPromptMessage
source-compatible.
Op: correct
Restores: spec:user-invoked-skill-prompt-self-identifies-and-exposes-skill-directory
Bonobo