- Raised default search results to 20 and context size to high.
- Added related questions parsing and return_related_questions request flag.
- Added API-key request-shape tests covering defaults and parsing.
Reviewer noted that an unconditional ExaProvider.isAvailable steered the auto chain into the public MCP fallback before any later-configured provider could run. Restored the credential-gated isAvailable, then split out isExplicitlyAvailable so resolveProviderChain still routes an explicit Exa selection through MCP without affecting other providers.\n\nRefs #1860
Restored Exa's unauthenticated MCP fallback when neither auth storage nor EXA_API_KEY provides credentials. Preserved API-key search ordering and bounded the MCP request with the web-search hard timeout. Updated provider copy and regression coverage for the no-key path.\n\nFixes #1860
ExaProvider.isAvailable() and searchExa() now consult AuthStorage so Exa credentials configured through the broker/credential store work alongside EXA_API_KEY, matching the other API-key search providers.
Refs #1695
- Passed ANTHROPIC_SEARCH_BASE_URL through to Anthropic web search calls that use authStorage fallback credentials instead of only applying it with ANTHROPIC_SEARCH_API_KEY.
- Added regression coverage asserting fallback Anthropic credentials use the search-specific base URL.
- Updated the environment and web_search docs to reflect the actual credential and base URL resolution order.
Fixes#1694
Replace the sunset V0 Search API with V1 (POST /api/v1/search) under the
existing `kagi` provider id instead of shipping a parallel `kagi-v1`
provider. Credentials still resolve through the shared AuthStorage broker
(Bearer token, KAGI_API_KEY, /login kagi), and recency now maps to a
UTC-deterministic filters.after date.
- Merge V1 client into src/web/kagi.ts (categorized result buckets, direct
answer, related/adjacent questions)
- Keep classifyProviderHttpError mapping for auth/quota signals
- Drop the kagi-v1 entries from the provider registry, order, type union,
and settings schema
- Consolidate tests into web-search-kagi.test.ts
- Added `ToolTier`, `ToolApproval`, and `ToolApprovalDecision` types and exported approval APIs.
- Updated approval-mode options from `auto|prompt|custom` to `always-ask|write|yolo` and defaulted mode to `yolo`.
- Changed approval resolution to apply per-tool decisions first, then mode-tier limits, with legacy-mode migration.
- Assigned read/write/exec `approval` and approval-detail prompts across built-in, custom, extension, and MCP tools.
Centralizing OAuth refresh in AuthStorage (e6893515) introduced five
follow-on bugs surfaced by an audit of the commit; this fixes all of
them and updates the tests that relied on the old refresh seam.
1. packages/ai/src/auth-storage.ts (#tryOAuthCredential):
For built-in providers the path went directly to `getOAuthApiKey`
with the (possibly still-expired) selection.credential when the
pre-refresh at line 2587 caught a transient error. `getOAuthApiKey`
then threw the "expired … must be refreshed via AuthStorage"
precondition error, which the disable classifier matched against
`/expired.*refresh/` and soft-disabled the row. A single network
blip during refresh could permanently kill a still-valid Anthropic /
OpenAI / Gemini-CLI / Copilot credential. Built-in providers now
route through the broker-aware single-flighted
`#refreshOAuthCredential` first, so transient failures surface as
network errors (5-min temp block) instead of definitive auth
failures.
2. packages/ai/src/auth-storage.ts (#fetchUsageUncached):
The usage refresh check only fired once `Date.now() >= expiresAt`,
missing the 60-second skew that `getApiKey` honors. A token
expiring inside the skew window was posted to the usage endpoint
and 401'd mid-flight, briefly hiding quota in the UI. Aligned with
`OAUTH_REFRESH_SKEW_MS`.
3. packages/coding-agent/src/web/search/index.ts (webSearchCustomTool):
The CustomTool counterpart of WebSearchTool dropped sessionId so
SDK callers that opted into `web_search` via toolNames lost
per-session credential stickiness — multi-account users saw the
provider round-robin between searches in the same session. Threads
`ctx.sessionManager.getSessionId()` through to `executeSearch`.
4. packages/coding-agent/src/web/search/providers/perplexity.ts
(findOAuthToken):
`authStorage.getApiKey("perplexity")` returns runtime/config
overrides, stored api_key credentials, OAuth bearers, and env keys.
Filtering only env keys meant a config-pinned `pplx-…` API key was
POSTed to `www.perplexity.ai/rest/sse/perplexity_ask` (the OAuth
endpoint) instead of falling through to
`api.perplexity.ai/chat/completions`, producing 401s. Switched to
`getOAuthAccess` so only true OAuth bearers reach the OAuth
branch; api_key credentials/overrides correctly fall through.
5. packages/ai/scripts/generate-models.ts:
`getOAuthApiKey` was being called directly with possibly-expired
credentials. The new contract throws on expired, the broad catch
swallowed it, and the build silently fell back to bundled models
instead of refreshing. Both helpers now route through
AuthStorage's `getApiKey` / `getOAuthAccess`, which trigger the
full broker-aware refresh pipeline.
Test updates:
- auth-storage-credential-disabled-event.test.ts,
sdk-credential-disabled-bridge.test.ts: the `failOAuthRefresh`
helper used to spy on `getOAuthApiKey` to inject invalid_grant.
With refresh now happening before that helper, the spy never fired.
Switched to spying on `refreshOAuthToken` so the simulated failure
reaches the disable classifier.
- auth-storage-rotation.test.ts: stub `refreshOAuthToken` so the test
doesn't hit a real OAuth endpoint when the seeded credential lands
inside the 60s skew window.
- packages/ai/test/issue-957-repro.test.ts now tests:
- refreshKimiToken applies the 5-minute server-side skew (Kimi-specific)
- AuthStorage refreshes kimi-code credentials inside its 60s skew window
- packages/ai/test/anthropic-stream-timeout.test.ts: raise the
streamFirstEventTimeoutMs from 10ms to 5000ms so slow CI scheduling
cannot fire the first-event watchdog before the mocked events arrive.
The test still exercises the (1ms) idle path it was written for.
fix(web): allow Parallel extract via PARALLEL_API_KEY env var without storage
The fetch tool and YouTube scraper previously gated the Parallel extract
branch behind `storage && findParallelApiKey(storage)`. With no
AgentStorage the env key was never consulted, so callers that ran
without a per-session storage (e.g. ReadTool sessions in unit tests, and
in practice any caller that has only an env API key) silently fell back
to raw-html / no-ytdlp paths.
- findCredential/findParallelApiKey now accept null or undefined storage
and rely solely on the env-first path when no storage is supplied.
- searchWithParallel/extractWithParallel mirror the same nullable shape.
- Drop the redundant `storage && ` guards in fetch.ts and youtube.ts;
the inner findParallelApiKey call already returns null when no
credential is available.
- Centralized OAuth access lifecycle in `AuthStorage`, returning identity metadata and new access-result types.
- Added 60-second skew and strict expiry checks, returning undefined/throws for stale or expired OAuth credentials.
- Removed provider-local token refresh flows from Gemini, Gemini CLI, Antigravity, Kimi, and related OAuth helpers.
- Migrated web-search providers from `AgentStorage` to `AuthStorage` session-aware lookup with `authStorage`/`sessionId`/`signal` flow.
- Replaced `findAnthropicAuth`/DB auth lookup with `buildAnthropicAuthConfig` and explicit base-url override/env fallback ordering.
- Cleaned up tests, made them more useful
- Fix a bug in the OpenAPI spec with Kagi v1 where 'trace' was mapped to
'id' incorrectly from Kagi's documentation
- Added OpenAI Codex and Gemini web search provider options with updated setup/auth descriptions.
- Updated Codex OAuth flow to refresh near-expiry tokens during web_search and persist the refreshed credentials.
- Plumbed AgentStorage through search orchestrator, scrapers, and fetch paths so providers share session credentials.
- Refactored web provider and credential helpers to accept caller-provided AgentStorage and resolve keys synchronously.
Replace relative time strings ("1d ago", "1w ago", "1mo ago", "1y ago") with
computed YYYY-MM-DD dates in the Kagi V1 search request's `filters.after`
field. The new `recencyToDate()` helper handles month/year drift and leap-day
rollover correctly.
This aligns with the Kagi V1 API's documented expectation of ISO date strings
in the filters object. Relative time strings were a carryover from the V0
integration and caused silent filter mismatches when the API stopped resolving
them on its end.
- Change filters.time_after/time_before → filters.after/before (V1 API
field names; old keys were silently ignored, breaking every recency query)
- Update KagiV1SearchData to include all V1 response categories
(podcast, adjacent_question, interesting_news, etc.)
- Update KagiV1SearchResultItem.props to Record<string, unknown>
- Parse related_search entries as result objects, extracting question
text from props.question/props.query/title
- Read direct_answer from standard result object fields (snippet/title)
instead of the non-existent .text field
Adds a shared classifyProviderHttpError helper that maps the well-known
failure shapes (status 401/402/403 and bodies matching credits / quota
/ insufficient) into compact SearchProviderError messages so the
orchestrator advances to the next provider instead of bailing.
Wired into every HTTP-talking provider (codex, exa, gemini, anthropic,
brave, jina, kimi, perplexity, searxng, synthetic, tavily, zai) and the
two wrapped-error providers (kagi via KagiApiError, parallel via
ParallelApiError). The orchestrator now collects per-provider failures
and emits a joined summary ("exa: 403 forbidden; codex: credits
exhausted; ...") when the whole chain fails.
Test updates: web-search-{exa,tavily,kagi} message expectations now
assert the compact "<id>: <status> <reason>" form.
Bun's WinHTTP backend can ignore AbortSignal once a TCP/TLS connection
stalls (oven-sh/bun#15275, oven-sh/bun#18536), so Esc never reached the
in-flight `web_search` fetch on Windows and the session froze until
Ctrl+C. Only kimi shipped any timeout at all (server-side); every other
provider passed `signal` to `fetch` with no client-side bound.
Introduced `withHardTimeout(signal, ms=60_000)` in providers/utils and
wired it into every web-search provider's outbound fetch — anthropic,
brave, codex, exa, gemini, jina, kagi, kimi, parallel, perplexity
(api-key and oauth), searxng, synthetic, tavily, z.ai. 60s tolerates
legitimate slow LLM-mediated responses while still guaranteeing the
request settles within a minute when Bun's abort fails to propagate.
Independently, `executeSearch`'s provider-fallback loop swallowed every
`AbortError` as a regular provider error and returned
"All web search providers failed", masking cancellation on every
platform. The catch block now calls `throwIfAborted(signal)` first so a
caller-initiated cancel propagates as `ToolAbortError`.
Fixes#1221
- Updated Perplexity JWT parsing in the ai OAuth utilities to return a far-future sentinel when `exp` is missing and use that when computing token expiry.
- Updated `getOAuthApiKey` to prefer the JWT expiry and normalized legacy one-hour `expires` values to a non-expiring value.
- Updated coding-agent web search token lookup to verify Perplexity credentials against the JWT `exp` claim and treat missing claims as non-expiring.
When the Codex backend returned the literal "(see attached image)" as
the answer for a text query and streamedAnswer was empty, the wrapper
accepted the placeholder as the response and the chain never advanced.
Throw SearchProviderError so the next provider is tried.
- Replaced all StringEnum(...) usages with z.enum([...]) across tools, examples, and tests.
- Removed StringEnum re-export from @oh-my-pi/pi-coding-agent public API.
- Condensed verbose tool parameter descriptions to minimal lowercase phrases.
- Renamed AuthCredentialStore to SqliteAuthCredentialStore at usage sites.
- Unified line-ending normalization to `replace(/\r\n?/g, "\\n")` in editor, scraper, benchmark, and utils modules.
- Added terminal-aware line sanitization in code-cell rendering to collapse inline carriage returns and avoid overwrite corruption.
- Tightened editor and paste sanitizers to trim control characters consistently after CR normalization.
- Added canonical `pi.zod` schema API exports and removed TypeBox package exports/imports.
- Migrated Tool schema typing from TypeBox to shared `TSchema`/Zod flow with legacy TypeBox compatibility.
- Updated AI provider adapters and MCP/agent builders to convert tool params through `toolWireSchema()`.
- Reworked schema validation from AJV to Zod-safe parsing with `fromTypeBox`, `toolWireSchema`, and meta schema checks.
- anthropic: add signal to AnthropicSearchParams, callSearch(), and
AnthropicProvider.search()
- exa: add signal to ExaSearchParams, callExaSearch(), and ExaProvider.search()
- jina: add signal to JinaSearchParams, callJinaSearch(), and JinaProvider.search()
- zai: add signal to ZaiSearchParams, callZaiTool(), and ZaiProvider.search()
- gemini: add signal to GeminiSearchParams, callGeminiSearch() and
buildInit() so both fetchWithRetry calls (initial + auth-refresh retry)
carry the signal
The five providers listed above never forwarded SearchParams.signal to the
underlying HTTP layer, so pressing Esc during a web_search call had no effect
and the session froze until the request resolved or Ctrl+C was pressed.
brave, kimi, perplexity, searxng, tavily, synthetic, codex, kagi, and parallel
already thread the signal correctly and are unchanged.
Fixes#1044
- Removed local `abortableSleep` in favour of Node's built-in `scheduler.wait` from `node:timers/promises`.
- Consolidated per-provider retry/fetch loops into a shared `fetchWithRetry` utility in `packages/utils`.
- Moved `extractHttpStatusFromError`, `isRetryableError`, and related helpers out of `packages/ai` into `packages/utils`.
- Deleted `extractRetryDelay` in favour of `extractRetryHint` with unified header and body parsing.
- Extended executeSearch to accept an optional AbortSignal parameter.
- Passed the provided signal through to the search provider invocation.
- Updated execute methods to forward incoming signals into executeSearch.
- Added optional `loadMode` and `summary` fields to `AgentTool` and related type declarations.
- Added `loadMode` and `summary` metadata to built-in tool classes for discoverable/essential behavior.
- Replaced `BUILTIN_TOOL_METADATA` with per-tool fields in discovery code paths.
- Updated `search_tool_bm25` and discovery indexing to use each tool's `summary` text.
- Updated discovery tests to validate tool `loadMode` and summary completeness.
- Converted systemPrompt APIs and state types to ordered `string[]` across agent, AI, and coding-agent surfaces.
- Added `normalizeSystemPrompts` and applied it to context normalization before building provider request payloads.
- Updated AI providers to emit separate normalized prompt blocks/messages instead of a single merged system prompt.
- Removed dedicated `projectPrompt` state and remapped that context into system-context buckets in session, dump, and token accounting.
- Aligned tests and changelogs to pass and assert `systemPrompt` as arrays with ordered prompt semantics.
- Consolidated AI provider imports through register-builtins and moved Gemini/Antigravity header helpers to a shared module.
- Added lazy loading for heavy providers and SDK-backed modules with cached initialization to trim startup cost.
- Converted markdown conversion helpers to async and awaited htmlToBasicMarkdown in affected scraper and kernel output paths.
- Parsed bundled agent definitions on-demand and moved BrowserTool prompt rendering behind a memoized getter.
- Added cached validation/error handling paths by replacing AJV runtime checks with Value.Check and trimming validation error output.
- Documented and removed `utils/oauth` from the `ai` package entrypoint, noting it as a breaking change.
- Refactored `cli`, `auth-storage`, and `utils/oauth` to load provider modules via scoped dynamic `import()` calls.
- Removed top-level provider imports and barrel exports from `utils/oauth/index.ts`, streamlining oauth module loading.
- Consolidated OAuth symbol, type, and provider imports in coding-agent and tests to `@oh-my-pi/pi-ai/utils/oauth` modules.
- Defined `DEFAULT_LOCAL_TOKEN` locally in model-registry and removed its cross-package OAuth import usage.
- Renamed the built-in `grep` content-search tool to `search` across settings, schemas, and SDK exports.
- Switched execution wiring so `Task`, `Plan`, cursor, and shell mapping now invoke `search` instead of `grep`.
- Updated prompts, plan-mode docs, and example tool lists to replace `grep`/`ls` references with `search` guidance.
- Aligned `Grep*`/`grep` event, renderer, and hook types to `Search*`/`search` across runtime and tests.
- Documented and fixed `search` result rendering budget behavior and added internal-URL/path-list transcript notes.
Add explicit Basic auth credentials for SearXNG while preserving Bearer token fallback.
Keep optional SearXNG settings out of the TUI and allow empty RFC 7617 username/password fields.
- Updated the ZAI provider constant to use the `web_search_prime` tool name.
- This aligned the search provider configuration with the expected MCP endpoint identifier.
- Updated the root `fix:ts` script to run `bun run --workspaces --if-present fix` after `fix:tools`.
- Updated `fix:ts:all` to run `fix:tools:all` followed by workspace-wide `fix` tasks.
- Applied minor SearXNG request cleanup, including direct `Authorization` header assignment and formatting-only formatting changes.
When the Codex Responses API synthesizes an answer without emitting
url_citation annotations, previously-empty sources made cited results
look ungrounded. Add:
- tool_choice: { type: "web_search" } so Codex must call the tool
- markdown-link + bare-URL extraction from the answer as a fallback
that only runs when no structured citations were returned
The model-resolution path is unchanged; getBundledModels already
returns a usable Codex catalog on main.
Closes#724
can1357