Treated SearXNG HTTP 200 responses with no usable sources and upstream engine failures as transient provider errors. Added a generic renderable-content guard so provider fallback continues instead of returning an invisible success.\n\nFixes #2571
- Extended GitHub URL parsing to recognize commit paths and extract commit refs.
- Implemented a commit renderer that fetches commit metadata, stats, and file patches from the GitHub API.
- Added a handler branch to render commit URLs to markdown with `github-commit` results metadata.
Anthropic OAuth web search now uses resolveAnthropicMetadataUserId so metadata.user_id matches the main streaming path's {session_id, account_uuid?, device_id} JSON envelope. API-key paths keep forwarding the raw session id.
Exported resolveAnthropicMetadataUserId from pi-ai. Extended the regression test to cover the OAuth shape.
Refs #2295
Forwarded the active web search session id as Anthropic Messages metadata.user_id so enterprise gateways can attribute and rate-limit search calls consistently with the main streaming path.
Added a focused Anthropic web search request-shape regression test.
Fixes#2295
Move bundled models, model cache/manager, thinking metadata, effort helpers,
provider descriptors/discovery, wire constants, and model identity utilities
into the new @oh-my-pi/pi-catalog package.
Update pi-ai to keep provider runtime/auth concerns, move catalog provider
metadata into CATALOG_PROVIDERS, and migrate coding-agent, agent, stats, docs,
and tests to import catalog values from pi-catalog.
Split coding-agent model registry helpers into discovery, roles, and models
config modules while preserving registry orchestration.
BREAKING CHANGE: @oh-my-pi/pi-ai no longer exports catalog subpaths such as
/models, /model-cache, /model-manager, /model-thinking, /effort,
/provider-models*, discovery helpers, and provider wire constants; use the
matching @oh-my-pi/pi-catalog subpaths instead.
- Introduced memoized dynamic import loaders for Babel parser, mnemopi modules, puppeteer, and HTML-related packages.
- Refactored eval import-rewrite helpers and runtime call sites to use asynchronous wrapping and parsing flows.
- Shifted fetch and web-scraper linkedom usage to on-demand imports so heavy modules load only when needed.
- Exported and applied wrapFetchForCch only for OAuth Anthropic web-search calls.
- Mapped model_context_window_exceeded to "length" and mapped unknown stop reasons to "stop".
- Adjusted header and param generation to preserve caller User-Agent and gate Claude Code betas.
- Updated stream and strict-tool retry handling to clear terminal errors and prevent regressions.
Completes the injectable-fetch transport wiring (15.10.8) that the feature
left half-done, fixing the deterministic CI test failures:
- compaction.compact() rebuilt summaryOptions field-by-field but dropped
`fetch`, so the injected transport never reached
requestOpenAiRemoteCompaction / generateSummary's remote path. Thread it.
- Read-tool URL pipeline had no fetch seam: renderHtmlToText gained a
fetchOverride param but renderUrl/ToolSession never carried one, so the
jina/parallel reader backends always used global fetch. Add
ToolSession.fetch -> renderUrl -> renderHtmlToText (defaults to global).
- searchWithParallel mirrored extractWithParallel but missed the fetch
option; add it.
- Repair tests whose deleted hookFetch interceptors were never replaced
with a FetchImpl seam (fetch-kagi-toggle, web-search-parallel,
issue-970 discovery).
- Update issue-1746 POSIX case to the #2154 preserved-scrollback contract:
unknown-viewport streaming deferral is now platform-independent.
- Added optional FetchImpl fields to compaction, proxy, AI, coding-agent, and mnemopi options.
- Threaded injected fetch implementations through OAuth, discovery, and search/LLM request flows.
- Removed exported hookFetch utility and its package entrypoint from utils.
- Replaced global-fetch test monkeypatching with per-test FetchImpl mocks across test suites.
- Derived descriptors, default-model map, env keys, login list, and refresh dispatch from one ProviderDefinition per provider.
- Disabled OpenAI Codex stream obfuscation and interrupted whitespace-only tool-call argument deltas.
- Derived auth-broker callback ports and paste-code login set from the registry.
- Added status.done and tool.* symbols to theme mappings and presets.
- Replaced generic success glyphs with contextual +/-, tool icons, and warnings.
- Mapped tool/task/job completions to status.done or status.enabled with icon overrides.
- Triggered runtime provider refresh after extension registration and warned on failure.
- Adjusted `LspTool` to retry only zero/decl-only references with two 250ms waits for project-aware servers.
- Removed raw-mode GitHub repo README API fallback in `read`, so `:raw` now renders the page directly.
- Replaced regex heuristics in `isImagePlaceholderAnswer` with a fixed normalized placeholder set.
- Expanded Codex placeholder detection to match common image-reference phrases and punctuation.
- Raised `codex` provider failure when final and streamed text are placeholders and no sources exist.
- Dropped placeholder prose from returned answers while preserving citation sources.
- Routed image-gen, inspect-image, and web search providers through `withAuth`.
- Used `reuseInitialApiKey`/`createAuthStorageResolver` for force-refresh and rotate retries.
- Attached HTTP status to thrown errors so the retry classifier detects retryable failures.
- Replaced `providers.parallelFetch` with a new `providers.fetch` enum in settings and added migration cleanup for the legacy key.
- Updated `renderHtmlToText` to follow configured reader preference with ordered fallback attempts and remote-reader timeout handling before local conversion.
- Updated YouTube and fetch tests to use `providers.fetch` and cover Jina-first stall fallback behavior.
- Parsed /actions/runs URLs into run and job render handlers.
- Rendered run metadata with per-job breakdown, showing steps for failed jobs.
- Fetched job logs via API token, stripping ISO timestamp prefixes.
- Forced authenticated ask requests to `experimental`, matching the anonymous fallback since the cookie session ignores pro upgrades.
- Kept TUI collapsed search answers full; capping now only applies in compact mode via `maxAnswerLines`.
- Preserved full multiline task pending preview instead of bounding it.
- Sent the OAuth token as `__Secure-next-auth.session-token` cookie since the ask endpoint ignores bearer headers and silently downgrades to `turbo`.
- Fell back to `result.title` when web results omit `name`.
- Renamed `callPerplexityOAuth` to `callPerplexityAsk` and removed a stray brace.
- Added tests covering OAuth, API-key, and anonymous request shapes.
- Added anonymous Perplexity authentication mode for unauthenticated web searches.
- Switched web-search setup checks to use `isExplicitlyAvailable` and removed key enforcement in doctor.
- Updated Perplexity OAuth flow to reuse auth handling for all non-key searches and anonymous responses.
- Updated CLI and provider option help text to mark the Perplexity key optional with fallback.
- Stopped prepending system_prompt to the consumer ask endpoint, which lacks a system slot and refused the meta-instruction.
- Kept system_prompt as a proper system message on the API-key path.
- Showed answer text in full in the TUI; kept the `omp q` compact cap.
- Rendered each source as a single title/domain/age line with the URL linked on the title.
- Collapsed the metadata block to one Provider line plus Usage.
- Rendered search errors as a framed panel matching the success layout.
- Stopped capping the synthesized answer at 12 lines while sources expanded in full.
- Rendered the answer through Markdown so headings, bold, lists, and code display formatted.
- Added regression tests for expanded/collapsed answer rendering.
- Raised default search results to 20 and context size to high.
- Added related questions parsing and return_related_questions request flag.
- Added API-key request-shape tests covering defaults and parsing.
Reviewer noted that an unconditional ExaProvider.isAvailable steered the auto chain into the public MCP fallback before any later-configured provider could run. Restored the credential-gated isAvailable, then split out isExplicitlyAvailable so resolveProviderChain still routes an explicit Exa selection through MCP without affecting other providers.\n\nRefs #1860
Restored Exa's unauthenticated MCP fallback when neither auth storage nor EXA_API_KEY provides credentials. Preserved API-key search ordering and bounded the MCP request with the web-search hard timeout. Updated provider copy and regression coverage for the no-key path.\n\nFixes #1860
ExaProvider.isAvailable() and searchExa() now consult AuthStorage so Exa credentials configured through the broker/credential store work alongside EXA_API_KEY, matching the other API-key search providers.
Refs #1695
- Passed ANTHROPIC_SEARCH_BASE_URL through to Anthropic web search calls that use authStorage fallback credentials instead of only applying it with ANTHROPIC_SEARCH_API_KEY.
- Added regression coverage asserting fallback Anthropic credentials use the search-specific base URL.
- Updated the environment and web_search docs to reflect the actual credential and base URL resolution order.
Fixes#1694
Replace the sunset V0 Search API with V1 (POST /api/v1/search) under the
existing `kagi` provider id instead of shipping a parallel `kagi-v1`
provider. Credentials still resolve through the shared AuthStorage broker
(Bearer token, KAGI_API_KEY, /login kagi), and recency now maps to a
UTC-deterministic filters.after date.
- Merge V1 client into src/web/kagi.ts (categorized result buckets, direct
answer, related/adjacent questions)
- Keep classifyProviderHttpError mapping for auth/quota signals
- Drop the kagi-v1 entries from the provider registry, order, type union,
and settings schema
- Consolidate tests into web-search-kagi.test.ts
- Added `ToolTier`, `ToolApproval`, and `ToolApprovalDecision` types and exported approval APIs.
- Updated approval-mode options from `auto|prompt|custom` to `always-ask|write|yolo` and defaulted mode to `yolo`.
- Changed approval resolution to apply per-tool decisions first, then mode-tier limits, with legacy-mode migration.
- Assigned read/write/exec `approval` and approval-detail prompts across built-in, custom, extension, and MCP tools.
Centralizing OAuth refresh in AuthStorage (e6893515) introduced five
follow-on bugs surfaced by an audit of the commit; this fixes all of
them and updates the tests that relied on the old refresh seam.
1. packages/ai/src/auth-storage.ts (#tryOAuthCredential):
For built-in providers the path went directly to `getOAuthApiKey`
with the (possibly still-expired) selection.credential when the
pre-refresh at line 2587 caught a transient error. `getOAuthApiKey`
then threw the "expired … must be refreshed via AuthStorage"
precondition error, which the disable classifier matched against
`/expired.*refresh/` and soft-disabled the row. A single network
blip during refresh could permanently kill a still-valid Anthropic /
OpenAI / Gemini-CLI / Copilot credential. Built-in providers now
route through the broker-aware single-flighted
`#refreshOAuthCredential` first, so transient failures surface as
network errors (5-min temp block) instead of definitive auth
failures.
2. packages/ai/src/auth-storage.ts (#fetchUsageUncached):
The usage refresh check only fired once `Date.now() >= expiresAt`,
missing the 60-second skew that `getApiKey` honors. A token
expiring inside the skew window was posted to the usage endpoint
and 401'd mid-flight, briefly hiding quota in the UI. Aligned with
`OAUTH_REFRESH_SKEW_MS`.
3. packages/coding-agent/src/web/search/index.ts (webSearchCustomTool):
The CustomTool counterpart of WebSearchTool dropped sessionId so
SDK callers that opted into `web_search` via toolNames lost
per-session credential stickiness — multi-account users saw the
provider round-robin between searches in the same session. Threads
`ctx.sessionManager.getSessionId()` through to `executeSearch`.
4. packages/coding-agent/src/web/search/providers/perplexity.ts
(findOAuthToken):
`authStorage.getApiKey("perplexity")` returns runtime/config
overrides, stored api_key credentials, OAuth bearers, and env keys.
Filtering only env keys meant a config-pinned `pplx-…` API key was
POSTed to `www.perplexity.ai/rest/sse/perplexity_ask` (the OAuth
endpoint) instead of falling through to
`api.perplexity.ai/chat/completions`, producing 401s. Switched to
`getOAuthAccess` so only true OAuth bearers reach the OAuth
branch; api_key credentials/overrides correctly fall through.
5. packages/ai/scripts/generate-models.ts:
`getOAuthApiKey` was being called directly with possibly-expired
credentials. The new contract throws on expired, the broad catch
swallowed it, and the build silently fell back to bundled models
instead of refreshing. Both helpers now route through
AuthStorage's `getApiKey` / `getOAuthAccess`, which trigger the
full broker-aware refresh pipeline.
Test updates:
- auth-storage-credential-disabled-event.test.ts,
sdk-credential-disabled-bridge.test.ts: the `failOAuthRefresh`
helper used to spy on `getOAuthApiKey` to inject invalid_grant.
With refresh now happening before that helper, the spy never fired.
Switched to spying on `refreshOAuthToken` so the simulated failure
reaches the disable classifier.
- auth-storage-rotation.test.ts: stub `refreshOAuthToken` so the test
doesn't hit a real OAuth endpoint when the seeded credential lands
inside the 60s skew window.
- packages/ai/test/issue-957-repro.test.ts now tests:
- refreshKimiToken applies the 5-minute server-side skew (Kimi-specific)
- AuthStorage refreshes kimi-code credentials inside its 60s skew window
- packages/ai/test/anthropic-stream-timeout.test.ts: raise the
streamFirstEventTimeoutMs from 10ms to 5000ms so slow CI scheduling
cannot fire the first-event watchdog before the mocked events arrive.
The test still exercises the (1ms) idle path it was written for.
fix(web): allow Parallel extract via PARALLEL_API_KEY env var without storage
The fetch tool and YouTube scraper previously gated the Parallel extract
branch behind `storage && findParallelApiKey(storage)`. With no
AgentStorage the env key was never consulted, so callers that ran
without a per-session storage (e.g. ReadTool sessions in unit tests, and
in practice any caller that has only an env API key) silently fell back
to raw-html / no-ytdlp paths.
- findCredential/findParallelApiKey now accept null or undefined storage
and rely solely on the env-first path when no storage is supplied.
- searchWithParallel/extractWithParallel mirror the same nullable shape.
- Drop the redundant `storage && ` guards in fetch.ts and youtube.ts;
the inner findParallelApiKey call already returns null when no
credential is available.
- Centralized OAuth access lifecycle in `AuthStorage`, returning identity metadata and new access-result types.
- Added 60-second skew and strict expiry checks, returning undefined/throws for stale or expired OAuth credentials.
- Removed provider-local token refresh flows from Gemini, Gemini CLI, Antigravity, Kimi, and related OAuth helpers.
- Migrated web-search providers from `AgentStorage` to `AuthStorage` session-aware lookup with `authStorage`/`sessionId`/`signal` flow.
- Replaced `findAnthropicAuth`/DB auth lookup with `buildAnthropicAuthConfig` and explicit base-url override/env fallback ordering.
- Cleaned up tests, made them more useful
- Fix a bug in the OpenAPI spec with Kagi v1 where 'trace' was mapped to
'id' incorrectly from Kagi's documentation
- Added OpenAI Codex and Gemini web search provider options with updated setup/auth descriptions.
- Updated Codex OAuth flow to refresh near-expiry tokens during web_search and persist the refreshed credentials.
- Plumbed AgentStorage through search orchestrator, scrapers, and fetch paths so providers share session credentials.
- Refactored web provider and credential helpers to accept caller-provided AgentStorage and resolve keys synchronously.
can1357