- Added optional `loadMode` and `summary` fields to `AgentTool` and related type declarations.
- Added `loadMode` and `summary` metadata to built-in tool classes for discoverable/essential behavior.
- Replaced `BUILTIN_TOOL_METADATA` with per-tool fields in discovery code paths.
- Updated `search_tool_bm25` and discovery indexing to use each tool's `summary` text.
- Updated discovery tests to validate tool `loadMode` and summary completeness.
- Converted systemPrompt APIs and state types to ordered `string[]` across agent, AI, and coding-agent surfaces.
- Added `normalizeSystemPrompts` and applied it to context normalization before building provider request payloads.
- Updated AI providers to emit separate normalized prompt blocks/messages instead of a single merged system prompt.
- Removed dedicated `projectPrompt` state and remapped that context into system-context buckets in session, dump, and token accounting.
- Aligned tests and changelogs to pass and assert `systemPrompt` as arrays with ordered prompt semantics.
- Consolidated AI provider imports through register-builtins and moved Gemini/Antigravity header helpers to a shared module.
- Added lazy loading for heavy providers and SDK-backed modules with cached initialization to trim startup cost.
- Converted markdown conversion helpers to async and awaited htmlToBasicMarkdown in affected scraper and kernel output paths.
- Parsed bundled agent definitions on-demand and moved BrowserTool prompt rendering behind a memoized getter.
- Added cached validation/error handling paths by replacing AJV runtime checks with Value.Check and trimming validation error output.
- Documented and removed `utils/oauth` from the `ai` package entrypoint, noting it as a breaking change.
- Refactored `cli`, `auth-storage`, and `utils/oauth` to load provider modules via scoped dynamic `import()` calls.
- Removed top-level provider imports and barrel exports from `utils/oauth/index.ts`, streamlining oauth module loading.
- Consolidated OAuth symbol, type, and provider imports in coding-agent and tests to `@oh-my-pi/pi-ai/utils/oauth` modules.
- Defined `DEFAULT_LOCAL_TOKEN` locally in model-registry and removed its cross-package OAuth import usage.
- Renamed the built-in `grep` content-search tool to `search` across settings, schemas, and SDK exports.
- Switched execution wiring so `Task`, `Plan`, cursor, and shell mapping now invoke `search` instead of `grep`.
- Updated prompts, plan-mode docs, and example tool lists to replace `grep`/`ls` references with `search` guidance.
- Aligned `Grep*`/`grep` event, renderer, and hook types to `Search*`/`search` across runtime and tests.
- Documented and fixed `search` result rendering budget behavior and added internal-URL/path-list transcript notes.
Add explicit Basic auth credentials for SearXNG while preserving Bearer token fallback.
Keep optional SearXNG settings out of the TUI and allow empty RFC 7617 username/password fields.
- Updated the ZAI provider constant to use the `web_search_prime` tool name.
- This aligned the search provider configuration with the expected MCP endpoint identifier.
- Updated the root `fix:ts` script to run `bun run --workspaces --if-present fix` after `fix:tools`.
- Updated `fix:ts:all` to run `fix:tools:all` followed by workspace-wide `fix` tasks.
- Applied minor SearXNG request cleanup, including direct `Authorization` header assignment and formatting-only formatting changes.
When the Codex Responses API synthesizes an answer without emitting
url_citation annotations, previously-empty sources made cited results
look ungrounded. Add:
- tool_choice: { type: "web_search" } so Codex must call the tool
- markdown-link + bare-URL extraction from the answer as a fallback
that only runs when no structured citations were returned
The model-resolution path is unchanged; getBundledModels already
returns a usable Codex catalog on main.
Closes#724
- Standardized missing-file read errors and now return `File not found: <path>` for absent edit targets.
- Centralized AI provider, usage, and OAuth helpers into shared modules to remove duplicated logic.
- Migrated OAuth/API-key login flows to shared factory helpers and removed inline prompt/token-exchange code.
- Reused shared tools and formatter utilities for discovery, stream tails, LSP batching, and source formatting.
- Consolidated repeated test helpers and fixtures into shared modules, replacing inline helper duplicates.
Removed unconditional topic:news coupling in buildRequestBody that scoped
Tavily index to news publications whenever recency was set. Technical queries
with --recency now search the general index filtered by time only.
Tightened SearchParams.recency contract in base.ts: providers MUST interpret
recency as a pure time filter and MUST NOT change topic scope as a side effect.
- SearXNG time_range only supports day/month/year; map week→month
- Add SearXNG option to settings-defs.ts submenu so users can
select it from the Settings UI (Providers tab)
- Import settings singleton directly (matches Exa provider pattern)
- Replace getEnvApiKey with process.env for env var fallbacks
(getEnvApiKey only works for registered provider IDs)
- Remove settings parameter from searchSearXNG/findEndpoint/findToken
- Add try/catch around settings.get calls for initialization safety
Fixes Codex review P1 issues on PR #712
Add SearXNG as a search backend for the web_search tool, enabling
privacy-friendly, self-hosted metasearch without API keys or usage limits.
Changes:
- New SearXNGProvider implementing SearchProvider interface
- Supports bearer token auth, categories, language, and recency filters
- Configurable via settings (searxng.endpoint, .token, .categories, .language)
or environment variables (SEARXNG_ENDPOINT, SEARXNG_TOKEN)
- Registered in provider chain and settings schema
- Maps SearXNG JSON API response to unified SearchResponse
Closes#545
- Added `extractReadableFromHtml()` utility function with dual-path content extraction using Readability library and CSS selector fallback.
- Integrated Turndown library with GitHub Flavored Markdown plugin for improved HTML-to-markdown conversion supporting tables, strikethrough, and task lists.
- Refactored `getPageReadable()` action to use new extraction function, consolidating content parsing logic and improving maintainability.
- Added TypeScript type declarations for turndown-plugin-gfm module with custom Turndown rules for enhanced markdown formatting.
- Fixed duplicate synthetic tool results by tracking persisted tool call IDs and skipping synthesis when real results exist.
- Fixed Codex search streamed answer handling to detect and skip image placeholder text, preferring final answer.
- Added test coverage for duplicate tool result prevention in message transformation pipeline.
- Extracted prompt rendering and formatting utilities from coding-agent to centralized pi-utils package with new API surface (prompt.render, prompt.format, prompt.registerHelper).
- Migrated parseFrontmatter utility from coding-agent to pi-utils package; updated 8 files to import from @oh-my-pi/pi-utils.
- Removed 170-line prompt-format.ts module and consolidated 192 lines of Handlebars helper registrations into pi-utils prompt module.
- Updated 60+ files across coding-agent and typescript-edit-benchmark to use new prompt.render() and prompt.format() API from pi-utils.
- Simplified prompt-templates.ts by delegating core functionality to pi-utils while retaining custom helper registrations (jtdToTypeScript, jsonStringify, etc.).
- Replaced Python-based markitdown CLI with native markit-ai library for document and notebook conversion.
- Added support for converting Jupyter notebooks (.ipynb) to markdown via markit-ai integration.
- Created markit utility module with file/buffer conversion wrappers and standardized error handling.
- Removed markitdown from Python tools manager and eliminated external CLI dependency.
- Updated fetch, read, and scraper tools to use new markit conversion API across codebase.
- Added test coverage for ipynb file conversion through markit utility.
- Removed code_search tool and all related Exa API integration from web search module.
- Deleted code-search.ts implementation including CodeSearchResponse types and executeCodeSearch() function.
- Updated getSearchTools() to return only webSearchCustomTool, removing dual-tool support.
- Removed code_search tool documentation from README and prompt files.
- Added pluggable code search provider system supporting Exa and grep.app with provider selection via `providers.codeSearch` setting.
- Removed Exa-specific tools (`exa_linkedin`, `exa_company`, `exa_search_deep`, `exa_crawl`) and simplified web search tools to focus on core functionality.
- Refactored code search from Exa-only to provider-agnostic architecture with new `code_search` tool supporting context-aware grep.app queries and Exa fallback.
- Removed `exa.enableLinkedin` and `exa.enableCompany` configuration settings in favor of provider-based architecture.
- Added comprehensive test coverage for code search functionality including grep.app result normalization and provider fallback behavior.
- Added Parallel AI provider integration for web search with fast and research modes.
- Added Parallel extract API for URL content and YouTube video extraction with fallback support.
- Added /login parallel command and PARALLEL_API_KEY environment variable authentication.
- Added providers.parallelFetch configuration setting to control Parallel extract usage.
- Integrated Parallel provider into web search priority order between Exa and Kagi.
- Updated HTML-to-text and YouTube scrapers to prefer Parallel extract over fallback providers.
- Removed provider parameter from web search tool schema; provider selection now handled internally.
- Removed deprecated no_fallback option from search parameters; fallback behavior is now automatic.
- Renamed SearchParams type to SearchToolParams and introduced SearchQueryParams for CLI queries.
- Updated executeSearch() to accept SearchQueryParams with optional provider selection.
- Added Tavily web search provider with API key authentication and credential discovery from environment or database.
- Integrated Tavily as highest-priority search provider in fallback chain with structured response mapping and error handling.
- Added Tavily OAuth login flow in CLI and auth-storage with manual API key input and validation.
- Added comprehensive test suite for Tavily provider covering registration, response mapping, error handling, and credential validation.
Fixes#313
- Removed Kagi Universal Summarizer integration from fetch tool and YouTube scraper.
- Removed `fetch.useKagiSummarizer` configuration setting from settings schema.
- Simplified renderHtmlToText() and renderUrl() functions by removing Kagi summarization fallback logic.
- Fixed indentation inconsistencies in test files from tabs to spaces.
findAnthropicAuth() only checked OAuth credentials in agent.db, missing
api_key type credentials entirely. Users authenticated via stored API key
(not env var, not OAuth) got null from isAvailable(), causing the provider
chain to skip Anthropic.
Added tier 4 (api_key in agent.db) between OAuth check and env var
fallback. Refactored store lifecycle so tiers 3-4 share one instance.
Also fixed ExaProvider.isAvailable() which unconditionally returned true,
ignoring exa.enabled/exa.enableSearch settings and never checking for
credentials. It now respects both settings and requires an actual API key.
- Fixed WebSocket stream fallback logic to safely replay buffered output over SSE when WebSocket fails after partial content has been streamed.
- Added tracking flag to prevent unsafe replays of tool calls and terminal events during fallback transitions.
- Enhanced error recovery to reset output state when replaying buffered content over SSE connection.
- Added docs.rs scraper for extracting Rust crate documentation from rustdoc JSON, supporting modules, functions, structs, traits, enums, and other Rust items with intelligent caching.
- Implemented rustdoc JSON parsing with type rendering for complex Rust types including generics, lifetimes, trait bounds, and qualified paths.
- Added caching layer for rustdoc JSON with date-based versioning for 'latest' releases to reduce repeated fetches.
- Added automatic Ollama model capability detection via /api/show endpoint to discover reasoning and input modality support.
- Improved Kagi API error handling with structured error parsing for JSON and plain text response formats.
- Fixed Cerebras streaming compatibility by omitting stream_options.include_usage parameter.
- Simplified API key credential storage to always replace credentials instead of merging for non-minimax providers.
- Updated Kagi Search API key format from 'kagi_...' to 'KG_...' and clarified beta access requirement in provider description.
Fixes#326.
Fixes#321.
Fixes#298.
- Added kebabToCamel and normalizeKeys utility functions to convert kebab-case keys to camelCase recursively. Updated parseFrontmatter to normalize all parsed keys, ensuring consistent camelCase property access throughout the codebase. Updated all frontmatter key accesses to use camelCase notation (e.g., thinkingLevel, spdxId).
- Removed await from stream() calls that return async iterables directly.
- Removed await from synchronous utility functions like findApiKey(), resolvePythonRuntime(), and getToolPath().
- Removed await from prerenderMermaid() and findExaKey() calls that don't require awaiting.
- Added Kagi Universal Summarizer integration for URL and YouTube video summarization with fallback support.
- Exported `searchWithKagi` and `summarizeUrlWithKagi` functions from new shared `web/kagi` module for reuse across components.
- Changed HTML-to-text rendering priority to attempt Kagi summarization first before Jina, Trafilatura, and Lynx.
- Refactored Kagi search provider to use shared utilities from `web/kagi` module, reducing code duplication.
- Added `KagiApiError` exception class for Kagi API-specific error handling with optional status code tracking.
- Added `/login <provider>` command for direct OAuth provider login with automatic selector routing.
- Added Kagi API key authentication support as new OAuth provider across ai and coding-agent packages.
- Added optional `providerId` parameter to `showOAuthSelector()` for direct provider selection without manual input.
- Simplified web search result formatting by removing empty sections (Answer, Sources, Meta, Related).
- Fixed MCP tool schema dereferencing and Ajv validation warnings for non-standard format keywords.
- Refactored OAuth login/logout logic into extracted handler methods for improved maintainability.
buildAnthropicSystemBlocks was unconditionally adding cache_control to system
blocks for OAuth cloaking. This triggered the early-return guard in
applyPromptCaching, which bailed before placing cache_control on conversation
messages. Result: only the system prompt (~21K tokens) was cached; conversation
history was sent as fresh uncached input every turn, dropping cache hit rate
from ~98% to ~40%.
Fix: make cache_control in buildAnthropicSystemBlocks opt-in via a cacheControl
option. The main streaming path (buildParams) omits it since applyPromptCaching
handles all cache breakpoint placement. External callers like the Anthropic
search provider that bypass applyPromptCaching pass cacheControl explicitly.
The guard in applyPromptCaching now only checks messages (for external callers
that pre-place breakpoints), not system/tool blocks.
Also removes dead hasCacheControlInBlocks function.
Introduced by: e7daffdda (feat: Claude fingerprint hardening)
Verified: cache hit rate restored from 20.8% to 99.3%.
- Add KagiProvider implementation (Search API v0)
- Register kagi in SearchProviderId type, provider map, and fallback order
- Add kagi to tool schema enum, SearchParams type, and settings UI
- Register KAGI_API_KEY in service provider map
- Update CHANGELOG
- Added exponential backoff retry mechanism with configurable max retries for transient failures. Implemented rate limit budget tracking to prevent excessive delays on 429 responses. Fixed HTTP status propagation to prevent network errors from being retried when explicit HTTP failures occur.
can1357