- 33d66643d removed the process-local URL response cache (#5803) but left
two fetch-kagi-toggle tests asserting the old reuse contract.
- Deleted the obsolete repeated-read reuse test (refetch behavior is
covered by fetch-raw-mode and search-url-paths regressions) and kept
the offset/limit selector contract without the no-network assertion.
Migrate 203 test files (356 call sites) from fs.rm/fs.rmSync to
removeWithRetries/removeSyncWithRetries to reduce EBUSY test failures
on Windows. removeWithRetries is now exported from @oh-my-pi/pi-utils.
The migration uses a regex-based approach that:
- Replaces fs.rm(path, { recursive, force }) → removeWithRetries(path)
- Replaces fs.rmSync(path, { recursive, force }) → removeSyncWithRetries(path)
- Replaces fs.rm(path) → removeWithRetries(path) (no options)
- Skips fs.rm/fs.rmSync inside template literals (bun --eval scripts)
- Adds imports to existing @oh-my-pi/pi-utils import or creates new one
- Removes unused fs imports where fs.rm was the only fs usage (4 files)
Completes the injectable-fetch transport wiring (15.10.8) that the feature
left half-done, fixing the deterministic CI test failures:
- compaction.compact() rebuilt summaryOptions field-by-field but dropped
`fetch`, so the injected transport never reached
requestOpenAiRemoteCompaction / generateSummary's remote path. Thread it.
- Read-tool URL pipeline had no fetch seam: renderHtmlToText gained a
fetchOverride param but renderUrl/ToolSession never carried one, so the
jina/parallel reader backends always used global fetch. Add
ToolSession.fetch -> renderUrl -> renderHtmlToText (defaults to global).
- searchWithParallel mirrored extractWithParallel but missed the fetch
option; add it.
- Repair tests whose deleted hookFetch interceptors were never replaced
with a FetchImpl seam (fetch-kagi-toggle, web-search-parallel,
issue-970 discovery).
- Update issue-1746 POSIX case to the #2154 preserved-scrollback contract:
unknown-viewport streaming deferral is now platform-independent.
- Added optional FetchImpl fields to compaction, proxy, AI, coding-agent, and mnemopi options.
- Threaded injected fetch implementations through OAuth, discovery, and search/LLM request flows.
- Removed exported hookFetch utility and its package entrypoint from utils.
- Replaced global-fetch test monkeypatching with per-test FetchImpl mocks across test suites.
- Replaced `providers.parallelFetch` with a new `providers.fetch` enum in settings and added migration cleanup for the legacy key.
- Updated `renderHtmlToText` to follow configured reader preference with ordered fallback attempts and remote-reader timeout handling before local conversion.
- Updated YouTube and fetch tests to use `providers.fetch` and cover Jina-first stall fallback behavior.
- Removed export leakage by demoting many helper and const symbols to module-local scope.
- Renamed underscore-prefixed internals and cache fields, then updated related references and `satisfies never` checks.
- Deleted obsolete logic branches and helpers, including harmony-stream interruption flow and unused benchmark runtime helpers.
- Updated Biome config and manifests by broadening lint coverage and removing an unused `@napi-rs/cli` dev dependency.
- Adjusted tests and utilities to use renamed test helpers and remove redundant private test-only helpers/locals.
- Expanded read tool range calculations to include optional leading and trailing context lines around user-requested offsets and limits.
- Added shared context range expansion logic so line-slice and streaming reads return anchor-safe windows while preserving existing truncation behavior.
- Updated read-tool tests to assert boundary-offset, limit, and combined offset-limit reads now include the expected ±3 lines of context.
- Updated read schema, path utilities, and dispatch to parse selectors from :raw/:L suffixes on path, removing standalone sel usage.
- Changed truncation/error notices to continue with :<nextOffset> and :1 guidance for read and sqlite pagination.
- Added summarizeCode support with tree-sitter summaries, read.summarize settings, and N-API Summary types/exports.
- Added tests and docs updates for path-embedded selectors, summary behavior, and explicit raw/offset SQL/read cases.
- Canonicalized file and CLI defaults from `read` to `open` across tool registration and prompts.
- Added `resolveToolAlias()` and applied alias-normalized tool selection so legacy `read` maps to `open`.
- Updated runtime, UI, and export layers to treat `open` as first-class while preserving `read` compatibility.
- Renamed read prompt docs to `open.md`/`open-chunk.md` and refreshed system guidance to recommend `open`.
- Updated tool-related tests and expectations from `read` to `open` (including test fixtures and aliases).
- Added support for embedded URL selectors with `:raw` and `:L#-L#` line range syntax in read command.
- Implemented `parseReadUrlTarget()` function to parse and validate URL read targets with line range support.
- Updated read CLI to delegate URL inputs through read tool pipeline instead of treating as local file paths.
- Added comprehensive test coverage for URL selector parsing and CLI URL delegation.
- Refactored URL handling in read tool to use structured `ParsedReadUrlTarget` object.
- Consolidated fetch tool into read tool with URL reading capability and caching support.
- Removed standalone fetch tool from all agent prompts and CLI documentation.
- Extended read tool schema with timeout and raw parameters for URL fetch control.
- Added URL caching mechanism to prevent redundant network requests during read operations.
- Refactored fetch module from class-based tool to standalone executeReadUrl function.
- Updated read tool documentation to describe multi-purpose capabilities including web pages, GitHub, Stack Overflow, Wikipedia, Reddit, NPM, arXiv, blogs, and feeds.
- Replaced Python-based markitdown CLI with native markit-ai library for document and notebook conversion.
- Added support for converting Jupyter notebooks (.ipynb) to markdown via markit-ai integration.
- Created markit utility module with file/buffer conversion wrappers and standardized error handling.
- Removed markitdown from Python tools manager and eliminated external CLI dependency.
- Updated fetch, read, and scraper tools to use new markit conversion API across codebase.
- Added test coverage for ipynb file conversion through markit utility.
- Migrated test mocking from bun:test mock API to vitest spyOn pattern across 9 test files.
- Extracted mock setup logic into reusable helper functions with Symbol.dispose cleanup pattern.
- Replaced manual beforeEach/afterEach and try-finally blocks with TypeScript 5.2 using declarations.
- Removed 178 lines of boilerplate mock initialization and restoration code from test suite.
- Added Parallel AI provider integration for web search with fast and research modes.
- Added Parallel extract API for URL content and YouTube video extraction with fallback support.
- Added /login parallel command and PARALLEL_API_KEY environment variable authentication.
- Added providers.parallelFetch configuration setting to control Parallel extract usage.
- Integrated Parallel provider into web search priority order between Exa and Kagi.
- Updated HTML-to-text and YouTube scrapers to prefer Parallel extract over fallback providers.
- Removed Kagi Universal Summarizer integration from fetch tool and YouTube scraper.
- Removed `fetch.useKagiSummarizer` configuration setting from settings schema.
- Simplified renderHtmlToText() and renderUrl() functions by removing Kagi summarization fallback logic.
- Fixed indentation inconsistencies in test files from tabs to spaces.
- Replaced vi.spyOn fetch mocking pattern with hookFetch utility across 5 test files.
- Migrated fetch mocks to use resource management with 'using' keyword for automatic cleanup.
- Removed @ts-expect-error comments related to fetch.preconnect type issues.
- Updated ts-hook-fetch rule documentation with expanded patterns and clearer lifecycle guidance.
- Moved llms.txt endpoint discovery to fallback strategy when rendered page content is low quality, prioritizing page-specific content over site-wide files.
- Enhanced llms.txt endpoint detection to scope candidates to the requested URL path, searching section-specific files before site-wide ones.
- Replaced getOrigin() with buildLlmEndpointCandidates() to generate path-scoped endpoint candidates with depth-based fallback strategy.
- Updated tryLlmEndpoints() to accept full URL and return endpoint metadata alongside content for better fallback tracking.
- Added 2 integration tests validating section-scoped llms.txt discovery and preference for rendered content over site-wide files.
* idiomatic rust fixes
* idiomatic rust fixes
* display an image if we are fetching an image
* MIME type strictness
* codex nagging me
* codex nagging
* handoff instead of compaction as context filled strategy and surfacing
* handoff instead of compaction as context filled strategy and surfacing p2
* handoff instead of compaction as context filled strategy and surfacing p3
* handoff instead of compaction as context filled strategy and surfacing p4
* handoff instead of compaction as context filled strategy and surfacing p5
* handoff instead of compaction as context filled strategy and surfacing, fixes
* failing fetch test from the fetch tool updates
* handoff focus prompt skeleton
* handoff focus prompt skeleton p2
* fetch bugs
* further codex improvements
* further codex improvements
---------
Co-authored-by: Brit <lol@no.com>
* idiomatic rust fixes
* idiomatic rust fixes
* display an image if we are fetching an image
* MIME type strictness
* codex nagging me
* resize so we do not blow up the terminal we are in
* codex nagging
* codex nagging
* codex nagging
* codex nagging
* codex nagging
---------
Co-authored-by: Brit <lol@no.com>
- Added `fetch.useKagiSummarizer` configuration setting to toggle Kagi Universal Summarizer usage in fetch tool.
- Updated fetch tool to conditionally apply Kagi summarization based on configuration setting.
- Added comprehensive test coverage for Kagi summarizer toggle behavior with mocked dependencies.
can1357