Commit Graph

183 Commits

Author SHA1 Message Date
can1357 63b39b7040 feat(coding-agent/utils): expanded tar extraction and validation logic
- Added archive and member size assertion limits along with path byte-length checks for PAX and GNU metadata targets.
- Added support for global PAX attributes, old-GNU name records, and signed GNU base-256 numeric header fields.
- Updated archive reading in WriteTool to accept a filesystem path instead of buffered bytes.
- Added test coverage for signed GNU base-256 values, PAX extensions, overlong path rejections, and oversized archives.
2026-08-12 03:04:17 +02:00
can1357 a4d8860a6c feat: added google reasoning controls mcp stream resumption and tar support
- Added Google provider thinking configuration parameters and force-reasoning-off controls.
- Implemented MCP SSE stream resumption using Last-Event-ID and `SSEResumeError`.
- Added support for TAR old-GNU sparse extension blocks, path length checks, and archive entry overrides.
- Restricted external thinking support to specific models and added semver fallback parsing.
2026-08-12 02:32:45 +02:00
can1357 7e2663360f fix(read): bounded tar directory-alias resolution and resolved aliased link targets
- Capped directory-alias rewrites per lookup (ELOOP-style, 40) so a directory
  symlink targeting its own subtree (a -> a/b) throws a catchable ToolError
  instead of looping forever growing the path.
- Deferred pending tar link resolution while any directory on the target path
  is itself an unresolved link, and rewrote targets through established
  directory aliases before the exact-path lookup, so file symlinks routed
  through directory aliases materialize instead of dangling.
- Regression tests reproduce both shapes: pre-fix the aliased symlink read
  failed with 'cannot be materialized' and the self-cycle read hung.
2026-08-12 02:02:33 +02:00
can1357 c63b475f47 Merge PR #8260: fix(read): avoid libarchive for tar reads (@roboomp) 2026-08-12 01:53:49 +02:00
roboomp f30a60b797 fix(read): resolved archive-root symlink targets
Symlink targets that normalize to the archive root (current -> ., dir/up -> ..) now resolve as directory aliases to the root instead of being treated as dangling links. The lookup normalizer distinguishes an empty root target from an escaping target, and ArchiveReader treats a resolved-empty path as the root directory.

Fixes #4774
2026-08-11 22:06:41 +00:00
roboomp 1df67cc667 fix(read): honored GNU sparse PAX member names
GNU 1.0 sparse PAX entries now list under GNU.sparse.name with GNU.sparse.realsize as the displayed size, so root listings no longer expose the internal GNUSparseFile path and reads of the real name reject as sparse instead of reporting the member missing. The on-disk header size still drives offset advance and truncation.

Fixes #4774
2026-08-11 21:57:06 +00:00
roboomp 40ce90866e fix(read): resolved tar directory symlinks lazily
Kept directory symlinks as one alias node and rewrote requested paths through aliases in ArchiveReader instead of cloning every target descendant during indexing.

Full archive materialization now fails explicitly on directory aliases rather than expanding them without a bound.

Fixes #4774
2026-08-11 21:47:45 +00:00
roboomp e1360bbb9d fix(read): preserved symlinks and rejected partial tar indexes
Resolved safe file and directory symlinks against indexed members, while retaining dangling links as listed nodes that fail explicitly when read or materialized.

Required fully buffered tar inputs to reach an end-of-archive zero block so truncated downloads cannot expose partial listings.

Fixes #4774
2026-08-11 21:37:18 +00:00
roboomp 861d5f5909 fix(read): preserved tar hard-link members
Resolved hard-link targets after indexing so forward links and chains reuse the referenced member's storage and size. Missing, directory, or cyclic targets now surface catchable archive errors instead of silently dropping paths.

Fixes #4774
2026-08-11 16:11:16 +00:00
roboomp b49e41fc7a fix(read): rejected non-tar gzip payloads while indexing
A gzip stream whose decompressed payload never presents a complete tar header or terminating zero block (a plain .txt.gz, or a tar truncated before the first header) now raises a catchable ToolError instead of returning an empty index rendered as '(empty archive directory)'. fetch falls back to binary rendering.

Fixes #4774
2026-08-11 16:02:00 +00:00
roboomp 1588f79ecf fix(read): rejected truncated tar members while indexing
A member header declaring more bytes than remain in the buffer now throws a ToolError during indexing instead of being listed as a valid entry that only fails on read.

Fixes #4774
2026-08-11 15:52:07 +00:00
roboomp 5892cd1ea4 fix(read): avoided libarchive for tar reads
- Parsed tar and tar.gz members in-process with bounded gzip inflation.

- Added UTF-8 ustar-prefix coverage for the minimal libarchive crash shape.

Fixes #4774
2026-08-11 15:41:51 +00:00
can1357 cc2265f681 feat: use simple form replace when replace is the edit mode 2026-08-03 01:00:51 +02:00
can1357 8f6305f772 fix(tools): preserve read continuation notice when spilling 2026-07-31 19:14:47 +02:00
Wolfgang Schoenberger 5f9558d17a fix(tools): keep truncation metadata when spilling read output
The artifact spill wrapper dropped the read truncation metadata, so a
spilled oversized read lost its next-offset pagination hint, and an
artifact:// read of an already-spilled result was spilled a second time.
Preserve the existing truncation metadata and skip re-spilling artifact
reads.
2026-07-30 16:25:48 -07:00
Wolfgang Schoenberger 9a6a1b40be fix(coding-agent): spill oversized read results to artifacts 2026-07-30 15:04:52 -07:00
can1357 bce36396ea test(coding-agent): stabilized auto-background output assertion
- Asserted early command output in the completed job delivery instead of the timing-dependent 10 ms background notice.
2026-07-28 04:11:28 +02:00
Kormákur 21a7725a09 feat: lift subagent async/auto-background limits via owner-routed delivery and quiescence
Three-piece architecture so subagents inherit async.enabled and
bash.autoBackground.enabled instead of having both force-disabled:

- Owner-routed delivery: AsyncJobManager gains registerDeliverySink /
  waitForOwnerJobs; every AgentSession registers a sink for its own agent
  id, so background job results inject into the owning agent's run.
  Owned deliveries with no live sink dead-letter (result retained on the
  job row) instead of misrouting into the first top-level session.

- Quiescence barrier: a subagent's final yield with owner jobs still
  running/undelivered is a scheduling pause, not completion. The run
  driver notifies the model once (hub wait/cancel), settles owner work,
  and folds results in as async-result follow-ups; teardown cancels and
  awaits surviving jobs before isolation worktree capture/cleanup.

- Steering soft channel: queued steering no longer hard-aborts
  non-interruptible tools; it aborts interruptible waits and raises a
  cooperative ToolCallContext.steeringSignal. The mid-batch watch runs
  for every batch, and auto-backgroundable bash backgrounds itself on
  steer so incoming messages inject promptly with no work lost.
2026-07-20 15:00:37 +00:00
can1357 29aefbca4a test: restored read-tool context-expansion contract tests
The exact-bounds rewrite matched reverted PR #5812; the ±context expansion
(1 leading + 3 trailing line) around explicit selectors is intended behavior
so edit anchors at range boundaries stay fresh.
2026-07-18 22:05:14 +02:00
can1357 32dc282517 test: aligned read-tool and logout tests with landed contracts
- tools.test.ts still asserted the pre-#5812 ±context expansion for offset/
  limit and archive-entry reads; updated to the exact-bounds contract.
- selector-controller-logout.test.ts mocked the old modelRegistry.refresh;
  #5786 switched logout to a provider-scoped refreshProvider(id, 'online'),
  so the mock never resolved and the test timed out.
2026-07-18 21:58:03 +02:00
roboomp 22a8524058 fix(tools): recognized zip-family archives and pruned unconvertible extensions
- Treated ZIP-based .jar/.war/.ear/.apk as zip archives in archiveFormatFromPath and parseArchivePathCandidates so read/write member access works.
- Shared one archive-extension alternation between format detection and path splitting to stop them drifting.
- Derived the markit convertible-extension set from a single source of truth (utils/markit) matching the registered converters (pdf/docx/pptx/xlsx/epub), dropping legacy .doc/.ppt/.xls/.rtf that had no converter and only produced Unsupported format errors.
- Updated read/write tool prompts to document the zip-family extensions.

Fixes #5808
2026-07-17 08:04:10 +00:00
can1357 206cb93fa1 test(bash): asserted timeout settles as flagged result
Local bash timeouts resolve with details.timedOut and a warning render
since #5546; ACP retains rejection semantics.
2026-07-17 05:46:57 +02:00
can1357 a049ed4ca7 merged PR #5422: fix(tools): stop column cap from faking window truncation 2026-07-16 03:32:03 +02:00
can1357 5ff277349c refactor(coding-agent): consolidated tool surface onto xd:// devices and hub
- Added the `xd://` virtual device protocol (`internal-urls/xd-protocol.ts`, `tools/xdev.ts`): tools declaring `loadMode: "discoverable"` are unmounted from the request tools array and driven via `read xd://` (list/docs+schema) and `write xd://<tool>` (execute), gated by the `tools.xdev` setting (default on) and inlined into the system prompt.
- Merged the `irc`, `job`, and `launch` tools into a single `hub` tool (`tools/hub/`, `async/job-manager.ts`): messaging keeps `send`/`inbox`/`list`, job control maps to `wait`/`cancel`/`jobs`, process supervision keeps `start`/`logs`/`stop`/`restart`/`describe` with `ps`, and the unified `wait` races background jobs against peer messages; SDK `IrcTool`/`JobTool`/`LaunchTool` are replaced by `HubTool`.
- Removed the hidden `resolve` tool in favor of the `xd://resolve`/`xd://reject`/`xd://propose` resolution devices, auto-including `write` whenever a deferrable tool or plan mode is present.
- Removed the BM25 tool-discovery system: the `search_tool_bm25` tool, the `tool-discovery` module, the `tools.discoveryMode`/`mcp.discoveryMode`/`mcp.discoveryDefaultServers`/`tools.essentialOverride` settings, per-tool MCP selection, and the `mcp_tool_selection` message type.
- Unified tool presentation on `ToolLoadMode` (`essential`|`discoverable`), replacing the custom-tool `xdev?: boolean` opt-out; custom, extension, MCP, RPC host, image-generation, and TTS tools now default to `discoverable`, and added a `satisfies` predicate to `SoftToolRequirement`.
- Removed the standalone `ssh` command tool and `ssh/ssh-executor` (the `ssh://` read/write/search protocol stays), and made `--tools` address hidden built-ins.
- Updated collab-web to render `xd://` dispatches and `hub` op families, dropped the `search_tool_bm25`/`ssh`/`report-finding` renderers, refreshed tool docs and prompts, and migrated the affected tests and changelogs.
2026-07-15 15:16:29 +02:00
can1357 a9c038818d feat(tools): removed separate selector args from read and grep APIs
- Removed `selector`/`sel` arguments from read and grep tool schemas and related execution arg handling.
- Reworked read and grep path processing to parse line selectors from `path` suffixes instead of separate fields, including inline range propagation.
- Updated delegation and execution call paths (including JS/Python preludes and executor tests) to pass selectors embedded in `path`.
- Updated read/grep prompt docs and changelog for the breaking inline-selector API, and removed obsolete selector-specific tests and expectations.
2026-07-15 00:04:02 +02:00
roboomp f697837659 fix(tools): stopped column cap from faking window truncation
Per-line column cap trims individual lines with a `…` marker but does not
truncate the output window. OutputSink.dump() nonetheless set truncated=true
whenever a line was capped, and truncationFromSummary then reported a byte
tail-window truncation, appending a bogus "Showing lines X-Y of Z (…B limit).
Read artifact://N for full output" footer even though every line was shown.

- OutputSink no longer flips #truncated on column-cap-only drops.
- OutputSummary carries columnMax; truncationFromSummary surfaces it as the
  "Some lines truncated to N chars" limit notice regardless of window state.

Fixes #4735
2026-07-14 16:27:06 +00:00
can1357 87a64b2f6a feat(coding-agent): improved background job lifecycle and display
- Stop propagating real-time updates for backgrounded Bash jobs to avoid UI flickering once a job enters the background.
- Refine background task tracking in `EventController` to distinguish between persistent background tasks and transient backgrounded Bash commands.
- Update UI rendering to display cleaner background job metadata in the footer instead of inline text notices.
2026-07-13 00:54:51 +02:00
can1357 2553495d00 fix(tools): treated empty read/grep selector fields as omitted
Models emit optional string args as empty strings; since ff3b0c795
(#4622) read/grep rejected a present-but-empty selector as invalid
instead of behaving like an omitted one. Normalize empty and
whitespace-only selector params to undefined before validation.
Adopted from PR #4881 minus unrelated prompt churn.

Fixes #4879
2026-07-09 18:27:20 +02:00
Christian Stewart 78d4978c51 fix(bash): support disabled command deadlines
Treat timeout 0 as an explicit no-deadline contract across the bash tool, executor, async job, and PTY paths.

Signed-off-by: Christian Stewart <christian@aperture.us>
2026-07-05 16:03:37 -07:00
can1357 95b91c7f73 feat(coding-agent/tools)!: replaced paths arrays with path strings
- Replaced `grep`, `glob`, and `ast_grep` `paths` inputs with optional single `path` strings while preserving default workspace-root behavior.
- Added shared `toPathList` normalization for legacy arrays and JSON-encoded arrays across tool execution and TUI renderers.
- Updated prompts, fixtures, shims, transcript summaries, and tests to send and display the new `path` argument.
- Updated collab-web search tool cards to read `path` while falling back to legacy `paths` for historical transcripts.
- Recorded the contiguous coding-agent changelog run for the tool-path breaking change and adjacent TTS entries.
2026-07-02 08:30:33 +02:00
can1357 e8090bb48a feat: introduced binary file detection to prevent encoding corruption
- Introduced `isProbablyBinary` utility to sniff file headers for NUL bytes or invalid UTF-8 sequences.
- Updated `ReadTool` to use the binary sniffer, preventing mojibake corruption in output when reading non-text files.
- Refined `file-mentions` auto-reads to skip binary files and mark them as `binary` in the message transcript.
- Added comprehensive unit tests for binary detection logic, covering NUL bytes, truncated multibyte characters, and path-based file sniffing.
2026-06-30 02:59:41 +02:00
can1357 ae1650d689 refactor: renamed search and find tools to grep and glob
- Renamed the `find` and `search` tools to `glob` and `grep` respectively across the codebase to improve command clarity.
- Implemented full-stack support for the renamed tools, including CLI arguments, system prompts, SDK exports, and tool registration.
- Added automated migration logic in `settings` to transform legacy `find` and `search` configuration keys to their new equivalents.
- Updated the `collab-web` renderer registry to ensure backwards compatibility with legacy tool outputs.
2026-06-27 00:57:55 +02:00
can1357 a6bdef85a2 feat(ai): added support for reasoning items in replay sanitization
- Added `sanitizeOpenAIResponsesReasoningItemForReplay` to process reasoning-type items by stripping unique identifiers and filtering properties.
- Updated the main sanitization utility to route reasoning items through the new logic.
2026-06-23 08:18:59 +02:00
can1357 2787b6dff7 chore: update changelogs 2026-06-23 08:18:29 +02:00
oldschoola a436dfbbeb fix(test): migrate fs.rmSync to removeSyncWithRetries in 5 more test files
Batch migration of 13 fs.rmSync calls to removeSyncWithRetries across:
- core/apply-patch.test.ts (4 calls)
- bash-executor.test.ts (4 calls)
- tools.test.ts (2 calls)
- compaction-hooks.test.ts (1 call)
- compaction-thinking-model.test.ts (2 calls)

Also exports removeSyncWithRetries from @oh-my-pi/pi-utils as a
standalone function for tests that manage their own temp dirs.

All tests pass: 139 pass, 0 fail across the 5 migrated files.
2026-06-19 17:30:58 -07:00
can1357 2de02f2119 Merge PR #3019: fix: Windows test failures — path handling, EBUSY, SQLite handle leaks (@oldschoola) 2026-06-19 17:17:04 +02:00
can1357 67f6518e42 feat: enhanced tool robustness, improve authentication flow, and update API parameters
- Implement JSON repair and strict argument validation to sanitize raw payloads and redact sensitive information from agent event logs.
- Add automatic authentication fallback for benchmark model resolution to ensure consistent performance testing across providers.
- Refactor search tool API parameters by replacing `i` with a case-sensitive `case` boolean flag for clarity.
- Update session history formatting to ensure empty objects are consistently serialized as `{}` instead of empty strings.
2026-06-19 16:46:07 +02:00
oldschoola 14252e71cb fix: Windows test failures — path handling, EBUSY, SQLite handle leaks
Fix all Windows-specific test failures caused by path handling problems
and EBUSY errors from unclosed SQLite database handles.

Root causes fixed:
1. POSIX path assumptions: replaced hard-coded file:///tmp, /repo, etc.
   with pathToFileURL/path.resolve/path.join computed expectations
2. shortenPath() now normalizes backslashes to forward slashes after ~
   and respects home directory boundaries
3. HistoryStorage.resetInstance() leaked its Database — added #close()
   that finalizes all prepared statements and closes the DB
4. AgentStorage gained the same resetInstance()/#close() pattern
5. SqliteAuthCredentialStore.close() leaked one-off prepared statements
   from inline this.#db.prepare() calls — wrapped each in try/finally
6. model-cache.ts used a process-global DB even for custom dbPath —
   now opens/closes per-call via withModelCacheDb
7. createAgentSession leaked AuthStorage on construction failure —
   added ownsAuthStorage cleanup in catch block
8. MnemopiBackend.removeDbFiles() now truly best-effort (catches errors)
9. TempDir retry window expanded from 4x10ms to 40x25ms
10. TempDir prefix convention: non-@ prefixes created dirs relative to
    cwd instead of os.tmpdir() — all test temp dirs now use @ prefix
11. Shell-escaped interpolated paths in bash tool tests
12. git core.autocrlf false in autoresearch test repo init

All 522 previously-failing Windows tests now pass.
2026-06-18 21:32:38 -07:00
can1357 38d1d3a2f5 feat(coding-agent): vendor relevant parts of markit 2026-06-18 19:11:22 +02:00
can1357 6385afdfb7 test(coding-agent): replaced Bun.sleep and wall-clock timing
- Replaced Bun.sleep and wall-clock timing with fake timers (vi.useFakeTimers), release gates, and deterministic polling across 15+ test files to eliminate flakiness and improve speed.
- Consolidated per-test fixture setup into beforeAll/afterAll lifecycle hooks across 20+ test files, reducing redundant initialization and improving test performance by reusing shared immutable fixtures.
- Stubbed network calls in ModelRegistry and test discovery to prevent unintended outbound requests during test execution.
- Replaced subprocess-based test coordination (file markers, Bun.sleep polling) with in-memory fakes (FakeWebSocket, FakeLspServer, VirtualClock) for deterministic, fast test execution.
2026-06-15 11:48:55 +02:00
can1357 f9a8aa1d96 fix(coding-agent): fixed queued steering drain after aborted and interrupted tool turns
- Queued steering now drains after session settlement, so aborted auto-continued turns no longer leave queued messages stranded.
- Resumable-state detection now treats tool-result messages as resumable so continue can process queued steering after an interrupted tool execution.
- Regression tests were added for queued steer draining after abort and after an interrupted tool result.
2026-06-13 21:25:48 +02:00
can1357 28df38ed37 feat(agent): added useless-result tagging and compaction dropping of tool outputs
- Added optional `useless` flags to tool result types and payload builders.
- Added `pruneUseless` and `dropUeless` options to control uneventful result pruning.
- Changed compaction and shake passes to prune or ignore non-error useless tool results.
- Changed conversation serialization to omit useless toolCall/toolResult pairs from output.
- Added coverage for useless tagging, pruning, and serialization behavior.
2026-06-12 16:26:38 +02:00
can1357 5406edeed0 refactor(coding-agent): made the job tool always available and dropped async-job gating
Removes isBackgroundJobSupportEnabled and JobTool.createIf; the tool is now registered unconditionally via `new JobTool(s)`. `async.enabled` now gates async bash commands only — the task tool runs asynchronously regardless. Deletes the async/support module and its barrel re-export.
2026-06-10 17:44:31 +02:00
can1357 82225e4444 fix(coding-agent): closed vault write approval bypass and fixed interaction tools
vault writes now rated write-tier and plan-mode enforced; .tar.gz rewrites keep gzip, are atomic, and write through symlinks; CRLF conflict detection works; conflict twins only invalidated when truly stale; ask discloses timeout auto-selection in result and transcript; todo rejects duplicate ids and stops persisting half-applied batches; auto-generated guard validates against mtime+size; ACP writes run post-write bookkeeping; irc errors set isError.
2026-06-10 01:27:17 +02:00
can1357 ee4df538f0 fix(tools): resolved search-path parsing and local sandbox checks
- Set `FindTool` to disable recursive glob traversal so `dir/*` stays shallow.
- Added `parseSearchPathPreferringLiteral` to prefer literal paths like `apps/[id]/page.tsx` when they exist.
- Updated `resolveToolSearchScope` to reject external URLs with a clear `read` usage error.
- Expanded plan-mode sandbox checks to allow absolute paths inside the local artifact root.
2026-06-08 02:04:59 +02:00
can1357 20d19e8002 test: replaced blind sleeps with shared fixtures and condition polling
- Shared immutable model registries and auth storage via beforeAll/afterAll.
- Swapped fixed-delay settle sleeps for predicate polling and signals.
- Stubbed network/timers to drop wall-clock waits in registry and history tests.
- Added resetDisplay invalidation tests and startup-timing breakdown lines.
2026-06-06 22:09:04 +02:00
can1357 492b454141 fix(archive): read zip central directory without inflating members
- Parsed zip metadata via central directory and lazy ranged reads.
- Inflated member contents only when a specific entry is read.
- Prevented large or corrupt zips from freezing directory reads.
2026-06-05 17:38:24 +02:00
can1357 1951081ad5 test(coding-agent): removed redundant AsyncJobManager.setInstance calls
- Relied on session-injected asyncJobManager instead of global instance.
2026-06-05 15:57:10 +02:00
can1357 07998fcc09 test(coding-agent): injected asyncJobManager via test session deps
- Passed asyncJobManager through test tool session dependencies.
- Replaced AsyncJobManager.setInstance with explicit session injection.
2026-06-05 15:55:25 +02:00
can1357 2eb39bae34 test(coding-agent): stabilized coding-agent tests with safer teardown and timing tweaks
- Updated interactive-mode plan review tests to capture shared fixtures, clear references, and run cleanup with explicit garbage collection before disposal.
- Increased the MCP HTTP transport test connection timeout from 200ms to 1,000ms.
- Adjusted the tool streaming command delay and tightened a start-pending-submission spy type in tests for better stability and type accuracy.
2026-05-31 04:25:14 +02:00