Brace expansion joined its results with spaces and re-parsed them as a single word, so tilde-at-word-start only fired on the leading element. Now each brace element is expanded as its own word, so `~/project/{a,b}` expands both tildes instead of leaving a literal `~/project/b`.
Fixes#5819
- Delayed reader cancellation so pipeline consumers can flush after producers are terminated.
- Kept the JavaScript watchdog behind bounded native timeout cleanup.
- Added native and executor regressions for timeout-time output draining.
Fixes#5316
- Rewrote logical /dev/fd operands to live OS descriptors before invoking in-process uutils.
- Added regression coverage for diff reading two process substitutions.
Fixes#5557
- Updated `run_fd_sync` to handle `io::ErrorKind::BrokenPipe` by returning exit code `141` without writing an error message.
- Added a regression test that verifies `fd` exits with `141` and empty stderr when stdout is closed early (e.g. `fd ... | head` scenario).
- Recorded the broken-pipe behavior fix in `packages/coding-agent/CHANGELOG.md`.
- Integrated 17 new coreutils-based shell builtins including `diff`, `date`, `ln`, `stat`, `seq`, `touch`, and others.
- Refactored vendored utilities to execute as in-process shell builtins by routing I/O, environment access, and path resolution through `pi_uutils_ctx`.
- Disabled process-level modifications (e.g., clock setting, hostname modification) to ensure safety and scope adherence within the shell environment.
- Implemented shell-specific features such as cancellation polling, custom exit code management, and efficient output streaming for all new builtins.
- Added base64, checksum utilities (md5, sha1, sha224, sha256, sha384, sha512, b2sum), path utilities (basename, dirname), and text processing tools (cut, tee, tr, paste, comm) to the shell.
- Registered new utility builtins in crates/pi-shell/src/coreutils.rs and crates/pi-shell/src/shell.rs.
- Updated Cargo.toml to include the necessary dependencies for the new core utilities.
- Removed the bash command fixup system and associated logic from both the Rust and TypeScript components.
- Deleted the redundant fixup module and its exported implementations.
- Updated technical documentation and configuration settings to reflect the removal of pre-execution bash command rewriting.
The non-PTY bash streaming bridge queued every decoded chunk into
flume::unbounded and fired ThreadsafeFunction callbacks NonBlocking
with no budget, so a producer outrunning the JS event loop grew the
native queue (and the napi queue behind it) without bound — measured
33.5 MB queued for a 32 MiB stream with a stalled consumer, and
multi-GB RSS on longer runs. The downstream OutputSink caps sit after
the N-API boundary and cannot bound either queue.
Bound the pipeline end to end without dropping data:
- pi-natives: bridge_chunks now creates flume::bounded(64) and the
drain task (extracted as pump_chunks) awaits on_chunk.call_async per
coalesced <=64 KiB batch, so at most one batch sits in the napi
queue and the JS event loop's real consumption rate backpressures
the whole pipeline. If the JS side is gone, the pump exits and
drops the receiver so senders fail fast.
- pi-shell: emit_chunk sends with send_async().await — a full bridge
queue parks the pipe reader, which parks the child on its
stdout/stderr pipe (ordinary pipe backpressure) instead of
buffering; a disconnected receiver fails immediately so child pipes
always keep draining.
Unlike a drop-after-cap design, every byte still reaches JS: the
rolling tail view, lossless [raw output: artifact://…] capture, and
totalBytes accounting keep working for outputs past the display cap.
E2E (darwin-arm64 addon): 32 MiB through a JS callback stalling 1 ms
per call — lossless, 472 coalesced callbacks, peak RSS +21.8 MiB.
Fixes#4078
Addresses the third review on #4606: `record` guarded pruning with
`spawned.len() >= PRUNE_THRESHOLD`. Once the recorded vec stabilized
above the threshold with entries the sweep could not remove — a run
whose live children exceed the threshold, e.g. `for i in {1..1000}; do
sleep 60 & done` — every subsequent `record` re-entered `prune_exited`
while holding the registry lock. Each sweep is O(N) (a status probe per
entry, plus a Toolhelp descendant walk on Windows for exited roots), so
the per-spawn cost climbed to O(n²) even though the doc-comment
promised amortized O(1).
The registry now tracks a `next_sweep_at` watermark alongside the
recorded vec (both under one mutex — the two fields are always
mutated together). A sweep fires only when `spawned.len()` crosses
that watermark; every sweep rescheds the next fire `PRUNE_THRESHOLD`
further records away from the current post-sweep size. Sweep frequency
is now capped at one per `PRUNE_THRESHOLD` records regardless of live
set size, restoring true amortized O(1) per spawn.
`build_targets` resets the watermark after its own sweep so the
record-time schedule stays consistent with the post-cancel live set.
Added `spawn_registry_watermark_bounds_sweep_frequency`: fills the vec
past threshold with permanently-live entries, records another 20, and
asserts the vec grew by exactly 20 (no sweep modified it) and the
watermark did not advance. Existing tests updated for the collapsed
`state` mutex.
Fixes#4605
Addresses the second review on #4606: `prune_exited` retained an entry
while its process was running OR its process group was alive. On Windows
`process_group_alive` is always `false` (no process groups), so the
predicate collapsed to "root running" — when a brush-spawned root exited
after starting a child that stayed alive (a `pwsh`/shell command that
launches a helper and exits, an MCP stdio wrapper handing off to a
long-running server), pruning immediately dropped the pinned handle.
Dropping that handle closes the last thing keeping the root pid slot
reserved. Windows can then recycle the pid onto an unrelated process,
reintroducing the exact race #4605 closes. It also strands the leftover
child: the next cancellation wave has no root to walk descendants from,
so `signal_tree` never reaches it.
The retain predicate is now:
- root process still running → keep (all platforms);
- Windows-only: root exited but the pinned handle still probes at least
one live descendant via Toolhelp → keep, because the handle is what
guarantees the walk targets the original subtree (recycled pids would
not be reachable while the handle holds the slot);
- pgid still alive → keep (Unix only; Windows falls through).
Unix stays unchanged because a child reparented onto init keeps its
pgid, so `process_group_alive` already catches "root gone, descendants
alive" without a per-entry tree walk.
Fixes#4605
Addresses the review on #4606: pinning a stable `Process` per spawn is
correct against pid reuse, but a long-running shell command that spawns
many short-lived external processes (a bash loop invoking a binary per
iteration) would otherwise retain one owned OS handle per historical
spawn — a pidfd on Linux, a process `HANDLE` on Windows — until the run
ends. Under enough iterations that hits the per-process FD/handle limit
and starts breaking `Process::from_pid` (or any other file operation) for
the rest of the run.
`SpawnRegistry` now sweeps entries whose pinned process and process group
are both gone. The sweep runs opportunistically inside `record` once the
recorded vec crosses a small threshold (`PRUNE_THRESHOLD = 64`) and
unconditionally at the top of `build_targets`, so the retained handle
count is bounded by the current live tree rather than the historical
spawn count. Amortized cost per spawn stays O(1); a sweep is O(N) probes
of `Process::status` (non-blocking pidfd `poll` on Linux, `WaitForSingle
Object(_, 0)` on Windows), running at most once per `PRUNE_THRESHOLD`
records.
The previous identity-pinning regression test was rewritten to defend
the actual invariant — the pinned handle carries into `build_targets`
while the child is alive, and the entry is dropped (never re-opened by
pid) once the child exits. A new regression asserts pruning keeps
retained entries under `PRUNE_THRESHOLD` after 2×threshold spawns of
short-lived children.
Fixes#4605
`SpawnRegistry` previously stored only the raw pid reported by brush's
`SpawnObserver` hook and deferred `Process::from_pid` to `build_targets`
at cancellation time. Between the moment a bash-spawned child exited and
the moment cancellation fired, Windows could recycle that freed pid onto
an unrelated process — typically another `pwsh.exe` or `powershell.exe`
in a different Cursor terminal tab, since PowerShell is the parent shell.
`Process::from_pid` at kill time then opened the impostor, and
`signal_tree` walked the current Toolhelp snapshot for `ppid == root`
matches and `TerminateProcess`'d whatever subtree happened to live under
that recycled pid. That is the reporter's Variant A symptom: OMP crashing
kills unrelated PowerShell sessions.
The `SpawnObserver` impl now pins a stable `Process` handle *at spawn
time*, before any pid recycling window can open:
- Windows: an open process handle keeps the pid slot reserved for the
handle's lifetime (Raymond Chen's documented invariant), so the pid
cannot be reassigned while the registry holds a reference.
- Linux: the pidfd carries identity independent of the numeric pid.
- macOS: the recorded `(pid, start_tvsec, start_tvusec)` triple detects
impersonation on every subsequent access.
`TerminationTargets::add_process` accepts a pre-pinned handle and skips
the `Process::from_pid` re-open entirely, and `build_targets` no longer
consults the raw pid at all — an entry the observer failed to pin (the
child exited before we could open a handle) becomes a no-op instead of
racing pid reuse.
Fixes#4605
Propagated the native shell working directory in ShellRunResult so AgentSession can refresh cwd without running a hidden pwd command in the persistent shell.
Added regression coverage for cd plus a failing command followed by echo $?, proving cwd sync no longer overwrites the user's last shell status.
Fixes#3958
The two search()-level cancel tests (`fallback_walk_observes_cancel_flag`
/ `fast_walk_observes_cancel_flag`) pre-set the cancel flag before
invoking search(), but search() and try_search_fast each have a
pre-loop guard that breaks out on cancelled before either walker call
is invoked. That means both tests still pass with the walker call
sites reverted to no-op heartbeats, so they don't defend the fix.
Replace them with symmetric walker-level tests that drive the two
walker APIs fd uses:
- `cancel_heartbeat_aborts_collect_with_heartbeat` — the fallback
path's `collect_with_heartbeat` call; asserts WalkError::Interrupted
(was `cancel_heartbeat_aborts_walker_when_flag_is_set`, renamed for
API-specific clarity and given a filler-file seed so the outcome
shows a real drain on regression).
- `cancel_heartbeat_aborts_for_each_entry_with_heartbeat` — the fast
path's streaming API; asserts WalkError::Interrupted AND that the
visitor never received any entry.
Both fail against the pre-fix no-op heartbeat with WalkStatus::Complete
/ a fully drained WalkOutcome. Keep
`walk_completes_normally_when_cancel_flag_is_unset` as the positive
regression pin — that path does exercise search() end to end because
the outer guard passes with cancelled=false.
The in-process fd builtin passed no-op heartbeats to pi_walker for
both its gitignore-respecting fallback path (`collect_with_heartbeat`
in `search`) and its fast path (`for_each_entry_with_heartbeat` in
`try_search_fast`), so cancellation of a large or slow directory walk
was deferred until traversal completed. The shell wrapper flips the
shared `AtomicBool` cancel flag when the runtime cancellation token
fires and then awaits the blocking task; with no heartbeat hookup the
walker had no way to observe the flag mid-walk and kept collecting the
whole tree before the wrapper could return exit 130.
Introduce `cancel_heartbeat(&AtomicBool)` — the walker-level heartbeat
that returns `io::ErrorKind::Interrupted` when the flag is set — and
plug it into both walker calls. Both call sites recognize the resulting
`WalkError::Interrupted` alongside `cancelled` and break silently
instead of surfacing an `fd:` diagnostic on stderr; the shell wrapper
owns the user-visible exit code.
Regression cover: a walker-level test pre-sets the cancel flag and
asserts `collect_with_heartbeat(cancel_heartbeat(&flag))` surfaces
`WalkError::Interrupted` instead of collecting the tree; two
higher-level `search` tests exercise the silent break for both the
fallback and fast paths; a fourth pins the non-cancelled contract so
the added heartbeat can't stall normal searches. Neuter the helper to
a no-op and the walker-level test fails with the pre-fix `WalkOutcome`
showing every entry scanned — the exact bug the issue reports.
Fixes#3949
- Removed the ignore-based fallback walker implementation and its associated dependencies.
- Simplified the directory traversal to consistently use native scanners across all platforms.
- Eliminated `WalkError::Unsupported` and `WalkStatus::Unsupported` error handling across all dependent crates.
- Added a depth-first sorting fallback for native traversal and implemented recursion detection using a symlink stack.
- Replaced standard and tokio mpsc channels with flume channels across workspace crates to simplify thread synchronization.
- Swapped standard Mutex guards for parking_lot Mutexes to avoid manual lock poisoning handling and improve performance.
- Declared workspace-wide dependencies for flume and parking_lot in root and member Cargo manifests.
- Replaced custom fast-walk and fs-cache implementations in pi-natives with a new dedicated pi-walker library.
- Integrated the thread-safe, parallel pi-walker library across pi-natives, pi-shell, pi-uu-grep, and uu-find.
- Rewrote file search, fuzzy finding, and glob-matching logic to leverage pi-walker configurations and visitor traits.
- Optimized shell process tracking in pi-shell by replacing global descendant-diff logic with an isolated, per-run SpawnRegistry.
- Added parallel rayon-based walking and optimized fast paths for directory scanning and entry classification.
- Switch build profile to `panic = "unwind"` to allow catching panics in vendored uutils code.
- Add `run_caught` to wrap utility execution in `catch_unwind`, converting panics into non-zero exit codes.
- Update `pi-natives` to distinguish between recoverable utility panics and fatal process crashes, preventing recovered panics from appearing in user-facing crash reports.
- Added an `fd` command-line utility for searching the filesystem.
- Implemented file filtering capabilities for type, size, modification time, and ownership.
- Integrated directory traversal and ignore-pattern support for search optimization.
- Registered the builtin in the shell and added comprehensive integration tests for filtering and globbing.
- Implemented `rg` shell builtin using ripgrep libraries to enable file and directory searching.
- Added `RgSink` to handle output formatting, context, and vimgrep compatibility.
- Updated shell context and IO flags to support stdin as search input.
- Added integration tests for directory recursion, ignore filtering, and stdin handling.
- Added support for `--quiet` (`-q`) and `--line-regexp` (`-x`) to the `grep` builtin.
- Enabled short-circuiting behavior for `-q` to suppress output and return early on the first match.
- Configured exit status logic to prioritize successful matches over error states when using `-q`.
- Added integration tests to verify correct exit status codes and line anchoring behavior.
- Ensure `-exec` commands run in the shell current working directory rather than the host process CWD.
- Update operand path resolution in `find` matchers to use the display path, matching behavior expected by shell-integrated utilities.
- Add an integration test to verify path substitution and execution context for shell-integrated `find`.
- Introduced `pi-uutils-ctx` to manage thread-local I/O redirection, environment context, and path resolution for in-process utilities.
- Integrated a suite of vendored coreutils (cat, find, grep, head, ls, mkdir, mv, rm, sort, tail, uniq, wc) as shell builtins.
- Added infrastructure for command cancellation, streaming I/O, and locale-aware error reporting within the shell environment.
- Configured shell-side dispatch logic to route commands through the new utility context, enhancing performance and binary integration.
- Introduce `detach_reparent` parameter to command execution to support process reparenting.
- Add `detach_session_reparent` to Unix command extensions using a double-fork technique to orphan processes from the shell descendant tree.
- Update background pipeline logic to automatically apply reparenting when unwrapping transparent wrappers like `nohup`.
- Remove unused `command_is_resolvable` helper.
- Added `Shell.liveBackgroundJobCount` to query active background processes.
- Retained per-call `:async:` shells if background jobs are still running upon turn completion.
- Reaped shells automatically once their last background process exits to prevent lingering processes.
- Updated various truncation and overflow messages across `pi-shell` and `coding-agent` to use the consistent `[...N units elided...]` format.
- Improved clarity of elided output by explicitly stating the count and type of omitted information.
- Standardized elision markers across all tool outputs and filters to use cohesive `[...N [type] elided...]`, `[...Nln elided...]`, and `[...xB elided...]` syntax.
- Updated documentation, prompts, and test expectations to reflect the unified elision format.
- Improved transcript viewer robustness by preventing content aliasing through path-inclusive signature hashing.
- Added logic to clear stale transcript content when associated session files are deleted, accompanied by verifying test cases.
- Updated line wrapping in module documentation and test blocks for consistent style.
- Adjusted variable assignment formatting in the segmented chain test to improve readability.
- Validated every stage of a pipeline against `simple_command_is_safe` instead of only the first stage to prevent improper segmentation of compound shell constructs.
- Guarded segment re-execution by verifying that each `Display`-reconstructed command parses back to the expected pipeline shape.
- Configured segmented-chain execution to fall back to an unsegmented, whole-command path whenever a reconstructed segment diverges from the original AST.
- Resolved a syntax error during command execution by preventing `Display` from stripping terminators from compound commands like `while` and `for` loops.
Some Kata/microVM guest kernels (e.g. the CI runner's 6.18.x) are built
without CONFIG_PROC_CHILDREN, so /proc/<pid>/task/<tid>/children does not
exist. The Linux children() relied on it with no fallback, making
children()/live_descendants() return empty and silently turning shell
cancellation cleanup into a no-op inside such containers.
Fall back to scanning /proc and grouping by parent pid (the same primitive
the macOS path already uses) when no children file is readable; kernels
with the file keep the cheap per-task fast path. Also fixes
process::tests::descendants_includes_freshly_spawned_child under Kata CI.
can1357