Commit Graph

212 Commits

Author SHA1 Message Date
can1357 2ee9943563 refactor: unify builtins in one place 2026-08-08 10:19:25 +02:00
can1357 b0a94a8fc0 chore: cleanup 2026-08-05 03:07:16 +02:00
can1357 1cf919099e style(pi-shell): wrapped long command string in test
- Split a long command string across multiple lines in shell test.
2026-08-03 23:41:07 +02:00
can1357 dce12984f2 fix(pi-shell): parsed state format specifier correctly in ps builtin
- Corrected the mapping of the "state" specifier to PsField::State in the ps format parser.
- Added a test to verify that the ps builtin successfully accepts tpgid and other job control columns.
2026-08-03 19:01:09 +02:00
can1357 451eb9842c fix: mapped tail builtin broken pipe to silent exit 141
- Handled broken pipe errors across tail output and follow paths by translating them to a silent SIGPIPE exit code.
- Prevented stderr noise when downstream pipeline readers exit early, matching native tail behavior.
2026-08-03 18:53:14 +02:00
can1357 cdd1d13079 feat(pi-shell): extended ps builtin format specifiers and process metrics
- Added support for extended ps format specifiers including tpgid, ruid, rgid, egid, pri, flags, min_flt, maj_flt, times, sz, s, ruser, rgroup, and tgid.
- Implemented group name resolution via `getgrgid_r` on Unix platforms.
- Suppressed broken pipe diagnostic output in the tail builtin to match standard tail behavior on downstream closure.
2026-08-03 18:51:23 +02:00
roboomp dc22ea5dad fix(pi-shell): kept recycled targets killable
Protect only the harness pid during cancellation sweeps. The host recorded parent pid can be stale on Windows and recycled onto the hung command; adding that raw pid to the protected set spared the cancellation target and pruned its whole subtree from cleanup.

Keep protected-subtree pruning rooted at the harness itself, which still spares its real workers while allowing the timed-out target to be reaped.

Fixes #7452
2026-08-03 03:47:32 +00:00
roboomp e5f954b0fe fix(pi-shell): prune protected subtrees from cancellation sweeps
The flattened descendant list can contain a protected node (the
harness, on a Windows PID-reuse false-descendant) together with that
node's real children, collected by recursing through it. Skipping only
the exact protected pid kept omp alive but still TerminateProcess'd its
unrelated worker/tool subprocesses.

signal_tree/terminate_tree now drop every node whose recorded parent
chain within the enumerated set passes through a protected pid, so a
false descendant of the harness can no longer drag the harness's real
children into the kill set.

Fixes #7452
2026-08-03 03:42:50 +00:00
roboomp 25633d357d fix(pi-shell): spare host process from run-cancellation sweeps
On Windows the descendant tree used to reap a cancelled bash run is
built from raw th32ParentProcessID links that outlive their recorded
parent. A recycled pid matching the harness's stale parent pid could
surface omp itself (or an ancestor) as a false descendant, and
signal_tree TerminateProcess'd it — killing the session with no
cleanup and no session_exit record when a blocking command hit its
timeout.

Add host_protected_pids() (harness pid plus its resolvable ancestor
chain) and skip those pids in signal_tree and terminate_tree. The
guard is cross-platform: a no-op on Unix, where the descendant walk is
already identity-pinned, and the safety net that keeps a tool timeout
from ever taking down the harness on Windows.

Fixes #7452
2026-08-03 03:34:44 +00:00
can1357 7d6eb47314 feat: implemented ps shell builtin with sorting and formatting options
- Added the `ps` shell builtin with BSD and procps selection forms, custom formatting, and sorting capabilities.
- Implemented the `sanitize_process_command` helper to clean process command names and arguments.
- Updated the bash prompt template and package changelogs to document the new builtin.
2026-08-02 20:40:19 +02:00
can1357 dda859b60f fix(ci): hardened logger probe timeouts and silenced async trait lint
- Raised probe-spawning logger tests to 30s timeouts; bun's 5s default
  SIGTERMed probe children (exit 143) on shared-core CI runners, matching
  the precedent documented in logger-contract.test.ts.
- Allowed clippy::unused_async_trait_impl on KillCommand::execute with an
  unknown_lints guard for the pinned CI nightly that predates the lint.
2026-08-01 22:34:39 +02:00
can1357 534f43eda6 style(pi-shell): wrapped long allow attributes per rustfmt 2026-08-01 22:12:13 +02:00
can1357 ed7d085708 fix(pi-shell): resolved clippy-strict errors in process builtins
- Made infallible ProcInfo accessors const and annotated Option-returning
  accessors with allow(unnecessary_wraps) since Option is the shared
  cross-platform contract.
- Boxed ParseProcResult::Options to shrink the enum.
- Restructured kill signal resolution to bind signal once instead of
  let-then-reassign sequences.
- Simplified liveness re-check with Option::is_none_or.
2026-08-01 22:08:55 +02:00
can1357 d326af8ed0 feat: implemented process management builtins and cross-platform snapshotting
- Implemented new shell builtins including `top`, `pgrep`, `pkill`, `pidwait`, and `kill`.
- Added a cross-platform process snapshot module supporting Linux, macOS, and Windows.
- Extended job and process handling utilities with process iteration and handle termination methods.
- Replaced test mutex locks with thread-local counters to prevent test races in the minimizer engine.
2026-08-01 21:58:09 +02:00
can1357 e5bdfc556d style(rust): formatted native and shell sources 2026-07-28 13:02:15 +02:00
can1357 92ac96ef9b refactor(vendor/brush-core): improved shell execution parameters and code cleanup
- Update job spawning functions to accept immutable shell references.
- Replace vector pop with index removal in word expansion.
2026-07-28 10:22:23 +02:00
can1357 34e88b344b refactor: adopted idiomatic combinators and updated the rust toolchain
- Updated the nightly rust toolchain channel in rust-toolchain.toml to nightly-2026-07-28.
- Adopted slice chunking and multiple-of helper methods across native and vendor crates.
- Replaced option map adapters and conditional patterns with idiomatic combinators.
2026-07-28 10:01:56 +02:00
can1357 2f63a07ba9 style(shell): removed unused mut in kill test helper 2026-07-27 17:11:37 +02:00
can1357 3d9f28dd46 refactor: resolved cross-platform build warnings and add windows-sys features
- Add conditional attributes to silence dead-code warnings on platform-specific code and fields.
- Update target dependencies in pi-walker/Cargo.toml with explicit windows-sys feature sets.
- Simplify time cast expressions in linux_reflink and rcopy modules.
2026-07-27 17:04:52 +02:00
can1357 8acd667005 Merge PR #6780: fix(shell): correct kill signal handling (@roboomp) 2026-07-27 15:57:47 +02:00
can1357 8facd237d5 feat(build): migrated native pipeline to bazel with remote caching
- Replaced the napi-cli/cargo-zigbuild/cargo-xwin/sccache build path with
  Bazel: rules_rust + crate_universe over Cargo.lock, hermetic zig cc
  toolchains (linux-gnu pinned to glibc 2.17, linux-musl), host Xcode for
  darwin, and a repo-local hermetic clang-cl + llvm-ml + xwin toolchain for
  windows-msvc (bazel/toolchains/msvc).
- All eight shipped addons build as //:natives-<target> via the release
  transition in bazel/defs.bzl (opt, thin LTO, cgu=16, stripped, canonical
  .node naming); scripts/bazel-natives.ts is the single driver for local
  dev and CI.
- Rust validation moved to bazel test + clippy aspects (strict workspace
  policy for opted-in crates, default lints elsewhere, mirroring cargo
  semantics) and the rustfmt aspect; cargo stays as the dev-iteration
  surface, with brush-core/brush-builtins promoted to workspace members
  and excluded from cargo dev tasks to keep their historical scope.
- CI caches through an in-cluster bazel-remote action cache (TLS + basic
  auth, cluster-internal only); GitHub-hosted runners never touch the
  infrastructure and use an actions/cache-backed disk cache instead.
- Deleted the hand-rolled caching machinery: ci-target-cache,
  ci-native-artifact-cache, ci-build-native, native-source-hash,
  find-native-artifacts, restore-linux-native, native-prewarm workflow,
  ensure-* toolchain actions, and all sccache/Swatinem wiring.
- Warm native rebuilds drop from ~20 minutes to seconds; a cold client
  with a warm remote cache rebuilds the linux x64 pair in ~2.5 minutes.
2026-07-27 12:22:19 +02:00
roboomp 0c1fd7f5d8 fix(shell): preserved negative pid when clap eats --
clap consumes the -- marker before execute for the default-signal and -s/-n forms, so kill -- -10 and kill -s TERM -- -10 previously misread the negative PID as a signal. Captured post-marker operands via a dedicated last=true field and treated a preselected -s/-n signal as closing the option position.

Added a regression covering both marker-consumed forms.

Fixes #6779
2026-07-27 09:42:32 +00:00
roboomp 0e0fa77a51 fix(shell): continued kill after target failures
Recorded per-target PID and jobspec errors while continuing through every remaining operand, then returned a non-zero aggregate status.

Added a regression with a stale PID between two live processes.

Fixes #6779
2026-07-27 09:32:21 +00:00
roboomp fb6a62c227 fix(shell): preserved negative pid operands in kill
Restricted -sigspec parsing to the option position and consumed the -- end-of-options marker, so negative PIDs (process groups) and post-marker operands are signaled rather than parsed as signals.

Added a process-group regression covering kill -TERM -- -<pgid> <pid>.

Fixes #6779
2026-07-27 09:27:57 +00:00
roboomp 38e143d7cf fix(shell): corrected kill signal handling
Accepted numeric signal specifications, signaled every process operand, and restored SIGTERM as the default.

Added process-level regressions for multi-target SIGKILL and graceful default termination.

Fixes #6779
2026-07-27 09:14:13 +00:00
can1357 f23bc266a8 feat(natives): enabled per-language rewrite rules for mixed-language paths
- Remove uniform language inference requirement, allowing mixed-language paths to rewrite each file in its own language.
- Update `ast_edit_blocking` in `crates/pi-natives/src/ast.rs` to compile rewrite rules per language and skip unsupported languages gracefully.
- Update `ast-edit.md` prompt documentation to reflect mixed-language path support.
- Add test coverage verifying mixed-language tree rewrites.
2026-07-24 21:52:29 +02:00
can1357 a569385b8e fix(shell): captured find -exec child stdio into scope streams
- find -exec/-execdir children inherited the omp process's real
  stdout/stderr, spamming output into the TUI terminal and bypassing
  shell redirects; they also inherited the host env instead of the
  shell's exported environment.
- Added pi_uutils_ctx::run_captured (moved from uu-xargs' private
  helper): stdin null, stdout streamed into scope stdout, stderr
  drained on a helper thread and forwarded after exit.
- uu-find exec matchers now use env_clear + env_snapshot and
  run_captured; MultiExecMatcher rebuilds a std Command from the
  argmax command's accumulated state (argmax only Derefs immutably).
- uu-xargs reuses the shared helper; added pi-shell regression test
  asserting -exec child stdout flows through the shell redirect with
  the exported env.
2026-07-24 20:06:29 +02:00
can1357 7ebf141743 chore: bump version to 17.1.2
- fixed rustfmt drift and clippy errors in pi-shell moreutils builtins (combine, errno, ifne, isutf8, sponge, ts) that blocked the release check gate
2026-07-24 16:53:52 +02:00
can1357 5d4f1cf6b5 feat(pi-shell/moreutils): implemented six moreutils-inspired shell builtins
- Added six builtin commands: ts, sponge, ifne, isutf8, combine, and errno to pi-shell.
- Created moreutils module with independent implementations for all tools.
- Added jiff dependency for timestamp and timezone functionality.
- Integrated builtins into shell execution pipeline with in-process execution.
- Added integration tests verifying pipe chains like ts | sponge with isutf8.
2026-07-24 14:52:16 +02:00
can1357 269c267397 feat(pi-shell): implemented cmp utility for embedded shell
- Added cmp builtin with full POSIX-compatible flag support including `-b`, `-i`, `-l`, `-s`, `-x`, `-h`, `-z`.
- Integrated into shell builtin registry and coreutils module.
- Added comprehensive tests using tempfile for temporary directories.
2026-07-24 08:05:12 +02:00
can1357 4d4090a578 style: formatted shell tilde-expansion test 2026-07-17 21:24:45 +02:00
roboomp 9e73821202 fix(shell): expanded tilde for every brace-expansion element
Brace expansion joined its results with spaces and re-parsed them as a single word, so tilde-at-word-start only fired on the leading element. Now each brace element is expanded as its own word, so `~/project/{a,b}` expands both tildes instead of leaving a literal `~/project/b`.

Fixes #5819
2026-07-17 09:15:30 +00:00
can1357 b71841c4cc merge PR #5694 via eval/pr-5694: ci: retry flaky workspace tests 2026-07-17 04:39:14 +02:00
can1357 4d7e51413b merge PR #5671 via eval/pr-5671: fix(bash): drained piped output before timeout return 2026-07-17 04:37:11 +02:00
David Andrews (LexGenius.ai) 5cb9533436 fix(shell): parse uv run --extra values 2026-07-16 08:39:37 -04:00
roboomp 3e38a5b514 fix(bash): drained piped output before timeout return
- Delayed reader cancellation so pipeline consumers can flush after producers are terminated.
- Kept the JavaScript watchdog behind bounded native timeout cleanup.
- Added native and executor regressions for timeout-time output draining.

Fixes #5316
2026-07-16 07:22:58 +00:00
can1357 b51fd7f06d Merge remote-tracking branch 'origin/farm/58511219/fix-bash-process-substitution-hang' 2026-07-15 12:38:47 +02:00
roboomp 8eb9b6d251 fix(shell): materialized process substitution fds for builtins
- Rewrote logical /dev/fd operands to live OS descriptors before invoking in-process uutils.
- Added regression coverage for diff reading two process substitutions.

Fixes #5557
2026-07-15 10:04:00 +00:00
can1357 e542f762ca fix(pi-shell): mapped BrokenPipe stdout errors to silent fd exit code 141
- Updated `run_fd_sync` to handle `io::ErrorKind::BrokenPipe` by returning exit code `141` without writing an error message.
- Added a regression test that verifies `fd` exits with `141` and empty stderr when stdout is closed early (e.g. `fd ... | head` scenario).
- Recorded the broken-pipe behavior fix in `packages/coding-agent/CHANGELOG.md`.
2026-07-15 10:09:41 +02:00
can1357 0ae8efd649 feat: integrated coreutils as in-process shell builtins
- Integrated 17 new coreutils-based shell builtins including `diff`, `date`, `ln`, `stat`, `seq`, `touch`, and others.
- Refactored vendored utilities to execute as in-process shell builtins by routing I/O, environment access, and path resolution through `pi_uutils_ctx`.
- Disabled process-level modifications (e.g., clock setting, hostname modification) to ensure safety and scope adherence within the shell environment.
- Implemented shell-specific features such as cancellation polling, custom exit code management, and efficient output streaming for all new builtins.
2026-07-11 20:37:52 +02:00
can1357 e889572cf0 test(pi-shell): added integration tests for shell builtins
- Verify `sed` correctly streams stdin and performs in-place directory-relative edits.
- Validate `xargs` behavior regarding stdout piping, argument batching, and GNU-compatible exit codes.
- Ensure `jq` correctly filters JSON input, resolves relative file paths, and propagates exit status.
2026-07-11 19:10:55 +02:00
can1357 5a73d65b7a feat(shell): integrated additional coreutils into shell builtins
- Added base64, checksum utilities (md5, sha1, sha224, sha256, sha384, sha512, b2sum), path utilities (basename, dirname), and text processing tools (cut, tee, tr, paste, comm) to the shell.
- Registered new utility builtins in crates/pi-shell/src/coreutils.rs and crates/pi-shell/src/shell.rs.
- Updated Cargo.toml to include the necessary dependencies for the new core utilities.
2026-07-11 19:07:01 +02:00
can1357 172691f6ec feat: removed redundant pre-execution bash command fixup logic
- Removed the bash command fixup system and associated logic from both the Rust and TypeScript components.
- Deleted the redundant fixup module and its exported implementations.
- Updated technical documentation and configuration settings to reflect the removal of pre-execution bash command rewriting.
2026-07-11 18:28:32 +02:00
can1357 5d1e25f834 fix(shell): bounded backpressured native bash output bridge
The non-PTY bash streaming bridge queued every decoded chunk into
flume::unbounded and fired ThreadsafeFunction callbacks NonBlocking
with no budget, so a producer outrunning the JS event loop grew the
native queue (and the napi queue behind it) without bound — measured
33.5 MB queued for a 32 MiB stream with a stalled consumer, and
multi-GB RSS on longer runs. The downstream OutputSink caps sit after
the N-API boundary and cannot bound either queue.

Bound the pipeline end to end without dropping data:

- pi-natives: bridge_chunks now creates flume::bounded(64) and the
  drain task (extracted as pump_chunks) awaits on_chunk.call_async per
  coalesced <=64 KiB batch, so at most one batch sits in the napi
  queue and the JS event loop's real consumption rate backpressures
  the whole pipeline. If the JS side is gone, the pump exits and
  drops the receiver so senders fail fast.
- pi-shell: emit_chunk sends with send_async().await — a full bridge
  queue parks the pipe reader, which parks the child on its
  stdout/stderr pipe (ordinary pipe backpressure) instead of
  buffering; a disconnected receiver fails immediately so child pipes
  always keep draining.

Unlike a drop-after-cap design, every byte still reaches JS: the
rolling tail view, lossless [raw output: artifact://…] capture, and
totalBytes accounting keep working for outputs past the display cap.

E2E (darwin-arm64 addon): 32 MiB through a JS callback stalling 1 ms
per call — lossless, 472 coalesced callbacks, peak RSS +21.8 MiB.

Fixes #4078
2026-07-09 18:30:48 +02:00
can1357 e66e2dc129 style(pi-shell): applied rustfmt to spawn registry tests 2026-07-08 16:44:39 +02:00
can1357 259078d9df merge PR #4606: fix(pi-shell): pin spawn observer identity against Windows pid reuse 2026-07-08 15:19:41 +02:00
can1357 48c53f8c56 chore: since perl crate decided to yank valid dists breaking my ci, im yanking it in return
who even uses perl?
2026-07-06 17:48:28 +02:00
roboomp 04258b98d7 fix(pi-shell): bounded SpawnRegistry sweeps by watermark not raw length
Addresses the third review on #4606: `record` guarded pruning with
`spawned.len() >= PRUNE_THRESHOLD`. Once the recorded vec stabilized
above the threshold with entries the sweep could not remove — a run
whose live children exceed the threshold, e.g. `for i in {1..1000}; do
sleep 60 & done` — every subsequent `record` re-entered `prune_exited`
while holding the registry lock. Each sweep is O(N) (a status probe per
entry, plus a Toolhelp descendant walk on Windows for exited roots), so
the per-spawn cost climbed to O(n²) even though the doc-comment
promised amortized O(1).

The registry now tracks a `next_sweep_at` watermark alongside the
recorded vec (both under one mutex — the two fields are always
mutated together). A sweep fires only when `spawned.len()` crosses
that watermark; every sweep rescheds the next fire `PRUNE_THRESHOLD`
further records away from the current post-sweep size. Sweep frequency
is now capped at one per `PRUNE_THRESHOLD` records regardless of live
set size, restoring true amortized O(1) per spawn.

`build_targets` resets the watermark after its own sweep so the
record-time schedule stays consistent with the post-cancel live set.

Added `spawn_registry_watermark_bounds_sweep_frequency`: fills the vec
past threshold with permanently-live entries, records another 20, and
asserts the vec grew by exactly 20 (no sweep modified it) and the
watermark did not advance. Existing tests updated for the collapsed
`state` mutex.

Fixes #4605
2026-07-05 10:27:39 +00:00
roboomp 1877c7ed27 fix(pi-shell): kept Windows spawn entries until descendants exit
Addresses the second review on #4606: `prune_exited` retained an entry
while its process was running OR its process group was alive. On Windows
`process_group_alive` is always `false` (no process groups), so the
predicate collapsed to "root running" — when a brush-spawned root exited
after starting a child that stayed alive (a `pwsh`/shell command that
launches a helper and exits, an MCP stdio wrapper handing off to a
long-running server), pruning immediately dropped the pinned handle.

Dropping that handle closes the last thing keeping the root pid slot
reserved. Windows can then recycle the pid onto an unrelated process,
reintroducing the exact race #4605 closes. It also strands the leftover
child: the next cancellation wave has no root to walk descendants from,
so `signal_tree` never reaches it.

The retain predicate is now:

- root process still running → keep (all platforms);
- Windows-only: root exited but the pinned handle still probes at least
  one live descendant via Toolhelp → keep, because the handle is what
  guarantees the walk targets the original subtree (recycled pids would
  not be reachable while the handle holds the slot);
- pgid still alive → keep (Unix only; Windows falls through).

Unix stays unchanged because a child reparented onto init keeps its
pgid, so `process_group_alive` already catches "root gone, descendants
alive" without a per-entry tree walk.

Fixes #4605
2026-07-05 10:21:26 +00:00
roboomp 79e083124c fix(pi-shell): pruned exited entries from SpawnRegistry
Addresses the review on #4606: pinning a stable `Process` per spawn is
correct against pid reuse, but a long-running shell command that spawns
many short-lived external processes (a bash loop invoking a binary per
iteration) would otherwise retain one owned OS handle per historical
spawn — a pidfd on Linux, a process `HANDLE` on Windows — until the run
ends. Under enough iterations that hits the per-process FD/handle limit
and starts breaking `Process::from_pid` (or any other file operation) for
the rest of the run.

`SpawnRegistry` now sweeps entries whose pinned process and process group
are both gone. The sweep runs opportunistically inside `record` once the
recorded vec crosses a small threshold (`PRUNE_THRESHOLD = 64`) and
unconditionally at the top of `build_targets`, so the retained handle
count is bounded by the current live tree rather than the historical
spawn count. Amortized cost per spawn stays O(1); a sweep is O(N) probes
of `Process::status` (non-blocking pidfd `poll` on Linux, `WaitForSingle
Object(_, 0)` on Windows), running at most once per `PRUNE_THRESHOLD`
records.

The previous identity-pinning regression test was rewritten to defend
the actual invariant — the pinned handle carries into `build_targets`
while the child is alive, and the entry is dropped (never re-opened by
pid) once the child exits. A new regression asserts pruning keeps
retained entries under `PRUNE_THRESHOLD` after 2×threshold spawns of
short-lived children.

Fixes #4605
2026-07-05 10:14:07 +00:00