Commit Graph
44 Commits
Author SHA1 Message Date
can1357 5d1e25f834 fix(shell): bounded backpressured native bash output bridge
The non-PTY bash streaming bridge queued every decoded chunk into
flume::unbounded and fired ThreadsafeFunction callbacks NonBlocking
with no budget, so a producer outrunning the JS event loop grew the
native queue (and the napi queue behind it) without bound — measured
33.5 MB queued for a 32 MiB stream with a stalled consumer, and
multi-GB RSS on longer runs. The downstream OutputSink caps sit after
the N-API boundary and cannot bound either queue.

Bound the pipeline end to end without dropping data:

- pi-natives: bridge_chunks now creates flume::bounded(64) and the
  drain task (extracted as pump_chunks) awaits on_chunk.call_async per
  coalesced <=64 KiB batch, so at most one batch sits in the napi
  queue and the JS event loop's real consumption rate backpressures
  the whole pipeline. If the JS side is gone, the pump exits and
  drops the receiver so senders fail fast.
- pi-shell: emit_chunk sends with send_async().await — a full bridge
  queue parks the pipe reader, which parks the child on its
  stdout/stderr pipe (ordinary pipe backpressure) instead of
  buffering; a disconnected receiver fails immediately so child pipes
  always keep draining.

Unlike a drop-after-cap design, every byte still reaches JS: the
rolling tail view, lossless [raw output: artifact://…] capture, and
totalBytes accounting keep working for outputs past the display cap.

E2E (darwin-arm64 addon): 32 MiB through a JS callback stalling 1 ms
per call — lossless, 472 coalesced callbacks, peak RSS +21.8 MiB.

Fixes #4078
2026-07-09 18:30:48 +02:00
roboomp ce76309065 fix(pi-shell): pinned spawn observer identity against Windows pid reuse
`SpawnRegistry` previously stored only the raw pid reported by brush's
`SpawnObserver` hook and deferred `Process::from_pid` to `build_targets`
at cancellation time. Between the moment a bash-spawned child exited and
the moment cancellation fired, Windows could recycle that freed pid onto
an unrelated process — typically another `pwsh.exe` or `powershell.exe`
in a different Cursor terminal tab, since PowerShell is the parent shell.
`Process::from_pid` at kill time then opened the impostor, and
`signal_tree` walked the current Toolhelp snapshot for `ppid == root`
matches and `TerminateProcess`'d whatever subtree happened to live under
that recycled pid. That is the reporter's Variant A symptom: OMP crashing
kills unrelated PowerShell sessions.

The `SpawnObserver` impl now pins a stable `Process` handle *at spawn
time*, before any pid recycling window can open:

- Windows: an open process handle keeps the pid slot reserved for the
  handle's lifetime (Raymond Chen's documented invariant), so the pid
  cannot be reassigned while the registry holds a reference.
- Linux: the pidfd carries identity independent of the numeric pid.
- macOS: the recorded `(pid, start_tvsec, start_tvusec)` triple detects
  impersonation on every subsequent access.

`TerminationTargets::add_process` accepts a pre-pinned handle and skips
the `Process::from_pid` re-open entirely, and `build_targets` no longer
consults the raw pid at all — an entry the observer failed to pin (the
child exited before we could open a handle) becomes a no-op instead of
racing pid reuse.

Fixes #4605
2026-07-05 10:06:00 +00:00
metaphorics 6680312a25 refactor(pi-shell): behavior-preserving cleanup\n\nOp: compress 2026-07-03 09:31:43 +09:00
Jeff Scott Ward 3da25618ea fix: preserve bash cwd without pwd probe 2026-06-30 23:51:28 -04:00
roboomp dff941d5fe fix(coding-agent): preserved bash status while syncing cwd
Propagated the native shell working directory in ShellRunResult so AgentSession can refresh cwd without running a hidden pwd command in the persistent shell.

Added regression coverage for cd plus a failing command followed by echo $?, proving cwd sync no longer overwrites the user's last shell status.

Fixes #3958
2026-07-01 02:27:05 +00:00
can1357 13b1b5b134 refactor: migrated concurrency primitives to flume and parking_lot
- Replaced standard and tokio mpsc channels with flume channels across workspace crates to simplify thread synchronization.
- Swapped standard Mutex guards for parking_lot Mutexes to avoid manual lock poisoning handling and improve performance.
- Declared workspace-wide dependencies for flume and parking_lot in root and member Cargo manifests.
2026-06-30 23:45:42 +02:00
can1357 31d9fb1145 refactor: consolidated filesystem traversal into a dedicated library
- Replaced custom fast-walk and fs-cache implementations in pi-natives with a new dedicated pi-walker library.
- Integrated the thread-safe, parallel pi-walker library across pi-natives, pi-shell, pi-uu-grep, and uu-find.
- Rewrote file search, fuzzy finding, and glob-matching logic to leverage pi-walker configurations and visitor traits.
- Optimized shell process tracking in pi-shell by replacing global descendant-diff logic with an isolated, per-run SpawnRegistry.
- Added parallel rayon-based walking and optimized fast paths for directory scanning and entry classification.
2026-06-30 23:25:50 +02:00
can1357 70a439ace3 feat(pi-shell): implemented fd filesystem search builtin
- Added an `fd` command-line utility for searching the filesystem.
- Implemented file filtering capabilities for type, size, modification time, and ownership.
- Integrated directory traversal and ignore-pattern support for search optimization.
- Registered the builtin in the shell and added comprehensive integration tests for filtering and globbing.
2026-06-26 17:35:59 +02:00
can1357 39a055a1e5 feat: implemented ripgrep-based rg shell builtin
- Implemented `rg` shell builtin using ripgrep libraries to enable file and directory searching.
- Added `RgSink` to handle output formatting, context, and vimgrep compatibility.
- Updated shell context and IO flags to support stdin as search input.
- Added integration tests for directory recursion, ignore filtering, and stdin handling.
2026-06-26 16:33:05 +02:00
can1357 c39b655ae7 feat: implemented -q and -x flags for in-process grep
- Added support for `--quiet` (`-q`) and `--line-regexp` (`-x`) to the `grep` builtin.
- Enabled short-circuiting behavior for `-q` to suppress output and return early on the first match.
- Configured exit status logic to prioritize successful matches over error states when using `-q`.
- Added integration tests to verify correct exit status codes and line anchoring behavior.
2026-06-26 15:47:24 +02:00
can1357 7bd326e6f4 fix(pi-shell): resolved find -exec paths against shell working directory
- Ensure `-exec` commands run in the shell current working directory rather than the host process CWD.
- Update operand path resolution in `find` matchers to use the display path, matching behavior expected by shell-integrated utilities.
- Add an integration test to verify path substitution and execution context for shell-integrated `find`.
2026-06-26 12:04:12 +02:00
can1357 b5dac65b31 feat: implemented coreutils shell builtins via in-process execution
- Introduced `pi-uutils-ctx` to manage thread-local I/O redirection, environment context, and path resolution for in-process utilities.
- Integrated a suite of vendored coreutils (cat, find, grep, head, ls, mkdir, mv, rm, sort, tail, uniq, wc) as shell builtins.
- Added infrastructure for command cancellation, streaming I/O, and locale-aware error reporting within the shell environment.
- Configured shell-side dispatch logic to route commands through the new utility context, enhancing performance and binary integration.
2026-06-26 12:00:34 +02:00
can1357 00dcd54597 feat: implemented reparenting for backgrounded wrappers
- Introduce `detach_reparent` parameter to command execution to support process reparenting.
- Add `detach_session_reparent` to Unix command extensions using a double-fork technique to orphan processes from the shell descendant tree.
- Update background pipeline logic to automatically apply reparenting when unwrapping transparent wrappers like `nohup`.
- Remove unused `command_is_resolvable` helper.
2026-06-23 08:06:54 +02:00
can1357 c6fcc155a2 fix: nohup 2026-06-23 06:51:11 +02:00
can1357 0fbcb63539 fix: preserved shells with running background jobs
- Added `Shell.liveBackgroundJobCount` to query active background processes.
- Retained per-call `:async:` shells if background jobs are still running upon turn completion.
- Reaped shells automatically once their last background process exits to prevent lingering processes.
2026-06-22 17:25:19 +02:00
can1357 8d671a71a7 refactor(pi-shell): improved test readability and formatting
- Updated line wrapping in module documentation and test blocks for consistent style.
- Adjusted variable assignment formatting in the segmented chain test to improve readability.
2026-06-19 03:35:53 +02:00
can1357 2e6e56711f fix: prevented shell command corruption in sequential chains
- Validated every stage of a pipeline against `simple_command_is_safe` instead of only the first stage to prevent improper segmentation of compound shell constructs.
- Guarded segment re-execution by verifying that each `Display`-reconstructed command parses back to the expected pipeline shape.
- Configured segmented-chain execution to fall back to an unsegmented, whole-command path whenever a reconstructed segment diverges from the original AST.
- Resolved a syntax error during command execution by preventing `Display` from stripping terminators from compound commands like `while` and `for` loops.
2026-06-19 02:39:44 +02:00
can1357 72f76280c9 test(pi-shell): tolerate PID-namespaced getsid(0) in session tests
Same containerized-CI issue as the pi-natives wrapper test: in a PID
namespace the host process's session leader lives outside the namespace,
so getsid(0) returns 0 (not -1). Relax the host_sid > 0 sanity asserts in
embedded_external_command_runs_in_its_own_session and
embedded_pipeline_stage_runs_in_its_own_session to host_sid >= 0; the
child-session invariants (own session, distinct from host) are unchanged.
2026-06-14 23:24:03 +02:00
oldschoolaandcan1357 8ac68b731f test(pi-shell): gate Unix job-control tests and fix nohup on Windows 2026-06-14 17:22:49 +02:00
can1357 4a2417c92f fix(pi-shell): fixed here-document chains to avoid segmented minimizer re-run failures
- Marked heredoc redirects as unsafe for chain segmentation so commands with here-docs run via the single execution path instead of being replayed.
- Added regression tests in minimizer and shell modules covering quoted, escaped, and chained here-doc pipelines.
- Recorded the fix in the natives changelog for multi-command quoted/escaped heredoc handling.
2026-06-14 08:55:35 +02:00
can1357 e14f7effa7 fix: remove flakey test 2026-06-13 22:14:34 +02:00
can1357 b921b67db7 fix(pi-shell): fixed quoted heredoc execution without trailing newline
- Added `ensure_trailing_newline_for_heredoc` to normalize commands by appending a newline for heredoc inputs missing one.
- Applied the helper in both one-shot and streaming shell execution paths before invoking `run_string`.
- Added a regression test for quoted heredocs at EOF in Unix and documented the fix in the natives changelog.
2026-06-13 14:38:36 +02:00
can1357 2e5b0b5757 fix(brush-core-vendored): fixed async background PID targeting for nohup wrapper
- Added a transparent_background_wrapper flag to builtin registrations and a helper constructor.
- Updated async background job creation to classify transparent wrapper builtins and launch the unwrapped command as the tracked job.
- Marked nohup as a transparent background wrapper and added a test asserting `nohup cmd &` exposes a child PID in `$!`.
2026-06-12 22:37:01 +02:00
can1357 961e7c10d3 feat(utils): added isMacosMallocStackLoggingEnvName() function
- Added isMacosMallocStackLoggingEnvName() function to identify MallocStackLogging and MallocStackLoggingNoCompact variables. Updated filterProcessEnv() and Bun.env initialization to skip these variables during environment filtering. Added test case to verify malloc stack logging toggles are dropped instead of forwarded.
2026-06-12 15:46:52 +02:00
David Andrews (LexGenius.ai)andcan1357 ce7ec45d42 feat(minimizer): deterministic shell-output minimizer on 15.10.8 2026-06-10 08:31:30 +02:00
can1357 9d457f73d9 test: migrated test imports to package subpath exports
- Replaced relative `../src` imports with `@oh-my-pi/pi-ai` and `@oh-my-pi/pi-agent-core` subpaths.
2026-06-08 19:03:55 +02:00
can1357 5ba7180d48 feat(pi-shell): added nohup builtin that doesn't mask SIGHUP
- Ran the operand as an ordinary descendant so it is reaped with the host instead of leaking as an orphan.
- Propagated the command's exit status; reported missing operand and exit 125 with no operand.
- Updated the bash tool prompt's daemon guidance away from nohup/setsid/disown detachment.
- Switched shell renderer success icon/state to "done".
2026-06-08 18:28:08 +02:00
can1357 b5582278a0 feat(builtins): implemented wait for pids, -f, -n, and -p flags
- Added process ID resolution and selectors for waited jobs.
- Implemented terminate-only waits and next-job polling.
- Recorded completed job IDs into the variable named by -p.
- Added shell integration tests covering the new wait paths.
2026-06-08 18:28:07 +02:00
can1357 cd9fc55572 fix: aborted internal background jobs during shell cancellation
- Added Job::abort_internal_tasks to abort internal async tasks and drop their join handles.
- Updated shell cancellation paths to call this abort logic and handle mutable shell job lists before signaling remaining process groups.
- Added Rust and TypeScript tests that verify cancellation prevents background shell jobs from completing after abort.
2026-06-02 19:30:40 +02:00
can1357 4ee35bec7a fix: corrected child-session behavior to DetachSession for non-term stdin
- Renamed non-terminal stdin pipeline tests and updated non-terminal expectations to DetachSession behavior.
- Updated child_session_action behavior so non-terminal, non-pipeline stdin now yields DetachSession instead of None.
- Updated execute_external_command to skip process_group for detached children and move take_foreground into its foreground arm.
- Added async regression coverage for detached pipeline stages and logged the fix in the natives CHANGELOG.
2026-05-30 20:55:55 +02:00
oldschoolaandcan1357 ae927b6f81 fix(pi-shell): unblock bash heredocs >4 KiB on Windows and >64 KiB on macOS
brush_core::interp::setup_open_file_with_contents wrote the entire heredoc/here-string body into an anonymous pipe synchronously before handing the reader to the downstream command. Bodies that exceed the OS pipe buffer (~4 KiB on Windows, 16-64 KiB on macOS) deadlocked the writer forever, and the bash tool tripped its 305 s hard timeout without ever launching the consumer. The Linux fast path still uses F_SETPIPE_SZ to grow the pipe inline; every other platform (and Linux bodies that overflow pipe-max-size) now decouples the write onto a fire-and-forget thread that terminates on drain or BrokenPipe.

Adds a 256 KiB regression test that exercises the worst-case shape (: builtin, which never drains stdin), guarded by tokio::time::timeout(10s) so a regression fails CI fast instead of hanging.
2026-05-26 20:12:39 +02:00
can1357 4a44bd1e52 fix(pi-shell): added Sync bound to descendant termination helper
- Updated `terminate_new_descendants` to require the `Sync` marker on its hash-set hasher.
- This tightened async helper bounds without changing its termination-wave logic.
2026-05-25 21:20:23 +02:00
roboomp 8e5c7c9bf1 fix(bash): kept persistent shells after cancel
Stopped marking persistent bash sessions as permanently broken when the JavaScript abort or timeout race wins.

Stopped the Rust descendant kill-wave helper once no cancellation targets remain so later commands are not swept into old cancels.

Fixes #1347
2026-05-25 18:40:22 +00:00
roboomp de5b0a51e2 fix(bash): forcefully terminated canceled shells
Added Rust-side descendant termination on shell cancellation paths so aborts and timeouts escalate to SIGKILL even if brush cleanup stalls.

Covered SIGTERM-ignoring shell workloads in native tests.

Fixes #1347
2026-05-25 18:13:49 +00:00
can1357 c7b2fec204 fix(run-rs-task): always ran fmt:rs regardless of Rust-affecting changes
- Exempted the `fmt:rs` task from the early-exit skip logic so formatting always runs.
- Reformatted `apply_command_env` signature in shell.rs to satisfy the formatter.
2026-05-15 07:10:41 +02:00
roboomp 737288676c fix(pi-shell): preserved $env:VAR refs through brush expansion
Brush-core applied POSIX parameter expansion to $env before dispatching a command, mangling PowerShell references like Write-Host $env:SystemRoot to :SystemRoot. Move the fix down to env-var application: every brush session now defines env=$env as an internal (non-exported) shell variable, so the bash expansion of $env yields the literal $env and PowerShell tokens reach the child intact. User assignments (env=prod; echo "$env:8080") still shadow the fallback in their command scope, so the POSIX bash contract is preserved.

Fixes #1079
2026-05-15 03:50:51 +00:00
can1357 ab4416f5bb fix(crates): address clippy lints surfaced on linux baseline 2026-05-12 14:55:39 +02:00
can1357 1cffb3473a fix: corrected worktree baseline capture with synthetic tree diff
- Added untracked worktree baseline capture via `untrackedPatch` and synthetic tree diffing.
- Added fallback-aware backend ordering by collecting host candidates and retrying alternates on unavailable PAL.
- Hardened overlay mount lifecycle by removing stale overlays and deleting upper/work dirs after unmount.
- Refined ZFS clone deletion to validate ownership and remove dataset+origin only when checks pass.
- Updated ProjFS integration to use extended-info callbacks and symlink metadata reads.
- Updated rcopy path handling to absolutize paths, and added `writeTree` plus combined shell timeout checks.
2026-05-12 14:36:31 +02:00
can1357 f81af06a2b feat(pi-shell): add execute_shell_streams with per-stream raw byte sinks 2026-05-12 14:11:55 +02:00
can1357 a4d86a075a feat(coding-agent): added verbatim unicode rule to hashline prompt 2026-05-12 05:25:55 +02:00
can1357 338ec7284f fix(pi-shell): fixed macOS descendant traversal on broken proc_listchildpids
- Replaced macOS `Process::children` and `descendants` traversal with a one-shot `proc_listallpids` scan grouped by `pbi_ppid`, avoiding the broken self-query path in `proc_listchildpids`.
- Built `snapshot_all_pids` and `build_process_tree` helpers and updated descendant collection to walk that tree once, de-duplicating by PID in post-order.
- Added a Unix regression test that verified a freshly spawned child appears in `live_descendants`, preventing silent no-op cleanup on affected darwin kernels.
2026-05-12 05:07:19 +02:00
can1357 9b536bd6e9 refactor(ps): moved process management logic to pi-shell crate
- Removed ~1,500 lines of platform-specific process code from pi-natives/ps.rs, delegating to pi_shell::process.
- Removed PTY execution support from pi-shell and pi-natives, including portable-pty dependency.
- Exposed terminate_tree and wait_for_exit as public methods on pi_shell::process::Process.
2026-05-12 03:43:32 +02:00
can1357 530942c0b2 feat(pi-shell): enabled PTY shell runs in pi-shell via PTY runner
- Added PTY flags to ShellRunOptions and ShellExecuteOptions and defaulted pty to false when unset.
- Routed PTY-enabled shell runs to a dedicated PTY runner and executed commands via `sh -lc` in a PTY.
- Handled PTY output with a dedicated thread, Chunk/Done events, child PGID cancellation, and drain-timeout cleanup.
- Added Unix-only PTY assertions and non-Windows native coverage for `executeShell` with `pty: true` and `/dev/` output checks.
- Added `portable-pty = "0.9"` (and lockfile update) to enable PTY execution in pi-shell.
2026-05-12 01:00:31 +02:00
can1357 41710a0a5f feat(cross-cutting): added pi-ast/pi-shell crates and rewired pi-natives
- Added `pi-ast` as a new crate with public `language`, `ops`, and `summary` modules and exported `SupportLang`.
- Added AST summary and rewrite APIs in `pi-ast`, including language inference, overlap-safe edits, and span normalization.
- Added `pi-shell` crate with new `Shell` and cross-platform `Process` APIs, session execution, and cancellation-aware runs.
- Updated `pi-natives` to use shared `pi_ast`/`pi_shell` APIs, switched to local path deps, and moved minimizer assets to `pi-shell`.
2026-05-11 03:43:58 +02:00