Commit Graph

62 Commits

Author SHA1 Message Date
can1357 52ad6516ef feat(diff): implemented native UTF-16 diff processing and removed jsdiff
- Implemented native UTF-16 text processing in Rust diff module with support for unpaired surrogates.
- Removed `similar` crate from Rust workspace and `diff` npm package from coding-agent, hashline, and natives.
- Removed jsdiff fallback wrappers and `isWellFormed()` guards from TypeScript diff implementations.
- Added comprehensive test suite for native diff functions covering random inputs and edge cases including surrogates and emoji.
- Renamed model `codex-auto-review` to `gpt-5.3-codex-spark` with updated pricing and context window.
2026-07-23 01:35:31 +02:00
Wolfgang Schoenberger ee6717872c test(natives): cover ill-formed UTF-16 rejection and jsdiff fallback 2026-07-22 04:32:46 -07:00
Wolfgang Schoenberger 81aa4ad27f test(natives): add seeded randomized word-diff parity coverage 2026-07-22 02:58:33 -07:00
Wolfgang Schoenberger 5f41203f00 test(natives): add jsdiff parity suite for native line/word diff 2026-07-22 01:25:31 -07:00
roboomp 044b74594f fix(launch): prevented finite PTY start hangs
- Reported the spawned PTY child PID through the native start callback.
- Replaced broker PID-file polling with the authoritative spawn event.
- Covered finite PTY startup without the legacy handoff in integration tests.

Fixes #5996
2026-07-18 14:54:53 +00:00
can1357 2c355102ce chore: applied biome formatting to merged sources 2026-07-16 03:52:48 +02:00
can1357 a29627142c fix(natives): verify disk before stale-process diagnosis 2026-07-16 03:32:03 +02:00
can1357 29b6649ff9 merged PR #5423: fix(natives): distinguish process-stale from disk-stale sentinel mismatch 2026-07-16 03:32:03 +02:00
roboomp 90c4726bae fix(launch): passed Windows PTY arguments directly
Added a direct-argv PTY entry point and used it for Windows launch sessions so portable-pty no longer re-quotes cmd.exe command text.

Rejected direct .bat and .cmd applications with guidance to use cmd.exe /c.

Fixes #5416
2026-07-14 20:17:03 +00:00
roboomp e4a10450ec fix(natives): distinguish process-stale from disk-stale sentinel mismatch
A long-lived process that survives an in-place upgrade keeps the previous
pi-natives NAPI addon resident. A tab worker spawned afterwards runs the
new JS loader, which expects the new version sentinel, but require returns
the resident old exports carrying the prior sentinel. validateLoadedBindings
previously reported "reinstall to re-sync" for this case even though disk was
already consistent, so only a restart helped.

Detect a versioned __piNativesV* export other than the expected one on the
loaded bindings and report that omp was upgraded mid-session and must be
restarted, reserving the reinstall guidance for genuinely disk-stale addons.

Fixes #4812
2026-07-14 16:32:04 +00:00
can1357 46b8ee737f feat(catalog): added Grok 4.5 model support
- Added Grok 4.5 to the model catalog and identity helper.
- Updated pricing and configuration settings for existing Grok models.
2026-07-09 22:43:34 +02:00
can1357 4f8c341745 test(natives): pinned bash timeout output-bridge crash contract
Regression test for the WSL crash where a timed-out bash command got OMP
OOM-killed: an output-heavy command (in-process 'yes | cat') with a short
timeoutMs must resolve near its deadline with bounded RSS, on both
executeShell and Shell.run. On the pre-fix bridge the same harness measured
~4 GiB RSS growth and minutes-late resolution (JS event loop starved by the
unbounded callback flood); with the bounded backpressured bridge it resolves
at the deadline with flat memory. Also added the user-facing changelog entry
for the crash symptom.

Fixes #4866
2026-07-09 18:39:56 +02:00
roboomp 4b2c3d524c fix(natives): made variant detection robust in worker threads
`pi_natives` failed to load in Bun worker threads on macOS x64 when the
host built only the `modern` (AVX2) variant. The runtime detector's
`child_process.spawnSync("sysctl", ...)` returned null from the worker
even though the build-time detector (`scripts/host-detect.ts`) succeeded
in the parent shell, so `loadNative()` resolved `variant=baseline` and
searched a file list that excluded the `pi_natives.darwin-x64-modern.node`
the build had actually produced.

Two compounding root causes in `packages/natives/native/loader-state.js`:

- `runCommand` only used `child_process.spawnSync`, which is the path
  observed to fail under Bun's worker shim on darwin. The build-time
  detector uses `Bun.spawnSync` and works fine.
- The darwin branch looked up `sysctl` via PATH. Login shells supply
  `/usr/sbin` so the build picks it up, but worker/embedded spawn
  contexts can ship without it.

Fix:

- `runCommand` now prefers `Bun.spawnSync` (matches the build-time
  detector) and falls back to `child_process.spawnSync` for non-Bun
  embeds.
- The darwin branch tries `/usr/sbin/sysctl` before bare `sysctl`.
- New private env key `__PI_NATIVE_VARIANT_CACHE`: once any context
  resolves the variant (the main thread does first), it is written
  there. Bun workers and child subprocesses inherit `process.env` at
  spawn, so they read the cache and skip detection — sidestepping the
  worker-context spawn flakiness end-to-end.
- New exported pure helper `selectCpuVariant({ arch, override, env,
  detectAvx2 })` codifies the override > cache > detect order and
  returns the cache write hints so the helper itself stays
  side-effect-free.

Regression test `packages/natives/test/issue-3238-repro.test.ts` pins
every branch of the resolution order, including the file-list shape
that surfaced the bug.

Fixes #3238
2026-06-22 07:17:50 +00:00
can1357 1f3f3cf5d1 feat: added ruby and julia language support to coding-agent
- Implemented persistent execution backends for Ruby and Julia using dedicated kernel processes and NDJSON-based IPC.
- Integrated language-specific prelude environments, runtime path resolution, and security-focused environment variable filtering.
- Exposed configuration options, tool schema updates, and lifecycle management for seamless agent interaction with both languages.
- Added comprehensive integration tests and updated prompt documentation to support the new evaluation capabilities.
2026-06-22 06:13:01 +02:00
ryjm c2a14a0afc fix(ast): infer extensionless Emacs init files 2026-06-15 18:58:55 -07:00
ryjm 18f11f5bfc feat(ast): add Emacs Lisp tree-sitter support
Register tree-sitter-elisp in the shared AST language registry so
.el files infer the emacs-lisp grammar across blockRangeAt,
summarizeCode, astGrep/astEdit, and native aliases.

This fixes edit-tool block operations on top-level Emacs Lisp forms
instead of returning unsupported-language block errors.

- Add canonical emacs-lisp aliases and .el extension inference.
- Teach summaries to fold Lisp forms without grouping arbitrary lists.
- Map .el rendering and highlighting aliases through coding-agent/native.
- Cover defun, ERT, use-package, with-eval-after-load, pcase,
  summary, astMatch, astEdit, and edit-tool insertion paths.
- Document the language and update package changelogs.
2026-06-15 18:41:24 -07:00
roboomp 001a3d1204 fix(natives): cleaned stale native cache versions
Removed stale per-version native cache directories after successful addon loads so updates do not leave old ~/.omp/natives/<version> trees behind.

Added a focused loader regression test for keeping the current version directory and non-directory files while removing stale version folders.

Fixes #2560
2026-06-14 11:57:56 +00:00
can1357 3d5572a0fc test(natives): documented AST conditions and expanded astMatch tests
- Updated `docs/ttsr-injection-lifecycle.md` to document `astCondition` behavior, including registration rules and tool-stream matching.
- Extended `packages/natives/test/native.test.ts` with new `astMatch` coverage for Smart matching, metavariable consistency, parse errors, and empty-language rejection.
2026-06-08 15:00:28 +02:00
can1357 a89bb76444 fix(tui): preserved visible ANSI content when truncating terminal rows
- Replaced the TUI row truncation path with per-character and ANSI-sequence iteration that tracks visible cells before clipping output.
- Added ANSI helpers to parse sequence boundaries and preserve OSC66 visible payloads while enforcing max source length during truncation.
- Added natives regression coverage for Ghostty super+alt backspace key matching and parsing.
2026-06-08 01:22:26 +02:00
roboomp c8919f3619 fix(natives): stream sorted glob admits only
Tied uncached sortByMtime onMatch callbacks to bounded heap admission so live progress and timeout partials reflect the same mtime-ranked set the call returns, with the JS-side mtime sort+truncate converging on the native top-N.
2026-06-03 06:26:26 +00:00
roboomp f73745235c fix(natives): streamed bounded glob progress
Preserved bounded onMatch progress during uncached sorted glob traversal so timed-out find scans can still return partial matches without unbounded callback growth.
2026-06-03 06:20:25 +00:00
roboomp d3477372a2 fix(natives): bounded sorted glob scans
Kept uncached sort-by-mtime glob traversal bounded to maxResults and emitted onMatch callbacks only for returned matches so broad find scans cannot grow parent memory independently of the limit.

Fixes #1761
2026-06-03 06:15:26 +00:00
can1357 34e099148a feat(natives): added per-platform leaf package publishing for native addons
- Added `gen-npm-packages.ts` script to generate per-platform leaf packages under `npm/-/`.
- Updated CI release publisher to generate and publish leaf packages before rewriting the core manifest with pinned `optionalDependencies`.
- Core package now ships only JS loader and declarations; installs fetch only the host platform's `.node` binary.
2026-05-30 17:08:23 +02:00
can1357 cc258b1757 feat(natives): added embedded addon tarball extraction in natives
- Added embedded addon tarball output in `embed-native.ts` using `embedded-addons.<platform>.tar.gz` artifacts.
- Added metadata-rich addon types with `size`, `filePath`, and optional `archive` fields.
- Updated extraction to prioritize archive unpacking and skip cached `.node` files when sizes match.
- Added `extractEmbeddedAddonArchive()` with archive parsing and safe validation of archive entry names and kinds.
- Adjusted release profile to disable line-table generation and strip symbols in `Cargo.toml`.
- Added CI-only ELF validation for forbidden sections and regression coverage for issue-823 archive extraction.
2026-05-28 11:48:32 +02:00
roboomp de5b0a51e2 fix(bash): forcefully terminated canceled shells
Added Rust-side descendant termination on shell cancellation paths so aborts and timeouts escalate to SIGKILL even if brush cleanup stalls.

Covered SIGTERM-ignoring shell workloads in native tests.

Fixes #1347
2026-05-25 18:13:49 +00:00
can1357 90b134ca4c test: replaced real timers and sleeps with deterministic test hooks
- Added `providerRetryWait` and `retryWait` hooks to stream/usage options so tests bypass real scheduler delays.
- Parameterized GitHub Copilot poll intervals and Copilot model retry base delay for fast test execution.
- Replaced `Bun.sleep`/`setTimeout` polling loops with `AbortSignal` event listeners in agent session tests.
- Consolidated auth-gateway E2E helpers into a shared `test/helpers` module, eliminating duplicated `checkGatewayAvailable` implementations.
- Migrated credential-disabled tests from SQLite-backed stores to an in-memory store, removing temp-dir lifecycle overhead.
2026-05-17 04:02:09 +02:00
can1357 22848e89ff refactor(natives): removed sanitizeText from pi-natives, use JS impl
- Moved @oh-my-pi/pi-natives from devDependencies to dependencies in pi-utils.
- Deleted Rust sanitizeText implementation and its JS bindings/types.
- Updated sanitize benchmark to use pi-utils implementation as baseline.
2026-05-16 20:21:36 +02:00
can1357 dac28f66e4 feat: added windows native staging candidates before runtime paths
- Added Windows node_modules staging checks so non-compiled addons are copied into versioned native directories.
- Updated loader candidate resolution to prefer embedded/staged/versioned candidates before default runtime paths when staging is enabled.
- Added runtime version-sentinel export __piNativesV15_0_1 and load-time validation to avoid stale binary drift.
- Added typing and release-flow updates to expose staging options and keep per-release sentinel IDs aligned.
2026-05-14 23:24:14 +02:00
can1357 f1f6516056 refactor: reorganized exports and removed obsolete helper branches
- Removed export leakage by demoting many helper and const symbols to module-local scope.
- Renamed underscore-prefixed internals and cache fields, then updated related references and `satisfies never` checks.
- Deleted obsolete logic branches and helpers, including harmony-stream interruption flow and unused benchmark runtime helpers.
- Updated Biome config and manifests by broadening lint coverage and removing an unused `@napi-rs/cli` dev dependency.
- Adjusted tests and utilities to use renamed test helpers and remove redundant private test-only helpers/locals.
2026-05-14 04:36:19 +02:00
can1357 975941aba4 chore: remove garbage tests 2026-05-12 04:09:33 +02:00
can1357 9b536bd6e9 refactor(ps): moved process management logic to pi-shell crate
- Removed ~1,500 lines of platform-specific process code from pi-natives/ps.rs, delegating to pi_shell::process.
- Removed PTY execution support from pi-shell and pi-natives, including portable-pty dependency.
- Exposed terminate_tree and wait_for_exit as public methods on pi_shell::process::Process.
2026-05-12 03:43:32 +02:00
can1357 530942c0b2 feat(pi-shell): enabled PTY shell runs in pi-shell via PTY runner
- Added PTY flags to ShellRunOptions and ShellExecuteOptions and defaulted pty to false when unset.
- Routed PTY-enabled shell runs to a dedicated PTY runner and executed commands via `sh -lc` in a PTY.
- Handled PTY output with a dedicated thread, Chunk/Done events, child PGID cancellation, and drain-timeout cleanup.
- Added Unix-only PTY assertions and non-Windows native coverage for `executeShell` with `pty: true` and `/dev/` output checks.
- Added `portable-pty = "0.9"` (and lockfile update) to enable PTY execution in pi-shell.
2026-05-12 01:00:31 +02:00
can1357 2e46257e9e feat: added listWorkspace binding and moved AGENTS.md lookup into tree
- Added `listWorkspace` native binding and API types, exporting bounded workspace trees with AGENTS.md candidates.
- Reworked `buildWorkspaceTree` and `buildDirectoryTree` to call `listWorkspace` with 5s timeout defaults.
- Replaced startup AGENTS.md discovery with workspace-tree-only scanning and removed legacy AgentsMdSearch session plumbing.
- Updated `WorkspaceTree` and system prompt context to expose `agentsMdFiles` and aligned tests/changelog expectations.
2026-05-10 07:59:53 +02:00
can1357 cc13fd3dce feat: added ScanDetail modes for fd, glob, and ast scans
- Added configurable `ScanDetail` modes across native fd/glob/ast flows and exposed `size` on `GlobMatch`/types.
- Added parallel grep processing with optional workers, buffered entry visits, and direct small-file reads.
- Changed `filesWithMatches` to stop at first match per file and report `totalMatches` as matching-file count.
- Fixed grep count/offset limits, timeout checks, and cancel handling by enforcing offsets in aggregate results.
- Updated PI_GREP_WORKERS and auto-absorb/read-summarize behavior/docs, including `read.summarize.prose` and pure-insert defaults.
- Added tests for filesWithMatches counts, cancel cases, and count-mode behavior, plus updated grep benchmark imports.
2026-05-06 17:00:45 +02:00
can1357 d76039a5bc fmt(natives): collapse single-line toContain assertion 2026-05-04 17:45:41 +02:00
can1357 36e2df7ed7 test(natives): align rust summarizer assertion with declaration_list elision 2026-05-04 17:44:24 +02:00
can1357 a019994091 feat: added tree-sitter summarizeCode support and N-API summary exports
- Updated read schema, path utilities, and dispatch to parse selectors from :raw/:L suffixes on path, removing standalone sel usage.
- Changed truncation/error notices to continue with :<nextOffset> and :1 guidance for read and sqlite pagination.
- Added summarizeCode support with tree-sitter summaries, read.summarize settings, and N-API Summary types/exports.
- Added tests and docs updates for path-embedded selectors, summary behavior, and explicit raw/offset SQL/read cases.
2026-05-04 04:24:33 +02:00
can1357 4eb286e2cc fix: type issues from esm 2026-05-02 08:11:35 +02:00
can1357 6105661b24 fix(natives): expose every public symbol via explicit module.exports
Fixes #892
2026-05-02 08:07:32 +02:00
can1357 7015f6dfd2 ci: drop zig 2026-04-30 14:25:01 +02:00
can1357 e39d3d9c76 fix(natives): detect compiled-binary mode via embedded-addon presence
Standalone Bun binaries on WSL (and any host where the user moves the
binary away from the build-host's checkout) failed to load
pi_natives.<platform>-<arch>*.node. The loader's isCompiledBinary
detection relied on two signals that are both false in shipped binaries:
process.env.PI_COMPILED (bun --define PI_COMPILED=true substitutes the
bare identifier, not property accesses on process.env) and
__filename.includes("$bunfs") (Bun retains the build-host absolute path
in __filename for required CJS modules — only import.meta.url is
rewritten). Detection therefore returned false, embedded-addon
extraction was skipped, and the only candidates probed were the
build-host nativeDir and execDir.

Make embedded-addon presence the authoritative compiled-mode signal
(it is null in the post-build --reset stub, populated when embed:native
ran for the standalone build), eagerly require the manifest, and
extract candidate-path computation into a pure helper covered by a
host-platform-agnostic unit test. Also fix the build-time --define so
process.env.PI_COMPILED is genuinely set at runtime as a defensive
fallback.

Fixes #823
2026-04-30 04:51:23 +02:00
can1357 5989fbf0d3 merge: PR #669 2026-04-13 00:31:38 +02:00
Bonobo 7c9edb610d fix(ci): tighten native x64 review guards 2026-04-11 01:46:16 +02:00
Bonobo 94c4c026cf fix(natives): eliminate unsafe avx512 x64 release artifacts
Fixes #601
2026-04-11 01:26:00 +02:00
can1357 b60c0cc3ce refactor: migrated tab width resolution to pi-utils with EditorConfig caching
- Extracted tab width resolution from pi-natives to pi-utils with EditorConfig caching support.
- Made tabWidth parameter required in native text functions (visibleWidth, truncateToWidth, wrapTextWithAnsi, sliceWithWidth, extractSegments).
- Removed process-wide tab width state management from pi-natives text module.
- Consolidated wrapper node promotion logic in chunk classification with abbreviated string representations.
- Removed path-clean and pathdiff dependencies from pi-natives.
- Updated all consuming packages to import tab width functions from pi-utils and pass explicit tabWidth parameters.
2026-04-10 14:18:36 +02:00
can1357 567197c34e feat: added macOS power assertion module to prevent idle-sleep during sessions
- Added MacOSPowerAssertion native module to prevent idle-sleep on macOS during active sessions.
- Integrated power assertion lifecycle management into coding-agent session with automatic start/stop.
- Implemented N-API bindings for macOS IOKit power assertions with RAII cleanup and cross-platform no-op fallback.
- Reformatted indentation from spaces to tabs across Rust chunk module for consistency.
2026-04-07 05:51:21 +02:00
can1357 9b78a6fa85 refactor(natives): migrated native bindings to NAPI-RS with auto-generated types
- Migrated native bindings from TypeScript wrappers to NAPI-RS generated modules with auto-generated type definitions and runtime enums.
- Replaced chunk tree API with stateful ChunkState class supporting render, edit, and resolve operations with improved error handling.
- Converted callback signatures to error-first pattern (error, result) for shell, PTY, glob, and grep operations.
- Introduced type-safe enums for MacOSAppearance, GrepOutputMode, KeyEventType, ImageFormat, and AstMatchStrictness replacing string literals.
- Refactored chunk tree implementation with dedicated modules for edit, indent, resolve, and state management with comprehensive validation.
- Moved clipboard utilities from native bindings to coding-agent package with improved OSC 52 and Termux compatibility.
2026-04-07 02:00:59 +02:00
can1357 9d5508980f perf(pi-natives): optimized pipe I/O to use async instead of blocking workers
- Replaced blocking Tokio worker with non-blocking async I/O for pipe reading to prevent stalling the runtime when child processes keep pipes open.
- Refactored background job termination to use process group and process tree APIs instead of deprecated signal parsing, improving reliability across Unix and Windows.
- Added non-blocking pipe registration and fcntl-based flag management for Unix platforms to enable efficient async I/O without blocking workers.
- Added test case verifying read_output cancellation before pipe EOF to ensure proper cleanup behavior.
2026-04-01 20:46:59 +02:00
zamo 69784b515f fix(natives): skipped grep special files (#565)
avoided opening fifos and other special filesystem nodes during grep and added fifo regressions in native and coding-agent tests.
2026-03-29 17:45:59 +02:00
can1357 b283abca73 feat(pi-natives): added automatic parenthesis escaping in regex patterns
- Added automatic escaping of unescaped parentheses in regex patterns when group syntax errors occur, enabling literal function call patterns like `fetchAnthropicProvider(` to work as search queries.
- Extracted regex matcher builder into separate function for reusability and error recovery logic.
- Added 2 test cases validating parenthesis escaping behavior for both escaped and literal parentheses.
- Fixed documentation formatting in sanitize_braces comment.
2026-03-27 12:11:17 +01:00