Commit Graph
82 Commits
Author SHA1 Message Date
zero 3d7e1c5dbc test(natives): make ConPTY OSC stripping non-greedy
Review follow-up (#7896): [^\u0007] also matches ESC, so the greedy OSC
body match ran past an ST (ESC \) terminator to the last one in the
buffer, over-stripping everything between two ST-terminated OSCs. Harmless
for the single BEL-terminated title ConPTY emits today, but wrong the day
it emits more.
2026-08-07 17:10:36 +08:00
zero 82436b0e93 fix(natives): make Windows-host builds and addon loading work end to end
Runtime: the loader's AVX2 probe used [System.Runtime.Intrinsics.X86.Avx2]
via powershell.exe, which only exists on .NET Core — stock Windows PowerShell
5.1 raised TypeNotFound, so every Windows host silently loaded the baseline
addon and paid ~270ms for the spawn on the startup path. Ask the kernel via
bun:ffi (IsProcessorFeaturePresent(PF_AVX2_INSTRUCTIONS_AVAILABLE), ~0.5ms)
and fall back to pwsh-then-powershell for Node embeds. scripts/host-detect.ts
shared the same broken probe for build-time variant selection.

Build: `bun run build:native` on a win32 host died deep inside the bazel msvc
repo rules (linux/mac exec hosts only, by design). The `host` pseudo-target
now delegates to the local napi build against real VS Build Tools, and other
targets fail fast with guidance. build-bindings.ts resolves the @napi-rs/cli
JS entry from its manifest (the node_modules/.bin shim is a PE launcher Bun
cannot parse) and auto-appends VS Build Tools' bundled CMake/Ninja to PATH
via vswhere so a vcvars prompt is no longer required. Cap maudio at
opt-level=1 to dodge a rustc ICE codegenning MaybeUninit<ma_fence> for
x86_64-pc-windows-msvc under the pinned nightly (Bazel's older pin is
unaffected).

Compile: Bun.Glob.scan yields backslash paths on Windows; the legacy Pi
virtual module used them verbatim for export keys and generated identifiers,
producing invalid JavaScript in compiled-binary builds.

Tests: strip ConPTY negotiation escapes in the pty argv test; ignore the
bazel-oh-my-pi convenience symlink.
2026-08-07 16:54:02 +08:00
can1357 f687a074d8 fix(natives): refresh active cache directories
(cherry picked from commit c62702b89d1f29bc48067d64575c739c65456bc8)
2026-08-05 02:32:35 +02:00
can1357 817a3648f9 Merge PR #6953: fix(natives): avoid cross-version cache cleanup races (@roboomp) 2026-08-05 02:32:35 +02:00
can1357 455493dfad feat: introduced native file lock bindings for cross-process advisory locking
- Added native `FileLock` bindings supporting cross-process advisory locking on Linux, Unix, and Windows.
- Replaced directory-based file locking and custom stale-lock reclamation with OS-backed native locks.
- Updated TypeScript declarations, native bindings, and package documentation for the new API.
- Added comprehensive unit tests and fixtures validating single-owner constraints and process death handoff.
2026-08-03 16:09:09 +02:00
Wolfie 7b63fc2740 test(natives): cover fresh cache cleanup grace 2026-08-03 01:02:59 -07:00
can1357 58c645e32b fix(natives): aligned desktop capability test with optional displayServer
- napi omits the displayServer key when the backend reports None (headless
  CI hits the unavailable backend); the declared contract is
  displayServer?: string, so assert the value type instead of key presence.
2026-08-02 23:25:16 +02:00
can1357 ef852af986 feat(computer): implemented modular desktop backend and script workflows
- Replaced monolithic desktop native bindings and action batching with a modular cross-platform backend structure supporting Wayland, X11, macOS, and Win32.
- Updated the computer tool schema and supervisor to execute persistent JavaScript script runs with timeout clamping and asynchronous tool calling.
- Integrated accessibility (AX) tree snapshotting, node querying, and bounds-based hit testing across platform desktop layers.
- Added native clipboard bindings and updated coding-agent prompts, renderers, and tests to validate script-based computer workflows.
2026-08-02 19:46:25 +02:00
can1357 7c3b24ddf2 feat: implemented cross-platform window discovery and targeting capabilities
- Added window discovery, targeting, and frame validation for macOS, Windows, and Linux platforms.
- Updated computer tool and protocol definitions to support listing and targeting specific application windows.
- Refactored native computer tool exposure to function-only representation across models.
- Updated documentation and test suites to reflect window-scoped computer control capabilities.
2026-08-02 17:35:29 +02:00
David Andrewsandcan1357 726844b019 fix(natives): preserve workspace addons 2026-07-31 19:42:52 +02:00
David Andrewsandcan1357 89f00d9a9b fix(natives): handle uppercase node_modules paths 2026-07-31 19:42:52 +02:00
David Andrewsandcan1357 fefbdf072e test(natives): restore loader variant cache 2026-07-31 19:42:52 +02:00
David Andrewsandcan1357 e2dedf3ade test(natives): cover workspace loader policy 2026-07-31 19:42:52 +02:00
br411 c1c162225b fix(natives): prevent deep HTML crashes and silent truncation
html-to-markdown-rs 2.30 could recurse through pathological HTML until
a native stack overflow aborted the entire OMP process. Upstream 3.9.2
bounds those traversals and reports omitted subtrees as a
machine-readable DepthLimitExceeded warning.

Upgrade html-to-markdown-rs to 3.9.2. Upstream treats a depth-limited
conversion as a successful partial Markdown result plus a warning;
deliberately promote that warning to a rejected `htmlToMarkdown`
promise, allowing the Read tool to fall back without terminating OMP.
Normal conversions and unrelated warnings keep their existing behavior.

Add a rejection-contract test plus a child-process regression proving
OMP survives deeply nested and malformed input.
2026-07-31 00:33:06 +02:00
can1357 87bf250484 Merge PR #6606: fix(natives): preserve newer cache versions (@wolfiesch) 2026-07-26 15:45:31 +02:00
Wolfgang Schoenberger 857ba713bd fix(natives): preserve newer cache versions 2026-07-25 04:40:57 -07:00
usr-bin-roygbiv 40937af750 test(computer): exercise packaged desktop session 2026-07-25 00:42:23 +00:00
can1357 b76c07ece2 feat(pi-natives): added LiveWebRtcPeer and deviceCheckGenerateToken bindings
- Replaced puppeteer-based WebRTC with native LiveWebRtcPeer for cross-platform live audio delivery.
- Added cross-platform microphone capture via miniaudio and Opus codec integration for live encoding/decoding.
- Added Apple DeviceCheck attestation token generation via raw Objective-C FFI for macOS.
- Updated live session model to "gpt-live-1-codex" and default voice to "sol" across protocol and controller.
- Added LiveWebRtcPeer and deviceCheckGenerateToken to the public native bindings API.
2026-07-24 08:22:22 +02:00
can1357andusr-bin-roygbiv 681d7daf65 feat(computer): unified native addon, /computer toggle, function tool
- Replaced the separate GUI-linked pi_natives.desktop.linux-x64 addon with
  a pure-Rust X11 backend (x11rb RustConnection capture via RandR/GetImage,
  XTest input with keysym mapping) compiled into the core addon on every
  published target; Linux arm64 and musl are now supported and headless
  hosts load the addon unaffected.
- Removed the native-desktop-linux cargo feature, desktop_unsupported.rs,
  lazy desktop loader, second napi build, desktop packaging/CI steps, GUI
  build dependencies, and the now-unreferenced vendored libspa crate;
  reverted setup-system-deps to main.
- Preserved the desktop input hardening semantics on the unified backend:
  XTest layouts reject negative origins and coordinates beyond 0..=32767,
  batch coordinates stay bound to the frame last returned to JS with
  intermediate screenshots deferred, coordinate input requires a
  previously returned frame, and failed chord releases still release
  every held key.
- Enforced a 60s worker-side execute deadline (DESKTOP_DEADLINE_EXCEEDED):
  no input is emitted after expiry and wait-heavy batches are rejected
  upfront.
- Added int32 fail-closed validation for coordinates, drag points, and
  scroll deltas at the JS ingress and gateway schema.
- Exposed computer to models without native OpenAI computer-use support as
  a regular function tool with a typed GA action schema across OpenAI,
  Azure, and Codex Responses providers, including named forced choice.
- Added the /computer slash command (on/off/status/toggle) for
  session-only enablement via runtime tool registration in SessionTools.
- Updated docs, changelogs, and contract tests accordingly.
2026-07-24 01:40:05 +00:00
usr-bin-roygbiv b9504f65e7 feat: add native Codex computer use 2026-07-24 01:40:04 +00:00
can1357 52ad6516ef feat(diff): implemented native UTF-16 diff processing and removed jsdiff
- Implemented native UTF-16 text processing in Rust diff module with support for unpaired surrogates.
- Removed `similar` crate from Rust workspace and `diff` npm package from coding-agent, hashline, and natives.
- Removed jsdiff fallback wrappers and `isWellFormed()` guards from TypeScript diff implementations.
- Added comprehensive test suite for native diff functions covering random inputs and edge cases including surrogates and emoji.
- Renamed model `codex-auto-review` to `gpt-5.3-codex-spark` with updated pricing and context window.
2026-07-23 01:35:31 +02:00
Wolfgang Schoenberger ee6717872c test(natives): cover ill-formed UTF-16 rejection and jsdiff fallback 2026-07-22 04:32:46 -07:00
Wolfgang Schoenberger 81aa4ad27f test(natives): add seeded randomized word-diff parity coverage 2026-07-22 02:58:33 -07:00
Wolfgang Schoenberger 5f41203f00 test(natives): add jsdiff parity suite for native line/word diff 2026-07-22 01:25:31 -07:00
roboomp 044b74594f fix(launch): prevented finite PTY start hangs
- Reported the spawned PTY child PID through the native start callback.
- Replaced broker PID-file polling with the authoritative spawn event.
- Covered finite PTY startup without the legacy handoff in integration tests.

Fixes #5996
2026-07-18 14:54:53 +00:00
can1357 2c355102ce chore: applied biome formatting to merged sources 2026-07-16 03:52:48 +02:00
can1357 a29627142c fix(natives): verify disk before stale-process diagnosis 2026-07-16 03:32:03 +02:00
can1357 29b6649ff9 merged PR #5423: fix(natives): distinguish process-stale from disk-stale sentinel mismatch 2026-07-16 03:32:03 +02:00
roboomp 90c4726bae fix(launch): passed Windows PTY arguments directly
Added a direct-argv PTY entry point and used it for Windows launch sessions so portable-pty no longer re-quotes cmd.exe command text.

Rejected direct .bat and .cmd applications with guidance to use cmd.exe /c.

Fixes #5416
2026-07-14 20:17:03 +00:00
roboomp e4a10450ec fix(natives): distinguish process-stale from disk-stale sentinel mismatch
A long-lived process that survives an in-place upgrade keeps the previous
pi-natives NAPI addon resident. A tab worker spawned afterwards runs the
new JS loader, which expects the new version sentinel, but require returns
the resident old exports carrying the prior sentinel. validateLoadedBindings
previously reported "reinstall to re-sync" for this case even though disk was
already consistent, so only a restart helped.

Detect a versioned __piNativesV* export other than the expected one on the
loaded bindings and report that omp was upgraded mid-session and must be
restarted, reserving the reinstall guidance for genuinely disk-stale addons.

Fixes #4812
2026-07-14 16:32:04 +00:00
can1357 46b8ee737f feat(catalog): added Grok 4.5 model support
- Added Grok 4.5 to the model catalog and identity helper.
- Updated pricing and configuration settings for existing Grok models.
2026-07-09 22:43:34 +02:00
can1357 4f8c341745 test(natives): pinned bash timeout output-bridge crash contract
Regression test for the WSL crash where a timed-out bash command got OMP
OOM-killed: an output-heavy command (in-process 'yes | cat') with a short
timeoutMs must resolve near its deadline with bounded RSS, on both
executeShell and Shell.run. On the pre-fix bridge the same harness measured
~4 GiB RSS growth and minutes-late resolution (JS event loop starved by the
unbounded callback flood); with the bounded backpressured bridge it resolves
at the deadline with flat memory. Also added the user-facing changelog entry
for the crash symptom.

Fixes #4866
2026-07-09 18:39:56 +02:00
roboomp 4b2c3d524c fix(natives): made variant detection robust in worker threads
`pi_natives` failed to load in Bun worker threads on macOS x64 when the
host built only the `modern` (AVX2) variant. The runtime detector's
`child_process.spawnSync("sysctl", ...)` returned null from the worker
even though the build-time detector (`scripts/host-detect.ts`) succeeded
in the parent shell, so `loadNative()` resolved `variant=baseline` and
searched a file list that excluded the `pi_natives.darwin-x64-modern.node`
the build had actually produced.

Two compounding root causes in `packages/natives/native/loader-state.js`:

- `runCommand` only used `child_process.spawnSync`, which is the path
  observed to fail under Bun's worker shim on darwin. The build-time
  detector uses `Bun.spawnSync` and works fine.
- The darwin branch looked up `sysctl` via PATH. Login shells supply
  `/usr/sbin` so the build picks it up, but worker/embedded spawn
  contexts can ship without it.

Fix:

- `runCommand` now prefers `Bun.spawnSync` (matches the build-time
  detector) and falls back to `child_process.spawnSync` for non-Bun
  embeds.
- The darwin branch tries `/usr/sbin/sysctl` before bare `sysctl`.
- New private env key `__PI_NATIVE_VARIANT_CACHE`: once any context
  resolves the variant (the main thread does first), it is written
  there. Bun workers and child subprocesses inherit `process.env` at
  spawn, so they read the cache and skip detection — sidestepping the
  worker-context spawn flakiness end-to-end.
- New exported pure helper `selectCpuVariant({ arch, override, env,
  detectAvx2 })` codifies the override > cache > detect order and
  returns the cache write hints so the helper itself stays
  side-effect-free.

Regression test `packages/natives/test/issue-3238-repro.test.ts` pins
every branch of the resolution order, including the file-list shape
that surfaced the bug.

Fixes #3238
2026-06-22 07:17:50 +00:00
can1357 1f3f3cf5d1 feat: added ruby and julia language support to coding-agent
- Implemented persistent execution backends for Ruby and Julia using dedicated kernel processes and NDJSON-based IPC.
- Integrated language-specific prelude environments, runtime path resolution, and security-focused environment variable filtering.
- Exposed configuration options, tool schema updates, and lifecycle management for seamless agent interaction with both languages.
- Added comprehensive integration tests and updated prompt documentation to support the new evaluation capabilities.
2026-06-22 06:13:01 +02:00
ryjm c2a14a0afc fix(ast): infer extensionless Emacs init files 2026-06-15 18:58:55 -07:00
ryjm 18f11f5bfc feat(ast): add Emacs Lisp tree-sitter support
Register tree-sitter-elisp in the shared AST language registry so
.el files infer the emacs-lisp grammar across blockRangeAt,
summarizeCode, astGrep/astEdit, and native aliases.

This fixes edit-tool block operations on top-level Emacs Lisp forms
instead of returning unsupported-language block errors.

- Add canonical emacs-lisp aliases and .el extension inference.
- Teach summaries to fold Lisp forms without grouping arbitrary lists.
- Map .el rendering and highlighting aliases through coding-agent/native.
- Cover defun, ERT, use-package, with-eval-after-load, pcase,
  summary, astMatch, astEdit, and edit-tool insertion paths.
- Document the language and update package changelogs.
2026-06-15 18:41:24 -07:00
roboomp 001a3d1204 fix(natives): cleaned stale native cache versions
Removed stale per-version native cache directories after successful addon loads so updates do not leave old ~/.omp/natives/<version> trees behind.

Added a focused loader regression test for keeping the current version directory and non-directory files while removing stale version folders.

Fixes #2560
2026-06-14 11:57:56 +00:00
can1357 3d5572a0fc test(natives): documented AST conditions and expanded astMatch tests
- Updated `docs/ttsr-injection-lifecycle.md` to document `astCondition` behavior, including registration rules and tool-stream matching.
- Extended `packages/natives/test/native.test.ts` with new `astMatch` coverage for Smart matching, metavariable consistency, parse errors, and empty-language rejection.
2026-06-08 15:00:28 +02:00
can1357 a89bb76444 fix(tui): preserved visible ANSI content when truncating terminal rows
- Replaced the TUI row truncation path with per-character and ANSI-sequence iteration that tracks visible cells before clipping output.
- Added ANSI helpers to parse sequence boundaries and preserve OSC66 visible payloads while enforcing max source length during truncation.
- Added natives regression coverage for Ghostty super+alt backspace key matching and parsing.
2026-06-08 01:22:26 +02:00
roboomp c8919f3619 fix(natives): stream sorted glob admits only
Tied uncached sortByMtime onMatch callbacks to bounded heap admission so live progress and timeout partials reflect the same mtime-ranked set the call returns, with the JS-side mtime sort+truncate converging on the native top-N.
2026-06-03 06:26:26 +00:00
roboomp f73745235c fix(natives): streamed bounded glob progress
Preserved bounded onMatch progress during uncached sorted glob traversal so timed-out find scans can still return partial matches without unbounded callback growth.
2026-06-03 06:20:25 +00:00
roboomp d3477372a2 fix(natives): bounded sorted glob scans
Kept uncached sort-by-mtime glob traversal bounded to maxResults and emitted onMatch callbacks only for returned matches so broad find scans cannot grow parent memory independently of the limit.

Fixes #1761
2026-06-03 06:15:26 +00:00
can1357 34e099148a feat(natives): added per-platform leaf package publishing for native addons
- Added `gen-npm-packages.ts` script to generate per-platform leaf packages under `npm/-/`.
- Updated CI release publisher to generate and publish leaf packages before rewriting the core manifest with pinned `optionalDependencies`.
- Core package now ships only JS loader and declarations; installs fetch only the host platform's `.node` binary.
2026-05-30 17:08:23 +02:00
can1357 cc258b1757 feat(natives): added embedded addon tarball extraction in natives
- Added embedded addon tarball output in `embed-native.ts` using `embedded-addons.<platform>.tar.gz` artifacts.
- Added metadata-rich addon types with `size`, `filePath`, and optional `archive` fields.
- Updated extraction to prioritize archive unpacking and skip cached `.node` files when sizes match.
- Added `extractEmbeddedAddonArchive()` with archive parsing and safe validation of archive entry names and kinds.
- Adjusted release profile to disable line-table generation and strip symbols in `Cargo.toml`.
- Added CI-only ELF validation for forbidden sections and regression coverage for issue-823 archive extraction.
2026-05-28 11:48:32 +02:00
roboomp de5b0a51e2 fix(bash): forcefully terminated canceled shells
Added Rust-side descendant termination on shell cancellation paths so aborts and timeouts escalate to SIGKILL even if brush cleanup stalls.

Covered SIGTERM-ignoring shell workloads in native tests.

Fixes #1347
2026-05-25 18:13:49 +00:00
can1357 90b134ca4c test: replaced real timers and sleeps with deterministic test hooks
- Added `providerRetryWait` and `retryWait` hooks to stream/usage options so tests bypass real scheduler delays.
- Parameterized GitHub Copilot poll intervals and Copilot model retry base delay for fast test execution.
- Replaced `Bun.sleep`/`setTimeout` polling loops with `AbortSignal` event listeners in agent session tests.
- Consolidated auth-gateway E2E helpers into a shared `test/helpers` module, eliminating duplicated `checkGatewayAvailable` implementations.
- Migrated credential-disabled tests from SQLite-backed stores to an in-memory store, removing temp-dir lifecycle overhead.
2026-05-17 04:02:09 +02:00
can1357 22848e89ff refactor(natives): removed sanitizeText from pi-natives, use JS impl
- Moved @oh-my-pi/pi-natives from devDependencies to dependencies in pi-utils.
- Deleted Rust sanitizeText implementation and its JS bindings/types.
- Updated sanitize benchmark to use pi-utils implementation as baseline.
2026-05-16 20:21:36 +02:00
can1357 dac28f66e4 feat: added windows native staging candidates before runtime paths
- Added Windows node_modules staging checks so non-compiled addons are copied into versioned native directories.
- Updated loader candidate resolution to prefer embedded/staged/versioned candidates before default runtime paths when staging is enabled.
- Added runtime version-sentinel export __piNativesV15_0_1 and load-time validation to avoid stale binary drift.
- Added typing and release-flow updates to expose staging options and keep per-release sentinel IDs aligned.
2026-05-14 23:24:14 +02:00
can1357 f1f6516056 refactor: reorganized exports and removed obsolete helper branches
- Removed export leakage by demoting many helper and const symbols to module-local scope.
- Renamed underscore-prefixed internals and cache fields, then updated related references and `satisfies never` checks.
- Deleted obsolete logic branches and helpers, including harmony-stream interruption flow and unused benchmark runtime helpers.
- Updated Biome config and manifests by broadening lint coverage and removing an unused `@napi-rs/cli` dev dependency.
- Adjusted tests and utilities to use renamed test helpers and remove redundant private test-only helpers/locals.
2026-05-14 04:36:19 +02:00
can1357 975941aba4 chore: remove garbage tests 2026-05-12 04:09:33 +02:00