Reported compositor-limited per-window input before reaching the AT-SPI focus path and stopped advertising foreground delivery on Wayland.
Removed the obsolete AT-SPI window-raise helper and updated public recovery guidance.
Fixes#7702
- Added window discovery, targeting, and frame validation for macOS, Windows, and Linux platforms.
- Updated computer tool and protocol definitions to support listing and targeting specific application windows.
- Refactored native computer tool exposure to function-only representation across models.
- Updated documentation and test suites to reflect window-scoped computer control capabilities.
- Replaced the separate GUI-linked pi_natives.desktop.linux-x64 addon with
a pure-Rust X11 backend (x11rb RustConnection capture via RandR/GetImage,
XTest input with keysym mapping) compiled into the core addon on every
published target; Linux arm64 and musl are now supported and headless
hosts load the addon unaffected.
- Removed the native-desktop-linux cargo feature, desktop_unsupported.rs,
lazy desktop loader, second napi build, desktop packaging/CI steps, GUI
build dependencies, and the now-unreferenced vendored libspa crate;
reverted setup-system-deps to main.
- Preserved the desktop input hardening semantics on the unified backend:
XTest layouts reject negative origins and coordinates beyond 0..=32767,
batch coordinates stay bound to the frame last returned to JS with
intermediate screenshots deferred, coordinate input requires a
previously returned frame, and failed chord releases still release
every held key.
- Enforced a 60s worker-side execute deadline (DESKTOP_DEADLINE_EXCEEDED):
no input is emitted after expiry and wait-heavy batches are rejected
upfront.
- Added int32 fail-closed validation for coordinates, drag points, and
scroll deltas at the JS ingress and gateway schema.
- Exposed computer to models without native OpenAI computer-use support as
a regular function tool with a typed GA action schema across OpenAI,
Azure, and Codex Responses providers, including named forced choice.
- Added the /computer slash command (on/off/status/toggle) for
session-only enablement via runtime tool registration in SessionTools.
- Updated docs, changelogs, and contract tests accordingly.
roboomp