smashimo/oh-my-pi
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(coding-agent): scope computer capture limits

Browse Source
This commit is contained in:
Wolfgang Schoenberger
2026-07-25 03:09:39 -07:00
parent 22789c73fe
commit fc68288477
8 changed files with 170 additions and 43 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/computer-use.md
+7 -7
View File
@@ -16,8 +16,8 @@ computer:
enabled: true
backend: auto
display: all
maxWidth: 1280
maxHeight: 900
maxWidth: 1920
maxHeight: 1200
tools:
approvalMode: write
@@ -46,7 +46,7 @@ omp config set computer.enabled true
omp config get computer.enabled
```
Inside a running session, the `/computer` slash command (`/computer`, `/computer on|off|status`) toggles the tool for that session only; it never writes settings files. Backend, display, and image-size settings still snapshot when the session's desktop controller is created, so change those in config and start a new session.
Inside a running session, the `/computer` slash command (`/computer`, `/computer on|off|status`) toggles the tool for that session only; it never writes settings files. Backend, display, and image-size settings snapshot when the desktop controller is created. A model switch that crosses the Claude-family sizing boundary recreates the controller and resnapshots those settings; changing config alone does not, so start a new session after a settings change.
### Settings
@@ -55,8 +55,8 @@ Inside a running session, the `/computer` slash command (`/computer`, `/computer
| `computer.enabled` | `false` | Register the essential `computer` tool. |
| `computer.backend` | `auto` | `auto` or `native`. Both require a native backend; neither falls back to browser or software automation. |
| `computer.display` | `all` | Composite every active display, or select one numeric native display ID. |
| `computer.maxWidth` | `1280` | Maximum composite screenshot width in pixels. Larger values are capped at `1280` to preserve screenshot-relative coordinates across providers. |
| `computer.maxHeight` | `900` | Maximum composite screenshot height in pixels. Larger values are capped at `900` to preserve screenshot-relative coordinates across providers. |
| `computer.maxWidth` | `1920` | Maximum composite screenshot width in pixels. Claude-family image transports cap the effective width at `1280`; other models retain the configured limit. |
| `computer.maxHeight` | `1200` | Maximum composite screenshot height in pixels. Claude-family image transports cap the effective height at `896`; other models retain the configured limit. |
The first successful result lists each display ID, name, logical rectangle, screenshot-pixel rectangle, scale, and primary status. Use one of those IDs as a string when you want a single display:
@@ -107,14 +107,14 @@ A batch containing only `screenshot` and `wait` is observation-only. Any click,
## Screenshot coordinates and image mapping
Always choose coordinates from the immediately preceding successful computer result. Every coordinate action in one batch maps through that same prior frame. Do not use OS logical coordinates, CSS pixels, terminal cell positions, coordinates copied from another screenshot, or an in-batch `screenshot` marker as a new frame.
Always choose coordinates from the immediately preceding successful computer result returned by the current desktop controller. Every coordinate action in one batch maps through that same prior frame. A model switch that crosses the Claude-family sizing boundary recreates the controller and invalidates the prior frame, so capture a fresh screenshot before the next coordinate action. Do not use OS logical coordinates, CSS pixels, terminal cell positions, coordinates copied from another screenshot, or an in-batch `screenshot` marker as a new frame.
For each capture, OMP:
1. Enumerates the selected native displays and their global logical rectangles.
2. Captures every selected display at native pixel density.
3. Builds one logical bounding rectangle, including negative monitor origins.
4. Chooses one render scale that preserves the desktop layout and stays within the configured and provider-safe `maxWidth` and `maxHeight` limits.
4. Chooses one render scale that preserves the desktop layout and stays within the configured `maxWidth` and `maxHeight` limits. Claude-family image transports additionally cap the effective frame at `1280×896`, below their verified resize threshold; other providers retain the configured limits.
5. Places each resized display image into the composite and returns a PNG.
Each result's `displays` metadata maps both spaces: