smashimo/oh-my-pi
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(coding-agent): capability-aware inspect_image with tri-state mode and /vision toggle

Browse Source 
Replace the inspect_image.enabled boolean with inspect_image.mode
(auto|on|off, default auto). In auto the tool is registered only when
the active model lacks native image input, so vision-capable models
(e.g. kimi-code/k3) read images inline with their own capabilities
instead of delegating to a separate vision model. on/off force
registration regardless of model capability.

- New utils/inspect-image-mode.ts resolves the effective state from the
  /vision session override, the persisted setting, and model capability
- read tool re-evaluates the effective state per image read and
  re-renders its description, so it returns decoded image blocks again
  whenever inspect_image is hidden
- /vision [on|off|auto|status] slash command (modeled on /computer)
  overrides the mode for the current session only
- Tool set is reconciled on model switch with a status notice when
  inspect_image appears/disappears
- Legacy inspect_image.enabled true/false migrates to mode on/off
This commit is contained in:
alexis@epsilver.xyz
2026-07-27 15:52:38 -04:00
parent 222e3569bd
commit c33b98e260
14 changed files with 311 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/tools/inspect_image.md
+1 -1
View File
@@ -75,7 +75,7 @@ TUI rendering adds presentation-only truncation from `packages/coding-agent/src/
## Limits & Caps
- Supported detected input formats: `image/png`, `image/jpeg`, `image/gif`, `image/webp` (`SUPPORTED_IMAGE_MIME_TYPES` in `packages/utils/src/mime.ts`).
- Metadata sniff cap: `DEFAULT_IMAGE_METADATA_HEADER_BYTES = 256 * 1024` bytes. Format detection only reads up to 256 KiB from the file header.
- Availability is gated by `inspect_image.enabled`, default `false`, in `packages/coding-agent/src/config/settings-schema.ts` / `packages/coding-agent/src/tools/index.ts`.
- Availability is gated by `inspect_image.mode` (`auto`|`on`|`off`, default `auto`) in `packages/coding-agent/src/config/settings-schema.ts`, resolved with the session-scoped `/vision` override and the active model's image capability in `packages/coding-agent/src/utils/inspect-image-mode.ts` / `packages/coding-agent/src/tools/index.ts`. `auto` registers the tool only when the active model lacks native image input; the legacy `inspect_image.enabled` boolean migrates to `mode` (`true`→`on`, `false`→`off`).
- Upload input cap: `MAX_IMAGE_INPUT_BYTES = 20 * 1024 * 1024` bytes (20 MiB) in `packages/coding-agent/src/utils/image-loading.ts`.
- Auto-resize defaults in `packages/coding-agent/src/utils/image-resize.ts`:
- `maxWidth: 1568`
docs/tools/read.md
+1 -1
View File
@@ -188,7 +188,7 @@ URL selectors are parsed separately in `packages/coding-agent/src/tools/fetch.ts
### Images
- Image detection is metadata-based (`readImageMetadata()`).
- Max accepted image size is `20 MiB` (`MAX_IMAGE_INPUT_BYTES`, re-exported as `MAX_IMAGE_SIZE`). Larger files throw.
- If `inspect_image.enabled` is true, `read` returns metadata only (MIME, bytes, dimensions, channels, alpha) plus a suggestion to call `inspect_image`.
- If the effective `inspect_image` state is active (mode `on`, or `auto` with an active model that lacks native image input), `read` returns metadata only (MIME, bytes, dimensions, channels, alpha) plus a suggestion to call `inspect_image`.
- Otherwise it calls `loadImageInput()` and returns:
- a text note from the image loader
- an inline image block