Commit Graph

52 Commits

Author SHA1 Message Date
can1357 19c0afcc0d feat: implemented external thinking flags and transport reasoning controls
- Added the `--external-thinking` CLI flag alongside model capability checks to gate external thinking tool availability.
- Updated Anthropic and Google transports to honor `forceReasoningOff` for native thinking-off controls.
- Renamed the `thoughts` property and parameter to `notes` across think fixtures, tools, and tests.
- Updated system prompt instructions and test suites to verify transport-specific thinking and tool activation.
2026-08-12 02:23:17 +02:00
can1357 5b7a606d8f Merge PR #8257: fix(snapcompact): extend high-res frame tier to opus 5+ (@roboomp) 2026-08-12 01:53:36 +02:00
can1357 c101452bb5 refactor: restructured and condensed agent prompts and system instructions
- Refactored and condensed numerous system prompts, agent instructions, and tool documentation files across packages.
- Streamlined workflow rules, formatting constraints, and execution guidelines for improved clarity and brevity.
- Updated discovery rules, recommendation criteria, and syntax standards in prompt templates.
2026-08-12 01:48:14 +02:00
roboomp 23a0543e38 fix(snapcompact): kept versionless fable aliases high-res
Versionless Fable/Mythos aliases (bundled claude-fable-latest) never parse a numeric version, so the parser path missed them and they fell to the 1568px default. Restored the /claude.*(fable|mythos)/i rule ahead of the generic Claude rule, mapping to the shared high-res tier.

Added regressions for anthropic/claude-fable-latest and its ~ prefixed alias.
2026-08-11 19:53:23 +00:00
roboomp 868de3c09a fix(snapcompact): used catalog parser for opus tiers
Replaced the local Opus version regex with the shared catalog Anthropic identity and semantic-version parser. This recognizes both kind-first and version-first model IDs, provider proxy prefixes, and multi-digit minor versions while preserving the 4.7 floor.

Added regressions for anthropic--claude-4.8-opus and claude-opus-4-10.
2026-08-11 19:48:04 +00:00
roboomp 913fee1d39 fix(snapcompact): extended high-res frame tier to opus 5+
MODEL_VARIANTS gated the 1932px frame tier on /claude-?opus-?4[.-][7-9]/i,
so claude-opus-5 fell through to the generic /claude/i entry and rendered at
the 1568px default reserved for older lines that downscale. The 1932px tier
tracks the Anthropic 4,784 visual-token cap — a family-wide billing property
opus-5 shares with opus-4-8 — so the version bound was stale rather than a
per-version eval gap.

Widened the pattern to cover Opus 5 and later; 4.0-4.6 still keep the safe
1568px default. Documented why 1932 specifically (largest square not
downscaled under the 4,784-patch cap, kept below the 2000px per-image limit).
Added regression assertions for opus-5/opus-6 (high-res) and opus-4-6
(default).

Fixes #8256
2026-08-11 19:40:57 +00:00
can1357 ee335ce9c2 feat(snapcompact): implemented squad context compression benchmark
- Added a new SQuAD-based context-compression benchmark script for evaluating recall conditions.
- Added model and shape command-line arguments along with pricing and shape configurations.
- Updated default model variants and added an unknown billing family to shape resolution.
- Updated shape resolution and model resolution tests to verify the new defaults.
2026-08-04 02:20:03 +02:00
roboomp 6c35f66231 fix(snapcompact): reported all foveated frame widths
- Collected distinct rendered frame widths for resume summaries.
- Covered mixed HQ/LQ archives with the existing foveation regression.
- Documented the fix in the snapcompact changelog.

Fixes #6712
2026-07-26 14:44:17 +00:00
can1357 4362cdf6ed fix(snapcompact): scrubbed legacy ¶think: sections from prior archives when thinking is excluded
- Re-compaction unfolds the prior archive's kept source verbatim, so archives written before includeThinking existed kept replaying reasoning to Claude (reasoning_extraction) even after the serializer fix; the prior text is now scrubbed when includeThinking is false, healing poisoned sessions at their next compaction.
2026-07-23 23:07:08 +02:00
roboomp 5a70eda7d7 fix(compaction): stop feeding reasoning back to Claude summarizer
Both compaction serializers reproduced prior assistant reasoning as text
bound for a Claude target, tripping Anthropic's reasoning_extraction
refusal and wedging Fable 5 sessions:

- context-full: serializeConversation rendered thinking verbatim inside
  <thinking> tags via the anthropic dialect renderer. Now drops thinking
  blocks when the summary target dialect is anthropic; other dialects
  (e.g. Harmony) keep native reasoning.
- snapcompact: emitted ¶think sections baked into replayed archive
  frames. Added an includeThinking serialize option (default true) and
  wired the agent session to disable it for Anthropic-dialect models.

Fixes #6093
2026-07-23 20:38:27 +00:00
can1357 711fa43123 feat(snapcompact): switched to compact scope markers for archived transcripts
- Replaced markdown-style headings with concise `¶user:`, `¶think:`, `¶ai:`, and `¶call:` scope markers.
- Updated the serializer to merge consecutive messages or blocks of the same type under a shared prefix.
- Updated the documentation prompt to reflect the new compact formatting and scope rules.
- Added regression tests verifying scope merging and correct formatting of tool calls and intents.
2026-07-13 01:35:40 +02:00
roboomp a8add580d9 fix(snapcompact): selected silver for cjk auto shape
- Made resolveShapeForText choose silver16-bw for CJK-heavy auto transcripts while preserving explicit variants and unsafe glyph protection.
- Added silver16-bw to the snapcompact shape settings submenu and renamed unsupported-glyph warnings.
- Covered auto shape selection, explicit variant precedence, unsafe glyph scans, and settings option parity.

Fixes #4486
2026-07-04 03:54:55 +00:00
can1357 fc24a70f5e Merge branch 'farm/ad81bc64/retry-anthropic-internal-server-error'
fix(compaction): cap snapcompact frame payloads (#3866)

Bound rebuilt snapcompact image payloads by a per-request base64 byte
budget so long sessions stop re-sending multi-megabyte standing image
archives on every provider request; auto-compaction falls back to
context-full summaries when snapcompact output is too large.

Resolved snapcompact.ts conflict against the main font-rendering refactor
by keeping both renderabilityProbeText and the frame-budget helpers.
Fixed historyBlocks to emit the omitted-frame notice before the kept
(newer) images, since the byte budget drops the oldest frames — keeping
reconstructed blocks oldest-to-newest (addresses Codex P2 review).

Fixes #3792
2026-06-30 09:59:03 +02:00
roboomp 39688620f8 fix(compaction): kept newest snapcompact frames
When legacy snapcompact archives exceed the per-request byte budget, retain frames from the newest end of the archived middle and restore oldest-to-newest order for the kept subset.
2026-06-30 05:50:12 +00:00
roboomp 88308f105f fix(compaction): capped snapcompact frame payloads
Bounded persisted snapcompact image archives by base64 byte size so large sessions stop re-sending multi-megabyte frame walls on every provider request.

Auto snapcompact now falls back to context-full summaries when rendered frame payloads exceed the byte budget, and legacy oversized archives omit over-budget frames during LLM context rebuilds.

Fixes #3792
2026-06-30 05:36:14 +00:00
can1357 242705c06c feat: enabled dynamic font rendering with CJK and unicode support
- Added Silver.ttf TrueType font support to `pi-natives` with automated fallback logic for bitmap font rendering.
- Implemented wide code point detection and cell-width calculation to improve CJK character handling and layout.
- Introduced dynamic font-aware preflight probing via `resolveShapeForText` and `renderabilityProbeText` for better font selection.
- Enabled semantic emoji folding and improved text normalization to handle non-Latin characters and emoji filtering.
2026-06-30 02:49:31 +02:00
can1357 93b80e5b1b refactor: consolidated duplicate
- Consolidated duplicate `stripSnapcompactPreserveData` functions into `snapcompact.stripPreservedArchive`.
- Added unit tests to verify archive removal and empty state collapse behavior.
2026-06-26 14:28:49 +02:00
OpenAI GPT-5.5 5c4e269900 fix(agent): migrate snapcompact archive on context compaction
Co-Authored-By: OpenAI GPT-5.5 <noreply@openai.com>
2026-06-26 08:15:17 +00:00
can1357 dd7c05a588 fix(snapcompact): adjusted umans provider image budget
- Added a 10-image budget for the umans provider to match its request cap.
- Updated the unit tests to verify the correct budget assignment for umans.

Fixes #3227
2026-06-22 04:46:27 +02:00
can1357 29f7b57fde feat: migrated core rendering and context transformation to async
- Updated `render`, `renderMany`, and native snapcompact methods to return promises, ensuring scalable async execution.
- Refactored `transformProviderContext` and `buildSideRequestContext` to support asynchronous operations in agent loops.
- Integrated `Promise.all` for improved concurrency when processing frame rendering and rendering batch operations.
- Updated all internal call sites, SDK hooks, and test suites to accommodate the asynchronous API signatures.
2026-06-21 00:32:13 +02:00
can1357 78a94c5a93 feat(snapcompact): improved archive text preservation and continuity
- Unify `previousText` resolution to correctly concatenate `textHead` and `textTail` during re-compaction.
- Ensure summary fallback logic correctly handles non-text legacy archives.
- Add test coverage for cross-compaction text retention and legacy archive continuity.
- Enable snapcompact strategy in agent plan reference re-injection tests.
2026-06-21 00:11:02 +02:00
can1357 7d28c60c86 refactor: renamed intent field from _i to i
- Renamed the global `INTENT_FIELD` constant from `_i` to `i`.
- Updated documentation strings, type annotations, and test expectations across packages to reflect the new field name.
- Ensured consistent usage of the constant in tool schema construction and intent serialization.
2026-06-19 16:42:33 +02:00
cagedbird043 5ff5f110af fix(agent): dynamically fallback from snapcompact to text summary on high CJK/non-ASCII rates 2026-06-19 21:50:16 +08:00
can1357 63b2c9603a refactor(snapcompact): replaced legacy frame pinning with full text-sourced re-rendering
- Removed legacy `pinnedFrames` logic and state, simplifying compaction to unconditionally re-render from the full source archive.
- Updated `historyBlocks` to reliably order archive regions (text head, imaged middle, text tail) based on the re-rendered state.
- Updated documentation and summary templates to reflect the transition to text-first compaction and the use of the new `FILES` section.
2026-06-19 07:48:46 +02:00
can1357 afef9fc68a feat(tui): added configurable box border support
- Added `BoxBorder` interface and optional border support to the `Box` component.
- Implemented `setBorder()` to allow dynamic toggling of box outlines.
- Added intelligent border dropping logic to prevent overflows in narrow containers.
- Added comprehensive unit tests for rendering, styling, and boundary conditions.
2026-06-19 07:39:24 +02:00
can1357 b137764f0c feat: implemented text-first foveated snapcompact archive layout
- Implemented text-first archive structure that incorporates bounded source text with foveated image frames (HQ edges, LQ middle) to improve context quality.
- Updated snapcompact core logic to use `historyBlocks` for reconstruction, transitioning away from reliance on previous PNG frame inheritance.
- Increased default frame limits to 80 and adjusted token estimation constants (5024) to enhance context budget accuracy.
- Added `pruneToolDescriptions` and improved multi-block summary token estimation to optimize tool spec usage.
2026-06-19 07:38:54 +02:00
can1357 f01b5e4c27 refactor(snapcompact): updated MAX_FRAMES_DEFAULT to 80 to better
- Update `MAX_FRAMES_DEFAULT` to 80 to better utilize high-capacity model context windows.
- Remove `providerFrameBudget` from `snapcompact` to decouple archival limits from provider-specific image caps.
- Update `compact` logic to treat `maxFrames` as a hard upper bound rather than a provider-clamped limit.
2026-06-19 06:33:19 +02:00
can1357 4f03180ae6 refactor(deps): moved intent field constant to pi-wire
- Moved the `INTENT_FIELD` constant from `@oh-my-pi/pi-agent-core` to the specialized `@oh-my-pi/pi-wire` package to permit broader usage across the monorepo.
- Updated all references across `agent`, `ai`, `coding-agent`, `collab-web`, and `snapcompact` packages to import the constant from the new location.
- Added `@oh-my-pi/pi-wire` as a dependency to all affected packages.
2026-06-19 04:58:29 +02:00
can1357 c224ef4df5 feat: standardized elision markers and improve transcript viewer robustness
- Standardized elision markers across all tool outputs and filters to use cohesive `[...N [type] elided...]`, `[...Nln elided...]`, and `[...xB elided...]` syntax.
- Updated documentation, prompts, and test expectations to reflect the unified elision format.
- Improved transcript viewer robustness by preventing content aliasing through path-inclusive signature hashing.
- Added logic to clear stale transcript content when associated session files are deleted, accompanied by verifying test cases.
2026-06-19 04:35:18 +02:00
can1357 d133add0e4 feat(snapcompact): improved text normalization and elision formatting
- Expand `CHAR_FOLD` dictionary to include more punctuation, dashes, arrows, and bullets.
- Added `foldToAscii` function using NFKD decomposition to better handle compatibility characters like fullwidth text, ligatures, and vulgar fractions.
- Updated elision markers from `[... N chars elided ...]` to `[... Nch elided ...]` to reduce character count and improve visual clarity.
2026-06-19 04:12:06 +02:00
can1357 2b843dc744 feat(coding-agent): reset advisor settings for protocol hosts
- Included advisor syncBacklog and immuneTurns in the default-resetting configuration for protocol hosts.
- Updated the lazy startup tests to verify that these settings are reset to defaults rather than inheriting user-defined values when a protocol host initiates.
2026-06-19 03:48:37 +02:00
can1357 3b0958509f feat(snapcompact): ignored empty text and thinking blocks
- Prevented the serialization of empty or whitespace-only content blocks to avoid extraneous headers.
- Updated the prompt template to improve clarity for message roles and formatting.
2026-06-19 03:46:42 +02:00
can1357 4a6d68d36d feat(snapcompact): filtered url-like paths and standardize heading markers
- Prevent file operations containing URL schemes from being included in the generated file lists.
- Introduce `HEADING_MARKER` constant to provide a consistent suffix for conversation headers.
2026-06-18 20:05:10 +02:00
can1357 ad14074701 feat: updated conversation rendering and tool result formatting
- Implemented structured markdown role headings and tool result merging to improve conversation readability.
- Added explicit `<out>` tags for tool result wrapping and enhanced rendering for thinking blocks.
- Refactored authentication snapshot validation to use manual structural checks instead of schema dependencies.
- Fixed instability in settings overlay scrolling and addressed assistant message splitting issues.
2026-06-18 19:54:01 +02:00
can1357 2a7cb56abd feat(agent): rendered conversation logs with preferred tool syntax
- Updated compaction, branch summarization, and session dump formatting to pass preferred model tool syntax into conversation serialization.
- Enhanced shared serializers to render assistant tool calls and tool results through grammar envelopes when syntax is available, with the prior compact format as fallback.
- Aligned prompt, preview, and test fixtures to the new transcript tags: `[Think]`, `[Tool Call]`, and `[Tool Result]`.
2026-06-15 12:32:47 +02:00
microproofs 50eb490d31 fix(snapcompact): budget openai-codex images
Signed-off-by: microproofs <kwhitemsg@gmail.com>
2026-06-14 16:31:27 -04:00
can1357 233ee852cd feat(snapcompact): added spacing-tuned snapcompact variants and updated provider defaults
- Added 8on22-bw and 11on16-bw variants with spacing-tuned defaults for snapcompact.
- Changed provider defaults/mappings to use 11on16-bw for Anthropic and 8on22-bw for OpenAI/Google.
- Updated model frame rules so claude/fable/opus and gemini/gpt families map to new shapes and 2048 billing behavior.
- Fixed non-multiplexer resize dragging by using alternate screen during transient resize frames.
2026-06-14 07:02:27 +02:00
can1357 28df38ed37 feat(agent): added useless-result tagging and compaction dropping of tool outputs
- Added optional `useless` flags to tool result types and payload builders.
- Added `pruneUseless` and `dropUeless` options to control uneventful result pruning.
- Changed compaction and shake passes to prune or ignore non-error useless tool results.
- Changed conversation serialization to omit useless toolCall/toolResult pairs from output.
- Added coverage for useless tagging, pruning, and serialization behavior.
2026-06-12 16:26:38 +02:00
can1357 b845ef6362 feat(snapcompact): implemented model-aware compaction billing and frame-size resolution
- Implemented model-specific frame-size billing for Anthropic, OpenAI, and Google.
- Changed compaction shape resolution to bind model variants to ideal frame sizes.
- Updated tests and docs to reflect new frame-size and budget behavior.
2026-06-12 09:27:03 +02:00
can1357 a9229b0f26 feat(snapcompact): clipped snapcompact PNG frame height to rendered text rows
- Adjusted snapcompact rendering to compute used rows from text, dim toggles, and doc line breaks, then derive output height from usedRows x lineRepeat x cellHeight.
- Updated indexed and RGB PNG encoders to accept explicit canvas width and height so native renders now emit non-square frames matching actual content.
- Expanded Rust and TypeScript tests and updated docs/changelogs to assert and describe variable-height frame behavior.
2026-06-12 09:11:41 +02:00
can1357 8eeab0a877 feat(packages/snapcompact): added model-aware shape + text-tail budgets
- Added support for `{api,id}` `ShapeTarget` in `resolveShape`, deriving variants from model IDs.
- Added provider budget APIs/constants and `providerFrameBudget` clamping with `MAX_FRAMES`.
- Added `Archive.textTail` and moved overflow pages into plain-text tail folding across frames.
- Updated summary prompt rendering to show continuation only when `textTail` exists and append it as text.
2026-06-12 09:00:04 +02:00
can1357 870d2b8981 feat(snapcompact): switched the Anthropic default shape from 8x8r-bw to 6x12-dim
- Repointed `SHAPES.anthropic` and the `anthropic-messages`/unknown-API fallback in `resolveShape` at the `6x12-dim` variant: production mono eval on claude-fable scored f1 .840 vs .877 for the repeated grid (within noise at n=25) at 37% lower cost, with no refusals.
- Reworded the `snapcompact.shape` descriptions in `settings-schema.ts` to drop per-provider eval-winner claims from the variant help text.
- Updated `snapcompact.test.ts` (new `6x12-dim` default render/compact assertions, `8x8r-bw` exercised via `resolveShape`), `snapcompact-inline.test.ts` frame math for the new geometry, and the `docs/compaction.md` shape sentence.
- Amended the snapcompact `[Unreleased]` entry that said the Anthropic default stayed `8x8r-bw` and added the default-switch entry.
2026-06-12 08:21:48 +02:00
can1357 300c1ada30 feat(coding-agent): added bench command and updated default compaction shapes
- Added a new `bench` CLI command with multi-model selectors and new options.
- Implemented `runBenchCommand` validation, per-run session handling, and failure exit reporting.
- Updated default compaction shapes to `8x8r-bw` and `doc-8on16-sent-dim` in code and schema.
- Documented `bench` flags, per-run errors, failure counts, and exit behavior.
2026-06-12 08:00:03 +02:00
can1357 a094b794bc fix: updated OpenAI defaults and corrected catalog grouping behavior
- Resolved OpenAI shape resolution to `openai` and default to `8on16-bw`.
- Fixed catalog generation to collapse effort tiers before provider grouping.
- Updated help text and schemas to describe `8on16-bw` as the OpenAI auto default.
- Added production `render_pages` and `mono_prod` scripts for end-to-end QA output.
2026-06-12 07:49:23 +02:00
can1357 118f32d37f feat(snapcompact): added the six research-eval winning frame variants
- Added `6x12-dim`, `8x13-bw`, `8on16-bw`, `doc-8on16-bw`, `doc-8on16-sent`, and `doc-8on16-sent-dim` to `SHAPE_VARIANTS`, backed by new `Shape` fields `stretch`, `columns`, and `stopwordDim` and the X.org `6x12`/`8x13` fonts.
- Added `dimStopwords()` (high-frequency function words printed in dim ink via zero-width markers, skipping already-dim spans) and `wrap()`, the greedy word-wrap used to typeset doc-layout pages; `geometry`/`render`/`renderMany`/`frames`/`compact` paginate doc shapes into `2 * rows`-line pages and persist `columns`/`stopwordDim` on compaction frames for mixed-shape detection.
- Changed `normalize()` to keep line structure (whitespace runs containing a line break collapse to `NEWLINE_GLYPH`, U+2588 FULL BLOCK) and to drop unrenderable characters — ANSI escape sequences, control characters, zero-width format characters, combining marks, lone surrogates — instead of printing `?` blanks.
- Explained the line-break marker in the frame-reading prompt `snapcompact-summary.md` and extended `snapcompact.test.ts` for the new variants, wrapping, and normalization.
- Added the `research/parity_check.py` and `research/parity_render.ts` parity tooling for the native renderer.
- Recorded the snapcompact changelog entries.
2026-06-12 07:36:36 +02:00
can1357 4f0d32a1c1 feat(snapcompact): added configurable compaction shape selection for snapcompact
- Added snapcompact.shape setting with auto and variant options in agent config.
- Implemented resolveShape support for forced variants, auto provider winners, and repricing.
- Threaded resolved shape into compaction and inline-image flows for pricing and rendering.
2026-06-12 07:00:34 +02:00
can1357 6b1ca33bf7 refactor(snapcompact): dropped the snapcompact qualifier from every export
- Renamed all functions, types, and constants in @oh-my-pi/snapcompact to namespace-relative names (`snapcompactCompact` → `compact`, `renderSnapcompactFrames` → `renderMany`, `snapcompactFrameCount` → `frames`, `SnapcompactShape` → `Shape`, `SNAPCOMPACT_SHAPES` → `SHAPES`, …).
- Converted every consumer to `import * as snapcompact` member access: `agent/compaction.ts`, `coding-agent` `agent-session.ts`/`session-manager.ts`/`snapcompact-inline.ts`, and all affected tests.
- Renamed internal `geometry` locals to `geo` in `snapcompact.ts` to avoid TDZ collisions with the new `geometry` export.
- Updated `docs/compaction.md` prose and added a Breaking Changes entry to the snapcompact changelog documenting the full rename map.
2026-06-12 03:27:50 +02:00
can1357 a82d68ef49 feat(coding-agent): added experimental snapcompact inline imaging for system prompt and tool results
- Added `renderSnapcompactFrames()` and `snapcompactFrameCount()` to @oh-my-pi/snapcompact for paging arbitrary text into PNG image blocks without dim-marker bookkeeping.
- Widened the agent loop's `transformProviderContext` hook to `(context, model) => Context` so per-request transforms can gate on the dispatch model's capabilities.
- Added `SnapcompactInlineTransformer` rendering the system prompt and large historical tool results as snapcompact frames on vision models: vision gate, per-provider image budgets, 3k-token floor, savings-margin gate, skip-last rule, and hash-keyed render caches swept to live tool calls.
- Added default-off `snapcompact.systemPrompt` and `snapcompact.toolResults` settings under a new Context → Experimental group, composed after secret obfuscation in `sdk.ts` so frames are built per-request and never persisted to session.jsonl.
- Added prompt stubs (`snapcompact-system-stub.md`, `snapcompact-system-frames-note.md`, `snapcompact-toolresult-note.md`) and unit tests covering frame paging, no-mutate guarantees, budget caps, gates, and render caching.
2026-06-12 03:27:50 +02:00
can1357 388354fe9c feat(cross-cutting): merged compaction file lists into one grouped tree with access markers 2026-06-10 23:13:57 +02:00
can1357 9d5587be81 feat(cross-cutting): updated snapcompact serialization to base64 with new truncation options
- Switched render outputs from byte arrays to base64 strings across native and TypeScript surfaces.
- Updated `RenderedFrame` and FFI declarations so `data` is a base64 string and `chars` are exposed.
- Added `SnapcompactSerializeOptions` with truncation, head/tail ratio and tool/arg/call limits.
- Tracked dim-marker state in rendering, reopening spans across chunk boundaries and stripping markers.
2026-06-10 22:59:25 +02:00