Addresses a broad audit of built-in shell utilities against their real counterparts: timeout gains signal delivery, -s/-k/--preserve-status/--foreground/-v, and GNU exit codes; diff defaults to normal format and gains -w/-b/-B/-i/-c/-x/-L/-s/--strip-trailing-cr and proper -r gating; find fixes -newerXY timestamp comparison direction, anchors -regex to whole paths, and gains BSD -perm +mode, -type lists, -size T/P suffixes, -E/-x/-s flags; date gains BSD -r epoch, -v adjustments, -j -f strptime, and non-greedy -I; tail/head accept obsolete -N/+N at any position with any file count; rg resolves case flags by last occurrence and gains --path-separator and clean -0 output; stat prints integer epochs for %X/%Y/%Z and gains BSD -s/-x/-t; cksum is registered as a builtin; truncate implements -o/--io-blocks and b/= size suffixes; sleep/timeout accept infinity; yes/errno/kill accept hyphen-prefixed operands; nohup -- cmd no longer runs --; which gains BSD -s.
The "Large Headings" feature renders Markdown H1 at 2x via the Kitty OSC 66
text-sizing protocol; a scale-s heading is s rows tall, so the s-1 blank rows
beneath it hold the multicell glyph's lower half. The render engine gave Kitty
image lines a dedicated re-emit path but had no OSC 66 awareness, so every
repaint through #lineRewriteSequence (incremental diff, full replay, resize
viewport) rewrote those reserved rows as reset + erase-to-end-of-line, clearing
the glyph and leaving reserved-but-invisible space.
Detect reserved rows from the heading's s= scale (#isOsc66SpacerRow) and emit
them untouched across every repaint path, covering all s-1 rows for scale >= 3.
Consolidate isOsc66Line into pi-tui/utils alongside a new osc66MaxScale.
Fixes#8318
Carry exact visible widths from Text and Box render results into their owners, and through Editor wrap/layout/render state. Stamp derived widths and render caches with a monotonic Hangul width-config epoch. Retain sidecar entries only by WeakMap owner identity, with immutable publication-time line/width proof, and conservatively remeasure escape-leading padded rows.
Before:
static_redraw 1633.11 us/op
editor_edits 1249.71 us/op
After hardened repair:
static_redraw 1321.51 us/op (1.236x)
editor_edits 1150.32 us/op (1.086x)
Win:
Exact byte/grid/scrollback hashes match across all five phases; every phase CV is below 20%, with no B/C regression.
Memory:
retained heap 251,393,541 -> 251,500,114 bytes (+0.042%).
Op: carry exact widths through render ownership and editor layout
Restores: repeated visible-width measurement in core TUI hot paths
`getWordNavKind` ran the `\p{P}` punctuation test before the `_` check, so
underscore (Unicode `Pc`) returned "delimiter" and `moveWordLeft` stopped at
each underscore. Ctrl+W deleted only the trailing segment of snake_case
identifiers. Reordered the underscore check ahead of the punctuation/symbol
test and removed the dead `ch === "_"` branch.
Fixes#4776
Four non-overlapping algorithmic complexity reductions in hot paths.
(Streaming-reveal throughput is owned separately by #3843.)
1. session/session-manager.ts pathTo: O(n^2) branch.unshift() leaf->root
walk -> O(n) push + single reverse(). Hot path (5-10x/turn via getBranch).
2. edit/streaming.ts extractAddedLines: O(n^2) progressive string
concat per streaming tick -> array push + single join.
3. edit/modes/patch.ts: collapseConsecutiveSharedLines O(n*m) filter
+ includes -> Set (O(n+m)); collapseRepeatedBlocks O(n^3) with
per-iteration slice allocations + every() -> index arithmetic with
a single shared.has() guard. Semantics preserved.
Honorable: tui/src/utils.ts replaceTabs reallocated
" ".repeat(DEFAULT_TAB_WIDTH) every call -> hoisted TAB_SPACES const.
- Adjusted the conditional check to correctly identify string-based ellipsis inputs.
- Resolved logic for applying default ellipsis behavior when invalid or empty types are provided.
Fixes#3492
Replace the compile-time cfg!(target_os = "macos") jamo-width heuristic
with a runtime override (process-global AtomicU8 in pi-natives, mirrored in
the TS width engine) plus terminal-identity detection: Ghostty renders
Hangul Compatibility Jamo (U+3131..U+318E) at 2 cells, so it is forced wide;
every other terminal keeps the platform default (macOS narrow, otherwise
UAX#11), making the override a no-op outside Ghostty.
Fixes doubled/ghosted jamo during Korean IME composition on Ghostty, where
the hardware cursor landed inside the typed text and the IME candidate window
drifted from the glyph. The width is resolved synchronously before the first
paint, so no stdin/CPR probe or async cache invalidation is needed.
A runtime DSR/CPR probe that auto-detects the width on unknown terminals is
tracked in a follow-up.
- Removed configurable tab width support and the `display.tabWidth` setting across all packages.
- Deleted obsolete utility functions `getIndentation`, `getIndentationNoescape`, and `setDefaultTabWidth`.
- Standardized tab expansion logic to use a fixed `DEFAULT_TAB_WIDTH` globally.
- Cleaned up related configuration schemas, test suites, and internal API signatures to remove path-dependency.
- Standardized elision markers across all tool outputs and filters to use cohesive `[...N [type] elided...]`, `[...Nln elided...]`, and `[...xB elided...]` syntax.
- Updated documentation, prompts, and test expectations to reflect the unified elision format.
- Improved transcript viewer robustness by preventing content aliasing through path-inclusive signature hashing.
- Added logic to clear stale transcript content when associated session files are deleted, accompanied by verifying test cases.
- Added new `tui.tight` setting with default `false` and documented behavior.
- Added global tight-mode APIs and `setIgnoreTight` propagation across Box, Text, and Markdown.
- Handled `tui.tight` updates to refresh UI borders and trigger rerendering.
- Updated key message components to ignore tight mode selectively and preserve intended spacing.
- Added animated message-color gating so loader timers run at 30fps only when needed.
- Added no-op checks in loader and text updates to skip work for unchanged values.
- Optimized key matching by canonicalizing key IDs and precomputing alias sets.
- Added prepared-line caching and bounded terminal/width scans to reduce repaint overhead.
- Fixed `visibleWidth` parity with native width using `Bun.stringWidth` ANSI stripping.
- Fixed tab and OSC66 width parsing with `s=`/`w=` metadata handling.
- Added ASCII and long-string fast paths in `visibleWidth` for faster width checks.
- Consolidated visible-width regression coverage by replacing removed IME/Arabic/Jamo tests with `visible-width.test.ts`.
- Added `textSizing` terminal capability and `tui.textSizing` setting for Kitty OSC66 scaling.
- Updated OSC66 range and visible-width handling to preserve grapheme slice behavior.
- Updated markdown rendering to apply heading text sizing only when enabled.
- Fixed DECCARA trailing-background fill tracking and DECRPM status 3/4 recognition.
- Added OSC 66 text-sizing utilities and `encodeTextSized` for safe, width-aware span emission.
- Enabled scale-2 OSC 66 output for H1 headings when text sizing is enabled and width permits.
- Handled OSC 66 heading rows as raw output, skipping wrap/background and preserving the reserved blank row.
- Mapped inline heading tokens to plain text to gate OSC 66 heading rendering by measured width.
- Introduced a shared missingSnapshotTagMessage helper and reused it from both hashline patcher and preview validation.
- Required snapshot tags in preview hashline sections for all edits, including head/tail inserts, so preview failures now match apply-time rejections.
- Updated TUI width calculation to strip ANSI escapes before grapheme splitting, preserving styled ZWJ emoji width behavior.
- Introduced `@oh-my-pi/pi-mnemosyne` workspace package with BeamMemory, recall/retain, FTS, and optional fastembed ONNX embeddings.
- Wired `memory.backend = "mnemosyne"` into the coding-agent settings schema and backend resolver.
- Added `MnemosyneSessionState` for per-session auto-recall on first turn and auto-retain after completed turns.
- Documented configuration, environment variables, and operational notes in `docs/mnemosyne-memory-backend.md`.
Keep the printable ASCII fast path, but send tabs, ANSI/control sequences, combining marks, and CJK through the same native width engine used by slicing and wrapping. This fixes Bun.stringWidth overcounting Arabic non-spacing marks and the old pure-ASCII tab double count.\n\nConstraint: Bun.stringWidth still reports Arabic combining marks as visible cells on Bun 1.3.14.\nRejected: Strip \p{Mn} in TypeScript | It would still leave tabs and future Unicode width mismatches divergent from native truncation.\nConfidence: high\nScope-risk: narrow\nDirective: Keep visibleWidth, sliceWithWidth, truncateToWidth, and wrapTextWithAnsi on one Unicode width model.\nTested: bun test packages/tui/test/text-utils.test.ts packages/tui/test/visible-width-jamo.test.ts packages/tui/test/wrap-ansi.test.ts packages/tui/test/truncate-to-width.test.ts packages/tui/test/render-regressions.test.ts; bun test packages/tui/test/*.test.ts; bun run check in packages/tui.\nNot-tested: Interactive terminal repaint with a live Arabic transcript.
The Hangul Compatibility Jamo width correction (U+3131..=U+318E → 1 cell)
was applied unconditionally, but macOS terminals (Ghostty, Terminal.app,
iTerm2) are the only ones that render jamo as 1 cell. WezTerm and most
Linux terminals follow UAX#11 and render them as 2 cells.
The unconditional correction caused a width mismatch between the TUI's
calculated column positions and the terminal's actual rendering, resulting
in corrupted layout and broken Korean input on non-macOS platforms.
- Rust: gate the correction with cfg!(target_os = "macos")
- TS: gate the correction with process.platform === "darwin"
- On non-macOS, use UnicodeWidthStr for multi-char graphemes (perf)
Constraint: macOS terminals render jamo as 1 cell despite UAX#11.
Rejected: removing the correction entirely — would break macOS.
Tested: native build succeeds, typecheck passes.
Not-tested: visual verification on macOS terminals.
Scope-risk: narrow
Korean IME composition (2-bul keyboard on macOS) emits Hangul Compatibility Jamo (U+3131..U+318E — ㄱ ㄴ ㄷ ㄹ ㅁ ㅂ ㅅ ㅇ ㅈ ㅊ ㅋ ㅌ ㅍ ㅎ + filler) for intermediate buffer state. `Bun.stringWidth` (and the underlying UAX#11 EAW tables) classify these as Wide (2 cells), but every macOS terminal we ship to (Ghostty, Terminal.app, iTerm2) renders them as a single cell in monospace fonts.
Symptom: while typing Korean into the OMP prompt under tmux + ghostty, the hardware cursor (and therefore the IME candidate window) appears N cells past the actual glyph — and the gap grows by 1 cell per jamo. `#extractCursorPosition` computes `col = visibleWidth(beforeMarker)` and feeds it to `\x1b[(col+1)G`; with jamo doubled in the width function, the cursor position is wrong by exactly N_jamo cells. Confirmed against a user-supplied screen recording (≈14 typed jamo, ≈14 cell gap to the next glyph).
`visibleWidthRaw` now subtracts 1 cell for each Compatibility Jamo character on the slow non-ASCII path. Allocation-free (charCodeAt loop, no regex object), runs only when at least one non-ASCII char is present.
Not affected:
• Hangul Syllables (U+AC00..U+D7A3, e.g. `안`) — 2 cells in both Bun and the terminal; leave alone.
• Halfwidth Hangul block (U+FFA0..U+FFDC) — already Narrow (1 cell) in Bun.
• Chinese, Japanese hiragana/katakana — unchanged.
Out of scope (follow-up): the Rust `pi-natives` width tables (used by `sliceWithWidth` / `truncateToWidth` / `wrapTextWithAnsi`) also count jamo as 2 cells. Truncation and word-wrap on jamo-heavy lines will still be slightly off. Invisible in normal use because the AI composes Korean as syllables, not jamo, and users type syllables once IME composition completes. Will reconcile when needed.
Tests: 9 new cases in `packages/tui/test/visible-width-jamo.test.ts` covering range edges, mixed strings, regression sanity for syllables and other CJK.
- Updated truncateToWidth to map a string ellipsis of "" to Ellipsis.Omit.
- Added a regression test for issue #848 to verify empty-string ellipsis produces a plain truncation result.
- Added normalizeTerminalOutput to convert precomposed Thai and Lao AM vowels into compatibility decompositions for terminal rendering.
- Converted ASCII classification tables to Uint8Array and updated whitespace and punctuation checks to use bounded code-point lookups.
renderContentPreview crashed on the Windows prebuilt with "Failed to
convert napi value Null into rust type `u8`" (and the related `bool`
variant) because the wrapper forwarded `null` to napi-rs 3's
`Option<Ellipsis>` / `Option<bool>` parameters; the Windows binding
rejects null where Linux/macOS tolerates it. `maxWidth` was also passed
through unchecked, so a stray null/undefined upstream blew up the
required `u32` everywhere. Pass concrete defaults that mirror the Rust
`unwrap_or`s and clamp `maxWidth` to a sane non-negative integer.
Fixes#848
Bun's JSC napi layer on Windows does not reliably report undefined as
napi_undefined, causing napi-rs Option<bool> to fall through to
bool::from_napi_value() and throw:
Error: Failed to convert napi value into rust type bool
Pass null instead of undefined for ellipsisKind, pad (truncateToWidth)
and strict (sliceWithWidth) via ?? null. null maps to napi_null which
napi-rs detects correctly on all platforms.
- Extracted tab width resolution from pi-natives to pi-utils with EditorConfig caching support.
- Made tabWidth parameter required in native text functions (visibleWidth, truncateToWidth, wrapTextWithAnsi, sliceWithWidth, extractSegments).
- Removed process-wide tab width state management from pi-natives text module.
- Consolidated wrapper node promotion logic in chunk classification with abbreviated string representations.
- Removed path-clean and pathdiff dependencies from pi-natives.
- Updated all consuming packages to import tab width functions from pi-utils and pass explicit tabWidth parameters.
- Migrated native bindings from TypeScript wrappers to NAPI-RS generated modules with auto-generated type definitions and runtime enums.
- Replaced chunk tree API with stateful ChunkState class supporting render, edit, and resolve operations with improved error handling.
- Converted callback signatures to error-first pattern (error, result) for shell, PTY, glob, and grep operations.
- Introduced type-safe enums for MacOSAppearance, GrepOutputMode, KeyEventType, ImageFormat, and AstMatchStrictness replacing string literals.
- Refactored chunk tree implementation with dedicated modules for edit, indent, resolve, and state management with comprehensive validation.
- Moved clipboard utilities from native bindings to coding-agent package with improved OSC 52 and Termux compatibility.
- Extracted word navigation logic into reusable `moveWordLeft()` and `moveWordRight()` utility functions.
- Consolidated Unicode-aware word navigation handling (whitespace, delimiters, CJK, word-joiner) into shared utilities.
- Refactored input and editor components to delegate word navigation to extracted utility functions, reducing duplication.
- Removed 212 lines of duplicated word navigation logic across components while maintaining identical behavior.
- Added a `display.tabWidth` setting to control default tab rendering.
- Integrated `.editorconfig` resolution for c ontext-aware tab indentation.
- Updated native text processing to use the resolved tab width for calculations.
- Improved tab visualization in diffs and other UI elements for consistency.
- Removed getCursorPosition() method from Component interface and implementations, eliminating hardware cursor positioning support.
- Changed PI_HARDWARE_CURSOR environment variable default from implicit true to explicit '1' with fallback pattern.
- Changed PI_CLEAR_ON_SHRINK environment variable default from implicit false to explicit '0' with fallback pattern.
- Added screen clearing on TUI startup to prevent shell prompts from bleeding into first rendered frame.
- Refactored full-render logic into reusable fullRender() helper function.
- Replaced regex-based character checks with lookup table approach for whitespace and punctuation detection.
- Introduced makeBoolArray helper to build ASCII lookup tables for O(1) character classification instead of regex matching.
- Added new public `wrapTextWithAnsi()` function to wrap text to a visible width while preserving ANSI escape codes across line breaks.
- Implemented ANSI state tracking and restoration logic to maintain color and formatting codes when wrapping multi-line text.
- Migrated `wrapTextWithAnsi` implementation from tui package to pi-natives package for better code organization and reusability.
- Added comprehensive test coverage for ANSI color code preservation across wrapped lines.
- Renamed `EllipsisKind` enum to `Ellipsis` across all packages for clearer API naming.
- Refactored Rust text module from 356 to 155 lines with simplified state tracking using u32 ColorVal instead of enum ColorCode and named attribute constants.
- Removed `Env` parameter from public NAPI functions `truncate_to_width` and `extract_segments` to simplify API surface.
- Changed return type of `truncate_to_width` from `JsString` to `Either<JsString, Utf16String>` to support zero-copy returns.
- Removed early return optimization in `truncateToWidth()` function.
- Reordered native module search paths to prioritize repository build artifacts over system paths.
- Changed TextInput type parameter to string in truncateToWidth(), visibleWidth(), sliceWithWidth(), and extractSegments() functions in native bindings.
- Removed TextInput type export from public API of pi-natives package.
- Removed format.ellipsis symbol from theme configuration, replacing all theme-based ellipsis references with hardcoded Unicode ellipsis character (...).
- Removed ellipsis property from SymbolTheme interface in pi-tui package.
- Removed truncateToWidth, sliceWithWidth, and extractSegments wrapper functions from pi-tui public API, now re-exporting directly from pi-natives.
- Added early return optimizations in truncateToWidth() and sliceWithWidth() native functions to skip native calls when possible.
- Added `visibleWidth()` function to measure visible text width while excluding ANSI codes.
- Optimized text processing from UTF-8 to UTF-16 implementation with direct JsString handling for improved performance.
- Refactored ANSI state tracking to use bitflags-based SgrState struct instead of individual boolean fields for more efficient storage.
- Introduced ColorCode enum to represent color codes (Basic, Indexed, Rgb) for better type safety.
- Added comprehensive test coverage for UTF-16 text processing, ANSI detection, and width calculations.
- Updated napi feature from 'napi8' to 'napi10' and added unicode-segmentation dependency.
- Added native Kitty keyboard protocol sequence matching via new `matchesKittySequence` function in pi-natives crate.
- Removed `visibleWidth` function from text utilities in favor of consistent Bun.stringWidth approach.
- Upgraded napi and napi-derive dependencies from version 2 to 3.
- Relaxed grep-regex, grep-searcher, and grep-matcher dependency version constraints to allow any 0.1.x version.
- Replaced unicode-segmentation dependency with bstr for improved byte-slice handling in text processing.
- Refactored text module to use byte-slice abstractions instead of string references for better performance and flexibility.
- Extracted manual space padding logic into a reusable `padding()` utility function across all UI components and utilities.
- Optimized padding operations by introducing a pre-allocated 512-space buffer in the `padding()` function to reduce repeated string allocations.
- Updated all imports across 30 files to use the new `padding()` function from pi-tui instead of inline `' '.repeat()` calls.
- Renamed local variables from `padding` to `pad`, `padSize`, `indent`, or `linePad` to avoid naming conflicts with the imported `padding()` function.
- Exported the `padding()` utility function from the tui package index for public use.
- Added native text utilities in pi-natives (visible width, truncation, slicing, segment extraction)
- Wired pi-tui to use native implementations with short-string JS fast path
- Optimized key matching via cached parsed key ids
- Refactored visual truncation to reuse Text instances
- Reduced theme color lookups by switching from Map to plain records
- Regenerated WASM artifacts
Benchmarks (after vs before, lower is better):
- truncateToWidth/ansi: 20.37ms vs 38.13ms (2000 iters)
- sliceWithWidth/ansi: 5.12ms vs 20.39ms
- extractSegments/ansi: 8.92ms vs 27.60ms
- truncateToVisualLines: 3.51ms vs 411.55ms (500 iters)
- visibleWidth/ansi: 0.24ms vs 0.31ms
- Added ToolChoice type and toolChoice parameter support across all AI providers (OpenAI, Azure OpenAI, Anthropic, Google) enabling fine-grained control over tool/function selection during LLM calls.
- Added toolChoice override capability to Agent.prompt() method and session prompt options allowing callers to control tool selection behavior per request.
- Added provider-specific tool choice mapping functions (mapAnthropicToolChoice, mapGoogleToolChoice, mapOpenAiToolChoice) to normalize tool choice formats across different LLM APIs.
- Removed kernel heartbeat/ping mechanism from PythonKernel, simplifying health monitoring by relying on direct isAlive() checks instead of periodic HTTP requests.
- Replaced external JSON/JSONL parsing libraries with Bun's built-in JSON5 and JSONL APIs across all packages.
- Removed dependencies: json5, ndjson, get-east-asian-width, json-stringify-safe, split2, through2, and @types/ndjson.
- Replaced custom text width and ANSI wrapping implementations with Bun.stringWidth() and Bun.wrapAnsi() APIs.
- Updated Bun type definitions from ^1.2.17 to ^1.2.18 and bun-types from ^1.3.5 to ^1.3.7.
- Refactored JSONL parsing throughout codebase to use Bun.JSONL.parse() and Bun.JSONL.parseChunk() with improved buffer management and error handling.
- Updated test assertions to use Bun.stringWidth() for dynamic width calculations instead of hardcoded values.
- Removed Prettier configuration files (.prettierignore and .prettierrc) and migrated formatting to Biome.
- Updated Biome configuration from version 2.3.11 to 2.3.12 and changed arrowParentheses rule from 'always' to 'asNeeded'.
- Pinned @biomejs/biome dependency to exact version 2.3.12 in package.json and bun.lock.
- Applied consistent arrow function formatting across 489 files by removing unnecessary parentheses around single parameters.
- Removed blank lines after comment blocks and reorganized imports for consistency across the codebase.
Provider additions:
- Added Amazon Bedrock provider with bedrock-converse-stream API
- Added MiniMax provider with OpenAI-compatible API
- Added EU cross-region inference model variants for Bedrock
Provider fixes:
- Fixed Gemini CLI retries with header parsing and empty stream retry logic
- Fixed Bedrock tool call transforms via transformMessages
- Fixed z.ai thinking/reasoning params
- Fixed OpenRouter+Anthropic cache control
- Fixed OpenAI responses timeout and service tier options
- Fixed tool call ID normalization for cross-provider switches
- Fixed thought signature validation for Google providers
- Fixed prompt cache key using session ID
TUI improvements:
- Added OverlayOptions API with CSS-like positioning (SizeValue, percentages)
- Added OverlayHandle for programmatic visibility control
- Added visible callback for responsive overlays
- Added pad parameter to truncateToWidth
- Added pageUp/pageDown key support
- Fixed numbered list items showing 1. when code blocks break continuity
- Fixed overlay width overflow crash with complex ANSI sequences
- Fixed light theme colors for WCAG AA compliance
Coding agent fixes:
- Fixed /new command to create new session file
- Fixed session selector to stay open when folder has no sessions
- Added session header emission in JSON print mode
- Added queued message hint with theme.tree.hook
- Exported highlightCode and getLanguageFromPath for extensions
Also renamed transorm-messages.ts to transform-messages.ts (typo fix)
Long whitespace tokens (e.g., lines with many spaces) were not being
truncated when they exceeded the line width, causing rendered lines
to exceed terminal width and crash the TUI.
can1357