Commit Graph

357 Commits

Author SHA1 Message Date
Sunil Srivatsa 301908560e perf(read): materialize a local file once per read
The local text read path opened the same file for every consumer. A ranged
read of a file within the snapshot cap cost four opens and three decodes:
an 8KiB binary sniff, a streaming scan for the rendered window, a whole-file
read for bracket context, and another whole-file read to hash the snapshot.
Whole-file reads under the structural summarizer paid a fifth. Two of those
readers also ran normalizeToLF over the same bytes.

Read the bytes once at or below SNAPSHOT_MAX_BYTES and derive every view
from them: sniff the leading 8KiB of the buffer, slice the rendered window
out of it under the identical line and byte budgets, index bracket context
into its addressable lines, and hand the normalized text to the snapshot
store and the summarizer. Past the cap nothing wants the whole file, so the
streaming reader stays.

Line byte lengths are walked out of the buffer rather than measured on the
decoded strings, so reported byte counts and the truncation boundary stay
exact for content that is not valid UTF-8.

The buffered text is BOM-stripped for hashing, matching the decoder the
patcher's live read uses. A whole-file read of a BOM file previously hashed
its tag from BOM-bearing text, so the following edit only applied through
stale-hash recovery and told the model the file had changed externally when
it had not.

Also stop rejoining lines into a fresh whole-file string on the way to
tree-sitter when the caller still holds that text, and drop the unread
selectedBytesTotal accounting from the streaming reader.

Measured on 966 differential cases across CRLF, BOM, lone-CR, invalid-UTF-8,
oversized-line, empty, no-trailing-newline, multi-range and raw shapes: the
spurious recovery warning is the only behavioral difference. Raw reads, which
skip the tree-sitter parse that dominates everything else, get 30-45% faster
(2.7MB: 10.3ms -> 5.7ms); non-raw reads 1-3%.
2026-08-17 14:15:15 -07:00
can1357 c7ecc3ef66 Merge PR #8598: fix(session): repair torn JSONL appends (@roboomp)
# Conflicts:
#	packages/coding-agent/src/session/session-loader.ts
#	packages/coding-agent/src/session/session-manager.ts
#	packages/coding-agent/test/session-loader-stream.test.ts
2026-08-16 02:48:12 +02:00
can1357 ffd838b256 fix(utils): detached deeper code after padded blanks 2026-08-16 02:03:17 +02:00
can1357 36465c4e1a fix(utils): preserved deeper lazy continuations 2026-08-16 02:03:17 +02:00
roboomp 9ae089d71b fix(session): repaired torn JSONL appends
Rolled partial file appends back to their pre-write size and marked malformed resumed sessions for an atomic rewrite.

Retried transient persistence failures from in-memory state and surfaced the first failure in the interactive TUI.

Fixes #8596
2026-08-15 00:34:41 +00:00
chan1103 dc0e2ebeed fix(utils): stop indented code interrupting paragraphs in marked lexer 2026-08-14 22:48:39 +09:00
roboomp 5a0b2d460d fix(utils): normalized aborts during retry backoff
Route both fetchWithRetry backoff paths through one abort-normalizing wait helper and cover response and network retry cancellation.

Fixes #8450
2026-08-14 00:02:14 +02:00
can1357 b279db1790 test: refactored test suites to eliminate time-based sleeps and polling loops
- Replaced time-based sleeps and polling loops with event-driven promise resolvers and fake timers across agent and tool tests.
- Migrated test suites to share in-memory auth storage and fixtures using lifecycle hooks.
- Updated catalog model definitions, metadata, and configurations.
2026-08-13 19:32:22 +02:00
can1357 157690aeb6 Merge PR #8101: fix(utils): mask routed lines beneath edge labels (@roboomp)
# Conflicts:
#	packages/utils/test/mermaid-ascii.test.ts
2026-08-13 01:15:29 +02:00
can1357 9c311d7a7b Merge PR #8040: fix(tui,utils): remove O(N²) hot paths when streaming long tool-call args (@AmecoCoding) 2026-08-13 01:14:45 +02:00
roboomp eaa5f46d32 fix(mcp): resumed interrupted HTTP response streams
Implemented 2025-11-25 Streamable HTTP polling semantics for POST SSE responses: retain event IDs and retry intervals, wait as instructed, and reconnect with GET plus Last-Event-ID until the originating JSON-RPC response arrives.

Extended the shared SSE parser to expose valid id/retry fields and control-only events so reconnecting consumers do not need to reparse raw lines.

Fixes #8264
2026-08-11 18:21:12 +00:00
can1357 b524dfe36f refactor: standardized outbound User-Agent headers on shared utility constant
- Define a centralized `USER_AGENT` constant in `@oh-my-pi/pi-utils` formatted as `omp/<version>`.
- Replace hardcoded and platform-specific user agent strings across AI providers, catalog scrapers, tools, and search providers with the unified `USER_AGENT`.
- Add unit tests for update-cli binary release distribution gating.
2026-08-11 15:38:32 +02:00
can1357 978db72a46 Merge PR #7120: fix(utils): retain one rolled log per process day (@wolfiesch) 2026-08-11 15:07:43 +02:00
can1357 164edcb466 chore: removed stray blank line 2026-08-11 15:06:16 +02:00
Shawn Koh cfbd567628 fix: preserve punctuation label colors 2026-08-11 01:19:54 +08:00
Shawn Koh e658d9f7be fix: address Mermaid rendering review 2026-08-11 01:14:29 +08:00
Shawn Koh a42868f550 fix: make terminal state diagrams readable 2026-08-11 01:00:35 +08:00
Wolfie dca28eee66 fix(utils): retain one rolled log per process day 2026-08-09 23:19:52 -07:00
roboomp 5d92336e19 fix(utils): masked routed lines beneath edge labels
Added opaque label-space cells so Mermaid ASCII edge labels erase routed paths while preserving spaces during serialization.

Added horizontal and vertical regression coverage.

Fixes #8098
2026-08-09 19:03:32 +00:00
lee 9e87d432bd fix(tui,utils): remove O(N^2) hot paths when streaming long tool-call args
Long write/edit streams made the TUI stutter or freeze for seconds at a
time (ui.loop-blocked warnings, keystrokes starving while the single JS
thread rebuilt previews). Two compounding quadratic paths:

1. parseStreamingJsonThrottled re-parsed the entire accumulated args
   buffer on a FIXED 256-byte cadence. The comment claimed this bounded
   mid-stream work to O(N), but a constant growth gate still parses an
   N-byte buffer N/256 times at O(N) each: O(N^2) with a smaller
   constant. The gate now scales geometrically (max(256, len/32)), so
   parse points form a geometric progression: O(log N) parses, O(N log N)
   total, with mid-stream snapshots staying within ~3% of the stream.
   Small buffers keep the exact fixed cadence as before.

2. write.ts formatStreamingContent normalized + split('\n') the WHOLE
   accumulated content on every 30Hz reveal tick (renderCall also ran a
   full-payload normalize first): O(N) per tick, O(N^2) per stream, per
   concurrent writer. The collapsed tail-window path now tracks the
   newline count incrementally (append-only resume keyed on the
   component's persistent render-state object via WeakMap) and extracts
   only the tail window with a backward scan: O(delta + preview lines)
   per tick, byte-identical output to the split-based reference. The
   expanded (Ctrl+O) path is unchanged in output and skips its
   split+join round-trip.

Tests: geometric-gate bounds + freshness + small-buffer cadence in
parse-streaming-json-throttled.test.ts; append-growth/reference-
window/CRLF/trailing-newline/restart battery in
write-streaming-incremental.test.ts. Full write/tool-render battery
(56 tests) and ai streaming-args tests (117 tests) pass.
2026-08-08 22:32:26 +01:00
can1357 850225a694 style: applied biome formatting after community fix merges 2026-08-07 13:40:52 +02:00
can1357 bd6e87b297 feat(utils): added repair and rawKeys options to parseFrontmatter
- repair: false disables lenient recovery (ambiguous-scalar quoting, tab
  replacement, leading HTML-comment stripping) so spec-conformant loaders
  reject malformed input instead of silently repairing it.
- rawKeys: true preserves frontmatter keys verbatim; exported
  normalizeFrontmatterKeys for callers that validate raw keys first.
2026-08-07 05:59:19 +02:00
can1357 6e40e3e9fd fix(utils/marked): closed lists at an end-of-input blank run
- A blank run at EOF now breaks the list without consuming the blank,
  matching real marked: '- item\n\n' lexes as a tight list plus a space
  token instead of a loose list whose raw includes the blank.
- Completes the 17.2.10 mid-document fix; same-marker continuation and
  indented item content across blanks are unaffected.
- Added list/blank boundary token-shape tests (verified against marked
  v15) since the tui incremental tests compare the lexer to itself.
2026-08-06 14:40:13 +02:00
can1357 0992c9314f fix(utils/template): added handlebars lookup builtin
- {{lookup obj key}} resolves proto-safe obj[key] like Handlebars'
  built-in; a user-registered lookup helper still takes precedence.
- Restores slash-command/prompt-template arg consumption via
  {{default (lookup . "arguments") "none"}}.
2026-08-06 14:18:32 +02:00
can1357 ef39946bfe fix(utils/marked): kept list-trailing blank line out of the list token
- A blank line before a non-continuing top-level line (including plain
  paragraphs) now closes the list without consuming the blank, so it
  always lexes as a separate space token like real marked.
- List token shape no longer depends on the follower's block type,
  restoring the TUI streaming lexer's freeze invariant (lex(prefix) ++
  lex(tail) == lex(full) under append-only growth).
- A list followed by a paragraph is now tight, not loose, per CommonMark.
2026-08-06 14:18:32 +02:00
can1357 dedd4449c9 refactor(utils/marked): removed biome lint ignore comment from generic token interface
- Removed the biome-ignore lint comment for explicit any on the Generic token interface.
2026-08-06 13:32:15 +02:00
can1357 6c84c3e8fc Merge PR #7691: fix(utils): support PowerShell as custom shellPath (@metaphorics) 2026-08-05 21:50:25 +02:00
can1357 e9888367d1 refactor: migrated packages to internal utility modules and removed external dependencies
- Implemented in-house, zero-dependency utility modules in `pi-utils` covering DOM manipulation, markdown parsing, templating, browser automation helpers, and terminal buffers.
- Migrated packages across the repository to consume the new internal utilities and `omptype` schema validators instead of external dependencies.
- Removed multiple external runtime and development dependencies including Zod, Marked, LRU cache, Turndown, and Puppeteer browser packages.
2026-08-05 13:39:09 +02:00
metaphorics b86f788ae7 fix(utils): support PowerShell as custom shellPath 2026-08-05 10:24:35 +00:00
brymko 01ed58d684 docs(utils): documented fatal recovery hint contract 2026-08-05 14:31:03 +08:00
brymko 56931915f1 feat(coding-agent): added fatal session recovery hints
Registered live session resume commands with postmortem handling so a
fatal rejection or exception identifies every recoverable agent before
cleanup. Escaped terminal control characters in recovery output.
2026-08-05 14:31:03 +08:00
metaphorics 7262d9762e fix(ai): honor account reset windows and statusless concurrency caps
Account-reset hint evaluated before short retry hints; account-scoped caps rotate on status 403 or undefined (Devin statusless trailer); statusless concurrency caps marked transient; transient same-model retries use the concurrency backoff.

Refuted: quota-worded concurrency caps were already excluded from rotation before the usage-limit text match.
(cherry picked from commit f2b9a18d715ddbcb6ae703670f2212da36bb2826)
2026-08-05 02:32:35 +02:00
metaphorics 9d51aabce9 fix(ai): address PR review feedback (#6958)
(cherry picked from commit 19327455a482ffdf8dfebcfb47b12fe0d4c888da)
2026-08-05 02:32:35 +02:00
can1357 84fc295f0c Merge PR #7586: refactor: centralize version comparison in pi-utils (@metaphorics) 2026-08-05 01:11:56 +02:00
can1357 e76d998c01 Merge PR #7498: fix(utils): honor current PATH in cached lookups (@usr-bin-roygbiv) 2026-08-05 01:11:56 +02:00
metaphorics 3802d2dd80 chore(ts): enforce noImplicitOverride 2026-08-05 02:49:01 +09:00
metaphorics 954894f1d4 refactor: centralize version comparison in pi-utils 2026-08-05 02:48:59 +09:00
can1357 455493dfad feat: introduced native file lock bindings for cross-process advisory locking
- Added native `FileLock` bindings supporting cross-process advisory locking on Linux, Unix, and Windows.
- Replaced directory-based file locking and custom stale-lock reclamation with OS-backed native locks.
- Updated TypeScript declarations, native bindings, and package documentation for the new API.
- Added comprehensive unit tests and fixtures validating single-owner constraints and process death handoff.
2026-08-03 16:09:09 +02:00
can1357 9fb082bf14 refactor(utils): consolidated file locking into pi-utils file-lock
- Moved the coding-agent lock-directory primitive to @oh-my-pi/pi-utils/file-lock
  and migrated settings, MCP config-writer, and security store imports.
- Replaced the stats aggregator's parallel ~200-line token/breaker lock protocol
  with the shared primitive: dead owners reclaimed immediately, live-but-wedged
  owners after STATS_SYNC_LOCK_STALE_MS, unstamped acquisitions after the new
  acquireStaleMs grace (10s).
- Shared primitive now treats EPERM kill probes as live owners.
- Rewrote the stats lock-reclamation regressions against the shared protocol
  and moved the file-lock contract test into pi-utils.
2026-08-03 15:25:03 +02:00
usr-bin-roygbiv 1bb5b445fe style(utils): format effective PATH lookup 2026-08-03 13:03:35 +00:00
usr-bin-roygbiv a5f5f53c35 fix(utils): honor current PATH in which lookups 2026-08-03 12:53:38 +00:00
usr-bin-roygbiv 55a5574bdd Revert "fix(utils): honor current PATH in which lookups"
This reverts commit 960c21baf9.
2026-08-03 12:45:29 +00:00
usr-bin-roygbiv 960c21baf9 fix(utils): honor current PATH in which lookups 2026-08-03 12:45:13 +00:00
can1357 f1c7eda7de Merge PR #7205: perf(cli): keep root help off runtime graph (@eggpeat)
# Conflicts:
#	packages/coding-agent/src/cli-commands.ts
#	packages/coding-agent/src/cli/args.ts
2026-08-02 20:59:12 +02:00
can1357 980c78538b Merge PR #7395: fix(postmortem): resolve native hard-exit per call (@roboomp) 2026-08-02 20:53:35 +02:00
can1357 3f61117994 Merge PR #7362: fix(auth): guard config-value resolvers against case-insensitive env hijack (@roboomp) 2026-08-02 20:53:20 +02:00
roboomp e27b0e81ec fix(postmortem): resolve native hard-exit per call
The postmortem module bound the native hard-exit once at module init
(process.reallyExit.bind(process)). The shipped bundle defers this
module's evaluation until first access, which can land inside a
withHostGuard window where process.reallyExit is the ExtensionExitError-
throwing stub; .bind() then froze that stub permanently, so every later
host-owned exit (SIGHUP 129, SIGINT 130, fatal 1) threw and re-entered
the unhandled-rejection fatal path in a loop (exit 129 storm).

Resolve the native exit on every call instead of binding at init, and
have withHostGuard stamp its throwing replacement with the native
primitive it shadows so a signal arriving mid-guard still exits (#6488)
without the guard poisoning later exits (#7393).

Fixes #7393
2026-08-02 17:32:12 +00:00
roboomp a6521f07ed fix(auth): guarded config-value resolvers against case-insensitive env hijack
On Windows process.env/Bun.env lookups are case-insensitive, so the
"env var name, else literal" resolvers turned a literal /login key like
`public` (OpenCode Zen's free key) into the built-in PUBLIC=C:\Users\Public,
sending `Authorization: Bearer C:\Users\Public` and 401ing every request.

Added `$envExact` in pi-utils, which trusts an env lookup only when an
exact-case key is enumerated (the only case-preserving signal on Windows;
the getter and hasOwnProperty/getOwnPropertyDescriptor traps are all
case-insensitive there). Wired it into all three resolvers:
resolve-config-value.ts, model-registry.ts, and auth-storage.ts.

Fixes #7361
2026-08-02 06:53:18 +00:00
can1357 92c79d80c7 feat: introduced OMP Browser Relay extension with CDP RPC execution
- Implement the OMP Browser Relay extension with WebSocket communication and CDP RPC execution.
- Add browser relay server, daemon management, and bridge multiplexing in the coding agent.
- Introduce CLI commands and settings schema options for configuring and installing the relay.
- Add utility functions and test suites supporting environment parsing and relay lifecycle handling.
2026-08-02 05:33:07 +02:00
can1357 72c66c87c1 fix(xdg): adopted legacy secret-placeholder.key and marketplaces.json at XDG paths 2026-08-01 20:46:25 +02:00