Commit Graph

258 Commits

Author SHA1 Message Date
can1357 362f89a2ec Merge remote-tracking branch 'origin/farm/fe301a84/reduce-stream-decoding-cpu' 2026-07-15 09:54:59 +02:00
can1357 404ebb0fb0 Merge remote-tracking branch 'origin/farm/ae7a5593/ttsr-inline-regex-flags-and-scope-quoting' 2026-07-15 09:54:46 +02:00
roboomp 22fa8f6623 perf(stream): batched response decoding
- Batched complete SSE lines into one UTF-8 decode per source chunk.

- Parsed Ollama NDJSON bytes directly without TextDecoder buffering.

Fixes #5542
2026-07-15 02:55:02 +00:00
can1357 0418d8622a fix(cli): handle native parser usage errors 2026-07-14 23:11:11 +02:00
can1357 33f61cc511 Merge PR #5496: fix(cli): print concise usage error instead of source dump (@roboomp) 2026-07-14 23:11:11 +02:00
roboomp 6dc48a8b02 fix(cli): print concise usage error instead of source dump
Argument/flag validation failures in the minimal CLI framework threw a
plain Error that bubbled to the process-level catch in cli.ts, which
dumps a Bun.inspect code frame — a minified dist/cli.js excerpt in
compiled binaries. A missing model on `omp bench` looked like a crash.

- Add CliUsageError; throw it from Command.parse for missing/invalid
  args and flags.
- Catch CliUsageError in run(): print `error: <msg>` plus the command
  usage line to stderr, exit 1, no stack.
- Render required variadic positionals as MODELS... in usage, not the
  misleading optional [MODELS]; extract commandUsageLine helper.

Fixes #5369
2026-07-14 19:16:59 +00:00
roboomp 86824c94e9 fix(ttsr): registered rules with inline regex flags and malformed scope
Rules whose condition led with a PCRE-style inline flag group (e.g.
`(?i)`) never registered: `new RegExp("(?i)...")` throws in Bun/JS, so
the condition failed to compile and `TtsrManager.addRule` dropped the
rule as having zero usable conditions.

- Add `compileRuleCondition` in capability/rule.ts translating a leading
  `(?i)`/`(?m)`/`(?s)` group into native RegExp flags; wire it into the
  TtsrManager and both ttsr-cli compile sites.
- Strip surrounding quotes from scope tokens so a malformed
  `scope: "text","thinking"` recovers to canonical `text`/`thinking`.
- Reparse each value in parseFrontmatter's YAML fallback so one bad line
  can't leave sibling values wrapped in literal quotes.

Fixes #4796
2026-07-14 19:01:47 +00:00
can1357 3efbce97b5 Merge PR #5295: fix(utils): bound Mermaid ASCII pathfinder (@roboomp) 2026-07-14 19:16:08 +02:00
can1357 c8afd0f967 Merge PR #5131: fix(tui): suppress unmanaged macOS stderr writes corrupting the viewport (@korri123) 2026-07-14 18:34:14 +02:00
roboomp cc2041ab7f fix(utils): bounded mermaid ascii pathfinder
- Added bounded A* routing extents plus an expansion backstop so unreachable attachment points return null instead of searching the unbounded quadrant.
- Covered the reported declaration-order graph and an enclosed destination attachment point.
- Documented the Mermaid ASCII routing fix in the utils changelog.

Fixes #5293
2026-07-12 18:54:28 +00:00
can1357 a3117c2842 feat(agent): migrated async-drain utility to shared package for reuse
- Relocated `AsyncDrain` class from `coding-agent` to `utils` package.
- Updated `HistoryStorage` to import `AsyncDrain` from shared utilities.
- Centralized the utility to allow reuse across the codebase.
2026-07-12 01:06:17 +02:00
Kormákur 17486d2009 fix(utils): create log dir before redirecting stderr guard
On a fresh profile ~/.omp/logs may not exist yet (the logger creates it
lazily), so opening getLogPath() with "a" threw and the guard fell back to
/dev/null — discarding macOS diagnostics and native crash reports instead
of preserving them in the omp log. mkdir the redirect target's parent
(recursive) before opening; the /dev/null fallback stays as the safety net.
2026-07-11 00:38:55 +00:00
Kormákur 4eaca82fa6 fix(tui): suppress unmanaged macOS stderr writes corrupting the viewport
On macOS, libmalloc writes runtime diagnostics (e.g. "MallocStackLogging:
can't turn off malloc stack logging because it was not enabled") directly
to fd 2 of the running TUI process at arbitrary times, painting into the
viewport. The existing env-strip only protects child processes.

Add a fd-level stderr guard in pi-utils (suppressTerminalStderr /
restoreTerminalStderr) that dup2-redirects fd 2 to the omp log file while
the TUI owns the terminal, and restores it at every ownership handoff
(external editor, Ctrl+Z suspend, shutdown, crash restore). Postmortem
fatal handlers restore fd 2 before printing so crash reports stay visible.

Mirrors openai/codex#24459.
2026-07-11 00:16:42 +00:00
can1357 531880c620 feat: improved json serialization for bigint values
- Introduced `stringifyJson` helper to preserve bigint precision by serializing them as decimal strings.
- Replaced native `JSON.stringify` across compaction and session management modules to prevent serialization errors when handling bigint values in tool arguments.
- Added regression tests in `agent` and `coding-agent` packages to ensure bigint tool arguments remain intact through compaction and persistence flows.
2026-07-11 00:12:29 +02:00
can1357 314b9a7c9d fix(config): shared yaml config discovery
Moved the config.yml/config.yaml filename order into pi-utils as
MAIN_CONFIG_FILENAMES and taught the auth-broker config reader to probe
both extensions with the same precedence as the settings loader.

Fixes #4914
2026-07-09 18:27:22 +02:00
can1357 ffaa4764ec merge PR #4729: fix(cli): stop exporting launch .env.local into commands 2026-07-08 15:19:37 +02:00
can1357 38486e56db feat(coding-agent): handled unawaited promise rejections in eval cells
- Introduced a rejection interception mechanism to capture unhandled promise rejections from eval cell code.
- Attributed floating rejections to specific runs to fail the owning cell instead of crashing the process or worker.
- Downgraded rejections occurring after a cell finished to warn logs to prevent silent failures.
2026-07-08 14:51:25 +02:00
roboomp 4b29248877 fix(cli): stopped exporting launch env local values
Filtered Bun-autoloaded launch .env.local entries out of child shell environments so nested commands can load their own dotenv files.

Added a regression test covering Convex-style inherited deployment variables while preserving ordinary inherited env values.

Fixes #4723
2026-07-06 15:23:59 +00:00
can1357 ff0419e412 feat(utils): enabled routine cleanup error tagging and deadlines
- Added markExpectedCleanupError and isExpectedCleanupError to identify and downgrade routine teardown errors to warnings.
- Updated global uncaughtException and unhandledRejection handlers to suppress process exit for tagged cleanup errors.
- Introduced a 10s execution deadline for cleanup callbacks to prevent process hanging during teardown.
2026-07-06 08:07:24 +02:00
can1357 d63c69d7c3 feat(utils): implemented buffer classification for stream parsing
- Implemented classifyJsonPrefix to categorize buffers as complete, valid prefix, or invalid based on RFC 8259 strictness.
- Added utility to support disambiguation of identifierless streaming tool-call deltas during model response processing.
- Validated classifier logic with comprehensive suite covering nested structures, escape sequences, and strict formatting rules.
2026-07-05 15:57:24 +02:00
can1357 5768615e5d feat(ai): enabled millisecond precision for transport retry delay headers
- Added support for the `retry-after-ms` header in `getRetryAfterMsFromHeaders` to parse millisecond-based retry delays.
- Implemented `parseRetryAfterMsHeader` utility to safely validate and convert header values to milliseconds.
- Updated `extractRetryHint` in `fetch-retry` to prioritize the `retry-after-ms` header during transport response processing.
- Included `x-ratelimit-reset-ms` header support in `extractRetryHint` to better handle rate limit reset timings.
2026-07-05 12:03:48 +02:00
can1357 17552b9135 feat(utils): recovered unquoted bareword strings in json parser
- Added support for parsing unquoted bareword strings in object and array value positions.
- Implemented safety checks to prevent bareword recovery from masking structure, consuming non-finite atoms, or swallowing valid JSON delimiters.
- Included logic to preserve URL-style and Windows-style paths containing colons while rejecting invalid or ambiguous syntax.
2026-07-02 07:52:12 +02:00
can1357 d5b26b6b97 build(utils): relocated tls-fetch utility to the utilities package
- Extracted the `tls-fetch` implementation and tests from `@oh-my-pi/pi-ai` to `@oh-my-pi/pi-utils`.
- Exported the `wrapFetchForExtraCa` and `withExtraCaFetch` utilities publicly from `@oh-my-pi/pi-utils`.
- Introduced `ExtraCaError` to replace the AI-specific `ValidationError` for missing `NODE_EXTRA_CA_CERTS` paths.
- Updated imports in `packages/ai/src/stream.ts` to consume the relocated utility.
2026-07-02 01:51:09 +02:00
can1357 6c552e39cf fix(utils): handled unquoted YAML plain scalars containing colon-space in frontmatter
- Added preprocessing to quote ambiguous plain scalars containing a colon-space sequence when standard YAML parsing fails.
- Preserves the parsed types of unaffected fields and prevents fallback warnings for common unquoted description strings.
- Added tests to verify successful recovery of unquoted values and continued fallback warning coverage for unrecoverable syntax.
2026-06-30 17:44:19 +02:00
roboomp 650d34f111 fix(utils): preserved long retry hint fail-fast
- Restored the fetchWithRetry early return for Retry-After and quota hints larger than maxDelayMs.

- Added coverage so oversized provider retry hints do not sleep and retry internally.
2026-06-30 13:24:47 +00:00
roboomp cd6ba0214f fix(ai): hardened ollama malformed tool-call handling
- Added a fetchWithRetry response-body gate so deterministic 5xx provider failures can skip retry delays.

- Stopped retrying llama.cpp malformed tool-call JSON responses from local Ollama and surfaced recovery guidance.

- Covered the Ollama parse-failure path and generic retry gate behavior.

Fixes #3899
2026-06-30 13:20:11 +00:00
can1357 e8090bb48a feat: introduced binary file detection to prevent encoding corruption
- Introduced `isProbablyBinary` utility to sniff file headers for NUL bytes or invalid UTF-8 sequences.
- Updated `ReadTool` to use the binary sniffer, preventing mojibake corruption in output when reading non-text files.
- Refined `file-mentions` auto-reads to skip binary files and mark them as `binary` in the message transcript.
- Added comprehensive unit tests for binary detection logic, covering NUL bytes, truncated multibyte characters, and path-based file sniffing.
2026-06-30 02:59:41 +02:00
roboomp 777b609e63 fix(cli): preserved windows extension paths
Rejoined split Windows extension module paths before launch parsing finishes and stripped extended-length Win32 prefixes before Bun import and worker spawn APIs see them.

Fixes #3804
2026-06-29 11:50:36 +00:00
can1357 45918a9ccf Merge remote-tracking branch 'origin/farm/afffb7cd/fix-ptree-bytes-arraybuffer-normalize' 2026-06-28 16:08:37 +02:00
can1357 fbad280b57 feat: implemented multi-advisor concurrent runtime with tui management
- Introduced comprehensive support for multiple concurrent, independently-configured advisors via `WATCHDOG.yml` files.
- Implemented a full-screen TUI overlay for managing advisor rosters, models, tools, and instructions.
- Added session-wide advisor initialization, telemetry aggregation, and named transcript isolation.
- Enhanced advisor security and observability with secret redaction in tool results and secure XML attribute encoding.
2026-06-28 12:55:09 +02:00
roboomp 7d6a5c6de0 fix(utils): normalized ptree.bytes() return type to Uint8Array
Bun's `Response(stream).bytes()` returns the raw `ArrayBuffer` once the
body arrives in more than one chunk, which happens for subprocess stdout
past ~128 KB. The public contract of `ptree.ChildProcess.bytes()` is
`Promise<Uint8Array>`, and callers — most visibly the `ssh://` read
path's `decodeUtf8Text` — rely on `Uint8Array` methods such as `.indexOf`
and `.subarray`. On larger remote text files this surfaced as:

  TypeError: bytes.indexOf is not a function

Normalize the result at the boundary: when `Response.bytes()` hands back
an `ArrayBuffer`, wrap it in a zero-copy `Uint8Array` view before
returning. Adds a regression test that drives a 256 KB stdout payload
through `ptree.spawn(...).bytes()` and asserts the contract.

Fixes #3712
2026-06-28 10:04:53 +00:00
can1357 b5da1099af fix(pr-3347): address integration review notes 2026-06-27 02:04:51 +02:00
can1357 82b63976d8 Merge PR #3347: fix(utils): export removeWithRetries + migrate ai test files (@oldschoola) 2026-06-27 02:04:50 +02:00
can1357 a241cb0999 test: restore directory profile state after cache tests 2026-06-27 01:39:34 +02:00
can1357 9dea09e238 Merge PR #3264: feat(coding-agent): cache successful document conversions (@wolfiesch) 2026-06-27 01:39:33 +02:00
can1357 ae1650d689 refactor: renamed search and find tools to grep and glob
- Renamed the `find` and `search` tools to `glob` and `grep` respectively across the codebase to improve command clarity.
- Implemented full-stack support for the renamed tools, including CLI arguments, system prompts, SDK exports, and tool registration.
- Added automated migration logic in `settings` to transform legacy `find` and `search` configuration keys to their new equivalents.
- Updated the `collab-web` renderer registry to ensure backwards compatibility with legacy tool outputs.
2026-06-27 00:57:55 +02:00
can1357 97845b3d69 feat(coding-agent): added configurable worktree base directory
- Added a `worktree.base` setting to allow users to specify a custom directory for agent-managed git worktrees.
- Updated the `worktree` command to ensure settings are initialized before operations to respect the configured base path.
- Enhanced directory resolution logic to support `~` expansion and enforce absolute paths for worktree environments.
2026-06-27 00:30:47 +02:00
can1357 eb4a02433c refactor: consolidated json parsing and stream utilities
- Centralized JSON parsing and stream processing logic by moving utilities from `packages/ai` to the shared `@oh-my-pi/pi-utils` package.
- Standardized import paths for JSON parsing and streaming across the agent, ai, and coding-agent packages.
- Refactored SSE stream handling to use consolidated `parseStreamingJson` logic and introduced robust error recovery for malformed container-shaped tail events.
- Cleaned up legacy bundled registry references and updated related module exports and tests to reflect the new utility structure.
2026-06-27 00:11:42 +02:00
oldschoola d87c3e9ed8 fix(utils): increase EBUSY retry delay from 25ms to 50ms
The retry-on-EBUSY logic in removeSyncWithRetries/removeWithRetries
used 40 retries × 25ms = 1 second total. Windows can hold file locks
on SQLite databases for up to ~1.5s after close(), so tests that
already used removeSyncWithRetries (e.g. settings-manager.test.ts,
sdk-credential-disabled-bridge.test.ts) still failed with EBUSY.

Fix: increase the delay to 50ms (40 × 50ms = 2s total window). This
only affects Windows — the retry logic only runs on win32. Linux CI
is unaffected.

(cherry picked from commit ac67e96f1285ce0d80b2022c40c383b7ca262354)
2026-06-26 23:43:03 +02:00
can1357 9cd9349d05 Merge PR #3362: fix(utils): handle trailing-truncated JSON in readSseJson (@usr-bin-roygbiv) 2026-06-26 23:27:38 +02:00
can1357 a13c8d2698 fix(utils): prevented winston warnings when transports are disabled
- Set logger to silent mode when no transports are active to avoid "no transports" errors during log emission.
- Added a regression test to verify that disabling all transports suppresses warnings and that log output resumes after re-enabling transports.
2026-06-25 12:46:49 +02:00
can1357 2c80863e55 fix(utils): silenced logger when no transports are configured
- Added a check for empty transport arrays during logger initialization.
- Set the logger to silent mode if no transports exist to prevent unnecessary warning messages on every log emit.
2026-06-25 12:45:02 +02:00
usr_bin_roygbiv 5f22e58a61 fix(utils): require string closed in hasValidPredecessorForComma 2026-06-23 23:53:04 -05:00
usr_bin_roygbiv 2a31d063b2 fix(utils): handle string escapes and partial unicode escapes at EOF in getClosingSuffix 2026-06-23 23:43:46 -05:00
usr_bin_roygbiv 4528f5414a fix(utils): gate literal completion to non-inString state in getClosingSuffix 2026-06-23 23:39:56 -05:00
usr_bin_roygbiv 3a91053566 fix(utils): transition arrays back to expect_value on comma in scanJsonState 2026-06-23 23:36:28 -05:00
usr_bin_roygbiv a971f1aa9b fix(utils): validate comma predecessor using parser state instead of character rules 2026-06-23 23:32:42 -05:00
usr_bin_roygbiv fde4a33a59 fix(utils): handle invalid comma predecessors in isJsonTruncated 2026-06-23 23:29:36 -05:00
usr_bin_roygbiv 5032116702 fix(utils): handle truncated JSON gracefully in readSseJson 2026-06-23 23:23:55 -05:00
usr_bin_roygbiv 4d8a85e028 fix(utils): handle truncated JSON gracefully in readSseJson 2026-06-23 23:20:17 -05:00