Commit Graph
29 Commits
Author SHA1 Message Date
can1357 00fedcf6ac feat(ai): extended codex and stream timeout defaults to 300 seconds
- Increased codex websocket first event timeout default to 300 seconds.
- Increased default stream idle and first event timeout thresholds to 300 seconds.
- Updated stream timeout test expectations to match new 300s global default.
2026-07-30 05:33:06 +02:00
roboomp 946b7d9bd2 fix(ai): preserved disabled first-event sentinel
- Kept zero timeout resolutions distinct from missing first-event policy so the iterator does not fall back to the idle watchdog.
- Added deterministic coverage for an SSE response that opens before local prompt prefill emits its first event.
2026-07-24 15:33:27 +00:00
roboomp a421ea8acd fix(ai): disabled local first-event watchdogs
- Added a per-model first-event watchdog policy and disabled it for local OpenAI-compatible backends while retaining inter-event stall detection.
- Applied the policy to Responses and chat-completions transports with regression coverage for resolver and runtime precedence.

Fixes #6524
2026-07-24 15:19:14 +00:00
roboomp c49c693b73 fix(ai): retried codex pre-response watchdog timeouts
Gave each Codex SSE transport attempt an independent pre-response watchdog while reserving the retry-loop signal for caller cancellation.

Added provider-level coverage for retry recovery and explicit abort behavior.

Fixes #5329
2026-07-14 18:26:35 +00:00
can1357 7f77104929 fix(ai): deferred stream idle watchdog while cursor exec tools run locally
The lazy stream wrapper registered streamCursor without provider-handled
timeouts, so iterateWithIdleTimeout treated every gap between
AssistantMessageEvents as potential provider death. During a Cursor
exec-channel round-trip the server is waiting on OUR local tool result
(shell/read/grep/write/MCP/...) and legitimately sends nothing, so any
local tool outliving the idle budget (120s default) tripped onIdle ->
abortLocally(StreamTimeoutError) and killed a healthy stream mid-task.

Fix at the watchdog seam instead of faking progress events:
- EventStream tracks consumer-side local work in flight
  (trackLocalWork / hasPendingLocalWork).
- iterateWithIdleTimeout accepts a hasPendingLocalWork probe; an expired
  idle or first-event deadline slides forward while it reports true and
  the pending iterator.next() is persisted across the extension so no
  item is dropped. Once local work finishes the watchdog re-arms with a
  full budget, so genuinely silent streams still abort.
- forwardStream wires the probe for any provider stream instance.
- cursor.ts marks the exec-server dispatch as local work, covering every
  exec case including shellStream and MCP.

Unlike synthesizing empty toolcall_delta keepalives (PR #4594), no
synthetic events reach consumers: post-toolcall_end deltas would clobber
reconstructed tool arguments in proxy adapters and re-trigger TTSR
argument checks.

Adopted from PR #4594: the setCursorProviderModule test seam.

Fixes #4593
2026-07-09 18:30:48 +02:00
can1357 c3f7e849e5 refactor: centralized AI error handling into a dedicated module
- Migrated 288 lines of scattered error classification logic from `utils/error-id.ts` into a cohesive `packages/ai/src/error/` module with 13 specialized submodules covering flags, classes, OAuth, providers, rate-limiting, and finalization.
- Replaced 100+ generic `Error` throws across 60+ provider and registry files with semantic `AIError.*` classes (e.g., `AIError.MissingApiKeyError`, `AIError.OAuthError`, `AIError.ProviderResponseError`), improving error diagnostics and retry logic.
- Consolidated error utility imports from `pi-utils` and scattered classification functions into a single `AIError` namespace, reducing coupling and simplifying error handling across all packages.
2026-06-27 10:44:13 +02:00
can1357 0386d44fec refactor(ai): replaced abort signal timeouts with clearable fetch timers
- Prevents absolute `AbortSignal.timeout` from cutting off active stream bodies by introducing a clearable pre-response timer.
- Clears the watchdog timer for Bedrock, Gemini, Ollama, and Codex providers once headers are received.
- Restores regular caller abort signaling capability on the combined fetch stream.
- Adds comprehensive fake-timer tests to cover the timeout arming and clearing lifecycle.
2026-06-18 01:57:23 +02:00
can1357 cbecab2be2 fix(ai): prevented OpenAI streams from hanging after terminal completion
- Added a terminal-aware async iterator that enforces a post-completion grace window, then ends iteration and optionally aborts the request.
- Wrapped OpenAI completions streaming with terminal detection of finish-reason and usage payloads so iteration stops once a response is logically complete.
- Stopped OpenAI Responses stream consumption at terminal response events instead of waiting for connection close timeouts.
2026-06-11 16:08:52 +02:00
can1357 e6ca4d5763 fix(ai): hardened core stream, retry, and abort infrastructure
abort-aware auth retry loop preserving resolver errors; EventStream.end() can no longer strand .result(); Copilot retry honors Retry-After; DSML hold-back only triggers on real section prefixes and marks capped params explicitly; idle-iterator hoists racers with bounded reaction retention; validation errors truncate embedded args; removed dead leaky iterateUntilAbort.
2026-06-10 01:26:30 +02:00
can1357 421ff27eb6 fix(ai): scoped Anthropic fallback state and fixed iterator early-close cleanup
- Scoped Anthropic provider session state by base URL and model so strict-tools/fast-mode fallback state no longer leaks across unrelated endpoints or models, and fast-mode re-arming clears all matching scoped entries.
- Dropped stale strict fallback error messages after successful retries and only enabled adaptive thinking display for models that advertise support, avoiding unsupported-model 400 errors.
- Updated idle stream iteration to close the upstream iterator on non-continuing exits (including consumer early break) and added coverage for upstream closure behavior.
2026-06-08 14:44:13 +02:00
roboomp 062ba5e836 fix(ai): kept openai first-event env honored with per-call idle
Always route OpenAI-family providers through getOpenAIStreamFirstEventTimeoutMs so PI_OPENAI_STREAM_FIRST_EVENT_TIMEOUT_MS wins even when callers pass per-call streamIdleTimeoutMs. The OpenAI helper now floors the first-event budget at the caller-resolved idle (which already encompasses per-call streamIdleTimeoutMs or PI_OPENAI_STREAM_IDLE_TIMEOUT_MS upstream), and explicit env disables ("0") on either knob continue to drop the watchdog.

Fixes #1603
2026-05-31 19:02:53 +00:00
roboomp 239bb9858d fix(ai): honored openai idle timeout for first events
OpenAI-compatible local servers can spend longer than the generic first-event budget processing large prompts before they emit response headers or SSE frames. The OpenAI-specific idle timeout now also acts as the OpenAI-family first-event floor unless an explicit OpenAI first-event timeout is configured.

Added regression coverage for OpenAI Responses request setup so a lower generic first-event watchdog no longer undercuts PI_OPENAI_STREAM_IDLE_TIMEOUT_MS.

Fixes #1603
2026-05-31 18:53:58 +00:00
roboomp 8b19a9c292 fix(ai): widened glm coding-plan stream watchdog
Raised the default OpenAI-compatible stream idle floor for slow GLM-5.x coding-plan endpoints and taught the OpenAI timeout helper to honor provider fallbacks.

Added regression coverage for OpenAI timeout fallback precedence and GLM coding-plan fallback selection.

Fixes #1494
2026-05-29 05:54:22 +00:00
can1357 796c437dc1 feat: overhauled stream timeout and eval session management
- Replaced external watchdog timers with per-request SDK timeouts for first-event budget across OpenAI, Anthropic, and Azure providers.
- Keyed Python shared kernels by (sessionId, cwd) to prevent cross-directory state bleed.
- Deduplicated concurrent cold-start session acquisition for JS and Python executors.
- Moved `isOpenAIResponsesProgressEvent` to shared module and scoped display output routing per run for interleaved async cells.
2026-05-26 16:49:11 +02:00
can1357 6db59ccbac feat(ai/stream-timeouts): added deadline-based first-event timeout handling in stream providers
- Refactored stream timeout handling to allow providers to manage first-event and idle timeouts directly.
- Extracted firstEventTimeoutMs calculations into reusable variables across OpenAI, Azure OpenAI, and Codex providers.
- Consolidated timeout enforcement logic in idle-iterator to use absolute deadline checking and early callback invocation.
- Added providerHandlesStreamTimeouts flag to LazyStreamLimits interface to skip framework-level timeouts when providers handle them.
- Exported isOpenAIResponsesProgressEvent function and removed deprecated MockModelHandle type alias.
2026-05-26 15:27:06 +02:00
can1357 b0258e575a feat(ai): added prompt cache key and per-provider stream watchdog with idle timeout filtering
- Restored per-provider stream watchdog with idle and first-event timeout support, progress-event filtering, and per-provider timeout overrides via `getStreamIdleTimeoutMs()` and `getStreamFirstEventTimeoutMs()`.
- Fixed silent multi-hour hangs on Codex WebSocket and z.ai/GLM-via-OpenRouter subagent runs by filtering keepalive frames from idle watchdog resets.
- Added `isOpenAICompletionsProgressChunk` export and progress-event filtering to multiple providers (Anthropic, OpenAI Responses/Completions, Azure OpenAI, Codex) to prevent keepalive frames from resetting idle timeouts.
- Un-deprecated `StreamOptions.streamIdleTimeoutMs` and wired it into all built-in providers and lazy stream forwarder with environment variable fallback support.
2026-05-26 14:44:13 +02:00
roboomp e3f9b5b773 fix(ai): restored pi-style provider streaming
Removed OMP-owned first-event and idle watchdogs from provider stream consumption while preserving caller abort handling. Updated provider stream tests to assert slow/silent streams wait for provider output or caller abort instead of surfacing watchdog errors.

Fixes #1392
2026-05-26 04:55:00 +00:00
can1357 eef35a1cb3 fix(ai): raised default first-event watchdog for google-gemini-cli streams
- Added optional per-provider fallback parameters to stream timeout helper functions so callers can widen default watchdog values safely.
- Threaded provider-specific lazy stream limits into stream creation and set Google Gemini CLI to a 300000ms first-event fallback by default.
- Added tests covering fallback defaults, env precedence, and global-default behavior for both idle and first-event timeouts.
2026-05-26 05:17:58 +02:00
can1357 752c021f31 fix(ai): restored 120s idle timeout and fixed watchdog first-item logic
- Restored default stream idle timeout from 30s back to 120s (regression from 15.0.0).
- Fixed `iterateWithIdleTimeout` to only flip `awaitingFirstItem` on real progress items, not synthetic `start` events.
- Marked provider `start` events as non-progress keepalives in the lazy-stream wrapper.
2026-05-15 14:46:54 +02:00
ephraimduncan bea95b056d fix(ai): recover stalled lazy provider streams 2026-05-14 15:28:59 +00:00
can1357 9f386dd6e5 feat(ai,coding-agent): raw SSE diagnostics + steady-state idle watchdog
Adds an opt-in onSseEvent callback across HTTP-streaming providers (Anthropic, OpenAI Responses/Completions, Azure OpenAI Responses, OpenAI Codex SSE, Google Gemini CLI, GitLab Duo, Kimi, Synthetic) so callers can inspect raw SSE frames without altering parsed output. Provider fetch wrapping only tees response bodies when an observer is wired; standalone packages/ai consumers without onSseEvent are not penalized.

Adds streamIdleTimeoutMs (env: PI_STREAM_IDLE_TIMEOUT_MS, with PI_OPENAI_STREAM_IDLE_TIMEOUT_MS as a backward-compatible alias). Anthropic now enforces a steady-state idle watchdog (default 120s) in addition to the first-event watchdog. OpenAI Responses, Azure Responses, and Codex (SSE + WebSocket) gain a semantic-progress predicate so response.in_progress-style keepalives no longer keep stalled tool calls alive forever.

Adds a coding-agent debug-panel raw SSE viewer backed by a per-session bounded buffer (1000 records / 512KB) that AgentSession populates unconditionally so users can post-hoc inspect a stuck stream from the TUI.
2026-05-10 04:53:41 +02:00
can1357 e52480a0f3 fix(ai): race openai stream iteration against caller abort signal
The OpenAI completions/responses providers fed the SDK stream into
iterateWithIdleTimeout, which only raced iterator.next() against the
idle/first-event watchdog (default 100s) and relied on the underlying
fetch body reader to surface caller aborts. When the transport does not
propagate AbortSignal to body reads — the path triggered by
github-copilot's gpt-5* responses on macOS — ESC was observed only after
the watchdog elapsed, so the spinner appeared to hang forever.

Race iterator.next() against the caller signal as well, threading
options.signal into iterateWithIdleTimeout. On signal abort the
iterator is closed via iterator.return() and the abort reason is thrown,
letting the provider catch surface a clean stopReason: "aborted"
within tens of milliseconds.

Fixes #912
2026-05-02 08:08:10 +02:00
can1357 381a658f6a refactor(ai): restructured stream first-event watchdog handling
- Reworked `idle-iterator` to replace the first-event watchdog wrapper with a shared timeout handle passed through stream iteration.
- Updated Anthropic and OpenAI/Azure response-completion streams to pass the watchdog into `iterateWithIdleTimeout` and rely on unified first-event timeout logic.
- Raised the default first-stream-event timeout from 60 seconds to 100 seconds.
2026-04-15 18:50:41 +02:00
can1357 64e2ac6f11 feat(ai): enabled 120000ms OpenAI stream timeout to keep streams alive
- Increased DEFAULT_OPENAI_STREAM_IDLE_TIMEOUT_MS to 120,000 ms so OpenAI streams stay alive longer.
2026-04-11 06:38:21 +02:00
can1357 593add00fb feat(lsp): implemented async LSP server discovery with real-time startup events
- Added asynchronous LSP server discovery and warmup at startup via `discoverStartupLspServers()` function.
- Added LSP startup event channel (`lsp:startup`) for real-time server status notifications during warmup.
- Added `LspStartupServerInfo` type to track LSP server status (connecting, ready, error) throughout lifecycle.
- Changed LSP server warmup from blocking synchronous operation to non-blocking background task.
- Refactored interactive mode to subscribe to LSP startup events and display real-time server status updates.
2026-04-08 07:15:11 +02:00
can1357 8bd942fcf1 fix(ai): corrected first-event timeout to exclude request setup overhead
- Deferred first-event watchdog initialization until after stream request creation in OpenAI-family providers to exclude slow request setup from timeout.
- Increased default first-event timeout from 15s to 45s to accommodate longer request setup times.
- Modified first-event watchdog to inherit idle timeout when it exceeds the default threshold.
- Refactored timeout calculation logic to use Math.max for proper threshold enforcement.
- Added comprehensive test coverage for first-event timeout behavior across default, inherited, and override scenarios.
2026-04-02 00:07:16 +02:00
can1357 906123c97d style: reformat 2026-04-01 04:52:47 +02:00
can1357 bcee42ae3b feat(ai): implemented first-event timeout detection for streaming responses
- Added first-event timeout detection for streaming responses to prevent stuck requests and enable faster recovery from stream stalls.
- Added PI_STREAM_FIRST_EVENT_TIMEOUT_MS environment variable to configure first-event timeout behavior independently from idle timeouts.
- Fixed Anthropic stream timeout errors to be properly retried with first-event recognition, distinguishing first-event from idle timeouts.
- Enhanced idle-iterator utility with FirstEventWatchdog mechanism and markFirstStreamEvent wrapper for unified timeout handling across OpenAI, Azure OpenAI, and Anthropic providers.
2026-04-01 04:52:03 +02:00
can1357 7136808b9a fix(ai): added idle timeout watchdog to OpenAI-family streaming transports
- Added idle-iterator utility to enforce maximum gaps between stream events, preventing indefinite hangs when providers stop sending mid-response. Applied watchdog to OpenAI completions, responses, Azure OpenAI responses, and Codex SSE streams with configurable timeout via PI_OPENAI_STREAM_IDLE_TIMEOUT_MS.
- Added separate first-event timeout for Codex websocket connections to quickly fall back to SSE when a prewarmed socket connects but never starts the response stream. Codex websocket now validates auth headers match before reusing connections.
- Fixed OAuth token refresh and usage lookups to respect request timeouts instead of waiting indefinitely during credential selection or rotation.
2026-03-09 15:40:39 +01:00