feat(core): add first-class retry fallback chains for model/provider failover (#541)
* feat(rules): implemented alwaysApply auto-injection into system prompt
rules with alwaysApply: true were parsed by all providers and used to
exclude the rule from rulebookRules, but the inclusion half was never
built — rule content was silently dropped. now:
- full content is injected directly into the system prompt (before the
rulebook rules section) in both default and custom prompt templates
- rules remain addressable via rule:// for re-reading
- ttsr rules still take priority (condition + alwaysApply goes to ttsr only)
updated rulebook-matching-pipeline.md to reflect the three-bucket split
(ttsr > always-apply > rulebook) and corrected the rule:// resolution
docs.
* feat(browser): implement screenshot path saving
- Add `browser.screenshotDir` setting (tools tab) for a persistent
default screenshot directory, configurable via /settings
- Honour the existing `path` parameter in the screenshot action,
which was declared in the schema but never consumed by the implementation
- Resolution order: params.path (abs) > join(screenshotDir, params.path)
> join(screenshotDir, screenshot-<timestamp>.png) > /tmp only
- Writes full-resolution buffer to disk (not the API-compressed copy)
- Creates destination directory recursively if it doesn't exist
- details.screenshotPath reflects the actual saved location
Fixes: path param silently ignored since removal in v11.5.0
* feat(browser): expand ~ in screenshotDir and path params
Users can now configure browser.screenshotDir as ~/Downloads or
~/Pictures and use params.path as ~/screenshots/foo.png without
needing to supply fully-qualified paths.
* fix(browser): write screenshot to exactly one location
Previously wrote to /tmp unconditionally then copied to user path,
resulting in two files. Now resolves a single destination upfront:
1. params.path (absolute, or relative to screenshotDir/cwd)
2. screenshotDir + auto-timestamp filename
3. /tmp fallback (original behaviour, unchanged)
Also: user-defined destinations receive the full-res buffer;
/tmp fallback retains the API-compressed copy as before.
* feat(settings): add text input support for plain string settings
Settings with type: "string" and no submenu now render as an editable
text field in the /settings TUI panel instead of being silently skipped.
- settings-defs.ts: add TextInputSettingDef interface; pathToSettingDef
falls through to { type: "text" } for any plain string schema entry
- settings-selector.ts: add TextInputSubmenu class (mirrors
ConfigInputSubmenu from plugin-settings.ts); add "text" case to
#defToItem; add #createTextInput method
This makes browser.screenshotDir visible and editable in the Tools tab.
Empty field on submit clears the setting (falls back to /tmp behavior).
* fix(settings): text input — cursor at end, block tab navigation
- Cursor: call handleInput(ctrl+e) after setValue to jump to end of
pre-filled string instead of leaving it at position 0
- Tab/arrow guard: add #textInputActive flag; suppress tab-switch and
left/right routing to tab bar while a TextInputSubmenu is open, so
arrow keys reach the Input component's cursor movement handlers
instead of switching settings tabs
* fix(browser): expand ~\ (Windows backslash) in screenshot paths
expandHome now handles both Unix ~/... and Windows ~\... separators,
matching user expectation on all platforms. Addresses Codex review.
* fix(coding-agent): use expandPath for screenshots
* docs: add changelog for screenshot path option
Bug fixes:
- Add browser.screenshotDir to settings-schema.ts (lost during rebase conflict)
- Fix stale expandHome comment to expandPath in settings-selector.ts
- Final screenshotDir description: Directory to save screenshots with ~ support
* fix: biome format corrections for screenshot-path-option branch
* fix: add missing #textInputActive field and screenshotDir default
* fix(browser): align screenshot metadata with saved file contents
When screenshotDir or params.path is set, the full-resolution PNG buffer
is written to disk. Previously mimeType/bytes in details still reflected
the resized payload sent to the model, making metadata inconsistent with
the actual saved file.
Now savedBuffer/savedMimeType track what is written, and details reflects
that. Display output distinguishes 'Saved' vs 'Model' when full-res is
used, and collapses to a single Format/Dimensions line for temp-only.
* fix(browser): resolve params.path relative to cwd regardless of screenshotDir
screenshotDir is a default save location, not an anchor for explicit
paths. A relative params.path should always resolve against cwd so its
semantics are stable and predictable regardless of user settings.
* fix(tui): enforce strict line budget for collapsed tool output
The grep, ast_grep, and ast_edit renderers used group-count-based
collapse that always included the first group unconditionally,
allowing collapsed output to remain visually large when a single
group contained many lines.
Add maxCollapsedLines to renderTreeList that enforces a strict
total-line cap in collapsed mode. Items that exceed the remaining
budget are skipped entirely (no broken fragments). The isLast tree
branch is computed after the budget check to avoid double-last
branches when a summary line follows.
Remove the per-tool getCollapsedMatchLimit / getCollapsedChangeLimit
helpers that are now redundant.
Fixes #455
Made-with: Cursor
* WIP
* WIP
* cleanup
* docs(coding-agent): update changelog for custom model tags and cycle order
* Fix custom model precedence across load and refresh
* feat(ask): add multiline editor support for custom input
* feat(extension-ui): add dialog options and abort signal support to editor
* docs(ask): add multiline editor and timeout behavior guidance
* docs(ask): simplify multiline input documentation
* fix(tui): preserve terminal scrollback during full redraws
Replace destructive \x1b[3J\x1b[2J\x1b[H full-redraw sequence with
scrollback-preserving repaint helpers:
- seedTranscript: first paint with no prior frame, writes full transcript
without clearing scrollback
- repaintViewport: trusted-frame repaints that scroll viewport-shift
delta into scrollback before overwriting visible rows in-place
- Height-increase handler that pushes revealed scrollback back before
reclaiming the display
Replace requestRender(true) state destruction with a one-shot
without resetting #previousLines or cursor bookkeeping.
Track #previousHeight to detect terminal height increases that pull
scrollback lines into the visible area.
fixes #507
* fix(tui): fix exit gaps, content shrink drift, and overlay cursor recovery
- Rewrite stop() to use viewport-relative cursor positioning instead of
content-length, preventing blank gaps when content is shorter than viewport
- Replace trailing \r\n\x1b[2K clear loops in repaintViewport() and
height-increase path with \r\n\x1b[J to avoid cursor drift past content
- Add 12 TUI regression tests for exit gaps, content shrink, and overlay
dismiss cursor recovery
- Add 6 coding-agent controller tests for /new and /tree commands
* fix(tui): redraw sparse height increases atomically
* fix(coding-agent,tui): address review regressions
* fix(tui): reseed after terminal resume
* test(ai): avoid leaking kagi module mocks
* fix(ask): keep multiline custom input in prompt gutter
* fix(ask): preserve multiselect choices on editor dismiss
* fix(ask): honor app interrupt in prompt editor
* fix(coding-agent): preserve new-session approval state
* feat(core): add session-scoped model/provider retry fallback policy
* feat(core): validate retry fallback chains on session startup
* fix(ask): preserve prior answer when custom editor dismissed in single-select
* fix(core): harden retry fallback policy semantics
* fix(core): correct retry fallback edge cases
* test(coding-agent): removed macOS fallback test case from theme detection
- Removed test case for macOS fallback behavior inside Zellij.
* refactor: simplified null checks using optional chaining across TypeScript and Rust modules
- Simplified null/empty checks across TypeScript codebase using optional chaining operator (?.) for improved readability.
- Replaced explicit null checks in validation logic with optional chaining in oauth-discovery, gemini-cli, claude, zai, and lsp modules.
- Updated error handling in Rust command invocation to use double question mark operator (??) for cmd_result.
- Consolidated null validation patterns across tools (bash-skill-urls, browser, gemini-image, resolve) and keybindings using optional chaining.
* chore: bump version to 13.15.1
* fix(core): address PR review on fallback retry behavior
* test(ai): refactored auth storage test to use spyOn for cleaner mocks
- Refactored auth storage test to use vi.spyOn() instead of vi.mock() for cleaner mock management.
- Simplified mock type definitions by leveraging bun:test's Mock type import.
* chore: bump version to 13.15.2
* Fix stale OpenAI Responses replay across session boundaries (#534)
* Fix stale OpenAI Responses replay across session boundaries
Fixes #505
* Fix CI tests for session replay change
* Harden session reload and switch rollback
* Guard session switch snapshots
* fix(coding-agent): preserve responses replay snapshots
* Normalize pasted image formats before attach (#543)
Co-authored-by: iter <itertoolz@gmail.com>
* Allow overriding the Codex web search model (#516)
* Allow codex web search model override
* Handle blank codex web search model
* feat(ai): add gemini-3.1-pro-preview models to google-vertex provider (#521)
Add gemini-3.1-pro-preview and gemini-3.1-pro-preview-customtools to
the google-vertex provider in models.json, matching the existing
google-generative-ai entries.
Fixes #520
Co-authored-by: Muness Castle <munesscastle@artium.ai>
* Make temporary model selector keybinding configurable (#539)
Fixes #533
* chore: bump models.json
* refactor(coding-agent): migrated test mocks to vitest spyOn with Symbol.dispose cleanup
- Migrated test mocking from bun:test mock API to vitest spyOn pattern across 9 test files.
- Extracted mock setup logic into reusable helper functions with Symbol.dispose cleanup pattern.
- Replaced manual beforeEach/afterEach and try-finally blocks with TypeScript 5.2 using declarations.
- Removed 178 lines of boilerplate mock initialization and restoration code from test suite.
* chore: bump version to 13.15.3
* fix(ask): restore prompt-style enter handling
* fix(tui): avoid blank scrollback regressions
* fix(models): keep same-id replacements authoritative
* fix(tui): enforce collapsed line budgets
* fix(models): unify selector role sources
* fix(rules): dedupe always-apply prompt injection
* fix(browser): show saved screenshot path
* style: format merged PR fixes
* refactor(coding-agent): restructured screenshot and prompt utilities into focused helpers
- Extracted screenshot formatting logic into dedicated `formatScreenshot()` function with options support.
- Consolidated prompt source deduplication into `dedupePromptSource()` helper to prevent rule duplication.
- Refactored editor text sanitization to use `replaceTabs()` utility for consistent tab width handling.
- Added test coverage verifying editor respects configured tab width when loading text programmatically.
* refactor(coding-agent): restructured validation and rendering for consistency
- Refactored theme color validation to use single source of truth with THEME_COLOR_RECORD object.
- Simplified model registry to defer per-model overrides to dedicated method and use constant for role IDs.
- Refactored tree list rendering to pre-render items once for consistent line counts across phases.
- Refactored question result formatting to use early returns and consistently include question ID in output.
- Updated hook editor hint text to include ctrl+g external editor option when prompt style is enabled.
- Removed unused isLogicalLineStart property from LayoutLine interface in editor component.
* revert: 535 due to TUI regressions
* test(coding-agent): corrected hook-editor assertion for keybinding render
- Corrected assertion in hook-editor test to verify external editor keybinding is rendered.
* feat(tools): added root path alias to resolve bare / to working directory
- Added root path alias feature to resolve bare `/` to session working directory in path resolution.
- Updated browser tool to use `resolveToCwd()` for consistent workspace-relative path handling.
- Added comprehensive test suite validating root path alias resolution across grep, read, find, ast_grep, and ast_edit tools.
* feat(prompts/tools): clarified hashline block boundary handling with examples
- Improved hashline tool documentation with clearer guidance on block boundary handling and closing delimiter duplication prevention.
- Added concrete example demonstrating correct anchor placement when replacing entire blocks including closing braces.
- Reorganized boundary duplication warnings into actionable self-check guidance with visual comparison steps.
* chore: bump version to 13.16.0
* fix(coding-agent): fixed python kernel startup hangs (#548)
* fix(coding-agent): fixed python kernel startup hangs
* fix(coding-agent): fixed startup timeout regressions
* fix(coding-agent): preserved startup cancellation typing
* perf(pi-natives): optimized memory allocation with MiMalloc integration
- Integrated MiMalloc as global allocator to improve memory allocation performance.
* feat: fff
- Added SearchDb class for stateful shared search database instances enabling persistent file indexing and frecency tracking across grep, glob, and fuzzyFind operations.
- Added optional db parameter to grep(), glob(), and fuzzyFind() functions for database-backed searching with improved performance via cached file indices.
- Replaced grep-searcher with fff-grep and added fff-search dependency for enhanced file discovery and search capabilities with memory-mapped file support.
- Migrated fuzzy file discovery from fd module to fff module with SearchDb integration for stateful caching and improved search performance.
- Exported SearchDb type from @oh-my-pi/pi-natives public API for type-safe usage in grep, glob, and fuzzyFind workflows.
* feat(pi-natives): added unified picker coordination for file search operations
- Added `wait_for_picker_scan()` function to search_db module with cancellation token support for polling picker scan completion.
- Integrated picker-based file search into glob matching logic with `collect_files_from_picker()` helper to reuse shared SearchDb picker results.
- Refactored fff and grep modules to use centralized `wait_for_picker_scan()` wrapper instead of direct FilePicker calls, improving cancellation handling.
- Propagated SearchDb instance through agent session initialization and input controller to enable unified file picker coordination across search operations.
* chore: bump version to 13.16.1
* fix: install zig in CI
* feat(tui): make inline image max-width configurable via tui.maxInlineImageColumns (#551)
* feat(tui): make inline image max-width configurable via tui.maxInlineImageColumns
* fix(tui): handle 0 as unlimited in maxInlineImageColumns; drop || undefined coercion
* feat(browser): auto-detect NixOS and use system Chromium (#550)
Puppeteer's bundled Chromium is a dynamically-linked FHS binary that
cannot run on NixOS. On startup, resolveSystemChromium() checks for
/etc/NIXOS and searches for a usable binary in order:
1. chromium on PATH
2. chromium-browser on PATH
3. ~/.nix-profile/bin/chromium
4. /run/current-system/sw/bin/chromium
The resolved path is passed as executablePath to puppeteer.launch().
Result is cached per process. On non-NixOS systems the function returns
undefined immediately, leaving Puppeteer's default resolution intact.
* feat(pi-natives): added automatic parenthesis escaping in regex patterns
- Added automatic escaping of unescaped parentheses in regex patterns when group syntax errors occur, enabling literal function call patterns like `fetchAnthropicProvider(` to work as search queries.
- Extracted regex matcher builder into separate function for reusability and error recovery logic.
- Added 2 test cases validating parenthesis escaping behavior for both escaped and literal parentheses.
- Fixed documentation formatting in sanitize_braces comment.
* style: reformat
* chore: bump version to 13.16.2
* fix: only show update banner when npm version is strictly newer (#552)
* fix(ai): corrected OAuth credential updates to replace in-place instead of accumulating soft-deleted rows
- Fixed OAuth credential updates to replace matching credentials in-place rather than creating disabled rows, preventing unbounded accumulation of soft-deleted credentials.
- Modified OAuth credential saving to preserve unrelated identities instead of replacing all credentials for a provider.
- Updated credential identity resolution to use provider context for more accurate email deduplication.
- Implemented upsertAuthCredentialForProvider method to handle credential matching and in-place updates.
- Added 5 test cases covering credential preservation across reauth, multi-account scenarios, and stale cache handling.
* chore: bump version to 13.16.3
* feat: introduced unified range API for hashline edits and model catalog updates
- Simplified hashline edit location API by replacing separate `line` and `block` properties with unified `range` property accepting `{ pos, end }` anchors.
- Renamed hashline helper functions from `hlineref`/`hlinefull` to `href`/`hline` for improved brevity and consistency.
- Added detection for `kysely-codegen` generated files in auto-generated file guard with corresponding test coverage.
- Added 13 new AI model configurations and updated token limits and pricing for existing models across multiple providers.
- Enhanced file type validation in grep native to reject symlinks, FIFOs, sockets, and non-regular files with improved error handling.
* chore: bump version to 13.16.4
* fix: pin rustc-hash to 2.1.1 to avoid SIGILL on CI
rustc-hash 2.1.2 (released today) refactored hash_bytes to use
split_first_chunk, which produces illegal instructions when compiled
with nightly + -C target-cpu=x86-64-v3 on CI runners.
* fix(ci): pin nightly to 2026-03-27 to avoid codegen SIGILL regression
Today's nightly produces illegal instructions when compiled with
-C target-cpu=x86-64-v3. Reverts the unnecessary rustc-hash pin from
the previous commit since the real cause is the nightly compiler.
Also lets Cargo.lock return to rustc-hash 2.1.2 (not the culprit).
* fix(ci): add rustup target fallback for pinned nightly cross-compile
* fix(coding-agent): do not prompt to use grep and find tools if they are disabled (#566)
Co-authored-by: le-cameleon <200889489+le-cameleon@users.noreply.github.com>
* fix(natives): skipped grep special files (#565)
avoided opening fifos and other special filesystem nodes during grep and added fifo regressions in native and coding-agent tests.
* fix: skill baseDir regex fails on Windows backslash paths (#554)
The regex that strips SKILL.md from the path to compute baseDir only
matches forward slashes. On Windows where paths use backslashes, the
replace is a no-op and baseDir equals the full SKILL.md file path.
This breaks sub-path resolution for skills: the subpath gets appended
to the SKILL.md file path instead of the skill directory.
Fix: use character class matching both path separators.
---------
Co-authored-by: deadcode-walker <268043493+deadcode-walker@users.noreply.github.com>
Co-authored-by: Rens Tillmann <rens@super-forms.com>
Co-authored-by: haiyang.zhou <haiyang.zhou@seamoney.com>
Co-authored-by: Leo P <junk@slact.net>
Co-authored-by: Zakhar Kogan <36503576+zaharkogan@users.noreply.github.com>
Co-authored-by: Vu Anh Nguyen <vuanhng00@gmail.com>
Co-authored-by: can1357 <me@can.ac>
Co-authored-by: daandden <64765666+daandden@users.noreply.github.com>
Co-authored-by: iter <72358817+itertea@users.noreply.github.com>
Co-authored-by: iter <itertoolz@gmail.com>
Co-authored-by: Cheol Kang <dev@cheol.me>
Co-authored-by: Muness Castle <931+muness@users.noreply.github.com>
Co-authored-by: Muness Castle <munesscastle@artium.ai>
Co-authored-by: zamo <falby97@proton.me>
Co-authored-by: elikoga <elikowa@gmail.com>
Co-authored-by: BayLee4 <63376748+BayLee4@users.noreply.github.com>
Co-authored-by: le-cameleon <200889489+le-cameleon@users.noreply.github.com>
Co-authored-by: Wiedzmin <56316383+art-wiedzmin@users.noreply.github.com>
This commit is contained in:
@@ -885,16 +885,33 @@ theme:
|
||||
dark: titanium
|
||||
light: light
|
||||
|
||||
enabledModels:
|
||||
- "anthropic/*"
|
||||
- "*gpt*"
|
||||
- "gemini-2.5-pro:high"
|
||||
|
||||
modelRoles:
|
||||
default: anthropic/claude-sonnet-4-20250514
|
||||
plan: anthropic/claude-opus-4-1:high
|
||||
smol: anthropic/claude-sonnet-4-20250514
|
||||
defaultThinkingLevel: high
|
||||
enabledModels:
|
||||
- anthropic/*
|
||||
- "*gpt*"
|
||||
- gemini-2.5-pro:high
|
||||
|
||||
retry:
|
||||
enabled: true
|
||||
# Number of retries before giving up on rate limits/server errors
|
||||
maxRetries: 3
|
||||
# Wait this long as a base (exponentially backed off) unless the API provides a retry-after-ms
|
||||
baseDelayMs: 2000
|
||||
# Configure role-specific model fallback chains
|
||||
fallbackChains:
|
||||
default:
|
||||
- "openai/gpt-4o-mini"
|
||||
- "openai/gpt-4o"
|
||||
plan:
|
||||
- "anthropic/claude-sonnet-4-6:high"
|
||||
- "openai/o3:high"
|
||||
# Whether to revert to the primary model when a fallback's cooldown expires
|
||||
fallbackRevertPolicy: cooldown-expiry
|
||||
steeringMode: one-at-a-time
|
||||
followUpMode: one-at-a-time
|
||||
interruptMode: immediate
|
||||
@@ -914,10 +931,6 @@ compaction:
|
||||
skills:
|
||||
enabled: true
|
||||
|
||||
retry:
|
||||
enabled: true
|
||||
maxRetries: 3
|
||||
baseDelayMs: 2000
|
||||
|
||||
terminal:
|
||||
showImages: true
|
||||
|
||||
@@ -28,6 +28,7 @@ import {
|
||||
import { isRecord, logger } from "@oh-my-pi/pi-utils";
|
||||
import { type Static, Type } from "@sinclair/typebox";
|
||||
import { type ConfigError, ConfigFile } from "../config";
|
||||
import { parseModelString } from "../config/model-resolver";
|
||||
import { isValidThemeColor, type ThemeColor } from "../modes/theme/theme";
|
||||
import type { AuthStorage, OAuthCredential } from "../session/auth-storage";
|
||||
import type { Settings } from "./settings";
|
||||
@@ -721,6 +722,14 @@ function buildCustomModel(
|
||||
return finalizeCustomModel(model, options);
|
||||
}
|
||||
|
||||
function normalizeSuppressedSelector(selector: string): string {
|
||||
const trimmed = selector.trim();
|
||||
if (!trimmed) return trimmed;
|
||||
const parsed = parseModelString(trimmed);
|
||||
if (!parsed) return trimmed;
|
||||
return `${parsed.provider}/${parsed.id}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Model registry - loads and manages models, resolves API keys via AuthStorage.
|
||||
*/
|
||||
@@ -737,6 +746,7 @@ export class ModelRegistry {
|
||||
#registeredProviderSources: Set<string> = new Set();
|
||||
#providerDiscoveryStates: Map<string, ProviderDiscoveryState> = new Map();
|
||||
#cacheDbPath?: string;
|
||||
#suppressedSelectors: Map<string, number> = new Map();
|
||||
#backgroundRefresh?: Promise<void>;
|
||||
#lastDiscoveryWarnings: Map<string, string> = new Map();
|
||||
|
||||
@@ -766,6 +776,7 @@ export class ModelRegistry {
|
||||
*/
|
||||
async refresh(strategy: ModelRefreshStrategy = "online-if-uncached"): Promise<void> {
|
||||
this.#reloadStaticModels();
|
||||
this.#suppressedSelectors.clear();
|
||||
await this.#refreshRuntimeDiscoveries(strategy);
|
||||
}
|
||||
|
||||
@@ -789,6 +800,11 @@ export class ModelRegistry {
|
||||
|
||||
async refreshProvider(providerId: string, strategy: ModelRefreshStrategy = "online"): Promise<void> {
|
||||
this.#reloadStaticModels();
|
||||
for (const selector of this.#suppressedSelectors.keys()) {
|
||||
if (selector.startsWith(`${providerId}/`)) {
|
||||
this.#suppressedSelectors.delete(selector);
|
||||
}
|
||||
}
|
||||
await this.#refreshRuntimeDiscoveries(strategy, new Set([providerId]));
|
||||
}
|
||||
|
||||
@@ -1825,6 +1841,27 @@ export class ModelRegistry {
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Suppress a specific model selector (e.g., "provider/id") until a specific timestamp.
|
||||
*/
|
||||
suppressSelector(selector: string, untilMs: number): void {
|
||||
this.#suppressedSelectors.set(normalizeSuppressedSelector(selector), untilMs);
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if a model selector is currently suppressed due to rate limits.
|
||||
*/
|
||||
isSelectorSuppressed(selector: string): boolean {
|
||||
const normalizedSelector = normalizeSuppressedSelector(selector);
|
||||
const suppressedUntil = this.#suppressedSelectors.get(normalizedSelector);
|
||||
if (!suppressedUntil) return false;
|
||||
if (suppressedUntil <= Date.now()) {
|
||||
this.#suppressedSelectors.delete(normalizedSelector);
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -525,6 +525,18 @@ export const SETTINGS_SCHEMA = {
|
||||
},
|
||||
|
||||
"retry.baseDelayMs": { type: "number", default: 2000 },
|
||||
"retry.fallbackChains": { type: "record", default: {} as Record<string, string[]> },
|
||||
"retry.fallbackRevertPolicy": {
|
||||
type: "enum",
|
||||
values: ["cooldown-expiry", "never"] as const,
|
||||
default: "cooldown-expiry",
|
||||
ui: {
|
||||
tab: "model",
|
||||
label: "Fallback Revert Policy",
|
||||
description: "When to return to the primary model after a fallback",
|
||||
submenu: true,
|
||||
},
|
||||
},
|
||||
|
||||
// ────────────────────────────────────────────────────────────────────────
|
||||
// Interaction
|
||||
|
||||
@@ -117,6 +117,15 @@ const OPTION_PROVIDERS: Partial<Record<SettingPath, OptionProvider>> = {
|
||||
{ value: "5", label: "5 retries" },
|
||||
{ value: "10", label: "10 retries" },
|
||||
],
|
||||
// Retry fallback revert policy
|
||||
"retry.fallbackRevertPolicy": [
|
||||
{
|
||||
value: "cooldown-expiry",
|
||||
label: "Cooldown expiry",
|
||||
description: "Return to the primary model after its suppression window ends",
|
||||
},
|
||||
{ value: "never", label: "Never", description: "Stay on the fallback model until manually changed" },
|
||||
],
|
||||
// Task max concurrency
|
||||
"task.maxConcurrency": [
|
||||
{ value: "0", label: "Unlimited" },
|
||||
|
||||
@@ -534,6 +534,16 @@ export class EventController {
|
||||
break;
|
||||
}
|
||||
|
||||
case "retry_fallback_applied": {
|
||||
this.ctx.showWarning(`Fallback: ${event.from} -> ${event.to}`);
|
||||
break;
|
||||
}
|
||||
|
||||
case "retry_fallback_succeeded": {
|
||||
this.ctx.showStatus(`Fallback succeeded on ${event.model}`);
|
||||
break;
|
||||
}
|
||||
|
||||
case "ttsr_triggered": {
|
||||
const component = new TtsrNotificationComponent(event.rules);
|
||||
component.setExpanded(this.ctx.toolOutputExpanded);
|
||||
|
||||
@@ -306,6 +306,11 @@ export class InteractiveMode implements InteractiveModeContext {
|
||||
|
||||
const startupQuiet = settings.get("startup.quiet");
|
||||
|
||||
for (const warning of this.session.configWarnings) {
|
||||
this.ui.addChild(new Text(theme.fg("warning", `Warning: ${warning}`), 1, 0));
|
||||
this.ui.addChild(new Spacer(1));
|
||||
}
|
||||
|
||||
if (!startupQuiet) {
|
||||
// Add welcome header
|
||||
const welcome = new WelcomeComponent(this.#version, modelName, providerName, recentSessions, lspServerInfo);
|
||||
|
||||
@@ -55,7 +55,12 @@ import { abortableSleep, getAgentDbPath, isEnoent, logger } from "@oh-my-pi/pi-u
|
||||
import type { AsyncJob, AsyncJobManager } from "../async";
|
||||
import type { Rule } from "../capability/rule";
|
||||
import { MODEL_ROLE_IDS, type ModelRegistry } from "../config/model-registry";
|
||||
import { extractExplicitThinkingSelector, parseModelString, resolveModelRoleValue } from "../config/model-resolver";
|
||||
import {
|
||||
extractExplicitThinkingSelector,
|
||||
formatModelString,
|
||||
parseModelString,
|
||||
resolveModelRoleValue,
|
||||
} from "../config/model-resolver";
|
||||
import { expandPromptTemplate, type PromptTemplate, renderPromptTemplate } from "../config/prompt-templates";
|
||||
import type { Settings, SkillsSettings } from "../config/settings";
|
||||
import { type BashResult, executeBash as executeBashCommand } from "../exec/bash-executor";
|
||||
@@ -170,6 +175,8 @@ export type AgentSessionEvent =
|
||||
}
|
||||
| { type: "auto_retry_start"; attempt: number; maxAttempts: number; delayMs: number; errorMessage: string }
|
||||
| { type: "auto_retry_end"; success: boolean; attempt: number; finalError?: string }
|
||||
| { type: "retry_fallback_applied"; from: string; to: string; role: string }
|
||||
| { type: "retry_fallback_succeeded"; model: string; role: string }
|
||||
| { type: "ttsr_triggered"; rules: Rule[] }
|
||||
| { type: "todo_reminder"; todos: TodoItem[]; attempt: number; maxAttempts: number }
|
||||
| { type: "todo_auto_clear" };
|
||||
@@ -315,6 +322,46 @@ interface HandoffOptions {
|
||||
|
||||
const AUTO_HANDOFF_THRESHOLD_FOCUS = renderPromptTemplate(autoHandoffThresholdFocusPrompt);
|
||||
|
||||
type RetryFallbackChains = Record<string, string[]>;
|
||||
|
||||
type RetryFallbackRevertPolicy = "never" | "cooldown-expiry";
|
||||
|
||||
interface RetryFallbackSelector {
|
||||
raw: string;
|
||||
provider: string;
|
||||
id: string;
|
||||
thinkingLevel: ThinkingLevel | undefined;
|
||||
}
|
||||
|
||||
interface ActiveRetryFallbackState {
|
||||
role: string;
|
||||
originalSelector: string;
|
||||
originalThinkingLevel: ThinkingLevel | undefined;
|
||||
lastAppliedFallbackThinkingLevel: ThinkingLevel | undefined;
|
||||
}
|
||||
|
||||
function parseRetryFallbackSelector(selector: string): RetryFallbackSelector | undefined {
|
||||
const trimmed = selector.trim();
|
||||
if (!trimmed) return undefined;
|
||||
const parsed = parseModelString(trimmed);
|
||||
if (!parsed) return undefined;
|
||||
return {
|
||||
raw: trimmed,
|
||||
provider: parsed.provider,
|
||||
id: parsed.id,
|
||||
thinkingLevel: parsed.thinkingLevel,
|
||||
};
|
||||
}
|
||||
|
||||
function formatRetryFallbackSelector(model: Model, thinkingLevel: ThinkingLevel | undefined): string {
|
||||
const selector = formatModelString(model);
|
||||
return thinkingLevel ? `${selector}:${thinkingLevel}` : selector;
|
||||
}
|
||||
|
||||
function formatRetryFallbackBaseSelector(selector: RetryFallbackSelector): string {
|
||||
return `${selector.provider}/${selector.id}`;
|
||||
}
|
||||
|
||||
const noOpUIContext: ExtensionUIContext = {
|
||||
select: async (_title, _options, _dialogOptions) => undefined,
|
||||
confirm: async (_title, _message, _dialogOptions) => false,
|
||||
@@ -352,6 +399,7 @@ export class AgentSession {
|
||||
readonly sessionManager: SessionManager;
|
||||
readonly settings: Settings;
|
||||
readonly searchDb: SearchDb | undefined;
|
||||
readonly configWarnings: string[] = [];
|
||||
|
||||
#asyncJobManager: AsyncJobManager | undefined = undefined;
|
||||
#scopedModels: Array<{ model: Model; thinkingLevel?: ThinkingLevel }>;
|
||||
@@ -391,7 +439,7 @@ export class AgentSession {
|
||||
#retryAttempt = 0;
|
||||
#retryPromise: Promise<void> | undefined = undefined;
|
||||
#retryResolve: (() => void) | undefined = undefined;
|
||||
|
||||
#activeRetryFallback: ActiveRetryFallbackState | undefined = undefined;
|
||||
// Todo completion reminder state
|
||||
#todoReminderCount = 0;
|
||||
#todoPhases: TodoPhase[] = [];
|
||||
@@ -478,6 +526,7 @@ export class AgentSession {
|
||||
this.#customCommands = config.customCommands ?? [];
|
||||
this.#skillsSettings = config.skillsSettings;
|
||||
this.#modelRegistry = config.modelRegistry;
|
||||
this.#validateRetryFallbackChains();
|
||||
this.#toolRegistry = config.toolRegistry ?? new Map();
|
||||
this.#transformContext = config.transformContext ?? (messages => messages);
|
||||
this.#onPayload = config.onPayload;
|
||||
@@ -787,6 +836,13 @@ export class AgentSession {
|
||||
assistantMsg.stopReason !== "aborted" &&
|
||||
this.#retryAttempt > 0
|
||||
) {
|
||||
if (this.#activeRetryFallback && this.model) {
|
||||
await this.#emitSessionEvent({
|
||||
type: "retry_fallback_succeeded",
|
||||
model: formatRetryFallbackSelector(this.model, this.thinkingLevel),
|
||||
role: this.#activeRetryFallback.role,
|
||||
});
|
||||
}
|
||||
await this.#emitSessionEvent({
|
||||
type: "auto_retry_end",
|
||||
success: true,
|
||||
@@ -985,6 +1041,7 @@ export class AgentSession {
|
||||
return;
|
||||
}
|
||||
try {
|
||||
await this.#maybeRestoreRetryFallbackPrimary();
|
||||
await this.agent.continue();
|
||||
} catch {
|
||||
options?.onError?.();
|
||||
@@ -2288,6 +2345,8 @@ export class AgentSession {
|
||||
// Reset todo reminder count on new user prompt
|
||||
this.#todoReminderCount = 0;
|
||||
|
||||
await this.#maybeRestoreRetryFallbackPrimary();
|
||||
|
||||
// Validate model
|
||||
if (!this.model) {
|
||||
throw new Error(
|
||||
@@ -3108,6 +3167,7 @@ export class AgentSession {
|
||||
throw new Error(`No API key for ${model.provider}/${model.id}`);
|
||||
}
|
||||
|
||||
this.#clearActiveRetryFallback();
|
||||
this.#setModelWithProviderSessionReset(model);
|
||||
this.sessionManager.appendModelChange(`${model.provider}/${model.id}`, role);
|
||||
this.settings.setModelRole(role, this.#formatRoleModelValue(role, model));
|
||||
@@ -3128,6 +3188,7 @@ export class AgentSession {
|
||||
throw new Error(`No API key for ${model.provider}/${model.id}`);
|
||||
}
|
||||
|
||||
this.#clearActiveRetryFallback();
|
||||
this.#setModelWithProviderSessionReset(model);
|
||||
this.sessionManager.appendModelChange(`${model.provider}/${model.id}`, "temporary");
|
||||
this.settings.getStorage()?.recordModelUsage(`${model.provider}/${model.id}`);
|
||||
@@ -3253,6 +3314,7 @@ export class AgentSession {
|
||||
const next = scopedModels[nextIndex];
|
||||
|
||||
// Apply model
|
||||
this.#clearActiveRetryFallback();
|
||||
this.#setModelWithProviderSessionReset(next.model);
|
||||
this.sessionManager.appendModelChange(`${next.model.provider}/${next.model.id}`);
|
||||
this.settings.setModelRole("default", this.#formatRoleModelValue("default", next.model));
|
||||
@@ -3281,11 +3343,11 @@ export class AgentSession {
|
||||
throw new Error(`No API key for ${nextModel.provider}/${nextModel.id}`);
|
||||
}
|
||||
|
||||
this.#clearActiveRetryFallback();
|
||||
this.#setModelWithProviderSessionReset(nextModel);
|
||||
this.sessionManager.appendModelChange(`${nextModel.provider}/${nextModel.id}`);
|
||||
this.settings.setModelRole("default", this.#formatRoleModelValue("default", nextModel));
|
||||
this.settings.getStorage()?.recordModelUsage(`${nextModel.provider}/${nextModel.id}`);
|
||||
|
||||
// Re-apply the current thinking level for the newly selected model
|
||||
this.setThinkingLevel(this.thinkingLevel);
|
||||
|
||||
@@ -4768,6 +4830,217 @@ export class AgentSession {
|
||||
);
|
||||
}
|
||||
|
||||
#getRetryFallbackChains(): RetryFallbackChains {
|
||||
const configuredChains = this.settings.get("retry.fallbackChains");
|
||||
if (!configuredChains || typeof configuredChains !== "object") return {};
|
||||
return configuredChains as RetryFallbackChains;
|
||||
}
|
||||
|
||||
#validateRetryFallbackChains(): void {
|
||||
const configuredChains = this.settings.get("retry.fallbackChains");
|
||||
if (configuredChains === undefined) return;
|
||||
if (!configuredChains || typeof configuredChains !== "object" || Array.isArray(configuredChains)) {
|
||||
const msg = "retry.fallbackChains must be a mapping of role names to selector arrays.";
|
||||
logger.warn(msg);
|
||||
this.configWarnings.push(msg);
|
||||
return;
|
||||
}
|
||||
|
||||
for (const [role, chain] of Object.entries(configuredChains)) {
|
||||
if (!Array.isArray(chain)) {
|
||||
const msg = `Fallback chain for role '${role}' must be an array of selector strings.`;
|
||||
logger.warn(msg);
|
||||
this.configWarnings.push(msg);
|
||||
continue;
|
||||
}
|
||||
for (const selectorStr of chain) {
|
||||
if (typeof selectorStr !== "string") {
|
||||
const msg = `Fallback chain for role '${role}' contains a non-string selector.`;
|
||||
logger.warn(msg);
|
||||
this.configWarnings.push(msg);
|
||||
continue;
|
||||
}
|
||||
const parsed = parseRetryFallbackSelector(selectorStr);
|
||||
if (!parsed) {
|
||||
const msg = `Invalid fallback selector format in role '${role}': ${selectorStr}`;
|
||||
logger.warn(msg);
|
||||
this.configWarnings.push(msg);
|
||||
continue;
|
||||
}
|
||||
const exists = this.#modelRegistry.find(parsed.provider, parsed.id);
|
||||
if (!exists) {
|
||||
const msg = `Fallback chain for role '${role}' references unknown model: ${selectorStr}`;
|
||||
logger.warn(msg);
|
||||
this.configWarnings.push(msg);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#getRetryFallbackRevertPolicy(): RetryFallbackRevertPolicy {
|
||||
return this.settings.get("retry.fallbackRevertPolicy") === "never" ? "never" : "cooldown-expiry";
|
||||
}
|
||||
|
||||
#getRetryFallbackPrimarySelector(role: string): RetryFallbackSelector | undefined {
|
||||
const configuredSelector = this.settings.getModelRole(role);
|
||||
return configuredSelector ? parseRetryFallbackSelector(configuredSelector) : undefined;
|
||||
}
|
||||
|
||||
#clearActiveRetryFallback(): void {
|
||||
this.#activeRetryFallback = undefined;
|
||||
}
|
||||
|
||||
#isRetryFallbackSelectorSuppressed(selector: RetryFallbackSelector): boolean {
|
||||
return this.#modelRegistry.isSelectorSuppressed(selector.raw);
|
||||
}
|
||||
|
||||
#noteRetryFallbackCooldown(currentSelector: string, retryAfterMs: number | undefined, errorMessage: string): void {
|
||||
let cooldownMs = retryAfterMs;
|
||||
if (!cooldownMs || cooldownMs <= 0) {
|
||||
const reason = parseRateLimitReason(errorMessage);
|
||||
cooldownMs = reason === "UNKNOWN" ? 5 * 60 * 1000 : calculateRateLimitBackoffMs(reason);
|
||||
}
|
||||
this.#modelRegistry.suppressSelector(currentSelector, Date.now() + cooldownMs);
|
||||
}
|
||||
|
||||
#resolveRetryFallbackRole(currentSelector: string): string | undefined {
|
||||
const parsedCurrent = parseRetryFallbackSelector(currentSelector);
|
||||
if (!parsedCurrent) return undefined;
|
||||
const currentBaseSelector = formatRetryFallbackBaseSelector(parsedCurrent);
|
||||
for (const role of Object.keys(this.#getRetryFallbackChains())) {
|
||||
const primarySelector = this.#getRetryFallbackPrimarySelector(role);
|
||||
if (!primarySelector) continue;
|
||||
if (primarySelector.raw === currentSelector) return role;
|
||||
if (formatRetryFallbackBaseSelector(primarySelector) === currentBaseSelector) return role;
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
#getRetryFallbackEffectiveChain(role: string): RetryFallbackSelector[] {
|
||||
const primarySelector = this.#getRetryFallbackPrimarySelector(role);
|
||||
if (!primarySelector) return [];
|
||||
const chain = [primarySelector];
|
||||
const seen = new Set<string>([primarySelector.raw]);
|
||||
for (const selector of this.#getRetryFallbackChains()[role] ?? []) {
|
||||
const parsed = parseRetryFallbackSelector(selector);
|
||||
if (!parsed || seen.has(parsed.raw)) continue;
|
||||
seen.add(parsed.raw);
|
||||
chain.push(parsed);
|
||||
}
|
||||
return chain;
|
||||
}
|
||||
|
||||
#findRetryFallbackCandidates(role: string, currentSelector: string): RetryFallbackSelector[] {
|
||||
const chain = this.#getRetryFallbackEffectiveChain(role);
|
||||
if (chain.length <= 1) return [];
|
||||
const parsedCurrent = parseRetryFallbackSelector(currentSelector);
|
||||
const currentBaseSelector = parsedCurrent ? formatRetryFallbackBaseSelector(parsedCurrent) : undefined;
|
||||
const exactIndex = chain.findIndex(selector => selector.raw === currentSelector);
|
||||
if (exactIndex >= 0) return chain.slice(exactIndex + 1);
|
||||
const baseIndex = currentBaseSelector
|
||||
? chain.findIndex(selector => formatRetryFallbackBaseSelector(selector) === currentBaseSelector)
|
||||
: -1;
|
||||
if (baseIndex >= 0) return chain.slice(baseIndex + 1);
|
||||
return chain.slice(1);
|
||||
}
|
||||
|
||||
async #applyRetryFallbackCandidate(
|
||||
role: string,
|
||||
selector: RetryFallbackSelector,
|
||||
currentSelector: string,
|
||||
): Promise<void> {
|
||||
const candidate = this.#modelRegistry.find(selector.provider, selector.id);
|
||||
if (!candidate) {
|
||||
throw new Error(`Retry fallback model not found: ${selector.raw}`);
|
||||
}
|
||||
const apiKey = await this.#modelRegistry.getApiKey(candidate, this.sessionId);
|
||||
if (!apiKey) {
|
||||
throw new Error(`No API key for retry fallback ${selector.raw}`);
|
||||
}
|
||||
|
||||
const currentThinkingLevel = this.thinkingLevel;
|
||||
const nextThinkingLevel = selector.thinkingLevel ?? currentThinkingLevel;
|
||||
|
||||
this.#setModelWithProviderSessionReset(candidate);
|
||||
this.sessionManager.appendModelChange(`${candidate.provider}/${candidate.id}`, "temporary");
|
||||
this.settings.getStorage()?.recordModelUsage(`${candidate.provider}/${candidate.id}`);
|
||||
this.setThinkingLevel(nextThinkingLevel);
|
||||
if (!this.#activeRetryFallback) {
|
||||
this.#activeRetryFallback = {
|
||||
role,
|
||||
originalSelector: currentSelector,
|
||||
originalThinkingLevel: currentThinkingLevel,
|
||||
lastAppliedFallbackThinkingLevel: nextThinkingLevel,
|
||||
};
|
||||
} else {
|
||||
this.#activeRetryFallback.lastAppliedFallbackThinkingLevel = nextThinkingLevel;
|
||||
}
|
||||
await this.#emitSessionEvent({
|
||||
type: "retry_fallback_applied",
|
||||
from: currentSelector,
|
||||
to: selector.raw,
|
||||
role,
|
||||
});
|
||||
}
|
||||
|
||||
async #tryRetryModelFallback(currentSelector: string): Promise<boolean> {
|
||||
const role = this.#activeRetryFallback?.role ?? this.#resolveRetryFallbackRole(currentSelector);
|
||||
if (!role) return false;
|
||||
|
||||
for (const selector of this.#findRetryFallbackCandidates(role, currentSelector)) {
|
||||
if (this.#isRetryFallbackSelectorSuppressed(selector)) continue;
|
||||
const candidate = this.#modelRegistry.find(selector.provider, selector.id);
|
||||
if (!candidate) continue;
|
||||
const apiKey = await this.#modelRegistry.getApiKey(candidate, this.sessionId);
|
||||
if (!apiKey) continue;
|
||||
await this.#applyRetryFallbackCandidate(role, selector, currentSelector);
|
||||
return true;
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
async #maybeRestoreRetryFallbackPrimary(): Promise<void> {
|
||||
if (!this.#activeRetryFallback) return;
|
||||
if (this.#getRetryFallbackRevertPolicy() !== "cooldown-expiry") return;
|
||||
|
||||
const {
|
||||
originalSelector: originalSelectorRaw,
|
||||
originalThinkingLevel,
|
||||
lastAppliedFallbackThinkingLevel,
|
||||
} = this.#activeRetryFallback;
|
||||
const originalSelector = parseRetryFallbackSelector(originalSelectorRaw);
|
||||
if (!originalSelector) {
|
||||
this.#clearActiveRetryFallback();
|
||||
return;
|
||||
}
|
||||
|
||||
const currentModel = this.model;
|
||||
if (!currentModel) return;
|
||||
const currentSelector = formatRetryFallbackSelector(currentModel, this.thinkingLevel);
|
||||
if (currentSelector === originalSelector.raw) {
|
||||
if (!this.#isRetryFallbackSelectorSuppressed(originalSelector)) {
|
||||
this.#clearActiveRetryFallback();
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (this.#isRetryFallbackSelectorSuppressed(originalSelector)) return;
|
||||
|
||||
const primaryModel = this.#modelRegistry.find(originalSelector.provider, originalSelector.id);
|
||||
if (!primaryModel) return;
|
||||
const apiKey = await this.#modelRegistry.getApiKey(primaryModel, this.sessionId);
|
||||
if (!apiKey) return;
|
||||
|
||||
const currentThinkingLevel = this.thinkingLevel;
|
||||
const thinkingToApply =
|
||||
currentThinkingLevel === lastAppliedFallbackThinkingLevel ? originalThinkingLevel : currentThinkingLevel;
|
||||
this.#setModelWithProviderSessionReset(primaryModel);
|
||||
this.sessionManager.appendModelChange(`${primaryModel.provider}/${primaryModel.id}`, "temporary");
|
||||
this.settings.getStorage()?.recordModelUsage(`${primaryModel.provider}/${primaryModel.id}`);
|
||||
this.setThinkingLevel(thinkingToApply);
|
||||
this.#clearActiveRetryFallback();
|
||||
}
|
||||
|
||||
#parseRetryAfterMsFromError(errorMessage: string): number | undefined {
|
||||
const now = Date.now();
|
||||
const retryAfterMsMatch = /retry-after-ms\s*[:=]\s*(\d+)/i.exec(errorMessage);
|
||||
@@ -4847,12 +5120,13 @@ export class AgentSession {
|
||||
}
|
||||
|
||||
const errorMessage = message.errorMessage || "Unknown error";
|
||||
const parsedRetryAfterMs = this.#parseRetryAfterMsFromError(errorMessage);
|
||||
let delayMs = retrySettings.baseDelayMs * 2 ** (this.#retryAttempt - 1);
|
||||
let switchedCredential = false;
|
||||
let switchedModel = false;
|
||||
|
||||
if (this.model && isUsageLimitError(errorMessage)) {
|
||||
const retryAfterMs =
|
||||
this.#parseRetryAfterMsFromError(errorMessage) ??
|
||||
calculateRateLimitBackoffMs(parseRateLimitReason(errorMessage));
|
||||
const retryAfterMs = parsedRetryAfterMs ?? calculateRateLimitBackoffMs(parseRateLimitReason(errorMessage));
|
||||
const switched = await this.#modelRegistry.authStorage.markUsageLimitReached(
|
||||
this.model.provider,
|
||||
this.sessionId,
|
||||
@@ -4862,6 +5136,7 @@ export class AgentSession {
|
||||
},
|
||||
);
|
||||
if (switched) {
|
||||
switchedCredential = true;
|
||||
delayMs = 0;
|
||||
} else if (retryAfterMs > delayMs) {
|
||||
// No more accounts to switch to — wait out the backoff
|
||||
@@ -4869,6 +5144,17 @@ export class AgentSession {
|
||||
}
|
||||
}
|
||||
|
||||
const currentSelector = this.model ? formatRetryFallbackSelector(this.model, this.thinkingLevel) : undefined;
|
||||
if (!switchedCredential && currentSelector) {
|
||||
this.#noteRetryFallbackCooldown(currentSelector, parsedRetryAfterMs, errorMessage);
|
||||
switchedModel = await this.#tryRetryModelFallback(currentSelector);
|
||||
if (switchedModel) {
|
||||
delayMs = 0;
|
||||
} else if (parsedRetryAfterMs && parsedRetryAfterMs > delayMs) {
|
||||
delayMs = parsedRetryAfterMs;
|
||||
}
|
||||
}
|
||||
|
||||
await this.#emitSessionEvent({
|
||||
type: "auto_retry_start",
|
||||
attempt: this.#retryAttempt,
|
||||
|
||||
@@ -0,0 +1,389 @@
|
||||
import { afterEach, beforeEach, describe, expect, it } from "bun:test";
|
||||
import * as path from "node:path";
|
||||
import { Agent } from "@oh-my-pi/pi-agent-core";
|
||||
import { type AssistantMessage, Effort, getBundledModel, type Model } from "@oh-my-pi/pi-ai";
|
||||
import { AssistantMessageEventStream } from "@oh-my-pi/pi-ai/utils/event-stream";
|
||||
import { ModelRegistry } from "@oh-my-pi/pi-coding-agent/config/model-registry";
|
||||
import { Settings } from "@oh-my-pi/pi-coding-agent/config/settings";
|
||||
import { AgentSession, type AgentSessionEvent } from "@oh-my-pi/pi-coding-agent/session/agent-session";
|
||||
import { AuthStorage } from "@oh-my-pi/pi-coding-agent/session/auth-storage";
|
||||
import { SessionManager } from "@oh-my-pi/pi-coding-agent/session/session-manager";
|
||||
import { TempDir } from "@oh-my-pi/pi-utils";
|
||||
|
||||
class MockAssistantStream extends AssistantMessageEventStream {}
|
||||
|
||||
function createAssistantMessage(
|
||||
model: Model,
|
||||
options: { text?: string; stopReason: "stop" | "error"; errorMessage?: string },
|
||||
): AssistantMessage {
|
||||
return {
|
||||
role: "assistant",
|
||||
content: options.text ? [{ type: "text", text: options.text }] : [],
|
||||
api: model.api,
|
||||
provider: model.provider,
|
||||
model: model.id,
|
||||
usage: {
|
||||
input: 0,
|
||||
output: 0,
|
||||
cacheRead: 0,
|
||||
cacheWrite: 0,
|
||||
totalTokens: 0,
|
||||
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 },
|
||||
},
|
||||
stopReason: options.stopReason,
|
||||
errorMessage: options.errorMessage,
|
||||
timestamp: Date.now(),
|
||||
};
|
||||
}
|
||||
|
||||
async function _waitFor(predicate: () => boolean, timeoutMs = 1000): Promise<void> {
|
||||
const deadline = Date.now() + timeoutMs;
|
||||
while (Date.now() < deadline) {
|
||||
if (predicate()) return;
|
||||
await Bun.sleep(10);
|
||||
}
|
||||
throw new Error("Timed out waiting for condition");
|
||||
}
|
||||
|
||||
describe("AgentSession retry fallback", () => {
|
||||
let tempDir: TempDir;
|
||||
let authStorage: AuthStorage;
|
||||
let modelRegistry: ModelRegistry;
|
||||
let session: AgentSession | undefined;
|
||||
|
||||
beforeEach(async () => {
|
||||
tempDir = TempDir.createSync("@pi-retry-fallback-");
|
||||
authStorage = await AuthStorage.create(path.join(tempDir.path(), "testauth.db"));
|
||||
authStorage.setRuntimeApiKey("anthropic", "anthropic-test-key");
|
||||
authStorage.setRuntimeApiKey("openai", "openai-test-key");
|
||||
modelRegistry = new ModelRegistry(authStorage);
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
if (session) {
|
||||
await session.dispose();
|
||||
session = undefined;
|
||||
}
|
||||
authStorage.close();
|
||||
tempDir.removeSync();
|
||||
});
|
||||
|
||||
it("advances through a role-keyed fallback chain across retries", async () => {
|
||||
const primaryModel = getBundledModel("anthropic", "claude-sonnet-4-5");
|
||||
const firstFallback = getBundledModel("openai", "gpt-4o-mini");
|
||||
const secondFallback = getBundledModel("openai", "gpt-4o");
|
||||
if (!primaryModel || !firstFallback || !secondFallback) {
|
||||
throw new Error("Expected bundled test models to exist");
|
||||
}
|
||||
|
||||
const requestedModels: string[] = [];
|
||||
const retryStartEvents: Array<Extract<AgentSessionEvent, { type: "auto_retry_start" }>> = [];
|
||||
const retryEndEvents: Array<Extract<AgentSessionEvent, { type: "auto_retry_end" }>> = [];
|
||||
const fallbackAppliedEvents: Array<Extract<AgentSessionEvent, { type: "retry_fallback_applied" }>> = [];
|
||||
const fallbackSucceededEvents: Array<Extract<AgentSessionEvent, { type: "retry_fallback_succeeded" }>> = [];
|
||||
|
||||
const agent = new Agent({
|
||||
getApiKey: provider => `${provider}-test-key`,
|
||||
initialState: {
|
||||
model: primaryModel,
|
||||
systemPrompt: "Test",
|
||||
tools: [],
|
||||
messages: [],
|
||||
},
|
||||
streamFn: model => {
|
||||
requestedModels.push(`${model.provider}/${model.id}`);
|
||||
const stream = new MockAssistantStream();
|
||||
queueMicrotask(() => {
|
||||
if (model.provider === primaryModel.provider && model.id === primaryModel.id) {
|
||||
const message = createAssistantMessage(model, {
|
||||
stopReason: "error",
|
||||
errorMessage: "overloaded_error: provider returned error 503",
|
||||
});
|
||||
stream.push({ type: "start", partial: message });
|
||||
stream.push({ type: "error", reason: "error", error: message });
|
||||
return;
|
||||
}
|
||||
if (model.provider === firstFallback.provider && model.id === firstFallback.id) {
|
||||
const message = createAssistantMessage(model, {
|
||||
stopReason: "error",
|
||||
errorMessage: "service unavailable: 503 overloaded",
|
||||
});
|
||||
stream.push({ type: "start", partial: message });
|
||||
stream.push({ type: "error", reason: "error", error: message });
|
||||
return;
|
||||
}
|
||||
if (model.provider === secondFallback.provider && model.id === secondFallback.id) {
|
||||
const message = createAssistantMessage(model, {
|
||||
text: "Recovered on second fallback",
|
||||
stopReason: "stop",
|
||||
});
|
||||
stream.push({
|
||||
type: "start",
|
||||
partial: createAssistantMessage(model, { text: "", stopReason: "stop" }),
|
||||
});
|
||||
stream.push({ type: "done", reason: "stop", message });
|
||||
return;
|
||||
}
|
||||
throw new Error(`Unexpected model requested during retry fallback test: ${model.provider}/${model.id}`);
|
||||
});
|
||||
return stream;
|
||||
},
|
||||
});
|
||||
|
||||
const settings = Settings.isolated({
|
||||
"compaction.enabled": false,
|
||||
"retry.baseDelayMs": 5,
|
||||
"retry.fallbackChains": {
|
||||
default: [
|
||||
`${firstFallback.provider}/${firstFallback.id}`,
|
||||
`${secondFallback.provider}/${secondFallback.id}`,
|
||||
],
|
||||
},
|
||||
});
|
||||
settings.setModelRole("default", `${primaryModel.provider}/${primaryModel.id}`);
|
||||
|
||||
session = new AgentSession({
|
||||
agent,
|
||||
sessionManager: SessionManager.inMemory(),
|
||||
settings,
|
||||
modelRegistry,
|
||||
});
|
||||
|
||||
session.subscribe(event => {
|
||||
if (event.type === "auto_retry_start") {
|
||||
retryStartEvents.push(event);
|
||||
}
|
||||
if (event.type === "auto_retry_end") {
|
||||
retryEndEvents.push(event);
|
||||
}
|
||||
if (event.type === "retry_fallback_applied") {
|
||||
fallbackAppliedEvents.push(event);
|
||||
}
|
||||
if (event.type === "retry_fallback_succeeded") {
|
||||
fallbackSucceededEvents.push(event);
|
||||
}
|
||||
});
|
||||
|
||||
await session.prompt("Recover from rate limits");
|
||||
await session.waitForIdle();
|
||||
|
||||
expect(requestedModels).toEqual([
|
||||
`${primaryModel.provider}/${primaryModel.id}`,
|
||||
`${firstFallback.provider}/${firstFallback.id}`,
|
||||
`${secondFallback.provider}/${secondFallback.id}`,
|
||||
]);
|
||||
expect(session.model?.provider).toBe(secondFallback.provider);
|
||||
expect(session.model?.id).toBe(secondFallback.id);
|
||||
expect(retryStartEvents.map(event => event.delayMs)).toEqual([0, 0]);
|
||||
expect(fallbackAppliedEvents).toEqual([
|
||||
{
|
||||
type: "retry_fallback_applied",
|
||||
from: `${primaryModel.provider}/${primaryModel.id}`,
|
||||
to: `${firstFallback.provider}/${firstFallback.id}`,
|
||||
role: "default",
|
||||
},
|
||||
{
|
||||
type: "retry_fallback_applied",
|
||||
from: `${firstFallback.provider}/${firstFallback.id}`,
|
||||
to: `${secondFallback.provider}/${secondFallback.id}`,
|
||||
role: "default",
|
||||
},
|
||||
]);
|
||||
expect(retryEndEvents).toHaveLength(1);
|
||||
expect(retryEndEvents[0]).toMatchObject({ success: true, attempt: 2 });
|
||||
expect(fallbackSucceededEvents).toEqual([
|
||||
{
|
||||
type: "retry_fallback_succeeded",
|
||||
model: `${secondFallback.provider}/${secondFallback.id}`,
|
||||
role: "default",
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it("suppresses cooled selectors and lazily reverts to the role primary after cooldown expiry", async () => {
|
||||
const primaryModel = getBundledModel("anthropic", "claude-sonnet-4-5");
|
||||
const fallbackModel = getBundledModel("openai", "gpt-4o-mini");
|
||||
if (!primaryModel || !fallbackModel) {
|
||||
throw new Error("Expected bundled test models to exist");
|
||||
}
|
||||
|
||||
const requestedModels: string[] = [];
|
||||
let primaryAttempts = 0;
|
||||
|
||||
const agent = new Agent({
|
||||
getApiKey: provider => `${provider}-test-key`,
|
||||
initialState: {
|
||||
model: primaryModel,
|
||||
systemPrompt: "Test",
|
||||
tools: [],
|
||||
messages: [],
|
||||
},
|
||||
streamFn: model => {
|
||||
requestedModels.push(`${model.provider}/${model.id}`);
|
||||
const stream = new MockAssistantStream();
|
||||
queueMicrotask(() => {
|
||||
if (model.provider === primaryModel.provider && model.id === primaryModel.id && primaryAttempts === 0) {
|
||||
primaryAttempts += 1;
|
||||
const message = createAssistantMessage(model, {
|
||||
stopReason: "error",
|
||||
errorMessage: "rate limit exceeded retry-after-ms=200",
|
||||
});
|
||||
stream.push({ type: "start", partial: message });
|
||||
stream.push({ type: "error", reason: "error", error: message });
|
||||
return;
|
||||
}
|
||||
const message = createAssistantMessage(model, {
|
||||
text: `ok:${model.provider}/${model.id}`,
|
||||
stopReason: "stop",
|
||||
});
|
||||
stream.push({ type: "start", partial: createAssistantMessage(model, { text: "", stopReason: "stop" }) });
|
||||
stream.push({ type: "done", reason: "stop", message });
|
||||
});
|
||||
return stream;
|
||||
},
|
||||
});
|
||||
|
||||
const settings = Settings.isolated({
|
||||
"compaction.enabled": false,
|
||||
"retry.baseDelayMs": 5,
|
||||
"retry.fallbackChains": {
|
||||
default: [`${fallbackModel.provider}/${fallbackModel.id}`],
|
||||
},
|
||||
"retry.fallbackRevertPolicy": "cooldown-expiry",
|
||||
});
|
||||
settings.setModelRole("default", `${primaryModel.provider}/${primaryModel.id}`);
|
||||
|
||||
session = new AgentSession({
|
||||
agent,
|
||||
sessionManager: SessionManager.inMemory(),
|
||||
settings,
|
||||
modelRegistry,
|
||||
});
|
||||
|
||||
await session.prompt("First prompt triggers fallback");
|
||||
await session.waitForIdle();
|
||||
expect(requestedModels).toEqual([
|
||||
`${primaryModel.provider}/${primaryModel.id}`,
|
||||
`${fallbackModel.provider}/${fallbackModel.id}`,
|
||||
]);
|
||||
expect(session.model?.provider).toBe(fallbackModel.provider);
|
||||
expect(session.model?.id).toBe(fallbackModel.id);
|
||||
|
||||
await session.prompt("Immediate second prompt should stay on fallback");
|
||||
await session.waitForIdle();
|
||||
expect(requestedModels).toEqual([
|
||||
`${primaryModel.provider}/${primaryModel.id}`,
|
||||
`${fallbackModel.provider}/${fallbackModel.id}`,
|
||||
`${fallbackModel.provider}/${fallbackModel.id}`,
|
||||
]);
|
||||
expect(session.model?.provider).toBe(fallbackModel.provider);
|
||||
expect(session.model?.id).toBe(fallbackModel.id);
|
||||
|
||||
await Bun.sleep(240);
|
||||
await session.prompt("Third prompt should lazily revert to primary");
|
||||
await session.waitForIdle();
|
||||
expect(requestedModels).toEqual([
|
||||
`${primaryModel.provider}/${primaryModel.id}`,
|
||||
`${fallbackModel.provider}/${fallbackModel.id}`,
|
||||
`${fallbackModel.provider}/${fallbackModel.id}`,
|
||||
`${primaryModel.provider}/${primaryModel.id}`,
|
||||
]);
|
||||
expect(session.model?.provider).toBe(primaryModel.provider);
|
||||
expect(session.model?.id).toBe(primaryModel.id);
|
||||
});
|
||||
|
||||
it("preserves thinking on bare fallback selectors and does not overwrite user thinking on restore", async () => {
|
||||
const primaryModel = getBundledModel("anthropic", "claude-sonnet-4-5");
|
||||
const fallbackModel = getBundledModel("openai", "gpt-4o-mini");
|
||||
if (!primaryModel || !fallbackModel) {
|
||||
throw new Error("Expected bundled test models to exist");
|
||||
}
|
||||
|
||||
const requestedModels: string[] = [];
|
||||
let primaryAttempts = 0;
|
||||
|
||||
const agent = new Agent({
|
||||
getApiKey: provider => `${provider}-test-key`,
|
||||
initialState: {
|
||||
model: primaryModel,
|
||||
systemPrompt: "Test",
|
||||
tools: [],
|
||||
messages: [],
|
||||
},
|
||||
streamFn: model => {
|
||||
requestedModels.push(`${model.provider}/${model.id}`);
|
||||
const stream = new MockAssistantStream();
|
||||
queueMicrotask(() => {
|
||||
if (model.provider === primaryModel.provider && model.id === primaryModel.id && primaryAttempts === 0) {
|
||||
primaryAttempts += 1;
|
||||
const message = createAssistantMessage(model, {
|
||||
stopReason: "error",
|
||||
errorMessage: "rate limit exceeded retry-after-ms=200",
|
||||
});
|
||||
stream.push({ type: "start", partial: message });
|
||||
stream.push({ type: "error", reason: "error", error: message });
|
||||
return;
|
||||
}
|
||||
const message = createAssistantMessage(model, {
|
||||
text: `ok:${model.provider}/${model.id}`,
|
||||
stopReason: "stop",
|
||||
});
|
||||
stream.push({ type: "start", partial: createAssistantMessage(model, { text: "", stopReason: "stop" }) });
|
||||
stream.push({ type: "done", reason: "stop", message });
|
||||
});
|
||||
return stream;
|
||||
},
|
||||
});
|
||||
|
||||
const settings = Settings.isolated({
|
||||
"compaction.enabled": false,
|
||||
"retry.baseDelayMs": 5,
|
||||
"retry.fallbackChains": {
|
||||
default: [`${fallbackModel.provider}/${fallbackModel.id}`],
|
||||
},
|
||||
"retry.fallbackRevertPolicy": "cooldown-expiry",
|
||||
});
|
||||
settings.setModelRole("default", `${primaryModel.provider}/${primaryModel.id}:high`);
|
||||
|
||||
session = new AgentSession({
|
||||
agent,
|
||||
sessionManager: SessionManager.inMemory(),
|
||||
settings,
|
||||
modelRegistry,
|
||||
thinkingLevel: Effort.High,
|
||||
});
|
||||
|
||||
await session.prompt("First prompt triggers bare-selector fallback");
|
||||
await session.waitForIdle();
|
||||
expect(requestedModels).toEqual([
|
||||
`${primaryModel.provider}/${primaryModel.id}`,
|
||||
`${fallbackModel.provider}/${fallbackModel.id}`,
|
||||
]);
|
||||
expect(session.model?.provider).toBe(fallbackModel.provider);
|
||||
expect(session.model?.id).toBe(fallbackModel.id);
|
||||
expect(session.thinkingLevel).toBeUndefined();
|
||||
|
||||
session.setThinkingLevel(Effort.Low);
|
||||
await Bun.sleep(240);
|
||||
await session.prompt("Second prompt should restore model but preserve user thinking change");
|
||||
await session.waitForIdle();
|
||||
expect(requestedModels).toEqual([
|
||||
`${primaryModel.provider}/${primaryModel.id}`,
|
||||
`${fallbackModel.provider}/${fallbackModel.id}`,
|
||||
`${primaryModel.provider}/${primaryModel.id}`,
|
||||
]);
|
||||
expect(session.model?.provider).toBe(primaryModel.provider);
|
||||
expect(session.model?.id).toBe(primaryModel.id);
|
||||
expect(session.thinkingLevel).toBeUndefined();
|
||||
});
|
||||
|
||||
it("normalizes suppression by base selector and clears it on model refresh", async () => {
|
||||
const future = Date.now() + 60_000;
|
||||
modelRegistry.suppressSelector("openai/gpt-4o:high", future);
|
||||
expect(modelRegistry.isSelectorSuppressed("openai/gpt-4o")).toBe(true);
|
||||
expect(modelRegistry.isSelectorSuppressed("openai/gpt-4o:low")).toBe(true);
|
||||
|
||||
await modelRegistry.refresh("offline");
|
||||
expect(modelRegistry.isSelectorSuppressed("openai/gpt-4o")).toBe(false);
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user