Commit Graph

49 Commits

Author SHA1 Message Date
can1357 e9888367d1 refactor: migrated packages to internal utility modules and removed external dependencies
- Implemented in-house, zero-dependency utility modules in `pi-utils` covering DOM manipulation, markdown parsing, templating, browser automation helpers, and terminal buffers.
- Migrated packages across the repository to consume the new internal utilities and `omptype` schema validators instead of external dependencies.
- Removed multiple external runtime and development dependencies including Zod, Marked, LRU cache, Turndown, and Puppeteer browser packages.
2026-08-05 13:39:09 +02:00
usr-bin-roygbiv 47ac33aa1f fix(ai): inspect snake-case schema children
(cherry picked from commit 41590a873ca838f9ac3b54e1008fd7c97424c3d3)
2026-07-30 01:27:35 +02:00
usr-bin-roygbiv 6dcd28c302 fix(ai): preserve negated enum semantics
(cherry picked from commit c96aa0d61445de12597abf76c397ce5f9707bcb8)
2026-07-30 01:27:35 +02:00
usr-bin-roygbiv b8c5f1544e style(ai): format schema traversal fix
(cherry picked from commit ffa704ed0abdd6f27d0ce56d8c97ccd64d025010)
2026-07-30 01:27:35 +02:00
usr-bin-roygbiv 2ee5a10923 fix(ai): preserve CCA literal payloads
(cherry picked from commit b026a96326e3d7df89771f3224476dc37eeeaa9c)
2026-07-30 01:27:35 +02:00
usr-bin-roygbiv 05d5b8c12d style(ai): format schema normalization
(cherry picked from commit aee3d7e5e26c3a5cb3e950266a2561ece4b1d20c)
2026-07-30 01:27:35 +02:00
usr-bin-roygbiv 34069920a6 fix(ai): normalize map value schemas
(cherry picked from commit 836951121076394b775771b29070f1022d7591c7)
2026-07-30 01:27:34 +02:00
usr-bin-roygbiv 2f579e1ac1 fix(ai): preserve literal schema payloads
(cherry picked from commit 3631bc5bd0a27eabb774bf800e0144523d25a55b)
2026-07-30 01:27:34 +02:00
usr-bin-roygbiv d17ae32b1d fix(ai): preserve enum keys in Google schema defaults
(cherry picked from commit 226f00efb3c330f5ed6b0583c4b65e2fa75b9f5b)
2026-07-30 01:27:34 +02:00
Roy 16e03728f0 fix(ai): omit unsupported Google enum values
(cherry picked from commit 6cc7916f9ff508ee0864229fe83a3c31a585f832)
2026-07-30 01:27:34 +02:00
can1357 18a24f8c45 style: formatted merge resolutions with biome 2026-07-18 19:59:20 +02:00
can1357 91bf49f897 fix(ai): traverse object-valued additionalProperties in grammar sanitizer
Object-form additionalProperties/unevaluatedProperties are genuine
subschemas whose interior may still hold the bare booleans emitted by
toolWireSchema; skipping them let llama.cpp reach an Unrecognized
schema: true. Boolean forms stay intact for closed/open-object grammar
semantics. Flagged by Codex review on #5919.
2026-07-18 19:58:23 +02:00
can1357 c2b241401b Merge PR #5919: fix(providers): sanitize boolean subschemas for grammar backends (@roboomp)
# Conflicts:
#	packages/catalog/src/compat/openai.ts
#	packages/catalog/src/types.ts
2026-07-18 19:58:22 +02:00
roboomp d4a9d50a5b fix(ai): normalized boolean schemas for moonshot
Coerced boolean subschemas into MFJS-compatible object forms while preserving boolean keyword values.

Fixes #5952
2026-07-18 03:52:16 +00:00
roboomp 2eb35e300f fix(providers): sanitize boolean subschemas for grammar backends
Local grammar-constrained OpenAI-compatible servers (llama.cpp, LM Studio,
vLLM) build a GBNF grammar from tool JSON Schemas and 400 with
"Unrecognized schema: true" on a bare boolean subschema. The task tool's
open outputSchema field normalizes to boolean `true` (issue #1179), which
the converter cannot compile, breaking every tool request.

Add a "grammar" tool-schema flavor, auto-detected for local backends, that
widens bare boolean/empty subschemas into a value-accepting primitive union
while preserving closed-object `additionalProperties: false`.

Fixes #5914
2026-07-17 22:35:44 +00:00
can1357 8932acb6f3 fix(schema): scope boolean coercion to google transports 2026-07-16 03:31:58 +02:00
roboomp c6c063ee00 fix(schema): rejected not schemas on cca transport
Added not to the CCA residual incompatibility gate so false boolean subschemas fall back before reaching the legacy object-shaped parameters wire.

Made residual scanning schema-map aware to avoid treating a property literally named not as the unsupported keyword, and added root, property, and dependentSchemas regressions.

Fixes #5604
2026-07-15 19:34:49 +00:00
roboomp da3e500637 fix(schema): handled boolean schema-map entries
Generalized the schema-map walk context beyond properties to include patternProperties, dependencies, dependentSchemas, $defs, and definitions. Each arbitrary map entry is now treated as a subschema, so bare booleans coerce before reaching Google/CCA.

Reused the shared map/array keyword tables in the Ollama sanitizer and added a dependentSchemas regression for both Google and CCA normalizers.

Fixes #5604
2026-07-15 19:27:00 +00:00
roboomp 508dbbbc5a fix(schema): coerce boolean subschemas for google/cca transport
Boolean JSON Schema subschemas (`true`/`false`, draft 6+) in MCP tool
inputs passed through normalizeSchemaForGoogle/normalizeSchemaForCCA
untouched. The Cloud Code Assist / Gemini protobuf Schema type has no
representation for a bare boolean, so requests bounced with a 400
INVALID_ARGUMENT before reaching the model.

Coerce booleans to their object equivalents (`true` -> `{}`,
`false` -> `{ not: {} }`) at the single normalizeSchemaNode choke point,
but only in genuine subschema slots (root, combiner branches,
subschema-valued keywords, property values). Keyword-slot booleans
(`nullable`, `enum` entries, `additionalProperties`) stay untouched so
Moonshot/MCP open-record markers survive.

Fixes #5604
2026-07-15 19:15:07 +00:00
lycaon 3f52e26a73 fix(ai): preserve CCA schemas with annotation conflicts 2026-07-11 00:10:47 -06:00
roboomp d4f331a450 fix(ai): kept ollama type unions conjunctive
- Wrapped generated multi-type branches in allOf so existing sibling anyOf constraints remain conjunctive.

- Added a regression case for schemas that combine type arrays with their own anyOf.

Refs #4488
2026-07-04 04:49:01 +00:00
roboomp d4545bc59f fix(ai): preserved multi-type ollama tool unions
- Kept nullable single-type arrays flattened for Ollama while expanding true multi-type arrays into anyOf alternatives.

- Covered non-null string/number and object/array unions in the Ollama provider payload regression test.

Refs #4488
2026-07-04 04:41:51 +00:00
roboomp df45f85885 fix(ai): widened open ollama subschemas instead of collapsing to {}
- Replaced boolean true / empty subschemas with an anyOf union over every primitive JSON type so grammar-constrained samplers (llama.cpp) keep advertising "any JSON value" for unconstrained fields.

- Dropped the WeakMap identity cache; the sanitizer no longer relies on module-external state.

- Updated the provider regression test and changelog entry to the widened shape.

Refs #4488
2026-07-04 04:31:14 +00:00
roboomp 676fefbfce fix(ai): sanitized ollama tool schemas
- Added an Ollama-specific tool schema sanitizer for boolean subschemas, boolean additionalProperties/unevaluatedProperties, and nullable type arrays.

- Applied the sanitizer in the native Ollama chat tool serializer and covered the provider payload contract with a regression test.

Fixes #4488
2026-07-04 04:19:03 +00:00
can1357 c3f7e849e5 refactor: centralized AI error handling into a dedicated module
- Migrated 288 lines of scattered error classification logic from `utils/error-id.ts` into a cohesive `packages/ai/src/error/` module with 13 specialized submodules covering flags, classes, OAuth, providers, rate-limiting, and finalization.
- Replaced 100+ generic `Error` throws across 60+ provider and registry files with semantic `AIError.*` classes (e.g., `AIError.MissingApiKeyError`, `AIError.OAuthError`, `AIError.ProviderResponseError`), improving error diagnostics and retry logic.
- Consolidated error utility imports from `pi-utils` and scattered classification functions into a single `AIError` namespace, reducing coupling and simplifying error handling across all packages.
2026-06-27 10:44:13 +02:00
can1357 291b3c74c2 feat: enhanced model reasoning, schema normalization, and loop guarding
- Integrated comprehensive loop guard support for DeepSeek and assistant prose patterns, including configurable stream checks.
- Implemented Moonshot Flavored JSON Schema (MFJS) normalization for improved tool compatibility and enum type inference.
- Added support for Ollama reasoning effort backfilling and Grok-specific service tier cost tracking across providers.
- Expanded model catalog with new entries and unified compatibility logic for improved OpenRouter API integration.
2026-06-18 04:51:43 +02:00
can1357 7c295faa52 feat: added native support for ArkType schemas alongside Zod
- Added native support for parsing, normalized validation, and serialization of ArkType schemas throughout the agent pipeline.
- Implemented pruning of unconstrained union branches and normalization helpers to handle unrepresentable strict-mode branches.
- Patched ArkType's schema package to preserve declared object key order during serialization.
- Integrated ArkType schemas into `agent-loop` and migrated coding agent tool parameters to ArkType format.
2026-06-18 00:59:56 +02:00
roboomp d35d1c1883 fix(ai): loosened openai pattern property fallback
Rewrote unsupported lookaround patternProperties keys to a supported catch-all pattern instead of dropping their value schemas, preserving dynamic-key tool arguments when additionalProperties is false.

Extended sanitizer and Codex conversion regression coverage for the closed dynamic-key case.

Fixes #2784
2026-06-17 00:23:37 +02:00
roboomp e4444a0fe7 fix(ai): normalized stripped openai schemas
Converted schema nodes emptied by OpenAI Responses lookaround stripping to boolean true so pattern-only nodes keep the existing empty-schema semantics.

Extended sanitizer and Codex regression coverage for pattern-only property and propertyNames schemas.

Fixes #2784
2026-06-16 15:54:50 +00:00
roboomp 965afbe377 fix(ai): stripped openai pattern property lookarounds
Dropped unsupported lookaround regex keys from OpenAI Responses patternProperties maps while preserving ordinary property names and supported regex keys.

Extended sanitizer and Codex tool conversion coverage for patternProperties.

Fixes #2784
2026-06-16 15:42:32 +00:00
roboomp a93a6c3908 fix(ai): stripped openai responses regex lookarounds
Removed JSON Schema pattern values containing regex lookaround from OpenAI Responses/Codex tool schemas so incompatible MCP tools do not poison the request.

Added schema-normalization and Codex conversion regression coverage for Figma-style fileKey patterns.

Fixes #2784
2026-06-16 15:24:54 +00:00
can1357 c341fe99a8 Merge PR #1837: fix(ai): preserve nullable enum tool schemas 2026-06-15 19:46:10 +02:00
can1357 e40c28146b fix(ai): reduced strict-mode retries by disabling strict tools after schema errors
- Tracked whether strict tool schemas were actually applied and used that flag when deciding strict-to-nonstrict retries.
- Recorded strict-tool failures in session state so later requests skip strict mode retries and avoid extra error rounds.
- Hardened strict schema normalization by flattening nested pure anyOf unions and expanded schema error matching for additional invalid-schema rejections.
2026-06-10 22:52:59 +02:00
roboomp 4428942b84 fix(ai): preserved constrained nullable unions
Only flatten optional anyOf schemas when the union wrapper has no sibling constraints that would exclude null.

Added a strict-schema regression for constrained anyOf properties so null remains an outer branch.
2026-06-10 20:18:39 +00:00
roboomp 4ccc9d7e88 fix(ai): flattened nullable strict unions
Flattened optional union tool schemas during strict enforcement so OpenRouter DeepSeek V4 no longer receives nested anyOf branches without a type.

Added schema-level and OpenRouter DeepSeek payload regressions for optional string-or-array tool parameters.

Fixes #2270
2026-06-10 20:13:54 +00:00
can1357 8a66c8786b fix(ai): surfaced Gemini stream errors and fixed Bedrock/AWS credential handling
in-band Gemini errors, promptFeedback blocks, and missing finishReason no longer report success; toolUse override stops masking SAFETY/MALFORMED finishes; schema normalization keeps DAG-shared subtrees while detecting true cycles; Google/AWS shared credential resolution detached from first caller's signal and bounded by own timeout; Bedrock keeps toolConfig under toolChoice none; eventstream cancels body on abnormal exit.
2026-06-10 01:26:31 +02:00
basedcorp99 f8ef2cf8e4 fix: format CCA schema normalization 2026-06-06 20:42:20 +02:00
basedcorp99 7c8fb4d8f6 fix(ai): also strip sibling type-specific keys during CCA mixed-type collapse
Address review feedback:
- Replace `as string` assertion with typed `chosenType` local
- Strip sibling keys from nextSchema that were copied via
  copySchemaWithout but belong to a type other than the chosen one
  (e.g. sibling `items` on a now-string-typed schema)
- Export ALL_CCA_TYPE_SPECIFIC_KEYS from fields.ts for sibling filtering
- Add regression test for the sibling-key edge case
2026-06-06 20:42:20 +02:00
basedcorp99 f3210ab862 fix(ai): strip type-specific keys when CCA mixed-type collapse picks non-matching type
When collapseMixedTypeCombinerVariants collapses an anyOf with mixed
types (e.g. string | array), it previously picked the first non-null
type but indiscriminately copied ALL mergedVariantFields — including
type-specific keys like "items" that only belong to array. This
produced schemas like {type: "string", items: {...}} which Google
Cloud Code Assist API rejects with 400.

Fix: filter mergedVariantFields against the chosen types allowed keys
(CLOUD_CODE_ASSIST_TYPE_SPECIFIC_KEYS) before copying, so array-only
keys are dropped when the winner is string (and vice versa).

Fixes 400 error on github tools "pr" parameter (anyOf string/array).
2026-06-06 20:42:20 +02:00
roboomp 89a47c8567 fix(ai): preserved nullable enum tool schemas
Distributed enum constraints across strict-mode type-array branches and kept null in collapsed nullable scalar enum wire schemas. Added regression coverage for the OpenAI Responses MCP nullable enum path.\n\nFixes #1835
2026-06-04 10:09:43 +00:00
can1357 debd90656c fix(ai): fixed CCA schema stripping for properties named properties
- Updated `normalizeSchemaNode` to keep `insideProperties` active only when entering an outer `properties` map, not when the key appears within an existing properties object.
- Added a regression test for `google-tool-schema` ensuring `propertyNames` and `additionalProperties` are removed from a schema property literally named `properties`.
2026-06-04 02:24:41 +02:00
roboomp ff9e007e1e fix(schema): apply empty-schema normalization to all providers, not just openai
Move {}→true normalization out of zodToWireSchema/postProcess (Zod-only)
into a dedicated, exported normalizeEmptySchemas helper that toolWireSchema
calls for both the Zod and TypeBox/raw-JSON-Schema branches. Every provider
that calls toolWireSchema (OpenAI, Anthropic, Google, Ollama, Bedrock,
Cursor) now receives the normalized schema regardless of how the tool was
authored.

Verified strict-mode opt-out across providers:
- OpenAI: hasUnrepresentableStrictObjectMap hits === true branch (was
  isJsonObject({}) branch), same result -> strict: false.
- Anthropic: normalizeAnthropicStrictSchemaNode opts out via
  additionalProperties !== false (still true for true) -> strict: false.
- Google: normalizeSchemaForGoogle strips additionalProperties regardless
  (UNSUPPORTED_SCHEMA_FIELDS, pre-existing behavior).

The normalizeOpenAIResponsesSchemaNode guard stays as a safety net for
callers that invoke sanitizeSchemaForOpenAIResponses directly on schemas
that bypass the wire-schema pipeline (e.g. fixtures, debug paths).

Adds cross-provider tests covering both Zod and TypeBox inputs and verifying
downstream provider behavior with the normalized form.

Fixes #1179
2026-05-19 04:58:19 +00:00
roboomp 09ea361331 fix(schema): normalize empty JSON schema {} to true for grammar-constrained models
z.unknown() emits {} (empty JSON Schema) as the value for additionalProperties,
items, and other schema-valued positions. This is semantically equivalent to
boolean true per JSON Schema draft 2020-12 §4.3.1, but grammar-constrained
samplers (llama.cpp, etc.) interpret the object form as "generate an empty
object" rather than "any JSON value". Models using such samplers therefore
emit {} for every open-typed field, including extra.title in plan-mode resolve
calls (issue #1179). This affected all tools with open schemas, not just the
resolve tool -- MCP tools and any tool using z.unknown() in value positions
have the same problem.

Fix: normalize any {} in schema-valued positions to true in two places:
- zodToWireSchema postProcess/walk: covers Zod-based tools across all providers
- normalizeOpenAIResponsesSchemaNode: covers all tools (Zod, TypeBox, MCP)
  going to openai-responses-compatible endpoints (llama.cpp, Azure Responses)

Adds isJsonObjectEmpty() helper to schema/types.ts (for...in based, no allocation).

Fixes #1179
2026-05-19 04:39:27 +00:00
Can Bölük 4024fbe7e2 fix(ai): decontaminate leaked Zod schema instances to valid JSON Schema
Rewrites JSON-roundtripped Zod 4 schema objects that leak Zod internals as
JSON Schema keywords (e.g., `type:"enum"`, `enum:{...}`) into valid
JSON Schema 2020-12. This prevents validation failures when such schemas
are used as tool input schemas (e.g., from MCP servers).

Updates `isZodSchema` to reject deserialized Zod impostors that retain
`_zod` but lose their prototype.

Wires `toJSON` methods onto TypeBox shim schemas to ensure `JSON.stringify`
produces clean JSON Schema, preventing future leaks.

Fixes #1101
2026-05-19 10:16:40 +09:00
roboomp 74461e6f53 fix(ai): broadened openai schema sanitization coverage
Added dependencies and contentSchema to the OpenAI Responses schema-position sets so draft-04..07 dependencies maps and draft 2019-09 contentSchema nodes also get the properties-on-object normalization. Recognized array-form type declarations that include object. Preserved malformed non-array oneOf payloads instead of dropping them. Documented the cycle-safe cache seed and added a self-referential regression test.

Refs #1147
2026-05-17 10:55:29 +00:00
roboomp 03623bdf5e fix(ai): normalized openai object tool schemas
Normalized OpenAI Responses schema sanitization so object schema nodes always include properties while only traversing schema-valued positions. Wired the same sanitizer into the OpenAI Codex Responses tool conversion path and added regression coverage for no-argument MCP-style schemas and literal payload preservation.

Fixes #1147
2026-05-17 09:28:59 +00:00
can1357 54a60a7702 fix(ai/schema): hoisted description to anyOf wrapper in strict-mode unions
- Extracted `description` from type-array and nullable branches so it lives on the wrapper, not duplicated onto each variant.
- Replaced inline enum-type inference with `inferStrictPrimitiveTypeFromEnumOrConst`, covering both `enum` and `const` in sanitize and enforce paths.
- Mixed-primitive enums and non-primitive consts now fall back to non-strict instead of producing a typeless schema that OpenAI rejects on the wire.
2026-05-16 20:22:39 +02:00
can1357 5c931ac135 refactor(ai/schema): merged strict-mode into normalize, removed strict-mode.ts
- Moved sanitizeSchemaForStrictMode, enforceStrictSchema, and tryEnforceStrictSchema into normalize.ts.
- Moved sanitizeSchemaForOpenAIResponses/rewriteOneOfToAnyOf into normalize.ts alongside other normalizers.
- Removed strict-mode.ts and its public re-export; adapt.ts now only exposes NO_STRICT and adaptSchemaForStrict.
- Dropped StringEnum helper from strict-mode.ts (already removed from public API).
2026-05-16 19:41:18 +02:00
can1357 e7200c2e4a feat(ai): added unified normalize flow for Google/CCA schema handling
- Implemented a unified normalization flow by switching Google/CCA handling to normalizeSchemaForGoogle/CCA.
- Added normalize.ts with recursive node normalization, nullable-union checks, and combiner collapsing.
- Removed sanitize-google.ts and normalize-cca.ts, replacing them with normalize exports in schema indexes.
- Added spill-to-description utilities with spill/paren modes and `$defs` exclusion for unsupported fields.
- Updated MCP bridge and schema tests to use normalizeSchemaFor* APIs with expanded compatibility checks.
- Documented normalization behavior changes and breaking rename in constraints and package changelog files.
2026-05-16 19:04:09 +02:00