fix(catalog): replayReasoningContent must not gate on spec.reasoning

The discovery paths for llama.cpp, LM Studio, and openai-models-list
hardcode reasoning: false because the upstream /models endpoints don't
advertise the capability. The original gate

    Boolean(spec.reasoning) && (LOCAL_PROVIDER || loopback)

therefore left replayReasoningContent off for the common setup the bug
targets — a discovered Qwen / DeepSeek model on local llama.cpp — and
the stream parser still recorded the upstream's reasoning_content
deltas as thinking blocks, so #3528 reproduced unchanged.

Drop the spec.reasoning gate. The encoder's own
'if (nonEmptyThinkingBlocks.length > 0)' guard ensures the flag stays a
no-op for pure-text turns, so always-on for local hosts adds nothing to
non-reasoning histories. Flip the matching test and add an encoder pin
that mirrors the discovery setup (reasoning: false + actual thinking
block must still ride as reasoning_content).

Addresses chatgpt-codex review on #3532.
This commit is contained in:
roboomp
2026-06-26 06:43:23 +00:00
parent e59466cca5
commit b6d81f22e1
3 changed files with 45 additions and 6 deletions
+33 -3
View File
@@ -145,10 +145,17 @@ describe("llama.cpp warm-prefix preservation (#3528)", () => {
expect(optedIn.replayReasoningContent).toBe(true);
});
it("leaves replayReasoningContent off for non-reasoning local models", () => {
// The flag only matters when thinking blocks could exist on prior turns.
it("still auto-enables replayReasoningContent when spec.reasoning is false on local hosts", () => {
// Runtime discovery for llama.cpp / lm-studio / openai-models-list
// hardcodes `reasoning: false` because the upstream `/models` endpoints
// don't advertise the capability — but the stream parser still records
// any incoming `reasoning_content` deltas as thinking blocks. Gating
// the flag on `spec.reasoning` would leave every discovered local Qwen
// model reproducing #3528. The encoder's own
// `nonEmptyThinkingBlocks.length > 0` guard makes the flag a no-op on
// pure-text histories, so it's safe to enable unconditionally.
const compat = llamaCppQwenModel({ reasoning: false }).compat;
expect(compat.replayReasoningContent).toBe(false);
expect(compat.replayReasoningContent).toBe(true);
});
it("leaves replayReasoningContent off for cloud OpenAI-compatible providers", () => {
@@ -195,6 +202,29 @@ describe("llama.cpp warm-prefix preservation (#3528)", () => {
expect(assistant.reasoning_content).toBe("Let me review the unpushed changes comprehensively.");
});
it("replays reasoning_content for discovered local models that omit spec.reasoning", () => {
// Mirrors the discovery setup: `discoverLlamaCppModels` builds specs
// with `reasoning: false` regardless of the actual model behaviour.
// When the model emits reasoning at runtime the stream parser still
// records it as a thinking block; the encoder MUST replay it as
// `reasoning_content` so llama.cpp's KV cache survives the next turn.
const target = llamaCppQwenModel({ reasoning: false });
const messages: Message[] = [
userMessage("Plan a refactor."),
assistantWithReasoning(
"Trace the call graph through service.ts and the registry.",
"Step 1: extract the loader. Step 2: rewire the factory.",
),
userMessage("Continue."),
];
const wire = convertMessages(target, { messages }, target.compat);
const assistant = findAssistantMessage(wire);
expect(assistant).toBeDefined();
if (!assistant) throw new Error("assistant message missing");
expect(assistant.reasoning_content).toBe("Trace the call graph through service.ts and the registry.");
});
it("honors the streamed signature when it identifies a recognized wire field", () => {
// Some llama.cpp builds stream reasoning under `reasoning` rather than
// `reasoning_content`. Round-trip into the same field so the chat
+1 -1
View File
@@ -4,7 +4,7 @@
### Added
- Added `OpenAICompat.replayReasoningContent` — auto-enabled for the built-in local OpenAI-compatible providers (`llama.cpp`, `lm-studio`, `vllm`, `ollama` on `openai-completions`) and for any provider pointed at a loopback / RFC1918 / `*.local` baseUrl when the model is reasoning-capable. Built-in proxy providers (currently `litellm`) are excluded from both checks because they forward to an unrelated upstream that gains no KV-cache benefit and may 400 on the extra field; users running a custom proxy in front of a llama.cpp-style backend can opt in via the sparse `compat.replayReasoningContent: true` override. Signals to the `openai-completions` encoder that preserved `thinking` blocks must be re-emitted as `reasoning_content` on every assistant turn so chat templates that reconstruct `<think>…</think>` from the field (Qwen3, DeepSeek-R1, GLM-5.x) keep the prior turn's tokens byte-stable and llama.cpp's prefix KV cache survives. ([#3528](https://github.com/can1357/oh-my-pi/issues/3528))
- Added `OpenAICompat.replayReasoningContent` — auto-enabled for the built-in local OpenAI-compatible providers (`llama.cpp`, `lm-studio`, `vllm`, `ollama` on `openai-completions`) and for any provider pointed at a loopback / RFC1918 / `*.local` baseUrl. NOT gated on `spec.reasoning`: the runtime discovery paths for `llama.cpp` / `lm-studio` / `openai-models-list` hardcode `reasoning: false` because the upstream `/models` endpoints don't advertise the capability, while the stream parser still records incoming `reasoning_content` deltas as thinking blocks — gating on the spec flag would leave every discovered local Qwen / DeepSeek model re-triggering #3528. The encoder only writes `reasoning_content` when a thinking block actually exists on the turn, so the flag is a no-op on pure-text histories. Built-in proxy providers (currently `litellm`) are excluded from both checks because they forward to an unrelated upstream that gains no KV-cache benefit and may 400 on the extra field; users running a custom proxy in front of a llama.cpp-style backend can opt in via the sparse `compat.replayReasoningContent: true` override. Signals to the `openai-completions` encoder that preserved `thinking` blocks must be re-emitted as `reasoning_content` on every assistant turn so chat templates that reconstruct `<think>…</think>` from the field (Qwen3, DeepSeek-R1, GLM-5.x) keep the prior turn's tokens byte-stable and llama.cpp's prefix KV cache survives. ([#3528](https://github.com/can1357/oh-my-pi/issues/3528))
## [16.1.20] - 2026-06-25
+11 -2
View File
@@ -454,9 +454,18 @@ export function buildOpenAICompat(spec: ModelSpec<"openai-completions">): Resolv
// content and forces full prompt re-processing (#3528). The
// `requires*ReasoningContent*` flags above stay off for these hosts —
// they accept but don't validate the field — so the encoder needs a
// distinct opt-in to replay on every reasoning turn.
// distinct opt-in to replay on every reasoning turn. NOT gated on
// `spec.reasoning`: the runtime discovery paths for `llama.cpp` /
// `lm-studio` / `openai-models-list` hardcode `reasoning: false`
// because the upstream `/models` endpoints don't advertise the
// capability, but the OpenAI stream parser still records incoming
// `reasoning_content` deltas as thinking blocks. Gating on the spec
// flag would leave every discovered local Qwen / DeepSeek model
// re-triggering #3528. The encoder only writes `reasoning_content`
// when a thinking block actually exists on the turn
// (`nonEmptyThinkingBlocks.length > 0`), so the flag is a no-op on
// pure-text histories.
replayReasoningContent:
Boolean(spec.reasoning) &&
!PROXY_OPENAI_COMPAT_PROVIDERS.has(provider) &&
(LOCAL_OPENAI_COMPAT_PROVIDERS.has(provider) || hasLocalLoopbackBaseUrl(baseUrl)),
requiresAssistantContentForToolCalls: isKimiModel || isDirectDeepseekReasoning,