fix(catalog): use LM Studio's loaded context window not the max

getLmStudioNativeContextWindow tried max_context_length first and never
read loaded_context_length, so omp believed a window the backend would
not accept. Models are routinely loaded below their architectural
maximum (user picks a smaller window, or MLX context auto-fit shrinks it
to fit unified memory), and compaction scheduled against the max window
never fired, killing sessions mid-run.

Prefer loaded_context_length when a model reports state: "loaded",
matching the runtime-over-max rule from #3754 for Ollama and llama.cpp.
Unloaded models report loaded_context_length: null and fall through to
the existing max/train chain unchanged.

Fixes #6082
This commit is contained in:
roboomp
2026-07-21 20:54:43 +00:00
parent 39c95e5e29
commit da64bc8cbf
4 changed files with 62 additions and 5 deletions
+4
View File
@@ -2,6 +2,10 @@
## [Unreleased]
### Fixed
- Fixed LM Studio discovery reporting a model's architectural maximum (`max_context_length`) instead of the window the running instance actually serves. `getLmStudioNativeContextWindow` now prefers `loaded_context_length` when a model reports `state: "loaded"`, so context accounting and compaction schedule against the real window ([#6082](https://github.com/can1357/oh-my-pi/issues/6082)).
## [17.0.5] - 2026-07-18
### Added
@@ -2631,7 +2631,16 @@ function getLmStudioNativeInput(entry: Record<string, unknown>): ("text" | "imag
}
function getLmStudioNativeContextWindow(entry: Record<string, unknown>): number | undefined {
// LM Studio serves `loaded_context_length` (the window the running instance
// actually accepts) alongside `max_context_length` (the architectural
// ceiling). A model is routinely loaded below its maximum — the user picks a
// smaller window, or MLX context auto-fit shrinks it to fit unified memory —
// so prefer what the runtime serves, matching the Ollama/llama.cpp rule from
// #3754. Unloaded models report `loaded_context_length: null` and fall
// through to the max/train chain unchanged.
const loadedContextWindow = entry.state === "loaded" ? toPositiveNumber(entry.loaded_context_length, null) : null;
return (
loadedContextWindow ??
toPositiveNumber(entry.max_context_length, null) ??
toPositiveNumber(entry.context_length, null) ??
toPositiveNumber(entry.max_model_len, null) ??
@@ -48,6 +48,54 @@ describe("lm studio local provider discovery", () => {
expect(text?.input).toEqual(["text"]);
});
test("prefers the loaded context window over the architectural maximum", async () => {
const fetchMock: FetchImpl = vi.fn(async input => {
const url = String(input);
if (url === "http://127.0.0.1:1234/api/v0/models") {
return new Response(
JSON.stringify({
data: [
{
id: "loaded-small",
type: "llm",
state: "loaded",
max_context_length: 262144,
loaded_context_length: 81920,
},
{
id: "unloaded",
type: "llm",
state: "not-loaded",
max_context_length: 262144,
loaded_context_length: null,
},
],
}),
{ status: 200, headers: { "Content-Type": "application/json" } },
);
}
if (url === "http://127.0.0.1:1234/v1/models") {
return new Response(
JSON.stringify({
data: [
{ id: "loaded-small", object: "model" },
{ id: "unloaded", object: "model" },
],
}),
{ status: 200, headers: { "Content-Type": "application/json" } },
);
}
throw new Error(`Unexpected URL: ${url}`);
});
const models = await lmStudioModelManagerOptions({ fetch: fetchMock }).fetchDynamicModels?.();
const loaded = models?.find(model => model.id === "loaded-small");
const unloaded = models?.find(model => model.id === "unloaded");
expect(loaded?.contextWindow).toBe(81920);
expect(unloaded?.contextWindow).toBe(262144);
});
test("falls back to the OpenAI-compatible catalog when native metadata hangs", async () => {
let nativeAborted = false;
let openAiCatalogStartedBeforeAbort = false;
@@ -47,7 +47,7 @@ For independent per-item chains (review → verify, fetch → extract → score)
schema: FINDINGS_SCHEMA,
});
return await parallel(found.findings.map((f) => async () => ({
...f,
…f,
verdict: await agent(
`Refute if you can (default refuted when unsure): ${f.title}`,
{ label: `verify:${f.file}`, schema: VERDICT_SCHEMA },
@@ -57,8 +57,6 @@ For independent per-item chains (review → verify, fetch → extract → score)
phase("Review");
const results = await parallel(DIMENSIONS.map((d) => async () => reviewAndVerify(d)));
const confirmed = results.flat().filter((f) => f.verdict.is_real);
Reach for `pipeline()` only when a stage genuinely needs ALL of the previous stage first — dedup/merge across the whole set, early-exit on zero, or "compare against the other findings" — because its inter-stage barrier makes every item wait for the slowest peer:
**Python (`eval`, Python backend):**
@@ -80,8 +78,6 @@ Reach for `pipeline()` only when a stage genuinely needs ALL of the previous sta
const verdicts = await parallel(findings.map((f) => async () =>
await agent(verifyPrompt(f), { schema: VERDICT_SCHEMA }),
));
Use ordinary code between calls to flatten/map/filter; don't add a barrier just for that. Nested `parallel()` pools each cap independently, so keep total fan-out sane.
</structure>