fix(catalog): use LM Studio's loaded context window not the max
getLmStudioNativeContextWindow tried max_context_length first and never read loaded_context_length, so omp believed a window the backend would not accept. Models are routinely loaded below their architectural maximum (user picks a smaller window, or MLX context auto-fit shrinks it to fit unified memory), and compaction scheduled against the max window never fired, killing sessions mid-run. Prefer loaded_context_length when a model reports state: "loaded", matching the runtime-over-max rule from #3754 for Ollama and llama.cpp. Unloaded models report loaded_context_length: null and fall through to the existing max/train chain unchanged. Fixes #6082
This commit is contained in:
@@ -2,6 +2,10 @@
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Fixed
|
||||
|
||||
- Fixed LM Studio discovery reporting a model's architectural maximum (`max_context_length`) instead of the window the running instance actually serves. `getLmStudioNativeContextWindow` now prefers `loaded_context_length` when a model reports `state: "loaded"`, so context accounting and compaction schedule against the real window ([#6082](https://github.com/can1357/oh-my-pi/issues/6082)).
|
||||
|
||||
## [17.0.5] - 2026-07-18
|
||||
|
||||
### Added
|
||||
|
||||
@@ -2631,7 +2631,16 @@ function getLmStudioNativeInput(entry: Record<string, unknown>): ("text" | "imag
|
||||
}
|
||||
|
||||
function getLmStudioNativeContextWindow(entry: Record<string, unknown>): number | undefined {
|
||||
// LM Studio serves `loaded_context_length` (the window the running instance
|
||||
// actually accepts) alongside `max_context_length` (the architectural
|
||||
// ceiling). A model is routinely loaded below its maximum — the user picks a
|
||||
// smaller window, or MLX context auto-fit shrinks it to fit unified memory —
|
||||
// so prefer what the runtime serves, matching the Ollama/llama.cpp rule from
|
||||
// #3754. Unloaded models report `loaded_context_length: null` and fall
|
||||
// through to the max/train chain unchanged.
|
||||
const loadedContextWindow = entry.state === "loaded" ? toPositiveNumber(entry.loaded_context_length, null) : null;
|
||||
return (
|
||||
loadedContextWindow ??
|
||||
toPositiveNumber(entry.max_context_length, null) ??
|
||||
toPositiveNumber(entry.context_length, null) ??
|
||||
toPositiveNumber(entry.max_model_len, null) ??
|
||||
|
||||
@@ -48,6 +48,54 @@ describe("lm studio local provider discovery", () => {
|
||||
expect(text?.input).toEqual(["text"]);
|
||||
});
|
||||
|
||||
test("prefers the loaded context window over the architectural maximum", async () => {
|
||||
const fetchMock: FetchImpl = vi.fn(async input => {
|
||||
const url = String(input);
|
||||
if (url === "http://127.0.0.1:1234/api/v0/models") {
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
data: [
|
||||
{
|
||||
id: "loaded-small",
|
||||
type: "llm",
|
||||
state: "loaded",
|
||||
max_context_length: 262144,
|
||||
loaded_context_length: 81920,
|
||||
},
|
||||
{
|
||||
id: "unloaded",
|
||||
type: "llm",
|
||||
state: "not-loaded",
|
||||
max_context_length: 262144,
|
||||
loaded_context_length: null,
|
||||
},
|
||||
],
|
||||
}),
|
||||
{ status: 200, headers: { "Content-Type": "application/json" } },
|
||||
);
|
||||
}
|
||||
if (url === "http://127.0.0.1:1234/v1/models") {
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
data: [
|
||||
{ id: "loaded-small", object: "model" },
|
||||
{ id: "unloaded", object: "model" },
|
||||
],
|
||||
}),
|
||||
{ status: 200, headers: { "Content-Type": "application/json" } },
|
||||
);
|
||||
}
|
||||
throw new Error(`Unexpected URL: ${url}`);
|
||||
});
|
||||
|
||||
const models = await lmStudioModelManagerOptions({ fetch: fetchMock }).fetchDynamicModels?.();
|
||||
const loaded = models?.find(model => model.id === "loaded-small");
|
||||
const unloaded = models?.find(model => model.id === "unloaded");
|
||||
|
||||
expect(loaded?.contextWindow).toBe(81920);
|
||||
expect(unloaded?.contextWindow).toBe(262144);
|
||||
});
|
||||
|
||||
test("falls back to the OpenAI-compatible catalog when native metadata hangs", async () => {
|
||||
let nativeAborted = false;
|
||||
let openAiCatalogStartedBeforeAbort = false;
|
||||
|
||||
@@ -47,7 +47,7 @@ For independent per-item chains (review → verify, fetch → extract → score)
|
||||
schema: FINDINGS_SCHEMA,
|
||||
});
|
||||
return await parallel(found.findings.map((f) => async () => ({
|
||||
...f,
|
||||
…f,
|
||||
verdict: await agent(
|
||||
`Refute if you can (default refuted when unsure): ${f.title}`,
|
||||
{ label: `verify:${f.file}`, schema: VERDICT_SCHEMA },
|
||||
@@ -57,8 +57,6 @@ For independent per-item chains (review → verify, fetch → extract → score)
|
||||
phase("Review");
|
||||
const results = await parallel(DIMENSIONS.map((d) => async () => reviewAndVerify(d)));
|
||||
const confirmed = results.flat().filter((f) => f.verdict.is_real);
|
||||
|
||||
|
||||
Reach for `pipeline()` only when a stage genuinely needs ALL of the previous stage first — dedup/merge across the whole set, early-exit on zero, or "compare against the other findings" — because its inter-stage barrier makes every item wait for the slowest peer:
|
||||
|
||||
**Python (`eval`, Python backend):**
|
||||
@@ -80,8 +78,6 @@ Reach for `pipeline()` only when a stage genuinely needs ALL of the previous sta
|
||||
const verdicts = await parallel(findings.map((f) => async () =>
|
||||
await agent(verifyPrompt(f), { schema: VERDICT_SCHEMA }),
|
||||
));
|
||||
|
||||
|
||||
Use ordinary code between calls to flatten/map/filter; don't add a barrier just for that. Nested `parallel()` pools each cap independently, so keep total fan-out sane.
|
||||
</structure>
|
||||
|
||||
|
||||
Reference in New Issue
Block a user