fix(mnemopi): lowered default embedding input cap to 8192 chars

Defaulted MNEMOPI_EMBEDDING_MAX_INPUT_CHARS to 8192 so the automatic guard matches bge-m3 and OpenAI text-embedding context limits by default. Larger local embedding servers such as Qwen3-Embedding with 32k ctx can still raise the cap, and 0 still disables truncation.

Fixes #3126
This commit is contained in:
roboomp
2026-06-20 13:20:45 +00:00
committed by can1357
parent 6666dec657
commit f2ad18692f
5 changed files with 11 additions and 11 deletions
+1 -1
View File
@@ -26,7 +26,7 @@
### Fixed
- Stopped Mnemopi retention from overflowing the embedding model's context window: `embed()` now caps each input at `MNEMOPI_EMBEDDING_MAX_INPUT_CHARS` (default 32000 chars) so a long multi-turn `MnemopiSessionState.retainMessages` transcript can't make llama.cpp's `/embeddings` server reject the request with `request (N tokens) exceeds the available context size` and silently drop vector recall for that memory ([#3126](https://github.com/can1357/oh-my-pi/issues/3126)).
- Stopped Mnemopi retention from overflowing the embedding model's context window: `embed()` now caps each input at `MNEMOPI_EMBEDDING_MAX_INPUT_CHARS` (default 8192 chars) so a long multi-turn `MnemopiSessionState.retainMessages` transcript can't make llama.cpp's `/embeddings` server reject the request with `request (N tokens) exceeds the available context size` and silently drop vector recall for that memory ([#3126](https://github.com/can1357/oh-my-pi/issues/3126)).
## [16.1.7] - 2026-06-20
+1 -1
View File
@@ -4,7 +4,7 @@
### Fixed
- Capped per-input length in `embed()` at `MNEMOPI_EMBEDDING_MAX_INPUT_CHARS` (default 32000 chars, override via the env var or `embeddings.maxInputChars` runtime option; `0` disables) so a long retention transcript can no longer overflow the embedding model's context window. llama.cpp's `/embeddings` server used to reject the request with `request (N tokens) exceeds the available context size`, silently dropping vector recall for that memory ([#3126](https://github.com/can1357/oh-my-pi/issues/3126)).
- Capped per-input length in `embed()` at `MNEMOPI_EMBEDDING_MAX_INPUT_CHARS` (default 8192 chars, override via the env var or `embeddings.maxInputChars` runtime option; `0` disables) so a long retention transcript can no longer overflow the embedding model's context window. llama.cpp's `/embeddings` server used to reject the request with `request (N tokens) exceeds the available context size`, silently dropping vector recall for that memory ([#3126](https://github.com/can1357/oh-my-pi/issues/3126)).
## [16.1.3] - 2026-06-19
+5 -5
View File
@@ -110,13 +110,13 @@ export function embeddingsDisabled(env: Env = process.env): boolean {
* source gives both backends deterministic behavior and prevents the silent
* recall degradation we saw in issue #3126.
*
* Default `32000` ≈ 8k English tokens (4 chars/token) or ~16k–32k CJK tokens —
* fits 8k-context models (bge-m3, text-embedding-3) for English and most CJK
* content. Lower it for 512-token models like `BAAI/bge-small-en-v1.5`, raise
* it for larger contexts. `0` disables the cap.
* Default `8192` chars is intentionally conservative for 8192-token embedding
* contexts (bge-m3, OpenAI text-embedding-3) and CJK-heavy transcripts. Raise
* it for larger local contexts (for example Qwen3-Embedding with 32k ctx).
* `0` disables the cap.
*/
export function embeddingMaxInputChars(env: Env = process.env): number {
return Math.max(0, envInt("MNEMOPI_EMBEDDING_MAX_INPUT_CHARS", 32000, env));
return Math.max(0, envInt("MNEMOPI_EMBEDDING_MAX_INPUT_CHARS", 8192, env));
}
export function isApiEmbeddingModel(model = embeddingModel(), env: Env = process.env): boolean {
+2 -2
View File
@@ -124,7 +124,7 @@ export function embeddingsDisabled(): boolean {
* Resolved per-input character cap for {@link embed}.
*
* Reads (in order): the active runtime scope's `embeddings.maxInputChars`, then
* `MNEMOPI_EMBEDDING_MAX_INPUT_CHARS`, then the bundled `32000` default. `0`
* `MNEMOPI_EMBEDDING_MAX_INPUT_CHARS`, then the bundled `8192` default. `0`
* disables the cap entirely.
*/
function effectiveMaxInputChars(): number {
@@ -132,7 +132,7 @@ function effectiveMaxInputChars(): number {
if (override !== undefined) return Math.max(0, Math.trunc(override));
const envValue = Number.parseInt($env.MNEMOPI_EMBEDDING_MAX_INPUT_CHARS ?? "", 10);
if (Number.isFinite(envValue) && envValue >= 0) return envValue;
return 32000;
return 8192;
}
/**
@@ -13,7 +13,7 @@ import { withMnemopiRuntimeOptions } from "@oh-my-pi/pi-mnemopi/core/runtime-opt
* overflow whatever ctx the embedding server was started with — llama.cpp
* rejects oversized requests with `request (N tokens) exceeds the available
* context size`, OpenAI silently right-truncates. `embed()` now caps each
* input to `MNEMOPI_EMBEDDING_MAX_INPUT_CHARS` (default 32000) before the
* input to `MNEMOPI_EMBEDDING_MAX_INPUT_CHARS` (default 8192) before the
* provider sees it.
*/
function captureProvider(): {
@@ -57,7 +57,7 @@ describe("embed() input cap (#3126)", () => {
expect(provider.calls).toHaveLength(1);
const [seenShort, seenHuge] = provider.calls[0] ?? [];
expect(seenShort).toBe("short");
expect(seenHuge?.length).toBe(32_000);
expect(seenHuge?.length).toBe(8192);
});
it("honors MNEMOPI_EMBEDDING_MAX_INPUT_CHARS env override", async () => {