fix(secrets): redact matching prefix before a cut placeholder
A regex match that starts in outside text and ends inside a previously
generated `#…#` placeholder's expanded value was skipped wholesale by
resuming the scan past the placeholder. An independently-matching outside
prefix was therefore left provider-visible — e.g. `[A-Z0-9]{8,12}` greedily
spanning `SECRETUV` into an `ABCDEFGH` placeholder returned `SECRETUV#…#`
even though `SECRETUV` satisfies the regex on its own.
The cut handling now re-runs the regex bounded to just before the
placeholder (full left context kept, so lookbehind still evaluates) and, when
the prefix forms a standalone match, redacts it — to its own reversible
placeholder in obfuscate mode, or a one-way redaction in replace mode — while
the cut secret stays as its existing placeholder. The replace default
redaction's fixed point is now verified against the placeholder-expanded view
re-obfuscation actually scans, so it does not drift when the adjacent
placeholder expands and connects to the redaction's trailing bytes.
Refs #2465
This commit is contained in:
Mathews-Tom
@@ -16,6 +16,7 @@
|
||||
- Fixed a secret regex whose match boundary falls inside a previously generated `#…#` placeholder's expanded value mishandling the cut. In obfuscate mode the boundary was snapped out to the whole token, so two such matches around one placeholder mapped to overlapping source ranges that clobbered on apply and dropped bytes from reversible deobfuscation (e.g. a plain `ABCDEFGH` secret plus `[A-Z]{8}` turned `YYBBABCDEFGHSECRETUV` into a placeholder that restored as `YYBBABCDEFGHETUV`, dropping `SECR`); in replace mode the same cut redacted only the bytes outside the snapped token with a deterministic scramble that drifted across re-obfuscation passes (`ZZgK#…#` → `ZZgZ#…#`). The regex scan now resumes just past the cut placeholder rather than consuming the straddled span, so the cut secret stays hidden as its existing placeholder, no bytes are lost, any trailing wholly-outside content (e.g. an adjacent 8-char run) is still obfuscated or redacted on its own, and re-obfuscation is a fixed point ([#2465](https://github.com/can1357/oh-my-pi/issues/2465)).
|
||||
- Fixed a `mode: "replace"` regex that depends on surrounding context (lookbehind/lookahead/`\b`) leaking the raw matched value on alternating turns. The deterministic-replacement collision search tested candidate redactions in isolation, so for a pattern like `(?<=api=)[AZ]` it accepted `api=A` for `api=Z` (a bare `A` does not match the lookbehind) — but the next obfuscate pass re-matched `A` in context and redacted it back to `api=Z`, shipping the secret every other turn. Candidate redactions are now evaluated in their surrounding text, and the deterministic replacement itself is verified to be a fixed point in context (not just against the `Z`/`ZZ` sentinel), so context-sensitive replace regexes resolve to a value the pattern never re-matches in place ([#2465](https://github.com/can1357/oh-my-pi/issues/2465)).
|
||||
- Fixed a multi-character `mode: "replace"` regex remainder (the bytes of a match outside a preserved `#…#` placeholder) drifting across an obfuscator restart, which invalidated provider prompt-cache prefixes even with a stable key. The remainder was redacted to a content-derived `ZZ`+hash marker that was only recognized as already-redacted within the generating session (via an in-memory set), so a fresh obfuscator reprocessing persisted text re-redacted it to a different value (`ZZPL#…#` → `ZZ7f#…#`). The remainder marker now derives from a keyed run of the per-install key and the remainder length, so any instance sharing the key reproduces it byte-identically (idempotent across restart) while staying unpredictable enough that raw sentinel-shaped bytes (`ZZZZ`) still differ from it and are redacted rather than passed through ([#2465](https://github.com/can1357/oh-my-pi/issues/2465)).
|
||||
- Fixed a secret regex match that starts in outside text and ends inside a previously generated `#…#` placeholder's expanded value leaving an independently-matching outside prefix provider-visible. Resuming the scan past the cut placeholder skipped the whole straddling span, so a pattern like `[A-Z0-9]{8,12}` greedily spanning `SECRETUV` into an `ABCDEFGH` placeholder returned `SECRETUV#…#` even though `SECRETUV` satisfies the regex on its own. The cut handling now re-runs the regex bounded to just before the placeholder (full left context kept, so lookbehind still evaluates) and redacts the standalone prefix match — to its own reversible placeholder in obfuscate mode, or a one-way redaction in replace mode — while the cut secret stays as its existing placeholder. The replace-mode redaction's fixed point is verified against the placeholder-expanded view re-obfuscation actually scans, so it does not drift when the adjacent placeholder expands ([#2465](https://github.com/can1357/oh-my-pi/issues/2465)).
|
||||
|
||||
## [16.2.6] - 2026-06-29
|
||||
|
||||
|
||||
@@ -694,13 +694,10 @@ export class SecretObfuscator {
|
||||
result = replaceRange(result, match.start, replaceEnd, redacted);
|
||||
origin = replaceRange(origin, match.start, replaceEnd, "I".repeat(redacted.length));
|
||||
} else {
|
||||
const replacement =
|
||||
entry.replacement ??
|
||||
this.#generateRegexReplacement(match.value, entry.regex, {
|
||||
text: result,
|
||||
start: match.start,
|
||||
end: match.end,
|
||||
});
|
||||
const replacement = entry.replacement ?? match.defaultReplacement;
|
||||
if (replacement === undefined) {
|
||||
throw new Error("regex replace match missing a generated replacement");
|
||||
}
|
||||
result = replaceRange(result, match.start, match.end, replacement);
|
||||
origin = replaceRange(origin, match.start, match.end, "I".repeat(replacement.length));
|
||||
}
|
||||
@@ -1082,6 +1079,7 @@ export class SecretObfuscator {
|
||||
inputPlaceholderOutsideIndependentlyMatches: boolean;
|
||||
inputPlaceholderOutsideStart: number;
|
||||
inputPlaceholderOutsideChunkCount: number;
|
||||
defaultReplacement: string | undefined;
|
||||
}> {
|
||||
const knownPlaceholderRanges = this.#knownPlaceholderRanges(text);
|
||||
const regexScan = buildReplaceRegexScan(text, knownPlaceholderRanges, this.#deobfuscateMap);
|
||||
@@ -1100,6 +1098,7 @@ export class SecretObfuscator {
|
||||
inputPlaceholderOutsideIndependentlyMatches: boolean;
|
||||
inputPlaceholderOutsideStart: number;
|
||||
inputPlaceholderOutsideChunkCount: number;
|
||||
defaultReplacement: string | undefined;
|
||||
}> = [];
|
||||
for (;;) {
|
||||
const match = regex.exec(scanText);
|
||||
@@ -1110,7 +1109,8 @@ export class SecretObfuscator {
|
||||
}
|
||||
let start = match.index;
|
||||
let end = match.index + match[0].length;
|
||||
const scanMatchLength = match[0].length;
|
||||
let scanMatchLength = match[0].length;
|
||||
let scanMatchValue = match[0];
|
||||
let canonicalValue = "";
|
||||
let recursive = false;
|
||||
let preserveGeneratedPlaceholders = false;
|
||||
@@ -1120,17 +1120,53 @@ export class SecretObfuscator {
|
||||
let inputPlaceholderOutsideStart = -1;
|
||||
let inputPlaceholderOutsideChunkCount = 0;
|
||||
|
||||
const mapped = mapReplaceRegexMatch(regexScan.segments, start, end);
|
||||
let mapped = mapReplaceRegexMatch(regexScan.segments, start, end);
|
||||
if (mapped.partialPlaceholderCut) {
|
||||
// The match straddles a generated placeholder (its boundary falls inside
|
||||
// the secret's expanded value). Rewriting across the token drops bytes
|
||||
// (obfuscate) or drifts the redaction across re-obfuscation passes
|
||||
// (replace), so resume scanning just past the placeholder instead — the
|
||||
// cut secret stays as its existing placeholder and any trailing
|
||||
// wholly-outside content is matched fresh on the next iteration.
|
||||
regex.lastIndex = mapped.cutResumeIndex;
|
||||
continue;
|
||||
// (replace), so the cut secret must stay as its existing placeholder.
|
||||
// But the wholly-outside prefix BEFORE the placeholder can itself be a
|
||||
// complete match (e.g. `[A-Z0-9]{8,12}` greedily spanning `SECRETUV` into
|
||||
// an `ABCDEFGH` placeholder): that prefix is provider-visible
|
||||
// secret-shaped content, not a drift artifact. Re-run the regex bounded
|
||||
// to just before the placeholder — full left context kept, so lookbehind
|
||||
// still evaluates — and redact the independent prefix match on its own;
|
||||
// otherwise skip past the placeholder and match trailing content fresh.
|
||||
const cutResumeIndex = mapped.cutResumeIndex;
|
||||
const prefixScanEnd = mapped.firstPlaceholderScanStart;
|
||||
let handledPrefix = false;
|
||||
if (prefixScanEnd > match.index) {
|
||||
regex.lastIndex = match.index;
|
||||
const prefixMatch = regex.exec(scanText.slice(0, prefixScanEnd));
|
||||
if (prefixMatch !== null && prefixMatch[0].length > 0) {
|
||||
const prefixStart = prefixMatch.index;
|
||||
const prefixEnd = prefixMatch.index + prefixMatch[0].length;
|
||||
const prefixMapped = mapReplaceRegexMatch(regexScan.segments, prefixStart, prefixEnd);
|
||||
if (!prefixMapped.partialPlaceholderCut) {
|
||||
start = prefixStart;
|
||||
end = prefixEnd;
|
||||
scanMatchValue = prefixMatch[0];
|
||||
scanMatchLength = prefixMatch[0].length;
|
||||
mapped = prefixMapped;
|
||||
regex.lastIndex = prefixEnd;
|
||||
handledPrefix = true;
|
||||
}
|
||||
}
|
||||
}
|
||||
if (!handledPrefix) {
|
||||
regex.lastIndex = cutResumeIndex;
|
||||
continue;
|
||||
}
|
||||
}
|
||||
// Scan-space coordinates of the match (placeholders expanded). The default
|
||||
// redaction's fixed-point check must run against this expanded view — the
|
||||
// view re-obfuscation actually scans — not the literal `#…#` text, or a
|
||||
// redaction adjacent to a placeholder (e.g. an outside prefix before a cut
|
||||
// secret) could drift when the placeholder expands and connects to it.
|
||||
const scanMatchStart = start;
|
||||
const scanMatchEnd = end;
|
||||
let defaultReplacement: string | undefined;
|
||||
start = mapped.start;
|
||||
end = mapped.end;
|
||||
preserveGeneratedPlaceholders = mapped.preserveGeneratedPlaceholders;
|
||||
@@ -1169,7 +1205,7 @@ export class SecretObfuscator {
|
||||
regex.lastIndex = resumeIndex;
|
||||
}
|
||||
if (mode === "replace") {
|
||||
canonicalValue = match[0];
|
||||
canonicalValue = scanMatchValue;
|
||||
recursive = mapped.recursive;
|
||||
} else {
|
||||
const overlappingRanges = knownPlaceholderRanges.filter(range => start < range.end && end > range.start);
|
||||
@@ -1188,10 +1224,20 @@ export class SecretObfuscator {
|
||||
recursive = canonical.recursive;
|
||||
}
|
||||
|
||||
if (mode === "replace" && replacement === undefined && !preserveGeneratedPlaceholders) {
|
||||
const savedLastIndex = regex.lastIndex;
|
||||
defaultReplacement = this.#generateRegexReplacement(scanMatchValue, regex, {
|
||||
text: scanText,
|
||||
start: scanMatchStart,
|
||||
end: scanMatchEnd,
|
||||
});
|
||||
regex.lastIndex = savedLastIndex;
|
||||
}
|
||||
matches.push({
|
||||
start,
|
||||
end,
|
||||
value: text.slice(start, end),
|
||||
defaultReplacement,
|
||||
canonicalValue,
|
||||
scanMatchLength,
|
||||
recursive,
|
||||
@@ -1516,6 +1562,7 @@ function mapReplaceRegexMatch(
|
||||
preserveGeneratedPlaceholders: boolean;
|
||||
partialPlaceholderCut: boolean;
|
||||
cutResumeIndex: number;
|
||||
firstPlaceholderScanStart: number;
|
||||
} {
|
||||
const startSegment = findScanSegment(segments, scanStart);
|
||||
const endSegment = findScanSegment(segments, scanEnd - 1);
|
||||
@@ -1537,17 +1584,29 @@ function mapReplaceRegexMatch(
|
||||
// When the match straddles a placeholder, resume scanning just past the last
|
||||
// overlapping placeholder so trailing wholly-outside content (e.g. an 8-char
|
||||
// run after the secret) still gets matched instead of being consumed by the
|
||||
// straddling span.
|
||||
// straddling span. `firstPlaceholderScanStart` marks where the leading
|
||||
// wholly-outside prefix ends, so a prefix that independently matches can be
|
||||
// redacted on its own rather than skipped along with the cut span.
|
||||
let cutResumeIndex = scanStart;
|
||||
let firstPlaceholderScanStart = -1;
|
||||
for (const segment of segments) {
|
||||
if (segment.scanStart >= scanEnd || segment.scanEnd <= scanStart) continue;
|
||||
recursive ||= segment.recursive;
|
||||
preserveGeneratedPlaceholders ||= segment.generatedPlaceholder;
|
||||
if (segment.generatedPlaceholder && segment.scanEnd > cutResumeIndex) {
|
||||
cutResumeIndex = segment.scanEnd;
|
||||
if (segment.generatedPlaceholder) {
|
||||
if (firstPlaceholderScanStart === -1) firstPlaceholderScanStart = segment.scanStart;
|
||||
if (segment.scanEnd > cutResumeIndex) cutResumeIndex = segment.scanEnd;
|
||||
}
|
||||
}
|
||||
return { start, end, recursive, preserveGeneratedPlaceholders, partialPlaceholderCut, cutResumeIndex };
|
||||
return {
|
||||
start,
|
||||
end,
|
||||
recursive,
|
||||
preserveGeneratedPlaceholders,
|
||||
partialPlaceholderCut,
|
||||
cutResumeIndex,
|
||||
firstPlaceholderScanStart,
|
||||
};
|
||||
}
|
||||
|
||||
function findScanSegment(segments: ReadonlyArray<RegexScanSegment>, scanIndex: number): RegexScanSegment {
|
||||
|
||||
@@ -460,6 +460,46 @@ describe("SecretObfuscator friendlyName placeholders", () => {
|
||||
expect(obfuscator.obfuscate(obfuscated)).toBe(obfuscated);
|
||||
});
|
||||
|
||||
it("redacts an independently matching prefix before a cut placeholder", () => {
|
||||
// A regex match that starts in outside text and ends inside a generated
|
||||
// placeholder's expanded value is not rewritten across the token (that drops
|
||||
// bytes / drifts the redaction). But when the wholly-outside prefix before the
|
||||
// placeholder is itself a complete match it is provider-visible secret-shaped
|
||||
// content, not a drift artifact. Regression: `[A-Z0-9]{8,12}` greedily
|
||||
// spanning `SECRETUV` into an `ABCDEFGH` placeholder returned `SECRETUV#…#`,
|
||||
// leaving `SECRETUV` visible even though it independently satisfies the regex.
|
||||
// Obfuscate mode: the prefix gets its own reversible placeholder and the whole
|
||||
// input still round-trips.
|
||||
const obf = new SecretObfuscator(
|
||||
[
|
||||
{ type: "plain", content: "ABCDEFGH" },
|
||||
{ type: "regex", content: "[A-Z0-9]{8,12}" },
|
||||
],
|
||||
"Q".repeat(43),
|
||||
);
|
||||
const out = obf.obfuscate("SECRETUVABCDEFGH");
|
||||
expect(out).not.toContain("SECRETUV");
|
||||
expect(out).not.toContain("ABCDEFGH");
|
||||
expect(obf.deobfuscate(out)).toBe("SECRETUVABCDEFGH");
|
||||
expect(obf.obfuscate(out)).toBe(out);
|
||||
|
||||
// Replace mode: the prefix is redacted one-way while the cut secret's
|
||||
// placeholder is preserved and still restores.
|
||||
const repl = new SecretObfuscator(
|
||||
[
|
||||
{ type: "plain", content: "ABCDEFGH" },
|
||||
{ type: "regex", content: "[A-Z0-9]{8,12}", mode: "replace" },
|
||||
],
|
||||
"Q".repeat(43),
|
||||
);
|
||||
const rout = repl.obfuscate("SECRETUVABCDEFGH");
|
||||
expect(rout).not.toContain("SECRETUV");
|
||||
expect(rout).not.toContain("ABCDEFGH");
|
||||
expect(repl.deobfuscate(rout)).toMatch(/ABCDEFGH$/);
|
||||
expect(repl.deobfuscate(rout)).not.toContain("SECRETUV");
|
||||
expect(repl.obfuscate(rout)).toBe(rout);
|
||||
});
|
||||
|
||||
it("keeps regex placeholders stable when inner friendly names change", () => {
|
||||
const sharedKey = "E".repeat(43);
|
||||
const before = new SecretObfuscator(
|
||||
|
||||
Reference in New Issue
Block a user