diff --git a/docs/ERRATA-GPT5-HARMONY.md b/docs/ERRATA-GPT5-HARMONY.md index 6b8cf4d27..194bcc9c3 100644 --- a/docs/ERRATA-GPT5-HARMONY.md +++ b/docs/ERRATA-GPT5-HARMONY.md @@ -138,7 +138,7 @@ The `edit` tool exists in two variants in the corpus: | Variant | Calls | Recovery | |--------------------------|------:|----------| -| Patch-DSL (`@PATH`/anchor/`~payload`) | 27 | **Recoverable** by op-truncation (§3.3) | +| Patch-DSL (`§PATH`/anchor/`«»≔` ops) | 27 | **Recoverable** by op-truncation (§3.3) | | JSON-schema (`{path,edits:[…]}`) | 11 | **Not recoverable** — contamination is escaped *inside* JSON strings, parser accepts it cleanly, content would be written verbatim into source files | For Patch-DSL leaks specifically: diff --git a/docs/tools/edit.md b/docs/tools/edit.md index 4e5b60769..13840de20 100644 --- a/docs/tools/edit.md +++ b/docs/tools/edit.md @@ -8,8 +8,8 @@ - Key collaborators: - `packages/coding-agent/src/utils/edit-mode.ts` — selects active edit mode - `packages/coding-agent/src/hashline/grammar.lark` — custom-tool grammar for hashline mode - - `packages/coding-agent/src/hashline/input.ts` — splits `@@ PATH` sections (legacy single-`@` headers are still accepted) - - `packages/coding-agent/src/hashline/parser.ts` — parses ops and payload lines + - `packages/coding-agent/src/hashline/input.ts` — splits `§PATH` sections + - `packages/coding-agent/src/hashline/parser.ts` — parses op-prefixed edits and verbatim payload lines - `packages/coding-agent/src/hashline/apply.ts` — validates anchors and applies edits - `packages/coding-agent/src/hashline/anchors.ts` — stale-anchor mismatch formatting - `packages/coding-agent/src/hashline/recovery.ts` — cache-based stale-anchor recovery @@ -26,16 +26,17 @@ | Field | Type | Required | Description | | --- | --- | --- | --- | -| `input` | `string` | Yes | One or more edit sections. First non-blank line must be `@@ PATH` (legacy single-`@` is still accepted) unless the caller supplies the legacy fallback `path` outside the model schema and the body already looks like hashline ops (`packages/coding-agent/src/hashline/input.ts`). Optional `*** Begin Patch` / `*** End Patch` envelope is ignored if present. | +| `input` | `string` | Yes | One or more edit sections. First non-blank line must be `§PATH` unless the caller supplies the legacy fallback `path` outside the model schema and the body already looks like hashline ops (`packages/coding-agent/src/hashline/input.ts`). Optional `*** Begin Patch` / `*** End Patch` envelope is ignored if present. | Patch language inside `input`: -- Section header: `@@ PATH` -- Insert after: `+ ANCHOR` -- Insert before: `< ANCHOR` -- Delete range: `- A..B` -- Replace range: `= A..B` -- Payload line: `~TEXT` by default; separator is `HL_EDIT_SEP` and can be overridden once at process start by `PI_HL_SEP` (`packages/coding-agent/src/hashline/hash.ts`) +- Section header: `§PATH` +- Insert after: `»ANCHOR` +- Insert before: `«ANCHOR` +- Replace/delete range: `≔A..B` +- Single-line replace/delete sugar: `≔A` means `≔A..A` +- `≔A..B` with no payload deletes the range. To keep a blank line, include one explicit empty payload line. +- Payload lines: verbatim file content after `»`, `«`, or `≔` - Special anchors: `BOF`, `EOF` - Anchor token: `<2-char-hash>`, for example `41th` @@ -66,17 +67,17 @@ Warnings: - While the model is still typing arguments, the TUI can compute a diff preview with `packages/coding-agent/src/edit/streaming.ts`; that preview is not a deferred action and does not block execution. ## Flow -1. `EditTool.execute()` in `packages/coding-agent/src/edit/index.ts` resolves the active mode. Default is `hashline`; `customFormat` exposes `packages/coding-agent/src/hashline/grammar.lark` with `$HFMT$` / `$HSEP$` placeholders filled from `packages/coding-agent/src/hashline/hash.ts`. -2. `executeHashlineSingle()` in `packages/coding-agent/src/hashline/execute.ts` splits the raw `input` into `@PATH` sections with `splitHashlineInputs()`. +1. `EditTool.execute()` in `packages/coding-agent/src/edit/index.ts` resolves the active mode. Default is `hashline`; `customFormat` exposes `packages/coding-agent/src/hashline/grammar.lark` with `$HFMT$` / `$HOP_INSERT_BEFORE$` / `$HOP_INSERT_AFTER$` / `$HOP_REPLACE$` / `$HOP_CHARS$` / `$HFILE$` placeholders filled from `packages/coding-agent/src/hashline/hash.ts`. +2. `executeHashlineSingle()` in `packages/coding-agent/src/hashline/execute.ts` splits the raw `input` into `§PATH` sections with `splitHashlineInputs()`. 3. If multiple sections target the same path, `mergeSamePathSections()` concatenates them before execution so every op still refers to the original file snapshot. 4. Multi-section calls run a preflight pass (`preflightHashlineSection()`): parse ops, enforce plan-mode write rules, load the current file, reject anchor-scoped edits against missing files, reject auto-generated files, apply edits in memory, and fail if the result is a no-op. This prevents partial batches. 5. `parseHashlineWithWarnings()` in `packages/coding-agent/src/hashline/parser.ts` tokenizes the diff body: - ignores blank lines and optional `*** Begin Patch` - stops at `*** End Patch` - stops at `*** Abort` and emits `ABORT_WARNING` - - turns `+` / `<` payload runs into one `insert` edit per payload line - - turns `- A..B` into one `delete` edit per line in the range - - turns `= A..B` into inserts before `A`, then deletes for `A..B`; no payload means replace with a single empty line + - turns `»` / `«` payload runs into one `insert` edit per payload line + - turns `≔A..B` with payload into inserts before `A`, then deletes for `A..B` + - turns `≔A..B` with no payload into one `delete` edit per line in the range; a blank-in-place edit requires one explicit empty payload line 6. `applyHashlineEdits()` in `packages/coding-agent/src/hashline/apply.ts` validates every referenced anchor before mutating anything. Each anchor hash is recomputed from current file content with `computeLineHash()`. 7. If any anchor hash differs, `applyHashlineEdits()` throws `HashlineMismatchError`. `execute.ts` catches only that class and calls `tryRecoverHashlineWithCache()`. 8. Recovery replays the edits against the most recent cached read/search snapshot for that path (`packages/coding-agent/src/edit/file-read-cache.ts`), then 3-way merges the result onto current disk content using `Diff.applyPatch(..., { fuzzFactor: 3 })` in `packages/coding-agent/src/hashline/recovery.ts`. On success the edit proceeds with a warning; on failure the original mismatch error is re-thrown. @@ -102,41 +103,56 @@ Warnings: Hashline op examples: ```text -@@ src/a.ts -+ 4fb -~const added = true; +§src/a.ts +»4fb +const added = true; ``` ```text -@@ src/a.ts -< 4fb -~const addedBefore = true; +§src/a.ts +«4fb +const addedBefore = true; ``` ```text -@@ src/a.ts -- 4fb..6qx +§src/a.ts +≔4fb..6qx ``` ```text -@@ src/a.ts -= 4fb..5dm -~const clean = (name || DEF).trim(); -~return clean.length === 0 ? DEF : clean.toUpperCase(); +§src/a.ts +≔4fb..5dm +const clean = (name || DEF).trim(); +return clean.length === 0 ? DEF : clean.toUpperCase(); ``` BOF/EOF examples: ```text -@@ src/a.ts -+ BOF -~const HEADER = true; +§src/a.ts +»BOF +const HEADER = true; ``` ```text -@@ src/a.ts -+ EOF -~export const done = true; +§src/a.ts +»EOF +export const done = true; +``` + +Delete / blank examples: + +```text +§src/a.ts +≔4fb +``` + +```text +§src/a.ts +≔4fb + +»EOF +export const done = true; ``` ## Side Effects @@ -162,26 +178,28 @@ BOF/EOF examples: - Stale-anchor recovery uses `fuzzFactor: 3` (`HASHLINE_RECOVERY_FUZZ_FACTOR`) in `packages/coding-agent/src/hashline/recovery.ts`. - The per-session read cache keeps at most 30 paths (`MAX_PATHS_PER_SESSION`) in `packages/coding-agent/src/edit/file-read-cache.ts`. - Hashline streaming chunk defaults are 200 lines or 64 KiB per chunk (`packages/coding-agent/src/hashline/types.ts`, consumed by `packages/coding-agent/src/hashline/stream.ts`). -- `HL_EDIT_SEP` defaults to `~`; `HL_BODY_SEP` is always `|` (`packages/coding-agent/src/hashline/hash.ts`). +- `HL_OP_INSERT_BEFORE` is `«`, `HL_OP_INSERT_AFTER` is `»`, `HL_OP_REPLACE` is `≔`, `HL_OP_CHARS` is `«»≔`, `HL_FILE_PREFIX` is `§`, and `HL_BODY_SEP` is `|` (`packages/coding-agent/src/hashline/hash.ts`). ## Errors - Missing section header: - - `input must begin with "@@ PATH" on the first non-blank line; got: ... Example: "@@ src/foo.ts" then edit ops.` + - `input must begin with "§PATH" on the first non-blank line; got: ... Example: "§src/foo.ts" then edit ops.` - Empty header: - - `Input header "@" is empty; provide a file path.` + - `Input header "§" is empty; provide a file path.` - Bad anchor token: - `line N: expected a full anchor such as "119sr"; got "...".` - Bad range syntax: - - `line N: explicit ranges are required for delete/replace...` + - `line N: explicit ranges are required for replacement...` - `line N: range must include exactly two full anchors separated by "..".` - `line N: range A..B ends before it starts.` - `line N: range A..B uses two different hashes for the same line.` -- Missing payload for `+` / `<`: - - `line N: + and < operations require at least one ~TEXT payload line.` +- Missing payload for `»` / `«`: + - `line N: » and « operations require at least one verbatim payload line.` - Stray payload line: - - `line N: payload line has no preceding +, <, or = operation.` + - `line N: payload line has no preceding », «, or ≔ operation.` - Unknown op: - - `line N: unrecognized op. Use < ANCHOR..., + ANCHOR..., - A..B..., = A..B...` + - `line N: unrecognized op. Use «ANCHOR..., »ANCHOR..., ≔A..B...` +- Delete vs blank: + - `≔A..B` with no payload deletes. To blank in place, include one explicit empty payload line before the next op/header/EOF. - Missing file for anchor-scoped edits: - `File not found: ` - Out-of-range anchor: @@ -194,12 +212,12 @@ BOF/EOF examples: ## Notes - `read` and `search` are the authoritative source of anchors. The edit parser does not want the trailing `|TEXT`; copy only the `LINEhh` token. - Multi-op patches are parsed against the original file snapshot. Do not renumber later anchors after earlier ops; `applyHashlineEdits()` buckets and applies them bottom-up. -- `= A..B` is not a primitive replace in the parser. It expands to inserts before `A` plus deletes for `A..B`, which is why stale-anchor checking still happens on the original range lines. +- `≔A..B` is not a primitive replace in the parser. With payload, it expands to inserts before `A` plus deletes for `A..B`; with no payload, it only deletes `A..B`. To blank in place, include one explicit empty payload line. Stale-anchor checking still happens on the original range lines. - Interior lines of a multi-line range use hash `**` (`RANGE_INTERIOR_HASH`) and are not individually verified; only the first and last anchor hashes are checked. - `computeLineHash()` trims trailing whitespace before hashing. Anchors survive line-ending changes and trailing-space-only changes, but not substantive line edits. - For punctuation-only lines, the hash mixes in the line number; identical `}` lines on different lines intentionally get different anchors. -- `splitHashlineInputs()` normalizes absolute `@@ PATH` headers back to a cwd-relative path when the file is inside the current working tree. Headers with any run of leading `@` chars (e.g. `@ foo.ts`, `@@ foo.ts`, `@@@foo.ts`) are accepted to absorb unified-diff-style drift; the canonical form is `@@ PATH`. -- Optional `*** Begin Patch` / `*** End Patch` markers are accepted in hashline mode, but the file sections are still `@@ PATH`-based, not Codex `*** Update File:` hunks. +- `splitHashlineInputs()` normalizes absolute `§PATH` headers back to a cwd-relative path when the file is inside the current working tree. Headers with any run of leading `§` chars (e.g. `§foo.ts`, `§§foo.ts`, `§§§foo.ts`) are accepted; the canonical form is `§PATH`. +- Optional `*** Begin Patch` / `*** End Patch` markers are accepted in hashline mode, but the file sections are still `§PATH`-based, not Codex `*** Update File:` hunks. - `*** Abort` terminates parsing early and returns `ABORT_WARNING`; ops parsed before the marker still apply. - File-read cache invalidation is conflict-based, not write-through invalidation. If `read` later records content for a line that disagrees with the cached snapshot, the entire snapshot for that path is replaced with the newly observed lines (`packages/coding-agent/src/edit/file-read-cache.ts`). - There is no resolve-style apply/discard phase for hashline edits. The only preview path is the transient TUI diff preview in `packages/coding-agent/src/edit/streaming.ts`. diff --git a/packages/coding-agent/CHANGELOG.md b/packages/coding-agent/CHANGELOG.md index 04bf14632..acb53aa6a 100644 --- a/packages/coding-agent/CHANGELOG.md +++ b/packages/coding-agent/CHANGELOG.md @@ -1,6 +1,25 @@ # Changelog ## [Unreleased] +### Breaking Changes + +- Replaced the legacy `@@` header and `+`/`<`/`=`/`-` hashline syntax with the new `§PATH` header and `«`/`»`/`≔` operation format, so existing hashline scripts and prompts using old symbols must be updated + +### Added + +- Added one-anchor `≔ANCHOR` shorthand equivalent to `≔ANCHOR..ANCHOR` for single-line replace/delete + +### Changed + +- Changed `≔A..B` so an omitted payload now deletes the range, and added an explicit empty payload line to keep a literal blank replacement line + +### Removed + +- Removed the `hsep` prompt helper and `PI_HL_SEP` payload-prefix configuration because hashline payloads are no longer line-prefixed + +### Fixed + +- Fixed hashline payload handling in parser and streaming preview to preserve blank lines as actual payload text until the next op, file header, or envelope marker ## [15.2.3] - 2026-05-22 ### Breaking Changes diff --git a/packages/coding-agent/src/config/prompt-templates.ts b/packages/coding-agent/src/config/prompt-templates.ts index 3c81470de..5f833e787 100644 --- a/packages/coding-agent/src/config/prompt-templates.ts +++ b/packages/coding-agent/src/config/prompt-templates.ts @@ -8,7 +8,7 @@ import { parseFrontmatter, prompt, } from "@oh-my-pi/pi-utils"; -import { computeLineHash, HL_BODY_SEP, HL_EDIT_SEP } from "../hashline/hash"; +import { computeLineHash, HL_BODY_SEP } from "../hashline/hash"; import { jtdToTypeScript } from "../tools/jtd-to-typescript"; import { parseCommandArgs, substituteArgs } from "../utils/command-args"; @@ -154,13 +154,6 @@ prompt.registerHelper("hline", function (this: unknown, ...args: unknown[]): str return `${ref}${HL_BODY_SEP}${text}`; }); -/** - * {{hsep}} — emit the configured hashline payload separator character. - * Stays in sync with {@link HL_EDIT_SEP} so edit prompt templates - * never have to hardcode the payload separator. - */ -prompt.registerHelper("hsep", (): string => HL_EDIT_SEP); - const INLINE_ARG_SHELL_PATTERN = /\$(?:ARGUMENTS|@(?:\[\d+(?::\d*)?\])?|\d+)/; const INLINE_ARG_TEMPLATE_PATTERN = /\{\{[\s\S]*?(?:\b(?:arguments|ARGUMENTS|args)\b|\barg\s+[^}]+)[\s\S]*?\}\}/; diff --git a/packages/coding-agent/src/edit/index.ts b/packages/coding-agent/src/edit/index.ts index e1e40360b..8285ebd0e 100644 --- a/packages/coding-agent/src/edit/index.ts +++ b/packages/coding-agent/src/edit/index.ts @@ -35,7 +35,7 @@ export * from "./apply-patch"; export * from "./diff"; export * from "./file-read-cache"; -// Resolve the `$HFMT$` and `$HSEP$` placeholders in the hashline Lark grammar. +// Resolve the `$HFMT$`, `$HOP_*$`, `$HOP_CHARS$`, and `$HFILE$` placeholders in the hashline Lark grammar. const hashlineGrammar = resolveHashlineGrammarPlaceholders(hashlineGrammarTemplate); export * from "../hashline"; diff --git a/packages/coding-agent/src/edit/renderer.ts b/packages/coding-agent/src/edit/renderer.ts index 1fd113a8e..542c0c789 100644 --- a/packages/coding-agent/src/edit/renderer.ts +++ b/packages/coding-agent/src/edit/renderer.ts @@ -6,6 +6,7 @@ import type { Component } from "@oh-my-pi/pi-tui"; import { Text, visibleWidth, wrapTextWithAnsi } from "@oh-my-pi/pi-tui"; import { sanitizeText } from "@oh-my-pi/pi-utils"; import type { RenderResultOptions } from "../extensibility/custom-tools/types"; +import { HL_FILE_PREFIX } from "../hashline/hash"; import type { FileDiagnosticsResult } from "../lsp"; import { renderDiff as renderDiffColored } from "../modes/components/diff"; import { getLanguageFromPath, type Theme } from "../modes/theme/theme"; @@ -328,7 +329,6 @@ function getCallPreview( } const MISSING_APPLY_PATCH_END_ERROR = "The last line of the patch must be '*** End Patch'"; -const HL_INPUT_HEADER_PREFIX = "@"; function normalizeHashlineInputPreviewPath(rawPath: string): string { const trimmed = rawPath.trim(); @@ -342,13 +342,11 @@ function normalizeHashlineInputPreviewPath(rawPath: string): string { } function parseHashlineInputPreviewHeader(line: string): string | null { - if (!line.startsWith(HL_INPUT_HEADER_PREFIX)) return null; - // The real parser (`parseHashlineHeaderLine` in `hashline/input.ts`) strips - // every leading "@" before resolving the path so canonical "@@ PATH" headers - // (and stray "@ PATH" / "@@@ PATH" runs) all route to the same file. Mirror - // that here so the renderer doesn't surface a literal "@ " in the title. + if (!line.startsWith(HL_FILE_PREFIX)) return null; + // Mirror hashline/input.ts: strip every leading file marker so canonical + // `§ PATH` headers and stray `§§ PATH` / `§§§PATH` runs render clean paths. let prefixEnd = 0; - while (prefixEnd < line.length && line[prefixEnd] === HL_INPUT_HEADER_PREFIX) prefixEnd++; + while (prefixEnd < line.length && line[prefixEnd] === HL_FILE_PREFIX) prefixEnd++; const body = line.slice(prefixEnd).trim(); const previewPath = normalizeHashlineInputPreviewPath(body); return previewPath.length > 0 ? previewPath : null; diff --git a/packages/coding-agent/src/edit/streaming.ts b/packages/coding-agent/src/edit/streaming.ts index e68ccaf3a..03754d43d 100644 --- a/packages/coding-agent/src/edit/streaming.ts +++ b/packages/coding-agent/src/edit/streaming.ts @@ -22,6 +22,8 @@ import { containsRecognizableHashlineOperations, END_PATCH_MARKER, type HashlineInputSection, + HL_FILE_PREFIX, + HL_OP_CHARS, splitHashlineInputs, } from "../hashline"; import type { Theme } from "../modes/theme/theme"; @@ -77,8 +79,19 @@ const STREAMING_FALLBACK_LINES = 12; const STREAMING_FALLBACK_WIDTH = 80; function isHashlineHeaderLine(line: string): boolean { + return line.trimEnd().startsWith(HL_FILE_PREFIX); +} + +function parseHashlineHeaderPath(line: string): string { const trimmed = line.trimEnd(); - return trimmed.startsWith("@") && trimmed.length > 1; + let prefixEnd = 0; + while (prefixEnd < trimmed.length && trimmed[prefixEnd] === HL_FILE_PREFIX) prefixEnd++; + return trimmed.slice(prefixEnd).trim(); +} + +function isHashlineOpLine(line: string): boolean { + const first = line[0]; + return first !== undefined && HL_OP_CHARS.includes(first); } function isHashlineEnvelopeMarkerLine(line: string): boolean { @@ -358,11 +371,11 @@ function buildApplyPatchNaturalOrderPreviews(input: string): PerFileDiffPreview[ } /** - * Hashline equivalent: emit each section's `~payload` lines as `+added` - * lines in the order the model typed them. We deliberately omit op headers - * and removal targets from the streaming preview because their content - * lives in the file and would require a costly re-apply per tick; the - * complete unified diff is shown once streaming finishes. + * Hashline equivalent: emit each payload line as a `+added` line in the + * order the model typed it. We deliberately omit op headers and removal + * targets from the streaming preview because their content lives in the file + * and would require a costly re-apply per tick; the complete unified diff is + * shown once streaming finishes. */ function buildHashlineNaturalOrderPreviews( input: string, @@ -382,13 +395,12 @@ function buildHashlineNaturalOrderPreviews( for (const raw of lines) { if (isHashlineEnvelopeMarkerLine(raw)) continue; if (isHashlineHeaderLine(raw)) { - currentPath = raw.trimEnd().slice(1).trim(); + currentPath = parseHashlineHeaderPath(raw); if (currentPath) ensure(currentPath); continue; } - if (raw.startsWith("~")) { - ensure(currentPath).push(`+${raw.slice(1)}`); - } + if (isHashlineOpLine(raw) || !currentPath) continue; + ensure(currentPath).push(`+${raw}`); } if (groups.size === 0) return null; const previews: PerFileDiffPreview[] = []; @@ -409,7 +421,7 @@ const hashlineStrategy: EditStreamingStrategy = { if (input.length === 0) return null; if (ctx.isStreaming) { // Skip the costly per-tick re-apply and avoid `Diff.structuredPatch` - // reordering by showing the model's `~payload` lines in input order. + // reordering by showing payload lines in input order. return buildHashlineNaturalOrderPreviews(input, args.path); } ctx.signal.throwIfAborted(); @@ -419,7 +431,7 @@ const hashlineStrategy: EditStreamingStrategy = { sections = splitHashlineInputs(input, { cwd: ctx.cwd, path: args.path }); } catch { // Single-section fallback keeps the original error rendering for the - // "haven't typed `@@ PATH` yet" case. + // "haven't typed `§ PATH` yet" case. const result = await computeHashlineDiff({ input, path: args.path }, ctx.cwd, { autoDropPureInsertDuplicates: ctx.hashlineAutoDropPureInsertDuplicates, }); diff --git a/packages/coding-agent/src/hashline/constants.ts b/packages/coding-agent/src/hashline/constants.ts index e40821991..0172a5290 100644 --- a/packages/coding-agent/src/hashline/constants.ts +++ b/packages/coding-agent/src/hashline/constants.ts @@ -4,9 +4,6 @@ export const MISMATCH_CONTEXT = 2; /** Filler hash used for the interior of a multi-line range; not validated. */ export const RANGE_INTERIOR_HASH = "**"; -/** Header marker introducing a new file section in multi-section input. */ -export const FILE_HEADER_PREFIX = "@"; - /** Optional patch envelope start marker; silently consumed when present. */ export const BEGIN_PATCH_MARKER = "*** Begin Patch"; diff --git a/packages/coding-agent/src/hashline/diff.ts b/packages/coding-agent/src/hashline/diff.ts index 014283a0a..36987e8b5 100644 --- a/packages/coding-agent/src/hashline/diff.ts +++ b/packages/coding-agent/src/hashline/diff.ts @@ -30,7 +30,7 @@ export async function computeHashlineSectionDiff( const rawContent = await readHashlineFileText(Bun.file(absolutePath), absolutePath, section.path); const { text: content } = stripBom(rawContent); const normalized = normalizeToLF(content); - const result = applyHashlineEdits(normalized, parseHashline(section.diff, { path: section.path }), options); + const result = applyHashlineEdits(normalized, parseHashline(section.diff), options); if (normalized === result.lines) return { error: `No changes would be made to ${section.path}.` }; return generateDiffString(normalized, result.lines); } catch (err) { diff --git a/packages/coding-agent/src/hashline/execute.ts b/packages/coding-agent/src/hashline/execute.ts index f502bfb56..1cf666712 100644 --- a/packages/coding-agent/src/hashline/execute.ts +++ b/packages/coding-agent/src/hashline/execute.ts @@ -106,7 +106,7 @@ async function preflightHashlineSection(options: ExecuteHashlineSingleOptions & const { session, path: sectionPath, diff } = options; const absolutePath = resolvePlanPath(session, sectionPath); - const { edits } = parseHashlineWithWarnings(diff, { path: sectionPath }); + const { edits } = parseHashlineWithWarnings(diff); enforcePlanModeWrite(session, sectionPath, { op: "update" }); const source = await readHashlineFile(absolutePath, sectionPath); @@ -139,7 +139,7 @@ async function executeHashlineSection( } = options; const absolutePath = resolvePlanPath(session, sourcePath); - const { edits, warnings: parseWarnings } = parseHashlineWithWarnings(diff, { path: sourcePath }); + const { edits, warnings: parseWarnings } = parseHashlineWithWarnings(diff); enforcePlanModeWrite(session, sourcePath, { op: "update" }); const source = await readHashlineFile(absolutePath, sourcePath); diff --git a/packages/coding-agent/src/hashline/grammar.lark b/packages/coding-agent/src/hashline/grammar.lark index b2d571d40..9d4fa3f7d 100644 --- a/packages/coding-agent/src/hashline/grammar.lark +++ b/packages/coding-agent/src/hashline/grammar.lark @@ -3,20 +3,19 @@ begin_patch: "*** Begin Patch" LF end_patch: "*** End Patch" LF? hunk: update_hunk -update_hunk: "@@ " filename LF line_op* +update_hunk: "$HFILE$" filename LF line_op* filename: /(.+)/ -line_op: insert_before | insert_after | replace | delete | blank -insert_before: ("<" | "< ") anchor LF payload+ -insert_after: ("+" | "+ ") anchor LF payload+ -replace: ("=" | "= ") range LF payload* -delete: ("-" | "- ") range LF -payload: $HSEP$ /(.*)/ LF +line_op: insert_before | insert_after | replace | blank +insert_before: "$HOP_INSERT_BEFORE$" anchor LF payload+ +insert_after: "$HOP_INSERT_AFTER$" anchor LF payload+ +replace: "$HOP_REPLACE$" range LF payload* +payload: /[^$HOP_CHARS$$HFILE$\n][^\n]*/ LF | LF blank: LF anchor: LID | "EOF" | "BOF" -range: LID ".." LID +range: LID (".." LID)? LID: /[1-9]\d*$HFMT$/ %import common.LF diff --git a/packages/coding-agent/src/hashline/hash.ts b/packages/coding-agent/src/hashline/hash.ts index 335174e0c..c0a169b2e 100644 --- a/packages/coding-agent/src/hashline/hash.ts +++ b/packages/coding-agent/src/hashline/hash.ts @@ -75,7 +75,13 @@ export function describeAnchorExamples(linePrefix = ""): string { * pass through unchanged. */ export function resolveHashlineGrammarPlaceholders(grammar: string): string { - return grammar.replaceAll("$HFMT$", "[a-z]{2}").replaceAll("$HSEP$", JSON.stringify(HL_EDIT_SEP)); + return grammar + .replaceAll("$HFMT$", "[a-z]{2}") + .replaceAll("$HOP_INSERT_BEFORE$", HL_OP_INSERT_BEFORE) + .replaceAll("$HOP_INSERT_AFTER$", HL_OP_INSERT_AFTER) + .replaceAll("$HOP_REPLACE$", HL_OP_REPLACE) + .replaceAll("$HOP_CHARS$", HL_OP_CHARS) + .replaceAll("$HFILE$", HL_FILE_PREFIX); } /** @deprecated Use {@link resolveHashlineGrammarPlaceholders}. */ @@ -84,51 +90,23 @@ export const resolveLarkLidPlaceholders = resolveHashlineGrammarPlaceholders; const regexEscape = (str: string): string => str.replace(/[.*+?^${}()|[\]\\]/g, "\\$&"); /** - * Single source of truth for the hashline edit payload separator. This is the - * configured separator that starts inserted/replacement payload lines in - * hashline edit input (`TEXT`) and separates inline modify ops from - * their appended/prepended text. + * Hashline edit input markers. File section headers start with {@link HL_FILE_PREFIX}; + * op lines start with a direction/action sigil: {@link HL_OP_INSERT_BEFORE}, + * {@link HL_OP_INSERT_AFTER}, or {@link HL_OP_REPLACE}. Payload lines are + * verbatim file content and have no per-line marker. * - * Override at runtime with the `PI_HL_SEP` env var (e.g. - * `PI_HL_SEP=">"`, `PI_HL_SEP="\\"`). The value is read once at module load; - * the edit grammar, prompt helper, and edit parser derive from it. - * - * Default is `~`, chosen empirically. Benchmark across 8 candidate separators - * x 3 models (glm-4.7:nitro, gpt-5.4-nano, claude-sonnet-4-6), 24-48 runs per - * cell, hashline variant, 12 sampled tasks per run: - * - * sep | task ✓ | edit ✓ | patch fail | tok/run - * ----|--------|--------|-----------------|-------- - * + | 70.8% | 78.0% | 27/125 (21.6%) | 32,127 - * ÷ | 70.7% | 90.6% | 22/211 (10.4%) | 31,666 - * ~ | 69.4% | 94.9% | 6/107 ( 5.6%) | 30,529 <-- default - * > | 69.2% | 91.5% | 21/219 ( 9.6%) | 30,777 - * : | 66.7% | 86.4% | 20/126 (15.9%) | 33,900 - * | | 65.9% | 86.9% | 20/127 (15.7%) | 34,589 - * \ | 65.5% | 89.8% | 16/124 (12.9%) | 36,010 - * % | 63.9% | 92.8% | 11/125 ( 8.8%) | 36,530 - * - * `~` wins because: - * - highest edit-tool success rate (94.9%) of any tested separator - * - lowest patch-failure rate (5.6%) — model rarely emits a malformed payload - * - cheapest in tokens alongside `>` (no retry overhead from format collisions) - * - no line-leading role in any mainstream language, markdown, diff, regex, - * or shell, so payload lines are unambiguous to both the parser and models - * - task-success is statistically tied with `>` and `÷` (within run-to-run - * noise), so the edit-reliability win is free - * - * `+` and `÷` lead on raw task-success but at the cost of ~2-4x more patch - * failures (the model retries until it lands a valid edit). `:`, `|`, `\` - * collide with line-leading syntax (label/object-key, body separator, escape) - * and degrade both edit reliability and intent-match. + * These constants are the single source of truth for the edit parser, grammar, + * renderer, and prompt. */ -export const HL_EDIT_SEP = (() => { - const sep = process.env.PI_HL_SEP?.trim(); - return sep?.length === 1 ? sep : "~"; -})(); +export const HL_OP_INSERT_BEFORE = "«"; +export const HL_OP_INSERT_AFTER = "»"; +export const HL_OP_REPLACE = "≔"; -/** Regex-escaped form of {@link HL_EDIT_SEP}, safe for regexes. */ -export const HL_EDIT_SEP_RE_RAW = regexEscape(HL_EDIT_SEP); +/** All hashline edit op sigils, concatenated for fast membership tests. */ +export const HL_OP_CHARS = `${HL_OP_INSERT_BEFORE}${HL_OP_INSERT_AFTER}${HL_OP_REPLACE}`; + +/** Hashline edit file section header marker. */ +export const HL_FILE_PREFIX = "§"; /** Stable separator for read/search/hashline display output. Intentionally not configurable. */ export const HL_BODY_SEP = "|"; diff --git a/packages/coding-agent/src/hashline/input.ts b/packages/coding-agent/src/hashline/input.ts index 2bedd0117..d2167d0af 100644 --- a/packages/coding-agent/src/hashline/input.ts +++ b/packages/coding-agent/src/hashline/input.ts @@ -1,8 +1,11 @@ import * as path from "node:path"; -import { ABORT_MARKER, BEGIN_PATCH_MARKER, END_PATCH_MARKER, FILE_HEADER_PREFIX } from "./constants"; -import { HL_EDIT_SEP } from "./hash"; +import { ABORT_MARKER, BEGIN_PATCH_MARKER, END_PATCH_MARKER } from "./constants"; +import { HL_FILE_PREFIX, HL_OP_CHARS } from "./hash"; import type { SplitHashlineOptions } from "./types"; +const regexEscape = (str: string): string => str.replace(/[.*+?^${}()|[\]\\]/g, "\\$&"); +const HASHLINE_OP_LINE_RE = new RegExp(`^[${regexEscape(HL_OP_CHARS)}]`); + export interface HashlineInputSection { path: string; diff: string; @@ -26,19 +29,18 @@ function normalizeHashlinePath(rawPath: string, cwd?: string): string { function parseHashlineHeaderLine(line: string, cwd?: string): HashlineInputSection | null { const trimmed = line.trimEnd(); - if (!trimmed.startsWith(FILE_HEADER_PREFIX)) return null; - // Some models occasionally emit unified-diff-style "@@ path" (or even longer - // runs of "@"). Strip every leading "@" before resolving the path so those - // stray headers still route to the right file. + if (!trimmed.startsWith(HL_FILE_PREFIX)) return null; + // Strip a run of leading header markers so canonical `§PATH` and + // runaway-prefix forms like `§§PATH` / `§§§PATH` route to the same file. let prefixEnd = 0; - while (prefixEnd < trimmed.length && trimmed[prefixEnd] === FILE_HEADER_PREFIX) prefixEnd++; + while (prefixEnd < trimmed.length && trimmed[prefixEnd] === HL_FILE_PREFIX) prefixEnd++; const rest = trimmed.slice(prefixEnd); if (rest.trim().length === 0) { - throw new Error(`Input header "${FILE_HEADER_PREFIX}" is empty; provide a file path.`); + throw new Error(`Input header "${HL_FILE_PREFIX}" is empty; provide a file path.`); } const parsedPath = normalizeHashlinePath(rest, cwd); if (parsedPath.length === 0) { - throw new Error(`Input header "${FILE_HEADER_PREFIX}" is empty; provide a file path.`); + throw new Error(`Input header "${HL_FILE_PREFIX}" is empty; provide a file path.`); } return { path: parsedPath, diff: "" }; } @@ -64,7 +66,7 @@ function stripLeadingBlankLines(input: string): string { export function containsRecognizableHashlineOperations(input: string): boolean { for (const line of input.split(/\r?\n/)) { - if (/^[+<=-]\s+/.test(line) || line.startsWith(HL_EDIT_SEP)) return true; + if (HASHLINE_OP_LINE_RE.test(line)) return true; } return false; } @@ -79,7 +81,7 @@ function normalizeFallbackInput(input: string, options: SplitHashlineOptions): s if (!options.path || !containsRecognizableHashlineOperations(input)) return input; const fallbackPath = normalizeHashlinePath(options.path, options.cwd); if (fallbackPath.length === 0) return input; - return `${FILE_HEADER_PREFIX} ${fallbackPath}\n${input}`; + return `${HL_FILE_PREFIX}${fallbackPath}\n${input}`; } export function splitHashlineInput(input: string, options: SplitHashlineOptions = {}): { path: string; diff: string } { @@ -95,8 +97,8 @@ export function splitHashlineInputs(input: string, options: SplitHashlineOptions if (parseHashlineHeaderLine(firstLine, options.cwd) === null) { const preview = JSON.stringify(firstLine.slice(0, 120)); throw new Error( - `input must begin with "@@ PATH" on the first non-blank line; got: ${preview}. ` + - `Example: "@@ src/foo.ts" then edit ops.`, + `input must begin with "${HL_FILE_PREFIX}PATH" on the first non-blank line; got: ${preview}. ` + + `Example: "${HL_FILE_PREFIX}src/foo.ts" then edit ops.`, ); } diff --git a/packages/coding-agent/src/hashline/parser.ts b/packages/coding-agent/src/hashline/parser.ts index 1b347e8e3..56a9c7a9d 100644 --- a/packages/coding-agent/src/hashline/parser.ts +++ b/packages/coding-agent/src/hashline/parser.ts @@ -1,8 +1,17 @@ import { ABORT_MARKER, ABORT_WARNING, BEGIN_PATCH_MARKER, END_PATCH_MARKER, RANGE_INTERIOR_HASH } from "./constants"; -import { describeAnchorExamples, HL_EDIT_SEP, HL_HASH_CAPTURE_RE_RAW } from "./hash"; +import { + describeAnchorExamples, + HL_FILE_PREFIX, + HL_HASH_CAPTURE_RE_RAW, + HL_OP_CHARS, + HL_OP_INSERT_AFTER, + HL_OP_INSERT_BEFORE, + HL_OP_REPLACE, +} from "./hash"; import type { Anchor, HashlineCursor, HashlineEdit } from "./types"; const LID_CAPTURE_RE = new RegExp(`^${HL_HASH_CAPTURE_RE_RAW}$`); +const regexEscape = (str: string): string => str.replace(/[.*+?^${}()|[\]\\]/g, "\\$&"); function parseLid(raw: string, lineNum: number): Anchor { const match = LID_CAPTURE_RE.exec(raw); @@ -22,12 +31,8 @@ interface ParsedRange { function parseRange(raw: string, lineNum: number): ParsedRange { if (!raw.includes("..")) { - throw new Error( - `line ${lineNum}: explicit ranges are required for delete/replace. ` + - `Repeat the same anchor on both sides for a one-line edit (for example, ` + - `${describeAnchorExamples("119")}..${describeAnchorExamples("119")}); ` + - `got ${JSON.stringify(raw)}.`, - ); + const start = parseLid(raw, lineNum); + return { start, end: { ...start } }; } const [startRaw, endRaw, extra] = raw.split(".."); if (extra !== undefined || !startRaw || !endRaw) { @@ -64,157 +69,61 @@ function parseInsertTarget(raw: string, lineNum: number, kind: "before" | "after return { kind: cursorKind, anchor: parseLid(raw, lineNum) }; } -const INSERT_BEFORE_OP_RE = /^<\s*(\S+)$/; -const INSERT_AFTER_OP_RE = /^\+\s*(\S+)$/; -const DELETE_OP_RE = /^-\s*(\S+)$/; -const REPLACE_OP_RE = /^=\s*(\S+)$/; +const INSERT_BEFORE_OP_RE = new RegExp(`^${regexEscape(HL_OP_INSERT_BEFORE)}\\s*(\\S+)\\s*$`); +const INSERT_AFTER_OP_RE = new RegExp(`^${regexEscape(HL_OP_INSERT_AFTER)}\\s*(\\S+)\\s*$`); +const REPLACE_OP_RE = new RegExp(`^${regexEscape(HL_OP_REPLACE)}\\s*([^\\s+<\\-=]\\S*)\\s*$`); + +function isEnvelopeOrAbortMarkerLine(line: string): boolean { + const trimmed = line.trimEnd(); + return trimmed === BEGIN_PATCH_MARKER || trimmed === END_PATCH_MARKER || trimmed === ABORT_MARKER; +} + +function isPayloadTerminatorLine(line: string): boolean { + const first = line[0]; + return ( + first === HL_FILE_PREFIX || + (first !== undefined && HL_OP_CHARS.includes(first)) || + isEnvelopeOrAbortMarkerLine(line) + ); +} export function cloneCursor(cursor: HashlineCursor): HashlineCursor { if (cursor.kind === "before_anchor") return { kind: "before_anchor", anchor: { ...cursor.anchor } }; if (cursor.kind === "after_anchor") return { kind: "after_anchor", anchor: { ...cursor.anchor } }; return cursor; } -/** - * Returns true when every non-empty payload line looks like the `~ TEXT` readability-padding - * typo: exactly one leading space followed by a non-space character (or a bare single space). - * - * Indented file content (Python 4-space, YAML/JSON/Markdown 2-space, etc.) starts with two or - * more leading spaces, so this heuristic ignores legitimate indentation while still flagging - * the common `~ beta` mistake that silently corrupts file content with a stray space. - */ -function hasUniformSeparatorPadding(payload: string[]): boolean { - let any = false; - for (const text of payload) { - if (text.length === 0) continue; - if (text.charCodeAt(0) !== 0x20) return false; - // Two or more leading spaces is real indentation, not separator padding. - if (text.length > 1 && text.charCodeAt(1) === 0x20) return false; - any = true; - } - return any; -} - -/** - * File extensions where leading single-space indentation is plausible legitimate file content - * (off-side-rule languages, structured-indent data formats, prose with continuation indent). - * For these we suppress the separator-padding warning entirely — the heuristic's false-positive - * cost on a real edit outweighs the rare chance it catches a `~ TEXT` typo. - */ -const INDENT_SENSITIVE_EXTS: Record = { - ".py": true, - ".pyi": true, - ".pyx": true, - ".pyw": true, - ".yml": true, - ".yaml": true, - ".md": true, - ".mdx": true, - ".markdown": true, - ".rst": true, - ".adoc": true, - ".asciidoc": true, - ".toml": true, - ".json": true, - ".jsonc": true, - ".json5": true, - ".ndjson": true, - ".jsonl": true, - ".tf": true, - ".tfvars": true, - ".hcl": true, - ".nix": true, - ".coffee": true, - ".litcoffee": true, - ".haml": true, - ".slim": true, - ".pug": true, - ".jade": true, - ".sass": true, - ".styl": true, - ".nim": true, - ".cr": true, - ".elm": true, - ".fs": true, - ".fsi": true, - ".fsx": true, -}; - -function isIndentationSensitivePath(path: string | undefined): boolean { - if (!path) return false; - const slash = Math.max(path.lastIndexOf("/"), path.lastIndexOf("\\")); - const dot = path.lastIndexOf("."); - if (dot <= slash) return false; - const ext = path.slice(dot).toLowerCase(); - return INDENT_SENSITIVE_EXTS[ext] === true; -} function collectPayload( lines: string[], startIndex: number, opLineNum: number, requirePayload: boolean, - checkPadding: boolean, -): { payload: string[]; nextIndex: number; paddingWarning?: string } { +): { payload: string[]; nextIndex: number } { const payload: string[] = []; let index = startIndex; while (index < lines.length) { const line = lines[index]; - if (line.startsWith(HL_EDIT_SEP)) { - payload.push(line.slice(HL_EDIT_SEP.length).trimEnd()); - index++; - continue; - } - // Silently recover from a missing payload prefix on an otherwise blank - // line: if more payload follows (possibly past further blanks), treat - // each intervening blank as an empty `${HL_EDIT_SEP}` payload line. - // Additionally, when the op explicitly requires payload (`+`/`<`) and - // we have not collected any yet, accept the blank(s) themselves as the - // empty payload — common typo of forgetting the `${HL_EDIT_SEP}` prefix - // when inserting a blank line. - if (line.length === 0) { - let lookahead = index + 1; - while (lookahead < lines.length && lines[lookahead].length === 0) { - lookahead++; - } - const followedByPayload = lookahead < lines.length && lines[lookahead].startsWith(HL_EDIT_SEP); - const acceptBareBlank = requirePayload && payload.length === 0; - if (followedByPayload || acceptBareBlank) { - for (let j = index; j < lookahead; j++) payload.push(""); - index = lookahead; - continue; - } - } - break; + if (isPayloadTerminatorLine(line)) break; + payload.push(line); + index++; } if (payload.length === 0 && requirePayload) { - throw new Error(`line ${opLineNum}: + and < operations require at least one ${HL_EDIT_SEP}TEXT payload line.`); + throw new Error( + `line ${opLineNum}: ${HL_OP_INSERT_BEFORE} and ${HL_OP_INSERT_AFTER} operations require at least one verbatim payload line.`, + ); } - const paddingWarning = - checkPadding && hasUniformSeparatorPadding(payload) - ? `line ${opLineNum}: every payload line begins with exactly one space before non-space content, ` + - `which looks like a readability gap after "${HL_EDIT_SEP}". The space becomes file content. ` + - `Drop it unless the file genuinely uses a one-space indent.` - : undefined; - return { payload, nextIndex: index, paddingWarning }; + return { payload, nextIndex: index }; } -export function parseHashline(diff: string, opts: ParseHashlineOptions = {}): HashlineEdit[] { - return parseHashlineWithWarnings(diff, opts).edits; +export function parseHashline(diff: string): HashlineEdit[] { + return parseHashlineWithWarnings(diff).edits; } -export interface ParseHashlineOptions { - /** File path the diff targets. Used to suppress indent-sensitive false-positive warnings. */ - path?: string; -} - -export function parseHashlineWithWarnings( - diff: string, - opts: ParseHashlineOptions = {}, -): { edits: HashlineEdit[]; warnings: string[] } { +export function parseHashlineWithWarnings(diff: string): { edits: HashlineEdit[]; warnings: string[] } { const edits: HashlineEdit[] = []; const warnings: string[] = []; const lines = diff.split(/\r?\n/); - const checkPadding = !isIndentationSensitivePath(opts.path); + if (diff.endsWith("\n") && lines.at(-1) === "") lines.pop(); let editIndex = 0; const pushInsert = (cursor: HashlineCursor, text: string, lineNum: number) => { @@ -240,15 +149,11 @@ export function parseHashlineWithWarnings( i++; continue; } - if (line.startsWith(HL_EDIT_SEP)) { - throw new Error(`line ${lineNum}: payload line has no preceding +, <, or = operation.`); - } const insertBeforeMatch = INSERT_BEFORE_OP_RE.exec(line); if (insertBeforeMatch) { const cursor = parseInsertTarget(insertBeforeMatch[1], lineNum, "before"); - const { payload, nextIndex, paddingWarning } = collectPayload(lines, i + 1, lineNum, true, checkPadding); - if (paddingWarning) warnings.push(paddingWarning); + const { payload, nextIndex } = collectPayload(lines, i + 1, lineNum, true); for (const text of payload) pushInsert(cursor, text, lineNum); i = nextIndex; continue; @@ -257,37 +162,26 @@ export function parseHashlineWithWarnings( const insertAfterMatch = INSERT_AFTER_OP_RE.exec(line); if (insertAfterMatch) { const cursor = parseInsertTarget(insertAfterMatch[1], lineNum, "after"); - const { payload, nextIndex, paddingWarning } = collectPayload(lines, i + 1, lineNum, true, checkPadding); - if (paddingWarning) warnings.push(paddingWarning); + const { payload, nextIndex } = collectPayload(lines, i + 1, lineNum, true); for (const text of payload) pushInsert(cursor, text, lineNum); i = nextIndex; continue; } - const deleteMatch = DELETE_OP_RE.exec(line); - if (deleteMatch) { - for (const anchor of expandRange(parseRange(deleteMatch[1], lineNum))) { - edits.push({ kind: "delete", anchor, lineNum, index: editIndex++ }); - } - i++; - continue; - } - const replaceMatch = REPLACE_OP_RE.exec(line); if (replaceMatch) { const range = parseRange(replaceMatch[1], lineNum); - const { payload, nextIndex, paddingWarning } = collectPayload(lines, i + 1, lineNum, false, checkPadding); - if (paddingWarning) warnings.push(paddingWarning); - // `= A..B` with no payload blanks the range to a single empty line. - const replacement = payload.length === 0 ? [""] : payload; - for (const text of replacement) { - edits.push({ - kind: "insert", - cursor: { kind: "before_anchor", anchor: { ...range.start } }, - text, - lineNum, - index: editIndex++, - }); + const { payload, nextIndex } = collectPayload(lines, i + 1, lineNum, false); + if (payload.length > 0) { + for (const text of payload) { + edits.push({ + kind: "insert", + cursor: { kind: "before_anchor", anchor: { ...range.start } }, + text, + lineNum, + index: editIndex++, + }); + } } for (const anchor of expandRange(range)) { edits.push({ kind: "delete", anchor, lineNum, index: editIndex++ }); @@ -296,8 +190,15 @@ export function parseHashlineWithWarnings( continue; } + if (isPayloadTerminatorLine(line) || /^[-@\u00B6]/u.test(line)) { + throw new Error( + `line ${lineNum}: unrecognized op. Use ${HL_OP_INSERT_BEFORE}ANCHOR (insert before), ${HL_OP_INSERT_AFTER}ANCHOR (insert after), or ${HL_OP_REPLACE}A..B (replace/delete). ` + + `Got ${JSON.stringify(line)}.`, + ); + } + throw new Error( - `line ${lineNum}: unrecognized op. Use < ANCHOR (insert before), + ANCHOR (insert after), - A..B (delete), = A..B (replace), or "${HL_EDIT_SEP}TEXT" payload lines. ` + + `line ${lineNum}: payload line has no preceding ${HL_OP_INSERT_BEFORE}, ${HL_OP_INSERT_AFTER}, or ${HL_OP_REPLACE} operation. ` + `Got ${JSON.stringify(line)}.`, ); } diff --git a/packages/coding-agent/src/prompts/tools/hashline.md b/packages/coding-agent/src/prompts/tools/hashline.md index 7ec445d63..f158b7046 100644 --- a/packages/coding-agent/src/prompts/tools/hashline.md +++ b/packages/coding-agent/src/prompts/tools/hashline.md @@ -1,58 +1,36 @@ Your patch language is a compact, line-anchored edit format. -A patch contains one or more file sections. The first non-blank line of every edit section MUST be `@@ PATH`. +A patch contains one or more file sections. The first non-blank line of every edit section MUST be `§PATH`. Operations reference lines in the file by their line number and hash, called "Anchors", e.g. `5th`, `123ab`. You MUST copy them verbatim from the latest output for the file you're editing. Purely textual format. The tool has NO awareness of language, indentation, brackets, fences, or table widths. You MUST emit valid syntax in replacements/insertions. -@@ PATH header: subsequent ops apply to PATH +§PATH header: subsequent ops apply to PATH Each op line is ONE of: -+ ANCHOR insert lines AFTER the anchored line (or EOF); payload follows as `{{hsep}}TEXT` lines -< ANCHOR insert lines BEFORE the anchored line (or BOF); payload follows as `{{hsep}}TEXT` lines -- A..B delete the line range (inclusive). -= A..B replace the range with payload `{{hsep}}TEXT` lines, or with one blank line if no payload follows. +»ANCHOR insert lines AFTER the anchored line (or EOF); payload follows on subsequent lines +«ANCHOR insert lines BEFORE the anchored line (or BOF); payload follows on subsequent lines +≔A..B replace the inclusive range A..B with payload; delete the range if no payload follows +≔A shorthand for ≔A..A - -Op lines carry no content — payload goes on the next line. - -WRONG: + 5pg| some code -WRONG: {{hsep}} some code -RIGHT: + 5pg -{{hsep}}some code - -A single `+`/`<`/`=` op accepts MANY `{{hsep}}` payload lines. To insert N consecutive lines, write ONE op followed by N payload lines — NEVER N ops with one payload each. - -WRONG (one op per inserted line, with fabricated anchors): - + 5pg - {{hsep}}first new line - + 6xx ← FABRICATED - {{hsep}}second new line - -RIGHT (one op, many payload lines): - + 5pg - {{hsep}}first new line - {{hsep}}second new line - - -- Every payload line MUST start with `{{hsep}}` immediately followed by payload text. Do NOT add a readability space after `{{hsep}}`. -- Every character after `{{hsep}}` is file content. If the target line intentionally starts with one space, write exactly one space after `{{hsep}}`; otherwise write none. - Payload text is verbatim — NEVER escape unicode. +- Payload ends at the next `»`, `«`, `≔`, `§`, envelope marker, or EOF. +- `≔A..B` with no payload deletes the range. To keep a blank line, include one explicit empty payload line. - **Payload is only what's NEW relative to your range:** - - `=` replaces inside; NEVER include lines outside. - - `+`/`<` adds at the anchor; NEVER repeat line A or neighbors. + - `≔` replaces inside; NEVER include lines outside. + - `»`/`«` adds at the anchor; NEVER repeat line A or neighbors. - Payload matching nearby content duplicates — drop it or widen. - **Pick a self-contained unit first.** Touching a multiline construct? Widen to the whole thing. -- Then smallest op: add → `+`/`<`; delete → `-`; `=` ONLY when modifying inside. +- Then smallest op: add → `»`/`«`; delete/replace → `≔`. When braces bound your edit, you SHOULD prefer these shapes: - **Whole block**: range spans `{` through matching `}`. -- **Signature only**: one-line `=` on the opener; body untouched. +- **Signature only**: one-line `≔` on the opener; body untouched. - **Insert inside**: anchor on `{` or last interior line; NEVER repeat the braces. - **End on `}`**: only when that `}` is part of the change. Otherwise extend or stop earlier. @@ -61,9 +39,9 @@ When braces bound your edit, you SHOULD prefer these shapes: - **NEVER replay past your range.** Stop before B+1; extend B if it must go. - **NEVER duplicate chunks inside one payload.** Caught re-emitting? Rewrite. - **Anchor only inside the visible region.** B+1 truncated? Re-`read` first. -- **You SHOULD prefer the narrowest self-contained edit.** Small `+`/`-` beats wide `=`. +- **You SHOULD prefer the narrowest self-contained edit.** Narrow range beats wide range. - **Anchors reference the file as last read.** NEVER shift for prior ops. -- **One `+`/`<` op per block, NOT per line.** N lines = ONE op, N payloads. Collapse adjacent ops. +- **One `»`/`«` op per block, NOT per line.** N lines = ONE op, N payloads. Collapse adjacent ops. - **NEVER fabricate anchor hashes.** Missing? Re-`read`. @@ -79,71 +57,74 @@ When braces bound your edit, you SHOULD prefer these shapes: # Replace one line (the payload must re-emit the original indentation) -@@ mod.ts -= {{hrefr 1}}..{{hrefr 1}} -{{hsep}}const TITLE = "Mrs"; +§mod.ts +≔{{hrefr 1}} +const TITLE = "Mrs"; # Replace a full multiline statement (widen to a self-contained boundary) -@@ mod.ts -= {{hrefr 3}}..{{hrefr 6}} -{{hsep}} return [ -{{hsep}} "Mrs", -{{hsep}} name?.trim() || "guest", -{{hsep}} ].join(" "); +§mod.ts +≔{{hrefr 3}}..{{hrefr 6}} + return [ + "Mrs", + name?.trim() || "guest", + ].join(" "); # Insert AFTER/BEFORE a line -@@ mod.ts -+ {{hrefr 4}} -{{hsep}} "Dr", -< {{hrefr 5}} -{{hsep}} "Dr", +§mod.ts +»{{hrefr 4}} + "Dr", +«{{hrefr 5}} + "Dr", # Append to file -@@ mod.ts -+ EOF -{{hsep}}export const done = true; +§mod.ts +»EOF +export const done = true; # Delete a line -@@ mod.ts -- {{hrefr 5}}..{{hrefr 5}} +§mod.ts +≔{{hrefr 5}} -# Blank a line (replace with LF) -@@ mod.ts -= {{hrefr 5}}..{{hrefr 5}} +# Blank a line (replace with LF: the empty payload is the blank line before `»EOF`) +§mod.ts +≔{{hrefr 5}} + +»EOF +export const done = true; # WRONG — replaces 2 lines just to add one. -@@ mod.ts -= {{hrefr 1}}..{{hrefr 2}} -{{hsep}}const TITLE = "Mr"; -{{hsep}}const DEBUG = false; -{{hsep}}export function greet(name) { +§mod.ts +≔{{hrefr 1}}..{{hrefr 2}} +const TITLE = "Mr"; +const DEBUG = false; +export function greet(name) { # RIGHT — same effect, one-line insert -@@ mod.ts -+ {{hrefr 1}} -{{hsep}}const DEBUG = false; +§mod.ts +»{{hrefr 1}} +const DEBUG = false; # WRONG — replace from the middle of a larger statement (error-prone) -@@ mod.ts -= {{hrefr 4}}..{{hrefr 5}} -{{hsep}} "Dr", -{{hsep}} name?.trim() || "guest", +§mod.ts +≔{{hrefr 4}}..{{hrefr 5}} + "Dr", + name?.trim() || "guest", # RIGHT — widen to the full statement -@@ mod.ts -= {{hrefr 3}}..{{hrefr 6}} -{{hsep}} return [ -{{hsep}} "Dr", -{{hsep}} name?.trim() || "guest", -{{hsep}} ].join(" "); +§mod.ts +≔{{hrefr 3}}..{{hrefr 6}} + return [ + "Dr", + name?.trim() || "guest", + ].join(" "); - Copy anchors verbatim (line number + 2-char hash); NEVER include the `|TEXT` body. -- Every payload line MUST start with `{{hsep}}`; raw content is invalid. -- NEVER write unified diff syntax. Header is `@@ PATH`; ops are `<`/`+`/`-`/`=`. -- `= A..B` deletes the range; payload is what's written. Edge line matches just outside? Widen, or it duplicates. -- Multiple ops are cheap. SHOULD prefer two narrow ops over one wide `=`. - - Before `= A..B`, mentally delete A..B. Splits an unclosed bracket/brace/string from above, or orphans a closer inside? You're bisecting a construct. +- NEVER write unified diff syntax. Headers are `§PATH`; ops are `»`/`«`/`≔`. +- `≔A..B` deletes the range when no payload follows. To keep a blank line, include one explicit empty payload line. +- `≔A..B` with payload writes exactly that payload. Edge line matches just outside? Widen, or it duplicates. +- Multiple ops are cheap. SHOULD prefer two narrow ops over one wide `≔`. + - Before `≔A..B`, mentally delete A..B. Splits an unclosed bracket/brace/string from above, or orphans a closer inside? You're bisecting a construct. - NEVER use this tool to reformat code (indentation, whitespace, line wrapping, style). Run the project's formatter instead. diff --git a/packages/coding-agent/test/core/hashline.test.ts b/packages/coding-agent/test/core/hashline.test.ts index 871ad3d6f..8c6130fc2 100644 --- a/packages/coding-agent/test/core/hashline.test.ts +++ b/packages/coding-agent/test/core/hashline.test.ts @@ -15,7 +15,6 @@ import { HashlineMismatchError, HL_BODY_SEP, HL_BODY_SEP_RE_RAW, - HL_EDIT_SEP, hashlineEditParamsSchema, parseHashline, parseHashlineWithWarnings, @@ -30,12 +29,7 @@ beforeAll(async () => { await Settings.init({ inMemory: true, cwd: process.cwd() }); }); -// Single source of truth for the payload separator under test. Every literal -// payload line in this file goes through `pl()` so flipping -// `HL_EDIT_SEP` (e.g. to ">" or "\\") flips the test inputs in -// lockstep without any `|`-vs-`>` churn. -const sep = HL_EDIT_SEP; -const pl = (text: string): string => `${sep}${text}`; +const pl = (text: string): string => text; const outputSep = HL_BODY_SEP; const outputSepRe = HL_BODY_SEP_RE_RAW; @@ -99,70 +93,71 @@ describe("hashline parser — block op syntax", () => { it("inserts payload before/after a Lid, and at BOF/EOF", () => { const diff = [ - `< ${tag(2, "bbb")}`, + `«${tag(2, "bbb")}`, pl("before b"), - `+ ${tag(2, "bbb")}`, + `»${tag(2, "bbb")}`, pl("after b"), - "+ BOF", + "»BOF", pl("top"), - "+ EOF", + "»EOF", pl("tail"), ].join("\n"); expect(applyDiff(content, diff)).toBe("top\naaa\nbefore b\nbbb\nafter b\nccc\ntail"); }); - it("inserts after the final line via `+ ANCHOR` instead of falling off the file", () => { - const diff = [`+ ${tag(3, "ccc")}`, pl("tail")].join("\n"); + it("inserts after the final line via `»ANCHOR` instead of falling off the file", () => { + const diff = [`»${tag(3, "ccc")}`, pl("tail")].join("\n"); expect(applyDiff(content, diff)).toBe("aaa\nbbb\nccc\ntail"); }); - it("blanks a line in place when `= A..A` has no payload", () => { - const diff = `= ${sameLineRange(tag(2, "bbb"))}`; + it("deletes one line or an inclusive range when `≔A..B` has no payload", () => { + expect(applyDiff(content, `≔${sameLineRange(tag(2, "bbb"))}`)).toBe("aaa\nccc"); + expect(applyDiff(content, `≔${tag(2, "bbb")}..${tag(3, "ccc")}`)).toBe("aaa"); + }); + + it("blanks a line in place with an explicit empty payload line", () => { + const diff = `≔${sameLineRange(tag(2, "bbb"))}\n\n`; expect(applyDiff(content, diff)).toBe("aaa\n\nccc"); }); - it("blanks a range to a single empty line when `= A..B` has no payload", () => { - const diff = `= ${tag(1, "aaa")}..${tag(2, "bbb")}`; - expect(applyDiff(content, diff)).toBe("\nccc"); - }); - - it("deletes one line or an inclusive range", () => { - expect(applyDiff(content, `- ${sameLineRange(tag(2, "bbb"))}`)).toBe("aaa\nccc"); - expect(applyDiff(content, `- ${tag(2, "bbb")}..${tag(3, "ccc")}`)).toBe("aaa"); - }); - it("replaces one line or an inclusive range with payload lines", () => { - const single = [`= ${sameLineRange(tag(2, "bbb"))}`, pl("BBB")].join("\n"); + const single = [`≔${tag(2, "bbb")}`, pl("BBB")].join("\n"); expect(applyDiff(content, single)).toBe("aaa\nBBB\nccc"); - const range = [`= ${tag(2, "bbb")}..${tag(3, "ccc")}`, pl("BBB"), pl("CCC")].join("\n"); + const range = [`≔${tag(2, "bbb")}..${tag(3, "ccc")}`, pl("BBB"), pl("CCC")].join("\n"); expect(applyDiff(content, range)).toBe("aaa\nBBB\nCCC"); }); + it("treats single-anchor replace sugar as equivalent to an explicit one-line range", () => { + const anchor = tag(2, "bbb"); + expect(parseHashline(`≔${anchor}\nBBB`)).toEqual(parseHashline(`≔${anchor}..${anchor}\nBBB`)); + expect(applyDiff(content, `≔${anchor}\nBBB`)).toBe(applyDiff(content, `≔${anchor}..${anchor}\nBBB`)); + }); + it("auto-absorbs duplicated multiline prefix boundaries during replacement", () => { const source = ["// one", "// two", "old();"].join("\n"); - const diff = [`= ${sameLineRange(tag(3, "old();"))}`, pl("// one"), pl("// two"), pl("new();")].join("\n"); + const diff = [`≔${sameLineRange(tag(3, "old();"))}`, pl("// one"), pl("// two"), pl("new();")].join("\n"); expect(applyDiff(source, diff)).toBe(["// one", "// two", "new();"].join("\n")); }); it("auto-absorbs duplicated multiline suffix boundaries during replacement", () => { const source = ["old();", "// one", "// two"].join("\n"); - const diff = [`= ${sameLineRange(tag(1, "old();"))}`, pl("new();"), pl("// one"), pl("// two")].join("\n"); + const diff = [`≔${sameLineRange(tag(1, "old();"))}`, pl("new();"), pl("// one"), pl("// two")].join("\n"); expect(applyDiff(source, diff)).toBe(["new();", "// one", "// two"].join("\n")); }); it("auto-absorbs a duplicated single structural suffix during replacement", () => { const source = ["old();", "};"].join("\n"); - const diff = [`= ${sameLineRange(tag(1, "old();"))}`, pl("new();"), pl("};")].join("\n"); + const diff = [`≔${sameLineRange(tag(1, "old();"))}`, pl("new();"), pl("};")].join("\n"); expect(applyDiff(source, diff)).toBe(["new();", "};"].join("\n")); }); it("auto-absorbs a duplicated single structural prefix during replacement", () => { const source = ["};", "old();"].join("\n"); - const diff = [`= ${sameLineRange(tag(2, "old();"))}`, pl("};"), pl("new();")].join("\n"); + const diff = [`≔${sameLineRange(tag(2, "old();"))}`, pl("};"), pl("new();")].join("\n"); expect(applyDiff(source, diff)).toBe(["};", "new();"].join("\n")); }); @@ -172,14 +167,14 @@ describe("hashline parser — block op syntax", () => { // `}` is a legitimate part of the new block, not a duplicate of the file's // existing `}`. The single-line structural absorb must NOT fire here. const source = ["old();", "}"].join("\n"); - const diff = [`= ${sameLineRange(tag(1, "old();"))}`, pl("if ok {"), pl("}")].join("\n"); + const diff = [`≔${sameLineRange(tag(1, "old();"))}`, pl("if ok {"), pl("}")].join("\n"); expect(applyDiff(source, diff)).toBe(["if ok {", "}", "}"].join("\n")); }); it("does not auto-absorb a single duplicated boundary line", () => { const source = ["keep", "old();"].join("\n"); - const diff = [`= ${sameLineRange(tag(2, "old();"))}`, pl("keep"), pl("new();")].join("\n"); + const diff = [`≔${sameLineRange(tag(2, "old();"))}`, pl("keep"), pl("new();")].join("\n"); expect(applyDiff(source, diff)).toBe(["keep", "keep", "new();"].join("\n")); }); @@ -190,11 +185,11 @@ describe("hashline parser — block op syntax", () => { // steal that anchor and turn the insert into a replacement. const source = ["A", "B", "X", "Y", "Z"].join("\n"); const diff = [ - `= ${tag(1, "A")}..${tag(2, "B")}`, + `≔${tag(1, "A")}..${tag(2, "B")}`, pl("alpha"), pl("X"), pl("Y"), - `< ${tag(4, "Y")}`, + `«${tag(4, "Y")}`, pl("extra"), ].join("\n"); @@ -203,7 +198,7 @@ describe("hashline parser — block op syntax", () => { it("surfaces a warning when boundary duplicates are auto-absorbed", () => { const source = ["// one", "// two", "old();"].join("\n"); - const diff = [`= ${sameLineRange(tag(3, "old();"))}`, pl("// one"), pl("// two"), pl("new();")].join("\n"); + const diff = [`≔${sameLineRange(tag(3, "old();"))}`, pl("// one"), pl("// two"), pl("new();")].join("\n"); const result = applyHashlineEdits(source, parseHashline(diff)); expect(result.lines).toBe(["// one", "// two", "new();"].join("\n")); @@ -218,94 +213,93 @@ describe("hashline parser — block op syntax", () => { // `autoDropPureInsertDuplicates` opt-in, unlike the single-line // structural absorb covered by the test below. const source = ["aaa", "bbb", "ccc"].join("\n"); - const diff = [`+ ${tag(2, "bbb")}`, pl("aaa"), pl("bbb"), pl("NEW")].join("\n"); + const diff = [`»${tag(2, "bbb")}`, pl("aaa"), pl("bbb"), pl("NEW")].join("\n"); expect(applyDiff(source, diff)).toBe("aaa\nbbb\naaa\nbbb\nNEW\nccc"); }); it("auto-drops a duplicated single structural suffix for pure insert by default", () => { const source = ["if ok {", " keep();", " }"].join("\n"); - const diff = [`< ${tag(3, " }")}`, pl(" added();"), pl(" }")].join("\n"); + const diff = [`«${tag(3, " }")}`, pl(" added();"), pl(" }")].join("\n"); expect(applyDiff(source, diff)).toBe(["if ok {", " keep();", " added();", " }"].join("\n")); }); it("auto-drops a duplicated single structural prefix for pure insert by default", () => { const source = [" });", "next();"].join("\n"); - const diff = [`+ ${tag(1, " });")}`, pl(" });"), pl("added();")].join("\n"); + const diff = [`»${tag(1, " });")}`, pl(" });"), pl("added();")].join("\n"); expect(applyDiff(source, diff)).toBe([" });", "added();", "next();"].join("\n")); }); - it("preserves an intentional non-structural anchor duplicate for `+ ANCHOR` by default", () => { + it("preserves an intentional non-structural anchor duplicate for `»ANCHOR` by default", () => { const source = ["aaa", "bbb", "ccc"].join("\n"); - const diff = [`+ ${tag(2, "bbb")}`, pl("bbb"), pl("NEW")].join("\n"); + const diff = [`»${tag(2, "bbb")}`, pl("bbb"), pl("NEW")].join("\n"); expect(applyDiff(source, diff)).toBe("aaa\nbbb\nbbb\nNEW\nccc"); }); - it("preserves an intentional non-structural anchor duplicate for `< ANCHOR` by default", () => { + it("preserves an intentional non-structural anchor duplicate for `«ANCHOR` by default", () => { const source = ["aaa", "bbb", "ccc"].join("\n"); - const diff = [`< ${tag(2, "bbb")}`, pl("NEW"), pl("bbb")].join("\n"); + const diff = [`«${tag(2, "bbb")}`, pl("NEW"), pl("bbb")].join("\n"); expect(applyDiff(source, diff)).toBe("aaa\nNEW\nbbb\nbbb\nccc"); }); it("does not drop a single structural pure-insert suffix when it preserves balance", () => { const source = ["if outer {", "}"].join("\n"); - const diff = [`< ${tag(2, "}")}`, pl("if inner {"), pl("}")].join("\n"); + const diff = [`«${tag(2, "}")}`, pl("if inner {"), pl("}")].join("\n"); expect(applyDiff(source, diff)).toBe(["if outer {", "if inner {", "}", "}"].join("\n")); }); - it("auto-absorbs duplicated leading payload of a pure `+ ANCHOR` insert", () => { - // `+ 2 ~aaa ~bbb ~NEW`: payload echoes the two file lines AT/ABOVE the - // insertion point (aaa, bbb), then adds NEW. The leading echo is absorbed. + it("auto-absorbs duplicated leading payload of a pure `»ANCHOR` insert", () => { + // Payload echoes the two file lines AT/ABOVE the insertion point + // (aaa, bbb), then adds NEW. The leading echo is absorbed. const source = ["aaa", "bbb", "ccc"].join("\n"); - const diff = [`+ ${tag(2, "bbb")}`, pl("aaa"), pl("bbb"), pl("NEW")].join("\n"); + const diff = [`»${tag(2, "bbb")}`, pl("aaa"), pl("bbb"), pl("NEW")].join("\n"); expect(applyDiffWithPureInsertAutoDrop(source, diff)).toBe("aaa\nbbb\nNEW\nccc"); }); - it("auto-absorbs context-wrap echo (leading-above + trailing-below) on `+ ANCHOR`", () => { - // `+ 2 ~aaa ~bbb ~NEW ~ccc ~ddd`: payload wraps NEW with context above - // (aaa, bbb) AND below (ccc, ddd). Both ends should be absorbed, leaving - // only NEW inserted after bbb. + it("auto-absorbs context-wrap echo (leading-above + trailing-below) on `»ANCHOR`", () => { + // Payload wraps NEW with context above (aaa, bbb) AND below (ccc, ddd). + // Both ends should be absorbed, leaving only NEW inserted after bbb. const source = ["aaa", "bbb", "ccc", "ddd"].join("\n"); - const diff = [`+ ${tag(2, "bbb")}`, pl("aaa"), pl("bbb"), pl("NEW"), pl("ccc"), pl("ddd")].join("\n"); + const diff = [`»${tag(2, "bbb")}`, pl("aaa"), pl("bbb"), pl("NEW"), pl("ccc"), pl("ddd")].join("\n"); expect(applyDiffWithPureInsertAutoDrop(source, diff)).toBe("aaa\nbbb\nNEW\nccc\nddd"); }); - it("auto-absorbs duplicated trailing payload of a pure `< ANCHOR` insert", () => { + it("auto-absorbs duplicated trailing payload of a pure `«ANCHOR` insert", () => { // Insert before line 3 ("ccc"). Trailing payload echoes the anchor and the // line after it. Drop the trailing duplicates. const source = ["aaa", "bbb", "ccc", "ddd"].join("\n"); - const diff = [`< ${tag(3, "ccc")}`, pl("NEW"), pl("ccc"), pl("ddd")].join("\n"); + const diff = [`«${tag(3, "ccc")}`, pl("NEW"), pl("ccc"), pl("ddd")].join("\n"); expect(applyDiffWithPureInsertAutoDrop(source, diff)).toBe("aaa\nbbb\nNEW\nccc\nddd"); }); it("auto-absorbs duplicated leading payload at EOF insert", () => { const source = ["aaa", "bbb", "ccc"].join("\n"); - // `+ EOF` payload echoes the last two file lines, then adds NEW. - const diff = ["+ EOF", pl("bbb"), pl("ccc"), pl("NEW")].join("\n"); + // `»EOF` payload echoes the last two file lines, then adds NEW. + const diff = ["»EOF", pl("bbb"), pl("ccc"), pl("NEW")].join("\n"); expect(applyDiffWithPureInsertAutoDrop(source, diff)).toBe("aaa\nbbb\nccc\nNEW"); }); it("auto-absorbs duplicated trailing payload at BOF insert", () => { const source = ["aaa", "bbb", "ccc"].join("\n"); - // `< BOF` payload prepends NEW but trails with the first two file lines. - const diff = ["< BOF", pl("NEW"), pl("aaa"), pl("bbb")].join("\n"); + // `«BOF` payload prepends NEW but trails with the first two file lines. + const diff = ["«BOF", pl("NEW"), pl("aaa"), pl("bbb")].join("\n"); expect(applyDiffWithPureInsertAutoDrop(source, diff)).toBe("NEW\naaa\nbbb\nccc"); }); it("auto-drops a single duplicated anchor line in a pure insert when generic duplicate absorption is enabled", () => { const source = ["aaa", "bbb", "ccc"].join("\n"); - const diff = [`+ ${tag(2, "bbb")}`, pl("bbb"), pl("NEW")].join("\n"); + const diff = [`»${tag(2, "bbb")}`, pl("bbb"), pl("NEW")].join("\n"); expect(applyDiffWithPureInsertAutoDrop(source, diff)).toBe("aaa\nbbb\nNEW\nccc"); }); it("surfaces a warning when pure-insert duplicates are auto-dropped", () => { const source = ["aaa", "bbb", "ccc"].join("\n"); - const diff = [`+ ${tag(2, "bbb")}`, pl("aaa"), pl("bbb"), pl("NEW")].join("\n"); + const diff = [`»${tag(2, "bbb")}`, pl("aaa"), pl("bbb"), pl("NEW")].join("\n"); const result = applyHashlineEdits(source, parseHashline(diff), { autoDropPureInsertDuplicates: true }); expect(result.lines).toBe("aaa\nbbb\nNEW\nccc"); expect(result.warnings).toBeDefined(); @@ -314,9 +308,9 @@ describe("hashline parser — block op syntax", () => { ); }); - it("preserves payload text exactly after the first separator", () => { + it("preserves payload text exactly", () => { const diff = [ - `= ${sameLineRange(tag(2, "bbb"))}`, + `≔${sameLineRange(tag(2, "bbb"))}`, pl(""), pl("# not a header"), pl("+ not an op"), @@ -326,64 +320,63 @@ describe("hashline parser — block op syntax", () => { }); it("treats blank lines inside a payload run as empty payload lines", () => { - // Truly blank lines (no leading separator) inside an active payload run - // are silently rewritten to empty payload lines as long as more payload - // follows. This recovers from a common typo where the model forgets the - // separator on what should be a blank inserted line. - const diff = [`= ${sameLineRange(tag(2, "bbb"))}`, pl("first"), "", "", pl("after")].join("\n"); + // Truly blank lines inside an active payload run are verbatim empty + // payload lines as long as more payload follows. + const diff = [`≔${sameLineRange(tag(2, "bbb"))}`, pl("first"), "", "", pl("after")].join("\n"); expect(applyDiff(content, diff)).toBe("aaa\nfirst\n\n\nafter\nccc"); }); - it("does not consume trailing blank lines between sections as payload", () => { - // Blanks that precede a non-payload op (here, another `=`) end the - // payload run cleanly — they're section separators, not payload. + it("treats blank lines before the next op as payload", () => { const diff = [ - `= ${sameLineRange(tag(1, "aaa"))}`, + `≔${sameLineRange(tag(1, "aaa"))}`, pl("AAA"), "", "", - `= ${sameLineRange(tag(3, "ccc"))}`, + `≔${sameLineRange(tag(3, "ccc"))}`, pl("CCC"), ].join("\n"); - expect(applyDiff(content, diff)).toBe("AAA\nbbb\nCCC"); + expect(applyDiff(content, diff)).toBe("AAA\n\n\nbbb\nCCC"); }); it("rejects missing payloads and orphan payload lines", () => { - expect(() => parseHashline(`+ ${tag(1, "aaa")}`)).toThrow(/require at least one/); + expect(() => parseHashline(`»${tag(1, "aaa")}`)).toThrow(/require at least one/); expect(() => parseHashline(pl("orphan"))).toThrow(/payload line has no preceding/); }); - it("leniently treats a bare blank line after + / < as an empty payload", () => { + it("leniently treats a bare blank line after « / » as an empty payload", () => { const hash = computeLineHash(5, "aaa"); const anchor = { line: 5, hash }; - expect(parseHashline(`< ${tag(5, "aaa")}\n`)).toEqual([ + expect(parseHashline(`«${tag(5, "aaa")}\n\n`)).toEqual([ { kind: "insert", cursor: { kind: "before_anchor", anchor }, text: "", lineNum: 1, index: 0 }, ]); - expect(parseHashline(`+ ${tag(5, "aaa")}\n`)).toEqual([ + expect(parseHashline(`»${tag(5, "aaa")}\n\n`)).toEqual([ { kind: "insert", cursor: { kind: "after_anchor", anchor }, text: "", lineNum: 1, index: 0 }, ]); }); it("rejects old cursor and equals-inline syntax after cutover", () => { expect(() => parseHashline(`@${tag(1, "aaa")}\n+old`)).toThrow(/unrecognized op/); - expect(() => parseHashline(`${tag(1, "aaa")}=AAA`)).toThrow(/unrecognized op/); + expect(() => parseHashline(`${tag(1, "aaa")}=AAA`)).toThrow(/payload line has no preceding/); }); - it("rejects single-anchor delete/replace shorthand", () => { - expect(() => parseHashline(`= ${tag(2, "bbb")}\n${pl("BBB")}`)).toThrow(/explicit ranges are required/); - expect(() => parseHashline(`- ${tag(2, "bbb")}`)).toThrow(/explicit ranges are required/); + it("rejects the retired delete op", () => { + expect(() => parseHashline(`≔-${sameLineRange(tag(2, "bbb"))}`)).toThrow(/unrecognized op/); + }); + + it("describes current sigils for unknown op syntax", () => { + expect(() => parseHashline(`-${sameLineRange(tag(2, "bbb"))}`)).toThrow(/Use «ANCHOR.*»ANCHOR.*≔A\.\.B/); }); }); describe("hashline — stale anchors", () => { it("throws HashlineMismatchError when a Lid hash no longer matches", () => { - const diff = [`= ${sameLineRange(mistag(2, "bbb"))}`, pl("BBB")].join("\n"); + const diff = [`≔${sameLineRange(mistag(2, "bbb"))}`, pl("BBB")].join("\n"); expect(() => applyDiff("aaa\nbbb\nccc", diff)).toThrow(HashlineMismatchError); }); it("rejects when an anchor's stored line shifted (no auto-rebase)", () => { const stale = tag(2, "bbb"); - const diff = [`= ${sameLineRange(stale)}`, pl("BBB")].join("\n"); + const diff = [`≔${sameLineRange(stale)}`, pl("BBB")].join("\n"); expect(() => applyDiff("aaa\nINSERTED\nbbb\nccc", diff)).toThrow(HashlineMismatchError); }); @@ -395,72 +388,72 @@ describe("hashline — stale anchors", () => { const collidingHash = computeLineHash(1, "x = 1"); // User points at line 4 (`z = 3`) with the colliding hash; without auto- // rebase, this is a plain mismatch. - const diff = [`= ${sameLineRange(`4${collidingHash}`)}`, pl("REPLACED")].join("\n"); + const diff = [`≔${sameLineRange(`4${collidingHash}`)}`, pl("REPLACED")].join("\n"); expect(() => applyDiff(file, diff)).toThrow(HashlineMismatchError); }); }); -describe("splitHashlineInput — @ headers", () => { - it("extracts path and diff body from @path header", () => { - const input = [`@src/foo.ts`, `= ${sameLineRange(tag(2, "bbb"))}`, pl("BBB")].join("\n"); +describe("splitHashlineInput — § headers", () => { + it("extracts path and diff body from §path header", () => { + const input = [`§src/foo.ts`, `≔${sameLineRange(tag(2, "bbb"))}`, pl("BBB")].join("\n"); expect(splitHashlineInput(input)).toEqual({ path: "src/foo.ts", - diff: `= ${sameLineRange(tag(2, "bbb"))}\n${pl("BBB")}`, + diff: `≔${sameLineRange(tag(2, "bbb"))}\n${pl("BBB")}`, }); }); it("strips leading blank lines and unquotes matching path quotes", () => { - expect(splitHashlineInput(`\n@"foo bar.ts"\n+ BOF\n${pl("x")}`)).toEqual({ + expect(splitHashlineInput(`\n§"foo bar.ts"\n»BOF\n${pl("x")}`)).toEqual({ path: "foo bar.ts", - diff: `+ BOF\n${pl("x")}`, + diff: `»BOF\n${pl("x")}`, }); }); it("normalizes cwd-prefixed absolute paths to cwd-relative paths", () => { const cwd = process.cwd(); const absolute = path.join(cwd, "src", "foo.ts"); - expect(splitHashlineInput(`@${absolute}\n+ BOF\n${pl("x")}`, { cwd }).path).toBe("src/foo.ts"); + expect(splitHashlineInput(`§${absolute}\n»BOF\n${pl("x")}`, { cwd }).path).toBe("src/foo.ts"); }); it("uses explicit fallback path only when input has recognizable operations", () => { - expect(splitHashlineInput(`+ BOF\n${pl("x")}`, { path: "a.ts" })).toEqual({ + expect(splitHashlineInput(`»BOF\n${pl("x")}`, { path: "a.ts" })).toEqual({ path: "a.ts", - diff: `+ BOF\n${pl("x")}`, + diff: `»BOF\n${pl("x")}`, }); expect(() => splitHashlineInput("plain text", { path: "a.ts" })).toThrow(/must begin with/); }); it("splits multiple edit sections", () => { - const input = ["@a.ts", "+ BOF", pl("a"), "@b.ts", "+ EOF", pl("b")].join("\n"); + const input = ["§a.ts", "»BOF", pl("a"), "§b.ts", "»EOF", pl("b")].join("\n"); expect(splitHashlineInputs(input)).toEqual([ - { path: "a.ts", diff: `+ BOF\n${pl("a")}` }, - { path: "b.ts", diff: `+ EOF\n${pl("b")}` }, + { path: "a.ts", diff: `»BOF\n${pl("a")}` }, + { path: "b.ts", diff: `»EOF\n${pl("b")}` }, ]); }); - it("tolerates extra '@' chars on the section header", () => { - const input = ["@@ a.ts", "+ BOF", pl("a"), "@@@b.ts", "+ EOF", pl("b")].join("\n"); + it("tolerates extra § chars on the section header", () => { + const input = ["§§a.ts", "»BOF", pl("a"), "§§§b.ts", "»EOF", pl("b")].join("\n"); expect(splitHashlineInputs(input)).toEqual([ - { path: "a.ts", diff: `+ BOF\n${pl("a")}` }, - { path: "b.ts", diff: `+ EOF\n${pl("b")}` }, + { path: "a.ts", diff: `»BOF\n${pl("a")}` }, + { path: "b.ts", diff: `»EOF\n${pl("b")}` }, ]); }); it("silently drops a duplicate header with no operations between them", () => { - const input = ["@@ src/foo.ts", "@@ src/foo.ts", `+ BOF`, pl("x")].join("\n"); - expect(splitHashlineInputs(input)).toEqual([{ path: "src/foo.ts", diff: `+ BOF\n${pl("x")}` }]); + const input = ["§§src/foo.ts", "§§src/foo.ts", `»BOF`, pl("x")].join("\n"); + expect(splitHashlineInputs(input)).toEqual([{ path: "src/foo.ts", diff: `»BOF\n${pl("x")}` }]); }); it("silently drops a trailing header with no operations", () => { - const input = ["@@ a.ts", "+ BOF", pl("a"), "@@ b.ts"].join("\n"); - expect(splitHashlineInputs(input)).toEqual([{ path: "a.ts", diff: `+ BOF\n${pl("a")}` }]); + const input = ["§§a.ts", "»BOF", pl("a"), "§§b.ts"].join("\n"); + expect(splitHashlineInputs(input)).toEqual([{ path: "a.ts", diff: `»BOF\n${pl("a")}` }]); }); }); describe("hashline executor", () => { it("creates a missing file with a file-scoped insert", async () => { await withTempDir(async tempDir => { - const input = `@new.ts\n+ BOF\n${pl("export const x = 1;")}\n`; + const input = `§new.ts\n»BOF\n${pl("export const x = 1;")}\n`; const result = await executeHashlineSingle(hashlineExecuteOptions(tempDir, input)); expect(result.content[0]?.type === "text" ? result.content[0].text : "").toContain("new.ts:"); expect(await Bun.file(path.join(tempDir, "new.ts")).text()).toBe("export const x = 1;"); @@ -471,7 +464,7 @@ describe("hashline executor", () => { await withTempDir(async tempDir => { const filePath = path.join(tempDir, "a.ts"); const source = ["aaa", "bbb", "ccc"].join("\n"); - const input = `@a.ts\n+ ${tag(2, "bbb")}\n${pl("aaa")}\n${pl("bbb")}\n${pl("NEW")}\n`; + const input = `§a.ts\n»${tag(2, "bbb")}\n${pl("aaa")}\n${pl("bbb")}\n${pl("NEW")}\n`; await Bun.write(filePath, source); await executeHashlineSingle(hashlineExecuteOptions(tempDir, input)); @@ -492,11 +485,11 @@ describe("hashline executor", () => { await Bun.write(aPath, "aaa\n"); await Bun.write(bPath, "bbb\n"); const input = [ - "@a.ts", - `= ${sameLineRange(tag(1, "aaa"))}`, + "§a.ts", + `≔${sameLineRange(tag(1, "aaa"))}`, pl("AAA"), - "@b.ts", - `= ${sameLineRange(mistag(1, "bbb"))}`, + "§b.ts", + `≔${sameLineRange(mistag(1, "bbb"))}`, pl("BBB"), ].join("\n"); @@ -520,8 +513,8 @@ describe("hashline executor", () => { // A naive sequential apply reads the modified disk and fails anchor // validation outright. const input = [ - "@a.ts", - `= ${sameLineRange(tag(2, "L2"))}`, + "§a.ts", + `≔${sameLineRange(tag(2, "L2"))}`, pl("L2a"), pl("L2b"), pl("L2c"), @@ -531,8 +524,8 @@ describe("hashline executor", () => { pl("L2g"), pl("L2h"), pl("L2i"), - "@a.ts", - `+ ${tag(8, "L8")}`, + "§a.ts", + `»${tag(8, "L8")}`, pl("INSERTED"), ].join("\n"); @@ -568,9 +561,7 @@ describe("hashline executor", () => { describe("hashlineEditParamsSchema — extra-field tolerance", () => { it("accepts extra `path` field alongside `input`", () => { - expect(hashlineEditParamsSchema.safeParse({ path: "x.ts", input: `@x.ts\n+ BOF\n${pl("x")}` }).success).toBe( - true, - ); + expect(hashlineEditParamsSchema.safeParse({ path: "x.ts", input: `§x.ts\n»BOF\n${pl("x")}` }).success).toBe(true); }); it("still requires `input`", () => { @@ -639,7 +630,7 @@ describe("hashline — anchor-stale recovery via read snapshot cache", () => { await Bun.write(filePath, `${v1Lines.join("\n")}\n`); // Model authors anchor against V0 — line 2 is "L2" in V0. - const input = `@a.ts\n= ${sameLineRange(tag(2, "L2"))}\n${pl("L2-MODEL")}\n`; + const input = `§a.ts\n≔${sameLineRange(tag(2, "L2"))}\n${pl("L2-MODEL")}\n`; const result = await executeHashlineSingle(hashlineExecuteOptions(tempDir, input, undefined, session)); const finalLines = (await Bun.file(filePath).text()).replace(/\n$/, "").split("\n"); @@ -670,7 +661,7 @@ describe("hashline — anchor-stale recovery via read snapshot cache", () => { v1Lines[5] = "L6-CHANGED"; await Bun.write(filePath, `${v1Lines.join("\n")}\n`); - const input = `@a.ts\n= ${sameLineRange(tag(6, "L6"))}\n${pl("L6-MODEL")}\n`; + const input = `§a.ts\n≔${sameLineRange(tag(6, "L6"))}\n${pl("L6-MODEL")}\n`; await expect( executeHashlineSingle(hashlineExecuteOptions(tempDir, input, undefined, session)), ).rejects.toThrow(HashlineMismatchError); @@ -687,7 +678,7 @@ describe("hashline — anchor-stale recovery via read snapshot cache", () => { // Live file is completely different — patch context cannot match even // with fuzz tolerance. const currentText = "totally\nunrelated\ncontent\nhere\nnow\n"; - const edits = parseHashline(`= ${sameLineRange(tag(2, "beta"))}\n${pl("BETA-MODEL")}`); + const edits = parseHashline(`≔${sameLineRange(tag(2, "beta"))}\n${pl("BETA-MODEL")}`); const recovered = tryRecoverHashlineWithCache({ cache, @@ -720,7 +711,7 @@ describe("hashline — anchor-stale recovery via read snapshot cache", () => { // First edit: change line 2 → BETA. After the write, the cache should // reflect V1 (post-edit), not V0. - const firstInput = `@a.ts\n= ${sameLineRange(tag(2, "beta"))}\n${pl("BETA")}\n`; + const firstInput = `§a.ts\n≔${sameLineRange(tag(2, "beta"))}\n${pl("BETA")}\n`; await executeHashlineSingle(hashlineExecuteOptions(tempDir, firstInput, undefined, session)); const v1Lines = ["alpha", "BETA", "gamma", "delta", "epsilon"]; expect(await Bun.file(filePath).text()).toBe(`${v1Lines.join("\n")}\n`); @@ -736,7 +727,7 @@ describe("hashline — anchor-stale recovery via read snapshot cache", () => { const v2Lines = ["H1", "H2", "H3", "H4", "H5", "H6", "H7", ...v1Lines]; await Bun.write(filePath, `${v2Lines.join("\n")}\n`); - const secondInput = `@a.ts\n= ${sameLineRange(tag(3, "gamma"))}\n${pl("GAMMA")}\n`; + const secondInput = `§a.ts\n≔${sameLineRange(tag(3, "gamma"))}\n${pl("GAMMA")}\n`; const result = await executeHashlineSingle(hashlineExecuteOptions(tempDir, secondInput, undefined, session)); const finalLines = (await Bun.file(filePath).text()).replace(/\n$/, "").split("\n"); @@ -787,7 +778,7 @@ describe("hashline *** Abort recovery sentinel (harmony-leak mitigation)", () => const sentinel = "*** Abort"; it("parser breaks at *** Abort and surfaces a warning", () => { - const diff = [`+ ${tag(1, "alpha")}`, pl("HELLO"), sentinel, `+ ${tag(99, "junk")}`, pl("never")].join("\n"); + const diff = [`»${tag(1, "alpha")}`, pl("HELLO"), sentinel, `»${tag(99, "junk")}`, pl("never")].join("\n"); const { edits, warnings } = parseHashlineWithWarnings(diff); expect(edits).toHaveLength(1); expect(edits[0]).toMatchObject({ kind: "insert", text: "HELLO" }); @@ -797,7 +788,7 @@ describe("hashline *** Abort recovery sentinel (harmony-leak mitigation)", () => it("appended sentinel from harmony-leak truncation: ops above are preserved", () => { // Mirrors the exact shape harmony-leak emits inside a single section. - const diff = `+ ${tag(1, "alpha")}\n${pl("KEPT")}\n*** Abort\n`; + const diff = `»${tag(1, "alpha")}\n${pl("KEPT")}\n*** Abort\n`; const { edits, warnings } = parseHashlineWithWarnings(diff); expect(edits).toHaveLength(1); expect(edits[0]).toMatchObject({ text: "KEPT" }); @@ -806,12 +797,12 @@ describe("hashline *** Abort recovery sentinel (harmony-leak mitigation)", () => it("splitter respects *** Abort like *** End Patch", () => { const input = [ - `@a.ts`, - `+ ${tag(1, "alpha")}`, + `§a.ts`, + `»${tag(1, "alpha")}`, pl("a-payload"), sentinel, - `@b.ts`, - `+ ${tag(1, "beta")}`, + `§b.ts`, + `»${tag(1, "beta")}`, pl("never-emitted"), ].join("\n"); const sections = splitHashlineInputs(input); @@ -821,74 +812,8 @@ describe("hashline *** Abort recovery sentinel (harmony-leak mitigation)", () => }); it("clean input without sentinel produces no warning", () => { - const diff = `+ ${tag(1, "alpha")}\n${pl("PAYLOAD")}\n`; + const diff = `»${tag(1, "alpha")}\n${pl("PAYLOAD")}\n`; const { warnings } = parseHashlineWithWarnings(diff); expect(warnings).toEqual([]); }); }); - -describe("hashline separator-padding warning", () => { - // Single-line typo of `~ beta` (sep + readability space + content). - it("warns on uniform single-space-before-content payload (~ TEXT typo)", () => { - const diff = `= ${sameLineRange(tag(1, "alpha"))}\n${pl(" beta")}\n`; - const { warnings } = parseHashlineWithWarnings(diff); - expect(warnings).toHaveLength(1); - expect(warnings[0]).toMatch(/exactly one space before non-space content/); - }); - - it("warns when every payload line uses the same single-space readability gap", () => { - const diff = [`+ ${tag(1, "alpha")}`, pl(" first"), pl(" second"), pl(" third")].join("\n"); - const { warnings } = parseHashlineWithWarnings(diff); - expect(warnings).toHaveLength(1); - expect(warnings[0]).toMatch(/exactly one space before non-space content/); - }); - - // Real YAML / JSON / Markdown edits indent in multiples of two spaces. - it("does not warn on uniform 2-space indentation (YAML/JSON/Markdown)", () => { - const diff = [`+ ${tag(1, "root:")}`, pl(" key: value"), pl(" list:"), pl(" - item")].join("\n"); - const { warnings } = parseHashlineWithWarnings(diff); - expect(warnings).toEqual([]); - }); - - // Python uses 4-space indentation; every payload line starts with whitespace. - it("does not warn on uniform 4-space indentation (Python)", () => { - const diff = [`+ ${tag(1, "def foo():")}`, pl(" x = 1"), pl(" y = 2"), pl(" return x + y")].join("\n"); - const { warnings } = parseHashlineWithWarnings(diff); - expect(warnings).toEqual([]); - }); - - // Mixed indentation: leading space on some lines but not all -> no warning. - it("does not warn when leading-space pattern is not uniform", () => { - const diff = [`+ ${tag(1, "alpha")}`, pl(" leading"), pl("flush")].join("\n"); - const { warnings } = parseHashlineWithWarnings(diff); - expect(warnings).toEqual([]); - }); - // Indent-sensitive file types: skip the padding check entirely so a one-space - // indent (e.g. YAML/Python at depth 1) does not surface a misleading warning. - it("skips padding check for indent-sensitive extensions (yaml/py/md/etc.)", () => { - const diff = `= ${sameLineRange(tag(1, "alpha"))}\n${pl(" beta")}\n`; - for (const path of [ - "config.yml", - "config.yaml", - "deep/nested/script.py", - "notes.md", - "data.json", - "infra.tf", - "/abs/path/file.toml", - "docs/guide.rst", - "C:\\proj\\file.yaml", - ]) { - const { warnings } = parseHashlineWithWarnings(diff, { path }); - expect(warnings).toEqual([]); - } - }); - - it("keeps padding check for non-indent-sensitive extensions (ts/rs/go/etc.)", () => { - const diff = `= ${sameLineRange(tag(1, "alpha"))}\n${pl(" beta")}\n`; - for (const path of ["app.ts", "main.rs", "server.go", "lib.c", "no-extension"]) { - const { warnings } = parseHashlineWithWarnings(diff, { path }); - expect(warnings).toHaveLength(1); - expect(warnings[0]).toMatch(/exactly one space before non-space content/); - } - }); -}); diff --git a/packages/coding-agent/test/edit-diff.test.ts b/packages/coding-agent/test/edit-diff.test.ts index 6a9c20a35..872a9934e 100644 --- a/packages/coding-agent/test/edit-diff.test.ts +++ b/packages/coding-agent/test/edit-diff.test.ts @@ -10,7 +10,6 @@ import { findMatch, formatLineHash, } from "@oh-my-pi/pi-coding-agent/edit"; -import { HL_EDIT_SEP } from "@oh-my-pi/pi-coding-agent/hashline/hash"; describe("findMatch", () => { describe("exact matching", () => { @@ -237,10 +236,10 @@ describe("computeHashlineDiff", () => { const line = "unchanged content"; await Bun.write(sourcePath, `${line}\n`); - // `= 1..1` with the same line as payload is a true no-op: the - // edit fires through computeHashlineDiff but produces identical content. + // `≔1` with the same line as payload is a true no-op: the edit + // fires through computeHashlineDiff but produces identical content. const anchor = formatLineHash(1, line); - const input = `@${sourcePath}\n= ${anchor}..${anchor}\n${HL_EDIT_SEP}${line}\n`; + const input = `§${sourcePath}\n≔${anchor}\n${line}\n`; const result = await computeHashlineDiff({ input }, tempDir); expect("error" in result).toBe(true); if ("error" in result) { @@ -252,14 +251,14 @@ describe("computeHashlineDiff", () => { const sourcePath = path.join(tempDir, "source.txt"); await Bun.write(sourcePath, "first\n"); - const result = await computeHashlineDiff({ input: `@${sourcePath}\n+ EOF\n${HL_EDIT_SEP}second` }, tempDir); + const result = await computeHashlineDiff({ input: `§${sourcePath}\n»EOF\nsecond` }, tempDir); expect("diff" in result).toBe(true); if ("diff" in result) { expect(result.diff).toContain("second"); } }); test("returns a handled error when the source path is a local URL", async () => { - const result = await computeHashlineDiff({ input: "@local://PLAN.md\n+ EOF\n" }, tempDir); + const result = await computeHashlineDiff({ input: "§local://PLAN.md\n»EOF\n" }, tempDir); expect("error" in result).toBe(true); if ("error" in result) { diff --git a/packages/coding-agent/test/edit-streaming-preview.test.ts b/packages/coding-agent/test/edit-streaming-preview.test.ts index 80ba74568..49da0503a 100644 --- a/packages/coding-agent/test/edit-streaming-preview.test.ts +++ b/packages/coding-agent/test/edit-streaming-preview.test.ts @@ -54,7 +54,7 @@ describe("hashline streaming preview (multi-section)", () => { const ctx = (cwd: string) => ({ cwd, signal: new AbortController().signal }); test("keeps section A's preview when section B's header just arrived", async () => { - const input = ["@a.ts", "+ BOF", "~// new", "@b.ts"].join("\n"); + const input = ["§a.ts", "»BOF", "// new", "§b.ts"].join("\n"); const previews = await strategy.computeDiffPreview({ input } as never, ctx(tmpDir) as never); expect(previews).not.toBeNull(); expect(previews).toHaveLength(1); @@ -64,8 +64,8 @@ describe("hashline streaming preview (multi-section)", () => { }); test("ignores parse errors from the trailing in-progress section", async () => { - // `+ 7` is a malformed anchor — the trailing section is still being typed. - const input = ["@a.ts", "+ BOF", "~// new", "@b.ts", "+ 7"].join("\n"); + // `»7` is a malformed anchor — the trailing section is still being typed. + const input = ["§a.ts", "»BOF", "// new", "§b.ts", "»7"].join("\n"); const previews = await strategy.computeDiffPreview({ input } as never, ctx(tmpDir) as never); expect(previews).not.toBeNull(); expect(previews).toHaveLength(1); @@ -74,7 +74,7 @@ describe("hashline streaming preview (multi-section)", () => { }); test("renders both sections once each has at least one valid op", async () => { - const input = ["@a.ts", "+ BOF", "~// new a", "@b.ts", "+ BOF", "~// new b"].join("\n"); + const input = ["§a.ts", "»BOF", "// new a", "§b.ts", "»BOF", "// new b"].join("\n"); const previews = await strategy.computeDiffPreview({ input } as never, ctx(tmpDir) as never); expect(previews).toHaveLength(2); expect(previews?.map(p => p.path).sort()).toEqual(["a.ts", "b.ts"]); diff --git a/packages/coding-agent/test/tools/edit-renderer.test.ts b/packages/coding-agent/test/tools/edit-renderer.test.ts index f6bd54f35..453d9b10f 100644 --- a/packages/coding-agent/test/tools/edit-renderer.test.ts +++ b/packages/coding-agent/test/tools/edit-renderer.test.ts @@ -1,11 +1,16 @@ -import { describe, expect, it } from "bun:test"; +import { beforeAll, describe, expect, it } from "bun:test"; import type { AgentTool } from "@oh-my-pi/pi-agent-core"; +import { resetSettingsForTest, Settings } from "@oh-my-pi/pi-coding-agent/config/settings"; import { editToolRenderer } from "@oh-my-pi/pi-coding-agent/edit/renderer"; -import { HL_EDIT_SEP } from "@oh-my-pi/pi-coding-agent/hashline/hash"; import { ToolExecutionComponent } from "@oh-my-pi/pi-coding-agent/modes/components/tool-execution"; import * as themeModule from "@oh-my-pi/pi-coding-agent/modes/theme/theme"; import type { TUI } from "@oh-my-pi/pi-tui"; +beforeAll(async () => { + resetSettingsForTest(); + await Settings.init({ inMemory: true, cwd: process.cwd() }); +}); + async function getUiTheme() { await themeModule.initTheme(false, undefined, undefined, "dark", "light"); const theme = await themeModule.getThemeByName("dark"); @@ -33,7 +38,7 @@ describe("editToolRenderer", () => { const uiTheme = await getUiTheme(); const component = editToolRenderer.renderCall( { - input: "@packages/coding-agent/src/edit/renderer.ts\n+ EOF\n|// preview", + input: "§packages/coding-agent/src/edit/renderer.ts\n»EOF\n// preview", }, { expanded: false, isPartial: true, spinnerFrame: 0, renderContext: { editMode: "hashline" } }, uiTheme, @@ -51,12 +56,9 @@ describe("editToolRenderer", () => { const component = new ToolExecutionComponent( "edit", { - input: [ - "*** Begin Patch", - "@crates/pi-natives/src/shell.rs", - "+ EOF", - `${HL_EDIT_SEP}pub fn streaming_preview() {`, - ].join("\n"), + input: ["*** Begin Patch", "§crates/pi-natives/src/shell.rs", "»EOF", "pub fn streaming_preview() {"].join( + "\n", + ), }, {}, hashlineTool, @@ -65,8 +67,8 @@ describe("editToolRenderer", () => { const rendered = Bun.stripANSI(component.render(160).join("\n")); expect(rendered).toContain("crates/pi-natives/src/shell.rs"); - expect(rendered).toContain("+ EOF"); - expect(rendered).toContain(`${HL_EDIT_SEP}pub fn streaming_preview() {`); + expect(rendered).toContain("»EOF"); + expect(rendered).toContain("pub fn streaming_preview() {"); expect(rendered).not.toContain("*** Begin Patch"); }); @@ -74,7 +76,7 @@ describe("editToolRenderer", () => { const uiTheme = await getUiTheme(); const compactComponent = editToolRenderer.renderCall( { - input: "@foo bar.ts\n+ BOF\n|// preview", + input: "§foo bar.ts\n»BOF\n// preview", }, { expanded: true, isPartial: true, spinnerFrame: 0, renderContext: { editMode: "hashline" } }, uiTheme, @@ -82,7 +84,7 @@ describe("editToolRenderer", () => { const quotedComponent = editToolRenderer.renderCall( { - input: "@'baz qux.ts'\n+ BOF\n|// preview", + input: "§'baz qux.ts'\n»BOF\n// preview", }, { expanded: false, isPartial: true, spinnerFrame: 0, renderContext: { editMode: "hashline" } }, uiTheme, @@ -94,14 +96,14 @@ describe("editToolRenderer", () => { expect(quotedRendered).toContain("baz qux.ts"); }); - it("strips canonical `@@` and longer `@` runs from hashline input headers", async () => { + it("strips canonical `§` and longer `§` runs from hashline input headers", async () => { const uiTheme = await getUiTheme(); - // Canonical `@@ PATH` form (two `@`s) — the parser strips both, the - // renderer used to keep one. Regression: title displayed `@ /path/...`. + // Canonical `§PATH` form — the parser strips the marker and the + // renderer keeps the title clean. const canonical = editToolRenderer.renderCall( { - input: "@@ packages/coding-agent/src/slash-commands/builtin-registry.ts\n+ BOF\n~// preview", + input: "§packages/coding-agent/src/slash-commands/builtin-registry.ts\n»BOF\n// preview", }, { expanded: true, isPartial: true, spinnerFrame: 0, renderContext: { editMode: "hashline" } }, uiTheme, @@ -109,7 +111,7 @@ describe("editToolRenderer", () => { // Even longer runs should still produce the clean path. const triple = editToolRenderer.renderCall( - { input: "@@@ a/b/c.ts\n+ BOF\n~// preview" }, + { input: "§§§a/b/c.ts\n»BOF\n// preview" }, { expanded: true, isPartial: true, spinnerFrame: 0, renderContext: { editMode: "hashline" } }, uiTheme, ); @@ -118,9 +120,9 @@ describe("editToolRenderer", () => { const tripleRendered = Bun.stripANSI(triple.render(160).join("\n")); expect(canonicalRendered).toContain("packages/coding-agent/src/slash-commands/builtin-registry.ts"); - expect(canonicalRendered).not.toMatch(/@ packages\/coding-agent/); + expect(canonicalRendered).not.toMatch(/§packages\/coding-agent/); expect(tripleRendered).toContain("a/b/c.ts"); - expect(tripleRendered).not.toMatch(/@+ a\/b\/c\.ts/); + expect(tripleRendered).not.toMatch(/§+a\/b\/c\.ts/); }); it("uses hashline input headers for completed single-file result path", async () => { @@ -136,7 +138,7 @@ describe("editToolRenderer", () => { { expanded: false, isPartial: false, renderContext: { editMode: "hashline" } }, uiTheme, { - input: "@packages/coding-agent/src/edit/renderer.ts\n+ EOF\n|// preview", + input: "§packages/coding-agent/src/edit/renderer.ts\n»EOF\n// preview", }, ); diff --git a/scripts/bench-edit-hashline-sep.ts b/scripts/bench-edit-hashline-sep.ts deleted file mode 100644 index b89e1f69e..000000000 --- a/scripts/bench-edit-hashline-sep.ts +++ /dev/null @@ -1,191 +0,0 @@ -#!/usr/bin/env bun -/** - * Run `bun run bench:edit --edit-variant hashline` across the cartesian product - * of `PI_HL_SEP` separator values and models, with a fixed concurrency cap. - * - * Each invocation writes its markdown report and a captured stdout/stderr log - * into `runs/hashline-sep-/`. - * - * Usage: - * bun scripts/bench-edit-hashline-sep.ts - */ -import * as fs from "node:fs/promises"; -import * as path from "node:path"; - -const SEPARATORS = ["~", "%", "÷", ">", ":"] as const; - -const MODELS = [ - "openrouter/z-ai/glm-4.7:nitro", - "openai/gpt-5.4-nano", - "anthropic/claude-sonnet-4-6", -] as const; - -const CONCURRENCY = 3; -const MAX_TASKS = "12"; -const VARIANT = "hashline"; - -const SEP_SLUGS: Record = { - "~": "tilde", - "%": "pct", - "÷": "div", - ">": "gt", - ":": "colon", -}; - -function slugifyModel(model: string): string { - return model.replace(/[^a-zA-Z0-9._-]+/g, "_"); -} - -interface Job { - sep: string; - sepSlug: string; - model: string; - output: string; - logPath: string; - tag: string; -} - -interface JobResult { - job: Job; - exitCode: number | null; - durationMs: number; -} - -const repoRoot = path.resolve(import.meta.dir, ".."); -const stamp = new Date().toISOString().replace(/[:.]/g, "-").replace(/Z$/, "Z"); -const runDir = path.join(repoRoot, "runs", `hashline-sep-${stamp}`); -await fs.mkdir(runDir, { recursive: true }); - -const jobs: Job[] = []; -for (const sep of SEPARATORS) { - for (const model of MODELS) { - const sepSlug = SEP_SLUGS[sep] ?? `sep_${sep.charCodeAt(0).toString(16)}`; - const slug = `${sepSlug}__${slugifyModel(model)}`; - jobs.push({ - sep, - sepSlug, - model, - output: path.join(runDir, `${slug}.md`), - logPath: path.join(runDir, `${slug}.log`), - tag: `[${sepSlug} ${model}]`, - }); - } -} - -console.log(`Total runs: ${jobs.length} concurrency: ${CONCURRENCY}`); -console.log(`Output dir: ${runDir}\n`); - -const results: JobResult[] = []; -let cursor = 0; -let finished = 0; - -async function pipeStream( - stream: ReadableStream, - sink: Bun.FileSink, - tag: string, -): Promise { - const reader = stream.getReader(); - const decoder = new TextDecoder(); - let buf = ""; - while (true) { - const { done, value } = await reader.read(); - if (done) break; - const text = decoder.decode(value, { stream: true }); - sink.write(text); - buf += text; - let nl = buf.indexOf("\n"); - while (nl !== -1) { - const line = buf.slice(0, nl); - buf = buf.slice(nl + 1); - console.log(`${tag} ${line}`); - nl = buf.indexOf("\n"); - } - } - const tail = decoder.decode(); - if (tail) { - sink.write(tail); - buf += tail; - } - if (buf) console.log(`${tag} ${buf}`); -} - -async function runJob(job: Job): Promise { - const started = Date.now(); - const sink = Bun.file(job.logPath).writer(); - sink.write(`# cmd: PI_HL_SEP=${JSON.stringify(job.sep)} bun run bench:edit \\\n`); - sink.write( - `# --edit-variant ${VARIANT} --model ${job.model} --max-tasks ${MAX_TASKS} --output ${job.output}\n\n`, - ); - - const proc = Bun.spawn({ - cmd: [ - "bun", - "run", - "bench:edit", - "--edit-variant", - VARIANT, - "--model", - job.model, - "--max-tasks", - MAX_TASKS, - "--output", - job.output, - ], - cwd: repoRoot, - env: { ...process.env, PI_HL_SEP: job.sep }, - stdout: "pipe", - stderr: "pipe", - }); - - await Promise.all([ - pipeStream(proc.stdout as ReadableStream, sink, job.tag), - pipeStream(proc.stderr as ReadableStream, sink, job.tag), - ]); - const exitCode = await proc.exited; - await sink.end(); - return { job, exitCode, durationMs: Date.now() - started }; -} - -async function worker(workerId: number): Promise { - while (true) { - const idx = cursor++; - if (idx >= jobs.length) return; - const job = jobs[idx]; - console.log(`${job.tag} starting (worker ${workerId}, ${idx + 1}/${jobs.length})`); - const result = await runJob(job); - results.push(result); - finished++; - const status = result.exitCode === 0 ? "ok" : `FAIL exit=${result.exitCode}`; - console.log( - `${job.tag} ${status} in ${(result.durationMs / 1000).toFixed(1)}s [${finished}/${jobs.length}]`, - ); - } -} - -const wallStart = Date.now(); -await Promise.all(Array.from({ length: CONCURRENCY }, (_, i) => worker(i + 1))); -const wallMs = Date.now() - wallStart; - -console.log("\n=== Summary ==="); -results.sort( - (a, b) => - SEPARATORS.indexOf(a.job.sep as (typeof SEPARATORS)[number]) - - SEPARATORS.indexOf(b.job.sep as (typeof SEPARATORS)[number]) || - MODELS.indexOf(a.job.model as (typeof MODELS)[number]) - - MODELS.indexOf(b.job.model as (typeof MODELS)[number]), -); -for (const r of results) { - const status = r.exitCode === 0 ? "ok " : `FAIL`; - console.log( - `${status} ${r.job.sepSlug.padEnd(7)} ${r.job.model.padEnd(40)} ${(r.durationMs / 1000) - .toFixed(1) - .padStart(6)}s ${path.relative(repoRoot, r.job.output)}`, - ); -} -console.log(`\nWall time: ${(wallMs / 1000).toFixed(1)}s`); - -const failures = results.filter(r => r.exitCode !== 0); -if (failures.length > 0) { - console.log(`${failures.length} job(s) failed`); - process.exit(1); -} diff --git a/scripts/session-stats/README.md b/scripts/session-stats/README.md index 53021900e..cabb086e9 100644 --- a/scripts/session-stats/README.md +++ b/scripts/session-stats/README.md @@ -47,7 +47,7 @@ All tables are prefixed `ss_` to avoid collision with `packages/stats`. |`ss_assistant_msgs`|per assistant message text + thinking blobs and token counts| |`ss_user_msgs`|per user message text and token count| |`ss_edit_calls`|per `edit` call: `success`, `warnings`, `raw_input_len`| -|`ss_edit_sections`|per `@PATH` section in an edit; precomputed `longest_repeat_*`, `dup_anchors`| +|`ss_edit_sections`|per `§PATH` section in an edit; precomputed `longest_repeat_*`, `dup_anchors`| Indexes on `(tool_name, timestamp)` and `(session_file, seq)` make per-tool aggregations and ordered session walks cheap. diff --git a/scripts/session-stats/harmony_backtest.py b/scripts/session-stats/harmony_backtest.py index 9b54bff9d..21acf9d95 100755 --- a/scripts/session-stats/harmony_backtest.py +++ b/scripts/session-stats/harmony_backtest.py @@ -53,13 +53,12 @@ SCRIPT_RUN_RE = re.compile( "]{2,}" ) -HEADER_RE = re.compile(r"^(?:@(?P\S.*)|\*\*\* Update File:\s+(?P\S.*))\s*$") +HEADER_RE = re.compile(r"^(?:§+(?P.*)|\*\*\* Update File:\s+(?P\S.*))\s*$") BEGIN_PATCH_RE = re.compile(r"^\*\*\* Begin Patch\s*$") END_PATCH_RE = re.compile(r"^\*\*\* End Patch\s*$") -INSERT_RE = re.compile(r"^[+<]\s*(?PBOF|EOF|[1-9][0-9]*[A-Za-z]{2})(?:\s*~(?P.*))?\s*$") +INSERT_RE = re.compile(r"^(?P[«»])\s*(?PBOF|EOF|[1-9][0-9]*[A-Za-z]{2})\s*$") RANGE_RE = re.compile(r"(?P[1-9][0-9]*[A-Za-z]{2})(?:\.\.(?P[1-9][0-9]*[A-Za-z]{2}))?") -DELETE_RE = re.compile(r"^-\s*(?P[1-9][0-9]*[A-Za-z]{2}(?:\.\.[1-9][0-9]*[A-Za-z]{2})?)\s*$") -REPLACE_RE = re.compile(r"^=\s*(?P[1-9][0-9]*[A-Za-z]{2}(?:\.\.[1-9][0-9]*[A-Za-z]{2})?)\s*$") +REPLACE_RE = re.compile(r"^≔\s*(?P[1-9][0-9]*[A-Za-z]{2}(?:\.\.[1-9][0-9]*[A-Za-z]{2})?)\s*$") JSON_DECODER = json.JSONDecoder() @@ -457,7 +456,7 @@ def parse_edit_boundary(text: str, *, legacy_loose_tail: bool = False) -> EditBo if header: if needs_payload and not saw_required_payload: break - target = (header.group("at") or header.group("upd") or "").strip() + target = (header.group("hl") or header.group("upd") or "").strip() cur = EditSection(target_file=target) sections.append(cur) parsed_end = end @@ -469,9 +468,7 @@ def parse_edit_boundary(text: str, *, legacy_loose_tail: bool = False) -> EditBo if cur is None: break - if line.startswith("~"): - if not payload_allowed: - break + if payload_allowed and line[:1] not in {"«", "»", "≔", "§"}: cur.payload_lines += 1 parsed_end = end saw_required_payload = True @@ -490,35 +487,17 @@ def parse_edit_boundary(text: str, *, legacy_loose_tail: bool = False) -> EditBo if needs_payload and not saw_required_payload: break - trimmed = line.lstrip() - ins = INSERT_RE.match(trimmed) + ins = INSERT_RE.match(line) if ins: cur.op_count += 1 - inline = ins.group("inline") - if inline is None: - needs_payload = True - payload_allowed = True - saw_required_payload = False - # Not complete until at least one payload line appears. - else: - cur.payload_lines += 1 - parsed_end = end - needs_payload = False - payload_allowed = False - saw_required_payload = False - continue - - dele = DELETE_RE.match(trimmed) - if dele: - cur.op_count += 1 - cur.deleted_lines += range_deleted_lines(dele.group("range")) - parsed_end = end - needs_payload = False - payload_allowed = False + needs_payload = True + payload_allowed = True saw_required_payload = False + # Not complete until at least one payload line appears. continue - repl = REPLACE_RE.match(trimmed) + + repl = REPLACE_RE.match(line) if repl: cur.op_count += 1 cur.deleted_lines += range_deleted_lines(repl.group("range")) diff --git a/scripts/session-stats/sync.py b/scripts/session-stats/sync.py index d0ced3207..385749f95 100644 --- a/scripts/session-stats/sync.py +++ b/scripts/session-stats/sync.py @@ -15,7 +15,7 @@ Schema (all tables prefixed `ss_` to avoid collision with packages/stats): ss_assistant_msgs one row per assistant message (text + thinking blobs) ss_user_msgs one row per user message (text blob) ss_edit_calls one row per edit toolCall (success + warnings paired in) - ss_edit_sections one row per @PATH section inside an edit toolCall, with + ss_edit_sections one row per §PATH section inside an edit toolCall, with precomputed detector outputs (longest_repeat_*, dup_anchors) Run: @@ -52,11 +52,11 @@ except ImportError: SESSIONS_ROOT = Path.home() / ".omp" / "agent" / "sessions" DB_PATH = Path.home() / ".omp" / "stats.db" TOKENIZER_NAME = "o200k_base" -SCHEMA_VERSION = 2 +SCHEMA_VERSION = 3 # Bump whenever parse_hashline_input / find_longest_repeat / duplicated_anchors # / looks_successful / extract_warnings semantics change. Bump invalidates # previously-stored ss_edit_* rows on next sync. -EDIT_PARSER_VERSION = 1 +EDIT_PARSER_VERSION = 4 SCHEMA_SQL = """ CREATE TABLE IF NOT EXISTS ss_sessions ( @@ -231,6 +231,8 @@ def batch_count_tokens(strings: list[str]) -> list[int]: _RANGE_RE = re.compile(r"^\s*(\d+)[a-z*]+(?:\.\.(\d+)[a-z*]+)?\s*$") _SINGLE_ANCHOR_RE = re.compile(r"^\s*(\d+)[a-z*]+\s*$") +_HASHLINE_OP_RE = re.compile(r"^([«»≔])\s*(\S+)\s*$") +_HASHLINE_ENVELOPE_MARKERS = {"*** Begin Patch", "*** End Patch", "*** Abort"} def _parse_range(raw: str) -> tuple[int, tuple[int, int] | None]: @@ -290,66 +292,57 @@ def parse_hashline_input(input_str: str) -> list[EditSection]: for raw_line in input_str.split("\n"): line = raw_line[:-1] if raw_line.endswith("\r") else raw_line + trimmed_end = line.rstrip() - if line.startswith("@"): + if trimmed_end in _HASHLINE_ENVELOPE_MARKERS: + if trimmed_end != "*** Begin Patch": + break + continue + + if line.startswith("§"): if cur is not None: sections.append(cur) - cur = EditSection(target_file=line[1:].strip()) + prefix_end = 0 + while prefix_end < len(line) and line[prefix_end] == "§": + prefix_end += 1 + cur = EditSection(target_file=line[prefix_end:].strip()) open_idx = None continue if cur is None: continue - if line.startswith("~"): - payload = line[1:] - if open_idx is None: + op_match = _HASHLINE_OP_RE.match(line) + if op_match: + op = op_match.group(1) + body = op_match.group(2) + if op in ("«", "»"): + anchor_trimmed = body.strip() + if anchor_trimmed and anchor_trimmed not in ("BOF", "EOF"): + cur.op_anchors.append(anchor_trimmed) + line_no = _parse_anchor_line(body) + if line_no is not None: + cur.touch(line_no) open_idx = open_new(cur) - cur.payload_blocks[open_idx].append(payload) - continue + cur.op_count += 1 + continue + if op == "≔": + size, lines = _parse_range(body) + cur.deleted_lines += size + if lines is not None: + cur.touch(lines[0]) + cur.touch(lines[1]) + for part in body.strip().split(".."): + t = part.strip() + if t: + cur.op_anchors.append(t) + cur.op_count += 1 + open_idx = open_new(cur) + continue - trimmed = line.lstrip() - if not trimmed: + if open_idx is not None: + cur.payload_blocks[open_idx].append(line) + elif not line.strip(): continue - op = trimmed[0] - if op in ("+", "<"): - body = trimmed[1:].lstrip() - if "~" in body: - anchor_part, tail = body.split("~", 1) - else: - anchor_part, tail = body, None - anchor_trimmed = anchor_part.strip() - if anchor_trimmed and anchor_trimmed not in ("BOF", "EOF"): - cur.op_anchors.append(anchor_trimmed) - line_no = _parse_anchor_line(anchor_part) - if line_no is not None: - cur.touch(line_no) - if tail is not None: - # Inline `+ ANCHOR~text`: replaces a single line. - if open_idx is None: - open_idx = open_new(cur) - cur.payload_blocks[open_idx].append(tail) - cur.deleted_lines += 1 - open_idx = None - else: - open_idx = open_new(cur) - cur.op_count += 1 - elif op in ("-", "="): - body = trimmed[1:].lstrip() - size, lines = _parse_range(body) - cur.deleted_lines += size - if lines is not None: - cur.touch(lines[0]) - cur.touch(lines[1]) - for part in body.strip().split(".."): - t = part.strip() - if t: - cur.op_anchors.append(t) - cur.op_count += 1 - if op == "=": - open_idx = open_new(cur) - else: - open_idx = None - # else: blank / unrecognized — keep payload state. if cur is not None: sections.append(cur) @@ -676,7 +669,7 @@ def _ingest_edit_call(rec, sf, seq, ts, call_id, arg_obj, arg_json) -> None: (sf, call_id, seq, ts, raw_input_len, EDIT_PARSER_VERSION) ) - if not input_str.lstrip().startswith("@"): + if not any(line.startswith("§") for line in input_str.lstrip("\ufeff").splitlines()): # Vim-mode or other shape — no sections to record. return