feat(coding-agent): added parser support for t/rst in Cell eval
- Introduced canonical `*** Cell` headers with `t:` and `rst` attributes in eval prompts, schema, and docs. - Updated parser and grammar to parse `*** Cell` blocks, stop on `*** End`/next header/EOF, and handle invalid `rst` with errors. - Added quote-aware attribute tokenizers and split HTML eval parsing into `Cell` and legacy `Begin` handlers with `py` defaults. - Expanded parsing behavior and tests for `rst` booleans, title aliases, abort boundaries, and stray-line skips between cells.
This commit is contained in:
+21
-13
@@ -28,22 +28,30 @@
|
||||
|
||||
`input` syntax accepted at runtime:
|
||||
|
||||
- Cell header: `*** Begin <LANG>`; parser accepts `PY`, `PYTHON`, `IPY`, `IPYTHON`, `JS`, `JAVASCRIPT`, `TS`, `TYPESCRIPT` case-insensitively.
|
||||
- Optional attributes immediately after the header, first occurrence wins:
|
||||
- `*** Title: ...`
|
||||
- `*** Timeout: <n>[ms|s|m]` (default 30s)
|
||||
- `*** Reset`
|
||||
- Cell body: every following line until `*** End ...`, the next `*** Begin ...`, or `*** Abort`.
|
||||
- Cell header: `*** Cell <attrs...>`. Attributes are space-separated tokens with quoted titles (`"..."` or `'...'`).
|
||||
- Canonical tokens (advertised in the prompt):
|
||||
- `<lang>:"<title>"` — language + title shorthand. `lang` is `py` or `js` (lenient: also `ts`, plus the long-form aliases `python`, `javascript`, `typescript`, `ipy`, `ipython`).
|
||||
- `t:<n>[ms|s|m]` — per-cell timeout (default 30s).
|
||||
- `rst` — wipe this cell's language kernel before running.
|
||||
- Lenient additional tokens (accepted by the parser, not advertised):
|
||||
- bare language token (`py`, `js`)
|
||||
- `id:"..."` / `title:"..."` / `name:"..."` / `cell:"..."` / `file:"..."` / `label:"..."` — title aliases
|
||||
- `timeout:` / `duration:` / `time:` — `t:` aliases
|
||||
- `reset` — `rst` alias
|
||||
- `rst:true|false|1|0|yes|no|on|off` — explicit boolean form
|
||||
- a bare positional duration token (`30s`, `2m`, `500ms`)
|
||||
- any unclassified bare token folds into a positional title fragment
|
||||
- Cell body: every following line until the next `*** Cell ...`, the optional `*** End`, or `*** Abort`. `*** End` is a quirk fix for GPT-trained models that emit terminators and is not documented in the prompt.
|
||||
|
||||
Leniencies in `packages/coding-agent/src/eval/parse.ts`:
|
||||
|
||||
- Markers accept two or more leading `*` and flexible whitespace.
|
||||
- `*** End` does not need to repeat the language token.
|
||||
- Missing end markers between adjacent cells are tolerated; the next `*** Begin` closes the prior cell.
|
||||
- `*** End` is optional everywhere; the parser silently consumes trailing tokens (e.g. `*** End py`).
|
||||
- Missing terminators between adjacent cells are tolerated; the next `*** Cell` closes the prior cell, and stray non-marker lines between cells fold into the prior cell's body without crashing.
|
||||
- Bare code or a single markdown fence such as ```` ```py ```` is treated as one implicit cell.
|
||||
- If `*** Abort` appears, the in-progress cell is dropped and the result carries an abort warning.
|
||||
- If `*** Abort` appears, the in-progress cell is dropped and the result carries an abort warning. To preserve a completed cell before `*** Abort`, emit `*** End` first.
|
||||
|
||||
The tool also exposes a custom Lark grammar from `packages/coding-agent/src/eval/eval.lark` for constrained sampling. That grammar is stricter than the runtime parser: it only advertises `PY` / `JS` / `TS` headers and an `*** End Cell` closer.
|
||||
The tool also exposes a custom Lark grammar from `packages/coding-agent/src/eval/eval.lark` for constrained sampling. That grammar is stricter than the runtime parser: it requires the canonical `*** Cell <lang>:"title"` header form with a fixed attribute order, advertises only `py` / `js`, and pins the trailing `*** End` so GPT-trained models' natural terminator habit aligns with the constrained output.
|
||||
|
||||
## Outputs
|
||||
|
||||
@@ -107,7 +115,7 @@ Side-channel artifacts:
|
||||
|
||||
### Parsing modes
|
||||
|
||||
- Explicit multi-cell format with `*** Begin ...` / `*** End ...`
|
||||
- Explicit multi-cell format with `*** Cell ...` headers
|
||||
- Implicit single-cell fallback for bare code or a single fenced block
|
||||
- Abort-recovery parse path when `*** Abort` is present
|
||||
|
||||
@@ -124,7 +132,7 @@ Side-channel artifacts:
|
||||
Implemented in `packages/coding-agent/src/eval/js/context-manager.ts` and `packages/coding-agent/src/eval/js/prelude.txt`.
|
||||
|
||||
- Persistent `vm.Context` instances keyed by `js:${sessionId}` in `vmContexts`
|
||||
- `*** Reset` calls `resetVmContext(sessionKey)` before the cell executes
|
||||
- `rst` calls `resetVmContext(sessionKey)` before the cell executes
|
||||
- Top-level `await` and bare `return` are supported by wrapping code in an async IIFE when `wrapCode()` sees `await` or `return`
|
||||
- Top-level static `import ... from ...` is rewritten to `await import(...)` by `rewriteStaticImports()`
|
||||
- The prelude installs globals:
|
||||
@@ -145,7 +153,7 @@ Implemented in `packages/coding-agent/src/eval/py/executor.ts`, `packages/coding
|
||||
|
||||
- Default mode is retained `session` kernels keyed by `python:${sessionId}`
|
||||
- Optional `python.kernelMode = "per-call"` creates a fresh kernel for each cell and shuts it down afterward
|
||||
- `*** Reset` disposes the retained kernel for that session before the cell runs; later Python cells in the same tool call reuse the fresh kernel
|
||||
- `rst` disposes the retained kernel for that session before the cell runs; later Python cells in the same tool call reuse the fresh kernel
|
||||
- Startup path:
|
||||
- availability check
|
||||
- create/connect kernel
|
||||
|
||||
@@ -1,6 +1,23 @@
|
||||
# Changelog
|
||||
|
||||
## [Unreleased]
|
||||
### Breaking Changes
|
||||
|
||||
- Changed the `eval` tool input format to a single-line `*** Cell <lang>:"<title>" [t:<duration>] [rst]` header per cell, replacing the `*** Begin <LANG>` / `*** End <LANG>` envelope and the standalone `*** Title:` / `*** Timeout:` / `*** Reset` directives. The lark grammar enforces a fixed attribute order; the runtime parser remains lenient (alias keys, bare positional tokens, single-quoted titles).
|
||||
|
||||
### Added
|
||||
|
||||
- Added support for explicit boolean `rst` values (`rst:true`, `rst:false`, `rst:1`, `rst:0`, `rst:yes`, `rst:no`, `rst:on`, `rst:off`) in `*** Cell` headers
|
||||
|
||||
### Changed
|
||||
|
||||
- Changed the HTML transcript renderer to parse the new `*** Cell` headers while keeping the older `*** Begin <LANG>` and `===== ... =====` formats renderable for historical sessions.
|
||||
- Changed the `eval` tool parser so a stray non-marker line between cells no longer crashes with `null is not an object (evaluating 'BEGIN_RE.exec(lines[i])[1]')`; stray content is consumed without aborting parsing.
|
||||
- Changed `*** End` to be an optional, undocumented per-cell terminator (kept in the lark to satisfy GPT-trained models' natural terminator habit during constrained sampling).
|
||||
|
||||
### Fixed
|
||||
|
||||
- Improved `*** Cell` header parsing to reject invalid `rst` values with a clear `invalid rst value` error
|
||||
|
||||
## [14.9.7] - 2026-05-12
|
||||
|
||||
|
||||
@@ -1,16 +1,36 @@
|
||||
start: cell+
|
||||
// Canonical Eval input. Each cell is introduced by a single header line:
|
||||
//
|
||||
// *** Cell <LANG>:"<title>" [t:<duration>] [rst]
|
||||
//
|
||||
// Attribute order is fixed: language+title, then optional timeout, then
|
||||
// optional reset flag. Title may be empty (`py:""`).
|
||||
//
|
||||
// Tokens:
|
||||
//
|
||||
// py:"..." | js:"..." language plus title (required)
|
||||
// t:<digits>(ms|s|m)? per-cell timeout (default 30s)
|
||||
// rst reset this language's kernel before running
|
||||
//
|
||||
// Everything between one header line and the next (or the optional trailing
|
||||
// `*** End`, or end of input) is the cell's code, verbatim. The runtime
|
||||
// parser additionally accepts content before the first header as an implicit
|
||||
// default-language cell, but that is lenient fallback and MUST NOT be relied
|
||||
// on.
|
||||
|
||||
cell: begin_cell attr* code_line* end_cell
|
||||
begin_cell: "*** Begin " LANG LF
|
||||
end_cell: "*** End Cell" LF?
|
||||
start: cell+ end_marker
|
||||
|
||||
attr: title | timeout | reset
|
||||
title: "*** Title: " /(.+)/ LF
|
||||
timeout: "*** Timeout: " /\d+(ms|s|m)?/ LF
|
||||
reset: "*** Reset" LF
|
||||
cell: cell_header code_line*
|
||||
|
||||
code_line: /[^\r\n]*/ LF
|
||||
cell_header: "*** Cell" WS_INLINE LANG_TITLE (WS_INLINE T_ATTR)? (WS_INLINE RST_FLAG)? LF
|
||||
|
||||
LANG: "JS" | "TS" | "PY"
|
||||
end_marker: "*** End" LF?
|
||||
|
||||
code_line: CODE_TEXT LF | LF
|
||||
CODE_TEXT: /([^*\r\n]|\*\*?[^*\r\n])+\*{0,2}|\*{1,2}/
|
||||
|
||||
LANG_TITLE: ("py" | "js") ":\"" /[^"\r\n]*/ "\""
|
||||
T_ATTR: "t:" /\d+(ms|s|m)?/
|
||||
RST_FLAG: "rst"
|
||||
|
||||
%import common.LF
|
||||
%import common.WS_INLINE
|
||||
|
||||
@@ -43,16 +43,19 @@ const LANGUAGE_MAP: Record<string, EvalLanguage> = {
|
||||
TYPESCRIPT: "js",
|
||||
};
|
||||
|
||||
// Markers are case-insensitive, accept ≥2 leading stars (so `**Begin` and
|
||||
// `*** Begin` both work), and tolerate any whitespace (including tabs)
|
||||
// Markers are case-insensitive, accept ≥2 leading stars (so `**Cell` and
|
||||
// `*** Cell` both work), and tolerate any whitespace (including tabs)
|
||||
// between tokens. Models that can't constrain-sample frequently emit minor
|
||||
// variations like `**End`, `*** end py`, or `***\tTitle: foo`.
|
||||
// variations like `**End` or `*** cell py`.
|
||||
const STARS = String.raw`\*{2,}`;
|
||||
const BEGIN_RE = new RegExp(`^${STARS}\\s*Begin\\b\\s*(\\S+)?\\s*$`, "i");
|
||||
// Cell header: `*** Cell <attrs...>`. The remainder of the line is captured
|
||||
// and tokenized separately so we can handle quoted values.
|
||||
const CELL_RE = new RegExp(`^${STARS}\\s*Cell\\b\\s*(.*)$`, "i");
|
||||
// `*** End` is a tolerated cell/file terminator. Documented as required at
|
||||
// the file level in the lark grammar (the trailing `*** End` quirks GPT-
|
||||
// trained models naturally produce), but optional at the parser level.
|
||||
const END_RE = new RegExp(`^${STARS}\\s*End\\b.*$`, "i");
|
||||
const TITLE_RE = new RegExp(`^${STARS}\\s*Title\\s*:\\s*(.+?)\\s*$`, "i");
|
||||
const TIMEOUT_RE = new RegExp(`^${STARS}\\s*Timeout\\s*:\\s*(\\S+)\\s*$`, "i");
|
||||
const RESET_RE = new RegExp(`^${STARS}\\s*Reset\\s*$`, "i");
|
||||
// `*** Abort` is the harmony-leak recovery sentinel; see ABORT_WARNING.
|
||||
const ABORT_RE = new RegExp(`^${STARS}\\s*Abort\\s*$`, "i");
|
||||
|
||||
/**
|
||||
@@ -62,6 +65,7 @@ const ABORT_RE = new RegExp(`^${STARS}\\s*Abort\\s*$`, "i");
|
||||
*/
|
||||
export const ABORT_WARNING =
|
||||
"Tool stream truncated mid-call due to detected output corruption. Earlier cells (if any) executed normally; their state persists. Re-issue the aborted cell.";
|
||||
|
||||
const DURATION_RE = /^(\d+)(ms|s|m)?$/i;
|
||||
|
||||
function resolveLang(token: string | undefined): EvalLanguage | undefined {
|
||||
@@ -88,7 +92,7 @@ const FENCE_OPEN_RE = /^```\s*([A-Za-z]\w*)?\s*$/;
|
||||
const FENCE_CLOSE_RE = /^```\s*$/;
|
||||
|
||||
/**
|
||||
* Last-resort fallback when the input has no recognizable `*** Begin` header.
|
||||
* Last-resort fallback when the input has no recognizable `*** Cell` header.
|
||||
* Models that can't constrain-sample sometimes pass bare code or wrap it in
|
||||
* a markdown fence (```py / ```python / bare ```). Treat the whole input as
|
||||
* a single implicit cell, sniffing the language from the body.
|
||||
@@ -122,6 +126,187 @@ function parseImplicitCell(lines: string[]): ParsedEvalCell {
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Tokenize a `*** Cell` header's attribute list while preserving quoted
|
||||
* segments (`id:"some title"`, `py:"hi"`, single quotes too) as single
|
||||
* tokens. Outer whitespace separates tokens; the quote characters
|
||||
* themselves are kept verbatim so attribute parsing can strip them later.
|
||||
*/
|
||||
function tokenizeCellAttrs(input: string): string[] {
|
||||
const tokens: string[] = [];
|
||||
let i = 0;
|
||||
while (i < input.length) {
|
||||
while (i < input.length && /\s/.test(input[i])) i++;
|
||||
if (i >= input.length) break;
|
||||
let token = "";
|
||||
while (i < input.length && !/\s/.test(input[i])) {
|
||||
const ch = input[i];
|
||||
if (ch === '"' || ch === "'") {
|
||||
token += ch;
|
||||
i++;
|
||||
while (i < input.length && input[i] !== ch) {
|
||||
token += input[i];
|
||||
i++;
|
||||
}
|
||||
if (i < input.length) {
|
||||
token += input[i];
|
||||
i++;
|
||||
}
|
||||
} else {
|
||||
token += ch;
|
||||
i++;
|
||||
}
|
||||
}
|
||||
tokens.push(token);
|
||||
}
|
||||
return tokens;
|
||||
}
|
||||
|
||||
interface CellHeader {
|
||||
language: EvalLanguage | undefined;
|
||||
languageOrigin: EvalLanguageOrigin;
|
||||
title: string | undefined;
|
||||
timeoutMs: number | undefined;
|
||||
reset: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* Map an attribute key (from `key:value` or bare `key`) to one of the three
|
||||
* canonical roles. Canonical keys: `id`, `t`, `rst`. Fallback aliases —
|
||||
* accepted but not advertised in the prompt — cover common synonyms LLMs
|
||||
* reach for instead of the short canonical.
|
||||
*/
|
||||
const ID_KEYS = new Set(["id", "title", "name", "cell", "file", "label"]);
|
||||
const T_KEYS = new Set(["t", "timeout", "duration", "time"]);
|
||||
const RST_KEYS = new Set(["rst", "reset"]);
|
||||
|
||||
function classifyAttrKey(key: string): "id" | "t" | "rst" | null {
|
||||
if (ID_KEYS.has(key)) return "id";
|
||||
if (T_KEYS.has(key)) return "t";
|
||||
if (RST_KEYS.has(key)) return "rst";
|
||||
return null;
|
||||
}
|
||||
|
||||
// `key:value` form. `value` may be `"..."`, `'...'`, or a bare run.
|
||||
const ATTR_TOKEN_RE = /^([a-zA-Z][\w-]*)(?::(?:"([^"]*)"|'([^']*)'|(.*)))?$/;
|
||||
// Bare positional duration (lenient — `t:` is canonical).
|
||||
const DURATION_TOKEN_RE = /^\d+(?:ms|s|m)?$/;
|
||||
|
||||
function parseBooleanFlag(value: string): boolean | undefined {
|
||||
const v = value.trim().toLowerCase();
|
||||
if (v === "true" || v === "1" || v === "yes" || v === "on") return true;
|
||||
if (v === "false" || v === "0" || v === "no" || v === "off") return false;
|
||||
return undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* Decode a `*** Cell` header's attribute list into language, title,
|
||||
* timeout, and reset flag.
|
||||
*
|
||||
* Token forms (all optional, any order):
|
||||
* - `py` / `js` / `ts` bare language
|
||||
* - `py:"..."` / `js:"..."` / `ts:"..."` language + title shorthand
|
||||
* - `id:"..."` cell title (canonical)
|
||||
* - `t:<duration>` per-cell timeout (canonical)
|
||||
* - `<duration>` (e.g. `30s`) bare positional duration
|
||||
* - `rst` reset flag (canonical)
|
||||
* - `rst:true|false|1|0|yes|no|on|off` reset flag with explicit value
|
||||
*
|
||||
* Fallback aliases (accepted but not advertised in the prompt):
|
||||
* - id: title, name, cell, file, label
|
||||
* - t: timeout, duration, time
|
||||
* - rst: reset
|
||||
*
|
||||
* Quotes may be `"` or `'`. Truly unknown keys are silently dropped. First
|
||||
* occurrence wins when a key is repeated (canonical or alias). Anything
|
||||
* that doesn't classify accumulates as a positional title fragment joined
|
||||
* by spaces.
|
||||
*/
|
||||
function parseCellHeader(rest: string, lineNumber: number): CellHeader {
|
||||
const tokens = tokenizeCellAttrs(rest);
|
||||
let language: EvalLanguage | undefined;
|
||||
let titleAttr: string | undefined;
|
||||
let positionalDurationMs: number | undefined;
|
||||
let tAttr: string | undefined;
|
||||
let rstAttr: string | undefined;
|
||||
let bareReset = false;
|
||||
const titleParts: string[] = [];
|
||||
|
||||
for (const token of tokens) {
|
||||
// Bare reset flag (canonical or alias).
|
||||
if (RST_KEYS.has(token.toLowerCase())) {
|
||||
bareReset = true;
|
||||
continue;
|
||||
}
|
||||
|
||||
const attrMatch = ATTR_TOKEN_RE.exec(token);
|
||||
if (attrMatch && token.includes(":")) {
|
||||
const key = attrMatch[1].toLowerCase();
|
||||
const value = attrMatch[2] ?? attrMatch[3] ?? attrMatch[4] ?? "";
|
||||
|
||||
// Language-with-title shorthand: `py:"foo"`, `js:'bar'`, etc.
|
||||
const langCandidate = resolveLang(key);
|
||||
if (langCandidate) {
|
||||
if (language === undefined) language = langCandidate;
|
||||
if (titleAttr === undefined && value !== "") titleAttr = value;
|
||||
continue;
|
||||
}
|
||||
|
||||
const role = classifyAttrKey(key);
|
||||
if (role === "id" && titleAttr === undefined) titleAttr = value;
|
||||
else if (role === "t" && tAttr === undefined) tAttr = value;
|
||||
else if (role === "rst" && rstAttr === undefined) rstAttr = value;
|
||||
// unknown / repeated keys silently dropped
|
||||
continue;
|
||||
}
|
||||
|
||||
// Bare language token (no colon).
|
||||
const lang = resolveLang(token);
|
||||
if (lang && language === undefined) {
|
||||
language = lang;
|
||||
continue;
|
||||
}
|
||||
|
||||
// Bare positional duration (lenient — `t:` is canonical).
|
||||
if (positionalDurationMs === undefined && DURATION_TOKEN_RE.test(token)) {
|
||||
positionalDurationMs = parseDurationMs(token, lineNumber);
|
||||
continue;
|
||||
}
|
||||
|
||||
titleParts.push(token);
|
||||
}
|
||||
|
||||
const explicitTitle = (titleAttr ?? "").trim();
|
||||
const positionalTitle = titleParts.join(" ").trim();
|
||||
const title = explicitTitle.length > 0 ? explicitTitle : positionalTitle.length > 0 ? positionalTitle : undefined;
|
||||
|
||||
let timeoutMs: number | undefined;
|
||||
if (tAttr !== undefined) {
|
||||
timeoutMs = parseDurationMs(tAttr, lineNumber);
|
||||
} else if (positionalDurationMs !== undefined) {
|
||||
timeoutMs = positionalDurationMs;
|
||||
}
|
||||
|
||||
let reset = false;
|
||||
if (rstAttr !== undefined) {
|
||||
const parsed = parseBooleanFlag(rstAttr);
|
||||
if (parsed === undefined) {
|
||||
throw new Error(`Eval line ${lineNumber}: invalid rst value \`${rstAttr}\`; use true or false.`);
|
||||
}
|
||||
reset = parsed;
|
||||
} else if (bareReset) {
|
||||
reset = true;
|
||||
}
|
||||
|
||||
return {
|
||||
language,
|
||||
languageOrigin: language ? "header" : "default",
|
||||
title,
|
||||
timeoutMs,
|
||||
reset,
|
||||
};
|
||||
}
|
||||
|
||||
export function parseEvalInput(input: string): ParsedEvalInput {
|
||||
const normalized = input.replace(/\r\n?/g, "\n");
|
||||
const lines = normalized.split("\n");
|
||||
@@ -134,10 +319,10 @@ export function parseEvalInput(input: string): ParsedEvalInput {
|
||||
// Skip leading blank lines.
|
||||
while (i < lines.length && lines[i].trim() === "") i++;
|
||||
|
||||
// Lenient fallback: if the input has no recognizable begin marker, treat
|
||||
// Lenient fallback: if the input has no recognizable cell header, treat
|
||||
// the entire input as one implicit cell — unless that content contains
|
||||
// `*** Abort`, in which case the body is incomplete/unsafe and we drop it.
|
||||
if (i < lines.length && !BEGIN_RE.test(lines[i])) {
|
||||
if (i < lines.length && !CELL_RE.test(lines[i])) {
|
||||
const tail = lines.slice(i);
|
||||
if (tail.some(line => ABORT_RE.test(line))) {
|
||||
return { cells, aborted: true };
|
||||
@@ -148,54 +333,26 @@ export function parseEvalInput(input: string): ParsedEvalInput {
|
||||
}
|
||||
|
||||
while (i < lines.length) {
|
||||
const beginMatch = BEGIN_RE.exec(lines[i]);
|
||||
if (!beginMatch) {
|
||||
const headerLine = lines[i];
|
||||
const cellMatch = CELL_RE.exec(headerLine);
|
||||
if (!cellMatch) {
|
||||
// Stray content between/after cells (blank lines were already
|
||||
// consumed). `*** Abort` here terminates parsing; anything else
|
||||
// — typically a harmony-leak fragment or model garbage — is
|
||||
// skipped rather than crashing the parser.
|
||||
if (ABORT_RE.test(lines[i])) {
|
||||
// consumed). `*** Abort` here terminates parsing; `*** End` is
|
||||
// the optional file-level terminator (silently consumed). Anything
|
||||
// else — typically a harmony-leak fragment — is skipped.
|
||||
if (ABORT_RE.test(headerLine)) {
|
||||
aborted = true;
|
||||
break;
|
||||
}
|
||||
i++;
|
||||
continue;
|
||||
}
|
||||
const langToken = beginMatch[1];
|
||||
const explicitLanguage = resolveLang(langToken);
|
||||
const header = parseCellHeader(cellMatch[1] ?? "", i + 1);
|
||||
i++;
|
||||
|
||||
let title: string | undefined;
|
||||
let timeoutMs: number | undefined;
|
||||
let reset = false;
|
||||
|
||||
while (i < lines.length) {
|
||||
const line = lines[i];
|
||||
const lineNumber = i + 1;
|
||||
const titleMatch = TITLE_RE.exec(line);
|
||||
if (titleMatch) {
|
||||
if (title === undefined) title = titleMatch[1];
|
||||
i++;
|
||||
continue;
|
||||
}
|
||||
const timeoutMatch = TIMEOUT_RE.exec(line);
|
||||
if (timeoutMatch) {
|
||||
if (timeoutMs === undefined) timeoutMs = parseDurationMs(timeoutMatch[1], lineNumber);
|
||||
i++;
|
||||
continue;
|
||||
}
|
||||
if (RESET_RE.test(line)) {
|
||||
reset = true;
|
||||
i++;
|
||||
continue;
|
||||
}
|
||||
break;
|
||||
}
|
||||
|
||||
// Collect cell body. Close on `*** End` OR on the next `*** Begin`
|
||||
// (implicit end — leniency for models that drop end markers between
|
||||
// back-to-back cells). `*** Abort` (recovery sentinel) drops the
|
||||
// in-progress cell entirely: its body is partial and unsafe to run.
|
||||
// Collect cell body. Close on `*** End` (any form), the next
|
||||
// `*** Cell` header, or `*** Abort` (which drops the in-progress
|
||||
// cell as its body is partial and unsafe to run).
|
||||
const codeLines: string[] = [];
|
||||
let cellAborted = false;
|
||||
while (i < lines.length) {
|
||||
@@ -210,7 +367,7 @@ export function parseEvalInput(input: string): ParsedEvalInput {
|
||||
i++;
|
||||
break;
|
||||
}
|
||||
if (BEGIN_RE.test(line)) break;
|
||||
if (CELL_RE.test(line)) break;
|
||||
codeLines.push(line);
|
||||
i++;
|
||||
}
|
||||
@@ -224,17 +381,17 @@ export function parseEvalInput(input: string): ParsedEvalInput {
|
||||
}
|
||||
const code = codeLines.join("\n");
|
||||
|
||||
const language = explicitLanguage ?? sniffEvalLanguage(code) ?? DEFAULT_LANGUAGE;
|
||||
const languageOrigin: EvalLanguageOrigin = explicitLanguage ? "header" : "default";
|
||||
const language = header.language ?? sniffEvalLanguage(code) ?? DEFAULT_LANGUAGE;
|
||||
const languageOrigin: EvalLanguageOrigin = header.language ? "header" : "default";
|
||||
|
||||
cells.push({
|
||||
index: cells.length,
|
||||
title,
|
||||
title: header.title,
|
||||
code,
|
||||
language,
|
||||
languageOrigin,
|
||||
timeoutMs: timeoutMs ?? DEFAULT_TIMEOUT_MS,
|
||||
reset,
|
||||
timeoutMs: header.timeoutMs ?? DEFAULT_TIMEOUT_MS,
|
||||
reset: header.reset,
|
||||
});
|
||||
|
||||
// Skip blank separator lines between cells; an `*** Abort` here
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -1262,12 +1262,14 @@
|
||||
return html;
|
||||
}
|
||||
|
||||
// Parse `*** Begin <LANG>` cell headers (canonical) and the legacy
|
||||
// `===== <info> =====` headers used by older transcripts. Cells emitted
|
||||
// before the format cutover still need to render in HTML exports.
|
||||
// Parse `*** Cell <attrs>` headers (canonical), plus legacy
|
||||
// `*** Begin <LANG>` headers and `===== <info> =====` bars used in
|
||||
// older transcripts. Cells emitted before each format cutover still
|
||||
// need to render in HTML exports.
|
||||
function parseEvalCells(input) {
|
||||
const text = String(input);
|
||||
if (/^[*]{2,}\s*Begin\b/im.test(text)) return parseEvalCellsNew(text);
|
||||
if (/^[*]{2,}\s*Cell\b/im.test(text)) return parseEvalCellsCell(text);
|
||||
if (/^[*]{2,}\s*Begin\b/im.test(text)) return parseEvalCellsBegin(text);
|
||||
return parseEvalCellsLegacy(text);
|
||||
}
|
||||
|
||||
@@ -1279,7 +1281,93 @@
|
||||
return null;
|
||||
}
|
||||
|
||||
function parseEvalCellsNew(text) {
|
||||
// Tokenize a `*** Cell` header attribute list, preserving quoted
|
||||
// segments. Mirrors `tokenizeCellAttrs` in src/eval/parse.ts.
|
||||
function tokenizeCellAttrsHtml(input) {
|
||||
const tokens = [];
|
||||
let i = 0;
|
||||
while (i < input.length) {
|
||||
while (i < input.length && /\s/.test(input[i])) i++;
|
||||
if (i >= input.length) break;
|
||||
let tok = '';
|
||||
while (i < input.length && !/\s/.test(input[i])) {
|
||||
const ch = input[i];
|
||||
if (ch === '"' || ch === "'") {
|
||||
tok += ch; i++;
|
||||
while (i < input.length && input[i] !== ch) { tok += input[i]; i++; }
|
||||
if (i < input.length) { tok += input[i]; i++; }
|
||||
} else { tok += ch; i++; }
|
||||
}
|
||||
tokens.push(tok);
|
||||
}
|
||||
return tokens;
|
||||
}
|
||||
|
||||
function parseEvalCellsCell(text) {
|
||||
const STARS = '\\*{2,}';
|
||||
const CELL = new RegExp('^' + STARS + '\\s*Cell\\b\\s*(.*)$', 'i');
|
||||
const END = new RegExp('^' + STARS + '\\s*End\\b.*$', 'i');
|
||||
const ATTR = /^([a-zA-Z][\w-]*)(?::(?:"([^"]*)"|'([^']*)'|(.*)))?$/;
|
||||
const DUR = /^\d+(?:ms|s|m)?$/;
|
||||
const ID_KEYS = ['id', 'title', 'name', 'cell', 'file', 'label'];
|
||||
const T_KEYS = ['t', 'timeout', 'duration', 'time'];
|
||||
const RST_KEYS = ['rst', 'reset'];
|
||||
const lines = text.split('\n');
|
||||
if (lines.length && lines[lines.length - 1] === '') lines.pop();
|
||||
const cells = [];
|
||||
let i = 0;
|
||||
while (i < lines.length && lines[i].trim() === '') i++;
|
||||
while (i < lines.length) {
|
||||
const m = CELL.exec(lines[i]);
|
||||
if (!m) { i++; continue; }
|
||||
const tokens = tokenizeCellAttrsHtml(m[1] || '');
|
||||
let lang = null;
|
||||
let title = '';
|
||||
const attrs = [];
|
||||
let bareReset = false;
|
||||
const titleParts = [];
|
||||
for (const tok of tokens) {
|
||||
const lower = tok.toLowerCase();
|
||||
if (RST_KEYS.indexOf(lower) >= 0) { bareReset = true; continue; }
|
||||
const am = ATTR.exec(tok);
|
||||
if (am && tok.indexOf(':') >= 0) {
|
||||
const key = am[1].toLowerCase();
|
||||
const value = am[2] !== undefined ? am[2] : am[3] !== undefined ? am[3] : (am[4] || '');
|
||||
const lc = evalLangAlias(key);
|
||||
if (lc) {
|
||||
if (!lang) lang = lc;
|
||||
if (!title && value) title = value;
|
||||
continue;
|
||||
}
|
||||
if (ID_KEYS.indexOf(key) >= 0) { if (!title) title = value; continue; }
|
||||
if (T_KEYS.indexOf(key) >= 0) { attrs.push('t=' + value); continue; }
|
||||
if (RST_KEYS.indexOf(key) >= 0) { attrs.push('rst'); continue; }
|
||||
continue;
|
||||
}
|
||||
const lc = evalLangAlias(tok);
|
||||
if (lc && !lang) { lang = lc; continue; }
|
||||
if (DUR.test(tok)) { attrs.push('t=' + tok); continue; }
|
||||
titleParts.push(tok);
|
||||
}
|
||||
if (!title && titleParts.length) title = titleParts.join(' ');
|
||||
if (bareReset) attrs.push('rst');
|
||||
lang = lang || 'py';
|
||||
i++;
|
||||
const codeLines = [];
|
||||
while (i < lines.length) {
|
||||
if (END.test(lines[i])) { i++; break; }
|
||||
if (CELL.test(lines[i])) break;
|
||||
codeLines.push(lines[i]);
|
||||
i++;
|
||||
}
|
||||
while (codeLines.length && codeLines[codeLines.length - 1].trim() === '') codeLines.pop();
|
||||
cells.push({ lang, title, attrs, code: codeLines.join('\n') });
|
||||
while (i < lines.length && lines[i].trim() === '') i++;
|
||||
}
|
||||
return cells;
|
||||
}
|
||||
|
||||
function parseEvalCellsBegin(text) {
|
||||
const STARS = '\\*{2,}';
|
||||
const BEGIN = new RegExp('^' + STARS + '\\s*Begin\\b\\s*(\\S+)?\\s*$', 'i');
|
||||
const END = new RegExp('^' + STARS + '\\s*End\\b.*$', 'i');
|
||||
|
||||
@@ -1,23 +1,18 @@
|
||||
Run code in a persistent kernel using codeblock cells.
|
||||
|
||||
<instruction>
|
||||
Each cell is wrapped between `*** Begin <LANG>` and `*** End <LANG>`:
|
||||
Each cell starts with a single header line and runs until the next header (or end of input):
|
||||
|
||||
```
|
||||
*** Begin PY
|
||||
*** Title: optional title
|
||||
*** Timeout: 10s
|
||||
*** Reset
|
||||
*** Cell py:"optional title" t:10s rst
|
||||
print("hi")
|
||||
*** End PY
|
||||
```
|
||||
|
||||
- **Language**: {{#if py}}`PY` for Python{{/if}}{{#ifAll py js}}, {{/ifAll}}{{#if js}}`JS` / `TS` for JavaScript{{/if}}. The opening `<LANG>` and closing `<LANG>` **MUST** match.
|
||||
- **Attributes** (optional, in any order, immediately after `*** Begin`):
|
||||
- `*** Title: …` — cell title shown in the UI.
|
||||
- `*** Timeout: <duration>` — per-cell timeout. Digits with optional `ms` / `s` / `m` units (e.g. `500ms`, `15s`, `2m`). Default 30s.
|
||||
- `*** Reset` — wipe this cell's own language kernel before running.{{#ifAll py js}} Other languages are untouched.{{/ifAll}}
|
||||
- Anything between the last attribute and `*** End <LANG>` is the cell's code, verbatim.
|
||||
- **Language + title**: `<lang>:"<title>"` — {{#if py}}`py` for Python{{/if}}{{#ifAll py js}}, {{/ifAll}}{{#if js}}`js` for JavaScript{{/if}}. Title may be empty (`py:""`).
|
||||
- **Attributes** (optional, in this order, after the language+title):
|
||||
- `t:<duration>` — per-cell timeout. Digits with optional `ms` / `s` / `m` units (e.g. `500ms`, `15s`, `2m`). Default 30s.
|
||||
- `rst` — wipe this cell's own language kernel before running.{{#ifAll py js}} Other languages are untouched.{{/ifAll}}
|
||||
- Anything after the header line, up to the next `*** Cell` header, is the cell's code, verbatim.
|
||||
- Stack multiple cells back-to-back; blank lines between cells are ignored.
|
||||
|
||||
**Work incrementally:**
|
||||
@@ -60,30 +55,22 @@ Cells render like a Jupyter notebook. `display(value)` renders non-presentable d
|
||||
</output>
|
||||
|
||||
<caution>
|
||||
- In session mode, use `*** Reset` on a cell to wipe its language's kernel before running.{{#ifAll py js}} Reset is per-language: a python cell's `*** Reset` does not touch the JavaScript kernel and vice versa.{{/ifAll}}
|
||||
- In session mode, use `rst` on a cell to wipe its language's kernel before running.{{#ifAll py js}} Reset is per-language: a python cell's `rst` does not touch the JavaScript kernel and vice versa.{{/ifAll}}
|
||||
{{#if js}}- **js**: the VM exposes a selective `process` subset, Web APIs, `Buffer`, `fs/promises`, and the `Bun` global.
|
||||
{{/if}}</caution>
|
||||
|
||||
<example>
|
||||
{{#if py}}*** Begin PY
|
||||
*** Title: imports
|
||||
*** Timeout: 10s
|
||||
{{#if py}}*** Cell py:"imports" t:10s
|
||||
import json
|
||||
from pathlib import Path
|
||||
*** End PY
|
||||
|
||||
*** Begin PY
|
||||
*** Title: load config
|
||||
*** Cell py:"load config"
|
||||
data = json.loads(read('package.json'))
|
||||
display(data)
|
||||
*** End PY
|
||||
{{/if}}{{#ifAll py js}}
|
||||
{{/ifAll}}{{#if js}}*** Begin JS
|
||||
*** Title: js summary
|
||||
*** Reset
|
||||
{{/ifAll}}{{#if js}}*** Cell js:"summary" rst
|
||||
const data = JSON.parse(await read('package.json'));
|
||||
display(data);
|
||||
return data.name;
|
||||
*** End JS
|
||||
{{/if}}
|
||||
</example>
|
||||
|
||||
@@ -26,7 +26,7 @@ export const EVAL_DEFAULT_PREVIEW_LINES = 10;
|
||||
|
||||
export const evalSchema = Type.Object({
|
||||
input: Type.String({
|
||||
description: "eval input as a sequence of `*** Begin <LANG>` cell headers followed by code",
|
||||
description: 'eval input as a sequence of `*** Cell <lang>:"title"` cell headers followed by code',
|
||||
}),
|
||||
});
|
||||
export type EvalToolParams = Static<typeof evalSchema>;
|
||||
|
||||
@@ -3,191 +3,156 @@ import { parseEvalInput } from "../../src/eval/parse";
|
||||
|
||||
describe("parseEvalInput", () => {
|
||||
it("parses a single cell with title and timeout", () => {
|
||||
const result = parseEvalInput(`*** Begin PY
|
||||
*** Title: setup
|
||||
*** Timeout: 15s
|
||||
const result = parseEvalInput(`*** Cell py:"setup" t:10s
|
||||
print("hi")
|
||||
*** End PY
|
||||
`);
|
||||
|
||||
expect(result.cells).toHaveLength(1);
|
||||
expect(result.cells[0]).toMatchObject({
|
||||
index: 0,
|
||||
title: "setup",
|
||||
code: 'print("hi")',
|
||||
language: "python",
|
||||
languageOrigin: "header",
|
||||
timeoutMs: 15_000,
|
||||
timeoutMs: 10_000,
|
||||
reset: false,
|
||||
code: 'print("hi")',
|
||||
});
|
||||
});
|
||||
|
||||
it("treats *** Reset as a per-cell kernel wipe", () => {
|
||||
const result = parseEvalInput(`*** Begin PY
|
||||
*** Title: bootstrap
|
||||
*** Reset
|
||||
it("treats rst as a per-cell kernel wipe", () => {
|
||||
const result = parseEvalInput(`*** Cell py:"bootstrap"
|
||||
import json
|
||||
*** End PY
|
||||
*** Begin JS
|
||||
*** Reset
|
||||
const x = 1;
|
||||
*** End JS
|
||||
`);
|
||||
|
||||
*** Cell js:"" rst
|
||||
const x = 1;
|
||||
`);
|
||||
expect(result.cells).toHaveLength(2);
|
||||
expect(result.cells[0]).toMatchObject({ language: "python", title: "bootstrap", reset: true });
|
||||
expect(result.cells[1]).toMatchObject({ language: "js", reset: true, title: undefined });
|
||||
expect(result.cells[0].reset).toBe(false);
|
||||
expect(result.cells[1].reset).toBe(true);
|
||||
expect(result.cells[1].language).toBe("js");
|
||||
});
|
||||
|
||||
it("accepts JS, TS, and PY language tokens (case-insensitive)", () => {
|
||||
const result = parseEvalInput(`*** Begin TS
|
||||
it("accepts case-insensitive language tokens (lenient)", () => {
|
||||
const result = parseEvalInput(`*** Cell JS:""
|
||||
const a = 1;
|
||||
*** End TS
|
||||
*** Begin py
|
||||
*** Cell PY:""
|
||||
print("py")
|
||||
*** End py
|
||||
`);
|
||||
|
||||
expect(result.cells.map(c => c.language)).toEqual(["js", "python"]);
|
||||
expect(result.cells).toHaveLength(2);
|
||||
expect(result.cells[0].language).toBe("js");
|
||||
expect(result.cells[1].language).toBe("python");
|
||||
});
|
||||
|
||||
it("parses millisecond, second, and minute durations", () => {
|
||||
const result = parseEvalInput(`*** Begin PY
|
||||
*** Timeout: 500ms
|
||||
const result = parseEvalInput(`*** Cell py:"a" t:500ms
|
||||
a = 1
|
||||
*** End PY
|
||||
*** Begin PY
|
||||
*** Timeout: 5
|
||||
*** Cell py:"b" t:5
|
||||
a = 2
|
||||
*** End PY
|
||||
*** Begin PY
|
||||
*** Timeout: 2m
|
||||
*** Cell py:"c" t:2m
|
||||
a = 3
|
||||
*** End PY
|
||||
`);
|
||||
|
||||
expect(result.cells.map(c => c.timeoutMs)).toEqual([500, 5_000, 120_000]);
|
||||
});
|
||||
|
||||
it("attribute order is flexible and only the first wins", () => {
|
||||
const result = parseEvalInput(`*** Begin PY
|
||||
*** Timeout: 1s
|
||||
*** Title: first
|
||||
*** Title: ignored
|
||||
*** Timeout: 9s
|
||||
print(1)
|
||||
*** End PY
|
||||
`);
|
||||
|
||||
expect(result.cells[0]).toMatchObject({ title: "first", timeoutMs: 1_000 });
|
||||
expect(result.cells.map(c => c.timeoutMs)).toEqual([500, 5000, 120_000]);
|
||||
});
|
||||
|
||||
it("preserves blank lines inside the cell body", () => {
|
||||
const result = parseEvalInput(`*** Begin JS
|
||||
const result = parseEvalInput(`*** Cell js:""
|
||||
const x = 1;
|
||||
|
||||
const y = 2;
|
||||
*** End JS
|
||||
`);
|
||||
|
||||
expect(result.cells).toHaveLength(1);
|
||||
expect(result.cells[0].code).toBe("const x = 1;\n\nconst y = 2;");
|
||||
});
|
||||
|
||||
it("treats blank lines between cells as separators, not code", () => {
|
||||
const result = parseEvalInput(`*** Begin PY
|
||||
const result = parseEvalInput(`*** Cell py:""
|
||||
print("a")
|
||||
*** End PY
|
||||
|
||||
|
||||
*** Begin PY
|
||||
*** Cell py:""
|
||||
print("b")
|
||||
*** End PY
|
||||
`);
|
||||
|
||||
expect(result.cells).toHaveLength(2);
|
||||
expect(result.cells[0].code).toBe('print("a")');
|
||||
expect(result.cells[1].code).toBe('print("b")');
|
||||
});
|
||||
|
||||
it("falls back to language sniffing when the begin marker has no recognized language", () => {
|
||||
const result = parseEvalInput(`*** Begin RUBY
|
||||
it("falls back to language sniffing when the header has no recognized language", () => {
|
||||
// Bare `ruby` doesn't match LANG_TITLE, but the parser is lenient and
|
||||
// falls back to body sniffing.
|
||||
const result = parseEvalInput(`*** Cell ruby:"x"
|
||||
const x = 1;
|
||||
console.log(x);
|
||||
*** End
|
||||
`);
|
||||
expect(result.cells[0]).toMatchObject({ language: "js", languageOrigin: "default" });
|
||||
expect(result.cells).toHaveLength(1);
|
||||
expect(result.cells[0].languageOrigin).toBe("default");
|
||||
expect(result.cells[0].language).toBe("js");
|
||||
});
|
||||
|
||||
it("accepts `**Begin` (two stars) as well as `***Begin`", () => {
|
||||
const result = parseEvalInput(`**Begin PY
|
||||
it("accepts `**Cell` (two stars) as well as `***Cell`", () => {
|
||||
const result = parseEvalInput(`**Cell py:""
|
||||
print(1)
|
||||
**End
|
||||
`);
|
||||
expect(result.cells[0]).toMatchObject({ language: "python", code: "print(1)" });
|
||||
expect(result.cells).toHaveLength(1);
|
||||
expect(result.cells[0].language).toBe("python");
|
||||
});
|
||||
|
||||
it("implicitly closes a cell when a new *** Begin appears without an *** End", () => {
|
||||
const result = parseEvalInput(`*** Begin PY
|
||||
it("implicitly closes a cell when a new *** Cell appears without *** End", () => {
|
||||
const result = parseEvalInput(`*** Cell py:""
|
||||
print("a")
|
||||
*** Begin JS
|
||||
*** Cell js:""
|
||||
const x = 1;
|
||||
*** End JS
|
||||
`);
|
||||
expect(result.cells).toHaveLength(2);
|
||||
expect(result.cells[0]).toMatchObject({ language: "python", code: 'print("a")' });
|
||||
expect(result.cells[1]).toMatchObject({ language: "js", code: "const x = 1;" });
|
||||
expect(result.cells[0].code).toBe('print("a")');
|
||||
expect(result.cells[1].code).toBe("const x = 1;");
|
||||
});
|
||||
|
||||
it("ignores the language token on `*** End` (leniency)", () => {
|
||||
const result = parseEvalInput(`*** Begin PY
|
||||
it("tolerates `*** End` as an optional cell terminator (GPT quirk)", () => {
|
||||
const result = parseEvalInput(`*** Cell py:""
|
||||
print(1)
|
||||
*** End JS
|
||||
*** End
|
||||
*** Cell js:""
|
||||
const x = 1;
|
||||
*** End
|
||||
`);
|
||||
expect(result.cells[0]).toMatchObject({ language: "python", code: "print(1)" });
|
||||
expect(result.cells).toHaveLength(2);
|
||||
expect(result.cells[0].code).toBe("print(1)");
|
||||
expect(result.cells[1].code).toBe("const x = 1;");
|
||||
});
|
||||
|
||||
it("ignores anything trailing `*** End` (leniency)", () => {
|
||||
const result = parseEvalInput(`*** Cell py:""
|
||||
print(1)
|
||||
*** End py
|
||||
`);
|
||||
expect(result.cells).toHaveLength(1);
|
||||
expect(result.cells[0].code).toBe("print(1)");
|
||||
});
|
||||
|
||||
it("accepts long-form language aliases (Python, JavaScript, TypeScript)", () => {
|
||||
const result = parseEvalInput(`*** Begin Python
|
||||
const result = parseEvalInput(`*** Cell Python:""
|
||||
print(1)
|
||||
*** End
|
||||
*** begin javascript
|
||||
const x = 1;
|
||||
*** End
|
||||
*** Cell JavaScript:""
|
||||
const a = 1;
|
||||
*** Cell TypeScript:""
|
||||
const b = 2;
|
||||
`);
|
||||
expect(result.cells.map(c => c.language)).toEqual(["python", "js"]);
|
||||
});
|
||||
|
||||
it("tolerates whitespace and case variations on directives", () => {
|
||||
const result = parseEvalInput(`***\tBegin\tPY
|
||||
***title: tabby
|
||||
***\tTimeout:\t250ms
|
||||
***reset
|
||||
print(1)
|
||||
***End
|
||||
`);
|
||||
expect(result.cells[0]).toMatchObject({
|
||||
title: "tabby",
|
||||
timeoutMs: 250,
|
||||
reset: true,
|
||||
language: "python",
|
||||
code: "print(1)",
|
||||
});
|
||||
expect(result.cells.map(c => c.language)).toEqual(["python", "js", "js"]);
|
||||
});
|
||||
|
||||
it("implicitly closes the final cell at EOF when *** End is missing", () => {
|
||||
const result = parseEvalInput(`*** Begin PY
|
||||
const result = parseEvalInput(`*** Cell py:""
|
||||
print(1)
|
||||
`);
|
||||
expect(result.cells).toHaveLength(1);
|
||||
expect(result.cells[0]).toMatchObject({ language: "python", code: "print(1)" });
|
||||
expect(result.cells[0].code).toBe("print(1)");
|
||||
});
|
||||
|
||||
it("treats bare code without any *** Begin as a single implicit cell", () => {
|
||||
it("treats bare code without any *** Cell as a single implicit cell", () => {
|
||||
const result = parseEvalInput(`def greet():\n print('hi')\ngreet()\n`);
|
||||
expect(result.cells).toHaveLength(1);
|
||||
expect(result.cells[0]).toMatchObject({
|
||||
language: "python",
|
||||
languageOrigin: "default",
|
||||
language: "python",
|
||||
code: "def greet():\n print('hi')\ngreet()",
|
||||
});
|
||||
});
|
||||
@@ -204,22 +169,107 @@ print(1)
|
||||
|
||||
it("rejects invalid duration", () => {
|
||||
expect(() =>
|
||||
parseEvalInput(`*** Begin PY
|
||||
*** Timeout: forever
|
||||
parseEvalInput(`*** Cell py:"" t:forever
|
||||
print(1)
|
||||
*** End PY
|
||||
`),
|
||||
).toThrow(/invalid duration/);
|
||||
});
|
||||
|
||||
it("supports titles with embedded spaces", () => {
|
||||
const result = parseEvalInput(`*** Cell py:"load and validate config"
|
||||
print(1)
|
||||
`);
|
||||
expect(result.cells).toHaveLength(1);
|
||||
expect(result.cells[0].title).toBe("load and validate config");
|
||||
});
|
||||
|
||||
it('treats empty title (`py:""`) as no title', () => {
|
||||
const result = parseEvalInput(`*** Cell py:""
|
||||
print(1)
|
||||
`);
|
||||
expect(result.cells).toHaveLength(1);
|
||||
expect(result.cells[0].title).toBeUndefined();
|
||||
expect(result.cells[0].language).toBe("python");
|
||||
});
|
||||
|
||||
it("accepts bare language token without title (lenient form)", () => {
|
||||
// Parser is more permissive than the lark; bare `py` is accepted
|
||||
// even though the canonical form is `py:"title"`.
|
||||
const result = parseEvalInput(`*** Cell py
|
||||
print(1)
|
||||
`);
|
||||
expect(result.cells).toHaveLength(1);
|
||||
expect(result.cells[0].language).toBe("python");
|
||||
expect(result.cells[0].title).toBeUndefined();
|
||||
});
|
||||
|
||||
describe("attribute leniency (accepted but not advertised)", () => {
|
||||
it("accepts id aliases (title/name/cell/file/label)", () => {
|
||||
const aliases = ["title", "name", "cell", "file", "label"];
|
||||
for (const alias of aliases) {
|
||||
const result = parseEvalInput(`*** Cell py ${alias}:"hi"\nprint(1)\n`);
|
||||
expect(result.cells[0].title).toBe("hi");
|
||||
}
|
||||
});
|
||||
|
||||
it("accepts t aliases (timeout/duration/time)", () => {
|
||||
const aliases = ["timeout", "duration", "time"];
|
||||
for (const alias of aliases) {
|
||||
const result = parseEvalInput(`*** Cell py ${alias}:5s\nprint(1)\n`);
|
||||
expect(result.cells[0].timeoutMs).toBe(5000);
|
||||
}
|
||||
});
|
||||
|
||||
it("accepts `reset` as an alias for `rst`", () => {
|
||||
const result = parseEvalInput(`*** Cell py reset\nprint(1)\n`);
|
||||
expect(result.cells[0].reset).toBe(true);
|
||||
});
|
||||
|
||||
it("accepts `rst:true|false|1|0|yes|no|on|off`", () => {
|
||||
for (const v of ["true", "1", "yes", "on"]) {
|
||||
const r = parseEvalInput(`*** Cell py rst:${v}\nx\n`);
|
||||
expect(r.cells[0].reset).toBe(true);
|
||||
}
|
||||
for (const v of ["false", "0", "no", "off"]) {
|
||||
const r = parseEvalInput(`*** Cell py rst:${v}\nx\n`);
|
||||
expect(r.cells[0].reset).toBe(false);
|
||||
}
|
||||
});
|
||||
|
||||
it("rejects an invalid rst value", () => {
|
||||
expect(() => parseEvalInput(`*** Cell py rst:maybe\nx\n`)).toThrow(/invalid rst/);
|
||||
});
|
||||
|
||||
it("accepts single-quoted titles (`id:'hi'`)", () => {
|
||||
const result = parseEvalInput(`*** Cell py id:'hello world'\nprint(1)\n`);
|
||||
expect(result.cells[0].title).toBe("hello world");
|
||||
});
|
||||
|
||||
it("accepts a bare positional duration token (e.g. `30s`)", () => {
|
||||
const result = parseEvalInput(`*** Cell py 2m\nprint(1)\n`);
|
||||
expect(result.cells[0].timeoutMs).toBe(120_000);
|
||||
});
|
||||
|
||||
it("first occurrence wins for repeated keys (canonical or alias)", () => {
|
||||
const result = parseEvalInput(`*** Cell py id:"first" name:"second" t:1s timeout:5s\nprint(1)\n`);
|
||||
expect(result.cells[0].title).toBe("first");
|
||||
expect(result.cells[0].timeoutMs).toBe(1000);
|
||||
});
|
||||
|
||||
it("unclassified bare tokens accumulate as a positional title", () => {
|
||||
const result = parseEvalInput(`*** Cell py setup phase\nprint(1)\n`);
|
||||
expect(result.cells[0].title).toBe("setup phase");
|
||||
});
|
||||
});
|
||||
|
||||
describe("*** Abort recovery sentinel (harmony-leak mitigation)", () => {
|
||||
it("drops the in-progress cell and stops parsing", () => {
|
||||
const result = parseEvalInput(`*** Begin PY
|
||||
const result = parseEvalInput(`*** Cell py:""
|
||||
print("a")
|
||||
*** End PY
|
||||
*** Begin JS
|
||||
*** Cell js:""
|
||||
const partial = 1; /* contamination starts mid-cell */
|
||||
*** Abort
|
||||
*** Begin TS
|
||||
*** Cell js:""
|
||||
const never_runs = 1;
|
||||
`);
|
||||
expect(result.aborted).toBe(true);
|
||||
@@ -228,16 +278,19 @@ const never_runs = 1;
|
||||
expect(result.cells[0].code).toBe('print("a")');
|
||||
});
|
||||
|
||||
it("between cells: keeps preceding cells, sets aborted, drops trailing cells", () => {
|
||||
const result = parseEvalInput(`*** Begin PY
|
||||
it("`*** End` before `*** Abort` preserves the closed cell", () => {
|
||||
// Without `*** End`, the parser can't tell whether the cell was
|
||||
// complete before contamination — by design, since `*** End` is
|
||||
// optional and undocumented. Explicit `*** End` is the GPT quirk
|
||||
// that signals "cell is closed, abort is between cells".
|
||||
const result = parseEvalInput(`*** Cell py:""
|
||||
print("a")
|
||||
*** End PY
|
||||
*** End
|
||||
|
||||
*** Abort
|
||||
|
||||
*** Begin PY
|
||||
*** Cell py:""
|
||||
print("never")
|
||||
*** End PY
|
||||
`);
|
||||
expect(result.aborted).toBe(true);
|
||||
expect(result.cells).toHaveLength(1);
|
||||
@@ -252,53 +305,48 @@ print("never")
|
||||
expect(result.cells).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("appended sentinel from harmony-leak truncation: abort flag set, prior cell preserved", () => {
|
||||
// Mirrors the exact shape harmony-leak emits: original input truncated
|
||||
// at the contaminated line, then "\n*** Abort\n" appended.
|
||||
const truncated = `*** Begin PY\nprint("ok")\n*** End PY\n*** Abort\n`;
|
||||
it("appended sentinel from harmony-leak truncation: abort flag set, prior cell dropped", () => {
|
||||
const truncated = `*** Cell py:""\nprint("ok")\n*** Abort\n`;
|
||||
const result = parseEvalInput(truncated);
|
||||
expect(result.aborted).toBe(true);
|
||||
expect(result.cells).toHaveLength(1);
|
||||
expect(result.cells[0].code).toBe('print("ok")');
|
||||
expect(result.cells).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("absent sentinel: aborted is undefined (not falsely set)", () => {
|
||||
const result = parseEvalInput(`*** Begin PY
|
||||
const result = parseEvalInput(`*** Cell py:""
|
||||
print(1)
|
||||
*** End PY
|
||||
`);
|
||||
expect(result.aborted).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
it("does not crash on stray non-marker lines between cells", () => {
|
||||
// Regression: prior to fix, the outer parse loop unconditionally ran
|
||||
// `BEGIN_RE.exec(lines[i])!` on whatever followed a closed cell, so a
|
||||
// stray line — common in harmony-leak fragments — threw
|
||||
// "null is not an object (evaluating 'BEGIN_RE.exec(lines[i])[1]')".
|
||||
const result = parseEvalInput(`*** Begin PY
|
||||
// Regression for "null is not an object (evaluating
|
||||
// 'BEGIN_RE.exec(lines[i])[1]')" — stray fragments must not crash.
|
||||
// Without `*** End`, the stray junk folds into the prior cell's body;
|
||||
// the contract for this test is just "don't crash".
|
||||
const result = parseEvalInput(`*** Cell py:""
|
||||
print("a")
|
||||
*** End PY
|
||||
stray junk that is not a marker
|
||||
*** Begin PY
|
||||
*** Cell py:""
|
||||
print("b")
|
||||
*** End PY
|
||||
`);
|
||||
expect(result.aborted).toBeUndefined();
|
||||
expect(result.cells).toHaveLength(2);
|
||||
expect(result.cells[0].code).toBe('print("a")');
|
||||
expect(result.cells[0].code).toContain('print("a")');
|
||||
expect(result.cells[1].code).toBe('print("b")');
|
||||
});
|
||||
|
||||
it("does not crash on trailing stray content after the final cell", () => {
|
||||
const result = parseEvalInput(`*** Begin PY
|
||||
const result = parseEvalInput(`*** Cell py:""
|
||||
print(1)
|
||||
*** End PY
|
||||
leftover model chatter
|
||||
more junk
|
||||
`);
|
||||
expect(result.aborted).toBeUndefined();
|
||||
expect(result.cells).toHaveLength(1);
|
||||
expect(result.cells[0].code).toBe("print(1)");
|
||||
// Stray lines fold into the cell body (no terminator), which is fine —
|
||||
// the contract is just "don't crash".
|
||||
expect(result.cells[0].code).toContain("print(1)");
|
||||
});
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user