Commit Graph

254 Commits

Author SHA1 Message Date
Kigbnajd 830d0a4dea fix(eval): allow awaiting Python workflow helpers 2026-08-12 19:18:25 +02:00
roboomp 78465e7aaf fix(eval): re-validate memoized kernel runner path
ensureRunnerScript() memoized the staged runner path in a module-level
RUNNER_SCRIPT_PATH and returned it on the warm path with no existence
check, defeating the cold path's own self-heal. If the tmpdir cache
(os.tmpdir()/omp-<lang>-runner) was cleared mid-session (e.g. a macOS
periodic clean_tmps sweep), the process kept handing out a path to a
deleted file and every later Python/Julia/Ruby kernel start failed to
spawn until restart.

Extract the three copy-pasted ensureRunnerScript implementations into a
single stageRunnerScript helper that re-checks fs.existsSync before
reusing the memoized path, so a swept runner is re-staged on the next
call. Route the Python, Julia, and Ruby kernels through it.

Fixes #8140
2026-08-10 07:03:13 +00:00
can1357 71af89cbaa refactor(coding-agent): extracted generic kernel session registry
- Python, Ruby and Julia each carried their own copy of the same session
  maps, acquire/reset/replace/dispose lifecycle and executeOnSession; one
  generic registry now owns it, parameterized by a per-language descriptor.
- Julia additionally re-implemented seven executor-base helpers locally; those
  copies are gone and executor-base gained sparse managed-env and timeout
  resolver hooks so Julia's differing behavior survives unchanged.
- All twelve exported entry points keep their names and signatures.
2026-08-08 06:32:00 +02:00
metaphorics 85177a29ec fix(coding-agent): fail loudly when no POSIX bash exists; add stdin-isolation regression test 2026-08-05 10:42:36 +00:00
metaphorics f03581dc22 fix(coding-agent): stop eval shell children inheriting the runner control channel; resolve Git Bash on Windows 2026-08-05 10:24:24 +00:00
Slava Zavadsky 76f04c4c77 fix(coding-agent/eval): reject legacy model kwarg in Julia agent() prelude
Python/Ruby raise on the removed keyword via their keyword-only
signatures; the Julia helper's kwargs... catch-all silently swallowed
agent("..."; model="default") and forwarded it to the bridge, where
the "+": "delete" strip discarded it. A caller could keep thinking a
model override was honored. Reject model loudly, matching the strict
removal decision on #7621.
2026-08-04 17:03:04 -04:00
Slava Zavadsky b8779dae63 fix(coding-agent/eval): remove per-call model override from agent() (#6438)
Completes the maintainer's removal of per-call model selection from
subagent spawns (9f8aa87dbf removed it from the task tool and the
model-facing agent() docs/prompt, but the eval agent() runtime and all
four preludes still accepted and forwarded a per-call model).

Subagents now always resolve through the selected agent's frontmatter
model and settings, so an explicit model: "default" can no longer
silently route children onto the parent session model.

- agent-bridge: drops "model?" from agentArgsSchema and the request
  forward; adds "+": "delete" so a legacy model argument is stripped
  (same contract as the task wire schemas).
- JS/Python/Ruby/Julia preludes: remove the model parameter from
  agent(); completion()'s tier selector is unchanged.
- docs (tools/eval.md, python-repl.md) updated to the removed surface.

Refs #6438
2026-08-04 08:35:01 -04:00
can1357 bc39ffa265 feat: introduced omptype validation package and migrated workspace dependencies
- Introduce `@oh-my-pi/omptype` as a new ArkType-compatible schema validation package featuring a lazy JIT runtime, JSON Schema emission, and compatibility adapters.
- Replace `arktype` across workspace packages and test utilities with `@oh-my-pi/omptype`.
- Add benchmark suites, tests, and documentation for the new validation engine and adapters.
- Update workspace build, test runner, and release configurations to include the new package.
2026-08-03 21:56:48 +02:00
can1357 b48aeacf25 fix(coding-agent/eval): made cells own every bridge call they start
- Split the Python bridge signal: the raw abort reaches tools (so
  subagents die with the turn) while a shielded signal governs how long
  the host waits, so a cancel can no longer settle a cell on top of a
  still-running isolation merge.
- Mirrored the contract in the JS runtime: abort in-flight tool calls
  immediately, then drain any deferExternalAbort phase before killing
  the worker, and refuse new bridge calls once cancelled.
- Held a finished worker result until the run's tool calls drain. A
  floated agent() previously settled the cell at once, dropping the
  run's abort listener and leaving the subagent running with nothing
  able to cancel it.
- Added regression coverage for all three, each verified to fail
  without its fix.
2026-08-03 19:55:20 +02:00
can1357 c7a3010d7c fix(coding-agent/eval): ensured tool bridge receives unshielded abort signal
- Prevent eval agent bridge calls from inheriting the kernel abort shield.
- Unset maxRuntimeMs in runEvalAgent to properly inherit task.maxRuntimeMs.
2026-08-03 18:57:22 +02:00
can1357 46c66ac03a fix(setup): always resolve interpreter during setup probe 2026-08-02 20:53:34 +02:00
roboomp 3fab985c14 fix(eval): preserved console for conpty python kernels
Combined native HWND and stdio TTY evidence so compiled Windows Terminal launches no longer set CREATE_NO_WINDOW for the Python eval kernel.

Fixes #7343
2026-08-02 05:05:52 +00:00
can1357 4d45e8154a fix(eval): consulted starting sessions when scoping kernel resets
- resolveOwnerScopedSessionKey's getOwners in the Python and JS executors
  only read live sessions, so a subagent reset issued while the shared
  kernel was still starting resolved to the base key, awaited the
  parent's startup, and shut its brand-new kernel down.
- Python and JS starting sessions are now owner-bearing records like
  Ruby/Julia's: owners attach synchronously before startup resolves,
  getOwners and per-owner disposal consult them, and the final
  sessions.set is identity-guarded so a disposed starting record cannot
  resurrect its kernel.
- Regression: deferred PythonKernel.start proves a concurrent subagent
  reset forks immediately and never reaps the parent's starting kernel
  (fails with the previous getOwners).
2026-07-31 20:15:07 +02:00
can1357 ccc9b900cf fix(eval): forked subagent kernel resets away from shared sessions
- Subagents inherit the parent's eval session id, so a child's
  reset: true destroyed the co-owned kernel and every sibling's
  interpreter state mid-session.
- resolveOwnerScopedSessionKey now routes a reset from a non-exclusive
  owner onto a deterministic per-owner fork key: the requester gets a
  fresh private kernel, co-owners keep the shared one, and the fork
  stays sticky for that owner until its teardown reaps it.
- Applied across Python, JavaScript, Ruby, and Julia executors; JS
  contexts gained an owner registry plus disposeVmContextsByOwner,
  wired into EvalRunner.disposeKernels and SDK session teardown.
- Covered by pure key-resolution contracts and an end-to-end JS test:
  co-owner reset forks, shared state survives, fork is sticky, and
  per-owner dispose reaps only the fork.
2026-07-31 20:13:23 +02:00
can1357 14dc5d4fcc fix(coding-agent/eval): bypassed environment proxies for python bridge calls
- Configure the python eval prelude to use a custom urllib opener that ignores environment proxies.
- Add test coverage verifying parallel tool bridge calls succeed when proxy variables are set.
2026-07-30 16:19:48 +02:00
can1357 d16a251777 chore: reorg tests 2026-07-27 16:43:53 +02:00
can1357 4eb94125b2 fix(coding-agent/eval): filtered internal runner frames from python cell error tracebacks
- Filter out runner-internal frames from runtime exception tracebacks to start at user code.
- Omit full tracebacks for cell syntax errors to render only the caret display with `<cell>` filename.
2026-07-27 07:28:22 +02:00
usr-bin-roygbiv 9ceebb2a4d fix(workers): isolate compiled host selectors 2026-07-25 08:31:54 +00:00
usr-bin-roygbiv 8e1bffb60f style(coding-agent): format worker host changes 2026-07-25 07:50:26 +00:00
usr-bin-roygbiv 237692deab fix(computer-use): re-enter single worker host 2026-07-25 07:45:56 +00:00
usr-bin-roygbiv a7e988b604 fix(computer-use): isolate worker process entry 2026-07-25 07:13:12 +00:00
usr-bin-roygbiv f1fb05df80 fix(eval): statically link rejection interceptor 2026-07-25 04:24:25 +00:00
usr-bin-roygbiv fe985551c5 fix(eval): preserve process bootstrap boundary 2026-07-25 01:30:04 +00:00
usr-bin-roygbiv d9a2782258 fix(eval): preserve worker after floated rejection 2026-07-25 00:42:23 +00:00
roboomp 31098f9248 fix(tools): closed spilled output descriptors on error/abort paths
OutputSink.dump() was the only path that closed the spill Bun.FileSink.
The bash and Python executors re-throw on failure and their finally
blocks never closed the sink, so any large-output command that errored
leaked the artifact descriptor until an unrelated read (e.g. a SKILL.md
load) hit EMFILE.

Added an idempotent OutputSink.dispose() that closes the sink exactly
once (awaiting any in-flight sink creation, guarding post-finalize
resurrection) and wired it into every executor's finally block.

Fixes #6463
2026-07-24 03:44:12 +00:00
roboomp 03489d1ebe fix(eval): coalesced python kernel replacement
Tracked retained Python kernel generations and shared one replacement promise per dead generation. Reset and disposal now invalidate and drain replacement work before allowing a new session to take ownership.

Added deterministic fake-kernel coverage for concurrent callers, cancellation, reset, owner/global disposal, and independent cwd keys.

Fixes #6367
2026-07-23 19:00:08 +00:00
can1357 b6e68fd243 fix(coding-agent): updated mentions of explore -> scout 2026-07-23 00:12:07 +02:00
can1357 242bcbef8c revert #6130: doesn't really work well 2026-07-22 23:54:15 +02:00
Kormákur 6f75cf0d5d feat(coding-agent): add task tool capture-only apply control
Add an optional `apply` parameter to the `task` tool so
`isolated: true, apply: false` captures patch/branch artifacts without
applying changes to the parent checkout. Available as a flat top-level
control and per `tasks[]` item. Shares the task/eval isolation-to-executor
translation via a single `toStructuredSubagentIsolationControls` adapter.
2026-07-20 18:18:23 +00:00
vmcall d53cf023b0 fix(task): reconciled structured subagents with upstream
- Preserved the plan-mode capability clamp after upstream removed report_finding.
- Updated persisted-revival coverage for mounted xdev tool activation.
- Applied current formatter output to conflicted runtime files.
2026-07-17 17:38:12 +02:00
vmcall b4952e27c4 refactor(eval): removed dead isolation recovery formatter 2026-07-17 17:38:12 +02:00
vmcall 414ef80c41 fix(task): restored goal activation outside restricted sessions
- Preserved goal-mode tool injection for ordinary explicit tool lists.
- Kept plan-mode LSP and IRC unavailable under the host capability clamp.
- Added regressions for both capability boundaries.
2026-07-17 17:36:59 +02:00
vmcall d944879f21 feat(task): unified structured subagent execution
- Added per-invocation task schemas with strict and permissive validation.
- Shared task and eval agent policy, artifacts, isolation, and lifecycle handling.
- Enabled host-restricted plan-mode eval agents and persisted their capability clamp.

Fixes #5279
2026-07-17 17:36:59 +02:00
can1357 55f5ebec49 chore: reformat 2026-07-15 00:08:50 +02:00
can1357 a9c038818d feat(tools): removed separate selector args from read and grep APIs
- Removed `selector`/`sel` arguments from read and grep tool schemas and related execution arg handling.
- Reworked read and grep path processing to parse line selectors from `path` suffixes instead of separate fields, including inline range propagation.
- Updated delegation and execution call paths (including JS/Python preludes and executor tests) to pass selectors embedded in `path`.
- Updated read/grep prompt docs and changelog for the breaking inline-selector API, and removed obsolete selector-specific tests and expectations.
2026-07-15 00:04:02 +02:00
can1357 37b1263a26 Merge PR #5494: fix(eval): delegate Python URI reads to host resolver (@roboomp) 2026-07-14 23:11:12 +02:00
can1357 6b381eb765 Merge PR #5451: fix(eval): honor timeout zero and classify session deadlines (@roboomp) 2026-07-14 22:58:49 +02:00
roboomp eef79705a6 fix(eval): passed js uri selectors separately
- Kept opaque MCP resource paths unchanged during JS pagination.
- Sent JS line ranges through the read tool selector field.
- Covered the shipped JS prelude and updated the changelog.

Fixes #5353
2026-07-14 20:31:58 +00:00
roboomp 2bfdf0ab07 fix(eval): passed uri selectors separately
- Kept opaque MCP resource paths unchanged during pagination.
- Sent line ranges through the read tool selector field.
- Covered paged artifact and MCP reads in the Python prelude test.

Fixes #5353
2026-07-14 20:20:42 +00:00
roboomp 5d6fcff2f1 fix(eval): delegated python uri reads to host resolver
- Routed non-local URI reads through the session read tool.
- Preserved offset and limit as host line selectors.
- Covered artifact delegation with the shipped Python prelude.

Fixes #5353
2026-07-14 19:12:06 +00:00
roboomp 8e6d26b1e8 fix(eval): honored unlimited cell timeouts
- Disabled the eval watchdog when timeout is explicitly zero.
- Classified session deadline aborts as TimeoutError while preserving their message.
- Documented and tested both timeout contracts.

Fixes #5250
2026-07-14 17:33:56 +00:00
can1357 5e74444c27 fix(test): repair auth-storage-rotation merge resolution and normalize changelogs 2026-07-14 18:50:47 +02:00
can1357 7a2b027e47 fix(eval): keep completion aborts interruptible 2026-07-14 18:45:03 +02:00
can1357 5d28a319fd Merge PR #5015: fix(eval): shield Python agent bridge aborts (@roboomp) 2026-07-14 18:45:03 +02:00
can1357 bfdca36f44 test(coding-agent): use portable eval isolation probe 2026-07-14 18:41:16 +02:00
can1357 79d7b74e24 Merge PR #5327: fix(coding-agent): isolate eval runtimes from terminal (@masonc15) 2026-07-14 18:41:16 +02:00
can1357 f9f6ed9e8d feat(coding-agent): replaced legacy pi/ role alias prefix with
- Replaced legacy `pi/` role alias prefix with canonical `@` syntax across model resolution, documentation, and tests.
- Added support for bare `*` default alias and multiple alias prefix detection with custom role resolution in `resolveConfiguredRolePattern()`.
- Enhanced thinking suffix parsing to accept unambiguous abbreviations (minimum 2 characters) for effort and level selectors.
- Extended `resolveCliModel()` and `filterAvailableModelsByEnabledPatterns()` to accept settings parameter for role alias resolution from `--model` flag.
2026-07-13 23:26:33 +02:00
Colin Mason ffa879ba2c keep process cwd while other cells are live 2026-07-13 09:57:20 -04:00
Colin Mason 640d7c83b9 propagate ipc clone errors to eval cells 2026-07-13 09:57:20 -04:00
Colin Mason 95620ccd6c preserve async js fallback ladder 2026-07-13 09:57:20 -04:00