test(coding-agent/mcp): defend getter-based tool descriptions against signature regression

Built-in tools whose prompt-rendered metadata depends on settings
(`TaskTool`, `SearchToolBm25Tool`, `EditTool`) expose `description`/
`label` via getters that re-evaluate on every access. The skip
optimization in `#applyActiveToolsByName` is correctness-safe for these
because `#computeAppliedToolSignature` reads `tool.description` live
each call, so a settings flip mutates the rendered string and differs
the signature on the next refresh.

This contract was implicit; a future refactor that caches per-tool
description strings would silently break it. Defending it explicitly:

- Added a regression test that wires a getter onto a CustomTool's
  `description`, verifies `refreshMCPTools` skips while the underlying
  state is unchanged, then mutates the state (without changing tool
  object identity) and verifies the rebuild fires.
- Expanded the `#computeAppliedToolSignature` docstring to document
  the getter-based coverage path and the SDK-init-time closure
  constants in `sdk.ts` that genuinely cannot change at runtime
  (`repeatToolDescriptions`, `eagerTasks`, `intentField`,
  `mcpDiscoveryEnabled`, `secretsEnabled`).

Triggered by a review question on whether the skip breaks settings-
based prompt changes. It does not, but the property is non-obvious.
This commit is contained in:
Miroslav Drbal
2026-04-30 14:57:53 +02:00
parent 08cb3f8555
commit 792b799e16
2 changed files with 66 additions and 4 deletions
@@ -2309,10 +2309,20 @@ export class AgentSession {
* embeds these in the appended prompt under "## MCP Server Instructions".
* A server upgrade can change instructions while keeping tools identical.
*
* Inputs NOT covered: tool input schemas, memory instructions read from disk,
* and any other ambient state the rebuild closure reads. Callers must explicitly
* call `refreshBaseSystemPrompt()` after such side-effecting changes; see e.g.
* the memory hooks and `#syncEditToolModeAfterModelChange`.
* Settings-driven tool metadata is covered automatically: built-in tools that
* depend on settings expose `description`/`label` via getters (see `TaskTool`,
* `SearchToolBm25Tool`, `EditTool`), and the signature reads them live on every
* call - so a settings flip that mutates the rendered string differs the signature
* the next time `#applyActiveToolsByName` runs. Do not refactor `describeTool` to
* cache per-tool strings without preserving this property.
*
* Inputs NOT covered: tool input schemas; memory instructions read from disk;
* and SDK-init-time closure constants in `sdk.ts` (`repeatToolDescriptions`,
* `eagerTasks`, `intentField`, `mcpDiscoveryEnabled`, `secretsEnabled`). The
* closure-captured ones cannot change at runtime regardless of skip behavior.
* For everything else, callers must explicitly call `refreshBaseSystemPrompt()`
* after side-effecting changes; see e.g. the memory hooks and
* `#syncEditToolModeAfterModelChange`.
*/
#computeAppliedToolSignature(toolNames: string[], tools: AgentTool[]): string {
// Order-preserving join: any reorder must produce a different signature so
@@ -334,4 +334,56 @@ describe("AgentSession refreshMCPTools rebuild skipping", () => {
expect(rebuildCount).toBe(3);
});
it("rebuilds when a tool's getter-based description reflects new settings state", async () => {
// Built-in tools whose prompt-rendered metadata depends on settings expose
// `description` via getters that re-evaluate on every access (TaskTool reads
// task.disabledAgents/maxConcurrency/isolation.mode/simple/async.enabled,
// SearchToolBm25Tool reads the discoverable MCP tool count, EditTool resolves
// through the current edit-mode definition). The signature reads `tool.description`
// live each call, so a settings flip that mutates the rendered string MUST differ
// the signature on the next `#applyActiveToolsByName`. Defending this contract
// against a future refactor that caches per-tool description strings.
let rebuildCount = 0;
const { session } = newSession(
async toolNames => {
rebuildCount++;
return `tools:${toolNames.join(",")}`;
},
// Discovery-on so the dynamic tool participates in the registry segment too,
// matching the production pattern where `TaskTool` and friends are always
// reachable via the registry.
{ mcpDiscoveryEnabled: true },
);
// Reuse the initially-active MCP name so the tool stays in the active list
// across refreshes - we want to defend the path where `tool.description` is read
// for the active descriptionSegment, not just the registrySegment.
const settingState = { disabled: "none" };
const dynamicTool = createMcpCustomTool("mcp__nucleus_search", "nucleus", "search", "placeholder");
Object.defineProperty(dynamicTool, "description", {
get: () => `dynamic disabled=${settingState.disabled}`,
enumerable: true,
configurable: true,
});
await session.refreshMCPTools([dynamicTool]);
const baseline = rebuildCount;
expect(baseline).toBeGreaterThanOrEqual(1);
// Same underlying state, same tool object identity: skip.
await session.refreshMCPTools([dynamicTool]);
expect(rebuildCount).toBe(baseline);
// Mutate the settings-backed state. The tool object identity does not change,
// but its `description` getter now returns a new string. The signature must
// pick this up live (no per-tool caching) and force a rebuild.
settingState.disabled = "plan,explore";
await session.refreshMCPTools([dynamicTool]);
expect(rebuildCount).toBe(baseline + 1);
// Same state again: skip.
await session.refreshMCPTools([dynamicTool]);
expect(rebuildCount).toBe(baseline + 1);
});
});