docs(coding-agent): completed full omp docs sync

Added a contributor-facing native crate map (docs/native-crates.md) covering pi-natives, pi-shell, pi-ast, pi-iso, pi-walker, pi_uu_grep, pi-uutils-ctx, and vendored brush crates, and linked it from natives-architecture.md and user-facing-packages.md. Added a docs-index tool coverage test asserting every BUILTIN_TOOL_NAMES entry and injected custom tool (generate_image, tts) has a docs/tools/<name>.md page served by omp://. Inlined tiny fail/buildPayloadText/checkDocsIndexFreshness helpers in generate-docs-index.ts per the project rule against single-expression named functions. Fixes #3934
2026-07-01 00:00:52 +00:00
parent 3c3cb2a76b
commit 92740d3c4f
6 changed files with 119 additions and 32 deletions
@@ -0,0 +1,48 @@
+# Native Crates
+
+Contributor-facing map of the Rust crates under `crates/`. These crates back
+`@oh-my-pi/pi-natives` and the embedded shell/PTY runtime. They are intentionally
+internal: end users see `@oh-my-pi/pi-natives` exports, not these crate APIs.
+
+For the consumer-side runtime contract see
+[`natives-architecture.md`](./natives-architecture.md). For inclusion policy
+covering when a crate should be promoted to user-facing docs, see
+[`user-facing-packages.md`](./user-facing-packages.md).
+
+## Crate map
+
+| Crate | Path | Role |
+| --- | --- | --- |
+| `pi-natives` | [`crates/pi-natives`](../crates/pi-natives) | Top-level N-API `cdylib`; aggregates the other crates and exposes the JS-visible API. |
+| `pi-shell` | [`crates/pi-shell`](../crates/pi-shell) | Embedded shell / PTY / process management split out of `pi-natives` (wraps `brush-*`). |
+| `pi-ast` | [`crates/pi-ast`](../crates/pi-ast) | tree-sitter-based code summarizer and AST utilities; 50+ language grammars. |
+| `pi-iso` | [`crates/pi-iso`](../crates/pi-iso) | Task isolation backend resolver: APFS clones, btrfs/zfs reflinks, overlayfs, projfs, rcopy. |
+| `pi-walker` | [`crates/pi-walker`](../crates/pi-walker) | Parallel filesystem walker (ignore + globset) shared by grep, glob, and fs-scan cache. |
+| `pi_uu_grep` | [`crates/pi-uu-grep`](../crates/pi-uu-grep) | `grep` re-implemented on `grep-regex` / `grep-searcher`; runs in-process as a shell builtin. Entry: `pi_uu_grep::run`. |
+| `pi-uutils-ctx` | [`crates/pi-uutils-ctx`](../crates/pi-uutils-ctx) | Thread-local stdio + cwd context shim for embedding vendored uutils as in-process shell builtins. |
+| `brush-core` | [`crates/vendor/brush-core`](../crates/vendor/brush-core) | Vendored fork of [brush-shell](https://github.com/reubeno/brush) for embedded bash execution. |
+| `brush-builtins` | [`crates/vendor/brush-builtins`](../crates/vendor/brush-builtins) | Vendored bash builtins (`cd`, `echo`, `test`, `printf`, `read`, `export`, ...). |
+
+## What lives where
+
+- Native API surface and loader (`@oh-my-pi/pi-natives`):
+  [`natives-architecture.md`](./natives-architecture.md),
+  [`natives-addon-loader-runtime.md`](./natives-addon-loader-runtime.md),
+  [`natives-binding-contract.md`](./natives-binding-contract.md),
+  [`natives-build-release-debugging.md`](./natives-build-release-debugging.md),
+  [`natives-media-system-utils.md`](./natives-media-system-utils.md),
+  [`natives-rust-task-cancellation.md`](./natives-rust-task-cancellation.md),
+  [`natives-shell-pty-process.md`](./natives-shell-pty-process.md),
+  [`natives-text-search-pipeline.md`](./natives-text-search-pipeline.md).
+- Porting cross-references:
+  [`porting-from-pi-mono.md`](./porting-from-pi-mono.md),
+  [`porting-to-natives.md`](./porting-to-natives.md).
+- Filesystem scan cache contract that consumes `pi-walker`:
+  [`fs-scan-cache-architecture.md`](./fs-scan-cache-architecture.md).
+
+## Policy
+
+These crates are implementation details. End-user docs live with the consuming
+package (`@oh-my-pi/pi-natives`) and the architecture pages above. Promote a
+crate to a dedicated user-facing doc only when it grows a standalone CLI or
+public API consumed outside `packages/natives`.
@@ -150,7 +150,7 @@ N-API exports are generated from Rust `#[napi]` functions/classes/objects/enums.
  - user-facing policy and fallbacks that are not built into the native API
  - higher-level rendering, artifact, shell-session, and command behavior

-For the root-docs inclusion policy that keeps internal Rust crates under native architecture docs unless promoted as user-facing, see [`user-facing-packages.md`](./user-facing-packages.md).
+For the contributor-facing crate map covering `pi-natives`, `pi-shell`, `pi-ast`, `pi-iso`, `pi-walker`, `pi_uu_grep`, `pi-uutils-ctx`, and the vendored `brush-*` crates, see [`native-crates.md`](./native-crates.md). The root-docs inclusion policy that keeps internal Rust crates under native architecture docs unless promoted as user-facing also lives in [`user-facing-packages.md`](./user-facing-packages.md).

 ## Runtime flow (high level)

@@ -7,7 +7,7 @@ This page indexes README-only user-facing package CLIs and features that need ro
 - **Include** root docs coverage for package-local CLIs, extension features, dashboards, and benchmark runners that users can run directly or through `omp`.
 - **Exclude explicitly** when a package/crate is internal implementation only; point to the architecture doc that owns it.
 - Package READMEs and manifests remain the source of truth for package-local setup and flags; root docs make the feature discoverable and link to exact source paths.
- Internal Rust crates remain covered by native architecture docs unless promoted as standalone user-facing commands or APIs. Today, `crates/pi-natives` is documented through [`natives-architecture.md`](./natives-architecture.md) and related native docs because it backs `@oh-my-pi/pi-natives` rather than exposing its own user CLI.
+- Internal Rust crates remain covered by native architecture docs unless promoted as standalone user-facing commands or APIs. The contributor-facing map lives at [`native-crates.md`](./native-crates.md); today every `crates/*` entry is internal to `@oh-my-pi/pi-natives` and the embedded shell, so [`natives-architecture.md`](./natives-architecture.md) and the surrounding native docs own them.

 ## Package CLIs and features

@@ -4,7 +4,7 @@

 ### Fixed

- Fixed `omp://` documentation coverage for managed memory/skill tools, image generation, speech generation, README-only user-facing package CLIs, and docs-index freshness checks. ([#3934](https://github.com/can1357/oh-my-pi/issues/3934))
+- Fixed `omp://` documentation coverage for the managed memory/skill tools, image generation, speech generation, README-only user-facing package CLIs, and the internal Rust crate map, and added a docs-index freshness check and a built-in tool docs coverage test. ([#3934](https://github.com/can1357/oh-my-pi/issues/3934))

 ## [16.2.10] - 2026-06-30

@@ -40,10 +40,6 @@ function isStringArray(value: unknown): value is string[] {
 	return Array.isArray(value) && value.every(item => typeof item === "string");
 }

-function fail(message: string): never {
-	throw new Error(message);
-}
-
 /** Build the exact two-line `omp://` docs embed from the source `docs/**\/*.md` corpus. */
 export async function buildDocsIndexPayload(): Promise<DocsIndexPayload> {
 	const glob = new Glob("**/*.md");
@@ -68,52 +64,55 @@ export function decodeDocsIndexPayload(embed: string): DecodedDocsIndexPayload |
 	if (newline === -1) return null;

 	const filenames: unknown = JSON.parse(embed.slice(0, newline));
-	if (!isStringArray(filenames)) fail("Embedded docs index filename line is not a JSON string array.");
+	if (!isStringArray(filenames)) {
+		throw new Error("Embedded docs index filename line is not a JSON string array.");
+	}

 	const inflated = gunzipSync(Buffer.from(embed.slice(newline + 1), "base64"));
 	const bodies: unknown = JSON.parse(inflated.toString("utf8"));
-	if (!isStringArray(bodies)) fail("Embedded docs index body blob is not a JSON string array.");
+	if (!isStringArray(bodies)) {
+		throw new Error("Embedded docs index body blob is not a JSON string array.");
+	}

 	return { files: filenames, bodies };
 }

-/** Assert that an embed payload is fresh against the current source docs payload. */
+/**
+ * Assert that an embed payload is fresh against the current source docs payload.
+ * An empty placeholder is accepted by round-tripping the expected payload (the
+ * dev tree and post-build reset state both checked-in placeholders).
+ */
 export function assertDocsIndexFresh(embed: string, expected: DecodedDocsIndexPayload): void {
-	const decoded =
-		embed.length === 0 ? decodeDocsIndexPayload(buildPayloadText(expected)) : decodeDocsIndexPayload(embed);
-	if (decoded === null) fail("Embedded docs index is malformed: missing newline separator.");
+	const source =
+		embed.length > 0
+			? embed
+			: `${JSON.stringify(expected.files)}\n${Buffer.from(gzipSync(Buffer.from(JSON.stringify(expected.bodies)), { level: 9 })).toString("base64")}`;
+	const decoded = decodeDocsIndexPayload(source);
+	if (decoded === null) {
+		throw new Error("Embedded docs index is malformed: missing newline separator.");
+	}
 	if (decoded.files.length !== expected.files.length) {
-		fail(`Embedded docs index has ${decoded.files.length} docs; source corpus has ${expected.files.length}.`);
+		throw new Error(
+			`Embedded docs index has ${decoded.files.length} docs; source corpus has ${expected.files.length}.`,
+		);
 	}
 	if (decoded.bodies.length !== expected.bodies.length) {
-		fail(`Embedded docs index has ${decoded.bodies.length} bodies; source corpus has ${expected.bodies.length}.`);
+		throw new Error(
+			`Embedded docs index has ${decoded.bodies.length} bodies; source corpus has ${expected.bodies.length}.`,
+		);
 	}
 	for (let i = 0; i < expected.files.length; i++) {
 		if (decoded.files[i] !== expected.files[i]) {
-			fail(
+			throw new Error(
 				`Embedded docs index filename mismatch at ${i}: ${decoded.files[i] ?? "<missing>"} !== ${expected.files[i]}.`,
 			);
 		}
 		if (decoded.bodies[i] !== expected.bodies[i]) {
-			fail(`Embedded docs index body mismatch for ${expected.files[i]}. Run \`bun run gen:docs\`.`);
+			throw new Error(`Embedded docs index body mismatch for ${expected.files[i]}. Run \`bun run gen:docs\`.`);
 		}
 	}
 }

-function buildPayloadText(payload: DecodedDocsIndexPayload): string {
-	const bodiesB64 = Buffer.from(gzipSync(Buffer.from(JSON.stringify(payload.bodies)), { level: 9 })).toString(
-		"base64",
-	);
-	return `${JSON.stringify(payload.files)}\n${bodiesB64}`;
-}
-
-async function checkDocsIndexFreshness(rel: string): Promise<void> {
-	const current = await buildDocsIndexPayload();
-	const embed = await Bun.file(outputPath).text();
-	assertDocsIndexFresh(embed, current);
-	process.stdout.write(`Docs index fresh for ${current.files.length} docs (${rel})\n`);
-}
-
 async function main(): Promise<void> {
 	const rel = path.relative(process.cwd(), outputPath);

@@ -124,7 +123,10 @@ async function main(): Promise<void> {
 	}

 	if (process.argv.includes(CHECK_FLAG)) {
-		await checkDocsIndexFreshness(rel);
+		const current = await buildDocsIndexPayload();
+		const embed = await Bun.file(outputPath).text();
+		assertDocsIndexFresh(embed, current);
+		process.stdout.write(`Docs index fresh for ${current.files.length} docs (${rel})\n`);
 		return;
 	}

@@ -0,0 +1,37 @@
+import { describe, expect, it } from "bun:test";
+import * as fs from "node:fs";
+import * as path from "node:path";
+import { BUILTIN_TOOL_NAMES } from "@oh-my-pi/pi-coding-agent/tools/builtin-names";
+
+// Every shipped built-in tool that is exposed to the model in normal sessions
+// must have a docs/tools/<name>.md root doc served by `omp://`. File names use
+// underscores or hyphens; the test accepts either form so renaming the on-disk
+// page does not require coordinating with the wire name.
+const docsToolsDir = path.resolve(import.meta.dir, "../../../../docs/tools");
+
+const expectedDocPaths = (name: string): string[] => [
+	path.join(docsToolsDir, `${name}.md`),
+	path.join(docsToolsDir, `${name.replace(/_/g, "-")}.md`),
+];
+
+// Custom tools injected by the SDK (`packages/coding-agent/src/sdk.ts`) when
+// their settings are enabled. Built-in tool factories live in BUILTIN_TOOLS but
+// these custom tools are not present there, so the coverage list is explicit.
+const CUSTOM_TOOL_NAMES = ["generate_image", "tts"] as const;
+
+describe("omp:// root docs coverage", () => {
+	it.each([...BUILTIN_TOOL_NAMES])("documents builtin tool %s", name => {
+		const candidates = expectedDocPaths(name);
+		const present = candidates.find(candidate => fs.existsSync(candidate));
+		expect(
+			present,
+			`Missing docs/tools/<name>.md for built-in tool "${name}". Tried: ${candidates.join(", ")}.`,
+		).toBeDefined();
+	});
+
+	it.each([...CUSTOM_TOOL_NAMES])("documents injected custom tool %s", name => {
+		const candidates = expectedDocPaths(name);
+		const present = candidates.find(candidate => fs.existsSync(candidate));
+		expect(present, `Missing docs/tools/<name>.md for injected custom tool "${name}".`).toBeDefined();
+	});
+});