oh-my-pi/bazel/toolchains/msvc/NOTES.md

# msvc cross toolchain — knobs & validation notes

Hermetic clang-cl + lld-link + xwin (MSVC CRT/SDK) cc toolchain for
`x86_64-pc-windows-msvc`, cross-linking from linux-x64 (CI) and darwin (dev)
exec hosts. Replaces cargo-xwin.

## Layout

| Piece | Where | Why separate |
| --- | --- | --- |
| `@llvm_msvc_tools` | `llvm.bzl` | LLVM 20.1.7 release archive for the fetching host, pruned to clang-cl/lld-link/llvm-lib/llvm-rc/llvm-mt + `lib/clang/*/include`. Downloads (~2 GiB) are sha256-pinned → Bazel repository cache. |
| `@xwin_sysroot` | `sysroot.bzl` | xwin 0.6.5 (pinned per-host sha256) runs `splat` in the repo rule. The ~1 GiB CRT/SDK payload comes from the Microsoft CDN via xwin itself and is **not** in Bazel's repo cache — a cold output base re-downloads it. Keep `sysroot.bzl` stable. |
| `@msvc_cc` | `cc.bzl` | Wrapper scripts + `cc_toolchain` + MSVC feature config (copied from the resolved rules_cc, like `@local_config_cc`). Cheap to regenerate — iterate flags here. |
| `toolchain()`s | `//bazel/toolchains` (`msvc-cc-from-*`) | One per exec host (linux-x64/arm64, darwin-arm64/x64), all pointing at `@msvc_cc//:cc_toolchain`; only the local host's variant can resolve. `target_compatible_with = [windows, x86_64]` ⇒ can never shadow zig on linux. |
| MODULE.bazel | `# --- msvc cross toolchain ---` section | `rules_cc` 0.2.17 (= Bazel 9.2's builtin pin) + the three `use_repo_rule` instantiations. Plus one target-suffixed env key inside the existing `audiopus_sys` annotation (see below). |

## Design decisions

- **No toolchains_llvm**: it wants to register full host cc toolchains, which
  risks shadowing the zig linux toolchains. Direct LLVM release fetch instead.
- **Wrappers self-locate from `$0`** (execroot-relative sibling repos), so they
  work from Bazel actions (cwd = execroot) *and* from build scripts, where
  rules_rust `${pwd}`-expands `CC`/`AR` to absolute paths and cc-rs/cmake spawn
  tools from other cwds.
- **CRT: dynamic `/MD`** (rules_cc's msvc branch default outside `dbg` without
  the `static_link_msvcrt` feature) — matches what napi/cc-rs produced under
  cargo-xwin (rust msvc targets default to dynamic CRT without `+crt-static`).
- **SSE floor in the wrapper, not annotations**: `-msse4.1 -msse4.2` live in the
  clang-cl wrapper, which only ever targets win32-x64 (baseline = x86-64-v2 ⊇
  SSE4.2). This is the old build-native.ts CFLAGS hack, windows-only by
  construction.
- **cmake generator via target-suffixed env**: `crate_universe` cannot select()
  annotations per platform, and a second `crate.annotation` for the same crate
  hard-fails the extension (`_insert_annotation` dupe check; `annotation_select`
  is unused/broken in rules_rust 0.71.3). Instead the existing `audiopus_sys`
  annotation carries `CMAKE_GENERATOR_x86_64_pc_windows_msvc=Ninja` — cmake-rs
  reads `VAR_<triple>` before `VAR`, so the key is inert for all other targets.
  cmake-rs sets `CMAKE_SYSTEM_NAME=Windows` itself when target ≠ host.
- **cmake tool discovery**: wrappers are named bare `clang-cl`, `lld-link`,
  `llvm-lib`, `llvm-rc`, `llvm-mt` (no `.sh`) because CMake's
  `CMakeFindBinUtils`/`find_program` probes for those names next to
  `CMAKE_C_COMPILER`, and cc-rs sniffs the MSVC/clang-cl tool family from the
  basename. The clang-cl wrapper also exports link.exe-style `LIB` and passes
  `-fuse-ld=lld-link` so compiler-driver links (cmake `try_compile` ABI checks)
  resolve CRT import libs without a toolchain file.
- **ring 0.17.14 needs no perl/nasm**: verified in its build.rs — crates.io
  tarballs use `pregenerated/*-nasm.o` objects directly for windows-msvc
  (`use_nasm()` path only shells out during the maintainer packaging step).
  The .o files flow through `cc::Build.object()` into llvm-lib.

## Reproducibility / cache implications

- First fetch per host: ~2 GiB LLVM (repo-cache backed) + ~15 MiB xwin +
  ~1 GiB MS CDN splat (not repo-cache backed). Splat repo ends up ~800 MiB.
- `--manifest-version 17` pins the VS2022 channel, but Microsoft advances the
  channel payload over time → splat is stable day-to-day, not bit-reproducible
  forever (same property cargo-xwin had). Action keys only depend on the files
  actually read, so remote-cache hit rates degrade gracefully after an MS bump.
- Toolchain binaries differ per exec host (linux vs mac clang) → win32 link
  actions do not share remote-cache entries across host OSes. CI is
  linux-x64-only for this target, so this only affects dev machines.

## Already verified (darwin-arm64 dev host, 2026-07-27)

- `bazel build --nobuild //:natives-win32-x64-baseline` analyzes clean;
  `cquery deps(...)` confirms `@msvc_cc//:cc_toolchain` (not the host Xcode
  toolchain) resolved for the windows target. Full fetch + splat took ~2.5 min
  on a fast link. Repin (`bun run bazel:repin`) already run for the
  `CMAKE_GENERATOR_x86_64_pc_windows_msvc` annotation key.
- Wrapper smoke test outside Bazel: `clang-cl /MD` compiled a windows.h +
  smmintrin.h SSE4.1 program and driver-linked it via `-fuse-ld=lld-link` +
  `LIB` into a valid PE32+ exe; the standalone `lld-link` wrapper (rustc's
  `-Clinker` path) and `llvm-lib` also produced a PE exe / ar archive.
- xwin's `10.0.26100 -> .` self-referential version symlinks broke Bazel's
  glob ("too many levels of symbolic links"); `sysroot.bzl` now prunes any
  readdir entry whose realpath equals its parent, post-splat.
- blake3 MASM (`ml64.exe` from cc-rs on non-windows hosts): `bin/ml64.exe` /
  `bin/ml64` shims exec `llvm-ml -m64`. All four blake3 1.8.5
  `blake3_*_x86-64_windows_msvc.asm` files assemble to valid amd64 COFF
  objects through the shim (invoked cc-rs-style via PATH with joined `/Fo`).
  cc-rs finds the shim through the blake3 crate.annotation, which prepends
  `$${pwd}/external/+msvc_cc_repository+msvc_cc/bin` to the build-script PATH
  (`$$` because annotation env runs through Bazel make-var expansion; the
  rules_rust runner then substitutes `${pwd}` → exec root). The wrapper dir
  reaches the sandbox via the cc toolchain's `all_files`. NOTE: the PATH entry
  hardcodes @msvc_cc's canonical repo name — keep in sync if the repo rule or
  repo name changes. Repin for this annotation already run.
- audiopus_sys/opus cmake exe links (can.internal finding #2): cmake's
  `vs_link_exe` demands rc/mt tools that `find_program` can't locate on a
  linux/mac PATH. @msvc_cc now generates `toolchain.cmake` (self-locating via
  `CMAKE_CURRENT_LIST_DIR`: compiler/linker/rc/mt = the wrappers) handed to
  cmake-rs through `CMAKE_TOOLCHAIN_FILE_x86_64_pc_windows_msvc` in the
  audiopus_sys annotation (same `$${pwd}` + canonical-repo-path mechanism as
  the blake3 shim). Second failure mode fixed in the same file: `try_compile`
  defaults to the Debug config → `/MDd` → `msvcrtd.lib`, which the lean splat
  (like cargo-xwin's) does not carry; toolchain.cmake pins
  `CMAKE_TRY_COMPILE_CONFIGURATION=Release`, `CMAKE_POLICY_DEFAULT_CMP0091=NEW`
  and `CMAKE_MSVC_RUNTIME_LIBRARY=MultiThreadedDLL` (/MD everywhere).
  Verified on darwin: scratch `project(C)` + `add_executable` configures with
  "Clang 20.1.7 with MSVC-like command-line" and links a valid PE32+ exe
  through vs_link_exe with the wrapper rc/mt/linker. Repin already run.

## What to verify on can.internal (linux-x64)

1. `bazel build //:natives-win32-x64-baseline` end-to-end link; check the
   produced `pi_natives.win32-x64-baseline.node` imports (dumpbin/llvm-readobj:
   expect VCRUNTIME140/api-ms-win-crt-* → `/MD`, no static CRT).
2. LLVM 20.1.7 Linux-X64 binaries are built on a newish Ubuntu: confirm the
   kata runner image's glibc is ≥ 2.35-ish and has `libtinfo6`/`libstdc++6`
   (usual LLVM release-binary runtime deps).
3. `ninja` + `cmake` must be on the audiopus_sys build-script PATH
   (`/usr/local/bin:/usr/bin:/bin`) on the kata image — same requirement the
   old ensure-cmake action satisfied for cargo-xwin.
4. audiopus_sys configure: cmake should report
   "Clang with MSVC-like command-line", take `toolchain.cmake` (log shows the
   vs_link_exe --rc/--mt pointing into the wrapper dir), and produce opus.lib
   with /MD objects. If mt is ever asked to actually merge manifests, note
   the official LLVM llvm-mt lacks libxml2 and would error — not hit today.
5. tree-sitter grammar compiles via cc-rs: wrapper is picked up as `CC`
   (family detection needs the basename to contain `clang-cl` — it does).
6. ring: no `nasm`/`perl` spawns in the build-script log; archive step uses
   the `llvm-lib` wrapper.
7. blake3: build-script log should show `ml64.exe` resolving to the shim (no
   "failed to find tool" error); spot-check the assembled objects land in the
   rlib.
8. Repo fetch time/disk on the pods (first fetch ~3 GiB, ~25 min worst case);
   consider pre-warming the bazel output base or a persistent
   `--repository_cache` volume if it hurts.

## Knobs

- LLVM version/sha256s: `llvm.bzl` (`_LLVM_VERSION`, `_LLVM_DISTS`). 20.1.7 is
  the newest release with archives for all four host tuples.
- xwin version + manifest channel: `sysroot.bzl` (`_XWIN_VERSION` 0.6.5 — last
  release with darwin binaries; `_XWIN_MANIFEST_VERSION` "17").
- Compile/link flags, include/libpath set, CRT choice: wrapper templates and
  `cc_toolchain_config` attrs in `cc.bzl`.
- Static CRT if ever needed: build with the standard `static_link_msvcrt`
  feature (`--features=static_link_msvcrt`) instead of editing flags.