feat: added cross-compilation support via CARGO_BUILD_TARGET and environment variable overrides

- Added support for cross-compilation via CARGO_BUILD_TARGET environment variable, enabling builds for non-native targets like ARM64 Linux.
- Added support for overriding platform and architecture detection via TARGET_PLATFORM and TARGET_ARCH environment variables.
- Modified native build script to search for release artifacts in target-specific directories when cross-compiling.
- Updated CI workflow to use cross-compilation toolchain for ARM64 Linux builds instead of native ARM64 runners.
- Created comprehensive porting guide documenting how to port hot paths to pi-natives with anatomy, checklist, and troubleshooting sections.
This commit is contained in:
can1357
2026-02-01 07:44:34 +01:00
parent c803338101
commit 1e8e058ead
4 changed files with 158 additions and 6 deletions
+16 -1
View File
@@ -41,9 +41,10 @@ jobs:
- os: ubuntu-latest
platform: linux
arch: x64
- os: ubuntu-24.04-arm64
- os: ubuntu-latest
platform: linux
arch: arm64
target: aarch64-unknown-linux-gnu
- os: macos-15-intel
platform: darwin
arch: x64
@@ -59,11 +60,25 @@ jobs:
with:
lfs: true
- uses: dtolnay/rust-toolchain@nightly
with:
targets: ${{ matrix.target }}
- uses: Swatinem/rust-cache@v2
with:
key: ${{ matrix.target }}
- uses: oven-sh/setup-bun@v2
with:
bun-version: "1.3"
- name: Install cross-compilation toolchain
if: matrix.target == 'aarch64-unknown-linux-gnu'
run: |
sudo apt-get update
sudo apt-get install -y gcc-aarch64-linux-gnu
- name: Build native addon
env:
CARGO_BUILD_TARGET: ${{ matrix.target }}
TARGET_PLATFORM: ${{ matrix.platform }}
TARGET_ARCH: ${{ matrix.arch }}
CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER: aarch64-linux-gnu-gcc
run: bun --cwd=packages/natives run build:native
- name: Upload native addon
uses: actions/upload-artifact@v4
+123
View File
@@ -0,0 +1,123 @@
# Porting to pi-natives (N-API) — Field Notes
This is a practical guide for moving hot paths into `crates/pi-natives` and wiring them through the JS bindings. It exists to avoid the same failures happening twice.
## When to port
Port when any of these are true:
- The hot path is called in render loops or large batches.
- JS allocates too much (string churn, regex backtracking, large arrays).
- You already have a JS implementation and can write a benchmark for it.
Do not port code that depends on JS-only state, dynamic imports, or anything async-first. N-API favors CPU-bound, synchronous work.
## Anatomy of a native export
**Rust side:**
- Implementation lives in `crates/pi-natives/src/<module>.rs`.
- Export lives either in that module **with** `#[napi]` or in `lib.rs` calling into the module.
**JS side:**
- `packages/natives/src/native.ts` declares the binding and validates it.
- `packages/natives/src/<module>/index.ts` wraps the binding.
- `packages/natives/src/index.ts` re-exports it.
- Call sites (often in `packages/tui/src/utils.ts`) use the wrapper.
## Porting checklist
1) **Add the Rust implementation**
- Put the core logic in a plain Rust function.
- Expose it with `#[napi(js_name = "...")]`.
- Keep signatures simple: `String`, `Vec<String>`, `Uint8Array`, numbers, bools.
2) **Wire JS bindings**
- Add the method to `NativeBindings` in `packages/natives/src/native.ts`.
- Add `checkFn("newExport")` in `validateNative`.
- Add a wrapper in `packages/natives/src/<module>/index.ts`.
- Re-export from `packages/natives/src/index.ts`.
3) **Add benchmarks**
- Put benchmarks in `packages/tui/bench/*.ts` (see `text-layout.ts`).
- Include a JS baseline and native version in the same run.
- Use `performance.now()` and a fixed iteration count.
- Keep the benchmark inputs small and realistic (actual data seen in the hot path).
4) **Build the native binary**
- `bun --cwd=packages/natives run build:native`
5) **Run the benchmark**
- `bun run packages/tui/bench/<bench>.ts`
6) **Decide on usage**
- If native is slower, **keep JS** and leave the native export unused.
- If native is faster, switch call sites to the native wrapper.
## Pain points and how to avoid them
### 1) Stale `pi_natives.node` prevents new exports
The build script prefers `target/release/pi_natives.node` if it exists. If it’s stale, the exported symbols won’t update even after a rebuild.
**Fix:** remove the stale file before rebuilding.
```bash
rm /work/pi/target/release/pi_natives.node
bun --cwd=packages/natives run build:native
```
Then verify the export exists in the binary:
```bash
bun -e "const mod = require('./packages/natives/native/pi_natives.linux-x64.node'); console.log(Object.keys(mod).includes('newExport'));"
```
### 2) “Missing exports” errors from `validateNative`
This is **good** — it prevents silent mismatches. When you see this:
```
Native addon missing exports ... Missing: applyLineResets
```
it means your binary is stale or the `#[napi]` export didn’t compile in. Fix the build, don’t weaken validation.
### 3) Rust signature mismatch
Keep it simple. `Vec<String>` works. Avoid references like `&str` in public exports. If you need complex types, wrap them in `#[napi(object)]` structs.
### 4) Benchmarking mistakes
- Don’t compare different inputs or allocations.
- Keep JS and native using identical input arrays.
- Run both in the same benchmark file to avoid skew.
## Benchmark template
```ts
const ITERATIONS = 2000;
function bench(name: string, fn: () => void): number {
const start = performance.now();
for (let i = 0; i < ITERATIONS; i++) fn();
const elapsed = performance.now() - start;
console.log(`${name}: ${elapsed.toFixed(2)}ms total (${(elapsed / ITERATIONS).toFixed(4)}ms/op)`);
return elapsed;
}
bench("feature/js", () => {
jsImpl(sample);
});
bench("feature/native", () => {
nativeImpl(sample);
});
```
## Verification checklist
- `validateNative` passes (no missing exports).
- `Object.keys(require(...))` includes your new export.
- Bench numbers recorded in the PR/notes.
- Call site updated **only if** native is faster or equal.
## Rule of thumb
- If native is slower, **do not switch**. Keep the export for future work, but the TUI should stay on the faster path.
- If native is faster, switch the call site and keep the benchmark in place to catch regressions.
+9 -1
View File
@@ -1,6 +1,14 @@
# Changelog
## [Unreleased]
### Added
- Support for cross-compilation via `CARGO_BUILD_TARGET` environment variable
- Support for overriding platform and architecture detection via `TARGET_PLATFORM` and `TARGET_ARCH` environment variables
### Changed
- Native build script now searches for release artifacts in target-specific directories when cross-compiling
## [9.5.0] - 2026-02-01
@@ -28,4 +36,4 @@
### Fixed
- Fixed potential crashes when updating native binaries by using safe copy strategy that avoids overwriting in-memory binaries
- Fixed potential crashes when updating native binaries by using safe copy strategy that avoids overwriting in-memory binaries
+10 -4
View File
@@ -4,18 +4,24 @@ import * as path from "node:path";
const repoRoot = path.join(import.meta.dir, "../../..");
const rustDir = path.join(repoRoot, "crates/pi-natives");
const cargoTarget = process.env.CARGO_BUILD_TARGET;
const targetRoots = [
process.env.CARGO_TARGET_DIR ? path.resolve(process.env.CARGO_TARGET_DIR) : undefined,
path.join(repoRoot, "target"),
path.join(rustDir, "target"),
].filter((value): value is string => Boolean(value));
const releaseDirs = targetRoots.map(root => path.join(root, "release"));
const releaseDirs = targetRoots.flatMap(root => {
const base = cargoTarget ? path.join(root, cargoTarget, "release") : path.join(root, "release");
return cargoTarget ? [base, path.join(root, "release")] : [base];
});
const nativeDir = path.join(import.meta.dir, "../native");
const platform = process.platform;
const arch = process.arch;
const platform = process.env.TARGET_PLATFORM || process.platform;
const arch = process.env.TARGET_ARCH || process.arch;
const buildResult = await $`cargo build --release`.cwd(rustDir).nothrow();
const cargoArgs = ["build", "--release"];
if (cargoTarget) cargoArgs.push("--target", cargoTarget);
const buildResult = await $`cargo ${cargoArgs}`.cwd(rustDir).nothrow();
if (buildResult.exitCode !== 0) {
const stderrText =
typeof buildResult.stderr === "string"