smashimo/oh-my-pi
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: streamlined native addon builds and caching in ci workflows

Browse Source 
- Enhance CI workflows to build and reuse native addon artifacts instead of recompiling.
- Update bazel cache actions with selective backend detection and separate remote and disk modes.
- Add `--source` CLI option to install prebuilt native targets without requiring Bazel.
- Increase Kata runner memory configuration from 12Gi to 24Gi.
This commit is contained in:
can1357
2026-07-28 02:06:13 +02:00
parent bbe0236a0f
commit ed4c78bc0f
9 changed files with 306 additions and 156 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/actions/bazel-cache/action.yml
+95 -65
View File
@@ -1,34 +1,22 @@
name: "Compose bazel cache config"
description: >
Single source of truth for how a CI job caches bazel work, emitted as a
bazelrc fragment (rc output) consumers pass via `bazelisk --bazelrc=...`.
Selects the CI cache backend and emits a bazelrc fragment. A shell probe
exposes the backend through step outputs before any action condition uses it.
omp-kata pods (detected via BAZEL_REMOTE_USER/BAZEL_REMOTE_PASSWORD from the
bazel-remote-ci secret) get read-write gRPC access to the in-cluster
bazel-remote service — an address that only resolves inside the cluster, so
nothing about the infrastructure leaks from this public repo. With
`export: true` (the main-push rust job), the kata job additionally writes a
bazel disk cache and saves it to the GitHub Actions cache at job end under
the main branch scope — that is what makes pull requests warm: PR-created
caches are never shared across PRs, main-created ones are readable by every
PR.
omp-kata jobs use the cluster remote cache. An exporting main job first
performs an exact GitHub cache lookup without downloading the archive. A
genuine miss switches that job to a local disk cache for one build. The
workflow saves that populated cache explicitly after the build.
GitHub-hosted runners never talk to the cluster: they restore the
main-exported disk cache (plus their own branch-scoped refresh saves).
GitHub-hosted jobs restore the exported disk cache and never contact the
cluster. Build callers can save a new exact-key archive after a miss.
inputs:
scope:
description: >
Disk-cache key discriminator. Every linux-family consumer (validation,
TS test jobs, linux/musl/win32 release jobs) shares the `linux` scope
so PRs hit the cache exported from main's rust job; darwin release
jobs keep per-target scopes and self-populate.
description: Disk-cache key discriminator shared by compatible consumers
required: true
export:
description: >
Write a disk cache during the build and save it to the GitHub cache
at job end (main-push rust job only). No-op when the key already
exists for the current lockfiles.
description: Prepare a portable disk cache after an exact lookup miss
required: false
default: "false"
@@ -36,78 +24,120 @@ outputs:
rc:
description: Path to the generated bazelrc fragment
value: ${{ steps.compose.outputs.rc }}
cache-key:
description: Exact GitHub cache key for a later explicit save
value: ${{ steps.backend.outputs.cache-key }}
export-needed:
description: Whether the remote exporter switched to a portable disk cache
value: ${{ steps.compose.outputs.export-needed }}
save-needed:
description: Whether a disk-cache build can save a new exact-key archive
value: ${{ steps.compose.outputs.save-needed }}
remote:
description: Whether the shell probe selected the cluster remote cache
value: ${{ steps.backend.outputs.remote }}
runs:
using: composite
steps:
- name: Restore bazel disk cache (GitHub-hosted)
if: env.BAZEL_REMOTE_USER == ''
uses: actions/cache@v4
- name: Detect bazel cache backend
id: backend
shell: bash
env:
CACHE_KEY: bazel-disk-${{ inputs.scope }}-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('Cargo.Bazel.lock', 'MODULE.bazel', 'rust-toolchain.toml') }}
run: |
set -euo pipefail
remote=false
if [ -n "${BAZEL_REMOTE_USER:-}" ] || [ -n "${BAZEL_REMOTE_PASSWORD:-}" ]; then
if [ -z "${BAZEL_REMOTE_USER:-}" ] || [ -z "${BAZEL_REMOTE_PASSWORD:-}" ]; then
echo "::error::BAZEL_REMOTE_USER and BAZEL_REMOTE_PASSWORD must both be set"
exit 1
fi
remote=true
fi
{
echo "remote=$remote"
echo "cache-key=$CACHE_KEY"
} >> "$GITHUB_OUTPUT"
- name: Restore bazel disk cache
id: restore
if: steps.backend.outputs.remote != 'true'
uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: |
~/.cache/omp-bazel-disk
~/.cache/omp-bazel-repo
key: bazel-disk-${{ inputs.scope }}-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('Cargo.Bazel.lock', 'MODULE.bazel', 'rust-toolchain.toml') }}
path: ~/.cache/omp-bazel-disk
key: ${{ steps.backend.outputs.cache-key }}
restore-keys: |
bazel-disk-${{ inputs.scope }}-${{ runner.os }}-${{ runner.arch }}-
# Combined restore+save: restore is a no-op on the first run for these
# lockfiles (that's exactly when we want to build the export), and the
# post-job save only fires on a primary-key miss — so the export is
# written once per lockfile change, from a trusted main push.
- name: Prepare disk cache export (omp-kata)
if: env.BAZEL_REMOTE_USER != '' && inputs.export == 'true'
- name: Look up bazel disk cache export
if: steps.backend.outputs.remote == 'true' && inputs.export == 'true'
id: export
uses: actions/cache@v4
uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: ~/.cache/omp-bazel-disk
key: bazel-disk-${{ inputs.scope }}-Linux-X64-${{ hashFiles('Cargo.Bazel.lock', 'MODULE.bazel', 'rust-toolchain.toml') }}
key: ${{ steps.backend.outputs.cache-key }}
lookup-only: true
- name: Compose cache config
id: compose
shell: bash
env:
EXPORT_DISK: ${{ inputs.export == 'true' && steps.export.outputs.cache-hit != 'true' && 'true' || 'false' }}
REMOTE: ${{ steps.backend.outputs.remote }}
EXPORT_REQUESTED: ${{ inputs.export }}
EXPORT_HIT: ${{ steps.export.outputs.cache-hit }}
RESTORE_HIT: ${{ steps.restore.outputs.cache-hit }}
run: |
set -euo pipefail
export_needed=false
if [ "$REMOTE" = "true" ] && [ "$EXPORT_REQUESTED" = "true" ] && [ "$EXPORT_HIT" != "true" ]; then
export_needed=true
fi
save_needed=$export_needed
if [ "$REMOTE" != "true" ] && [ "$RESTORE_HIT" != "true" ]; then
save_needed=true
fi
rc="$RUNNER_TEMP/bazel-cache.rc"
if [ -n "${BAZEL_REMOTE_USER:-}" ]; then
auth="$(printf %s "${BAZEL_REMOTE_USER}:${BAZEL_REMOTE_PASSWORD}" | base64 | tr -d '\n')"
if [ "$REMOTE" = "true" ]; then
{
# The PVC repo-cache mount lives OUTSIDE $HOME (/opt): kubelet
# creates missing mountpoint parents root-owned, and a
# root-owned $HOME/.cache breaks bazel's default
# output_user_root and zig's wrapper cache. Pods are
# single-job ephemeral, so RUNNER_TEMP hosts the output root.
# The PVC mount lives outside $HOME. Pods are single-job and
# use RUNNER_TEMP for Bazel's output root.
echo "startup --output_user_root=$RUNNER_TEMP/bazel-root"
echo "common --config=ci"
echo "common --config=cache-rw"
echo "common --remote_cache=grpcs://bazel-remote.bazel-cache.svc.cluster.local:9092"
echo "common --tls_certificate=infra/bazel-remote/ca.crt"
echo "common --remote_header='authorization=Basic ${auth}'"
# PVC-backed shared repository cache (pods are ephemeral; without
# it every job re-downloads toolchains + crate archives). The
# xwin MSVC splat reuses the same PVC via OMP_XWIN_CACHE_DIR
# (its ~1GiB CDN payload is not repository-cacheable).
echo "common --repository_cache=/opt/bazel-repo-cache"
echo "common --repo_env=OMP_XWIN_CACHE_DIR=/opt/bazel-repo-cache/xwin"
if [ "$EXPORT_DISK" = "true" ]; then
# Export runs (once per lockfile change) write the disk cache
# PRs restore. They must download everything: with top-level-
# only downloading, remote hits would export action entries
# whose blobs were never materialized.
echo "common --disk_cache=$HOME/.cache/omp-bazel-disk"
else
# Cache-hit work stays metadata-only; only requested top-level
# outputs (the .node addons) are actually downloaded.
echo "common --remote_download_toplevel"
fi
} > "$rc"
if [ "$export_needed" = "true" ]; then
# Do not combine remote and disk caches. Remote hits do not
# materialize a portable disk cache for hosted runners.
mkdir -p "$HOME/.cache/omp-bazel-disk"
echo "common --disk_cache=$HOME/.cache/omp-bazel-disk" >> "$rc"
else
raw_auth="${BAZEL_REMOTE_USER}:${BAZEL_REMOTE_PASSWORD}"
auth="$(printf %s "$raw_auth" | base64 | tr -d '\n')"
echo "::add-mask::$raw_auth"
echo "::add-mask::$auth"
{
echo "common --config=cache-rw"
echo "common --remote_cache=grpcs://bazel-remote.bazel-cache.svc.cluster.local:9092"
echo "common --tls_certificate=infra/bazel-remote/ca.crt"
echo "common --remote_header='authorization=Basic ${auth}'"
echo "common --remote_download_toplevel"
} >> "$rc"
fi
else
mkdir -p "$HOME/.cache/omp-bazel-disk" "$HOME/.cache/omp-bazel-repo"
{
echo "common --config=ci"
echo "common --disk_cache=$HOME/.cache/omp-bazel-disk"
echo "common --repository_cache=$HOME/.cache/omp-bazel-repo"
} > "$rc"
fi
echo "rc=$rc" >> "$GITHUB_OUTPUT"
{
echo "rc=$rc"
echo "export-needed=$export_needed"
echo "save-needed=$save_needed"
} >> "$GITHUB_OUTPUT"
.github/actions/bazel-natives/action.yml
+12 -2
View File
@@ -28,10 +28,20 @@ runs:
- name: Build native addons
shell: bash
env:
OMP_BAZEL_RC: ${{ steps.cache.outputs.rc }}
NATIVE_TARGETS: ${{ inputs.targets }}
NATIVE_DEST: ${{ inputs.dest }}
run: |
set -euo pipefail
# Raise-only fd guard: huge toolchain input trees exhaust low soft
# limits during sandbox setup.
if [ "$(ulimit -Sn)" != unlimited ] && [ "$(ulimit -Sn)" -lt 65536 ]; then ulimit -Sn 65536 || true; fi
export OMP_BAZEL_RC="${{ steps.cache.outputs.rc }}"
bun scripts/bazel-natives.ts ${{ inputs.targets }} --dest "${{ inputs.dest }}"
read -r -a targets <<< "$NATIVE_TARGETS"
bun scripts/bazel-natives.ts "${targets[@]}" --dest "$NATIVE_DEST"
- name: Save bazel disk cache
if: steps.cache.outputs.save-needed == 'true'
uses: actions/cache/save@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: ~/.cache/omp-bazel-disk
key: ${{ steps.cache.outputs.cache-key }}
.github/actions/native-artifacts/action.yml
+38
View File
@@ -0,0 +1,38 @@
name: "Install native addon artifacts"
description: >
Downloads the native addons built once by the Rust job and installs exact
targets without invoking Bazel.
inputs:
targets:
description: Space-separated scripts/bazel-natives.ts target names
required: true
dest:
description: Destination directory for the .node files
required: false
default: packages/natives/native
runs:
using: composite
steps:
- name: Download native addon artifacts
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: native-addons
path: ${{ runner.temp }}/omp-native-artifacts
- name: Install native addon targets
shell: bash
env:
NATIVE_TARGETS: ${{ inputs.targets }}
NATIVE_DEST: ${{ inputs.dest }}
run: |
set -euo pipefail
read -r -a targets <<< "$NATIVE_TARGETS"
if [ "${#targets[@]}" -eq 0 ]; then
echo "::error::At least one native addon target is required"
exit 1
fi
bun scripts/bazel-natives.ts "${targets[@]}" \
--source "$RUNNER_TEMP/omp-native-artifacts" \
--dest "$NATIVE_DEST"
.github/workflows/ci.yml
+52 -43
View File
@@ -118,21 +118,16 @@ jobs:
- name: Build collab web
run: bun run collab:web:build
# Bazel validation and cache warm — replaces the old cargo pipeline
# (rust_validation + native build matrices + hand-rolled artifact caching).
# `bazel test` covers the Rust suite, the clippy/rustfmt aspect configs cover
# linting, and on main pushes (omp-kata, read-write cache) an additional
# //:natives-linux-all build populates the shared bazel-remote cache so every
# downstream job — TS tests, releases, PR runners — gets cache hits instead
# of rebuilding. No toolchain setup: bazelisk is on the GitHub images and
# baked into the kata runner image; bazel fetches the rest hermetically.
# One Bazel job validates Rust changes and builds native addons for every
# downstream job. Main builds all Linux-hosted targets. Pull requests build
# only the Linux x64 pair required by tests.
rust:
name: Validate Rust workspace (bazel)
runs-on: ${{ github.event_name == 'pull_request' && 'ubuntu-22.04' || 'omp-kata' }}
steps:
- uses: actions/checkout@v4
# TS-only PRs skip Rust validation entirely; addons for the TS test
# jobs come from the main-exported disk cache. Pushes always run.
# TS-only PRs skip Rust validation. They still materialize the Linux
# x64 addons once from the main-exported disk cache.
- name: Detect Rust-affecting changes
id: changes
shell: bash
@@ -153,7 +148,6 @@ jobs:
- if: steps.changes.outputs.rust == 'true'
uses: ./.github/actions/bun-install
- id: cache
if: steps.changes.outputs.rust == 'true'
uses: ./.github/actions/bazel-cache
with:
scope: linux
@@ -184,11 +178,30 @@ jobs:
- name: Rustfmt
if: steps.changes.outputs.rust == 'true'
run: bazelisk --bazelrc="${{ steps.cache.outputs.rc }}" build --config=rustfmt //crates/...
- name: Warm native addon cache (main push)
if: github.event_name != 'pull_request'
- name: Build native addons once
env:
EVENT_NAME: ${{ github.event_name }}
run: |
set -euo pipefail
if [ "$(ulimit -Sn)" != unlimited ] && [ "$(ulimit -Sn)" -lt 65536 ]; then ulimit -Sn 65536 || true; fi
bazelisk --bazelrc="${{ steps.cache.outputs.rc }}" build //:natives-linux-all
targets=(//:natives-linux-x64-baseline //:natives-linux-x64-modern)
if [ "$EVENT_NAME" != "pull_request" ]; then
targets=(//:natives-linux-all)
fi
bazelisk --bazelrc="${{ steps.cache.outputs.rc }}" build "${targets[@]}"
- name: Save Bazel disk cache
if: steps.cache.outputs.save-needed == 'true'
uses: actions/cache/save@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: ~/.cache/omp-bazel-disk
key: ${{ steps.cache.outputs.cache-key }}
- name: Upload native addon artifacts
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: native-addons
path: bazel-bin/natives-*/*.node
if-no-files-found: error
retention-days: 1
test_workspace:
name: Test TS workspace fast
@@ -200,10 +213,9 @@ jobs:
- uses: actions/checkout@v4
- uses: ./.github/actions/setup-system-deps
- uses: ./.github/actions/bun-install
- uses: ./.github/actions/bazel-natives
- uses: ./.github/actions/native-artifacts
with:
targets: linux-x64-baseline linux-x64-modern
cache-scope: linux
- name: Test workspace packages and repo scripts (TS)
env:
OMP_TEST_CONCURRENCY: "4"
@@ -219,10 +231,9 @@ jobs:
- uses: actions/checkout@v4
- uses: ./.github/actions/setup-system-deps
- uses: ./.github/actions/bun-install
- uses: ./.github/actions/bazel-natives
- uses: ./.github/actions/native-artifacts
with:
targets: linux-x64-baseline linux-x64-modern
cache-scope: linux
- name: Test coding-agent singleton/global-state bucket
# Keep global Settings/env/fake-timer tests serial; native addon
# artifacts are still available like every other coding-agent bucket.
@@ -238,10 +249,9 @@ jobs:
- uses: actions/checkout@v4
- uses: ./.github/actions/setup-system-deps
- uses: ./.github/actions/bun-install
- uses: ./.github/actions/bazel-natives
- uses: ./.github/actions/native-artifacts
with:
targets: linux-x64-baseline linux-x64-modern
cache-scope: linux
- name: Test native/TUI/browser-ish packages (TS)
env:
OMP_TEST_CONCURRENCY: "4"
@@ -257,10 +267,9 @@ jobs:
- uses: actions/checkout@v4
- uses: ./.github/actions/setup-system-deps
- uses: ./.github/actions/bun-install
- uses: ./.github/actions/bazel-natives
- uses: ./.github/actions/native-artifacts
with:
targets: linux-x64-baseline linux-x64-modern
cache-scope: linux
- name: Test coding-agent UI/TUI bucket
env:
OMP_TEST_CONCURRENCY: "2"
@@ -276,10 +285,9 @@ jobs:
- uses: actions/checkout@v4
- uses: ./.github/actions/setup-system-deps
- uses: ./.github/actions/bun-install
- uses: ./.github/actions/bazel-natives
- uses: ./.github/actions/native-artifacts
with:
targets: linux-x64-baseline linux-x64-modern
cache-scope: linux
- name: Test coding-agent runtime bucket
# Runtime/session tests import native-backed barrels too; keep this
# separate for concurrency, not as a native-free guardrail.
@@ -297,10 +305,9 @@ jobs:
- uses: actions/checkout@v4
- uses: ./.github/actions/setup-system-deps
- uses: ./.github/actions/bun-install
- uses: ./.github/actions/bazel-natives
- uses: ./.github/actions/native-artifacts
with:
targets: linux-x64-baseline linux-x64-modern
cache-scope: linux
- name: Test coding-agent native/unit bucket
env:
OMP_TEST_CONCURRENCY: "4"
@@ -316,10 +323,9 @@ jobs:
- uses: actions/checkout@v4
- uses: ./.github/actions/setup-system-deps
- uses: ./.github/actions/bun-install
- uses: ./.github/actions/bazel-natives
- uses: ./.github/actions/native-artifacts
with:
targets: linux-x64-baseline linux-x64-modern
cache-scope: linux
- name: CLI smoke test
run: bun run ci:test:smoke
@@ -332,10 +338,9 @@ jobs:
- uses: actions/checkout@v4
- uses: ./.github/actions/setup-system-deps
- uses: ./.github/actions/bun-install
- uses: ./.github/actions/bazel-natives
- uses: ./.github/actions/native-artifacts
with:
targets: linux-x64-baseline linux-x64-modern
cache-scope: linux
- name: Install method smoke tests
env:
OMP_INSTALL_TEST_SKIP_NATIVE_BUILD: "1"
@@ -398,7 +403,7 @@ jobs:
arch: x64,
target_id: darwin-x64,
binary_path: packages/coding-agent/binaries/omp-darwin-x64,
native_targets: darwin-all,
native_targets: darwin-x64-baseline,
}
- {
os: macos-14,
@@ -406,7 +411,7 @@ jobs:
arch: arm64,
target_id: darwin-arm64,
binary_path: packages/coding-agent/binaries/omp-darwin-arm64,
native_targets: darwin-all,
native_targets: darwin-arm64,
}
- {
os: ubuntu-22.04,
@@ -442,14 +447,20 @@ jobs:
path: ~/.bun/install/cache
key: bun-${{ runner.os }}-${{ hashFiles('**/bun.lock') }}
- run: bun install --frozen-lockfile
# Release runners are GitHub-hosted and never touch the private
# cluster cache: they build with the actions/cache-backed disk cache,
# so repeat releases with unchanged Rust are mostly local cache hits.
- name: Build native addon(s) (bazel)
# Linux and Windows addons come from the Rust job. Darwin runners
# build only their own architecture because cross-hosted artifacts do
# not exist for macOS.
- name: Install prebuilt native addon(s)
if: matrix.platform != 'darwin'
uses: ./.github/actions/native-artifacts
with:
targets: ${{ matrix.native_targets }}
- name: Build native addon (bazel)
if: matrix.platform == 'darwin'
uses: ./.github/actions/bazel-natives
with:
targets: ${{ matrix.native_targets }}
cache-scope: ${{ startsWith(matrix.target_id, 'darwin') && format('release-{0}', matrix.target_id) || 'linux' }}
cache-scope: release-${{ matrix.target_id }}
- name: Build release binary
env:
RELEASE_TARGETS: ${{ matrix.target_id }}
@@ -609,14 +620,12 @@ jobs:
path: ~/.bun/install/cache
key: bun-${{ runner.os }}-${{ hashFiles('**/bun.lock') }}
- run: bun install --frozen-lockfile
# The pi-coding-agent prepack executes workspace code (bundle-dist
# imports the pi-utils barrel, which loads the pi-natives addon), so
# this job needs the Linux x64 native addons just like TS tests do.
- name: Build native addons (bazel)
uses: ./.github/actions/bazel-natives
# The prepack executes workspace code which loads the Linux x64
# addon, so install the Rust job's artifact before publishing.
- name: Install prebuilt native addons
uses: ./.github/actions/native-artifacts
with:
targets: linux-x64-baseline linux-x64-modern
cache-scope: linux
- name: Publish to npm
env:
# Fallback auth: setup-node wrote an .npmrc referencing