perf(ci): cut warm CI time via download-skipping, splat reuse, PR gating
Four levers on top of the green pipeline: - kata jobs pass --remote_download_toplevel, so fully cache-hit builds stay metadata-only instead of pulling every intermediate artifact from bazel-remote (the bulk of the previous 6-minute TS-only main runs). - the xwin MSVC splat caches its ~1GiB CDN payload on the runner-cache PVC (OMP_XWIN_CACHE_DIR), instead of re-downloading per ephemeral pod. - main-push rust jobs export their bazel disk cache to the GitHub cache (once per lockfile change, shared linux scope). GitHub only shares default-branch caches across PRs, and main runs on kata where actions/cache never saved — so every fresh PR was building cold. - TS-only pull requests skip Rust validation entirely (gh pr diff path gate); their test jobs restore addons from the main-exported cache. Export runs disable top-level-only downloading: remote hits would otherwise export action entries whose blobs were never materialized.
This commit is contained in:
can1357
@@ -2,21 +2,35 @@ name: "Compose bazel cache config"
|
||||
description: >
|
||||
Single source of truth for how a CI job caches bazel work, emitted as a
|
||||
bazelrc fragment (rc output) consumers pass via `bazelisk --bazelrc=...`.
|
||||
|
||||
omp-kata pods (detected via BAZEL_REMOTE_USER/BAZEL_REMOTE_PASSWORD from the
|
||||
bazel-remote-ci secret) get read-write gRPC access to the in-cluster
|
||||
bazel-remote service — an address that only resolves inside the cluster, so
|
||||
nothing about the infrastructure leaks from this public repo. GitHub-hosted
|
||||
runners never talk to that infrastructure: they use a local bazel disk cache
|
||||
persisted with actions/cache, keyed on the crate lockfile and module
|
||||
definition.
|
||||
nothing about the infrastructure leaks from this public repo. With
|
||||
`export: true` (the main-push rust job), the kata job additionally writes a
|
||||
bazel disk cache and saves it to the GitHub Actions cache at job end under
|
||||
the main branch scope — that is what makes pull requests warm: PR-created
|
||||
caches are never shared across PRs, main-created ones are readable by every
|
||||
PR.
|
||||
|
||||
GitHub-hosted runners never talk to the cluster: they restore the
|
||||
main-exported disk cache (plus their own branch-scoped refresh saves).
|
||||
|
||||
inputs:
|
||||
scope:
|
||||
description: >
|
||||
Disk-cache key discriminator for GitHub-hosted runners; jobs building
|
||||
different target sets (linux pair, darwin-all, msvc, validation) use
|
||||
separate scopes so they don't evict each other's entries.
|
||||
Disk-cache key discriminator. Every linux-family consumer (validation,
|
||||
TS test jobs, linux/musl/win32 release jobs) shares the `linux` scope
|
||||
so PRs hit the cache exported from main's rust job; darwin release
|
||||
jobs keep per-target scopes and self-populate.
|
||||
required: true
|
||||
export:
|
||||
description: >
|
||||
Write a disk cache during the build and save it to the GitHub cache
|
||||
at job end (main-push rust job only). No-op when the key already
|
||||
exists for the current lockfiles.
|
||||
required: false
|
||||
default: "false"
|
||||
|
||||
outputs:
|
||||
rc:
|
||||
@@ -37,9 +51,23 @@ runs:
|
||||
restore-keys: |
|
||||
bazel-disk-${{ inputs.scope }}-${{ runner.os }}-${{ runner.arch }}-
|
||||
|
||||
# Combined restore+save: restore is a no-op on the first run for these
|
||||
# lockfiles (that's exactly when we want to build the export), and the
|
||||
# post-job save only fires on a primary-key miss — so the export is
|
||||
# written once per lockfile change, from a trusted main push.
|
||||
- name: Prepare disk cache export (omp-kata)
|
||||
if: env.BAZEL_REMOTE_USER != '' && inputs.export == 'true'
|
||||
id: export
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: ~/.cache/omp-bazel-disk
|
||||
key: bazel-disk-${{ inputs.scope }}-Linux-X64-${{ hashFiles('Cargo.Bazel.lock', 'MODULE.bazel', 'rust-toolchain.toml') }}
|
||||
|
||||
- name: Compose cache config
|
||||
id: compose
|
||||
shell: bash
|
||||
env:
|
||||
EXPORT_DISK: ${{ inputs.export == 'true' && steps.export.outputs.cache-hit != 'true' && 'true' || 'false' }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
rc="$RUNNER_TEMP/bazel-cache.rc"
|
||||
@@ -58,8 +86,22 @@ runs:
|
||||
echo "common --tls_certificate=infra/bazel-remote/ca.crt"
|
||||
echo "common --remote_header='authorization=Basic ${auth}'"
|
||||
# PVC-backed shared repository cache (pods are ephemeral; without
|
||||
# it every job re-downloads toolchains + crate archives).
|
||||
# it every job re-downloads toolchains + crate archives). The
|
||||
# xwin MSVC splat reuses the same PVC via OMP_XWIN_CACHE_DIR
|
||||
# (its ~1GiB CDN payload is not repository-cacheable).
|
||||
echo "common --repository_cache=/opt/bazel-repo-cache"
|
||||
echo "common --repo_env=OMP_XWIN_CACHE_DIR=/opt/bazel-repo-cache/xwin"
|
||||
if [ "$EXPORT_DISK" = "true" ]; then
|
||||
# Export runs (once per lockfile change) write the disk cache
|
||||
# PRs restore. They must download everything: with top-level-
|
||||
# only downloading, remote hits would export action entries
|
||||
# whose blobs were never materialized.
|
||||
echo "common --disk_cache=$HOME/.cache/omp-bazel-disk"
|
||||
else
|
||||
# Cache-hit work stays metadata-only; only requested top-level
|
||||
# outputs (the .node addons) are actually downloaded.
|
||||
echo "common --remote_download_toplevel"
|
||||
fi
|
||||
} > "$rc"
|
||||
else
|
||||
{
|
||||
|
||||
+40
-12
@@ -131,12 +131,37 @@ jobs:
|
||||
runs-on: ${{ github.event_name == 'pull_request' && 'ubuntu-22.04' || 'omp-kata' }}
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: ./.github/actions/bun-install
|
||||
# TS-only PRs skip Rust validation entirely; addons for the TS test
|
||||
# jobs come from the main-exported disk cache. Pushes always run.
|
||||
- name: Detect Rust-affecting changes
|
||||
id: changes
|
||||
shell: bash
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
run: |
|
||||
if [ "${{ github.event_name }}" != "pull_request" ]; then
|
||||
echo "rust=true" >> "$GITHUB_OUTPUT"
|
||||
exit 0
|
||||
fi
|
||||
if gh pr diff ${{ github.event.pull_request.number }} --name-only \
|
||||
| grep -qE '^(crates/|Cargo\.(toml|lock)|Cargo\.Bazel\.lock|MODULE\.bazel|BUILD\.bazel|\.bazelrc|\.bazelversion|bazel/|rust-toolchain\.toml|rustfmt\.toml|scripts/bazel-natives|\.github/actions/bazel-|\.github/workflows/ci\.yml)'; then
|
||||
echo "rust=true" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "No Rust-affecting changes; skipping validation."
|
||||
echo "rust=false" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
- if: steps.changes.outputs.rust == 'true'
|
||||
uses: ./.github/actions/bun-install
|
||||
- id: cache
|
||||
if: steps.changes.outputs.rust == 'true'
|
||||
uses: ./.github/actions/bazel-cache
|
||||
with:
|
||||
scope: validation
|
||||
scope: linux
|
||||
# Main pushes export the disk cache PRs restore (once per
|
||||
# lockfile change; no-op otherwise).
|
||||
export: ${{ github.event_name != 'pull_request' }}
|
||||
- name: Rust tests
|
||||
if: steps.changes.outputs.rust == 'true'
|
||||
# The ulimit guard runs in the step that launches the bazel server
|
||||
# (limits are per-process and the server persists across steps).
|
||||
run: |
|
||||
@@ -147,14 +172,17 @@ jobs:
|
||||
# `[lints] workspace = true` get the workspace policy, the vendored
|
||||
# brush fork is exempt (same as run-rs-task.ts's cargo excludes).
|
||||
- name: Clippy (workspace lint policy on opted-in crates)
|
||||
if: steps.changes.outputs.rust == 'true'
|
||||
run: |
|
||||
bazelisk query "kind('rust_library|rust_shared_library', //crates/pi-ast/... + //crates/pi-iso/... + //crates/pi-natives/... + //crates/pi-shell/... + //crates/pi-walker/...)" \
|
||||
| xargs bazelisk --bazelrc="${{ steps.cache.outputs.rc }}" build --config=clippy-strict --
|
||||
- name: Clippy (default lints elsewhere)
|
||||
if: steps.changes.outputs.rust == 'true'
|
||||
run: |
|
||||
bazelisk query "kind('rust_library|rust_shared_library', //crates/... - (//crates/pi-ast/... + //crates/pi-iso/... + //crates/pi-natives/... + //crates/pi-shell/... + //crates/pi-walker/...) - //crates/vendor/brush-core/... - //crates/vendor/brush-builtins/...)" \
|
||||
| xargs bazelisk --bazelrc="${{ steps.cache.outputs.rc }}" build --config=clippy --
|
||||
- name: Rustfmt
|
||||
if: steps.changes.outputs.rust == 'true'
|
||||
run: bazelisk --bazelrc="${{ steps.cache.outputs.rc }}" build --config=rustfmt //crates/...
|
||||
- name: Warm native addon cache (main push)
|
||||
if: github.event_name != 'pull_request'
|
||||
@@ -175,7 +203,7 @@ jobs:
|
||||
- uses: ./.github/actions/bazel-natives
|
||||
with:
|
||||
targets: linux-x64-baseline linux-x64-modern
|
||||
cache-scope: linux-x64-pair
|
||||
cache-scope: linux
|
||||
- name: Test workspace packages and repo scripts (TS)
|
||||
env:
|
||||
OMP_TEST_CONCURRENCY: "4"
|
||||
@@ -194,7 +222,7 @@ jobs:
|
||||
- uses: ./.github/actions/bazel-natives
|
||||
with:
|
||||
targets: linux-x64-baseline linux-x64-modern
|
||||
cache-scope: linux-x64-pair
|
||||
cache-scope: linux
|
||||
- name: Test coding-agent singleton/global-state bucket
|
||||
# Keep global Settings/env/fake-timer tests serial; native addon
|
||||
# artifacts are still available like every other coding-agent bucket.
|
||||
@@ -213,7 +241,7 @@ jobs:
|
||||
- uses: ./.github/actions/bazel-natives
|
||||
with:
|
||||
targets: linux-x64-baseline linux-x64-modern
|
||||
cache-scope: linux-x64-pair
|
||||
cache-scope: linux
|
||||
- name: Test native/TUI/browser-ish packages (TS)
|
||||
env:
|
||||
OMP_TEST_CONCURRENCY: "4"
|
||||
@@ -232,7 +260,7 @@ jobs:
|
||||
- uses: ./.github/actions/bazel-natives
|
||||
with:
|
||||
targets: linux-x64-baseline linux-x64-modern
|
||||
cache-scope: linux-x64-pair
|
||||
cache-scope: linux
|
||||
- name: Test coding-agent UI/TUI bucket
|
||||
env:
|
||||
OMP_TEST_CONCURRENCY: "2"
|
||||
@@ -251,7 +279,7 @@ jobs:
|
||||
- uses: ./.github/actions/bazel-natives
|
||||
with:
|
||||
targets: linux-x64-baseline linux-x64-modern
|
||||
cache-scope: linux-x64-pair
|
||||
cache-scope: linux
|
||||
- name: Test coding-agent runtime bucket
|
||||
# Runtime/session tests import native-backed barrels too; keep this
|
||||
# separate for concurrency, not as a native-free guardrail.
|
||||
@@ -272,7 +300,7 @@ jobs:
|
||||
- uses: ./.github/actions/bazel-natives
|
||||
with:
|
||||
targets: linux-x64-baseline linux-x64-modern
|
||||
cache-scope: linux-x64-pair
|
||||
cache-scope: linux
|
||||
- name: Test coding-agent native/unit bucket
|
||||
env:
|
||||
OMP_TEST_CONCURRENCY: "4"
|
||||
@@ -291,7 +319,7 @@ jobs:
|
||||
- uses: ./.github/actions/bazel-natives
|
||||
with:
|
||||
targets: linux-x64-baseline linux-x64-modern
|
||||
cache-scope: linux-x64-pair
|
||||
cache-scope: linux
|
||||
- name: CLI smoke test
|
||||
run: bun run ci:test:smoke
|
||||
|
||||
@@ -307,7 +335,7 @@ jobs:
|
||||
- uses: ./.github/actions/bazel-natives
|
||||
with:
|
||||
targets: linux-x64-baseline linux-x64-modern
|
||||
cache-scope: linux-x64-pair
|
||||
cache-scope: linux
|
||||
- name: Install method smoke tests
|
||||
env:
|
||||
OMP_INSTALL_TEST_SKIP_NATIVE_BUILD: "1"
|
||||
@@ -421,7 +449,7 @@ jobs:
|
||||
uses: ./.github/actions/bazel-natives
|
||||
with:
|
||||
targets: ${{ matrix.native_targets }}
|
||||
cache-scope: release-${{ matrix.target_id }}
|
||||
cache-scope: ${{ startsWith(matrix.target_id, 'darwin') && format('release-{0}', matrix.target_id) || 'linux' }}
|
||||
- name: Build release binary
|
||||
env:
|
||||
RELEASE_TARGETS: ${{ matrix.target_id }}
|
||||
@@ -588,7 +616,7 @@ jobs:
|
||||
uses: ./.github/actions/bazel-natives
|
||||
with:
|
||||
targets: linux-x64-baseline linux-x64-modern
|
||||
cache-scope: linux-x64-pair
|
||||
cache-scope: linux
|
||||
- name: Publish to npm
|
||||
env:
|
||||
# Fallback auth: setup-node wrote an .npmrc referencing
|
||||
|
||||
Reference in New Issue
Block a user