fix(ci): serialized native addon builds to avoid kata pod OOM
- The aggregate //:natives-linux-all build links all six addon cdylibs concurrently; rustc RSS peaks OOMed the pod and the kernel killed the bazel server (exit 37, runs 30556752623 / 30557524371, twice at the same spot). - Build one addon target per invocation so the persistent server shares analysis and cached actions while the heavy links run one at a time; a final aggregate build stays as a completeness no-op.
This commit is contained in:
@@ -250,7 +250,26 @@ jobs:
|
||||
run: |
|
||||
set -eo pipefail
|
||||
if [ "$(ulimit -Sn)" != unlimited ] && [ "$(ulimit -Sn)" -lt 65536 ]; then ulimit -Sn 65536 || true; fi
|
||||
bazelisk --bazelrc="${{ steps.cache.outputs.rc }}" build //:natives-linux-all 2>&1 | tee "$RUNNER_TEMP/bazel-build.log"
|
||||
# The addon cdylib links each peak at several GiB of rustc RSS;
|
||||
# the aggregate //:natives-linux-all build runs all six
|
||||
# concurrently, which OOMs the kata pod and takes the bazel
|
||||
# server with it (exit 37 "Server terminated abruptly", runs
|
||||
# 30556752623 / 30557524371). Build one addon per invocation —
|
||||
# analysis and cached actions are shared through the persistent
|
||||
# server, so only the heavy links serialize — then assemble the
|
||||
# aggregate as a no-op (also catches targets added to the
|
||||
# filegroup but missing from this list).
|
||||
: > "$RUNNER_TEMP/bazel-build.log"
|
||||
for target in \
|
||||
natives-linux-arm64 \
|
||||
natives-linux-musl-arm64 \
|
||||
natives-linux-musl-x64-baseline \
|
||||
natives-linux-x64-baseline \
|
||||
natives-linux-x64-modern \
|
||||
natives-win32-x64-baseline; do
|
||||
bazelisk --bazelrc="${{ steps.cache.outputs.rc }}" build "//:$target" 2>&1 | tee -a "$RUNNER_TEMP/bazel-build.log"
|
||||
done
|
||||
bazelisk --bazelrc="${{ steps.cache.outputs.rc }}" build //:natives-linux-all 2>&1 | tee -a "$RUNNER_TEMP/bazel-build.log"
|
||||
# Cache-hit visibility: a supposedly warm build that executes
|
||||
# thousands of actions is the failure mode that made CI slow — make
|
||||
# it visible in the run summary instead of discovering it weeks in.
|
||||
|
||||
Reference in New Issue
Block a user