Commit Graph

18 Commits

Author SHA1 Message Date
can1357 17ea2d65fa docs(infra): reconciled runner sizing docs with burstable scale set
- Updated the arc-omp-values example block and resources bullet to the live
  3cpu/10Gi request + 8cpu/14Gi limit shape instead of the old guaranteed
  8cpu/24Gi sizing.
- Documented the Kata boot floor vs request vs hotplug-limit relationship in
  tune-kata-runtime.sh so boot defaults are kept at or below pod requests.
2026-07-30 08:16:19 +02:00
can1357 fcddf46e12 infra(scope): configured burstable runner resources and scale-up
- Increased maximum concurrent runner pods from 4 to 8.
- Configured burstable runner resource requests and limits to improve bin-packing.
- Updated documentation to reflect the new runner sizing and scale limits.
2026-07-30 07:42:55 +02:00
can1357 e74a70cf96 chore(infra): updated runner reload script to use heredoc for remote directory
- Updated ssh execution in infra/reload-runner.sh to use a heredoc for creating the remote context directory.
2026-07-28 10:22:57 +02:00
can1357 83f3105e6e ci(infra): raised kata guest file descriptor limit
- Raised fs.nr_open from 1,048,576 to 8,388,608 through the Kata guest kernel command line.
- Added a smoke check for guest, soft, and hard open-file limits.
- Documented the cold Bazel sandbox failure mode and lazy descriptor allocation.
2026-07-28 03:19:42 +02:00
can1357 ed4c78bc0f feat: streamlined native addon builds and caching in ci workflows
- Enhance CI workflows to build and reuse native addon artifacts instead of recompiling.
- Update bazel cache actions with selective backend detection and separate remote and disk modes.
- Add `--source` CLI option to install prebuilt native targets without requiring Bazel.
- Increase Kata runner memory configuration from 12Gi to 24Gi.
2026-07-28 02:06:13 +02:00
can1357 a7abeff1b7 perf(ci): cut warm CI time via download-skipping, splat reuse, PR gating
Four levers on top of the green pipeline:
- kata jobs pass --remote_download_toplevel, so fully cache-hit builds
  stay metadata-only instead of pulling every intermediate artifact from
  bazel-remote (the bulk of the previous 6-minute TS-only main runs).
- the xwin MSVC splat caches its ~1GiB CDN payload on the runner-cache
  PVC (OMP_XWIN_CACHE_DIR), instead of re-downloading per ephemeral pod.
- main-push rust jobs export their bazel disk cache to the GitHub cache
  (once per lockfile change, shared linux scope). GitHub only shares
  default-branch caches across PRs, and main runs on kata where
  actions/cache never saved — so every fresh PR was building cold.
- TS-only pull requests skip Rust validation entirely (gh pr diff path
  gate); their test jobs restore addons from the main-exported cache.

Export runs disable top-level-only downloading: remote hits would
otherwise export action entries whose blobs were never materialized.
2026-07-27 14:31:24 +02:00
can1357 02c50eb6f3 fix(ci): moved bazel repo cache mount outside the runner home
kubelet creates missing subPath mountpoint parents as root, so mounting
the PVC repository cache under ~/.cache left the directory root-owned
and broke both bazel's default output root and zig's wrapper cache
compile (AccessDenied). The mount now lives at /opt/bazel-repo-cache.
2026-07-27 12:27:31 +02:00
can1357 2092f9330c fix(ci): moved bazel output root off the root-owned kata cache dir
kubelet materializes /home/runner/.cache as root when creating the
omp-bazel-repo subPath mountpoint, so bazel's default output_user_root
under it fails with EACCES. Kata jobs now point output_user_root at
RUNNER_TEMP via the bazel-cache rc fragment (pods are single-job
ephemeral; toolchain/crate downloads stay on the PVC repository cache),
and the runner image pre-owns ~/.cache for the next rebake.
2026-07-27 12:24:35 +02:00
can1357 8facd237d5 feat(build): migrated native pipeline to bazel with remote caching
- Replaced the napi-cli/cargo-zigbuild/cargo-xwin/sccache build path with
  Bazel: rules_rust + crate_universe over Cargo.lock, hermetic zig cc
  toolchains (linux-gnu pinned to glibc 2.17, linux-musl), host Xcode for
  darwin, and a repo-local hermetic clang-cl + llvm-ml + xwin toolchain for
  windows-msvc (bazel/toolchains/msvc).
- All eight shipped addons build as //:natives-<target> via the release
  transition in bazel/defs.bzl (opt, thin LTO, cgu=16, stripped, canonical
  .node naming); scripts/bazel-natives.ts is the single driver for local
  dev and CI.
- Rust validation moved to bazel test + clippy aspects (strict workspace
  policy for opted-in crates, default lints elsewhere, mirroring cargo
  semantics) and the rustfmt aspect; cargo stays as the dev-iteration
  surface, with brush-core/brush-builtins promoted to workspace members
  and excluded from cargo dev tasks to keep their historical scope.
- CI caches through an in-cluster bazel-remote action cache (TLS + basic
  auth, cluster-internal only); GitHub-hosted runners never touch the
  infrastructure and use an actions/cache-backed disk cache instead.
- Deleted the hand-rolled caching machinery: ci-target-cache,
  ci-native-artifact-cache, ci-build-native, native-source-hash,
  find-native-artifacts, restore-linux-native, native-prewarm workflow,
  ensure-* toolchain actions, and all sccache/Swatinem wiring.
- Warm native rebuilds drop from ~20 minutes to seconds; a cold client
  with a warm remote cache rebuilds the linux x64 pair in ~2.5 minutes.
2026-07-27 12:22:19 +02:00
can1357 5f988a8270 ci: configured native artifact caching and parallel execution in ci workflows
- Enhanced CI workflows and GitHub actions to support native artifact caching and parallel builds.
- Added composite actions and scripts for computing sources, finding artifacts, and managing caches.
- Updated infrastructure documentation and runner deployment scripts with revised resource limits.
2026-07-27 07:53:28 +02:00
can1357 69307261c3 chore(infra): baked cmake and ninja into the omp-kata runner image
- Pinned to the same versions as .github/actions/ensure-cmake (cmake 4.1.2, ninja 1.13.1), which now no-ops on the preloaded image.
- Extended both reload smoke-test tool lists with cmake/ninja.
- Rolled out as omp-kata-runner:2026-07-24-113811.
2026-07-24 11:43:10 +02:00
can1357 12950af096 perf(scripts): raised workspace test workers and ARC CPU limits
- Updated workspace-mode CI to run `workspaceTestCommand` with 8 workers instead of 4.
- Raised the documented ARC runner CPU limit from 8 to 16 in caching docs.
2026-06-15 02:44:16 +02:00
can1357 1f5307fdec feat(infra): migrated runner caches to PVC-backed Bun/Cargo and RustFS sccache
- Updated bun-install action to set mounted cache mode and use PVC cache paths.
- Removed RustFS Bun restore/save and maintenance scripts, replacing them with mounted cache setup.
- Removed zstd from runner image installation and baked-tool verification checks.
- Updated infra docs to describe split caching with RustFS for sccache and PVC for Bun/Cargo.
2026-06-15 02:24:35 +02:00
can1357 a6aa0c4ae8 fix(actions): fixed runner tooling setup and version parsing
- Updated Rust toolchain checks to use a prefix-aware grep pattern when validating installed components.
- Simplified the Zig installer action to extract into ~/.local and add the archive directory directly to PATH.
- Adjusted runner checks to parse sccache and gh version output via positional shell fields for stability.
2026-06-15 01:15:25 +02:00
can1357 551c599672 feat(infra): added infra automation for runner, cache, and kata runtime operations
- Enabled containerd-first runner builds with auto backend defaults, tool bootstrap, and docker fallback.
- Added RustFS cache maintenance report/prune modes with S3-driven listing, cache key grouping, and threshold exits.
- Added kata runtime tuning automation via SSH that patches config, runs smoke tests, and cleans up resources.
- Expanded runner-image smoke checks to verify clang, lld, sccache, zig, and cargo-* requirements.
2026-06-15 01:08:06 +02:00
can1357 ff5b06d0c2 ci: added CI workflows and actions for pinned toolchain-native builds
- Added composite GitHub actions to ensure rust toolchains and cargo helpers.
- Added a kata-native build action with variant checks and platform artifact uploads.
- Reworked CI matrices to split native cross-platform jobs and gate releases accordingly.
- Updated runner bootstrap and image to preinstall pinned build tools for CI consistency.
2026-06-15 01:01:45 +02:00
can1357 37d356f7bc test(mnemopi): centralized embedding test gating through shared Bun setup
- Added a Bun test preload config in packages/mnemopi/bunfig.toml to run shared setup automatically.
- Moved embedding opt-out logic into test/setup.ts, using EMBEDDINGS=1 to enable embeddings and defaulting MNEMOPI_NO_EMBEDDINGS otherwise.
- Updated embedding-focused tests to import RUN_EMBEDDINGS and skip suites unless embeddings are enabled.
2026-06-15 00:58:26 +02:00
can1357 f4e11db5b8 feat(infra): added kata-based ARC CI docs and reload tooling for k3s
- Added k3s host and cluster setup docs, including pinned install, module checks, and readiness commands.
- Added kata-runtime setup docs with containerd integration, RuntimeClass, and guest-kernel validation steps.
- Added ARC ARC/runner-and-cache deployment docs covering helm values, secrets, and pod network policy.
- Added reload-runner automation and a preloaded runner image Dockerfile with pinned toolchain and rollout checks.
2026-06-15 00:52:47 +02:00