Commit Graph

36 Commits

Author SHA1 Message Date
can1357 053dbfa605 fix(ci): stopped mtime prune from gutting extracted bazel repos
- The hosted disk-cache prune swept ~/.cache/omp-bazel-repo file-by-file;
  extracted repository contents keep upstream-archive mtimes (months old),
  so a restored archive lost most of rules_rust while bazel still trusted
  the entry's recorded_inputs — both darwin release legs failed with
  'BUILD file not found' in release run 30519253683. Prune only the
  action disk cache, whose files carry bazel-written mtimes.
- Gave the logger burst-order contract an explicit 30 s budget: two probe
  children measure ~4.4 s unloaded and bun's 5 s default test timeout
  SIGTERMed them (exit 143) on shared-core runners.
2026-07-30 08:33:27 +02:00
can1357 52bd191b34 build: optimized Bazel repository and toolchain caching for CI
- Pin Rust toolchain component checksums in MODULE.bazel to enable Bazel's repo contents cache.
- Opt MSVC LLVM tools, XWin sysroot, and Zig repositories into reproducible repo metadata caching.
- Extend GitHub Actions cache paths to include the Bazel repository download and contents cache.
2026-07-30 07:21:43 +02:00
can1357 a38a2f25cf ci: optimized github actions caching and workflows
- Updated GitHub Actions workflows and custom actions to optimize caching strategies and runners.
- Configured separate restore and save steps for bun store caching with non-PR restrictions.
- Added darwin release bazel cache seeding and fallback keys for cache restore.
- Removed native-inputs workflow action and disabled PR-side Rust validation.
2026-07-30 04:56:47 +02:00
can1357 b550858265 ci: upgraded continuous integration workflows and migrated bazel dependency locking
- Updated CI workflows and GitHub actions to enhance Bazel cache keying, credential masking, and validation checks.
- Migrated dependency locking from Cargo.Bazel.lock to MODULE.bazel.lock using rules_rust crate_universe.
- Updated build configuration, documentation, and tooling scripts to reflect the lockfile and cache changes.
2026-07-28 12:53:23 +02:00
can1357 0820085890 ci: restructured workflow pipelines and introduced bazel cache actions
- Updated bazel cache key generation with v2 schema version and streamlined remote cache usage.
- Added native-inputs composite action to centralize change detection and artifact caching.
- Added a scheduled workflow to warm the hosted bazel disk cache.
- Split the rust validation job and integrated prebuilt native addon caching into the CI pipeline.
2026-07-28 11:55:57 +02:00
can1357 ed4c78bc0f feat: streamlined native addon builds and caching in ci workflows
- Enhance CI workflows to build and reuse native addon artifacts instead of recompiling.
- Update bazel cache actions with selective backend detection and separate remote and disk modes.
- Add `--source` CLI option to install prebuilt native targets without requiring Bazel.
- Increase Kata runner memory configuration from 12Gi to 24Gi.
2026-07-28 02:06:13 +02:00
can1357 a7abeff1b7 perf(ci): cut warm CI time via download-skipping, splat reuse, PR gating
Four levers on top of the green pipeline:
- kata jobs pass --remote_download_toplevel, so fully cache-hit builds
  stay metadata-only instead of pulling every intermediate artifact from
  bazel-remote (the bulk of the previous 6-minute TS-only main runs).
- the xwin MSVC splat caches its ~1GiB CDN payload on the runner-cache
  PVC (OMP_XWIN_CACHE_DIR), instead of re-downloading per ephemeral pod.
- main-push rust jobs export their bazel disk cache to the GitHub cache
  (once per lockfile change, shared linux scope). GitHub only shares
  default-branch caches across PRs, and main runs on kata where
  actions/cache never saved — so every fresh PR was building cold.
- TS-only pull requests skip Rust validation entirely (gh pr diff path
  gate); their test jobs restore addons from the main-exported cache.

Export runs disable top-level-only downloading: remote hits would
otherwise export action entries whose blobs were never materialized.
2026-07-27 14:31:24 +02:00
can1357 4fd3662114 fix(ci): reused sandbox trees to stop fd exhaustion on kata pods
The zig and xwin toolchains stage ~10k-file input trees per action;
building and async-deleting thousands of sandbox trees exhausted file
descriptors (EMFILE in unix_jni during sandbox setup). --reuse_sandbox_directories
under --config=ci removes the churn, with a raise-only ulimit guard in
the bazel-launching steps as belt and braces.
2026-07-27 12:48:57 +02:00
can1357 02c50eb6f3 fix(ci): moved bazel repo cache mount outside the runner home
kubelet creates missing subPath mountpoint parents as root, so mounting
the PVC repository cache under ~/.cache left the directory root-owned
and broke both bazel's default output root and zig's wrapper cache
compile (AccessDenied). The mount now lives at /opt/bazel-repo-cache.
2026-07-27 12:27:31 +02:00
can1357 2092f9330c fix(ci): moved bazel output root off the root-owned kata cache dir
kubelet materializes /home/runner/.cache as root when creating the
omp-bazel-repo subPath mountpoint, so bazel's default output_user_root
under it fails with EACCES. Kata jobs now point output_user_root at
RUNNER_TEMP via the bazel-cache rc fragment (pods are single-job
ephemeral; toolchain/crate downloads stay on the PVC repository cache),
and the runner image pre-owns ~/.cache for the next rebake.
2026-07-27 12:24:35 +02:00
can1357 8facd237d5 feat(build): migrated native pipeline to bazel with remote caching
- Replaced the napi-cli/cargo-zigbuild/cargo-xwin/sccache build path with
  Bazel: rules_rust + crate_universe over Cargo.lock, hermetic zig cc
  toolchains (linux-gnu pinned to glibc 2.17, linux-musl), host Xcode for
  darwin, and a repo-local hermetic clang-cl + llvm-ml + xwin toolchain for
  windows-msvc (bazel/toolchains/msvc).
- All eight shipped addons build as //:natives-<target> via the release
  transition in bazel/defs.bzl (opt, thin LTO, cgu=16, stripped, canonical
  .node naming); scripts/bazel-natives.ts is the single driver for local
  dev and CI.
- Rust validation moved to bazel test + clippy aspects (strict workspace
  policy for opted-in crates, default lints elsewhere, mirroring cargo
  semantics) and the rustfmt aspect; cargo stays as the dev-iteration
  surface, with brush-core/brush-builtins promoted to workspace members
  and excluded from cargo dev tasks to keep their historical scope.
- CI caches through an in-cluster bazel-remote action cache (TLS + basic
  auth, cluster-internal only); GitHub-hosted runners never touch the
  infrastructure and use an actions/cache-backed disk cache instead.
- Deleted the hand-rolled caching machinery: ci-target-cache,
  ci-native-artifact-cache, ci-build-native, native-source-hash,
  find-native-artifacts, restore-linux-native, native-prewarm workflow,
  ensure-* toolchain actions, and all sccache/Swatinem wiring.
- Warm native rebuilds drop from ~20 minutes to seconds; a cold client
  with a warm remote cache rebuilds the linux x64 pair in ~2.5 minutes.
2026-07-27 12:22:19 +02:00
can1357 5f988a8270 ci: configured native artifact caching and parallel execution in ci workflows
- Enhanced CI workflows and GitHub actions to support native artifact caching and parallel builds.
- Added composite actions and scripts for computing sources, finding artifacts, and managing caches.
- Updated infrastructure documentation and runner deployment scripts with revised resource limits.
2026-07-27 07:53:28 +02:00
can1357 59619623e1 feat: enabled cargo target caching and conditional validation in workflows
- Add scripts/ci-target-cache.ts to snapshot and restore Cargo target directories to S3 storage on omp-kata runners.
- Update .github/actions/build-native/action.yml to support target cache restoration, saving, and compiler launcher configurations.
- Add skip_validation input in GitHub workflow actions to bypass clippy and Rust test checks on release runs.
2026-07-25 02:34:14 +02:00
can1357 69307261c3 chore(infra): baked cmake and ninja into the omp-kata runner image
- Pinned to the same versions as .github/actions/ensure-cmake (cmake 4.1.2, ninja 1.13.1), which now no-ops on the preloaded image.
- Extended both reload smoke-test tool lists with cmake/ninja.
- Rolled out as omp-kata-runner:2026-07-24-113811.
2026-07-24 11:43:10 +02:00
can1357 90e0a8af28 fix(ci): repaired native builds broken by native audio stack deps
- Added ensure-cmake action installing pinned cmake/ninja on omp-kata pods; audiopus_sys builds bundled libopus via CMake (Ninja for MSVC cross).
- Set CMAKE_POLICY_VERSION_MINIMUM=3.5 globally and in build-native.ts: the bundled opus tree declares cmake_minimum_required below 3.5, which CMake 4.x refuses.
- Dropped the -C target-cpu=native fallback for non-x64 native builds: it baked build-host CPU features into shipped darwin arm64 addons and trips ring 0.17's aarch64-apple const assertion.
2026-07-24 09:36:31 +02:00
roboomp 6efcdfb66e feat(release): added musl-linked linux builds
Built x64 and arm64 musl release artifacts with matching native addons, Alpine smoke coverage, and installer detection.

Fixes #3367
2026-07-22 19:06:20 +00:00
can1357 75ab023434 fix(actions/bun-install): retried bun install with job-local cache on first-attempt failure
- Updated the bun-install action to retry `bun install --frozen-lockfile` when the first attempt fails.
- Added a fallback path that creates a temporary job-local cache directory and reruns install with `--cache-dir`.
- Emitted a warning to note the shared-store failure before the retry path is used.
2026-06-15 09:25:02 +02:00
can1357 bc6130aad4 fix(natives): build linux addons against a glibc 2.17 floor via cargo-zigbuild
Native linux-x64/arm64 builds moved onto the Ubuntu 24.04 (glibc 2.39)
omp-kata runner. The x64 addon was a plain host build that linked the
runner's glibc and failed to dlopen with `version 'GLIBC_2.39' not found`
on older distros; the arm64 cross-build floated up to GLIBC_2.30. Build
the shipped linux-gnu addons through cargo-zigbuild against a pinned 2.17
floor so they load on any glibc >= 2.17.

- build-native.ts: key the tree-sitter-just `-UNDEBUG` CFLAGS off the
  bare triple (cargo-zigbuild strips the `.2.17` glibc suffix before
  invoking cargo) and symlink the suffixed target dir napi 3.7.0 expects
  to the bare dir cargo-zigbuild writes, so postBuild copyArtifact finds
  the cdylib.
- build-native action: add a `glibc` input plus a resolve step deriving
  the zigbuild cross_target (suffixed) and the rustup bare_target
  (stripped); gate zig/cargo-zigbuild install on cross_target so the
  host-arch x64 build still runs native Rust tests.
- ci.yml: GLIBC_FLOOR=2.17 fed to the linux-x64 and linux-arm64 native
  jobs.

Re-tags 15.13.1, whose release failed at the linux-x64 binary smoke
before any publish step ran.
2026-06-15 05:39:02 +02:00
can1357 05fd499551 Merge PR #1435: feat: added isolated profiles with --profile and --alias
Closes #1435

# Conflicts:
#	.github/actions/bun-install/action.yml
2026-06-15 02:47:12 +02:00
Ogrodev 932ebe9a48 Merge remote-tracking branch 'upstream/main' into feat/profiles-and-alias
# Conflicts:
#	.github/actions/build-native/action.yml
#	.github/workflows/ci.yml
2026-06-14 21:26:03 -03:00
can1357 1f5307fdec feat(infra): migrated runner caches to PVC-backed Bun/Cargo and RustFS sccache
- Updated bun-install action to set mounted cache mode and use PVC cache paths.
- Removed RustFS Bun restore/save and maintenance scripts, replacing them with mounted cache setup.
- Removed zstd from runner image installation and baked-tool verification checks.
- Updated infra docs to describe split caching with RustFS for sccache and PVC for Bun/Cargo.
2026-06-15 02:24:35 +02:00
Ogrodev c7537eb1f4 fix(ci): disable rustfs bun cache on test jobs 2026-06-14 21:20:59 -03:00
can1357 a6aa0c4ae8 fix(actions): fixed runner tooling setup and version parsing
- Updated Rust toolchain checks to use a prefix-aware grep pattern when validating installed components.
- Simplified the Zig installer action to extract into ~/.local and add the archive directory directly to PATH.
- Adjusted runner checks to parse sccache and gh version output via positional shell fields for stability.
2026-06-15 01:15:25 +02:00
can1357 ff5b06d0c2 ci: added CI workflows and actions for pinned toolchain-native builds
- Added composite GitHub actions to ensure rust toolchains and cargo helpers.
- Added a kata-native build action with variant checks and platform artifact uploads.
- Reworked CI matrices to split native cross-platform jobs and gate releases accordingly.
- Updated runner bootstrap and image to preinstall pinned build tools for CI consistency.
2026-06-15 01:01:45 +02:00
can1357 198b1cdec2 fix(ci): close bun-install cache backend branch
The backend-detection shell block in .github/actions/bun-install/action.yml was missing its closing fi, so every job that touched the composite failed immediately with . Restore the RustFS/GHA branch correctly and validate with YAML parse + bash -n.
2026-06-15 01:00:19 +02:00
can1357 82871093bd ci(actions): skipped bun setup and rust-cache on shared-sccache runners
- Updated the bun-install composite action to detect preinstalled Bun and only fetch it when missing.
- Added cache-backend detection and wiring so Bun dependencies use RustFS cache when SCCACHE credentials are present.
- Conditionally skipped rust-cache in build-native and CI jobs when shared sccache runners are available, relying on the existing RustFS/sccache layer instead.
2026-06-15 00:52:32 +02:00
can1357 5959bb0ad3 test(mnemopi): disabled embeddings in mnemopi tests
- Added beforeEach and afterEach hooks in mnemopi tests to set and clear MNEMOPI_NO_EMBEDDINGS so embeddings are skipped during those runs.
- Updated the bun-install cache script to archive only node_modules paths that exist as directories.
2026-06-15 00:29:24 +02:00
can1357 2be8256af0 ci(workflows): shared bun caching in CI via backend-aware install action
- Updated CI dependency install flow to share bun cache orchestration across jobs.
- Added RustFS-backed bun cache restore/save script keyed by bun.lock hash.
2026-06-15 00:18:43 +02:00
can1357 6d8bd80392 ci(workflows): share rust sccache via in-cluster RustFS S3 on self-hosted runners
Self-hosted omp-kata runners now inject a shared S3 (RustFS, in-cluster)
sccache backend via pod env (SCCACHE_BUCKET/ENDPOINT/REGION + AWS creds).
The Enable-sccache step branches on SCCACHE_BUCKET: when set, sccache reads
the S3 config from the inherited environment; otherwise GitHub-hosted
runners (macOS, ubuntu-arm) keep the GHA cache backend since they can't
reach the private RustFS.
2026-06-14 23:07:54 +02:00
can1357 b8e4da23d0 ci(ci): refactored CI setup and test-state isolation for coding-agent workflows
- Added a setup-system-deps action with preloaded-runner guards and apt fallbacks.
- Updated CI workflows to download Linux x64 native artifacts and gate on native job success.
- Renamed coding-agent fast mode to singleton in scripts and test partitioning logic.
- Added settings test-state begin/restore helpers with recursive cleanup in affected tests.
2026-06-14 22:37:05 +02:00
can1357 61c2e29532 ci(ci): aligned CI release metadata flow after job and output renames
- Renamed the gate and native jobs in CI for consistent release naming.
- Renamed reusable-artifact output keys for native lookup compatibility.
- Rewired release and test jobs to read tags, flags, and hashes from metadata outputs.
2026-06-08 12:26:31 +02:00
can1357 71fe258320 ci: migrated release trigger from tag-push to branch-push
- Removed `tags: ["v*"]` trigger; release now fires from the single atomic `main` push that carries the tag.
- Replaced `gate.skip` with `gate.is-release` and `gate.release-tag` so downstream jobs detect the tag via `git tag --points-at HEAD`.
- Passed `release-tag` explicitly to release steps (gh-release, curl, release notes) since `github.ref` is now `refs/heads/main`, not the tag.
- Fixed rust-cache key collision on macOS x64 by setting `RUSTFLAGS` (target-cpu) before cache restore and including the native source hash in the shared key.
2026-05-31 02:33:19 +02:00
can1357 58d95ae258 ci: set CARGO_INCREMENTAL=0 to prevent sccache from being bypassed
- sccache silently skips caching when incremental compilation is enabled, making the wrapper a no-op.
- Applied fix to both the shared build-native action and the CI workflow setup step.
2026-05-27 02:09:14 +02:00
can1357 3514ea3cfa ci(actions): enabled cross-compilation toolchains across CI native and release builds
- Updated the build-native action to install cargo-zigbuild for non-MSVC targets and cargo-xwin with LLVM tooling for MSVC targets.
- Removed the fixed aarch64 linker variable and narrowed Rust test execution to skip duplicate macOS runs.
- Reworked CI matrices to build win32-x64 artifacts on ubuntu with x86_64-pc-windows-msvc and simplified smoke testing to skip those Windows binaries via matrix gating.
2026-05-27 01:54:26 +02:00
can1357 db3362be95 ci(workflows): optimized CI native artifact reuse and added sccache fallback
- Updated `rust-hash` to return separate `linux-run-id` and `release-run-id` outputs by checking each prior run for required native artifacts.
- Rewired `native_linux` and `native_release` gating to use those outputs, enabling Linux cache reuse on main and release pre-warm only when all cross-platform artifacts are present.
- Enabled sccache in both the shared native build action and CI setup, and excluded macOS workspace Rust tests from the generic native test step due to duplicated coverage.
2026-05-27 01:33:09 +02:00
can1357 7abac9f96e ci: split native job into linux and release-only platform jobs
- Extracted repeated build steps into a reusable `build-native` composite action.
- Split `native` into `native_linux` (runs on every PR) and `native_release` (tags only).
- Release jobs now always use same-run artifacts, removing cross-run artifact resolution logic.
2026-05-17 05:14:18 +02:00