Skip to content

[ci] Consolidate the OOP-JIT and in-process LLVM caches.#939

Open
vgvassilev wants to merge 1 commit intocompiler-research:mainfrom
vgvassilev:ci-oop-cache-cleanup
Open

[ci] Consolidate the OOP-JIT and in-process LLVM caches.#939
vgvassilev wants to merge 1 commit intocompiler-research:mainfrom
vgvassilev:ci-oop-cache-cleanup

Conversation

@vgvassilev
Copy link
Copy Markdown
Contributor

Two OOP-JIT cells (ubu24-x86-gcc12-llvm22-oop-vg, osx26-arm-clang-llvm22-oop) used a dedicated -oop-suffixed cache slot. Build_LLVM baked llvm-jitlink-executor and orc_rt-<arch> into that slot under a matrix.oop-jit == 'On' gate, and the matrix rows additionally enabled
llvm_enable_projects: clang;compiler-rt so the orc_rt target existed in the LLVM build. The whole arrangement existed because OOP-JIT used to require clang20-1-out-of-process.patch (a 966-line backport of clang::Interpreter::JITConfig), which genuinely changed the cached binary.

That patch retired in fb595ac (#924) when OOP-JIT moved to upstream release/22.x's clang::IncrementalExecutorBuilder. The OOP and in-process LLVM builds are now identical modulo the two runtime-only artifacts (llvm-jitlink-executor and orc_rt). Drop the dedicated cache slot and source the runtime parts from packages instead, so OOP rows free-ride the same cache as their in-process counterparts.

Cache-side:

  • main.yml: drop the ${{ matrix.oop-jit == 'On' && '-oop' || '' }} segment from the cache key. Every non-OOP key is bit-identical to before, so existing cache entries continue to hit -- no invalidation. The previously -oop-suffixed entries become orphaned (still in storage until eviction, just unreferenced).
  • main.yml matrix: drop llvm_enable_projects: clang;compiler-rt from both OOP rows so they share build config, and therefore cache content, with their non-OOP twins on the same axes.
  • Build_LLVM/action.yml: drop the if oop-jit == On: ninja llvm-jitlink-executor orc_rt-<suffix> block. The cache now contains a vanilla LLVM build regardless of matrix.oop-jit.

Runtime-side, OOP coverage preserved:

  • main.yml adds an Install OOP-JIT runtime step gated on matrix.oop-jit == 'On'. It installs the runtime parts via apt.llvm.org (llvm-${runtime} for the executor binary, libclang-rt-${runtime}-dev for the orc_rt static archive) on Linux, or brew install llvm@${runtime} on macOS. The install step then symlinks both into the test binary's expected resource-dir layout (<test-bin-dir>/lib/clang/<major>/{bin,lib/<triple>}/). That path matters: at test time clang::IncrementalExecutor Builder derives the resource directory from getMainExecutable() (the test binary), and the upstream safety check refuses to use an orc_rt whose path isn't under that directory. Symlinking before Build_and_Test_ CppInterOp runs is fine -- CppInterOp's build doesn't overwrite those paths.
  • Build_and_Test_CppInterOp/action.yml keeps both the -DLLVM_BUILT_WITH_OOP_JIT=${{ matrix.oop-jit }} cmake flag and the OOP-specific Valgrind suppression branch. The OOP test fixtures from PR Out-Of-Process Interpreter for CppInterOp #717 still parameterize over both in-process and OOP modes on those rows.

The cached LLVM stays LLVM_ENABLE_ASSERTIONS=ON, so the JIT compiler that runs in-process during CppInterOp's tests still fires its internal assertions -- which is the assertion signal worth keeping. orc_rt and llvm-jitlink-executor are runtime support code, not assertion-rich, so sourcing them from packages doesn't lose coverage.

Two follow-ups out of scope for this commit:

  • Retire the LLVM_BUILT_WITH_OOP_JIT macro from CppInterOp source. The compile-time gate becomes an LLVM-version / platform check, and a runtime check decides whether to actually use OOP. After that, Build_and_Test_CppInterOp no longer needs the cmake flag and the matrix's oop-jit field becomes purely a Valgrind-mode hint.
  • Bundle orc_rt (and ship llvm-jitlink-executor) inside libclangCppInterOp.so's distribution so the library is self-contained. Removes the install step entirely.

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 26, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.87%. Comparing base (fb595ac) to head (22aed11).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #939   +/-   ##
=======================================
  Coverage   80.87%   80.87%           
=======================================
  Files          15       15           
  Lines        4707     4707           
=======================================
  Hits         3807     3807           
  Misses        900      900           
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Two OOP-JIT cells (`ubu24-x86-gcc12-llvm22-oop-vg`,
`osx26-arm-clang-llvm22-oop`) used a dedicated `-oop`-suffixed
cache slot. `Build_LLVM` baked `llvm-jitlink-executor` and
`orc_rt-<arch>` into that slot under a `matrix.oop-jit == 'On'`
gate, and the matrix rows additionally enabled
`llvm_enable_projects: clang;compiler-rt` so the orc_rt target
existed in the LLVM build. The whole arrangement existed because
OOP-JIT used to require `clang20-1-out-of-process.patch` (a
966-line backport of `clang::Interpreter::JITConfig`), which
genuinely changed the cached binary.

That patch retired in fb595ac (compiler-research#924) when OOP-JIT moved to
upstream `release/22.x`'s `clang::IncrementalExecutorBuilder`.
The OOP and in-process LLVM builds are now identical modulo the
two runtime-only artifacts (`llvm-jitlink-executor` and
`orc_rt`). Drop the dedicated cache slot and source the runtime
parts from packages instead, so OOP rows free-ride the same
cache as their in-process counterparts.

Cache-side:

  - `main.yml`: drop the `${{ matrix.oop-jit == 'On' && '-oop' ||
    '' }}` segment from the cache key. Every non-OOP key is
    bit-identical to before, so existing cache entries continue
    to hit -- no invalidation. The previously `-oop`-suffixed
    entries become orphaned (still in storage until eviction,
    just unreferenced).
  - `main.yml` matrix: drop `llvm_enable_projects:
    clang;compiler-rt` from both OOP rows so they share build
    config, and therefore cache content, with their non-OOP
    twins on the same axes.
  - `Build_LLVM/action.yml`: drop the `if oop-jit == On: ninja
    llvm-jitlink-executor orc_rt-<suffix>` block. The cache now
    contains a vanilla LLVM build regardless of `matrix.oop-jit`.

Runtime-side, OOP coverage preserved:

  - `main.yml` adds an `Install OOP-JIT runtime` step gated on
    `matrix.oop-jit == 'On'`. It installs the runtime parts via
    apt.llvm.org (`llvm-${runtime}` for the executor binary,
    `libclang-rt-${runtime}-dev` for the orc_rt static archive)
    on Linux, or `brew install llvm@${runtime}` on macOS. The
    install step then symlinks both into the test binary's
    expected resource-dir layout
    (`<test-bin-dir>/lib/clang/<major>/{bin,lib/<triple>}/`).
    That path matters: at test time `clang::IncrementalExecutor
    Builder` derives the resource directory from
    `getMainExecutable()` (the test binary), and the upstream
    safety check refuses to use an `orc_rt` whose path isn't
    under that directory. Symlinking before `Build_and_Test_
    CppInterOp` runs is fine -- CppInterOp's build doesn't
    overwrite those paths.
  - `Build_and_Test_CppInterOp/action.yml` keeps both the
    `-DLLVM_BUILT_WITH_OOP_JIT=${{ matrix.oop-jit }}` cmake flag
    and the OOP-specific Valgrind suppression branch. The OOP
    test fixtures from PR compiler-research#717 still parameterize over both
    in-process and OOP modes on those rows.

The cached LLVM stays `LLVM_ENABLE_ASSERTIONS=ON`, so the JIT
compiler that runs in-process during CppInterOp's tests still
fires its internal assertions -- which is the assertion signal
worth keeping. `orc_rt` and `llvm-jitlink-executor` are runtime
support code, not assertion-rich, so sourcing them from packages
doesn't lose coverage.

Two follow-ups out of scope for this commit:

  - Retire the `LLVM_BUILT_WITH_OOP_JIT` macro from CppInterOp
    source. The compile-time gate becomes an LLVM-version /
    platform check, and a runtime check decides whether to
    actually use OOP. After that, `Build_and_Test_CppInterOp`
    no longer needs the cmake flag and the matrix's `oop-jit`
    field becomes purely a Valgrind-mode hint.
  - Bundle `orc_rt` (and ship `llvm-jitlink-executor`) inside
    `libclangCppInterOp.so`'s distribution so the library is
    self-contained. Removes the install step entirely.
@vgvassilev vgvassilev force-pushed the ci-oop-cache-cleanup branch from 63f9753 to 22aed11 Compare April 26, 2026 09:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant