[ci] Consolidate the OOP-JIT and in-process LLVM caches.#939
Open
vgvassilev wants to merge 1 commit intocompiler-research:mainfrom
Open
[ci] Consolidate the OOP-JIT and in-process LLVM caches.#939vgvassilev wants to merge 1 commit intocompiler-research:mainfrom
vgvassilev wants to merge 1 commit intocompiler-research:mainfrom
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #939 +/- ##
=======================================
Coverage 80.87% 80.87%
=======================================
Files 15 15
Lines 4707 4707
=======================================
Hits 3807 3807
Misses 900 900 🚀 New features to boost your workflow:
|
Two OOP-JIT cells (`ubu24-x86-gcc12-llvm22-oop-vg`, `osx26-arm-clang-llvm22-oop`) used a dedicated `-oop`-suffixed cache slot. `Build_LLVM` baked `llvm-jitlink-executor` and `orc_rt-<arch>` into that slot under a `matrix.oop-jit == 'On'` gate, and the matrix rows additionally enabled `llvm_enable_projects: clang;compiler-rt` so the orc_rt target existed in the LLVM build. The whole arrangement existed because OOP-JIT used to require `clang20-1-out-of-process.patch` (a 966-line backport of `clang::Interpreter::JITConfig`), which genuinely changed the cached binary. That patch retired in fb595ac (compiler-research#924) when OOP-JIT moved to upstream `release/22.x`'s `clang::IncrementalExecutorBuilder`. The OOP and in-process LLVM builds are now identical modulo the two runtime-only artifacts (`llvm-jitlink-executor` and `orc_rt`). Drop the dedicated cache slot and source the runtime parts from packages instead, so OOP rows free-ride the same cache as their in-process counterparts. Cache-side: - `main.yml`: drop the `${{ matrix.oop-jit == 'On' && '-oop' || '' }}` segment from the cache key. Every non-OOP key is bit-identical to before, so existing cache entries continue to hit -- no invalidation. The previously `-oop`-suffixed entries become orphaned (still in storage until eviction, just unreferenced). - `main.yml` matrix: drop `llvm_enable_projects: clang;compiler-rt` from both OOP rows so they share build config, and therefore cache content, with their non-OOP twins on the same axes. - `Build_LLVM/action.yml`: drop the `if oop-jit == On: ninja llvm-jitlink-executor orc_rt-<suffix>` block. The cache now contains a vanilla LLVM build regardless of `matrix.oop-jit`. Runtime-side, OOP coverage preserved: - `main.yml` adds an `Install OOP-JIT runtime` step gated on `matrix.oop-jit == 'On'`. It installs the runtime parts via apt.llvm.org (`llvm-${runtime}` for the executor binary, `libclang-rt-${runtime}-dev` for the orc_rt static archive) on Linux, or `brew install llvm@${runtime}` on macOS. The install step then symlinks both into the test binary's expected resource-dir layout (`<test-bin-dir>/lib/clang/<major>/{bin,lib/<triple>}/`). That path matters: at test time `clang::IncrementalExecutor Builder` derives the resource directory from `getMainExecutable()` (the test binary), and the upstream safety check refuses to use an `orc_rt` whose path isn't under that directory. Symlinking before `Build_and_Test_ CppInterOp` runs is fine -- CppInterOp's build doesn't overwrite those paths. - `Build_and_Test_CppInterOp/action.yml` keeps both the `-DLLVM_BUILT_WITH_OOP_JIT=${{ matrix.oop-jit }}` cmake flag and the OOP-specific Valgrind suppression branch. The OOP test fixtures from PR compiler-research#717 still parameterize over both in-process and OOP modes on those rows. The cached LLVM stays `LLVM_ENABLE_ASSERTIONS=ON`, so the JIT compiler that runs in-process during CppInterOp's tests still fires its internal assertions -- which is the assertion signal worth keeping. `orc_rt` and `llvm-jitlink-executor` are runtime support code, not assertion-rich, so sourcing them from packages doesn't lose coverage. Two follow-ups out of scope for this commit: - Retire the `LLVM_BUILT_WITH_OOP_JIT` macro from CppInterOp source. The compile-time gate becomes an LLVM-version / platform check, and a runtime check decides whether to actually use OOP. After that, `Build_and_Test_CppInterOp` no longer needs the cmake flag and the matrix's `oop-jit` field becomes purely a Valgrind-mode hint. - Bundle `orc_rt` (and ship `llvm-jitlink-executor`) inside `libclangCppInterOp.so`'s distribution so the library is self-contained. Removes the install step entirely.
63f9753 to
22aed11
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two OOP-JIT cells (
ubu24-x86-gcc12-llvm22-oop-vg,osx26-arm-clang-llvm22-oop) used a dedicated-oop-suffixed cache slot.Build_LLVMbakedllvm-jitlink-executorandorc_rt-<arch>into that slot under amatrix.oop-jit == 'On'gate, and the matrix rows additionally enabledllvm_enable_projects: clang;compiler-rtso the orc_rt target existed in the LLVM build. The whole arrangement existed because OOP-JIT used to requireclang20-1-out-of-process.patch(a 966-line backport ofclang::Interpreter::JITConfig), which genuinely changed the cached binary.That patch retired in fb595ac (#924) when OOP-JIT moved to upstream
release/22.x'sclang::IncrementalExecutorBuilder. The OOP and in-process LLVM builds are now identical modulo the two runtime-only artifacts (llvm-jitlink-executorandorc_rt). Drop the dedicated cache slot and source the runtime parts from packages instead, so OOP rows free-ride the same cache as their in-process counterparts.Cache-side:
main.yml: drop the${{ matrix.oop-jit == 'On' && '-oop' || '' }}segment from the cache key. Every non-OOP key is bit-identical to before, so existing cache entries continue to hit -- no invalidation. The previously-oop-suffixed entries become orphaned (still in storage until eviction, just unreferenced).main.ymlmatrix: dropllvm_enable_projects: clang;compiler-rtfrom both OOP rows so they share build config, and therefore cache content, with their non-OOP twins on the same axes.Build_LLVM/action.yml: drop theif oop-jit == On: ninja llvm-jitlink-executor orc_rt-<suffix>block. The cache now contains a vanilla LLVM build regardless ofmatrix.oop-jit.Runtime-side, OOP coverage preserved:
main.ymladds anInstall OOP-JIT runtimestep gated onmatrix.oop-jit == 'On'. It installs the runtime parts via apt.llvm.org (llvm-${runtime}for the executor binary,libclang-rt-${runtime}-devfor the orc_rt static archive) on Linux, orbrew install llvm@${runtime}on macOS. The install step then symlinks both into the test binary's expected resource-dir layout (<test-bin-dir>/lib/clang/<major>/{bin,lib/<triple>}/). That path matters: at test timeclang::IncrementalExecutor Builderderives the resource directory fromgetMainExecutable()(the test binary), and the upstream safety check refuses to use anorc_rtwhose path isn't under that directory. Symlinking beforeBuild_and_Test_ CppInterOpruns is fine -- CppInterOp's build doesn't overwrite those paths.Build_and_Test_CppInterOp/action.ymlkeeps both the-DLLVM_BUILT_WITH_OOP_JIT=${{ matrix.oop-jit }}cmake flag and the OOP-specific Valgrind suppression branch. The OOP test fixtures from PR Out-Of-Process Interpreter for CppInterOp #717 still parameterize over both in-process and OOP modes on those rows.The cached LLVM stays
LLVM_ENABLE_ASSERTIONS=ON, so the JIT compiler that runs in-process during CppInterOp's tests still fires its internal assertions -- which is the assertion signal worth keeping.orc_rtandllvm-jitlink-executorare runtime support code, not assertion-rich, so sourcing them from packages doesn't lose coverage.Two follow-ups out of scope for this commit:
LLVM_BUILT_WITH_OOP_JITmacro from CppInterOp source. The compile-time gate becomes an LLVM-version / platform check, and a runtime check decides whether to actually use OOP. After that,Build_and_Test_CppInterOpno longer needs the cmake flag and the matrix'soop-jitfield becomes purely a Valgrind-mode hint.orc_rt(and shipllvm-jitlink-executor) insidelibclangCppInterOp.so's distribution so the library is self-contained. Removes the install step entirely.