Skip to content

Commit 62f0acf

Browse files
vgvassilevvgvassilev
committed
[ci] Make the LLVM/Cling cache leaner and the cling prefix readable.
Six tied-together hygiene fixes on the cache and Build_LLVM action. None of them is individually large enough to ship alone; together they shrink the cached artifact, fix a latent WASM trim bug, clean up the cling part of the cache key, and unblock the parallel ROOT-integration work which needs `clangInterpreter` and the source-tree `lib/` drop. Source-tree shrink. `Build_LLVM` and `Build_LLVM_WASM` were keeping the source `lib/` dirs of `llvm-project/{llvm,clang}` in the cache even though nothing on the consumer include path references them (`Build_and_Test_CppInterOp/action.yml` only adds `llvm/include` and `clang/include` to `CPLUS_INCLUDE_PATH`) and ROOT with `builtin_*=OFF` links the pre-built `.a` files instead of recompiling. Drop them from the keep-lists. Estimated savings ~50-100 MB compressed per cache row, ~1-1.5 GB across the pool. Build a leaner LLVM. The cling cmake call gains `-DLLVM_INCLUDE_BENCHMARKS=OFF -DLLVM_INCLUDE_EXAMPLES=OFF`, mirroring the repl branch. These suppress targets that never end up in the cache but cost configure time and incremental dependency tracking. Notably we do NOT pass `-DLLVM_INCLUDE_TESTS=OFF` here: cling's `CMakeLists.txt:437-452` builds its runtime `CLING_INCLUDE_PATHS` (used by `CIFactory.cpp` to find `cling/Interpreter/RuntimeUniverse.h` at runtime) inside an `if(CLING_INCLUDE_TESTS)` block, and `CLING_INCLUDE_TESTS` defaults to `${LLVM_INCLUDE_TESTS}`. Disabling it leaves the path list empty, so cling at runtime cannot locate its own headers. The repl branch (LLVM_INCLUDE_TESTS=OFF, no cling) is unaffected. Add `clangInterpreter` to the cling ninja line. ROOT's bundled `interpreter/cling/lib/Interpreter/CMakeLists.txt:14` lists `clangInterpreter` in its `LIBS`, but cling v1.3 (the version we cache against) does not. Without it, downstream consumers that rebuild ROOT's bundled cling fail at link with `libclangInterpreter.a ... missing and no known rule to make it`: the imported target from `ClangTargets.cmake` points at a file that was never built. The Build_LLVM trim deletes the top-level `llvm/CMakeLists.txt`, so the lib cannot be ninja'd later from the cached build -- it must be built here. Fix Build_LLVM_WASM source-trim. The Linux block's `find` was missing its closing `)`, leaving the substitution unterminated and swallowing the next half-dozen lines. Cling-on-WASM trims have been broken since the action was last refactored; correct on its own merits and required for the source-`lib/` shrink to actually take effect on the WASM side. Quote `cling: 'On'` on all matrix rows. YAML 1.1 parsers (notably nektos/act) interpret bare `On` as boolean true; the upper-cased comparison in Save_PR_Info against the literal string `"ON"` then fails, the row gets `CLING_HASH=Repl`, and the cache key drifts off real CI. GitHub Actions own parser preserves bare `On` as a string, so real CI was unaffected; quoting eliminates the act-only divergence. CLING_HASH and LLVM_HASH cleanup. The previous values embedded `git ls-remote` internals verbatim: `tr '\t' '-'` on the `<sha>\t<ref>` line produced `<40-char-sha>-refs/tags/v1.3` -- a 60-char string with a refspec prefix that nothing downstream wants in a cache key. Save_PR_Info now extracts the SHA via `awk '{print $1}'`, truncates to 8 chars, and prefixes a readable label, giving `cling-v1.3-c2beed49`. The cache-key formula in `main.yml` is left as-is, so non-cling rows (`CLING_HASH=Repl`) keep hitting their existing cache entries; only cling rows are invalidated by the new prefix. Save_PR_Info is also converted to LF line endings (it was the only CRLF file in the action set, which made every prior diff to the file noisy with `^M`).
1 parent 674ac58 commit 62f0acf

4 files changed

Lines changed: 115 additions & 76 deletions

File tree

‎.github/actions/Build_LLVM/action.yml‎

Lines changed: 31 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -33,10 +33,22 @@ runs:
3333
-DCLANG_ENABLE_ARCMT=OFF \
3434
-DCLANG_ENABLE_FORMAT=OFF \
3535
-DCLANG_ENABLE_BOOTSTRAP=OFF \
36+
-DLLVM_INCLUDE_BENCHMARKS=OFF \
37+
-DLLVM_INCLUDE_EXAMPLES=OFF \
3638
-G Ninja \
3739
../llvm
3840
ninja clang -j ${{ env.ncpus }}
3941
ninja LLVMOrcDebugging -j ${{ env.ncpus }}
42+
# `clangInterpreter` (clang-repl's Interpreter library) is not
43+
# a transitive dep of the `clang` driver, and the cling version
44+
# we cache (v1.3) doesn't list it in its `LIBS`. ROOT's bundled
45+
# `interpreter/cling/lib/Interpreter/CMakeLists.txt:14` does,
46+
# so downstream consumers (the ROOT integration job) link
47+
# against `libclangInterpreter.a` and fail with `missing and
48+
# no known rule to make it` if it isn't in the cached build.
49+
# The trim removes top-level `llvm/CMakeLists.txt`, so we can't
50+
# ninja it later from the cache -- it has to be built here.
51+
ninja clangInterpreter -j ${{ env.ncpus }}
4052
ninja clingInterpreter -j ${{ env.ncpus }}
4153
else
4254
if [[ "${{ matrix.oop-jit }}" == "On" ]]; then
@@ -74,18 +86,24 @@ runs:
7486
fi
7587
cd ../
7688
rm -rf $(find . -maxdepth 1 ! -name "build" ! -name "llvm" ! -name "clang" ! -name ".")
89+
# Source-tree `lib/` dirs hold .cpp already linked into
90+
# build/lib/*.a. CppInterOp consumers reference llvm/include and
91+
# clang/include via CPLUS_INCLUDE_PATH, never the source `lib/`
92+
# (see Build_and_Test_CppInterOp/action.yml). cling/lib stays --
93+
# ROOT and downstream consumers may pick up cling header internals
94+
# from there.
7795
if [[ "${cling_on}" == "ON" ]]; then
7896
cd ./llvm/
79-
rm -rf $(find . -maxdepth 1 ! -name "include" ! -name "lib" ! -name "cmake" ! -name "utils" ! -name ".")
97+
rm -rf $(find . -maxdepth 1 ! -name "include" ! -name "cmake" ! -name "utils" ! -name ".")
8098
cd ../clang/
81-
rm -rf $(find . -maxdepth 1 ! -name "include" ! -name "lib" ! -name "cmake" ! -name "utils" ! -name ".")
99+
rm -rf $(find . -maxdepth 1 ! -name "include" ! -name "cmake" ! -name "utils" ! -name ".")
82100
cd ../../cling/
83101
rm -rf $(find . -maxdepth 1 ! -name "include" ! -name "lib" ! -name "cmake" ! -name ".")
84102
else # repl
85103
cd ./llvm/
86-
rm -rf $(find . -maxdepth 1 ! -name "include" ! -name "lib" ! -name "cmake" ! -name ".")
104+
rm -rf $(find . -maxdepth 1 ! -name "include" ! -name "cmake" ! -name ".")
87105
cd ../clang/
88-
rm -rf $(find . -maxdepth 1 ! -name "include" ! -name "lib" ! -name "cmake" ! -name ".")
106+
rm -rf $(find . -maxdepth 1 ! -name "include" ! -name "cmake" ! -name ".")
89107
cd ../..
90108
fi
91109
@@ -124,9 +142,13 @@ runs:
124142
-DCLANG_ENABLE_ARCMT=OFF `
125143
-DCLANG_ENABLE_FORMAT=OFF `
126144
-DCLANG_ENABLE_BOOTSTRAP=OFF `
145+
-DLLVM_INCLUDE_BENCHMARKS=OFF `
146+
-DLLVM_INCLUDE_EXAMPLES=OFF `
127147
..\llvm
128148
cmake --build . --config Release --target clang --parallel ${{ env.ncpus }}
129149
cmake --build . --config Release --target LLVMOrcDebugging --parallel ${{ env.ncpus }}
150+
# See Linux block above for why clangInterpreter is built here.
151+
cmake --build . --config Release --target clangInterpreter --parallel ${{ env.ncpus }}
130152
cmake --build . --config Release --target clingInterpreter --parallel ${{ env.ncpus }}
131153
}
132154
else
@@ -146,20 +168,21 @@ runs:
146168
}
147169
cd ..\
148170
rm -r -force $(find.exe . -maxdepth 1 ! -name "build" ! -name "llvm" ! -name "clang" ! -name ".")
171+
# See Linux block above for why source `lib/` is dropped.
149172
if ( "${{ matrix.cling }}" -imatch "On" )
150173
{
151174
cd .\llvm\
152-
rm -r -force $(find.exe . -maxdepth 1 ! -name "include" ! -name "lib" ! -name "cmake" ! -name "utils" ! -name ".")
175+
rm -r -force $(find.exe . -maxdepth 1 ! -name "include" ! -name "cmake" ! -name "utils" ! -name ".")
153176
cd ..\clang\
154-
rm -r -force $(find.exe . -maxdepth 1 ! -name "include" ! -name "lib" ! -name "cmake" ! -name "utils" ! -name ".")
177+
rm -r -force $(find.exe . -maxdepth 1 ! -name "include" ! -name "cmake" ! -name "utils" ! -name ".")
155178
cd ..\..\cling\
156179
rm -r -force $(find.exe . -maxdepth 1 ! -name "include" ! -name "lib" ! -name "cmake" ! -name ".")
157180
}
158181
else
159182
{
160183
cd .\llvm\
161-
rm -r -force $(find.exe . -maxdepth 1 ! -name "include" ! -name "lib" ! -name "cmake" ! -name ".")
184+
rm -r -force $(find.exe . -maxdepth 1 ! -name "include" ! -name "cmake" ! -name ".")
162185
cd ..\clang\
163-
rm -r -force $(find.exe . -maxdepth 1 ! -name "include" ! -name "lib" ! -name "cmake" ! -name ".")
186+
rm -r -force $(find.exe . -maxdepth 1 ! -name "include" ! -name "cmake" ! -name ".")
164187
cd ..\..
165188
}

‎.github/actions/Build_LLVM_WASM/action.yml‎

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -80,11 +80,13 @@ runs:
8080
cd ..
8181
8282
# Trim source trees so the cache stays small. Cling rows additionally
83-
# need llvm/{utils} for runtime tablegen lookups.
84-
rm -rf $(find . -maxdepth 1 ! -name "build" ! -name "llvm" ! -name "clang" ! -name "." ! -name "native_build"
85-
keep=('!' -name include '!' -name lib '!' -name cmake '!' -name .)
83+
# need llvm/{utils} for runtime tablegen lookups. Source-tree
84+
# `lib/` dirs hold .cpp already linked into build/lib/*.a and
85+
# nothing on the consumer include path references them.
86+
rm -rf $(find . -maxdepth 1 ! -name "build" ! -name "llvm" ! -name "clang" ! -name "." ! -name "native_build")
87+
keep=('!' -name include '!' -name cmake '!' -name .)
8688
if [[ "${cling_on}" == "ON" ]]; then
87-
keep=('!' -name include '!' -name lib '!' -name cmake '!' -name utils '!' -name .)
89+
keep=('!' -name include '!' -name cmake '!' -name utils '!' -name .)
8890
fi
8991
for d in llvm clang; do
9092
(cd "$d" && rm -rf $(find . -maxdepth 1 "${keep[@]}"))
@@ -176,9 +178,9 @@ runs:
176178
# Trim source trees so the cache stays small. Cling rows additionally
177179
# need llvm/{utils} for runtime tablegen lookups.
178180
rm -r -force $(find.exe . -maxdepth 1 ! -name "build" ! -name "llvm" ! -name "clang" ! -name "." ! -name "native_build")
179-
$keep = @("!", "-name", "include", "!", "-name", "lib", "!", "-name", "cmake", "!", "-name", ".")
181+
$keep = @("!", "-name", "include", "!", "-name", "cmake", "!", "-name", ".")
180182
if ( $cling_on ) {
181-
$keep = @("!", "-name", "include", "!", "-name", "lib", "!", "-name", "cmake", "!", "-name", "utils", "!", "-name", ".")
183+
$keep = @("!", "-name", "include", "!", "-name", "cmake", "!", "-name", "utils", "!", "-name", ".")
182184
}
183185
foreach ($d in @("llvm", "clang")) {
184186
Push-Location $d

‎.github/actions/Miscellaneous/Save_PR_Info/action.yml‎

Lines changed: 72 additions & 58 deletions
Original file line numberDiff line numberDiff line change
@@ -1,58 +1,72 @@
1-
name: 'Save PR Info'
2-
description: 'This PR saves the PR Info for the job'
3-
4-
runs:
5-
using: composite
6-
steps:
7-
8-
- name: Save PR Info on Unix Systems
9-
if: ${{ runner.os != 'Windows' }}
10-
shell: bash
11-
run: |
12-
mkdir -p ./pr
13-
echo ${{ github.event.number }} > ./pr/NR
14-
echo ${{ github.repository }} > ./pr/REPO
15-
16-
cling_on=$(echo "${{ matrix.cling }}" | tr '[:lower:]' '[:upper:]')
17-
if [[ "$cling_on" == "ON" ]]; then
18-
export CLING_HASH=$(git ls-remote https://github.com/root-project/cling.git refs/tags/v${{ matrix.cling-version || env.CLING_VERSION }} | tr '\t' '-')
19-
export LLVM_HASH=$(git ls-remote https://github.com/root-project/llvm-project.git cling-llvm${{ matrix.clang-runtime}} | tr '\t' '-')
20-
else
21-
export CLING_HASH="Repl"
22-
# May need to revert back to both having same llvm_hash, as below cause llvm to be rebuilt everytime commit is made to llvm/llvm-project for release a.x
23-
# which could be quite often for new releases
24-
export LLVM_HASH=$(git ls-remote https://github.com/llvm/llvm-project.git refs/heads/release/${{ matrix.clang-runtime}}.x | tr '\t' '-')
25-
fi
26-
27-
echo "CLING_HASH=$CLING_HASH" >> $GITHUB_ENV
28-
echo "LLVM_HASH=$LLVM_HASH" >> $GITHUB_ENV
29-
30-
- name: Save PR Info on Windows systems
31-
if: runner.os == 'Windows'
32-
shell: powershell
33-
run: |
34-
#can be found
35-
mkdir ./pr
36-
echo "${{ github.event.number }}" > ./pr/NR
37-
echo ${{ github.repository }} > ./pr/REPO
38-
39-
if ( "${{ matrix.cling }}" -imatch "On" )
40-
{
41-
$env:CLING_HASH_TEMP = ( git ls-remote https://github.com/root-project/cling.git refs/tags/v${{ matrix.cling-version || env.CLING_VERSION }} )
42-
$env:CLING_HASH = $env:CLING_HASH_TEMP -replace "\t","-"
43-
}
44-
else
45-
{
46-
$env:CLING_HASH="Repl"
47-
# May need to revert back to both having same llvm_hash, as below cause llvm to be rebuilt everytime commit is made to llvm/llvm-project for release a.x
48-
# which could be quite often for new releases
49-
$env:LLVM_HASH_TEMP = (git ls-remote https://github.com/llvm/llvm-project.git refs/heads/release/${{ matrix.clang-runtime}}.x )
50-
$env:LLVM_HASH = $env:LLVM_HASH_TEMP -replace "\t","-"
51-
}
52-
53-
echo "CLING_HASH=$env:CLING_HASH"
54-
echo "LLVM_HASH=$env:LLVM_HASH"
55-
56-
echo "CLING_HASH=$env:CLING_HASH" >> $GITHUB_ENV
57-
echo "LLVM_HASH=$env:LLVM_HASH" >> $GITHUB_ENV
58-
1+
name: 'Save PR Info'
2+
description: 'This PR saves the PR Info for the job'
3+
4+
runs:
5+
using: composite
6+
steps:
7+
8+
- name: Save PR Info on Unix Systems
9+
if: ${{ runner.os != 'Windows' }}
10+
shell: bash
11+
run: |
12+
mkdir -p ./pr
13+
echo ${{ github.event.number }} > ./pr/NR
14+
echo ${{ github.repository }} > ./pr/REPO
15+
16+
# Build a readable, short cache-key tag from `git ls-remote`'s
17+
# SHA. Previously we piped the whole `<sha>\t<ref>` line through
18+
# `tr '\t' '-'`, which leaked the `refs/tags/...` git internals
19+
# into the cache key. Format: `<label>-<short-sha>` -- label first
20+
# so the key is greppable, 8-char SHA so it's still unique across
21+
# the few tags we care about without the 40-char eyesore.
22+
cling_on=$(echo "${{ matrix.cling }}" | tr '[:lower:]' '[:upper:]')
23+
if [[ "$cling_on" == "ON" ]]; then
24+
cling_version='${{ matrix.cling-version || env.CLING_VERSION }}'
25+
cling_sha=$(git ls-remote https://github.com/root-project/cling.git "refs/tags/v${cling_version}" | awk '{print $1}')
26+
export CLING_HASH="cling-v${cling_version}-${cling_sha:0:8}"
27+
llvm_sha=$(git ls-remote https://github.com/root-project/llvm-project.git "cling-llvm${{ matrix.clang-runtime }}" | awk '{print $1}')
28+
export LLVM_HASH="cling-llvm${{ matrix.clang-runtime }}-${llvm_sha:0:8}"
29+
else
30+
export CLING_HASH="Repl"
31+
# May need to revert back to both having same llvm_hash, as below cause llvm to be rebuilt everytime commit is made to llvm/llvm-project for release a.x
32+
# which could be quite often for new releases
33+
llvm_sha=$(git ls-remote https://github.com/llvm/llvm-project.git "refs/heads/release/${{ matrix.clang-runtime }}.x" | awk '{print $1}')
34+
export LLVM_HASH="llvm-release-${{ matrix.clang-runtime }}.x-${llvm_sha:0:8}"
35+
fi
36+
37+
echo "CLING_HASH=$CLING_HASH" >> $GITHUB_ENV
38+
echo "LLVM_HASH=$LLVM_HASH" >> $GITHUB_ENV
39+
40+
- name: Save PR Info on Windows systems
41+
if: runner.os == 'Windows'
42+
shell: powershell
43+
run: |
44+
#can be found
45+
mkdir ./pr
46+
echo "${{ github.event.number }}" > ./pr/NR
47+
echo ${{ github.repository }} > ./pr/REPO
48+
49+
# See bash block above for the format rationale.
50+
if ( "${{ matrix.cling }}" -imatch "On" )
51+
{
52+
$cling_version = "${{ matrix.cling-version || env.CLING_VERSION }}"
53+
$cling_sha = ( git ls-remote https://github.com/root-project/cling.git "refs/tags/v$cling_version" ).Split("`t")[0]
54+
$env:CLING_HASH = "cling-v$cling_version-$($cling_sha.Substring(0, 8))"
55+
$llvm_sha = ( git ls-remote https://github.com/root-project/llvm-project.git "cling-llvm${{ matrix.clang-runtime }}" ).Split("`t")[0]
56+
$env:LLVM_HASH = "cling-llvm${{ matrix.clang-runtime }}-$($llvm_sha.Substring(0, 8))"
57+
}
58+
else
59+
{
60+
$env:CLING_HASH="Repl"
61+
# May need to revert back to both having same llvm_hash, as below cause llvm to be rebuilt everytime commit is made to llvm/llvm-project for release a.x
62+
# which could be quite often for new releases
63+
$llvm_sha = ( git ls-remote https://github.com/llvm/llvm-project.git "refs/heads/release/${{ matrix.clang-runtime }}.x" ).Split("`t")[0]
64+
$env:LLVM_HASH = "llvm-release-${{ matrix.clang-runtime }}.x-$($llvm_sha.Substring(0, 8))"
65+
}
66+
67+
echo "CLING_HASH=$env:CLING_HASH"
68+
echo "LLVM_HASH=$env:LLVM_HASH"
69+
70+
echo "CLING_HASH=$env:CLING_HASH" >> $GITHUB_ENV
71+
echo "LLVM_HASH=$env:LLVM_HASH" >> $GITHUB_ENV
72+

‎.github/workflows/main.yml‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -90,7 +90,7 @@ jobs:
9090
- { name: ubu24-x86-clang22-llvm22-asan-ubsan, os: ubuntu-24.04, compiler: clang-22, clang-runtime: '22', sanitizer: "Address;Undefined" }
9191
- { name: ubu24-x86-gcc12-llvm21-cppyy-vg, os: ubuntu-24.04, compiler: gcc-12, clang-runtime: '21', cppyy: On, Valgrind: On }
9292
- { name: ubu24-x86-gcc12-llvm22-cppyy, os: ubuntu-24.04, compiler: gcc-12, clang-runtime: '22', cppyy: On }
93-
- { name: ubu24-x86-gcc14-cling-llvm20-cppyy, os: ubuntu-24.04, compiler: gcc-14, clang-runtime: '20', cling: On, cppyy: On, root-llvm-tag: &root_llvm_tag 'cling-llvm20-20260119-01' }
93+
- { name: ubu24-x86-gcc14-cling-llvm20-cppyy, os: ubuntu-24.04, compiler: gcc-14, clang-runtime: '20', cling: 'On', cppyy: On, root-llvm-tag: &root_llvm_tag 'cling-llvm20-20260119-01' }
9494
- name: selfh-ubu22-x86-gcc12-llvm22-cuda
9595
os: [self-hosted, cuda, heavy]
9696
compiler: gcc-12
@@ -107,7 +107,7 @@ jobs:
107107
# MacOS Arm
108108
- { name: osx26-arm-clang-llvm22, os: macos-26, compiler: clang, clang-runtime: '22', llvm_targets_to_build: host }
109109
- { name: osx26-arm-clang-llvm21-cppyy, os: macos-26, compiler: clang, clang-runtime: '21', cppyy: On, llvm_targets_to_build: host }
110-
- { name: osx26-arm-clang-cling-llvm20-cppyy, os: macos-26, compiler: clang, clang-runtime: '20', cling: On, cppyy: On, root-llvm-tag: *root_llvm_tag }
110+
- { name: osx26-arm-clang-cling-llvm20-cppyy, os: macos-26, compiler: clang, clang-runtime: '20', cling: 'On', cppyy: On, root-llvm-tag: *root_llvm_tag }
111111
- name: osx26-arm-clang-llvm20-oop
112112
os: macos-26
113113
compiler: clang
@@ -118,11 +118,11 @@ jobs:
118118
python-version: '3.13'
119119
# MacOS X86
120120
- { name: osx26-x86-clang-llvm21-cppyy, os: macos-26-intel, compiler: clang, clang-runtime: '21', cppyy: On, llvm_targets_to_build: host }
121-
- { name: osx26-x86-clang-cling-llvm20-cppyy, os: macos-26-intel, compiler: clang, clang-runtime: '20', cling: On, cppyy: On, root-llvm-tag: *root_llvm_tag }
121+
- { name: osx26-x86-clang-cling-llvm20-cppyy, os: macos-26-intel, compiler: clang, clang-runtime: '20', cling: 'On', cppyy: On, root-llvm-tag: *root_llvm_tag }
122122
# Windows
123123
- { name: win11-msvc-llvm22, os: windows-11-arm, compiler: msvc, clang-runtime: '22' }
124124
- { name: win2025-msvc-llvm22, os: windows-2025, compiler: msvc, clang-runtime: '22' }
125-
- { name: win2025-msvc-cling-llvm20, os: windows-2025, compiler: msvc, clang-runtime: '20', cling: On, root-llvm-tag: *root_llvm_tag }
125+
- { name: win2025-msvc-cling-llvm20, os: windows-2025, compiler: msvc, clang-runtime: '20', cling: 'On', root-llvm-tag: *root_llvm_tag }
126126

127127
steps:
128128
- uses: actions/checkout@v6

0 commit comments

Comments
 (0)