Skip to content

Remove non-deterministic files from OUT_DIR to fix TreeArtifact cache misses#3973

Merged
UebelAndre merged 2 commits intobazelbuild:mainfrom
jsharpe:push-rnwossulpwxl
Apr 22, 2026
Merged

Remove non-deterministic files from OUT_DIR to fix TreeArtifact cache misses#3973
UebelAndre merged 2 commits intobazelbuild:mainfrom
jsharpe:push-rnwossulpwxl

Conversation

@jsharpe
Copy link
Copy Markdown
Contributor

@jsharpe jsharpe commented Apr 16, 2026

Recursively remove files from OUT_DIR whose names appear in a known list (config.log, config.log.old, config.status, Makefile, Makefile.config, config.cache, commit_hash) or have a .d extension before Bazel captures OUT_DIR as a TreeArtifact.

These files embed sandbox-specific paths, timestamps, or volatile values that make the TreeArtifact hash non-deterministic, causing cache misses for all downstream rustc compilations on every action run.

Copy link
Copy Markdown
Collaborator

@UebelAndre UebelAndre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank! Can you add a test for this somewhere?

@jsharpe jsharpe force-pushed the push-rnwossulpwxl branch 2 times, most recently from 2bced6c to 94a2144 Compare April 16, 2026 23:58
Comment thread cargo/tests/cargo_build_script/nondeterministic_out_dir/test.rs
Comment thread cargo/private/cargo_build_script_runner/bin.rs Outdated
@jsharpe jsharpe force-pushed the push-rnwossulpwxl branch 7 times, most recently from d893275 to 5ac1e80 Compare April 22, 2026 15:16
Comment thread cargo/settings/settings.bzl Outdated
Comment thread cargo/tests/cargo_build_script/nondeterministic_out_dir/test.rs Outdated
@jsharpe jsharpe force-pushed the push-rnwossulpwxl branch from 5dc032c to df32184 Compare April 22, 2026 15:36
jsharpe and others added 2 commits April 22, 2026 17:21
… misses

Recursively remove files from OUT_DIR whose names appear in the
`//cargo/settings:out_dir_volatile_file_basenames` string_list_flag, or have
a .d or .pc extension, before Bazel captures OUT_DIR as a TreeArtifact.

The flag value (default: config.log, config.log.old, config.status, Makefile,
Makefile.config, config.cache, commit_hash, jemalloc-config, jemalloc.sh) is
passed to the runner via the `RULES_RUST_OUT_DIR_VOLATILE_BASENAMES` env var.
The runner reads the env var at runtime; if absent, no named-file removal
occurs (extension-based removal of .d and .pc files is unconditional).

These files embed sandbox-specific paths, timestamps, or volatile values
that make the TreeArtifact hash non-deterministic, causing cache misses for
all downstream rustc compilations on every action run.

An integration test verifies both that legitimate outputs survive and that
each category of volatile file is absent from the captured TreeArtifact.
Co-authored-by: UebelAndre <github@uebelandre.com>
@jsharpe jsharpe force-pushed the push-rnwossulpwxl branch from df32184 to 044c39f Compare April 22, 2026 16:21
Copy link
Copy Markdown
Collaborator

@UebelAndre UebelAndre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@UebelAndre UebelAndre enabled auto-merge April 22, 2026 16:33
@UebelAndre UebelAndre added this pull request to the merge queue Apr 22, 2026
Merged via the queue into bazelbuild:main with commit 6d85959 Apr 22, 2026
3 checks passed
@jsharpe jsharpe deleted the push-rnwossulpwxl branch April 22, 2026 21:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants