Skip to content

toolchain: allow to configure build_tar target - #1937

Merged
gravypod merged 3 commits into
bazelbuild:masterfrom
dataoleg:build-tar-target
Dec 6, 2021
Merged

toolchain: allow to configure build_tar target#1937
gravypod merged 3 commits into
bazelbuild:masterfrom
dataoleg:build-tar-target

Conversation

@dataoleg

@dataoleg dataoleg commented Oct 8, 2021

Copy link
Copy Markdown
Contributor

PR Checklist

Please check if your PR fulfills the following requirements:

  • Tests for the changes have been added (for bug fixes / features)
  • Docs have been added / updated (for bug fixes / features)

PR Type

What kind of change does this PR introduce?

  • Bugfix
  • Feature
  • Code style update (formatting, local variables)
  • Refactoring (no functional changes, no api changes)
  • Build related changes
  • CI related changes
  • Documentation content changes
  • Other... Please describe:

What is the current behavior?

Current behavior of build_tar tool used by container_layer is not hermetic, because it implicitly depends on the python toolchain, and as demonstrated by #1680, this can lead to non-reproducible builds.

What is the new behavior?

Similar to #1881, this PR adds an optional attribute for toolchain configuration, allowing to pass a hermetic build_tar_target executable.

Does this PR introduce a breaking change?

  • Yes
  • No

dataoleg and others added 3 commits October 4, 2021 14:55
python/cpython#18080 hasn't been back-ported to python3.8, which happens to be
the default interpreter version in a lot of popular base docker images.

This change in python's tarfile implementation affects the behavior of
https://github.com/bazelbuild/rules_docker/blob/bdea23404dd256c7ae2f8e02d346428044cc7d91/container/archive.py#L144
used by a popular container_image rule. This means that images built by bazel
in environments with different python3 versions will result in different image
digests.

Until the hermeticity of python toolchains is guaranteed, it may be useful to
have a workaround for users who can provide their own build_tar tool, akin to
https://github.com/kubernetes/repo-infra/blob/72a6f5d05659f7d255b51f152e0725adfe970718/tools/build_tar/buildtar.go
@google-cla

google-cla Bot commented Oct 8, 2021

Copy link
Copy Markdown

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.


What to do if you already signed the CLA

Individual signers
Corporate signers

ℹ️ Googlers: Go here for more info.

@google-cla google-cla Bot added the cla: no label Oct 8, 2021
@dataoleg

dataoleg commented Oct 8, 2021

Copy link
Copy Markdown
Contributor Author

@googlebot I signed it!

@google-cla google-cla Bot added cla: yes and removed cla: no labels Oct 8, 2021
@dataoleg
dataoleg marked this pull request as ready for review October 8, 2021 14:40

@gravypod gravypod left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing this! Is this part of a longer series of changes you are working on? I was hitting some performance problems with build_tar.py (and wanted to remove our python dep). If you're interested in helping to rewrite this tool let me know!

@gravypod
gravypod merged commit 86c54e5 into bazelbuild:master Dec 6, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants