Experimental: migrate cloudbuild to buildkite

[pcj]

Remove travis and cloudbuild, update buildkite.

I started this PR intending to replace cloudbuild with github actions. However, as I got into it I realized there was already some overlap/duplication between the cloudbuild coverage and buildkite coverage. Given we want to migrate everything to BK eventually anyway, I skipped the intermediate step.

NOTE: the cloudbuild account currently in use for rules_docker also hosts google internal jobs and logs cannot be shared. As a result, they are not suitable for community usage.

[google-cla]

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

[google-cla]

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

[pcj]

@googlebot I fixed it.

[pcj]

Note: turning on --remote in github action caused:

ERROR: Failed to init auth credentials: The Application Default Credentials are not available. They are available if running in Google Compute Engine. Otherwise, the environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.

Need to resolve this (add a secret) to use remote execution in this context.

[pcj]

List of failing tests on github action:

//tests/docker/package_managers:install_pkgs_reproducibility_test

Does not necessarily imply the test is bad, just that it is not passing in github action

Planning to exclude these

[pcj]

Issue migrating //run_instruction_arbitrary:all to buildkite:

(06:25:53) ERROR: /workdir/testing/examples/run_instruction_arbitrary/BUILD:65:16: //run_instruction_arbitrary:bazel_gcloud_dockerfile_wrapper depends on @bazel_gcloud_dockerfile//image:dockerfile_image.tar in repository @bazel_gcloud_dockerfile which failed to fetch. no such package '@bazel_gcloud_dockerfile//image': docker build command failed: The command '/bin/sh -c wget https://bootstrap.pypa.io/3.5/get-pip.py && python3 get-pip.py && python3 -m pip install --upgrade setuptools wheel' returned a non-zero code: 1

[pcj]

OK, pausing work on this for now. Here's where it stands: I migrated a bunch of stuff off cloudbuild to buildkite without too much difficulty. The remaining ones have special requirements and/or I don't know what they are supposed to do:

1. tests/contrib/cloudbuild.yaml: this pulls in gcr.io/asci-toolchain/container_release_tools/dependency_update/validators/semantics:latest to help test //tests/contrib/automatic_container_release:all. I need more context on this.
2. tests/docker/security/cloudbuild.yaml requires gcr.io/asci-toolchain/nosla-ubuntu16_04-bazel-docker-gcloud to test //tests/docker/security/.... Again, need more context here.
3. container/go/cloudbuild.yaml builds the puller for multiple platforms and uploadsthe artifacts. This is fairly straightforward to run in BK, but are these artifacts used by downstream clients somewhere? Need more info about this.
4. testing/examples/run_instruction_apt_pkgs/cloudbuild.yaml is somewhat more complex with a custom bazel output_base. It looks a bit more daunting so I skipped it for the moment.
5. testing/examples/run_instruction_apt_pkgs/cloudbuild.yaml failed on buildkite probably because the BK image is missing more or more tools in /bin/sh -c wget https://bootstrap.pypa.io/3.5/get-pip.py && python3 get-pip.py && python3 -m pip install --upgrade setuptools wheel' returned a non-zero code: 1.

Also, at this intermediate stage the presubmit is only testing on ubuntu. Need to restore the macos coverage.

[pcj]

@smukherj1 @gravypod do you want to sync on this sometime later in the week?

[pcj]

cc @alexeagle

[alexeagle]

Looks like we should discuss at next maintainer meeting? Do you want a review on this now?

[pcj]

Resolving some of those unknowns I enumerated above best during a meeting. Any other PR comments certainly welcome.

[pcj]

The ubuntu keyserver gpg key sha256 changed. Here is the value I am currently seeing:

-----BEGIN PGP PUBLIC KEY BLOCK-----
Comment: Hostname: 
Version: Hockeypuck ~unreleased

xo0ES8OmlQEEALy8ttT3KgmjoqCkeU7S04/j615RDivV0CHv+5mdJFQY10wo6v6k
rmXxK757eIoRDN1B3ztl8vjqLHxi8oA+f4inZcrrIZYjW0MghO+IcKEbKCrPUPjD
fQBSIvZD5ZaiibMNwZJ6TidS2r6nSAH+aoMRgXYycQPGkAHDcaPKY54zABEBAAHN
JExhdW5jaHBhZCBPcGVuSkRLIGJ1aWxkcyAoYWxsIGFyY2hzKcK2BBMBAgAgBQJL
w6aVAhsDBgsJCAcDAgQVAggDBBYCAwECHgECF4AACgkQ65sdiIb0TiraSgQAr+WD
+qGcW5yrkS2QBMq+g+Ew2tLqeDLw8Y2xZ4BhnRzB3WljBfwj7wt/KOugjJWaEWuY
GiuLBf6vy+At5VtXs0FSk6oIZR0XZvUVamFPXeTkZqUiw4oTNlBTeOhAxrevRWzn
mLuRAIWlr0dARJyLCK5MMVVhesouoG8WpOrniXs=
=JbYw
-----END PGP PUBLIC KEY BLOCK-----

http_file(
    name = "launchpad_openjdk_gpg",
    sha256 = "e9a596d0c194a562be9fd2c2a0994d7885505a1145fed0fbd5ae4c11d56220a0",
    urls = ["https://keyserver.ubuntu.com/pks/lookup?op=get&fingerprint=on&search=0xEB9B1D8886F44E2A"],
)

[pcj]

Subsumed by #1859