Skip to content

Commit 079fe86

Browse files
ci: install protobuf well-known types in the harness builder (#237)
## Problem The [Nightly E2E Harness](https://github.com/lance-format/lance-context/actions/workflows/nightly-harness.yml) has failed on **every** scheduled run since it landed in #226 — 15 consecutive runs, 2026-08-01 through 2026-08-15 — always at the `Bring up the test stack` step. Most recent failure: [run 31870495892](https://github.com/lance-format/lance-context/actions/runs/31870495892). ## Root cause Debian's `protobuf-compiler` package ships only the `protoc` binary. The well-known types that the `lance-*` protos import (`google/protobuf/empty.proto` and friends) live in `libprotobuf-dev`, which `protobuf-compiler` merely **Recommends** — so `--no-install-recommends` in the builder stage drops it. Straight from the failing build's apt step (`#23`): ``` The following additional packages will be installed: libprotobuf32 libprotoc32 Recommended packages: libprotobuf-dev The following NEW packages will be installed: libprotobuf32 libprotoc32 protobuf-compiler ``` `libprotobuf-dev` is listed as Recommended and is absent from the install set. The build then gets ~22 minutes into the release compile before dying (`#26`): ``` error: failed to run custom build command for `lance-encoding v7.0.0` Error: Custom { kind: Other, error: "protoc failed: google/protobuf/empty.proto: File not found. encodings_v2_0.proto:8:1: Import \"google/protobuf/empty.proto\" was not found or had errors. encodings_v2_0.proto:343:5: \"google.protobuf.Empty\" is not defined." } ``` ## Why the cargo jobs stayed green This Dockerfile was the only build surface with neither source of the includes: | Surface | protoc source | Gets well-known types? | | --- | --- | --- | | `rust-test.yml` | `apt install protobuf-compiler` (no `--no-install-recommends`) | Yes, via the Recommends | | `python-test.yml` | vendors upstream protoc 25.6, `cp -R .../include/* /usr/local/include/` | Yes, explicitly | | `test/Dockerfile` | `apt install --no-install-recommends protobuf-compiler` | **No** | `nightly-harness.yml` has no `pull_request` trigger, so the compose path was never exercised before merge — this is a latent defect from #226, not a regression from a later change. ## Fix Add `libprotobuf-dev` to the builder stage, with a comment explaining why it can't be pruned as redundant later. ## Verification The root cause is confirmed directly from the failing run's logs (both the apt Recommends line and the `protoc` error above), and reproduced locally: protoc 3.21.12 without `libprotobuf-dev` fails identically on the lance proto sources, and installing `libprotobuf-dev` makes the same compile pass. **Not yet verified end-to-end:** the full release build inside Docker. Recommend a `workflow_dispatch` run of the nightly once this merges to confirm the stack comes all the way up, rather than waiting for the 06:37 UTC cron. ## Possible follow-ups (not in this PR) - `rust-test.yml` currently gets `libprotobuf-dev` only as a side effect of apt's Recommends. Listing it explicitly would make that surface robust against the same class of breakage. - `python-test.yml` deliberately vendors protoc 25.6 rather than trusting bookworm's 3.21.12. If a crate later needs the newer protoc, mirroring that vendoring in the builder stage is the more durable fix than the distro package. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
1 parent 79c1f08 commit 079fe86

1 file changed

Lines changed: 6 additions & 1 deletion

File tree

test/Dockerfile

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,8 +29,13 @@ RUN npm run build
2929
FROM rust:1-bookworm AS builder
3030
# Native deps mirror what CI installs: protoc for the lance protobufs, openssl
3131
# for etcd TLS + object-store, make/perl for the openssl-sys vendored build.
32+
# libprotobuf-dev is not optional here: Debian's protobuf-compiler ships only
33+
# the protoc binary, and the well-known types (google/protobuf/empty.proto and
34+
# friends, which the lance-* protos import) live in libprotobuf-dev. It is a
35+
# Recommends, so --no-install-recommends drops it and lance-encoding's build
36+
# script fails with "google/protobuf/empty.proto: File not found".
3237
RUN apt-get update && apt-get install -y --no-install-recommends \
33-
protobuf-compiler libssl-dev pkg-config make perl ca-certificates \
38+
protobuf-compiler libprotobuf-dev libssl-dev pkg-config make perl ca-certificates \
3439
&& rm -rf /var/lib/apt/lists/*
3540
WORKDIR /build
3641
COPY . .

0 commit comments

Comments
 (0)