Skip to content

Make the test fixture local and auditable - #241

Merged
lyrixx merged 4 commits into
mainfrom
tests
Sep 7, 2026
Merged

Make the test fixture local and auditable#241
lyrixx merged 4 commits into
mainfrom
tests

Conversation

@lyrixx

@lyrixx lyrixx commented Sep 7, 2026

Copy link
Copy Markdown
Member

Summary

  • Replace the network clone of https://github.com/gitonomy/foobar.git in AbstractTestCase with a local git bundle (tests/fixtures/foobar.bundle). Tests no longer need network access, no longer depend on an external repository outside our control, and a contributor can add a new fixture scenario without needing push access to a separate repo.
  • Add tests/fixtures/verify-bundle.sh, run in a new CI job on every push/PR, guarding the bundle against unreviewable changes: since it's a binary file, GitHub shows any PR touching it as an opaque diff. The script checks bundle integrity (git bundle verify), the ref list against an explicit allow-list, and a size ceiling (200KB, current size ~39KB).

Test plan

  • Full suite passes locally against the bundle: vendor/bin/phpunit (224 tests, 562 assertions)
  • Confirmed offline: same result with http_proxy/https_proxy forced to an unreachable address
  • tests/fixtures/verify-bundle.sh passes against the real bundle
  • Verified the guard actually rejects tampering: an added branch ref, and an oversized file, both correctly fail the script

The test suite cloned https://github.com/gitonomy/foobar.git over the
network on every run, and asserted on hardcoded commit SHAs from that
external repository's history. This made the tests fail without
network access, coupled them to a repository outside our control, and
made it impossible for a contributor to add a new fixture scenario
without push access to that separate repository.

tests/fixtures/foobar.bundle is a git bundle of the same repository
(all branches and tags, including the GPG-signed commit used by
PushReferenceTest). Admin::cloneTo/cloneBranchTo/cloneRepository just
shell out to `git clone <url> <path>`, so cloning from a local bundle
file behaves identically to cloning the remote repository, but works
offline and can't be affected by changes made upstream.
Since tests/fixtures/foobar.bundle is a binary file, GitHub shows a PR
touching it as an opaque "binary file not shown" diff: a contributor
could regenerate it with extra refs or bloated content while keeping
the commit SHAs the tests rely on intact, and a reviewer would have no
way to see it from the diff alone.

tests/fixtures/verify-bundle.sh runs `git bundle verify`, checks the
ref list against an explicit allow-list of the 5 branches and 2 tags
the fixture actually needs, and fails if the file grows past 200KB
(current size: ~39KB). Any legitimate change to the fixture's shape or
size requires editing this script in the same PR, making the change
explicit and reviewable instead of silent. Wired into CI as a new
verify-fixture job that runs on every push and pull request.
Explains what tests/fixtures/foobar.bundle is, how to regenerate it to
add a new fixture scenario, and why tests/fixtures/verify-bundle.sh
exists and needs updating alongside any intentional change to the
bundle's refs or size.
The docs and comments still pointed at cloning gitonomy/foobar over
the network to add a new fixture scenario, defeating the point of
moving to a local bundle. The bundle already contains the full history
(all branches and tags), so regenerating it only ever needs to clone
tests/fixtures/foobar.bundle itself — verified this reproduces a
byte-identical bundle with no network access.
@lyrixx
lyrixx merged commit 94cf24e into main Sep 7, 2026
6 checks passed
@lyrixx
lyrixx deleted the tests branch September 7, 2026 14:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant