feat: add annotate-coverage action - #2026
Conversation
Adds a composite action that annotates uncovered lines in a PR diff using Go coverage data. Parses Go coverage files from a directory, intersects them with a git diff, and emits text, Markdown, or GitHub Actions workflow commands (::notice file=...,line=...::...) that GitHub renders as PR annotations on the changed lines. The action ships its own Go binary (cmd/annotate-coverage) and supports PR-diff, branch-vs-HEAD, single-commit, and working-tree diff modes. Wires the new action into release-please-config.json and the manifest with an initial version of 0.1.0. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The repo root .gitignore excludes any path named `coverage/`, which silently dropped the Go source files under internal/coverage/. Re-allow internal/coverage/ alongside testdata/coverage/ in the action's own .gitignore, and re-add the missing files. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR introduces a new composite GitHub Action (actions/annotate-coverage) backed by a Go CLI that parses Go coverage profiles, intersects them with a git diff, and emits results as text/Markdown or GitHub Actions annotation workflow commands. It also wires the action into release-please so it can be versioned and released like the other actions in this repo.
Changes:
- Added
actions/annotate-coveragecomposite action + README with usage and inputs. - Implemented Go CLI (
cmd/annotate-coverage) and supporting internal packages for diff parsing, coverage parsing/merging, coverage-vs-diff analysis, and formatting. - Updated release-please configuration/manifest to include
actions/annotate-coveragestarting atv0.1.0.
Reviewed changes
Copilot reviewed 32 out of 52 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| release-please-config.json | Adds release-please package entry for actions/annotate-coverage. |
| .release-please-manifest.json | Adds initial manifest version for actions/annotate-coverage. |
| actions/annotate-coverage/README.md | Documents action purpose, usage example, inputs, diff modes, and output formats. |
| actions/annotate-coverage/action.yaml | Defines composite action steps to build and run the Go binary with provided inputs. |
| actions/annotate-coverage/CHANGELOG.md | Initializes changelog for the new action. |
| actions/annotate-coverage/.gitignore | Ignores local build artifacts and coverage output while keeping fixtures tracked. |
| actions/annotate-coverage/go.mod | Declares the Go module and dependencies for the action implementation. |
| actions/annotate-coverage/go.sum | Locks Go dependency checksums for reproducible builds. |
| actions/annotate-coverage/cmd/annotate-coverage/main.go | Implements CLI entrypoint, flags, and diff-mode selection. |
| actions/annotate-coverage/internal/local/runner.go | Orchestrates diff acquisition, diff parsing, coverage loading/merging, analysis, and output. |
| actions/annotate-coverage/internal/local/runner_test.go | Adds tests around coverage loading/merging and runner behavior. |
| actions/annotate-coverage/internal/diff/diff.go | Defines the DiffSource abstraction. |
| actions/annotate-coverage/internal/diff/local.go | Implements working-tree diff acquisition (git add -N + git diff). |
| actions/annotate-coverage/internal/diff/local_test.go | Tests the local diff source using a temporary git repo. |
| actions/annotate-coverage/internal/diff/commit.go | Implements single-commit diff acquisition via git diff-tree. |
| actions/annotate-coverage/internal/diff/commit_test.go | Tests commit diff source behavior (valid/invalid SHAs, root commits). |
| actions/annotate-coverage/internal/diff/base.go | Implements base-vs-head diff acquisition via git diff base..commit. |
| actions/annotate-coverage/internal/diff/base_test.go | Tests base diff source behavior across SHAs and branch names. |
| actions/annotate-coverage/internal/coverage/parser.go | Parses Go coverage profiles (and coverage inside zip archives) into internal structures. |
| actions/annotate-coverage/internal/coverage/parser_test.go | Tests coverage parsing/validation and zip extraction behavior via fixtures. |
| actions/annotate-coverage/internal/coverage/merger.go | Merges multiple coverage profiles using a gocovmerge-style block merge. |
| actions/annotate-coverage/internal/coverage/merger_test.go | Tests merge behavior across modes, overlaps, and ordering. |
| actions/annotate-coverage/internal/coverage/diff.go | Parses unified diffs and extracts added line numbers per file (Go files only). |
| actions/annotate-coverage/internal/coverage/diff_test.go | Tests diff parsing across multiple scenarios (hunks, renames, deletions, binaries, etc.). |
| actions/annotate-coverage/internal/coverage/analysis.go | Computes uncovered added lines by intersecting coverage blocks with added diff lines; generates annotations. |
| actions/annotate-coverage/internal/coverage/analysis_test.go | Tests analysis logic, stats, comparisons, and annotation generation. |
| actions/annotate-coverage/internal/format/formatter.go | Factory for selecting output formatter by name. |
| actions/annotate-coverage/internal/format/text.go | Plain-text formatting of uncovered lines and summary coverage. |
| actions/annotate-coverage/internal/format/text_test.go | Tests text formatting and range compaction output. |
| actions/annotate-coverage/internal/format/markdown.go | Markdown table formatting of uncovered lines and summary coverage. |
| actions/annotate-coverage/internal/format/markdown_test.go | Tests Markdown formatting output. |
| actions/annotate-coverage/internal/format/github_annotations.go | Emits ::notice ...::... workflow commands for PR annotations. |
| actions/annotate-coverage/internal/format/github_annotations_test.go | Tests annotation command output formatting. |
| actions/annotate-coverage/internal/github/annotations.go | Sorts and groups line numbers into consecutive ranges. |
| actions/annotate-coverage/internal/github/annotations_test.go | Tests line grouping/sorting logic. |
| actions/annotate-coverage/testdata/diffs/simple_addition.diff | Diff fixture for simple additions. |
| actions/annotate-coverage/testdata/diffs/multiple_hunks.diff | Diff fixture for multiple hunks in one file. |
| actions/annotate-coverage/testdata/diffs/mixed_changes.diff | Diff fixture with additions and modifications. |
| actions/annotate-coverage/testdata/diffs/file_rename.diff | Diff fixture for renames. |
| actions/annotate-coverage/testdata/diffs/file_deletion.diff | Diff fixture for deletions. |
| actions/annotate-coverage/testdata/diffs/binary_file.diff | Diff fixture for binary changes. |
| actions/annotate-coverage/testdata/coverage/valid_single.out | Coverage fixture with multiple files/blocks. |
| actions/annotate-coverage/testdata/coverage/valid_count.out | Coverage fixture for count mode. |
| actions/annotate-coverage/testdata/coverage/valid_atomic.out | Coverage fixture for atomic mode. |
| actions/annotate-coverage/testdata/coverage/no_mode.out | Coverage fixture missing a mode header (parse failure). |
| actions/annotate-coverage/testdata/coverage/multiple_files_1.out | Coverage fixture used for merge tests (set mode). |
| actions/annotate-coverage/testdata/coverage/multiple_files_2.out | Additional coverage fixture used for merge tests. |
| actions/annotate-coverage/testdata/coverage/malformed.out | Malformed coverage fixture (parse failure). |
| actions/annotate-coverage/testdata/coverage/empty.out | Empty coverage fixture (parse failure). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- format/github_annotations.go: emit `::notice::<msg>` instead of
`::notice <msg>`; the workflow-command syntax for a notice without
metadata requires a double-colon separator.
- diff/local.go: capture stderr explicitly for `git add -N` (cmd.Run
doesn't populate ExitError.Stderr); use %w-wrapping on both git
failures so the underlying error is preserved.
- local/runner.go: correct Run's doc-comment — errors propagate to the
CLI as a non-zero exit code, not "always exits 0".
- local/runner.go: use filepath.Join + strings.HasSuffix to build coverage
file paths so the action also works on Windows runners.
- local/runner_test.go: probe `git rev-parse --is-inside-work-tree`
instead of `os.Stat("../.git")` so the integration test actually runs
from internal/local (the repo root is several levels up); supply
Format: "Text" so Runner.Run can construct a formatter.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Runs golangci-lint and `go test -race -short ./...` whenever anything under actions/annotate-coverage/ or the workflow itself changes, on push to main, pull requests, and merge_group. Mirrors the pattern used by test-techdocs-rewrite-relative-links. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The new test-annotate-coverage workflow exposed lint issues that were not flagged by `go vet`/`go test` alone: - errcheck: silence stdout-bound `fmt.Fprintln/Fprintf` writes in the text and markdown formatters with `_, _ =` (caller picks the writer and we use these for human-facing output). Propagate writer errors from the github-annotations formatter (production CI path) and from `SerializeProfiles`. Discard `rc.Close()` in the zip reader. - errcheck (tests): introduce `runGit(t, dir, args...)` in internal/diff/testutil_test.go and route the temp-repo setup calls in base_test.go, commit_test.go, and local_test.go through it. Combined output is surfaced in failures. - staticcheck SA4017: drop the empty `if !strings.HasPrefix(line, "---")` branch in `coverage.ParseDiff`; the `---` header was already a no-op for the '-' case. - staticcheck QF1003: convert merger_test.go's `if p.FileName == ...` chain to a tagged switch. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 34 out of 54 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (1)
actions/annotate-coverage/internal/local/runner.go:1
- This unconditional stdout logging will intermix with formatter output. In
GitHubAnnotationsmode, it will add extra non-annotation lines to stdout, which can be noisy and makes the output harder to consume in pipelines. Prefer writing progress logs to stderr, gating behind a verbose flag, or removing this line.
# Conflicts: # .release-please-manifest.json # release-please-config.json
ricky-undeadcoders
left a comment
There was a problem hiding this comment.
Nice work — clean structure and great test coverage. I ran it locally: go test -race -short ./... passes in all five packages, and the binary builds and runs fine. It follows the same pattern as go-flaky-tests, and building into RUNNER_TEMP keeps the user's checkout clean.
A couple of notes.
Should fix:
-
internal/coverage/analysis.go:38— matching a coverage file to a diff file by suffix is too loose. It can match the wrong file: a coverage profile for.../myhandler.gowould match a diff entry forhandler.go, becausemyhandler.goends withhandler.go. So coverage can get attributed to the wrong file.Line 41 already has the correct check (it requires a
/right before the name, sohandler.goonly matches a real path segment). But that line never runs — line 38 matches everything line 41 would, and returns first. So the safe check is effectively dead code.Fix: drop the loose check on line 38 and keep the boundary one, plus an exact match for files at the repo root:
if profileFile == diffFile || strings.HasSuffix(profileFile, "/"+diffFile) { return diffFile, addedLines, true }
One more thing: this loops over a map, and Go map order is random. If two diff files both match, which one wins is nondeterministic. The fix above makes matches stricter, which helps, but worth keeping in mind.
Optional: E2E CI Test
test-annotate-coverage.yaml only runs go test. Some of the other test-* workflows in this repo also run the action itself and check its output — see test-component-change-detection.yml, which does uses: ./actions/component-change-detection and then validates the outputs. Right now nothing checks the action wiring end to end (inputs → command-line flags → the ::notice output GitHub renders). Adding this is good practice to future proof against regressions, but also to demonstrate that it currently works end-to-end.
It's a little more setup here, since the action computes the diff from live git state rather than taking a diff as input — so a meaningful test needs a small git repo plus a coverage file to intersect. If that's too much for this PR, a screenshot of annotations on a real downstream PR would at least confirm GitHub renders the output.
The coverage-profile path was matched to a diff file with a plain suffix check, so a profile for ".../myhandler.go" would be attributed to a diff entry for "handler.go". Require a path-segment boundary (leading "/"), or an exact match for repo-root files. This also removes the boundary check that was previously dead code behind the looser one. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add an e2e job that runs annotate-coverage against this module's own coverage and asserts it annotates an intentionally-uncovered demo function. This exercises the full wiring (inputs -> flags -> git diff -> coverage intersection -> ::notice) end to end and serves as a live signal that the action still detects uncovered code. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Addressed the review comments, please take another look |
Summary
annotate-coveragethat highlights uncovered lines in a PR diff using Go coverage data. It parses Go coverage files from a directory, intersects them with a git diff, and emits text, Markdown, or GitHub Actions workflow commands (::notice file=...,line=...::...) that GitHub renders as PR annotations on the changed lines — no GitHub API client involved.cmd/annotate-coverage, with the existing canopy internal packages (coverage,diff,format,github,local) and their full test suites. Supports four diff modes: PR diff (base-ref+commit-sha), branch-vs-HEAD (base-ref), single-commit (commit-sha), and working-tree (no inputs).release-please-config.jsonand.release-please-manifest.jsonwith an initial version of0.1.0, following the same pattern asactions/go-flaky-tests.Test plan
go test -race -short ./...passes insideactions/annotate-coverage/(verified locally).go build ./cmd/annotate-coverageproduces a working binary (verified locally;annotate-coverage versionruns).prettier --checkpasses onaction.yaml,README.md, and the release-please configs (verified locally).::noticeannotations appear in the GitHub UI on the expected lines.🤖 Generated with Claude Code