Skip to content

feat: add annotate-coverage action - #2026

Merged
ricky-undeadcoders merged 8 commits into
grafana:mainfrom
oleg-kozlyuk-grafana:feat/annotate-coverage
Jun 26, 2026
Merged

feat: add annotate-coverage action#2026
ricky-undeadcoders merged 8 commits into
grafana:mainfrom
oleg-kozlyuk-grafana:feat/annotate-coverage

Conversation

@oleg-kozlyuk-grafana

@oleg-kozlyuk-grafana oleg-kozlyuk-grafana commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds a new composite action annotate-coverage that highlights uncovered lines in a PR diff using Go coverage data. It parses Go coverage files from a directory, intersects them with a git diff, and emits text, Markdown, or GitHub Actions workflow commands (::notice file=...,line=...::...) that GitHub renders as PR annotations on the changed lines — no GitHub API client involved.
  • The utility is lifted verbatim from https://github.com/oleg-kozlyuk-grafana/go-canopy
  • Ships its own Go binary at cmd/annotate-coverage, with the existing canopy internal packages (coverage, diff, format, github, local) and their full test suites. Supports four diff modes: PR diff (base-ref + commit-sha), branch-vs-HEAD (base-ref), single-commit (commit-sha), and working-tree (no inputs).
  • Wires the new action into release-please-config.json and .release-please-manifest.json with an initial version of 0.1.0, following the same pattern as actions/go-flaky-tests.

Test plan

  • go test -race -short ./... passes inside actions/annotate-coverage/ (verified locally).
  • go build ./cmd/annotate-coverage produces a working binary (verified locally; annotate-coverage version runs).
  • prettier --check passes on action.yaml, README.md, and the release-please configs (verified locally).
  • Try the action end-to-end on a downstream PR to confirm ::notice annotations appear in the GitHub UI on the expected lines.

🤖 Generated with Claude Code

Adds a composite action that annotates uncovered lines in a PR diff using
Go coverage data. Parses Go coverage files from a directory, intersects
them with a git diff, and emits text, Markdown, or GitHub Actions workflow
commands (::notice file=...,line=...::...) that GitHub renders as PR
annotations on the changed lines.

The action ships its own Go binary (cmd/annotate-coverage) and supports
PR-diff, branch-vs-HEAD, single-commit, and working-tree diff modes.

Wires the new action into release-please-config.json and the manifest
with an initial version of 0.1.0.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The repo root .gitignore excludes any path named `coverage/`, which
silently dropped the Go source files under internal/coverage/. Re-allow
internal/coverage/ alongside testdata/coverage/ in the action's own
.gitignore, and re-add the missing files.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Comment thread release-please-config.json

@electron0zero electron0zero left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a new composite GitHub Action (actions/annotate-coverage) backed by a Go CLI that parses Go coverage profiles, intersects them with a git diff, and emits results as text/Markdown or GitHub Actions annotation workflow commands. It also wires the action into release-please so it can be versioned and released like the other actions in this repo.

Changes:

  • Added actions/annotate-coverage composite action + README with usage and inputs.
  • Implemented Go CLI (cmd/annotate-coverage) and supporting internal packages for diff parsing, coverage parsing/merging, coverage-vs-diff analysis, and formatting.
  • Updated release-please configuration/manifest to include actions/annotate-coverage starting at v0.1.0.

Reviewed changes

Copilot reviewed 32 out of 52 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
release-please-config.json Adds release-please package entry for actions/annotate-coverage.
.release-please-manifest.json Adds initial manifest version for actions/annotate-coverage.
actions/annotate-coverage/README.md Documents action purpose, usage example, inputs, diff modes, and output formats.
actions/annotate-coverage/action.yaml Defines composite action steps to build and run the Go binary with provided inputs.
actions/annotate-coverage/CHANGELOG.md Initializes changelog for the new action.
actions/annotate-coverage/.gitignore Ignores local build artifacts and coverage output while keeping fixtures tracked.
actions/annotate-coverage/go.mod Declares the Go module and dependencies for the action implementation.
actions/annotate-coverage/go.sum Locks Go dependency checksums for reproducible builds.
actions/annotate-coverage/cmd/annotate-coverage/main.go Implements CLI entrypoint, flags, and diff-mode selection.
actions/annotate-coverage/internal/local/runner.go Orchestrates diff acquisition, diff parsing, coverage loading/merging, analysis, and output.
actions/annotate-coverage/internal/local/runner_test.go Adds tests around coverage loading/merging and runner behavior.
actions/annotate-coverage/internal/diff/diff.go Defines the DiffSource abstraction.
actions/annotate-coverage/internal/diff/local.go Implements working-tree diff acquisition (git add -N + git diff).
actions/annotate-coverage/internal/diff/local_test.go Tests the local diff source using a temporary git repo.
actions/annotate-coverage/internal/diff/commit.go Implements single-commit diff acquisition via git diff-tree.
actions/annotate-coverage/internal/diff/commit_test.go Tests commit diff source behavior (valid/invalid SHAs, root commits).
actions/annotate-coverage/internal/diff/base.go Implements base-vs-head diff acquisition via git diff base..commit.
actions/annotate-coverage/internal/diff/base_test.go Tests base diff source behavior across SHAs and branch names.
actions/annotate-coverage/internal/coverage/parser.go Parses Go coverage profiles (and coverage inside zip archives) into internal structures.
actions/annotate-coverage/internal/coverage/parser_test.go Tests coverage parsing/validation and zip extraction behavior via fixtures.
actions/annotate-coverage/internal/coverage/merger.go Merges multiple coverage profiles using a gocovmerge-style block merge.
actions/annotate-coverage/internal/coverage/merger_test.go Tests merge behavior across modes, overlaps, and ordering.
actions/annotate-coverage/internal/coverage/diff.go Parses unified diffs and extracts added line numbers per file (Go files only).
actions/annotate-coverage/internal/coverage/diff_test.go Tests diff parsing across multiple scenarios (hunks, renames, deletions, binaries, etc.).
actions/annotate-coverage/internal/coverage/analysis.go Computes uncovered added lines by intersecting coverage blocks with added diff lines; generates annotations.
actions/annotate-coverage/internal/coverage/analysis_test.go Tests analysis logic, stats, comparisons, and annotation generation.
actions/annotate-coverage/internal/format/formatter.go Factory for selecting output formatter by name.
actions/annotate-coverage/internal/format/text.go Plain-text formatting of uncovered lines and summary coverage.
actions/annotate-coverage/internal/format/text_test.go Tests text formatting and range compaction output.
actions/annotate-coverage/internal/format/markdown.go Markdown table formatting of uncovered lines and summary coverage.
actions/annotate-coverage/internal/format/markdown_test.go Tests Markdown formatting output.
actions/annotate-coverage/internal/format/github_annotations.go Emits ::notice ...::... workflow commands for PR annotations.
actions/annotate-coverage/internal/format/github_annotations_test.go Tests annotation command output formatting.
actions/annotate-coverage/internal/github/annotations.go Sorts and groups line numbers into consecutive ranges.
actions/annotate-coverage/internal/github/annotations_test.go Tests line grouping/sorting logic.
actions/annotate-coverage/testdata/diffs/simple_addition.diff Diff fixture for simple additions.
actions/annotate-coverage/testdata/diffs/multiple_hunks.diff Diff fixture for multiple hunks in one file.
actions/annotate-coverage/testdata/diffs/mixed_changes.diff Diff fixture with additions and modifications.
actions/annotate-coverage/testdata/diffs/file_rename.diff Diff fixture for renames.
actions/annotate-coverage/testdata/diffs/file_deletion.diff Diff fixture for deletions.
actions/annotate-coverage/testdata/diffs/binary_file.diff Diff fixture for binary changes.
actions/annotate-coverage/testdata/coverage/valid_single.out Coverage fixture with multiple files/blocks.
actions/annotate-coverage/testdata/coverage/valid_count.out Coverage fixture for count mode.
actions/annotate-coverage/testdata/coverage/valid_atomic.out Coverage fixture for atomic mode.
actions/annotate-coverage/testdata/coverage/no_mode.out Coverage fixture missing a mode header (parse failure).
actions/annotate-coverage/testdata/coverage/multiple_files_1.out Coverage fixture used for merge tests (set mode).
actions/annotate-coverage/testdata/coverage/multiple_files_2.out Additional coverage fixture used for merge tests.
actions/annotate-coverage/testdata/coverage/malformed.out Malformed coverage fixture (parse failure).
actions/annotate-coverage/testdata/coverage/empty.out Empty coverage fixture (parse failure).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread actions/annotate-coverage/internal/format/github_annotations.go Outdated
Comment thread actions/annotate-coverage/internal/diff/local.go
Comment thread actions/annotate-coverage/internal/local/runner.go Outdated
Comment thread actions/annotate-coverage/internal/local/runner.go Outdated
Comment thread actions/annotate-coverage/internal/local/runner_test.go
oleg-kozlyuk-grafana and others added 3 commits June 5, 2026 10:14
- format/github_annotations.go: emit `::notice::<msg>` instead of
  `::notice <msg>`; the workflow-command syntax for a notice without
  metadata requires a double-colon separator.
- diff/local.go: capture stderr explicitly for `git add -N` (cmd.Run
  doesn't populate ExitError.Stderr); use %w-wrapping on both git
  failures so the underlying error is preserved.
- local/runner.go: correct Run's doc-comment — errors propagate to the
  CLI as a non-zero exit code, not "always exits 0".
- local/runner.go: use filepath.Join + strings.HasSuffix to build coverage
  file paths so the action also works on Windows runners.
- local/runner_test.go: probe `git rev-parse --is-inside-work-tree`
  instead of `os.Stat("../.git")` so the integration test actually runs
  from internal/local (the repo root is several levels up); supply
  Format: "Text" so Runner.Run can construct a formatter.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Runs golangci-lint and `go test -race -short ./...` whenever anything
under actions/annotate-coverage/ or the workflow itself changes, on
push to main, pull requests, and merge_group. Mirrors the pattern used
by test-techdocs-rewrite-relative-links.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The new test-annotate-coverage workflow exposed lint issues that were
not flagged by `go vet`/`go test` alone:

- errcheck: silence stdout-bound `fmt.Fprintln/Fprintf` writes in the
  text and markdown formatters with `_, _ =` (caller picks the writer
  and we use these for human-facing output). Propagate writer errors
  from the github-annotations formatter (production CI path) and from
  `SerializeProfiles`. Discard `rc.Close()` in the zip reader.
- errcheck (tests): introduce `runGit(t, dir, args...)` in
  internal/diff/testutil_test.go and route the temp-repo setup calls in
  base_test.go, commit_test.go, and local_test.go through it. Combined
  output is surfaced in failures.
- staticcheck SA4017: drop the empty `if !strings.HasPrefix(line, "---")`
  branch in `coverage.ParseDiff`; the `---` header was already a no-op
  for the '-' case.
- staticcheck QF1003: convert merger_test.go's `if p.FileName == ...`
  chain to a tagged switch.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 34 out of 54 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

actions/annotate-coverage/internal/local/runner.go:1

  • This unconditional stdout logging will intermix with formatter output. In GitHubAnnotations mode, it will add extra non-annotation lines to stdout, which can be noisy and makes the output harder to consume in pipelines. Prefer writing progress logs to stderr, gating behind a verbose flag, or removing this line.

# Conflicts:
#	.release-please-manifest.json
#	release-please-config.json

@ricky-undeadcoders ricky-undeadcoders left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work — clean structure and great test coverage. I ran it locally: go test -race -short ./... passes in all five packages, and the binary builds and runs fine. It follows the same pattern as go-flaky-tests, and building into RUNNER_TEMP keeps the user's checkout clean.

A couple of notes.

Should fix:

  • internal/coverage/analysis.go:38 — matching a coverage file to a diff file by suffix is too loose. It can match the wrong file: a coverage profile for .../myhandler.go would match a diff entry for handler.go, because myhandler.go ends with handler.go. So coverage can get attributed to the wrong file.

    Line 41 already has the correct check (it requires a / right before the name, so handler.go only matches a real path segment). But that line never runs — line 38 matches everything line 41 would, and returns first. So the safe check is effectively dead code.

    Fix: drop the loose check on line 38 and keep the boundary one, plus an exact match for files at the repo root:

    if profileFile == diffFile || strings.HasSuffix(profileFile, "/"+diffFile) {
        return diffFile, addedLines, true
    }

    One more thing: this loops over a map, and Go map order is random. If two diff files both match, which one wins is nondeterministic. The fix above makes matches stricter, which helps, but worth keeping in mind.

Optional: E2E CI Test

test-annotate-coverage.yaml only runs go test. Some of the other test-* workflows in this repo also run the action itself and check its output — see test-component-change-detection.yml, which does uses: ./actions/component-change-detection and then validates the outputs. Right now nothing checks the action wiring end to end (inputs → command-line flags → the ::notice output GitHub renders). Adding this is good practice to future proof against regressions, but also to demonstrate that it currently works end-to-end.

It's a little more setup here, since the action computes the diff from live git state rather than taking a diff as input — so a meaningful test needs a small git repo plus a coverage file to intersect. If that's too much for this PR, a screenshot of annotations on a real downstream PR would at least confirm GitHub renders the output.

oleg-kozlyuk-grafana and others added 2 commits June 22, 2026 11:17
The coverage-profile path was matched to a diff file with a plain suffix
check, so a profile for ".../myhandler.go" would be attributed to a diff
entry for "handler.go". Require a path-segment boundary (leading "/"), or
an exact match for repo-root files. This also removes the boundary check
that was previously dead code behind the looser one.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add an e2e job that runs annotate-coverage against this module's own
coverage and asserts it annotates an intentionally-uncovered demo
function. This exercises the full wiring (inputs -> flags -> git diff ->
coverage intersection -> ::notice) end to end and serves as a live signal
that the action still detects uncovered code.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@oleg-kozlyuk-grafana

Copy link
Copy Markdown
Contributor Author

Addressed the review comments, please take another look

@ricky-undeadcoders
ricky-undeadcoders added this pull request to the merge queue Jun 26, 2026
Merged via the queue into grafana:main with commit a56a230 Jun 26, 2026
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants