Skip to content

ci: make typecheck-ratchet base collection robust (fix false positive)#49

Merged
hblanken merged 1 commit into
collabfrom
fix/typecheck-ratchet-base-collection
Jun 15, 2026
Merged

ci: make typecheck-ratchet base collection robust (fix false positive)#49
hblanken merged 1 commit into
collabfrom
fix/typecheck-ratchet-base-collection

Conversation

@hblanken

Copy link
Copy Markdown

Problem

The typecheck ratchet (#46) fails PRs that contain no TypeScript changes at all — e.g. the Dockerfile-only #48. Root cause is in the base-baseline collection, not the PRs:

  • It ran bun turbo typecheck inside a detached git worktree with symlinked node_modules. turbo/tsgo produce no task output in a linked worktree (it has a .git file, symlinked deps), so the base baseline came back empty.
  • collect-type-errors.sh's final grep … | sed | sort -u exits 1 when grep matches nothing — and as the script's last command that propagated out, failing the step under bash -e (contradicting the script's own "always exits 0" header contract).
  • Net effect: base baseline empty → every real head error (the fork's ~20 known baseline errors) looks "new" → false failure.

Evidence: on #48 the base step died in 1.5 s (far too fast for a real tsgo run), while the head step — same repo, same script, run in the normal checkout — passed.

Fix

  • Collect the base baseline in place: git checkout --detach $BASE_SHA in the same working tree (reusing head's installed node_modules) instead of a linked worktree, so turbo behaves identically for base and head. Head's collector script is copied to $RUNNER_TEMP before the checkout, so the up-to-date collector runs even when BASE predates this fix.
  • Honor the "always exits 0" contract in collect-type-errors.sh (|| true + explicit exit 0), and warn to stderr when the typecheck emitted no turbo task lines (i.e. didn't actually run) — so a real tooling failure is visible instead of silently producing an empty baseline.
  • Sanity gate in the compare step: base=0 && head>0 means base collection didn't run → fail with a clear diagnostic rather than misreporting N "new" errors.

Verification

  • bash -n clean; YAML parses (both jobs + all 6 ratchet steps intact).
  • Functional: running collect-type-errors.sh where the typecheck can't run (no bun) now exits 0 and prints the "type checker did NOT run" diagnostic — previously it would exit 1 and fail the step.
  • This PR only touches .github/, so head and base TypeScript are identical → expected ratchet result is 0 new errors (and a non-empty base, so the new guard stays quiet).

Follow-up

After merge, rebase #48 onto collab so it re-runs against the fixed gate (also picks up #47) → its typecheck goes green.

🤖 Generated with Claude Code

The base-baseline step ran `bun turbo typecheck` inside a detached `git
worktree` with symlinked node_modules. turbo/tsgo produced no task output in
that linked worktree (it has a `.git` *file*, not dir), so the base set came
back empty and the ratchet flagged every real head error as "new" — failing
Dockerfile-only / no-TS PRs (e.g. #48) with a misleading "introduces N new
errors". The base step itself also died because collect-type-errors.sh's final
grep exits 1 on zero matches.

- Collect the base baseline by checking BASE out IN PLACE in the same working
  tree (reusing head's node_modules) instead of a linked worktree, so turbo
  behaves identically for base and head. Head's collector is copied to a temp
  path before checkout so the up-to-date collector runs even when BASE predates
  this fix.
- collect-type-errors.sh: honor its documented "always exits 0" contract (the
  final grep exited 1 on no matches and, as the last command, failed the step
  under `bash -e`). Warn to stderr when the typecheck emitted no turbo task
  lines (i.e. it did not actually run).
- Sanity gate: base=0 signatures while head>0 means base collection did not run
  — fail with a clear diagnostic instead of misreporting N "new" errors.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@hblanken hblanken merged commit 7a35ee1 into collab Jun 15, 2026
2 of 3 checks passed
@hblanken hblanken deleted the fix/typecheck-ratchet-base-collection branch June 15, 2026 03:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant