Skip to content

workflows/incomplete-prs: allow deleting PR template comments - #22290

Merged
MikeMcQuaid merged 2 commits into
Homebrew:mainfrom
gibfahn:incomplete_pr_description_check
May 22, 2026
Merged

workflows/incomplete-prs: allow deleting PR template comments#22290
MikeMcQuaid merged 2 commits into
Homebrew:mainfrom
gibfahn:incomplete_pr_description_check

Conversation

@gibfahn

@gibfahn gibfahn commented May 15, 2026

Copy link
Copy Markdown
Contributor

  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same change?
  • Have you added an explanation of what your changes do and why you'd like us to include them? Performance claims (e.g. "this is faster") must include Hyperfine benchmarks.
  • Have you written new tests (excluding integration tests) for your changes? Here's an example.
  • Have you successfully run brew lgtm (style, typechecking and tests) with your changes locally?

Unsure how to test this. Will the changed check run on this PR?


  • AI was used to generate or assist with generating this PR.

AI did the initial investigation, I made the changes.


The goal is to ensure people include the key lines from the template,
while not requiring them to exactly keep the text verbatim. In my
PR I accidentally changed too many lines, which caused my PR to
be auto-closed, despite me including all the checkboxes.

This makes the following changes:

  1. Don't consider --- markdown ruler lines as required
  2. Don't consider <!-- HTML comments --> as required
  3. Use <!-- HTML --> comments for the AI note as well as for the other
    ones.

Refs: #22249 (comment)

Full AI investigation:

The check works by normalizing lines (stripping, replacing any [ ]/[x]/[X] with [ ]) and checking that ≥75% of the template's unique non-empty lines appear in the PR body.

The template has 9 unique non-empty lines after normalization:

1. -----
2. <!-- Do not tick a checkbox if you haven't performed its action... -->
3. <!-- Use [x] to mark item done... -->
4–8. The five main checkboxes
4. - [ ] AI was used to generate or assist with generating this PR. *Please specify below how you used AI to help you, and what steps you have taken to manually verify the changes*.
Non-maintainers may only have one AI-assisted/generated PR open at a time.

Your PR body matches 6 of those 9:

- Misses the 2 HTML comments — you stripped them (as most authors do)
- Misses the AI checkbox line — you replaced the full template text with AI was used to generate or assist with generating this PR -> Did the design myself, used AI to write the Ruby., which
 no longer matches the template text

6/9 = 66.7%, which is below the 75% threshold.

If you'd only dropped the HTML comments (8/9 = 88.9%) or only modified the AI checkbox (8/9 = 88.9%), each would pass on its own. Both together pushes it under.

Fix: Keep the AI checkbox text verbatim as it appears in the template, and put your note on a separate line below it (which is exactly what "Please specify below" intends):

- [x] AI was used to generate or assist with generating this PR. *Please specify below how you used AI to help you, and what steps you have taken to manually verify the changes*.
Non-maintainers may only have one AI-assisted/generated PR open at a time.

@gibfahn
gibfahn force-pushed the incomplete_pr_description_check branch from 17b8b5f to 38dbbac Compare May 15, 2026 13:15
@gibfahn gibfahn changed the title workflows/incomplete-prs: improve matching checks workflows/incomplete-prs: allow deleting PR template comments May 15, 2026

@MikeMcQuaid MikeMcQuaid left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @gibfahn, a few comments but seems like a good idea.

Comment thread .github/workflows/incomplete-prs.yml Outdated
Comment thread .github/workflows/incomplete-prs.yml Outdated
Comment thread .github/PULL_REQUEST_TEMPLATE.md Outdated
gibfahn and others added 2 commits May 22, 2026 12:30
The goal is to ensure people include the key lines from the template,
while not requiring them to exactly keep the text verbatim. In my
PR I accidentally changed too many lines, which caused my PR to
be auto-closed, despite me including all the checkboxes.

This makes the following changes:

1. Don't consider `---` markdown ruler lines as required
2. Don't consider `<!-- HTML comments -->` as required
3. Use `<!-- HTML -->` comments for the AI note as well as for the other
   ones.

Refs: Homebrew#22249 (comment)
- Keep template-only scaffolding out of both sides of the match so
  untouched template comments are not treated as user content.
- Leave the AI details prompt in the editable body because contributors
  need that context when ticking the checkbox.
- Use named regexes for the ignored line types so the workflow intent is
  readable without extra explanation.
@MikeMcQuaid
MikeMcQuaid force-pushed the incomplete_pr_description_check branch from 06d4ded to 4902e57 Compare May 22, 2026 11:36

@MikeMcQuaid MikeMcQuaid left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@MikeMcQuaid
MikeMcQuaid enabled auto-merge May 22, 2026 11:38
@MikeMcQuaid
MikeMcQuaid added this pull request to the merge queue May 22, 2026
Merged via the queue into Homebrew:main with commit 09c6423 May 22, 2026
55 of 56 checks passed
@gibfahn
gibfahn deleted the incomplete_pr_description_check branch May 22, 2026 14:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants