chore(repo): add workflow to track beginning-contributor PRs needing CI approval - #38220
Conversation
|
👋 It looks like your PR description references an issue, but not in the expected location. The issue number must appear in the first section of the description (the first two lines), following the template format: Please move your issue reference to the top of the description. |
d88b654 to
d1bf27b
Compare
d1bf27b to
196541b
Compare
…I approval Add a daily scheduled workflow that monitors beginning-contributor PRs and categorizes them into two priority buckets: 1. **Pending Approval** — CI workflow is in `action_required` state, waiting for a maintainer to approve the run. 2. **Pending Build Result** — CI was approved but no build result has been reported back (stuck pipeline or infra issue). The workflow creates/updates a tracking issue with labels `automation`, `ci-pending`, `tracking` so maintainers have a single dashboard for the backlog. Closes aws#38196
196541b to
55ea47b
Compare
✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.
| steps: | ||
| - name: Check for PRs pending CI approval | ||
| id: check-prs | ||
| uses: actions/github-script@v7 |
There was a problem hiding this comment.
Can we use the latest actions/github-script@v9 here ?
| : `CI Pending Approval - No PRs`; | ||
|
|
||
| // Tracking issue number — update this after the issue is created | ||
| const TRACKING_ISSUE_NUMBER = 9000; |
There was a problem hiding this comment.
Will we create new single tracking issue ?
| const repo = 'aws-cdk'; | ||
|
|
||
| // Search for beginning-contributor PRs | ||
| const { data: searchResults } = await github.rest.search.issuesAndPullRequests({ |
There was a problem hiding this comment.
Do we need the Search API here or can we just use github.rest.issues.listForRepo?
Also, per_page: 100 on Search will only return the first page, so this will miss beginning-contributor PRs if they exceed 100.
|
|
||
| // Check if build has completed with a result | ||
| const buildRun = workflowRuns.workflow_runs.find(run => | ||
| run.name === 'PR Build' || run.name === 'Codebuild PR Build' |
There was a problem hiding this comment.
Can we avoid identifying the build workflow by run.name here since these are display name and can be renamed anytime. We can instead match on run.path which is more stable identifier.
| }); | ||
|
|
||
| if (status.state === 'pending' && status.total_count === 0 && !buildRun) { | ||
| // No CI activity at all — but also no action_required |
There was a problem hiding this comment.
This comment and logging below seems inaccurate since buildRun only matches runs named PR Build or Codebuild PR Build, it's undefined whenever the build workflow hasn't produced a run, even if the PR has other CI activity. For example PR #38083 has other CI activity but no build-named run.
| }); | ||
| console.log(`Updated tracking issue #${TRACKING_ISSUE_NUMBER}: ${pendingApproval.length} pending approval, ${pendingBuildResult.length} pending build result`); | ||
|
|
||
| console.log(`Result: ${pendingApproval.length} pending approval, ${pendingBuildResult.length} pending build result`); |
There was a problem hiding this comment.
This logging seems redundant.
| return { pendingApproval, pendingBuildResult }; | ||
|
|
||
| - name: Update tracking issue | ||
| uses: actions/github-script@v7 |
There was a problem hiding this comment.
Can we use the latest actions/github-script@v9 here ?
| const total = pendingApproval.length + pendingBuildResult.length; | ||
| const owner = 'aws'; | ||
| const repo = 'aws-cdk'; | ||
| const labels = ['automation', 'ci-pending', 'tracking']; |
There was a problem hiding this comment.
Why are we adding 3 new labels?
| with: | ||
| github-token: ${{ secrets.PROJEN_GITHUB_TOKEN }} | ||
| script: | | ||
| const results = ${{ steps.check-prs.outputs.result }}; |
There was a problem hiding this comment.
This interpolates the previous step's output directly into the source of this script. Can we instead pass the values via env to avoid any potential syntax errors:
| const results = ${{ steps.check-prs.outputs.result }}; | |
| env: | |
| CHECK_RESULT: ${{ steps.check-prs.outputs.result }} | |
| with: | |
| script: | | |
| const results = JSON.parse(process.env.CHECK_RESULT); | |
| issue_number: TRACKING_ISSUE_NUMBER, | ||
| title: issueTitle, | ||
| body: issueBody, | ||
| labels: labels |
There was a problem hiding this comment.
The issues.update call will replace existing labels on the issues based on API doc. Is this intended?
Addresses CodeQL js/incomplete-sanitization: a title containing '\|' was previously escaped to '\\|', which Markdown reads as a literal backslash followed by an unescaped cell delimiter, breaking the tracking issue table.
…check - Raise workflow-run fetch to per_page 100 (API max): a head SHA can accumulate 20+ runs, and an action_required run past the old 20-run cap would be silently missed, dropping a pending PR from the report - Use the Monday's calendar year in the week key so the date part always reads as a real date at year boundaries (2025-December29-W01, not 2026-December29-W01) - Bump actions/checkout to v7 to match repo convention - Refresh README wording for the week key format
|
@kumsmrit Thanks for the review! All of your comments are addressed in the current revision — the threads show as outdated because the workflow was restructured (the old
Additional hardening from self-review (c896fa4): workflow-run fetch raised to End-to-end PoC with live |
| jobs: | ||
| update_tracking_issue: | ||
| # scheduled runs should only happen in the upstream repo; manual dispatch is | ||
| # allowed anywhere (e.g. to validate changes in a fork) |
There was a problem hiding this comment.
Why is this workflow allowed to run in forks? If fork execution was only added for validation during development, can we please remove that and simplify this to upstream-only guard and avoid the secrets.GITHUB_TOKEN fallback?
There was a problem hiding this comment.
Good point — the fork execution was only there for validation during development. Removed in 70c30bf: the job now uses the plain github.repository == 'aws/aws-cdk' guard and PROJEN_GITHUB_TOKEN only (no GITHUB_TOKEN fallback), matching the other project-prioritization workflows. If anyone needs to validate future changes in a fork, defining PROJEN_GITHUB_TOKEN in the fork and temporarily adjusting the guard works, same as with the other scheduled automation.
Per review feedback: drop the fork workflow_dispatch carve-out (it only existed for validation during development) and the secrets.GITHUB_TOKEN fallback. The job now uses the same guard and single-token pattern as the other project-prioritization workflows. Fork validation remains possible by defining PROJEN_GITHUB_TOKEN in the fork, as sync-from-upstream.yml already expects.
| repo, | ||
| ref: pr.head.sha, | ||
| }); | ||
| if (status.state === 'pending' && status.total_count === 0) { |
There was a problem hiding this comment.
This no_ci_activity logic seems narrower than the first commit and could miss PRs that still have the required build check missing/expected (Example: #37984 has non-build workflow runs, but no PR Build / Codebuild PR Build run)
In aws/aws-cdk PRs, non-build pull_request_target workflows such as PR Linter, PR Prioritization, etc. usually run on the head SHA, so this check runs.length === 0 will make this branch rare to reach.
Should this be based on the absence of the build workflow check instead of absence of all workflow runs?
There was a problem hiding this comment.
This is a good catch — with pull_request_target workflows always running on the head SHA, runs.length === 0 made this branch nearly unreachable and PRs like #37984 fell through as ok. Fixed in db16761: the branch now triggers on !buildRun (absence of a build workflow run, matched by run.path), restoring the first revision's semantics on top of the stable-path matching. Kept the combined-status guard as a backstop in case CI ever reports via commit statuses outside Actions. Verified the detection against mocked scenarios: action_required → pending_approval, non-build runs only (the #37984 shape) → no_ci_activity, completed build run → ok, zero runs → no_ci_activity.
…w runs Per review feedback: pull_request_target workflows (PR Linter, prioritization, etc.) run on the head SHA without CI approval, so 'runs.length === 0' almost never held and PRs whose build workflow never started (e.g. aws#37984) were silently classified as ok. The branch now triggers on the absence of a build workflow run (matched by run.path), restoring the first revision's !buildRun semantics on top of the stable-path matching. The combined-status guard is kept as a backstop for CI reporting via commit statuses.
|
Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork). |
Merge Queue Status
This pull request spent 12 seconds in the queue, with no time running CI. ReasonThe pull request can't be updated
HintYou should update or rebase your pull request manually. If you do, this pull request will automatically be requeued once the queue conditions match again. Requeued — the merge queue status continues in this comment ↓. |
|
Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork). |
Merge Queue Status
This pull request spent 11 seconds in the queue, including 1 second running CI. Required conditions to merge
|
|
Comments on closed issues and PRs are hard for our team to see. |
Issue
Closes #38196
Reason for this change
There are 100+ open PRs from beginning contributors waiting for maintainer action. Many have been waiting weeks with no CI feedback. There is no dashboard or alert for this backlog.
Description of changes
Adds a daily scheduled GitHub Actions workflow (
pending-maintainer-action-check.yml) that:beginning-contributorlabel (paginated, so it scales past 100 PRs)action_required; a maintainer needs to click Approvepull_request_targetworkflows (PR Linter, prioritization, etc.) ranci-pending-trackingmarker label plus a week key in the title — the ISO week number combined with the date of the Monday that week starts on (e.g.CI pending maintainer action: week 2026-July13-W29) — no hardcoded issue number. First run of the week creates the issue; subsequent daily runs update its body in place.Update from the previous revision (per review feedback): the project-board approach has been reverted back to issue-based tracking, but with a weekly issue keyed by marker label + week key instead of a single hardcoded issue. The speculative
build_completed_no_statusdetection branch has been dropped — it never corresponded to an observed real scenario; we can revisit if stuck pipelines are actually seen.Update from the latest review round: the workflow is now upstream-only (plain
github.repository == 'aws/aws-cdk'guard,PROJEN_GITHUB_TOKENonly — noGITHUB_TOKENfallback), matching the other project-prioritization workflows; and the no-CI-activity check is scoped to the absence of a build workflow run rather than all runs, so PRs whose build never started aren't masked bypull_request_targetworkflow runs (e.g. #37984).Design notes:
2026-July13-W29) so maintainers can see at a glance which calendar week an issue covers without decoding ISO week numbersaws/aws-cdk(same guard and single-token pattern as the other project-prioritization workflows);owner/repostill come from the Actionscontextrather than being hardcodeddry_runinput on manual dispatch to log intended issue writes without performing them🔍 For reviewer: notes before merge
ci-pending-trackinglabel does not exist inaws/aws-cdkyet.issues.createwith a nonexistent label creates it automatically when the token has push access, but creating it up front (with a description/color) is cleaner.Description of how you validated changes
Validated end-to-end in the
pahud/aws-cdkfork before updating this PR. (An earlier revision allowedworkflow_dispatchin forks for this purpose; the final workflow is upstream-only per review.)PoC / live-data preview: pahud#21 shows the exact production output — the unmodified script run against real
aws/aws-cdkPR data (69 pending PRs detected as of the latest run), with only the issue write redirected to the fork. The issue title demonstrates the week-key format:CI pending maintainer action: week 2026-July13-W29.Fork validation details:
beginning-contributorwith no workflow runs and no commit statuses on its head commit (theno_ci_activitystate)PR #20: needs attention (no_ci_activity)) and created a tracking issue labeledci-pending-tracking, listing the PR with author and reasonci-pending-trackingissue after both runs (no duplicates)2026-07-13 → 2026-July13-W29,2026-07-19 → 2026-July13-W29(same week),2026-07-20 → 2026-July20-W30(Monday rollover), and year boundaries (2026-01-01 → 2025-December29-W01,2027-01-03 → 2026-December28-W53) — the year prefix uses the Monday's calendar year so the date part always reads as a real dateaction_requiredrun →pending_approval; non-build runs only (the fix(bedrock-agentcore-alpha): add tracing resource policy opt-out #37984 shape) →no_ci_activity; completed build run → ok; zero runs →no_ci_activityChecklist