Skip to content

chore(repo): add workflow to track beginning-contributor PRs needing CI approval - #38220

Merged
mergify[bot] merged 14 commits into
aws:mainfrom
pahud:feat/monitor-ci-pending-approval
Jul 20, 2026
Merged

chore(repo): add workflow to track beginning-contributor PRs needing CI approval#38220
mergify[bot] merged 14 commits into
aws:mainfrom
pahud:feat/monitor-ci-pending-approval

Conversation

@pahud

@pahud pahud commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Issue

Closes #38196

Reason for this change

There are 100+ open PRs from beginning contributors waiting for maintainer action. Many have been waiting weeks with no CI feedback. There is no dashboard or alert for this backlog.

Description of changes

Adds a daily scheduled GitHub Actions workflow (pending-maintainer-action-check.yml) that:

  1. Fetches open PRs with the beginning-contributor label (paginated, so it scales past 100 PRs)
  2. For each non-draft PR, checks whether it needs maintainer CI action:
    • Pending approval — a workflow run for the PR's head commit has conclusion action_required; a maintainer needs to click Approve
    • No build CI activity — no build workflow run (matched by workflow file path) and no commit statuses on the head commit; CI approval was skipped or never triggered, even if non-build pull_request_target workflows (PR Linter, prioritization, etc.) ran
  3. Creates (or updates) a weekly tracking issue listing all pending PRs. The issue is identified by the ci-pending-tracking marker label plus a week key in the title — the ISO week number combined with the date of the Monday that week starts on (e.g. CI pending maintainer action: week 2026-July13-W29) — no hardcoded issue number. First run of the week creates the issue; subsequent daily runs update its body in place.

Update from the previous revision (per review feedback): the project-board approach has been reverted back to issue-based tracking, but with a weekly issue keyed by marker label + week key instead of a single hardcoded issue. The speculative build_completed_no_status detection branch has been dropped — it never corresponded to an observed real scenario; we can revisit if stuck pipelines are actually seen.

Update from the latest review round: the workflow is now upstream-only (plain github.repository == 'aws/aws-cdk' guard, PROJEN_GITHUB_TOKEN only — no GITHUB_TOKEN fallback), matching the other project-prioritization workflows; and the no-CI-activity check is scoped to the absence of a build workflow run rather than all runs, so PRs whose build never started aren't masked by pull_request_target workflow runs (e.g. #37984).

Design notes:

  • The week key includes the week's start date (2026-July13-W29) so maintainers can see at a glance which calendar week an issue covers without decoding ISO week numbers
  • The issue body is a Markdown table (PR link, title, author, pending-since date) with a per-reason summary line. PR titles are sanitized before embedding in table cells (pipes escaped, whitespace collapsed) — addressing the title-sanitization concern from earlier review
  • Issues from previous weeks are left open for maintainers to review and close manually; each new week gets a fresh issue
  • If no PRs are pending and no issue exists for the current week, nothing is created (no noise)
  • The workflow runs only in aws/aws-cdk (same guard and single-token pattern as the other project-prioritization workflows); owner/repo still come from the Actions context rather than being hardcoded
  • Supports a dry_run input on manual dispatch to log intended issue writes without performing them
┌────────────────────────────────────────────────────────────────┐
│        pending-maintainer-action-check.yml                     │
│        Daily at 08:00 UTC or manual dispatch (dry_run opt-in) │
│        (upstream repo only)                                    │
└──────────────────────────┬─────────────────────────────────────┘
                           ▼
┌────────────────────────────────────────────────────────────────┐
│  List open 'beginning-contributor' PRs (paginated)             │
└──────────────────────────┬─────────────────────────────────────┘
                           ▼
               ┌───────────────────────┐
               │  For each PR:         │
               │  - skip drafts        │
               │  - get workflow runs  │
               │    for head SHA       │
               └───────────┬───────────┘
                           │
             ┌─────────────┴─────────────┐
             ▼                           ▼
   ┌──────────────────┐       ┌──────────────────────┐
   │ Any run has      │       │ No build workflow    │
   │ conclusion:      │       │ run AND zero commit  │
   │ action_required? │       │ statuses?            │
   └────────┬─────────┘       └──────────┬───────────┘
            │ pending_approval           │ no_ci_activity
            └─────────────┬──────────────┘
                          ▼
┌────────────────────────────────────────────────────────────────┐
│  Weekly tracking issue (label: ci-pending-tracking)            │
│                                                                │
│  "CI pending maintainer action: week 2026-July13-W29"          │
│  - exists for this week?  → update body in place               │
│  - doesn't exist?         → create it                          │
│  - nothing pending, none exists → do nothing                   │
└────────────────────────────────────────────────────────────────┘

🔍 For reviewer: notes before merge

  • The ci-pending-tracking label does not exist in aws/aws-cdk yet. issues.create with a nonexistent label creates it automatically when the token has push access, but creating it up front (with a description/color) is cleaner.
  • Previous weeks' issues are intentionally left open for a human to close after review. If you'd prefer the workflow to auto-close the prior week's issue when creating a new one, that's a small follow-up.

Description of how you validated changes

Validated end-to-end in the pahud/aws-cdk fork before updating this PR. (An earlier revision allowed workflow_dispatch in forks for this purpose; the final workflow is upstream-only per review.)

PoC / live-data preview: pahud#21 shows the exact production output — the unmodified script run against real aws/aws-cdk PR data (69 pending PRs detected as of the latest run), with only the issue write redirected to the fork. The issue title demonstrates the week-key format: CI pending maintainer action: week 2026-July13-W29.

Fork validation details:

  • Set up a test PR (pahud#20) labeled beginning-contributor with no workflow runs and no commit statuses on its head commit (the no_ci_activity state)
  • First dispatch: correctly detected the PR (PR #20: needs attention (no_ci_activity)) and created a tracking issue labeled ci-pending-tracking, listing the PR with author and reason
  • Second dispatch: found the existing week's issue via the marker label + week key and updated its body in place — exactly one open ci-pending-tracking issue after both runs (no duplicates)
  • Week key logic unit-checked locally, including boundary cases: 2026-07-13 → 2026-July13-W29, 2026-07-19 → 2026-July13-W29 (same week), 2026-07-20 → 2026-July20-W30 (Monday rollover), and year boundaries (2026-01-01 → 2025-December29-W01, 2027-01-03 → 2026-December28-W53) — the year prefix uses the Monday's calendar year so the date part always reads as a real date
  • The build-scoped detection (latest revision) verified against mocked workflow-run scenarios: action_required run → pending_approval; non-build runs only (the fix(bedrock-agentcore-alpha): add tracing resource policy opt-out #37984 shape) → no_ci_activity; completed build run → ok; zero runs → no_ci_activity

Checklist

@aws-cdk-automation
aws-cdk-automation requested a review from a team June 30, 2026 19:06
@github-actions github-actions Bot added the p1 label Jun 30, 2026
@mergify mergify Bot added the contribution/core This is a PR that came from AWS. label Jun 30, 2026
@mergify
mergify Bot temporarily deployed to automation June 30, 2026 19:07 Inactive
@mergify
mergify Bot temporarily deployed to automation June 30, 2026 19:07 Inactive
@github-actions

github-actions Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

👋 It looks like your PR description references an issue, but not in the expected location.

The issue number must appear in the first section of the description (the first two lines), following the template format:

### Issue # (if applicable)

Closes #123.

Please move your issue reference to the top of the description.

@aws-cdk-automation aws-cdk-automation left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(This review is outdated)

@pahud
pahud force-pushed the feat/monitor-ci-pending-approval branch from d88b654 to d1bf27b Compare June 30, 2026 19:10
@pahud
pahud force-pushed the feat/monitor-ci-pending-approval branch from d1bf27b to 196541b Compare June 30, 2026 19:12
…I approval

Add a daily scheduled workflow that monitors beginning-contributor PRs
and categorizes them into two priority buckets:

1. **Pending Approval** — CI workflow is in `action_required` state,
   waiting for a maintainer to approve the run.
2. **Pending Build Result** — CI was approved but no build result has
   been reported back (stuck pipeline or infra issue).

The workflow creates/updates a tracking issue with labels
`automation`, `ci-pending`, `tracking` so maintainers have a
single dashboard for the backlog.

Closes aws#38196
@pahud
pahud force-pushed the feat/monitor-ci-pending-approval branch from 196541b to 55ea47b Compare June 30, 2026 19:15
@pahud pahud changed the title feat(repo): add workflow to track beginning-contributor PRs needing CI approval chore(repo): add workflow to track beginning-contributor PRs needing CI approval Jun 30, 2026
@aws-cdk-automation
aws-cdk-automation dismissed their stale review June 30, 2026 19:18

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.

@aws-cdk-automation aws-cdk-automation added the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Jun 30, 2026
steps:
- name: Check for PRs pending CI approval
id: check-prs
uses: actions/github-script@v7

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use the latest actions/github-script@v9 here ?

: `CI Pending Approval - No PRs`;

// Tracking issue number — update this after the issue is created
const TRACKING_ISSUE_NUMBER = 9000;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will we create new single tracking issue ?

const repo = 'aws-cdk';

// Search for beginning-contributor PRs
const { data: searchResults } = await github.rest.search.issuesAndPullRequests({

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need the Search API here or can we just use github.rest.issues.listForRepo?
Also, per_page: 100 on Search will only return the first page, so this will miss beginning-contributor PRs if they exceed 100.


// Check if build has completed with a result
const buildRun = workflowRuns.workflow_runs.find(run =>
run.name === 'PR Build' || run.name === 'Codebuild PR Build'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we avoid identifying the build workflow by run.name here since these are display name and can be renamed anytime. We can instead match on run.path which is more stable identifier.

});

if (status.state === 'pending' && status.total_count === 0 && !buildRun) {
// No CI activity at all — but also no action_required

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment and logging below seems inaccurate since buildRun only matches runs named PR Build or Codebuild PR Build, it's undefined whenever the build workflow hasn't produced a run, even if the PR has other CI activity. For example PR #38083 has other CI activity but no build-named run.

});
console.log(`Updated tracking issue #${TRACKING_ISSUE_NUMBER}: ${pendingApproval.length} pending approval, ${pendingBuildResult.length} pending build result`);

console.log(`Result: ${pendingApproval.length} pending approval, ${pendingBuildResult.length} pending build result`);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This logging seems redundant.

return { pendingApproval, pendingBuildResult };

- name: Update tracking issue
uses: actions/github-script@v7

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use the latest actions/github-script@v9 here ?

const total = pendingApproval.length + pendingBuildResult.length;
const owner = 'aws';
const repo = 'aws-cdk';
const labels = ['automation', 'ci-pending', 'tracking'];

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we adding 3 new labels?

with:
github-token: ${{ secrets.PROJEN_GITHUB_TOKEN }}
script: |
const results = ${{ steps.check-prs.outputs.result }};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This interpolates the previous step's output directly into the source of this script. Can we instead pass the values via env to avoid any potential syntax errors:

Suggested change
const results = ${{ steps.check-prs.outputs.result }};
env:
CHECK_RESULT: ${{ steps.check-prs.outputs.result }}
with:
script: |
const results = JSON.parse(process.env.CHECK_RESULT);

issue_number: TRACKING_ISSUE_NUMBER,
title: issueTitle,
body: issueBody,
labels: labels

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issues.update call will replace existing labels on the issues based on API doc. Is this intended?

@aws-cdk-automation aws-cdk-automation removed the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Jul 9, 2026
Addresses CodeQL js/incomplete-sanitization: a title containing '\|' was
previously escaped to '\\|', which Markdown reads as a literal backslash
followed by an unescaped cell delimiter, breaking the tracking issue table.
…check

- Raise workflow-run fetch to per_page 100 (API max): a head SHA can
  accumulate 20+ runs, and an action_required run past the old 20-run cap
  would be silently missed, dropping a pending PR from the report
- Use the Monday's calendar year in the week key so the date part always
  reads as a real date at year boundaries (2025-December29-W01, not
  2026-December29-W01)
- Bump actions/checkout to v7 to match repo convention
- Refresh README wording for the week key format
@pahud

pahud commented Jul 17, 2026

Copy link
Copy Markdown
Contributor Author

@kumsmrit Thanks for the review! All of your comments are addressed in the current revision — the threads show as outdated because the workflow was restructured (the old monitor-ci-pending-approval.yml was replaced by pending-maintainer-action-check.yml + scripts/prioritization/pending-maintainer-action.js). Here's a map of each concern to its resolution:

# Concern Resolution Commit
1 Use latest actions/github-script@v9 Bumped to v9 c97eb5b
2 Will we create a new single tracking issue? Redesigned to a weekly tracking issue, keyed by the ci-pending-tracking marker label + week key in the title (e.g. 2026-July13-W29) — no hardcoded issue number; first run of the week creates it, later runs update in place 9fddf1e, 65bcffa
3 Search API vs issues.listForRepo; Search per_page: 100 only returns first page Switched to github.paginate(github.rest.issues.listForRepo, ...) — fully paginated, no Search API c97eb5b
4 Don't match build workflow by display run.name (renameable) Now matches on run.path against stable workflow file paths (CONFIG.buildWorkflowPaths) c97eb5b
5 Inaccurate comment/logging around buildRun Detection logic restructured; the misleading comment/log paths were removed in the rewrite 9fddf1e
6 "Have we seen any PRs for such scenario?" (speculative build_completed_no_status branch) Dropped — it never corresponded to an observed real scenario; can revisit if stuck pipelines are actually seen 9fddf1e
7 Escape/sanitize contributor-controlled fields in the Markdown table sanitizeTitle() collapses whitespace and escapes backslashes + pipes (backslashes first, so they can't neutralize the pipe escaping — also resolves the CodeQL js/incomplete-sanitization alert) c97eb5b, ea29659
8 Redundant logging Removed in the rewrite 9fddf1e
9 Why 3 new labels? Down to a single marker label (ci-pending-tracking); reasons are rendered as sections in the issue body instead 9fddf1e
10 Step output interpolated into script source Workflow is now a single step; the script receives everything via {github, context} and env (DRY_RUN) — no cross-step interpolation c97eb5b, 9fddf1e
11 issues.update replaces existing labels The update call now passes only body — labels are set once at creation and never touched on update 9fddf1e

Additional hardening from self-review (c896fa4): workflow-run fetch raised to per_page: 100 (a head SHA on this repo can accumulate 20+ runs; an action_required run past the old cap would have been silently missed), actions/checkout bumped to v7 to match repo convention, and the week key's year prefix now uses the Monday's calendar year so year-boundary titles read as real dates (2025-December29-W01).

End-to-end PoC with live aws/aws-cdk data: pahud#21 (only the issue write redirected to the fork). Would appreciate a re-review when you get a chance!

@pahud
pahud requested a review from kumsmrit July 17, 2026 16:42

@kumsmrit kumsmrit left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for addressing all the comments; I have added few follow-up comment.

jobs:
update_tracking_issue:
# scheduled runs should only happen in the upstream repo; manual dispatch is
# allowed anywhere (e.g. to validate changes in a fork)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this workflow allowed to run in forks? If fork execution was only added for validation during development, can we please remove that and simplify this to upstream-only guard and avoid the secrets.GITHUB_TOKEN fallback?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point — the fork execution was only there for validation during development. Removed in 70c30bf: the job now uses the plain github.repository == 'aws/aws-cdk' guard and PROJEN_GITHUB_TOKEN only (no GITHUB_TOKEN fallback), matching the other project-prioritization workflows. If anyone needs to validate future changes in a fork, defining PROJEN_GITHUB_TOKEN in the fork and temporarily adjusting the guard works, same as with the other scheduled automation.

Per review feedback: drop the fork workflow_dispatch carve-out (it only
existed for validation during development) and the secrets.GITHUB_TOKEN
fallback. The job now uses the same guard and single-token pattern as
the other project-prioritization workflows. Fork validation remains
possible by defining PROJEN_GITHUB_TOKEN in the fork, as
sync-from-upstream.yml already expects.
repo,
ref: pr.head.sha,
});
if (status.state === 'pending' && status.total_count === 0) {

@kumsmrit kumsmrit Jul 20, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This no_ci_activity logic seems narrower than the first commit and could miss PRs that still have the required build check missing/expected (Example: #37984 has non-build workflow runs, but no PR Build / Codebuild PR Build run)
In aws/aws-cdk PRs, non-build pull_request_target workflows such as PR Linter, PR Prioritization, etc. usually run on the head SHA, so this check runs.length === 0 will make this branch rare to reach.

Should this be based on the absence of the build workflow check instead of absence of all workflow runs?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good catch — with pull_request_target workflows always running on the head SHA, runs.length === 0 made this branch nearly unreachable and PRs like #37984 fell through as ok. Fixed in db16761: the branch now triggers on !buildRun (absence of a build workflow run, matched by run.path), restoring the first revision's semantics on top of the stable-path matching. Kept the combined-status guard as a backstop in case CI ever reports via commit statuses outside Actions. Verified the detection against mocked scenarios: action_required → pending_approval, non-build runs only (the #37984 shape) → no_ci_activity, completed build run → ok, zero runs → no_ci_activity.

…w runs

Per review feedback: pull_request_target workflows (PR Linter,
prioritization, etc.) run on the head SHA without CI approval, so
'runs.length === 0' almost never held and PRs whose build workflow never
started (e.g. aws#37984) were silently classified as ok. The branch now
triggers on the absence of a build workflow run (matched by run.path),
restoring the first revision's !buildRun semantics on top of the
stable-path matching. The combined-status guard is kept as a backstop
for CI reporting via commit statuses.
@mergify

mergify Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@mergify

mergify Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Merge Queue Status

  • Entered queue2026-07-20 15:09 UTC · Rule: default-squash · triggered by rule automatic merge
  • 🚫 Left the queue2026-07-20 15:09 UTC · at db16761ded5c51f2066ee6ba901601c920cd61c9

This pull request spent 12 seconds in the queue, with no time running CI.

Reason

The pull request can't be updated

For security reasons, Mergify can't update this pull request. Try updating locally.
GitHub response: refusing to allow a GitHub App to create or update workflow .github/workflows/analytics-metadata-updater.yml without workflows permission

Hint

You should update or rebase your pull request manually. If you do, this pull request will automatically be requeued once the queue conditions match again.
If you think this was a flaky issue, you can requeue the pull request, without updating it, by posting a @mergifyio queue comment.

Requeued — the merge queue status continues in this comment ↓.

@mergify

mergify Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@mergify

mergify Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Merge Queue Status

  • Entered queue2026-07-20 22:32 UTC · Rule: default-squash · triggered by rule automatic merge
  • Checks skipped · PR is already up-to-date
  • Merged2026-07-20 22:32 UTC · at c523b237db549e1ae124989fe674e9a6ed1485f9 · squash

This pull request spent 11 seconds in the queue, including 1 second running CI.

Required conditions to merge

@mergify
mergify Bot merged commit 031918c into aws:main Jul 20, 2026
18 of 19 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

Comments on closed issues and PRs are hard for our team to see.
If you need help, please open a new issue that references this one.

@github-actions github-actions Bot locked as resolved and limited conversation to collaborators Jul 20, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

contribution/core This is a PR that came from AWS. p1

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(repo): add workflow to track beginning-contributor PRs needing maintainer action

4 participants