Skip to content

chore(misc): calibrate PR review pipeline from maintainer review feedback - #36433

Merged
AgentEnder merged 4 commits into
masterfrom
chore/review-pipeline-calibration
Jul 21, 2026
Merged

chore(misc): calibrate PR review pipeline from maintainer review feedback#36433
AgentEnder merged 4 commits into
masterfrom
chore/review-pipeline-calibration

Conversation

@FrozenPandaz

Copy link
Copy Markdown
Contributor

Current Behavior

The in-repo PR review pipeline (.claude/skills/review-pr + analyzer agents) diverged from maintainer judgment on a recent batch: it verdicted lgtm on a security-hardening PR the maintainer then requested changes on. Structurally:

  • Debatable design decisions (fail-open vs fail-closed validation, normalize-then-compare, world-shared tmpdir opt-outs) are silently endorsed inside agent reasoning — invisible to the human who'd want to veto them.
  • Concrete nice-to-have suggestions are discarded by design, yet they make up most of what the maintainer actually posts in reviews.
  • The SHA-based dedup pins drafts produced by older, weaker pipeline generations forever.
  • Docs changes are checked for accuracy but not editorial direction (a page can accurately recommend a harmful practice and be praised for it).
  • The performance agent's cold-path rule exempts full-workspace iteration from any scale-testing consideration.

Expected Behavior

  • Review drafts include a ### Maintainer calls section — one line per endorsed debatable decision (choice / stricter alternative / trade-off) — and a capped, non-verdict-driving ### Suggestions tier.
  • The security analyzer runs a design pass on security-mechanism PRs: prefer fail-closed, exact comparison, permissions at creation time; sockets in world-shared temp dirs are a finding; deliberate laxer choices surface as maintainer calls.
  • The performance analyzer weighs a stress-test suggestion heavily (advisory, not mechanical) when changed code iterates the full project set — even on cold/error paths — asking when per-item cost is non-obvious and skipping when it's trivially constant.
  • A PIPELINE_VERSION constant in the dedup ages out drafts from older criteria generations instead of pinning them at an unchanged head SHA.
  • astro-docs diffs get an editorial-direction check, not just an accuracy check.

Related Issue(s)

N/A — calibration derived from diffing the pipeline's drafts against the maintainer's manual reviews of the same PRs (#36370, #35863).


View session information ↗

…back

Derived from diffing the pipeline's drafts against the maintainer's manual
reviews of the same PRs (nx#36370, nx#35863):

- review-pr: add PIPELINE_VERSION to the dedup so drafts from older
  criteria generations re-review at an unchanged head SHA; keep a capped
  non-verdict-driving Suggestions tier instead of discarding it; require a
  Maintainer-calls section surfacing endorsed debatable design decisions;
  add an editorial-direction check for astro-docs changes
- security-analyzer: design pass for security-mechanism PRs (fail-closed
  over fail-open, exact comparison, perms at creation, throw on
  world-shared tmpdir); laxer deliberate choices become maintainer calls
- performance-analyzer: full-workspace iteration earns an advisory
  stress-test ask even on cold paths
@netlify

netlify Bot commented Jul 21, 2026

Copy link
Copy Markdown

Deploy Preview for nx-docs ready!

Name Link
🔨 Latest commit 566e79b
🔍 Latest deploy log https://app.netlify.com/projects/nx-docs/deploys/6a5fe8125713bf00080957f6
😎 Deploy Preview https://deploy-preview-36433--nx-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@netlify

netlify Bot commented Jul 21, 2026

Copy link
Copy Markdown

Deploy Preview for nx-dev ready!

Name Link
🔨 Latest commit 566e79b
🔍 Latest deploy log https://app.netlify.com/projects/nx-dev/deploys/6a5fe81221990400089430b8
😎 Deploy Preview https://deploy-preview-36433--nx-dev.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@nx-cloud

nx-cloud Bot commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

View your CI Pipeline Execution ↗ for commit da071d1

Command Status Duration Result
nx run-many -t check-imports check-lock-files c... ✅ Succeeded 3s View ↗
nx affected --targets=lint,test,build,e2e,e2e-c... ✅ Succeeded <1s View ↗
nx-cloud record -- pnpm nx-cloud conformance:check ✅ Succeeded 1m 2s View ↗
nx build workspace-plugin ✅ Succeeded <1s View ↗
nx-cloud record -- nx sync:check ✅ Succeeded 18s View ↗
nx-cloud record -- nx format:check ✅ Succeeded 5s View ↗

☁️ Nx Cloud last updated this comment at 2026-07-21 21:48:39 UTC

@FrozenPandaz
FrozenPandaz marked this pull request as ready for review July 21, 2026 21:48
@FrozenPandaz
FrozenPandaz requested a review from a team as a code owner July 21, 2026 21:48
@FrozenPandaz
FrozenPandaz requested a review from MaxKless July 21, 2026 21:48
@AgentEnder
AgentEnder merged commit 3db1c91 into master Jul 21, 2026
18 checks passed
@AgentEnder
AgentEnder deleted the chore/review-pipeline-calibration branch July 21, 2026 22:21
FrozenPandaz added a commit that referenced this pull request Jul 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants