Skip to content

Semantic review convergence: evidenced dispositions and visible pending review #3693

Description

@EffortlessSteven

Current synthesis

GitHub can enforce conversation resolution, but resolution alone does not prove that a finding was addressed. Review also needs a durable visible result when it is deliberately requested.

The original issue bound review convergence to the latest PR head and treated any later push as making prior review stale. PR #5784 and #5778 supersede that rule. In this squash-merge repository, review currentness is semantic and changed-seam based; the head SHA remains observation metadata and merge compare-and-swap protection.

Governing rule

zero unresolved substantive conversations
+ zero current change-request reviews
+ every resolved substantive finding has an evidence-backed disposition
+ deliberately requested review is no longer pending
+ provider facts are usable
= review converged

A changed SHA does not create work by itself.

Durable review record

Use native GitHub surfaces:

  • requested reviewer/team for visible pending judgment;
  • submitted review with useful scope, evidence/falsifiers, findings or explicit clean result, prior dispositions, what is established, residual risk, and next action;
  • inline review thread for one localized finding;
  • evidence-backed reply before resolution;
  • checks/rulesets for machine and merge evidence.

Do not use lifecycle labels, task lists, agent identities, review-run markers, claim digests, or exact-head comments as review authority.

Semantic currentness

Review conclusions attach to their semantic subjects:

  • material claim and governing authority;
  • production or consumer route;
  • proof stimulus and oracle;
  • changed implementation seam;
  • negative, fallback, stale, refusal, and error behavior;
  • compatibility, security, packaging, migration, support, and rollback boundaries.

Currentness rules:

unrelated main movement
→ no effect

behind-only branch movement
→ no effect

formatting/editorial/generated-evidence refresh
→ no broad re-review unless meaning changed

focused finding repair
→ recheck the finding, affected proof, and changed seam
→ inspect cumulative interaction proportionately

material premise/authority/production/risk/claim change
→ refresh affected broader conclusions

actual conflict or combined-tree repair
→ refresh conflict-affected proof and review

The observed head may be recorded for traceability. It is not the verdict.

Evidence-backed disposition

Before resolving a substantive finding, post:

Disposition: fixed | refuted | superseded | follow-up
Evidence: current source, focused test/oracle, governing authority, or linked issue

The checker distinguishes:

  • unresolved thread;
  • resolved after evidence-bearing response;
  • resolved silently or through bulk clearing;
  • editorial/automated informational thread;
  • outdated unresolved thread;
  • finding fixed on a later commit;
  • finding refuted against primary evidence;
  • accepted bounded follow-up;
  • provider/instrument failure.

Do not require one magic sentence when the disposition and evidence are otherwise machine-discoverable.

Result classes

CONVERGED
  native review requests, change requests, threads, and dispositions converge

PENDING
  PR is draft for an active reason or a deliberate review request remains

FINDINGS_OPEN
  change request, unresolved thread, or unsupported resolution remains

NOT_APPLICABLE
  live policy requires no review and none was submitted

NOT_PROVEN
  provider facts are partial, rate-limited, unavailable, malformed, or contradictory

STALE is reserved for evidence that is substantively stale—for example, the reviewed production route or claim was replaced—not merely a different SHA.

Structured output

Expose bounded facts to merge-readiness consumers:

{
  "schema": "review-convergence.v2",
  "observed_head": "...",
  "pending_reviewers": 0,
  "current_change_requests": 0,
  "unresolved_total": 0,
  "resolved_without_disposition": 0,
  "useful_submitted_reviews": 1,
  "material_dimensions_requiring_refresh": [],
  "result": "converged"
}

Legacy review-run/claim-receipt fields may remain diagnostic during migration but never feed result.

Implementation slices

R1 — native review/thread substrate

  • enumerate requested reviews, submitted reviews, and all threads including outdated;
  • expose provider failure as NOT_PROVEN;
  • retain the observed candidate head as metadata.

R2 — evidenced dispositions

  • require evidence for substantive fixed/refuted/superseded/follow-up outcomes;
  • expose silent resolution separately;
  • keep a light path for editorial/informational threads.

R3 — visible pending review

  • consume native review requests/submissions;
  • distinguish no-review-required from pending and unavailable review;
  • do not mirror private workflow liveness.

R4 — semantic changed-seam currentness

  • preserve useful review across unrelated movement;
  • refresh only affected findings/proof after focused repair;
  • refresh broader conclusions only after material semantic change;
  • cover reviewer-applied repairs without requiring identity theatre.

R5 — integration and measured promotion

  • emit one bounded result to merge readiness;
  • run advisory first;
  • measure false blocks, silent resolutions, unclassified threads, and NOT_PROVEN states;
  • promote only after every merge path consumes the same semantics.

Incident fixtures

  • resolved-to-clear bulk thread closure;
  • real P1 fixed and rechecked at the affected seam;
  • bot finding refuted against primary source;
  • outdated unresolved thread;
  • reviewer applies a focused fix while unaffected conclusions remain usable;
  • review deliberately requested and still pending;
  • H1 review followed by formatting-only H2;
  • H1 review followed by material production-path H2;
  • editorial typo thread;
  • GitHub API/rate-limit/instrument failure;
  • legacy review-run marker present but non-authoritative.

Acceptance

  • Resolved substantive thread without an evidence-bearing response blocks.
  • Supported fixed/refuted/superseded/follow-up disposition satisfies the thread rule.
  • Unresolved outdated thread remains visible.
  • Deliberately requested review remains pending.
  • Unrelated SHA movement does not invalidate useful review.
  • Focused repair refreshes only affected review dimensions.
  • Material semantic change refreshes affected broader conclusions.
  • Clean review is valid; no finding is manufactured.
  • Rate limit or instrument failure is NOT_PROVEN, not success.
  • Output feeds merge readiness without labels, runtime identity, claim digests, or review-run receipts.

Non-goals

  • No label-based review state machine.
  • No fixed reviewer persona or model sequence.
  • No requirement that reviewers be unable to edit.
  • No per-agent mutation ledger.
  • No merge operation in this checker.
  • No claim that every automated comment is substantive.
  • No exact-head review ceremony.

What this establishes

Whether useful semantic review state has converged for the current candidate, without turning ordinary commits into automatic review invalidation.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions