fix(reward,extradup): resolve identity every call, and stop selling a bound nobody checked

[abrichr]

Two shipped defects, both found by adversarial review. Every reproduction below was run against published openadapt-evals 0.97.0 with openadapt-types 0.17.0 in a clean venv, and every one of them succeeded.

1. A certificate was trusted on its shape

DevelopmentSigner.issue_certificate took issuer and calibration_scope as plain parameters, and the types validator refused exactly one pair, self_signed + production. So:

cert = signer.issue_certificate(CONTRACT, ..., issuer="organization", calibration_scope="production")
receipt = signer.issue_receipt(contract=CONTRACT, ..., certificate=cert)
# certified: True   calibration_scope: production   production_certified: True

No worker, no oracle, no read. Both parameters are gone. This signer holds a key derived from a seed and nobody verifies that key, so its certificate can honestly claim one thing: someone computed a bound on a synthetic corpus. It always mints synthetic and self_signed.

Second reproduction, same run: RewardCertificateV1.satisfies had one caller in the workspace and it was a test. A certificate whose measured epsilon was 0.248885, against a contract whose certificate_policy demanded 0.05, produced certified anyway. A certificate issued for a completely different contract did too.

issue_receipt now certifies only a certificate that names this contract by digest and clears its certificate_policy. It raises on a certificate for another contract, because that is a wiring bug and not a weak bound.

assess_receipt takes an optional certificate_policy. Give it one and a certificate weaker than the contract asked for stops counting as certified, with both bounds logged side by side. CertifiedRewardFunction and the verl manager pass it through, and the proof harness supplies contract.certificate_policy. A trainer handed only a contract digest has nothing to compare against, so omitting it leaves the receipt's own flag standing.

The pin

openadapt-types moves to >=0.17.0,<0.18.0, the bound openadapt-flow already uses. It is 0-based semver with major_on_zero = false, so a breaking change arrives as a minor bump and an unbounded floor takes it silently. openadapt-types#40 narrows these contracts and changes score() to require the reward contract; widening this bound and updating assess_receipt belong in one PR after that release, which is why nothing here depends on it.

2. The identity check was guarded away

openadapt_evals/extradup/checkers.py, sor_check:

if added and not records_under(identity_of(spec), added):

Identity was resolved only when nothing at all had landed under the contract identity. Set expected_new=2, add one correct row and one on the decoy patient, and sor_check returns PASS while identity_check returns FAIL and names the decoy. Both gold specs use expected_new=1, where a right count forces a right subject, so no shipped fixture could reach it. That is the same fixture-shaped blindness that hid the wrong-record defect this guard was added to fix.

The resolution now runs on every call and decides the verdict. The reason strings do not move for the frozen families: the identity sentence is left out when every added row is already under the contract identity, because the cardinality line then reports the same number about the same rows. That condition governs wording only and can never suppress a FAIL, and test_the_wording_rule_never_decides_the_verdict enumerates every arrangement of up to three rows to keep it that way.

What a reviewer should check

The hub environment is unchanged. openadapt-mockmed-extradup is live and pinned to openadapt-evals>=0.97.0. Its self-test, its pytest file, and check_fails_closed.py --num-examples 2 all ran here against this branch: the same eight cases, the same per-case rewards, the same reasons, and the same 95% Clopper-Pearson upper bound of 0.004270473020189125. Nothing needs pushing to the hub.

The M-freeze is amended, not quietly rehashed. checkers.py and proof.py are pinned by hash in M_FREEZE_CERTIFIED_REWARD_RL_PILOT_2026_09_02.json. Both hashes move, both are repinned, and amendment_note records what changed and why, in the same form the 2026-09-02 amendment used. proof_2026-09-01.json and proof_2026-09-02.json are byte-identical, OPERATORS, MUTANTS, the seed schedule and the certificate digest are untouched, and the freeze still reports no result.

Nine new tests, eight of which fail on 0.97.0. tests/test_reward_certificate_narrowing.py and two additions to tests/test_extradup_kit.py. The ninth is a positive assertion that the scope stays synthetic, which held before and still holds.

Opened by an agent session, not the founder.

🤖 Generated with Claude Code