Conversation
… oracle First code of the rewrite. A Lean 4 model with no mathlib dependency, pinned to v4.32.2, building a library and an executable that checks itself and exits nonzero on disagreement -- so CI verifies behaviour rather than only that the model compiles. 71 checks pass. Types are finite, with recursion through an explicit TypeTable. MiniCandid represents types as a CoInductive T; Lean 4 accepts `coinductive` only for predicates, and the infinite-tree representation cannot be executed anyway. The table is what the binary format does and what candid_types is specified to do with arena indices. Subtyping is written twice, deliberately. Candid/SubtypeSpec.lean holds the coinductive relation, mirroring spec/Candid.md rule for rule; Candid/Subtype.lean holds the decision procedure. The theorem connecting them is stated in prose and is the next slice's first obligation. Keeping them apart is what makes the model a specification and an oracle at once, rather than one or the other. Three things the spec turned out to say that the design had to follow: - The negative premises in the four `opt` rules are eliminable. Pairing each negative rule with its positive twin collapses them to `t <: opt t'` for every t and t'. This is not a shortcut -- a rule functional with negative premises is non-monotone and has no greatest fixed point, so the relation would not be coinductively definable at all. - `principal` is a <primtype> (spec:80), not a reference type. It lives in Prim, which also makes `principal <: principal` follow from prim reflexivity. - The type table may only hold composite types (spec:1227, "no <primtype>"), which is stronger than the no-bare-references invariant it replaces and rules out unbounded reference walks for a reason that is cited rather than derived. Subtyping relates two tables, not one. A TypeExpr holding a ref means nothing without its table, and the case that matters most compares a wire type table against the receiver's own type graph. Separating them forces the memo onto reference pairs -- bounded by |A|x|B|, where unfolded expressions are not -- and exposes that func contravariance swaps the tables along with the types. Deferred and named in lean/README.md rather than left implicit: `fuel` bounds recursion depth instead of a termination measure, and returns none rather than false when exhausted so the model never reports an answer it did not compute; decSubtype_iff; productive-recursion checking; Verso. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… disproved Three corrections that writing the first slice forced, none of which were visible from the plan alone. **The policy boundary was in the wrong place.** §5 said "only code churn lives on the branch" and listed directory reservations as policy, which made every README inside a reserved directory a master-PR item -- so recording a design decision cost a review round trip. A decision that has to wait for a policy review is a decision that gets made in someone's head and written down later, or not at all. The boundary is now location: everything under lean/, crates/ and conformance/ belongs to the working branch, including their README.md and CLAUDE.md; policy is what sits outside them. Location is decidable from a diff, where "is this policy?" is arguable. Two claims in §5 were also simply false once we started editing: "existing files are not modified" and a review criterion of "touches nothing existing". Both now name the real constraint -- nothing under rust/, spec/, test/, coq/ or tools/ is touched, and REWRITE.md is in scope because it is where the plan gets corrected. **§6's coq/ deletion condition was wrong on both halves.** It asked that Lean reproduce every MiniCandid theorem and that the two models be diffed, which treats them as the same kind of artifact differing in coverage. MiniCandid is a justification device: it shows that non-obvious design decisions are sound and that a proposed spec change can be accommodated. The Lean model answers what happens to a given input. So the condition is that each theorem's purpose is discharged, and there is no structural diff to perform once one side has finite types and an explicit table. The scoped nine-constructor comparison survives as a finding worth hunting, not a gate. **The layering diagram named a structure that does not exist.** candid_types listed TypeEnv, but rust/'s TypeEnv is a BTreeMap<String, Type> -- a name-keyed environment of .did declarations, not an index-keyed table, and both will exist. The diagram now names TypeExpr/TypeTable/TypeRef/FieldId/ClosedType, matching the Lean identifiers so that "candid_subtype reads as a transcription of its Lean counterpart" is checkable rather than aspirational, with a naming table in crates/README.md and the rule restated in crates/CLAUDE.md. TypeEnv stays reserved for candid_syntax, where "environment" is the accurate word. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…mise
The function rule's parameter premise is contravariant, so it swaps the two type
tables -- but it passed `seen` through unchanged. A memo of (A-index, B-index)
pairs was then read as (B-index, A-index), which asserts a transposed and
generally different question, since subtyping is not symmetric. The result was
`<:` reported for unrelated types.
The new `contra` checks in Main.lean are the witness: `record { f : func (vec nat)
-> () }` against `record { f : func (vec text) -> () }` reaches its parameter
premise with `(1, 2)` in the memo, reads it transposed, and answers `<:` where the
answer is `!<:`. Both checks fail without the one-line fix.
Also records a second gap, which this does not fix. `T.0 = vec (vec T.0)` makes
`ref 0` and `vec (ref 0)` the same infinite type, but every state on that cycle
has a reference on exactly one side, so the reference-pair memo never fires and
no budget suffices. That makes the completeness half of the `decSubtype_iff`
obligation false of the current procedure rather than merely unproved, so the
comments claiming a termination measure exists are corrected too.
The two `vecOmega` checks state the answer the model owes and are marked as known
gaps: reported, not build failures -- and a known gap that starts passing is
itself a failure, so the fix cannot land unnoticed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A table entry is now a `Composite`, its children are `Slot`s, and a `Slot` is a primitive or an index -- never an inline composite. That is the wire format's own shape rather than an invention of this model (`spec/Candid.md:1207`), and the spec draws the conclusion the change is built on: "Because recursion goes through `T`, this format by construction rules out non-well-founded definitions like `type t = t`." What it buys: - The subtype procedure can only recurse through a pair of *references*; every other slot pair is decided outright. So it carries `todo` -- the reference pairs it has not yet assumed -- descends by removing one, and `todo.length` is a termination measure. The membership test that decides the branch is exactly the fact the measure needs, so no invariant is threaded through the recursion. - `fuel`, `budgetFor`, `TypeExpr.depth` and the whole `Verdict` (`Option Bool`) layer are gone. `decSubtype` returns `Bool` and is total, and the public API has no "unanswered" state to explain. - The two `vecOmega` checks recorded as known gaps in the previous commit now pass. A cycle whose states hold a reference on alternating sides used to dodge the reference-pair memo entirely; flat, the inner composite has to be an entry of its own, so the same cycle passes through `(0, 1)` and `(1, 0)` and stops. - Both `mutual` blocks in the type language collapse: well-formedness is now a per-entry check over one flat node, and "the type table may only contain composite types" (`spec/Candid.md:1227`) holds by construction. That rule still has force at the decoder, which must reject a primitive opcode in an entry position. - `Subty` splits into two mutually coinductive predicates, `Subty` on slots and `SubtyC` on composites, mirroring the procedure. `decSubtype_iff` can now be stated with no side condition about a budget. What it costs: a type means nothing without its table, and even `vec nat` needs an entry, so hand-written types are built with `intern`/`close`. The `.did` surface syntax is nested, so the parser will need a flattening pass -- which is also what an encoder does when it emits a type table. Verified beyond the 79 checks by differencing against the previous model over 250,000 random pairs -- 200,000 flat (expressible in both representations, so the answers must agree exactly) and 50,000 nested and flattened. Zero mismatches in answers and in well-formedness. Also: `ClosedType.ofPrim` replaces `ofSlot`, so the helper cannot build a type whose root is a dangling reference; `simp_wf` is dropped from the termination proofs, unnecessary since Lean 4.12; and a dangling reference is no longer handed the top or bottom rule without being resolved first, which `<:` reported without looking would be the dangerous direction for a compatibility gate. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…n crates/ The module was named after a type it no longer defines. It holds `Slot`, `Composite`, `TypeTable` and `ClosedType`, so `Types.lean` is what it is. `crates/README.md` and `REWRITE.md` carried the same stale name in the planned `candid_types` layer, still describing `TypeExpr` as "one structural node; may contain references" -- the shape the model dropped. Identifiers are shared between `lean/` and `crates/` precisely so that "candid_subtype reads as a transcription of its Lean counterpart" stays checkable, so the naming table now records `Slot` and `Composite` with the reason neither takes the `Type` prefix: neither is a type, and both are named after the grammar position they occupy in the spec. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`decSubtype` was roughly cubic. The procedure carried `todo`, the reference pairs
not yet assumed, seeded with all `|A| x |B|` of them and narrowed by
`todo.erase (i, j)` at each descent -- and `List.erase` copies the list up to the
erased element. Compiled, on a table of `n` entries in one long cycle:
n=100 23 ms
n=300 370 ms
n=1000 14.3 s
n=2000 >100 s
A 1000-entry type table is 2-3 KB on the wire, so that is the same shape as a
cheap-bytes/expensive-work bug, in the artifact meant to become a differential
fuzzing oracle.
The complement was only ever there to be a termination measure. So the procedure now
carries `seen` -- the pairs this path *has* assumed, one cons per descent, membership
by a scan no longer than the path -- and the measure moves into `remaining`, which
counts the pairs `seen` does not record. `termination_by` measures are erased, so the
pair space is never built when the procedure runs. Same total process time on the
same tables:
n=1000 <10 ms n=10000 70 ms
n=50000 1.6 s n=400000 113 s, and no stack overflow at that depth
The two facts the measure needs are produced by the branch that makes the decision:
the guard says the pair is fresh, and the table lookups say its indices are in range.
So there is still no invariant threaded through the recursion. `remaining` counts
both orientations of the pair space, which is what makes the function rule's table
swap leave the measure alone -- proved as `remaining_swap` rather than assumed.
The one general lemma this needs is `countP_cons_lt`: a `countP` over a list drops
when the predicate flips to `false` on one member that is present, and nowhere gains.
That is the whole of the proof obligation; no pigeonhole argument and no mathlib.
Verified unchanged by differencing against the pre-rewrite model again: 50,000 random
flat pairs and 20,000 random nested pairs, zero mismatches, plus reflexivity, top,
bottom and transitivity over 20,000 random well-formed types.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
**Well-formedness.** The model accepted two kinds of type the spec forbids:
oneway with a result: wellFormed = true -- spec/Candid.md:211
service with a prim method: wellFormed = true -- spec/Candid.md:1223
`Composite.annotsOk` covers the first. The second needs the table, since a method's
type is a slot like any other, so it lives in `TypeTable.wellFormed` as
`methodsDenoteFuncs` -- the only well-formedness rule here that looks through the
table. Six checks pin both, including the cases that must stay well formed (`query`
with a result, `oneway` without one, a method that is a function).
One rule is left out on purpose, with the reason recorded: "The list of parameters
must be shorter than 2^32 values" (`spec/Candid.md:209`) cannot be violated by
anything that fits in memory. It is not idle, though -- `indexedFrom` labels
positional arguments with `UInt32`, which wraps, so that bound is what keeps a
function's argument labels distinct.
**Citations.** `spec/Candid.md:1207` is a bare code fence; the `I` block it introduces
starts at 1208. `spec/Candid.md:1221` is the note about multiple representations, not
the method-type rule, which is 1223. All ten citations in `lean/` were checked against
the spec; the rest were exact.
**Vocabulary.** "Slice" was used seven times and defined nowhere, and REWRITE.md does
not use the word at all. It now has a one-line definition where it first appears -- a
label for what landed in a merge window, applied after the fact -- and the two
forward-looking uses in the sources are gone, since the plan is the coverage checklist
and the tiers in REWRITE.md §3.
**`lake test`.** `testDriver = "oracle"`, so the model checking itself is reachable
the ordinary way. CI keeps running the binary explicitly so the check names show up in
the log.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
lean/: a Lean 4 model of Candid's type language and subtyping, as both arelation and an executable decision procedure, with a self-checking binary that CI
runs. Additive only — nothing on
masterreferences any of it.What is here
Candid/Types.leanSlot,Composite,TypeTable,ClosedType, well-formednessCandid/Hash.leanCandid/Subtype.leandecSubtype, the decision procedureCandid/SubtypeSpec.leanSubty/SubtyC, the same rules as a coinductive relationMain.leanoraclebinary: 85 checks, nonzero exit on any disagreement.github/workflows/lean.ymllake testruns the oracle.The three decisions that shaped it
The type table is the only recursion. A table entry is a
Composite, its childrenare
Slots, and aSlotis a primitive or an index — never an inline composite. Thatis the wire format's own shape (
spec/Candid.md:1208), and the spec draws theconclusion the model is built on: "Because recursion goes through
T, this format byconstruction rules out non-well-founded definitions like
type t = t."It pays for itself in the procedure. The only way to recurse is through a pair of
references, so
decSubtypecarriesseen— the pairs this path has assumed — anddescends by recording one. Termination is
remaining, the count of pairsseendoesnot record; it is named only by
termination_by, so the|A| × |B|pair space isnever built at run time. No fuel, no
Option Bool, no "unanswered" state in the API,and the two facts the measure needs are produced by the branch that makes the
decision. The one general lemma required is
countP_cons_lt; no mathlib.Subtyping relates two type tables, not one. A
Slotholding arefmeans nothingwithout its table, and the case that matters most compares a table that arrived on the
wire against the receiver's own type graph.
rust/candid/src/types/subtype.rs:19takesa single
envfor both, which works only because callers merge tables first.The spec's negative premises are eliminable. Two of the four
optrules carrynegative premises, and a non-monotone rule functional has no greatest fixed point — so
the relation would not be definable coinductively at all. Pairing each negative rule
with its positive counterpart collapses them:
t <: opt t'holds for everytandt', which is whatsubtype.rs:293implements as a warning-only catch-all.Two bugs the model found in itself
The memo did not swap with the tables. The function rule's parameter premise is
contravariant, so it swaps the two tables — but it passed
seenthrough unchanged, soa memo of (A-index, B-index) pairs was read as (B-index, A-index). Since subtyping is
not symmetric, that answered a transposed question from the memo and reported
<:forunrelated types. The
contrachecks are the witness and fail without the fix.decSubtypewas roughly cubic. An earlier iteration carried the complement ofthe memo, seeded with all
|A| × |B|pairs and narrowed withList.erase, whichcopies up to the erased element. Compiled, on a table of n entries in one cycle:
23 ms at n=100, 370 ms at n=300, 14.3 s at n=1000 — and a 1000-entry type table is
2–3 KB on the wire. Recording assumed pairs instead of carrying their complement makes
the same query <10 ms, 70 ms at n=10,000, 1.6 s at n=50,000, with no stack overflow at
400,000.
What it is checked against
width subtyping, the four
optrules collapsed, records, variants, functions inboth variance directions, services, recursive types across two independent tables,
and well-formedness.
flat type pairs (expressible in both representations, so the answers must agree
exactly) and 20,000 random nested pairs flattened into the new form. Zero mismatches
in answers and in well-formedness.
t <: reserved,empty <: t, and transitivity on every triple that satisfied both premises.Well-formedness
References resolve, labels do not repeat, and two rules the shape of a
Compositedoes not enforce on its own: a
onewayfunction has no results(
spec/Candid.md:211), and a service's method type denotes a function(
spec/Candid.md:1223) — the only rule that has to look through the table. The 2^32bound on parameter lists (
spec/Candid.md:209) is deliberately not checked, with thereason recorded: it cannot be violated by anything that fits in memory, but it is what
keeps
indexedFrom'sUInt32argument labels distinct.Deliberately not here
the next PR from its own branch. The generator used for the differential above lives
outside the repo today, which is exactly why it should land as a module.
decSubtype_iffis stated as prose next to the relation, not as asorry.Files outside
lean/REWRITE.md(§5's policy boundary and what slice 1 disproved),crates/README.mdandcrates/CLAUDE.md(thecandid_typesnaming table now recordsSlotandComposite), and.github/workflows/lean.yml, which §5 explicitly allows to ride theworking branch so that it runs while the code is being written.