fix(json): parse big integer literals to correctly rounded doubles - #3996
Conversation
Integer literals beyond 2^53 - 1 should parse to the correctly rounded double with the exact text preserved in repr, like every mainstream JSON implementation, instead of collapsing to the Number(Infinity, repr=...) sentinel. The sentinel behavior currently means: - "10000000000000000000" (1e19, exactly representable) parses to Infinity while "1e19" parses to 1.0e19 - the same value differs by spelling - all big integer texts compare equal via Eq (value is always Infinity) - from_json : Double raises on values doubles represent fine These tests encode the desired behavior and fail; the fix lands in the next commit. The infinity sentinel remains correct - and tested - for integers genuinely beyond double range. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
lex_integer_end conflated "cannot be represented exactly in 2^53" with
"out of double range": once the exact-integer accumulator overflowed
SAFE_INTEGER_LIMIT it returned the Number(Infinity, repr=...) sentinel
immediately, without ever consulting strconv. The decimal/exponent path
already falls back to strconv for correct rounding, so the same value
parsed differently depending on spelling ("10000000000000000000" gave
Infinity while "1e19" gave 1.0e19).
On overflow, fall back to @internal/strconv.parse_double for the
correctly rounded value and keep the exact source text in repr, so
stringify remains lossless. The infinity sentinel now only applies to
literals strconv itself rejects (beyond double range), matching how
mainstream implementations behave (JS rounds; serde_json/Jackson/
Python keep exact or rounded values; nobody yields Infinity in range).
This makes the red tests from the previous commit pass, fixes Eq
collapsing all big-integer texts into one equivalence class, and lets
from_json : Double accept in-range values, while the existing
out-of-range sentinel tests keep passing unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR fixes JSON integer-literal parsing so that integer-shaped literals exceeding the exact-integer range of IEEE-754 doubles (above 2^53−1) fall back to @internal/strconv.parse_double for correct rounding, while preserving the original source text in repr for lossless stringify. This aligns integer-literal behavior with the existing decimal/exponent path and prevents large integer texts from collapsing to an Infinity sentinel unless they are genuinely beyond double range.
Changes:
- Update
lex_integer_endto parse oversized integer literals via@internal/strconv.parse_double, reserving the infinity sentinel for truly out-of-range values. - Add targeted regression tests for big integer literal parsing, equality behavior, and
from_json : Doubledecoding. - Update existing number lexer tests to match the new correctly-rounded behavior.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| json/moon.pkg | Adds core/double to test dependencies for new/updated tests. |
| json/lex_number.mbt | Implements the overflow fallback from exact-integer accumulation to correctly-rounded parse_double, preserving repr. |
| json/lex_number_test.mbt | Updates expected debug output for large integer parsing to reflect correct rounding + preserved repr. |
| json/big_integer_parse_test.mbt | Adds regression coverage for big integer literals (rounding, equality, from_json : Double, and out-of-range sentinel behavior). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Coverage Report for CI Build 5822Coverage increased (+0.001%) to 90.324%Details
Uncovered ChangesNo uncovered changes found. Coverage RegressionsNo coverage regressions found. Coverage Stats
💛 - Coveralls |
With #3996 merged, integer-shaped literals beyond 2^53 - 1 parse to correctly rounded doubles, so the two-regime number contract collapses into the property we originally wanted: every finite double survives printing and re-parsing bit-exactly, and re-stringify reproduces the text (via repr for big integers, via shortest-repr determinism otherwise). Drop the sentinel-window predicate and the generator sanitization that worked around it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
With #3996 merged, integer-shaped literals beyond 2^53 - 1 parse to correctly rounded doubles, so the two-regime number contract collapses into the property we originally wanted: every finite double survives printing and re-parsing bit-exactly, and re-stringify reproduces the text (via repr for big integers, via shortest-repr determinism otherwise). Drop the sentinel-window predicate and the generator sanitization that worked around it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Problem
lex_integer_endconflated "cannot be represented exactly in 2^53" with "out of double range": once the exact-integer accumulator overflowedSAFE_INTEGER_LIMIT(2^53 − 1), it returned theNumber(Infinity, repr=...)sentinel immediately, never consulting strconv. The decimal/exponent path already falls back to strconv for correct rounding, so the same value parsed differently depending on spelling:"1e19"Number(1.0e19)Number(1.0e19)(unchanged)"10000000000000000000"(= 1e19)Number(Infinity, repr=Some(...))Number(1.0e19, repr=Some(...))"9007199254740993"(2^53 + 1)Number(Infinity, repr=Some(...))Number(9007199254740992, repr=Some(...))"9".repeat(400)(beyond double range)Number(Infinity, repr=Some(...))Number(Infinity, repr=Some(...))(unchanged)Consequences of the old behavior:
parse("9007199254740993") == parse("99999999999999999999")wastrue—Eqcompares only the double value, so every big-integer text collapsed into one equivalence classfrom_json : Doubleraised on values doubles represent fine (1e19 is exactly representable)JSON.parserounds correctly; serde_json / Jackson / Python keep exact or correctly rounded values; raw-text preservation (whichrepralready provides) is the opt-in layer elsewhere (Gojson.Number, serdearbitrary_precision,JSON.rawJSON)Found by the QuickCheck property suite in #3995 — the naive
parse(stringify(j)) == jproperty falsified in 30 cases and shrank to a bare bigNumber.Fix
On accumulator overflow, fall back to
@internal/strconv.parse_doublefor the correctly rounded value and keep the exact source text inrepr, sostringifyremains byte-lossless. The infinity sentinel now only applies to literals strconv itself rejects (genuinely beyond double range).Structured as red/green commits:
json/big_integer_parse_test.mbt)lex_integer_endfallback + updated expectations inlex_number_test.mbt(the out-of-range sentinel tests pass unchanged)Validation
moon fmt,moon info— no public API changesNote
If this merges before #3995, the two-regime number-contract test there simplifies back to the universal
parse(stringify(j)) == jproperty; I'll update that branch accordingly.🤖 Generated with Claude Code