Skip to content

fix(json): parse big integer literals to correctly rounded doubles - #3996

Merged
bobzhang merged 2 commits into
mainfrom
json-bigint-correct-rounding
Aug 7, 2026
Merged

fix(json): parse big integer literals to correctly rounded doubles#3996
bobzhang merged 2 commits into
mainfrom
json-bigint-correct-rounding

Conversation

@bobzhang

@bobzhang bobzhang commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Problem

lex_integer_end conflated "cannot be represented exactly in 2^53" with "out of double range": once the exact-integer accumulator overflowed SAFE_INTEGER_LIMIT (2^53 − 1), it returned the Number(Infinity, repr=...) sentinel immediately, never consulting strconv. The decimal/exponent path already falls back to strconv for correct rounding, so the same value parsed differently depending on spelling:

Input Before After
"1e19" Number(1.0e19) Number(1.0e19) (unchanged)
"10000000000000000000" (= 1e19) Number(Infinity, repr=Some(...)) Number(1.0e19, repr=Some(...))
"9007199254740993" (2^53 + 1) Number(Infinity, repr=Some(...)) Number(9007199254740992, repr=Some(...))
"9".repeat(400) (beyond double range) Number(Infinity, repr=Some(...)) Number(Infinity, repr=Some(...)) (unchanged)

Consequences of the old behavior:

  • parse("9007199254740993") == parse("99999999999999999999") was trueEq compares only the double value, so every big-integer text collapsed into one equivalence class
  • from_json : Double raised on values doubles represent fine (1e19 is exactly representable)
  • No mainstream implementation behaves this way: JS JSON.parse rounds correctly; serde_json / Jackson / Python keep exact or correctly rounded values; raw-text preservation (which repr already provides) is the opt-in layer elsewhere (Go json.Number, serde arbitrary_precision, JSON.rawJSON)

Found by the QuickCheck property suite in #3995 — the naive parse(stringify(j)) == j property falsified in 30 cases and shrank to a bare big Number.

Fix

On accumulator overflow, fall back to @internal/strconv.parse_double for the correctly rounded value and keep the exact source text in repr, so stringify remains byte-lossless. The infinity sentinel now only applies to literals strconv itself rejects (genuinely beyond double range).

Structured as red/green commits:

  1. test(json): failing tests encoding the desired behavior (json/big_integer_parse_test.mbt)
  2. fix(json): the lex_integer_end fallback + updated expectations in lex_number_test.mbt (the out-of-range sentinel tests pass unchanged)

Validation

  • Full repo suite: 7029/7029 passing
  • moon fmt, moon info — no public API changes

Note

If this merges before #3995, the two-regime number-contract test there simplifies back to the universal parse(stringify(j)) == j property; I'll update that branch accordingly.

🤖 Generated with Claude Code

bobzhang and others added 2 commits August 7, 2026 15:31
Integer literals beyond 2^53 - 1 should parse to the correctly rounded
double with the exact text preserved in repr, like every mainstream
JSON implementation, instead of collapsing to the Number(Infinity,
repr=...) sentinel. The sentinel behavior currently means:

- "10000000000000000000" (1e19, exactly representable) parses to
  Infinity while "1e19" parses to 1.0e19 - the same value differs by
  spelling
- all big integer texts compare equal via Eq (value is always Infinity)
- from_json : Double raises on values doubles represent fine

These tests encode the desired behavior and fail; the fix lands in the
next commit. The infinity sentinel remains correct - and tested - for
integers genuinely beyond double range.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
lex_integer_end conflated "cannot be represented exactly in 2^53" with
"out of double range": once the exact-integer accumulator overflowed
SAFE_INTEGER_LIMIT it returned the Number(Infinity, repr=...) sentinel
immediately, without ever consulting strconv. The decimal/exponent path
already falls back to strconv for correct rounding, so the same value
parsed differently depending on spelling ("10000000000000000000" gave
Infinity while "1e19" gave 1.0e19).

On overflow, fall back to @internal/strconv.parse_double for the
correctly rounded value and keep the exact source text in repr, so
stringify remains lossless. The infinity sentinel now only applies to
literals strconv itself rejects (beyond double range), matching how
mainstream implementations behave (JS rounds; serde_json/Jackson/
Python keep exact or rounded values; nobody yields Infinity in range).

This makes the red tests from the previous commit pass, fixes Eq
collapsing all big-integer texts into one equivalence class, and lets
from_json : Double accept in-range values, while the existing
out-of-range sentinel tests keep passing unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings August 7, 2026 07:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes JSON integer-literal parsing so that integer-shaped literals exceeding the exact-integer range of IEEE-754 doubles (above 2^53−1) fall back to @internal/strconv.parse_double for correct rounding, while preserving the original source text in repr for lossless stringify. This aligns integer-literal behavior with the existing decimal/exponent path and prevents large integer texts from collapsing to an Infinity sentinel unless they are genuinely beyond double range.

Changes:

  • Update lex_integer_end to parse oversized integer literals via @internal/strconv.parse_double, reserving the infinity sentinel for truly out-of-range values.
  • Add targeted regression tests for big integer literal parsing, equality behavior, and from_json : Double decoding.
  • Update existing number lexer tests to match the new correctly-rounded behavior.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File Description
json/moon.pkg Adds core/double to test dependencies for new/updated tests.
json/lex_number.mbt Implements the overflow fallback from exact-integer accumulation to correctly-rounded parse_double, preserving repr.
json/lex_number_test.mbt Updates expected debug output for large integer parsing to reflect correct rounding + preserved repr.
json/big_integer_parse_test.mbt Adds regression coverage for big integer literals (rounding, equality, from_json : Double, and out-of-range sentinel behavior).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@coveralls

Copy link
Copy Markdown
Collaborator

Coverage Report for CI Build 5822

Coverage increased (+0.001%) to 90.324%

Details

  • Coverage increased (+0.001%) from the base build.
  • Patch coverage: 4 of 4 lines across 1 file are fully covered (100%).
  • No coverage regressions found.

Uncovered Changes

No uncovered changes found.

Coverage Regressions

No coverage regressions found.


Coverage Stats

Coverage Status
Relevant Lines: 17973
Covered Lines: 16234
Line Coverage: 90.32%
Coverage Strength: 169605.65 hits per line

💛 - Coveralls

@bobzhang
bobzhang enabled auto-merge (rebase) August 7, 2026 07:40
@bobzhang
bobzhang merged commit 9bcbc00 into main Aug 7, 2026
16 checks passed
@bobzhang
bobzhang deleted the json-bigint-correct-rounding branch August 7, 2026 07:42
bobzhang added a commit that referenced this pull request Aug 7, 2026
With #3996 merged, integer-shaped literals beyond 2^53 - 1 parse to
correctly rounded doubles, so the two-regime number contract collapses
into the property we originally wanted: every finite double survives
printing and re-parsing bit-exactly, and re-stringify reproduces the
text (via repr for big integers, via shortest-repr determinism
otherwise). Drop the sentinel-window predicate and the generator
sanitization that worked around it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
bobzhang added a commit that referenced this pull request Aug 7, 2026
With #3996 merged, integer-shaped literals beyond 2^53 - 1 parse to
correctly rounded doubles, so the two-regime number contract collapses
into the property we originally wanted: every finite double survives
printing and re-parsing bit-exactly, and re-stringify reproduces the
text (via repr for big integers, via shortest-repr determinism
otherwise). Drop the sentinel-window predicate and the generator
sanitization that worked around it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants