Skip to content

Honor allow_exceptions=false for excessive array/object size (out_of_range.408) - #5467

Open
nlohmann wants to merge 3 commits into
fix/parser-callback-duplicate-keyfrom
fix/allow-exceptions-408
Open

Honor allow_exceptions=false for excessive array/object size (out_of_range.408)#5467
nlohmann wants to merge 3 commits into
fix/parser-callback-duplicate-keyfrom
fix/allow-exceptions-408

Conversation

@nlohmann

@nlohmann nlohmann commented Sep 5, 2026

Copy link
Copy Markdown
Owner

Summary

The size sanity check in json_sax_dom_parser/json_sax_dom_callback_parser's start_object()/start_array() (include/nlohmann/detail/input/json_sax.hpp) throws out_of_range.408 directly via JSON_THROW, instead of routing through parse_error() — the only place that honors the SAX consumer's allow_exceptions flag. Every other malformed-input error in the binary readers (truncated input, invalid bytes, bad UTF-8 in CBOR sizes, etc.) correctly yields a discarded value when allow_exceptions is false; this one path didn't, and could also terminate JSON_NOEXCEPTION builds.

json::from_cbor(std::vector<uint8_t>{0x9b, 0x80,0,0,0,0,0,0,0}, true, false);  // CBOR array, length 2^63
// before: throws out_of_range.408, even though allow_exceptions=false
// after:  returns a discarded value, per the documented allow_exceptions contract

Affects CBOR (array and map), UBJSON, and BJData. MessagePack cannot trigger it on 64-bit platforms (lengths are at most 32-bit) and BSON has no element counts, so neither needs a code change.

Fix

Replace the four JSON_THROW(out_of_range::create(408, ...)) call sites with return parse_error(0, "", out_of_range::create(408, ...)), matching the pattern already used by every other error path in these two classes. The binary readers already propagate a false return from sax->start_object()/sax->start_array() correctly (the same mechanism every other error condition uses), so no changes were needed there.

This is behavior-preserving when allow_exceptions == true (still throws the identical exception) — the only behavior change is for allow_exceptions == false.

Test plan

  • Added a regression test to tests/src/unit-regression2.cpp covering CBOR array/map, UBJSON, and BJData with an oversized declared length, both with allow_exceptions=false (now is_discarded()) and the default allow_exceptions=true (still throws, message unchanged), plus a truncated-input regression guard.
  • Verified the new is_discarded() assertions fail (throw) without the fix and pass with it.
  • Regenerated single_include/nlohmann/json.hpp via make amalgamate.

Breaking change?

No breaking changes to the public API. allow_exceptions=true (the default) behavior is byte-for-byte unchanged. allow_exceptions=false now correctly returns a discarded value instead of throwing/aborting for this specific malformed-input case, matching the documented contract for every other malformed-binary-input case — code that relied on the previous (undocumented, inconsistent) throwing behavior under allow_exceptions=false would need to check is_discarded() instead, same as it already must for every other kind of parse error.

🤖 Generated with Claude Code

@nlohmann
nlohmann changed the base branch from develop to fix/parser-callback-duplicate-key September 5, 2026 14:53
@nlohmann
nlohmann force-pushed the fix/allow-exceptions-408 branch from 825fa4a to fdc6a0d Compare September 5, 2026 14:57
@nlohmann
nlohmann force-pushed the fix/allow-exceptions-408 branch from fdc6a0d to 2cee81a Compare September 5, 2026 15:06
@nlohmann nlohmann added aspect: binary formats BSON, CBOR, MessagePack, UBJSON review needed It would be great if someone could review the proposed changes. labels Sep 5, 2026
@nlohmann
nlohmann force-pushed the fix/allow-exceptions-408 branch from 2cee81a to e7761fd Compare September 5, 2026 16:27
@nlohmann
nlohmann force-pushed the fix/allow-exceptions-408 branch from e7761fd to 6ddf16e Compare September 5, 2026 16:43
@nlohmann
nlohmann force-pushed the fix/allow-exceptions-408 branch from f66897e to 8e3cdd0 Compare September 5, 2026 20:01
@nlohmann nlohmann added 🚀 ready to merge Ready to merge - just waiting for CI to complete. and removed review needed It would be great if someone could review the proposed changes. labels Sep 8, 2026
@nlohmann nlohmann added this to the Release 3.13.0 milestone Sep 9, 2026
…range.408)

The SAX DOM parsers' start_object()/start_array() threw out_of_range.408
directly via JSON_THROW when a binary format (CBOR/UBJSON/BJData) declared
a container size exceeding max_size(), bypassing the allow_exceptions flag
that every other malformed-input error path in these classes honors via
parse_error(). This meant that json::from_cbor(data, true, false) etc.
could still throw (or abort under JSON_NOEXCEPTION) instead of returning a
discarded value, contrary to the allow_exceptions=false contract.

Route all four call sites (two in json_sax_dom_parser, two in
json_sax_dom_callback_parser) through parse_error() instead, matching the
existing error-handling pattern used elsewhere in this file. Behavior is
unchanged when allow_exceptions is true (the default); the exception
message and type are identical.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The allow_exceptions=false regression test checked the exact message
text produced when allow_exceptions=true (the default). On platforms
where std::size_t is 32-bit (e.g. mingw x86, MSVC Win32 builds), a
declared CBOR length of 2^63 is intercepted earlier, by
get_cbor_container_size()'s own (pre-existing, already correct)
length-narrowing check, with different wording than this fix's
start_array()/start_object() size check -- same error code, same
"still throws when allow_exceptions=true" guarantee, different text.

CHECK_THROWS_AS already verifies the behavior this test cares about
(still throws json::out_of_range, unchanged); drop the exact-message
assertion since it isn't portable across size_t widths and doesn't
add coverage of this fix specifically.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
…test

from_cbor() is [[nodiscard]]; CHECK_THROWS_AS() otherwise discards its
result, which GCC flags under -Werror. Assign to a throwaway json, as
the rest of the suite already does for from_cbor()/from_msgpack().

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
@nlohmann
nlohmann force-pushed the fix/allow-exceptions-408 branch from 8e3cdd0 to a7f38e7 Compare September 9, 2026 11:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

aspect: binary formats BSON, CBOR, MessagePack, UBJSON M 🚀 ready to merge Ready to merge - just waiting for CI to complete. tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants