Skip to content

Reject MessagePack/BSON binary subtypes that don't fit their wire format - #5469

Open
nlohmann wants to merge 2 commits into
fix/insert-array-iterator-checkfrom
fix/msgpack-bson-subtype-range
Open

Reject MessagePack/BSON binary subtypes that don't fit their wire format#5469
nlohmann wants to merge 2 commits into
fix/insert-array-iterator-checkfrom
fix/msgpack-bson-subtype-range

Conversation

@nlohmann

@nlohmann nlohmann commented Sep 5, 2026

Copy link
Copy Markdown
Owner

Summary

byte_container_with_subtype::subtype() is a std::uint64_t, but the MessagePack ext-type byte and the BSON binary subtype byte are each a single byte on the wire. Both writers (include/nlohmann/detail/output/binary_writer.hpp) cast the subtype down to a single byte without any range check, silently wrapping modulo 256 instead of erroring — a silent data-corruption bug:

json::from_msgpack(json::to_msgpack(json::binary({1, 2}, 256))).get_binary().subtype();    // 0, not 256
json::from_msgpack(json::to_msgpack(json::binary({1, 2}, 70000))).get_binary().subtype();  // 112, not 70000

The valid range for both formats is actually the full 0–255 (a full unsigned byte), not just 0–127 — the MessagePack reader already round-trips 128–255 correctly via a signed/unsigned reinterpretation of the ext-type byte, and the BSON reader reads the subtype directly as uint8_t. Only subtypes ≥ 256 are affected. The CBOR writer already handles the full 64-bit range correctly (choosing 1/2/4/8-byte tag encodings), and the BSON writer already throws out_of_range.407-adjacent errors for other values it can't represent, so a check here is consistent with existing behavior.

Fix

Add a range check (subtype() > 255) before each of the two truncating casts, throwing a new out_of_range.413 (the next free out_of_range code after the existing .412) instead of silently truncating. A binary value with no subtype at all is unaffected.

Test plan

  • Added tests/src/unit-regression2.cpp coverage: MessagePack/BSON subtype 0/200/255 still round-trip correctly (regression guard for the now-confirmed-correct 128–255 range); subtype 256/70000/300 now throw out_of_range.413 with the exact expected message; CBOR (already correct) remains unaffected at subtype 70000; a binary value with no subtype at all is unaffected.
  • Documented the new exception code in docs/mkdocs/docs/home/exceptions.md (json.exception.out_of_range.413), matching the existing style.
  • Regenerated single_include/nlohmann/json.hpp via make amalgamate.

Breaking change?

No breaking changes to the public API — this adds a new exception code (out_of_range.413) and only affects inputs that previously silently corrupted data (a subtype ≥ 256 being written and read back as a different, wrong value). No valid/previously-correct round-trip changes behavior; serializing a binary value whose subtype is ≥ 256 now throws instead of silently producing a wrong result.

🤖 Generated with Claude Code

@nlohmann
nlohmann changed the base branch from develop to fix/insert-array-iterator-check September 5, 2026 14:53
@nlohmann
nlohmann force-pushed the fix/msgpack-bson-subtype-range branch from 1331cd0 to a1a69a8 Compare September 5, 2026 14:57
@nlohmann
nlohmann force-pushed the fix/msgpack-bson-subtype-range branch from a1a69a8 to 2e83234 Compare September 5, 2026 15:07
@nlohmann nlohmann added aspect: binary formats BSON, CBOR, MessagePack, UBJSON review needed It would be great if someone could review the proposed changes. labels Sep 5, 2026
@nlohmann
nlohmann force-pushed the fix/msgpack-bson-subtype-range branch from 2e83234 to 7705208 Compare September 5, 2026 16:27
@nlohmann
nlohmann force-pushed the fix/msgpack-bson-subtype-range branch from 7705208 to 51ff50b Compare September 5, 2026 16:43
@nlohmann
nlohmann force-pushed the fix/msgpack-bson-subtype-range branch from 51ff50b to 572c308 Compare September 5, 2026 18:00
@nlohmann
nlohmann force-pushed the fix/msgpack-bson-subtype-range branch from 572c308 to 93a1a8a Compare September 5, 2026 19:59
@nlohmann
nlohmann force-pushed the fix/msgpack-bson-subtype-range branch 2 times, most recently from bda5371 to 6a81cf3 Compare September 6, 2026 11:26
@github-actions github-actions Bot added L and removed M labels Sep 6, 2026
Comment thread docs/mkdocs/docs/api/basic_json/insert.md
@nlohmann
nlohmann force-pushed the fix/msgpack-bson-subtype-range branch 3 times, most recently from 6a81cf3 to 795b11f Compare September 8, 2026 18:11
@github-actions github-actions Bot added M and removed L labels Sep 8, 2026
@github-actions github-actions Bot added L M and removed M L labels Sep 8, 2026
Both formats store byte_container_with_subtype's subtype (a uint64_t)
in a single byte. The writers cast to std::int8_t/std::uint8_t without
a range check, so subtypes above 255 were silently truncated modulo
256 instead of raising an error. Throw out_of_range.413 instead when
the subtype exceeds the representable range of 0-255.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
unit-regression2.cpp is already at the edge of what the MinGW linker
can relocate; adding this test's ~26 lines tips test-regression2_cpp20
(clang, Windows) over into "relocation truncated to fit:
IMAGE_REL_AMD64_REL32 against `.rdata'" (see 8ce64b9 / b82717c for
the same failure mode). Split the test along format lines instead:
MessagePack assertions move to unit-msgpack.cpp, BSON assertions to
unit-bson.cpp. The CBOR round-trip guard is dropped as redundant --
unit-cbor.cpp's "Tagged values" section already round-trips subtypes
up to 8589934590, far past the 70000 checked here.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
@nlohmann
nlohmann force-pushed the fix/msgpack-bson-subtype-range branch from 795b11f to 1b2b595 Compare September 9, 2026 11:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

aspect: binary formats BSON, CBOR, MessagePack, UBJSON documentation M review needed It would be great if someone could review the proposed changes. tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants