amorfo77/json

0 Fork 0

mirror of https://github.com/nlohmann/json.git synced 2026-10-27 00:08:55 +00:00

Files

T

Niels Lohmann dc768a7d5d Fix BSON binary subtype 0x02 and check the declared document size earlier (#5930 )

* Write and read the inner length of BSON binary subtype 0x02

The BSON specification defines the payload of the "binary (old)" subtype
0x02 as an int32 followed by that many bytes. to_bson wrote the plain
payload instead, so decoders such as pymongo and libbson-based readers
rejected the bytes, and from_bson returned the inner length as part of the
payload (documented as lenient behavior in gh-5333).

The writer now emits the outer length (payload + 4), the subtype, the inner
int32 holding the payload size, and the payload. The reader requires an
outer length of at least 4 and an inner length equal to the outer length
minus 4 (the cases the bson-corpus lists as decode errors), reports parse
error 112 otherwise, and returns only the payload. Both directions change
together so that the round trip keeps working.

The single-reservation fast path for scalar entries declines subtype 0x02
because it has no room for the inner length.

This is a deliberate exception to the opt-in rule for behavior changes:
the previous output was invalid BSON.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Check the declared BSON document size before reading its content

The BSON reader compared a document's declared size with the number of bytes
read only when it reached the terminator. The declared size never bounded
the reads, so a SAX handler saw the keys and values of a document whose size
was too small before parse error 112 was reported.

The container frame now stores the position where the document ends
according to its declared size. open_bson_document rejects a size below 5
(size prefix plus terminator) and an embedded document that does not end
before the terminator of its enclosing document; the element loop rejects an
element whose type byte leaves no room for the name terminator and the
document terminator. The terminator check stays and still catches a declared
size that is too large. No input that was accepted before is rejected now;
documents with a wrong size are reported earlier, with fewer SAX events, a
different byte position and, in three cases, a different message. The error
id (parse_error.112) is unchanged and strict=false still does not skip the
check.

The tests that asserted the old messages and positions are updated, with the
outer sizes of two inputs corrected so that they still exercise the embedded
document. New tests cover the SAX events, a size below 5, and an embedded
document that fits exactly.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Document BSON subtype 0x02, document size checks, and lenient reads

Describe the inner length of binary subtype 0x02 that to_bson now writes and
from_bson checks and removes. Add the document size checks (parse_error.112,
not affected by strict=false) to bson.md and from_bson.md, and extend the
list of lenient reads with the UUID and MD5 payload lengths (not checked),
duplicate keys (last value wins), embedded NUL bytes in strings, and the
ignored trailing bytes with strict=false. State that the members of an object
are written in iteration order.

Correct the statement that element names are never validated on read: they
are checked and sanitized like string values by the error handler. Add the
BSON message to the examples of out_of_range.415 and type_error.321, the
subtype limit of 255 to the binary values page, CBOR to the formats that use
byte_container_with_subtype, and fix a typo in the incomplete mapping box.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Regenerate single_include and apply astyle

Result of make amalgamate (including make pretty) for the preceding changes.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Address review of the BSON documentation

Describe exactly when a too small document size is detected and note that subtype 2 data written by earlier versions is not readable.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: use the Oxford comma, drop discarded-value remarks

Add the serial comma in a list added by this PR and remove the BSON
example of type_error.321 (with the type_error.317 remark) so the docs do
not mention discarded values for BSON; restore the original singular
heading of that section. No tables were changed by this PR.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix -Weffc++ in the BSON recorder test and a clang-tidy use-auto hit

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix clang-tidy readability-redundant-member-init in BSON test

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

2026-10-26 18:25:46 +01:00

5.9 KiB

Raw Blame History

nlohmann::basic_json::to_bson

// (1)
static std::vector<std::uint8_t> to_bson(const basic_json& j,
                                         const error_handler_t error_handler = error_handler_t::keep);

// (2)
static void to_bson(const basic_json& j, detail::output_adapter<std::uint8_t> o,
                    const error_handler_t error_handler = error_handler_t::keep);
static void to_bson(const basic_json& j, detail::output_adapter<char> o,
                    const error_handler_t error_handler = error_handler_t::keep);

BSON (Binary JSON) is a binary format in which zero or more ordered key/value pairs are stored as a single entity (a so-called document). The members of an object are written in the order in which iteration visits them.

Returns a byte vector containing the BSON serialization.
Writes the BSON serialization to an output adapter.

The exact mapping and its limitations are described on a dedicated page.

Parameters

j (in): JSON value to serialize
o (in): output adapter to write serialization to
error_handler (in): how to treat a string or object key in j that is not valid UTF-8; see error_handler_t. The default, keep, writes the ill-formed bytes to the output as is, as every version of to_bson did before this parameter was added; strict throws; replace/ignore sanitize it the same way dump would. If JSON_STRICT_BINARY_UTF8 is enabled, the default is strict instead.

Return value

BSON serialization as a byte vector
(none)

Exception safety

Strong guarantee: if an exception is thrown, there are no changes in the JSON value.

Exceptions

Throws type_error.317 if the top-level type of the JSON value is not an object; example: "to serialize to BSON, top-level type must be object, but is string"
Throws out_of_range.407 if j contains a signed integer outside the range of int64 or an unsigned integer outside the range of uint64, which is only possible with a number type wider than 64 bits; example: "integer number 9223372036854775808 cannot be represented by BSON as it does not fit int64"
Throws out_of_range.409 if a key in the JSON object contains a null byte (code point U+0000); example: "BSON key cannot contain code point U+0000 (at byte 2)"
Throws out_of_range.412 if the length of a document, array, string, or binary value exceeds the range of the 32-bit BSON length field; example: "BSON length 2147483661 exceeds maximum of 2147483647"
Throws out_of_range.415 if the subtype of a binary value exceeds 255, the maximum of the BSON binary subtype; example: "subtype 70000 is too large for the BSON binary subtype (max 255)"
Throws type_error.316 if a string or object key is not valid UTF-8 and error_handler is strict (the default only if JSON_STRICT_BINARY_UTF8 is enabled)
Throws type_error.321 if a value nested in j is discarded (the top-level value itself is covered by type_error.317 above, since it must be an object); example: "cannot serialize discarded value to BSON"

Complexity

Linear in the size of the JSON value j. The length prefixes of all nested documents and arrays are computed in one pass before anything is written.

Examples

??? example "Example: serialize a JSON value to BSON"

The example shows the serialization of a JSON value to a byte vector in BSON format.
 
```cpp
--8<-- "examples/to_bson.cpp"
```

Output:

```json
--8<-- "examples/to_bson.output"
```

??? example "Example: out_of_range.409 exception"

The example shows how serializing a JSON object whose key contains a null byte (U+0000) throws an exception, because
BSON keys are null-terminated C strings and cannot contain U+0000 themselves.

```cpp
--8<-- "examples/to_bson__exception.cpp"
```

Output:

```json
--8<-- "examples/to_bson__exception.output"
```

See also

from_bson create a JSON value from an input in BSON format
to_cbor create a CBOR serialization of a JSON value
to_msgpack create a MessagePack serialization of a JSON value
to_ubjson create a UBJSON serialization of a JSON value
to_bjdata create a BJData serialization of a JSON value
to_bon8 create a BON8 serialization of a JSON value

Version history

Added in version 3.4.0.
Throws out_of_range.412 and out_of_range.415 since version 3.13.0.
Linear in the size of j, and no longer limited by the call stack for deeply nested values, since version 3.13.0.
out_of_range.415 is now detected before anything is written, like the other exceptions above, since version 3.13.0.
Throws type_error.321 for a discarded value nested in j since version 3.13.0; previously, it was silently skipped, producing a document whose declared size did not match what was actually written.
Added error_handler parameter in version 3.13.0. Its default, keep, writes the bytes of a string or object key that is not valid UTF-8 unchanged, as before; strict (the default if JSON_STRICT_BINARY_UTF8 is enabled) throws type_error.316 before anything is written.
Throws out_of_range.407 for integers that do not fit 64 bits since version 3.13.0; previously, integers of a number type wider than 64 bits were silently truncated.