mirror of
https://github.com/nlohmann/json.git
synced 2026-09-07 16:57:59 +00:00
Bound UBJSON optimized arrays of a valueless type
An element of type 'Z' (null), 'T' (true) or 'F' (false) is encoded by its type marker alone, so an optimized UBJSON array of one of those has no payload: reading an element consumes no input at all. Its declared count is therefore the only thing that decides how much is allocated, and nothing bounded it. "[$Z#l" and a four-byte count is nine bytes of input describing two billion values; #2793 reports 35 GB and 150 seconds from ten bytes, and OSS-Fuzz has an out-of-memory and a timeout report for the same shape. Every other type costs at least one byte per element, so the end of the input bounds it. 'N' (no-op) is already skipped rather than stored. Objects are not affected either: each element is preceded by its key, which costs bytes. And BJData already refuses these markers as an optimized type, so this is a plain UBJSON matter. Reject a count above 1,048,576 elements for those three types with out_of_range.408, the code this reader already uses for a declared size it will not honour. The check runs before the SAX start event, so no container is opened and then abandoned. Rejecting on the read side alone would break the guarantee that anything to_ubjson() writes can be read back, and would trip the round-trip assertion in fuzzer-parse_ubjson.cpp. So the writer falls back to the unoptimized encoding, one byte per element, for arrays of these types above the same limit. Its decision depends only on the array's size, which is identical for a value and for anything parsed back from it, so the round trip is stable. No existing test changes: the largest such count in the test suite is 65,793. The excessive-size test that already used this shape still passes, now rejected a little earlier than by the max_size() check it used to reach. Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
@@ -69,6 +69,12 @@ The library uses the following mapping from JSON values types to UBJSON types ac
|
||||
Note that `use_size = true` alone may result in larger representations - the benefit of this parameter is that the
|
||||
receiving side is immediately informed on the number of elements of the container.
|
||||
|
||||
An array whose type marker is `Z` (null), `T` (true) or `F` (false) stores no payload at all, because the marker
|
||||
already is the value. Its declared count is therefore the only thing that decides how much memory the receiving side
|
||||
allocates, and a handful of bytes can describe billions of elements. `from_ubjson` rejects such an array with
|
||||
[`out_of_range.408`](../../home/exceptions.md#jsonexceptionout_of_range408) when the count exceeds 1,048,576, and
|
||||
`to_ubjson` writes longer arrays of these types without the annotation, so any value it produces can be read back.
|
||||
|
||||
!!! info "Binary values"
|
||||
|
||||
If the JSON data contains the binary type, the value stored is a list of integers, as suggested by the UBJSON
|
||||
|
||||
@@ -868,6 +868,12 @@ The size of an array or object in a [binary format](../features/binary_formats/i
|
||||
the size following `#` for [UBJSON](../features/binary_formats/ubjson.md)/[BJData](../features/binary_formats/bjdata.md),
|
||||
or the encoded length for [CBOR](../features/binary_formats/cbor.md).
|
||||
|
||||
The exception is also thrown for a [UBJSON](../features/binary_formats/ubjson.md) array of a type that is encoded by its
|
||||
marker alone (`Z`, `T` or `F`) whose declared count exceeds 1,048,576. Such an array has no payload, so its count alone
|
||||
decides how much memory is allocated, and a handful of bytes would otherwise describe billions of values.
|
||||
[`to_ubjson`](../api/basic_json/to_ubjson.md) writes longer arrays of these types without the size and type annotation,
|
||||
so any value it produces can still be read back.
|
||||
|
||||
!!! failure "Example messages"
|
||||
|
||||
```
|
||||
@@ -879,6 +885,9 @@ or the encoded length for [CBOR](../features/binary_formats/cbor.md).
|
||||
```
|
||||
[json.exception.out_of_range.408] syntax error while parsing CBOR size: excessive map size
|
||||
```
|
||||
```
|
||||
[json.exception.out_of_range.408] syntax error while parsing UBJSON size: excessive array size
|
||||
```
|
||||
|
||||
### json.exception.out_of_range.409
|
||||
|
||||
|
||||
Reference in New Issue
Block a user