Report an empty array for UBJSON [$N#<n>] instead of n elements (#6079)

* Report an empty array for UBJSON `[$N#<n>` instead of n elements

A no-op (N) is not a value, so an optimized array of type N holds no
element whatever count it declares. The reader skipped the elements but
still passed the declared count to the SAX parser: a handler was told
start_array(n) and no element followed, and the DOM parser reserved
min(n, 16384) values (256 KB) for an array that stays empty. Nine bytes
(`[$N#l 0x7FFFFFFF`) therefore committed 256 KB: 2000 of them in one
input, 18 KB, committed 512 MB of heap (capacity 163,840,000 values for
the 10000 arrays of the original probe: 2.6 GB for 70 KB of input).
Untouched pages keep the resident size small, but the commit charge and
a 32-bit address space are consumed.

The reader now announces 0 elements for an optimized array of type N.
The loop that skips the elements is unchanged (it never ran). A count
that does not fit into std::size_t is still rejected by the count
reader; a count that is too large for the array type of the DOM (before:
out_of_range.408 "excessive array size" from the DOM handler) is
accepted now, because the array is empty.

Behaviour change: sax_parse reports start_array(0) for `[$N#n` (before:
n); from_ubjson no longer throws out_of_range.408 for an N-typed array
whose count exceeds the maximum size of the array type.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Clarify the reservation note for announced array counts

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
Niels Lohmann authored and GitHub committed 2026-11-11 22:37:23 +01:00
1 parent 71ec3b4306
commit 5187b6ecc7
6 files changed
+116 -26

No files matched your search

@@ -89,9 +89,10 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
- Throws [out_of_range.406](../../home/exceptions.md#jsonexceptionout_of_range406) if a high-precision number, or a
float64 when `number_float_t` is `float`, does not fit into `number_float_t`
- Throws [out_of_range.408](../../home/exceptions.md#jsonexceptionout_of_range408) if the count of an optimized
container exceeds the capacity of the container (or of `std::size_t`), if an optimized array of `Z`, `T`, or `F` has
more than 1,048,576 elements, or if the optimized arrays of `Z`, `T`, and `F` in the input together have more than
1,048,576 elements plus 4 for every byte read so far
container exceeds the capacity of the container (or of `std::size_t`; an array of type `N` is empty, so only the
latter applies to it), if an optimized array of `Z`, `T`, or `F` has more than 1,048,576 elements, or if the
optimized arrays of `Z`, `T`, and `F` in the input together have more than 1,048,576 elements plus 4 for every byte
read so far
## Complexity
@@ -133,6 +134,8 @@ Linear in the size of the input.
- Extended container support (1) to include types with lvalue-only ADL `begin`/`end` (matching `std::begin`/`std::end` semantics) in version 3.13.0.
- Extended overload (2) to accept heterogeneous iterator+sentinel pairs (C++20 ranges support) in version 3.13.0.
- Added `error_handler` parameter in version 3.13.0.
- Report `start_array(0)` to a SAX handler for an optimized array of type `N` (`[$N#n`), whatever its count, in version
3.13.0.
- Limited the total number of elements of optimized arrays of `Z`, `T`, and `F` in one input in version 3.13.0.
!!! warning "Deprecation"