Correct and complete the UBJSON documentation

Fix statements that disagree with the behavior (checked with probes):
- `Z` is null; the unused marker is `N`.
- `$` requires `#`; use_type with use_size=false throws other_error.502 only
  for non-empty arrays, objects and binary values.
- NaN and infinity are written as IEEE 754 float64 and read back unchanged;
  the UBJSON specification encodes infinity as null.
- `H` is documented (semantics, parse_error.115, out_of_range.406); the
  "complete mapping" claim is qualified.
- use_type "same type" means same marker.
- Long Z/T/F arrays lose only the `$` annotation; the `#` count and the
  element markers are kept.
- from_ubjson lists the 112, 113, 115, 406 and 408 causes that occur for
  UBJSON, without the BJData-only ND-array wording.
- Duplicate object keys keep the last value; the `$` type marker is
  validated even for count 0.
- Smaller fixes: float32 row, "map" -> "object", https link, exception 110
  strict case, to_ubjson history of discarded values.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
Niels Lohmann committed 2026-10-25 20:05:27 +01:00
1 parent d1d142633a
commit 5ac95db5a9
4 files changed
+63 -21

No files matched your search

+11 -4
View File
@@ -79,11 +79,18 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
- Throws [parse_error.110](../../home/exceptions.md#jsonexceptionparse_error110) if the given input ends prematurely or
the end of the file was not reached when `strict` was set to true
- Throws [parse_error.112](../../home/exceptions.md#jsonexceptionparse_error112) if a parse error occurs
- Throws [parse_error.112](../../home/exceptions.md#jsonexceptionparse_error112) if an invalid type marker is read
(including a type marker after `$` of an optimized container, even if its count is 0), or if `$` is not followed by `#`
- Throws [parse_error.113](../../home/exceptions.md#jsonexceptionparse_error113) if a string could not be parsed
successfully, or if a string value or object key is not valid UTF-8 and `error_handler` is `strict`
- Throws [out_of_range.408](../../home/exceptions.md#jsonexceptionout_of_range408) if the size of an optimized container
or n-dimensional array cannot be represented by `std::size_t`
successfully (negative or non-integer length, `C` followed by a byte above 0x7F), if the count of an optimized
container is negative, or if a string value or object key is not valid UTF-8 and `error_handler` is `strict`
- Throws [parse_error.115](../../home/exceptions.md#jsonexceptionparse_error115) if the text of a high-precision number
(`H`) is not a valid JSON number
- Throws [out_of_range.406](../../home/exceptions.md#jsonexceptionout_of_range406) if a high-precision number, or a
float64 when `number_float_t` is `float`, does not fit into `number_float_t`
- Throws [out_of_range.408](../../home/exceptions.md#jsonexceptionout_of_range408) if the count of an optimized
container exceeds the capacity of the container (or of `std::size_t`), or if an optimized array of `Z`, `T` or `F` has
more than 1,048,576 elements
## Complexity
+2 -2
View File
@@ -114,5 +114,5 @@ Linear in the size of the JSON value `j`.
- Added `error_handler` parameter in version 3.13.0. Its default, `keep`, writes the bytes of a string or object key
that is not valid UTF-8 unchanged, as before; `strict` (the default if
[`JSON_STRICT_BINARY_UTF8`](../macros/json_strict_binary_utf8.md) is enabled) throws `type_error.316`.
- Throws `type_error.321` for a discarded value since version 3.13.0; previously, a discarded value nested in an
array or object was silently skipped, producing invalid UBJSON.
- Throws `type_error.321` for a discarded value since version 3.13.0; previously, a discarded array element was
skipped, and an object member was written as a key without a value, producing invalid UBJSON.
@@ -5,11 +5,11 @@ achieve the generality of JSON, combined with being much easier to process than
!!! abstract "References"
- [UBJSON Website](http://ubjson.org)
- [UBJSON Website](https://ubjson.org)
## Byte layout
UBJSON is built from just a one-byte, human-readable ASCII *type marker* (for example, `i` for int8, `S` for string, `[` for array) followed by the payload of that type. Strings and the like are prefixed with their length, which is itself a UBJSON integer value including its marker. Numbers are stored in big-endian byte order. Containers can be written in two ways: as an *unoptimized* container that is opened with `[` or `{` and closed with `]` or `}`, or as an *optimized* container that declares an element type (`$`) and/or a count (`#`) after the opening marker, which saves per-element markers and (for a count) the closing marker.
UBJSON is built from just a one-byte, human-readable ASCII *type marker* (for example, `i` for int8, `S` for string, `[` for array) followed by the payload of that type. Strings and the like are prefixed with their length, which is itself a UBJSON integer value including its marker. Numbers are stored in big-endian byte order. Containers can be written in two ways: as an *unoptimized* container that is opened with `[` or `{` and closed with `]` or `}`, or as an *optimized* container that declares a count (`#`) after the opening marker, optionally preceded by an element type (`$`); an element type is only allowed together with a count. An optimized container saves the closing marker and, with a type, the per-element markers.
Consider the following JSON value (shown formatted for readability):
@@ -63,10 +63,10 @@ The library uses the following mapping from JSON values types to UBJSON types ac
| number_unsigned | 32768..2147483647 | int32 | `l` |
| number_unsigned | 2147483648..9223372036854775807 | int64 | `L` |
| number_unsigned | 9223372036854775808..18446744073709551615 | high-precision | `H` |
| number_float | *any value* | float64 | `D` |
| number_float | *any value* | float64 (float32 if `number_float_t` is `float`) | `D` (`d`) |
| string | *with shortest length indicator* | string | `S` |
| array | *see notes on optimized format* | array | `[` |
| object | *see notes on optimized format* | map | `{` |
| object | *see notes on optimized format* | object | `{` |
!!! success "Complete mapping"
@@ -93,20 +93,27 @@ The library uses the following mapping from JSON values types to UBJSON types ac
The following markers are not used in the conversion:
- `Z`: no-op values are not created.
- `N`: no-op values are never written (they are skipped when reading).
- `C`: single-byte strings are serialized with `S` markers.
!!! info "NaN/infinity handling"
If NaN or Infinity are stored inside a JSON number, they are serialized properly. This behavior differs from the
[`dump()`](../../api/basic_json/dump.md) function which serializes NaN or Infinity to `null`.
NaN and infinity are written as IEEE 754 float64 values (`D`) and read back unchanged. This differs from
[`dump()`](../../api/basic_json/dump.md), which serializes them to `null`, and from the UBJSON specification, which
encodes infinity as `null` (it does not mention NaN); other UBJSON readers may therefore not accept these values.
!!! info "Optimized formats"
The optimized formats for containers are supported: Parameter `use_size` adds size information to the beginning of a
container and removes the closing marker. Parameter `use_type` further checks whether all elements of a container
have the same type and adds the type marker to the beginning of the container. The `use_type` parameter must only be
used together with `use_size = true`.
used together with `use_size = true`: combining `use_type` with `use_size = false` throws
[`other_error.502`](../../home/exceptions.md#jsonexceptionother_error502) if the value contains a non-empty array,
object or binary value.
Two elements have the same type if they are written with the same type marker. For example, `[1, 2]` is written with
the type `i`, but `[1, 200]` (`i`, `U`), `[true, false]` (`T`, `F`) and `[1, 2.5]` (`i`, `D`) are not; empty
containers (`[]`, `{}`) are never typed.
Note that `use_size = true` alone may result in larger representations - the benefit of this parameter is that the
receiving side is immediately informed on the number of elements of the container.
@@ -115,8 +122,9 @@ The library uses the following mapping from JSON values types to UBJSON types ac
already is the value. Its declared count is therefore the only thing that decides how much memory the receiving side
allocates, and a handful of bytes can describe billions of elements. `from_ubjson` rejects such an array with
[`out_of_range.408`](../../home/exceptions.md#jsonexceptionout_of_range408) when the count exceeds 1,048,576
(`1 << 20`), and `to_ubjson` writes longer arrays of these types without the annotation, so any value it produces
can be read back.
(`1 << 20`), and `to_ubjson` writes longer arrays of these types in the counted but untyped form: the `$` type marker is
dropped, the `#` count is kept, and every element is written with its own marker (`[#l 1048577 Z Z ...`), so any value
it produces can be read back. Objects and untyped counted arrays are not subject to this limit.
!!! info "Binary values"
@@ -155,12 +163,38 @@ The library maps UBJSON types to JSON value types as follows:
| int64 | number_integer | `L` |
| string | string | `S` |
| char | string | `C` |
| high-precision | number_unsigned, number_integer or number_float | `H` |
| array | array (optimized values are supported) | `[` |
| object | object (optimized values are supported) | `{` |
!!! info "High-precision numbers"
The payload of `H` is a length-prefixed string that is parsed as a JSON number (surrounding whitespace is ignored).
Integer text in the range of `number_unsigned_t` (non-negative) or `number_integer_t` (negative) is stored as such;
any other valid number text (larger or smaller integers, fractions, exponents such as `1e2`) is stored as
`number_float`, so precision may be lost. Text that is not a valid JSON number raises
[`parse_error.115`](../../home/exceptions.md#jsonexceptionparse_error115), a number outside the range of
`number_float_t` (`1e999`) raises [`out_of_range.406`](../../home/exceptions.md#jsonexceptionout_of_range406).
!!! success "Complete mapping"
The mapping is **complete** in the sense that any UBJSON value can be converted to a JSON value.
Every well-formed UBJSON value can be converted to a JSON value, with these exceptions: a high-precision number
outside the range of `number_float_t` ([`out_of_range.406`](../../home/exceptions.md#jsonexceptionout_of_range406)),
an optimized array of `Z`, `T` or `F` with more than 1,048,576 elements, and a count that exceeds the capacity of the
container ([`out_of_range.408`](../../home/exceptions.md#jsonexceptionout_of_range408)). High-precision numbers that
do not fit an integer type are stored as floating-point numbers and may lose precision. Ill-formed input is rejected
with a parse error.
!!! info "Duplicate object keys"
If an object contains the same key more than once, the last value is kept (the UBJSON specification does not
address duplicate keys).
!!! info "Type marker of optimized containers"
The type marker after `$` must be one of `Z`, `N`, `T`, `F`, `i`, `U`, `I`, `l`, `L`, `d`, `D`, `H`, `C`, `S`, `[` or
`{`. It is checked even if the count is 0; any other marker raises
[`parse_error.112`](../../home/exceptions.md#jsonexceptionparse_error112).
!!! warning "Ill-formed UTF-8 in string values and object keys"
+5 -4
View File
@@ -330,7 +330,7 @@ where they cannot be resolved.
### json.exception.parse_error.110
When parsing a [binary format](../features/binary_formats/index.md), the byte vector ends before the complete value has
been read.
been read, or (with `strict = true`) further bytes follow the value.
!!! failure "Example messages"
@@ -1045,8 +1045,8 @@ or the encoded length for [CBOR](../features/binary_formats/cbor.md).
The exception is also thrown for a [UBJSON](../features/binary_formats/ubjson.md) array of a type that is encoded by its
marker alone (`Z`, `T` or `F`) whose declared count exceeds 1,048,576 (`1 << 20`). Such an array has no payload, so its
count alone decides how much memory is allocated, and a handful of bytes would otherwise describe billions of values.
[`to_ubjson`](../api/basic_json/to_ubjson.md) writes longer arrays of these types without the size and type annotation,
so any value it produces can still be read back.
[`to_ubjson`](../api/basic_json/to_ubjson.md) writes longer arrays of these types without the `$` type annotation (the `#`
count and one marker per element are kept), so any value it produces can still be read back.
!!! failure "Example messages"
@@ -1229,7 +1229,8 @@ A JSON Patch operation 'test' failed. The unsuccessful operation is also printed
### json.exception.other_error.502
[`to_ubjson`](../api/basic_json/to_ubjson.md) and [`to_bjdata`](../api/basic_json/to_bjdata.md) were called with
`use_type = true` but `use_size = false`. UBJSON requires a size marker (`#`) after a type marker (`$`).
`use_type = true` but `use_size = false` for a value that contains a non-empty array, object or binary value. UBJSON
requires a size marker (`#`) after a type marker (`$`); scalars and empty containers do not throw.
!!! failure "Example message"