mirror of
https://github.com/nlohmann/json.git
synced 2026-10-25 23:38:58 +00:00
Correct and complete the UBJSON documentation
Fix statements that disagree with the behavior (checked with probes): - `Z` is null; the unused marker is `N`. - `$` requires `#`; use_type with use_size=false throws other_error.502 only for non-empty arrays, objects and binary values. - NaN and infinity are written as IEEE 754 float64 and read back unchanged; the UBJSON specification encodes infinity as null. - `H` is documented (semantics, parse_error.115, out_of_range.406); the "complete mapping" claim is qualified. - use_type "same type" means same marker. - Long Z/T/F arrays lose only the `$` annotation; the `#` count and the element markers are kept. - from_ubjson lists the 112, 113, 115, 406 and 408 causes that occur for UBJSON, without the BJData-only ND-array wording. - Duplicate object keys keep the last value; the `$` type marker is validated even for count 0. - Smaller fixes: float32 row, "map" -> "object", https link, exception 110 strict case, to_ubjson history of discarded values. Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
4 files changed
+63
-21
No files matched your search
@@ -79,11 +79,18 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
|
||||
|
||||
- Throws [parse_error.110](../../home/exceptions.md#jsonexceptionparse_error110) if the given input ends prematurely or
|
||||
the end of the file was not reached when `strict` was set to true
|
||||
- Throws [parse_error.112](../../home/exceptions.md#jsonexceptionparse_error112) if a parse error occurs
|
||||
- Throws [parse_error.112](../../home/exceptions.md#jsonexceptionparse_error112) if an invalid type marker is read
|
||||
(including a type marker after `$` of an optimized container, even if its count is 0), or if `$` is not followed by `#`
|
||||
- Throws [parse_error.113](../../home/exceptions.md#jsonexceptionparse_error113) if a string could not be parsed
|
||||
successfully, or if a string value or object key is not valid UTF-8 and `error_handler` is `strict`
|
||||
- Throws [out_of_range.408](../../home/exceptions.md#jsonexceptionout_of_range408) if the size of an optimized container
|
||||
or n-dimensional array cannot be represented by `std::size_t`
|
||||
successfully (negative or non-integer length, `C` followed by a byte above 0x7F), if the count of an optimized
|
||||
container is negative, or if a string value or object key is not valid UTF-8 and `error_handler` is `strict`
|
||||
- Throws [parse_error.115](../../home/exceptions.md#jsonexceptionparse_error115) if the text of a high-precision number
|
||||
(`H`) is not a valid JSON number
|
||||
- Throws [out_of_range.406](../../home/exceptions.md#jsonexceptionout_of_range406) if a high-precision number, or a
|
||||
float64 when `number_float_t` is `float`, does not fit into `number_float_t`
|
||||
- Throws [out_of_range.408](../../home/exceptions.md#jsonexceptionout_of_range408) if the count of an optimized
|
||||
container exceeds the capacity of the container (or of `std::size_t`), or if an optimized array of `Z`, `T` or `F` has
|
||||
more than 1,048,576 elements
|
||||
|
||||
## Complexity
|
||||
|
||||
|
||||
@@ -114,5 +114,5 @@ Linear in the size of the JSON value `j`.
|
||||
- Added `error_handler` parameter in version 3.13.0. Its default, `keep`, writes the bytes of a string or object key
|
||||
that is not valid UTF-8 unchanged, as before; `strict` (the default if
|
||||
[`JSON_STRICT_BINARY_UTF8`](../macros/json_strict_binary_utf8.md) is enabled) throws `type_error.316`.
|
||||
- Throws `type_error.321` for a discarded value since version 3.13.0; previously, a discarded value nested in an
|
||||
array or object was silently skipped, producing invalid UBJSON.
|
||||
- Throws `type_error.321` for a discarded value since version 3.13.0; previously, a discarded array element was
|
||||
skipped, and an object member was written as a key without a value, producing invalid UBJSON.
|
||||
@@ -5,11 +5,11 @@ achieve the generality of JSON, combined with being much easier to process than
|
||||
|
||||
!!! abstract "References"
|
||||
|
||||
- [UBJSON Website](http://ubjson.org)
|
||||
- [UBJSON Website](https://ubjson.org)
|
||||
|
||||
## Byte layout
|
||||
|
||||
UBJSON is built from just a one-byte, human-readable ASCII *type marker* (for example, `i` for int8, `S` for string, `[` for array) followed by the payload of that type. Strings and the like are prefixed with their length, which is itself a UBJSON integer value including its marker. Numbers are stored in big-endian byte order. Containers can be written in two ways: as an *unoptimized* container that is opened with `[` or `{` and closed with `]` or `}`, or as an *optimized* container that declares an element type (`$`) and/or a count (`#`) after the opening marker, which saves per-element markers and (for a count) the closing marker.
|
||||
UBJSON is built from just a one-byte, human-readable ASCII *type marker* (for example, `i` for int8, `S` for string, `[` for array) followed by the payload of that type. Strings and the like are prefixed with their length, which is itself a UBJSON integer value including its marker. Numbers are stored in big-endian byte order. Containers can be written in two ways: as an *unoptimized* container that is opened with `[` or `{` and closed with `]` or `}`, or as an *optimized* container that declares a count (`#`) after the opening marker, optionally preceded by an element type (`$`); an element type is only allowed together with a count. An optimized container saves the closing marker and, with a type, the per-element markers.
|
||||
|
||||
Consider the following JSON value (shown formatted for readability):
|
||||
|
||||
@@ -63,10 +63,10 @@ The library uses the following mapping from JSON values types to UBJSON types ac
|
||||
| number_unsigned | 32768..2147483647 | int32 | `l` |
|
||||
| number_unsigned | 2147483648..9223372036854775807 | int64 | `L` |
|
||||
| number_unsigned | 9223372036854775808..18446744073709551615 | high-precision | `H` |
|
||||
| number_float | *any value* | float64 | `D` |
|
||||
| number_float | *any value* | float64 (float32 if `number_float_t` is `float`) | `D` (`d`) |
|
||||
| string | *with shortest length indicator* | string | `S` |
|
||||
| array | *see notes on optimized format* | array | `[` |
|
||||
| object | *see notes on optimized format* | map | `{` |
|
||||
| object | *see notes on optimized format* | object | `{` |
|
||||
|
||||
!!! success "Complete mapping"
|
||||
|
||||
@@ -93,20 +93,27 @@ The library uses the following mapping from JSON values types to UBJSON types ac
|
||||
|
||||
The following markers are not used in the conversion:
|
||||
|
||||
- `Z`: no-op values are not created.
|
||||
- `N`: no-op values are never written (they are skipped when reading).
|
||||
- `C`: single-byte strings are serialized with `S` markers.
|
||||
|
||||
!!! info "NaN/infinity handling"
|
||||
|
||||
If NaN or Infinity are stored inside a JSON number, they are serialized properly. This behavior differs from the
|
||||
[`dump()`](../../api/basic_json/dump.md) function which serializes NaN or Infinity to `null`.
|
||||
NaN and infinity are written as IEEE 754 float64 values (`D`) and read back unchanged. This differs from
|
||||
[`dump()`](../../api/basic_json/dump.md), which serializes them to `null`, and from the UBJSON specification, which
|
||||
encodes infinity as `null` (it does not mention NaN); other UBJSON readers may therefore not accept these values.
|
||||
|
||||
!!! info "Optimized formats"
|
||||
|
||||
The optimized formats for containers are supported: Parameter `use_size` adds size information to the beginning of a
|
||||
container and removes the closing marker. Parameter `use_type` further checks whether all elements of a container
|
||||
have the same type and adds the type marker to the beginning of the container. The `use_type` parameter must only be
|
||||
used together with `use_size = true`.
|
||||
used together with `use_size = true`: combining `use_type` with `use_size = false` throws
|
||||
[`other_error.502`](../../home/exceptions.md#jsonexceptionother_error502) if the value contains a non-empty array,
|
||||
object or binary value.
|
||||
|
||||
Two elements have the same type if they are written with the same type marker. For example, `[1, 2]` is written with
|
||||
the type `i`, but `[1, 200]` (`i`, `U`), `[true, false]` (`T`, `F`) and `[1, 2.5]` (`i`, `D`) are not; empty
|
||||
containers (`[]`, `{}`) are never typed.
|
||||
|
||||
Note that `use_size = true` alone may result in larger representations - the benefit of this parameter is that the
|
||||
receiving side is immediately informed on the number of elements of the container.
|
||||
@@ -115,8 +122,9 @@ The library uses the following mapping from JSON values types to UBJSON types ac
|
||||
already is the value. Its declared count is therefore the only thing that decides how much memory the receiving side
|
||||
allocates, and a handful of bytes can describe billions of elements. `from_ubjson` rejects such an array with
|
||||
[`out_of_range.408`](../../home/exceptions.md#jsonexceptionout_of_range408) when the count exceeds 1,048,576
|
||||
(`1 << 20`), and `to_ubjson` writes longer arrays of these types without the annotation, so any value it produces
|
||||
can be read back.
|
||||
(`1 << 20`), and `to_ubjson` writes longer arrays of these types in the counted but untyped form: the `$` type marker is
|
||||
dropped, the `#` count is kept, and every element is written with its own marker (`[#l 1048577 Z Z ...`), so any value
|
||||
it produces can be read back. Objects and untyped counted arrays are not subject to this limit.
|
||||
|
||||
!!! info "Binary values"
|
||||
|
||||
@@ -155,12 +163,38 @@ The library maps UBJSON types to JSON value types as follows:
|
||||
| int64 | number_integer | `L` |
|
||||
| string | string | `S` |
|
||||
| char | string | `C` |
|
||||
| high-precision | number_unsigned, number_integer or number_float | `H` |
|
||||
| array | array (optimized values are supported) | `[` |
|
||||
| object | object (optimized values are supported) | `{` |
|
||||
|
||||
!!! info "High-precision numbers"
|
||||
|
||||
The payload of `H` is a length-prefixed string that is parsed as a JSON number (surrounding whitespace is ignored).
|
||||
Integer text in the range of `number_unsigned_t` (non-negative) or `number_integer_t` (negative) is stored as such;
|
||||
any other valid number text (larger or smaller integers, fractions, exponents such as `1e2`) is stored as
|
||||
`number_float`, so precision may be lost. Text that is not a valid JSON number raises
|
||||
[`parse_error.115`](../../home/exceptions.md#jsonexceptionparse_error115), a number outside the range of
|
||||
`number_float_t` (`1e999`) raises [`out_of_range.406`](../../home/exceptions.md#jsonexceptionout_of_range406).
|
||||
|
||||
!!! success "Complete mapping"
|
||||
|
||||
The mapping is **complete** in the sense that any UBJSON value can be converted to a JSON value.
|
||||
Every well-formed UBJSON value can be converted to a JSON value, with these exceptions: a high-precision number
|
||||
outside the range of `number_float_t` ([`out_of_range.406`](../../home/exceptions.md#jsonexceptionout_of_range406)),
|
||||
an optimized array of `Z`, `T` or `F` with more than 1,048,576 elements, and a count that exceeds the capacity of the
|
||||
container ([`out_of_range.408`](../../home/exceptions.md#jsonexceptionout_of_range408)). High-precision numbers that
|
||||
do not fit an integer type are stored as floating-point numbers and may lose precision. Ill-formed input is rejected
|
||||
with a parse error.
|
||||
|
||||
!!! info "Duplicate object keys"
|
||||
|
||||
If an object contains the same key more than once, the last value is kept (the UBJSON specification does not
|
||||
address duplicate keys).
|
||||
|
||||
!!! info "Type marker of optimized containers"
|
||||
|
||||
The type marker after `$` must be one of `Z`, `N`, `T`, `F`, `i`, `U`, `I`, `l`, `L`, `d`, `D`, `H`, `C`, `S`, `[` or
|
||||
`{`. It is checked even if the count is 0; any other marker raises
|
||||
[`parse_error.112`](../../home/exceptions.md#jsonexceptionparse_error112).
|
||||
|
||||
!!! warning "Ill-formed UTF-8 in string values and object keys"
|
||||
|
||||
|
||||
@@ -330,7 +330,7 @@ where they cannot be resolved.
|
||||
### json.exception.parse_error.110
|
||||
|
||||
When parsing a [binary format](../features/binary_formats/index.md), the byte vector ends before the complete value has
|
||||
been read.
|
||||
been read, or (with `strict = true`) further bytes follow the value.
|
||||
|
||||
!!! failure "Example messages"
|
||||
|
||||
@@ -1045,8 +1045,8 @@ or the encoded length for [CBOR](../features/binary_formats/cbor.md).
|
||||
The exception is also thrown for a [UBJSON](../features/binary_formats/ubjson.md) array of a type that is encoded by its
|
||||
marker alone (`Z`, `T` or `F`) whose declared count exceeds 1,048,576 (`1 << 20`). Such an array has no payload, so its
|
||||
count alone decides how much memory is allocated, and a handful of bytes would otherwise describe billions of values.
|
||||
[`to_ubjson`](../api/basic_json/to_ubjson.md) writes longer arrays of these types without the size and type annotation,
|
||||
so any value it produces can still be read back.
|
||||
[`to_ubjson`](../api/basic_json/to_ubjson.md) writes longer arrays of these types without the `$` type annotation (the `#`
|
||||
count and one marker per element are kept), so any value it produces can still be read back.
|
||||
|
||||
!!! failure "Example messages"
|
||||
|
||||
@@ -1229,7 +1229,8 @@ A JSON Patch operation 'test' failed. The unsuccessful operation is also printed
|
||||
### json.exception.other_error.502
|
||||
|
||||
[`to_ubjson`](../api/basic_json/to_ubjson.md) and [`to_bjdata`](../api/basic_json/to_bjdata.md) were called with
|
||||
`use_type = true` but `use_size = false`. UBJSON requires a size marker (`#`) after a type marker (`$`).
|
||||
`use_type = true` but `use_size = false` for a value that contains a non-empty array, object or binary value. UBJSON
|
||||
requires a size marker (`#`) after a type marker (`$`); scalars and empty containers do not throw.
|
||||
|
||||
!!! failure "Example message"
|
||||
|
||||
|
||||
Reference in new issue
Block a user