mirror of
https://github.com/nlohmann/json.git
synced 2026-10-27 00:08:55 +00:00
Correct and complete the UBJSON documentation
Fix statements that disagree with the behavior (checked with probes): - `Z` is null; the unused marker is `N`. - `$` requires `#`; use_type with use_size=false throws other_error.502 only for non-empty arrays, objects and binary values. - NaN and infinity are written as IEEE 754 float64 and read back unchanged; the UBJSON specification encodes infinity as null. - `H` is documented (semantics, parse_error.115, out_of_range.406); the "complete mapping" claim is qualified. - use_type "same type" means same marker. - Long Z/T/F arrays lose only the `$` annotation; the `#` count and the element markers are kept. - from_ubjson lists the 112, 113, 115, 406 and 408 causes that occur for UBJSON, without the BJData-only ND-array wording. - Duplicate object keys keep the last value; the `$` type marker is validated even for count 0. - Smaller fixes: float32 row, "map" -> "object", https link, exception 110 strict case, to_ubjson history of discarded values. Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
4 files changed
+63
-21
No files matched your search
@@ -5,11 +5,11 @@ achieve the generality of JSON, combined with being much easier to process than
|
||||
|
||||
!!! abstract "References"
|
||||
|
||||
- [UBJSON Website](http://ubjson.org)
|
||||
- [UBJSON Website](https://ubjson.org)
|
||||
|
||||
## Byte layout
|
||||
|
||||
UBJSON is built from just a one-byte, human-readable ASCII *type marker* (for example, `i` for int8, `S` for string, `[` for array) followed by the payload of that type. Strings and the like are prefixed with their length, which is itself a UBJSON integer value including its marker. Numbers are stored in big-endian byte order. Containers can be written in two ways: as an *unoptimized* container that is opened with `[` or `{` and closed with `]` or `}`, or as an *optimized* container that declares an element type (`$`) and/or a count (`#`) after the opening marker, which saves per-element markers and (for a count) the closing marker.
|
||||
UBJSON is built from just a one-byte, human-readable ASCII *type marker* (for example, `i` for int8, `S` for string, `[` for array) followed by the payload of that type. Strings and the like are prefixed with their length, which is itself a UBJSON integer value including its marker. Numbers are stored in big-endian byte order. Containers can be written in two ways: as an *unoptimized* container that is opened with `[` or `{` and closed with `]` or `}`, or as an *optimized* container that declares a count (`#`) after the opening marker, optionally preceded by an element type (`$`); an element type is only allowed together with a count. An optimized container saves the closing marker and, with a type, the per-element markers.
|
||||
|
||||
Consider the following JSON value (shown formatted for readability):
|
||||
|
||||
@@ -63,10 +63,10 @@ The library uses the following mapping from JSON values types to UBJSON types ac
|
||||
| number_unsigned | 32768..2147483647 | int32 | `l` |
|
||||
| number_unsigned | 2147483648..9223372036854775807 | int64 | `L` |
|
||||
| number_unsigned | 9223372036854775808..18446744073709551615 | high-precision | `H` |
|
||||
| number_float | *any value* | float64 | `D` |
|
||||
| number_float | *any value* | float64 (float32 if `number_float_t` is `float`) | `D` (`d`) |
|
||||
| string | *with shortest length indicator* | string | `S` |
|
||||
| array | *see notes on optimized format* | array | `[` |
|
||||
| object | *see notes on optimized format* | map | `{` |
|
||||
| object | *see notes on optimized format* | object | `{` |
|
||||
|
||||
!!! success "Complete mapping"
|
||||
|
||||
@@ -93,20 +93,27 @@ The library uses the following mapping from JSON values types to UBJSON types ac
|
||||
|
||||
The following markers are not used in the conversion:
|
||||
|
||||
- `Z`: no-op values are not created.
|
||||
- `N`: no-op values are never written (they are skipped when reading).
|
||||
- `C`: single-byte strings are serialized with `S` markers.
|
||||
|
||||
!!! info "NaN/infinity handling"
|
||||
|
||||
If NaN or Infinity are stored inside a JSON number, they are serialized properly. This behavior differs from the
|
||||
[`dump()`](../../api/basic_json/dump.md) function which serializes NaN or Infinity to `null`.
|
||||
NaN and infinity are written as IEEE 754 float64 values (`D`) and read back unchanged. This differs from
|
||||
[`dump()`](../../api/basic_json/dump.md), which serializes them to `null`, and from the UBJSON specification, which
|
||||
encodes infinity as `null` (it does not mention NaN); other UBJSON readers may therefore not accept these values.
|
||||
|
||||
!!! info "Optimized formats"
|
||||
|
||||
The optimized formats for containers are supported: Parameter `use_size` adds size information to the beginning of a
|
||||
container and removes the closing marker. Parameter `use_type` further checks whether all elements of a container
|
||||
have the same type and adds the type marker to the beginning of the container. The `use_type` parameter must only be
|
||||
used together with `use_size = true`.
|
||||
used together with `use_size = true`: combining `use_type` with `use_size = false` throws
|
||||
[`other_error.502`](../../home/exceptions.md#jsonexceptionother_error502) if the value contains a non-empty array,
|
||||
object or binary value.
|
||||
|
||||
Two elements have the same type if they are written with the same type marker. For example, `[1, 2]` is written with
|
||||
the type `i`, but `[1, 200]` (`i`, `U`), `[true, false]` (`T`, `F`) and `[1, 2.5]` (`i`, `D`) are not; empty
|
||||
containers (`[]`, `{}`) are never typed.
|
||||
|
||||
Note that `use_size = true` alone may result in larger representations - the benefit of this parameter is that the
|
||||
receiving side is immediately informed on the number of elements of the container.
|
||||
@@ -115,8 +122,9 @@ The library uses the following mapping from JSON values types to UBJSON types ac
|
||||
already is the value. Its declared count is therefore the only thing that decides how much memory the receiving side
|
||||
allocates, and a handful of bytes can describe billions of elements. `from_ubjson` rejects such an array with
|
||||
[`out_of_range.408`](../../home/exceptions.md#jsonexceptionout_of_range408) when the count exceeds 1,048,576
|
||||
(`1 << 20`), and `to_ubjson` writes longer arrays of these types without the annotation, so any value it produces
|
||||
can be read back.
|
||||
(`1 << 20`), and `to_ubjson` writes longer arrays of these types in the counted but untyped form: the `$` type marker is
|
||||
dropped, the `#` count is kept, and every element is written with its own marker (`[#l 1048577 Z Z ...`), so any value
|
||||
it produces can be read back. Objects and untyped counted arrays are not subject to this limit.
|
||||
|
||||
!!! info "Binary values"
|
||||
|
||||
@@ -155,12 +163,38 @@ The library maps UBJSON types to JSON value types as follows:
|
||||
| int64 | number_integer | `L` |
|
||||
| string | string | `S` |
|
||||
| char | string | `C` |
|
||||
| high-precision | number_unsigned, number_integer or number_float | `H` |
|
||||
| array | array (optimized values are supported) | `[` |
|
||||
| object | object (optimized values are supported) | `{` |
|
||||
|
||||
!!! info "High-precision numbers"
|
||||
|
||||
The payload of `H` is a length-prefixed string that is parsed as a JSON number (surrounding whitespace is ignored).
|
||||
Integer text in the range of `number_unsigned_t` (non-negative) or `number_integer_t` (negative) is stored as such;
|
||||
any other valid number text (larger or smaller integers, fractions, exponents such as `1e2`) is stored as
|
||||
`number_float`, so precision may be lost. Text that is not a valid JSON number raises
|
||||
[`parse_error.115`](../../home/exceptions.md#jsonexceptionparse_error115), a number outside the range of
|
||||
`number_float_t` (`1e999`) raises [`out_of_range.406`](../../home/exceptions.md#jsonexceptionout_of_range406).
|
||||
|
||||
!!! success "Complete mapping"
|
||||
|
||||
The mapping is **complete** in the sense that any UBJSON value can be converted to a JSON value.
|
||||
Every well-formed UBJSON value can be converted to a JSON value, with these exceptions: a high-precision number
|
||||
outside the range of `number_float_t` ([`out_of_range.406`](../../home/exceptions.md#jsonexceptionout_of_range406)),
|
||||
an optimized array of `Z`, `T` or `F` with more than 1,048,576 elements, and a count that exceeds the capacity of the
|
||||
container ([`out_of_range.408`](../../home/exceptions.md#jsonexceptionout_of_range408)). High-precision numbers that
|
||||
do not fit an integer type are stored as floating-point numbers and may lose precision. Ill-formed input is rejected
|
||||
with a parse error.
|
||||
|
||||
!!! info "Duplicate object keys"
|
||||
|
||||
If an object contains the same key more than once, the last value is kept (the UBJSON specification does not
|
||||
address duplicate keys).
|
||||
|
||||
!!! info "Type marker of optimized containers"
|
||||
|
||||
The type marker after `$` must be one of `Z`, `N`, `T`, `F`, `i`, `U`, `I`, `l`, `L`, `d`, `D`, `H`, `C`, `S`, `[` or
|
||||
`{`. It is checked even if the count is 0; any other marker raises
|
||||
[`parse_error.112`](../../home/exceptions.md#jsonexceptionparse_error112).
|
||||
|
||||
!!! warning "Ill-formed UTF-8 in string values and object keys"
|
||||
|
||||
|
||||
Reference in new issue
Block a user