Fix narrowing of the optimized-container count in BJData and UBJSON tests

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

docs/mkdocs/docs/api/basic_json/binary_t.md

@@ -90,8 +90,9 @@ type `#!cpp binary_t*` must be dereferenced.
 
 - MessagePack
     - If a subtype is given and the binary array contains exactly 1, 2, 4, 8, or 16 elements, the fixext family (fixext1,
-      fixext2, fixext4, fixext8) is used. For other sizes, the ext family (ext8, ext16, ext32) is used. The subtype is
-      then added as a signed 8-bit integer.
+      fixext2, fixext4, fixext8, fixext16) is used. For other sizes, the ext family (ext8, ext16, ext32) is used. The
+      subtype is then written as the signed 8-bit ext type: subtypes 128 to 255 map to the negative (reserved) ext
+      types, and on reading the ext type is exposed as an unsigned subtype from 0 to 255.
     - If no subtype is given, the bin family (bin8, bin16, bin32) is used.
 
 - BSON

docs/mkdocs/docs/api/basic_json/from_ubjson.md

@@ -79,11 +79,18 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
 
 - Throws [parse_error.110](../../home/exceptions.md#jsonexceptionparse_error110) if the given input ends prematurely or
   the end of the file was not reached when `strict` was set to true
-- Throws [parse_error.112](../../home/exceptions.md#jsonexceptionparse_error112) if a parse error occurs
+- Throws [parse_error.112](../../home/exceptions.md#jsonexceptionparse_error112) if an invalid type marker is read
+  (including a type marker after `$` of an optimized container, even if its count is 0), or if `$` is not followed by `#`
 - Throws [parse_error.113](../../home/exceptions.md#jsonexceptionparse_error113) if a string could not be parsed
-  successfully, or if a string value or object key is not valid UTF-8 and `error_handler` is `strict`
-- Throws [out_of_range.408](../../home/exceptions.md#jsonexceptionout_of_range408) if the size of an optimized container
-  or n-dimensional array cannot be represented by `std::size_t`
+  successfully (negative or non-integer length, `C` followed by a byte above 0x7F), if the count of an optimized
+  container is negative, or if a string value or object key is not valid UTF-8 and `error_handler` is `strict`
+- Throws [parse_error.115](../../home/exceptions.md#jsonexceptionparse_error115) if the text of a high-precision number
+  (`H`) is not a valid JSON number
+- Throws [out_of_range.406](../../home/exceptions.md#jsonexceptionout_of_range406) if a high-precision number, or a
+  float64 when `number_float_t` is `float`, does not fit into `number_float_t`
+- Throws [out_of_range.408](../../home/exceptions.md#jsonexceptionout_of_range408) if the count of an optimized
+  container exceeds the capacity of the container (or of `std::size_t`), or if an optimized array of `Z`, `T`, or `F` has
+  more than 1,048,576 elements
 
 ## Complexity
 

docs/mkdocs/docs/api/basic_json/to_msgpack.md

@@ -53,7 +53,7 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
   value, array, or object exceeds 4294967295, the maximum MessagePack can store; example:
   `"MessagePack length 4294967296 exceeds maximum of 4294967295"`
 - Throws [`out_of_range.415`](../../home/exceptions.md#jsonexceptionout_of_range415) if the subtype of a binary value
-  exceeds 255, the maximum of the MessagePack ext type; example:
+  exceeds 255, the largest value the library can store in the one-byte MessagePack ext type; example:
   `"subtype 70000 is too large for the MessagePack ext type (max 255)"`
 - Throws [type_error.316](../../home/exceptions.md#jsonexceptiontype_error316) if a string or object key in `j` is
   not valid UTF-8 and `error_handler` is `strict`

docs/mkdocs/docs/features/binary_formats/index.md

@@ -22,7 +22,7 @@ to efficiently encode JSON values to byte vectors and to decode such vectors.
 | BON8        | incomplete: no unsigned integers above int64  | complete                                     |
 | BSON        | incomplete: top-level value must be an object | incomplete, but all JSON types are supported |
 | CBOR        | complete                                      | incomplete, but all JSON types are supported |
-| MessagePack | complete                                      | complete                                     |
+| MessagePack | complete                                      | incomplete, but all JSON types are supported |
 | UBJSON      | complete                                      | complete                                     |
 
 ### Binary values

docs/mkdocs/docs/features/binary_formats/messagepack.md

@@ -11,7 +11,7 @@ extra byte in addition to the strings themselves.
 
 ## Byte layout
 
-MessagePack encodes each value as a one-byte *format marker*, optionally followed by a length or the payload. The marker range is partitioned to keep common values small: small non-negative integers (0 to 127) and small negative integers (-32 to -1) are the marker byte itself; short strings, arrays, and maps (up to 31 or 15 elements) hide their length in the marker's lower bits. Larger values use dedicated markers that announce the width (8, 16, 32, or 64 bits) of the length or number that follows, in big-endian byte order. Like CBOR, containers are length-prefixed, so no terminator is needed.
+MessagePack encodes each value as a one-byte *format marker*, optionally followed by a length or the payload. The marker range is partitioned to keep common values small: small non-negative integers (0 to 127) and small negative integers (-32 to -1) are the marker byte itself; short strings, arrays, and maps (up to 31 bytes for strings and up to 15 elements for arrays and maps) hide their length in the marker's lower bits. Larger values use dedicated markers that announce the width (8, 16, 32, or 64 bits) of the length or number that follows, in big-endian byte order. Like CBOR, containers are length-prefixed, so no terminator is needed.
 
 Consider the following JSON value (shown formatted for readability):
 
@@ -109,8 +109,9 @@ specification:
 !!! note
 
     Prior to version 3.13.0, NaN and Infinity were instead serialized as a MessagePack float 64 (type 0xCB, 9 bytes
-    total), because the check used to select the smaller float 32 encoding compared magnitudes with NaN, which is
-    always `false` and caused the float 32 path to be skipped.
+    total), because the check used to select the smaller float 32 encoding failed for NaN (all comparisons are `false`)
+    and for Infinity and -Infinity (outside the float range checked against `std::numeric_limits<float>::lowest()` and
+    `max()`), so the float 32 path was skipped.
 
 ??? example "Example: serialize a JSON value to MessagePack"
 
@@ -135,6 +136,7 @@ The library maps MessagePack types to JSON value types as follows:
 | fixarray         | array           | 0x90..0x9F |
 | fixstr           | string          | 0xA0..0xBF |
 | nil              | `null`          | 0xC0       |
+| (never used)     | –               | 0xC1       |
 | false            | `false`         | 0xC2       |
 | true             | `true`          | 0xC3       |
 | float 32         | number_float    | 0xCA       |
@@ -167,6 +169,30 @@ The library maps MessagePack types to JSON value types as follows:
 | fixext 16        | binary          | 0xD8       |
 | negative fixint  | number_integer  | 0xE0-0xFF  |
 
+The byte 0xC1 is reserved as "never used" by the specification. As a value it is rejected with
+[`parse_error.112`](../../home/exceptions.md#jsonexceptionparse_error112) (`invalid byte: 0xC1`); as a map key it is
+rejected with [`parse_error.113`](../../home/exceptions.md#jsonexceptionparse_error113) and the generic "expected length
+specification" message, because the key is read as a string.
+
+Duplicate keys in a map are accepted; the last value wins.
+
+### Extension types (ext, fixext)
+
+MessagePack ext and fixext values are read as binary values, and the ext type byte is stored as the binary subtype.
+Although the specification defines the type as a signed 8-bit integer (0 to 127 for applications, -128 to -1 reserved,
+with -1 being the Timestamp extension), the library reads it as an unsigned byte: a negative ext type `n` becomes
+subtype `256+n`, so ext type -1 is read as subtype 255 and -128 as subtype 128. The Timestamp extension is not
+interpreted; it is returned as binary with subtype 255. Use
+[`has_subtype()`](../../api/byte_container_with_subtype/has_subtype.md) to tell a `bin` value (no subtype) from an ext
+value (subtype, even if it is 0).
+
+When serializing, a binary value with a subtype is always written as fixext (1, 2, 4, 8, or 16 bytes) or ext, even
+for subtype 0 (`d6 00 01 02 03 04`); a binary value without a subtype is written as bin. Subtypes 128 to 255 are
+written as the reserved negative ext types (255 as -1), so `json::binary({1,2,3,4}, 255)` becomes
+`d6 ff 01 02 03 04`, which other MessagePack libraries decode as a 32-bit timestamp. A subtype above 255 throws
+[`out_of_range.415`](../../home/exceptions.md#jsonexceptionout_of_range415). The payload length is limited to
+4294967295 bytes like all other lengths.
+
 !!! info
 
     Any MessagePack output created by `to_msgpack` can be successfully parsed by `from_msgpack`.
@@ -177,7 +203,8 @@ The library maps MessagePack types to JSON value types as follows:
     JSON-compatible [profile](https://github.com/msgpack/msgpack/blob/master/spec.md#profile) sketched in the
     MessagePack specification, this library restricts map keys to `str` values. Maps with keys of any other type are
     rejected with a [`parse_error.113`](../../home/exceptions.md#jsonexceptionparse_error113) exception (or, with
-    `allow_exceptions` set to `false`, a discarded value) naming the type of the key that was found, for instance:
+    `allow_exceptions` set to `false`, a discarded value). For most key types the message names the type that was
+    found, for instance:
 
     ```
     [json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack object key: only string keys are supported, but found nil; last byte: 0xC0
@@ -189,7 +216,7 @@ The library maps MessagePack types to JSON value types as follows:
 !!! warning "Ill-formed UTF-8 in string values"
 
     The MessagePack specification explicitly allows a `str` value (`fixstr`, `str 8`, `str 16`, `str 32`) to contain
-    a byte sequence that is not valid UTF-8, and expects a deserializer to hand the original bytes back unchanged.
+    a byte sequence that is not valid UTF-8, and recommends that a deserializer provide a way to obtain the original bytes.
     This library follows that by default: with its
     [`error_handler`](../../api/basic_json/from_msgpack.md) parameter left at `keep` (the default),
     `from_msgpack()` reads `str` bytes (object keys included) as-is, without validating them, so such a value

docs/mkdocs/docs/features/binary_formats/ubjson.md

@@ -5,11 +5,11 @@ achieve the generality of JSON, combined with being much easier to process than
 
 !!! abstract "References"
 
-    - [UBJSON Website](http://ubjson.org)
+    - [UBJSON Website](https://ubjson.org)
 
 ## Byte layout
 
-UBJSON is built from just a one-byte, human-readable ASCII *type marker* (for example, `i` for int8, `S` for string, `[` for array) followed by the payload of that type. Strings and the like are prefixed with their length, which is itself a UBJSON integer value including its marker. Numbers are stored in big-endian byte order. Containers can be written in two ways: as an *unoptimized* container that is opened with `[` or `{` and closed with `]` or `}`, or as an *optimized* container that declares an element type (`$`) and/or a count (`#`) after the opening marker, which saves per-element markers and (for a count) the closing marker.
+UBJSON is built from just a one-byte, human-readable ASCII *type marker* (for example, `i` for int8, `S` for string, `[` for array) followed by the payload of that type. Strings and the like are prefixed with their length, which is itself a UBJSON integer value including its marker. Numbers are stored in big-endian byte order. Containers can be written in two ways: as an *unoptimized* container that is opened with `[` or `{` and closed with `]` or `}`, or as an *optimized* container that declares a count (`#`) after the opening marker, optionally preceded by an element type (`$`); an element type is only allowed together with a count. An optimized container saves the closing marker and, with a type, the per-element markers.
 
 Consider the following JSON value (shown formatted for readability):
 
@@ -45,7 +45,7 @@ Consider the following JSON value (shown formatted for readability):
 The library uses the following mapping from JSON values types to UBJSON types according to the UBJSON specification:
 
 | JSON value type | value/range                               | UBJSON type                                      | marker    |
-|-----------------|-------------------------------------------|----------------|--------|
+|-----------------|-------------------------------------------|--------------------------------------------------|-----------|
 | null            | `null`                                    | null                                             | `Z`       |
 | boolean         | `true`                                    | true                                             | `T`       |
 | boolean         | `false`                                   | false                                            | `F`       |
@@ -63,10 +63,10 @@ The library uses the following mapping from JSON values types to UBJSON types ac
 | number_unsigned | 32768..2147483647                         | int32                                            | `l`       |
 | number_unsigned | 2147483648..9223372036854775807           | int64                                            | `L`       |
 | number_unsigned | 9223372036854775808..18446744073709551615 | high-precision                                   | `H`       |
-| number_float    | *any value*                               | float64        | `D`    |
+| number_float    | *any value*                               | float64 (float32 if `number_float_t` is `float`) | `D` (`d`) |
 | string          | *with shortest length indicator*          | string                                           | `S`       |
 | array           | *see notes on optimized format*           | array                                            | `[`       |
-| object          | *see notes on optimized format*           | map            | `{`    |
+| object          | *see notes on optimized format*           | object                                           | `{`       |
 
 !!! success "Complete mapping"
 
@@ -93,20 +93,27 @@ The library uses the following mapping from JSON values types to UBJSON types ac
 
     The following markers are not used in the conversion:
     
-    - `Z`: no-op values are not created.
+    - `N`: no-op values are never written (they are skipped when reading).
     - `C`: single-byte strings are serialized with `S` markers.
 
 !!! info "NaN/infinity handling"
 
-    If NaN or Infinity are stored inside a JSON number, they are serialized properly. This behavior differs from the
-    [`dump()`](../../api/basic_json/dump.md) function which serializes NaN or Infinity to `null`.
+    NaN and infinity are written as IEEE 754 float64 values (`D`) and read back unchanged. This differs from
+    [`dump()`](../../api/basic_json/dump.md), which serializes them to `null`, and from the UBJSON specification, which
+    encodes infinity as `null` (it does not mention NaN); other UBJSON readers may therefore not accept these values.
 
 !!! info "Optimized formats"
 
     The optimized formats for containers are supported: Parameter `use_size` adds size information to the beginning of a
     container and removes the closing marker. Parameter `use_type` further checks whether all elements of a container
     have the same type and adds the type marker to the beginning of the container. The `use_type` parameter must only be
-    used together with `use_size = true`.
+    used together with `use_size = true`: combining `use_type` with `use_size = false` throws
+    [`other_error.502`](../../home/exceptions.md#jsonexceptionother_error502) if the value contains a non-empty array,
+    object, or binary value.
+
+    Two elements have the same type if they are written with the same type marker. For example, `[1, 2]` is written with
+    the type `i`, but `[1, 200]` (`i`, `U`), `[true, false]` (`T`, `F`), and `[1, 2.5]` (`i`, `D`) are not; an empty
+    container is never given a `$` annotation of its own (an array of empty arrays is still typed `[`).
 
     Note that `use_size = true` alone may result in larger representations - the benefit of this parameter is that the
     receiving side is immediately informed on the number of elements of the container.
@@ -115,8 +122,9 @@ The library uses the following mapping from JSON values types to UBJSON types ac
     already is the value. Its declared count is therefore the only thing that decides how much memory the receiving side
     allocates, and a handful of bytes can describe billions of elements. `from_ubjson` rejects such an array with
     [`out_of_range.408`](../../home/exceptions.md#jsonexceptionout_of_range408) when the count exceeds 1,048,576
-    (`1 << 20`), and `to_ubjson` writes longer arrays of these types without the annotation, so any value it produces
-    can be read back.
+    (`1 << 20`), and `to_ubjson` writes longer arrays of these types in the counted but untyped form: the `$` type marker is
+    dropped, the `#` count is kept, and every element is written with its own marker (`[#l 1048577 Z Z ...`), so any value
+    it produces can be read back. Objects and untyped counted arrays are not subject to this limit.
 
 !!! info "Binary values"
 
@@ -141,7 +149,7 @@ The library uses the following mapping from JSON values types to UBJSON types ac
 The library maps UBJSON types to JSON value types as follows:
 
 | UBJSON type    | JSON value type                                  | marker |
-|-------------|-----------------------------------------|--------|
+|----------------|--------------------------------------------------|--------|
 | no-op          | *no value, next value is read*                   | `N`    |
 | null           | `null`                                           | `Z`    |
 | false          | `false`                                          | `F`    |
@@ -155,12 +163,39 @@ The library maps UBJSON types to JSON value types as follows:
 | int64          | number_integer                                   | `L`    |
 | string         | string                                           | `S`    |
 | char           | string                                           | `C`    |
+| high-precision | number_unsigned, number_integer, or number_float | `H`    |
 | array          | array (optimized values are supported)           | `[`    |
 | object         | object (optimized values are supported)          | `{`    |
 
+!!! info "High-precision numbers"
+
+    The payload of `H` is a length-prefixed string that is parsed as a JSON number (surrounding whitespace is ignored).
+    Integer text in the range of `number_unsigned_t` (non-negative) or `number_integer_t` (negative) is stored as such;
+    any other valid number text (larger or smaller integers, fractions, and exponents such as `1e2`) is stored as
+    `number_float`, so precision may be lost. Text that is not a valid JSON number raises
+    [`parse_error.115`](../../home/exceptions.md#jsonexceptionparse_error115), a number outside the range of
+    `number_float_t` (`1e999`) raises [`out_of_range.406`](../../home/exceptions.md#jsonexceptionout_of_range406).
+
 !!! success "Complete mapping"
 
-    The mapping is **complete** in the sense that any UBJSON value can be converted to a JSON value.
+    Every well-formed UBJSON value can be converted to a JSON value, with these exceptions: a high-precision number
+    outside the range of `number_float_t` ([`out_of_range.406`](../../home/exceptions.md#jsonexceptionout_of_range406)),
+    an optimized array of `Z`, `T`, or `F` with more than 1,048,576 elements, and a count that exceeds the capacity of the
+    container ([`out_of_range.408`](../../home/exceptions.md#jsonexceptionout_of_range408)). High-precision numbers that
+    do not fit an integer type are stored as floating-point numbers and may lose precision. Ill-formed input is rejected
+    with a parse error.
+
+!!! info "Duplicate object keys"
+
+    If an object contains the same key more than once, the last value is kept (the UBJSON specification does not
+    address duplicate keys).
+
+!!! info "Type marker of optimized containers"
+
+    The type marker after `$` must be one of `Z`, `N`, `T`, `F`, `i`, `U`, `I`, `l`, `L`, `d`, `D`, `H`, `C`, `S`, `[`, or
+    `{`. It is checked even if the count is 0; any other marker raises
+    [`parse_error.112`](../../home/exceptions.md#jsonexceptionparse_error112). A counted object typed `N` (`{$N#i1 ...`)
+    is not supported.
 
 !!! warning "Ill-formed UTF-8 in string values and object keys"
 

docs/mkdocs/docs/features/binary_values.md

@@ -324,8 +324,9 @@ representation using the length of the byte array.
 ### MessagePack
 
 [MessagePack](binary_formats/messagepack.md) supports binary values and subtypes. If a subtype is given, the ext family
-is used. The library will choose the smallest representation among fixext1, fixext2, fixext4, fixext8, ext8, ext16, and
-ext32. The subtype is then added as a signed 8-bit integer.
+is used. The library will choose the smallest representation among fixext1, fixext2, fixext4, fixext8, fixext16, ext8,
+ext16, and ext32. The subtype is then written as the signed 8-bit ext type: subtypes 128 to 255 map to the negative
+(reserved) ext types, and on reading the ext type is exposed as an unsigned subtype from 0 to 255.
 
 If no subtype is given, the bin family (bin8, bin16, bin32) is used.
 

docs/mkdocs/docs/home/exceptions.md

@@ -330,7 +330,7 @@ where they cannot be resolved.
 ### json.exception.parse_error.110
 
 When parsing a [binary format](../features/binary_formats/index.md), the byte vector ends before the complete value has
-been read.
+been read, or (with `strict = true`) further bytes follow the value.
 
 !!! failure "Example messages"
 
@@ -1045,8 +1045,8 @@ or the encoded length for [CBOR](../features/binary_formats/cbor.md).
 The exception is also thrown for a [UBJSON](../features/binary_formats/ubjson.md) array of a type that is encoded by its
 marker alone (`Z`, `T` or `F`) whose declared count exceeds 1,048,576 (`1 << 20`). Such an array has no payload, so its
 count alone decides how much memory is allocated, and a handful of bytes would otherwise describe billions of values.
-[`to_ubjson`](../api/basic_json/to_ubjson.md) writes longer arrays of these types without the size and type annotation,
-so any value it produces can still be read back.
+[`to_ubjson`](../api/basic_json/to_ubjson.md) writes longer arrays of these types without the `$` type annotation (the `#`
+count and one marker per element are kept), so any value it produces can still be read back.
 
 !!! failure "Example messages"
 
@@ -1153,7 +1153,7 @@ A JSON Patch `move` operation's `"from"` location is a proper prefix of its `"pa
 ### json.exception.out_of_range.415
 
 MessagePack's ext type and BSON's binary subtype are each stored in a single byte. This exception is thrown when serializing a
-[`byte_container_with_subtype`](../api/byte_container_with_subtype/index.md) whose subtype exceeds 255.
+[`byte_container_with_subtype`](../api/byte_container_with_subtype/index.md) whose subtype exceeds 255, the largest value the library can store in that byte.
 
 !!! failure "Example message"
 
@@ -1229,7 +1229,8 @@ A JSON Patch operation 'test' failed. The unsuccessful operation is also printed
 ### json.exception.other_error.502
 
 [`to_ubjson`](../api/basic_json/to_ubjson.md) and [`to_bjdata`](../api/basic_json/to_bjdata.md) were called with
-`use_type = true` but `use_size = false`. UBJSON requires a size marker (`#`) after a type marker (`$`).
+`use_type = true` but `use_size = false` for a value that contains a non-empty array, object, or binary value. UBJSON
+requires a size marker (`#`) after a type marker (`$`); scalars and empty containers do not throw.
 
 !!! failure "Example message"
 

include/nlohmann/detail/input/binary_reader.hpp

@@ -3000,6 +3000,14 @@ class binary_reader
                 return false;
             }
 
+            // validate the type marker up front: otherwise it would only be
+            // checked when an element is read, and a container with count 0
+            // would accept any marker
+            if (JSON_HEDLEY_UNLIKELY(!is_valid_optimized_type(result.second)))
+            {
+                return optimized_type_error();
+            }
+
             get_ignore_noop();
             if (JSON_HEDLEY_UNLIKELY(current != '#'))
             {
@@ -4496,6 +4504,44 @@ class binary_reader
     std::size_t bon8_pushback_size = 0;
 
   JSON_PRIVATE_UNLESS_TESTED:
+    /*!
+    @brief check whether @a marker is a type marker that may follow '$'
+    UBJSON: Z N T F i U I l L d D H C S [ {; BJData: the numeric markers and C
+    (BJData's excluded markers are rejected separately).
+    */
+    bool is_valid_optimized_type(const char_int_type marker) const noexcept
+    {
+        switch (marker)
+        {
+            case 'i':
+            case 'U':
+            case 'I':
+            case 'l':
+            case 'L':
+            case 'd':
+            case 'D':
+            case 'C':
+                return true;
+            case 'Z':
+            case 'N':
+            case 'T':
+            case 'F':
+            case 'H':
+            case 'S':
+            case '[':
+            case '{':
+                return input_format != input_format_t::bjdata;
+            case 'B':
+            case 'u':
+            case 'm':
+            case 'M':
+            case 'h':
+                return input_format == input_format_t::bjdata;
+            default:
+                return false;
+        }
+    }
+
     /*!
     @brief whether @a marker is excluded from BJData's optimized ND-array types
     @return whether @a marker is one of 'F', 'H', 'N', 'S', 'T', 'Z', '[', '{'

single_include/nlohmann/json.hpp

@@ -20467,6 +20467,14 @@ class binary_reader
                 return false;
             }
 
+            // validate the type marker up front: otherwise it would only be
+            // checked when an element is read, and a container with count 0
+            // would accept any marker
+            if (JSON_HEDLEY_UNLIKELY(!is_valid_optimized_type(result.second)))
+            {
+                return optimized_type_error();
+            }
+
             get_ignore_noop();
             if (JSON_HEDLEY_UNLIKELY(current != '#'))
             {
@@ -21963,6 +21971,44 @@ class binary_reader
     std::size_t bon8_pushback_size = 0;
 
   JSON_PRIVATE_UNLESS_TESTED:
+    /*!
+    @brief check whether @a marker is a type marker that may follow '$'
+    UBJSON: Z N T F i U I l L d D H C S [ {; BJData: the numeric markers and C
+    (BJData's excluded markers are rejected separately).
+    */
+    bool is_valid_optimized_type(const char_int_type marker) const noexcept
+    {
+        switch (marker)
+        {
+            case 'i':
+            case 'U':
+            case 'I':
+            case 'l':
+            case 'L':
+            case 'd':
+            case 'D':
+            case 'C':
+                return true;
+            case 'Z':
+            case 'N':
+            case 'T':
+            case 'F':
+            case 'H':
+            case 'S':
+            case '[':
+            case '{':
+                return input_format != input_format_t::bjdata;
+            case 'B':
+            case 'u':
+            case 'm':
+            case 'M':
+            case 'h':
+                return input_format == input_format_t::bjdata;
+            default:
+                return false;
+        }
+    }
+
     /*!
     @brief whether @a marker is excluded from BJData's optimized ND-array types
     @return whether @a marker is one of 'F', 'H', 'N', 'S', 'T', 'Z', '[', '{'

tests/src/unit-bjdata.cpp

@@ -3134,7 +3134,7 @@ TEST_CASE("BJData")
                 std::vector<uint8_t> const v1 = {'[', '#', 'i', 0xF1};
                 std::vector<uint8_t> const v2 = {'[', '$', 'I', '#', 'i', 0xF2};
                 std::vector<uint8_t> const v3 = {'[', '$', 'I', '#', '[', 'i', 0xF4, 'i', 0x02, ']'};
-                std::vector<uint8_t> const v4 = {'[', '$', 0xF6, '#', 'i', 0xF7};
+                std::vector<uint8_t> const v4 = {'[', '$', 'l', '#', 'i', 0xF7};
                 std::vector<uint8_t> const v5 = {'[', '$', 'I', '#', '[', 'i', 0xF5, 'i', 0xF1, ']'};
                 std::vector<uint8_t> const v6 = {'[', '#', '[', 'i', 0xF3, 'i', 0x02, ']'};
 
@@ -3455,7 +3455,7 @@ TEST_CASE("BJData")
             CHECK(json::from_bjdata(v0, true, false).is_discarded());
 
             std::vector<uint8_t> const vi = {'[', '$', '#'};
-            CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vi), "[json.exception.parse_error.110] parse error at byte 4: syntax error while parsing BJData value: unexpected end of input", json::parse_error&);
+            CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vi), "[json.exception.parse_error.112] parse error at byte 3: syntax error while parsing BJData type: marker 0x23 is not a permitted optimized array type", json::parse_error&);
             CHECK(json::from_bjdata(vi, true, false).is_discarded());
 
             std::vector<uint8_t> const vU = {'[', '$', 'U'};
@@ -4622,3 +4622,60 @@ TEST_CASE("BJData large strings and binaries (chunked reader)")
         }
     }
 }
+
+TEST_CASE("BJData optimized container type marker is validated for count 0")
+{
+    // the '$' type marker must be a legal marker even if no element follows
+    SECTION("invalid marker")
+    {
+        // 'Z', 'T', 'F', 'N', 'S', 'H', '[' and '{' are excluded in BJData
+        for (const char marker :
+                {'x', 'E', 'Z', 'T', 'F', 'N', 'S', 'H', '[', '{'
+                })
+        {
+            for (const char open :
+                    {'[', '{'
+                    })
+            {
+                for (const int count :
+                        {
+                            0, 1
+                        })
+                {
+                    CAPTURE(marker)
+                    CAPTURE(open)
+                    CAPTURE(count)
+                    const std::vector<std::uint8_t> v = {static_cast<std::uint8_t>(open), '$', static_cast<std::uint8_t>(marker), '#', 'i', static_cast<std::uint8_t>(count)};
+                    json _;
+                    CHECK_THROWS_AS(_ = json::from_bjdata(v), json::parse_error&);
+                    CHECK(json::from_bjdata(v, true, false).is_discarded());
+                }
+            }
+        }
+    }
+
+    SECTION("error message")
+    {
+        const std::vector<std::uint8_t> v = {'[', '$', 'x', '#', 'i', 0};
+        json _;
+        CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v), "[json.exception.parse_error.112] parse error at byte 3: syntax error while parsing BJData type: marker 0x78 is not a permitted optimized array type", json::parse_error&);
+    }
+
+    SECTION("valid marker")
+    {
+        for (const char marker :
+                {'i', 'U', 'I', 'u', 'l', 'm', 'L', 'M', 'h', 'd', 'D', 'C'
+                })
+        {
+            CAPTURE(marker)
+            const std::vector<std::uint8_t> va = {'[', '$', static_cast<std::uint8_t>(marker), '#', 'i', 0};
+            const std::vector<std::uint8_t> vo = {'{', '$', static_cast<std::uint8_t>(marker), '#', 'i', 0};
+            CHECK(json::from_bjdata(va) == json::array());
+            CHECK(json::from_bjdata(vo) == json::object());
+        }
+
+        // an optimized array of bytes is read as a binary value
+        const std::vector<std::uint8_t> vb = {'[', '$', 'B', '#', 'i', 0};
+        CHECK(json::from_bjdata(vb) == json::binary({}));
+    }
+}

tests/src/unit-ubjson.cpp

@@ -2023,7 +2023,7 @@ TEST_CASE("UBJSON")
             CHECK(json::from_ubjson(v0, true, false).is_discarded());
 
             std::vector<uint8_t> const vi = {'[', '$', '#'};
-            CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vi), "[json.exception.parse_error.110] parse error at byte 4: syntax error while parsing UBJSON value: unexpected end of input", json::parse_error&);
+            CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vi), "[json.exception.parse_error.112] parse error at byte 3: syntax error while parsing UBJSON type: marker 0x23 is not a permitted optimized array type", json::parse_error&);
             CHECK(json::from_ubjson(vi, true, false).is_discarded());
 
             std::vector<uint8_t> const vT = {'[', '$', 'T'};
@@ -3348,3 +3348,55 @@ TEST_CASE("UBJSON large strings (chunked reader)")
         }
     }
 }
+
+TEST_CASE("UBJSON optimized container type marker is validated for count 0")
+{
+    // the '$' type marker must be a legal marker even if no element follows
+    SECTION("invalid marker")
+    {
+        for (const char marker :
+                {'x', 'E', 'B', 'u', 'm', 'M', 'h'
+                })
+        {
+            for (const char open :
+                    {'[', '{'
+                    })
+            {
+                for (const int count :
+                        {
+                            0, 1
+                        })
+                {
+                    CAPTURE(marker)
+                    CAPTURE(open)
+                    CAPTURE(count)
+                    const std::vector<std::uint8_t> v = {static_cast<std::uint8_t>(open), '$', static_cast<std::uint8_t>(marker), '#', 'i', static_cast<std::uint8_t>(count)};
+                    json _;
+                    CHECK_THROWS_AS(_ = json::from_ubjson(v), json::parse_error&);
+                    CHECK(json::from_ubjson(v, true, false).is_discarded());
+                }
+            }
+        }
+    }
+
+    SECTION("error message")
+    {
+        const std::vector<std::uint8_t> v = {'[', '$', 'x', '#', 'i', 0};
+        json _;
+        CHECK_THROWS_WITH_AS(_ = json::from_ubjson(v), "[json.exception.parse_error.112] parse error at byte 3: syntax error while parsing UBJSON type: marker 0x78 is not a permitted optimized array type", json::parse_error&);
+    }
+
+    SECTION("valid marker")
+    {
+        for (const char marker :
+                {'Z', 'N', 'T', 'F', 'i', 'U', 'I', 'l', 'L', 'd', 'D', 'H', 'C', 'S', '[', '{'
+                })
+        {
+            CAPTURE(marker)
+            const std::vector<std::uint8_t> va = {'[', '$', static_cast<std::uint8_t>(marker), '#', 'i', 0};
+            const std::vector<std::uint8_t> vo = {'{', '$', static_cast<std::uint8_t>(marker), '#', 'i', 0};
+            CHECK(json::from_ubjson(va) == json::array());
+            CHECK(json::from_ubjson(vo) == json::object());
+        }
+    }
+}