Reserve output capacity up front for binary serialization

The vector-returning to_cbor/to_msgpack/to_ubjson/to_bjdata/to_bson grew
the output buffer purely by geometric reallocation. Reserving an estimate
up front avoids the early reallocations, which is the dominant per-byte
cost for array/object-heavy output.

The estimate (binary_reserve_hint) is deliberately conservative and safe
against untrusted input: it consults only the top-level element count
(O(1), no walk of the DOM), guards the multiplication against overflow,
and clamps the result to a fixed 1 MiB ceiling, so a large or hostile DOM
can never force an oversized allocation here. The buffer still grows
geometrically past the hint, so an underestimate only costs a few later
reallocations; scalars/strings/binary are written in one shot and get no
hint. Reserving capacity does not change the bytes produced.

Throughput (g++/clang -O3, vs the previous commit):
  cbor int array     +10% / +13%
  cbor object array  +20% / +38%

Output is byte-for-byte identical to develop across the binary
differential corpus.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

.github/workflows/codeql-analysis.yml

@@ -38,14 +38,14 @@ jobs:
 
     # Initializes the CodeQL tools for scanning.
     - name: Initialize CodeQL
-      uses: github/codeql-action/init@e0647621c2984b5ed2f768cb892365bf2a616ad1 # v4.37.2
+      uses: github/codeql-action/init@e4fba868fa4b1b91e1fdab776edc8cfbe6e9fb81 # v4.37.3
       with:
         languages: c-cpp
 
     # Autobuild attempts to build any compiled languages  (C/C++, C#, or Java).
     # If this step fails, then you should remove it and run the build manually (see below)
     - name: Autobuild
-      uses: github/codeql-action/autobuild@e0647621c2984b5ed2f768cb892365bf2a616ad1 # v4.37.2
+      uses: github/codeql-action/autobuild@e4fba868fa4b1b91e1fdab776edc8cfbe6e9fb81 # v4.37.3
 
     - name: Perform CodeQL Analysis
-      uses: github/codeql-action/analyze@e0647621c2984b5ed2f768cb892365bf2a616ad1 # v4.37.2
+      uses: github/codeql-action/analyze@e4fba868fa4b1b91e1fdab776edc8cfbe6e9fb81 # v4.37.3

.github/workflows/flawfinder.yml

@@ -43,6 +43,6 @@ jobs:
           output: 'flawfinder_results.sarif'
 
       - name: Upload analysis results to GitHub Security tab
-        uses: github/codeql-action/upload-sarif@e0647621c2984b5ed2f768cb892365bf2a616ad1 # v4.37.2
+        uses: github/codeql-action/upload-sarif@e4fba868fa4b1b91e1fdab776edc8cfbe6e9fb81 # v4.37.3
         with:
           sarif_file: ${{github.workspace}}/flawfinder_results.sarif

.github/workflows/scorecards.yml

@@ -76,6 +76,6 @@ jobs:
 
       # Upload the results to GitHub's code scanning dashboard.
       - name: "Upload to code-scanning"
-        uses: github/codeql-action/upload-sarif@e0647621c2984b5ed2f768cb892365bf2a616ad1 # v4.37.2
+        uses: github/codeql-action/upload-sarif@e4fba868fa4b1b91e1fdab776edc8cfbe6e9fb81 # v4.37.3
         with:
           sarif_file: results.sarif

.github/workflows/semgrep.yml

@@ -61,7 +61,7 @@ jobs:
 
       # Upload SARIF file generated in previous step
       - name: Upload SARIF file
-        uses: github/codeql-action/upload-sarif@e0647621c2984b5ed2f768cb892365bf2a616ad1 # v4.37.2
+        uses: github/codeql-action/upload-sarif@e4fba868fa4b1b91e1fdab776edc8cfbe6e9fb81 # v4.37.3
         with:
           sarif_file: semgrep.sarif
         if: always()

.github/workflows/stale.yml

@@ -20,7 +20,7 @@ jobs:
         with:
           egress-policy: audit
 
-      - uses: actions/stale@1e223db275d687790206a7acac4d1a11bd6fe629 # v10.4.0
+      - uses: actions/stale@4391f3da665fdf50b6810c1a66712fb9ba21aa93 # v11.0.0
         with:
           stale-issue-label: 'state: stale'
           stale-pr-label: 'state: stale'

.github/workflows/ubuntu.yml

@@ -142,7 +142,7 @@ jobs:
           name: code-coverage-report
           path: ${{ github.workspace }}/build/html
       - name: Publish report to Coveralls
-        uses: coverallsapp/github-action@5cbfd81b66ca5d10c19b062c04de0199c215fb6e # v2.3.7
+        uses: coverallsapp/github-action@8d6379e14d29928660c4ba802d8e85393440b329 # v2.3.8
         with:
           github-token: ${{ secrets.GITHUB_TOKEN }}
           path-to-lcov: ${{ github.workspace }}/build/json.info.filtered.noexcept

README.md

@@ -1801,13 +1801,13 @@ The library itself consists of a single header file licensed under the MIT licen
 - [**amalgamate.py - Amalgamate C source and header files**](https://github.com/edlund/amalgamate) to create a single header file
 - [**American fuzzy lop**](https://lcamtuf.coredump.cx/afl/) for fuzz testing
 - [**AppVeyor**](https://www.appveyor.com) for [continuous integration](https://ci.appveyor.com/project/nlohmann/json) on Windows
-- [**Artistic Style**](http://astyle.sourceforge.net) for automatic source code indentation
+- [**Artistic Style**](https://astyle.sourceforge.net) for automatic source code indentation
 - [**Clang**](https://clang.llvm.org) for compilation with code sanitizers
 - [**CMake**](https://cmake.org) for build automation
 - [**Codacy**](https://www.codacy.com) for further [code analysis](https://app.codacy.com/gh/nlohmann/json/dashboard)
 - [**Coveralls**](https://coveralls.io) to measure [code coverage](https://coveralls.io/github/nlohmann/json)
 - [**Coverity Scan**](https://scan.coverity.com) for [static analysis](https://scan.coverity.com/projects/nlohmann-json)
-- [**cppcheck**](http://cppcheck.sourceforge.net) for static analysis
+- [**cppcheck**](https://cppcheck.sourceforge.io) for static analysis
 - [**doctest**](https://github.com/onqtam/doctest) for the unit tests
 - [**GitHub Changelog Generator**](https://github.com/skywinder/github-changelog-generator) to generate the [ChangeLog](https://github.com/nlohmann/json/blob/develop/ChangeLog.md)
 - [**Google Benchmark**](https://github.com/google/benchmark) to implement the benchmarks
@@ -1822,6 +1822,15 @@ The library itself consists of a single header file licensed under the MIT licen
 
 ## Notes
 
+### Standards compliance
+
+The library targets strict conformance with [RFC 8259](https://tools.ietf.org/html/rfc8259.html). Both the original [JSONTestSuite](https://github.com/nst/JSONTestSuite) and its updated revision are exercised in CI; their test data is downloaded from [`nlohmann/json_test_data`](https://github.com/nlohmann/json_test_data) at configure time rather than committed to this repository (see [`tests/src/unit-testsuites.cpp`](https://github.com/nlohmann/json/blob/develop/tests/src/unit-testsuites.cpp)):
+
+- The updated revision runs all mandatory `y_` (must-accept) and `n_` (must-reject) cases through the strict [`parse()`](https://json.nlohmann.me/api/basic_json/parse/) entry point; the original suite runs its `n_` cases through `parse()` and its `y_` cases through [`operator>>`](https://json.nlohmann.me/api/operator_gtgt/).
+- The `i_` (implementation-defined) cases are, by RFC 8259, free to be accepted *or* rejected, so "passing all `i_` cases" is not a meaningful conformance metric. The library makes deliberate, documented choices there: nesting depth is not artificially limited, a leading UTF-8 byte order mark is silently ignored, [Unicode noncharacters](https://www.unicode.org/faq/private_use.html#nonchar1) are forwarded unchanged, invalid UTF-8 and lone/unpaired UTF-16 surrogates are rejected (stricter than required), and a number that cannot be stored without becoming `NaN`/`INF` raises [`out_of_range.406`](https://json.nlohmann.me/home/exceptions/#jsonexceptionout_of_range406).
+
+One behavioral nuance is worth calling out, because a superficial test often misreads it as non-compliance: [`parse()`](https://json.nlohmann.me/api/basic_json/parse/) is strict and rejects trailing data after a value, whereas [`operator>>`](https://json.nlohmann.me/api/operator_gtgt/) follows relaxed iostream semantics — it parses a single value and leaves the stream positioned right after it. Feeding "a valid document followed by trailing bytes" through `operator>>` reports success; the same input through `parse()` is rejected. This is a documented two-API design, not a conformance gap. See [**parsing**](https://json.nlohmann.me/features/parsing/) for details.
+
 ### Character encoding
 
 The library supports **Unicode input** as follows:

cmake/clang_flags.cmake

@@ -5,6 +5,9 @@
 # -Wno-extra-semi-stmt            The library uses assert which triggers this warning.
 # -Wno-padded                     We do not care about padding warnings.
 # -Wno-covered-switch-default     All switches list all cases and a default case.
+# -Wno-c2y-extensions             Clang 22.1 diagnoses __COUNTER__ as a C2y extension, also in
+#                                 C++ mode. The library does not use __COUNTER__; the warnings
+#                                 all come from vendored Doctest (SECTION/TEST_CASE macros).
 # -Wno-unsafe-buffer-usage        Pervasive: the library's own low-level numeric/buffer code
 #                                 (to_chars, serializer, lexer, binary reader/writer, input
 #                                 adapters, json_pointer) plus vendored Doctest itself (~208
@@ -20,5 +23,6 @@ set(CLANG_CXXFLAGS
     -Wno-extra-semi-stmt
     -Wno-padded
     -Wno-covered-switch-default
+    -Wno-c2y-extensions
     -Wno-unsafe-buffer-usage
 )

docs/mkdocs/docs/api/basic_json/parser_callback_t.md

@@ -29,7 +29,14 @@ Discarding a value (i.e., returning `#!cpp false`) has different effects dependi
 called:
 
 - Discarded values in structured types are skipped. That is, the parser will behave as if the discarded value was never
-  read.
+  read. This holds for every value type and for both kinds of parent: a discarded element is removed from the
+  surrounding array, and a discarded member is removed from the surrounding object together with its key.
+- Arrays and objects can be discarded either at their `parse_event_t::array_start`/`parse_event_t::object_start` event
+  or at their `parse_event_t::array_end`/`parse_event_t::object_end` event, and both remove the whole value. Discarding
+  it at the start event also means the callback is called neither for the content of the value nor for its matching end
+  event.
+- Discarding a `parse_event_t::key` event discards the whole object member. The callback is still called for the
+  associated value, but its return value has no further effect.
 - In case a value outside a structured type is skipped, it is replaced with `null`. This case happens if the top-level
   element is skipped.
 
@@ -49,7 +56,7 @@ called:
 ## Return value
 
 Whether the JSON value which called the function during parsing should be kept (`#!cpp true`) or not (`#!cpp false`). In
-the latter case, it is either skipped completely or replaced by an empty discarded object.
+the latter case, it is skipped completely, or replaced by `null` if it is the top-level value.
 
 ## Examples
 
@@ -68,6 +75,21 @@ the latter case, it is either skipped completely or replaced by an empty discard
     --8<-- "examples/parse__string__parser_callback_t.output"
     ```
 
+??? example
+
+    The example below shows where discarded values are removed. The array and the number are discarded in different
+    ways, but in each case the parse result contains neither the value nor its key.
+
+    ```cpp
+    --8<-- "examples/parser_callback_t.cpp"
+    ```
+
+    Output:
+
+    ```json
+    --8<-- "examples/parser_callback_t.output"
+    ```
+
 ## See also
 
 - [parse](parse.md) deserialize from a compatible input
@@ -76,3 +98,5 @@ the latter case, it is either skipped completely or replaced by an empty discard
 ## Version history
 
 - Added in version 1.0.0.
+- Fixed in version 3.13.0 to also remove discarded values from a parent object; before, discarding an array or a value
+  stored under an object key left a discarded member behind, which made the parse result serialize to invalid JSON.

docs/mkdocs/docs/api/operator_gtgt.md

@@ -33,17 +33,44 @@ A UTF-8 byte order mark is silently ignored.
 Invalid Unicode escapes and unpaired surrogates in the input are reported as
 [`parse_error.101`](../home/exceptions.md#jsonexceptionparse_error101) with a detailed message.
 
-`operator>>` parses exactly one JSON value and leaves the stream positioned right after it, so it can be called
+`operator>>` parses exactly one JSON value, so it can be called repeatedly to read a sequence of concatenated JSON
-repeatedly to read a sequence of concatenated JSON values from the same stream:
+values from the same stream:
 
 ```cpp
 json j1, j2;
-input >> j1;  // parses the first value, stream now positioned right after it
+input >> j1;  // parses the first value
 input >> j2;  // parses the next value
 ```
 
-Note this does **not** work for [JSON Lines](../features/parsing/json_lines.md) (newline-delimited JSON) input --
+!!! warning "A number must be followed by whitespace"
-see that page for why and for the recommended alternative.
+
+    A number is only terminated by the character that follows it. That character is read from the stream to detect the
+    end of the number, and it is **not** put back. When a value that is a number is immediately followed by the next
+    value, the first character of that next value is lost:
+
+    ```cpp
+    std::istringstream input("1true");
+    json j1, j2;
+    input >> j1;  // j1 == 1
+    input >> j2;  // throws parse_error.101: the stream now starts at "rue"
+    ```
+
+    Separating the values with whitespace avoids this, because the character that is eaten is then the separator:
+
+    ```cpp
+    std::istringstream input("1 true");
+    json j1, j2;
+    input >> j1;  // j1 == 1
+    input >> j2;  // j2 == true
+    ```
+
+    Only numbers are affected. Values ending in a self-delimiting character do not read past themselves, so
+    `truefalse`, `[1][2]`, `{"a":1}{"b":2}`, and `"a""b"` can be read back to back without a separator.
+
+    This is tracked in [#5340](https://github.com/nlohmann/json/issues/5340).
+
+Note that reading concatenated values does **not** work for [JSON Lines](../features/parsing/json_lines.md)
+(newline-delimited JSON) input -- see that page for why and for the recommended alternative.
 
 !!! warning "Deprecation"
 

docs/mkdocs/docs/api/ordered_json.md

@@ -13,6 +13,12 @@ Therefore, adding object elements can yield a reallocation in which case all ite
 [`end()`](basic_json/end.md) iterator) and all references to the elements are invalidated. Also, any iterator or
 reference after the insertion point will point to the same index, which is now a different value.
 
+## Complexity
+
+[`ordered_map`](ordered_map.md) has no lookup index: every key-based object operation is a linear scan, so building or
+parsing an object of `n` keys costs O(n²) rather than O(n log n). See
+[`ordered_map` complexity](ordered_map.md#complexity) for the per-operation table and for measured numbers.
+
 ## Examples
 
 ??? example

docs/mkdocs/docs/api/ordered_map.md

@@ -56,6 +56,48 @@ std::equal_to<>     // since C++14
 - **find**
 - **insert**
 
+## Complexity
+
+Because the elements are stored in a `std::vector` in insertion order, there is no index to look a key up by. Every
+key-based operation performs a **linear scan** over the stored elements. With `n` denoting the number of elements in the
+container:
+
+| Operation                              | Complexity     | Note                                                     |
+|----------------------------------------|----------------|----------------------------------------------------------|
+| **emplace**                            | O(n)           | scans for an existing key, then appends (amortized O(1)) |
+| **operator\[\]**                       | O(n)           | delegates to **emplace** (non-const) or **at** (const)   |
+| **at**                                 | O(n)           | throws `#!cpp std::out_of_range` if the key is not found |
+| **find**                               | O(n)           |                                                          |
+| **count**                              | O(n)           | the result is always 0 or 1                              |
+| **erase(key)**                         | O(n)           | scan, then move the remaining elements one position down |
+| **erase(pos)**, **erase(first, last)** | O(n)           | moves all elements after the erased range                |
+| **insert(value)**                      | O(n)           | equivalent to **emplace**                                |
+| **insert(first, last)**                | O((n + m) * m) | for `m` inserted elements                                |
+
+This differs from `#!cpp std::map`, where the same operations are O(log n).
+
+!!! warning "Quadratic cost of building large objects"
+
+    Because every insertion scans all elements inserted so far, building an object of `n` distinct keys costs
+    **O(n²)** in total. This applies to filling an [`ordered_json`](ordered_json.md) object key by key as well as to
+    parsing one, since the parser inserts each key as it is read.
+
+    The cost is negligible for the object sizes typically found in configuration files or API payloads, but it grows
+    steeply for machine-generated objects with many thousands of keys. Measured with `-O2 -DNDEBUG` for parsing a flat
+    object of `n` keys, relative to `#!cpp nlohmann::json` (which uses `#!cpp std::map`):
+
+    | `n`    | `json` | `ordered_json` | factor |
+    |--------|--------|----------------|--------|
+    | 2000   | 0.7 ms | 3.6 ms         | 5×     |
+    | 4000   | 0.8 ms | 14.0 ms        | 19×    |
+    | 8000   | 1.6 ms | 67.8 ms        | 43×    |
+    | 16 000 | 3.3 ms | 181.6 ms       | 54×    |
+
+    If key order matters for objects of that size, consider a container with a lookup index, such as
+    [`tsl::ordered_map`](https://github.com/Tessil/ordered-map)
+    ([integration](https://github.com/nlohmann/json/issues/546#issuecomment-304447518)), as the object type -- see
+    [object order](../features/object_order.md).
+
 ## Examples
 
 ??? example

docs/mkdocs/docs/examples/parser_callback_t.cpp

@@ -0,0 +1,47 @@
+#include <iostream>
+#include <nlohmann/json.hpp>
+
+using json = nlohmann::json;
+
+int main()
+{
+    // a JSON text with an array and a number inside an object
+    auto text = R"({"IDs": [116, 943], "Width": 800})";
+
+    // discard the array when the parser reads its opening bracket
+    json j_array_start = json::parse(text, [](int /*depth*/, json::parse_event_t event, json & /*parsed*/)
+    {
+        return event != json::parse_event_t::array_start;
+    });
+
+    // discard the same array when the parser reads its closing bracket
+    json j_array_end = json::parse(text, [](int /*depth*/, json::parse_event_t event, json & /*parsed*/)
+    {
+        return event != json::parse_event_t::array_end;
+    });
+
+    // discard the number, but keep its key
+    json j_value = json::parse(text, [](int /*depth*/, json::parse_event_t event, json & parsed)
+    {
+        return !(event == json::parse_event_t::value && parsed == json(800));
+    });
+
+    // discard the key of the number
+    json j_key = json::parse(text, [](int /*depth*/, json::parse_event_t event, json & parsed)
+    {
+        return !(event == json::parse_event_t::key && parsed == json("Width"));
+    });
+
+    // discard the top-level object
+    json j_root = json::parse(text, [](int /*depth*/, json::parse_event_t event, json & /*parsed*/)
+    {
+        return event != json::parse_event_t::object_end;
+    });
+
+    // in every case, the discarded value is removed together with its key
+    std::cout << j_array_start << '\n'
+              << j_array_end << '\n'
+              << j_value << '\n'
+              << j_key << '\n'
+              << j_root << '\n';
+}

docs/mkdocs/docs/examples/parser_callback_t.output

@@ -0,0 +1,5 @@
+{"Width":800}
+{"Width":800}
+{"IDs":[116,943]}
+{"IDs":[116,943]}
+null

docs/mkdocs/docs/features/binary_formats/bson.md

@@ -35,6 +35,19 @@ The library uses the following mapping from JSON values types to BSON types:
     The mapping is **incomplete**, since only JSON-objects (and things contained therein) can be serialized to BSON.
     Also, keys may not contain U+0000, since they are serialized a zero-terminated c-strings.
 
+!!! warning "BSON type 0x11 interoperability"
+
+    The BSON specification defines type `0x11` as a Timestamp. This library uses marker `0x11` when serializing
+    `number_unsigned` values in the range `9223372036854775808..18446744073709551615`. Other BSON implementations may
+    therefore interpret these values as Timestamps instead of unsigned integers.
+
+!!! info "Binary values without a subtype"
+
+    BSON requires every binary value to have a subtype. If a binary value has no subtype, this library serializes it
+    with the generic subtype `0x00`. After deserialization, `has_subtype()` returns `true` and `subtype()` returns `0`.
+    As a result, serializing and deserializing a JSON object containing such a value produces a different JSON object,
+    even though the binary data is unchanged.
+
 ??? example
 
     ```cpp
@@ -82,8 +95,8 @@ The library maps BSON record types to JSON value types as follows:
 
 !!! note "Handling of BSON type 0x11"
 
-    BSON type 0x11 is used to represent uint64 numbers. This library treats these values purely as uint64 numbers 
+    This library deserializes BSON type `0x11` (Timestamp) as a `number_unsigned` value. The 64-bit value is preserved,
-    and does not parse them into date-related formats.
+    but the Timestamp type information is not.
 
 ??? example
 

docs/mkdocs/docs/features/conversions.md

@@ -66,8 +66,14 @@ auto t = j.get<std::tuple<double, std::string, int>>();  // {1.0, "hello", 42}
     std::get<1>(refs) = "world";  // modifies j[1] in place
     ```
 
-    A referenced type must be one the library actually stores (or an arithmetic type it can convert to/from);
+    A referenced element must name the type the library actually *stores* — one of [`boolean_t`](../api/basic_json/boolean_t.md),
-    otherwise this is a compile error.
+    [`number_integer_t`](../api/basic_json/number_integer_t.md), [`number_unsigned_t`](../api/basic_json/number_unsigned_t.md),
+    [`number_float_t`](../api/basic_json/number_float_t.md), [`string_t`](../api/basic_json/string_t.md),
+    [`binary_t`](../api/basic_json/binary_t.md), [`array_t`](../api/basic_json/array_t.md), or
+    [`object_t`](../api/basic_json/object_t.md). There is nothing else to refer to, so a reference to any other type is a
+    compile error even when a conversion would exist: `#!cpp std::tuple<int&>` is rejected, because the library stores a
+    `#!cpp number_integer_t` (`#!cpp std::int64_t` by default) and not an `#!cpp int`. This restriction applies only to
+    reference elements — a plain `#!cpp std::tuple<int>` converts by value as usual.
 
 ## Implicit conversions
 
@@ -116,17 +122,34 @@ which forces the explicit `get` form and can catch unintended conversions at com
     with a custom `adl_serializer<std::optional<T>>` specialization. Prefer `get<std::optional<T>>()`/`get_to()`
     over `static_cast` for optional types.
 
-!!! warning "Converting to a fixed-size `std::array` does not check length"
+!!! warning "Converting to a fixed-size destination does not check the array size"
 
-    Converting a JSON array to `#!cpp std::array<T, N>` does not check that the JSON array's size matches `N`:
+    Some destination types have a size that is fixed by their C++ type rather than by the JSON value:
-    if the JSON array is longer, the extra elements are silently dropped; if it is shorter, the remaining
+    `#!cpp std::pair<A, B>`, `#!cpp std::tuple<Ts...>`, `#!cpp std::array<T, N>`, C arrays `#!cpp T[N]`, and
-    `std::array` elements are left default-constructed. No exception is thrown in either case.
+    `#!cpp std::map`/`#!cpp std::unordered_map` with a non-string key type (which is read from an array of
+    two-element arrays). All of them read exactly as many elements as they need via
+    [`at`](../api/basic_json/at.md) and **never compare the JSON array's size to that number**. The two
+    mismatch directions therefore behave differently:
+
+    - The JSON array has **too many** elements: the surplus is **silently discarded**, and no exception is
+      thrown.
+    - The JSON array has **too few** elements: `at` throws
+      [`out_of_range.401`](../home/exceptions.md#jsonexceptionout_of_range401) for the first missing index --
+      an out-of-range error, not a [`type_error`](../home/exceptions.md#type-errors), even though the cause
+      is a shape mismatch.
 
     ```cpp
     json j = {1, 2, 3, 4, 5};
-    auto a = j.get<std::array<int, 3>>();  // {1, 2, 3} -- elements 4 and 5 silently dropped
+
+    auto a = j.get<std::array<int, 3>>();       // {1, 2, 3} -- elements 4 and 5 silently dropped
+    auto p = j.get<std::pair<int, int>>();      // (1, 2)    -- elements 3, 4, and 5 silently dropped
+
+    json k = {1};
+    auto q = k.get<std::pair<int, int>>();      // ❌ throws out_of_range.401
     ```
 
+    If a size mismatch is an error in your application, check the size yourself before converting.
+
 ## Omitting a field when serializing `std::optional`
 
 By default, `to_json` for `std::optional<T>` writes either the value or `#!json null` -- there is no built-in way

docs/mkdocs/docs/features/object_order.md

@@ -53,6 +53,12 @@ If you do want to preserve the **insertion order**, you can use the type [`nlohm
 
 Alternatively, you can use a more sophisticated ordered map like [`tsl::ordered_map`](https://github.com/Tessil/ordered-map) ([integration](https://github.com/nlohmann/json/issues/546#issuecomment-304447518)) or [`nlohmann::fifo_map`](https://github.com/nlohmann/fifo_map) ([integration](https://github.com/nlohmann/json/issues/485#issuecomment-333652309)).
 
+The [`ordered_map`](../api/ordered_map.md) behind `nlohmann::ordered_json` is deliberately minimal and has no lookup
+index, so every key access is a linear scan and building an object of `n` keys costs O(n²). This is unnoticeable at
+typical object sizes but becomes significant for objects with many thousands of keys; see
+[`ordered_map` complexity](../api/ordered_map.md#complexity). The alternatives above keep a lookup index and do not
+have this cost.
+
 ### Notes on parsing
 
 Note that you also need to call the right [`parse`](../api/basic_json/parse.md) function when reading from a file.

docs/mkdocs/docs/features/parsing/index.md

@@ -28,6 +28,22 @@ Inputs consisting of multiple values separated by newlines are handled by the [J
 By default, the library rejects comments and trailing commas. Both can be enabled with parameters of the `parse`
 function — see [comments](../comments.md) and [trailing commas](../trailing_commas.md).
 
+## Strictness and trailing data
+
+[`parse`](../../api/basic_json/parse.md) reads a single JSON value and requires the whole input to be consumed: any
+non-whitespace data after the value is reported as a parse error. Use it when you want to guarantee that an input is
+exactly one complete JSON document.
+
+[`operator>>`](../../api/operator_gtgt.md) follows relaxed `#!cpp std::istream` semantics instead: it parses one JSON
+value and leaves the stream positioned right after it, without requiring the rest of the stream to be consumed. This is
+what makes it possible to read several concatenated values from the same stream, but it also means that "a valid
+document followed by trailing bytes" is accepted rather than rejected. If you are validating conformance, or need to
+reject any input that is not exactly one JSON document, prefer `parse`.
+
+When using `operator>>` to read several concatenated values this way, a value that is a number must be followed by
+whitespace, because `operator>>` consumes the character that terminates a number — see the
+[`operator>>` notes](../../api/operator_gtgt.md#notes) for details and examples.
+
 ## SAX vs. DOM parsing
 
 The library offers two parsing models:

docs/mkdocs/docs/features/parsing/json_lines.md

@@ -49,4 +49,5 @@ JSON Lines input with more than one value is treated as invalid JSON by the [`pa
     with a JSON Lines input does not work, because the parser will try to parse one value after the last one.
 
     This is different from parsing a stream of *concatenated* (non-newline-delimited) JSON values, for which
-    `operator>>` does work -- see its [notes](../../api/operator_gtgt.md#notes) for details.
+    `operator>>` does work, provided that a value that is a number is followed by whitespace -- see its
+    [notes](../../api/operator_gtgt.md#notes) for details.

docs/mkdocs/docs/home/exceptions.md

@@ -291,9 +291,10 @@ A JSON Pointer array index must be a number.
 
 ### json.exception.parse_error.110
 
-When parsing CBOR or MessagePack, the byte vector ends before the complete value has been read.
+When parsing a [binary format](../features/binary_formats/index.md), the byte vector ends before the complete value has
+been read.
 
-!!! failure "Example message"
+!!! failure "Example messages"
 
     ```
     [json.exception.parse_error.110] parse error at byte 5: syntax error while parsing CBOR string: unexpected end of input
@@ -301,6 +302,9 @@ When parsing CBOR or MessagePack, the byte vector ends before the complete value
     ```
     [json.exception.parse_error.110] parse error at byte 2: syntax error while parsing UBJSON value: expected end of input; last byte: 0x5A
     ```
+    ```
+    [json.exception.parse_error.110] parse error at byte 8: syntax error while parsing BSON number: unexpected end of input
+    ```
 
 ### json.exception.parse_error.112
 
@@ -329,10 +333,14 @@ An unexpected byte was read in a [binary format](../features/binary_formats/inde
     ```
     [json.exception.parse_error.112] parse error at byte 9: syntax error while parsing CBOR value: negative integer overflow
     ```
+    ```
+    [json.exception.parse_error.112] parse error at byte 5: syntax error while parsing BSON document: document size 6 does not match the number of bytes read (5)
+    ```
 
 ### json.exception.parse_error.113
 
-While parsing a map key, a value that is not a string has been read.
+A string could not be read from a [binary format](../features/binary_formats/index.md): either a value that is not a
+string was read where one was required (for instance as a map key), or the string's length specification is invalid.
 
 !!! failure "Example messages"
 
@@ -345,6 +353,9 @@ While parsing a map key, a value that is not a string has been read.
     ```
     [json.exception.parse_error.113] parse error at byte 2: syntax error while parsing UBJSON char: byte after 'C' must be in range 0x00..0x7F; last byte: 0x82
     ```
+    ```
+    [json.exception.parse_error.113] parse error at byte 3: syntax error while parsing BJData string: string length must not be negative
+    ```
 
 ### json.exception.parse_error.114
 
@@ -853,13 +864,21 @@ and this exception no longer occurs.
 
 ### json.exception.out_of_range.408
 
-The size (following `#`) of an UBJSON array or object exceeds the maximal capacity.
+The size of an array or object in a [binary format](../features/binary_formats/index.md) exceeds the maximal capacity:
+the size following `#` for [UBJSON](../features/binary_formats/ubjson.md)/[BJData](../features/binary_formats/bjdata.md),
+or the encoded length for [CBOR](../features/binary_formats/cbor.md).
 
-!!! failure "Example message"
+!!! failure "Example messages"
 
     ```
     excessive array size: 8658170730974374167
     ```
+    ```
+    [json.exception.out_of_range.408] syntax error while parsing CBOR size: excessive array size
+    ```
+    ```
+    [json.exception.out_of_range.408] syntax error while parsing CBOR size: excessive map size
+    ```
 
 ### json.exception.out_of_range.409
 

include/nlohmann/detail/input/binary_reader.hpp

@@ -704,13 +704,15 @@ class binary_reader
             case 0x9A: // array (four-byte uint32_t for n follow)
             {
                 std::uint32_t len{};
-                return get_number(input_format_t::cbor, len) && get_cbor_array(conditional_static_cast<std::size_t>(len), tag_handler);
+                std::size_t size{};
+                return get_number(input_format_t::cbor, len) && get_cbor_container_size(len, size, "array") && get_cbor_array(size, tag_handler);
             }
 
             case 0x9B: // array (eight-byte uint64_t for n follow)
             {
                 std::uint64_t len{};
-                return get_number(input_format_t::cbor, len) && get_cbor_array(conditional_static_cast<std::size_t>(len), tag_handler);
+                std::size_t size{};
+                return get_number(input_format_t::cbor, len) && get_cbor_container_size(len, size, "array") && get_cbor_array(size, tag_handler);
             }
 
             case 0x9F: // array (indefinite length)
@@ -758,13 +760,15 @@ class binary_reader
             case 0xBA: // map (four-byte uint32_t for n follow)
             {
                 std::uint32_t len{};
-                return get_number(input_format_t::cbor, len) && get_cbor_object(conditional_static_cast<std::size_t>(len), tag_handler);
+                std::size_t size{};
+                return get_number(input_format_t::cbor, len) && get_cbor_container_size(len, size, "map") && get_cbor_object(size, tag_handler);
             }
 
             case 0xBB: // map (eight-byte uint64_t for n follow)
             {
                 std::uint64_t len{};
-                return get_number(input_format_t::cbor, len) && get_cbor_object(conditional_static_cast<std::size_t>(len), tag_handler);
+                std::size_t size{};
+                return get_number(input_format_t::cbor, len) && get_cbor_container_size(len, size, "map") && get_cbor_object(size, tag_handler);
             }
 
             case 0xBF: // map (indefinite length)
@@ -807,25 +811,37 @@ class binary_reader
                             case 0xD8:
                             {
                                 std::uint8_t subtype_to_ignore{};
-                                get_number(input_format_t::cbor, subtype_to_ignore);
+                                if (!get_number(input_format_t::cbor, subtype_to_ignore))
+                                {
+                                    return false;
+                                }
                                 break;
                             }
                             case 0xD9:
                             {
                                 std::uint16_t subtype_to_ignore{};
-                                get_number(input_format_t::cbor, subtype_to_ignore);
+                                if (!get_number(input_format_t::cbor, subtype_to_ignore))
+                                {
+                                    return false;
+                                }
                                 break;
                             }
                             case 0xDA:
                             {
                                 std::uint32_t subtype_to_ignore{};
-                                get_number(input_format_t::cbor, subtype_to_ignore);
+                                if (!get_number(input_format_t::cbor, subtype_to_ignore))
+                                {
+                                    return false;
+                                }
                                 break;
                             }
                             case 0xDB:
                             {
                                 std::uint64_t subtype_to_ignore{};
-                                get_number(input_format_t::cbor, subtype_to_ignore);
+                                if (!get_number(input_format_t::cbor, subtype_to_ignore))
+                                {
+                                    return false;
+                                }
                                 break;
                             }
                             default:
@@ -843,28 +859,40 @@ class binary_reader
                             case 0xD8:
                             {
                                 std::uint8_t subtype{};
-                                get_number(input_format_t::cbor, subtype);
+                                if (!get_number(input_format_t::cbor, subtype))
+                                {
+                                    return false;
+                                }
                                 b.set_subtype(detail::conditional_static_cast<typename binary_t::subtype_type>(subtype));
                                 break;
                             }
                             case 0xD9:
                             {
                                 std::uint16_t subtype{};
-                                get_number(input_format_t::cbor, subtype);
+                                if (!get_number(input_format_t::cbor, subtype))
+                                {
+                                    return false;
+                                }
                                 b.set_subtype(detail::conditional_static_cast<typename binary_t::subtype_type>(subtype));
                                 break;
                             }
                             case 0xDA:
                             {
                                 std::uint32_t subtype{};
-                                get_number(input_format_t::cbor, subtype);
+                                if (!get_number(input_format_t::cbor, subtype))
+                                {
+                                    return false;
+                                }
                                 b.set_subtype(detail::conditional_static_cast<typename binary_t::subtype_type>(subtype));
                                 break;
                             }
                             case 0xDB:
                             {
                                 std::uint64_t subtype{};
-                                get_number(input_format_t::cbor, subtype);
+                                if (!get_number(input_format_t::cbor, subtype))
+                                {
+                                    return false;
+                                }
                                 b.set_subtype(detail::conditional_static_cast<typename binary_t::subtype_type>(subtype));
                                 break;
                             }
@@ -1155,6 +1183,31 @@ class binary_reader
         }
     }
 
+    /*!
+    @brief narrow a definite CBOR array/map length to std::size_t
+
+    A definite length is rejected if it does not fit in std::size_t or if it
+    equals detail::unknown_size(), which is reserved to mark an indefinite-
+    length container and would otherwise make the length read as indefinite.
+    Both cases exceed any container's max_size(), so no representable input
+    is affected.
+
+    @param[in]  len      the declared length
+    @param[out] result   the length narrowed to std::size_t
+    @param[in]  context  "array" or "map", for the error message
+    @return whether the length is usable
+    */
+    bool get_cbor_container_size(const std::uint64_t len, std::size_t& result, const char* context)
+    {
+        if (JSON_HEDLEY_UNLIKELY(!value_in_range_of<std::size_t>(len) || len == detail::unknown_size()))
+        {
+            return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408,
+                                    exception_message(input_format_t::cbor, concat("excessive ", context, " size"), "size"), nullptr));
+        }
+        result = conditional_static_cast<std::size_t>(len);
+        return true;
+    }
+
     /*!
     @param[in] len  the length of the array or detail::unknown_size() for an
                     array of indefinite size
@@ -2825,7 +2878,17 @@ class binary_reader
             case token_type::value_unsigned:
                 return sax->number_unsigned(number_lexer.get_number_unsigned());
             case token_type::value_float:
-                return sax->number_float(number_lexer.get_number_float(), std::move(number_string));
+            {
+                const auto parsed_float = number_lexer.get_number_float();
+                if (JSON_HEDLEY_UNLIKELY(!std::isfinite(parsed_float)))
+                {
+                    return sax->parse_error(
+                               chars_read,
+                               number_string,
+                               out_of_range::create(406, concat("number overflow parsing '", number_string, '\''), nullptr));
+                }
+                return sax->number_float(parsed_float, std::move(number_string));
+            }
             case token_type::uninitialized:
             case token_type::literal_true:
             case token_type::literal_false:

include/nlohmann/detail/input/json_sax.hpp

@@ -626,14 +626,7 @@ class json_sax_dom_callback_parser
         if (!ref_stack.empty() && ref_stack.back() && ref_stack.back()->is_structured())
         {
             // remove discarded value
-            for (auto it = ref_stack.back()->begin(); it != ref_stack.back()->end(); ++it)
+            remove_discarded_value(*ref_stack.back());
-            {
-                if (it->is_discarded())
-                {
-                    ref_stack.back()->erase(it);
-                    break;
-                }
-            }
         }
 
         return true;
@@ -674,8 +667,9 @@ class json_sax_dom_callback_parser
     bool end_array()
     {
         bool keep = true;
+        const bool stored = ref_stack.back() != nullptr;
 
-        if (ref_stack.back())
+        if (stored)
         {
             keep = callback(static_cast<int>(ref_stack.size()) - 1, parse_event_t::array_end, *ref_stack.back());
             if (keep)
@@ -709,9 +703,19 @@ class json_sax_dom_callback_parser
         keep_stack.pop_back();
 
         // remove discarded value
-        if (!keep && !ref_stack.empty() && ref_stack.back()->is_array())
+        if (!ref_stack.empty() && ref_stack.back())
         {
-            ref_stack.back()->m_data.m_value.array->pop_back();
+            if (!keep && ref_stack.back()->is_array())
+            {
+                ref_stack.back()->m_data.m_value.array->pop_back();
+            }
+            else if ((!keep || !stored) && ref_stack.back()->is_object())
+            {
+                // the array is either still stored under its key or was never
+                // stored, leaving the placeholder key() wrote; both show up as
+                // a discarded member of the parent object
+                remove_discarded_value(*ref_stack.back());
+            }
         }
 
         return true;
@@ -801,6 +805,19 @@ class json_sax_dom_callback_parser
     }
 #endif
 
+    /// remove the discarded value the callback rejected from its parent
+    static void remove_discarded_value(BasicJsonType& parent)
+    {
+        for (auto it = parent.begin(); it != parent.end(); ++it)
+        {
+            if (it->is_discarded())
+            {
+                parent.erase(it);
+                break;
+            }
+        }
+    }
+
     /*!
     @param[in] v  value to add to the JSON value we build during parsing
     @param[in] skip_callback  whether we should skip calling the callback
@@ -841,6 +858,18 @@ class json_sax_dom_callback_parser
         // do not handle this value if we just learnt it shall be discarded
         if (!keep)
         {
+            // if the value was to become an object member, key() already
+            // stored a placeholder for it that has to be removed again
+            if (!ref_stack.empty() && ref_stack.back() && ref_stack.back()->is_object())
+            {
+                JSON_ASSERT(!key_keep_stack.empty());
+                const bool placeholder_stored = key_keep_stack.back();
+                key_keep_stack.pop_back();
+                if (placeholder_stored)
+                {
+                    remove_discarded_value(*ref_stack.back());
+                }
+            }
             return {false, nullptr};
         }
 

include/nlohmann/detail/meta/type_traits.hpp

@@ -231,7 +231,9 @@ struct char_traits<signed char> : std::char_traits<char>
     // Redefine to_int_type function
     static int_type to_int_type(char_type c) noexcept
     {
-        return static_cast<int_type>(c);
+        // cast via unsigned char: sign-extending a negative char_type would make
+        // byte 0xFF indistinguishable from eof()
+        return static_cast<int_type>(static_cast<unsigned char>(c));
     }
 
     static char_type to_char_type(int_type i) noexcept
@@ -699,21 +701,35 @@ struct is_json_pointer_of<A, ::nlohmann::json_pointer<A>> : std::true_type {};
 template <typename A>
 struct is_json_pointer_of<A, ::nlohmann::json_pointer<A>&> : std::true_type {};
 
-// checks if A and B are comparable using Compare functor
+// checks if A and B are comparable using Compare functor, assuming that
+// neither A nor B is a json_pointer type (that case is handled by
+// is_comparable below, which never instantiates this helper otherwise)
 template<typename Compare, typename A, typename B, typename = void>
-struct is_comparable : std::false_type {};
+struct is_comparable_no_json_pointer : std::false_type {};
 
-// We exclude json_pointer here, because the checks using Compare(A, B) will
-// use json_pointer::operator string_t() which triggers a deprecation warning
-// for GCC. See https://github.com/nlohmann/json/issues/4621. The call to
-// is_json_pointer_of can be removed once the deprecated function has been
-// removed.
 template<typename Compare, typename A, typename B>
-struct is_comparable < Compare, A, B, enable_if_t < !is_json_pointer_of<A, B>::value
+struct is_comparable_no_json_pointer < Compare, A, B, enable_if_t <
-&& std::is_constructible <decltype(std::declval<Compare>()(std::declval<A>(), std::declval<B>()))>::value
+std::is_constructible <decltype(std::declval<Compare>()(std::declval<A>(), std::declval<B>()))>::value
 && std::is_constructible <decltype(std::declval<Compare>()(std::declval<B>(), std::declval<A>()))>::value
 >> : std::true_type {};
 
+// checks if A and B are comparable using Compare functor
+// We dispatch on is_json_pointer_of as a plain bool (rather than folding it
+// into a single enable_if_t condition together with the checks below) so
+// that the Compare(A, B) checks are only ever written - and thus only ever
+// instantiated - when A/B are not a json_pointer/string pair. Those checks
+// use json_pointer::operator string_t() (GCC, see #4621) resp. the
+// deprecated json_pointer/string operator== (Clang, see #5288), and merely
+// naming them as later operands of a plain && chain is not sufficient to
+// avoid their instantiation on all compilers, even when the first operand
+// is false. The dispatch on is_json_pointer_of can be removed once the
+// deprecated json_pointer comparison operators have been removed.
+template<typename Compare, typename A, typename B, bool = is_json_pointer_of<A, B>::value>
+struct is_comparable : std::false_type {};
+
+template<typename Compare, typename A, typename B>
+struct is_comparable<Compare, A, B, false> : is_comparable_no_json_pointer<Compare, A, B> {};
+
 template<typename T>
 using detect_is_transparent = typename T::is_transparent;
 

include/nlohmann/detail/output/output_adapters.hpp

@@ -13,6 +13,7 @@
 #include <iterator> // back_inserter
 #include <memory> // shared_ptr, make_shared
 #include <string> // basic_string
+#include <utility> // move
 #include <vector> // vector
 
 #ifndef JSON_NO_IO
@@ -118,6 +119,72 @@ class output_string_adapter : public output_adapter_protocol<CharType>
     StringType& str;
 };
 
+/// @brief non-virtual output sink writing into a std::vector
+///
+/// Unlike output_vector_adapter, this sink is not part of the virtual
+/// output_adapter_protocol hierarchy: it is passed to binary_writer by value as
+/// a template parameter, so write_character()/write_characters() are ordinary
+/// (inlinable) calls with no vtable lookup and no shared_ptr. It is used for the
+/// common `to_cbor`/`to_msgpack`/... into a std::vector.
+template<typename CharType, typename AllocatorType = std::allocator<CharType>>
+class output_vector_sink
+{
+  public:
+    explicit output_vector_sink(std::vector<CharType, AllocatorType>& vec) noexcept
+        : v(vec)
+    {}
+
+    void write_character(CharType c)
+    {
+        v.push_back(c);
+    }
+
+    // no JSON_HEDLEY_NON_NULL here: binary_writer legitimately passes a null
+    // pointer with length 0 for empty strings/binary values. Appending an empty
+    // range is a no-op; the type-erased path tolerates this via the (unattributed)
+    // virtual base, and the concrete sink must do the same.
+    void write_characters(const CharType* s, std::size_t length)
+    {
+        v.insert(v.end(), s, s + length);
+    }
+
+  private:
+    std::vector<CharType, AllocatorType>& v;
+};
+
+/// @brief output sink forwarding to a type-erased output adapter
+///
+/// Wraps the polymorphic output_adapter_t so the same binary_writer template can
+/// also target arbitrary adapters (output streams, strings, user-provided
+/// adapters) via the `output_adapter`-based overloads. Each write still goes
+/// through one virtual call, exactly as before; only the concrete sinks above
+/// avoid it.
+template<typename CharType>
+class output_adapter_sink
+{
+  public:
+    explicit output_adapter_sink(output_adapter_t<CharType> adapter)
+        : oa(std::move(adapter))
+    {
+        JSON_ASSERT(oa);
+    }
+
+    void write_character(CharType c)
+    {
+        oa->write_character(c);
+    }
+
+    // no JSON_HEDLEY_NON_NULL: forwards (null, 0) for empty payloads, exactly as
+    // the type-erased path already did before this sink existed
+    void write_characters(const CharType* s, std::size_t length)
+    {
+        oa->write_characters(s, length);
+    }
+
+  private:
+    output_adapter_t<CharType> oa = nullptr;
+};
+
 template<typename CharType, typename StringType = std::basic_string<CharType>>
 class output_adapter
 {

include/nlohmann/json.hpp

@@ -140,7 +140,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
     friend ::nlohmann::detail::serializer<basic_json>;
     template<typename BasicJsonType>
     friend class ::nlohmann::detail::iter_impl;
-    template<typename BasicJsonType, typename CharType>
+    template<typename BasicJsonType, typename CharType, typename OutputSinkType>
     friend class ::nlohmann::detail::binary_writer;
     template<typename BasicJsonType, typename InputType, typename SAX>
     friend class ::nlohmann::detail::binary_reader;
@@ -4327,7 +4327,9 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
     static std::vector<std::uint8_t> to_cbor(const basic_json& j)
     {
         std::vector<std::uint8_t> result;
-        to_cbor(j, result);
+        result.reserve(detail::binary_reserve_hint(j));
+        detail::binary_writer<basic_json, std::uint8_t, detail::output_vector_sink<std::uint8_t>>(
+            detail::output_vector_sink<std::uint8_t>(result)).write_cbor(j);
         return result;
     }
 
@@ -4350,7 +4352,9 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
     static std::vector<std::uint8_t> to_msgpack(const basic_json& j)
     {
         std::vector<std::uint8_t> result;
-        to_msgpack(j, result);
+        result.reserve(detail::binary_reserve_hint(j));
+        detail::binary_writer<basic_json, std::uint8_t, detail::output_vector_sink<std::uint8_t>>(
+            detail::output_vector_sink<std::uint8_t>(result)).write_msgpack(j);
         return result;
     }
 
@@ -4375,7 +4379,9 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
             const bool use_type = false)
     {
         std::vector<std::uint8_t> result;
-        to_ubjson(j, result, use_size, use_type);
+        result.reserve(detail::binary_reserve_hint(j));
+        detail::binary_writer<basic_json, std::uint8_t, detail::output_vector_sink<std::uint8_t>>(
+            detail::output_vector_sink<std::uint8_t>(result)).write_ubjson(j, use_size, use_type);
         return result;
     }
 
@@ -4403,7 +4409,9 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
             const bjdata_version_t version = bjdata_version_t::draft2)
     {
         std::vector<std::uint8_t> result;
-        to_bjdata(j, result, use_size, use_type, version);
+        result.reserve(detail::binary_reserve_hint(j));
+        detail::binary_writer<basic_json, std::uint8_t, detail::output_vector_sink<std::uint8_t>>(
+            detail::output_vector_sink<std::uint8_t>(result)).write_ubjson(j, use_size, use_type, true, true, version);
         return result;
     }
 
@@ -4430,7 +4438,9 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
     static std::vector<std::uint8_t> to_bson(const basic_json& j)
     {
         std::vector<std::uint8_t> result;
-        to_bson(j, result);
+        result.reserve(detail::binary_reserve_hint(j));
+        detail::binary_writer<basic_json, std::uint8_t, detail::output_vector_sink<std::uint8_t>>(
+            detail::output_vector_sink<std::uint8_t>(result)).write_bson(j);
         return result;
     }
 

tests/src/unit-32bit.cpp

@@ -132,3 +132,37 @@ TEST_CASE("BJData")
         }
     }
 }
+
+TEST_CASE("CBOR")
+{
+    SECTION("parse errors")
+    {
+        SECTION("array/map size larger than std::size_t")
+        {
+            // declared lengths do not fit in a 32-bit std::size_t and must not be truncated
+            std::vector<uint8_t> const varr = {0x9B, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x05};
+            std::vector<uint8_t> const vmap = {0xBB, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x05};
+
+            json _;
+            CHECK_THROWS_WITH_AS(_ = json::from_cbor(varr), "[json.exception.out_of_range.408] syntax error while parsing CBOR size: excessive array size", json::out_of_range&);
+            CHECK(json::from_cbor(varr, true, false).is_discarded());
+
+            CHECK_THROWS_WITH_AS(_ = json::from_cbor(vmap), "[json.exception.out_of_range.408] syntax error while parsing CBOR size: excessive map size", json::out_of_range&);
+            CHECK(json::from_cbor(vmap, true, false).is_discarded());
+        }
+
+        SECTION("array/map size equal to the indefinite-length sentinel")
+        {
+            // on 32-bit platforms a four-byte length of 0xFFFFFFFF aliases unknown_size()
+            std::vector<uint8_t> const varr = {0x9A, 0xFF, 0xFF, 0xFF, 0xFF};
+            std::vector<uint8_t> const vmap = {0xBA, 0xFF, 0xFF, 0xFF, 0xFF};
+
+            json _;
+            CHECK_THROWS_WITH_AS(_ = json::from_cbor(varr), "[json.exception.out_of_range.408] syntax error while parsing CBOR size: excessive array size", json::out_of_range&);
+            CHECK(json::from_cbor(varr, true, false).is_discarded());
+
+            CHECK_THROWS_WITH_AS(_ = json::from_cbor(vmap), "[json.exception.out_of_range.408] syntax error while parsing CBOR size: excessive map size", json::out_of_range&);
+            CHECK(json::from_cbor(vmap, true, false).is_discarded());
+        }
+    }
+}

tests/src/unit-bjdata.cpp

@@ -1347,6 +1347,8 @@ TEST_CASE("BJData")
                     CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vec2), "[json.exception.parse_error.115] parse error at byte 5: syntax error while parsing BJData high-precision number: invalid number text: 1A", json::parse_error);
                     std::vector<uint8_t> const vec3 = {'H', 'i', 2, '1', '.'};
                     CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vec3), "[json.exception.parse_error.115] parse error at byte 5: syntax error while parsing BJData high-precision number: invalid number text: 1.", json::parse_error);
+                    std::vector<uint8_t> const vec_overflow = {'H', 'i', 5, '1', 'e', '4', '0', '0'};
+                    CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vec_overflow), "[json.exception.out_of_range.406] number overflow parsing '1e400'", json::out_of_range);
                     std::vector<uint8_t> const vec4 = {'H', 2, '1', '0'};
                     CHECK_THROWS_WITH_AS(_ = json::from_bjdata(vec4), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing BJData size: expected length type specification (U, i, u, I, m, l, M, L) after '#'; last byte: 0x02", json::parse_error);
                 }
@@ -2587,6 +2589,69 @@ TEST_CASE("BJData")
                 CHECK(json::to_bjdata(json::from_bjdata(v_B), true, true) == v_B);
             }
 
+            SECTION("ndarray with data not matching _ArrayType_ is written as an object")
+            {
+                // A JData-annotated object is only serialized as an ndarray when
+                // its _ArrayData_ elements are actually stored as the number kind
+                // named by _ArrayType_. Otherwise the writer would read the wrong
+                // union member (e.g. a std::string's heap pointer as a uint64) and
+                // emit it, so such an object falls back to a plain object encoding
+                // that still round-trips.
+
+                // string data declared as a uint64 array
+                json const j_str = json({{"_ArrayType_", "uint64"}, {"_ArraySize_", {1}}, {"_ArrayData_", {"pointer"}}});
+                const auto out_str = json::to_bjdata(j_str);
+                CHECK(out_str.at(0) == '{');
+                CHECK(json::from_bjdata(out_str) == j_str);
+
+                // integer data declared as a double array
+                json const j_float = json({{"_ArrayType_", "double"}, {"_ArraySize_", {2}}, {"_ArrayData_", {1, 2}}});
+                const auto out_float = json::to_bjdata(j_float);
+                CHECK(out_float.at(0) == '{');
+                CHECK(json::from_bjdata(out_float) == j_float);
+
+                // a non-integer shape entry is likewise not treated as an ndarray
+                json const j_size = json({{"_ArrayType_", "uint8"}, {"_ArraySize_", {"x"}}, {"_ArrayData_", {1}}});
+                const auto out_size = json::to_bjdata(j_size);
+                CHECK(out_size.at(0) == '{');
+                CHECK(json::from_bjdata(out_size) == j_size);
+
+                // a negative shape entry is not a usable dimension either
+                json const j_neg = json::parse(R"({"_ArrayType_":"uint8","_ArraySize_":[-1],"_ArrayData_":[1]})");
+                const auto out_neg = json::to_bjdata(j_neg);
+                CHECK(out_neg.at(0) == '{');
+                CHECK(json::from_bjdata(out_neg) == j_neg);
+            }
+
+            SECTION("ndarray parsed from text is written as a typed array")
+            {
+                // json::parse stores a non-negative integer as number_unsigned while
+                // the C++ API stores an int literal as number_integer, so _ArrayType_
+                // names the wire type rather than the storage. Both storages have to
+                // produce the same typed array for every type.
+                for (const char* type :
+                        {"uint8", "int8", "uint16", "int16", "uint32", "int32", "uint64", "int64", "char", "byte"
+                        })
+                {
+                    CAPTURE(type);
+                    const std::string text = std::string(R"({"_ArrayType_":")") + type +
+                                             R"(","_ArraySize_":[2,3],"_ArrayData_":[1,2,3,4,5,6]})";
+                    const auto from_text = json::to_bjdata(json::parse(text));
+                    CHECK(from_text.at(0) == '[');
+                    CHECK(from_text == json::to_bjdata(json({{"_ArrayType_", type}, {"_ArraySize_", {2, 3}}, {"_ArrayData_", {1, 2, 3, 4, 5, 6}}})));
+                }
+
+                // negative values under a signed type behave the same way
+                const auto from_neg = json::to_bjdata(json::parse(R"({"_ArrayType_":"int32","_ArraySize_":[2],"_ArrayData_":[-5,7]})"));
+                CHECK(from_neg.at(0) == '[');
+                CHECK(from_neg == json::to_bjdata(json({{"_ArrayType_", "int32"}, {"_ArraySize_", {2}}, {"_ArrayData_", {-5, 7}}})));
+
+                // and so do the floating point types
+                const auto from_float = json::to_bjdata(json::parse(R"({"_ArrayType_":"double","_ArraySize_":[2],"_ArrayData_":[1.5,2.5]})"));
+                CHECK(from_float.at(0) == '[');
+                CHECK(from_float == json::to_bjdata(json({{"_ArrayType_", "double"}, {"_ArraySize_", {2}}, {"_ArrayData_", {1.5, 2.5}}})));
+            }
+
             SECTION("optimized ndarray (type and vector-size as 1D array)")
             {
                 // create vector with two elements of the same type

tests/src/unit-bson.cpp

@@ -529,6 +529,41 @@ TEST_CASE("BSON")
             CHECK(json::from_bson(result, true, false) == j);
         }
 
+        SECTION("non-empty object with binary member without subtype")
+        {
+            const size_t N = 10;
+            const auto s = std::vector<std::uint8_t>(N, 'x');
+            json const j =
+            {
+                { "entry", json::binary(s) }
+            };
+
+            CHECK(!j.at("entry").get_binary().has_subtype());
+
+            std::vector<std::uint8_t> const expected =
+            {
+                0x1B, 0x00, 0x00, 0x00, // size (little endian)
+                0x05, // entry: binary
+                'e', 'n', 't', 'r', 'y', '\x00',
+
+                0x0A, 0x00, 0x00, 0x00, // size of binary (little endian)
+                0x00, // Generic binary subtype
+                0x78, 0x78, 0x78, 0x78, 0x78, 0x78, 0x78, 0x78, 0x78, 0x78,
+
+                0x00 // end marker
+            };
+
+            const auto result = json::to_bson(j);
+            CHECK(result == expected);
+
+            // roundtrip adds the generic binary subtype
+            const auto roundtrip = json::from_bson(result);
+            CHECK(roundtrip != j);
+            CHECK(roundtrip.at("entry").get_binary().has_subtype());
+            CHECK(roundtrip.at("entry").get_binary().subtype() == 0);
+            CHECK(json::from_bson(result, true, false) == roundtrip);
+        }
+
         SECTION("non-empty object with binary member with subtype")
         {
             // an MD5 hash

tests/src/unit-cbor.cpp

@@ -1999,6 +1999,42 @@ TEST_CASE("CBOR regressions")
 }
 #endif
 
+TEST_CASE("CBOR definite length equal to the indefinite-length sentinel")
+{
+    // A definite-length array or map whose declared element count equals the
+    // reserved unknown_size() sentinel (SIZE_MAX) must be rejected. Otherwise
+    // it is read as an indefinite-length container and the following bytes are
+    // silently accepted instead of the (impossible) count being reported.
+    json _;
+
+    SECTION("array")
+    {
+        // 0x9B: array with eight-byte length; length = 0xFFFFFFFFFFFFFFFF
+        const std::vector<uint8_t> input = {0x9B, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x01, 0x02, 0xFF};
+        CHECK_THROWS_WITH_AS(_ = json::from_cbor(input), "[json.exception.out_of_range.408] syntax error while parsing CBOR size: excessive array size", json::out_of_range&);
+    }
+
+    SECTION("map")
+    {
+        // 0xBB: map with eight-byte length; length = 0xFFFFFFFFFFFFFFFF
+        const std::vector<uint8_t> input = {0xBB, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x61, 0x61, 0x01, 0xFF};
+        CHECK_THROWS_WITH_AS(_ = json::from_cbor(input), "[json.exception.out_of_range.408] syntax error while parsing CBOR size: excessive map size", json::out_of_range&);
+    }
+
+    SECTION("indefinite-length containers are unaffected")
+    {
+        CHECK(json::from_cbor(std::vector<uint8_t>({0x9F, 0x01, 0x02, 0xFF})) == json({1, 2}));
+        CHECK(json::from_cbor(std::vector<uint8_t>({0xBF, 0x61, 0x61, 0x01, 0xFF})) == json({{"a", 1}}));
+    }
+
+    SECTION("ordinary four-byte length containers are unaffected")
+    {
+        // 0x9A/0xBA carry a four-byte length; a normal count still parses
+        CHECK(json::from_cbor(std::vector<uint8_t>({0x9A, 0x00, 0x00, 0x00, 0x02, 0x01, 0x02})) == json({1, 2}));
+        CHECK(json::from_cbor(std::vector<uint8_t>({0xBA, 0x00, 0x00, 0x00, 0x01, 0x61, 0x61, 0x01})) == json({{"a", 1}}));
+    }
+}
+
 TEST_CASE("CBOR roundtrips" * doctest::skip())
 {
     SECTION("input from flynn")

tests/src/unit-class_parser.cpp

@@ -1440,6 +1440,13 @@ TEST_CASE("parser class")
             ]
         )";
 
+        const auto* structured_object = R"(
+            {
+                "foo": [1, 2],
+                "bar": 3
+            }
+        )";
+
         SECTION("filter nothing")
         {
             const json j_object = json::parse(s_object, [](int /*unused*/, json::parse_event_t /*unused*/, const json& /*unused*/) noexcept
@@ -1515,6 +1522,48 @@ TEST_CASE("parser class")
             CHECK (j_filtered2 == json({1}));
         }
 
+        SECTION("filter array in object")
+        {
+            // the array is discarded once it is already stored under its key
+            const json j_filtered1 = json::parse(structured_object, [](int /*unused*/, json::parse_event_t e, const json& /*parsed*/) noexcept
+            {
+                return e != json::parse_event_t::array_end;
+            });
+
+            CHECK (j_filtered1 == json({{"bar", 3}}));
+
+            // the array is discarded before it is stored, leaving the
+            // placeholder the key event wrote
+            const json j_filtered2 = json::parse(structured_object, [](int /*unused*/, json::parse_event_t e, const json& /*parsed*/) noexcept
+            {
+                return e != json::parse_event_t::array_start;
+            });
+
+            CHECK (j_filtered2 == json({{"bar", 3}}));
+        }
+
+        SECTION("filter value in object")
+        {
+            // the value is discarded after its key was kept, leaving the
+            // placeholder the key event wrote
+            const json j_filtered1 = json::parse(structured_object, [](int /*unused*/, json::parse_event_t e, const json & parsed) noexcept
+            {
+                return !(e == json::parse_event_t::value && parsed == json(3));
+            });
+
+            CHECK (j_filtered1 == json({{"foo", {1, 2}}}));
+
+            // the same value is discarded together with its key, so no
+            // placeholder was stored for it
+            const json j_filtered2 = json::parse(structured_object, [](int /*unused*/, json::parse_event_t e, const json & parsed) noexcept
+            {
+                return !((e == json::parse_event_t::key && parsed == json("bar")) ||
+                         (e == json::parse_event_t::value && parsed == json(3)));
+            });
+
+            CHECK (j_filtered2 == json({{"foo", {1, 2}}}));
+        }
+
         SECTION("filter specific events")
         {
             SECTION("first closing event")

tests/src/unit-deserialization.cpp

@@ -427,6 +427,30 @@ TEST_CASE("deserialization")
                 CHECK(l.events == std::vector<std::string>({"boolean(true)"}));
             }
 
+            SECTION("from std::vector<signed char>")
+            {
+                std::vector<signed char> const v = {'t', 'r', 'u', 'e'};
+                CHECK(json::parse(v) == json(true));
+                CHECK(json::accept(v));
+
+                SaxEventLogger l;
+                CHECK(json::sax_parse(v, &l));
+                CHECK(l.events.size() == 1);
+                CHECK(l.events == std::vector<std::string>({"boolean(true)"}));
+
+                // bytes outside ASCII are negative here and must not be sign-extended;
+                // 0xC3 and 0xA9 do not fit in signed char (MSVC C4309), so spell them as negative values
+                std::vector<signed char> const umlaut = {'"', static_cast<signed char>(0xC3 - 0x100), static_cast<signed char>(0xA9 - 0x100), '"'};
+                CHECK(json::parse(umlaut) == json("\xC3\xA9"));
+                CHECK(json::accept(umlaut));
+
+                // 0xFF (spelled as -1 to stay in range) must not be reported as end of input
+                std::vector<signed char> const trailing = {'t', 'r', 'u', 'e', static_cast<signed char>(0xFF - 0x100)};
+                json _;
+                CHECK_THROWS_WITH_AS(_ = json::parse(trailing), "[json.exception.parse_error.101] parse error at line 1, column 5: syntax error while parsing value - invalid literal; last read: 'true\xFF'; expected end of input", json::parse_error&);
+                CHECK(!json::accept(trailing));
+            }
+
             SECTION("from std::array")
             {
                 std::array<uint8_t, 5> const v { {'t', 'r', 'u', 'e'} };

tests/src/unit-regression2.cpp

@@ -1530,4 +1530,29 @@ TEST_CASE("issue #4320 - custom base class must not leak nlohmann::detail into A
     CHECK(j == json({{"x", 1.0}, {"y", 2.0}, {"z", 3.0}}));
 }
 
+TEST_CASE("issue #5338 - truncated CBOR tagged binary subtype is rejected")
+{
+    const std::vector<std::vector<std::uint8_t>> truncated_tags =
+    {
+        {0xD8},
+        {0xD9, 0x00},
+        {0xDA, 0x00, 0x00, 0x00},
+        {0xDB, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}
+    };
+
+    for (const auto& data : truncated_tags)
+    {
+        CAPTURE(data);
+        for (const auto tag_handler :
+                {
+                    json::cbor_tag_handler_t::ignore, json::cbor_tag_handler_t::store
+                })
+        {
+            CAPTURE(tag_handler);
+            const auto result = json::from_cbor(data, true, false, tag_handler);
+            CHECK(result.is_discarded());
+        }
+    }
+}
+
 DOCTEST_CLANG_SUPPRESS_WARNING_POP

tests/src/unit-type_traits.cpp

@@ -53,4 +53,34 @@ TEST_CASE("type traits")
             // NOLINTEND(hicpp-avoid-c-arrays,modernize-avoid-c-arrays,cppcoreguidelines-avoid-c-arrays)
         }
     }
+
+    SECTION("char_traits")
+    {
+        SECTION("to_int_type does not sign-extend")
+        {
+            using unsigned_traits = nlohmann::detail::char_traits<unsigned char>;
+            using signed_traits = nlohmann::detail::char_traits<signed char>;
+
+            CHECK(unsigned_traits::to_int_type(static_cast<unsigned char>(0x7F)) == 0x7F);
+            CHECK(unsigned_traits::to_int_type(static_cast<unsigned char>(0x80)) == 0x80);
+            CHECK(unsigned_traits::to_int_type(static_cast<unsigned char>(0xFF)) == 0xFF);
+
+            CHECK(signed_traits::to_int_type(static_cast<signed char>(0x7F)) == 0x7F);
+            // 0x80 and 0xFF do not fit in signed char (MSVC C4309), so spell them as negative values
+            CHECK(signed_traits::to_int_type(static_cast<signed char>(0x80 - 0x100)) == 0x80);
+            CHECK(signed_traits::to_int_type(static_cast<signed char>(0xFF - 0x100)) == 0xFF);
+        }
+
+        SECTION("no byte value collides with eof")
+        {
+            using unsigned_traits = nlohmann::detail::char_traits<unsigned char>;
+            using signed_traits = nlohmann::detail::char_traits<signed char>;
+
+            for (int i = 0; i < 256; ++i)
+            {
+                CHECK(unsigned_traits::to_int_type(static_cast<unsigned char>(i)) != unsigned_traits::eof());
+                CHECK(signed_traits::to_int_type(static_cast<signed char>(i)) != signed_traits::eof());
+            }
+        }
+    }
 }

tests/src/unit-ubjson.cpp

@@ -819,6 +819,8 @@ TEST_CASE("UBJSON")
                     CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vec2), "[json.exception.parse_error.115] parse error at byte 5: syntax error while parsing UBJSON high-precision number: invalid number text: 1A", json::parse_error);
                     std::vector<uint8_t> const vec3 = {'H', 'i', 2, '1', '.'};
                     CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vec3), "[json.exception.parse_error.115] parse error at byte 5: syntax error while parsing UBJSON high-precision number: invalid number text: 1.", json::parse_error);
+                    std::vector<uint8_t> const vec_overflow = {'H', 'i', 5, '1', 'e', '4', '0', '0'};
+                    CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vec_overflow), "[json.exception.out_of_range.406] number overflow parsing '1e400'", json::out_of_range&);
                     std::vector<uint8_t> const vec4 = {'H', 2, '1', '0'};
                     CHECK_THROWS_WITH_AS(_ = json::from_ubjson(vec4), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing UBJSON size: expected length type specification (U, i, I, l, L) after '#'; last byte: 0x02", json::parse_error);
                 }