State the frame stack invariants of json_sax_dom_parser as assertions

clang-tidy's static analyzer analyzes the forwarding SAX handlers of the
error recovery tests (RecoveringDomParser, BasicRecoveringParser) as entry
points, with the state of their json_sax_dom_parser member unknown. It then
assumes a frame stack of capacity 0 that grow() cannot enlarge
(clang-analyzer-security.ArrayBound) and a collected container without an
enclosing frame (clang-analyzer-core.NonNullParamChecker). Neither can
happen: the stack starts with local_capacity frames, and only a container
inside another one is collected. Assert both, which also lets the analyzer
discard these paths.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

.github/CONTRIBUTING.md

@@ -111,6 +111,8 @@ context which existing file needs to be extended, and only very few cases requir
 When fixing a bug, edit `unit-regression3.cpp` and add a section referencing the fixed issue.
 `unit-regression2.cpp` holds the older tests; the two files exist because a single one grew large enough for the
 MinGW linker to fail relocating it, so please keep adding to the smaller file rather than growing the larger one.
+Tests that call `sax_parse` go into `unit-sax_parse.cpp` rather than into a large test file: every call instantiates
+the parser and the binary reader that recover from errors, which grows a test file considerably.
 
 #### Exceptions
 

README.md

@@ -495,7 +495,7 @@ bool key(string_t& val);
 bool parse_error(std::size_t position, const std::string& last_token, const detail::exception& ex);
 ```
 
-The return value of each function determines whether parsing should proceed.
+The return value of each function determines whether parsing should proceed. For `parse_error`, returning `true` [recovers from the error](https://json.nlohmann.me/features/parsing/error_recovery/): the parser repairs the input and continues.
 
 To implement your own SAX handler, proceed as follows:
 
@@ -503,7 +503,7 @@ To implement your own SAX handler, proceed as follows:
 2. Create an object of your SAX interface class, e.g. `my_sax`.
 3. Call `bool json::sax_parse(input, &my_sax)`; where the first parameter can be any input like a string or an input stream and the second parameter is a pointer to your SAX interface.
 
-Note the `sax_parse` function only returns a `bool` indicating the result of the last executed SAX event. It does not return a  `json` value - it is up to you to decide what to do with the SAX events. Furthermore, no exceptions are thrown in case of a parse error -- it is up to you what to do with the exception object passed to your `parse_error` implementation. Internally, the SAX interface is used for the DOM parser (class `json_sax_dom_parser`) as well as the acceptor (`json_sax_acceptor`), see file [`json_sax.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/input/json_sax.hpp).
+Note the `sax_parse` function only returns a `bool` indicating whether the input was parsed without errors and no SAX event returned `false`. It does not return a  `json` value - it is up to you to decide what to do with the SAX events. Furthermore, no exceptions are thrown in case of a parse error -- it is up to you what to do with the exception object passed to your `parse_error` implementation. Internally, the SAX interface is used for the DOM parser (class `json_sax_dom_parser`) as well as the acceptor (`json_sax_acceptor`), see file [`json_sax.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/input/json_sax.hpp).
 
 ### STL-like access
 

docs/mkdocs/docs/api/basic_json/sax_parse.md

@@ -96,7 +96,9 @@ The SAX event lister must follow the interface of [`json_sax`](../json_sax/index
 
 ## Return value
 
-return value of the last processed SAX event
+`#!cpp true` if the input was parsed without errors and no SAX event returned `#!cpp false`; `#!cpp false` otherwise.
+In particular, the result is `#!cpp false` for input with errors, even if the SAX parser recovered from all of them
+(see [error recovery](../../features/parsing/error_recovery.md)).
 
 ## Exception safety
 
@@ -153,6 +155,7 @@ use the same file with stdio functions, but it must not wait for another thread
 - Added `ignore_trailing_commas` in version 3.13.0.
 - Added `tag_handler` in version 3.13.0.
 - Extended container support (1) to include types with lvalue-only ADL `begin`/`end` (matching `std::begin`/`std::end` semantics) in version 3.13.0.
+- Recovering from parse errors (see [`parse_error`](../json_sax/parse_error.md)) added in version 3.13.0.
 - Extended overload (2) to accept heterogeneous iterator+sentinel pairs (C++20 ranges support) in version 3.13.0.
 - `JSON_PRECISE_STREAM_POSITION` added in version 3.13.0 to optionally leave a `#!cpp std::istream` positioned right
   after the parsed value when `strict` is `#!cpp false`.

docs/mkdocs/docs/api/json_sax/index.md

@@ -7,7 +7,8 @@ struct json_sax;
 
 This class describes the SAX interface used by [sax_parse](../basic_json/sax_parse.md). Each function is called in
 different situations while the input is parsed. The boolean return value informs the parser whether to continue
-processing the input.
+processing the input; for [`parse_error`](parse_error.md), it decides whether to
+[recover from the error](../../features/parsing/error_recovery.md).
 
 For instance, parsing the JSON text `{"a": [1, true]}` triggers the following callbacks, in order:
 

docs/mkdocs/docs/api/json_sax/parse_error.md

@@ -21,11 +21,18 @@ A parse error occurred.
 
 ## Return value
 
-Whether parsing should proceed (**must return `#!cpp false`**).
+Whether to recover from the error:
+
+- `#!cpp false` stops parsing.
+- `#!cpp true` recovers from the error: the error is repaired and parsing continues. If that is not possible, which
+  happens in the binary formats when the end of the item with the error is unknown, the value read so far is completed
+  and parsing stops. See [error recovery](../../features/parsing/error_recovery.md) for how errors are repaired.
+
+Either way, [`sax_parse`](../basic_json/sax_parse.md) returns `#!cpp false`.
 
 ## Examples
 
-??? example
+??? example "Example: (1) the SAX interface"
 
     The example below shows how the SAX interface is used.
 
@@ -39,12 +46,29 @@ Whether parsing should proceed (**must return `#!cpp false`**).
     --8<-- "examples/sax_parse.output"
     ```
 
+??? example "Example: (2) recovering from errors"
+
+    The example below shows how a SAX parser recovers from errors.
+
+    ```cpp
+    --8<-- "examples/sax_parse__error_recovery.cpp"
+    ```
+    
+    Output:
+    
+    ```
+    --8<-- "examples/sax_parse__error_recovery.output"
+    ```
+
 ## See also
 
 - [sax_parse](../basic_json/sax_parse.md) - SAX parser
 - [Parsing and Exceptions](../../features/parsing/parse_exceptions.md) - the article on handling parse errors without
   exceptions
+- [Error Recovery](../../features/parsing/error_recovery.md) - the article on recovering from parse errors
 
 ## Version history
 
 - Added in version 3.2.0.
+- Returning `#!cpp true` recovers from the error since version 3.13.0; before, parsing stopped, but the result of
+  [`sax_parse`](../basic_json/sax_parse.md) could be wrong.

docs/mkdocs/docs/examples/sax_parse__error_recovery.cpp

@@ -0,0 +1,43 @@
+#include <iostream>
+#include <iomanip>
+#include <nlohmann/json.hpp>
+
+using json = nlohmann::json;
+
+// a SAX parser that creates a JSON value like json::parse does, but that
+// recovers from parse errors instead of stopping at the first one
+class recovering_parser : public nlohmann::detail::json_sax_dom_parser<json>
+{
+  public:
+    explicit recovering_parser(json& result)
+        : nlohmann::detail::json_sax_dom_parser<json>(result, false)
+    {}
+
+    bool parse_error(std::size_t position,
+                     const std::string& /*last_token*/,
+                     const json::exception& ex)
+    {
+        std::cout << "byte " << position << ": " << ex.what() << '\n';
+
+        // repair the input and continue
+        return true;
+    }
+};
+
+int main()
+{
+    // JSON text with several mistakes that ends too early
+    const std::string text = R"({
+    "name": "Hello World",
+    "tags": ["a" "b",],
+    "valid": tru,
+    "size": 1.,
+    "nested": {"x": 1)";
+
+    json result;
+    recovering_parser sax(result);
+    const bool valid = json::sax_parse(text, &sax);
+
+    std::cout << "\nvalid JSON: " << std::boolalpha << valid << '\n'
+              << std::setw(4) << result << std::endl;
+}

docs/mkdocs/docs/examples/sax_parse__error_recovery.output

@@ -0,0 +1,19 @@
+byte 49: [json.exception.parse_error.101] parse error at line 3, column 20: syntax error while parsing array - unexpected string literal; expected ']'
+byte 51: [json.exception.parse_error.101] parse error at line 3, column 22: syntax error while parsing value - unexpected ']'; expected '[', '{', or a literal
+byte 70: [json.exception.parse_error.101] parse error at line 4, column 17: syntax error while parsing value - invalid literal; last read: '"valid": tru,'
+byte 86: [json.exception.parse_error.101] parse error at line 5, column 15: syntax error while parsing value - invalid number; expected digit after '.'; last read: '1.,'
+byte 109: [json.exception.parse_error.101] parse error at line 6, column 22: syntax error while parsing object - unexpected end of input; expected '}'
+
+valid JSON: false
+{
+    "name": "Hello World",
+    "nested": {
+        "x": 1
+    },
+    "size": 1,
+    "tags": [
+        "a",
+        "b"
+    ],
+    "valid": null
+}

docs/mkdocs/docs/features/parsing/error_recovery.md

@@ -0,0 +1,122 @@
+# Error Recovery
+
+By default, parsing stops at the first error. With the [SAX interface](sax_interface.md), you can instead ask the
+parser to *recover*: to repair the error and continue, so that you get as much as possible out of malformed input, for
+instance a file that was cut off, JSON edited by hand, or the output of a language model.
+
+## Recovering from errors
+
+The SAX parser's [`parse_error`](../../api/json_sax/parse_error.md) function is called for every error. Its return value
+decides what happens next:
+
+- `#!cpp false` stops parsing. This is what the SAX parsers of the library do, so [`parse`](../../api/basic_json/parse.md)
+  and [`accept`](../../api/basic_json/accept.md) never recover.
+- `#!cpp true` repairs the error and continues parsing.
+
+When recovering, the SAX parser still receives well-formed events: every `start_object` or `start_array` is followed by
+the matching `end_object` or `end_array`, and every `key` is followed by exactly one value. A SAX parser that creates a
+JSON value, such as the one in the example below, therefore gets a complete value. Parsing always ends, and
+[`sax_parse`](../../api/basic_json/sax_parse.md) returns `#!cpp false` for input that is not valid JSON, even if every
+error was repaired. Each token is reported at most once, and the SAX parser can stop at any error by returning
+`#!cpp false`.
+
+!!! example
+
+    The example below derives a SAX parser from the library's parser for `json` values (`json_sax_dom_parser`),
+    and recovers from all errors.
+
+    ```cpp
+    --8<-- "examples/sax_parse__error_recovery.cpp"
+    ```
+    
+    Output:
+    
+    ```
+    --8<-- "examples/sax_parse__error_recovery.output"
+    ```
+
+## How errors are repaired
+
+Each error is repaired with the smallest local edit: a missing separator is inserted, a stray token is removed, what can
+be read of a broken string or number is kept, and a value that cannot be read at all becomes `#!json null`.
+
+| Mistake                   | Repair                                                                         | Example                                  | Result                     |
+|---------------------------|--------------------------------------------------------------------------------|------------------------------------------|----------------------------|
+| missing `,` or `:`        | inserted                                                                       | `#!json [1 2]`, `#!json {"a" 1}`         | `[1,2]`, `{"a":1}`         |
+| missing value             | `#!json null` for an object key or between commas in an array                  | `#!json {"a":}`, `#!json [1,,2]`         | `{"a":null}`, `[1,null,2]` |
+| trailing comma            | removed                                                                        | `#!json [1,2,]`                          | `[1,2]`                    |
+| broken string             | invalid escapes and bytes are replaced (see below); a line break ends the string | `#!json ["a\qb"]`                        | `["aqb"]`                  |
+| broken number             | the longest valid beginning is kept                                            | `#!json [1., 2e+]`                       | `[1,2]`                    |
+| unreadable value          | `#!json null`                                                                  | `#!json [1, NaN, tru]`                   | `[1,null,null]`            |
+| number too large          | passed as infinity, together with its text                                     | `#!json [1e999]`                         | infinity (see below)       |
+| stray `:`                 | removed                                                                        | `#!json ["a":1]`                         | `["a",1]`                  |
+| member without a key      | skipped up to the next `,` or `}`                                              | `#!json {1:2, "b":3}`                    | `{"b":3}`                  |
+| wrong closing bracket     | closes the innermost array or object                                           | `#!json {"a":[1,2}, "b":3}`              | `{"a":[1,2],"b":3}`        |
+| input ends too early      | all open arrays and objects are closed                                         | `#!json {"a":[1,2`                       | `{"a":[1,2]}`              |
+| text before the value     | skipped                                                                        | `#!json )]}'{"a":1}`                     | `{"a":1}`                  |
+
+In a string, an unknown escape like `\q` stands for the escaped character (`q`), as in JavaScript. An invalid `\u`
+escape, a lone surrogate, and ill-formed UTF-8 are each replaced by U+FFFD (REPLACEMENT CHARACTER), and control
+characters are kept. A string without its closing quote ends at the next line break or at the end of the input.
+
+The input after the top-level value is not repaired: as without recovery, it is reported as an error, and parsing stops.
+
+## Binary formats
+
+The binary formats ([BJData](../binary_formats/bjdata.md), [BON8](../binary_formats/bon8.md),
+[BSON](../binary_formats/bson.md), [CBOR](../binary_formats/cbor.md), [MessagePack](../binary_formats/messagepack.md),
+and [UBJSON](../binary_formats/ubjson.md)) have no delimiters to find the next value by. So what can be repaired depends
+on whether the end of the item with the error is known, a distinction that
+[RFC 8949, Section 5.3](https://www.rfc-editor.org/rfc/rfc8949.html#section-5.3) makes for CBOR, too.
+
+If the item is complete, but cannot be passed on as it is, it is replaced, and parsing continues after it:
+
+| Mistake                                                             | Formats                 | Repair                                                                  |
+|---------------------------------------------------------------------|-------------------------|-------------------------------------------------------------------------|
+| tag, if `tag_handler` is `cbor_tag_handler_t::error` (the default)  | CBOR                    | ignored                                                                 |
+| simple value other than `false`, `true`, and `null`, like undefined | CBOR                    | `#!json null`                                                           |
+| number too large for a custom `number_float_t`, like `float`        | all                     | infinity                                                                |
+| character (`C`) that is not ASCII                                   | BJData, UBJSON          | U+FFFD                                                                  |
+| invalid high-precision number (`H`)                                 | BJData, UBJSON          | the longest valid beginning is kept, as for JSON text, or `#!json null` |
+| high-precision number too large                                     | BJData, UBJSON          | passed as infinity, together with its text                              |
+| object key that is not a string                                     | BON8, CBOR, MessagePack | the member is skipped                                                   |
+| element of a type the library does not read, like ObjectId or date  | BSON                    | `#!json null`                                                           |
+| string without its terminator                                       | BSON                    | kept                                                                    |
+| document whose size does not match its content                      | BSON                    | kept                                                                    |
+
+CBOR tags and simple values are repaired as [RFC 8949, Section 6.1](https://www.rfc-editor.org/rfc/rfc8949.html#section-6.1)
+suggests for converting CBOR to JSON. By default, [`sax_parse`](../../api/basic_json/sax_parse.md) reports every tag as
+an error; when recovering, tags are then ignored like with
+[`cbor_tag_handler_t::ignore`](../../api/basic_json/cbor_tag_handler_t.md). Strings that are not valid UTF-8 are no
+error: like [`from_cbor`](../../api/basic_json/from_cbor.md) and the other functions by default, `sax_parse` passes
+them on as they are.
+
+After any other error, the end of the item is unknown: the input ended, a byte is not a valid type marker, or a size
+cannot be right. Parsing then stops, and the value read so far is completed: a key that waits for its value gets
+`#!json null`, and all open arrays and objects are closed. This keeps everything before the error of an input that was
+cut off. The exception is BSON, which stores the size of every document: an element whose end is unknown gets
+`#!json null`, the rest of its document is skipped, and parsing continues after the document.
+
+## Limitations
+
+- A repair is a guess. For example, `#!json {"a" "b": 1}` could be meant as `#!json {"a": "b"}` or as
+  `#!json {"a": null, "b": 1}`; it is repaired to the former. Treat recovered values as a best effort, and check the
+  reported errors.
+- A closing bracket always closes the innermost array or object. If a bracket is missing rather than wrong, the
+  repair differs from the intention: `#!json {"a": {"b": [1, 2}, "c": 3}` is repaired to
+  `#!json {"a": {"b": [1, 2], "c": 3}}`, although `#!json {"a": {"b": [1, 2]}, "c": 3}` may have been meant.
+- Keys without quotes, and strings in single quotes, are not supported; such members are skipped.
+- In the binary formats, a member that is skipped because its key is not a string is lost, and so are the elements of a
+  BSON document after one whose end is unknown.
+- A number that is too large for `number_float_t` is passed as positive or negative infinity. The SAX parser's
+  `number_float` also gets the number's text, but a JSON value cannot store it, and
+  [`dump`](../../api/basic_json/dump.md) serializes infinity as `#!json null`.
+- When parsing is not strict (see [`sax_parse`](../../api/basic_json/sax_parse.md)), a repair may read parts of the
+  input after the value, for instance of the next value in a stream of concatenated values.
+
+## See also
+
+- [SAX interface](sax_interface.md) - implement a custom SAX handler
+- [`parse_error`](../../api/json_sax/parse_error.md) - the SAX event for parse errors
+- [`sax_parse`](../../api/basic_json/sax_parse.md) - generate SAX events
+- [parsing and exceptions](parse_exceptions.md) - control error handling

docs/mkdocs/docs/features/parsing/index.md

@@ -103,7 +103,7 @@ You can influence a DOM parse without switching to the SAX interface by passing
 When the input is not valid JSON, the `parse` function throws an exception by default. If exceptions are undesired or
 unavailable, the parser can instead return a discarded value, or [`accept`](../../api/basic_json/accept.md) can be used
 to only check whether an input is valid JSON. See [parsing and exceptions](parse_exceptions.md) for the available
-options.
+options. To get as much as possible out of malformed input, a SAX parser can [recover from errors](error_recovery.md).
 
 ## See also
 
@@ -114,4 +114,5 @@ options.
 - [parser callbacks](parser_callbacks.md) - influence the parsing by a callback function
 - [SAX interface](sax_interface.md) - implement a custom SAX handler
 - [parsing and exceptions](parse_exceptions.md) - control error handling
+- [error recovery](error_recovery.md) - get as much as possible out of malformed input
 - [parsing untrusted input](untrusted_input.md) - what to consider when parsing input from untrusted sources

docs/mkdocs/docs/features/parsing/parse_exceptions.md

@@ -64,7 +64,8 @@ bool parse_error(std::size_t position,
                  const json::exception& ex);
 ```
 
-The return value indicates whether the parsing should continue, so the function should usually return `#!cpp false`.
+The return value decides whether to stop parsing (`#!cpp false`) or to repair the error and continue
+(`#!cpp true`); see [error recovery](error_recovery.md) for the latter.
 
 ??? example "Example: report parse errors without exceptions"
 

docs/mkdocs/docs/features/parsing/sax_interface.md

@@ -60,7 +60,8 @@ bool key(string_t& val);
 bool parse_error(std::size_t position, const std::string& last_token, const json::exception& ex);
 ```
 
-The return value of each function determines whether parsing should proceed.
+The return value of each function determines whether parsing should proceed. For `parse_error`, returning
+`#!cpp true` [recovers from the error](error_recovery.md).
 
 To implement your own SAX handler, proceed as follows:
 
@@ -68,7 +69,7 @@ To implement your own SAX handler, proceed as follows:
 2. Create an object of your SAX interface class, e.g. `my_sax`.
 3. Call `#!cpp bool json::sax_parse(input, &my_sax);` where the first parameter can be any input like a string or an input stream and the second parameter is a pointer to your SAX interface.
 
-Note the `sax_parse` function only returns a `#!cpp bool` indicating the result of the last executed SAX event. It does not return `json` value - it is up to you to decide what to do with the SAX events. Furthermore, no exceptions are thrown in case of a parse error - it is up to you what to do with the exception object passed to your `parse_error` implementation. Internally, the SAX interface is used for the DOM parser (class `json_sax_dom_parser`) as well as the acceptor (`json_sax_acceptor`), see file `json_sax.hpp`.
+Note the `sax_parse` function only returns a `#!cpp bool` indicating whether the input was parsed without errors and no SAX event returned `#!cpp false`. It does not return `json` value - it is up to you to decide what to do with the SAX events. Furthermore, no exceptions are thrown in case of a parse error - it is up to you what to do with the exception object passed to your `parse_error` implementation. Internally, the SAX interface is used for the DOM parser (class `json_sax_dom_parser`) as well as the acceptor (`json_sax_acceptor`), see file `json_sax.hpp`.
 
 ## See also
 

docs/mkdocs/mkdocs.yml

@@ -88,6 +88,7 @@ nav:
       - features/performance.md
       - Parsing:
           - features/parsing/index.md
+          - features/parsing/error_recovery.md
           - features/parsing/json_lines.md
           - features/parsing/parse_exceptions.md
           - features/parsing/parser_callbacks.md

include/nlohmann/detail/conversions/from_json.hpp

@@ -8,7 +8,7 @@
 
 #pragma once
 
-#include <algorithm> // min, transform
+#include <algorithm> // transform
 #include <array> // array
 #include <forward_list> // forward_list
 #include <iterator> // inserter, front_inserter, end
@@ -69,10 +69,12 @@ void from_json(const BasicJsonType& j, std::optional<T>& opt)
 }
 #endif // JSON_HAS_CPP_17
 
-// read a number value of any of the three number types into val;
-// returns false (leaving val untouched) if j is not a number
-template<typename BasicJsonType, typename ArithmeticType>
-bool get_number_value(const BasicJsonType& j, ArithmeticType& val)
+// overloads for basic_json template parameters
+template < typename BasicJsonType, typename ArithmeticType,
+           enable_if_t < std::is_arithmetic<ArithmeticType>::value&&
+                         !std::is_same<ArithmeticType, typename BasicJsonType::boolean_t>::value,
+                         int > = 0 >
+void get_arithmetic_value(const BasicJsonType& j, ArithmeticType& val)
 {
     switch (static_cast<value_t>(j))
     {
@@ -83,17 +85,17 @@ bool get_number_value(const BasicJsonType& j, ArithmeticType& val)
             const auto* ptr = j.template get_ptr<const typename BasicJsonType::number_unsigned_t*>();
             JSON_ASSERT(ptr != nullptr);
             val = static_cast<ArithmeticType>(*ptr);
-            return true;
+            break;
         }
         case value_t::number_integer:
         {
             val = static_cast<ArithmeticType>(*j.template get_ptr<const typename BasicJsonType::number_integer_t*>());
-            return true;
+            break;
         }
         case value_t::number_float:
         {
             val = static_cast<ArithmeticType>(*j.template get_ptr<const typename BasicJsonType::number_float_t*>());
-            return true;
+            break;
         }
 
         case value_t::null:
@@ -104,20 +106,7 @@ bool get_number_value(const BasicJsonType& j, ArithmeticType& val)
         case value_t::binary:
         case value_t::discarded:
         default:
-            return false;
-    }
-}
-
-// overloads for basic_json template parameters
-template < typename BasicJsonType, typename ArithmeticType,
-           enable_if_t < std::is_arithmetic<ArithmeticType>::value&&
-                         !std::is_same<ArithmeticType, typename BasicJsonType::boolean_t>::value,
-                         int > = 0 >
-void get_arithmetic_value(const BasicJsonType& j, ArithmeticType& val)
-{
-    if (JSON_HEDLEY_UNLIKELY(!get_number_value(j, val)))
-    {
-        throw_type_must_be("number", j);
+            throw_type_must_be("number", j);
     }
 }
 
@@ -256,44 +245,29 @@ inline void from_json_array_impl(const BasicJsonType& j, typename BasicJsonType:
     arr = *j.template get_ptr<const typename BasicJsonType::array_t*>();
 }
 
-// fill a std::array from the JSON array j: the size is checked once and the
-// elements are read in order, so a short array converts the elements it has and
-// then throws the out_of_range error at() would throw for the first missing one
-template<typename BasicJsonType, typename T, std::size_t N>
-void from_json_std_array(const BasicJsonType& j, std::array<T, N>& arr)
-{
-    const auto& inner_array = *j.template get_ptr<const typename BasicJsonType::array_t*>();
-    const std::size_t size = (std::min)(N, inner_array.size());
-    for (std::size_t i = 0; i < size; ++i)
-    {
-        arr[i] = inner_array[i].template get<T>();
-    }
-    if (JSON_HEDLEY_UNLIKELY(size < N))
-    {
-        static_cast<void>(j.at(size));
-    }
-}
-
 template<typename BasicJsonType, typename T, std::size_t N>
 auto from_json_array_impl(const BasicJsonType& j, std::array<T, N>& arr,
                           priority_tag<2> /*unused*/)
 -> decltype(j.template get<T>(), void())
 {
-    from_json_std_array(j, arr);
+    for (std::size_t i = 0; i < N; ++i)
+    {
+        arr[i] = j.at(i).template get<T>();
+    }
 }
 
-// reserve() is called through this pair (used for arrays and objects alike)
-// so from_json_array_impl and the object from_json have a single body for both
-// containers that support reserve() and those that don't.
-template<typename ConstructibleContainerType>
-auto reserve_if_possible(ConstructibleContainerType& c, typename ConstructibleContainerType::size_type size, priority_tag<1> /*unused*/)
--> decltype(c.reserve(size), void())
+// reserve() is called through this pair (modeled on from_json_object_reserve)
+// so from_json_array_impl below has a single body for both ConstructibleArrayType
+// that support reserve() and those that don't.
+template<typename ConstructibleArrayType>
+auto from_json_array_reserve(ConstructibleArrayType& arr, typename ConstructibleArrayType::size_type size, priority_tag<1> /*unused*/)
+-> decltype(arr.reserve(size), void())
 {
-    c.reserve(size);
+    arr.reserve(size);
 }
 
-template<typename ConstructibleContainerType>
-void reserve_if_possible(ConstructibleContainerType& /*c*/, std::size_t /*size*/, priority_tag<0> /*unused*/)
+template<typename ConstructibleArrayType>
+void from_json_array_reserve(ConstructibleArrayType& /*arr*/, std::size_t /*size*/, priority_tag<0> /*unused*/)
 {}
 
 // append through push_back() where available; containers without it (such as
@@ -323,7 +297,7 @@ auto from_json_array_impl(const BasicJsonType& j, ConstructibleArrayType& arr, p
 {
     ConstructibleArrayType ret;
     const auto* inner_array = j.template get_ptr<const typename BasicJsonType::array_t*>();
-    reserve_if_possible(ret, inner_array->size(), priority_tag<1> {});
+    from_json_array_reserve(ret, inner_array->size(), priority_tag<1> {});
     for (const auto& elem : *inner_array)
     {
         // get<BasicJsonType>() returns *this, this won't call a from_json
@@ -362,37 +336,6 @@ std::array<T, sizeof...(Idx)> from_json_inplace_array_impl(const BasicJsonType&
     return { { j.at(Idx).template get<T>()... } };
 }
 
-// std::is_trivial is deprecated in C++26 (see binary_writer::to_char_type);
-// a struct rather than an alias, as astyle indents an #ifdef block of aliases
-template<typename T>
-struct is_trivial_element
-#ifdef JSON_HAS_CPP_26
-    : conjunction<std::is_trivially_copyable<T>, std::is_trivially_default_constructible<T>>
-#else
-    : std::is_trivial<T>
-#endif
-{};
-
-// the element type may not be default constructible: construct every element
-// in place (a pack expansion that is unrolled N times)
-template < typename BasicJsonType, typename T, std::size_t N >
-std::array<T, N> from_json_inplace_array(const BasicJsonType& j, identity_tag<std::array<T, N>> tag,
-        std::false_type /*is_trivial_element*/)
-{
-    return from_json_inplace_array_impl(j, tag, make_index_sequence<N> {});
-}
-
-// a trivial element type cannot tell default construction plus assignment from
-// construction in place, so use a loop instead of unrolling N conversions
-template < typename BasicJsonType, typename T, std::size_t N >
-std::array<T, N> from_json_inplace_array(const BasicJsonType& j, identity_tag<std::array<T, N>> /*unused*/,
-        std::true_type /*is_trivial_element*/)
-{
-    std::array<T, N> arr; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): every element is assigned or an exception is thrown
-    from_json_std_array(j, arr);
-    return arr;
-}
-
 template < typename BasicJsonType, typename T, std::size_t N >
 auto from_json(const BasicJsonType& j, identity_tag<std::array<T, N>> tag)
 -> decltype(from_json_inplace_array_impl(j, tag, make_index_sequence<N> {}))
@@ -402,7 +345,7 @@ auto from_json(const BasicJsonType& j, identity_tag<std::array<T, N>> tag)
         throw_type_must_be("array", j);
     }
 
-    return from_json_inplace_array(j, tag, is_trivial_element<T> {});
+    return from_json_inplace_array_impl(j, tag, make_index_sequence<N> {});
 }
 
 template<typename BasicJsonType>
@@ -435,6 +378,17 @@ inline void from_json(const BasicJsonType& j, CompatibleArrayType& bin)
     }
 }
 
+template<typename ConstructibleObjectType>
+auto from_json_object_reserve(ConstructibleObjectType& obj, typename ConstructibleObjectType::size_type size, priority_tag<1> /*unused*/)
+-> decltype(obj.reserve(size), void())
+{
+    obj.reserve(size);
+}
+
+template<typename ConstructibleObjectType>
+inline void from_json_object_reserve(ConstructibleObjectType& /*obj*/, std::size_t /*size*/, priority_tag<0> /*unused*/)
+{}
+
 // the keys of the source are sorted for the usual object types, so hinting the
 // end makes every insertion constant-time (for containers that order their keys
 // the same way); containers without emplace_hint() (such as ordered_map) fall
@@ -463,7 +417,7 @@ inline void from_json(const BasicJsonType& j, ConstructibleObjectType& obj)
 
     ConstructibleObjectType ret;
     const auto* inner_object = j.template get_ptr<const typename BasicJsonType::object_t*>();
-    reserve_if_possible(ret, inner_object->size(), priority_tag<1> {});
+    from_json_object_reserve(ret, inner_object->size(), priority_tag<1> {});
     for (const auto& p : *inner_object)
     {
         from_json_object_emplace(ret, p.first, p.second.template get<typename ConstructibleObjectType::mapped_type>(), priority_tag<1> {});
@@ -483,17 +437,42 @@ template < typename BasicJsonType, typename ArithmeticType,
                int > = 0 >
 inline void from_json(const BasicJsonType& j, ArithmeticType& val)
 {
-    if (get_number_value(j, val))
+    switch (static_cast<value_t>(j))
     {
-        return;
-    }
+        case value_t::number_unsigned:
+        {
+            // never null: a basic_json whose number_unsigned_t is number_integer_t
+            // stores no number_unsigned values (#2573)
+            const auto* ptr = j.template get_ptr<const typename BasicJsonType::number_unsigned_t*>();
+            JSON_ASSERT(ptr != nullptr);
+            val = static_cast<ArithmeticType>(*ptr);
+            break;
+        }
+        case value_t::number_integer:
+        {
+            val = static_cast<ArithmeticType>(*j.template get_ptr<const typename BasicJsonType::number_integer_t*>());
+            break;
+        }
+        case value_t::number_float:
+        {
+            val = static_cast<ArithmeticType>(*j.template get_ptr<const typename BasicJsonType::number_float_t*>());
+            break;
+        }
+        case value_t::boolean:
+        {
+            val = static_cast<ArithmeticType>(*j.template get_ptr<const typename BasicJsonType::boolean_t*>());
+            break;
+        }
 
-    // unlike get_arithmetic_value(), a boolean is accepted here
-    if (JSON_HEDLEY_UNLIKELY(!j.is_boolean()))
-    {
-        throw_type_must_be("number", j);
+        case value_t::null:
+        case value_t::object:
+        case value_t::array:
+        case value_t::string:
+        case value_t::binary:
+        case value_t::discarded:
+        default:
+            throw_type_must_be("number", j);
     }
-    val = static_cast<ArithmeticType>(*j.template get_ptr<const typename BasicJsonType::boolean_t*>());
 }
 
 template<typename BasicJsonType, typename Type>

include/nlohmann/detail/exceptions.hpp

@@ -66,7 +66,6 @@ enum class exception_id : int
     invalid_string_or_size = 113, ///< a binary input contains an invalid string or size specification
     bson_unsupported_type = 114, ///< a BSON record type is not supported
     invalid_high_precision_number = 115, ///< a UBJSON/BJData high-precision number cannot be parsed
-    invalid_document_image = 116, ///< json_document::load() rejects an image
     // invalid_iterator
     iterators_incompatible = 201, ///< the iterators of a range belong to different values
     iterator_from_other_value = 202, ///< an iterator does not belong to the value it is used with
@@ -101,8 +100,6 @@ enum class exception_id : int
     invalid_utf8 = 316, ///< dump() finds a string that is not valid UTF-8
     type_not_serializable = 317, ///< a value cannot be serialized to the requested binary format
     enum_key_duplicate = 318, ///< an enum-keyed map has two keys that serialize to the same string
-    document_binary_value = 319, ///< a binary value is stored in a json_document
-    document_image_unsupported = 320, ///< a json_document image cannot be saved or loaded (discarded document, big-endian target)
     discarded_value_used = 321, ///< a discarded value is used
     // out_of_range
     array_index_out_of_range = 401, ///< an array index is out of range
@@ -120,7 +117,6 @@ enum class exception_id : int
     patch_remove_parent_not_container = 413, ///< the parent of a JSON Patch 'remove' target is not a container
     patch_move_into_child = 414, ///< a JSON Patch 'move' moves a value into one of its children
     subtype_out_of_range = 415, ///< a binary subtype does not fit into one byte
-    document_too_large = 416, ///< a json_document input, its edit storage, or an image reaches the 32-bit (4 GiB) limit
     // other_error
     internal_error = 500, ///< unreachable code was reached
     patch_test_failed = 501, ///< a JSON Patch 'test' operation failed
@@ -150,13 +146,6 @@ class exception : public std::exception
         return concat("[json.exception.", ename, '.', std::to_string(id_), "] ");
     }
 
-    /// build the message "[json.exception.<ename>.<id>] <diagnostics><what_arg>"
-    template<typename BasicJsonContext>
-    static std::string what_impl(const std::string& ename, int id_, const std::string& what_arg, BasicJsonContext context)
-    {
-        return concat(name(ename, id_), diagnostics(context), what_arg);
-    }
-
     static std::string diagnostics(std::nullptr_t /*leaf_element*/)
     {
         return "";
@@ -169,33 +158,48 @@ class exception : public std::exception
         std::vector<std::string> tokens;
         for (const auto* current = leaf_element; current != nullptr && current->m_parent != nullptr; current = current->m_parent)
         {
-            const auto* parent = current->m_parent;
-            if (parent->is_array())
+            switch (current->m_parent->type())
             {
-                for (std::size_t i = 0; i < parent->m_data.m_value.array->size(); ++i)
+                case value_t::array:
                 {
-                    if (&parent->m_data.m_value.array->operator[](i) == current)
+                    for (std::size_t i = 0; i < current->m_parent->m_data.m_value.array->size(); ++i)
                     {
-                        tokens.emplace_back(std::to_string(i));
-                        break;
+                        if (&current->m_parent->m_data.m_value.array->operator[](i) == current)
+                        {
+                            tokens.emplace_back(std::to_string(i));
+                            break;
+                        }
                     }
+                    break;
                 }
-            }
-            else if (parent->is_object())
-            {
-                for (const auto& element : *parent->m_data.m_value.object)
+
+                case value_t::object:
                 {
-                    if (&element.second == current)
+                    for (const auto& element : *current->m_parent->m_data.m_value.object)
                     {
-                        // data() is null-terminated, so a key containing
-                        // a null byte is cut short here rather than
-                        // truncating the whole message at what()
-                        tokens.emplace_back(element.first.data());
-                        break;
+                        if (&element.second == current)
+                        {
+                            // data() is null-terminated, so a key containing
+                            // a null byte is cut short here rather than
+                            // truncating the whole message at what()
+                            tokens.emplace_back(element.first.data());
+                            break;
+                        }
                     }
+                    break;
                 }
+
+                case value_t::null: // LCOV_EXCL_LINE
+                case value_t::string: // LCOV_EXCL_LINE
+                case value_t::boolean: // LCOV_EXCL_LINE
+                case value_t::number_integer: // LCOV_EXCL_LINE
+                case value_t::number_unsigned: // LCOV_EXCL_LINE
+                case value_t::number_float: // LCOV_EXCL_LINE
+                case value_t::binary: // LCOV_EXCL_LINE
+                case value_t::discarded: // LCOV_EXCL_LINE
+                default:   // LCOV_EXCL_LINE
+                    break; // LCOV_EXCL_LINE
             }
-            // all other parent types (primitives) contribute no token
         }
 
         if (tokens.empty())
@@ -311,7 +315,7 @@ class invalid_iterator : public exception
     template<typename BasicJsonContext, enable_if_t<is_basic_json_context<BasicJsonContext>::value, int> = 0>
     static invalid_iterator create(int id_, const std::string& what_arg, BasicJsonContext context)
     {
-        const std::string w = exception::what_impl("invalid_iterator", id_, what_arg, context);
+        const std::string w = concat(exception::name("invalid_iterator", id_), exception::diagnostics(context), what_arg);
         return {id_, w.c_str()};
     }
 
@@ -335,7 +339,7 @@ class type_error : public exception
     template<typename BasicJsonContext, enable_if_t<is_basic_json_context<BasicJsonContext>::value, int> = 0>
     static type_error create(int id_, const std::string& what_arg, BasicJsonContext context)
     {
-        const std::string w = exception::what_impl("type_error", id_, what_arg, context);
+        const std::string w = concat(exception::name("type_error", id_), exception::diagnostics(context), what_arg);
         return {id_, w.c_str()};
     }
 
@@ -358,7 +362,7 @@ class out_of_range : public exception
     template<typename BasicJsonContext, enable_if_t<is_basic_json_context<BasicJsonContext>::value, int> = 0>
     static out_of_range create(int id_, const std::string& what_arg, BasicJsonContext context)
     {
-        const std::string w = exception::what_impl("out_of_range", id_, what_arg, context);
+        const std::string w = concat(exception::name("out_of_range", id_), exception::diagnostics(context), what_arg);
         return {id_, w.c_str()};
     }
 
@@ -381,7 +385,7 @@ class other_error : public exception
     template<typename BasicJsonContext, enable_if_t<is_basic_json_context<BasicJsonContext>::value, int> = 0>
     static other_error create(int id_, const std::string& what_arg, BasicJsonContext context)
     {
-        const std::string w = exception::what_impl("other_error", id_, what_arg, context);
+        const std::string w = concat(exception::name("other_error", id_), exception::diagnostics(context), what_arg);
         return {id_, w.c_str()};
     }
 

include/nlohmann/detail/input/input_adapters.hpp

@@ -67,36 +67,6 @@ namespace detail
 /// the supported input formats
 enum class input_format_t { json, cbor, msgpack, ubjson, bson, bjdata, bon8 };
 
-/// @return the name of a binary input format, as used in exception messages
-inline const char* input_format_name(const input_format_t format) noexcept
-{
-    switch (format)
-    {
-        case input_format_t::cbor:
-            return "CBOR";
-        case input_format_t::msgpack:
-            return "MessagePack";
-        case input_format_t::ubjson:
-            return "UBJSON";
-        case input_format_t::bson:
-            return "BSON";
-        case input_format_t::bjdata:
-            return "BJData";
-        case input_format_t::bon8:
-            return "BON8";
-        case input_format_t::json: // LCOV_EXCL_LINE
-        default:            // LCOV_EXCL_LINE
-            JSON_ASSERT(false); // NOLINT(cert-dcl03-c,hicpp-static-assert,misc-static-assert) LCOV_EXCL_LINE
-            return ""; // LCOV_EXCL_LINE
-    }
-}
-
-/// @return the message of the parse_error thrown or reported for an empty input
-inline const char* empty_input_message() noexcept
-{
-    return "attempting to parse an empty input; check that your input string or stream contains the expected JSON";
-}
-
 ////////////////////
 // input adapters //
 ////////////////////
@@ -796,17 +766,22 @@ class iterator_input_adapter
         return get_consumed_count() + eof_reads;
     }
 
-    // number of newlines in the consumed range [0, last_index); @a line_start
-    // is set to the index after the last of them (0 if there is none); only
-    // valid when supports_seek
-    std::size_t count_consumed_lines(std::size_t last_index, std::size_t& line_start) const
+    // number of newlines in the consumed range [first_index, last_index); if
+    // there is one, @a line_start is set to the index after the last of them,
+    // otherwise it is left unchanged; only valid when supports_seek
+    std::size_t count_consumed_lines(std::size_t first_index, std::size_t last_index, std::size_t& line_start) const
     {
-        const auto to = std::next(begin, static_cast<typename std::iterator_traits<IteratorType>::difference_type>(last_index));
+        using difference_type = typename std::iterator_traits<IteratorType>::difference_type;
+        const auto from = std::next(begin, static_cast<difference_type>(first_index));
+        const auto to = std::next(from, static_cast<difference_type>(last_index - first_index));
         const auto newline = char_traits<char_type>::to_char_type('\n');
         using reverse_it = std::reverse_iterator<IteratorType>;
-        const auto last_newline = std::find(reverse_it(to), reverse_it(begin), newline);
-        line_start = static_cast<std::size_t>(std::distance(begin, last_newline.base()));
-        return static_cast<std::size_t>(std::count(begin, last_newline.base(), newline));
+        const auto last_newline = std::find(reverse_it(to), reverse_it(from), newline);
+        if (last_newline.base() != from)
+        {
+            line_start = static_cast<std::size_t>(std::distance(begin, last_newline.base()));
+        }
+        return static_cast<std::size_t>(std::count(from, last_newline.base(), newline));
     }
 
     // append the already-consumed characters in the half-open range
@@ -1298,7 +1273,7 @@ inline file_input_adapter input_adapter(std::FILE* file)
 {
     if (file == nullptr)
     {
-        JSON_THROW(parse_error::create(exception_id::syntax_error, 0, empty_input_message(), nullptr));
+        JSON_THROW(parse_error::create(exception_id::syntax_error, 0, "attempting to parse an empty input; check that your input string or stream contains the expected JSON", nullptr));
     }
     return file_input_adapter(file);
 }
@@ -1307,7 +1282,7 @@ inline input_stream_adapter input_adapter(std::istream& stream)
 {
     if (stream.rdbuf() == nullptr)
     {
-        JSON_THROW(parse_error::create(exception_id::syntax_error, 0, empty_input_message(), nullptr));
+        JSON_THROW(parse_error::create(exception_id::syntax_error, 0, "attempting to parse an empty input; check that your input string or stream contains the expected JSON", nullptr));
     }
     return input_stream_adapter(stream);
 }
@@ -1332,7 +1307,7 @@ contiguous_bytes_input_adapter input_adapter(CharT b)
 {
     if (b == nullptr)
     {
-        JSON_THROW(parse_error::create(exception_id::syntax_error, 0, empty_input_message(), nullptr));
+        JSON_THROW(parse_error::create(exception_id::syntax_error, 0, "attempting to parse an empty input; check that your input string or stream contains the expected JSON", nullptr));
     }
     auto length = std::strlen(reinterpret_cast<const char*>(b));
     const auto* ptr = reinterpret_cast<const char*>(b);

include/nlohmann/detail/input/json_sax.hpp

@@ -138,7 +138,9 @@ struct json_sax
     @param[in] position    the position in the input where the error occurs
     @param[in] last_token  the last read token
     @param[in] ex          an exception object describing the error
-    @return whether parsing should proceed (must return false)
+    @return whether to recover from the error: false stops parsing; true
+            repairs the error and continues, or, if that is not possible,
+            stops after completing the value read so far
     */
     virtual bool parse_error(std::size_t position,
                              const std::string& last_token,
@@ -321,52 +323,6 @@ struct diagnostic_positions
 };
 #endif
 
-/*!
-@brief error state shared by the DOM-building SAX parsers
-
-Holds whether a parse error occurred and whether errors are thrown, and
-implements the parse_error() and is_errored() members of the SAX interface for
-json_sax_dom_parser and json_sax_dom_callback_parser.
-*/
-class json_sax_dom_error_state
-{
-  public:
-    explicit json_sax_dom_error_state(const bool allow_exceptions_) noexcept
-        : allow_exceptions(allow_exceptions_)
-    {}
-
-    // make class move-only
-    json_sax_dom_error_state(const json_sax_dom_error_state&) = delete;
-    json_sax_dom_error_state(json_sax_dom_error_state&&) = default; // NOLINT(hicpp-noexcept-move,performance-noexcept-move-constructor)
-    json_sax_dom_error_state& operator=(const json_sax_dom_error_state&) = delete;
-    json_sax_dom_error_state& operator=(json_sax_dom_error_state&&) = delete; // the const member prevents assignment
-    ~json_sax_dom_error_state() = default;
-
-    template<class Exception>
-    bool parse_error(std::size_t /*unused*/, const std::string& /*unused*/,
-                     const Exception& ex)
-    {
-        errored = true;
-        static_cast<void>(ex);
-        if (allow_exceptions)
-        {
-            JSON_THROW(ex);
-        }
-        return false;
-    }
-
-    bool is_errored() const
-    {
-        return errored;
-    }
-
-  private:
-    /// whether a syntax error occurred
-    bool errored = false;
-    /// whether to throw exceptions in case of errors
-    const bool allow_exceptions = true;
-};
-
 /*!
 @brief a hash table that finds the position of a key in an object whose
        elements are kept in a vector (ordered_map)
@@ -469,13 +425,16 @@ constructor contains the parsed value. Without @a Collect, every value is
 stored in place as it arrives, so after a parse error the value holds what was
 read up to it.
 
-@tparam BasicJsonType  the JSON type
-@tparam Collect  whether to collect the values of containers of unknown size on
-                 a stack (only for a caller that does not use the value after a
-                 parse error)
+@tparam BasicJsonType     the JSON type
+@tparam InputAdapterType  the input adapter of the lexer that can be passed to
+                          the constructor to record diagnostic positions; it
+                          does not matter if no lexer is passed
+@tparam Collect           whether to collect the values of containers of
+                          unknown size on a stack (only for a caller that does
+                          not use the value after a parse error)
 */
-template<typename BasicJsonType, typename InputAdapterType, bool Collect = false>
-class json_sax_dom_parser : public json_sax_dom_error_state
+template<typename BasicJsonType, typename InputAdapterType = string_input_adapter_type, bool Collect = false>
+class json_sax_dom_parser
 {
   public:
     using number_integer_t = typename BasicJsonType::number_integer_t;
@@ -491,7 +450,7 @@ class json_sax_dom_parser : public json_sax_dom_error_state
     @param[in] allow_exceptions_  whether parse errors yield exceptions
     */
     explicit json_sax_dom_parser(BasicJsonType& r, const bool allow_exceptions_ = true, lexer_t* lexer_ = nullptr)
-        : json_sax_dom_error_state(allow_exceptions_), root(r), m_lexer_ref(lexer_)
+        : root(r), allow_exceptions(allow_exceptions_), m_lexer_ref(lexer_)
     {}
 
     // make class move-only
@@ -642,6 +601,24 @@ class json_sax_dom_parser : public json_sax_dom_error_state
         return true;
     }
 
+    template<class Exception>
+    bool parse_error(std::size_t /*unused*/, const std::string& /*unused*/,
+                     const Exception& ex)
+    {
+        errored = true;
+        static_cast<void>(ex);
+        if (allow_exceptions)
+        {
+            JSON_THROW(ex);
+        }
+        return false;
+    }
+
+    constexpr bool is_errored() const
+    {
+        return errored;
+    }
+
   private:
     using object_t = typename BasicJsonType::object_t;
     using array_t = typename BasicJsonType::array_t;
@@ -772,6 +749,8 @@ class json_sax_dom_parser : public json_sax_dom_error_state
         JSON_HEDLEY_NEVER_INLINE
         void grow()
         {
+            // the stack starts with local_capacity frames and only grows
+            JSON_ASSERT(capacity != 0);
             const std::size_t new_capacity = capacity * 2;
             std::unique_ptr<frame[]> new_heap(new frame[new_capacity]); // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
             std::copy(data, data + count, new_heap.get());
@@ -937,6 +916,8 @@ class json_sax_dom_parser : public json_sax_dom_error_state
     /// directly if the value was the one too many.
     void place_container(const frame& f, frame* enclosing, BasicJsonType&& value, const bool enclosing_is_innermost)
     {
+        // a container without a slot of its own is collected for an enclosing one
+        JSON_ASSERT(f.slot != nullptr || enclosing != nullptr);
         if (f.slot != nullptr)
         {
             *f.slot = std::move(value);
@@ -1353,6 +1334,10 @@ class json_sax_dom_parser : public json_sax_dom_error_state
     /// the array that stores the elements of the innermost array as they
     /// arrive (frames.back().array), or nullptr
     array_t* stored_array = nullptr;
+    /// whether a syntax error occurred
+    bool errored = false;
+    /// whether to throw exceptions in case of errors
+    const bool allow_exceptions = true;
     /// the lexer reference to obtain the current position
     lexer_t* m_lexer_ref = nullptr;
 };
@@ -1377,7 +1362,7 @@ constructor contains the parsed value.
 @tparam BasicJsonType  the JSON type
 */
 template<typename BasicJsonType, typename InputAdapterType>
-class json_sax_dom_binary_parser : public json_sax_dom_error_state
+class json_sax_dom_binary_parser
 {
   public:
     using number_integer_t = typename BasicJsonType::number_integer_t;
@@ -1393,7 +1378,7 @@ class json_sax_dom_binary_parser : public json_sax_dom_error_state
     @param[in] allow_exceptions_  whether parse errors yield exceptions
     */
     explicit json_sax_dom_binary_parser(BasicJsonType& r, const bool allow_exceptions_ = true, lexer_t* lexer_ = nullptr)
-        : json_sax_dom_error_state(allow_exceptions_), root(r), m_lexer_ref(lexer_)
+        : root(r), allow_exceptions(allow_exceptions_), m_lexer_ref(lexer_)
     {}
 
     // make class move-only
@@ -1522,6 +1507,24 @@ class json_sax_dom_binary_parser : public json_sax_dom_error_state
         return true;
     }
 
+    template<class Exception>
+    bool parse_error(std::size_t /*unused*/, const std::string& /*unused*/,
+                     const Exception& ex)
+    {
+        errored = true;
+        static_cast<void>(ex);
+        if (allow_exceptions)
+        {
+            JSON_THROW(ex);
+        }
+        return false;
+    }
+
+    constexpr bool is_errored() const
+    {
+        return errored;
+    }
+
   private:
 
     /*!
@@ -1575,12 +1578,16 @@ class json_sax_dom_binary_parser : public json_sax_dom_error_state
     std::vector<BasicJsonType*> ref_stack {};
     /// helper to hold the reference for the next object element
     BasicJsonType* object_element = nullptr;
+    /// whether a syntax error occurred
+    bool errored = false;
+    /// whether to throw exceptions in case of errors
+    const bool allow_exceptions = true;
     /// the lexer reference to obtain the current position
     lexer_t* m_lexer_ref = nullptr;
 };
 
-template<typename BasicJsonType, typename InputAdapterType>
-class json_sax_dom_callback_parser : public json_sax_dom_error_state
+template<typename BasicJsonType, typename InputAdapterType = string_input_adapter_type>
+class json_sax_dom_callback_parser
 {
   public:
     using number_integer_t = typename BasicJsonType::number_integer_t;
@@ -1596,7 +1603,7 @@ class json_sax_dom_callback_parser : public json_sax_dom_error_state
                                  parser_callback_t cb,
                                  const bool allow_exceptions_ = true,
                                  lexer_t* lexer_ = nullptr)
-        : json_sax_dom_error_state(allow_exceptions_), root(r), callback(std::move(cb)), m_lexer_ref(lexer_)
+        : root(r), callback(std::move(cb)), allow_exceptions(allow_exceptions_), m_lexer_ref(lexer_)
     {
         keep_stack.push_back(true);
     }
@@ -1860,6 +1867,24 @@ class json_sax_dom_callback_parser : public json_sax_dom_error_state
         return true;
     }
 
+    template<class Exception>
+    bool parse_error(std::size_t /*unused*/, const std::string& /*unused*/,
+                     const Exception& ex)
+    {
+        errored = true;
+        static_cast<void>(ex);
+        if (allow_exceptions)
+        {
+            JSON_THROW(ex);
+        }
+        return false;
+    }
+
+    constexpr bool is_errored() const
+    {
+        return errored;
+    }
+
   private:
 
     /*!
@@ -2126,8 +2151,12 @@ class json_sax_dom_callback_parser : public json_sax_dom_error_state
     /// value is later rejected by the callback, instead of erasing the
     /// member entirely
     std::vector<std::pair<BasicJsonType*, BasicJsonType>> duplicate_key_stash {};
+    /// whether a syntax error occurred
+    bool errored = false;
     /// callback function
     const parser_callback_t callback = nullptr;
+    /// whether to throw exceptions in case of errors
+    const bool allow_exceptions = true;
     /// a discarded value for the callback
     BasicJsonType discarded = BasicJsonType::value_t::discarded;
     /// the value of the key event for the callback (see key_value())

include/nlohmann/detail/input/parser.hpp

@@ -153,25 +153,72 @@ class parser
     bool accept(const bool strict = true)
     {
         json_sax_acceptor<BasicJsonType> sax_acceptor;
-        return sax_parse(&sax_acceptor, strict);
+        return sax_parse_impl<false>(&sax_acceptor, strict);
     }
 
+    /*!
+    @brief public SAX interface
+
+    If the SAX parser's parse_error() returns true, the parser recovers from
+    the error: it repairs the input and continues (see #3989).
+
+    @param[in] sax     the SAX parser
+    @param[in] strict  whether to expect the last token to be EOF
+    @return whether the input was parsed without errors and no SAX event
+            returned false
+    */
     template<typename SAX>
     JSON_HEDLEY_NON_NULL(2)
     bool sax_parse(SAX* sax, const bool strict = true)
     {
-        (void)detail::is_sax_static_asserts<SAX, BasicJsonType> {};
-        const bool result = sax_parse_internal(sax);
-
-        if (result)
-        {
-            return finish_input(*sax, strict);
-        }
-
-        return false;
+        return sax_parse_impl<true>(sax, strict);
     }
 
   private:
+    /// what sax_parse_internal() does after an object key was expected
+    enum class next_step : std::uint8_t
+    {
+        /// stop parsing
+        stop,
+        /// parse a value that begins with last_token
+        parse_value,
+        /// evaluate the state of the innermost container, which reads
+        /// last_token again
+        evaluate_state
+    };
+
+    template<bool AllowRecovery, typename SAX>
+    JSON_HEDLEY_NON_NULL(2)
+    bool sax_parse_impl(SAX* sax, const bool strict)
+    {
+        (void)detail::is_sax_static_asserts<SAX, BasicJsonType> {};
+        const bool result = sax_parse_internal<AllowRecovery>(sax);
+
+        if (result)
+        {
+            if (strict)
+            {
+                // strict mode: next byte must be EOF; after recovering from an
+                // error, the end of the input may already have been read
+                if (last_token != token_type::end_of_input && get_token() != token_type::end_of_input)
+                {
+                    // the value is complete, so there is nothing to recover
+                    static_cast<void>(report_error(sax, parse_error::create(exception_id::syntax_error, m_lexer.get_position(), exception_message(token_type::end_of_input, "value"), nullptr),
+                                                   std::integral_constant<bool, AllowRecovery> {}));
+                    return false;
+                }
+            }
+            else
+            {
+                // the caller keeps using the input: position it right after
+                // the value by leaving the character that terminated it
+                m_lexer.release_lookahead();
+            }
+        }
+
+        return result && !error_reported;
+    }
+
     /// the text of the current float token for @a sax (see sax_ignores_number_text)
     template<typename SAX>
     const string_t& number_float_text(SAX* /*sax*/)
@@ -189,39 +236,6 @@ class parser
         return m_lexer.get_string();
     }
 
-    /*!
-    @brief apply the end-of-input handling after a value was parsed
-
-    In strict mode, the next token must be the end of the input; otherwise
-    parse_error.101 is reported through @a sax. In non-strict mode, the
-    lookahead is released so the caller can keep reading the input right after
-    the parsed value.
-
-    @param[in,out] sax  the SAX parser to report an error to
-    @param[in] strict   whether to expect the last token to be EOF
-    @return true, or what @a sax returned for the error
-    */
-    template<typename SAX>
-    bool finish_input(SAX& sax, const bool strict)
-    {
-        if (strict)
-        {
-            // strict mode: next byte must be EOF
-            if (get_token() != token_type::end_of_input)
-            {
-                return syntax_error(sax, exception_message(token_type::end_of_input, "value"));
-            }
-        }
-        else
-        {
-            // the caller keeps using the input: position it right after
-            // the value by leaving the character that terminated it
-            m_lexer.release_lookahead();
-        }
-
-        return true;
-    }
-
     /*!
     @brief run a DOM SAX parser to completion and position the lexer
 
@@ -239,17 +253,39 @@ class parser
     template<typename DomSax>
     bool parse_dom(DomSax& sdp, const bool strict)
     {
-        sax_parse_internal(&sdp);
+        sax_parse_internal<false>(&sdp);
 
-        static_cast<void>(finish_input(sdp, strict));
+        if (strict)
+        {
+            // in strict mode, input must be completely read
+            if (get_token() != token_type::end_of_input)
+            {
+                syntax_error(sdp, exception_message(token_type::end_of_input, "value"));
+            }
+        }
+        else
+        {
+            // the caller keeps using the input: position it right after
+            // the value by leaving the character that terminated it
+            m_lexer.release_lookahead();
+        }
 
         return !sdp.is_errored();
     }
 
-    template<typename SAX>
+    /*!
+    @brief parse a JSON value and pass it to a SAX parser
+
+    @tparam AllowRecovery  whether to recover from an error if the SAX parser's
+                           parse_error() returns true; false for the SAX parsers
+                           of parse() and accept(), which never do, so that no
+                           code for recovering is generated for them
+    */
+    template<bool AllowRecovery, typename SAX>
     JSON_HEDLEY_NON_NULL(2)
     bool sax_parse_internal(SAX* sax)
     {
+        const std::integral_constant<bool, AllowRecovery> allow_recovery{};
         // value to avoid a goto (see comment where set to true)
         bool skip_to_state_evaluation = false;
 
@@ -288,10 +324,18 @@ class parser
                             break;
                         }
 
-                        // parse key
+                        // remember we are now inside an object
+                        states.push_back(0);
+
+                        // parse key (the steps of parse_key(), which are
+                        // repeated here and below for speed)
                         if (JSON_HEDLEY_UNLIKELY(last_token != token_type::value_string))
                         {
-                            return syntax_error(*sax, exception_message(token_type::value_string, "object key"));
+                            if (!continue_after(key_error(sax, allow_recovery, false), skip_to_state_evaluation))
+                            {
+                                return false;
+                            }
+                            continue;
                         }
                         if (JSON_HEDLEY_UNLIKELY(!sax->key(m_lexer.get_string_token())))
                         {
@@ -301,12 +345,13 @@ class parser
                         // parse separator (:)
                         if (JSON_HEDLEY_UNLIKELY(!get_token_expecting(token_type::name_separator)))
                         {
-                            return syntax_error(*sax, exception_message(token_type::name_separator, "object separator"));
+                            if (!continue_after(key_error(sax, allow_recovery, true), skip_to_state_evaluation))
+                            {
+                                return false;
+                            }
+                            continue;
                         }
 
-                        // remember we are now inside an object
-                        states.push_back(0);
-
                         // parse values
                         get_token();
                         continue;
@@ -342,9 +387,11 @@ class parser
 
                         if (JSON_HEDLEY_UNLIKELY(!std::isfinite(res)))
                         {
-                            return sax->parse_error(m_lexer.get_position(),
-                                                    m_lexer.get_token_string(),
-                                                    out_of_range::create(exception_id::number_overflow, concat("number overflow parsing '", m_lexer.get_token_string(), '\''), nullptr));
+                            if (!overflow_error(sax, res, allow_recovery))
+                            {
+                                return false;
+                            }
+                            break;
                         }
 
                         if (JSON_HEDLEY_UNLIKELY(!sax->number_float(res, number_float_text(sax))))
@@ -412,16 +459,63 @@ class parser
                     case token_type::parse_error:
                     {
                         // using "uninitialized" to avoid an "expected" message
-                        return syntax_error(*sax, exception_message(token_type::uninitialized, "value"));
+                        if (!report_error(sax, parse_error::create(exception_id::syntax_error, m_lexer.get_position(), exception_message(token_type::uninitialized, "value"), nullptr), allow_recovery))
+                        {
+                            return false;
+                        }
+
+                        // recover: keep what can be read of the token
+                        recover_token(allow_recovery);
+                        if (last_token != token_type::uninitialized)
+                        {
+                            // a string or a number
+                            continue;
+                        }
+                        if (states.empty())
+                        {
+                            // look for the value after the garbage
+                            if (!skip_to_value(allow_recovery))
+                            {
+                                return false;
+                            }
+                            continue;
+                        }
+                        // nothing could be read
+                        if (JSON_HEDLEY_UNLIKELY(!sax->null()))
+                        {
+                            return false;
+                        }
+                        break;
                     }
                     case token_type::end_of_input:
                     {
                         if (JSON_HEDLEY_UNLIKELY(m_lexer.get_chars_read_total() == 1))
                         {
-                            return syntax_error(*sax, empty_input_message());
+                            // there is nothing to recover
+                            static_cast<void>(report_error(sax, parse_error::create(exception_id::syntax_error, m_lexer.get_position(),
+                                                           "attempting to parse an empty input; check that your input string or stream contains the expected JSON", nullptr), allow_recovery));
+                            return false;
                         }
 
-                        return syntax_error(*sax, exception_message(token_type::literal_or_value, "value"));
+                        if (!report_error(sax, parse_error::create(exception_id::syntax_error, m_lexer.get_position(), exception_message(token_type::literal_or_value, "value"), nullptr), allow_recovery))
+                        {
+                            return false;
+                        }
+
+                        // recover: the input ends where a value is missing
+                        if (states.empty())
+                        {
+                            // there is no value
+                            return false;
+                        }
+                        if (!recover_missing_value(sax, states, allow_recovery))
+                        {
+                            return false;
+                        }
+                        // the state evaluation reads the token again
+                        unget_token(allow_recovery);
+                        skip_to_state_evaluation = true;
+                        continue;
                     }
                     case token_type::uninitialized:
                     case token_type::end_array:
@@ -431,7 +525,35 @@ class parser
                     case token_type::literal_or_value:
                     default: // the last token was unexpected
                     {
-                        return syntax_error(*sax, exception_message(token_type::literal_or_value, "value"));
+                        if (!report_error(sax, parse_error::create(exception_id::syntax_error, m_lexer.get_position(), exception_message(token_type::literal_or_value, "value"), nullptr), allow_recovery))
+                        {
+                            return false;
+                        }
+
+                        // recover
+                        if (states.empty())
+                        {
+                            // look for the value after the garbage
+                            if (!skip_to_value(allow_recovery))
+                            {
+                                return false;
+                            }
+                            continue;
+                        }
+                        if (last_token == token_type::name_separator)
+                        {
+                            // a stray ':'; the value may follow
+                            get_token();
+                            continue;
+                        }
+                        if (!recover_missing_value(sax, states, allow_recovery))
+                        {
+                            return false;
+                        }
+                        // the state evaluation reads the token again
+                        unget_token(allow_recovery);
+                        skip_to_state_evaluation = true;
+                        continue;
                     }
                 }
             }
@@ -482,7 +604,30 @@ class parser
                     continue;
                 }
 
-                return syntax_error(*sax, exception_message(token_type::end_array, "array"));
+                if (!report_error(sax, parse_error::create(exception_id::syntax_error, m_lexer.get_position(), exception_message(token_type::end_array, "array"), nullptr), allow_recovery))
+                {
+                    return false;
+                }
+
+                // recover
+                if (last_token == token_type::end_of_input)
+                {
+                    // the input ends inside the array
+                    return close_containers(sax, states, allow_recovery);
+                }
+                if (last_token == token_type::end_object)
+                {
+                    // a wrong closing bracket closes the innermost container
+                    if (JSON_HEDLEY_UNLIKELY(!sax->end_array()))
+                    {
+                        return false;
+                    }
+                    states.pop_back();
+                    skip_to_state_evaluation = true;
+                }
+                // otherwise, a missing ',' (or a stray ':', which value
+                // parsing drops): the next value begins here
+                continue;
             }
 
             // states.back() is false -> object
@@ -499,9 +644,12 @@ class parser
                     // parse key
                     if (JSON_HEDLEY_UNLIKELY(last_token != token_type::value_string))
                     {
-                        return syntax_error(*sax, exception_message(token_type::value_string, "object key"));
+                        if (!continue_after(key_error(sax, allow_recovery, false), skip_to_state_evaluation))
+                        {
+                            return false;
+                        }
+                        continue;
                     }
-
                     if (JSON_HEDLEY_UNLIKELY(!sax->key(m_lexer.get_string_token())))
                     {
                         return false;
@@ -510,7 +658,11 @@ class parser
                     // parse separator (:)
                     if (JSON_HEDLEY_UNLIKELY(!get_token_expecting(token_type::name_separator)))
                     {
-                        return syntax_error(*sax, exception_message(token_type::name_separator, "object separator"));
+                        if (!continue_after(key_error(sax, allow_recovery, true), skip_to_state_evaluation))
+                        {
+                            return false;
+                        }
+                        continue;
                     }
 
                     // parse values
@@ -538,10 +690,546 @@ class parser
                 continue;
             }
 
-            return syntax_error(*sax, exception_message(token_type::end_object, "object"));
+            if (!report_error(sax, parse_error::create(exception_id::syntax_error, m_lexer.get_position(), exception_message(token_type::end_object, "object"), nullptr), allow_recovery))
+            {
+                return false;
+            }
+
+            // recover
+            if (last_token == token_type::end_of_input)
+            {
+                // the input ends inside the object
+                return close_containers(sax, states, allow_recovery);
+            }
+            if (last_token == token_type::end_array)
+            {
+                // a wrong closing bracket closes the innermost container
+                if (JSON_HEDLEY_UNLIKELY(!sax->end_object()))
+                {
+                    return false;
+                }
+                states.pop_back();
+                skip_to_state_evaluation = true;
+                continue;
+            }
+            if (!continue_after(recover_member(sax, allow_recovery), skip_to_state_evaluation))
+            {
+                return false;
+            }
         }
     }
 
+    /*!
+    @brief continue sax_parse_internal() after a recovery
+    @return whether to continue parsing
+    */
+    bool continue_after(const next_step step, bool& skip_to_state_evaluation)
+    {
+        if (step == next_step::evaluate_state)
+        {
+            // the state evaluation reads the token again
+            m_lexer.unget_token();
+            skip_to_state_evaluation = true;
+        }
+        return step != next_step::stop;
+    }
+
+    /// the parser for parse() and accept() never recovers: stop parsing
+    JSON_INTERNAL_ALWAYS_INLINE
+    static std::false_type continue_after(std::false_type /*step*/, bool& /*skip_to_state_evaluation*/) noexcept
+    {
+        return {};
+    }
+
+    /*!
+    @brief parse an object key and the name separator (:) after it
+
+    last_token is the token where the key is expected. sax_parse_internal()
+    repeats these steps rather than calling this function, which is used
+    when recovering from an error.
+
+    @return next_step::parse_value if the value follows, with last_token its
+            first token; next_step::evaluate_state if the object's state is
+            to be evaluated after recovering from an error; next_step::stop
+            to stop parsing
+    */
+    template<typename SAX>
+    next_step parse_key(SAX* sax)
+    {
+        const std::true_type allow_recovery{};
+
+        if (JSON_HEDLEY_UNLIKELY(last_token != token_type::value_string))
+        {
+            return key_error(sax, allow_recovery, false);
+        }
+
+        if (JSON_HEDLEY_UNLIKELY(!sax->key(m_lexer.get_string_token())))
+        {
+            return next_step::stop;
+        }
+
+        // parse separator (:)
+        if (JSON_HEDLEY_UNLIKELY(get_token() != token_type::name_separator))
+        {
+            return key_error(sax, allow_recovery, true);
+        }
+
+        // the value begins with the next token
+        get_token();
+        return next_step::parse_value;
+    }
+
+    /*!
+    @brief report a number that is too large for number_float_t, and recover
+           from the error by passing the value on; the SAX parser gets the
+           number's text as well
+
+    This is a separate function, as reading other numbers is measurably
+    slower if the error is handled where they are read.
+
+    @param[in] sax    the SAX parser
+    @param[in] value  the value that is not finite
+    @return whether to continue parsing
+    */
+    template<typename SAX>
+    bool overflow_error(SAX* sax, const number_float_t value, std::true_type allow_recovery)
+    {
+        if (!report_error(sax, out_of_range::create(exception_id::number_overflow, concat("number overflow parsing '", m_lexer.get_token_string(), '\''), nullptr), allow_recovery))
+        {
+            return false;
+        }
+        return sax->number_float(value, number_float_text(sax));
+    }
+
+    /// @copydoc overflow_error
+    template<typename SAX>
+    JSON_INTERNAL_ALWAYS_INLINE
+    std::false_type overflow_error(SAX* sax, const number_float_t /*value*/, std::false_type allow_recovery)
+    {
+        return report_error(sax, out_of_range::create(exception_id::number_overflow, concat("number overflow parsing '", m_lexer.get_token_string(), '\''), nullptr), allow_recovery);
+    }
+
+    /*!
+    @brief report a missing key, or a missing name separator (:) after the
+           key; the parser for parse() and accept() never recovers
+
+    @param[in] key_read  whether the key was read, so that the name separator
+                         is missing
+    @return std::false_type, see report_error()
+    */
+    template<typename SAX>
+    JSON_INTERNAL_ALWAYS_INLINE
+    std::false_type key_error(SAX* sax, std::false_type allow_recovery, const bool key_read)
+    {
+        return report_error(sax, parse_error::create(exception_id::syntax_error, m_lexer.get_position(), key_read
+                            ? exception_message(token_type::name_separator, "object separator")
+                            : exception_message(token_type::value_string, "object key"), nullptr), allow_recovery);
+    }
+
+    /*!
+    @brief report a missing key, or a missing name separator (:) after the
+           key, and recover from it
+
+    @param[in] key_read  whether the key was read, so that the name separator
+                         is missing
+    */
+    template<typename SAX>
+    next_step key_error(SAX* sax, std::true_type allow_recovery, const bool key_read)
+    {
+        if (!key_read)
+        {
+            if (!report_error(sax, parse_error::create(exception_id::syntax_error, m_lexer.get_position(), exception_message(token_type::value_string, "object key"), nullptr), allow_recovery))
+            {
+                return next_step::stop;
+            }
+            return recover_key(sax);
+        }
+
+        if (!report_error(sax, parse_error::create(exception_id::syntax_error, m_lexer.get_position(), exception_message(token_type::name_separator, "object separator"), nullptr), allow_recovery))
+        {
+            return next_step::stop;
+        }
+        return recover_name_separator(sax);
+    }
+
+    /////////////////////
+    // error recovery
+    /////////////////////
+
+    /*
+    The functions below repair an error after the SAX parser's parse_error()
+    returned true (see #3989). Each mistake is repaired by the smallest local
+    edit: a missing ',' or ':' is inserted, a stray token is removed, what can
+    be read of an invalid string or number is kept (see
+    lexer::recover_token()), a missing value becomes null, a wrong closing
+    bracket closes the innermost container, and the end of the input closes
+    all of them. The events stay balanced, and every key() is followed by
+    exactly one value.
+
+    A repair hands a token to the state evaluation, by returning it to the
+    lexer (lexer::unget_token()) so that the state evaluation reads it again,
+    only if it is ',', ']', '}', or the end of the input. The state evaluation
+    hands a token to value or key parsing only if it is none of them, so a
+    token is never handed back and forth. Every other step reads a token or
+    closes a container, so parsing always ends.
+    */
+
+    /*!
+    @brief report an error to the SAX parser; the parser for parse() and
+           accept() never recovers
+
+    @return std::false_type rather than false: its value is known where the
+            function is called even if the call is not inlined, so the code
+            for recovering is not generated
+    */
+    template<typename SAX, typename Exception>
+    JSON_INTERNAL_ALWAYS_INLINE
+    std::false_type report_error(SAX* sax, const Exception& ex, std::false_type /*allow_recovery*/)
+    {
+        static_cast<void>(sax); // MSVC 2015 does not count calling a static parse_error() as using it
+        error_reported = true;
+        static_cast<void>(sax->parse_error(m_lexer.get_position(), m_lexer.get_token_string(), ex));
+        return {};
+    }
+
+    /*!
+    @brief report an error to the SAX parser
+    @return whether to recover from the error
+    */
+    template<typename SAX, typename Exception>
+    bool report_error(SAX* sax, const Exception& ex, std::true_type /*allow_recovery*/)
+    {
+        static_cast<void>(sax); // MSVC 2015 does not count calling a static parse_error() as using it
+        const std::size_t position = m_lexer.get_position().chars_read_total;
+        if (error_reported && position == last_error_position && last_token == last_error_token)
+        {
+            // a repair handed on the token of the error it repaired; the
+            // token was reported already, and the SAX parser asked to recover
+            return true;
+        }
+
+        error_reported = true;
+        last_error_position = position;
+        last_error_token = last_token;
+
+        if (!sax->parse_error(m_lexer.get_position(), m_lexer.get_token_string(), ex))
+        {
+            return false;
+        }
+
+        // the token string of the next error begins here
+        m_lexer.restart_token_string();
+        return true;
+    }
+
+    /*!
+    @brief keep what can be read of the token that the lexer rejected
+
+    The error was reported for the rejected token, so it is not reported again
+    for the token it is repaired to (see lexer::recover_token()).
+    */
+    token_type recover_token()
+    {
+        last_token = m_lexer.recover_token();
+        last_error_position = m_lexer.get_position().chars_read_total;
+        last_error_token = last_token;
+        return last_token;
+    }
+
+    // The functions below are called from sax_parse_internal() after an error
+    // was reported. The parser for parse() and accept() stopped there, so for
+    // it they are stubs: the code for recovering is then not even referenced,
+    // and unoptimized builds do not emit it either.
+
+    /// @copydoc recover_token()
+    JSON_INTERNAL_ALWAYS_INLINE
+    token_type recover_token(std::true_type /*allow_recovery*/)
+    {
+        return recover_token();
+    }
+
+    /// @copydoc recover_token()
+    JSON_INTERNAL_ALWAYS_INLINE
+    static std::false_type recover_token(std::false_type /*allow_recovery*/) noexcept
+    {
+        return {};
+    }
+
+    /// return the token that was read last to the lexer (see lexer::unget_token())
+    JSON_INTERNAL_ALWAYS_INLINE
+    void unget_token(std::true_type /*allow_recovery*/)
+    {
+        m_lexer.unget_token();
+    }
+
+    /// @copydoc unget_token(std::true_type)
+    JSON_INTERNAL_ALWAYS_INLINE
+    static void unget_token(std::false_type /*allow_recovery*/) noexcept {}
+
+    /// @copydoc skip_to_value
+    JSON_INTERNAL_ALWAYS_INLINE
+    static std::false_type skip_to_value(std::false_type /*allow_recovery*/) noexcept
+    {
+        return {};
+    }
+
+    /// @copydoc recover_missing_value
+    template<typename SAX>
+    JSON_INTERNAL_ALWAYS_INLINE
+    static std::false_type recover_missing_value(SAX* /*sax*/, const std::vector<std::uint8_t>& /*states*/, std::false_type /*allow_recovery*/) noexcept
+    {
+        return {};
+    }
+
+    /// pass the end events of all open containers
+    template<typename SAX>
+    bool close_containers(SAX* sax, std::vector<std::uint8_t>& states, std::true_type /*allow_recovery*/)
+    {
+        while (!states.empty())
+        {
+            const bool is_array = states.back() != 0;
+            states.pop_back();
+            if (JSON_HEDLEY_UNLIKELY(is_array ? !sax->end_array() : !sax->end_object()))
+            {
+                return false;
+            }
+        }
+        return true;
+    }
+
+    /// @copydoc close_containers
+    template<typename SAX>
+    JSON_INTERNAL_ALWAYS_INLINE
+    static std::false_type close_containers(SAX* /*sax*/, std::vector<std::uint8_t>& /*states*/, std::false_type /*allow_recovery*/) noexcept
+    {
+        return {};
+    }
+
+    /*!
+    @brief read tokens until one begins a value, skipping everything before
+           the top-level value
+    @return whether a value begins with last_token
+    */
+    bool skip_to_value(std::true_type /*allow_recovery*/)
+    {
+        while (true)
+        {
+            switch (get_token())
+            {
+                case token_type::begin_array:
+                case token_type::begin_object:
+                case token_type::literal_false:
+                case token_type::literal_null:
+                case token_type::literal_true:
+                case token_type::value_float:
+                case token_type::value_integer:
+                case token_type::value_string:
+                case token_type::value_unsigned:
+                    return true;
+
+                case token_type::end_of_input:
+                    return false;
+
+                case token_type::parse_error:
+                    recover_token();
+                    if (last_token != token_type::uninitialized)
+                    {
+                        return true;
+                    }
+                    break;
+
+                case token_type::uninitialized:
+                case token_type::end_array:
+                case token_type::end_object:
+                case token_type::name_separator:
+                case token_type::value_separator:
+                case token_type::literal_or_value:
+                default:
+                    break;
+            }
+        }
+    }
+
+    /*!
+    @brief skip the rest of an object member that cannot be read
+
+    Reads tokens, beginning with last_token, until a ',', '}', or ']' that is
+    not inside a container that begins in the skipped tokens, or the end of
+    the input.
+    */
+    void skip_member()
+    {
+        std::size_t depth = 0;
+        while (true)
+        {
+            switch (last_token)
+            {
+                case token_type::begin_array:
+                case token_type::begin_object:
+                    ++depth;
+                    break;
+
+                case token_type::end_array:
+                case token_type::end_object:
+                    if (depth == 0)
+                    {
+                        return;
+                    }
+                    --depth;
+                    break;
+
+                case token_type::value_separator:
+                    if (depth == 0)
+                    {
+                        return;
+                    }
+                    break;
+
+                case token_type::end_of_input:
+                    return;
+
+                case token_type::parse_error:
+                    recover_token();
+                    break;
+
+                case token_type::uninitialized:
+                case token_type::literal_true:
+                case token_type::literal_false:
+                case token_type::literal_null:
+                case token_type::value_string:
+                case token_type::value_unsigned:
+                case token_type::value_integer:
+                case token_type::value_float:
+                case token_type::name_separator:
+                case token_type::literal_or_value:
+                default:
+                    break;
+            }
+            get_token();
+        }
+    }
+
+    /*!
+    @brief pass a value where it is missing
+
+    last_token is ',', ']', '}', or the end of the input, where a value was
+    expected. In an object, the key gets null; in an array, a ',' where a
+    value is missing stands for null (as in JavaScript), while an array that
+    ends there just ends.
+    */
+    template<typename SAX>
+    bool recover_missing_value(SAX* sax, const std::vector<std::uint8_t>& states, std::true_type /*allow_recovery*/)
+    {
+        JSON_ASSERT(!states.empty());
+        if (states.back() == 0 || last_token == token_type::value_separator)
+        {
+            return sax->null();
+        }
+        return true;
+    }
+
+    /// recover from a missing key; last_token is where it was expected
+    template<typename SAX>
+    next_step recover_key(SAX* sax)
+    {
+        switch (last_token)
+        {
+            case token_type::value_separator:
+            case token_type::end_object:
+            case token_type::end_array:
+            case token_type::end_of_input:
+                // no member: the object's state handles the token
+                return next_step::evaluate_state;
+
+            case token_type::parse_error:
+                recover_token();
+                if (last_token == token_type::value_string)
+                {
+                    // a key that could be repaired
+                    return parse_key(sax);
+                }
+                skip_member();
+                return next_step::evaluate_state;
+
+            case token_type::uninitialized:
+            case token_type::literal_true:
+            case token_type::literal_false:
+            case token_type::literal_null:
+            case token_type::value_string:
+            case token_type::value_unsigned:
+            case token_type::value_integer:
+            case token_type::value_float:
+            case token_type::begin_array:
+            case token_type::begin_object:
+            case token_type::name_separator:
+            case token_type::literal_or_value:
+            default:
+                // a member without a key
+                skip_member();
+                return next_step::evaluate_state;
+        }
+    }
+
+    /// recover from a missing name separator (:) after the key; last_token
+    /// is where it was expected
+    template<typename SAX>
+    next_step recover_name_separator(SAX* sax)
+    {
+        switch (last_token)
+        {
+            case token_type::value_separator:
+            case token_type::end_object:
+            case token_type::end_array:
+            case token_type::end_of_input:
+                // the value is missing as well
+                return sax->null() ? next_step::evaluate_state : next_step::stop;
+
+            case token_type::uninitialized:
+            case token_type::literal_true:
+            case token_type::literal_false:
+            case token_type::literal_null:
+            case token_type::value_string:
+            case token_type::value_unsigned:
+            case token_type::value_integer:
+            case token_type::value_float:
+            case token_type::begin_array:
+            case token_type::begin_object:
+            case token_type::name_separator:
+            case token_type::parse_error:
+            case token_type::literal_or_value:
+            default:
+                // a missing ':'; the value begins here
+                return next_step::parse_value;
+        }
+    }
+
+    /// recover from a token after an object member that is neither ',' nor
+    /// '}' (nor ']' or the end of the input, which the caller handles)
+    template<typename SAX>
+    next_step recover_member(SAX* sax, std::true_type /*allow_recovery*/)
+    {
+        if (last_token == token_type::parse_error)
+        {
+            recover_token();
+        }
+        if (last_token == token_type::value_string)
+        {
+            // a missing ','; the next key begins here
+            return parse_key(sax);
+        }
+        skip_member();
+        return next_step::evaluate_state;
+    }
+
+    /// the parser for parse() and accept() never recovers (and does not come
+    /// here, as report_error() returned false)
+    template<typename SAX>
+    JSON_INTERNAL_ALWAYS_INLINE
+    std::false_type recover_member(SAX* /*sax*/, std::false_type /*allow_recovery*/) const noexcept
+    {
+        return {};
+    }
+
     /// get next token from lexer
     token_type get_token()
     {
@@ -612,6 +1300,12 @@ class parser
     const bool allow_exceptions = true;
     /// whether trailing commas in objects and arrays should be ignored (true) or signaled as errors (false)
     const bool ignore_trailing_commas = false;
+    /// whether an error was reported to the SAX parser
+    bool error_reported = false;
+    /// the position of the last reported error
+    std::size_t last_error_position = 0;
+    /// the token of the last reported error
+    token_type last_error_token = token_type::uninitialized;
 };
 
 }  // namespace detail

include/nlohmann/detail/input/string_scan.hpp

@@ -96,7 +96,7 @@ JSON_HEDLEY_ALWAYS_INLINE std::size_t find_string_special(const unsigned char* d
 // printable ASCII except '"' and '\\' - is emitted unchanged. Note this differs
 // from is_string_special() only in that 0x7F is also a stop (it is escaped as
 // \u007f under ensure_ascii).
-constexpr bool is_ascii_copyable(unsigned char c) noexcept
+inline bool is_ascii_copyable(unsigned char c) noexcept
 {
     return c >= 0x20u && c < 0x7Fu && c != '"' && c != '\\';
 }

include/nlohmann/detail/iterators/iter_impl.hpp

@@ -108,18 +108,33 @@ class iter_impl // NOLINT(cppcoreguidelines-special-member-functions,hicpp-speci
     {
         JSON_ASSERT(m_object != nullptr);
 
-        if (m_object->is_object())
+        switch (m_object->m_data.m_type)
         {
-            m_it.object_iterator = typename object_t::iterator();
-        }
-        else if (m_object->is_array())
-        {
-            m_it.array_iterator = typename array_t::iterator();
-        }
-        else
-        {
-            // primitive (is_primitive(), including discarded)
-            m_it.primitive_iterator = primitive_iterator_t();
+            case value_t::object:
+            {
+                m_it.object_iterator = typename object_t::iterator();
+                break;
+            }
+
+            case value_t::array:
+            {
+                m_it.array_iterator = typename array_t::iterator();
+                break;
+            }
+
+            case value_t::null:
+            case value_t::string:
+            case value_t::boolean:
+            case value_t::number_integer:
+            case value_t::number_unsigned:
+            case value_t::number_float:
+            case value_t::binary:
+            case value_t::discarded:
+            default:
+            {
+                m_it.primitive_iterator = primitive_iterator_t();
+                break;
+            }
         }
     }
 
@@ -234,28 +249,33 @@ class iter_impl // NOLINT(cppcoreguidelines-special-member-functions,hicpp-speci
     {
         JSON_ASSERT(m_object != nullptr);
 
-        if (m_object->is_object())
+        switch (m_object->m_data.m_type)
         {
-            m_it.object_iterator = m_object->m_data.m_value.object->end();
-        }
-        else if (m_object->is_array())
-        {
-            m_it.array_iterator = m_object->m_data.m_value.array->end();
-        }
-        else
-        {
-            // primitive (is_primitive(), including discarded)
-            m_it.primitive_iterator.set_end();
-        }
-    }
+            case value_t::object:
+            {
+                m_it.object_iterator = m_object->m_data.m_value.object->end();
+                break;
+            }
 
-    /// @throw invalid_iterator.212 if @a other points to another container
-    template<typename IterImpl>
-    JSON_HEDLEY_ALWAYS_INLINE void check_same_container(const IterImpl& other) const
-    {
-        if (JSON_HEDLEY_UNLIKELY(m_object != other.m_object))
-        {
-            throw_literal<invalid_iterator>(exception_id::iterators_compare_different_values, "cannot compare iterators of different containers", m_object);
+            case value_t::array:
+            {
+                m_it.array_iterator = m_object->m_data.m_value.array->end();
+                break;
+            }
+
+            case value_t::null:
+            case value_t::string:
+            case value_t::boolean:
+            case value_t::number_integer:
+            case value_t::number_unsigned:
+            case value_t::number_float:
+            case value_t::binary:
+            case value_t::discarded:
+            default:
+            {
+                m_it.primitive_iterator.set_end();
+                break;
+            }
         }
     }
 
@@ -404,7 +424,10 @@ class iter_impl // NOLINT(cppcoreguidelines-special-member-functions,hicpp-speci
     bool operator==(const IterImpl& other) const
     {
         // if objects are not the same, the comparison is undefined
-        check_same_container(other);
+        if (JSON_HEDLEY_UNLIKELY(m_object != other.m_object))
+        {
+            throw_literal<invalid_iterator>(exception_id::iterators_compare_different_values, "cannot compare iterators of different containers", m_object);
+        }
 
         // value-initialized forward iterators can be compared, and must compare equal to other value-initialized iterators of the same type #4493
         if (m_object == nullptr)
@@ -443,7 +466,10 @@ class iter_impl // NOLINT(cppcoreguidelines-special-member-functions,hicpp-speci
     bool operator<(const iter_impl& other) const
     {
         // if objects are not the same, the comparison is undefined
-        check_same_container(other);
+        if (JSON_HEDLEY_UNLIKELY(m_object != other.m_object))
+        {
+            throw_literal<invalid_iterator>(exception_id::iterators_compare_different_values, "cannot compare iterators of different containers", m_object);
+        }
 
         // value-initialized forward iterators can be compared, and must compare equal to other value-initialized iterators of the same type #4493
         if (m_object == nullptr)
@@ -452,18 +478,25 @@ class iter_impl // NOLINT(cppcoreguidelines-special-member-functions,hicpp-speci
             return false;
         }
 
-        if (m_object->is_object())
+        switch (m_object->m_data.m_type)
         {
-            throw_literal<invalid_iterator>(exception_id::iterator_order_on_object, "cannot compare order of object iterators", m_object);
-        }
+            case value_t::object:
+                throw_literal<invalid_iterator>(exception_id::iterator_order_on_object, "cannot compare order of object iterators", m_object);
 
-        if (m_object->is_array())
-        {
-            return (m_it.array_iterator < other.m_it.array_iterator);
-        }
+            case value_t::array:
+                return (m_it.array_iterator < other.m_it.array_iterator);
 
-        // primitive (is_primitive(), including discarded)
-        return (m_it.primitive_iterator < other.m_it.primitive_iterator);
+            case value_t::null:
+            case value_t::string:
+            case value_t::boolean:
+            case value_t::number_integer:
+            case value_t::number_unsigned:
+            case value_t::number_float:
+            case value_t::binary:
+            case value_t::discarded:
+            default:
+                return (m_it.primitive_iterator < other.m_it.primitive_iterator);
+        }
     }
 
     /*!
@@ -501,19 +534,30 @@ class iter_impl // NOLINT(cppcoreguidelines-special-member-functions,hicpp-speci
     {
         JSON_ASSERT(m_object != nullptr);
 
-        if (m_object->is_object())
+        switch (m_object->m_data.m_type)
         {
-            throw_literal<invalid_iterator>(exception_id::iterator_arithmetic_on_object, "cannot use offsets with object iterators", m_object);
-        }
+            case value_t::object:
+                throw_literal<invalid_iterator>(exception_id::iterator_arithmetic_on_object, "cannot use offsets with object iterators", m_object);
 
-        if (m_object->is_array())
-        {
-            std::advance(m_it.array_iterator, i);
-        }
-        else
-        {
-            // primitive (is_primitive(), including discarded)
-            m_it.primitive_iterator += i;
+            case value_t::array:
+            {
+                std::advance(m_it.array_iterator, i);
+                break;
+            }
+
+            case value_t::null:
+            case value_t::string:
+            case value_t::boolean:
+            case value_t::number_integer:
+            case value_t::number_unsigned:
+            case value_t::number_float:
+            case value_t::binary:
+            case value_t::discarded:
+            default:
+            {
+                m_it.primitive_iterator += i;
+                break;
+            }
         }
 
         return *this;
@@ -569,18 +613,25 @@ class iter_impl // NOLINT(cppcoreguidelines-special-member-functions,hicpp-speci
     {
         JSON_ASSERT(m_object != nullptr);
 
-        if (m_object->is_object())
+        switch (m_object->m_data.m_type)
         {
-            throw_literal<invalid_iterator>(exception_id::iterator_arithmetic_on_object, "cannot use offsets with object iterators", m_object);
-        }
+            case value_t::object:
+                throw_literal<invalid_iterator>(exception_id::iterator_arithmetic_on_object, "cannot use offsets with object iterators", m_object);
 
-        if (m_object->is_array())
-        {
-            return m_it.array_iterator - other.m_it.array_iterator;
-        }
+            case value_t::array:
+                return m_it.array_iterator - other.m_it.array_iterator;
 
-        // primitive (is_primitive(), including discarded)
-        return m_it.primitive_iterator - other.m_it.primitive_iterator;
+            case value_t::null:
+            case value_t::string:
+            case value_t::boolean:
+            case value_t::number_integer:
+            case value_t::number_unsigned:
+            case value_t::number_float:
+            case value_t::binary:
+            case value_t::discarded:
+            default:
+                return m_it.primitive_iterator - other.m_it.primitive_iterator;
+        }
     }
 
     /*!

include/nlohmann/detail/iterators/iteration_proxy.hpp

@@ -109,25 +109,35 @@ template<typename IteratorType> class iteration_proxy_value
     {
         JSON_ASSERT(anchor.m_object != nullptr);
 
-        if (anchor.m_object->is_array())
+        switch (anchor.m_object->type())
         {
             // use integer array index as key
-            if (array_index != array_index_last)
+            case value_t::array:
             {
-                int_to_string( array_index_str, array_index );
-                array_index_last = array_index;
+                if (array_index != array_index_last)
+                {
+                    int_to_string( array_index_str, array_index );
+                    array_index_last = array_index;
+                }
+                return array_index_str;
             }
-            return array_index_str;
-        }
 
-        if (anchor.m_object->is_object())
-        {
             // use key from the object
-            return anchor.key();
-        }
+            case value_t::object:
+                return anchor.key();
 
-        // use an empty key for all primitive types
-        return empty_str;
+            // use an empty key for all primitive types
+            case value_t::null:
+            case value_t::string:
+            case value_t::boolean:
+            case value_t::number_integer:
+            case value_t::number_unsigned:
+            case value_t::number_float:
+            case value_t::binary:
+            case value_t::discarded:
+            default:
+                return empty_str;
+        }
     }
 
     /// return value of the iterator

include/nlohmann/detail/json_pointer.hpp

@@ -38,102 +38,6 @@ NLOHMANN_JSON_NAMESPACE_BEGIN
 namespace detail
 {
 struct json_pointer_access;
-
-/*!
-@brief result of parse_array_index()
-
-json_pointer::array_index() maps each value to the corresponding
-parse_error/out_of_range exception; json_pointer::contains() and
-get_checked_or_null(), which must not throw for an out-of-range or
-unrepresentable index, and the json_view pointer resolution switch on it
-directly instead.
-*/
-enum class array_index_status
-{
-    ok,                ///< @a s is a valid, representable array index
-    leading_zero,      ///< @a s begins with '0' but has more than one character
-    not_a_number,      ///< @a s is neither empty nor "-" and does not begin with a digit
-    unresolved,        ///< @a s could not be converted to an integer
-    exceeds_size_type  ///< @a s converts to an integer that exceeds SizeType
-};
-
-/*!
-@brief convert a JSON Pointer reference token into an array index (RFC 6901, Sect. 4)
-
-@param[in] s    reference token to be converted into an array index; its data()
-                must be NUL-terminated
-@param[out] idx the integer representation of @a s if array_index_status::ok
-                is returned; left unchanged otherwise
-
-@return whether @a s is a valid array index, and if not, why
-
-@note this function never throws; json_pointer::array_index(), the callers that
-      must not throw, and the json_view pointer resolution build on it instead
-      of each re-implementing the RFC 6901 digit rules and the range check
-@note force-inlined: as a free function template it is no longer inlined into
-      the pointer resolution, which makes JsonPointerAt up to 17% slower (#5891)
-*/
-template<typename SizeType, typename StringType>
-JSON_HEDLEY_ALWAYS_INLINE array_index_status parse_array_index(const StringType& s, SizeType& idx) noexcept
-{
-    // error condition (cf. RFC 6901, Sect. 4)
-    if (JSON_HEDLEY_UNLIKELY(s.size() > 1 && s[0] == '0'))
-    {
-        return array_index_status::leading_zero;
-    }
-
-    // error condition (cf. RFC 6901, Sect. 4); this also covers single-
-    // character tokens, so "/x" and "/xy" fail alike; "-" and the empty
-    // token are left to the conversion below and are reported as unresolved
-    if (JSON_HEDLEY_UNLIKELY(!s.empty() && !(s.size() == 1 && s[0] == '-') && !(s[0] >= '0' && s[0] <= '9')))
-    {
-        return array_index_status::not_a_number;
-    }
-
-    // fast path: up to 18 decimal digits cannot overflow unsigned long long or
-    // SizeType (9 digits for a 32-bit SizeType), so strtoull and errno are not needed
-    constexpr std::size_t max_fast_digits = sizeof(SizeType) >= 8 ? 18 : 9;
-    if (!s.empty() && s.size() <= max_fast_digits)
-    {
-        SizeType acc = 0;
-        for (std::size_t i = 0; i < s.size(); ++i)
-        {
-            const char c = s[i];
-            if (JSON_HEDLEY_UNLIKELY(c < '0' || c > '9'))
-            {
-                break; // not a plain number: let strtoull decide
-            }
-            acc = static_cast<SizeType>((acc * 10) + static_cast<SizeType>(c - '0'));
-            if (JSON_HEDLEY_UNLIKELY(i == s.size() - 1))
-            {
-                idx = acc;
-                return array_index_status::ok;
-            }
-        }
-    }
-
-    const char* p = s.data();
-    char* p_end = nullptr; // NOLINT(misc-const-correctness)
-    errno = 0; // strtoull doesn't reset errno
-    const unsigned long long res = std::strtoull(p, &p_end, 10); // NOLINT(runtime/int)
-    if (p == p_end // invalid input or empty string
-            || errno == ERANGE // out of range
-            || JSON_HEDLEY_UNLIKELY(static_cast<std::size_t>(p_end - p) != s.size())) // incomplete read
-    {
-        return array_index_status::unresolved;
-    }
-
-    // the index does not fit into SizeType; on 64-bit platforms this is
-    // only SIZE_MAX itself (see #2203 and #5395)
-    if (res >= static_cast<unsigned long long>((std::numeric_limits<SizeType>::max)()))  // NOLINT(runtime/int)
-    {
-        return array_index_status::exceeds_size_type;
-    }
-
-    idx = static_cast<SizeType>(res);
-    return array_index_status::ok;
-}
-
 }  // namespace detail
 
 /// @brief JSON Pointer defines a string syntax for identifying a specific value within a JSON document
@@ -398,28 +302,6 @@ class json_pointer
         JSON_THROW(detail::out_of_range::create(detail::exception_id::patch_on_root, "JSON pointer has no parent", nullptr));
     }
 
-    /// whether the '~' at @a pos of @a s is not followed by '0' or '1'
-    JSON_HEDLEY_ALWAYS_INLINE static bool is_invalid_escape(const string_t& s, std::size_t pos)
-    {
-        return pos == s.size() - 1 || (s[pos + 1] != '0' && s[pos + 1] != '1');
-    }
-
-    /// @throw parse_error.107 always
-    JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE static void throw_missing_slash(const string_t& pointer)
-    {
-        static_cast<void>(pointer); // unused when JSON_NOEXCEPTION is defined
-        JSON_THROW(detail::parse_error::create(detail::exception_id::pointer_missing_slash, 1, detail::concat("JSON pointer must be empty or begin with '/' - was: '", pointer, "'"), nullptr));
-    }
-
-    /// @throw parse_error.107 if a nonempty @a pointer does not begin with '/'
-    JSON_HEDLEY_ALWAYS_INLINE static void check_begins_with_slash(const string_t& pointer)
-    {
-        if (JSON_HEDLEY_UNLIKELY(pointer[0] != '/'))
-        {
-            throw_missing_slash(pointer);
-        }
-    }
-
     /// @throw out_of_range.404 always
     template<typename BasicJsonContext>
     JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE static void throw_unresolved(const string_t& reference_token, BasicJsonContext context)
@@ -429,7 +311,21 @@ class json_pointer
         JSON_THROW(detail::out_of_range::create(detail::exception_id::pointer_unresolved, detail::concat("unresolved reference token '", reference_token, "'"), context));
     }
 
-    using array_index_status = detail::array_index_status;
+    /*!
+    @brief result of @ref parse_array_index
+
+    @ref array_index maps each value to the corresponding parse_error/out_of_range
+    exception; @ref contains and @ref get_checked_or_null, which must not throw for
+    an out-of-range or unrepresentable index, switch on it directly instead.
+    */
+    enum class array_index_status
+    {
+        ok,                ///< @a s is a valid, representable array index
+        leading_zero,      ///< @a s begins with '0' but has more than one character
+        not_a_number,      ///< @a s is neither empty nor "-" and does not begin with a digit
+        unresolved,        ///< @a s could not be converted to an integer
+        exceeds_size_type  ///< @a s converts to an integer that exceeds size_type
+    };
 
     /// whether a reference token is "-" (a plain comparison with a string literal is not always inlined)
     static bool is_dash(const string_t& s) noexcept
@@ -437,11 +333,78 @@ class json_pointer
         return s.size() == 1 && s[0] == '-';
     }
 
-    /// detail::parse_array_index() for the size_type of @a BasicJsonType
+    /*!
+    @param[in] s    reference token to be converted into an array index
+    @param[out] idx the integer representation of @a s if @ref array_index_status::ok
+                     is returned; left unchanged otherwise
+
+    @return whether @a s is a valid array index, and if not, why
+
+    @note this function never throws; @ref array_index and the callers that must not
+          throw (@ref contains, @ref get_checked_or_null) build on it instead of each
+          re-implementing the RFC 6901 digit rules and the @a size_type range check
+    */
     template<typename BasicJsonType>
     static array_index_status parse_array_index(const string_t& s, typename BasicJsonType::size_type& idx) noexcept
     {
-        return detail::parse_array_index(s, idx);
+        using size_type = typename BasicJsonType::size_type;
+
+        // error condition (cf. RFC 6901, Sect. 4)
+        if (JSON_HEDLEY_UNLIKELY(s.size() > 1 && s[0] == '0'))
+        {
+            return array_index_status::leading_zero;
+        }
+
+        // error condition (cf. RFC 6901, Sect. 4); this also covers single-
+        // character tokens, so "/x" and "/xy" fail alike; "-" and the empty
+        // token are left to the conversion below and are reported as unresolved
+        if (JSON_HEDLEY_UNLIKELY(!s.empty() && !is_dash(s) && !(s[0] >= '0' && s[0] <= '9')))
+        {
+            return array_index_status::not_a_number;
+        }
+
+        // fast path: up to 18 decimal digits cannot overflow unsigned long long or
+        // size_type (9 digits for a 32-bit size_type), so strtoull and errno are not needed
+        constexpr std::size_t max_fast_digits = sizeof(size_type) >= 8 ? 18 : 9;
+        if (!s.empty() && s.size() <= max_fast_digits)
+        {
+            size_type acc = 0;
+            for (std::size_t i = 0; i < s.size(); ++i)
+            {
+                const char c = s[i];
+                if (JSON_HEDLEY_UNLIKELY(c < '0' || c > '9'))
+                {
+                    break; // not a plain number: let strtoull decide
+                }
+                acc = static_cast<size_type>((acc * 10) + static_cast<size_type>(c - '0'));
+                if (JSON_HEDLEY_UNLIKELY(i == s.size() - 1))
+                {
+                    idx = acc;
+                    return array_index_status::ok;
+                }
+            }
+        }
+
+        const char* p = s.data();
+        char* p_end = nullptr; // NOLINT(misc-const-correctness)
+        errno = 0; // strtoull doesn't reset errno
+        const unsigned long long res = std::strtoull(p, &p_end, 10); // NOLINT(runtime/int)
+        if (p == p_end // invalid input or empty string
+                || errno == ERANGE // out of range
+                || JSON_HEDLEY_UNLIKELY(static_cast<std::size_t>(p_end - p) != s.size())) // incomplete read
+        {
+            return array_index_status::unresolved;
+        }
+
+        // the index does not fit into size_type; on 64-bit platforms this is
+        // only SIZE_MAX itself (see #2203 and #5395)
+        if (res >= static_cast<unsigned long long>((std::numeric_limits<size_type>::max)()))  // NOLINT(runtime/int)
+        {
+            return array_index_status::exceeds_size_type;
+        }
+
+        idx = static_cast<size_type>(res);
+        return array_index_status::ok;
     }
 
     /*!
@@ -491,7 +454,10 @@ class json_pointer
             return reference_tokens.empty();
         }
 
-        check_begins_with_slash(reference_string);
+        if (JSON_HEDLEY_UNLIKELY(reference_string[0] != '/'))
+        {
+            JSON_THROW(detail::parse_error::create(detail::exception_id::pointer_missing_slash, 1, detail::concat("JSON pointer must be empty or begin with '/' - was: '", reference_string, "'"), nullptr));
+        }
 
         // the whole string is validated, whether or not it matches
         const std::size_t size = reference_string.size();
@@ -499,7 +465,7 @@ class json_pointer
                 pos != string_t::npos;
                 pos = reference_string.find_first_of('~', pos + 1))
         {
-            if (JSON_HEDLEY_UNLIKELY(is_invalid_escape(reference_string, pos)))
+            if (JSON_HEDLEY_UNLIKELY(pos == size - 1 || (reference_string[pos + 1] != '0' && reference_string[pos + 1] != '1')))
             {
                 JSON_THROW(detail::parse_error::create(detail::exception_id::pointer_invalid_escape, 0, "escape character '~' must be followed with '0' or '1'", nullptr));
             }
@@ -777,7 +743,10 @@ class json_pointer
         }
 
         const char* const data = key.data();
-        check_begins_with_slash(key);
+        if (JSON_HEDLEY_UNLIKELY(data[0] != '/'))
+        {
+            JSON_THROW(detail::parse_error::create(detail::exception_id::pointer_missing_slash, 1, detail::concat("JSON pointer must be empty or begin with '/' - was: '", key, "'"), nullptr));
+        }
 
         // The beginning of the key that it has in common with the previous
         // one, which was checked already, is skipped: the scan starts with
@@ -810,7 +779,7 @@ class json_pointer
             else if (data[i] == '~')
             {
                 // ~ must be followed by 0 or 1
-                if (JSON_HEDLEY_UNLIKELY(is_invalid_escape(key, i)))
+                if (JSON_HEDLEY_UNLIKELY(i + 1 == size || (data[i + 1] != '0' && data[i + 1] != '1')))
                 {
                     JSON_THROW(detail::parse_error::create(detail::exception_id::pointer_invalid_escape, 0, "escape character '~' must be followed with '0' or '1'", nullptr));
                 }
@@ -909,46 +878,58 @@ class json_pointer
                 ++end;
             }
 
-            if (ptr->is_object())
+            switch (ptr->type())
             {
-                token.clear();
-                token.append(begin, static_cast<std::size_t>(end - begin));
-                const auto it = ptr->m_data.m_value.object->find(token);
-                if (JSON_HEDLEY_UNLIKELY(it == ptr->m_data.m_value.object->end()))
+                case detail::value_t::object:
                 {
-                    return nullptr;
-                }
-                ptr = &it->second;
-            }
-            else if (ptr->is_array())
-            {
-                // an index of up to 18 digits without a leading zero fits into size_type
-                // (9 digits for a 32-bit size_type); everything else is left to json_pointer
-                constexpr std::size_t max_digits = sizeof(typename BasicJsonType::size_type) >= 8 ? 18 : 9;
-                const auto digits = static_cast<std::size_t>(end - begin);
-                if (JSON_HEDLEY_UNLIKELY(digits == 0 || digits > max_digits || (digits > 1 && *begin == '0')))
-                {
-                    return nullptr;
-                }
-                typename BasicJsonType::size_type idx = 0;
-                for (const char* c = begin; c != end; ++c)
-                {
-                    if (JSON_HEDLEY_UNLIKELY(*c < '0' || *c > '9'))
+                    token.clear();
+                    token.append(begin, static_cast<std::size_t>(end - begin));
+                    const auto it = ptr->m_data.m_value.object->find(token);
+                    if (JSON_HEDLEY_UNLIKELY(it == ptr->m_data.m_value.object->end()))
                     {
                         return nullptr;
                     }
-                    idx = static_cast<typename BasicJsonType::size_type>((idx * 10) + static_cast<typename BasicJsonType::size_type>(*c - '0'));
+                    ptr = &it->second;
+                    break;
                 }
-                if (JSON_HEDLEY_UNLIKELY(idx >= ptr->m_data.m_value.array->size()))
+
+                case detail::value_t::array:
                 {
-                    return nullptr;
+                    // an index of up to 18 digits without a leading zero fits into size_type
+                    // (9 digits for a 32-bit size_type); everything else is left to json_pointer
+                    constexpr std::size_t max_digits = sizeof(typename BasicJsonType::size_type) >= 8 ? 18 : 9;
+                    const auto digits = static_cast<std::size_t>(end - begin);
+                    if (JSON_HEDLEY_UNLIKELY(digits == 0 || digits > max_digits || (digits > 1 && *begin == '0')))
+                    {
+                        return nullptr;
+                    }
+                    typename BasicJsonType::size_type idx = 0;
+                    for (const char* c = begin; c != end; ++c)
+                    {
+                        if (JSON_HEDLEY_UNLIKELY(*c < '0' || *c > '9'))
+                        {
+                            return nullptr;
+                        }
+                        idx = static_cast<typename BasicJsonType::size_type>((idx * 10) + static_cast<typename BasicJsonType::size_type>(*c - '0'));
+                    }
+                    if (JSON_HEDLEY_UNLIKELY(idx >= ptr->m_data.m_value.array->size()))
+                    {
+                        return nullptr;
+                    }
+                    ptr = &(*ptr->m_data.m_value.array)[idx];
+                    break;
                 }
-                ptr = &(*ptr->m_data.m_value.array)[idx];
-            }
-            else
-            {
-                // primitive value with a reference token left
-                return nullptr;
+
+                case detail::value_t::null:
+                case detail::value_t::string:
+                case detail::value_t::boolean:
+                case detail::value_t::number_integer:
+                case detail::value_t::number_unsigned:
+                case detail::value_t::number_float:
+                case detail::value_t::binary:
+                case detail::value_t::discarded:
+                default:
+                    return nullptr;
             }
             pos = static_cast<std::size_t>(end - path.data());
             steps.emplace_back(pos, ptr);
@@ -998,28 +979,40 @@ class json_pointer
                        : detail::value_t::object;
             }
 
-            if (ptr->is_object())
+            switch (ptr->type())
             {
-                // use unchecked object access
-                ptr = &object_member(*ptr, reference_token);
-            }
-            else if (ptr->is_array())
-            {
-                if (reference_token == "-")
+                case detail::value_t::object:
                 {
-                    // explicitly treat "-" as index beyond the end
-                    ptr = &ptr->operator[](ptr->m_data.m_value.array->size());
+                    // use unchecked object access
+                    ptr = &object_member(*ptr, reference_token);
+                    break;
                 }
-                else
+
+                case detail::value_t::array:
                 {
-                    // convert array index to number; unchecked access
-                    ptr = &ptr->operator[](array_index<BasicJsonType>(reference_token));
+                    if (reference_token == "-")
+                    {
+                        // explicitly treat "-" as index beyond the end
+                        ptr = &ptr->operator[](ptr->m_data.m_value.array->size());
+                    }
+                    else
+                    {
+                        // convert array index to number; unchecked access
+                        ptr = &ptr->operator[](array_index<BasicJsonType>(reference_token));
+                    }
+                    break;
                 }
-            }
-            else
-            {
-                // primitive value with a reference token left
-                throw_unresolved(reference_token, ptr);
+
+                case detail::value_t::null:
+                case detail::value_t::string:
+                case detail::value_t::boolean:
+                case detail::value_t::number_integer:
+                case detail::value_t::number_unsigned:
+                case detail::value_t::number_float:
+                case detail::value_t::binary:
+                case detail::value_t::discarded:
+                default:
+                    throw_unresolved(reference_token, ptr);
             }
         }
 
@@ -1037,39 +1030,51 @@ class json_pointer
     {
         for (const auto& reference_token : reference_tokens)
         {
-            if (ptr->is_object())
+            switch (ptr->type())
             {
-                const auto it = ptr->m_data.m_value.object->find(reference_token);
-                if (JSON_HEDLEY_UNLIKELY(it == ptr->m_data.m_value.object->end()))
+                case detail::value_t::object:
                 {
-                    // at() throws out_of_range.403 with the usual message
-                    ptr->at(reference_token);
-                }
-                ptr = &it->second;
-            }
-            else if (ptr->is_array())
-            {
-                if (JSON_HEDLEY_UNLIKELY(is_dash(reference_token)))
-                {
-                    // "-" always fails the range check
-                    JSON_THROW(detail::out_of_range::create(detail::exception_id::pointer_past_the_end_index, detail::concat(
-                            "array index '-' (", std::to_string(ptr->m_data.m_value.array->size()),
-                            ") is out of range"), ptr));
+                    const auto it = ptr->m_data.m_value.object->find(reference_token);
+                    if (JSON_HEDLEY_UNLIKELY(it == ptr->m_data.m_value.object->end()))
+                    {
+                        // at() throws out_of_range.403 with the usual message
+                        ptr->at(reference_token);
+                    }
+                    ptr = &it->second;
+                    break;
                 }
 
-                const auto idx = array_index<BasicJsonType>(reference_token);
-                // Bounds check before access to avoid exception with JSON_NOEXCEPTION
-                if (JSON_HEDLEY_UNLIKELY(idx >= ptr->m_data.m_value.array->size()))
+                case detail::value_t::array:
                 {
-                    JSON_THROW(detail::out_of_range::create(detail::exception_id::array_index_out_of_range, detail::concat(
-                            "array index ", std::to_string(idx), " is out of range"), ptr));
+                    if (JSON_HEDLEY_UNLIKELY(is_dash(reference_token)))
+                    {
+                        // "-" always fails the range check
+                        JSON_THROW(detail::out_of_range::create(detail::exception_id::pointer_past_the_end_index, detail::concat(
+                                "array index '-' (", std::to_string(ptr->m_data.m_value.array->size()),
+                                ") is out of range"), ptr));
+                    }
+
+                    const auto idx = array_index<BasicJsonType>(reference_token);
+                    // Bounds check before access to avoid exception with JSON_NOEXCEPTION
+                    if (JSON_HEDLEY_UNLIKELY(idx >= ptr->m_data.m_value.array->size()))
+                    {
+                        JSON_THROW(detail::out_of_range::create(detail::exception_id::array_index_out_of_range, detail::concat(
+                                "array index ", std::to_string(idx), " is out of range"), ptr));
+                    }
+                    ptr = &(*ptr->m_data.m_value.array)[idx];
+                    break;
                 }
-                ptr = &(*ptr->m_data.m_value.array)[idx];
-            }
-            else
-            {
-                // primitive value with a reference token left
-                throw_unresolved(reference_token, ptr);
+
+                case detail::value_t::null:
+                case detail::value_t::string:
+                case detail::value_t::boolean:
+                case detail::value_t::number_integer:
+                case detail::value_t::number_unsigned:
+                case detail::value_t::number_float:
+                case detail::value_t::binary:
+                case detail::value_t::discarded:
+                default:
+                    throw_unresolved(reference_token, ptr);
             }
         }
 
@@ -1098,32 +1103,44 @@ class json_pointer
     {
         for (const auto& reference_token : reference_tokens)
         {
-            if (ptr->is_object())
+            switch (ptr->type())
             {
-                // use unchecked object access; the const operator[]
-                // asserts that the key exists
-                const auto it = ptr->m_data.m_value.object->find(reference_token);
-                JSON_ASSERT(it != ptr->m_data.m_value.object->end());
-                ptr = &it->second;
-            }
-            else if (ptr->is_array())
-            {
-                if (JSON_HEDLEY_UNLIKELY(is_dash(reference_token)))
+                case detail::value_t::object:
                 {
-                    // "-" cannot be used for const access
-                    JSON_THROW(detail::out_of_range::create(detail::exception_id::pointer_past_the_end_index, detail::concat("array index '-' (", std::to_string(ptr->m_data.m_value.array->size()), ") is out of range"), ptr));
+                    // use unchecked object access; the const operator[]
+                    // asserts that the key exists
+                    const auto it = ptr->m_data.m_value.object->find(reference_token);
+                    JSON_ASSERT(it != ptr->m_data.m_value.object->end());
+                    ptr = &it->second;
+                    break;
                 }
 
-                // use unchecked array access; the const operator[]
-                // asserts that the index exists
-                const auto idx = array_index<BasicJsonType>(reference_token);
-                JSON_ASSERT(idx < ptr->m_data.m_value.array->size());
-                ptr = &(*ptr->m_data.m_value.array)[idx];
-            }
-            else
-            {
-                // primitive value with a reference token left
-                throw_unresolved(reference_token, ptr);
+                case detail::value_t::array:
+                {
+                    if (JSON_HEDLEY_UNLIKELY(is_dash(reference_token)))
+                    {
+                        // "-" cannot be used for const access
+                        JSON_THROW(detail::out_of_range::create(detail::exception_id::pointer_past_the_end_index, detail::concat("array index '-' (", std::to_string(ptr->m_data.m_value.array->size()), ") is out of range"), ptr));
+                    }
+
+                    // use unchecked array access; the const operator[]
+                    // asserts that the index exists
+                    const auto idx = array_index<BasicJsonType>(reference_token);
+                    JSON_ASSERT(idx < ptr->m_data.m_value.array->size());
+                    ptr = &(*ptr->m_data.m_value.array)[idx];
+                    break;
+                }
+
+                case detail::value_t::null:
+                case detail::value_t::string:
+                case detail::value_t::boolean:
+                case detail::value_t::number_integer:
+                case detail::value_t::number_unsigned:
+                case detail::value_t::number_float:
+                case detail::value_t::binary:
+                case detail::value_t::discarded:
+                default:
+                    throw_unresolved(reference_token, ptr);
             }
         }
 
@@ -1147,54 +1164,66 @@ class json_pointer
     {
         for (const auto& reference_token : reference_tokens)
         {
-            if (ptr->is_object())
+            switch (ptr->type())
             {
-                const auto it = ptr->m_data.m_value.object->find(reference_token);
-                if (JSON_HEDLEY_UNLIKELY(it == ptr->m_data.m_value.object->end()))
+                case detail::value_t::object:
                 {
-                    return nullptr;
-                }
-                ptr = &it->second;
-            }
-            else if (ptr->is_array())
-            {
-                if (JSON_HEDLEY_UNLIKELY(is_dash(reference_token)))
-                {
-                    // "-" always fails the range check
-                    return nullptr;
-                }
-
-                // a malformed index throws parse_error.106/109; an
-                // index that is syntactically valid but cannot be
-                // represented (out_of_range.404/410) is treated like an
-                // out-of-range index below
-                typename BasicJsonType::size_type idx{};
-                switch (parse_array_index<BasicJsonType>(reference_token, idx))
-                {
-                    // the branches differ in their messages, not after JSON_THROW's expansion
-                    // NOLINTNEXTLINE(bugprone-branch-clone)
-                    case array_index_status::leading_zero:
-                        JSON_THROW(detail::parse_error::create(detail::exception_id::pointer_index_leading_zero, 0, detail::concat("array index '", reference_token, "' must not begin with '0'"), nullptr));
-                    case array_index_status::not_a_number:
-                        JSON_THROW(detail::parse_error::create(detail::exception_id::pointer_index_not_a_number, 0, detail::concat("array index '", reference_token, "' is not a number"), nullptr));
-                    case array_index_status::unresolved:
-                    case array_index_status::exceeds_size_type:
+                    const auto it = ptr->m_data.m_value.object->find(reference_token);
+                    if (JSON_HEDLEY_UNLIKELY(it == ptr->m_data.m_value.object->end()))
+                    {
                         return nullptr;
-                    case array_index_status::ok:
-                    default:
-                        break;
+                    }
+                    ptr = &it->second;
+                    break;
                 }
 
-                if (JSON_HEDLEY_UNLIKELY(idx >= ptr->m_data.m_value.array->size()))
+                case detail::value_t::array:
                 {
-                    return nullptr;
+                    if (JSON_HEDLEY_UNLIKELY(is_dash(reference_token)))
+                    {
+                        // "-" always fails the range check
+                        return nullptr;
+                    }
+
+                    // a malformed index throws parse_error.106/109; an
+                    // index that is syntactically valid but cannot be
+                    // represented (out_of_range.404/410) is treated like an
+                    // out-of-range index below
+                    typename BasicJsonType::size_type idx{};
+                    switch (parse_array_index<BasicJsonType>(reference_token, idx))
+                    {
+                        // the branches differ in their messages, not after JSON_THROW's expansion
+                        // NOLINTNEXTLINE(bugprone-branch-clone)
+                        case array_index_status::leading_zero:
+                            JSON_THROW(detail::parse_error::create(detail::exception_id::pointer_index_leading_zero, 0, detail::concat("array index '", reference_token, "' must not begin with '0'"), nullptr));
+                        case array_index_status::not_a_number:
+                            JSON_THROW(detail::parse_error::create(detail::exception_id::pointer_index_not_a_number, 0, detail::concat("array index '", reference_token, "' is not a number"), nullptr));
+                        case array_index_status::unresolved:
+                        case array_index_status::exceeds_size_type:
+                            return nullptr;
+                        case array_index_status::ok:
+                        default:
+                            break;
+                    }
+
+                    if (JSON_HEDLEY_UNLIKELY(idx >= ptr->m_data.m_value.array->size()))
+                    {
+                        return nullptr;
+                    }
+                    ptr = &(*ptr->m_data.m_value.array)[idx];
+                    break;
                 }
-                ptr = &(*ptr->m_data.m_value.array)[idx];
-            }
-            else
-            {
-                // primitive value with a reference token left
-                return nullptr;
+
+                case detail::value_t::null:
+                case detail::value_t::string:
+                case detail::value_t::boolean:
+                case detail::value_t::number_integer:
+                case detail::value_t::number_unsigned:
+                case detail::value_t::number_float:
+                case detail::value_t::binary:
+                case detail::value_t::discarded:
+                default:
+                    return nullptr;
             }
         }
 
@@ -1210,48 +1239,63 @@ class json_pointer
     {
         for (const auto& reference_token : reference_tokens)
         {
-            if (ptr->is_object())
+            switch (ptr->type())
             {
-                const auto it = ptr->m_data.m_value.object->find(reference_token);
-                if (it == ptr->m_data.m_value.object->end())
+                case detail::value_t::object:
                 {
-                    // we did not find the key in the object
-                    return false;
+                    const auto it = ptr->m_data.m_value.object->find(reference_token);
+                    if (it == ptr->m_data.m_value.object->end())
+                    {
+                        // we did not find the key in the object
+                        return false;
+                    }
+
+                    ptr = &it->second;
+                    break;
                 }
 
-                ptr = &it->second;
-            }
-            else if (ptr->is_array())
-            {
-                if (JSON_HEDLEY_UNLIKELY(is_dash(reference_token)))
+                case detail::value_t::array:
                 {
-                    // "-" always fails the range check
-                    return false;
+                    if (JSON_HEDLEY_UNLIKELY(is_dash(reference_token)))
+                    {
+                        // "-" always fails the range check
+                        return false;
+                    }
+
+                    // any parse failure (malformed index, or one that is syntactically
+                    // valid but not representable as size_type) means the reference
+                    // token cannot denote an existing array element -- contains() must
+                    // not throw (see #5395), so it is treated as "not found"
+                    typename BasicJsonType::size_type idx{};
+                    if (JSON_HEDLEY_UNLIKELY(parse_array_index<BasicJsonType>(reference_token, idx) != array_index_status::ok))
+                    {
+                        return false;
+                    }
+
+                    if (idx >= ptr->m_data.m_value.array->size())
+                    {
+                        // index out of range
+                        return false;
+                    }
+
+                    ptr = &(*ptr->m_data.m_value.array)[idx];
+                    break;
                 }
 
-                // any parse failure (malformed index, or one that is syntactically
-                // valid but not representable as size_type) means the reference
-                // token cannot denote an existing array element -- contains() must
-                // not throw (see #5395), so it is treated as "not found"
-                typename BasicJsonType::size_type idx{};
-                if (JSON_HEDLEY_UNLIKELY(parse_array_index<BasicJsonType>(reference_token, idx) != array_index_status::ok))
+                case detail::value_t::null:
+                case detail::value_t::string:
+                case detail::value_t::boolean:
+                case detail::value_t::number_integer:
+                case detail::value_t::number_unsigned:
+                case detail::value_t::number_float:
+                case detail::value_t::binary:
+                case detail::value_t::discarded:
+                default:
                 {
+                    // we do not expect primitive values if there is still a
+                    // reference token to process
                     return false;
                 }
-
-                if (idx >= ptr->m_data.m_value.array->size())
-                {
-                    // index out of range
-                    return false;
-                }
-
-                ptr = &(*ptr->m_data.m_value.array)[idx];
-            }
-            else
-            {
-                // we do not expect primitive values if there is still a
-                // reference token to process
-                return false;
             }
         }
 
@@ -1279,7 +1323,10 @@ class json_pointer
         }
 
         // check if a nonempty reference string begins with slash
-        check_begins_with_slash(reference_string);
+        if (JSON_HEDLEY_UNLIKELY(reference_string[0] != '/'))
+        {
+            JSON_THROW(detail::parse_error::create(detail::exception_id::pointer_missing_slash, 1, detail::concat("JSON pointer must be empty or begin with '/' - was: '", reference_string, "'"), nullptr));
+        }
 
         // every slash starts a reference token
         const std::size_t size = reference_string.size();
@@ -1320,7 +1367,9 @@ class json_pointer
                     JSON_ASSERT(reference_token[pos] == '~');
 
                     // ~ must be followed by 0 or 1
-                    if (JSON_HEDLEY_UNLIKELY(is_invalid_escape(reference_token, pos)))
+                    if (JSON_HEDLEY_UNLIKELY(pos == reference_token.size() - 1 ||
+                                             (reference_token[pos + 1] != '0' &&
+                                              reference_token[pos + 1] != '1')))
                     {
                         JSON_THROW(detail::parse_error::create(detail::exception_id::pointer_invalid_escape, 0, "escape character '~' must be followed with '0' or '1'", nullptr));
                     }
@@ -1477,34 +1526,50 @@ class json_pointer
         auto hint = object.end();
         const auto enter = [&stack, &path, &object, &hint](const BasicJsonType & v)
         {
-            if (v.is_array())
+            switch (v.type())
             {
-                if (v.m_data.m_value.array->empty())
+                case detail::value_t::array:
                 {
-                    // flatten empty array as null
-                    add_flattened(object, hint, path, nullptr, detail::priority_tag<2> {});
+                    if (v.m_data.m_value.array->empty())
+                    {
+                        // flatten empty array as null
+                        add_flattened(object, hint, path, nullptr, detail::priority_tag<2> {});
+                    }
+                    else
+                    {
+                        stack.emplace_back(&v, object_const_iterator(), path.size());
+                    }
+                    return;
                 }
-                else
+
+                case detail::value_t::object:
                 {
-                    stack.emplace_back(&v, object_const_iterator(), path.size());
+                    if (v.m_data.m_value.object->empty())
+                    {
+                        // flatten empty object as null
+                        add_flattened(object, hint, path, nullptr, detail::priority_tag<2> {});
+                    }
+                    else
+                    {
+                        stack.emplace_back(&v, v.m_data.m_value.object->begin(), path.size());
+                    }
+                    return;
                 }
-            }
-            else if (v.is_object())
-            {
-                if (v.m_data.m_value.object->empty())
+
+                case detail::value_t::null:
+                case detail::value_t::string:
+                case detail::value_t::boolean:
+                case detail::value_t::number_integer:
+                case detail::value_t::number_unsigned:
+                case detail::value_t::number_float:
+                case detail::value_t::binary:
+                case detail::value_t::discarded:
+                default:
                 {
-                    // flatten empty object as null
-                    add_flattened(object, hint, path, nullptr, detail::priority_tag<2> {});
+                    // add a primitive value with its reference string
+                    add_flattened(object, hint, path, v, detail::priority_tag<2> {});
+                    return;
                 }
-                else
-                {
-                    stack.emplace_back(&v, v.m_data.m_value.object->begin(), path.size());
-                }
-            }
-            else
-            {
-                // add a primitive value with its reference string
-                add_flattened(object, hint, path, v, detail::priority_tag<2> {});
             }
         };
 

include/nlohmann/detail/macro_scope.hpp

@@ -215,6 +215,17 @@
     #define JSON_NO_UNIQUE_ADDRESS
 #endif
 
+// Inlines small functions even in unoptimized builds, so that they are not
+// emitted. The parsers for parse(), accept(), and from_*() use it for the
+// functions that stand in for the code recovering from errors (see #3989).
+// MSVC is left to decide, as it warns (C4714) where it does not inline a
+// __forceinline function.
+#if defined(_MSC_VER) && !defined(__clang__)
+    #define JSON_INTERNAL_ALWAYS_INLINE
+#else
+    #define JSON_INTERNAL_ALWAYS_INLINE JSON_HEDLEY_ALWAYS_INLINE
+#endif
+
 // Clang targeting MinGW does not survive the thread_local storage the copy
 // constructor uses to bound its descent: every test that copies a value
 // segfaults with clang 11.0.1 and clang 18.1.8, while the same tests pass with

include/nlohmann/detail/macro_unscope.hpp

@@ -19,6 +19,7 @@
 #undef NLOHMANN_CAN_CALL_STD_FUNC_IMPL
 #undef JSON_INLINE_VARIABLE
 #undef JSON_NO_UNIQUE_ADDRESS
+#undef JSON_INTERNAL_ALWAYS_INLINE
 #undef JSON_DISABLE_ENUM_SERIALIZATION
 #undef JSON_DISABLE_TUPLE_REFERENCE_CONVERSION
 #undef JSON_UNLOCKED_FILE_IO

include/nlohmann/detail/output/binary_writer.hpp

@@ -21,7 +21,6 @@
 
 #include <nlohmann/detail/bit_ops.hpp>
 #include <nlohmann/detail/input/binary_reader.hpp>
-#include <nlohmann/detail/input/input_adapters.hpp>
 #include <nlohmann/detail/input/string_scan.hpp>
 #include <nlohmann/detail/macro_scope.hpp>
 #include <nlohmann/detail/output/error_handler.hpp>
@@ -312,7 +311,9 @@ class binary_writer
                 write_cbor_head(0x40, N);
 
                 // step 2: write each element
-                write_bytes(j.m_data.m_value.binary->data(), N);
+                oa.write_characters(
+                      reinterpret_cast<const CharType*>(j.m_data.m_value.binary->data()),
+                      N);
 
                 break;
             }
@@ -325,7 +326,7 @@ class binary_writer
 
             case value_t::discarded:
             default:
-                throw_on_discarded(j, input_format_t::cbor);
+                throw_on_discarded(j, "CBOR");
         }
     }
 
@@ -525,7 +526,9 @@ class binary_writer
                 }
 
                 // step 2: write the byte string
-                write_bytes(j.m_data.m_value.binary->data(), N);
+                oa.write_characters(
+                      reinterpret_cast<const CharType*>(j.m_data.m_value.binary->data()),
+                      N);
 
                 break;
             }
@@ -539,7 +542,7 @@ class binary_writer
 
             case value_t::discarded:
             default:
-                throw_on_discarded(j, input_format_t::msgpack);
+                throw_on_discarded(j, "MessagePack");
         }
     }
 
@@ -623,7 +626,7 @@ class binary_writer
             case value_t::array:
             {
                 bool prefix_required = true;
-                const bool write_closer = write_ubjson_start_container<true>(j, use_count, use_type, add_prefix, use_bjdata, prefix_required);
+                const bool write_closer = write_ubjson_start_array(j, use_count, use_type, add_prefix, use_bjdata, prefix_required);
 
                 {
                     const auto& array = *j.m_data.m_value.array;
@@ -673,7 +676,9 @@ class binary_writer
 
                 if (use_type)
                 {
-                    write_bytes(j.m_data.m_value.binary->data(), j.m_data.m_value.binary->size());
+                    oa.write_characters(
+                          reinterpret_cast<const CharType*>(j.m_data.m_value.binary->data()),
+                          j.m_data.m_value.binary->size());
                 }
                 else
                 {
@@ -699,7 +704,7 @@ class binary_writer
                 }
 
                 bool prefix_required = true;
-                const bool write_closer = write_ubjson_start_container<false>(j, use_count, use_type, add_prefix, use_bjdata, prefix_required);
+                const bool write_closer = write_ubjson_start_object(j, use_count, use_type, add_prefix, use_bjdata, prefix_required);
 
                 write_ubjson_members(j, use_count, use_type, prefix_required, use_bjdata, bjdata_version, depth);
 
@@ -713,7 +718,7 @@ class binary_writer
 
             case value_t::discarded:
             default:
-                throw_on_discarded(j, use_bjdata ? input_format_t::bjdata : input_format_t::ubjson);
+                throw_on_discarded(j, use_bjdata ? "BJData" : "UBJSON");
         }
     }
 
@@ -739,11 +744,11 @@ class binary_writer
     @brief throws because @a j is discarded and cannot be serialized
     @throw type_error.321 always
     */
-    JSON_HEDLEY_NO_RETURN static void throw_on_discarded(const BasicJsonType& j, const input_format_t format)
+    JSON_HEDLEY_NO_RETURN static void throw_on_discarded(const BasicJsonType& j, const char* format_name)
     {
         static_cast<void>(j); // unused when JSON_NOEXCEPTION is defined
-        static_cast<void>(format);
-        JSON_THROW(type_error::create(exception_id::discarded_value_used, concat("cannot serialize discarded value to ", input_format_name(format)), &j));
+        static_cast<void>(format_name);
+        JSON_THROW(type_error::create(exception_id::discarded_value_used, concat("cannot serialize discarded value to ", format_name), &j));
     }
 
     /// how the range of integer type @a NumberType relates to that of @a TargetType
@@ -863,11 +868,11 @@ class binary_writer
     }
 
     /*!
-    @brief throws because the integer @a j is too large for @a format
+    @brief throws because the integer @a j is too large for @a format_name
     @throw out_of_range.407 always
     */
     JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE
-    static void throw_integer_out_of_range(const BasicJsonType& j, const input_format_t format, const char* range)
+    static void throw_integer_out_of_range(const BasicJsonType& j, const char* format_name, const char* range)
     {
         // neither std::to_string(), which has no overload for integer types
         // wider than 64 bits, nor dump(), which would also instantiate the
@@ -878,9 +883,9 @@ class binary_writer
                                    ? integer_to_string(j.m_data.m_value.number_unsigned, std::integral_constant<bool, unsigned_limits::is_signed>())
                                    : integer_to_string(j.m_data.m_value.number_integer, std::integral_constant<bool, integer_limits::is_signed>());
         static_cast<void>(number); // unused when JSON_NOEXCEPTION is defined
-        static_cast<void>(format);
+        static_cast<void>(format_name);
         static_cast<void>(range);
-        JSON_THROW(out_of_range::create(exception_id::integer_too_large, concat("integer number ", number, " cannot be represented by ", input_format_name(format), " as it does not fit ", range), &j));
+        JSON_THROW(out_of_range::create(exception_id::integer_too_large, concat("integer number ", number, " cannot be represented by ", format_name, " as it does not fit ", range), &j));
     }
 
     // The exceptions below are thrown from functions of their own, out of line:
@@ -923,56 +928,53 @@ class binary_writer
         JSON_THROW(other_error::create(exception_id::size_marker_required, "use_type requires use_size = true", &j));
     }
 
-    /// @brief write @a size bytes starting at @a data to the output adapter
-    JSON_HEDLEY_ALWAYS_INLINE
-    void write_bytes(const void* data, const std::size_t size)
-    {
-        oa.write_characters(static_cast<const CharType*>(data), size);
-    }
-
-    /// @brief write the MessagePack prefix of an array or object with @a N
-    ///        elements: the fix marker @a fix_marker | n, or @a marker_16 or
-    ///        @a marker_32 followed by the length
-    JSON_HEDLEY_ALWAYS_INLINE
-    void write_msgpack_container_prefix(const std::size_t N, const BasicJsonType& j,
-                                        const std::uint8_t fix_marker, const std::uint8_t marker_16, const std::uint8_t marker_32)
+    void write_msgpack_array_prefix(const std::size_t N, const BasicJsonType& j)
     {
         const auto n = to_msgpack_length(N, j);
         if (n <= 15)
         {
-            // fixarray, fixmap
-            oa.write_character(to_char_type(static_cast<std::uint8_t>(fix_marker | n)));
+            // fixarray
+            oa.write_character(to_char_type(static_cast<std::uint8_t>(0x90 | n)));
         }
         else if (n <= (std::numeric_limits<std::uint16_t>::max)())
         {
-            // array 16, map 16
-            oa.write_character(to_char_type(marker_16));
+            // array 16
+            oa.write_character(to_char_type(0xDC));
             write_number(static_cast<std::uint16_t>(n));
         }
         else
         {
-            // array 32, map 32
-            oa.write_character(to_char_type(marker_32));
+            // array 32
+            oa.write_character(to_char_type(0xDD));
             write_number(static_cast<std::uint32_t>(n));
         }
     }
 
-    JSON_HEDLEY_ALWAYS_INLINE
-    void write_msgpack_array_prefix(const std::size_t N, const BasicJsonType& j)
-    {
-        write_msgpack_container_prefix(N, j, 0x90, 0xDC, 0xDD);
-    }
-
-    JSON_HEDLEY_ALWAYS_INLINE
     void write_msgpack_object_prefix(const std::size_t N, const BasicJsonType& j)
     {
-        write_msgpack_container_prefix(N, j, 0x80, 0xDE, 0xDF);
+        const auto n = to_msgpack_length(N, j);
+        if (n <= 15)
+        {
+            // fixmap
+            oa.write_character(to_char_type(static_cast<std::uint8_t>(0x80 | (n & 0xF))));
+        }
+        else if (n <= (std::numeric_limits<std::uint16_t>::max)())
+        {
+            // map 16
+            oa.write_character(to_char_type(0xDE));
+            write_number(static_cast<std::uint16_t>(n));
+        }
+        else
+        {
+            // map 32
+            oa.write_character(to_char_type(0xDF));
+            write_number(static_cast<std::uint32_t>(n));
+        }
     }
 
-    /// @brief a CBOR, MessagePack, BON8, or (as the base of @ref ubjson_frame)
-    ///        UBJSON array or object whose elements @ref write_container_iterative,
-    ///        @ref write_bon8_iterative, or @ref write_ubjson_iterative is still
-    ///        writing
+    /// @brief a CBOR, MessagePack, or BON8 array or object whose elements
+    ///        @ref write_cbor_iterative, @ref write_msgpack_iterative, or
+    ///        @ref write_bon8_iterative is still writing
     struct binary_container_frame
     {
         explicit binary_container_frame(const BasicJsonType* value_) noexcept
@@ -1008,82 +1010,19 @@ class binary_writer
         typename BasicJsonType::array_t::const_iterator array_it{};
     };
 
-    /// @brief the CBOR specifics of @ref write_container_iterative
-    struct cbor_walk
-    {
-        static void array_head(binary_writer& w, const BasicJsonType& j)
-        {
-            w.write_cbor_head(0x80, j.m_data.m_value.array->size());
-        }
-
-        static void object_head(binary_writer& w, const BasicJsonType& j)
-        {
-            w.write_cbor_head(0xA0, j.m_data.m_value.object->size());
-        }
-
-        static void scalar(binary_writer& w, const BasicJsonType& j)
-        {
-            w.write_cbor(j);
-        }
-
-        static void string(binary_writer& w, const string_t& key, const BasicJsonType& context)
-        {
-            w.write_cbor_string(key, context);
-        }
-
-        template<typename KeyType>
-        static void key(binary_writer& w, const KeyType& k, const BasicJsonType& object)
-        {
-            w.write_cbor_key(k, object);
-        }
-    };
-
-    /// @brief the MessagePack specifics of @ref write_container_iterative
-    struct msgpack_walk
-    {
-        static void array_head(binary_writer& w, const BasicJsonType& j)
-        {
-            w.write_msgpack_array_prefix(j.m_data.m_value.array->size(), j);
-        }
-
-        static void object_head(binary_writer& w, const BasicJsonType& j)
-        {
-            w.write_msgpack_object_prefix(j.m_data.m_value.object->size(), j);
-        }
-
-        static void scalar(binary_writer& w, const BasicJsonType& j)
-        {
-            w.write_msgpack(j);
-        }
-
-        static void string(binary_writer& w, const string_t& key, const BasicJsonType& context)
-        {
-            w.write_msgpack_string(key, context);
-        }
-
-        template<typename KeyType>
-        static void key(binary_writer& w, const KeyType& k, const BasicJsonType& object)
-        {
-            w.write_msgpack_key(k, object);
-        }
-    };
-
     /*!
-    @brief write @a j with @a Walk::scalar, or write its header and push a
-           frame for @ref write_container_iterative to continue with its
-           elements
+    @brief write @a j with @ref write_cbor, or write its header and push a
+           frame for @ref write_cbor_iterative to continue with its elements
 
     A scalar, and an empty array or object, are written out in full: there is
-    nothing below them for @ref write_container_iterative to come back to, so
+    nothing below them for @ref write_cbor_iterative to come back to, so
     nothing is pushed for them.
     */
-    template<typename Walk>
-    JSON_HEDLEY_ALWAYS_INLINE
-    void write_container_value_or_push(const BasicJsonType& j, std::vector<binary_container_frame>& stack)
+    void write_cbor_value_or_push(const BasicJsonType& j, std::vector<binary_container_frame>& stack)
     {
         if (j.is_array())
         {
-            Walk::array_head(*this, j);
+            write_cbor_head(0x80, j.m_data.m_value.array->size());
             if (!j.m_data.m_value.array->empty())
             {
                 stack.emplace_back(&j);
@@ -1093,7 +1032,7 @@ class binary_writer
 
         if (j.is_object())
         {
-            Walk::object_head(*this, j);
+            write_cbor_head(0xA0, j.m_data.m_value.object->size());
             if (!j.m_data.m_value.object->empty())
             {
                 stack.emplace_back(&j);
@@ -1101,26 +1040,22 @@ class binary_writer
             return;
         }
 
-        Walk::scalar(*this, j);
+        write_cbor(j);
     }
 
     /*!
     @brief write out @a root and everything below it without the call stack
 
-    Emits the same bytes as the recursive writer, keeping the containers it has
+    Emits the same bytes as @ref write_cbor, keeping the containers it has
     entered on an explicit stack instead of descending into them. Only reached
     for values nested deeper than @ref recursion_depth_limit, which is why it
-    is not written for speed. Shared by CBOR and MessagePack, which differ only
-    in the @a Walk policy (@ref cbor_walk, @ref msgpack_walk); BON8 has its own
-    walker, as it tracks an open string run and closes a container with a
-    marker.
+    is not written for speed.
     */
-    template<typename Walk>
-    void write_container_iterative(const BasicJsonType& root)
+    void write_cbor_iterative(const BasicJsonType& root)
     {
-        // only a container with elements is ever pushed; see write_container_value_or_push
+        // only a container with elements is ever pushed; see write_cbor_value_or_push
         std::vector<binary_container_frame> stack;
-        write_container_value_or_push<Walk>(root, stack);
+        write_cbor_value_or_push(root, stack);
 
         while (!stack.empty())
         {
@@ -1138,7 +1073,7 @@ class binary_writer
                 // read the child before pushing: entering it can move every frame
                 const BasicJsonType* child = &(*current.array_it);
                 ++stack.back().array_it;
-                write_container_value_or_push<Walk>(*child, stack);
+                write_cbor_value_or_push(*child, stack);
             }
             else
             {
@@ -1150,53 +1085,110 @@ class binary_writer
                 }
 
                 // the key is written directly (not via a temporary basic_json),
-                // with the object as diagnostics context, as in the recursive writer
-                Walk::key(*this, current.object_it->first, *current.value);
+                // with the object as diagnostics context, as in write_cbor
+                write_cbor_key(current.object_it->first, *current.value);
                 const BasicJsonType* child = &(current.object_it->second);
                 ++stack.back().object_it;
-                write_container_value_or_push<Walk>(*child, stack);
+                write_cbor_value_or_push(*child, stack);
             }
         }
     }
 
-    void write_cbor_iterative(const BasicJsonType& root)
-    {
-        write_container_iterative<cbor_walk>(root);
-    }
+    /*!
+    @brief write @a j with @ref write_msgpack, or write its header and push a
+           frame for @ref write_msgpack_iterative to continue with its elements
 
-    void write_msgpack_iterative(const BasicJsonType& root)
+    @sa @ref write_cbor_value_or_push
+    */
+    void write_msgpack_value_or_push(const BasicJsonType& j, std::vector<binary_container_frame>& stack)
     {
-        write_container_iterative<msgpack_walk>(root);
+        if (j.is_array())
+        {
+            write_msgpack_array_prefix(j.m_data.m_value.array->size(), j);
+            if (!j.m_data.m_value.array->empty())
+            {
+                stack.emplace_back(&j);
+            }
+            return;
+        }
+
+        if (j.is_object())
+        {
+            write_msgpack_object_prefix(j.m_data.m_value.object->size(), j);
+            if (!j.m_data.m_value.object->empty())
+            {
+                stack.emplace_back(&j);
+            }
+            return;
+        }
+
+        write_msgpack(j);
     }
 
     /*!
-    @brief write the start of an array (@a IsArray) or object: its opening
-           marker, the optimized type, and the count
+    @brief write out @a root and everything below it without the call stack
 
-    @return true when a closing ']' or '}' still has to be written after the
-            elements
+    @sa @ref write_cbor_iterative
     */
-    template<bool IsArray>
-    bool write_ubjson_start_container(const BasicJsonType& j, const bool use_count, const bool use_type,
-                                      const bool add_prefix, const bool use_bjdata, bool& prefix_required)
+    void write_msgpack_iterative(const BasicJsonType& root)
+    {
+        std::vector<binary_container_frame> stack;
+        write_msgpack_value_or_push(root, stack);
+
+        while (!stack.empty())
+        {
+            const binary_container_frame current = stack.back();
+
+            if (current.value->is_array())
+            {
+                const auto& array = *current.value->m_data.m_value.array;
+                if (current.array_it == array.cend())
+                {
+                    stack.pop_back();
+                    continue;
+                }
+
+                const BasicJsonType* child = &(*current.array_it);
+                ++stack.back().array_it;
+                write_msgpack_value_or_push(*child, stack);
+            }
+            else
+            {
+                const auto& object = *current.value->m_data.m_value.object;
+                if (current.object_it == object.cend())
+                {
+                    stack.pop_back();
+                    continue;
+                }
+
+                // as in write_cbor_iterative, the key is written directly with
+                // the object as diagnostics context
+                write_msgpack_key(current.object_it->first, *current.value);
+                const BasicJsonType* child = &(current.object_it->second);
+                ++stack.back().object_it;
+                write_msgpack_value_or_push(*child, stack);
+            }
+        }
+    }
+
+    /// @return true when a closing ']' still has to be written after the elements
+    bool write_ubjson_start_array(const BasicJsonType& j, const bool use_count, const bool use_type,
+                                  const bool add_prefix, const bool use_bjdata, bool& prefix_required)
     {
         prefix_required = true;
         if (add_prefix)
         {
-            oa.write_character(to_char_type(IsArray ? '[' : '{'));
+            oa.write_character(to_char_type('['));
         }
 
-        const std::size_t size = IsArray ? j.m_data.m_value.array->size() : j.m_data.m_value.object->size();
-
-        if (use_type && size != 0)
+        if (use_type && !j.m_data.m_value.array->empty())
         {
             if (!use_count)
             {
                 throw_size_marker_required(j);
             }
             const CharType first_prefix = ubjson_prefix(j.front(), use_bjdata);
-            // the first element of an array has been compared with itself
-            const bool same_prefix = std::all_of(IsArray ? j.begin() + 1 : j.begin(), j.end(),
+            const bool same_prefix = std::all_of(j.begin() + 1, j.end(),
                                                  [this, first_prefix, use_bjdata](const BasicJsonType & v)
             {
                 return ubjson_prefix(v, use_bjdata) == first_prefix;
@@ -1208,9 +1200,8 @@ class binary_writer
             // those, at one byte per element, so the result can be read back.
             // Objects are not affected: every element is preceded by its key.
             const bool valueless_type = (first_prefix == 'Z' || first_prefix == 'T' || first_prefix == 'F');
-            const bool excessive_valueless = IsArray
-                                             && valueless_type
-                                             && size > detail::max_valueless_container_size;
+            const bool excessive_valueless = valueless_type
+                                             && j.m_data.m_value.array->size() > detail::max_valueless_container_size;
 
             if (same_prefix && !excessive_valueless
                     && !(use_bjdata && is_bjdata_excluded_type_marker(first_prefix)))
@@ -1223,7 +1214,46 @@ class binary_writer
 
         if (use_count)
         {
-            write_ubjson_count(size, use_bjdata);
+            write_ubjson_count(j.m_data.m_value.array->size(), use_bjdata);
+        }
+
+        return !use_count;
+    }
+
+    /// @return true when a closing '}' still has to be written after the elements
+    bool write_ubjson_start_object(const BasicJsonType& j, const bool use_count, const bool use_type,
+                                   const bool add_prefix, const bool use_bjdata, bool& prefix_required)
+    {
+        prefix_required = true;
+        if (add_prefix)
+        {
+            oa.write_character(to_char_type('{'));
+        }
+
+        if (use_type && !j.m_data.m_value.object->empty())
+        {
+            if (!use_count)
+            {
+                throw_size_marker_required(j);
+            }
+            const CharType first_prefix = ubjson_prefix(j.front(), use_bjdata);
+            const bool same_prefix = std::all_of(j.begin(), j.end(),
+                                                 [this, first_prefix, use_bjdata](const BasicJsonType & v)
+            {
+                return ubjson_prefix(v, use_bjdata) == first_prefix;
+            });
+
+            if (same_prefix && !(use_bjdata && is_bjdata_excluded_type_marker(first_prefix)))
+            {
+                prefix_required = false;
+                oa.write_character(to_char_type('$'));
+                oa.write_character(first_prefix);
+            }
+        }
+
+        if (use_count)
+        {
+            write_ubjson_count(j.m_data.m_value.object->size(), use_bjdata);
         }
 
         return !use_count;
@@ -1251,12 +1281,21 @@ class binary_writer
 
     /// @brief an object or array @ref write_ubjson_iterative is still writing
     ///        the elements of
-    struct ubjson_frame : binary_container_frame
+    struct ubjson_frame
     {
         ubjson_frame(const BasicJsonType* value_, const bool prefix_required_) noexcept
-            : binary_container_frame(value_)
+            : value(value_)
             , prefix_required(prefix_required_)
-        {}
+        {
+            if (value->is_object())
+            {
+                object_it = value->m_data.m_value.object->cbegin();
+            }
+            else
+            {
+                array_it = value->m_data.m_value.array->cbegin();
+            }
+        }
 
         // declared for GCC's -Weffc++, which asks for them in a class with
         // pointer members and a non-trivial destructor; the exception
@@ -1268,9 +1307,13 @@ class binary_writer
         ubjson_frame& operator=(ubjson_frame&&) = default;
         ~ubjson_frame() = default;
 
+        /// the array or object being written
+        const BasicJsonType* value;
         /// whether value's elements each carry their own type marker; an
         /// optimized ($type) container writes it once for all of them instead
         bool prefix_required;
+        typename BasicJsonType::object_t::const_iterator object_it{};
+        typename BasicJsonType::array_t::const_iterator array_it{};
     };
 
     /*!
@@ -1281,7 +1324,7 @@ class binary_writer
                elements of an optimized container, and everything below the
                top level, never repeat it)
 
-    @sa @ref write_container_value_or_push
+    @sa @ref write_cbor_value_or_push
     */
     void write_ubjson_value_or_push(const BasicJsonType& j, const bool add_prefix, const bool use_count,
                                     const bool use_type, const bool use_bjdata, const bjdata_version_t bjdata_version,
@@ -1304,11 +1347,11 @@ class binary_writer
         bool prefix_required = true;
         if (is_array)
         {
-            write_ubjson_start_container<true>(j, use_count, use_type, add_prefix, use_bjdata, prefix_required);
+            write_ubjson_start_array(j, use_count, use_type, add_prefix, use_bjdata, prefix_required);
         }
         else
         {
-            write_ubjson_start_container<false>(j, use_count, use_type, add_prefix, use_bjdata, prefix_required);
+            write_ubjson_start_object(j, use_count, use_type, add_prefix, use_bjdata, prefix_required);
         }
 
         const bool empty = is_array ? j.m_data.m_value.array->empty() : j.m_data.m_value.object->empty();
@@ -1318,7 +1361,7 @@ class binary_writer
             return;
         }
 
-        // write_ubjson_start_container returns !use_count, i.e. whether a
+        // write_ubjson_start_array/_object return !use_count, i.e. whether a
         // closer still has to be written; use_count is constant for the whole
         // document, so that is recomputed here instead of being carried along
         if (!use_count)
@@ -1507,15 +1550,14 @@ class binary_writer
     {
         oa.write_character(to_char_type(element_type));
 
-        if (JSON_HEDLEY_LIKELY(error_handler == error_handler_t::keep || error_handler == error_handler_t::strict))
+        if (error_handler == error_handler_t::keep || error_handler == error_handler_t::strict || is_valid_utf8(name))
         {
-            write_bytes(name.data(), name.size());
+            oa.write_characters(reinterpret_cast<const CharType*>(name.data()), name.size());
         }
         else
         {
-            string_t storage;
-            const string_t& written = sanitize_utf8_for_bson_write(name, storage);
-            write_bytes(written.data(), written.size());
+            const string_t sanitized = sanitize_utf8(name, error_handler);
+            oa.write_characters(reinterpret_cast<const CharType*>(sanitized.data()), sanitized.size());
         }
 
         // the terminating null byte is written explicitly rather than taken
@@ -1591,11 +1633,16 @@ class binary_writer
     {
         write_bson_entry_header(name, 0x02);
 
-        string_t storage;
-        const string_t& written = sanitize_utf8_for_bson_write(value, storage);
+        const bool sanitize = error_handler != error_handler_t::keep
+                              && error_handler != error_handler_t::strict
+                              && !is_valid_utf8(value);
+        const string_t sanitized = sanitize ? sanitize_utf8(value, error_handler) : string_t{};
+        const string_t& written = sanitize ? sanitized : value;
 
         write_number<std::int32_t>(to_bson_length(written.size() + 1ul), true);
-        write_bytes(written.data(), written.size());
+        oa.write_characters(
+              reinterpret_cast<const CharType*>(written.data()),
+              written.size());
         // the terminating null byte is written explicitly rather than taken
         // from the buffer, so that string_t::data() need not be null-terminated
         oa.write_character(to_char_type(0x00));
@@ -1701,7 +1748,7 @@ class binary_writer
 
         write_number(value.has_subtype() ? static_cast<std::uint8_t>(value.subtype()) : static_cast<std::uint8_t>(0x00));
 
-        write_bytes(value.data(), value.size());
+        oa.write_characters(reinterpret_cast<const CharType*>(value.data()), value.size());
     }
 
     /*!
@@ -1731,14 +1778,14 @@ class binary_writer
             case value_t::number_integer:
                 if (JSON_HEDLEY_UNLIKELY(!integer_fits<std::int64_t>(j.m_data.m_value.number_integer)))
                 {
-                    throw_integer_out_of_range(j, input_format_t::bson, "int64");
+                    throw_integer_out_of_range(j, "BSON", "int64");
                 }
                 return calc_bson_integer_size(static_cast<std::int64_t>(j.m_data.m_value.number_integer));
 
             case value_t::number_unsigned:
                 if (JSON_HEDLEY_UNLIKELY(!integer_fits_max<std::uint64_t>(j.m_data.m_value.number_unsigned)))
                 {
-                    throw_integer_out_of_range(j, input_format_t::bson, "uint64");
+                    throw_integer_out_of_range(j, "BSON", "uint64");
                 }
                 return calc_bson_unsigned_size(conditional_static_cast<std::uint64_t>(j.m_data.m_value.number_unsigned));
 
@@ -1749,7 +1796,7 @@ class binary_writer
                 return 0ul;
 
             case value_t::discarded:
-                throw_on_discarded(j, input_format_t::bson);
+                throw_on_discarded(j, "BSON");
 
             // LCOV_EXCL_START
             case value_t::object:
@@ -1913,7 +1960,7 @@ class binary_writer
             // the terminating null byte is written explicitly rather than
             // taken from the buffer, so that string_t::data() need not be
             // null-terminated
-            write_bytes(value.data(), value.size());
+            oa.write_characters(reinterpret_cast<const CharType*>(value.data()), value.size());
             oa.write_character(to_char_type(0x00));
             return true;
         }
@@ -1930,7 +1977,7 @@ class binary_writer
             p += sizeof(std::int32_t);
             *p++ = value.has_subtype() ? to_char_type(static_cast<std::uint8_t>(value.subtype())) : to_char_type(0x00);
             oa.commit(static_cast<std::size_t>(p - out));
-            write_bytes(value.data(), value.size());
+            oa.write_characters(reinterpret_cast<const CharType*>(value.data()), value.size());
             return true;
         }
 
@@ -1978,7 +2025,7 @@ class binary_writer
                 return write_bson_null(name);
 
             case value_t::discarded:
-                throw_on_discarded(j, input_format_t::bson);
+                throw_on_discarded(j, "BSON");
 
             // LCOV_EXCL_START
             case value_t::object:
@@ -2759,7 +2806,7 @@ class binary_writer
         {
             if (JSON_HEDLEY_UNLIKELY(!integer_fits_max<std::uint64_t>(j.m_data.m_value.number_unsigned)))
             {
-                throw_integer_out_of_range(j, input_format_t::cbor, "uint64");
+                throw_integer_out_of_range(j, "CBOR", "uint64");
             }
             write_cbor_head(0x00, conditional_static_cast<std::uint64_t>(j.m_data.m_value.number_unsigned));
             return;
@@ -2769,7 +2816,7 @@ class binary_writer
         {
             if (JSON_HEDLEY_UNLIKELY(!integer_fits_max<std::uint64_t>(j.m_data.m_value.number_integer)))
             {
-                throw_integer_out_of_range(j, input_format_t::cbor, "[-2^64, 2^64-1]");
+                throw_integer_out_of_range(j, "CBOR", "[-2^64, 2^64-1]");
             }
             // CBOR does not differentiate between positive signed
             // integers and unsigned integers
@@ -2780,7 +2827,7 @@ class binary_writer
             // a negative integer n is encoded as -1 - n
             if (JSON_HEDLEY_UNLIKELY(!integer_fits_max<std::uint64_t>(-1 - j.m_data.m_value.number_integer)))
             {
-                throw_integer_out_of_range(j, input_format_t::cbor, "[-2^64, 2^64-1]");
+                throw_integer_out_of_range(j, "CBOR", "[-2^64, 2^64-1]");
             }
             write_cbor_head(0x20, static_cast<std::uint64_t>(-1 - j.m_data.m_value.number_integer));
         }
@@ -2856,7 +2903,7 @@ class binary_writer
         {
             if (JSON_HEDLEY_UNLIKELY(!integer_fits_max<std::uint64_t>(j.m_data.m_value.number_unsigned)))
             {
-                throw_integer_out_of_range(j, input_format_t::msgpack, "uint64");
+                throw_integer_out_of_range(j, "MessagePack", "uint64");
             }
             write_msgpack_unsigned(static_cast<std::uint64_t>(j.m_data.m_value.number_unsigned));
             return;
@@ -2866,7 +2913,7 @@ class binary_writer
         {
             if (JSON_HEDLEY_UNLIKELY(!integer_fits_max<std::uint64_t>(j.m_data.m_value.number_integer)))
             {
-                throw_integer_out_of_range(j, input_format_t::msgpack, "[-2^63, 2^64-1]");
+                throw_integer_out_of_range(j, "MessagePack", "[-2^63, 2^64-1]");
             }
             // MessagePack does not differentiate between positive
             // signed integers and unsigned integers.
@@ -2876,7 +2923,7 @@ class binary_writer
         {
             if (JSON_HEDLEY_UNLIKELY(!integer_fits_min<std::int64_t>(j.m_data.m_value.number_integer)))
             {
-                throw_integer_out_of_range(j, input_format_t::msgpack, "[-2^63, 2^64-1]");
+                throw_integer_out_of_range(j, "MessagePack", "[-2^63, 2^64-1]");
             }
             if (j.m_data.m_value.number_integer >= -32)
             {
@@ -3010,37 +3057,40 @@ class binary_writer
     }
 
     /*!
-    @brief write an object key as a CBOR or MessagePack string (@a Walk)
+    @brief write an object key as a CBOR text string
 
     A key convertible to string_t is written directly. Other key types (only
     an explicit conversion, or only a to_json overload) go through a temporary
     basic_json, as in version 3.12.0; the temporary is then the diagnostics
     context for strict UTF-8 checks.
     */
-    template<typename Walk, typename Key = typename BasicJsonType::object_t::key_type,
+    template<typename Key = typename BasicJsonType::object_t::key_type,
              enable_if_t<std::is_convertible<Key, string_t>::value, int> = 0>
-    void write_object_key(const typename BasicJsonType::object_t::key_type& key, const BasicJsonType& context)
-    {
-        Walk::string(*this, key, context);
-    }
-
-    template<typename Walk, typename Key = typename BasicJsonType::object_t::key_type,
-             enable_if_t<negation<std::is_convertible<Key, string_t>>::value, int> = 0>
-    void write_object_key(const typename BasicJsonType::object_t::key_type& key, const BasicJsonType& /*context*/)
-    {
-        Walk::scalar(*this, BasicJsonType(key));
-    }
-
-    /// @brief write an object key as a CBOR text string
     void write_cbor_key(const typename BasicJsonType::object_t::key_type& key, const BasicJsonType& context)
     {
-        write_object_key<cbor_walk>(key, context);
+        write_cbor_string(key, context);
     }
 
-    /// @brief write an object key as a MessagePack str
+    template < typename Key = typename BasicJsonType::object_t::key_type,
+               enable_if_t < !std::is_convertible<Key, string_t>::value, int > = 0 >
+    void write_cbor_key(const typename BasicJsonType::object_t::key_type& key, const BasicJsonType& /*context*/)
+    {
+        write_cbor(BasicJsonType(key));
+    }
+
+    /// @brief write an object key as a MessagePack str, as in @ref write_cbor_key
+    template<typename Key = typename BasicJsonType::object_t::key_type,
+             enable_if_t<std::is_convertible<Key, string_t>::value, int> = 0>
     void write_msgpack_key(const typename BasicJsonType::object_t::key_type& key, const BasicJsonType& context)
     {
-        write_object_key<msgpack_walk>(key, context);
+        write_msgpack_string(key, context);
+    }
+
+    template < typename Key = typename BasicJsonType::object_t::key_type,
+               enable_if_t < !std::is_convertible<Key, string_t>::value, int > = 0 >
+    void write_msgpack_key(const typename BasicJsonType::object_t::key_type& key, const BasicJsonType& /*context*/)
+    {
+        write_msgpack(BasicJsonType(key));
     }
 
     /// the longest string that is written, with its head of one or two
@@ -3115,7 +3165,9 @@ class binary_writer
         write_cbor_head(0x60, sanitized.size());
 
         // step 2: write the string
-        write_bytes(sanitized.data(), sanitized.size());
+        oa.write_characters(
+              reinterpret_cast<const CharType*>(sanitized.data()),
+              sanitized.size());
     }
 
     /////////////
@@ -3196,7 +3248,9 @@ class binary_writer
         }
 
         // step 2: write the string
-        write_bytes(sanitized.data(), sanitized.size());
+        oa.write_characters(
+              reinterpret_cast<const CharType*>(sanitized.data()),
+              sanitized.size());
     }
 
     ////////////
@@ -3479,7 +3533,9 @@ class binary_writer
             oa.write_character(to_char_type('S'));
         }
         write_number_with_ubjson_prefix(sanitized.size(), true, use_bjdata);
-        write_bytes(sanitized.data(), sanitized.size());
+        oa.write_characters(
+              reinterpret_cast<const CharType*>(sanitized.data()),
+              sanitized.size());
     }
 
     /*!
@@ -4279,7 +4335,7 @@ class binary_writer
             {
                 if (JSON_HEDLEY_UNLIKELY(!integer_fits_max<std::int64_t>(j.m_data.m_value.number_unsigned)))
                 {
-                    throw_integer_out_of_range(j, input_format_t::bon8, "int64");
+                    throw_integer_out_of_range(j, "BON8", "int64");
                 }
                 write_bon8_integer(static_cast<std::int64_t>(j.m_data.m_value.number_unsigned));
                 string_open = false;
@@ -4290,7 +4346,7 @@ class binary_writer
             {
                 if (JSON_HEDLEY_UNLIKELY(!integer_fits<std::int64_t>(j.m_data.m_value.number_integer)))
                 {
-                    throw_integer_out_of_range(j, input_format_t::bon8, "int64");
+                    throw_integer_out_of_range(j, "BON8", "int64");
                 }
                 write_bon8_integer(static_cast<std::int64_t>(j.m_data.m_value.number_integer));
                 string_open = false;
@@ -4358,7 +4414,7 @@ class binary_writer
 
             case value_t::discarded:
             default:
-                throw_on_discarded(j, input_format_t::bon8);
+                throw_on_discarded(j, "BON8");
         }
     }
 
@@ -4369,7 +4425,7 @@ class binary_writer
     A scalar, and an empty array or object, are written out in full and not
     pushed.
 
-    @sa @ref write_container_value_or_push
+    @sa @ref write_cbor_value_or_push
     */
     void write_bon8_value_or_push(const BasicJsonType& j, bool& string_open, std::vector<binary_container_frame>& stack)
     {
@@ -4755,7 +4811,7 @@ class binary_writer
         }
         else
         {
-            write_bytes(s.data(), s.size());
+            oa.write_characters(reinterpret_cast<const CharType*>(s.data()), s.size());
             string_open = true;
         }
     }
@@ -4846,31 +4902,6 @@ class binary_writer
         }
     }
 
-    /*!
-    @brief the write-pass counterpart of @ref sanitize_utf8_for_write for BSON
-
-    The size pass (@ref calc_bson_entry_header_size, @ref
-    calc_bson_string_size) has already validated @a s, and thrown for @ref
-    error_handler_t::strict if it was ill-formed. Here, @ref
-    error_handler_t::strict therefore behaves like @ref error_handler_t::keep,
-    and @ref error_handler_t::replace / @ref error_handler_t::ignore produce
-    exactly the bytes the size pass measured. Hence no context is needed.
-
-    @param[in] s        the string (value or object key) to write
-    @param[out] storage  backing storage for a sanitized copy
-
-    @return a reference to @a s, or to @a storage once it holds a sanitized copy
-    */
-    const string_t& sanitize_utf8_for_bson_write(const string_t& s, string_t& storage) const
-    {
-        if (error_handler == error_handler_t::keep || error_handler == error_handler_t::strict || is_valid_utf8(s))
-        {
-            return s; // NOLINT(bugprone-return-const-ref-from-parameter): callers pass lvalues that outlive the call
-        }
-        storage = sanitize_utf8(s, error_handler);
-        return storage;
-    }
-
     /// deleted: anything but a string_t would bind a temporary that dies before the returned reference is used
     template < typename T, enable_if_t < !std::is_same<T, string_t>::value, int > = 0 >
     const string_t& sanitize_utf8_for_write(const T& /*s*/, const BasicJsonType& /*context*/, string_t& /*storage*/) const = delete; // NOLINT(hicpp-use-equals-delete,modernize-use-equals-delete): a private helper's guard, not part of the interface

include/nlohmann/detail/output/serializer.hpp

@@ -966,28 +966,6 @@ class serializer
         }
     }
 
-    /// write out the bytes pending in @ref string_buffer and reset @a bytes
-    JSON_HEDLEY_ALWAYS_INLINE void emit_pending(std::size_t& bytes)
-    {
-        if (bytes != 0)
-        {
-            put_buffer(string_buffer, bytes);
-            bytes = 0;
-        }
-    }
-
-    /// write buffer and reset index if fewer than 13 bytes are left, as this
-    /// is the maximal number of bytes to be written ("\uxxxx\uxxxx\0") for one
-    /// code point
-    JSON_HEDLEY_ALWAYS_INLINE void reserve_string_buffer(std::size_t& bytes)
-    {
-        if (string_buffer.size() - bytes < 13)
-        {
-            put_buffer(string_buffer, bytes);
-            bytes = 0;
-        }
-    }
-
     /*!
     @brief worker for @ref dump_escaped
 
@@ -1046,7 +1024,11 @@ class serializer
                 {
                     // emit any bytes still pending in string_buffer first to
                     // preserve output order, then write the run directly
-                    emit_pending(bytes);
+                    if (bytes != 0)
+                    {
+                        put_buffer(string_buffer, bytes);
+                        bytes = 0;
+                    }
                     put_run(s, i, run);
                     bytes_after_last_accept = 0;
                     undumped_chars = 0;
@@ -1067,10 +1049,11 @@ class serializer
                 {
                     // emit any bytes still pending in string_buffer first to
                     // preserve output order
-                    emit_pending(bytes);
-                    // deliberately not put_buffer(): a fixed 8-byte memcpy
-                    // has no asserts and no write-through branch, which
-                    // matters on the hot path
+                    if (bytes != 0)
+                    {
+                        put_buffer(string_buffer, bytes);
+                        bytes = 0;
+                    }
                     const auto& escape = ascii_escapes()[data[i]];
                     if (JSON_HEDLEY_UNLIKELY(write_buffer.size() - write_buffer_pos < escape.size()))
                     {
@@ -1145,7 +1128,14 @@ class serializer
                         string_buffer[bytes++] = s[i];
                     }
 
-                    reserve_string_buffer(bytes);
+                    // write buffer and reset index; there must be 13 bytes
+                    // left, as this is the maximal number of bytes to be
+                    // written ("\uxxxx\uxxxx\0") for one code point
+                    if (string_buffer.size() - bytes < 13)
+                    {
+                        put_buffer(string_buffer, bytes);
+                        bytes = 0;
+                    }
 
                     // remember the byte position of this accept
                     bytes_after_last_accept = bytes;
@@ -1197,7 +1187,14 @@ class serializer
                                     string_buffer[bytes++] = '\xBD';
                                 }
 
-                                reserve_string_buffer(bytes);
+                                // write buffer and reset index; there must be 13 bytes
+                                // left, as this is the maximal number of bytes to be
+                                // written ("\uxxxx\uxxxx\0") for one code point
+                                if (string_buffer.size() - bytes < 13)
+                                {
+                                    put_buffer(string_buffer, bytes);
+                                    bytes = 0;
+                                }
 
                                 bytes_after_last_accept = bytes;
                             }
@@ -1230,7 +1227,14 @@ class serializer
                                 string_buffer[bytes++] = s[i];
                             }
 
-                            reserve_string_buffer(bytes);
+                            // write buffer and reset index; there must be 13 bytes
+                            // left, as this is the maximal number of bytes to be
+                            // written ("\uxxxx\uxxxx\0") for one code point
+                            if (string_buffer.size() - bytes < 13)
+                            {
+                                put_buffer(string_buffer, bytes);
+                                bytes = 0;
+                            }
 
                             bytes_after_last_accept = bytes;
                             undumped_chars = 0;
@@ -1596,17 +1600,30 @@ class serializer
     */
     static const std::array<std::uint8_t, 256>& ascii_copyable() noexcept
     {
-        // constant-initialized from the predicate: no runtime loop, no guard
-        static constexpr std::array<std::uint8_t, 256> copyable = make_ascii_copyable(make_index_sequence<256> {});
+        static const std::array<std::uint8_t, 256> copyable =
+        {
+            {
+                0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // 0x00
+                0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // 0x10
+                1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, // 0x20
+                1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, // 0x30
+                1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, // 0x40
+                1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, // 0x50
+                1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, // 0x60
+                1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, // 0x70
+                0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // 0x80
+                0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // 0x90
+                0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // 0xA0
+                0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // 0xB0
+                0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // 0xC0
+                0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // 0xD0
+                0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // 0xE0
+                0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 // 0xF0
+            }
+        };
         return copyable;
     }
 
-    template<std::size_t... Is>
-    static constexpr std::array<std::uint8_t, 256> make_ascii_copyable(index_sequence<Is...> /*unused*/) noexcept
-    {
-        return {{ static_cast<std::uint8_t>(is_ascii_copyable(static_cast<unsigned char>(Is)) ? 1 : 0)... }};
-    }
-
     /*!
     @brief the JSON form of each character below 0x60
 
@@ -1735,6 +1752,7 @@ class serializer
     */
     JSON_HEDLEY_NEVER_INLINE std::size_t write_u_escapes(const unsigned char* data, std::size_t n)
     {
+        constexpr const char* nibble_to_hex = "0123456789abcdef";
         std::size_t i = 0;
         while (i < n && data[i] >= 0x80)
         {
@@ -1771,21 +1789,26 @@ class serializer
             char* d = write_buffer.data() + write_buffer_pos;
             if (codepoint > 0xFFFFu)
             {
-                d = put_u_escape(d, static_cast<std::uint16_t>(0xD7C0u + (codepoint >> 10u)));
+                d = put_u_escape(d, static_cast<std::uint16_t>(0xD7C0u + (codepoint >> 10u)), nibble_to_hex);
                 codepoint = 0xDC00u + (codepoint & 0x3FFu);
             }
-            d = put_u_escape(d, static_cast<std::uint16_t>(codepoint));
+            d = put_u_escape(d, static_cast<std::uint16_t>(codepoint), nibble_to_hex);
             write_buffer_pos = static_cast<std::size_t>(d - write_buffer.data());
             i += length;
         }
         return i;
     }
 
-    static char* put_u_escape(char* d, const std::uint16_t codeunit) noexcept
+    static char* put_u_escape(char* d, const std::uint16_t codeunit, const char* nibble_to_hex) noexcept
     {
+        const auto unit = static_cast<std::uint32_t>(codeunit);
         d[0] = '\\';
         d[1] = 'u';
-        return hex4_lower(codeunit, d + 2);
+        d[2] = nibble_to_hex[(unit >> 12u) & 0x0Fu];
+        d[3] = nibble_to_hex[(unit >> 8u) & 0x0Fu];
+        d[4] = nibble_to_hex[(unit >> 4u) & 0x0Fu];
+        d[5] = nibble_to_hex[unit & 0x0Fu];
+        return d + 6;
     }
 
     /*!
@@ -1794,7 +1817,7 @@ class serializer
      * Branch-free replacement for `snprintf(buf, 7, "\\u%04x", codeunit)` in the
      * string escaping hot path. It writes exactly six characters ('\\', 'u' and
      * four hex digits) at position @a pos of @a string_buffer via a nibble
-     * lookup table (@ref hex4_lower), avoiding the format-string parsing and locale machinery of
+     * lookup table, avoiding the format-string parsing and locale machinery of
      * `snprintf`. Advances @a pos by the number of bytes written (6).
      *
      * @param[in] pos       position in @a string_buffer to write at; there must
@@ -1804,9 +1827,13 @@ class serializer
     void write_u_escape(std::size_t& pos, std::uint16_t codeunit) noexcept
     {
         JSON_ASSERT(string_buffer.size() - pos >= 6);
+        constexpr const char* nibble_to_hex = "0123456789abcdef";
         string_buffer[pos + 0] = '\\';
         string_buffer[pos + 1] = 'u';
-        hex4_lower(codeunit, &string_buffer[pos + 2]);
+        string_buffer[pos + 2] = nibble_to_hex[(codeunit >> 12u) & 0x0Fu];
+        string_buffer[pos + 3] = nibble_to_hex[(codeunit >> 8u) & 0x0Fu];
+        string_buffer[pos + 4] = nibble_to_hex[(codeunit >> 4u) & 0x0Fu];
+        string_buffer[pos + 5] = nibble_to_hex[codeunit & 0x0Fu];
         pos += 6;
     }
 

include/nlohmann/detail/string_utils.hpp

@@ -13,6 +13,7 @@
 #include <cstddef> // size_t
 #include <cstdint> // uint8_t, uint32_t
 #include <string> // string, to_string
+#include <utility> // move
 
 #include <nlohmann/detail/abi_macros.hpp>
 #include <nlohmann/detail/macro_scope.hpp>
@@ -48,24 +49,6 @@ inline std::string hex_byte(const std::uint8_t byte)
     return result;
 }
 
-/*!
-@brief write a UTF-16 code unit as four lowercase hexadecimal digits
-
-The lowercase counterpart of hex_byte(), used for the `\\uXXXX` escapes of the
-serializers, whose output is lowercase. Writes exactly four characters to
-@a out and returns the pointer past them.
-*/
-JSON_HEDLEY_ALWAYS_INLINE char* hex4_lower(const std::uint16_t codeunit, char* out) noexcept
-{
-    constexpr const char* nibble_to_hex = "0123456789abcdef";
-    const unsigned int unit = codeunit;
-    out[0] = nibble_to_hex[(unit >> 12u) & 0x0Fu];
-    out[1] = nibble_to_hex[(unit >> 8u) & 0x0Fu];
-    out[2] = nibble_to_hex[(unit >> 4u) & 0x0Fu];
-    out[3] = nibble_to_hex[unit & 0x0Fu];
-    return out + 4;
-}
-
 ///////////////////
 // UTF-8 encoding //
 ///////////////////
@@ -121,66 +104,6 @@ void encode_utf8(std::uint32_t cp, const Out& out)
     }
 }
 
-/*!
-@brief the shape of a well-formed UTF-8 sequence, by its lead byte
-
-@a length is 1 for ASCII, 2 to 4 for a valid lead byte, and 0 for a byte that
-cannot start a well-formed sequence (a continuation byte, 0xC0, 0xC1, or 0xF5
-to 0xFF). For a multibyte sequence, the second byte must lie in
-[@a second_lo, @a second_hi], and every later byte in [0x80, 0xBF]
-(RFC 3629, section 4; Unicode Table 3-7). The narrower second-byte ranges
-exclude overlong forms (0xE0, 0xF0), surrogates (0xED), and code points
-beyond U+10FFFF (0xF4).
-*/
-struct utf8_sequence_shape
-{
-    std::uint8_t length;
-    std::uint8_t second_lo;
-    std::uint8_t second_hi;
-};
-
-/// @return the shape of a UTF-8 sequence starting with @a lead, see utf8_sequence_shape
-inline utf8_sequence_shape utf8_shape(const std::uint8_t lead) noexcept
-{
-    if (lead < 0x80u)
-    {
-        return {1, 0, 0};
-    }
-    if (lead < 0xC2u)
-    {
-        return {0, 0, 0};
-    }
-    if (lead <= 0xDFu)
-    {
-        return {2, 0x80u, 0xBFu};
-    }
-    if (lead == 0xE0u)
-    {
-        return {3, 0xA0u, 0xBFu};
-    }
-    if (lead == 0xEDu)
-    {
-        return {3, 0x80u, 0x9Fu};
-    }
-    if (lead <= 0xEFu)
-    {
-        return {3, 0x80u, 0xBFu};
-    }
-    if (lead == 0xF0u)
-    {
-        return {4, 0x90u, 0xBFu};
-    }
-    if (lead <= 0xF3u)
-    {
-        return {4, 0x80u, 0xBFu};
-    }
-    if (lead == 0xF4u)
-    {
-        return {4, 0x80u, 0x8Fu};
-    }
-    return {0, 0, 0};
-}
-
 ///////////////////
 // UTF-8 decoding //
 ///////////////////
@@ -196,7 +119,7 @@ This is a single-byte step of a "shift-based" UTF-8 decoder originally
 written by Björn Hoehrmann. See
 http://bjoern.hoehrmann.de/utf-8/decoder/dfa/ for details.
 
-The library checks UTF-8 well-formedness (RFC 3629, section 4) in five
+The library checks UTF-8 well-formedness (RFC 3629, section 4) in three
 places, which differ in speed, diagnostics, and how they read the input:
 
 - decode() below: the serializer, to escape and, in strict mode, reject
@@ -210,15 +133,10 @@ places, which differ in speed, diagnostics, and how they read the input:
   bulk string scan, the bulk path of the BON8 reader, and the BON8 writer.
   They must accept exactly what the lexer's switch accepts.
 - the byte path of binary_reader::get_bon8_string(): BON8 input without bulk
-  access, and the bytes the bulk path leaves to it. It keeps its own ranges:
-  taking them from utf8_shape() made BON8 decoding 2-3% slower (#5891).
-- check_utf8() in detail/view/edit.hpp: strings stored in an editable
-  json_document, with the index of the offending byte. It also uses
-  utf8_shape().
+  access, and the bytes the bulk path leaves to it.
 
-All five must accept the same set of sequences, which the test "UTF-8
-validators accept the same sequences" in unit-class_lexer.cpp checks
-exhaustively against decode() and utf8_shape().
+All four must accept the same set of sequences, so a change to one needs a
+matching change to the others.
 
 @param[in,out] state  the current decoder state
 @param[in,out] codep  codepoint (valid only if resulting state is UTF8_ACCEPT)

include/nlohmann/detail/value_t.hpp

@@ -177,36 +177,6 @@ inline bool owns_memory(const value_t t) noexcept
     }
 }
 
-/*!
-@brief whether a value of the given type is neither an object nor an array
-
-Unlike basic_json::is_primitive(), this includes value_t::discarded: it names
-the types that iterators walk with a primitive_iterator_t (a single value, or no
-value for null and discarded) and that JSON Pointer resolution and the
-serializer treat as leaves. Every enumerator is listed so that -Wswitch-enum
-flags this function when a new type is added.
-*/
-inline bool is_primitive(const value_t t) noexcept
-{
-    switch (t)
-    {
-        case value_t::object:
-        case value_t::array:
-            return false;
-
-        case value_t::null:
-        case value_t::string:
-        case value_t::boolean:
-        case value_t::number_integer:
-        case value_t::number_unsigned:
-        case value_t::number_float:
-        case value_t::binary:
-        case value_t::discarded:
-        default:
-            return true;
-    }
-}
-
 
 /*!
 @brief compare an integer with a floating point number without precision loss

include/nlohmann/detail/view/edit.hpp

@@ -51,6 +51,11 @@ struct first_true<false, Conditions...> : std::integral_constant < int, 1 + firs
 inline void check_utf8(const char* s, std::size_t n)
 {
     const auto* const p = reinterpret_cast<const unsigned char*>(s); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
+    const auto hex = [](unsigned char c)
+    {
+        constexpr const char* digits = "0123456789ABCDEF";
+        return std::string{digits[c >> 4u], digits[c & 0xFu]};
+    };
     for (std::size_t i = 0; i < n;)
     {
         const unsigned char c = p[i];
@@ -59,22 +64,39 @@ inline void check_utf8(const char* s, std::size_t n)
             ++i;
             continue;
         }
-        const utf8_sequence_shape shape = utf8_shape(c);
-        const std::size_t len = shape.length;
-        if (len == 0)
+        std::size_t len = 0;
+        unsigned char lo = 0x80;
+        unsigned char hi = 0xBF;
+        if (c >= 0xC2 && c <= 0xDF)
         {
-            throw_type_error(exception_id::invalid_utf8, concat("invalid UTF-8 byte at index ", std::to_string(i), ": 0x", hex_byte(c)));
+            len = 2;
+        }
+        else if (c >= 0xE0 && c <= 0xEF)
+        {
+            len = 3;
+            lo = c == 0xE0 ? 0xA0 : 0x80;
+            hi = c == 0xED ? 0x9F : 0xBF;
+        }
+        else if (c >= 0xF0 && c <= 0xF4)
+        {
+            len = 4;
+            lo = c == 0xF0 ? 0x90 : 0x80;
+            hi = c == 0xF4 ? 0x8F : 0xBF;
+        }
+        else
+        {
+            throw_type_error(316, concat("invalid UTF-8 byte at index ", std::to_string(i), ": 0x", hex(c)));
         }
         for (std::size_t k = 1; k < len; ++k)
         {
             if (i + k == n)
             {
-                throw_type_error(exception_id::invalid_utf8, concat("incomplete UTF-8 string; last byte: 0x", hex_byte(p[n - 1])));
+                throw_type_error(316, concat("incomplete UTF-8 string; last byte: 0x", hex(p[n - 1])));
             }
             const unsigned char b = p[i + k];
-            if (b < (k == 1 ? shape.second_lo : 0x80) || b > (k == 1 ? shape.second_hi : 0xBF))
+            if (b < (k == 1 ? lo : 0x80) || b > (k == 1 ? hi : 0xBF))
             {
-                throw_type_error(exception_id::invalid_utf8, concat("invalid UTF-8 byte at index ", std::to_string(i + k), ": 0x", hex_byte(b)));
+                throw_type_error(316, concat("invalid UTF-8 byte at index ", std::to_string(i + k), ": 0x", hex(b)));
             }
         }
         i += len;
@@ -122,7 +144,7 @@ class editor
         node* const o = own(object);
         if (o->kind != static_cast<std::uint8_t>(value_t::object) && o->kind != static_cast<std::uint8_t>(value_t::null))
         {
-            throw_type_error(exception_id::subscript_wrong_type, "cannot use operator[] with a string argument with ", object.type_name());
+            throw_type_error(305, "cannot use operator[] with a string argument with ", object.type_name());
         }
         check_utf8(key.data(), key.size());
         const encoded e = encode(std::forward<V>(value));
@@ -173,7 +195,7 @@ class editor
         node* const a = own(array);
         if (a->kind != static_cast<std::uint8_t>(value_t::array))
         {
-            throw_type_error(exception_id::subscript_wrong_type, "cannot use operator[] with a numeric argument with ", array.type_name());
+            throw_type_error(305, "cannot use operator[] with a numeric argument with ", array.type_name());
         }
         check_index(idx, a->len);
         const encoded e = encode(std::forward<V>(value));
@@ -190,7 +212,7 @@ class editor
         node* const a = own(array);
         if (a->kind != static_cast<std::uint8_t>(value_t::array) && a->kind != static_cast<std::uint8_t>(value_t::null))
         {
-            throw_type_error(exception_id::push_back_wrong_type, "cannot use push_back() with ", array.type_name());
+            throw_type_error(308, "cannot use push_back() with ", array.type_name());
         }
         const encoded e = encode(std::forward<V>(value));
         if (a->kind == static_cast<std::uint8_t>(value_t::null))
@@ -214,7 +236,7 @@ class editor
         node* const a = own(array);
         if (a->kind != static_cast<std::uint8_t>(value_t::array))
         {
-            throw_type_error(exception_id::insert_wrong_type, "cannot use insert() with ", array.type_name());
+            throw_type_error(309, "cannot use insert() with ", array.type_name());
         }
         check_index(idx, a->len + 1);
         const encoded e = encode(std::forward<V>(value));
@@ -234,7 +256,7 @@ class editor
         node* const o = own(object);
         if (o->kind != static_cast<std::uint8_t>(value_t::object))
         {
-            throw_type_error(exception_id::erase_wrong_type, "cannot use erase() with ", object.type_name());
+            throw_type_error(307, "cannot use erase() with ", object.type_name());
         }
         for (const node* k = nav::first(m_doc, o), *end = nav::end(m_doc, o); k != end; k = document_data::after(k + 1))
         {
@@ -252,7 +274,7 @@ class editor
         node* const a = own(array);
         if (a->kind != static_cast<std::uint8_t>(value_t::array))
         {
-            throw_type_error(exception_id::erase_wrong_type, "cannot use erase() with ", array.type_name());
+            throw_type_error(307, "cannot use erase() with ", array.type_name());
         }
         check_index(idx, a->len);
         node* const h = block_of(m_doc, a, 0);
@@ -275,7 +297,7 @@ class editor
     {
         if (NLOHMANN_VIEW_UNLIKELY(v.m_doc != &m_doc || v.m_node == nullptr))
         {
-            throw_invalid_iterator(exception_id::iterator_from_other_value, "view does not belong to this document");
+            throw_invalid_iterator(202, "view does not belong to this document");
         }
         edit_state_of(m_doc);
         return const_cast<node*>(v.m_node); // NOLINT(cppcoreguidelines-pro-type-const-cast): the nodes belong to this document
@@ -285,7 +307,7 @@ class editor
     {
         if (idx >= limit)
         {
-            throw_array_index_out_of_range(idx);
+            throw_out_of_range(401, concat("array index ", std::to_string(idx), " is out of range"));
         }
     }
 
@@ -426,7 +448,7 @@ class editor
     {
         if (NLOHMANN_VIEW_UNLIKELY(v.m_node == nullptr))
         {
-            throw_type_error(exception_id::type_mismatch, "type must be a value, but is ", "discarded");
+            throw_type_error(302, "type must be a value, but is ", "discarded");
         }
         encoded r;
         if (!is_container(*v.m_node))
@@ -576,7 +598,7 @@ class editor
     {
         if (NLOHMANN_VIEW_UNLIKELY(len >= 0xFFFFFFFFu))
         {
-            throw_out_of_range(exception_id::document_too_large, "strings of 4 GiB or more are not supported by json_document"); // LCOV_EXCL_LINE
+            throw_out_of_range(416, "strings of 4 GiB or more are not supported by json_document"); // LCOV_EXCL_LINE
         }
         node n = plain_node(value_t::string);
         n.flags = static_cast<std::uint8_t>(n.flags | node_flags::edited);
@@ -651,12 +673,12 @@ class editor
                 return string_node(s.data(), s.size());
             }
             case value_t::binary:
-                throw_type_error(exception_id::document_binary_value, "cannot store a binary value in a json_document", "");
+                throw_type_error(319, "cannot store a binary value in a json_document", "");
             case value_t::discarded:
             case value_t::object:
             case value_t::array:
             default:
-                throw_type_error(exception_id::type_mismatch, "type must be a value, but is ", "discarded");
+                throw_type_error(302, "type must be a value, but is ", "discarded");
         }
     }
 

include/nlohmann/detail/view/edit_storage.hpp

@@ -84,7 +84,7 @@ inline std::size_t text_capacity(std::size_t cap, std::size_t used, std::size_t
     constexpr std::size_t limit = 0xFFFFFFFFu;
     if (NLOHMANN_VIEW_UNLIKELY(used > limit || n > limit - used))
     {
-        throw_out_of_range(exception_id::document_too_large, "edits of 4 GiB or more are not supported by json_document");
+        throw_out_of_range(416, "edits of 4 GiB or more are not supported by json_document");
     }
     const std::size_t needed = used + n;
     const std::size_t wanted = needed + (std::min)(limit - needed, std::size_t{256});
@@ -135,7 +135,7 @@ inline void reserve_moved(document_data& d)
     document_data::edit_state& e = edit_state_of(d);
     if (e.moved.size() >= 0xFFFFFFFFu)
     {
-        throw_out_of_range(exception_id::document_too_large, "more than 4294967295 edited arrays and objects are not supported by json_document"); // LCOV_EXCL_LINE
+        throw_out_of_range(416, "more than 4294967295 edited arrays and objects are not supported by json_document"); // LCOV_EXCL_LINE
     }
     if (e.moved.size() == e.moved.capacity() || e.moved_cap.size() == e.moved_cap.capacity())
     {

include/nlohmann/detail/view/errors.hpp

@@ -25,52 +25,32 @@ namespace view
 // Exceptions are thrown out of line, so that the accessors that may throw stay
 // small enough to be inlined.
 
-[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_type_error(exception_id id, const char* prefix, const char* type)
+[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_type_error(int id, const char* prefix, const char* type)
 {
     NLOHMANN_VIEW_THROW(type_error::create(id, concat(prefix, type), nullptr));
 }
 
-[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_type_error(exception_id id, const std::string& msg)
+[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_type_error(int id, const std::string& msg)
 {
     NLOHMANN_VIEW_THROW(type_error::create(id, msg, nullptr));
 }
 
-[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_out_of_range(exception_id id, const std::string& msg)
+[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_out_of_range(int id, const std::string& msg)
 {
     NLOHMANN_VIEW_THROW(out_of_range::create(id, msg, nullptr));
 }
 
-[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_invalid_iterator(exception_id id, const char* msg)
+[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_invalid_iterator(int id, const char* msg)
 {
     NLOHMANN_VIEW_THROW(invalid_iterator::create(id, msg, nullptr));
 }
 
 /// a parse error without a position (as those of json_pointer)
-[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_parse_error(exception_id id, const std::string& msg)
+[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_parse_error(int id, const std::string& msg)
 {
     NLOHMANN_VIEW_THROW(parse_error::create(id, 0, msg, nullptr));
 }
 
-/// out_of_range.401: the array index @a idx is not below the size
-template<typename Index>
-[[noreturn]] NLOHMANN_VIEW_NOINLINE void throw_array_index_out_of_range(const Index idx)
-{
-    throw_out_of_range(exception_id::array_index_out_of_range, concat("array index ", std::to_string(idx), " is out of range"));
-}
-
-/// out_of_range.402: the JSON Pointer token '-' of an array of @a size elements
-[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_array_index_past_end(std::size_t size)
-{
-    throw_out_of_range(exception_id::pointer_past_the_end_index, concat("array index '-' (", std::to_string(size), ") is out of range"));
-}
-
-/// out_of_range.403: the object has no member @a key
-template<typename Key>
-[[noreturn]] NLOHMANN_VIEW_NOINLINE void throw_key_not_found(const Key& key)
-{
-    throw_out_of_range(exception_id::key_not_found, concat("key '", key, "' not found"));
-}
-
 /// SAX handler of the error replay: accepts every event without storing it,
 /// and throws the exception the parser reports
 template<typename BasicJsonType>
@@ -144,11 +124,21 @@ struct throwing_sax
 
     /// (a template, as in json_sax_dom_parser: the parser passes the concrete
     /// exception type, which is thrown as is)
+    ///
+    /// It is not [[noreturn]]: sax_parse() recovers from an error if
+    /// parse_error() returns true, and MSVC reports that code as unreachable
+    /// (C4702) for a handler that never returns. The flag hides that from it.
     template<class Exception>
-    [[noreturn]] bool parse_error(std::size_t /*unused*/, const std::string& /*unused*/, const Exception& ex)
+    bool parse_error(std::size_t /*unused*/, const std::string& /*unused*/, const Exception& ex)
     {
-        NLOHMANN_VIEW_THROW(ex);
+        if (throwing)
+        {
+            NLOHMANN_VIEW_THROW(ex);
+        }
+        return false; // LCOV_EXCL_LINE
     }
+
+    bool throwing = true;
 };
 
 /*!
@@ -169,7 +159,7 @@ template<typename BasicJsonType>
     if (f.code == error_code::input_too_large)
     {
         // (the limit is detail::view::max_input_size(): 4 GiB minus 16 bytes)
-        NLOHMANN_VIEW_THROW(out_of_range::create(exception_id::document_too_large, "input of 4294967280 bytes or more is not supported by json_document", nullptr));
+        NLOHMANN_VIEW_THROW(out_of_range::create(416, "input of 4294967280 bytes or more is not supported by json_document", nullptr));
     }
     throwing_sax<BasicJsonType> sax;
     const bool accepted = BasicJsonType::sax_parse(src, src + size, &sax, input_format_t::json, true, ignore_comments, ignore_trailing_commas);
@@ -189,7 +179,7 @@ template<typename BasicJsonType>
         }
     }
     pos.chars_read_current_line = off + 1 - line_start;
-    NLOHMANN_VIEW_THROW(parse_error::create(exception_id::syntax_error, pos, "syntax error while parsing value", nullptr));
+    NLOHMANN_VIEW_THROW(parse_error::create(101, pos, "syntax error while parsing value", nullptr));
     // LCOV_EXCL_STOP
 }
 

include/nlohmann/detail/view/image.hpp

@@ -186,7 +186,7 @@ inline void compact_nodes(const document_data& d, std::size_t arena_size, std::v
 inline std::vector<std::uint8_t> save_image(const document_data& d)
 {
 #if !NLOHMANN_VIEW_LITTLE_ENDIAN
-    throw_type_error(exception_id::document_image_unsupported, "json_document images need a little-endian target"); // LCOV_EXCL_LINE
+    throw_type_error(320, "json_document images need a little-endian target"); // LCOV_EXCL_LINE
 #endif
     const std::size_t arena_size = d.arena_size;
     std::vector<node> compacted;
@@ -203,7 +203,7 @@ inline std::vector<std::uint8_t> save_image(const document_data& d)
     if (NLOHMANN_VIEW_UNLIKELY(text_size >= image_limit || total_arena >= image_limit || count >= image_limit))
     {
         // LCOV_EXCL_START (4 GiB)
-        throw_out_of_range(exception_id::document_too_large, "images of 4 GiB or more are not supported by json_document");
+        throw_out_of_range(416, "images of 4 GiB or more are not supported by json_document");
         // LCOV_EXCL_STOP
     }
     image_header h{};
@@ -660,7 +660,7 @@ inline bool check_image(const node* nodes, std::size_t count, const unsigned cha
 
 [[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_invalid_image(const char* what)
 {
-    throw_parse_error(exception_id::invalid_document_image, concat("invalid json_document image: ", what));
+    throw_parse_error(116, concat("invalid json_document image: ", what));
 }
 
 /// Read an image into d. The text and the decoded strings stay in the image;
@@ -668,7 +668,7 @@ inline bool check_image(const node* nodes, std::size_t count, const unsigned cha
 inline void load_image(document_data& d, const std::uint8_t* image, std::size_t size, image_check check)
 {
 #if !NLOHMANN_VIEW_LITTLE_ENDIAN
-    throw_type_error(exception_id::document_image_unsupported, "json_document images need a little-endian target"); // LCOV_EXCL_LINE
+    throw_type_error(320, "json_document images need a little-endian target"); // LCOV_EXCL_LINE
 #endif
     if (image == nullptr || size < sizeof(image_header))
     {

include/nlohmann/detail/view/iterator.hpp

@@ -110,7 +110,7 @@ class view_iterator
     {
         if (NLOHMANN_VIEW_UNLIKELY(m_value_offset == 0))
         {
-            throw_invalid_iterator(exception_id::iterator_key_not_object, "cannot use key() for non-object iterators");
+            throw_invalid_iterator(207, "cannot use key() for non-object iterators");
         }
         return string_view_t(m_doc->str(*m_pos), m_pos->len);
     }

include/nlohmann/detail/view/pointer.hpp

@@ -32,15 +32,51 @@ enum class pointer_mode
     contains,  ///< as basic_json::contains(): no exceptions at all
 };
 
-/// the outcome of reading an array index: detail::parse_array_index(), shared with json_pointer
-using index_status = detail::array_index_status;
+/// the outcome of reading an array index from a reference token
+enum class index_status
+{
+    ok,
+    leading_zero, ///< parse_error.106
+    not_number,   ///< parse_error.109
+    unresolved,   ///< out_of_range.404
+    too_large,    ///< out_of_range.410
+};
 
 /// reads an array index like json_pointer::array_index (RFC 6901, Sect. 4),
 /// but reports errors instead of throwing them
 template<typename StringType>
 index_status array_index(const StringType& s, std::size_t& idx) noexcept
 {
-    return detail::parse_array_index(s, idx);
+    // as basic_json: the empty token and "-" (past the last element) are
+    // unresolved rather than not a number
+    if (s.empty() || (s.size() == 1 && s[0] == '-'))
+    {
+        return index_status::unresolved;
+    }
+    if (s[0] < '0' || s[0] > '9')
+    {
+        return index_status::not_number;
+    }
+    if (s.size() > 1 && s[0] == '0')
+    {
+        return index_status::leading_zero;
+    }
+    std::uint64_t v = 0;
+    for (const auto c : s)
+    {
+        const auto d = static_cast<unsigned>(static_cast<unsigned char>(c)) - '0';
+        if (d > 9 || v > ((std::numeric_limits<std::uint64_t>::max)() - d) / 10)
+        {
+            return index_status::unresolved; // not a number, or beyond unsigned long long
+        }
+        v = (v * 10) + d;
+    }
+    if (v >= (std::numeric_limits<std::size_t>::max)()) // (std::size_t converts to std::uint64_t implicitly)
+    {
+        return index_status::too_large;
+    }
+    idx = static_cast<std::size_t>(v);
+    return index_status::ok;
 }
 
 /// throws the exception json_pointer::array_index throws for this status
@@ -50,15 +86,15 @@ template<typename StringType>
     switch (status)
     {
         case index_status::leading_zero:
-            throw_parse_error(exception_id::pointer_index_leading_zero, concat("array index '", s, "' must not begin with '0'"));
-        case index_status::not_a_number:
-            throw_parse_error(exception_id::pointer_index_not_a_number, concat("array index '", s, "' is not a number"));
-        case index_status::exceeds_size_type:
-            throw_out_of_range(exception_id::value_out_of_range, concat("array index ", s, " exceeds size_type")); // LCOV_EXCL_LINE
+            throw_parse_error(106, concat("array index '", s, "' must not begin with '0'"));
+        case index_status::not_number:
+            throw_parse_error(109, concat("array index '", s, "' is not a number"));
+        case index_status::too_large:
+            throw_out_of_range(410, concat("array index ", s, " exceeds size_type")); // LCOV_EXCL_LINE
         case index_status::unresolved:
         case index_status::ok:
         default:
-            throw_out_of_range(exception_id::pointer_unresolved, concat("unresolved reference token '", s, "'"));
+            throw_out_of_range(404, concat("unresolved reference token '", s, "'"));
     }
 }
 
@@ -83,7 +119,7 @@ View resolve_pointer(View cur, const Tokens& tokens, pointer_mode mode)
             {
                 if (mode == pointer_mode::checked)
                 {
-                    throw_key_not_found(token);
+                    throw_out_of_range(403, concat("key '", token, "' not found"));
                 }
                 return View();
             }
@@ -95,7 +131,7 @@ View resolve_pointer(View cur, const Tokens& tokens, pointer_mode mode)
             {
                 if (throwing)
                 {
-                    throw_array_index_past_end(cur.size());
+                    throw_out_of_range(402, concat("array index '-' (", std::to_string(cur.size()), ") is out of range"));
                 }
                 return View();
             }
@@ -103,7 +139,7 @@ View resolve_pointer(View cur, const Tokens& tokens, pointer_mode mode)
             const index_status status = array_index(token, idx);
             if (status != index_status::ok)
             {
-                const bool parse_error = status == index_status::leading_zero || status == index_status::not_a_number;
+                const bool parse_error = status == index_status::leading_zero || status == index_status::not_number;
                 if (throwing || (mode == pointer_mode::value && parse_error))
                 {
                     throw_array_index_error(status, token);
@@ -114,7 +150,7 @@ View resolve_pointer(View cur, const Tokens& tokens, pointer_mode mode)
             {
                 if (mode == pointer_mode::checked)
                 {
-                    throw_array_index_out_of_range(idx);
+                    throw_out_of_range(401, concat("array index ", std::to_string(idx), " is out of range"));
                 }
                 return View();
             }
@@ -124,7 +160,7 @@ View resolve_pointer(View cur, const Tokens& tokens, pointer_mode mode)
         {
             if (throwing)
             {
-                throw_out_of_range(exception_id::pointer_unresolved, concat("unresolved reference token '", token, "'"));
+                throw_out_of_range(404, concat("unresolved reference token '", token, "'"));
             }
             return View();
         }

include/nlohmann/detail/view/serializer.hpp

@@ -1073,17 +1073,21 @@ class view_serializer
             m_out.put("null");
             return;
         }
-        write_float(x, std::integral_constant<bool, ::nlohmann::detail::has_native_float_format<number_float_t>::value>());
+        static constexpr bool is_ieee_single_or_double
+            = std::numeric_limits<number_float_t>::is_iec559
+              && ((std::numeric_limits<number_float_t>::digits == 24 && std::numeric_limits<number_float_t>::max_exponent == 128)
+                  || (std::numeric_limits<number_float_t>::digits == 53 && std::numeric_limits<number_float_t>::max_exponent == 1024));
+        write_float(x, std::integral_constant<bool, is_ieee_single_or_double>());
     }
 
-    void write_float(number_float_t x, std::true_type /*has_native_float_format*/)
+    void write_float(number_float_t x, std::true_type /*is_ieee_single_or_double*/)
     {
         std::array<char, 64> buf{};
         const char* const end = ::nlohmann::detail::to_chars(buf.data(), buf.data() + buf.size(), x);
         m_out.put(buf.data(), static_cast<std::size_t>(end - buf.data()));
     }
 
-    void write_float(number_float_t x, std::false_type /*has_native_float_format*/)
+    void write_float(number_float_t x, std::false_type /*is_ieee_single_or_double*/)
     {
         // other types (e.g. long double) are rare: the library writes them
         const string_t s = BasicJsonType(x).dump();
@@ -1144,8 +1148,8 @@ class view_serializer
 
     void write_u_escape(std::uint32_t u)
     {
-        std::array<char, 6> e = {{'\\', 'u', 0, 0, 0, 0}};
-        hex4_lower(static_cast<std::uint16_t>(u), &e[2]);
+        static constexpr const char* hex = "0123456789abcdef";
+        const std::array<char, 6> e = {{'\\', 'u', hex[(u >> 12u) & 0xFu], hex[(u >> 8u) & 0xFu], hex[(u >> 4u) & 0xFu], hex[u & 0xFu]}};
         m_out.put(e.data(), e.size());
     }
 

include/nlohmann/detail/view/value.hpp

@@ -60,7 +60,7 @@ NLOHMANN_VIEW_ALWAYS_INLINE T arithmetic_value(const document_data& d, const nod
         case value_t::binary:
         case value_t::discarded:
         default:
-            throw_type_error(exception_id::type_mismatch, "type must be number, but is ", value_type_name(static_cast<value_t>(n.kind)));
+            throw_type_error(302, "type must be number, but is ", value_type_name(static_cast<value_t>(n.kind)));
     }
 }
 
@@ -70,7 +70,7 @@ std::vector<U, A> vector_value(const View& v)
 {
     if (NLOHMANN_VIEW_UNLIKELY(!v.is_array()))
     {
-        throw_type_error(exception_id::type_mismatch, "type must be array, but is ", v.type_name());
+        throw_type_error(302, "type must be array, but is ", v.type_name());
     }
     std::vector<U, A> r;
     r.reserve(v.size());
@@ -88,7 +88,7 @@ Map map_value(const View& v)
 {
     if (NLOHMANN_VIEW_UNLIKELY(!v.is_object()))
     {
-        throw_type_error(exception_id::type_mismatch, "type must be object, but is ", v.type_name());
+        throw_type_error(302, "type must be object, but is ", v.type_name());
     }
     Map r;
     for (auto it = v.begin(); it != v.end(); ++it)

include/nlohmann/json.hpp

@@ -55,7 +55,7 @@
 #include <nlohmann/detail/abi_macros.hpp> // IWYU pragma: export
 #include <nlohmann/detail/conversions/from_json.hpp> // IWYU pragma: keep
 #include <nlohmann/detail/conversions/to_json.hpp> // IWYU pragma: keep
-#include <nlohmann/detail/exceptions.hpp> // IWYU pragma: export
+#include <nlohmann/detail/exceptions.hpp>
 #include <nlohmann/detail/hash.hpp>
 #include <nlohmann/detail/input/binary_reader.hpp>
 #include <nlohmann/detail/input/input_adapters.hpp>
@@ -157,7 +157,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
     friend class ::nlohmann::detail::iter_impl;
     template<typename BasicJsonType, typename CharType, typename OutputSinkType>
     friend class ::nlohmann::detail::binary_writer;
-    template<typename BasicJsonType, typename InputType, typename SAX>
+    template<typename BasicJsonType, typename InputType, typename SAX, bool AllowRecovery>
     friend class ::nlohmann::detail::binary_reader;
     template<typename BasicJsonType, typename InputAdapterType, bool Collect>
     friend class ::nlohmann::detail::json_sax_dom_parser;
@@ -376,9 +376,9 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
         result["name"] = "JSON for Modern C++";
         result["url"] = "https://github.com/nlohmann/json";
         result["version"]["string"] =
-            detail::concat<string_t>(detail::to_string<string_t>(static_cast<std::size_t>(NLOHMANN_JSON_VERSION_MAJOR)), '.',
-                                     detail::to_string<string_t>(static_cast<std::size_t>(NLOHMANN_JSON_VERSION_MINOR)), '.',
-                                     detail::to_string<string_t>(static_cast<std::size_t>(NLOHMANN_JSON_VERSION_PATCH)));
+            detail::concat(std::to_string(NLOHMANN_JSON_VERSION_MAJOR), '.',
+                           std::to_string(NLOHMANN_JSON_VERSION_MINOR), '.',
+                           std::to_string(NLOHMANN_JSON_VERSION_PATCH));
         result["version"]["major"] = NLOHMANN_JSON_VERSION_MAJOR;
         result["version"]["minor"] = NLOHMANN_JSON_VERSION_MINOR;
         result["version"]["patch"] = NLOHMANN_JSON_VERSION_PATCH;
@@ -400,10 +400,10 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
 #elif defined(__clang__)
         result["compiler"] = {{"family", "clang"}, {"version", __clang_version__}};
 #elif defined(__GNUC__) || defined(__GNUG__)
-        result["compiler"] = {{"family", "gcc"}, {"version", detail::concat<string_t>(
-                    detail::to_string<string_t>(static_cast<std::size_t>(__GNUC__)), '.',
-                    detail::to_string<string_t>(static_cast<std::size_t>(__GNUC_MINOR__)), '.',
-                    detail::to_string<string_t>(static_cast<std::size_t>(__GNUC_PATCHLEVEL__)))
+        result["compiler"] = {{"family", "gcc"}, {"version", detail::concat(
+                    std::to_string(__GNUC__), '.',
+                    std::to_string(__GNUC_MINOR__), '.',
+                    std::to_string(__GNUC_PATCHLEVEL__))
             }
         };
 #elif defined(__HP_aCC)
@@ -421,9 +421,9 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
 #endif
 
 #if defined(_MSVC_LANG)
-        result["compiler"]["c++"] = detail::to_string<string_t>(static_cast<std::size_t>(_MSVC_LANG));
+        result["compiler"]["c++"] = std::to_string(_MSVC_LANG);
 #elif defined(__cplusplus)
-        result["compiler"]["c++"] = detail::to_string<string_t>(static_cast<std::size_t>(__cplusplus));
+        result["compiler"]["c++"] = std::to_string(__cplusplus);
 #else
         result["compiler"]["c++"] = "unknown";
 #endif
@@ -734,15 +734,23 @@ private:
         // true if v is not an array/object, or is an already-empty one
         static bool has_no_children(const basic_json& v) noexcept
         {
-            if (v.m_data.m_type == value_t::array)
+            switch (v.m_data.m_type)
             {
-                return v.m_data.m_value.array->empty();
+                case value_t::array:
+                    return v.m_data.m_value.array->empty();
+                case value_t::object:
+                    return v.m_data.m_value.object->empty();
+                case value_t::null:
+                case value_t::string:
+                case value_t::boolean:
+                case value_t::number_integer:
+                case value_t::number_unsigned:
+                case value_t::number_float:
+                case value_t::binary:
+                case value_t::discarded:
+                default:
+                    return true;
             }
-            if (v.m_data.m_type == value_t::object)
-            {
-                return v.m_data.m_value.object->empty();
-            }
-            return true;
         }
 
         // true if no child of the non-empty array/object v has children of
@@ -847,17 +855,28 @@ private:
         }
 
 public:
-        template<typename T>
-        static void destroy_pointer(T*& ptr) noexcept
+        void destroy_string() noexcept
         {
-            if (ptr == nullptr)
+            if (string == nullptr)
             {
                 // not initialized (e.g., due to exception in the ctor)
                 return;
             }
-            AllocatorType<T> alloc;
-            std::allocator_traits<decltype(alloc)>::destroy(alloc, ptr);
-            std::allocator_traits<decltype(alloc)>::deallocate(alloc, ptr, 1);
+            AllocatorType<string_t> alloc;
+            std::allocator_traits<decltype(alloc)>::destroy(alloc, string);
+            std::allocator_traits<decltype(alloc)>::deallocate(alloc, string, 1);
+        }
+
+        void destroy_binary() noexcept
+        {
+            if (binary == nullptr)
+            {
+                // not initialized (e.g., due to exception in the ctor)
+                return;
+            }
+            AllocatorType<binary_t> alloc;
+            std::allocator_traits<decltype(alloc)>::destroy(alloc, binary);
+            std::allocator_traits<decltype(alloc)>::deallocate(alloc, binary, 1);
         }
 
         // t must be value_t::array or value_t::object
@@ -1023,11 +1042,11 @@ public:
             switch (t)
             {
                 case value_t::string:
-                    destroy_pointer(string);
+                    destroy_string();
                     break;
 
                 case value_t::binary:
-                    destroy_pointer(binary);
+                    destroy_binary();
                     break;
 
                 case value_t::object:
@@ -1089,19 +1108,36 @@ public:
     void set_parents()
     {
 #if JSON_DIAGNOSTICS
-        if (m_data.m_type == value_t::array)
+        switch (m_data.m_type)
         {
-            for (auto& element : *m_data.m_value.array)
+            case value_t::array:
             {
-                element.m_parent = this;
+                for (auto& element : *m_data.m_value.array)
+                {
+                    element.m_parent = this;
+                }
+                break;
             }
-        }
-        else if (m_data.m_type == value_t::object)
-        {
-            for (auto& element : *m_data.m_value.object)
+
+            case value_t::object:
             {
-                element.second.m_parent = this;
+                for (auto& element : *m_data.m_value.object)
+                {
+                    element.second.m_parent = this;
+                }
+                break;
             }
+
+            case value_t::null:
+            case value_t::string:
+            case value_t::boolean:
+            case value_t::number_integer:
+            case value_t::number_unsigned:
+            case value_t::number_float:
+            case value_t::binary:
+            case value_t::discarded:
+            default:
+                break;
         }
 #endif
     }
@@ -2549,7 +2585,8 @@ public:
             case value_t::string:
             case value_t::binary:
             {
-                if (JSON_HEDLEY_UNLIKELY(!primitive_range_is_complete(first, last)))
+                if (JSON_HEDLEY_UNLIKELY(!first.m_it.primitive_iterator.is_begin()
+                                         || !last.m_it.primitive_iterator.is_end()))
                 {
                     detail::throw_literal<invalid_iterator>(detail::exception_id::iterator_range_out_of_range, "iterators out of range", first.m_object);
                 }
@@ -3023,134 +3060,6 @@ public:
         JSON_THROW(std::length_error(detail::concat("array index ", std::to_string(idx), " exceeds size_type")));
     }
 
-    /// @throw type_error.309 always, as insert() is not defined for the type of @a j
-    JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE
-    static void throw_insert_wrong_type(const basic_json& j)
-    {
-        detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", j);
-    }
-
-    /// @throw invalid_iterator.202 always, as an iterator belongs to another value than @a obj
-    JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE
-    static void throw_iterator_from_other_value(const basic_json* obj)
-    {
-        detail::throw_literal<invalid_iterator>(detail::exception_id::iterator_from_other_value, "iterator does not fit current value", obj);
-    }
-
-    /// @throw type_error.310 always, as swap() with the given operand is not defined for the type of @a j
-    JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE
-    static void throw_swap_wrong_type(const char* operation, const basic_json& j)
-    {
-        detail::throw_cannot_use_with(detail::exception_id::swap_wrong_type, operation, j);
-    }
-
-    /// exchange the stored @a Stored value (if the value has that type) with @a other
-    template<typename Stored, typename Other>
-    void swap_impl(Other& other, const char* operation)
-    {
-        Stored* const stored = get_impl_ptr(static_cast<Stored*>(nullptr));
-        if (JSON_HEDLEY_UNLIKELY(stored == nullptr))
-        {
-            throw_swap_wrong_type(operation, *this);
-        }
-
-        using std::swap;
-        swap(*stored, other);
-    }
-
-    /// whether [first, last) spans the whole of a primitive value
-    template<typename IteratorType>
-    JSON_HEDLEY_ALWAYS_INLINE
-    static bool primitive_range_is_complete(const IteratorType& first, const IteratorType& last) noexcept
-    {
-        return first.m_it.primitive_iterator.is_begin() && last.m_it.primitive_iterator.is_end();
-    }
-
-    /// destroy the current (primitive) value and become null
-    void reset_to_null_impl()
-    {
-        m_data.m_value.destroy(m_data.m_type);
-        m_data.m_value = {};
-        m_data.m_type = value_t::null;
-        assert_invariant();
-    }
-
-    /// append @a val (moved or copied) to the array; a null value becomes an array
-    template<typename ValueType>
-    JSON_HEDLEY_ALWAYS_INLINE
-    void push_back_array_impl(ValueType&& val)
-    {
-        // push_back only works for null objects or arrays
-        if (JSON_HEDLEY_UNLIKELY(!(is_null() || is_array())))
-        {
-            detail::throw_cannot_use_with(detail::exception_id::push_back_wrong_type, "push_back()", *this);
-        }
-
-        // transform a null object into an array
-        if (is_null())
-        {
-            convert_null_to<array_t>();
-        }
-
-        // add the element to the array
-        const auto old_capacity = container_capacity();
-        m_data.m_value.array->push_back(std::forward<ValueType>(val));
-        set_parent_after_insert(m_data.m_value.array->back(), old_capacity);
-    }
-
-    /// parse the input behind an already created adapter
-    template<typename InputAdapterType>
-    static basic_json parse_impl(InputAdapterType&& adapter,
-                                 parser_callback_t cb,
-                                 const bool allow_exceptions,
-                                 const bool ignore_comments,
-                                 const bool ignore_trailing_commas)
-    {
-        basic_json result;
-        parser(std::forward<InputAdapterType>(adapter), std::move(cb), allow_exceptions, ignore_comments, ignore_trailing_commas).parse(true, result);
-        return result;
-    }
-
-    /// check the input behind an already created adapter
-    template<typename InputAdapterType>
-    static bool accept_impl(InputAdapterType&& adapter,
-                            const bool ignore_comments,
-                            const bool ignore_trailing_commas)
-    {
-        return parser(std::forward<InputAdapterType>(adapter), nullptr, false, ignore_comments, ignore_trailing_commas, true).accept(true);
-    }
-
-    /// generate SAX events from the input behind an already created adapter
-    template<typename InputAdapterType, typename SAX>
-    static bool sax_parse_impl(InputAdapterType ia, SAX* sax,
-                               const input_format_t format,
-                               const bool strict,
-                               const bool ignore_comments,
-                               const bool ignore_trailing_commas,
-                               const cbor_tag_handler_t tag_handler)
-    {
-        if (format == input_format_t::json)
-        {
-            return parser(std::move(ia), nullptr, true, ignore_comments, ignore_trailing_commas).sax_parse(sax, strict);
-        }
-        return detail::binary_reader<basic_json, InputAdapterType, SAX>(std::move(ia), format).sax_parse(sax, strict, tag_handler);
-    }
-
-    /// serialize @a j into a new vector; @a write receives the vector-backed writer
-    template<typename WriteFunction>
-    static std::vector<std::uint8_t> to_binary_vector(const basic_json& j, const error_handler_t error_handler, WriteFunction write)
-    {
-        std::vector<std::uint8_t> result;
-        result.reserve(detail::binary_initial_capacity(j));
-        {
-            // the sink sets the size of result when it is destroyed, which
-            // must happen before result is returned (without NRVO, it is moved)
-            auto writer = vector_writer(result, error_handler);
-            write(writer);
-        }
-        return result;
-    }
-
     /// @throw out_of_range.401 always, as @a idx is out of range of @a j
     JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE
     static void throw_array_index_out_of_range(size_type idx, const basic_json& j)
@@ -4133,7 +4042,7 @@ public:
         // make sure the iterator fits the current value
         if (JSON_HEDLEY_UNLIKELY(this != pos.m_object))
         {
-            throw_iterator_from_other_value(this);
+            detail::throw_literal<invalid_iterator>(detail::exception_id::iterator_from_other_value, "iterator does not fit current value", this);
         }
 
         IteratorType result = end();
@@ -4152,7 +4061,10 @@ public:
                     detail::throw_literal<invalid_iterator>(detail::exception_id::iterator_out_of_range, "iterator out of range", this);
                 }
 
-                reset_to_null_impl();
+                m_data.m_value.destroy(m_data.m_type);
+                m_data.m_value = {};
+                m_data.m_type = value_t::null;
+                assert_invariant();
                 break;
             }
 
@@ -4202,12 +4114,16 @@ public:
             case value_t::string:
             case value_t::binary:
             {
-                if (JSON_HEDLEY_UNLIKELY(!primitive_range_is_complete(first, last)))
+                if (JSON_HEDLEY_UNLIKELY(!first.m_it.primitive_iterator.is_begin()
+                                         || !last.m_it.primitive_iterator.is_end()))
                 {
                     detail::throw_literal<invalid_iterator>(detail::exception_id::iterator_range_out_of_range, "iterators out of range", this);
                 }
 
-                reset_to_null_impl();
+                m_data.m_value.destroy(m_data.m_type);
+                m_data.m_value = {};
+                m_data.m_type = value_t::null;
+                assert_invariant();
                 break;
             }
 
@@ -4688,20 +4604,34 @@ public:
     JSON_HEDLEY_WARN_UNUSED_RESULT
     size_type max_size() const noexcept
     {
-        if (m_data.m_type == value_t::array)
+        switch (m_data.m_type)
         {
-            // delegate call to array_t::max_size()
-            return m_data.m_value.array->max_size();
-        }
+            case value_t::array:
+            {
+                // delegate call to array_t::max_size()
+                return m_data.m_value.array->max_size();
+            }
 
-        if (m_data.m_type == value_t::object)
-        {
-            // delegate call to object_t::max_size()
-            return m_data.m_value.object->max_size();
-        }
+            case value_t::object:
+            {
+                // delegate call to object_t::max_size()
+                return m_data.m_value.object->max_size();
+            }
 
-        // all other types have max_size() == size()
-        return size();
+            case value_t::null:
+            case value_t::string:
+            case value_t::boolean:
+            case value_t::number_integer:
+            case value_t::number_unsigned:
+            case value_t::number_float:
+            case value_t::binary:
+            case value_t::discarded:
+            default:
+            {
+                // all other types have max_size() == size()
+                return size();
+            }
+        }
     }
 
     /// @}
@@ -4779,7 +4709,22 @@ public:
     /// @sa https://json.nlohmann.me/api/basic_json/push_back/
     void push_back(basic_json&& val)
     {
-        push_back_array_impl(std::move(val));
+        // push_back only works for null objects or arrays
+        if (JSON_HEDLEY_UNLIKELY(!(is_null() || is_array())))
+        {
+            detail::throw_cannot_use_with(detail::exception_id::push_back_wrong_type, "push_back()", *this);
+        }
+
+        // transform a null object into an array
+        if (is_null())
+        {
+            convert_null_to<array_t>();
+        }
+
+        // add the element to the array (move semantics)
+        const auto old_capacity = container_capacity();
+        m_data.m_value.array->push_back(std::move(val));
+        set_parent_after_insert(m_data.m_value.array->back(), old_capacity);
         // if val is moved from, basic_json move constructor marks it null, so we do not call the destructor
     }
 
@@ -4795,7 +4740,22 @@ public:
     /// @sa https://json.nlohmann.me/api/basic_json/push_back/
     void push_back(const basic_json& val)
     {
-        push_back_array_impl(val);
+        // push_back only works for null objects or arrays
+        if (JSON_HEDLEY_UNLIKELY(!(is_null() || is_array())))
+        {
+            detail::throw_cannot_use_with(detail::exception_id::push_back_wrong_type, "push_back()", *this);
+        }
+
+        // transform a null object into an array
+        if (is_null())
+        {
+            convert_null_to<array_t>();
+        }
+
+        // add the element to the array
+        const auto old_capacity = container_capacity();
+        m_data.m_value.array->push_back(val);
+        set_parent_after_insert(m_data.m_value.array->back(), old_capacity);
     }
 
     /// @brief add an object to an array
@@ -4941,13 +4901,13 @@ public:
         // insert only works for arrays
         if (JSON_HEDLEY_UNLIKELY(!is_array()))
         {
-            throw_insert_wrong_type(*this);
+            detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
         }
 
         // check if iterator pos fits to this JSON value
         if (JSON_HEDLEY_UNLIKELY(pos.m_object != this))
         {
-            throw_iterator_from_other_value(this);
+            detail::throw_literal<invalid_iterator>(detail::exception_id::iterator_from_other_value, "iterator does not fit current value", this);
         }
 
         // insert to array and return iterator
@@ -4961,13 +4921,13 @@ public:
         // insert only works for arrays
         if (JSON_HEDLEY_UNLIKELY(!is_array()))
         {
-            throw_insert_wrong_type(*this);
+            detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
         }
 
         // check if iterator pos fits to this JSON value
         if (JSON_HEDLEY_UNLIKELY(pos.m_object != this))
         {
-            throw_iterator_from_other_value(this);
+            detail::throw_literal<invalid_iterator>(detail::exception_id::iterator_from_other_value, "iterator does not fit current value", this);
         }
 
         // moving into a local first keeps this safe even if val aliases
@@ -4983,13 +4943,13 @@ public:
         // insert only works for arrays
         if (JSON_HEDLEY_UNLIKELY(!is_array()))
         {
-            throw_insert_wrong_type(*this);
+            detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
         }
 
         // check if iterator pos fits to this JSON value
         if (JSON_HEDLEY_UNLIKELY(pos.m_object != this))
         {
-            throw_iterator_from_other_value(this);
+            detail::throw_literal<invalid_iterator>(detail::exception_id::iterator_from_other_value, "iterator does not fit current value", this);
         }
 
         // insert to array and return iterator
@@ -5003,13 +4963,13 @@ public:
         // insert only works for arrays
         if (JSON_HEDLEY_UNLIKELY(!is_array()))
         {
-            throw_insert_wrong_type(*this);
+            detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
         }
 
         // check if iterator pos fits to this JSON value
         if (JSON_HEDLEY_UNLIKELY(pos.m_object != this))
         {
-            throw_iterator_from_other_value(this);
+            detail::throw_literal<invalid_iterator>(detail::exception_id::iterator_from_other_value, "iterator does not fit current value", this);
         }
 
         // check if range iterators belong to the same JSON object
@@ -5040,13 +5000,13 @@ public:
         // insert only works for arrays
         if (JSON_HEDLEY_UNLIKELY(!is_array()))
         {
-            throw_insert_wrong_type(*this);
+            detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
         }
 
         // check if iterator pos fits to this JSON value
         if (JSON_HEDLEY_UNLIKELY(pos.m_object != this))
         {
-            throw_iterator_from_other_value(this);
+            detail::throw_literal<invalid_iterator>(detail::exception_id::iterator_from_other_value, "iterator does not fit current value", this);
         }
 
         // copy the values first: ilist may refer to elements of this array
@@ -5068,7 +5028,7 @@ public:
         // insert only works for objects
         if (JSON_HEDLEY_UNLIKELY(!is_object()))
         {
-            throw_insert_wrong_type(*this);
+            detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
         }
 
         // check if range iterators belong to the same JSON object
@@ -5310,37 +5270,82 @@ public:
     /// @sa https://json.nlohmann.me/api/basic_json/swap/
     void swap(array_t& other) // NOLINT(bugprone-exception-escape,cppcoreguidelines-noexcept-swap,performance-noexcept-swap)
     {
-        swap_impl<array_t>(other, "swap(array_t&)");
-        set_parents();
+        // swap only works for arrays
+        if (JSON_HEDLEY_LIKELY(is_array()))
+        {
+            using std::swap;
+            swap(*(m_data.m_value.array), other);
+            set_parents();
+        }
+        else
+        {
+            detail::throw_cannot_use_with(detail::exception_id::swap_wrong_type, "swap(array_t&)", *this);
+        }
     }
 
     /// @brief exchanges the values
     /// @sa https://json.nlohmann.me/api/basic_json/swap/
     void swap(object_t& other) // NOLINT(bugprone-exception-escape,cppcoreguidelines-noexcept-swap,performance-noexcept-swap)
     {
-        swap_impl<object_t>(other, "swap(object_t&)");
-        set_parents();
+        // swap only works for objects
+        if (JSON_HEDLEY_LIKELY(is_object()))
+        {
+            using std::swap;
+            swap(*(m_data.m_value.object), other);
+            set_parents();
+        }
+        else
+        {
+            detail::throw_cannot_use_with(detail::exception_id::swap_wrong_type, "swap(object_t&)", *this);
+        }
     }
 
     /// @brief exchanges the values
     /// @sa https://json.nlohmann.me/api/basic_json/swap/
     void swap(string_t& other) // NOLINT(bugprone-exception-escape,cppcoreguidelines-noexcept-swap,performance-noexcept-swap)
     {
-        swap_impl<string_t>(other, "swap(string_t&)");
+        // swap only works for strings
+        if (JSON_HEDLEY_LIKELY(is_string()))
+        {
+            using std::swap;
+            swap(*(m_data.m_value.string), other);
+        }
+        else
+        {
+            detail::throw_cannot_use_with(detail::exception_id::swap_wrong_type, "swap(string_t&)", *this);
+        }
     }
 
     /// @brief exchanges the values
     /// @sa https://json.nlohmann.me/api/basic_json/swap/
     void swap(binary_t& other) // NOLINT(bugprone-exception-escape,cppcoreguidelines-noexcept-swap,performance-noexcept-swap)
     {
-        swap_impl<binary_t>(other, "swap(binary_t&)");
+        // swap only works for binary values
+        if (JSON_HEDLEY_LIKELY(is_binary()))
+        {
+            using std::swap;
+            swap(*(m_data.m_value.binary), other);
+        }
+        else
+        {
+            detail::throw_cannot_use_with(detail::exception_id::swap_wrong_type, "swap(binary_t&)", *this);
+        }
     }
 
     /// @brief exchanges the values
     /// @sa https://json.nlohmann.me/api/basic_json/swap/
     void swap(typename binary_t::container_type& other) // NOLINT(bugprone-exception-escape)
     {
-        swap_impl<binary_t>(other, "swap(binary_t::container_type&)");
+        // swap only works for binary values
+        if (JSON_HEDLEY_LIKELY(is_binary()))
+        {
+            using std::swap;
+            swap(*(m_data.m_value.binary), other);
+        }
+        else
+        {
+            detail::throw_cannot_use_with(detail::exception_id::swap_wrong_type, "swap(binary_t::container_type&)", *this);
+        }
     }
 
     /// @}
@@ -6021,7 +6026,10 @@ public:
                             const bool ignore_comments = false,
                             const bool ignore_trailing_commas = false)
     {
-        return parse_impl(detail::input_adapter(std::forward<InputType>(i)), std::move(cb), allow_exceptions, ignore_comments, ignore_trailing_commas);
+        basic_json result;
+        auto p = parser(detail::input_adapter(std::forward<InputType>(i)), std::move(cb), allow_exceptions, ignore_comments, ignore_trailing_commas);
+        p.parse(true, result);
+        return result;
     }
 
     /// @brief deserialize from a pair of character iterators (or an iterator+sentinel pair, C++20 ranges support)
@@ -6036,7 +6044,10 @@ public:
                             const bool ignore_comments = false,
                             const bool ignore_trailing_commas = false)
     {
-        return parse_impl(detail::input_adapter(std::move(first), std::move(last)), std::move(cb), allow_exceptions, ignore_comments, ignore_trailing_commas);
+        basic_json result;
+        auto p = parser(detail::input_adapter(std::move(first), std::move(last)), std::move(cb), allow_exceptions, ignore_comments, ignore_trailing_commas);
+        p.parse(true, result);
+        return result;
     }
 
     JSON_HEDLEY_WARN_UNUSED_RESULT
@@ -6050,7 +6061,10 @@ public:
         = delete;
 #else
     {
-        return parse_impl(i.get(), std::move(cb), allow_exceptions, ignore_comments, ignore_trailing_commas);
+        basic_json result;
+        auto p = parser(i.get(), std::move(cb), allow_exceptions, ignore_comments, ignore_trailing_commas);
+        p.parse(true, result);
+        return result;
     }
 #endif
 
@@ -6062,7 +6076,7 @@ public:
                        const bool ignore_comments = false,
                        const bool ignore_trailing_commas = false)
     {
-        return accept_impl(detail::input_adapter(std::forward<InputType>(i)), ignore_comments, ignore_trailing_commas);
+        return parser(detail::input_adapter(std::forward<InputType>(i)), nullptr, false, ignore_comments, ignore_trailing_commas, true).accept(true);
     }
 
     /// @brief check if the input is valid JSON (iterator pair, or iterator+sentinel pair for C++20 ranges support)
@@ -6074,7 +6088,7 @@ public:
                        const bool ignore_comments = false,
                        const bool ignore_trailing_commas = false)
     {
-        return accept_impl(detail::input_adapter(std::move(first), std::move(last)), ignore_comments, ignore_trailing_commas);
+        return parser(detail::input_adapter(std::move(first), std::move(last)), nullptr, false, ignore_comments, ignore_trailing_commas, true).accept(true);
     }
 
     JSON_HEDLEY_WARN_UNUSED_RESULT
@@ -6086,7 +6100,7 @@ public:
         = delete;
 #else
     {
-        return accept_impl(i.get(), ignore_comments, ignore_trailing_commas);
+        return parser(i.get(), nullptr, false, ignore_comments, ignore_trailing_commas, true).accept(true);
     }
 #endif
 
@@ -6101,7 +6115,10 @@ public:
                           const bool ignore_trailing_commas = false,
                           const cbor_tag_handler_t tag_handler = cbor_tag_handler_t::error)
     {
-        return sax_parse_impl(detail::input_adapter(std::forward<InputType>(i)), sax, format, strict, ignore_comments, ignore_trailing_commas, tag_handler);
+        auto ia = detail::input_adapter(std::forward<InputType>(i));
+        return format == input_format_t::json
+               ? parser(std::move(ia), nullptr, true, ignore_comments, ignore_trailing_commas).sax_parse(sax, strict)
+               : detail::binary_reader<basic_json, decltype(ia), SAX, true>(std::move(ia), format).sax_parse(sax, strict, tag_handler);
     }
 
     /// @brief generate SAX events (iterator pair, or iterator+sentinel pair for C++20 ranges support)
@@ -6116,7 +6133,10 @@ public:
                           const bool ignore_trailing_commas = false,
                           const cbor_tag_handler_t tag_handler = cbor_tag_handler_t::error)
     {
-        return sax_parse_impl(detail::input_adapter(std::move(first), std::move(last)), sax, format, strict, ignore_comments, ignore_trailing_commas, tag_handler);
+        auto ia = detail::input_adapter(std::move(first), std::move(last));
+        return format == input_format_t::json
+               ? parser(std::move(ia), nullptr, true, ignore_comments, ignore_trailing_commas).sax_parse(sax, strict)
+               : detail::binary_reader<basic_json, decltype(ia), SAX, true>(std::move(ia), format).sax_parse(sax, strict, tag_handler);
     }
 
     /// @brief generate SAX events
@@ -6150,7 +6170,12 @@ public:
     = delete;
 #else
     {
-        return sax_parse_impl(i.get(), sax, format, strict, ignore_comments, ignore_trailing_commas, tag_handler);
+        auto ia = i.get();
+        return format == input_format_t::json
+               // NOLINTNEXTLINE(hicpp-move-const-arg,performance-move-const-arg)
+               ? parser(std::move(ia), nullptr, true, ignore_comments, ignore_trailing_commas).sax_parse(sax, strict)
+               // NOLINTNEXTLINE(hicpp-move-const-arg,performance-move-const-arg)
+               : detail::binary_reader<basic_json, decltype(ia), SAX, true>(std::move(ia), format).sax_parse(sax, strict, tag_handler);
     }
 #endif
 #if defined(__clang__)
@@ -6298,10 +6323,10 @@ public:
     static std::vector<std::uint8_t> to_cbor(const basic_json& j,
             const error_handler_t error_handler = detail::binary_writer_default_error_handler())
     {
-        return to_binary_vector(j, error_handler, [&](vector_binary_writer<std::uint8_t>& w)
-        {
-            w.write_cbor(j);
-        });
+        std::vector<std::uint8_t> result;
+        result.reserve(detail::binary_initial_capacity(j));
+        vector_writer(result, error_handler).write_cbor(j);
+        return result;
     }
 
     /// @brief create a CBOR serialization of a given JSON value
@@ -6329,10 +6354,10 @@ public:
     static std::vector<std::uint8_t> to_msgpack(const basic_json& j,
             const error_handler_t error_handler = error_handler_t::keep)
     {
-        return to_binary_vector(j, error_handler, [&](vector_binary_writer<std::uint8_t>& w)
-        {
-            w.write_msgpack(j);
-        });
+        std::vector<std::uint8_t> result;
+        result.reserve(detail::binary_initial_capacity(j));
+        vector_writer(result, error_handler).write_msgpack(j);
+        return result;
     }
 
     /// @brief create a MessagePack serialization of a given JSON value
@@ -6362,10 +6387,10 @@ public:
             const bool use_type = false,
             const error_handler_t error_handler = detail::binary_writer_default_error_handler())
     {
-        return to_binary_vector(j, error_handler, [&](vector_binary_writer<std::uint8_t>& w)
-        {
-            w.write_ubjson(j, use_size, use_type);
-        });
+        std::vector<std::uint8_t> result;
+        result.reserve(detail::binary_initial_capacity(j));
+        vector_writer(result, error_handler).write_ubjson(j, use_size, use_type);
+        return result;
     }
 
     /// @brief create a UBJSON serialization of a given JSON value
@@ -6398,10 +6423,10 @@ public:
             const bjdata_version_t version = bjdata_version_t::draft2,
             const error_handler_t error_handler = detail::binary_writer_default_error_handler())
     {
-        return to_binary_vector(j, error_handler, [&](vector_binary_writer<std::uint8_t>& w)
-        {
-            w.write_ubjson(j, use_size, use_type, true, true, version);
-        });
+        std::vector<std::uint8_t> result;
+        result.reserve(detail::binary_initial_capacity(j));
+        vector_writer(result, error_handler).write_ubjson(j, use_size, use_type, true, true, version);
+        return result;
     }
 
     /// @brief create a BJData serialization of a given JSON value
@@ -6433,10 +6458,10 @@ public:
     static std::vector<std::uint8_t> to_bson(const basic_json& j,
             const error_handler_t error_handler = detail::binary_writer_default_error_handler())
     {
-        return to_binary_vector(j, error_handler, [&](vector_binary_writer<std::uint8_t>& w)
-        {
-            w.write_bson(j);
-        });
+        std::vector<std::uint8_t> result;
+        result.reserve(detail::binary_initial_capacity(j));
+        vector_writer(result, error_handler).write_bson(j);
+        return result;
     }
 
     /// @brief create a BSON serialization of a given JSON value
@@ -6463,10 +6488,10 @@ public:
     /// @sa https://json.nlohmann.me/api/basic_json/to_bon8/
     static std::vector<std::uint8_t> to_bon8(const basic_json& j)
     {
-        return to_binary_vector(j, detail::binary_writer_default_error_handler(), [&](vector_binary_writer<std::uint8_t>& w)
-        {
-            w.write_bon8(j);
-        });
+        std::vector<std::uint8_t> result;
+        result.reserve(detail::binary_initial_capacity(j));
+        vector_writer(result).write_bon8(j);
+        return result;
     }
 
     /// @brief create a BON8 serialization of a given JSON value
@@ -7802,68 +7827,82 @@ public:
             return;
         }
 
-        if (detail::is_primitive(source.type()))
+        switch (source.type())
         {
-            // both primitive types: replace value
-            diff_replace(result, path, target);
-            return;
-        }
-
-        if (source.is_array())
-        {
-            const std::size_t path_length = path.size();
-
-            // first pass: traverse common elements
-            std::size_t i = 0;
-            while (i < source.size() && i < target.size())
+            case value_t::array:
             {
-                // recursive call to compare array values at index i
-                if (!(source[i] == target[i]))
+                const std::size_t path_length = path.size();
+
+                // first pass: traverse common elements
+                std::size_t i = 0;
+                while (i < source.size() && i < target.size())
                 {
-                    diff_append_index(path, i);
-                    diff_unequal(result, source[i], target[i], path, depth + 1);
-                    path.resize(path_length);
+                    // recursive call to compare array values at index i
+                    if (!(source[i] == target[i]))
+                    {
+                        diff_append_index(path, i);
+                        diff_unequal(result, source[i], target[i], path, depth + 1);
+                        path.resize(path_length);
+                    }
+                    ++i;
                 }
-                ++i;
+
+                // We now reached the end of at least one array
+                // in a second pass, traverse the remaining elements
+                diff_array_tails(result, source, target, path, i);
+                break;
             }
 
-            // We now reached the end of at least one array
-            // in a second pass, traverse the remaining elements
-            diff_array_tails(result, source, target, path, i);
-        }
-        else
-        {
-            diff_target_values target_values;
-            std::vector<const_iterator> added;
-            if (diff_object_keys(result, source, target, path, target_values, added))
+            case value_t::object:
             {
-                // fast path: target_values holds, for every member of
-                // source in order, its value in target -- so no further
-                // lookup is needed. Deleted keys are interleaved here
-                // too, in source's original order, to match the
-                // historical (pre-reordering-aware) output order.
-                const std::size_t path_length = path.size();
-                std::size_t i = 0;
-                for (auto it = source.cbegin(); it != source.cend(); ++it, ++i)
+                diff_target_values target_values;
+                std::vector<const_iterator> added;
+                if (diff_object_keys(result, source, target, path, target_values, added))
                 {
-                    if (target_values[i] != nullptr)
+                    // fast path: target_values holds, for every member of
+                    // source in order, its value in target -- so no further
+                    // lookup is needed. Deleted keys are interleaved here
+                    // too, in source's original order, to match the
+                    // historical (pre-reordering-aware) output order.
+                    const std::size_t path_length = path.size();
+                    std::size_t i = 0;
+                    for (auto it = source.cbegin(); it != source.cend(); ++it, ++i)
                     {
-                        if (!(it.value() == *target_values[i]))
+                        if (target_values[i] != nullptr)
                         {
-                            diff_append_key(path, it.key());
-                            diff_unequal(result, it.value(), *target_values[i], path, depth + 1);
-                            path.resize(path_length);
+                            if (!(it.value() == *target_values[i]))
+                            {
+                                diff_append_key(path, it.key());
+                                diff_unequal(result, it.value(), *target_values[i], path, depth + 1);
+                                path.resize(path_length);
+                            }
+                        }
+                        else
+                        {
+                            // found a key that is not in target -> remove it
+                            diff_remove(result, diff_key_path(path, it.key()));
                         }
                     }
-                    else
-                    {
-                        // found a key that is not in target -> remove it
-                        diff_remove(result, diff_key_path(path, it.key()));
-                    }
-                }
 
-                // append the "add" operations for brand-new keys
-                diff_added(result, added, path);
+                    // append the "add" operations for brand-new keys
+                    diff_added(result, added, path);
+                }
+                break;
+            }
+
+            case value_t::null:
+            case value_t::string:
+            case value_t::boolean:
+            case value_t::number_integer:
+            case value_t::number_unsigned:
+            case value_t::number_float:
+            case value_t::binary:
+            case value_t::discarded:
+            default:
+            {
+                // both primitive types: replace value
+                diff_replace(result, path, target);
+                break;
             }
         }
     }
@@ -7909,31 +7948,46 @@ public:
                 return;
             }
 
-            if (detail::is_primitive(s.type()))
+            switch (s.type())
             {
-                // both primitive types: replace value
-                diff_replace(result, current_path, t);
-                return;
-            }
+                case value_t::array:
+                {
+                    stack.emplace_back(&s, &t, current_path.size());
+                    return;
+                }
 
-            if (s.is_array())
-            {
-                stack.emplace_back(&s, &t, current_path.size());
-                return;
+                case value_t::object:
+                {
+                    std::vector<const basic_json*> target_values;
+                    std::vector<const_iterator> added;
+                    if (diff_object_keys(result, s, t, current_path, target_values, added))
+                    {
+                        // fast path: the frame walks source in lockstep with
+                        // common_keys, as diff_recursively does, and appends
+                        // added_ops once all members are done
+                        stack.emplace_back(&s, &t, current_path.size());
+                        stack.back().member = s.cbegin();
+                        stack.back().target_values = std::move(target_values);
+                        stack.back().added = std::move(added);
+                    }
+                    return;
+                }
+
+                case value_t::null:
+                case value_t::string:
+                case value_t::boolean:
+                case value_t::number_integer:
+                case value_t::number_unsigned:
+                case value_t::number_float:
+                case value_t::binary:
+                case value_t::discarded:
+                default:
+                {
+                    // both primitive types: replace value
+                    diff_replace(result, current_path, t);
+                    return;
+                }
             }
-            std::vector<const basic_json*> target_values;
-            std::vector<const_iterator> added;
-            if (diff_object_keys(result, s, t, current_path, target_values, added))
-            {
-                // fast path: the frame walks source in lockstep with
-                // common_keys, as diff_recursively does, and appends
-                // added_ops once all members are done
-                stack.emplace_back(&s, &t, current_path.size());
-                stack.back().member = s.cbegin();
-                stack.back().target_values = std::move(target_values);
-                stack.back().added = std::move(added);
-            }
-            return;
         };
 
         enter(source, target);

include/nlohmann/json_view.hpp

@@ -268,7 +268,7 @@ class basic_json_view
             {
                 return {};
             }
-            detail::view::throw_type_error(detail::exception_id::subscript_wrong_type, "cannot use operator[] with a string argument with ", type_name());
+            detail::view::throw_type_error(305, "cannot use operator[] with a string argument with ", type_name());
         }
         return lookup(key);
     }
@@ -294,7 +294,7 @@ class basic_json_view
             {
                 return {};
             }
-            detail::view::throw_type_error(detail::exception_id::subscript_wrong_type, "cannot use operator[] with a numeric argument with ", type_name());
+            detail::view::throw_type_error(305, "cannot use operator[] with a numeric argument with ", type_name());
         }
         return idx < m_node->len ? element(idx) : basic_json_view();
     }
@@ -327,12 +327,12 @@ class basic_json_view
     {
         if (NLOHMANN_VIEW_UNLIKELY(!is_object()))
         {
-            detail::view::throw_type_error(detail::exception_id::at_wrong_type, "cannot use at() with ", type_name());
+            detail::view::throw_type_error(304, "cannot use at() with ", type_name());
         }
         const basic_json_view r = lookup(key);
         if (NLOHMANN_VIEW_UNLIKELY(r.is_discarded()))
         {
-            detail::view::throw_key_not_found(std::string(key.data(), key.size()));
+            detail::view::throw_out_of_range(403, detail::concat("key '", std::string(key.data(), key.size()), "' not found"));
         }
         return r;
     }
@@ -353,11 +353,11 @@ class basic_json_view
     {
         if (NLOHMANN_VIEW_UNLIKELY(!is_array()))
         {
-            detail::view::throw_type_error(detail::exception_id::at_wrong_type, "cannot use at() with ", type_name());
+            detail::view::throw_type_error(304, "cannot use at() with ", type_name());
         }
         if (NLOHMANN_VIEW_UNLIKELY(idx >= m_node->len))
         {
-            detail::view::throw_array_index_out_of_range(idx);
+            detail::view::throw_out_of_range(401, detail::concat("array index ", std::to_string(idx), " is out of range"));
         }
         return element(idx);
     }
@@ -385,7 +385,7 @@ class basic_json_view
     {
         if (NLOHMANN_VIEW_UNLIKELY(!is_object()))
         {
-            detail::view::throw_type_error(detail::exception_id::value_wrong_type, "cannot use value() with ", type_name());
+            detail::view::throw_type_error(306, "cannot use value() with ", type_name());
         }
         const basic_json_view r = lookup(key);
         return r.is_discarded() ? default_value : r.template get<T>();
@@ -410,7 +410,7 @@ class basic_json_view
     {
         if (NLOHMANN_VIEW_UNLIKELY(!is_structured()))
         {
-            detail::view::throw_type_error(detail::exception_id::value_wrong_type, "cannot use value() with ", type_name());
+            detail::view::throw_type_error(306, "cannot use value() with ", type_name());
         }
         const basic_json_view r = detail::view::resolve_pointer(*this, detail::json_pointer_access::reference_tokens(ptr), detail::view::pointer_mode::value);
         return r.is_discarded() ? default_value : r.template get<T>();
@@ -433,7 +433,7 @@ class basic_json_view
         const iterator it = begin();
         if (NLOHMANN_VIEW_UNLIKELY(it == end()))
         {
-            detail::view::throw_invalid_iterator(detail::exception_id::iterator_value_unavailable, "cannot get value");
+            detail::view::throw_invalid_iterator(214, "cannot get value");
         }
         return *it;
     }
@@ -586,7 +586,7 @@ class basic_json_view
     {
         if (NLOHMANN_VIEW_UNLIKELY(!is_string()))
         {
-            detail::view::throw_type_error(detail::exception_id::type_mismatch, "type must be string, but is ", type_name());
+            detail::view::throw_type_error(302, "type must be string, but is ", type_name());
         }
         return {m_doc->str(*m_node), m_node->len};
     }
@@ -598,7 +598,7 @@ class basic_json_view
     {
         if (NLOHMANN_VIEW_UNLIKELY(!is_number()))
         {
-            detail::view::throw_type_error(detail::exception_id::type_mismatch, "type must be number, but is ", type_name());
+            detail::view::throw_type_error(302, "type must be number, but is ", type_name());
         }
         return {m_doc->str(*m_node), detail::view::number_length(*m_node)};
     }
@@ -855,7 +855,7 @@ class basic_json_view
     {
         if (NLOHMANN_VIEW_UNLIKELY(!is_boolean()))
         {
-            detail::view::throw_type_error(detail::exception_id::type_mismatch, "type must be boolean, but is ", type_name());
+            detail::view::throw_type_error(302, "type must be boolean, but is ", type_name());
         }
         return (m_node->flags & detail::view::node_flags::is_true) != 0;
     }
@@ -865,7 +865,7 @@ class basic_json_view
     {
         if (NLOHMANN_VIEW_UNLIKELY(m_node == nullptr))
         {
-            detail::view::throw_type_error(detail::exception_id::type_mismatch, "type must be number, but is ", type_name());
+            detail::view::throw_type_error(302, "type must be number, but is ", type_name());
         }
         return detail::view::arithmetic_value<T, BasicJsonType>(*m_doc, *m_node);
     }
@@ -874,7 +874,7 @@ class basic_json_view
     {
         if (NLOHMANN_VIEW_UNLIKELY(!is_null()))
         {
-            detail::view::throw_type_error(detail::exception_id::type_mismatch, "type must be null, but is ", type_name());
+            detail::view::throw_type_error(302, "type must be null, but is ", type_name());
         }
         return nullptr;
     }
@@ -1183,7 +1183,7 @@ class basic_json_document
     {
         if (NLOHMANN_VIEW_UNLIKELY(!m_data || m_data->discarded))
         {
-            detail::view::throw_type_error(detail::exception_id::document_image_unsupported, "cannot save a discarded json_document");
+            detail::view::throw_type_error(320, "cannot save a discarded json_document");
         }
         return detail::view::save_image(*m_data);
     }
@@ -1284,7 +1284,7 @@ class basic_json_document
                 const std::size_t idx = pointer_index(token);
                 if (idx >= 0xFFFFFFFFu)
                 {
-                    detail::view::throw_array_index_out_of_range(idx);
+                    detail::view::throw_out_of_range(401, detail::concat("array index ", std::to_string(idx), " is out of range"));
                 }
                 for (std::size_t i = 0; i < idx; ++i)
                 {
@@ -1340,7 +1340,7 @@ class basic_json_document
     {
         if (ptr.empty())
         {
-            detail::view::throw_out_of_range(detail::exception_id::patch_on_root, "JSON pointer has no parent");
+            detail::view::throw_out_of_range(405, "JSON pointer has no parent");
         }
         const view_type parent = root().at(ptr.parent_pointer());
         const auto& token = ptr.back();
@@ -1360,7 +1360,7 @@ class basic_json_document
         static_assert(Editable, "only an editable document can be edited: use basic_json_document<BasicJsonType, true> (json_editable_document)");
         if (NLOHMANN_VIEW_UNLIKELY(!m_data || m_data->discarded))
         {
-            detail::view::throw_invalid_iterator(detail::exception_id::iterator_from_other_value, "view does not belong to this document");
+            detail::view::throw_invalid_iterator(202, "view does not belong to this document");
         }
         return detail::view::editor<BasicJsonType, view_type>(*m_data);
     }
@@ -1377,7 +1377,7 @@ class basic_json_document
     {
         if (idx < 0)
         {
-            detail::view::throw_array_index_out_of_range(idx);
+            detail::view::throw_out_of_range(401, detail::concat("array index ", std::to_string(idx), " is out of range"));
         }
         return static_cast<std::size_t>(idx);
     }

include/nlohmann/ordered_map.hpp

@@ -125,19 +125,6 @@ private:
         Container::pop_back();
     }
 
-    /// @brief remove the entry with the given key; returns the number of removed entries
-    template<class K>
-    size_type erase_impl(const K& key)
-    {
-        const auto it = find_impl(*this, key);
-        if (it != this->end())
-        {
-            erase_at(it);
-            return 1;
-        }
-        return 0;
-    }
-
 public:
     template<class V, detail::enable_if_t<
                  detail::is_constructible<T, V>::value, int> = 0>
@@ -227,14 +214,26 @@ public:
 
     size_type erase(const key_type& key)
     {
-        return erase_impl(key);
+        const auto it = find_impl(*this, key);
+        if (it != this->end())
+        {
+            erase_at(it);
+            return 1;
+        }
+        return 0;
     }
 
     template<class KeyType, detail::enable_if_t<
                  detail::is_usable_as_key_type<key_compare, key_type, KeyType>::value, int> = 0>
     size_type erase(KeyType && key) // NOLINT(cppcoreguidelines-missing-std-forward)
     {
-        return erase_impl(key);
+        const auto it = find_impl(*this, key);
+        if (it != this->end())
+        {
+            erase_at(it);
+            return 1;
+        }
+        return 0;
     }
 
     iterator erase(iterator pos)

single_include/nlohmann/json_view.hpp

@@ -2819,52 +2819,32 @@ namespace view
 // Exceptions are thrown out of line, so that the accessors that may throw stay
 // small enough to be inlined.
 
-[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_type_error(exception_id id, const char* prefix, const char* type)
+[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_type_error(int id, const char* prefix, const char* type)
 {
     NLOHMANN_VIEW_THROW(type_error::create(id, concat(prefix, type), nullptr));
 }
 
-[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_type_error(exception_id id, const std::string& msg)
+[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_type_error(int id, const std::string& msg)
 {
     NLOHMANN_VIEW_THROW(type_error::create(id, msg, nullptr));
 }
 
-[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_out_of_range(exception_id id, const std::string& msg)
+[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_out_of_range(int id, const std::string& msg)
 {
     NLOHMANN_VIEW_THROW(out_of_range::create(id, msg, nullptr));
 }
 
-[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_invalid_iterator(exception_id id, const char* msg)
+[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_invalid_iterator(int id, const char* msg)
 {
     NLOHMANN_VIEW_THROW(invalid_iterator::create(id, msg, nullptr));
 }
 
 /// a parse error without a position (as those of json_pointer)
-[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_parse_error(exception_id id, const std::string& msg)
+[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_parse_error(int id, const std::string& msg)
 {
     NLOHMANN_VIEW_THROW(parse_error::create(id, 0, msg, nullptr));
 }
 
-/// out_of_range.401: the array index @a idx is not below the size
-template<typename Index>
-[[noreturn]] NLOHMANN_VIEW_NOINLINE void throw_array_index_out_of_range(const Index idx)
-{
-    throw_out_of_range(exception_id::array_index_out_of_range, concat("array index ", std::to_string(idx), " is out of range"));
-}
-
-/// out_of_range.402: the JSON Pointer token '-' of an array of @a size elements
-[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_array_index_past_end(std::size_t size)
-{
-    throw_out_of_range(exception_id::pointer_past_the_end_index, concat("array index '-' (", std::to_string(size), ") is out of range"));
-}
-
-/// out_of_range.403: the object has no member @a key
-template<typename Key>
-[[noreturn]] NLOHMANN_VIEW_NOINLINE void throw_key_not_found(const Key& key)
-{
-    throw_out_of_range(exception_id::key_not_found, concat("key '", key, "' not found"));
-}
-
 /// SAX handler of the error replay: accepts every event without storing it,
 /// and throws the exception the parser reports
 template<typename BasicJsonType>
@@ -2938,11 +2918,21 @@ struct throwing_sax
 
     /// (a template, as in json_sax_dom_parser: the parser passes the concrete
     /// exception type, which is thrown as is)
+    ///
+    /// It is not [[noreturn]]: sax_parse() recovers from an error if
+    /// parse_error() returns true, and MSVC reports that code as unreachable
+    /// (C4702) for a handler that never returns. The flag hides that from it.
     template<class Exception>
-    [[noreturn]] bool parse_error(std::size_t /*unused*/, const std::string& /*unused*/, const Exception& ex)
+    bool parse_error(std::size_t /*unused*/, const std::string& /*unused*/, const Exception& ex)
     {
-        NLOHMANN_VIEW_THROW(ex);
+        if (throwing)
+        {
+            NLOHMANN_VIEW_THROW(ex);
+        }
+        return false; // LCOV_EXCL_LINE
     }
+
+    bool throwing = true;
 };
 
 /*!
@@ -2963,7 +2953,7 @@ template<typename BasicJsonType>
     if (f.code == error_code::input_too_large)
     {
         // (the limit is detail::view::max_input_size(): 4 GiB minus 16 bytes)
-        NLOHMANN_VIEW_THROW(out_of_range::create(exception_id::document_too_large, "input of 4294967280 bytes or more is not supported by json_document", nullptr));
+        NLOHMANN_VIEW_THROW(out_of_range::create(416, "input of 4294967280 bytes or more is not supported by json_document", nullptr));
     }
     throwing_sax<BasicJsonType> sax;
     const bool accepted = BasicJsonType::sax_parse(src, src + size, &sax, input_format_t::json, true, ignore_comments, ignore_trailing_commas);
@@ -2983,7 +2973,7 @@ template<typename BasicJsonType>
         }
     }
     pos.chars_read_current_line = off + 1 - line_start;
-    NLOHMANN_VIEW_THROW(parse_error::create(exception_id::syntax_error, pos, "syntax error while parsing value", nullptr));
+    NLOHMANN_VIEW_THROW(parse_error::create(101, pos, "syntax error while parsing value", nullptr));
     // LCOV_EXCL_STOP
 }
 
@@ -3058,7 +3048,7 @@ inline std::size_t text_capacity(std::size_t cap, std::size_t used, std::size_t
     constexpr std::size_t limit = 0xFFFFFFFFu;
     if (NLOHMANN_VIEW_UNLIKELY(used > limit || n > limit - used))
     {
-        throw_out_of_range(exception_id::document_too_large, "edits of 4 GiB or more are not supported by json_document");
+        throw_out_of_range(416, "edits of 4 GiB or more are not supported by json_document");
     }
     const std::size_t needed = used + n;
     const std::size_t wanted = needed + (std::min)(limit - needed, std::size_t{256});
@@ -3109,7 +3099,7 @@ inline void reserve_moved(document_data& d)
     document_data::edit_state& e = edit_state_of(d);
     if (e.moved.size() >= 0xFFFFFFFFu)
     {
-        throw_out_of_range(exception_id::document_too_large, "more than 4294967295 edited arrays and objects are not supported by json_document"); // LCOV_EXCL_LINE
+        throw_out_of_range(416, "more than 4294967295 edited arrays and objects are not supported by json_document"); // LCOV_EXCL_LINE
     }
     if (e.moved.size() == e.moved.capacity() || e.moved_cap.size() == e.moved_cap.capacity())
     {
@@ -3739,6 +3729,11 @@ struct first_true<false, Conditions...> : std::integral_constant < int, 1 + firs
 inline void check_utf8(const char* s, std::size_t n)
 {
     const auto* const p = reinterpret_cast<const unsigned char*>(s); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
+    const auto hex = [](unsigned char c)
+    {
+        constexpr const char* digits = "0123456789ABCDEF";
+        return std::string{digits[c >> 4u], digits[c & 0xFu]};
+    };
     for (std::size_t i = 0; i < n;)
     {
         const unsigned char c = p[i];
@@ -3747,22 +3742,39 @@ inline void check_utf8(const char* s, std::size_t n)
             ++i;
             continue;
         }
-        const utf8_sequence_shape shape = utf8_shape(c);
-        const std::size_t len = shape.length;
-        if (len == 0)
+        std::size_t len = 0;
+        unsigned char lo = 0x80;
+        unsigned char hi = 0xBF;
+        if (c >= 0xC2 && c <= 0xDF)
         {
-            throw_type_error(exception_id::invalid_utf8, concat("invalid UTF-8 byte at index ", std::to_string(i), ": 0x", hex_byte(c)));
+            len = 2;
+        }
+        else if (c >= 0xE0 && c <= 0xEF)
+        {
+            len = 3;
+            lo = c == 0xE0 ? 0xA0 : 0x80;
+            hi = c == 0xED ? 0x9F : 0xBF;
+        }
+        else if (c >= 0xF0 && c <= 0xF4)
+        {
+            len = 4;
+            lo = c == 0xF0 ? 0x90 : 0x80;
+            hi = c == 0xF4 ? 0x8F : 0xBF;
+        }
+        else
+        {
+            throw_type_error(316, concat("invalid UTF-8 byte at index ", std::to_string(i), ": 0x", hex(c)));
         }
         for (std::size_t k = 1; k < len; ++k)
         {
             if (i + k == n)
             {
-                throw_type_error(exception_id::invalid_utf8, concat("incomplete UTF-8 string; last byte: 0x", hex_byte(p[n - 1])));
+                throw_type_error(316, concat("incomplete UTF-8 string; last byte: 0x", hex(p[n - 1])));
             }
             const unsigned char b = p[i + k];
-            if (b < (k == 1 ? shape.second_lo : 0x80) || b > (k == 1 ? shape.second_hi : 0xBF))
+            if (b < (k == 1 ? lo : 0x80) || b > (k == 1 ? hi : 0xBF))
             {
-                throw_type_error(exception_id::invalid_utf8, concat("invalid UTF-8 byte at index ", std::to_string(i + k), ": 0x", hex_byte(b)));
+                throw_type_error(316, concat("invalid UTF-8 byte at index ", std::to_string(i + k), ": 0x", hex(b)));
             }
         }
         i += len;
@@ -3810,7 +3822,7 @@ class editor
         node* const o = own(object);
         if (o->kind != static_cast<std::uint8_t>(value_t::object) && o->kind != static_cast<std::uint8_t>(value_t::null))
         {
-            throw_type_error(exception_id::subscript_wrong_type, "cannot use operator[] with a string argument with ", object.type_name());
+            throw_type_error(305, "cannot use operator[] with a string argument with ", object.type_name());
         }
         check_utf8(key.data(), key.size());
         const encoded e = encode(std::forward<V>(value));
@@ -3861,7 +3873,7 @@ class editor
         node* const a = own(array);
         if (a->kind != static_cast<std::uint8_t>(value_t::array))
         {
-            throw_type_error(exception_id::subscript_wrong_type, "cannot use operator[] with a numeric argument with ", array.type_name());
+            throw_type_error(305, "cannot use operator[] with a numeric argument with ", array.type_name());
         }
         check_index(idx, a->len);
         const encoded e = encode(std::forward<V>(value));
@@ -3878,7 +3890,7 @@ class editor
         node* const a = own(array);
         if (a->kind != static_cast<std::uint8_t>(value_t::array) && a->kind != static_cast<std::uint8_t>(value_t::null))
         {
-            throw_type_error(exception_id::push_back_wrong_type, "cannot use push_back() with ", array.type_name());
+            throw_type_error(308, "cannot use push_back() with ", array.type_name());
         }
         const encoded e = encode(std::forward<V>(value));
         if (a->kind == static_cast<std::uint8_t>(value_t::null))
@@ -3902,7 +3914,7 @@ class editor
         node* const a = own(array);
         if (a->kind != static_cast<std::uint8_t>(value_t::array))
         {
-            throw_type_error(exception_id::insert_wrong_type, "cannot use insert() with ", array.type_name());
+            throw_type_error(309, "cannot use insert() with ", array.type_name());
         }
         check_index(idx, a->len + 1);
         const encoded e = encode(std::forward<V>(value));
@@ -3922,7 +3934,7 @@ class editor
         node* const o = own(object);
         if (o->kind != static_cast<std::uint8_t>(value_t::object))
         {
-            throw_type_error(exception_id::erase_wrong_type, "cannot use erase() with ", object.type_name());
+            throw_type_error(307, "cannot use erase() with ", object.type_name());
         }
         for (const node* k = nav::first(m_doc, o), *end = nav::end(m_doc, o); k != end; k = document_data::after(k + 1))
         {
@@ -3940,7 +3952,7 @@ class editor
         node* const a = own(array);
         if (a->kind != static_cast<std::uint8_t>(value_t::array))
         {
-            throw_type_error(exception_id::erase_wrong_type, "cannot use erase() with ", array.type_name());
+            throw_type_error(307, "cannot use erase() with ", array.type_name());
         }
         check_index(idx, a->len);
         node* const h = block_of(m_doc, a, 0);
@@ -3963,7 +3975,7 @@ class editor
     {
         if (NLOHMANN_VIEW_UNLIKELY(v.m_doc != &m_doc || v.m_node == nullptr))
         {
-            throw_invalid_iterator(exception_id::iterator_from_other_value, "view does not belong to this document");
+            throw_invalid_iterator(202, "view does not belong to this document");
         }
         edit_state_of(m_doc);
         return const_cast<node*>(v.m_node); // NOLINT(cppcoreguidelines-pro-type-const-cast): the nodes belong to this document
@@ -3973,7 +3985,7 @@ class editor
     {
         if (idx >= limit)
         {
-            throw_array_index_out_of_range(idx);
+            throw_out_of_range(401, concat("array index ", std::to_string(idx), " is out of range"));
         }
     }
 
@@ -4114,7 +4126,7 @@ class editor
     {
         if (NLOHMANN_VIEW_UNLIKELY(v.m_node == nullptr))
         {
-            throw_type_error(exception_id::type_mismatch, "type must be a value, but is ", "discarded");
+            throw_type_error(302, "type must be a value, but is ", "discarded");
         }
         encoded r;
         if (!is_container(*v.m_node))
@@ -4264,7 +4276,7 @@ class editor
     {
         if (NLOHMANN_VIEW_UNLIKELY(len >= 0xFFFFFFFFu))
         {
-            throw_out_of_range(exception_id::document_too_large, "strings of 4 GiB or more are not supported by json_document"); // LCOV_EXCL_LINE
+            throw_out_of_range(416, "strings of 4 GiB or more are not supported by json_document"); // LCOV_EXCL_LINE
         }
         node n = plain_node(value_t::string);
         n.flags = static_cast<std::uint8_t>(n.flags | node_flags::edited);
@@ -4339,12 +4351,12 @@ class editor
                 return string_node(s.data(), s.size());
             }
             case value_t::binary:
-                throw_type_error(exception_id::document_binary_value, "cannot store a binary value in a json_document", "");
+                throw_type_error(319, "cannot store a binary value in a json_document", "");
             case value_t::discarded:
             case value_t::object:
             case value_t::array:
             default:
-                throw_type_error(exception_id::type_mismatch, "type must be a value, but is ", "discarded");
+                throw_type_error(302, "type must be a value, but is ", "discarded");
         }
     }
 
@@ -4972,7 +4984,7 @@ inline void compact_nodes(const document_data& d, std::size_t arena_size, std::v
 inline std::vector<std::uint8_t> save_image(const document_data& d)
 {
 #if !NLOHMANN_VIEW_LITTLE_ENDIAN
-    throw_type_error(exception_id::document_image_unsupported, "json_document images need a little-endian target"); // LCOV_EXCL_LINE
+    throw_type_error(320, "json_document images need a little-endian target"); // LCOV_EXCL_LINE
 #endif
     const std::size_t arena_size = d.arena_size;
     std::vector<node> compacted;
@@ -4989,7 +5001,7 @@ inline std::vector<std::uint8_t> save_image(const document_data& d)
     if (NLOHMANN_VIEW_UNLIKELY(text_size >= image_limit || total_arena >= image_limit || count >= image_limit))
     {
         // LCOV_EXCL_START (4 GiB)
-        throw_out_of_range(exception_id::document_too_large, "images of 4 GiB or more are not supported by json_document");
+        throw_out_of_range(416, "images of 4 GiB or more are not supported by json_document");
         // LCOV_EXCL_STOP
     }
     image_header h{};
@@ -5446,7 +5458,7 @@ inline bool check_image(const node* nodes, std::size_t count, const unsigned cha
 
 [[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_invalid_image(const char* what)
 {
-    throw_parse_error(exception_id::invalid_document_image, concat("invalid json_document image: ", what));
+    throw_parse_error(116, concat("invalid json_document image: ", what));
 }
 
 /// Read an image into d. The text and the decoded strings stay in the image;
@@ -5454,7 +5466,7 @@ inline bool check_image(const node* nodes, std::size_t count, const unsigned cha
 inline void load_image(document_data& d, const std::uint8_t* image, std::size_t size, image_check check)
 {
 #if !NLOHMANN_VIEW_LITTLE_ENDIAN
-    throw_type_error(exception_id::document_image_unsupported, "json_document images need a little-endian target"); // LCOV_EXCL_LINE
+    throw_type_error(320, "json_document images need a little-endian target"); // LCOV_EXCL_LINE
 #endif
     if (image == nullptr || size < sizeof(image_header))
     {
@@ -5842,7 +5854,7 @@ class view_iterator
     {
         if (NLOHMANN_VIEW_UNLIKELY(m_value_offset == 0))
         {
-            throw_invalid_iterator(exception_id::iterator_key_not_object, "cannot use key() for non-object iterators");
+            throw_invalid_iterator(207, "cannot use key() for non-object iterators");
         }
         return string_view_t(m_doc->str(*m_pos), m_pos->len);
     }
@@ -6234,15 +6246,51 @@ enum class pointer_mode
     contains,  ///< as basic_json::contains(): no exceptions at all
 };
 
-/// the outcome of reading an array index: detail::parse_array_index(), shared with json_pointer
-using index_status = detail::array_index_status;
+/// the outcome of reading an array index from a reference token
+enum class index_status
+{
+    ok,
+    leading_zero, ///< parse_error.106
+    not_number,   ///< parse_error.109
+    unresolved,   ///< out_of_range.404
+    too_large,    ///< out_of_range.410
+};
 
 /// reads an array index like json_pointer::array_index (RFC 6901, Sect. 4),
 /// but reports errors instead of throwing them
 template<typename StringType>
 index_status array_index(const StringType& s, std::size_t& idx) noexcept
 {
-    return detail::parse_array_index(s, idx);
+    // as basic_json: the empty token and "-" (past the last element) are
+    // unresolved rather than not a number
+    if (s.empty() || (s.size() == 1 && s[0] == '-'))
+    {
+        return index_status::unresolved;
+    }
+    if (s[0] < '0' || s[0] > '9')
+    {
+        return index_status::not_number;
+    }
+    if (s.size() > 1 && s[0] == '0')
+    {
+        return index_status::leading_zero;
+    }
+    std::uint64_t v = 0;
+    for (const auto c : s)
+    {
+        const auto d = static_cast<unsigned>(static_cast<unsigned char>(c)) - '0';
+        if (d > 9 || v > ((std::numeric_limits<std::uint64_t>::max)() - d) / 10)
+        {
+            return index_status::unresolved; // not a number, or beyond unsigned long long
+        }
+        v = (v * 10) + d;
+    }
+    if (v >= (std::numeric_limits<std::size_t>::max)()) // (std::size_t converts to std::uint64_t implicitly)
+    {
+        return index_status::too_large;
+    }
+    idx = static_cast<std::size_t>(v);
+    return index_status::ok;
 }
 
 /// throws the exception json_pointer::array_index throws for this status
@@ -6252,15 +6300,15 @@ template<typename StringType>
     switch (status)
     {
         case index_status::leading_zero:
-            throw_parse_error(exception_id::pointer_index_leading_zero, concat("array index '", s, "' must not begin with '0'"));
-        case index_status::not_a_number:
-            throw_parse_error(exception_id::pointer_index_not_a_number, concat("array index '", s, "' is not a number"));
-        case index_status::exceeds_size_type:
-            throw_out_of_range(exception_id::value_out_of_range, concat("array index ", s, " exceeds size_type")); // LCOV_EXCL_LINE
+            throw_parse_error(106, concat("array index '", s, "' must not begin with '0'"));
+        case index_status::not_number:
+            throw_parse_error(109, concat("array index '", s, "' is not a number"));
+        case index_status::too_large:
+            throw_out_of_range(410, concat("array index ", s, " exceeds size_type")); // LCOV_EXCL_LINE
         case index_status::unresolved:
         case index_status::ok:
         default:
-            throw_out_of_range(exception_id::pointer_unresolved, concat("unresolved reference token '", s, "'"));
+            throw_out_of_range(404, concat("unresolved reference token '", s, "'"));
     }
 }
 
@@ -6285,7 +6333,7 @@ View resolve_pointer(View cur, const Tokens& tokens, pointer_mode mode)
             {
                 if (mode == pointer_mode::checked)
                 {
-                    throw_key_not_found(token);
+                    throw_out_of_range(403, concat("key '", token, "' not found"));
                 }
                 return View();
             }
@@ -6297,7 +6345,7 @@ View resolve_pointer(View cur, const Tokens& tokens, pointer_mode mode)
             {
                 if (throwing)
                 {
-                    throw_array_index_past_end(cur.size());
+                    throw_out_of_range(402, concat("array index '-' (", std::to_string(cur.size()), ") is out of range"));
                 }
                 return View();
             }
@@ -6305,7 +6353,7 @@ View resolve_pointer(View cur, const Tokens& tokens, pointer_mode mode)
             const index_status status = array_index(token, idx);
             if (status != index_status::ok)
             {
-                const bool parse_error = status == index_status::leading_zero || status == index_status::not_a_number;
+                const bool parse_error = status == index_status::leading_zero || status == index_status::not_number;
                 if (throwing || (mode == pointer_mode::value && parse_error))
                 {
                     throw_array_index_error(status, token);
@@ -6316,7 +6364,7 @@ View resolve_pointer(View cur, const Tokens& tokens, pointer_mode mode)
             {
                 if (mode == pointer_mode::checked)
                 {
-                    throw_array_index_out_of_range(idx);
+                    throw_out_of_range(401, concat("array index ", std::to_string(idx), " is out of range"));
                 }
                 return View();
             }
@@ -6326,7 +6374,7 @@ View resolve_pointer(View cur, const Tokens& tokens, pointer_mode mode)
         {
             if (throwing)
             {
-                throw_out_of_range(exception_id::pointer_unresolved, concat("unresolved reference token '", token, "'"));
+                throw_out_of_range(404, concat("unresolved reference token '", token, "'"));
             }
             return View();
         }
@@ -7419,17 +7467,21 @@ class view_serializer
             m_out.put("null");
             return;
         }
-        write_float(x, std::integral_constant<bool, ::nlohmann::detail::has_native_float_format<number_float_t>::value>());
+        static constexpr bool is_ieee_single_or_double
+            = std::numeric_limits<number_float_t>::is_iec559
+              && ((std::numeric_limits<number_float_t>::digits == 24 && std::numeric_limits<number_float_t>::max_exponent == 128)
+                  || (std::numeric_limits<number_float_t>::digits == 53 && std::numeric_limits<number_float_t>::max_exponent == 1024));
+        write_float(x, std::integral_constant<bool, is_ieee_single_or_double>());
     }
 
-    void write_float(number_float_t x, std::true_type /*has_native_float_format*/)
+    void write_float(number_float_t x, std::true_type /*is_ieee_single_or_double*/)
     {
         std::array<char, 64> buf{};
         const char* const end = ::nlohmann::detail::to_chars(buf.data(), buf.data() + buf.size(), x);
         m_out.put(buf.data(), static_cast<std::size_t>(end - buf.data()));
     }
 
-    void write_float(number_float_t x, std::false_type /*has_native_float_format*/)
+    void write_float(number_float_t x, std::false_type /*is_ieee_single_or_double*/)
     {
         // other types (e.g. long double) are rare: the library writes them
         const string_t s = BasicJsonType(x).dump();
@@ -7490,8 +7542,8 @@ class view_serializer
 
     void write_u_escape(std::uint32_t u)
     {
-        std::array<char, 6> e = {{'\\', 'u', 0, 0, 0, 0}};
-        hex4_lower(static_cast<std::uint16_t>(u), &e[2]);
+        static constexpr const char* hex = "0123456789abcdef";
+        const std::array<char, 6> e = {{'\\', 'u', hex[(u >> 12u) & 0xFu], hex[(u >> 8u) & 0xFu], hex[(u >> 4u) & 0xFu], hex[u & 0xFu]}};
         m_out.put(e.data(), e.size());
     }
 
@@ -7688,7 +7740,7 @@ NLOHMANN_VIEW_ALWAYS_INLINE T arithmetic_value(const document_data& d, const nod
         case value_t::binary:
         case value_t::discarded:
         default:
-            throw_type_error(exception_id::type_mismatch, "type must be number, but is ", value_type_name(static_cast<value_t>(n.kind)));
+            throw_type_error(302, "type must be number, but is ", value_type_name(static_cast<value_t>(n.kind)));
     }
 }
 
@@ -7698,7 +7750,7 @@ std::vector<U, A> vector_value(const View& v)
 {
     if (NLOHMANN_VIEW_UNLIKELY(!v.is_array()))
     {
-        throw_type_error(exception_id::type_mismatch, "type must be array, but is ", v.type_name());
+        throw_type_error(302, "type must be array, but is ", v.type_name());
     }
     std::vector<U, A> r;
     r.reserve(v.size());
@@ -7716,7 +7768,7 @@ Map map_value(const View& v)
 {
     if (NLOHMANN_VIEW_UNLIKELY(!v.is_object()))
     {
-        throw_type_error(exception_id::type_mismatch, "type must be object, but is ", v.type_name());
+        throw_type_error(302, "type must be object, but is ", v.type_name());
     }
     Map r;
     for (auto it = v.begin(); it != v.end(); ++it)
@@ -7955,7 +8007,7 @@ class basic_json_view
             {
                 return {};
             }
-            detail::view::throw_type_error(detail::exception_id::subscript_wrong_type, "cannot use operator[] with a string argument with ", type_name());
+            detail::view::throw_type_error(305, "cannot use operator[] with a string argument with ", type_name());
         }
         return lookup(key);
     }
@@ -7981,7 +8033,7 @@ class basic_json_view
             {
                 return {};
             }
-            detail::view::throw_type_error(detail::exception_id::subscript_wrong_type, "cannot use operator[] with a numeric argument with ", type_name());
+            detail::view::throw_type_error(305, "cannot use operator[] with a numeric argument with ", type_name());
         }
         return idx < m_node->len ? element(idx) : basic_json_view();
     }
@@ -8014,12 +8066,12 @@ class basic_json_view
     {
         if (NLOHMANN_VIEW_UNLIKELY(!is_object()))
         {
-            detail::view::throw_type_error(detail::exception_id::at_wrong_type, "cannot use at() with ", type_name());
+            detail::view::throw_type_error(304, "cannot use at() with ", type_name());
         }
         const basic_json_view r = lookup(key);
         if (NLOHMANN_VIEW_UNLIKELY(r.is_discarded()))
         {
-            detail::view::throw_key_not_found(std::string(key.data(), key.size()));
+            detail::view::throw_out_of_range(403, detail::concat("key '", std::string(key.data(), key.size()), "' not found"));
         }
         return r;
     }
@@ -8040,11 +8092,11 @@ class basic_json_view
     {
         if (NLOHMANN_VIEW_UNLIKELY(!is_array()))
         {
-            detail::view::throw_type_error(detail::exception_id::at_wrong_type, "cannot use at() with ", type_name());
+            detail::view::throw_type_error(304, "cannot use at() with ", type_name());
         }
         if (NLOHMANN_VIEW_UNLIKELY(idx >= m_node->len))
         {
-            detail::view::throw_array_index_out_of_range(idx);
+            detail::view::throw_out_of_range(401, detail::concat("array index ", std::to_string(idx), " is out of range"));
         }
         return element(idx);
     }
@@ -8072,7 +8124,7 @@ class basic_json_view
     {
         if (NLOHMANN_VIEW_UNLIKELY(!is_object()))
         {
-            detail::view::throw_type_error(detail::exception_id::value_wrong_type, "cannot use value() with ", type_name());
+            detail::view::throw_type_error(306, "cannot use value() with ", type_name());
         }
         const basic_json_view r = lookup(key);
         return r.is_discarded() ? default_value : r.template get<T>();
@@ -8097,7 +8149,7 @@ class basic_json_view
     {
         if (NLOHMANN_VIEW_UNLIKELY(!is_structured()))
         {
-            detail::view::throw_type_error(detail::exception_id::value_wrong_type, "cannot use value() with ", type_name());
+            detail::view::throw_type_error(306, "cannot use value() with ", type_name());
         }
         const basic_json_view r = detail::view::resolve_pointer(*this, detail::json_pointer_access::reference_tokens(ptr), detail::view::pointer_mode::value);
         return r.is_discarded() ? default_value : r.template get<T>();
@@ -8120,7 +8172,7 @@ class basic_json_view
         const iterator it = begin();
         if (NLOHMANN_VIEW_UNLIKELY(it == end()))
         {
-            detail::view::throw_invalid_iterator(detail::exception_id::iterator_value_unavailable, "cannot get value");
+            detail::view::throw_invalid_iterator(214, "cannot get value");
         }
         return *it;
     }
@@ -8273,7 +8325,7 @@ class basic_json_view
     {
         if (NLOHMANN_VIEW_UNLIKELY(!is_string()))
         {
-            detail::view::throw_type_error(detail::exception_id::type_mismatch, "type must be string, but is ", type_name());
+            detail::view::throw_type_error(302, "type must be string, but is ", type_name());
         }
         return {m_doc->str(*m_node), m_node->len};
     }
@@ -8285,7 +8337,7 @@ class basic_json_view
     {
         if (NLOHMANN_VIEW_UNLIKELY(!is_number()))
         {
-            detail::view::throw_type_error(detail::exception_id::type_mismatch, "type must be number, but is ", type_name());
+            detail::view::throw_type_error(302, "type must be number, but is ", type_name());
         }
         return {m_doc->str(*m_node), detail::view::number_length(*m_node)};
     }
@@ -8542,7 +8594,7 @@ class basic_json_view
     {
         if (NLOHMANN_VIEW_UNLIKELY(!is_boolean()))
         {
-            detail::view::throw_type_error(detail::exception_id::type_mismatch, "type must be boolean, but is ", type_name());
+            detail::view::throw_type_error(302, "type must be boolean, but is ", type_name());
         }
         return (m_node->flags & detail::view::node_flags::is_true) != 0;
     }
@@ -8552,7 +8604,7 @@ class basic_json_view
     {
         if (NLOHMANN_VIEW_UNLIKELY(m_node == nullptr))
         {
-            detail::view::throw_type_error(detail::exception_id::type_mismatch, "type must be number, but is ", type_name());
+            detail::view::throw_type_error(302, "type must be number, but is ", type_name());
         }
         return detail::view::arithmetic_value<T, BasicJsonType>(*m_doc, *m_node);
     }
@@ -8561,7 +8613,7 @@ class basic_json_view
     {
         if (NLOHMANN_VIEW_UNLIKELY(!is_null()))
         {
-            detail::view::throw_type_error(detail::exception_id::type_mismatch, "type must be null, but is ", type_name());
+            detail::view::throw_type_error(302, "type must be null, but is ", type_name());
         }
         return nullptr;
     }
@@ -8870,7 +8922,7 @@ class basic_json_document
     {
         if (NLOHMANN_VIEW_UNLIKELY(!m_data || m_data->discarded))
         {
-            detail::view::throw_type_error(detail::exception_id::document_image_unsupported, "cannot save a discarded json_document");
+            detail::view::throw_type_error(320, "cannot save a discarded json_document");
         }
         return detail::view::save_image(*m_data);
     }
@@ -8971,7 +9023,7 @@ class basic_json_document
                 const std::size_t idx = pointer_index(token);
                 if (idx >= 0xFFFFFFFFu)
                 {
-                    detail::view::throw_array_index_out_of_range(idx);
+                    detail::view::throw_out_of_range(401, detail::concat("array index ", std::to_string(idx), " is out of range"));
                 }
                 for (std::size_t i = 0; i < idx; ++i)
                 {
@@ -9027,7 +9079,7 @@ class basic_json_document
     {
         if (ptr.empty())
         {
-            detail::view::throw_out_of_range(detail::exception_id::patch_on_root, "JSON pointer has no parent");
+            detail::view::throw_out_of_range(405, "JSON pointer has no parent");
         }
         const view_type parent = root().at(ptr.parent_pointer());
         const auto& token = ptr.back();
@@ -9047,7 +9099,7 @@ class basic_json_document
         static_assert(Editable, "only an editable document can be edited: use basic_json_document<BasicJsonType, true> (json_editable_document)");
         if (NLOHMANN_VIEW_UNLIKELY(!m_data || m_data->discarded))
         {
-            detail::view::throw_invalid_iterator(detail::exception_id::iterator_from_other_value, "view does not belong to this document");
+            detail::view::throw_invalid_iterator(202, "view does not belong to this document");
         }
         return detail::view::editor<BasicJsonType, view_type>(*m_data);
     }
@@ -9064,7 +9116,7 @@ class basic_json_document
     {
         if (idx < 0)
         {
-            detail::view::throw_array_index_out_of_range(idx);
+            detail::view::throw_out_of_range(401, detail::concat("array index ", std::to_string(idx), " is out of range"));
         }
         return static_cast<std::size_t>(idx);
     }

tests/src/fuzzer-parse_bjdata.cpp

@@ -47,6 +47,10 @@ dumps is stable under exactly the same values that break operator==.
 The unit tests run the same checks on a fixed corpus (see the "BJData round-trip
 invariants" test case), so keep both in sync.
 
+Furthermore, it reads data with a SAX parser that recovers from every error
+and checks that the events are balanced, that reading ends, and that it
+reports an error exactly when from_bjdata() fails (see #3989).
+
 Finally, it reads the data from a stream, a non-contiguous iterator range, and
 a FILE*, which read byte by byte, and checks that each gives the same value or
 error (id, message, byte position) as the pointer range, which takes the bulk
@@ -57,6 +61,7 @@ drivers.
 */
 
 #include "fuzzer_common.hpp"
+#include "fuzzer-recovering_checker.hpp"
 
 namespace
 {
@@ -81,6 +86,9 @@ static bool is_value_stable(const json& lhs, const json& rhs)
 // see http://llvm.org/docs/LibFuzzer.html
 extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
 {
+    // recover from all errors, reading from memory and from a stream
+    const bool recovered_without_errors = check_recovering_parse(data, size, json::input_format_t::bjdata).errors == 0;
+
     std::vector<uint8_t> const vec1(data, data + size);
 
     // step 0: parse input without exceptions
@@ -99,6 +107,9 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
         // without exceptions, the same input must give the same value
         assert(!noexcept_threw && !j_noexcept.is_discarded() && same_value(j_noexcept, j1));
 
+        // the recovering parser must not have reported an error either
+        assert(recovered_without_errors);
+
         try
         {
             // step 2.1: round trip without adding size annotations to container types
@@ -132,6 +143,7 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
     {
         // parse errors are ok, because input may be random bytes
         assert(parsed || noexcept_threw || j_noexcept.is_discarded());
+        assert(parsed || !recovered_without_errors);
     }
     catch (const json::type_error&)
     {
@@ -142,6 +154,7 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
     {
         // out of range errors may happen if provided sizes are excessive
         assert(parsed || noexcept_threw || j_noexcept.is_discarded());
+        assert(parsed || !recovered_without_errors);
     }
 
     // step 3: the other input adapters must read the data alike

tests/src/fuzzer-parse_bon8.cpp

@@ -17,6 +17,10 @@ array data, it performs the following steps:
 - j2 = from_bon8(vec)
 - assert(to_bon8(j2) == vec)
 
+Furthermore, it reads data with a SAX parser that recovers from every error
+and checks that the events are balanced, that reading ends, and that it
+reports an error exactly when from_bon8() fails (see #3989).
+
 Finally, it reads the data from a stream, a non-contiguous iterator range, and
 a FILE*, which read byte by byte, and checks that each gives the same value or
 error (id, message, byte position) as the pointer range, which takes the bulk
@@ -27,6 +31,7 @@ drivers.
 */
 
 #include "fuzzer_common.hpp"
+#include "fuzzer-recovering_checker.hpp"
 
 namespace
 {
@@ -44,6 +49,9 @@ struct read_bon8
 // see http://llvm.org/docs/LibFuzzer.html
 extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
 {
+    // recover from all errors, reading from memory and from a stream
+    const bool recovered_without_errors = check_recovering_parse(data, size, json::input_format_t::bon8).errors == 0;
+
     std::vector<uint8_t> const vec1(data, data + size);
 
     // step 0: parse input without exceptions
@@ -62,6 +70,9 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
         // without exceptions, the same input must give the same value
         assert(!noexcept_threw && !j_noexcept.is_discarded() && same_value(j_noexcept, j1));
 
+        // the recovering parser must not have reported an error either
+        assert(recovered_without_errors);
+
         try
         {
             // step 2: round trip
@@ -83,6 +94,7 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
     {
         // parse errors are ok, because input may be random bytes
         assert(parsed || noexcept_threw || j_noexcept.is_discarded());
+        assert(parsed || !recovered_without_errors);
     }
     catch (const json::type_error&)
     {
@@ -93,6 +105,7 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
     {
         // out of range errors may happen if provided sizes are excessive
         assert(parsed || noexcept_threw || j_noexcept.is_discarded());
+        assert(parsed || !recovered_without_errors);
     }
 
     // step 3: the other input adapters must read the data alike

tests/src/fuzzer-parse_bson.cpp

@@ -17,6 +17,10 @@ array data, it performs the following steps:
 - j2 = from_bson(vec)
 - assert(to_bson(j2) == vec)
 
+Furthermore, it reads data with a SAX parser that recovers from every error
+and checks that the events are balanced, that reading ends, and that it
+reports an error exactly when from_bson() fails (see #3989).
+
 Finally, it reads the data from a stream, a non-contiguous iterator range, and
 a FILE*, which read byte by byte, and checks that each gives the same value or
 error (id, message, byte position) as the pointer range, which takes the bulk
@@ -27,6 +31,7 @@ drivers.
 */
 
 #include "fuzzer_common.hpp"
+#include "fuzzer-recovering_checker.hpp"
 
 namespace
 {
@@ -44,6 +49,9 @@ struct read_bson
 // see http://llvm.org/docs/LibFuzzer.html
 extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
 {
+    // recover from all errors, reading from memory and from a stream
+    const bool recovered_without_errors = check_recovering_parse(data, size, json::input_format_t::bson).errors == 0;
+
     std::vector<uint8_t> const vec1(data, data + size);
 
     // step 0: parse input without exceptions
@@ -62,6 +70,9 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
         // without exceptions, the same input must give the same value
         assert(!noexcept_threw && !j_noexcept.is_discarded() && same_value(j_noexcept, j1));
 
+        // the recovering parser must not have reported an error either
+        assert(recovered_without_errors);
+
         try
         {
             // step 2: round trip
@@ -83,6 +94,7 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
     {
         // parse errors are ok, because input may be random bytes
         assert(parsed || noexcept_threw || j_noexcept.is_discarded());
+        assert(parsed || !recovered_without_errors);
     }
     catch (const json::type_error&)
     {
@@ -93,6 +105,7 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
     {
         // out of range errors can occur during parsing, too
         assert(parsed || noexcept_threw || j_noexcept.is_discarded());
+        assert(parsed || !recovered_without_errors);
     }
 
     // step 3: the other input adapters must read the data alike

tests/src/fuzzer-parse_cbor.cpp

@@ -17,6 +17,10 @@ array data, it performs the following steps:
 - j2 = from_cbor(vec)
 - assert(to_cbor(j2) == vec)
 
+Furthermore, it reads data with a SAX parser that recovers from every error
+and checks that the events are balanced, that reading ends, and that it
+reports an error exactly when from_cbor() fails (see #3989).
+
 Finally, it reads the data from a stream, a non-contiguous iterator range, and
 a FILE*, which read byte by byte, and checks that each gives the same value or
 error (id, message, byte position) as the pointer range, which takes the bulk
@@ -27,6 +31,7 @@ drivers.
 */
 
 #include "fuzzer_common.hpp"
+#include "fuzzer-recovering_checker.hpp"
 
 namespace
 {
@@ -44,6 +49,9 @@ struct read_cbor
 // see http://llvm.org/docs/LibFuzzer.html
 extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
 {
+    // recover from all errors, reading from memory and from a stream
+    const bool recovered_without_errors = check_recovering_parse(data, size, json::input_format_t::cbor).errors == 0;
+
     std::vector<uint8_t> const vec1(data, data + size);
 
     // step 0: parse input without exceptions
@@ -62,6 +70,9 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
         // without exceptions, the same input must give the same value
         assert(!noexcept_threw && !j_noexcept.is_discarded() && same_value(j_noexcept, j1));
 
+        // the recovering parser must not have reported an error either
+        assert(recovered_without_errors);
+
         try
         {
             // step 2: round trip
@@ -83,6 +94,7 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
     {
         // parse errors are ok, because input may be random bytes
         assert(parsed || noexcept_threw || j_noexcept.is_discarded());
+        assert(parsed || !recovered_without_errors);
     }
     catch (const json::type_error&)
     {
@@ -93,6 +105,7 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
     {
         // out of range errors can occur during parsing, too
         assert(parsed || noexcept_threw || j_noexcept.is_discarded());
+        assert(parsed || !recovered_without_errors);
     }
 
     // step 3: the other input adapters must read the data alike

tests/src/fuzzer-parse_json.cpp

@@ -24,11 +24,16 @@ array data, it performs the following steps:
 The pointer range takes the bulk fast paths, the other input adapters read byte
 by byte; see check_input_adapters().
 
+Furthermore, it parses data with a SAX parser that recovers from every error
+and checks that the events are balanced, that parsing ends, and that valid
+input is parsed without errors (see #3989).
+
 The provided function `LLVMFuzzerTestOneInput` can be used in different fuzzer
 drivers.
 */
 
 #include "fuzzer_common.hpp"
+#include "fuzzer-recovering_checker.hpp"
 
 namespace
 {
@@ -59,6 +64,13 @@ struct accept_json
 // see http://llvm.org/docs/LibFuzzer.html
 extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
 {
+    // recover from all errors, reading from memory and from a stream
+    {
+        const auto checker = check_recovering_parse(data, size, json::input_format_t::json);
+        assert(checker.events <= (4 * size) + 4);
+        assert((checker.errors == 0) == json::accept(data, data + size));
+    }
+
     // step 0: parse input without exceptions
     bool noexcept_threw = false;
     json const j_noexcept = parse_without_exceptions([&] { return json::parse(data, data + size, nullptr, false); }, noexcept_threw);

tests/src/fuzzer-parse_msgpack.cpp

@@ -17,6 +17,10 @@ array data, it performs the following steps:
 - j2 = from_msgpack(vec)
 - assert(to_msgpack(j2) == vec)
 
+Furthermore, it reads data with a SAX parser that recovers from every error
+and checks that the events are balanced, that reading ends, and that it
+reports an error exactly when from_msgpack() fails (see #3989).
+
 Finally, it reads the data from a stream, a non-contiguous iterator range, and
 a FILE*, which read byte by byte, and checks that each gives the same value or
 error (id, message, byte position) as the pointer range, which takes the bulk
@@ -27,6 +31,7 @@ drivers.
 */
 
 #include "fuzzer_common.hpp"
+#include "fuzzer-recovering_checker.hpp"
 
 namespace
 {
@@ -44,6 +49,9 @@ struct read_msgpack
 // see http://llvm.org/docs/LibFuzzer.html
 extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
 {
+    // recover from all errors, reading from memory and from a stream
+    const bool recovered_without_errors = check_recovering_parse(data, size, json::input_format_t::msgpack).errors == 0;
+
     std::vector<uint8_t> const vec1(data, data + size);
 
     // step 0: parse input without exceptions
@@ -62,6 +70,9 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
         // without exceptions, the same input must give the same value
         assert(!noexcept_threw && !j_noexcept.is_discarded() && same_value(j_noexcept, j1));
 
+        // the recovering parser must not have reported an error either
+        assert(recovered_without_errors);
+
         try
         {
             // step 2: round trip
@@ -83,6 +94,7 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
     {
         // parse errors are ok, because input may be random bytes
         assert(parsed || noexcept_threw || j_noexcept.is_discarded());
+        assert(parsed || !recovered_without_errors);
     }
     catch (const json::type_error&)
     {
@@ -93,6 +105,7 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
     {
         // out of range errors may happen if provided sizes are excessive
         assert(parsed || noexcept_threw || j_noexcept.is_discarded());
+        assert(parsed || !recovered_without_errors);
     }
 
     // step 3: the other input adapters must read the data alike

tests/src/fuzzer-parse_ubjson.cpp

@@ -26,6 +26,10 @@ array data, it performs the following steps:
 The unit tests run the same checks on a fixed corpus (see the "UBJSON round-trip
 invariants" test case), so keep both in sync.
 
+Furthermore, it reads data with a SAX parser that recovers from every error
+and checks that the events are balanced, that reading ends, and that it
+reports an error exactly when from_ubjson() fails (see #3989).
+
 Finally, it reads the data from a stream, a non-contiguous iterator range, and
 a FILE*, which read byte by byte, and checks that each gives the same value or
 error (id, message, byte position) as the pointer range, which takes the bulk
@@ -36,6 +40,7 @@ drivers.
 */
 
 #include "fuzzer_common.hpp"
+#include "fuzzer-recovering_checker.hpp"
 
 namespace
 {
@@ -53,6 +58,9 @@ struct read_ubjson
 // see http://llvm.org/docs/LibFuzzer.html
 extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
 {
+    // recover from all errors, reading from memory and from a stream
+    const bool recovered_without_errors = check_recovering_parse(data, size, json::input_format_t::ubjson).errors == 0;
+
     std::vector<uint8_t> const vec1(data, data + size);
 
     // step 0: parse input without exceptions
@@ -71,6 +79,9 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
         // without exceptions, the same input must give the same value
         assert(!noexcept_threw && !j_noexcept.is_discarded() && same_value(j_noexcept, j1));
 
+        // the recovering parser must not have reported an error either
+        assert(recovered_without_errors);
+
         try
         {
             // step 2.1: round trip without adding size annotations to container types
@@ -102,6 +113,7 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
     {
         // parse errors are ok, because input may be random bytes
         assert(parsed || noexcept_threw || j_noexcept.is_discarded());
+        assert(parsed || !recovered_without_errors);
     }
     catch (const json::type_error&)
     {
@@ -112,6 +124,7 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
     {
         // out of range errors may happen if provided sizes are excessive
         assert(parsed || noexcept_threw || j_noexcept.is_discarded());
+        assert(parsed || !recovered_without_errors);
     }
 
     // step 3: the other input adapters must read the data alike

tests/src/fuzzer-recovering_checker.hpp

@@ -0,0 +1,154 @@
+//     __ _____ _____ _____
+//  __|  |   __|     |   | |  JSON for Modern C++ (supporting code)
+// |  |  |__   |  |  | | | |  version 3.12.0
+// |_____|_____|_____|_|___|  https://github.com/nlohmann/json
+//
+// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
+// SPDX-License-Identifier: MIT
+
+#pragma once
+
+#include <cassert>
+#include <cstddef>
+#include <cstdint>
+#include <sstream>
+#include <string>
+#include <vector>
+#include <nlohmann/json.hpp>
+
+namespace
+{
+// a SAX parser that recovers from every error and checks that the events are
+// balanced and that every key is followed by exactly one value
+class recovering_checker : public nlohmann::json_sax<nlohmann::json>
+{
+  public:
+    bool null() override
+    {
+        return value();
+    }
+
+    bool boolean(bool /*val*/) override
+    {
+        return value();
+    }
+
+    bool number_integer(number_integer_t /*val*/) override
+    {
+        return value();
+    }
+
+    bool number_unsigned(number_unsigned_t /*val*/) override
+    {
+        return value();
+    }
+
+    bool number_float(number_float_t /*val*/, const string_t& /*s*/) override
+    {
+        return value();
+    }
+
+    bool string(string_t& /*val*/) override
+    {
+        return value();
+    }
+
+    bool binary(binary_t& /*val*/) override
+    {
+        return value();
+    }
+
+    bool start_object(std::size_t /*elements*/) override
+    {
+        value();
+        stack.push_back('o');
+        return true;
+    }
+
+    bool key(string_t& /*val*/) override
+    {
+        ++events;
+        assert(!stack.empty() && stack.back() == 'o');
+        stack.back() = 'v';
+        return true;
+    }
+
+    bool end_object() override
+    {
+        ++events;
+        assert(!stack.empty() && stack.back() == 'o');
+        stack.pop_back();
+        return true;
+    }
+
+    bool start_array(std::size_t /*elements*/) override
+    {
+        value();
+        stack.push_back('a');
+        return true;
+    }
+
+    bool end_array() override
+    {
+        ++events;
+        assert(!stack.empty() && stack.back() == 'a');
+        stack.pop_back();
+        return true;
+    }
+
+    bool parse_error(std::size_t /*position*/, const std::string& /*last_token*/, const nlohmann::detail::exception& /*ex*/) override
+    {
+        ++errors;
+        return true;
+    }
+
+    bool complete() const
+    {
+        return stack.empty();
+    }
+
+    std::size_t events = 0;
+    std::size_t errors = 0;
+
+  private:
+    bool value()
+    {
+        ++events;
+        if (!stack.empty())
+        {
+            // an array element, or the value of a key
+            assert(stack.back() != 'o');
+            if (stack.back() == 'v')
+            {
+                stack.back() = 'o';
+            }
+        }
+        return true;
+    }
+
+    // 'a' for an array, 'o' for an object that expects a key, 'v' for an
+    // object that expects the value of a key
+    std::vector<char> stack {}; // NOLINT(readability-redundant-member-init)
+};
+
+/// parses @a data with a recovering_checker from memory and from a stream,
+/// checks that both see the same, that the events are balanced, and that the
+/// number of errors is bounded, and returns the checker (see #3989)
+inline recovering_checker check_recovering_parse(const std::uint8_t* data, const std::size_t size, const nlohmann::json::input_format_t format)
+{
+    recovering_checker checker;
+    const bool ok = nlohmann::json::sax_parse(data, data + size, &checker, format);
+    assert(checker.complete());
+    assert(checker.errors <= size + 1);
+    assert(ok == (checker.errors == 0));
+
+    std::istringstream stream(std::string(reinterpret_cast<const char*>(data), size));
+    recovering_checker stream_checker;
+    assert(nlohmann::json::sax_parse(stream, &stream_checker, format) == ok);
+    assert(stream_checker.complete());
+    assert(stream_checker.events == checker.events);
+    assert(stream_checker.errors == checker.errors);
+
+    return checker;
+}
+} // namespace

tests/src/sax_event_loggers.hpp

@@ -131,11 +131,13 @@ struct SaxEventLogger : public nlohmann::json_sax<nlohmann::json>
     {
         errored = true;
         events.push_back("parse_error(" + std::to_string(position) + ")");
-        return false;
+        return recover;
     }
 
     std::vector<std::string> events {}; // NOLINT(readability-redundant-member-init)
     bool errored = false;
+    /// whether parse_error() asks the parser to recover from the error (see #3989)
+    bool recover = false;
 };
 
 struct SaxEventLoggerExitAfterStartObject : public SaxEventLogger

tests/src/unit-alt-string.cpp

@@ -415,6 +415,46 @@ TEST_CASE("alternative string type")
         CHECK(j2.dump() == R"({"/foo/0":"bar","/foo/1":"baz"})");
     }
 
+    SECTION("error recovery")
+    {
+        // a SAX parser that recovers from every error (see #3989)
+        struct recovering_parser : nlohmann::detail::json_sax_dom_parser<alt_json>
+        {
+            explicit recovering_parser(alt_json& j)
+                : nlohmann::detail::json_sax_dom_parser<alt_json>(j, false)
+            {}
+
+            // sax_parse() calls the SAX parser's own parse_error(), so hiding
+            // the one of the base class is what recovering takes
+            // NOLINTNEXTLINE(bugprone-derived-method-shadowing-base-method)
+            bool parse_error(std::size_t /*unused*/, const std::string& /*unused*/, const nlohmann::detail::exception& /*unused*/)
+            {
+                ++errors;
+                return true;
+            }
+
+            std::size_t errors = 0;
+        };
+
+        alt_json j;
+        recovering_parser sax(j);
+        // not inside CHECK(): MSVC reads the escape in a stringized raw string
+        const std::string input = R"([1., "a\qb", tru, {"k" 2}])";
+        CHECK(!alt_json::sax_parse(input, &sax));
+        CHECK(sax.errors == 4);
+        CHECK(j.dump() == R"([1,"aqb",null,{"k":2}])");
+
+        // a UBJSON high-precision number, a CBOR key that is not a string
+        alt_json u;
+        recovering_parser ubjson_sax(u);
+        CHECK(!alt_json::sax_parse(std::vector<std::uint8_t> {'[', 'H', 'i', 2, '1', '.', ']'}, &ubjson_sax, alt_json::input_format_t::ubjson));
+        CHECK(u.dump() == "[1]");
+        alt_json c;
+        recovering_parser cbor_sax(c);
+        CHECK(!alt_json::sax_parse(std::vector<std::uint8_t> {0xA2, 0x01, 0x02, 0x61, 'a', 0x03}, &cbor_sax, alt_json::input_format_t::cbor));
+        CHECK(c.dump() == R"({"a":3})");
+    }
+
     SECTION("conversion between basic_json specializations (#2649)")
     {
         // explicit conversions are always possible

tests/src/unit-class_lexer.cpp

@@ -2042,118 +2042,3 @@ TEST_CASE("string scanning kernels")
     CHECK(nlohmann::detail::read_eight_bytes(bytes.data() + 8) == 0xFF0F0E0D0C0B0A09u);
     CHECK(nlohmann::detail::read_eight_bytes(reinterpret_cast<const char*>(bytes.data()) + 3) == 0x0B0A090807060504u);
 }
-
-TEST_CASE("find_ascii_copyable_run agrees with is_ascii_copyable for every byte")
-{
-    // the SWAR mask in find_ascii_copyable_run() encodes the predicate a
-    // second time; check every byte value at every position of a 24-byte buffer
-    // (three words) and for several lengths, with the byte at the end or inside
-    for (unsigned int value = 0; value < 256; ++value)
-    {
-        for (std::size_t position = 0; position < 24; ++position)
-        {
-            std::array<unsigned char, 24> buffer{};
-            buffer.fill(static_cast<unsigned char>('a'));
-            buffer[position] = static_cast<unsigned char>(value);
-            const bool copyable = nlohmann::detail::is_ascii_copyable(static_cast<unsigned char>(value));
-            for (const std::size_t n :
-                    {
-                        position, position + 1, std::size_t{24}
-                    })
-            {
-                if (n > 24)
-                {
-                    continue;
-                }
-                const std::size_t expected = (copyable || n <= position) ? n : position;
-                CAPTURE(value)
-                CAPTURE(position)
-                CAPTURE(n)
-                CHECK(nlohmann::detail::find_ascii_copyable_run(buffer.data(), n) == expected);
-            }
-        }
-    }
-}
-
-TEST_CASE("UTF-8 validators accept the same sequences")
-{
-    // The library checks UTF-8 in several places: detail::decode (serializer),
-    // the per-lead-byte switch of the lexer (json::accept), validate_one_utf8
-    // (bulk string scan), utf8_shape, and the byte path of the BON8 reader.
-    // They must accept exactly the same sequences (RFC 3629, section 4).
-    using nlohmann::json;
-
-    // bytes for the third and fourth position: the first two pairs are checked
-    // against all validators, the others only against the cheap ones
-    const std::array<std::array<unsigned char, 2>, 6> tails =
-    {
-        {
-            {{0x80, 0x80}}, {{0xBF, 0xBF}}, {{0x7F, 0x80}}, {{0xC0, 0x80}}, {{0x80, 0x7F}}, {{0x80, 0xC0}}
-        }
-    };
-
-    for (unsigned lead = 0x80; lead <= 0xFF; ++lead)
-    {
-        const auto shape = nlohmann::detail::utf8_shape(static_cast<std::uint8_t>(lead));
-        for (unsigned second = 0; second <= 0xFF; ++second)
-        {
-            for (std::size_t t = 0; t < tails.size(); ++t)
-            {
-                const std::array<unsigned char, 4> bytes = {{static_cast<unsigned char>(lead), static_cast<unsigned char>(second), tails[t][0], tails[t][1]}};
-
-                // expectation from the shape: the sequence is bytes[0..length)
-                const std::size_t length = shape.length;
-                bool expected = length >= 2 && second >= shape.second_lo && second <= shape.second_hi;
-                for (std::size_t i = 2; expected && i < length; ++i)
-                {
-                    expected = bytes[i] >= 0x80 && bytes[i] <= 0xBF;
-                }
-
-                // the DFA of the serializer
-                bool dfa = length >= 2;
-                if (dfa)
-                {
-                    std::uint8_t state = nlohmann::detail::UTF8_ACCEPT;
-                    std::uint32_t codep = 0;
-                    for (std::size_t i = 0; i < length; ++i)
-                    {
-                        nlohmann::detail::decode(state, codep, bytes[i]);
-                    }
-                    dfa = state == nlohmann::detail::UTF8_ACCEPT;
-                }
-
-                // the bulk path: exactly the sequence, and one byte short
-                const std::size_t bulk = nlohmann::detail::validate_one_utf8(bytes.data(), length);
-                const std::size_t bulk_short = length > 0 ? nlohmann::detail::validate_one_utf8(bytes.data(), length - 1) : 0;
-
-                CAPTURE(lead)
-                CAPTURE(second)
-                CAPTURE(t)
-                CHECK((dfa == expected && bulk == (expected ? length : 0) && bulk_short == 0));
-
-                if (t < 2 && length >= 2)
-                {
-                    // the lexer's byte path
-                    const std::string sequence(reinterpret_cast<const char*>(bytes.data()), length); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
-                    CHECK(json::accept("\"" + sequence + "\"") == expected);
-
-                    // the byte path of the BON8 reader (a string ends at 0xFF)
-                    if (lead >= 0xC2 && lead <= 0xF7)
-                    {
-                        std::vector<std::uint8_t> packed(bytes.begin(), bytes.begin() + static_cast<std::ptrdiff_t>(length));
-                        packed.push_back(0xFF);
-                        const json j = json::from_bon8(packed, true, false);
-                        if (expected)
-                        {
-                            CHECK(j == json(sequence));
-                        }
-                        else
-                        {
-                            CHECK_FALSE((j.is_string() && j.get<std::string>() == sequence));
-                        }
-                    }
-                }
-            }
-        }
-    }
-}

tests/src/unit-class_parser.cpp

@@ -2953,6 +2953,651 @@ TEST_CASE("diagnostic positions: value lifetime, input adapters, and SAX")
 }
 #endif
 
+namespace
+{
+/// builds a value like json::parse(), but asks the parser to recover from
+/// errors (see #3989), and checks that the events it receives are balanced
+class RecoveringDomParser
+{
+  public:
+    explicit RecoveringDomParser(json& j, std::size_t max_errors_ = static_cast<std::size_t>(-1))
+        : dom(j, false)
+        , max_errors(max_errors_)
+    {}
+
+    bool null()
+    {
+        value();
+        return dom.null();
+    }
+
+    bool boolean(bool val)
+    {
+        value();
+        return dom.boolean(val);
+    }
+
+    bool number_integer(json::number_integer_t val)
+    {
+        value();
+        return dom.number_integer(val);
+    }
+
+    bool number_unsigned(json::number_unsigned_t val)
+    {
+        value();
+        return dom.number_unsigned(val);
+    }
+
+    bool number_float(json::number_float_t val, const std::string& s)
+    {
+        value();
+        return dom.number_float(val, s);
+    }
+
+    bool string(std::string& val)
+    {
+        value();
+        return dom.string(val);
+    }
+
+    bool binary(json::binary_t& val)
+    {
+        value();
+        return dom.binary(val);
+    }
+
+    bool start_object(std::size_t elements)
+    {
+        value();
+        stack.push_back('o');
+        return dom.start_object(elements);
+    }
+
+    bool key(std::string& val)
+    {
+        ++events;
+        if (stack.empty() || stack.back() != 'o')
+        {
+            well_formed = false;
+            return false;
+        }
+        stack.back() = 'v';
+        return dom.key(val);
+    }
+
+    bool end_object()
+    {
+        ++events;
+        if (stack.empty() || stack.back() != 'o')
+        {
+            well_formed = false;
+            return false;
+        }
+        stack.pop_back();
+        return dom.end_object();
+    }
+
+    bool start_array(std::size_t elements)
+    {
+        value();
+        stack.push_back('a');
+        return dom.start_array(elements);
+    }
+
+    bool end_array()
+    {
+        ++events;
+        if (stack.empty() || stack.back() != 'a')
+        {
+            well_formed = false;
+            return false;
+        }
+        stack.pop_back();
+        return dom.end_array();
+    }
+
+    bool parse_error(std::size_t /*unused*/, const std::string& /*unused*/, const json::exception& ex)
+    {
+        errors.emplace_back(ex.what());
+        return errors.size() < max_errors;
+    }
+
+    /// whether the events were balanced and every key was followed by a value
+    bool balanced() const
+    {
+        return well_formed && stack.empty();
+    }
+
+    /// builds the value
+    nlohmann::detail::json_sax_dom_parser<json> dom;
+    std::vector<std::string> errors {}; // NOLINT(readability-redundant-member-init)
+    std::size_t events = 0;
+    /// the open containers: 'a' for an array, 'o' for an object that expects
+    /// a key, 'v' for an object that expects the value of a key
+    std::vector<char> stack {}; // NOLINT(readability-redundant-member-init)
+    bool well_formed = true;
+    std::size_t max_errors;
+
+  private:
+    /// a value is passed: it is an array element, or the value of a key
+    void value()
+    {
+        ++events;
+        if (!stack.empty())
+        {
+            if (stack.back() == 'v')
+            {
+                stack.back() = 'o';
+            }
+            else if (stack.back() == 'o')
+            {
+                // a value without a key
+                well_formed = false;
+            }
+        }
+    }
+
+};
+
+struct RecoveryResult
+{
+    json value;
+    std::vector<std::string> errors;
+    std::size_t events;
+    bool ok;
+    bool balanced;
+};
+
+template<typename InputType>
+RecoveryResult parse_recovering(InputType&& input, const bool strict = true,
+                                const bool ignore_comments = false, const bool ignore_trailing_commas = false)
+{
+    json j;
+    RecoveringDomParser sax(j);
+    const bool ok = json::sax_parse(std::forward<InputType>(input), &sax, json::input_format_t::json,
+                                    strict, ignore_comments, ignore_trailing_commas);
+    return {j, sax.errors, sax.events, ok, sax.balanced()};
+}
+
+/// stops after a number of events, but recovers from errors
+class RecoveringCountdown
+{
+  public:
+    explicit RecoveringCountdown(const int count)
+        : countdown(count)
+    {}
+
+    bool null()
+    {
+        return countdown.null();
+    }
+
+    bool boolean(bool val)
+    {
+        return countdown.boolean(val);
+    }
+
+    bool number_integer(json::number_integer_t val)
+    {
+        return countdown.number_integer(val);
+    }
+
+    bool number_unsigned(json::number_unsigned_t val)
+    {
+        return countdown.number_unsigned(val);
+    }
+
+    bool number_float(json::number_float_t val, const std::string& s)
+    {
+        return countdown.number_float(val, s);
+    }
+
+    bool string(std::string& val)
+    {
+        return countdown.string(val);
+    }
+
+    bool binary(json::binary_t& val)
+    {
+        return countdown.binary(val);
+    }
+
+    bool start_object(std::size_t elements)
+    {
+        return countdown.start_object(elements);
+    }
+
+    bool key(std::string& val)
+    {
+        return countdown.key(val);
+    }
+
+    bool end_object()
+    {
+        return countdown.end_object();
+    }
+
+    bool start_array(std::size_t elements)
+    {
+        return countdown.start_array(elements);
+    }
+
+    bool end_array()
+    {
+        return countdown.end_array();
+    }
+
+    bool parse_error(std::size_t /*position*/, const std::string& /*last_token*/, const json::exception& /*ex*/) // NOLINT(readability-convert-member-functions-to-static)
+    {
+        return true;
+    }
+
+  private:
+    SaxCountdown countdown;
+};
+
+/// a repaired input: the value it is repaired to, and the number of errors
+struct Repair
+{
+    const char* input;
+    const char* expected;
+    std::size_t errors;
+};
+} // namespace
+
+TEST_CASE("parser error recovery (#3989)")
+{
+    SECTION("repairs")
+    {
+        const std::vector<Repair> repairs =
+        {
+            // a missing separator is inserted
+            {"[1 2]", "[1,2]", 1},
+            {R"({"a":1 "b":2})", R"({"a":1,"b":2})", 1},
+            {R"({"a" 1})", R"({"a":1})", 1},
+            {"[1 tru 2]", "[1,null,2]", 2},
+            {R"({"a" "b": 1})", R"({"a":"b"})", 2},
+
+            // a missing value is null in an object; in an array, a ',' stands
+            // for null, while an array that ends there just ends
+            {R"({"a":})", R"({"a":null})", 1},
+            {R"({"a"})", R"({"a":null})", 1},
+            {R"({"a","b":1})", R"({"a":null,"b":1})", 1},
+            {"[1,,2]", "[1,null,2]", 1},
+            {"[,1]", "[null,1]", 1},
+            {"[1,]", "[1]", 1},
+            {"[1,2,3,]", "[1,2,3]", 1},
+            {R"({"a":1,})", R"({"a":1})", 1},
+
+            // a broken string keeps what can be read
+            {R"(["a\qb"])", R"(["aqb"])", 1},
+            {R"({"na\me":1})", R"({"name":1})", 1},
+            {"[\"\xFF\"]", R"(["\uFFFD"])", 1},
+            {"[\"a\xC3(\"]", R"(["a\uFFFD("])", 1},
+            {"[\"\xE2\x82\"]", R"(["\uFFFD"])", 1},
+            {"[\"\xC3\\\\\", 1]", R"(["\uFFFD\\",1])", 1},
+            {R"(["\u12"])", R"(["\uFFFD"])", 1},
+            {R"(["\u12G4"])", R"(["\uFFFDG4"])", 1},
+            {R"(["\uDC00x"])", R"(["\uFFFDx"])", 1},
+            {R"(["\uD800x"])", R"(["\uFFFDx"])", 1},
+            {R"(["\uD800\u0041"])", R"(["\uFFFDA"])", 1},
+            {R"(["\uD800\uD800\uDC00"])", R"(["\uFFFD\uD800\uDC00"])", 1},
+            {R"(["\uD800\uD800\uD800x"])", R"(["\uFFFD\uFFFD\uFFFDx"])", 1},
+            {
+                R"(["\uD800\"x", 1])", R"(["\uFFFD\"x",1])", 1
+            },
+            {R"(["\uD800\q"])", R"(["\uFFFDq"])", 1},
+            {"[\"a\tb\"]", R"(["a\tb"])", 1},
+            {R"(["a\qb\u0041\x"])", R"(["aqbAx"])", 1},
+
+            // a broken number keeps its longest valid prefix
+            {"[1.]", "[1]", 1},
+            {"[-2.]", "[-2]", 1},
+            {"[1.5e]", "[1.5]", 1},
+            {"[1e+]", "[1]", 1},
+            {"[1.x2, 3]", "[1,3]", 1},
+
+            // what cannot be read at all is null
+            {"[1,NaN,3]", "[1,null,3]", 1},
+            {"[tru]", "[null]", 1},
+            {"[-]", "[null]", 1},
+            {R"({"a":Infinity})", R"({"a":null})", 1},
+
+            // a stray token is dropped
+            {"[:1]", "[1]", 1},
+            {R"(["a":1])", R"(["a",1])", 1},
+            {R"({"a"::1})", R"({"a":1})", 1},
+
+            // a member that cannot be read is skipped
+            {R"({1:2,"b":3})", R"({"b":3})", 1},
+            {R"({"a":1 2})", R"({"a":1})", 1},
+            {R"({,"a":1})", R"({"a":1})", 1},
+            {R"({"a":1,,"b":2})", R"({"a":1,"b":2})", 1},
+            {"{a:1}", "{}", 1},
+            {R"({"a":1 [1,{"b":2}], "c":3})", R"({"a":1,"c":3})", 1},
+            {R"([{1}, "a"])", R"([{},"a"])", 1},
+
+            // a wrong closing bracket closes the innermost container
+            {R"({"a":[1,2}, "b":3})", R"({"a":[1,2],"b":3})", 1},
+            {R"([{"a":1], 2])", R"([{"a":1},2])", 1},
+            {"{]", "{}", 1},
+            {"[}", "[]", 1},
+
+            // the end of the input closes all containers
+            {R"({"a":[1,2)", R"({"a":[1,2]})", 1},
+            {"[", "[]", 1},
+            {"{", "{}", 1},
+            {R"({"a")", R"({"a":null})", 1},
+            {R"({"a":)", R"({"a":null})", 1},
+            {"[1,", "[1]", 1},
+            {"[[[1", "[[[1]]]", 1},
+            {
+                R"(["abc)", R"(["abc"])", 2
+            },
+            {"[1,tr", "[1,null]", 2},
+            {"\"abc", "\"abc\"", 1},
+            {"[\"ab\ncd\"]", R"(["ab",null,"]"])", 4},
+
+            // what comes before the top-level value is skipped
+            {")]}'\n{\"a\":1}", R"({"a":1})", 1},
+            {R"(data: {"a":1})", R"({"a":1})", 1},
+            {"\xEF\xBB[1]", "[1]", 1},
+
+            // what comes after it is an error that ends parsing
+            {R"({"a":1}})", R"({"a":1})", 1},
+            {"[1}]", "[1]", 2},
+            {"[1] [2]", "[1]", 1},
+        };
+
+        for (const auto& repair : repairs)
+        {
+            CAPTURE(repair.input)
+            const auto result = parse_recovering(std::string(repair.input));
+            CHECK(!result.ok);
+            CHECK(result.balanced);
+            CHECK(result.value == json::parse(repair.expected));
+            CHECK(result.errors.size() == repair.errors);
+        }
+    }
+
+    SECTION("number overflow")
+    {
+        const auto result = parse_recovering(std::string("[1e999,-1e999]"));
+        CHECK(!result.ok);
+        CHECK(result.balanced);
+        CHECK(result.errors.size() == 2);
+        CHECK(result.errors[0] == "[json.exception.out_of_range.406] number overflow parsing '1e999'");
+        REQUIRE(result.value.size() == 2);
+        CHECK(result.value[0].is_number_float());
+        CHECK(result.value[0].get<double>() == std::numeric_limits<double>::infinity());
+        CHECK(result.value[1].get<double>() == -std::numeric_limits<double>::infinity());
+
+        // the SAX parser gets the number's text
+        SaxEventLogger logger;
+        logger.recover = true;
+        CHECK(!json::sax_parse("1e999", &logger));
+        CHECK(logger.events == std::vector<std::string>({"parse_error(5)", "number_float(1e999)"}));
+    }
+
+    SECTION("nothing to recover")
+    {
+        for (const std::string s :
+                {
+                    "", "   ", "]", "tru", "NaN", ",:", "/* comment"
+                })
+        {
+            CAPTURE(s)
+            const auto result = parse_recovering(s, true, true);
+            CHECK(!result.ok);
+            CHECK(result.balanced);
+            CHECK(result.events == 0);
+            CHECK(result.value == nullptr);
+            CHECK(result.errors.size() == 1);
+        }
+    }
+
+    SECTION("error messages")
+    {
+        // the first error is reported as without recovery
+        for (const std::string s :
+                {
+                    "[1 2]", R"({"a":1 "b":2})", R"({"a" 1})", R"({"a":})", "[1,]", "[1.]",
+                    R"(["a\qb"])", "[1e999]", "{1:2}", R"({"a":[1,2}})", "[1,", "[1] [2]", "{a:1}"
+                })
+        {
+            CAPTURE(s)
+            const auto result = parse_recovering(s);
+            REQUIRE(!result.errors.empty());
+            json _;
+            CHECK_THROWS_WITH_STD_STR(_ = json::parse(s), result.errors.front());
+        }
+
+        // the token of an error begins where the previous error was
+        const auto result = parse_recovering(std::string("[tru, fals, nul]"));
+        CHECK(result.errors == std::vector<std::string>(
+        {
+            "[json.exception.parse_error.101] parse error at line 1, column 5: syntax error while parsing value - invalid literal; last read: '[tru,'",
+            "[json.exception.parse_error.101] parse error at line 1, column 11: syntax error while parsing value - invalid literal; last read: ', fals,'",
+            "[json.exception.parse_error.101] parse error at line 1, column 16: syntax error while parsing value - invalid literal; last read: ', nul]'"
+        }));
+        CHECK(result.value == json::parse("[null,null,null]"));
+    }
+
+    SECTION("events")
+    {
+        // see #4522
+        SaxEventLogger logger;
+        logger.recover = true;
+        CHECK(!json::sax_parse(R"([{1}, "a"])", &logger));
+        CHECK(logger.events == std::vector<std::string>(
+        {
+            "start_array()", "start_object()", "parse_error(3)", "end_object()", "string(a)", "end_array()"
+        }));
+    }
+
+    SECTION("options")
+    {
+        SECTION("strict")
+        {
+            const auto result = parse_recovering(std::string("[1 2] [3]"), false);
+            CHECK(!result.ok);
+            CHECK(result.value == json::parse("[1,2]"));
+            CHECK(result.errors.size() == 1);
+        }
+
+        SECTION("ignore_trailing_commas")
+        {
+            for (const std::string s :
+                    {
+                        "[1,]", R"({"a":1,})", "[[1,],]"
+                    })
+            {
+                CAPTURE(s)
+                const auto result = parse_recovering(s, true, false, true);
+                CHECK(result.ok);
+                CHECK(result.errors.empty());
+            }
+
+            auto result = parse_recovering(std::string("[1,,]"), true, false, true);
+            CHECK(result.value == json::parse("[1,null]"));
+            CHECK(result.errors.size() == 1);
+
+            result = parse_recovering(std::string(R"({"a":1,,})"), true, false, true);
+            CHECK(result.value == json::parse(R"({"a":1})"));
+            CHECK(result.errors.size() == 1);
+        }
+
+        SECTION("ignore_comments")
+        {
+            auto result = parse_recovering(std::string("[1 /* one */ 2]"), true, true);
+            CHECK(result.value == json::parse("[1,2]"));
+            CHECK(result.errors.size() == 1);
+
+            // a comment that is not closed runs to the end of the input, which
+            // is not reported again
+            result = parse_recovering(std::string("[1, 2 /* unterminated"), true, true);
+            CHECK(result.balanced);
+            CHECK(result.value == json::parse("[1,2]"));
+            CHECK(result.errors.size() == 1);
+
+            // a '/' that does not begin a comment is garbage
+            result = parse_recovering(std::string("[1, /x, 2]"), true, true);
+            CHECK(result.balanced);
+            CHECK(result.value == json::parse("[1,null,2]"));
+            CHECK(result.errors.size() == 1);
+        }
+    }
+
+    SECTION("null bytes")
+    {
+        // a null byte ends the input, unless JSON_STRICT_NUL_HANDLING is set
+        const auto result = parse_recovering(std::string("[1,\0x", 5));
+        CHECK(result.balanced);
+        CHECK(!result.ok);
+#ifdef JSON_TEST_STRICT_NUL_HANDLING_ENABLED
+        CHECK(result.value == json::parse("[1,null]"));
+#else
+        CHECK(result.value == json::parse("[1]"));
+        CHECK(result.errors.size() == 1);
+#endif
+
+        const auto in_string = parse_recovering(std::string("[\"a\0b\"]", 7));
+        CHECK(in_string.balanced);
+#ifdef JSON_TEST_STRICT_NUL_HANDLING_ENABLED
+        CHECK(in_string.value == json::array({std::string("a\0b", 3)}));
+#else
+        CHECK(in_string.value == json::parse(R"(["a"])"));
+#endif
+    }
+
+    SECTION("the SAX parser stops recovering")
+    {
+        json j;
+        RecoveringDomParser sax(j, 2);
+        CHECK(!json::sax_parse("[1 2 3 4 5]", &sax));
+        CHECK(sax.errors.size() == 2);
+
+        // an error at a delimiter that an invalid token consumed is reported
+        // to the SAX parser, too
+        json j2;
+        RecoveringDomParser sax2(j2, 2);
+        CHECK(!json::sax_parse("[tru}, 1]", &sax2));
+        CHECK(sax2.errors.size() == 2);
+    }
+
+    SECTION("an event stops parsing during a repair")
+    {
+        // start_object() and key() are passed, then null() for the missing
+        // value returns false
+        RecoveringCountdown countdown(2);
+        CHECK(!json::sax_parse(R"({"a":})", &countdown));
+
+        // the end of the input: end_array() for the second array returns false
+        RecoveringCountdown countdown2(4);
+        CHECK(!json::sax_parse("[[1", &countdown2));
+    }
+
+    SECTION("input adapters")
+    {
+        // the lexer reads contiguous and streaming input differently, and it
+        // puts back a character that ended an invalid token
+        for (const std::string s :
+                {
+                    "[1 2]", "[tru}, 1]", R"({"a" "b\q", "c":[1.x, 2}})", "[\"\xFF\xC3(\", -, 1e+]", "{a:1,\"b\":2", ")]}' [1]"
+                })
+        {
+            CAPTURE(s)
+            const auto reference = parse_recovering(s);
+            CHECK(reference.balanced);
+
+            const auto from_c_string = parse_recovering(s.c_str());
+            CHECK(from_c_string.value == reference.value);
+            CHECK(from_c_string.errors == reference.errors);
+
+            const std::list<char> l(s.begin(), s.end());
+            json j;
+            RecoveringDomParser sax(j);
+            CHECK(!json::sax_parse(l.begin(), l.end(), &sax));
+            CHECK(j == reference.value);
+            CHECK(sax.errors == reference.errors);
+
+            std::istringstream ss(s);
+            const auto from_stream = parse_recovering(ss);
+            CHECK(from_stream.value == reference.value);
+            CHECK(from_stream.errors == reference.errors);
+        }
+    }
+
+    SECTION("long runs of errors")
+    {
+        // no error may copy all the input read before it
+        const auto closing = parse_recovering("[" + std::string(100000, '}'));
+        CHECK(closing.balanced);
+        CHECK(closing.value == json::array());
+
+        const auto garbage = parse_recovering("[" + std::string(100000, 'x') + "]");
+        CHECK(garbage.balanced);
+        CHECK(garbage.errors.size() == 1);
+
+        const auto commas = parse_recovering("{" + std::string(100000, ',') + "}");
+        CHECK(commas.balanced);
+        CHECK(commas.value == json::object());
+    }
+
+    SECTION("mutations of valid input")
+    {
+        // whatever the input, the events are balanced, every error is reported
+        // at most once, and valid input is parsed as usual
+        const std::vector<std::string> documents =
+        {
+            R"({"name": "value", "list": [1, -2.5, true, null, {"x": [[]]}], "e": "\u00e9"})",
+            R"([{"a": [1, 2, {"b": "c"}]}, [], {}, "\ud83d\ude00", 1e10])",
+            "{\"\xC3\xA9\": \"\xF0\x9F\x98\x80\"}",
+            R"(  {"k" : [ "v" , 0 ] }  )",
+        };
+        // each character that can be inserted, including a null byte
+        const std::string insertions("[]{},:\"x\\\0\xFF", 11);
+
+        std::vector<std::string> inputs;
+        for (const auto& doc : documents)
+        {
+            for (std::size_t i = 0; i <= doc.size(); ++i)
+            {
+                inputs.push_back(doc.substr(0, i));
+                if (i < doc.size())
+                {
+                    inputs.push_back(doc.substr(0, i) + doc.substr(i + 1));
+                }
+                for (const char c : insertions)
+                {
+                    inputs.push_back(doc.substr(0, i) + c + doc.substr(i));
+                }
+            }
+        }
+
+        for (const auto& s : inputs)
+        {
+            CAPTURE(s)
+            const auto result = parse_recovering(s);
+            CHECK(result.balanced);
+            CHECK(result.errors.size() <= s.size() + 1);
+            CHECK(result.events <= (4 * s.size()) + 4);
+            if (json::accept(s))
+            {
+                CHECK(result.ok);
+                CHECK(result.errors.empty());
+                CHECK(result.value == json::parse(s));
+            }
+            else
+            {
+                CHECK(!result.ok);
+                CHECK(!result.errors.empty());
+            }
+        }
+    }
+}
+
 TEST_CASE("accept() agrees with parse() on floats around the overflow limit")
 {
     // accept() does not convert a float that cannot overflow; the tokens

tests/src/unit-constructor1.cpp

@@ -361,40 +361,6 @@ TEST_CASE("constructors")
             CHECK_THROWS_WITH_AS((j.get<std::pair<int, int>>()), "[json.exception.out_of_range.401] array index 1 is out of range", json::out_of_range&);
             CHECK_THROWS_WITH_AS((j.get<std::tuple<int, int>>()), "[json.exception.out_of_range.401] array index 1 is out of range", json::out_of_range&);
             CHECK_THROWS_WITH_AS((j.get<std::array<int, 3>>()), "[json.exception.out_of_range.401] array index 1 is out of range", json::out_of_range&);
-#if JSON_DIAGNOSTICS
-            // the exception names the element that has the wrong type
-            CHECK_THROWS_WITH_AS((j.get<std::array<std::string, 3>>()), "[json.exception.type_error.302] (/0) type must be string, but is number", json::type_error&);
-#else
-            CHECK_THROWS_WITH_AS((j.get<std::array<std::string, 3>>()), "[json.exception.type_error.302] type must be string, but is number", json::type_error&);
-#endif
-
-            // the elements are converted in order, so a type error comes before a missing element
-            json const j2{1, "two"};
-            std::array<int, 3> a{{0, 0, 0}};
-#if JSON_DIAGNOSTICS
-            CHECK_THROWS_WITH_AS((j2.get<std::array<int, 3>>()), "[json.exception.type_error.302] (/1) type must be number, but is string", json::type_error&);
-            CHECK_THROWS_WITH_AS(j2.get_to(a), "[json.exception.type_error.302] (/1) type must be number, but is string", json::type_error&);
-#else
-            CHECK_THROWS_WITH_AS((j2.get<std::array<int, 3>>()), "[json.exception.type_error.302] type must be number, but is string", json::type_error&);
-            CHECK_THROWS_WITH_AS(j2.get_to(a), "[json.exception.type_error.302] type must be number, but is string", json::type_error&);
-#endif
-            CHECK_THROWS_WITH_AS(j.get_to(a), "[json.exception.out_of_range.401] array index 1 is out of range", json::out_of_range&);
-#if !defined(JSON_NOEXCEPTION) // without exceptions, the get_to calls in CHECK_THROWS_WITH_AS do not run
-            CHECK(a[0] == 1);
-#endif
-
-            json const j3{"one", "two"};
-            CHECK_THROWS_WITH_AS((j3.get<std::array<std::string, 3>>()), "[json.exception.out_of_range.401] array index 2 is out of range", json::out_of_range&);
-        }
-
-        SECTION("std::array from a longer array or from booleans")
-        {
-            json const j{1, 2.5, true, 4u};
-            CHECK(j.get<std::array<int, 3>>() == (std::array<int, 3> {{1, 2, 1}}));
-            std::array<int, 4> a{{0, 0, 0, 0}};
-            j.get_to(a);
-            CHECK(a == (std::array<int, 4> {{1, 2, 1, 4}}));
-            CHECK(j.get<std::array<int, 0>>().empty());
         }
 
         SECTION("std::forward_list<json>")

tests/src/unit-msgpack.cpp

@@ -31,8 +31,6 @@ using nlohmann::json;
 #include "make_test_data_available.hpp"
 #include "round_trip_corpus.hpp"
 #include "test_utils.hpp"
-#include "sax_countdown.hpp"
-using utils::SaxCountdown;
 
 
 TEST_CASE("MessagePack")
@@ -1645,30 +1643,6 @@ TEST_CASE("MessagePack")
             }
         }
     }
-
-    SECTION("SAX aborts")
-    {
-        SECTION("start_array(len)")
-        {
-            std::vector<uint8_t> const v = {0x93, 0x01, 0x02, 0x03};
-            SaxCountdown scp(0);
-            CHECK(!json::sax_parse(v, &scp, json::input_format_t::msgpack));
-        }
-
-        SECTION("start_object(len)")
-        {
-            std::vector<uint8_t> const v = {0x81, 0xa3, 0x66, 0x6F, 0x6F, 0xc2};
-            SaxCountdown scp(0);
-            CHECK(!json::sax_parse(v, &scp, json::input_format_t::msgpack));
-        }
-
-        SECTION("key()")
-        {
-            std::vector<uint8_t> const v = {0x81, 0xa3, 0x66, 0x6F, 0x6F, 0xc2};
-            SaxCountdown scp(1);
-            CHECK(!json::sax_parse(v, &scp, json::input_format_t::msgpack));
-        }
-    }
 }
 
 TEST_CASE("issue #5405 - array reserve for definite-length MessagePack arrays")
@@ -1739,21 +1713,6 @@ TEST_CASE("issue #5405 - array reserve for definite-length MessagePack arrays")
             CHECK(json::from_msgpack(packed) == j);
         }
     }
-
-    SECTION("a user-defined SAX consumer is unaffected by the internal DOM reserve optimization")
-    {
-        // the reserve() call is local to json_sax_dom_parser / json_sax_dom_callback_parser;
-        // a custom SAX consumer that does not touch a DOM array sees identical events
-        json j = json::array();
-        for (int i = 0; i < 100; ++i)
-        {
-            j.push_back(i);
-        }
-        const auto packed = json::to_msgpack(j);
-
-        SaxCountdown scp(1000000); // large enough to never trigger an abort
-        CHECK(json::sax_parse(packed, &scp, json::input_format_t::msgpack));
-    }
 }
 
 TEST_CASE("regression test - MessagePack ext type rejects a subtype that doesn't fit a single byte")
@@ -1792,15 +1751,6 @@ TEST_CASE("MessagePack nesting does not consume the call stack")
         CHECK(json::from_msgpack(input, true, false).is_discarded());
     }
 
-    SECTION("a well-formed deep value is read through the SAX interface")
-    {
-        std::vector<uint8_t> input(300000, 0x91);
-        input.push_back(0x01); // innermost value
-
-        SaxCountdown accept_all(600001);
-        CHECK(json::sax_parse(input, &accept_all, json::input_format_t::msgpack));
-    }
-
     SECTION("a well-formed deep value is read into a value")
     {
         const std::size_t depth = 10000;
@@ -1848,31 +1798,6 @@ TEST_CASE("MessagePack input that cannot be read is discarded by every overload"
 #endif
 }
 
-TEST_CASE("MessagePack SAX parsing stops at every event")
-{
-    // Containers are opened and closed by the loop that reads them; a SAX
-    // handler that rejects any event - including the end of a nested
-    // container - must stop the parse right there.
-    const auto count_events = [](const std::vector<std::uint8_t>& input)
-    {
-        int events = 0;
-        while (true)
-        {
-            SaxCountdown scp(events);
-            if (json::sax_parse(input, &scp, json::input_format_t::msgpack))
-            {
-                return events;
-            }
-            ++events;
-            REQUIRE(events < 1000);
-        }
-    };
-
-    // 20 events: every container kind closes inside another one
-    const json j = json::parse(R"({"a": [1, {"b": []}], "c": {"d": [[2]]}})");
-    CHECK(count_events(json::to_msgpack(j)) == 20);
-}
-
 TEST_CASE("single MessagePack roundtrip")
 {
     SECTION("sample.json")

tests/src/unit-regression3.cpp

@@ -22,14 +22,6 @@
 // scoped enum, so get<std::byte>() (needed below to get<std::vector<std::byte>>()
 // from a plain JSON array, not just from an already-binary value) relies on
 // enum serialization being enabled
-// capture whether JSON_DELETE_DEPRECATED_FUNCTIONS was enabled on the command
-// line *before* including json.hpp, since the library #undefs it once the header
-// has been fully processed (see include/nlohmann/detail/macro_unscope.hpp); the
-// tests of deprecated functions are skipped if these functions are deleted
-#if defined(JSON_DELETE_DEPRECATED_FUNCTIONS) && (JSON_DELETE_DEPRECATED_FUNCTIONS == 1)
-    #define JSON_TEST_DEPRECATED_FUNCTIONS_DELETED
-#endif
-
 #if defined(JSON_DISABLE_ENUM_SERIALIZATION) && (JSON_DISABLE_ENUM_SERIALIZATION == 1)
     #define SKIP_TESTS_FOR_ENUM_SERIALIZATION
 #endif
@@ -810,50 +802,6 @@ TEST_CASE("issue #5338 - truncated CBOR tagged binary subtype is rejected")
     }
 }
 
-TEST_CASE("issue #5676 - SAX parsing of CBOR tags")
-{
-    const json expected = json::binary({1, 2, 3}, 42);
-    const auto cbor = json::to_cbor(expected);
-
-    nlohmann::detail::json_sax_acceptor<json> acceptor;
-    CHECK_FALSE(json::sax_parse(cbor, &acceptor, json::input_format_t::cbor));
-    CHECK_FALSE(json::sax_parse(cbor, &acceptor, json::input_format_t::cbor,
-                                true, false, false, json::cbor_tag_handler_t::error));
-
-    CHECK(json::sax_parse(cbor, &acceptor, json::input_format_t::cbor,
-                          true, false, false, json::cbor_tag_handler_t::ignore));
-
-    json parsed;
-    nlohmann::detail::json_sax_dom_parser<json, nlohmann::detail::string_input_adapter_type> sax(parsed);
-    CHECK(json::sax_parse(cbor, &sax, json::input_format_t::cbor,
-                          true, false, false, json::cbor_tag_handler_t::store));
-    CHECK(parsed == expected);
-
-    json iterator_parsed;
-    nlohmann::detail::json_sax_dom_parser<json, nlohmann::detail::string_input_adapter_type> iterator_sax(iterator_parsed);
-    CHECK(json::sax_parse(cbor.begin(), cbor.end(), &iterator_sax, json::input_format_t::cbor,
-                          true, false, false, json::cbor_tag_handler_t::store));
-    CHECK(iterator_parsed == expected);
-
-#ifndef JSON_TEST_DEPRECATED_FUNCTIONS_DELETED
-    json span_parsed;
-    nlohmann::detail::json_sax_dom_parser<json, nlohmann::detail::string_input_adapter_type> span_sax(span_parsed);
-    CHECK(json::sax_parse(nlohmann::detail::span_input_adapter(cbor.data(), cbor.size()), &span_sax,
-                          json::input_format_t::cbor, true, false, false, json::cbor_tag_handler_t::store));
-    CHECK(span_parsed == expected);
-#endif
-
-    const std::string text = "null";
-    CHECK(json::sax_parse(text, &acceptor, json::input_format_t::json,
-                          true, false, false, json::cbor_tag_handler_t::store));
-    CHECK(json::sax_parse(text.begin(), text.end(), &acceptor, json::input_format_t::json,
-                          true, false, false, json::cbor_tag_handler_t::store));
-#ifndef JSON_TEST_DEPRECATED_FUNCTIONS_DELETED
-    CHECK(json::sax_parse(nlohmann::detail::span_input_adapter(text.data(), text.size()), &acceptor,
-                          json::input_format_t::json, true, false, false, json::cbor_tag_handler_t::store));
-#endif
-}
-
 TEST_CASE("issue #5402 - update(merge_objects=true) overwrites a primitive with an object")
 {
     json t = {{"k", 1}};

tests/src/unit-sax_parse.cpp

@@ -0,0 +1,768 @@
+//     __ _____ _____ _____
+//  __|  |   __|     |   | |  JSON for Modern C++ (supporting code)
+// |  |  |__   |  |  | | | |  version 3.12.0
+// |_____|_____|_____|_|___|  https://github.com/nlohmann/json
+//
+// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
+// SPDX-License-Identifier: MIT
+
+/////////////////////////////////////////////////////////////////////
+// Tests that call basic_json::sax_parse have a file of their own: every
+// sax_parse call instantiates the parser and binary reader that recover from
+// errors (see #3989), and in unit-regression2.cpp, unit-regression3.cpp, and
+// unit-msgpack.cpp this made the objects too large for the MinGW linker to
+// relocate (see #5511).
+/////////////////////////////////////////////////////////////////////
+
+#include "doctest_compatibility.h"
+
+// capture whether JSON_DELETE_DEPRECATED_FUNCTIONS was enabled on the command
+// line *before* including json.hpp, since the library #undefs it once the header
+// has been fully processed (see include/nlohmann/detail/macro_unscope.hpp); the
+// tests of deprecated functions are skipped if these functions are deleted
+#if defined(JSON_DELETE_DEPRECATED_FUNCTIONS) && (JSON_DELETE_DEPRECATED_FUNCTIONS == 1)
+    #define JSON_TEST_DEPRECATED_FUNCTIONS_DELETED
+#endif
+
+#include <nlohmann/json.hpp>
+using json = nlohmann::json;
+
+#include <cmath>
+#include <cstddef>
+#include <cstdint>
+#include <initializer_list>
+#include <map>
+#include <string>
+#include <utility>
+#include <vector>
+
+#include "sax_countdown.hpp"
+using utils::SaxCountdown;
+
+// a narrow number_float_t, so that a double read from binary input can
+// overflow it
+using float_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int64_t, std::uint64_t, float>;
+
+DOCTEST_CLANG_SUPPRESS_WARNING_PUSH
+DOCTEST_CLANG_SUPPRESS_WARNING("-Wexit-time-destructors")
+
+namespace
+{
+/// builds a value from SAX events, asks the parser to recover from its first
+/// 100 errors, and checks that the events are balanced (see #3989)
+template<typename BasicJsonType>
+class BasicRecoveringParser
+{
+  public:
+    explicit BasicRecoveringParser(BasicJsonType& j)
+        : dom(j, false)
+    {}
+
+    bool null()
+    {
+        value();
+        return dom.null();
+    }
+
+    bool boolean(bool val)
+    {
+        value();
+        return dom.boolean(val);
+    }
+
+    bool number_integer(typename BasicJsonType::number_integer_t val)
+    {
+        value();
+        return dom.number_integer(val);
+    }
+
+    bool number_unsigned(typename BasicJsonType::number_unsigned_t val)
+    {
+        value();
+        return dom.number_unsigned(val);
+    }
+
+    bool number_float(typename BasicJsonType::number_float_t val, const std::string& s)
+    {
+        value();
+        return dom.number_float(val, s);
+    }
+
+    bool string(std::string& val)
+    {
+        value();
+        return dom.string(val);
+    }
+
+    bool binary(typename BasicJsonType::binary_t& val)
+    {
+        value();
+        return dom.binary(val);
+    }
+
+    bool start_object(std::size_t elements)
+    {
+        value();
+        stack.push_back('o');
+        return dom.start_object(elements);
+    }
+
+    bool key(std::string& val)
+    {
+        if (stack.empty() || stack.back() != 'o')
+        {
+            well_formed = false;
+            return false;
+        }
+        stack.back() = 'v';
+        return dom.key(val);
+    }
+
+    bool end_object()
+    {
+        if (stack.empty() || stack.back() != 'o')
+        {
+            well_formed = false;
+            return false;
+        }
+        stack.pop_back();
+        return dom.end_object();
+    }
+
+    bool start_array(std::size_t elements)
+    {
+        value();
+        stack.push_back('a');
+        return dom.start_array(elements);
+    }
+
+    bool end_array()
+    {
+        if (stack.empty() || stack.back() != 'a')
+        {
+            well_formed = false;
+            return false;
+        }
+        stack.pop_back();
+        return dom.end_array();
+    }
+
+    bool parse_error(std::size_t /*unused*/, const std::string& /*unused*/, const json::exception& ex)
+    {
+        messages.emplace_back(ex.what());
+        // a limit, so that a reader that does not stop fails the test
+        // instead of making it hang
+        return ++errors < 100;
+    }
+
+    /// whether the events were balanced and every key was followed by a value
+    bool balanced() const
+    {
+        return well_formed && stack.empty();
+    }
+
+    /// builds the value
+    nlohmann::detail::json_sax_dom_parser<BasicJsonType> dom;
+    std::size_t errors = 0;
+    std::vector<std::string> messages {}; // NOLINT(readability-redundant-member-init)
+    std::vector<char> stack {}; // NOLINT(readability-redundant-member-init)
+    bool well_formed = true;
+
+  private:
+    void value()
+    {
+        if (!stack.empty())
+        {
+            if (stack.back() == 'v')
+            {
+                stack.back() = 'o';
+            }
+            else if (stack.back() == 'o')
+            {
+                well_formed = false;
+            }
+        }
+    }
+
+};
+
+using RecoveringParser = BasicRecoveringParser<json>;
+
+struct BinaryParseResult
+{
+    json value;
+    std::size_t errors;
+    std::vector<std::string> messages;
+    bool ok;
+    bool balanced;
+};
+
+BinaryParseResult parse_binary_recovering(const std::vector<std::uint8_t>& input, const json::input_format_t format)
+{
+    json j;
+    RecoveringParser sax(j);
+    const bool ok = json::sax_parse(input, &sax, format);
+    return {j, sax.errors, sax.messages, ok, sax.balanced()};
+}
+
+#if !defined(JSON_NOEXCEPTION)
+/// the message of the exception that reading @a input into a JSON value
+/// throws, or an empty string if reading succeeds
+std::string binary_error_message(const std::vector<std::uint8_t>& input, const json::input_format_t format)
+{
+    try
+    {
+        json _;
+        switch (format)
+        {
+            case json::input_format_t::cbor:
+                _ = json::from_cbor(input);
+                break;
+            case json::input_format_t::msgpack:
+                _ = json::from_msgpack(input);
+                break;
+            case json::input_format_t::ubjson:
+                _ = json::from_ubjson(input);
+                break;
+            case json::input_format_t::bjdata:
+                _ = json::from_bjdata(input);
+                break;
+            case json::input_format_t::bson:
+                _ = json::from_bson(input);
+                break;
+            case json::input_format_t::bon8:
+                _ = json::from_bon8(input);
+                break;
+            case json::input_format_t::json:
+            default:
+                break;
+        }
+    }
+    catch (const json::exception& e)
+    {
+        return e.what();
+    }
+    return "";
+}
+#endif
+
+/// a BSON element: its type, its name, and its value
+std::vector<std::uint8_t> bson_element(const std::uint8_t type, const std::string& name, const std::vector<std::uint8_t>& value)
+{
+    std::vector<std::uint8_t> result = {type};
+    result.insert(result.end(), name.begin(), name.end());
+    result.push_back(0x00);
+    result.insert(result.end(), value.begin(), value.end());
+    return result;
+}
+
+/// a BSON document of the given elements; @a size_offset is added to the
+/// size it declares
+std::vector<std::uint8_t> bson_document(const std::vector<std::vector<std::uint8_t>>& elements, const int size_offset = 0)
+{
+    std::vector<std::uint8_t> body;
+    for (const auto& element : elements)
+    {
+        body.insert(body.end(), element.begin(), element.end());
+    }
+    const auto size = static_cast<std::uint32_t>(static_cast<int>(body.size()) + 5 + size_offset);
+    std::vector<std::uint8_t> result = {static_cast<std::uint8_t>(size & 0xFFu), static_cast<std::uint8_t>((size >> 8u) & 0xFFu),
+                                        static_cast<std::uint8_t>((size >> 16u) & 0xFFu), static_cast<std::uint8_t>((size >> 24u) & 0xFFu)
+                                       };
+    result.insert(result.end(), body.begin(), body.end());
+    result.push_back(0x00);
+    return result;
+}
+
+/// a BSON int32 value
+std::vector<std::uint8_t> bson_int32(const std::int32_t value)
+{
+    const auto u = static_cast<std::uint32_t>(value);
+    return {static_cast<std::uint8_t>(u & 0xFFu), static_cast<std::uint8_t>((u >> 8u) & 0xFFu),
+            static_cast<std::uint8_t>((u >> 16u) & 0xFFu), static_cast<std::uint8_t>((u >> 24u) & 0xFFu)};
+}
+
+/// a BSON string value, whose length is @a length_offset off
+std::vector<std::uint8_t> bson_string(const std::string& value, const std::int32_t length_offset = 0)
+{
+    auto result = bson_int32(static_cast<std::int32_t>(value.size() + 1) + length_offset);
+    result.insert(result.end(), value.begin(), value.end());
+    result.push_back(0x00);
+    return result;
+}
+
+/// @a count bytes of value 0xAB
+std::vector<std::uint8_t> bytes(const std::size_t count)
+{
+    return std::vector<std::uint8_t>(count, 0xAB);
+}
+
+/// the bytes of @a parts, one after the other
+std::vector<std::uint8_t> concatenated(std::initializer_list<std::vector<std::uint8_t>> parts)
+{
+    std::vector<std::uint8_t> result;
+    for (const auto& part : parts)
+    {
+        result.insert(result.end(), part.begin(), part.end());
+    }
+    return result;
+}
+
+/// U+FFFD REPLACEMENT CHARACTER
+std::string replacement_character()
+{
+    return "\xEF\xBF\xBD";
+}
+}  // namespace
+
+TEST_CASE("regression test - #3989 SAX parse_error() returning true")
+{
+    SECTION("binary formats complete what was read before the input ends")
+    {
+        const json j = {{"a", {1, -2, {{"b", "c"}}, json::array()}}, {"d", {{"e", nullptr}, {"f", true}}}, {"g", 1.5}, {"h", json::binary({1, 2, 3})}};
+
+        const std::vector<std::pair<json::input_format_t, std::vector<std::uint8_t>>> encodings =
+        {
+            {json::input_format_t::cbor, json::to_cbor(j)},
+            {json::input_format_t::msgpack, json::to_msgpack(j)},
+            {json::input_format_t::ubjson, json::to_ubjson(j)},
+            {json::input_format_t::ubjson, json::to_ubjson(j, true, true)},
+            {json::input_format_t::bjdata, json::to_bjdata(j)},
+            {json::input_format_t::bjdata, json::to_bjdata(j, true, true)},
+            {json::input_format_t::bson, json::to_bson(j)},
+            {json::input_format_t::bon8, json::to_bon8(j)},
+        };
+
+        for (const auto& encoding : encodings)
+        {
+            const auto format = encoding.first;
+            const auto& bytes = encoding.second;
+            CAPTURE(format)
+
+            // every prefix is truncated input
+            for (std::size_t length = 0; length < bytes.size(); ++length)
+            {
+                CAPTURE(length)
+                const auto result = parse_binary_recovering(std::vector<std::uint8_t>(bytes.begin(), bytes.begin() + static_cast<std::ptrdiff_t>(length)), format);
+                CHECK(!result.ok);
+                CHECK(result.errors == 1);
+                CHECK(result.balanced);
+            }
+
+            // the complete input is read as usual (binary values do not
+            // round-trip through every format, so compare with a plain parse)
+            json expected;
+            nlohmann::detail::json_sax_dom_parser<json> dom(expected);
+            CHECK(json::sax_parse(bytes, &dom, format));
+            const auto complete = parse_binary_recovering(bytes, format);
+            CHECK(complete.ok);
+            CHECK(complete.errors == 0);
+            CHECK(complete.value == expected);
+
+            // a byte after the value
+            auto trailing_bytes = bytes;
+            trailing_bytes.push_back(0x01);
+            const auto trailing = parse_binary_recovering(trailing_bytes, format);
+            CHECK(!trailing.ok);
+            CHECK(trailing.errors == 1);
+            CHECK(trailing.value == expected);
+        }
+    }
+
+    SECTION("containers without an end")
+    {
+        // these made the readers loop, or read on, after the error
+        const auto cbor_array = parse_binary_recovering({0x9F}, json::input_format_t::cbor);
+        CHECK(cbor_array.errors == 1);
+        CHECK(cbor_array.value == json::array());
+
+        const auto cbor_map = parse_binary_recovering({0xBF, 0x61, 'a'}, json::input_format_t::cbor);
+        CHECK(cbor_map.errors == 1);
+        CHECK(cbor_map.value == json({{"a", nullptr}}));
+
+        const auto msgpack_array = parse_binary_recovering({0xDD, 0xFF, 0xFF, 0xFF, 0xFF}, json::input_format_t::msgpack);
+        CHECK(msgpack_array.errors == 1);
+        CHECK(msgpack_array.value == json::array());
+
+        const auto msgpack_map = parse_binary_recovering({0x81, 0xA1, 'a', 0x92, 0x01}, json::input_format_t::msgpack);
+        CHECK(msgpack_map.errors == 1);
+        CHECK(msgpack_map.value == json({{"a", {1}}}));
+    }
+
+    SECTION("BJData ndarray")
+    {
+        // a 2x3 int8 array with two of its six elements; the annotated array
+        // format opens an object and two arrays of its own
+        const auto result = parse_binary_recovering({'[', '$', 'i', '#', '[', '$', 'i', '#', 'i', 2, 2, 3, 1, 2}, json::input_format_t::bjdata);
+        CHECK(result.errors == 1);
+        CHECK(result.balanced);
+        CHECK(result.value == json({{"_ArrayType_", "int8"}, {"_ArraySize_", {2, 3}}, {"_ArrayData_", {1, 2}}}));
+    }
+
+    SECTION("binary formats repair items whose end is known")
+    {
+        struct Repair
+        {
+            json::input_format_t format;
+            std::vector<std::uint8_t> input;
+            json expected;
+            std::size_t errors;
+        };
+
+        const std::vector<Repair> repairs =
+        {
+            // CBOR: tags are ignored (here tag 1 and the self-describe tag 55799)
+            {json::input_format_t::cbor, {0x82, 0xC1, 0x05, 0xD9, 0xD9, 0xF7, 0x06}, {5, 6}, 2},
+            // CBOR: undefined and other simple values become null
+            {json::input_format_t::cbor, {0x84, 0xF7, 0xE0, 0xF8, 0x20, 0x01}, {nullptr, nullptr, nullptr, 1}, 3},
+            // CBOR: members whose key is not a string are skipped, whatever their key and value
+            {json::input_format_t::cbor, {0xA4, 0x01, 0x02, 0x82, 0x01, 0x02, 0xA1, 0x61, 'x', 0x9F, 0xFF, 0xC1, 0x01, 0x5F, 0x41, 0x00, 0xFF, 0x61, 'a', 0x03}, {{"a", 3}}, 3},
+            {json::input_format_t::cbor, {0xBF, 0xF5, 0xBF, 0x61, 'x', 0x7F, 0x61, 'y', 0xFF, 0xFF, 0x61, 'a', 0x03, 0xFF}, {{"a", 3}}, 1},
+            // MessagePack: members whose key is not a string are skipped
+            {json::input_format_t::msgpack, {0x84, 0x01, 0x02, 0x81, 0xA1, 'x', 0x01, 0x92, 0x01, 0x02, 0xD4, 0x01, 0x02, 0xC0, 0xA1, 'a', 0x04}, {{"a", 4}}, 3},
+            // UBJSON: a char that is not ASCII becomes U+FFFD
+            {json::input_format_t::ubjson, {'[', 'C', 0x80, 'C', 'A', ']'}, {replacement_character(), "A"}, 1},
+            // UBJSON: the longest beginning of a high-precision number is kept
+            {json::input_format_t::ubjson, {'[', 'H', 'i', 5, '1', '2', 'a', 'b', 'c', 'H', 'i', 2, '1', '.', 'H', 'i', 3, 'a', 'b', 'c', 'H', 'i', 3, '4', '.', '5', ']'}, {12, 1, nullptr, 4.5}, 3},
+            // BJData, too
+            {json::input_format_t::bjdata, {'[', 'C', 0xFF, 'H', 'i', 2, '-', '1', 'H', 'i', 2, '-', 'x', ']'}, {replacement_character(), -1, nullptr}, 2},
+            // a NUL ends a high-precision number, as it ends JSON text
+            {json::input_format_t::ubjson, {'[', 'H', 'i', 4, '1', '2', 0, '9', 'H', 'i', 2, 0, '1', 'i', 3, ']'}, {12, nullptr, 3}, 2},
+            // BON8: members whose key is not a string are skipped
+            {json::input_format_t::bon8, {0x89, 0x91, 0x92, 0xC9, 0x40, 0x82, 0x91, 0x92, 0x61, 0x93}, {{"a", 3}}, 2},
+            {json::input_format_t::bon8, {0x8B, 0x91, 0x85, 0x91, 0xFE, 0xFA, 0x8B, 'x', 0x91, 0xFE, 0x61, 0x93, 0xFE}, {{"a", 3}}, 2},
+            // BSON: elements of types the library does not read become null
+            {
+                json::input_format_t::bson, bson_document(
+                {
+                    bson_element(0x07, "_id", bytes(12)),                         // ObjectId
+                    bson_element(0x09, "date", bytes(8)),                         // UTC datetime
+                    bson_element(0x13, "decimal", bytes(16)),                     // 128-bit decimal
+                    bson_element(0x0B, "regex", {'a', '+', 0, 'i', 0}),           // regular expression
+                    bson_element(0x0D, "code", bson_string("f()")),               // JavaScript code
+                    bson_element(0x0E, "symbol", bson_string("s")),               // symbol
+                    bson_element(0x0C, "pointer", concatenated({bson_string("c"), bytes(12)})), // DBPointer
+                    bson_element(0x0F, "scope", concatenated({bson_int32(15), bson_string("g"), bson_document({})})), // code with scope
+                    bson_element(0x06, "undefined", {}),                          // undefined
+                    bson_element(0xFF, "min", {}),                                // min key
+                    bson_element(0x7F, "max", {}),                                // max key
+                    bson_element(0x10, "z", bson_int32(7)),
+                }),
+                {{"_id", nullptr}, {"date", nullptr}, {"decimal", nullptr}, {"regex", nullptr}, {"code", nullptr}, {"symbol", nullptr}, {"pointer", nullptr}, {"scope", nullptr}, {"undefined", nullptr}, {"min", nullptr}, {"max", nullptr}, {"z", 7}},
+                11
+            },
+            // BSON: an element of an unknown type becomes null, and the rest of its document is skipped
+            {
+                json::input_format_t::bson, bson_document(
+                {
+                    bson_element(0x03, "inner", bson_document({bson_element(0x10, "a", bson_int32(1)), bson_element(0x42, "x", bytes(3)), bson_element(0x10, "b", bson_int32(2))})),
+                    bson_element(0x04, "array", bson_document({bson_element(0x10, "0", bson_int32(1)), bson_element(0x42, "1", bytes(3))})),
+                    bson_element(0x10, "after", bson_int32(3)),
+                }),
+                {{"inner", {{"a", 1}, {"x", nullptr}}}, {"array", {1, nullptr}}, {"after", 3}},
+                2
+            },
+            // BSON: so does a string or byte array whose length cannot be right
+            {
+                json::input_format_t::bson, bson_document(
+                {
+                    bson_element(0x03, "inner", bson_document({bson_element(0x02, "s", bson_string("abc", -10)), bson_element(0x10, "b", bson_int32(2))})),
+                    bson_element(0x03, "bin", bson_document({bson_element(0x05, "b", concatenated({bson_int32(-1), bytes(1)})), bson_element(0x10, "b", bson_int32(2))})),
+                    bson_element(0x10, "after", bson_int32(3)),
+                }),
+                {{"inner", {{"s", nullptr}}}, {"bin", {{"b", nullptr}}}, {"after", 3}},
+                2
+            },
+            // BSON: a string without its terminator, and a document whose size does not match, are kept
+            {
+                json::input_format_t::bson, bson_document(
+                {
+                    bson_element(0x02, "s", {2, 0, 0, 0, 'a', 'X'}),
+                    bson_element(0x03, "inner", bson_document({bson_element(0x10, "a", bson_int32(1))}, 1)),
+                }),
+                {{"s", "a"}, {"inner", {{"a", 1}}}},
+                2
+            },
+        };
+
+        for (const auto& repair : repairs)
+        {
+            CAPTURE(repair.format)
+            CAPTURE(repair.input)
+            const auto result = parse_binary_recovering(repair.input, repair.format);
+            CHECK(!result.ok);
+            CHECK(result.balanced);
+            CHECK(result.errors == repair.errors);
+            CHECK(result.value == repair.expected);
+            REQUIRE(!result.messages.empty());
+#if !defined(JSON_NOEXCEPTION)
+            // the first error is the one reported without recovering; under
+            // JSON_NOEXCEPTION, reading without recovering aborts instead of
+            // throwing, so there is no message to compare with
+            CHECK(result.messages.front() == binary_error_message(repair.input, repair.format));
+#endif
+        }
+    }
+
+    SECTION("binary formats repair numbers that are out of range")
+    {
+        // CBOR: a double too large for a float number_float_t
+        float_json cbor;
+        BasicRecoveringParser<float_json> sax(cbor);
+        const std::vector<std::uint8_t> cbor_input = {0x82, 0xFB, 0x7E, 0x37, 0xE4, 0x3C, 0x88, 0x00, 0x75, 0x9C, 0x01}; // [1e300, 1]
+        CHECK(!float_json::sax_parse(cbor_input, &sax, float_json::input_format_t::cbor));
+        CHECK(sax.errors == 1);
+        CHECK(sax.messages.front() == "[json.exception.out_of_range.406] syntax error while parsing CBOR value: number overflow");
+        REQUIRE(cbor.size() == 2);
+        CHECK(std::isinf(cbor[0].get<float>()));
+        CHECK(cbor[1] == 1);
+
+        // UBJSON: a high-precision number too large for number_float_t
+        const auto ubjson = parse_binary_recovering({'H', 'i', 5, '1', 'e', '9', '9', '9'}, json::input_format_t::ubjson);
+        CHECK(ubjson.errors == 1);
+        CHECK(ubjson.value.is_number_float());
+        CHECK(std::isinf(ubjson.value.get<double>()));
+    }
+
+    SECTION("binary formats stop where the end of an item is not known")
+    {
+        // a byte that begins no item
+        const auto cbor = parse_binary_recovering({0x82, 0x01, 0x1C, 0x02}, json::input_format_t::cbor);
+        CHECK(cbor.errors == 1);
+        CHECK(cbor.value == json({1}));
+
+        // a key that is no item: the unused MessagePack byte, a CBOR break
+        // in a map of known size, and the end of a BON8 container
+        const auto msgpack = parse_binary_recovering({0x82, 0xA1, 'a', 0x01, 0xC1, 0x02}, json::input_format_t::msgpack);
+        CHECK(msgpack.errors == 1);
+        CHECK(msgpack.value == json({{"a", 1}}));
+        const auto cbor_break = parse_binary_recovering({0xA2, 0x61, 'a', 0x01, 0xFF, 0x02}, json::input_format_t::cbor);
+        CHECK(cbor_break.errors == 1);
+        CHECK(cbor_break.value == json({{"a", 1}}));
+        const auto bon8 = parse_binary_recovering({0x88, 0x61, 0x91, 0xFE}, json::input_format_t::bon8);
+        CHECK(bon8.errors == 1);
+        CHECK(bon8.value == json({{"a", 1}}));
+
+        // an indefinite-length string inside an indefinite-length string
+        const auto nested = parse_binary_recovering({0x82, 0x01, 0x7F, 0x7F, 0x61, 'a', 0xFF, 0xFF}, json::input_format_t::cbor);
+        CHECK(nested.errors == 1);
+        CHECK(nested.value == json({1}));
+
+        // a BJData ndarray whose element type has no name: the object that
+        // holds the ndarray was already begun
+        const auto ndarray = parse_binary_recovering({'[', '$', 0x01, '#', '[', '$', 'i', '#', 'i', 2, 2, 3}, json::input_format_t::bjdata);
+        CHECK(ndarray.errors == 1);
+        CHECK(ndarray.balanced);
+        CHECK(ndarray.value == json::object());
+
+        // a skipped member that the input ends in
+        const auto truncated = parse_binary_recovering({0xA2, 0x01, 0x82, 0x01}, json::input_format_t::cbor);
+        CHECK(truncated.errors == 2);
+        CHECK(truncated.balanced);
+        CHECK(truncated.value == json::object());
+
+        // a BSON element of an unknown type in a document whose size cannot be right
+        const auto bson = parse_binary_recovering(bson_document({bson_element(0x10, "a", bson_int32(1)), bson_element(0x42, "x", bytes(3))}, -10), json::input_format_t::bson);
+        CHECK(bson.errors == 1);
+        CHECK(bson.value == json({{"a", 1}, {"x", nullptr}}));
+    }
+
+    SECTION("changed bytes in binary input")
+    {
+        const json j = {{"a", {1, -2, {{"b", "c"}}, json::array()}}, {"d", {{"e", nullptr}, {"f", true}}}, {"g", 1.5}, {"h", json::binary({1, 2, 3})}, {"i", "\xC3\xA4"}};
+
+        const std::vector<std::pair<json::input_format_t, std::vector<std::uint8_t>>> encodings =
+        {
+            {json::input_format_t::cbor, json::to_cbor(j)},
+            {json::input_format_t::msgpack, json::to_msgpack(j)},
+            {json::input_format_t::ubjson, json::to_ubjson(j)},
+            {json::input_format_t::ubjson, json::to_ubjson(j, true, true)},
+            {json::input_format_t::bjdata, json::to_bjdata(j)},
+            {json::input_format_t::bjdata, json::to_bjdata(j, true, true)},
+            {json::input_format_t::bson, json::to_bson(j)},
+            {json::input_format_t::bon8, json::to_bon8(j)},
+        };
+        const std::vector<std::uint8_t> replacements = {0x00, 0x01, 0x7F, 0x80, 0xC1, 0xD9, 0xE0, 0xF7, 0xFE, 0xFF};
+
+        for (const auto& encoding : encodings)
+        {
+            const auto format = encoding.first;
+            const auto& original = encoding.second;
+            CAPTURE(format)
+
+            std::vector<std::vector<std::uint8_t>> inputs;
+            for (std::size_t position = 0; position < original.size(); ++position)
+            {
+                for (const auto replacement : replacements)
+                {
+                    auto changed = original;
+                    changed[position] = replacement;
+                    inputs.push_back(changed);
+                }
+                auto removed = original;
+                removed.erase(removed.begin() + static_cast<std::ptrdiff_t>(position));
+                inputs.push_back(removed);
+            }
+
+            for (const auto& input : inputs)
+            {
+                CAPTURE(input)
+                const auto result = parse_binary_recovering(input, format);
+                CHECK(result.balanced);
+                CHECK(result.errors <= input.size() + 1);
+#if !defined(JSON_NOEXCEPTION)
+                // an error is reported exactly if reading into a JSON value
+                // fails, and the first one is the same (under JSON_NOEXCEPTION,
+                // that reading aborts instead of throwing)
+                const auto message = binary_error_message(input, format);
+                CHECK(result.ok == message.empty());
+                if (!result.ok && result.errors < 100)
+                {
+                    CHECK(result.messages.front() == message);
+                }
+#endif
+            }
+        }
+    }
+
+    SECTION("JSON text")
+    {
+        // the parser stopped, but reported success
+        json j;
+        RecoveringParser sax(j);
+        CHECK(!json::sax_parse("[1,2,3,]", &sax));
+        CHECK(sax.errors == 1);
+        CHECK(j == json({1, 2, 3}));
+    }
+
+    SECTION("the SAX parsers of the library stop")
+    {
+        json _;
+        CHECK(json::from_cbor(std::vector<std::uint8_t> {0x9F}, true, false).is_discarded());
+        CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector<std::uint8_t> {0x9F}), "[json.exception.parse_error.110] parse error at byte 2: syntax error while parsing CBOR value: unexpected end of input", json::parse_error&);
+        CHECK(json::parse("[1,2,3,]", nullptr, false).is_discarded());
+        CHECK(!json::accept("[1,2,3,]"));
+    }
+}
+
+
+TEST_CASE("issue #5676 - SAX parsing of CBOR tags")
+{
+    const json expected = json::binary({1, 2, 3}, 42);
+    const auto cbor = json::to_cbor(expected);
+
+    nlohmann::detail::json_sax_acceptor<json> acceptor;
+    CHECK_FALSE(json::sax_parse(cbor, &acceptor, json::input_format_t::cbor));
+    CHECK_FALSE(json::sax_parse(cbor, &acceptor, json::input_format_t::cbor,
+                                true, false, false, json::cbor_tag_handler_t::error));
+
+    CHECK(json::sax_parse(cbor, &acceptor, json::input_format_t::cbor,
+                          true, false, false, json::cbor_tag_handler_t::ignore));
+
+    json parsed;
+    nlohmann::detail::json_sax_dom_parser<json, nlohmann::detail::string_input_adapter_type> sax(parsed);
+    CHECK(json::sax_parse(cbor, &sax, json::input_format_t::cbor,
+                          true, false, false, json::cbor_tag_handler_t::store));
+    CHECK(parsed == expected);
+
+    json iterator_parsed;
+    nlohmann::detail::json_sax_dom_parser<json, nlohmann::detail::string_input_adapter_type> iterator_sax(iterator_parsed);
+    CHECK(json::sax_parse(cbor.begin(), cbor.end(), &iterator_sax, json::input_format_t::cbor,
+                          true, false, false, json::cbor_tag_handler_t::store));
+    CHECK(iterator_parsed == expected);
+
+#ifndef JSON_TEST_DEPRECATED_FUNCTIONS_DELETED
+    json span_parsed;
+    nlohmann::detail::json_sax_dom_parser<json, nlohmann::detail::string_input_adapter_type> span_sax(span_parsed);
+    CHECK(json::sax_parse(nlohmann::detail::span_input_adapter(cbor.data(), cbor.size()), &span_sax,
+                          json::input_format_t::cbor, true, false, false, json::cbor_tag_handler_t::store));
+    CHECK(span_parsed == expected);
+#endif
+
+    const std::string text = "null";
+    CHECK(json::sax_parse(text, &acceptor, json::input_format_t::json,
+                          true, false, false, json::cbor_tag_handler_t::store));
+    CHECK(json::sax_parse(text.begin(), text.end(), &acceptor, json::input_format_t::json,
+                          true, false, false, json::cbor_tag_handler_t::store));
+#ifndef JSON_TEST_DEPRECATED_FUNCTIONS_DELETED
+    CHECK(json::sax_parse(nlohmann::detail::span_input_adapter(text.data(), text.size()), &acceptor,
+                          json::input_format_t::json, true, false, false, json::cbor_tag_handler_t::store));
+#endif
+}
+
+
+TEST_CASE("MessagePack SAX aborts")
+{
+    SECTION("start_array(len)")
+    {
+        std::vector<uint8_t> const v = {0x93, 0x01, 0x02, 0x03};
+        SaxCountdown scp(0);
+        CHECK(!json::sax_parse(v, &scp, json::input_format_t::msgpack));
+    }
+
+    SECTION("start_object(len)")
+    {
+        std::vector<uint8_t> const v = {0x81, 0xa3, 0x66, 0x6F, 0x6F, 0xc2};
+        SaxCountdown scp(0);
+        CHECK(!json::sax_parse(v, &scp, json::input_format_t::msgpack));
+    }
+
+    SECTION("key()")
+    {
+        std::vector<uint8_t> const v = {0x81, 0xa3, 0x66, 0x6F, 0x6F, 0xc2};
+        SaxCountdown scp(1);
+        CHECK(!json::sax_parse(v, &scp, json::input_format_t::msgpack));
+    }
+}
+
+TEST_CASE("issue #5405 - a user-defined SAX consumer is unaffected by the internal DOM reserve optimization")
+{
+    // the reserve() call is local to json_sax_dom_parser / json_sax_dom_callback_parser;
+    // a custom SAX consumer that does not touch a DOM array sees identical events
+    json j = json::array();
+    for (int i = 0; i < 100; ++i)
+    {
+        j.push_back(i);
+    }
+    const auto packed = json::to_msgpack(j);
+
+    SaxCountdown scp(1000000); // large enough to never trigger an abort
+    CHECK(json::sax_parse(packed, &scp, json::input_format_t::msgpack));
+}
+
+TEST_CASE("MessagePack nesting does not consume the call stack - SAX interface")
+{
+    // see the test case of the same name in unit-msgpack.cpp (#5104)
+    std::vector<uint8_t> input(300000, 0x91);
+    input.push_back(0x01); // innermost value
+
+    SaxCountdown accept_all(600001);
+    CHECK(json::sax_parse(input, &accept_all, json::input_format_t::msgpack));
+}
+
+TEST_CASE("MessagePack SAX parsing stops at every event")
+{
+    // Containers are opened and closed by the loop that reads them; a SAX
+    // handler that rejects any event - including the end of a nested
+    // container - must stop the parse right there.
+    const auto count_events = [](const std::vector<std::uint8_t>& input)
+    {
+        int events = 0;
+        while (true)
+        {
+            SaxCountdown scp(events);
+            if (json::sax_parse(input, &scp, json::input_format_t::msgpack))
+            {
+                return events;
+            }
+            ++events;
+            REQUIRE(events < 1000);
+        }
+    };
+
+    // 20 events: every container kind closes inside another one
+    const json j = json::parse(R"({"a": [1, {"b": []}], "c": {"d": [[2]]}})");
+    CHECK(count_events(json::to_msgpack(j)) == 20);
+}
+
+DOCTEST_CLANG_SUPPRESS_WARNING_POP