Reject ill-formed UTF-8 in CBOR/UBJSON/BJData/BSON writers, accept it in MessagePack reader

to_cbor(), to_ubjson(), to_bjdata(), and to_bson() wrote a string or object
key with ill-formed UTF-8 byte for byte, but from_cbor()/from_ubjson()/
from_bjdata()/from_bson() reject such strings with parse_error.113 since
d19f7f5dc (#5185, not yet released): a value that serialized without error
could not be read back by the same library (#5651).

CBOR (RFC 8949 Section 5), UBJSON, BJData, and BSON all require text strings
and object keys/element names to be valid UTF-8, so their writers now
validate and throw type_error.316, like to_bon8() already does. For BSON,
the check runs in the size-computation pass, before any byte is written, the
same way the binary subtype check works. check_bon8_utf8() is renamed to
check_utf8() since it is now shared by all of these writers.

MessagePack's specification explicitly allows a str object to contain an
invalid byte sequence and expects a deserializer to hand the bytes back
unchanged, so its writer is unaffected and from_msgpack() (object keys
included) no longer validates UTF-8, restoring its pre-#5185 behavior.
dump() still rejects ill-formed UTF-8 with type_error.316 unless an error
handler is passed.

Docs: add the type_error.316 exception to to_cbor/to_ubjson/to_bjdata/to_bson,
document the writer-side check on the cbor/bson/ubjson/bjdata format pages,
and rewrite the MessagePack UTF-8 warning to describe the round-trip and
dump() behavior instead of a validation requirement the spec does not have.

Tests: add ill-formed value/key cases (invalid byte, truncated sequence,
encoded surrogate, overlong encoding) expecting type_error.316 to
unit-cbor.cpp, unit-ubjson.cpp, unit-bjdata.cpp, and unit-bson.cpp (which
also checks the output vector stays empty), and turn unit-msgpack.cpp's
former parse_error.113 ill-formed-UTF-8 tests into byte-for-byte round-trip
tests for both values and keys.

This text was written by Claude Code.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

.github/workflows/ubuntu.yml

@@ -100,7 +100,7 @@ jobs:
     container: ubuntu:focal
     strategy:
       matrix:
-        target: [ci_cmake_flags, ci_test_diagnostics, ci_test_diagnostic_positions, ci_test_noexceptions, ci_test_noimplicitconversions, ci_test_legacycomparison, ci_test_noglobaludls, ci_test_disableenumserialization, ci_test_skiplibraryversioncheck, ci_test_simdutf, ci_test_strict_nul_handling, ci_test_no_thread_local]
+        target: [ci_cmake_flags, ci_test_diagnostics, ci_test_diagnostic_positions, ci_test_noexceptions, ci_test_noimplicitconversions, ci_test_legacycomparison, ci_test_noglobaludls, ci_test_disableenumserialization, ci_test_disabletuplereferenceconversion, ci_test_skiplibraryversioncheck, ci_test_simdutf, ci_test_strict_nul_handling, ci_test_no_thread_local]
     steps:
       - name: Install build-essential
         run: apt-get update ; apt-get install -y build-essential unzip wget git libssl-dev

CMakeLists.txt

@@ -55,6 +55,7 @@ option(JSON_Diagnostic_Positions           "Enable diagnostic positions." OFF)
 option(JSON_GlobalUDLs                     "Place user-defined string literals in the global namespace." ON)
 option(JSON_ImplicitConversions            "Enable implicit conversions." ON)
 option(JSON_DisableEnumSerialization       "Disable default integer enum serialization." OFF)
+option(JSON_DisableTupleReferenceConversion "Disable conversion from a one-element tuple of a JSON reference." OFF)
 option(JSON_LegacyDiscardedValueComparison "Enable legacy discarded value comparison." OFF)
 option(JSON_Install                        "Install CMake targets during install step." ${MAIN_PROJECT})
 option(JSON_MultipleHeaders                "Use non-amalgamated version of the library." ON)
@@ -101,6 +102,10 @@ if (JSON_DisableEnumSerialization)
     message(STATUS "Enum integer serialization is disabled (JSON_DISABLE_ENUM_SERIALIZATION=1)")
 endif()
 
+if (JSON_DisableTupleReferenceConversion)
+    message(STATUS "Tuple reference conversion is disabled (JSON_DISABLE_TUPLE_REFERENCE_CONVERSION=1)")
+endif()
+
 if (JSON_LegacyDiscardedValueComparison)
     message(STATUS "Legacy discarded value comparison enabled (JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON=1)")
 endif()
@@ -143,6 +148,7 @@ target_compile_definitions(
     $<$<NOT:$<BOOL:${JSON_GlobalUDLs}>>:JSON_USE_GLOBAL_UDLS=0>
     $<$<NOT:$<BOOL:${JSON_ImplicitConversions}>>:JSON_USE_IMPLICIT_CONVERSIONS=0>
     $<$<BOOL:${JSON_DisableEnumSerialization}>:JSON_DISABLE_ENUM_SERIALIZATION=1>
+    $<$<BOOL:${JSON_DisableTupleReferenceConversion}>:JSON_DISABLE_TUPLE_REFERENCE_CONVERSION=1>
     $<$<BOOL:${JSON_Diagnostics}>:JSON_DIAGNOSTICS=1>
     $<$<BOOL:${JSON_Diagnostic_Positions}>:JSON_DIAGNOSTIC_POSITIONS=1>
     $<$<BOOL:${JSON_LegacyDiscardedValueComparison}>:JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON=1>

README.md

@@ -1395,7 +1395,7 @@ THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR I
 - The class contains a slightly modified version of the Grisu2 algorithm from Florian Loitsch which is licensed under the [MIT License](https://opensource.org/licenses/MIT) (see above). Copyright © 2009 [Florian Loitsch](https://florian.loitsch.com/)
 - The class contains a copy of [Hedley](https://nemequ.github.io/hedley/) from Evan Nemerson which is licensed as [CC0-1.0](https://creativecommons.org/publicdomain/zero/1.0/).
 - The class contains parts of [Google Abseil](https://github.com/abseil/abseil-cpp) which is licensed under the [Apache 2.0 License](https://opensource.org/licenses/Apache-2.0).
-- The class contains an adapted version of the Eisel-Lemire algorithm, its table of powers of five, and its digit comparison for long numbers from [fast_float](https://github.com/fastfloat/fast_float) by Daniel Lemire and contributors, which is available under the [MIT License](https://opensource.org/licenses/MIT) (used here), the Apache 2.0 License, and the Boost Software License. Copyright © 2021 The fast_float authors
+- The class contains an adapted version of the Eisel-Lemire algorithm and its table of powers of five from [fast_float](https://github.com/fastfloat/fast_float) by Daniel Lemire and contributors, which is available under the [MIT License](https://opensource.org/licenses/MIT) (used here), the Apache 2.0 License, and the Boost Software License. Copyright © 2021 The fast_float authors
 
 <img align="right" src="https://git.fsfe.org/reuse/reuse-ci/raw/branch/master/reuse-horizontal.png" alt="REUSE Software">
 

cmake/ci.cmake

@@ -276,6 +276,20 @@ add_custom_target(ci_test_disableenumserialization
     COMMENT "Compile and test with enum serialization disabled"
 )
 
+###############################################################################
+# Disable conversion from a one-element tuple of a JSON reference.
+###############################################################################
+
+add_custom_target(ci_test_disabletuplereferenceconversion
+    COMMAND ${CMAKE_COMMAND}
+    -DCMAKE_BUILD_TYPE=Debug -GNinja
+    -DJSON_BuildTests=ON -DJSON_FastTests=ON -DJSON_DisableTupleReferenceConversion=ON
+    -S${PROJECT_SOURCE_DIR} -B${PROJECT_BINARY_DIR}/build_disabletuplereferenceconversion
+    COMMAND ${CMAKE_COMMAND} --build ${PROJECT_BINARY_DIR}/build_disabletuplereferenceconversion
+    COMMAND cd ${PROJECT_BINARY_DIR}/build_disabletuplereferenceconversion && ${CMAKE_CTEST_COMMAND} --parallel ${N} --output-on-failure
+    COMMENT "Compile and test with tuple reference conversion disabled"
+)
+
 ###############################################################################
 # Skip the multiple-inclusion library version check.
 ###############################################################################

docs/docset/docSet.sql

@@ -210,6 +210,7 @@ INSERT INTO searchIndex(name, type, path) VALUES ('JSON_CATCH_USER', 'Macro', 'a
 INSERT INTO searchIndex(name, type, path) VALUES ('JSON_DIAGNOSTICS', 'Macro', 'api/macros/json_diagnostics/index.html');
 INSERT INTO searchIndex(name, type, path) VALUES ('JSON_DIAGNOSTIC_POSITIONS', 'Macro', 'api/macros/json_diagnostic_positions/index.html');
 INSERT INTO searchIndex(name, type, path) VALUES ('JSON_DISABLE_ENUM_SERIALIZATION', 'Macro', 'api/macros/json_disable_enum_serialization/index.html');
+INSERT INTO searchIndex(name, type, path) VALUES ('JSON_DISABLE_TUPLE_REFERENCE_CONVERSION', 'Macro', 'api/macros/json_disable_tuple_reference_conversion/index.html');
 INSERT INTO searchIndex(name, type, path) VALUES ('JSON_HAS_CPP_11', 'Macro', 'api/macros/json_has_cpp_11/index.html');
 INSERT INTO searchIndex(name, type, path) VALUES ('JSON_HAS_CPP_14', 'Macro', 'api/macros/json_has_cpp_11/index.html');
 INSERT INTO searchIndex(name, type, path) VALUES ('JSON_HAS_CPP_17', 'Macro', 'api/macros/json_has_cpp_11/index.html');

docs/mkdocs/docs/api/basic_json/basic_json.md

@@ -159,6 +159,8 @@ basic_json(basic_json&& other) noexcept;
     - `CompatibleType` is not `basic_json` (to avoid hijacking copy/move constructors),
     - `CompatibleType` is not a different `basic_json` type (i.e. with different template arguments)
     - `CompatibleType` is not a `basic_json` nested type (e.g., `json_pointer`, `iterator`, etc.)
+    - if [`JSON_DISABLE_TUPLE_REFERENCE_CONVERSION`](../macros/json_disable_tuple_reference_conversion.md) is defined
+      to `1`: `CompatibleType` is not a one-element `std::tuple` holding a reference to `basic_json`
     - `json_serializer<U>` (with `U = uncvref_t<CompatibleType>`) has a `to_json(basic_json_t&, CompatibleType&&)`
        method
 

docs/mkdocs/docs/api/basic_json/number_float_t.md

@@ -23,10 +23,9 @@ type to use.
 ## Template parameters
 
 `NumberFloatType`
-:   the type to store floating-point numbers. The parser converts `#!cpp float`, `#!cpp double`, and a
-    `#!cpp long double` that is IEEE 754 binary64 itself and other `#!cpp long double` formats with
-    `#!cpp std::from_chars` or `#!cpp std::strtold`, and serialization falls back to `#!cpp std::snprintf`, so the
-    type must be `#!cpp float`, `#!cpp double`, or `#!cpp long double`. The
+:   the type to store floating-point numbers. Parsing and serialization are implemented in terms of
+    `#!cpp std::strtof`/`#!cpp std::strtod`/`#!cpp std::strtold` and `#!cpp std::snprintf`, so the type must be
+    `#!cpp float`, `#!cpp double`, or `#!cpp long double`. The
     [binary formats](../../features/binary_formats/index.md) additionally require `#!cpp float` or `#!cpp double`,
     because they have no encoding for `#!cpp long double`. See
     [Template Parameter Requirements](../../features/types/template_parameters.md#numberfloattype).

docs/mkdocs/docs/api/basic_json/to_bjdata.md

@@ -56,6 +56,8 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
 
 - Throws [`other_error.502`](../../home/exceptions.md#jsonexceptionother_error502) if `use_type` is true and `use_size`
   is false.
+- Throws [type_error.316](../../home/exceptions.md#jsonexceptiontype_error316) if a string or object key in `j` is
+  not valid UTF-8
 
 ## Complexity
 
@@ -89,4 +91,5 @@ Linear in the size of the JSON value `j`.
 ## Version history
 
 - Added in version 3.11.0.
-- BJData version parameter (for draft3 binary encoding) added in version 3.12.0.
+- BJData version parameter (for draft3 binary encoding) added in version 3.12.0.
+- Throwing `type_error.316` for a string or object key that is not valid UTF-8 added in version 3.13.0.

docs/mkdocs/docs/api/basic_json/to_bson.md

@@ -46,6 +46,8 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
 - Throws [`out_of_range.415`](../../home/exceptions.md#jsonexceptionout_of_range415) if the subtype of a binary value
   exceeds 255, the maximum of the BSON binary subtype; example:
   `"subtype 70000 is too large for the BSON binary subtype (max 255)"`
+- Throws [type_error.316](../../home/exceptions.md#jsonexceptiontype_error316) if a string or object key is
+  not valid UTF-8
 
 ## Complexity
 
@@ -82,3 +84,5 @@ pass before anything is written.
 - Added in version 3.4.0.
 - Linear in the size of `j`, and no longer limited by the call stack for deeply nested values, since version 3.13.0.
 - `out_of_range.415` is now detected before anything is written, like the other exceptions above, since version 3.13.0.
+- Throwing `type_error.316` for a string value or object key that is not valid UTF-8, detected before anything is
+  written, added in version 3.13.0.

docs/mkdocs/docs/api/basic_json/to_cbor.md

@@ -35,6 +35,11 @@ The exact mapping and its limitations are described on a [dedicated page](../../
 
 Strong guarantee: if an exception is thrown, there are no changes in the JSON value.
 
+## Exceptions
+
+- Throws [type_error.316](../../home/exceptions.md#jsonexceptiontype_error316) if a string or object key in `j` is
+  not valid UTF-8
+
 ## Complexity
 
 Linear in the size of the JSON value `j`.
@@ -68,3 +73,4 @@ Linear in the size of the JSON value `j`.
 
 - Added in version 2.0.9.
 - Compact representation of floating-point numbers added in version 3.8.0.
+- Throwing `type_error.316` for a string or object key that is not valid UTF-8 added in version 3.13.0.

docs/mkdocs/docs/api/basic_json/to_ubjson.md

@@ -49,6 +49,8 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
 
 - Throws [`other_error.502`](../../home/exceptions.md#jsonexceptionother_error502) if `use_type` is true and `use_size`
   is false.
+- Throws [type_error.316](../../home/exceptions.md#jsonexceptiontype_error316) if a string or object key in `j` is
+  not valid UTF-8
 
 ## Complexity
 
@@ -82,3 +84,4 @@ Linear in the size of the JSON value `j`.
 ## Version history
 
 - Added in version 3.1.0.
+- Throwing `type_error.316` for a string or object key that is not valid UTF-8 added in version 3.13.0.

docs/mkdocs/docs/api/macros/index.md

@@ -53,6 +53,7 @@ header. See also the [macro overview page](../../features/macros.md).
 
 - [**JSON_BRACE_INIT_COPY_SEMANTICS**](json_brace_init_copy_semantics.md) - opt in to copy/move semantics for single-element brace initialization
 - [**JSON_DISABLE_ENUM_SERIALIZATION**](json_disable_enum_serialization.md) - switch off default serialization/deserialization functions for enums
+- [**JSON_DISABLE_TUPLE_REFERENCE_CONVERSION**](json_disable_tuple_reference_conversion.md) - switch off conversion from a one-element tuple of a JSON reference
 - [**JSON_USE_IMPLICIT_CONVERSIONS**](json_use_implicit_conversions.md) - control implicit conversions
 
 ## Comparison behavior

docs/mkdocs/docs/api/macros/json_disable_tuple_reference_conversion.md

@@ -0,0 +1,114 @@
+# JSON_DISABLE_TUPLE_REFERENCE_CONVERSION
+
+```cpp
+#define JSON_DISABLE_TUPLE_REFERENCE_CONVERSION /* value */
+```
+
+When defined to `1`, a `basic_json` value can no longer be constructed from a one-element `std::tuple` whose element is
+a reference to that `basic_json` type, such as `std::tuple<json&>`, `std::tuple<const json&>`, or `std::tuple<json&&>`.
+These are the tuples created by `std::forward_as_tuple(j)`.
+
+## Default definition
+
+The default value is `0` (disabled — existing behavior is preserved).
+
+```cpp
+#define JSON_DISABLE_TUPLE_REFERENCE_CONVERSION 0
+```
+
+## Notes
+
+!!! note "Background"
+
+    By default, `basic_json` can be constructed from any `std::tuple` whose elements can be converted to JSON; the result
+    is an array. This includes `std::tuple<json&>`, which becomes a one-element array.
+
+    `std::tuple` only converts another tuple element by element if its element type cannot be constructed from the whole
+    source tuple. Because `json` *can* be constructed from `std::tuple<json&>`, `std::tuple` instead converts the whole
+    tuple into a single `json` value. This has two surprising effects:
+
+    ```cpp
+    json j = true;
+
+    // rejected by some standard libraries (e.g., libc++); with others, the
+    // reference binds to a temporary that is destroyed right away
+    std::tuple<const json&> t1(std::forward_as_tuple(j));
+
+    // compiles, but std::get<0>(t2) is [true], not true
+    std::tuple<json> t2(std::forward_as_tuple(j));
+    ```
+
+    Enabling this macro removes the conversion, so both tuples are converted element by element: `std::get<0>(t1)`
+    refers to `j`, and `std::get<0>(t2)` is a copy of `j` (see [#2226](https://github.com/nlohmann/json/issues/2226)).
+
+!!! warning "Opt-in only"
+
+    This macro must be defined **before** including `<nlohmann/json.hpp>`. Defining it after the include has no effect.
+
+!!! note "Affected conversions"
+
+    Only one-element tuples holding a reference to the **same** `basic_json` type are affected. Constructing a JSON value
+    from them no longer compiles:
+
+    ```cpp
+    json j = true;
+    json a = std::forward_as_tuple(j);  // error with the macro enabled
+    json b = json::array({j});          // use this instead: [true]
+    ```
+
+    Tuples holding a JSON value (`std::make_tuple(j)`), tuples with more than one element, and tuples holding references
+    to other types (including other `basic_json` specializations) are converted to arrays as before.
+
+!!! hint "CMake option"
+
+    This behavior can also be controlled with the CMake option
+    [`JSON_DisableTupleReferenceConversion`](../../integration/cmake.md#json_disabletuplereferenceconversion)
+    (`OFF` by default) which defines `JSON_DISABLE_TUPLE_REFERENCE_CONVERSION` accordingly.
+
+## Examples
+
+??? example "Default behavior (macro not defined)"
+
+    ```cpp
+    #include <nlohmann/json.hpp>
+
+    using json = nlohmann::json;
+
+    int main()
+    {
+        json j = true;
+
+        std::tuple<json> t(std::forward_as_tuple(j));
+        // std::get<0>(t) is [true] -- the whole tuple was converted
+    }
+    ```
+
+??? example "Conversion disabled (macro defined to 1)"
+
+    ```cpp
+    #define JSON_DISABLE_TUPLE_REFERENCE_CONVERSION 1
+    #include <nlohmann/json.hpp>
+
+    using json = nlohmann::json;
+
+    int main()
+    {
+        json j = true;
+
+        std::tuple<json> t(std::forward_as_tuple(j));
+        // std::get<0>(t) is true -- a copy of j
+
+        std::tuple<const json&> r(std::forward_as_tuple(j));
+        // std::get<0>(r) refers to j
+    }
+    ```
+
+## See also
+
+- [**basic_json(CompatibleType&&)**](../basic_json/basic_json.md) - the affected constructor
+- [:simple-cmake: JSON_DisableTupleReferenceConversion](../../integration/cmake.md#json_disabletuplereferenceconversion) -
+  CMake option to control the macro
+
+## Version history
+
+- Added in version 3.13.0.

docs/mkdocs/docs/features/binary_formats/bjdata.md

@@ -63,6 +63,12 @@ The library uses the following mapping from JSON values types to BJData types ac
 
       - strings with more than 18446744073709551615 bytes, i.e., $2^{64}-1$ bytes (theoretical)
 
+!!! warning "UTF-8 validation of string values and object keys"
+
+    BJData strings must use UTF-8 encoding. `to_bjdata()` validates the bytes of every string value and object key
+    and throws [`type_error.316`](../../home/exceptions.md#jsonexceptiontype_error316) for ill-formed UTF-8, so a
+    value with such a string cannot be serialized in the first place.
+
 !!! info "Unused BJData markers"
 
     The following markers are not used in the conversion:
@@ -208,6 +214,13 @@ The library maps BJData types to JSON value types as follows:
 
     The mapping is **complete** in the sense that any BJData value can be converted to a JSON value.
 
+!!! warning "UTF-8 validation of string values and object keys"
+
+    This library validates the bytes of every string value and object key at decode time and rejects ill-formed
+    UTF-8 with a [`parse_error.113`](../../home/exceptions.md#jsonexceptionparse_error113) exception (or, with
+    `allow_exceptions` set to `false`, a discarded value), rather than only failing later when the resulting value
+    is dumped.
+
 !!! info "Round trips"
 
     A value returned by [`from_bjdata`](../../api/basic_json/from_bjdata.md) can be serialized with

docs/mkdocs/docs/features/binary_formats/bson.md

@@ -115,8 +115,12 @@ The library maps BSON record types to JSON value types as follows:
     bytes of every such string at decode time and rejects ill-formed UTF-8 with a
     [`parse_error.113`](../../home/exceptions.md#jsonexceptionparse_error113) exception (or, with `allow_exceptions`
     set to `false`, a discarded value), rather than only failing later when the resulting value is dumped. Element
-    (key) names and `binary` values (type `0x05`) are unaffected and are never validated, since they are read
-    byte-by-byte as a C string, or are not required to hold text, respectively.
+    (key) names and `binary` values (type `0x05`) are unaffected and are never validated on read, since they are read
+    byte-by-byte as a C string, or are not required to hold text, respectively. `to_bson()` validates both string
+    values and element names and throws
+    [`type_error.316`](../../home/exceptions.md#jsonexceptiontype_error316) for ill-formed UTF-8 in either, so an
+    object with such a key or value cannot be produced in the first place, even though `from_bson()` would accept it
+    from another source.
 
 ??? example
 

docs/mkdocs/docs/features/binary_formats/cbor.md

@@ -197,7 +197,9 @@ The library maps CBOR types to JSON value types as follows:
     [`parse_error.113`](../../home/exceptions.md#jsonexceptionparse_error113) exception (or, with
     `allow_exceptions` set to `false`, a discarded value), rather than only failing later when the resulting value is
     dumped. Byte strings (major type 2) are unaffected and are never validated, since they are not required to hold
-    text.
+    text. `to_cbor()` validates string values and object keys the same way and throws
+    [`type_error.316`](../../home/exceptions.md#jsonexceptiontype_error316) for ill-formed UTF-8, so a value with
+    such a string cannot be serialized in the first place.
 
 !!! warning "Tagged items"
 

docs/mkdocs/docs/features/binary_formats/messagepack.md

@@ -153,14 +153,15 @@ The library maps MessagePack types to JSON value types as follows:
     This applies to the [SAX interface](../parsing/sax_interface.md) as well, as the key is read before it is passed
     on. Such input needs a general-purpose MessagePack library instead.
 
-!!! warning "UTF-8 validation of string values"
+!!! warning "Ill-formed UTF-8 in string values"
 
-    The MessagePack specification requires `str` values (`fixstr`, `str 8`, `str 16`, `str 32`) to be valid UTF-8.
-    This library validates the bytes of every such string (object keys included) at decode time and rejects
-    ill-formed UTF-8 with a [`parse_error.113`](../../home/exceptions.md#jsonexceptionparse_error113) exception (or,
-    with `allow_exceptions` set to `false`, a discarded value), rather than only failing later when the resulting
-    value is dumped. `bin`/`ext`/`fixext` values are unaffected and are never validated, since they are not required
-    to hold text.
+    The MessagePack specification explicitly allows a `str` value (`fixstr`, `str 8`, `str 16`, `str 32`) to contain
+    a byte sequence that is not valid UTF-8, and expects a deserializer to hand the original bytes back unchanged.
+    This library follows that: `from_msgpack()` reads `str` bytes (object keys included) as-is, without validating
+    them, and `to_msgpack()` writes them back as-is, so such a value round-trips through `from_msgpack(to_msgpack(j))`
+    byte for byte. However, [`dump()`](../../api/basic_json/dump.md) still requires valid UTF-8 and throws
+    [`type_error.316`](../../home/exceptions.md#jsonexceptiontype_error316) for a value read this way, unless an
+    error handler is passed that replaces or ignores the ill-formed bytes.
 
 ??? example
 

docs/mkdocs/docs/features/binary_formats/ubjson.md

@@ -47,6 +47,12 @@ The library uses the following mapping from JSON values types to UBJSON types ac
 
       - strings with more than 9223372036854775807 bytes (theoretical)
 
+!!! warning "UTF-8 validation of string values and object keys"
+
+    UBJSON's required string encoding is UTF-8. `to_ubjson()` validates the bytes of every string value and object
+    key and throws [`type_error.316`](../../home/exceptions.md#jsonexceptiontype_error316) for ill-formed UTF-8, so
+    a value with such a string cannot be serialized in the first place.
+
 !!! info "Unused UBJSON markers"
 
     The following markers are not used in the conversion:
@@ -120,6 +126,13 @@ The library maps UBJSON types to JSON value types as follows:
 
     The mapping is **complete** in the sense that any UBJSON value can be converted to a JSON value.
 
+!!! warning "UTF-8 validation of string values and object keys"
+
+    This library validates the bytes of every string value and object key at decode time and rejects ill-formed
+    UTF-8 with a [`parse_error.113`](../../home/exceptions.md#jsonexceptionparse_error113) exception (or, with
+    `allow_exceptions` set to `false`, a discarded value), rather than only failing later when the resulting value
+    is dumped.
+
 ??? example
 
     ```cpp

docs/mkdocs/docs/features/macros.md

@@ -83,6 +83,13 @@ When defined, default parse and serialize functions for enums are excluded and h
 
 See [full documentation of `JSON_DISABLE_ENUM_SERIALIZATION`](../api/macros/json_disable_enum_serialization.md).
 
+## `JSON_DISABLE_TUPLE_REFERENCE_CONVERSION`
+
+When defined to `1`, a JSON value can no longer be created from a one-element `std::tuple` holding a reference to a JSON
+value, such as the result of `std::forward_as_tuple(j)`. This lets `std::tuple` convert such tuples element-wise.
+
+See [full documentation of `JSON_DISABLE_TUPLE_REFERENCE_CONVERSION`](../api/macros/json_disable_tuple_reference_conversion.md).
+
 ## `JSON_NO_AUTOMATIC_UDLS`
 
 When defined, `<nlohmann/json.hpp>` does not include `<nlohmann/json_literals.hpp>` with the user-defined string literals

docs/mkdocs/docs/features/types/number_handling.md

@@ -71,11 +71,10 @@ otherwise, it uses unsigned integer storage.
 
     - Numbers with a decimal digit or scientific notation are always stored as `#!c double`.
     - The number types can be changed, see [Template number types](#template-number-types). 
-    - The library converts integers and floating-point numbers itself, independent of the locale. Floating-point
-      numbers are correctly rounded (to nearest, ties to even). Only a `#!c long double` that is not IEEE 754 binary64
-      (e.g., the 80-bit x87 format) is converted with `#!cpp std::from_chars` where available, or with
-      [`std::strtold`](https://en.cppreference.com/w/cpp/string/byte/strtof), which gets the decimal point of the
-      current locale, also one longer than one byte (e.g., in `fa_IR.UTF-8`).
+    - As of version 3.9.1, the conversion is realized by
+      [`std::strtoull`](https://en.cppreference.com/w/cpp/string/byte/strtoul),
+      [`std::strtoll`](https://en.cppreference.com/w/cpp/string/byte/strtol), and
+      [`std::strtod`](https://en.cppreference.com/w/cpp/string/byte/strtof), respectively.
 
 !!! example "Examples"
 
@@ -86,10 +85,10 @@ otherwise, it uses unsigned integer storage.
 ### Number limits
 
 - Any 64-bit signed or unsigned integer can be stored without loss of precision.
-- Numbers exceeding the limits of `#!c double` (i.e., numbers whose rounded value is not satisfying
+- Numbers exceeding the limits of `#!c double` (i.e., numbers that after conversion via
+[`std::strtod`](https://en.cppreference.com/w/cpp/string/byte/strtof) are not satisfying
 [`std::isfinite`](https://en.cppreference.com/w/cpp/numeric/math/isfinite) such as `#!c 1E400`) will throw exception
-[`json.exception.out_of_range.406`](../../home/exceptions.md#jsonexceptionout_of_range406) during parsing. Numbers too
-small for `#!c double` (such as `#!c 1E-400`) become zero, with the sign of the number.
+[`json.exception.out_of_range.406`](../../home/exceptions.md#jsonexceptionout_of_range406) during parsing.
 - Floating-point numbers are rounded to the next number representable as `double`. For instance
 `#!c 3.141592653589793238462643383279` is stored as [`0x400921fb54442d18`](https://float.exposed/0x400921fb54442d18).
 This is the same behavior as the code `#!c double x = 3.141592653589793238462643383279;`.

docs/mkdocs/docs/features/types/template_parameters.md

@@ -26,9 +26,8 @@ Requirements are split into two groups:
     diagnosed with dedicated error messages, and violating most of them results in a compiler error somewhere inside
     the library. Four violations are not caught at compile time at all:
 
-    - A [`StringType`](#stringtype) whose `data()` is not null-terminated compiles and silently misparses numbers
-      stored as a `#!cpp long double` that is not IEEE 754 binary64 (e.g., the 80-bit x87 format), because the lexer
-      hands the buffer to `#!cpp std::strtold`.
+    - A [`StringType`](#stringtype) whose `data()` is not null-terminated compiles and silently misparses numbers,
+      because the lexer hands the buffer to `#!cpp std::strtoull`/`#!cpp std::strtoll`/`#!cpp std::strtod`.
     - A stateful [`AllocatorType`](#allocatortype) compiles and silently ignores its state: allocation, deallocation,
       and [`get_allocator()`](../../api/basic_json/get_allocator.md) each use a different default-constructed instance.
     - The two [cross-specialization conversions](#cross-specialization-conversions) below. These abort on an assertion
@@ -536,10 +535,8 @@ therefore silently changes parse results rather than raising an error. See
 
 `NumberFloatType` must be one of `#!cpp float`, `#!cpp double`, or `#!cpp long double`:
 
-- The [parser](../parsing/index.md) converts number literals to `#!cpp float`, `#!cpp double`, and a
-  `#!cpp long double` that is IEEE 754 binary64 itself; other `#!cpp long double` formats are converted with
-  `#!cpp std::from_chars` where available, or with `#!cpp std::strtold`. The library provides overloads for exactly
-  these three types.
+- The [parser](../parsing/index.md) converts number literals with `#!cpp std::strtof`, `#!cpp std::strtod`, or
+  `#!cpp std::strtold`; the library provides overloads for exactly these three types.
 - [`dump`](../../api/basic_json/dump.md) falls back to `#!cpp std::snprintf` with the `%g` and `%Lg` conversion
   specifiers, for which the library likewise provides only `#!cpp double` and `#!cpp long double` overloads
   (`#!cpp float` is promoted to `#!cpp double`).

docs/mkdocs/docs/home/license.md

@@ -20,4 +20,4 @@ The class contains a slightly modified version of the Grisu2 algorithm from Flor
 
 The class contains a copy of [Hedley](https://nemequ.github.io/hedley/) from Evan Nemerson which is licensed as [CC0-1.0](https://creativecommons.org/publicdomain/zero/1.0/).
 
-The class contains an adapted version of the Eisel-Lemire algorithm, its table of powers of five, and its digit comparison for long numbers from [fast_float](https://github.com/fastfloat/fast_float) by Daniel Lemire and contributors, which is available under the [MIT License](https://opensource.org/licenses/MIT) (used here), the Apache 2.0 License, and the Boost Software License. Copyright © 2021 The fast_float authors
+The class contains an adapted version of the Eisel-Lemire algorithm and its table of powers of five from [fast_float](https://github.com/fastfloat/fast_float) by Daniel Lemire and contributors, which is available under the [MIT License](https://opensource.org/licenses/MIT) (used here), the Apache 2.0 License, and the Boost Software License. Copyright © 2021 The fast_float authors

docs/mkdocs/docs/integration/cmake.md

@@ -169,6 +169,12 @@ Enable position diagnostics by defining macro [`JSON_DIAGNOSTIC_POSITIONS`](../a
 Disable default `enum` serialization by defining the macro
 [`JSON_DISABLE_ENUM_SERIALIZATION`](../api/macros/json_disable_enum_serialization.md). This option is `OFF` by default.
 
+### `JSON_DisableTupleReferenceConversion`
+
+Disable the conversion from a one-element `std::tuple` holding a reference to a JSON value by defining the macro
+[`JSON_DISABLE_TUPLE_REFERENCE_CONVERSION`](../api/macros/json_disable_tuple_reference_conversion.md). This option is
+`OFF` by default.
+
 ### `JSON_FastTests`
 
 Skip expensive/slow test suites. This option is `OFF` by default. Depends on `JSON_BuildTests`.

docs/mkdocs/mkdocs.yml

@@ -287,6 +287,7 @@ nav:
           - 'JSON_DIAGNOSTICS': api/macros/json_diagnostics.md
           - 'JSON_DIAGNOSTIC_POSITIONS': api/macros/json_diagnostic_positions.md
           - 'JSON_DISABLE_ENUM_SERIALIZATION': api/macros/json_disable_enum_serialization.md
+          - 'JSON_DISABLE_TUPLE_REFERENCE_CONVERSION': api/macros/json_disable_tuple_reference_conversion.md
           - 'JSON_HAS_CPP_11, JSON_HAS_CPP_14, JSON_HAS_CPP_17, JSON_HAS_CPP_20': api/macros/json_has_cpp_11.md
           - 'JSON_HAS_EXPERIMENTAL_FILESYSTEM, JSON_HAS_FILESYSTEM': api/macros/json_has_filesystem.md
           - 'JSON_HAS_RANGES': api/macros/json_has_ranges.md

include/nlohmann/detail/bit_ops.hpp

@@ -39,25 +39,6 @@ inline int count_leading_zeros(std::uint64_t x) noexcept
 #endif
 }
 
-/// number of trailing zero bits of x (x != 0)
-inline int count_trailing_zeros(std::uint64_t x) noexcept
-{
-#if defined(__GNUC__) || defined(__clang__)
-    return __builtin_ctzll(x);
-#else
-    int n = 0;
-    for (int shift = 32; shift != 0; shift >>= 1)
-    {
-        if ((x << (64 - shift)) == 0)
-        {
-            n += shift;
-            x >>= shift;
-        }
-    }
-    return n;
-#endif
-}
-
 /// the 128-bit product of two 64-bit numbers
 struct uint128_parts
 {
@@ -87,19 +68,14 @@ inline uint128_parts full_multiplication(std::uint64_t a, std::uint64_t b) noexc
 
 /// eight bytes as a little-endian word (compilers fold this into one load on
 /// little-endian targets)
-inline std::uint64_t read_eight_bytes(const unsigned char* b) noexcept
+inline std::uint64_t read_eight_bytes(const char* p) noexcept
 {
+    const auto* b = reinterpret_cast<const unsigned char*>(p); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
     return static_cast<std::uint64_t>(b[0]) | (static_cast<std::uint64_t>(b[1]) << 8u)
            | (static_cast<std::uint64_t>(b[2]) << 16u) | (static_cast<std::uint64_t>(b[3]) << 24u)
            | (static_cast<std::uint64_t>(b[4]) << 32u) | (static_cast<std::uint64_t>(b[5]) << 40u)
            | (static_cast<std::uint64_t>(b[6]) << 48u) | (static_cast<std::uint64_t>(b[7]) << 56u);
 }
 
-/// eight bytes as a little-endian word
-inline std::uint64_t read_eight_bytes(const char* p) noexcept
-{
-    return read_eight_bytes(reinterpret_cast<const unsigned char*>(p)); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
-}
-
 }  // namespace detail
 NLOHMANN_JSON_NAMESPACE_END

include/nlohmann/detail/conversions/to_json.hpp

@@ -471,11 +471,13 @@ inline void to_json_tuple_impl(BasicJsonType& j, const Tuple& t, index_sequence<
     j = { std::get<Idx>(t)... };
 }
 
-#if JSON_BRACE_INIT_COPY_SEMANTICS
-// JSON_BRACE_INIT_COPY_SEMANTICS makes a one-element braced list copy its
-// element instead of wrapping it, which would serialize std::tuple<int>{5} as 5
-// rather than [5]. Build what the default deduction builds instead: an object
-// if the element is a [string, value] pair, a one-element array otherwise.
+// A one-element braced list does not reliably wrap its element: with
+// JSON_BRACE_INIT_COPY_SEMANTICS it copies it, which would serialize
+// std::tuple<int>{5} as 5 rather than [5], and some compilers (e.g., Apple clang
+// 15 and 16) copy an element that is itself a basic_json even without it, so
+// std::tuple<json>{true} became true rather than [true]. Build what the default
+// deduction builds instead: an object if the element is a [string, value] pair,
+// a one-element array otherwise.
 template<typename BasicJsonType, typename Tuple>
 inline void to_json_tuple_impl(BasicJsonType& j, const Tuple& t, index_sequence<0> /*unused*/)
 {
@@ -493,7 +495,6 @@ inline void to_json_tuple_impl(BasicJsonType& j, const Tuple& t, index_sequence<
         j = BasicJsonType::array({std::move(element)});
     }
 }
-#endif
 
 template<typename BasicJsonType, typename Tuple>
 inline void to_json_tuple_impl(BasicJsonType& j, const Tuple& /*unused*/, index_sequence<> /*unused*/)

include/nlohmann/detail/input/binary_reader.hpp

@@ -4111,12 +4111,16 @@ class binary_reader
             return false;
         }
 
-        // RFC 8949 (CBOR) §3.1 and the MessagePack/BSON/UBJSON specifications
-        // all require text strings to be valid UTF-8; reject anything else
-        // right here so malformed input is caught at decode time instead of
-        // only surfacing later as a type_error.316 when the value is dumped
-        // (which would defeat allow_exceptions=false / strict discarding).
-        if (JSON_HEDLEY_UNLIKELY(!is_valid_utf8(result, old_size)))
+        // RFC 8949 (CBOR) §3.1 and the BSON/UBJSON specifications require
+        // text strings to be valid UTF-8; reject anything else right here so
+        // malformed input is caught at decode time instead of only surfacing
+        // later as a type_error.316 when the value is dumped (which would
+        // defeat allow_exceptions=false / strict discarding). The MessagePack
+        // specification explicitly allows a str object to contain an invalid
+        // byte sequence and expects deserializers to hand back the original
+        // bytes, so msgpack strings (and map keys, which go through this
+        // function as well) are exempt.
+        if (format != input_format_t::msgpack && JSON_HEDLEY_UNLIKELY(!is_valid_utf8(result, old_size)))
         {
             return sax->parse_error(chars_read, get_token_string(),
                                     parse_error::create(113, chars_read,

include/nlohmann/detail/input/lexer.hpp

@@ -219,44 +219,6 @@ class lexer : public lexer_base<BasicJsonType>
     // scan functions
     /////////////////////
 
-    /// contiguous input: try to decode the 4 hex digits following `\u`
-    /// directly from the input buffer via hex_codepoint(), instead of 4 calls
-    /// to get(). On success, advances the adapter and the position counters
-    /// exactly as those 4 get() calls would (a hex digit is never '\n', so
-    /// only the flat counters move) and leaves @a current holding the last of
-    /// the 4 digits, just as the last such get() would; the codepoint is
-    /// written to @a out. Makes no state change and returns false - for a
-    /// pending unget, fewer than 4 remaining bytes, or any of the 4 bytes not
-    /// being a hex digit - so the caller falls back unchanged to the
-    /// per-character loop, which then reports the same diagnostic (stopping
-    /// at the first invalid digit) as before this optimization.
-    bool get_codepoint_bulk(std::true_type /*bulk*/, int& out)
-    {
-        if (next_unget || ia.bulk_remaining() < 4)
-        {
-            return false;
-        }
-        const char_type* const raw = ia.bulk_data();
-        const int codepoint = hex_codepoint(reinterpret_cast<const unsigned char*>(raw));
-        if (codepoint < 0)
-        {
-            return false;
-        }
-        ia.bulk_skip(4);
-        // a hex digit is never a newline, so only the flat counters advance
-        position.chars_read_total += 4;
-        position.chars_read_current_line += 4;
-        current = char_traits<char_type>::to_int_type(raw[3]);
-        out = codepoint;
-        return true;
-    }
-
-    /// streaming input: no bulk fast path
-    bool get_codepoint_bulk(std::false_type /*bulk*/, int& /*out*/) const noexcept
-    {
-        return false;
-    }
-
     /*!
     @brief get codepoint from 4 hex characters following `\u`
 
@@ -276,14 +238,6 @@ class lexer : public lexer_base<BasicJsonType>
     {
         // this function only makes sense after reading `\u`
         JSON_ASSERT(current == 'u');
-
-        // contiguous input: decode all 4 hex digits directly from the buffer
-        int fast_codepoint = 0;
-        if (get_codepoint_bulk(std::integral_constant<bool, bulk_scan> {}, fast_codepoint))
-        {
-            return fast_codepoint;
-        }
-
         int codepoint = 0;
 
         const auto factors = { 12u, 8u, 4u, 0u };
@@ -1105,11 +1059,9 @@ class lexer : public lexer_base<BasicJsonType>
             token_type::parse_error otherwise
 
     @note The scanner is independent of the current locale: token_buffer
-          always holds `.`. The conversion of float and double does not use
-          the locale either. Only the std::strtold fallback of
-          convert_number() for long double formats other than binary64
-          depends on it, and it looks up the decimal point right before
-          converting (see detail::convert_float_locale_aware()).
+          always holds `.`. Only the std::strtod fallback of convert_number()
+          depends on the locale, and it looks up the decimal point right
+          before converting (see detail::convert_float_locale_aware()).
     */
     token_type scan_number()  // lgtm [cpp/use-of-goto] `goto` is used in this function to implement the number-parsing state machine described above. By design, any finite input will eventually reach the "done" state or return token_type::parse_error. In each intermediate state, 1 byte of the input is appended to the token_buffer vector, and only the already initialized variables token_buffer, number_type, and error_message are manipulated.
     {
@@ -1122,7 +1074,7 @@ class lexer : public lexer_base<BasicJsonType>
 
         // offset just past the last mantissa byte in token_buffer (i.e. the
         // index of 'e'/'E', or the whole token when there is no exponent).
-        // convert_number() uses it to split the token; npos means
+        // convert_number() uses it to count significant digits; npos means
         // "not seen an exponent yet" and is resolved at scan_number_done
         std::size_t mantissa_end = std::string::npos;
 
@@ -1452,8 +1404,8 @@ scan_number_done:
     @param[in] mantissa_end  offset just past the last mantissa byte in
                              token_buffer (the index of 'e'/'E', or
                              token_buffer.size() when there is no exponent);
-                             with decimal_point_position, it locates the parts
-                             of a float token without scanning it again
+                             used to skip Clinger's fast path when it cannot
+                             possibly succeed - see detail::mantissa_fits_clinger()
     */
     token_type convert_number(token_type number_type, std::size_t mantissa_end)
     {
@@ -1522,11 +1474,10 @@ scan_number_done:
         }
 
         // this code is reached if we parse a floating-point number or if an
-        // integer conversion above overflowed. float and double (and long
-        // double where it is binary64) are converted by the library itself,
-        // correctly rounded and independent of the locale; other long double
-        // formats use std::from_chars when available, otherwise the
-        // locale-aware strtold.
+        // integer conversion above overflowed. Prefer std::from_chars
+        // (Eisel-Lemire, locale-independent, correctly rounded) when available;
+        // otherwise the exact Clinger fast path (double only); otherwise the
+        // locale-aware strtof/strtod/strtold.
         if (convert_float_fast(num_begin, num_end, decimal_point_position, mantissa_end, value_float))
         {
             return token_type::value_float;

include/nlohmann/detail/input/string_scan.hpp

@@ -8,12 +8,10 @@
 
 #pragma once
 
-#include <array> // array
 #include <cstddef> // size_t
-#include <cstdint> // uint64_t, uint8_t
+#include <cstdint> // uint64_t
 #include <cstring> // memcpy
 
-#include <nlohmann/detail/bit_ops.hpp>
 #include <nlohmann/detail/macro_scope.hpp>
 
 // Optional SIMD backend for bulk UTF-8 validation. This is an opt-in external
@@ -71,12 +69,18 @@ inline std::size_t find_string_special(const unsigned char* data, std::size_t n)
     std::size_t i = 0;
     for (; i + 8 <= n; i += 8)
     {
-        const std::uint64_t special = swar_string_special(read_eight_bytes(data + i));
-        if (special != 0)
+        std::uint64_t word = 0;
+        std::memcpy(&word, data + i, sizeof(word));
+        if (swar_string_special(word) != 0)
         {
-            // the lowest flagged byte is the first special one: the borrows of
-            // the subtractions can only flag bytes above a true hit
-            return i + (static_cast<std::size_t>(count_trailing_zeros(special)) / 8);
+            // a special byte is in this word; locate it (endian-agnostic)
+            for (std::size_t j = 0; j < 8; ++j)
+            {
+                if (is_string_special(data[i + j]))
+                {
+                    return i + j;
+                }
+            }
         }
     }
     for (; i < n; ++i)
@@ -110,7 +114,8 @@ inline std::size_t find_ascii_copyable_run(const unsigned char* data, std::size_
     std::size_t i = 0;
     for (; i + 8 <= n; i += 8)
     {
-        const std::uint64_t v = read_eight_bytes(data + i);
+        std::uint64_t v = 0;
+        std::memcpy(&v, data + i, sizeof(v));
         const std::uint64_t q = v ^ 0x2222222222222222ull; // '"'  (0x22)
         const std::uint64_t b = v ^ 0x5C5C5C5C5C5C5C5Cull; // '\\' (0x5C)
         const std::uint64_t d = v ^ 0x7F7F7F7F7F7F7F7Full; // DEL  (0x7F)
@@ -121,9 +126,7 @@ inline std::size_t find_ascii_copyable_run(const unsigned char* data, std::size_
                                    | (v & high);               // >= 0x80
         if (stop != 0)
         {
-            // the lowest flagged byte is the first one to stop at (see
-            // find_string_special())
-            return i + (static_cast<std::size_t>(count_trailing_zeros(stop)) / 8);
+            break;
         }
     }
     for (; i < n; ++i)
@@ -250,18 +253,12 @@ inline std::size_t scalar_string_bulk_run(const unsigned char* data, std::size_t
         {
             break; // end of buffer, or a quote/escape/control byte
         }
-        // a run of multi-byte sequences (e.g. CJK text) is validated sequence
-        // by sequence without searching for the next special byte in between
-        do
+        const std::size_t seq = validate_one_utf8(data + pos, n - pos);
+        if (seq == 0)
         {
-            const std::size_t seq = validate_one_utf8(data + pos, n - pos);
-            if (seq == 0)
-            {
-                return pos; // ill-formed or truncated: let the byte path diagnose it
-            }
-            pos += seq;
+            break; // ill-formed or truncated: let the byte path diagnose it
         }
-        while (pos < n && data[pos] >= 0x80u);
+        pos += seq;
     }
     return pos;
 }
@@ -276,7 +273,8 @@ inline std::size_t find_string_delimiter(const unsigned char* data, std::size_t
     std::size_t i = 0;
     for (; i + 8 <= n; i += 8)
     {
-        const std::uint64_t v = read_eight_bytes(data + i);
+        std::uint64_t v = 0;
+        std::memcpy(&v, data + i, sizeof(v));
         const std::uint64_t q = v ^ 0x2222222222222222ull;
         const std::uint64_t b = v ^ 0x5C5C5C5C5C5C5C5Cull;
         const std::uint64_t hit = ((q - ones) & ~q & high)
@@ -284,8 +282,14 @@ inline std::size_t find_string_delimiter(const unsigned char* data, std::size_t
                                   | ((v - 0x2020202020202020ull) & ~v & high);
         if (hit != 0)
         {
-            // the lowest flagged byte is the first delimiter (see find_string_special())
-            return i + (static_cast<std::size_t>(count_trailing_zeros(hit)) / 8);
+            for (std::size_t j = 0; j < 8; ++j)
+            {
+                const unsigned char c = data[i + j];
+                if (c == '\"' || c == '\\' || c < 0x20u)
+                {
+                    return i + j;
+                }
+            }
         }
     }
     for (; i < n; ++i)
@@ -316,50 +320,5 @@ inline std::size_t string_bulk_run(const unsigned char* data, std::size_t n) noe
     return scalar_string_bulk_run(data, n);
 }
 
-// Decode the 4 hex digits at [data, data+4) - the digits following a `\u`
-// escape - into a codepoint 0x0000..0xFFFF via one table lookup per byte
-// (after yyjson's read_hex_u16), or return -1 if any of the 4 bytes is not a
-// hex digit ('0'..'9', 'A'..'F', 'a'..'f'). The caller must already have
-// checked that 4 bytes are available; used by lexer::get_codepoint()'s
-// contiguous fast path. On -1 it falls back to the byte-at-a-time loop, which
-// stops at the first invalid digit, so the reported error and position are
-// unaffected by this fast path.
-inline int hex_codepoint(const unsigned char* data) noexcept
-{
-    static const std::array<std::uint8_t, 256> hex_digit_table = // NOLINT(cppcoreguidelines-avoid-non-const-global-variables)
-    {
-        {
-            0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // 00..0F
-            0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // 10..1F
-            0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // 20..2F
-            0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // 30..3F ('0'..'9')
-            0xFF, 0x0A, 0x0B, 0x0C, 0x0D, 0x0E, 0x0F, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // 40..4F ('A'..'F')
-            0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // 50..5F
-            0xFF, 0x0A, 0x0B, 0x0C, 0x0D, 0x0E, 0x0F, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // 60..6F ('a'..'f')
-            0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // 70..7F
-            0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // 80..8F
-            0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // 90..9F
-            0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // A0..AF
-            0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // B0..BF
-            0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // C0..CF
-            0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // D0..DF
-            0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // E0..EF
-            0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF  // F0..FF
-        }
-    };
-
-    const std::uint8_t d0 = hex_digit_table[data[0]];
-    const std::uint8_t d1 = hex_digit_table[data[1]];
-    const std::uint8_t d2 = hex_digit_table[data[2]];
-    const std::uint8_t d3 = hex_digit_table[data[3]];
-    // every valid digit is <= 0xF; the combined OR only exceeds it if at
-    // least one of the four bytes was not a hex digit (looked up as 0xFF)
-    if ((d0 | d1 | d2 | d3) > 0x0F)
-    {
-        return -1;
-    }
-    return (d0 << 12) | (d1 << 8) | (d2 << 4) | d3;
-}
-
 }  // namespace detail
 NLOHMANN_JSON_NAMESPACE_END

include/nlohmann/detail/macro_scope.hpp

@@ -915,3 +915,7 @@ void templated_json_throw(ExceptionType exception)
 #ifndef JSON_DISABLE_ENUM_SERIALIZATION
     #define JSON_DISABLE_ENUM_SERIALIZATION 0
 #endif
+
+#ifndef JSON_DISABLE_TUPLE_REFERENCE_CONVERSION
+    #define JSON_DISABLE_TUPLE_REFERENCE_CONVERSION 0
+#endif

include/nlohmann/detail/macro_unscope.hpp

@@ -25,6 +25,7 @@
 #undef JSON_INLINE_VARIABLE
 #undef JSON_NO_UNIQUE_ADDRESS
 #undef JSON_DISABLE_ENUM_SERIALIZATION
+#undef JSON_DISABLE_TUPLE_REFERENCE_CONVERSION
 
 #ifndef JSON_TEST_KEEP_MACROS
     #undef JSON_CATCH

include/nlohmann/detail/meta/type_traits.hpp

@@ -636,6 +636,18 @@ template<typename BasicJsonType, typename CompatibleType>
 struct is_compatible_type
     : is_compatible_type_impl<BasicJsonType, CompatibleType> {};
 
+// a one-element std::tuple holding a reference to BasicJsonType, as created by
+// std::forward_as_tuple(j); see JSON_DISABLE_TUPLE_REFERENCE_CONVERSION
+template<typename BasicJsonType, typename T>
+struct is_basic_json_reference_tuple : std::false_type {};
+
+template<typename BasicJsonType, typename T>
+struct is_basic_json_reference_tuple<BasicJsonType, std::tuple<T>>
+{
+    static constexpr bool value =
+        std::is_reference<T>::value && std::is_same<uncvref_t<T>, BasicJsonType>::value;
+};
+
 template<typename BasicJsonType, typename CompatibleArrayType>
 struct is_compatible_binary_type
 {

include/nlohmann/detail/output/binary_writer.hpp

@@ -116,6 +116,8 @@ class binary_writer
     /*!
     @param[in] j  JSON value to serialize
     @pre       j.type() == value_t::object
+    @throw type_error.316 if a string value or an object key is not valid
+           UTF-8
     */
     void write_bson(const BasicJsonType& j)
     {
@@ -145,6 +147,8 @@ class binary_writer
 
     /*!
     @param[in] j  JSON value to serialize
+    @throw type_error.316 if a string value or an object key is not valid
+           UTF-8
     */
     void write_cbor(const BasicJsonType& j)
     {
@@ -211,6 +215,8 @@ class binary_writer
 
             case value_t::string:
             {
+                check_utf8(*j.m_data.m_value.string, j);
+
                 // step 1: write control byte and the string length
                 write_cbor_head(0x60, j.m_data.m_value.string->size());
 
@@ -280,6 +286,11 @@ class binary_writer
                 // step 2: write each element
                 for (const auto& el : *j.m_data.m_value.object)
                 {
+                    // el.first is checked here, against the object as
+                    // diagnostics context, because write_cbor(el.first)
+                    // converts it to a temporary basic_json that would be
+                    // used as the context instead
+                    check_utf8(el.first, j);
                     write_cbor(el.first);
                     write_cbor(el.second);
                 }
@@ -643,6 +654,8 @@ class binary_writer
     @param[in] add_prefix  whether prefixes need to be used for this value
     @param[in] use_bjdata  whether write in BJData format, default is false
     @param[in] bjdata_version  which BJData version to use, default is draft2
+    @throw type_error.316 if a string value or an object key is not valid
+           UTF-8
     */
     void write_ubjson(const BasicJsonType& j, const bool use_count,
                       const bool use_type, const bool add_prefix = true,
@@ -692,6 +705,8 @@ class binary_writer
 
             case value_t::string:
             {
+                check_utf8(*j.m_data.m_value.string, j);
+
                 if (add_prefix)
                 {
                     oa.write_character(to_char_type('S'));
@@ -854,6 +869,7 @@ class binary_writer
 
                 for (const auto& el : *j.m_data.m_value.object)
                 {
+                    check_utf8(el.first, j);
                     write_number_with_ubjson_prefix(el.first.size(), true, use_bjdata);
                     oa.write_characters(
                           reinterpret_cast<const CharType*>(el.first.data()),
@@ -898,6 +914,10 @@ class binary_writer
     /*!
     @return The size of a BSON document entry header, including the id marker
             and the entry name size (and its null-terminator).
+    @throw out_of_range.409 if @a name contains U+0000, before anything is
+           written
+    @throw type_error.316 if @a name is not valid UTF-8, before anything is
+           written
     */
     static std::size_t calc_bson_entry_header_size(const string_t& name, const BasicJsonType& j)
     {
@@ -907,7 +927,8 @@ class binary_writer
             JSON_THROW(out_of_range::create(409, concat("BSON key cannot contain code point U+0000 (at byte ", std::to_string(it), ")"), &j));
         }
 
-        static_cast<void>(j);
+        check_utf8(name, j);
+
         return /*id*/ 1ul + name.size() + /*zero-terminator*/1u;
     }
 
@@ -963,9 +984,21 @@ class binary_writer
 
     /*!
     @return The size of the BSON-encoded string in @a value
+    @throw type_error.316 if @a value is not valid UTF-8, before anything is
+           written
+
+    @note The UTF-8 check is skipped if @a value is already too long for the
+          32-bit BSON length field (@ref to_bson_length rejects it later, once
+          the size of the whole document is known); this also keeps the check
+          from reading past a StringType that reports a size larger than what
+          it actually holds.
     */
-    static std::size_t calc_bson_string_size(const string_t& value)
+    static std::size_t calc_bson_string_size(const string_t& value, const BasicJsonType& j)
     {
+        if (JSON_HEDLEY_LIKELY(value_in_range_of<std::int32_t>(value.size())))
+        {
+            check_utf8(value, j);
+        }
         return sizeof(std::int32_t) + value.size() + 1ul;
     }
 
@@ -1094,6 +1127,8 @@ class binary_writer
             is neither an object nor an array
     @throw out_of_range.415 if @a j is binary with a subtype that does not fit
            into a byte, before anything is written
+    @throw type_error.316 if @a j is a string that is not valid UTF-8, before
+           anything is written
     */
     static std::size_t calc_bson_value_size(const BasicJsonType& j)
     {
@@ -1115,7 +1150,7 @@ class binary_writer
                 return calc_bson_unsigned_size(j.m_data.m_value.number_unsigned);
 
             case value_t::string:
-                return calc_bson_string_size(*j.m_data.m_value.string);
+                return calc_bson_string_size(*j.m_data.m_value.string, j);
 
             case value_t::null:
                 return 0ul;
@@ -1228,6 +1263,8 @@ class binary_writer
            written
     @throw out_of_range.415 if a binary value's subtype does not fit into a
            byte, before anything is written
+    @throw type_error.316 if a string value or a key is not valid UTF-8,
+           before anything is written
     */
     static std::size_t calc_bson_sizes(const BasicJsonType& document, std::vector<std::size_t>& nested_sizes)
     {
@@ -2252,7 +2289,7 @@ class binary_writer
     */
     void write_bon8_string(const string_t& s, bool& string_open, const BasicJsonType& context)
     {
-        check_bon8_utf8(s, context);
+        check_utf8(s, context);
 
         // a string that follows another string terminates it
         if (string_open)
@@ -2282,7 +2319,7 @@ class binary_writer
     @throw type_error.316 if @a s is not valid UTF-8; the message names the
            first byte of the first invalid or incomplete sequence
     */
-    static void check_bon8_utf8(const string_t& s, const BasicJsonType& context)
+    static void check_utf8(const string_t& s, const BasicJsonType& context)
     {
         static_cast<void>(context); // only used when exceptions are enabled
         const auto* data = reinterpret_cast<const unsigned char*>(s.data());

include/nlohmann/json.hpp

@@ -906,11 +906,11 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
     /*!
     @brief how many levels the operation going on in this thread has descended into
 
-    Copying a value and comparing two values share this count. The library never
-    nests one inside the other - copying a value does not compare one, and
-    comparing two values does not copy them - and where user code nests them
-    anyway, sharing the count only ends a descent sooner than it had to, which
-    costs a little speed and is never wrong.
+    Copying a value, converting one from another specialization, and comparing
+    two values share this count. The library never nests one of them inside
+    another - none of them does either of the other two on the way - and where
+    user code nests them anyway, sharing the count only ends a descent sooner
+    than it had to, which costs a little speed and is never wrong.
 
     A byte is enough: the count never exceeds the limit by more than the single
     level that notices the limit has been reached.
@@ -1267,6 +1267,222 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
         copy_iteratively(src);
     }
 
+    /*!
+    @brief convert the value @a val of another specialization into this null
+           value; @a val must be neither an object nor an array
+
+    Converting such a value never descends, so both ways of converting an
+    object or an array (@ref convert_structured) leave their elements of this
+    kind to the converting constructor, which leaves them to this.
+    */
+    template<typename BasicJsonType>
+    void convert_leaf(const BasicJsonType& val)
+    {
+        using other_boolean_t = typename BasicJsonType::boolean_t;
+        using other_number_float_t = typename BasicJsonType::number_float_t;
+        using other_number_integer_t = typename BasicJsonType::number_integer_t;
+        using other_number_unsigned_t = typename BasicJsonType::number_unsigned_t;
+        using other_string_t = typename BasicJsonType::string_t;
+        using other_binary_t = typename BasicJsonType::binary_t;
+
+        switch (val.type())
+        {
+            case value_t::boolean:
+                JSONSerializer<other_boolean_t>::to_json(*this, val.template get<other_boolean_t>());
+                break;
+            case value_t::number_float:
+                JSONSerializer<other_number_float_t>::to_json(*this, val.template get<other_number_float_t>());
+                break;
+            case value_t::number_integer:
+                JSONSerializer<other_number_integer_t>::to_json(*this, val.template get<other_number_integer_t>());
+                break;
+            case value_t::number_unsigned:
+                JSONSerializer<other_number_unsigned_t>::to_json(*this, val.template get<other_number_unsigned_t>());
+                break;
+            case value_t::string:
+                JSONSerializer<other_string_t>::to_json(*this, val.template get_ref<const other_string_t&>());
+                break;
+            case value_t::binary:
+                JSONSerializer<other_binary_t>::to_json(*this, val.template get_ref<const other_binary_t&>());
+                break;
+            case value_t::null:
+                // m_data.m_type is already value_t::null
+                break;
+            case value_t::discarded:
+                m_data.m_type = value_t::discarded;
+                break;
+            case value_t::object: // LCOV_EXCL_LINE
+            case value_t::array:  // LCOV_EXCL_LINE
+            default:              // LCOV_EXCL_LINE
+                JSON_ASSERT(false); // NOLINT(cert-dcl03-c,hicpp-static-assert,misc-static-assert) LCOV_EXCL_LINE
+        }
+    }
+
+    /// scratch space for the converted elements of the arrays that
+    /// @ref convert_iteratively has yet to create
+    using convert_scratch_t = std::vector<basic_json, AllocatorType<basic_json>>;
+
+    /*!
+    @brief create the object or array @a val converted into this null value
+
+    Its converted elements are the last `val.size()` entries of @a elements (an
+    array) or of @a members (an object); they are moved into the container in
+    one go and then removed.
+    */
+    template<typename BasicJsonType>
+    void convert_level(const BasicJsonType& val, convert_scratch_t& elements, copy_scratch_t& members)
+    {
+        if (val.is_object())
+        {
+            const auto first = members.end() - static_cast<typename copy_scratch_t::difference_type>(val.size());
+            m_data.m_value.object = create<object_t>(std::make_move_iterator(first),
+                                    std::make_move_iterator(members.end()));
+            // only now that the object exists may this stop being a null value
+            m_data.m_type = value_t::object;
+            members.erase(first, members.end());
+        }
+        else
+        {
+            const auto first = elements.end() - static_cast<typename convert_scratch_t::difference_type>(val.size());
+            m_data.m_value.array = create<array_t>(std::make_move_iterator(first),
+                                                   std::make_move_iterator(elements.end()));
+            // only now that the array exists may this stop being a null value
+            m_data.m_type = value_t::array;
+            elements.erase(first, elements.end());
+        }
+
+        set_parents();
+    }
+
+    /*!
+    @brief convert the object or array @a val of another specialization into
+           this null value without recursing
+
+    The containers whose conversion has begun are kept on an explicit stack
+    rather than on the call stack. Unlike @ref copy_iteratively, this builds
+    every container from the bottom up: all its elements are converted first,
+    and the container is then created from them in one go, the way the range
+    constructor that converts the levels above the bound does. The two object
+    types need not enumerate their members in the same order, so the members
+    could not be paired up by position anyway, and building from a range keeps
+    what the range constructor does with keys that become equal on conversion.
+
+    Every value is complete before it is handed on, and a container gets its
+    type only once it exists, so whatever throws, every value left behind can
+    be destroyed.
+    */
+    template<typename BasicJsonType>
+    void convert_iteratively(const BasicJsonType& val)
+    {
+        using other_const_iterator = typename BasicJsonType::const_iterator;
+
+        // the containers whose conversion has begun, innermost last, each with
+        // its element to convert next
+        std::vector<std::pair<const BasicJsonType*, other_const_iterator>> pending;
+
+        // the converted elements of the pending arrays and the converted
+        // members of the pending objects, those of the innermost one last
+        convert_scratch_t elements;
+        copy_scratch_t members;
+
+        pending.emplace_back(&val, val.cbegin());
+
+        for (;;)
+        {
+            const BasicJsonType& container = *pending.back().first;
+            // a copy, as descending below can reallocate pending; the
+            // iterator kept in pending is only advanced through pending.back()
+            const other_const_iterator next = pending.back().second;
+
+            if (next != container.cend())
+            {
+                if (next->is_structured())
+                {
+                    // convert its elements first; next stays where it is until
+                    // the converted container is handed back to this one
+                    pending.emplace_back(&*next, next->cbegin());
+                    continue;
+                }
+
+                // the converting constructor does not descend into this value
+                if (container.is_object())
+                {
+                    members.emplace_back(next.key(), *next);
+                }
+                else
+                {
+                    elements.emplace_back(*next);
+                }
+                ++pending.back().second;
+                continue;
+            }
+
+            // all elements of the container are converted: create it
+            pending.pop_back();
+
+            if (pending.empty())
+            {
+                convert_level(container, elements, members);
+                return;
+            }
+
+            basic_json converted;
+            converted.convert_level(container, elements, members);
+#if JSON_DIAGNOSTIC_POSITIONS
+            converted.start_position = container.start_pos();
+            converted.end_position = container.end_pos();
+#endif
+
+            // hand it to the container it is an element of
+            if (pending.back().first->is_object())
+            {
+                members.emplace_back(pending.back().second.key(), std::move(converted));
+            }
+            else
+            {
+                elements.push_back(std::move(converted));
+            }
+            ++pending.back().second;
+        }
+    }
+
+    /*!
+    @brief convert the object or array @a val of another specialization into
+           this null value
+
+    Converting a container converts its elements, so a value nested deeply
+    enough used to exhaust the call stack. The descent is bounded here as in
+    @ref copy_structured: the first `detail::recursion_depth_limit()` levels
+    are converted by the containers' range constructors, just as they always were,
+    and anything below that is converted without the call stack by
+    @ref convert_iteratively.
+
+    @sa https://github.com/nlohmann/json/issues/5650
+    */
+    template<typename BasicJsonType>
+    void convert_structured(const BasicJsonType& val)
+    {
+        const nesting_depth_guard guard;
+
+        if (JSON_HEDLEY_LIKELY(guard.okay()))
+        {
+            // every element comes back to the converting constructor
+            if (val.is_object())
+            {
+                using other_object_t = typename BasicJsonType::object_t;
+                JSONSerializer<other_object_t>::to_json(*this, val.template get_ref<const other_object_t&>());
+            }
+            else
+            {
+                using other_array_t = typename BasicJsonType::array_t;
+                JSONSerializer<other_array_t>::to_json(*this, val.template get_ref<const other_array_t&>());
+            }
+            return;
+        }
+
+        convert_iteratively(val);
+    }
+
 
     /// the result of comparing two values, including values that cannot be
     /// ordered at all, such as a discarded value or a NaN
@@ -1596,7 +1812,12 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
     template < typename CompatibleType,
                typename U = detail::uncvref_t<CompatibleType>,
                detail::enable_if_t <
-                   !detail::is_basic_json<U>::value && detail::is_compatible_type<basic_json_t, U>::value, int > = 0 >
+                   !detail::is_basic_json<U>::value && detail::is_compatible_type<basic_json_t, U>::value
+#if JSON_DISABLE_TUPLE_REFERENCE_CONVERSION
+                   // see https://github.com/nlohmann/json/issues/2226
+                   && !detail::is_basic_json_reference_tuple<basic_json_t, U>::value
+#endif
+                   , int > = 0 >
     basic_json(CompatibleType && val) noexcept(noexcept( // NOLINT(bugprone-forwarding-reference-overload,bugprone-exception-escape)
                 JSONSerializer<U>::to_json(std::declval<basic_json_t&>(),
                                            std::forward<CompatibleType>(val))))
@@ -1617,49 +1838,13 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
           end_position(val.end_pos())
 #endif
     {
-        using other_boolean_t = typename BasicJsonType::boolean_t;
-        using other_number_float_t = typename BasicJsonType::number_float_t;
-        using other_number_integer_t = typename BasicJsonType::number_integer_t;
-        using other_number_unsigned_t = typename BasicJsonType::number_unsigned_t;
-        using other_string_t = typename BasicJsonType::string_t;
-        using other_object_t = typename BasicJsonType::object_t;
-        using other_array_t = typename BasicJsonType::array_t;
-        using other_binary_t = typename BasicJsonType::binary_t;
-
-        switch (val.type())
+        if (val.is_structured())
         {
-            case value_t::boolean:
-                JSONSerializer<other_boolean_t>::to_json(*this, val.template get<other_boolean_t>());
-                break;
-            case value_t::number_float:
-                JSONSerializer<other_number_float_t>::to_json(*this, val.template get<other_number_float_t>());
-                break;
-            case value_t::number_integer:
-                JSONSerializer<other_number_integer_t>::to_json(*this, val.template get<other_number_integer_t>());
-                break;
-            case value_t::number_unsigned:
-                JSONSerializer<other_number_unsigned_t>::to_json(*this, val.template get<other_number_unsigned_t>());
-                break;
-            case value_t::string:
-                JSONSerializer<other_string_t>::to_json(*this, val.template get_ref<const other_string_t&>());
-                break;
-            case value_t::object:
-                JSONSerializer<other_object_t>::to_json(*this, val.template get_ref<const other_object_t&>());
-                break;
-            case value_t::array:
-                JSONSerializer<other_array_t>::to_json(*this, val.template get_ref<const other_array_t&>());
-                break;
-            case value_t::binary:
-                JSONSerializer<other_binary_t>::to_json(*this, val.template get_ref<const other_binary_t&>());
-                break;
-            case value_t::null:
-                *this = nullptr;
-                break;
-            case value_t::discarded:
-                m_data.m_type = value_t::discarded;
-                break;
-            default:            // LCOV_EXCL_LINE
-                JSON_ASSERT(false); // NOLINT(cert-dcl03-c,hicpp-static-assert,misc-static-assert) LCOV_EXCL_LINE
+            convert_structured(val);
+        }
+        else
+        {
+            convert_leaf(val);
         }
         JSON_ASSERT(m_data.m_type == val.type());
 

tests/src/float_hard_cases.hpp

@@ -1,599 +0,0 @@
-//     __ _____ _____ _____
-//  __|  |   __|     |   | |  JSON for Modern C++ (supporting code)
-// |  |  |__   |  |  | | | |  version 3.12.0
-// |_____|_____|_____|_|___|  https://github.com/nlohmann/json
-//
-// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
-// SPDX-License-Identifier: MIT
-
-#pragma once
-
-#include <array> // array
-#include <cstdint> // uint32_t, uint64_t
-
-// Number tokens that are hard to round correctly, with the IEEE-754 binary64
-// and binary32 bits of their correctly rounded values (ties to even; infinity
-// for an overflow, a signed zero for an underflow).
-//
-// For doubles and floats around 0, the smallest normal number, 1, 2^24, 2^53,
-// 0.1, and the largest finite number, and for random ones, the exact midpoint
-// m to the next number gives: m, m with one unit more and less in the last
-// digit, m with "01" and "0...01" appended, m with trailing zeros, and m cut
-// after 17 to 30 digits (rounded down and up, so that the rounding is decided
-// after the 19th digit), in fixed and exponent notation, 30% of them negative.
-// Tokens longer than 80 characters are left out, except for four of 700 digits
-// and more. Zeros, underflow, overflow, huge exponents, and integers beyond 64
-// bits complete the set. Of the 508 tokens, 134 (as double) and 150 (as
-// float) need the exact comparison with the midpoint (detail::digit_comparison()).
-//
-// The expected bits were computed with exact rational arithmetic in Python
-// (fractions.Fraction) and cross-checked with Python's float(); strtod_l and
-// strtof_l of Apple's libc and of glibc agree. Generated by
-// compact_hard_cases.py 5 (with hard_cases.py), see the pull request that
-// added this file.
-
-namespace float_hard_cases
-{
-
-struct hard_case
-{
-    const char* token;
-    std::uint64_t bits64;
-    std::uint32_t bits32;
-};
-
-inline const std::array<hard_case, 508>& cases()
-{
-    static const std::array<hard_case, 508> table =
-    {
-        {
-            {"-2.4703282292062327e-324", 0x8000000000000000u, 0x80000000u},
-            {"24703282292062328e-340", 0x0000000000000001u, 0x00000000u},
-            {"247032822920623272e-341", 0x0000000000000000u, 0x00000000u},
-            {"-0.2470328229206232721e-323", 0x8000000000000001u, 0x80000000u},
-            {"-0.24703282292062327208e-323", 0x8000000000000000u, 0x80000000u},
-            {"-2.4703282292062327209e-324", 0x8000000000000001u, 0x80000000u},
-            {"2.47032822920623272088e-324", 0x0000000000000000u, 0x00000000u},
-            {"247032822920623272089e-344", 0x0000000000000001u, 0x00000000u},
-            {"-247032822920623272088284396434e-353", 0x8000000000000000u, 0x80000000u},
-            {"0.247032822920623272088284396435e-323", 0x0000000000000001u, 0x00000000u},
-            {"-74109846876186981e-340", 0x8000000000000001u, 0x80000000u},
-            {"0.74109846876186982e-323", 0x0000000000000002u, 0x00000000u},
-            {"-0.7410984687618698162e-323", 0x8000000000000001u, 0x80000000u},
-            {"-7.410984687618698163e-324", 0x8000000000000002u, 0x80000000u},
-            {"7.4109846876186981626e-324", 0x0000000000000001u, 0x00000000u},
-            {"-74109846876186981627e-343", 0x8000000000000002u, 0x80000000u},
-            {"-741098468761869816264e-344", 0x8000000000000001u, 0x80000000u},
-            {"0.741098468761869816265e-323", 0x0000000000000002u, 0x00000000u},
-            {"0.741098468761869816264853189302e-323", 0x0000000000000001u, 0x00000000u},
-            {"-7.41098468761869816264853189303e-324", 0x8000000000000002u, 0x80000000u},
-            {"0.22250738585072006e-307", 0x000FFFFFFFFFFFFEu, 0x00000000u},
-            {"2.2250738585072007e-308", 0x000FFFFFFFFFFFFFu, 0x00000000u},
-            {"2.225073858507200641e-308", 0x000FFFFFFFFFFFFEu, 0x00000000u},
-            {"-2225073858507200642e-326", 0x800FFFFFFFFFFFFFu, 0x80000000u},
-            {"22250738585072006419e-327", 0x000FFFFFFFFFFFFEu, 0x00000000u},
-            {"0.2225073858507200642e-307", 0x000FFFFFFFFFFFFFu, 0x00000000u},
-            {"0.222507385850720064199e-307", 0x000FFFFFFFFFFFFEu, 0x00000000u},
-            {"2.225073858507200642e-308", 0x000FFFFFFFFFFFFFu, 0x00000000u},
-            {"-2.22507385850720064199176395546e-308", 0x800FFFFFFFFFFFFEu, 0x80000000u},
-            {"222507385850720064199176395547e-337", 0x000FFFFFFFFFFFFFu, 0x00000000u},
-            {"-2.2250738585072011e-308", 0x800FFFFFFFFFFFFFu, 0x80000000u},
-            {"-22250738585072012e-324", 0x8010000000000000u, 0x80000000u},
-            {"-2225073858507201136e-326", 0x800FFFFFFFFFFFFFu, 0x80000000u},
-            {"0.2225073858507201137e-307", 0x0010000000000000u, 0x00000000u},
-            {"0.2225073858507201136e-307", 0x000FFFFFFFFFFFFFu, 0x00000000u},
-            {"-2.2250738585072011361e-308", 0x8010000000000000u, 0x80000000u},
-            {"2.22507385850720113605e-308", 0x000FFFFFFFFFFFFFu, 0x00000000u},
-            {"222507385850720113606e-328", 0x0010000000000000u, 0x00000000u},
-            {"22250738585072011360574097967e-336", 0x000FFFFFFFFFFFFFu, 0x00000000u},
-            {"0.222507385850720113605740979671e-307", 0x0010000000000000u, 0x00000000u},
-            {"22250738585072016e-324", 0x0010000000000000u, 0x00000000u},
-            {"0.22250738585072017e-307", 0x0010000000000001u, 0x00000000u},
-            {"0.222507385850720163e-307", 0x0010000000000000u, 0x00000000u},
-            {"2.225073858507201631e-308", 0x0010000000000001u, 0x00000000u},
-            {"-2.2250738585072016301e-308", 0x8010000000000000u, 0x80000000u},
-            {"22250738585072016302e-327", 0x0010000000000001u, 0x00000000u},
-            {"-222507385850720163012e-328", 0x8010000000000000u, 0x80000000u},
-            {"0.222507385850720163013e-307", 0x0010000000000001u, 0x00000000u},
-            {"0.222507385850720163012305563795e-307", 0x0010000000000000u, 0x00000000u},
-            {"-2.22507385850720163012305563796e-308", 0x8010000000000001u, 0x80000000u},
-            {"0.17976931348623156E+309", 0x7FEFFFFFFFFFFFFEu, 0x7F800000u},
-            {"1.7976931348623157e308", 0x7FEFFFFFFFFFFFFFu, 0x7F800000u},
-            {"1.797693134862315608e308", 0x7FEFFFFFFFFFFFFEu, 0x7F800000u},
-            {"-1797693134862315609e290", 0xFFEFFFFFFFFFFFFFu, 0xFF800000u},
-            {"-17976931348623156083e289", 0xFFEFFFFFFFFFFFFEu, 0xFF800000u},
-            {"-0.17976931348623156084E+309", 0xFFEFFFFFFFFFFFFFu, 0xFF800000u},
-            {"0.179769313486231560835E+309", 0x7FEFFFFFFFFFFFFEu, 0x7F800000u},
-            {"-1.79769313486231560836e308", 0xFFEFFFFFFFFFFFFFu, 0xFF800000u},
-            {"1.79769313486231560835325876058e308", 0x7FEFFFFFFFFFFFFEu, 0x7F800000u},
-            {"179769313486231560835325876059e279", 0x7FEFFFFFFFFFFFFFu, 0x7F800000u},
-            {"1.7976931348623158e308", 0x7FEFFFFFFFFFFFFFu, 0x7F800000u},
-            {"17976931348623159e292", 0x7FF0000000000000u, 0x7F800000u},
-            {"1797693134862315807e290", 0x7FEFFFFFFFFFFFFFu, 0x7F800000u},
-            {"0.1797693134862315808E+309", 0x7FF0000000000000u, 0x7F800000u},
-            {"0.17976931348623158079E+309", 0x7FEFFFFFFFFFFFFFu, 0x7F800000u},
-            {"-1.797693134862315808e308", 0xFFF0000000000000u, 0xFF800000u},
-            {"1.79769313486231580793e308", 0x7FEFFFFFFFFFFFFFu, 0x7F800000u},
-            {"179769313486231580794e288", 0x7FF0000000000000u, 0x7F800000u},
-            {"179769313486231580793728971405e279", 0x7FEFFFFFFFFFFFFFu, 0x7F800000u},
-            {"-0.179769313486231580793728971406E+309", 0xFFF0000000000000u, 0xFF800000u},
-            {"100000000000000011102230246251565404236316680908203125e-53", 0x3FF0000000000000u, 0x3F800000u},
-            {"-1.00000000000000011102230246251565404236316680908203126", 0xBFF0000000000001u, 0xBF800000u},
-            {"1.00000000000000011102230246251565404236316680908203124e0", 0x3FF0000000000000u, 0x3F800000u},
-            {"10000000000000001110223024625156540423631668090820312501e-55", 0x3FF0000000000001u, 0x3F800000u},
-            {"1.00000000000000011102230246251565404236316680908203125000000000000000000001", 0x3FF0000000000001u, 0x3F800000u},
-            {"10000000000000001e-16", 0x3FF0000000000000u, 0x3F800000u},
-            {"1.0000000000000002", 0x3FF0000000000001u, 0x3F800000u},
-            {"1.000000000000000111", 0x3FF0000000000000u, 0x3F800000u},
-            {"1.000000000000000112e0", 0x3FF0000000000001u, 0x3F800000u},
-            {"1.000000000000000111e0", 0x3FF0000000000000u, 0x3F800000u},
-            {"-10000000000000001111e-19", 0xBFF0000000000001u, 0xBF800000u},
-            {"-100000000000000011102e-20", 0xBFF0000000000000u, 0xBF800000u},
-            {"-1.00000000000000011103", 0xBFF0000000000001u, 0xBF800000u},
-            {"1.00000000000000011102230246251", 0x3FF0000000000000u, 0x3F800000u},
-            {"1.00000000000000011102230246252e0", 0x3FF0000000000001u, 0x3F800000u},
-            {"-0.999999999999999944488848768742172978818416595458984375", 0xBFF0000000000000u, 0xBF800000u},
-            {"-9.99999999999999944488848768742172978818416595458984376e-1", 0xBFF0000000000000u, 0xBF800000u},
-            {"999999999999999944488848768742172978818416595458984374e-54", 0x3FEFFFFFFFFFFFFFu, 0x3F800000u},
-            {"0.99999999999999994448884876874217297881841659545898437501", 0x3FF0000000000000u, 0x3F800000u},
-            {"9.99999999999999944488848768742172978818416595458984375000000000000000000001e-1", 0x3FF0000000000000u, 0x3F800000u},
-            {"-0.99999999999999994", 0xBFEFFFFFFFFFFFFFu, 0xBF800000u},
-            {"9.9999999999999995e-1", 0x3FF0000000000000u, 0x3F800000u},
-            {"9.999999999999999444e-1", 0x3FEFFFFFFFFFFFFFu, 0x3F800000u},
-            {"9999999999999999445e-19", 0x3FF0000000000000u, 0x3F800000u},
-            {"99999999999999994448e-20", 0x3FEFFFFFFFFFFFFFu, 0x3F800000u},
-            {"-0.99999999999999994449", 0xBFF0000000000000u, 0xBF800000u},
-            {"0.999999999999999944488", 0x3FEFFFFFFFFFFFFFu, 0x3F800000u},
-            {"-9.99999999999999944489e-1", 0xBFF0000000000000u, 0xBF800000u},
-            {"9.99999999999999944488848768742e-1", 0x3FEFFFFFFFFFFFFFu, 0x3F800000u},
-            {"999999999999999944488848768743e-30", 0x3FF0000000000000u, 0x3F800000u},
-            {"-9.007199254740993e15", 0xC340000000000000u, 0xDA000000u},
-            {"9007199254740994e0", 0x4340000000000001u, 0x5A000000u},
-            {"9007199254740992", 0x4340000000000000u, 0x5A000000u},
-            {"9.00719925474099301e15", 0x4340000000000001u, 0x5A000000u},
-            {"9007199254740993000000000000000000001e-21", 0x4340000000000001u, 0x5A000000u},
-            {"-9007199254740993.000000000000000000000000000000", 0xC340000000000000u, 0xDA000000u},
-            {"90071992547409915e-1", 0x4340000000000000u, 0x5A000000u},
-            {"-9007199254740991.6", 0xC340000000000000u, 0xDA000000u},
-            {"9.0071992547409914e15", 0x433FFFFFFFFFFFFFu, 0x5A000000u},
-            {"-9007199254740991501e-3", 0xC340000000000000u, 0xDA000000u},
-            {"9007199254740991.5000000000000000000001", 0x4340000000000000u, 0x5A000000u},
-            {"9.0071992547409915000000000000000000000000000000e15", 0x4340000000000000u, 0x5A000000u},
-            {"0.100000000000000012490009027033011079765856266021728515625", 0x3FB999999999999Au, 0x3DCCCCCDu},
-            {"1.00000000000000012490009027033011079765856266021728515626e-1", 0x3FB999999999999Bu, 0x3DCCCCCDu},
-            {"100000000000000012490009027033011079765856266021728515624e-57", 0x3FB999999999999Au, 0x3DCCCCCDu},
-            {"0.10000000000000001249000902703301107976585626602172851562501", 0x3FB999999999999Bu, 0x3DCCCCCDu},
-            {"0.10000000000000001", 0x3FB999999999999Au, 0x3DCCCCCDu},
-            {"1.0000000000000002e-1", 0x3FB999999999999Bu, 0x3DCCCCCDu},
-            {"-1.000000000000000124e-1", 0xBFB999999999999Au, 0xBDCCCCCDu},
-            {"1000000000000000125e-19", 0x3FB999999999999Bu, 0x3DCCCCCDu},
-            {"-10000000000000001249e-20", 0xBFB999999999999Au, 0xBDCCCCCDu},
-            {"0.1000000000000000125", 0x3FB999999999999Bu, 0x3DCCCCCDu},
-            {"0.10000000000000001249", 0x3FB999999999999Au, 0x3DCCCCCDu},
-            {"1.00000000000000012491e-1", 0x3FB999999999999Bu, 0x3DCCCCCDu},
-            {"1.00000000000000012490009027033e-1", 0x3FB999999999999Au, 0x3DCCCCCDu},
-            {"100000000000000012490009027034e-30", 0x3FB999999999999Bu, 0x3DCCCCCDu},
-            {"2.45134755833537796875e14", 0x42EBDE5C4164D83Au, 0x575EF2E2u},
-            {"-245134755833537796876e-6", 0xC2EBDE5C4164D83Au, 0xD75EF2E2u},
-            {"-245134755833537.796874", 0xC2EBDE5C4164D839u, 0xD75EF2E2u},
-            {"2.4513475583353779687501e14", 0x42EBDE5C4164D83Au, 0x575EF2E2u},
-            {"245134755833537796875000000000000000000001e-27", 0x42EBDE5C4164D83Au, 0x575EF2E2u},
-            {"245134755833537.796875000000000000000000000000000000", 0x42EBDE5C4164D83Au, 0x575EF2E2u},
-            {"2.4513475583353779e14", 0x42EBDE5C4164D839u, 0x575EF2E2u},
-            {"2451347558335378e-1", 0x42EBDE5C4164D83Au, 0x575EF2E2u},
-            {"2451347558335377968e-4", 0x42EBDE5C4164D839u, 0x575EF2E2u},
-            {"245134755833537.7969", 0x42EBDE5C4164D83Au, 0x575EF2E2u},
-            {"245134755833537.79687", 0x42EBDE5C4164D839u, 0x575EF2E2u},
-            {"2.4513475583353779688e14", 0x42EBDE5C4164D83Au, 0x575EF2E2u},
-            {"181510327827821147441864013671875e-23", 0x41DB0C11CB91CE38u, 0x4ED8608Eu},
-            {"-1815103278.27821147441864013671876", 0xC1DB0C11CB91CE38u, 0xCED8608Eu},
-            {"1.81510327827821147441864013671874e9", 0x41DB0C11CB91CE37u, 0x4ED8608Eu},
-            {"18151032782782114744186401367187501e-25", 0x41DB0C11CB91CE38u, 0x4ED8608Eu},
-            {"1815103278.27821147441864013671875000000000000000000001", 0x41DB0C11CB91CE38u, 0x4ED8608Eu},
-            {"-1.81510327827821147441864013671875000000000000000000000000000000e9", 0xC1DB0C11CB91CE38u, 0xCED8608Eu},
-            {"18151032782782114e-7", 0x41DB0C11CB91CE37u, 0x4ED8608Eu},
-            {"1815103278.2782115", 0x41DB0C11CB91CE38u, 0x4ED8608Eu},
-            {"1815103278.278211474", 0x41DB0C11CB91CE37u, 0x4ED8608Eu},
-            {"-1.815103278278211475e9", 0xC1DB0C11CB91CE38u, 0xCED8608Eu},
-            {"1.8151032782782114744e9", 0x41DB0C11CB91CE37u, 0x4ED8608Eu},
-            {"-18151032782782114745e-10", 0xC1DB0C11CB91CE38u, 0xCED8608Eu},
-            {"181510327827821147441e-11", 0x41DB0C11CB91CE37u, 0x4ED8608Eu},
-            {"1815103278.27821147442", 0x41DB0C11CB91CE38u, 0x4ED8608Eu},
-            {"1815103278.27821147441864013671", 0x41DB0C11CB91CE37u, 0x4ED8608Eu},
-            {"1.81510327827821147441864013672e9", 0x41DB0C11CB91CE38u, 0x4ED8608Eu},
-            {"3809325632181785344", 0x43CA6EB8BD69FE2Au, 0x5E5375C6u},
-            {"3.809325632181785345e18", 0x43CA6EB8BD69FE2Au, 0x5E5375C6u},
-            {"3809325632181785343e0", 0x43CA6EB8BD69FE29u, 0x5E5375C6u},
-            {"3809325632181785344.01", 0x43CA6EB8BD69FE2Au, 0x5E5375C6u},
-            {"3.809325632181785344000000000000000000001e18", 0x43CA6EB8BD69FE2Au, 0x5E5375C6u},
-            {"3809325632181785344000000000000000000000000000000e-30", 0x43CA6EB8BD69FE2Au, 0x5E5375C6u},
-            {"3809325632181785300", 0x43CA6EB8BD69FE29u, 0x5E5375C6u},
-            {"3.8093256321817854e18", 0x43CA6EB8BD69FE2Au, 0x5E5375C6u},
-            {"4.046966549916366943359375e12", 0x428D7210076CE2F0u, 0x546B9080u},
-            {"4046966549916366943359376e-12", 0x428D7210076CE2F0u, 0x546B9080u},
-            {"4046966549916.366943359374", 0x428D7210076CE2EFu, 0x546B9080u},
-            {"4.04696654991636694335937501e12", 0x428D7210076CE2F0u, 0x546B9080u},
-            {"4046966549916366943359375000000000000000000001e-33", 0x428D7210076CE2F0u, 0x546B9080u},
-            {"-4046966549916.366943359375000000000000000000000000000000", 0xC28D7210076CE2F0u, 0xD46B9080u},
-            {"4.0469665499163669e12", 0x428D7210076CE2EFu, 0x546B9080u},
-            {"4046966549916367e-3", 0x428D7210076CE2F0u, 0x546B9080u},
-            {"4046966549916366943e-6", 0x428D7210076CE2EFu, 0x546B9080u},
-            {"4046966549916.366944", 0x428D7210076CE2F0u, 0x546B9080u},
-            {"4046966549916.3669433", 0x428D7210076CE2EFu, 0x546B9080u},
-            {"4.0469665499163669434e12", 0x428D7210076CE2F0u, 0x546B9080u},
-            {"-4.04696654991636694335e12", 0xC28D7210076CE2EFu, 0xD46B9080u},
-            {"404696654991636694336e-8", 0x428D7210076CE2F0u, 0x546B9080u},
-            {"28093802557000874e154", 0x63529C3B77330BDBu, 0x7F800000u},
-            {"0.28093802557000875E+171", 0x63529C3B77330BDCu, 0x7F800000u},
-            {"0.2809380255700087447E+171", 0x63529C3B77330BDBu, 0x7F800000u},
-            {"2.809380255700087448e170", 0x63529C3B77330BDCu, 0x7F800000u},
-            {"2.8093802557000874472e170", 0x63529C3B77330BDBu, 0x7F800000u},
-            {"28093802557000874473e151", 0x63529C3B77330BDCu, 0x7F800000u},
-            {"280938025570008744728e150", 0x63529C3B77330BDBu, 0x7F800000u},
-            {"-0.280938025570008744729E+171", 0xE3529C3B77330BDCu, 0xFF800000u},
-            {"-0.280938025570008744728403667979E+171", 0xE3529C3B77330BDBu, 0xFF800000u},
-            {"2.8093802557000874472840366798e170", 0x63529C3B77330BDCu, 0x7F800000u},
-            {"0.39523280297734525e-154", 0x1FE0F51BF17FD374u, 0x00000000u},
-            {"-3.9523280297734526e-155", 0x9FE0F51BF17FD375u, 0x80000000u},
-            {"3.952328029773452547e-155", 0x1FE0F51BF17FD374u, 0x00000000u},
-            {"3952328029773452548e-173", 0x1FE0F51BF17FD375u, 0x00000000u},
-            {"-39523280297734525478e-174", 0x9FE0F51BF17FD374u, 0x80000000u},
-            {"0.39523280297734525479e-154", 0x1FE0F51BF17FD375u, 0x00000000u},
-            {"-0.395232802977345254787e-154", 0x9FE0F51BF17FD374u, 0x80000000u},
-            {"-3.95232802977345254788e-155", 0x9FE0F51BF17FD375u, 0x80000000u},
-            {"-3.95232802977345254787245825501e-155", 0x9FE0F51BF17FD374u, 0x80000000u},
-            {"395232802977345254787245825502e-184", 0x1FE0F51BF17FD375u, 0x00000000u},
-            {"-1.0790205420931879e-276", 0x86A3209CA6233255u, 0x80000000u},
-            {"-1079020542093188e-291", 0x86A3209CA6233256u, 0x80000000u},
-            {"-1079020542093187947e-294", 0x86A3209CA6233255u, 0x80000000u},
-            {"0.1079020542093187948e-275", 0x06A3209CA6233256u, 0x00000000u},
-            {"0.1079020542093187947e-275", 0x06A3209CA6233255u, 0x00000000u},
-            {"1.0790205420931879471e-276", 0x06A3209CA6233256u, 0x00000000u},
-            {"1.07902054209318794701e-276", 0x06A3209CA6233255u, 0x00000000u},
-            {"-107902054209318794702e-296", 0x86A3209CA6233256u, 0x80000000u},
-            {"107902054209318794701153285302e-305", 0x06A3209CA6233255u, 0x00000000u},
-            {"-0.107902054209318794701153285303e-275", 0x86A3209CA6233256u, 0x80000000u},
-            {"58530471071351308e-228", 0x1413B446E6A16A3Bu, 0x00000000u},
-            {"-0.58530471071351309e-211", 0x9413B446E6A16A3Cu, 0x80000000u},
-            {"0.5853047107135130893e-211", 0x1413B446E6A16A3Bu, 0x00000000u},
-            {"-5.853047107135130894e-212", 0x9413B446E6A16A3Cu, 0x80000000u},
-            {"5.853047107135130893e-212", 0x1413B446E6A16A3Bu, 0x00000000u},
-            {"58530471071351308931e-231", 0x1413B446E6A16A3Cu, 0x00000000u},
-            {"-585304710713513089304e-232", 0x9413B446E6A16A3Bu, 0x80000000u},
-            {"0.585304710713513089305e-211", 0x1413B446E6A16A3Cu, 0x00000000u},
-            {"-0.585304710713513089304248824438e-211", 0x9413B446E6A16A3Bu, 0x80000000u},
-            {"5.85304710713513089304248824439e-212", 0x1413B446E6A16A3Cu, 0x00000000u},
-            {"0.19334214893983531e-78", 0x2F96ECBF1CFB10F6u, 0x00000000u},
-            {"1.9334214893983532e-79", 0x2F96ECBF1CFB10F7u, 0x00000000u},
-            {"1.933421489398353102e-79", 0x2F96ECBF1CFB10F6u, 0x00000000u},
-            {"1933421489398353103e-97", 0x2F96ECBF1CFB10F7u, 0x00000000u},
-            {"19334214893983531023e-98", 0x2F96ECBF1CFB10F6u, 0x00000000u},
-            {"0.19334214893983531024e-78", 0x2F96ECBF1CFB10F7u, 0x00000000u},
-            {"0.193342148939835310231e-78", 0x2F96ECBF1CFB10F6u, 0x00000000u},
-            {"1.93342148939835310232e-79", 0x2F96ECBF1CFB10F7u, 0x00000000u},
-            {"-1.93342148939835310231359014704e-79", 0xAF96ECBF1CFB10F6u, 0x80000000u},
-            {"193342148939835310231359014705e-108", 0x2F96ECBF1CFB10F7u, 0x00000000u},
-            {"2.9873358928024455e227", 0x6F2938807814E8A2u, 0x7F800000u},
-            {"29873358928024456e211", 0x6F2938807814E8A3u, 0x7F800000u},
-            {"298733589280244551e210", 0x6F2938807814E8A2u, 0x7F800000u},
-            {"0.2987335892802445511E+228", 0x6F2938807814E8A3u, 0x7F800000u},
-            {"0.29873358928024455109E+228", 0x6F2938807814E8A2u, 0x7F800000u},
-            {"2.987335892802445511e227", 0x6F2938807814E8A3u, 0x7F800000u},
-            {"-2.98733589280244551098e227", 0xEF2938807814E8A2u, 0xFF800000u},
-            {"-298733589280244551099e207", 0xEF2938807814E8A3u, 0xFF800000u},
-            {"298733589280244551098081559931e198", 0x6F2938807814E8A2u, 0x7F800000u},
-            {"0.298733589280244551098081559932E+228", 0x6F2938807814E8A3u, 0x7F800000u},
-            {"7.0064923216240853e-46", 0x3690000000000000u, 0x00000000u},
-            {"70064923216240854e-62", 0x3690000000000000u, 0x00000001u},
-            {"7006492321624085354e-64", 0x3690000000000000u, 0x00000000u},
-            {"-0.7006492321624085355e-45", 0xB690000000000000u, 0x80000001u},
-            {"0.70064923216240853546e-45", 0x3690000000000000u, 0x00000000u},
-            {"7.0064923216240853547e-46", 0x3690000000000000u, 0x00000001u},
-            {"7.00649232162408535461e-46", 0x3690000000000000u, 0x00000000u},
-            {"-700649232162408535462e-66", 0xB690000000000000u, 0x80000001u},
-            {"-700649232162408535461864791644e-75", 0xB690000000000000u, 0x80000000u},
-            {"0.700649232162408535461864791645e-45", 0x3690000000000000u, 0x00000001u},
-            {"21019476964872256e-61", 0x36A8000000000000u, 0x00000001u},
-            {"0.21019476964872257e-44", 0x36A8000000000000u, 0x00000002u},
-            {"-0.2101947696487225606e-44", 0xB6A8000000000000u, 0x80000001u},
-            {"-2.101947696487225607e-45", 0xB6A8000000000000u, 0x80000002u},
-            {"2.1019476964872256063e-45", 0x36A8000000000000u, 0x00000001u},
-            {"21019476964872256064e-64", 0x36A8000000000000u, 0x00000002u},
-            {"-210194769648722560638e-65", 0xB6A8000000000000u, 0x80000001u},
-            {"-0.210194769648722560639e-44", 0xB6A8000000000000u, 0x80000002u},
-            {"-0.210194769648722560638559437493e-44", 0xB6A8000000000000u, 0x80000001u},
-            {"2.10194769648722560638559437494e-45", 0x36A8000000000000u, 0x00000002u},
-            {"0.11754941406275178e-37", 0x380FFFFFA0000000u, 0x007FFFFEu},
-            {"1.1754941406275179e-38", 0x380FFFFFA0000000u, 0x007FFFFFu},
-            {"-1.175494140627517859e-38", 0xB80FFFFFA0000000u, 0x807FFFFEu},
-            {"117549414062751786e-55", 0x380FFFFFA0000000u, 0x007FFFFFu},
-            {"-11754941406275178592e-57", 0xB80FFFFFA0000000u, 0x807FFFFEu},
-            {"0.11754941406275178593e-37", 0x380FFFFFA0000000u, 0x007FFFFFu},
-            {"0.117549414062751785924e-37", 0x380FFFFFA0000000u, 0x007FFFFEu},
-            {"1.17549414062751785925e-38", 0x380FFFFFA0000000u, 0x007FFFFFu},
-            {"-1.17549414062751785924617589866e-38", 0xB80FFFFFA0000000u, 0x807FFFFEu},
-            {"117549414062751785924617589867e-67", 0x380FFFFFA0000000u, 0x007FFFFFu},
-            {"1.1754942807573642e-38", 0x380FFFFFDFFFFFFFu, 0x007FFFFFu},
-            {"11754942807573643e-54", 0x380FFFFFE0000000u, 0x00800000u},
-            {"1175494280757364291e-56", 0x380FFFFFE0000000u, 0x007FFFFFu},
-            {"0.1175494280757364292e-37", 0x380FFFFFE0000000u, 0x00800000u},
-            {"0.11754942807573642917e-37", 0x380FFFFFE0000000u, 0x007FFFFFu},
-            {"-1.1754942807573642918e-38", 0xB80FFFFFE0000000u, 0x80800000u},
-            {"-1.17549428075736429172e-38", 0xB80FFFFFE0000000u, 0x807FFFFFu},
-            {"-117549428075736429173e-58", 0xB80FFFFFE0000000u, 0x80800000u},
-            {"117549428075736429172788299103e-67", 0x380FFFFFE0000000u, 0x007FFFFFu},
-            {"0.117549428075736429172788299104e-37", 0x380FFFFFE0000000u, 0x00800000u},
-            {"11754944208872107e-54", 0x3810000010000000u, 0x00800000u},
-            {"-0.11754944208872108e-37", 0xB810000010000000u, 0x80800001u},
-            {"0.1175494420887210724e-37", 0x3810000010000000u, 0x00800000u},
-            {"1.175494420887210725e-38", 0x3810000010000000u, 0x00800001u},
-            {"1.1754944208872107242e-38", 0x3810000010000000u, 0x00800000u},
-            {"-11754944208872107243e-57", 0xB810000010000000u, 0x80800001u},
-            {"-11754944208872107242e-57", 0xB810000010000000u, 0x80800000u},
-            {"0.117549442088721072421e-37", 0x3810000010000000u, 0x00800001u},
-            {"0.11754944208872107242095900834e-37", 0x3810000010000000u, 0x00800000u},
-            {"-1.17549442088721072420959008341e-38", 0xB810000010000000u, 0x80800001u},
-            {"340282336497324057985868971510891282432", 0x47EFFFFFD0000000u, 0x7F7FFFFEu},
-            {"-3.40282336497324057985868971510891282433e38", 0xC7EFFFFFD0000000u, 0xFF7FFFFFu},
-            {"340282336497324057985868971510891282431e0", 0x47EFFFFFD0000000u, 0x7F7FFFFEu},
-            {"340282336497324057985868971510891282432.01", 0x47EFFFFFD0000000u, 0x7F7FFFFFu},
-            {"3.40282336497324057985868971510891282432000000000000000000001e38", 0x47EFFFFFD0000000u, 0x7F7FFFFFu},
-            {"340282336497324057985868971510891282432000000000000000000000000000000e-30", 0x47EFFFFFD0000000u, 0x7F7FFFFEu},
-            {"340282336497324050000000000000000000000", 0x47EFFFFFD0000000u, 0x7F7FFFFEu},
-            {"3.4028233649732406e38", 0x47EFFFFFD0000000u, 0x7F7FFFFFu},
-            {"3.402823364973240579e38", 0x47EFFFFFD0000000u, 0x7F7FFFFEu},
-            {"340282336497324058e21", 0x47EFFFFFD0000000u, 0x7F7FFFFFu},
-            {"34028233649732405798e19", 0x47EFFFFFD0000000u, 0x7F7FFFFEu},
-            {"340282336497324057990000000000000000000", 0x47EFFFFFD0000000u, 0x7F7FFFFFu},
-            {"340282336497324057985000000000000000000", 0x47EFFFFFD0000000u, 0x7F7FFFFEu},
-            {"3.40282336497324057986e38", 0x47EFFFFFD0000000u, 0x7F7FFFFFu},
-            {"3.4028233649732405798586897151e38", 0x47EFFFFFD0000000u, 0x7F7FFFFEu},
-            {"340282336497324057985868971511e9", 0x47EFFFFFD0000000u, 0x7F7FFFFFu},
-            {"3.40282356779733661637539395458142568448e38", 0x47EFFFFFF0000000u, 0x7F800000u},
-            {"-340282356779733661637539395458142568449e0", 0xC7EFFFFFF0000000u, 0xFF800000u},
-            {"340282356779733661637539395458142568447", 0x47EFFFFFF0000000u, 0x7F7FFFFFu},
-            {"3.4028235677973366163753939545814256844801e38", 0x47EFFFFFF0000000u, 0x7F800000u},
-            {"340282356779733661637539395458142568448000000000000000000001e-21", 0x47EFFFFFF0000000u, 0x7F800000u},
-            {"340282356779733661637539395458142568448.000000000000000000000000000000", 0x47EFFFFFF0000000u, 0x7F800000u},
-            {"3.4028235677973366e38", 0x47EFFFFFF0000000u, 0x7F7FFFFFu},
-            {"-34028235677973367e22", 0xC7EFFFFFF0000000u, 0xFF800000u},
-            {"3402823567797336616e20", 0x47EFFFFFF0000000u, 0x7F7FFFFFu},
-            {"340282356779733661700000000000000000000", 0x47EFFFFFF0000000u, 0x7F800000u},
-            {"340282356779733661630000000000000000000", 0x47EFFFFFF0000000u, 0x7F7FFFFFu},
-            {"3.4028235677973366164e38", 0x47EFFFFFF0000000u, 0x7F800000u},
-            {"3.40282356779733661637e38", 0x47EFFFFFF0000000u, 0x7F7FFFFFu},
-            {"340282356779733661638e18", 0x47EFFFFFF0000000u, 0x7F800000u},
-            {"-340282356779733661637539395458e9", 0xC7EFFFFFF0000000u, 0xFF7FFFFFu},
-            {"340282356779733661637539395459000000000", 0x47EFFFFFF0000000u, 0x7F800000u},
-            {"-1000000059604644775390625e-24", 0xBFF0000010000000u, 0xBF800000u},
-            {"-1.000000059604644775390626", 0xBFF0000010000000u, 0xBF800001u},
-            {"-1.000000059604644775390624e0", 0xBFF0000010000000u, 0xBF800000u},
-            {"100000005960464477539062501e-26", 0x3FF0000010000000u, 0x3F800001u},
-            {"1.000000059604644775390625000000000000000000001", 0x3FF0000010000000u, 0x3F800001u},
-            {"1.000000059604644775390625000000000000000000000000000000e0", 0x3FF0000010000000u, 0x3F800000u},
-            {"-10000000596046447e-16", 0xBFF0000010000000u, 0xBF800000u},
-            {"1.0000000596046448", 0x3FF0000010000000u, 0x3F800001u},
-            {"1.000000059604644775", 0x3FF0000010000000u, 0x3F800000u},
-            {"-1.000000059604644776e0", 0xBFF0000010000000u, 0xBF800001u},
-            {"-1.0000000596046447753e0", 0xBFF0000010000000u, 0xBF800000u},
-            {"10000000596046447754e-19", 0x3FF0000010000000u, 0x3F800001u},
-            {"100000005960464477539e-20", 0x3FF0000010000000u, 0x3F800000u},
-            {"-1.0000000596046447754", 0xBFF0000010000000u, 0xBF800001u},
-            {"0.9999999701976776123046875", 0x3FEFFFFFF0000000u, 0x3F800000u},
-            {"9.999999701976776123046876e-1", 0x3FEFFFFFF0000000u, 0x3F800000u},
-            {"9999999701976776123046874e-25", 0x3FEFFFFFF0000000u, 0x3F7FFFFFu},
-            {"0.999999970197677612304687501", 0x3FEFFFFFF0000000u, 0x3F800000u},
-            {"-9.999999701976776123046875000000000000000000001e-1", 0xBFEFFFFFF0000000u, 0xBF800000u},
-            {"-9999999701976776123046875000000000000000000000000000000e-55", 0xBFEFFFFFF0000000u, 0xBF800000u},
-            {"0.99999997019767761", 0x3FEFFFFFF0000000u, 0x3F7FFFFFu},
-            {"-9.9999997019767762e-1", 0xBFEFFFFFF0000000u, 0xBF800000u},
-            {"-9.999999701976776123e-1", 0xBFEFFFFFF0000000u, 0xBF7FFFFFu},
-            {"9999999701976776124e-19", 0x3FEFFFFFF0000000u, 0x3F800000u},
-            {"9999999701976776123e-19", 0x3FEFFFFFF0000000u, 0x3F7FFFFFu},
-            {"0.99999997019767761231", 0x3FEFFFFFF0000000u, 0x3F800000u},
-            {"-0.999999970197677612304", 0xBFEFFFFFF0000000u, 0xBF7FFFFFu},
-            {"9.99999970197677612305e-1", 0x3FEFFFFFF0000000u, 0x3F800000u},
-            {"-1.6777217e7", 0xC170000010000000u, 0xCB800000u},
-            {"16777218e0", 0x4170000020000000u, 0x4B800001u},
-            {"16777216", 0x4170000000000000u, 0x4B800000u},
-            {"-1.677721701e7", 0xC17000001028F5C3u, 0xCB800001u},
-            {"-16777217000000000000000000001e-21", 0xC170000010000000u, 0xCB800001u},
-            {"16777217.000000000000000000000000000000", 0x4170000010000000u, 0x4B800000u},
-            {"167772155e-1", 0x416FFFFFF0000000u, 0x4B800000u},
-            {"16777215.6", 0x416FFFFFF3333333u, 0x4B800000u},
-            {"1.67772154e7", 0x416FFFFFECCCCCCDu, 0x4B7FFFFFu},
-            {"16777215501e-3", 0x416FFFFFF0083127u, 0x4B800000u},
-            {"-16777215.5000000000000000000001", 0xC16FFFFFF0000000u, 0xCB800000u},
-            {"-1.67772155000000000000000000000000000000e7", 0xC16FFFFFF0000000u, 0xCB800000u},
-            {"0.1000000052154064178466796875", 0x3FB99999B0000000u, 0x3DCCCCCEu},
-            {"1.000000052154064178466796876e-1", 0x3FB99999B0000000u, 0x3DCCCCCEu},
-            {"-1000000052154064178466796874e-28", 0xBFB99999B0000000u, 0xBDCCCCCDu},
-            {"0.100000005215406417846679687501", 0x3FB99999B0000000u, 0x3DCCCCCEu},
-            {"1.000000052154064178466796875000000000000000000001e-1", 0x3FB99999B0000000u, 0x3DCCCCCEu},
-            {"-1000000052154064178466796875000000000000000000000000000000e-58", 0xBFB99999B0000000u, 0xBDCCCCCEu},
-            {"-0.10000000521540641", 0xBFB99999AFFFFFFFu, 0xBDCCCCCDu},
-            {"-1.0000000521540642e-1", 0xBFB99999B0000000u, 0xBDCCCCCEu},
-            {"1.000000052154064178e-1", 0x3FB99999B0000000u, 0x3DCCCCCDu},
-            {"-1000000052154064179e-19", 0xBFB99999B0000000u, 0xBDCCCCCEu},
-            {"10000000521540641784e-20", 0x3FB99999B0000000u, 0x3DCCCCCDu},
-            {"0.10000000521540641785", 0x3FB99999B0000000u, 0x3DCCCCCEu},
-            {"0.100000005215406417846", 0x3FB99999B0000000u, 0x3DCCCCCDu},
-            {"1.00000005215406417847e-1", 0x3FB99999B0000000u, 0x3DCCCCCEu},
-            {"5.429001220703125e3", 0x40B5350050000000u, 0x45A9A802u},
-            {"-5429001220703126e-12", 0xC0B5350050000001u, 0xC5A9A803u},
-            {"-5429.001220703124", 0xC0B535004FFFFFFFu, 0xC5A9A802u},
-            {"5.42900122070312501e3", 0x40B5350050000000u, 0x45A9A803u},
-            {"5429001220703125000000000000000000001e-33", 0x40B5350050000000u, 0x45A9A803u},
-            {"-5429.001220703125000000000000000000000000000000", 0xC0B5350050000000u, 0xC5A9A802u},
-            {"503719056e0", 0x41BE062490000000u, 0x4DF03124u},
-            {"503719057", 0x41BE062491000000u, 0x4DF03125u},
-            {"5.03719055e8", 0x41BE06248F000000u, 0x4DF03124u},
-            {"50371905601e-2", 0x41BE062490028F5Cu, 0x4DF03125u},
-            {"503719056.000000000000000000001", 0x41BE062490000000u, 0x4DF03125u},
-            {"5.03719056000000000000000000000000000000e8", 0x41BE062490000000u, 0x4DF03124u},
-            {"-92331620", 0xC196037990000000u, 0xCCB01BCCu},
-            {"9.233163e7", 0x41960379B8000000u, 0x4CB01BCEu},
-            {"9233161e1", 0x4196037968000000u, 0x4CB01BCBu},
-            {"92331620.1", 0x4196037990666666u, 0x4CB01BCDu},
-            {"9.233162000000000000000000001e7", 0x4196037990000000u, 0x4CB01BCDu},
-            {"9233162000000000000000000000000000000e-29", 0x4196037990000000u, 0x4CB01BCCu},
-            {"3.002458625e6", 0x4146E82D50000000u, 0x4A37416Au},
-            {"3002458626e-3", 0x4146E82D5020C49Cu, 0x4A37416Bu},
-            {"3002458.624", 0x4146E82D4FDF3B64u, 0x4A37416Au},
-            {"-3.00245862501e6", 0xC146E82D500053E3u, 0xCA37416Bu},
-            {"3002458625000000000000000000001e-24", 0x4146E82D50000000u, 0x4A37416Bu},
-            {"3002458.625000000000000000000000000000000", 0x4146E82D50000000u, 0x4A37416Au},
-            {"-1095485584696182596504479582065262592e1", 0xC7A07BA830000000u, 0xFD03DD42u},
-            {"10954855846961825965044795820652625930", 0x47A07BA830000000u, 0x7D03DD42u},
-            {"1.095485584696182596504479582065262591e37", 0x47A07BA830000000u, 0x7D03DD41u},
-            {"109548558469618259650447958206526259201e-1", 0x47A07BA830000000u, 0x7D03DD42u},
-            {"-10954855846961825965044795820652625920.00000000000000000001", 0xC7A07BA830000000u, 0xFD03DD42u},
-            {"1.095485584696182596504479582065262592000000000000000000000000000000e37", 0x47A07BA830000000u, 0x7D03DD42u},
-            {"10954855846961825e21", 0x47A07BA830000000u, 0x7D03DD41u},
-            {"10954855846961826000000000000000000000", 0x47A07BA830000000u, 0x7D03DD42u},
-            {"10954855846961825960000000000000000000", 0x47A07BA830000000u, 0x7D03DD41u},
-            {"1.095485584696182597e37", 0x47A07BA830000000u, 0x7D03DD42u},
-            {"1.0954855846961825965e37", 0x47A07BA830000000u, 0x7D03DD41u},
-            {"-10954855846961825966e18", 0xC7A07BA830000000u, 0xFD03DD42u},
-            {"10954855846961825965e18", 0x47A07BA830000000u, 0x7D03DD41u},
-            {"10954855846961825965100000000000000000", 0x47A07BA830000000u, 0x7D03DD42u},
-            {"10954855846961825965044795820600000000", 0x47A07BA830000000u, 0x7D03DD41u},
-            {"-1.09548558469618259650447958207e37", 0xC7A07BA830000000u, 0xFD03DD42u},
-            {"1.6449216019182103706535606608388384863861375606575165875256061553955078126e-21", 0x3B9F125A50000000u, 0x1CF892D3u},
-            {"-16449216019182103706535606608388384863861375606575165875256061553955078124e-94", 0xBB9F125A50000000u, 0x9CF892D2u},
-            {"-0.0000000000000000000016449216019182103", 0xBB9F125A50000000u, 0x9CF892D2u},
-            {"-1.6449216019182104e-21", 0xBB9F125A50000000u, 0x9CF892D3u},
-            {"-1.64492160191821037e-21", 0xBB9F125A50000000u, 0x9CF892D2u},
-            {"-1644921601918210371e-39", 0xBB9F125A50000000u, 0x9CF892D3u},
-            {"-16449216019182103706e-40", 0xBB9F125A50000000u, 0x9CF892D2u},
-            {"0.0000000000000000000016449216019182103707", 0x3B9F125A50000000u, 0x1CF892D3u},
-            {"0.00000000000000000000164492160191821037065", 0x3B9F125A50000000u, 0x1CF892D2u},
-            {"1.64492160191821037066e-21", 0x3B9F125A50000000u, 0x1CF892D3u},
-            {"1.64492160191821037065356066083e-21", 0x3B9F125A50000000u, 0x1CF892D2u},
-            {"164492160191821037065356066084e-50", 0x3B9F125A50000000u, 0x1CF892D3u},
-            {"6.565061509609222412109375e-1", 0x3FE5021930000000u, 0x3F2810CAu},
-            {"6565061509609222412109376e-25", 0x3FE5021930000000u, 0x3F2810CAu},
-            {"0.6565061509609222412109374", 0x3FE5021930000000u, 0x3F2810C9u},
-            {"6.56506150960922241210937501e-1", 0x3FE5021930000000u, 0x3F2810CAu},
-            {"6565061509609222412109375000000000000000000001e-46", 0x3FE5021930000000u, 0x3F2810CAu},
-            {"-0.6565061509609222412109375000000000000000000000000000000", 0xBFE5021930000000u, 0xBF2810CAu},
-            {"6.5650615096092224e-1", 0x3FE5021930000000u, 0x3F2810C9u},
-            {"-65650615096092225e-17", 0xBFE5021930000000u, 0xBF2810CAu},
-            {"6565061509609222412e-19", 0x3FE5021930000000u, 0x3F2810C9u},
-            {"0.6565061509609222413", 0x3FE5021930000000u, 0x3F2810CAu},
-            {"0.65650615096092224121", 0x3FE5021930000000u, 0x3F2810C9u},
-            {"6.5650615096092224122e-1", 0x3FE5021930000000u, 0x3F2810CAu},
-            {"-6.5650615096092224121e-1", 0xBFE5021930000000u, 0xBF2810C9u},
-            {"656506150960922241211e-21", 0x3FE5021930000000u, 0x3F2810CAu},
-            {"18014627239033005156980393746124491372029297053813934326171875e-77", 0x3CA9F63970000000u, 0x254FB1CCu},
-            {"0.00000000000000018014627239033005156980393746124491372029297053813934326171876", 0x3CA9F63970000000u, 0x254FB1CCu},
-            {"-1.8014627239033005156980393746124491372029297053813934326171874e-16", 0xBCA9F63970000000u, 0xA54FB1CBu},
-            {"1801462723903300515698039374612449137202929705381393432617187501e-79", 0x3CA9F63970000000u, 0x254FB1CCu},
-            {"18014627239033005e-32", 0x3CA9F63970000000u, 0x254FB1CBu},
-            {"0.00000000000000018014627239033006", 0x3CA9F63970000000u, 0x254FB1CCu},
-            {"0.0000000000000001801462723903300515", 0x3CA9F63970000000u, 0x254FB1CBu},
-            {"-1.801462723903300516e-16", 0xBCA9F63970000000u, 0xA54FB1CCu},
-            {"-1.8014627239033005156e-16", 0xBCA9F63970000000u, 0xA54FB1CBu},
-            {"18014627239033005157e-35", 0x3CA9F63970000000u, 0x254FB1CCu},
-            {"180146272390330051569e-36", 0x3CA9F63970000000u, 0x254FB1CBu},
-            {"-0.00000000000000018014627239033005157", 0xBCA9F63970000000u, 0xA54FB1CCu},
-            {"0.000000000000000180146272390330051569803937461", 0x3CA9F63970000000u, 0x254FB1CBu},
-            {"1.80146272390330051569803937462e-16", 0x3CA9F63970000000u, 0x254FB1CCu},
-            {"0.05534819327294826507568359375", 0x3FAC569930000000u, 0x3D62B4CAu},
-            {"-5.534819327294826507568359376e-2", 0xBFAC569930000000u, 0xBD62B4CAu},
-            {"5534819327294826507568359374e-29", 0x3FAC569930000000u, 0x3D62B4C9u},
-            {"-0.0553481932729482650756835937501", 0xBFAC569930000000u, 0xBD62B4CAu},
-            {"5.534819327294826507568359375000000000000000000001e-2", 0x3FAC569930000000u, 0x3D62B4CAu},
-            {"5534819327294826507568359375000000000000000000000000000000e-59", 0x3FAC569930000000u, 0x3D62B4CAu},
-            {"0.055348193272948265", 0x3FAC569930000000u, 0x3D62B4C9u},
-            {"5.5348193272948266e-2", 0x3FAC569930000000u, 0x3D62B4CAu},
-            {"5.534819327294826507e-2", 0x3FAC569930000000u, 0x3D62B4C9u},
-            {"5534819327294826508e-20", 0x3FAC569930000000u, 0x3D62B4CAu},
-            {"55348193272948265075e-21", 0x3FAC569930000000u, 0x3D62B4C9u},
-            {"-0.055348193272948265076", 0xBFAC569930000000u, 0xBD62B4CAu},
-            {"0.0553481932729482650756", 0x3FAC569930000000u, 0x3D62B4C9u},
-            {"-5.53481932729482650757e-2", 0xBFAC569930000000u, 0xBD62B4CAu},
-            {"5.179692133247783258005389047985340416e36", 0x478F2C9450000000u, 0x7C7964A2u},
-            {"5179692133247783258005389047985340417e0", 0x478F2C9450000000u, 0x7C7964A3u},
-            {"5179692133247783258005389047985340415", 0x478F2C9450000000u, 0x7C7964A2u},
-            {"-5.17969213324778325800538904798534041601e36", 0xC78F2C9450000000u, 0xFC7964A3u},
-            {"-5179692133247783258005389047985340416000000000000000000001e-21", 0xC78F2C9450000000u, 0xFC7964A3u},
-            {"-5179692133247783258005389047985340416.000000000000000000000000000000", 0xC78F2C9450000000u, 0xFC7964A2u},
-            {"5.1796921332477832e36", 0x478F2C9450000000u, 0x7C7964A2u},
-            {"51796921332477833e20", 0x478F2C9450000000u, 0x7C7964A3u},
-            {"-5179692133247783258e18", 0xC78F2C9450000000u, 0xFC7964A2u},
-            {"5179692133247783259000000000000000000", 0x478F2C9450000000u, 0x7C7964A3u},
-            {"5179692133247783258000000000000000000", 0x478F2C9450000000u, 0x7C7964A2u},
-            {"5.1796921332477832581e36", 0x478F2C9450000000u, 0x7C7964A3u},
-            {"-5.179692133247783258e36", 0xC78F2C9450000000u, 0xFC7964A2u},
-            {"-517969213324778325801e16", 0xC78F2C9450000000u, 0xFC7964A3u},
-            {"517969213324778325800538904798e7", 0x478F2C9450000000u, 0x7C7964A2u},
-            {"5179692133247783258005389047990000000", 0x478F2C9450000000u, 0x7C7964A3u},
-            {
-                "0.22250738585072011360574097967091319759348195463516456480234261097248222220210769455165295239081350"
-                "8791414915891303962110687008643869459464552765720740782062174337998814106326732925355228688137214901"
-                "2981122451451889849057222307285255133155755015914397476397983411801999323962548289017107081850690630"
-                "6666559949382757725720157630626906633326475653000092458883164330377797918696120494973903778297049050"
-                "5108060994073026293712895895000358379996720725430436028407889577179615094551674824347103070260914462"
-                "1572289880258182545180325707018860872113128079512233426288368622321503775666622503982534335974568884"
-                "4239002654981983854879482922068947216898310996983658468140228542433306603398508864458040010349339704"
-                "2756718644338377048603786162277173854562306587467901408672332763671875e-307", 0x0010000000000000u, 0x00000000u
-            },
-            {
-                "2.22507385850720113605740979670913197593481954635164564802342610972482222202107694551652952390813508"
-                "7914149158913039621106870086438694594645527657207407820621743379988141063267329253552286881372149012"
-                "9811224514518898490572223072852551331557550159143974763979834118019993239625482890171070818506906306"
-                "6665599493827577257201576306269066333264756530000924588831643303777979186961204949739037782970490505"
-                "1080609940730262937128958950003583799967207254304360284078895771796150945516748243471030702609144621"
-                "5722898802581825451803257070188608721131280795122334262883686223215037756666225039825343359745688844"
-                "2390026549819838548794829220689472168983109969836584681402285424333066033985088644580400103493397042"
-                "756718644338377048603786162277173854562306587467901408672332763671875000000000000000000001e-308", 0x0010000000000000u, 0x00000000u
-            },
-            {
-                "0.11754942807573642917278829910357665133228589927589904276829631184250030649651730385585324256680905"
-                "8189392089843750000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
-                "0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
-                "0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
-                "0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
-                "0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
-                "0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
-                "00000000000000000000000000000000000000000000000000000000000000000e-37", 0x380FFFFFE0000000u, 0x00800000u
-            },
-            {
-                "1175494280757364291727882991035766513322858992758990427682963118425003064965173038558532425668090581"
-                "8939208984375000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
-                "0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
-                "0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
-                "0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
-                "0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
-                "0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
-                "00000000000001e-751", 0x380FFFFFE0000000u, 0x00800000u
-            },
-            {"0", 0x0000000000000000u, 0x00000000u},
-            {"-0", 0x8000000000000000u, 0x80000000u},
-            {"0.0", 0x0000000000000000u, 0x00000000u},
-            {"-0.0", 0x8000000000000000u, 0x80000000u},
-            {"0e999999999999999999999", 0x0000000000000000u, 0x00000000u},
-            {"-0.000e-99999", 0x8000000000000000u, 0x80000000u},
-            {"1e-400", 0x0000000000000000u, 0x00000000u},
-            {"-1e-400", 0x8000000000000000u, 0x80000000u},
-            {"1e400", 0x7FF0000000000000u, 0x7F800000u},
-            {"-1e400", 0xFFF0000000000000u, 0xFF800000u},
-            {"1e-50", 0x358DEE7A4AD4B81Fu, 0x00000000u},
-            {"-1e-50", 0xB58DEE7A4AD4B81Fu, 0x80000000u},
-            {"1e39", 0x48078287F49C4A1Du, 0x7F800000u},
-            {"-1e39", 0xC8078287F49C4A1Du, 0xFF800000u},
-            {"1e99999999999999999999999999", 0x7FF0000000000000u, 0x7F800000u},
-            {"1e-99999999999999999999999999", 0x0000000000000000u, 0x00000000u},
-            {"1e0000000000000000000000000000000000000000308", 0x7FE1CCF385EBC8A0u, 0x7F800000u},
-            {"123456789012345678901234567890e-30", 0x3FBF9ADD3746F65Fu, 0x3DFCD6EAu},
-            {"18446744073709551615", 0x43F0000000000000u, 0x5F800000u},
-            {"18446744073709551616", 0x43F0000000000000u, 0x5F800000u},
-            {"-9223372036854775808", 0xC3E0000000000000u, 0xDF000000u},
-            {"-9223372036854775809", 0xC3E0000000000000u, 0xDF000000u},
-        }
-    };
-    return table;
-}
-
-} // namespace float_hard_cases

tests/src/unit-allocator.cpp

@@ -479,6 +479,77 @@ TEST_CASE("deep copy uses the provided allocator")
     CHECK(copy == j);
 }
 
+namespace
+{
+// the number of constructions countdown_allocator lets happen, including the
+// one that fails; 0 means none ever fails
+std::size_t constructions_until_failure = 0;
+
+template<class T>
+struct countdown_allocator : std::allocator<T>
+{
+    using std::allocator<T>::allocator;
+
+    template<class U, class... Args>
+    void construct(U* p, Args&& ... args)
+    {
+        if (constructions_until_failure != 0 && --constructions_until_failure == 0)
+        {
+            throw std::bad_alloc();
+        }
+
+        ::new (static_cast<void*>(p)) U(std::forward<Args>(args)...);
+    }
+
+    template <class U>
+    struct rebind
+    {
+        using other = countdown_allocator<U>;
+    };
+};
+} // namespace
+
+TEST_CASE("converting a deeply nested value from another specialization fails cleanly (#5650)")
+{
+    using countdown_json = nlohmann::basic_json<std::map,
+          std::vector,
+          std::string,
+          bool,
+          std::int64_t,
+          std::uint64_t,
+          double,
+          countdown_allocator>;
+
+    // deeper than the 128 levels the converting constructor descends into, so
+    // that failures land on both sides of the bound - or, built with
+    // JSON_NO_THREAD_LOCAL, all in the iterative conversion
+    json j = {1, "two", {{"three", 3}}};
+    for (std::size_t i = 0; i < 150; ++i)
+    {
+        j = json{{"a", json::array({j, "sibling"})}};
+    }
+
+    // Fail every construction in turn. Each failure has to reach the caller,
+    // and everything built until then has to be destroyed cleanly.
+    std::size_t failures = 0;
+    for (std::size_t n = 1;; ++n)
+    {
+        constructions_until_failure = n;
+        try
+        {
+            const countdown_json converted = j;
+            constructions_until_failure = 0;
+            CHECK(converted.dump() == j.dump());
+            break;
+        }
+        catch (const std::bad_alloc&)
+        {
+            ++failures;
+        }
+    }
+    CHECK(failures > 0);
+}
+
 namespace
 {
 template<class T>

tests/src/unit-bjdata.cpp

@@ -4025,6 +4025,30 @@ TEST_CASE("Universal Binary JSON Specification Examples 1")
             CHECK(json::to_bjdata(j) == v);
             CHECK(json::from_bjdata(v) == j);
         }
+
+        SECTION("ill-formed UTF-8 (see #5651)")
+        {
+            // a string value whose bytes are not valid UTF-8 (0xC0 0xAE is an
+            // overlong encoding of '.') is rejected at decode time, matching
+            // every other kind of malformed binary input, and to_bjdata()
+            // rejects it as well, so a value it accepts can always be read
+            // back
+            const std::vector<uint8_t> v = {'S', 'i', 2, 0xc0, 0xae};
+            json _;
+            CHECK_THROWS_WITH_AS(_ = json::from_bjdata(v), "[json.exception.parse_error.113] parse error at byte 5: syntax error while parsing BJData string: invalid string: ill-formed UTF-8 byte", json::parse_error&);
+            CHECK(json::from_bjdata(v, true, false).is_discarded());
+
+            CHECK_THROWS_WITH_AS(json::to_bjdata(json("\xFF")), "[json.exception.type_error.316] invalid UTF-8 byte at index 0: 0xFF", json::type_error&);
+            // a truncated multi-byte sequence
+            CHECK_THROWS_WITH_AS(json::to_bjdata(json("\xC3")), "[json.exception.type_error.316] invalid UTF-8 byte at index 0: 0xC3", json::type_error&);
+            // an encoded surrogate half (U+D800)
+            CHECK_THROWS_WITH_AS(json::to_bjdata(json("\xED\xA0\x80")), "[json.exception.type_error.316] invalid UTF-8 byte at index 0: 0xED", json::type_error&);
+            // an overlong encoding of '.'
+            CHECK_THROWS_WITH_AS(json::to_bjdata(json("\xC0\xAF")), "[json.exception.type_error.316] invalid UTF-8 byte at index 0: 0xC0", json::type_error&);
+
+            // an object key with ill-formed UTF-8 is rejected the same way
+            CHECK_THROWS_WITH_AS(json::to_bjdata(json{{"\xFF", 1}}), "[json.exception.type_error.316] invalid UTF-8 byte at index 0: 0xFF", json::type_error&);
+        }
     }
 
     SECTION("Array Type")

tests/src/unit-bson.cpp

@@ -151,6 +151,44 @@ TEST_CASE("BSON")
 #endif
     }
 
+    SECTION("ill-formed UTF-8 (see #5651)")
+    {
+        // a BSON document {"s": "\xC0\xAE"} (0xC0 0xAE is an overlong
+        // encoding of '.'); the reader rejects an ill-formed string value at
+        // decode time
+        const std::vector<uint8_t> v =
+        {
+            0x0F, 0x00, 0x00, 0x00, // document length
+            0x02, 's', 0x00,        // type 0x02 (string), key "s"
+            0x03, 0x00, 0x00, 0x00, // string length (including null)
+            0xc0, 0xae, 0x00,       // string content and its null terminator
+            0x00                    // document terminator
+        };
+        json _;
+        CHECK_THROWS_WITH_AS(_ = json::from_bson(v), "[json.exception.parse_error.113] parse error at byte 13: syntax error while parsing BSON string: invalid string: ill-formed UTF-8 byte", json::parse_error&);
+        CHECK(json::from_bson(v, true, false).is_discarded());
+
+        // to_bson() rejects the same kind of ill-formed string value, before
+        // any bytes reach the output adapter (the BSON document length
+        // prefix must be known up front, so nothing is written incrementally)
+        std::vector<std::uint8_t> out{0x42}; // a sentinel byte the writer must not touch
+        CHECK_THROWS_WITH_AS(json::to_bson(json{{"s", "\xFF"}}, nlohmann::detail::output_adapter<std::uint8_t>(out)), "[json.exception.type_error.316] invalid UTF-8 byte at index 0: 0xFF", json::type_error&);
+        CHECK(out == std::vector<std::uint8_t> {0x42});
+
+        CHECK_THROWS_WITH_AS(json::to_bson(json{{"s", "\xFF"}}), "[json.exception.type_error.316] invalid UTF-8 byte at index 0: 0xFF", json::type_error&);
+        // a truncated multi-byte sequence
+        CHECK_THROWS_WITH_AS(json::to_bson(json{{"s", "\xC3"}}), "[json.exception.type_error.316] invalid UTF-8 byte at index 0: 0xC3", json::type_error&);
+        // an encoded surrogate half (U+D800)
+        CHECK_THROWS_WITH_AS(json::to_bson(json{{"s", "\xED\xA0\x80"}}), "[json.exception.type_error.316] invalid UTF-8 byte at index 0: 0xED", json::type_error&);
+        // an overlong encoding of '.'
+        CHECK_THROWS_WITH_AS(json::to_bson(json{{"s", "\xC0\xAF"}}), "[json.exception.type_error.316] invalid UTF-8 byte at index 0: 0xC0", json::type_error&);
+
+        // an object key with ill-formed UTF-8 is rejected as well; unlike
+        // the reader (which never validates element names), the writer
+        // checks both string values and object keys
+        CHECK_THROWS_WITH_AS(json::to_bson(json{{"\xFF", 1}}), "[json.exception.type_error.316] invalid UTF-8 byte at index 0: 0xFF", json::type_error&);
+    }
+
     SECTION("lengths exceeding INT32_MAX cannot be serialized to BSON")
     {
         // out_of_range.412 is thrown from a single shared helper

tests/src/unit-cbor.cpp

@@ -1896,6 +1896,25 @@ TEST_CASE("CBOR")
             CHECK(json::from_cbor(json::to_cbor(j)) == j);
         }
 
+        SECTION("to_cbor rejects ill-formed UTF-8 (see #5651)")
+        {
+            // to_cbor() must reject the same ill-formed strings from_cbor()
+            // rejects, so a value it accepts can always be read back
+            CHECK_THROWS_WITH_AS(json::to_cbor(json("\xFF")), "[json.exception.type_error.316] invalid UTF-8 byte at index 0: 0xFF", json::type_error&);
+            // a truncated multi-byte sequence
+            CHECK_THROWS_WITH_AS(json::to_cbor(json("\xC3")), "[json.exception.type_error.316] invalid UTF-8 byte at index 0: 0xC3", json::type_error&);
+            // an encoded surrogate half (U+D800)
+            CHECK_THROWS_WITH_AS(json::to_cbor(json("\xED\xA0\x80")), "[json.exception.type_error.316] invalid UTF-8 byte at index 0: 0xED", json::type_error&);
+            // an overlong encoding of '.'
+            CHECK_THROWS_WITH_AS(json::to_cbor(json("\xC0\xAF")), "[json.exception.type_error.316] invalid UTF-8 byte at index 0: 0xC0", json::type_error&);
+
+            // an object key with ill-formed UTF-8 is rejected the same way
+            CHECK_THROWS_WITH_AS(json::to_cbor(json{{"\xFF", 1}}), "[json.exception.type_error.316] invalid UTF-8 byte at index 0: 0xFF", json::type_error&);
+
+            // binary values are not text and are unaffected
+            CHECK_NOTHROW(json::to_cbor(json::binary(std::vector<std::uint8_t>({0xFF}))));
+        }
+
         SECTION("invalid UTF-8 in indefinite-length string")
         {
             json _;

tests/src/unit-class_lexer.cpp

@@ -13,19 +13,15 @@
 using nlohmann::json;
 
 #include <array> // array
+#include <cfloat> // FLT_EVAL_METHOD
 #include <cstdint> // uint32_t, uint64_t
-#include <cstdio> // snprintf
 #include <cstdlib> // strtod
 #include <cstring> // memcpy
-#include <map> // map
-#include <random> // mt19937
 #include <sstream> // stringstream
 #include <string> // string
 #include <utility> // pair
 #include <vector> // vector
 
-#include "float_hard_cases.hpp"
-
 namespace
 {
 // shortcut to scan a string literal
@@ -261,7 +257,7 @@ TEST_CASE("lexer number fast path")
             "123456789012345678901234567890",  // huge -> float
             "0.30000000000000004", "2.2250738585072014e-308", "1e308",
             // high-precision / wide-exponent values that exercise the
-            // Eisel-Lemire path beyond the Clinger subset
+            // std::from_chars (Eisel-Lemire) path beyond the Clinger subset
             "1.7976931348623157e308", "1.2345678901234567e-250",
             "9007199254740993", "5e-324", "1e-320"
         };
@@ -283,18 +279,20 @@ TEST_CASE("lexer number fast path")
         }
     }
 
-    SECTION("significant digits around Clinger's fast path")
+    SECTION("significant-digit gate for the Clinger fast path")
     {
-        // Clinger's fast path needs a significand of at most 2^53, which
-        // tokens with 17 or more significant digits exceed. The conversion
-        // splits the token at the positions the scanners recorded, so leading
-        // zeros must not count as digits - "0.1234567890123456" has 16
-        // significant digits, not 17 - and both scanners must agree.
+        // Clinger's fast path needs a significand below 2^53, so it cannot
+        // succeed once the mantissa has 17 or more significant digits (the
+        // significand would be at least 10^16). The lexer skips the attempt
+        // there. That is only allowed to save work: every value must still come
+        // out bit-exactly, and both scanners must agree. In particular the gate
+        // must not fire for tokens whose leading zeros merely look like extra
+        // digits - "0.1234567890123456" has 16 significant digits, not 17.
         const std::vector<std::string> numbers =
         {
             "1234567890123456",                  // 16 significant digits
-            "12345678901234567",                 // 17
-            "123456789012345678",                // 18
+            "12345678901234567",                 // 17 -> attempt skipped
+            "123456789012345678",                // 18 -> attempt skipped
             "0.1234567890123456",                // 16: the leading "0" is not significant
             "0.12345678901234567",               // 17
             "0.00000000000000001",               // 1, in a long token
@@ -665,323 +663,46 @@ TEST_CASE("lexer string fast path")
     }
 }
 
-TEST_CASE("lexer escape fast path")
+TEST_CASE("parse_float_fast declines what it cannot convert exactly")
 {
-    // json::accept() never throws, so this section stays covered without
-    // exceptions; it pins which of the cases below are valid/invalid and
-    // checks the contiguous and streaming paths agree on that classification.
-    SECTION("accept() parity")
+    // The lexer only hands well-formed numbers to parse_float_fast, so the
+    // malformed ones below can only be passed to it directly. Declining is
+    // always safe: the caller then falls back to a slower, exact conversion.
+    const auto fast = [](const std::string & s, double & out)
     {
-        const std::vector<std::pair<std::string, bool>> cases =
-        {
-            {"\\u0041", true}, {"\\u00e4", true}, {"\\u00E4", true},
-            {"\\uD83D\\uDE00", true},
-            {"\\u12", false}, {"\\u12G4", false}, {"\\uXYZW", false},
-            {"\\uD800", false}, {"\\uD800A", false}, {"\\uD800\\u0041", false},
-            {"\\uDC00", false}, {"\\u", false}
-        };
-
-        for (const auto& c : cases)
-        {
-            for (const std::size_t offset :
-                    {
-                        std::size_t{0}, std::size_t{9}
-                    })
-            {
-                const std::string doc = "[\"" + std::string(offset, 'a') + c.first + "\"]";
-                CAPTURE(doc);
-                CHECK(json::accept(doc) == c.second);
-                std::stringstream ss(doc);
-                CHECK(json::accept(ss) == c.second);
-            }
-        }
-    }
-
-#if !defined(JSON_NOEXCEPTION)
-    // the full outcome of parsing @a doc: the parsed value, or the exact
-    // error message, so a mismatch in either is caught
-    const auto outcome = [](const std::string & doc, bool streaming) -> std::string
-    {
-        try
-        {
-            if (streaming)
-            {
-                std::stringstream ss(doc);
-                const json j = json::parse(ss);
-                return j.dump();
-            }
-            const json j = json::parse(doc);
-            return j.dump();
-        }
-        catch (const json::exception& e)
-        {
-            return {e.what()};
-        }
+        return nlohmann::detail::parse_float_fast(s.data(), s.data() + s.size(), out);
     };
+    double out = 0;
 
-    SECTION("contiguous vs streaming parity")
-    {
-        const std::vector<std::string> escapes =
-        {
-            "\\u0041",             // "A"
-            "\\u00e4",             // "ä" (lowercase hex)
-            "\\u00E4",             // "ä" (uppercase hex)
-            "\\uD83D\\uDE00",      // valid surrogate pair (an emoji)
-            "\\u12",               // truncated: only 2 hex digits before the closing quote
-            "\\u12G4",             // invalid hex digit at the 3rd position
-            "\\uXYZW",             // all 4 bytes invalid
-            "\\uD800",             // lone high surrogate, string ends right after
-            "\\uD800A",            // high surrogate not followed by another \u escape
-            "\\uD800\\u0041",      // high surrogate followed by \u, but not a low surrogate
-            "\\uDC00",             // lone low surrogate
-            "\\u",                 // '\u' with nothing after (closing quote right away)
-        };
-
-        // once at the start of the string and once past the first 8-byte SWAR
-        // word of the outer string_bulk_run, so the escape is reached both
-        // right after the opening quote and mid-run
-        for (const auto& escape : escapes)
-        {
-            for (const std::size_t offset :
-                    {
-                        std::size_t{0}, std::size_t{9}
-                    })
-            {
-                const std::string doc = "[\"" + std::string(offset, 'a') + escape + "\"]";
-                CAPTURE(doc);
-                CHECK(outcome(doc, false) == outcome(doc, true));
-            }
-
-            // the escape is the last thing before end of input: no closing
-            // quote at all
-            const std::string truncated_doc = "[\"" + escape;
-            CAPTURE(truncated_doc);
-            CHECK(outcome(truncated_doc, false) == outcome(truncated_doc, true));
-        }
-    }
-
-    SECTION("truncated \\u escape at every distance from the end of input")
-    {
-        // ia.bulk_remaining() must correctly report fewer than 4 bytes for
-        // every possible count of trailing hex-looking bytes (0, 1, 2, or 3)
-        // before end of input, so the fast path declines and the byte path
-        // alone reports the "must be followed by 4 hex digits" error, at the
-        // same position, in every case
-        for (const std::string& tail :
-                {
-                    std::string{}, std::string("1"), std::string("12"), std::string("123")
-                })
-        {
-            const std::string doc = "[\"\\u" + tail;
-            CAPTURE(doc);
-            CHECK(outcome(doc, false) == outcome(doc, true));
-            CHECK(outcome(doc, false).find("must be followed by 4 hex digits") != std::string::npos);
-        }
-    }
-
-    SECTION("invalid hex digit at every position of the 4")
-    {
-        // the fast path must decline for *any* invalid byte among the 4, not
-        // just the first, and the byte path must then stop at exactly that
-        // position - same as it always has
-        for (std::size_t bad_pos = 0; bad_pos < 4; ++bad_pos)
-        {
-            std::string digits = "1234";
-            digits[bad_pos] = 'g'; // not a hex digit
-            const std::string doc = "[\"\\u" + digits + "\"]";
-            CAPTURE(doc);
-            CHECK(outcome(doc, false) == outcome(doc, true));
-            CHECK(outcome(doc, false).find("must be followed by 4 hex digits") != std::string::npos);
-        }
-    }
-
-    SECTION("random escapes")
-    {
-        // A seeded PRNG builds the 4 bytes following `\u` from a mix of hex
-        // digits and non-hex bytes, at varying distances from the start of
-        // the string, to compare the two scanners on many more shapes than
-        // are practical to enumerate by hand.
-        std::mt19937 gen(7654321); // NOLINT(cert-msc32-c,cert-msc51-cpp)
-        const std::string hex_alphabet = "0123456789AaBbCcDdEeFf";
-        std::uniform_int_distribution<std::size_t> pick_hex(0, hex_alphabet.size() - 1);
-        std::uniform_int_distribution<int> pick_byte(1, 255); // never NUL
-        std::uniform_int_distribution<int> pick_is_hex(0, 4); // 4-in-5 chance of a hex digit
-        std::uniform_int_distribution<std::size_t> pick_offset(0, 12);
-
-        std::vector<std::string> mismatches;
-        for (int iter = 0; iter < 3000; ++iter)
-        {
-            std::string digits;
-            for (int i = 0; i < 4; ++i)
-            {
-                if (pick_is_hex(gen) != 0)
-                {
-                    digits += hex_alphabet[pick_hex(gen)];
-                }
-                else
-                {
-                    char c = static_cast<char>(pick_byte(gen));
-                    if (c == '"' || c == '\\')
-                    {
-                        // keep the string well-formed apart from the escape
-                        // itself, so any mismatch is attributable to the \u
-                        // handling and not to an unrelated quote/escape
-                        c = 'z';
-                    }
-                    digits += c;
-                }
-            }
-            const std::string doc = "[\"" + std::string(pick_offset(gen), 'a') + "\\u" + digits + "\"]";
-            if (outcome(doc, false) != outcome(doc, true))
-            {
-                mismatches.push_back(doc);
-            }
-        }
-        CAPTURE(mismatches);
-        CHECK(mismatches.empty());
-    }
+#if defined(FLT_EVAL_METHOD) && FLT_EVAL_METHOD != 0
+    // without true double precision, the fast path declines everything
+    CHECK_FALSE(fast("1.5", out));
+#else
+    CHECK(fast("1.5", out));
+    CHECK(out == 1.5);
+    CHECK(fast("+2.5e1", out));
+    CHECK(out == 25.0);
+    CHECK(fast("-25E-1", out));
+    CHECK(out == -2.5);
+    CHECK(fast("1e", out));
+    CHECK(out == 1.0);
 #endif
-}
 
-namespace
-{
-// the index of the decimal point (or npos) and of the end of the mantissa of a
-// number token, which the lexer records while scanning it
-std::pair<std::size_t, std::size_t> float_token_layout(const std::string& s)
-{
-    std::size_t dot = std::string::npos;
-    std::size_t mantissa_end = s.size();
-    for (std::size_t i = 0; i < s.size(); ++i)
-    {
-        if (s[i] == '.')
-        {
-            dot = i;
-        }
-        else if (s[i] == 'e' || s[i] == 'E')
-        {
-            mantissa_end = i;
-            break;
-        }
-    }
-    return {dot, mantissa_end};
-}
+    // not a number
+    CHECK_FALSE(fast("", out));
+    CHECK_FALSE(fast("-", out));
+    CHECK_FALSE(fast(".", out));
+    CHECK_FALSE(fast("1.2.3", out));
+    CHECK_FALSE(fast("1x", out));
+    CHECK_FALSE(fast("1e+", out));
+    CHECK_FALSE(fast("1e1x", out));
 
-template<typename FloatType>
-FloatType parse_native(const std::string& s)
-{
-    const auto layout = float_token_layout(s);
-    return nlohmann::detail::parse_float_native<FloatType>(s.data(), s.data() + s.size(), layout.first, layout.second);
-}
-
-std::uint64_t bits_of(double d)
-{
-    std::uint64_t b = 0;
-    std::memcpy(&b, &d, sizeof(b));
-    return b;
-}
-
-std::uint32_t bits_of(float f)
-{
-    std::uint32_t b = 0;
-    std::memcpy(&b, &f, sizeof(b));
-    return b;
-}
-
-std::uint64_t native_bits64(const std::string& s)
-{
-    return bits_of(parse_native<double>(s));
-}
-
-std::uint32_t native_bits32(const std::string& s)
-{
-    return bits_of(parse_native<float>(s));
-}
-} // namespace
-
-TEST_CASE("parse_float_native rounds correctly")
-{
-    SECTION("double")
-    {
-        CHECK(native_bits64("1.5") == 0x3FF8000000000000u);
-        CHECK(native_bits64("0.1") == 0x3FB999999999999Au);
-        CHECK(native_bits64("-0.0") == 0x8000000000000000u);
-        CHECK(native_bits64("0e999999999999999999999") == 0u);
-        // 2^53 + 1 is exactly between two doubles: ties to even, unless more digits follow
-        CHECK(native_bits64("9007199254740993") == 0x4340000000000000u);
-        CHECK(native_bits64("9007199254740993.0000000000000000001") == 0x4340000000000001u);
-        CHECK(native_bits64("9007199254740992.9999999999999999999") == 0x4340000000000000u);
-        // 1 + 2^-53 exactly (a tie), and one unit in the 55th digit around it
-        CHECK(native_bits64("1.00000000000000011102230246251565404236316680908203125") == 0x3FF0000000000000u);
-        CHECK(native_bits64("1.00000000000000011102230246251565404236316680908203126") == 0x3FF0000000000001u);
-        CHECK(native_bits64("1.00000000000000011102230246251565404236316680908203124") == 0x3FF0000000000000u);
-        // subnormal and overflow boundaries
-        CHECK(native_bits64("2.4703282292062327e-324") == 0u);
-        CHECK(native_bits64("2.4703282292062328e-324") == 1u);
-        CHECK(native_bits64("2.2250738585072011e-308") == 0x000FFFFFFFFFFFFFu);
-        CHECK(native_bits64("2.2250738585072012e-308") == 0x0010000000000000u);
-        CHECK(native_bits64("1.7976931348623157e308") == 0x7FEFFFFFFFFFFFFFu);
-        CHECK(native_bits64("1.7976931348623159e308") == 0x7FF0000000000000u);
-        CHECK(native_bits64("-1e400") == 0xFFF0000000000000u);
-        CHECK(native_bits64("-1e-400") == 0x8000000000000000u);
-        // exponents and zeros far beyond the range cancel out
-        CHECK(native_bits64("0." + std::string(1000, '0') + "1e1001") == 0x3FF0000000000000u);
-        CHECK(native_bits64("1" + std::string(1000, '0') + "e-1000") == 0x3FF0000000000000u);
-        CHECK(native_bits64("1e-99999999999999999999999") == 0u);
-        CHECK(native_bits64("1E+99999999999999999999999") == 0x7FF0000000000000u);
-        // more digits than any midpoint has (769): only whether a nonzero digit follows matters
-        const std::string tie = "1.00000000000000011102230246251565404236316680908203125";
-        CHECK(native_bits64(tie + std::string(800, '0')) == 0x3FF0000000000000u);
-        CHECK(native_bits64(tie + std::string(800, '0') + "1") == 0x3FF0000000000001u);
-    }
-
-    SECTION("float")
-    {
-        CHECK(native_bits32("1.5") == 0x3FC00000u);
-        CHECK(native_bits32("0.1") == 0x3DCCCCCDu);
-        CHECK(native_bits32("-0.0") == 0x80000000u);
-        // 2^24 + 1 is exactly between two floats
-        CHECK(native_bits32("16777217") == 0x4B800000u);
-        CHECK(native_bits32("16777217.000000000000000000001") == 0x4B800001u);
-        CHECK(native_bits32("16777218.999999999999999999999") == 0x4B800001u);
-        CHECK(native_bits32("16777219") == 0x4B800002u);
-        // subnormal and overflow boundaries
-        CHECK(native_bits32("3.4028235677973366e38") == 0x7F7FFFFFu);
-        CHECK(native_bits32("3.4028235677973367e38") == 0x7F800000u);
-        CHECK(native_bits32("7.006492321624085e-46") == 0u);
-        CHECK(native_bits32("7.006492321624086e-46") == 1u);
-        CHECK(native_bits32("1.1754942e-38") == 0x007FFFFFu);
-        CHECK(native_bits32("-1.17549435e-38") == 0x80800000u);
-        CHECK(native_bits32("1e39") == 0x7F800000u);
-        CHECK(native_bits32("-1e-50") == 0x80000000u);
-        // not rounded through double: its double would round to another float
-        CHECK(native_bits32("1.00000005960464477539062500000000001") == 0x3F800001u);
-        CHECK(native_bits32("9007199254740993") == 0x5A000000u);
-    }
-
-    SECTION("the conversion shared with other parsers")
-    {
-        // convert_float() gives the lexer's results, for every type
-        const std::vector<std::string> tokens =
-        {
-            "0", "-0.0", "1.5", "0.1", "1e-400", "-2.5E+3", "123456789012345678901234567890",
-            "9007199254740993.0000000000000000001", "4.9406564584124654e-324"
-        };
-        using float_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int64_t, std::uint64_t, float>;
-        using long_double_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int64_t, std::uint64_t, long double>;
-        for (const auto& t : tokens)
-        {
-            CAPTURE(t);
-            const auto layout = float_token_layout(t);
-            const char* const first = t.data();
-            const char* const last = first + t.size();
-            const auto d = nlohmann::detail::convert_float<double>(first, last, layout.first, layout.second);
-            const auto f = nlohmann::detail::convert_float<float>(first, last, layout.first, layout.second);
-            const auto ld = nlohmann::detail::convert_float<long double>(first, last, layout.first, layout.second);
-            CHECK(bits_of(d) == bits_of(json::parse(t).get<double>()));
-            CHECK(bits_of(f) == bits_of(float_json::parse(t).get<float>()));
-            CHECK(ld == long_double_json::parse(t).get<long double>());
-        }
-    }
+    // numbers that are not represented exactly on the fast path
+    CHECK_FALSE(fast("12345678901234567890", out));
+    CHECK_FALSE(fast("1e10000", out));
+    CHECK_FALSE(fast("9007199254740993", out));
+    CHECK_FALSE(fast("1e23", out));
+    CHECK_FALSE(fast("1e-23", out));
 }
 
 namespace
@@ -1085,6 +806,40 @@ std::size_t big_bit_length(const big_uint& a)
     }
     return n;
 }
+
+std::uint64_t bits_of(double d)
+{
+    std::uint64_t b = 0;
+    std::memcpy(&b, &d, sizeof(b));
+    return b;
+}
+
+bool eisel_lemire(const std::string& s, double& out)
+{
+    return nlohmann::detail::parse_float_eisel_lemire(s.data(), s.data() + s.size(), out);
+}
+
+// significant digits of a token, without trailing zeros
+std::size_t significant_digits(const std::string& s)
+{
+    std::string digits;
+    for (const char c : s)
+    {
+        if (c == 'e' || c == 'E')
+        {
+            break;
+        }
+        if (c >= '0' && c <= '9' && !(digits.empty() && c == '0'))
+        {
+            digits += c;
+        }
+    }
+    while (!digits.empty() && digits.back() == '0')
+    {
+        digits.pop_back();
+    }
+    return digits.size();
+}
 } // namespace
 
 TEST_CASE("Eisel-Lemire float conversion")
@@ -1482,33 +1237,26 @@ TEST_CASE("Eisel-Lemire float conversion")
         for (const auto& c : known)
         {
             CAPTURE(c.first);
-            CHECK(native_bits64(c.first) == c.second);
+            double out = 0;
+            if (eisel_lemire(c.first, out))
+            {
+                CHECK(bits_of(out) == c.second);
+            }
+            else
+            {
+                // only tokens with more than 19 significant digits are left to
+                // strtod: those whose value lies too close to a tie
+                CHECK(significant_digits(c.first) > 19);
+            }
         }
     }
 
-    SECTION("binary32")
-    {
-        using binary32 = nlohmann::detail::ieee_binary_format<24>;
-        CHECK(nlohmann::detail::eisel_lemire<binary32>(0, 1) == 0x3F800000u);
-        CHECK(nlohmann::detail::eisel_lemire<binary32>(-1, 1) == 0x3DCCCCCDu);
-        CHECK(nlohmann::detail::eisel_lemire<binary32>(-1, 15) == 0x3FC00000u);
-        CHECK(nlohmann::detail::eisel_lemire<binary32>(0, 16777217) == 0x4B800000u); // tie, to even
-        CHECK(nlohmann::detail::eisel_lemire<binary32>(0, 16777219) == 0x4B800002u); // tie, to even
-        CHECK(nlohmann::detail::eisel_lemire<binary32>(-45, 1) == 0x00000001u);
-        CHECK(nlohmann::detail::eisel_lemire<binary32>(-46, 7) == 0x00000000u);
-        CHECK(nlohmann::detail::eisel_lemire<binary32>(-46, 8) == 0x00000001u);
-        CHECK(nlohmann::detail::eisel_lemire<binary32>(-65, 9999999999999999999u) == 0x00000000u);
-        CHECK(nlohmann::detail::eisel_lemire<binary32>(20, 3402823466385288598u) == 0x7F7FFFFFu);
-        CHECK(nlohmann::detail::eisel_lemire<binary32>(20, 3402823669209384635u) == 0x7F800000u);
-        CHECK(nlohmann::detail::eisel_lemire<binary32>(39, 1) == 0x7F800000u);
-        CHECK(nlohmann::detail::eisel_lemire<binary32>(-5, 0) == 0x00000000u);
-    }
-
     SECTION("round trip")
     {
-        // every double written by to_chars and read back, and its 17-digit
-        // form with trailing digits that make the token longer than 19 digits
+        // every double written by to_chars and read back, also with trailing
+        // digits that make the token longer than 19 digits
         std::uint64_t state = 5295;
+        std::size_t declined = 0;
         for (int i = 0; i < 200000; ++i)
         {
             state ^= state << 13u;
@@ -1530,51 +1278,30 @@ TEST_CASE("Eisel-Lemire float conversion")
             const char* end = nlohmann::detail::to_chars(buffer.data(), buffer.data() + buffer.size(), d);
             const std::string token(buffer.data(), static_cast<std::size_t>(end - buffer.data()));
             CAPTURE(token);
-            CHECK(native_bits64(token) == b);
+            double out = 0;
+            REQUIRE(eisel_lemire(token, out));
+            CHECK(bits_of(out) == b);
 
-            // insert digits before the exponent of the 17-digit form: that
-            // form lies strictly inside the rounding interval of the double
-            // (the shortest one may lie on its boundary), and the digits move
-            // it by far less than the distance to the boundary, so the value
-            // must not change
-            std::array<char, 64> digits17{};
-            static_cast<void>(std::snprintf(digits17.data(), digits17.size(), "%.17g", d)); // NOLINT(cppcoreguidelines-pro-type-vararg,hicpp-vararg)
-            std::string longer = digits17.data();
+            // insert digits before the exponent: the value moves by far less
+            // than the distance to the rounding boundary, so it must not change
+            std::string longer = token;
             const std::size_t e = longer.find('e');
             const std::size_t dot = longer.find('.');
             const std::string extra = dot == std::string::npos ? ".000000000000000000001" : "000000000000000000001";
             longer.insert(e == std::string::npos ? longer.size() : e, extra);
             CAPTURE(longer);
-            CHECK(native_bits64(longer) == b);
-        }
-    }
-
-    SECTION("round trip, binary32")
-    {
-        std::uint32_t state = 5295;
-        for (int i = 0; i < 100000; ++i)
-        {
-            state ^= state << 13u;
-            state ^= state >> 17u;
-            state ^= state << 5u;
-            std::uint32_t b = state;
-            if ((b & 0x7F800000u) == 0x7F800000u)
+            if (eisel_lemire(longer, out))
             {
-                continue; // infinity or NaN
+                CHECK(bits_of(out) == b);
             }
-            if (i % 4 == 0)
+            else
             {
-                b &= 0x807FFFFFu; // subnormals
+                // w and w + 1 round differently: only when the value is very
+                // close to a rounding boundary
+                ++declined;
             }
-            float f = 0;
-            std::memcpy(&f, &b, sizeof(f));
-
-            std::array<char, 64> buffer{};
-            const char* end = nlohmann::detail::to_chars(buffer.data(), buffer.data() + buffer.size(), f);
-            const std::string token(buffer.data(), static_cast<std::size_t>(end - buffer.data()));
-            CAPTURE(token);
-            CHECK(native_bits32(token) == b);
         }
+        CHECK(declined < 1000); // 107 of the 200,000
     }
 
     SECTION("used by the lexer")
@@ -1588,177 +1315,3 @@ TEST_CASE("Eisel-Lemire float conversion")
                              "[json.exception.out_of_range.406] number overflow parsing '1.7976931348623159e308'", json::out_of_range&);
     }
 }
-
-namespace
-{
-using float_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int64_t, std::uint64_t, float>;
-
-// the bits of the float that parse() gives for a token, via both scanners;
-// the value must be the same for both
-template<typename Json, typename Bits>
-void check_parse(const std::string& token, Bits expected, Bits infinity)
-{
-    std::stringstream stream(token);
-    if ((expected & ~(Bits{1} << (8 * sizeof(Bits) - 1))) == infinity)
-    {
-        Json _;
-        CHECK_THROWS_WITH_AS(_ = Json::parse(token), ("[json.exception.out_of_range.406] number overflow parsing '" + token + "'").c_str(), typename Json::out_of_range&);
-        CHECK_THROWS_WITH_AS(_ = Json::parse(stream), ("[json.exception.out_of_range.406] number overflow parsing '" + token + "'").c_str(), typename Json::out_of_range&);
-        return;
-    }
-    const Json contiguous = Json::parse(token);
-    const Json streamed = Json::parse(stream);
-    if (contiguous.is_number_float()) // not an integer that fits
-    {
-        CHECK(bits_of(contiguous.template get<typename Json::number_float_t>()) == expected);
-        CHECK(bits_of(streamed.template get<typename Json::number_float_t>()) == expected);
-    }
-    else
-    {
-        CHECK(streamed.is_number_integer());
-    }
-}
-} // namespace
-
-TEST_CASE("float conversion of hard cases")
-{
-    // see float_hard_cases.hpp
-    for (const auto& c : float_hard_cases::cases())
-    {
-        const std::string token = c.token;
-        CAPTURE(token);
-        CHECK(native_bits64(token) == c.bits64);
-        CHECK(native_bits32(token) == c.bits32);
-        check_parse<json>(token, c.bits64, std::uint64_t{0x7FF0000000000000u});
-        check_parse<float_json>(token, c.bits32, std::uint32_t{0x7F800000u});
-    }
-}
-
-TEST_CASE("float overflow and underflow in the parser")
-{
-    SECTION("double")
-    {
-        check_parse<json>("1.7976931348623157e308", std::uint64_t{0x7FEFFFFFFFFFFFFFu}, std::uint64_t{0x7FF0000000000000u});
-        check_parse<json>("1.7976931348623159e308", std::uint64_t{0x7FF0000000000000u}, std::uint64_t{0x7FF0000000000000u});
-        check_parse<json>("-1e309", std::uint64_t{0xFFF0000000000000u}, std::uint64_t{0x7FF0000000000000u});
-        check_parse<json>("1" + std::string(400, '0'), std::uint64_t{0x7FF0000000000000u}, std::uint64_t{0x7FF0000000000000u});
-        check_parse<json>("1e99999999999999999999", std::uint64_t{0x7FF0000000000000u}, std::uint64_t{0x7FF0000000000000u});
-        // an underflow gives a zero with the sign of the token
-        check_parse<json>("1e-400", std::uint64_t{0}, std::uint64_t{0x7FF0000000000000u});
-        check_parse<json>("-1e-400", std::uint64_t{0x8000000000000000u}, std::uint64_t{0x7FF0000000000000u});
-        check_parse<json>("-2.4703282292062327e-324", std::uint64_t{0x8000000000000000u}, std::uint64_t{0x7FF0000000000000u});
-        check_parse<json>("0." + std::string(400, '0') + "1", std::uint64_t{0}, std::uint64_t{0x7FF0000000000000u});
-    }
-
-    SECTION("float")
-    {
-        check_parse<float_json>("3.4028234e38", std::uint32_t{0x7F7FFFFFu}, std::uint32_t{0x7F800000u});
-        check_parse<float_json>("3.4028236e38", std::uint32_t{0x7F800000u}, std::uint32_t{0x7F800000u});
-        check_parse<float_json>("-1e39", std::uint32_t{0xFF800000u}, std::uint32_t{0x7F800000u});
-        check_parse<float_json>("1e-46", std::uint32_t{0}, std::uint32_t{0x7F800000u});
-        check_parse<float_json>("-1e-46", std::uint32_t{0x80000000u}, std::uint32_t{0x7F800000u});
-        check_parse<float_json>("-7.006492321624085e-46", std::uint32_t{0x80000000u}, std::uint32_t{0x7F800000u});
-        check_parse<float_json>("-7.006492321624086e-46", std::uint32_t{0x80000001u}, std::uint32_t{0x7F800000u});
-    }
-}
-
-TEST_CASE("string scanning kernels")
-{
-    // the word-at-a-time kernels must stop exactly where a byte-by-byte scan
-    // stops, for any content, length, and alignment
-    const auto reference_special = [](const unsigned char* data, std::size_t n)
-    {
-        std::size_t i = 0;
-        while (i < n && !nlohmann::detail::is_string_special(data[i]))
-        {
-            ++i;
-        }
-        return i;
-    };
-    const auto reference_copyable = [](const unsigned char* data, std::size_t n)
-    {
-        std::size_t i = 0;
-        while (i < n && nlohmann::detail::is_ascii_copyable(data[i]))
-        {
-            ++i;
-        }
-        return i;
-    };
-    const auto reference_bulk_run = [](const unsigned char* data, std::size_t n)
-    {
-        std::size_t i = 0;
-        while (i < n)
-        {
-            if (data[i] < 0x80u)
-            {
-                if (nlohmann::detail::is_string_special(data[i]))
-                {
-                    break;
-                }
-                ++i;
-                continue;
-            }
-            const std::size_t seq = nlohmann::detail::validate_one_utf8(data + i, n - i);
-            if (seq == 0)
-            {
-                break;
-            }
-            i += seq;
-        }
-        return i;
-    };
-
-    // pieces: ordinary ASCII, stops, DEL, well-formed sequences of every
-    // length, and ill-formed or truncated ones
-    const std::vector<std::string> pieces =
-    {
-        "a", "Z", " ", "~", "0123456789", "\"", "\\", std::string(1, '\0'), "\n", "\x1F", "\x7F",
-        "\xC3\xA4", "\xE2\x82\xAC", "\xE6\x97\xA5\xE6\x9C\xAC", "\xF0\x9F\x98\x80", "\xED\x9F\xBF",
-        "\x80", "\xC0\x80", "\xC3", "\xE2\x82", "\xED\xA0\x80", "\xF4\x90\x80\x80", "\xFF",
-    };
-    std::uint64_t state = 5295;
-    const auto next = [&state]()
-    {
-        state ^= state << 13u;
-        state ^= state >> 7u;
-        state ^= state << 17u;
-        return state;
-    };
-    // the upper half as a 32-bit value: converts to std::size_t implicitly on
-    // every platform (a cast of std::uint64_t is useless where both are the
-    // same type, and required where std::size_t is 32 bits wide)
-    const auto next_small = [&next]()
-    {
-        return static_cast<std::uint32_t>(next() >> 32u);
-    };
-    for (int round = 0; round < 100000; ++round)
-    {
-        // mostly ordinary text, so that runs span several words
-        std::string text(next_small() % 8u, '.');
-        const std::size_t count = next_small() % 12u;
-        for (std::size_t k = 0; k < count; ++k)
-        {
-            const std::size_t p = (next() % 4 == 0) ? next_small() % pieces.size() : 0;
-            text += pieces[p];
-            text += std::string(next_small() % 10u, 'x');
-        }
-        const auto* data = reinterpret_cast<const unsigned char*>(text.data()); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
-        for (std::size_t offset = 0; offset < 3 && offset <= text.size(); ++offset)
-        {
-            const std::size_t n = text.size() - offset;
-            CAPTURE(text);
-            CAPTURE(offset);
-            CHECK(nlohmann::detail::find_string_special(data + offset, n) == reference_special(data + offset, n));
-            CHECK(nlohmann::detail::find_ascii_copyable_run(data + offset, n) == reference_copyable(data + offset, n));
-            CHECK(nlohmann::detail::scalar_string_bulk_run(data + offset, n) == reference_bulk_run(data + offset, n));
-        }
-    }
-
-    // the trailing-zero count, whichever implementation the compiler gets
-    for (int k = 0; k < 64; ++k)
-    {
-        const std::uint64_t bit = std::uint64_t{1} << k;
-        CHECK(nlohmann::detail::count_trailing_zeros(bit) == k);
-        CHECK(nlohmann::detail::count_trailing_zeros(bit | (bit << 1u) | 0x8000000000000000u) == k);
-    }
-}

tests/src/unit-diagnostic-positions.cpp

@@ -141,6 +141,58 @@ TEST_CASE("Better diagnostics with positions")
         check_objects(300);
     }
 
+    SECTION("converting keeps the positions of nested values (#5650)")
+    {
+        // Values nested deeper than the converting constructor's descent bound
+        // are converted without the call stack, on a path that has to carry the
+        // positions of every value over itself. Objects and arrays take turns,
+        // and the innermost value is null, which used to lose its positions.
+        const auto check_conversion = [](std::size_t depth)
+        {
+            CAPTURE(depth)
+
+            std::string text;
+            std::string closing;
+            for (std::size_t i = 0; i < depth; ++i)
+            {
+                text += (i % 2 == 0) ? "[12, " : R"({"b":1, "a":)";
+                closing += (i % 2 == 0) ? ']' : '}';
+            }
+            text += "null";
+            text.append(closing.rbegin(), closing.rend());
+
+            const json original = json::parse(text);
+            const nlohmann::ordered_json converted = original;
+
+            const json* o = &original;
+            const nlohmann::ordered_json* c = &converted;
+            for (std::size_t level = 0; level <= depth; ++level)
+            {
+                CAPTURE(level)
+                REQUIRE(c->start_pos() == o->start_pos());
+                REQUIRE(c->end_pos() == o->end_pos());
+
+                if (level < depth)
+                {
+                    // the number beside the value nested next
+                    const json& o_number = o->is_object() ? o->at("b") : o->at(0);
+                    const nlohmann::ordered_json& c_number = c->is_object() ? c->at("b") : c->at(0);
+                    REQUIRE(c_number.start_pos() == o_number.start_pos());
+                    REQUIRE(c_number.end_pos() == o_number.end_pos());
+
+                    o = o->is_object() ? &o->at("a") : &o->at(1);
+                    c = c->is_object() ? &c->at("a") : &c->at(1);
+                }
+            }
+        };
+
+        check_conversion(1);
+        check_conversion(127);
+        check_conversion(128);
+        check_conversion(129);
+        check_conversion(300);
+    }
+
     SECTION("JSON patch add to primitive parent (#4292)")
     {
         // the JSON Patch "add" target /foo/bar/baz has a string parent

tests/src/unit-diagnostics.cpp

@@ -341,6 +341,36 @@ TEST_CASE("Regression tests for extended diagnostics")
         }
     }
 
+    SECTION("Regression test for issue #5650 - converting keeps the parents of nested values")
+    {
+        // A value nested deeper than the converting constructor's descent bound
+        // is converted without the call stack. Every container that path creates
+        // has to have the parents of its children set, or the JSON Pointer in the
+        // diagnostic is cut short. Objects and arrays take turns.
+        const std::size_t pairs = 150;
+
+        json j = "not a number";
+        std::string pointer;
+        for (std::size_t i = 0; i < pairs; ++i)
+        {
+            j = json{{"a", json::array({j})}};
+            pointer += "/a/0";
+        }
+
+        const nlohmann::ordered_json converted = j;
+
+        const nlohmann::ordered_json* inner = &converted;
+        for (std::size_t i = 0; i < pairs; ++i)
+        {
+            inner = &inner->at("a").at(0);
+        }
+
+        std::string const expected = "[json.exception.type_error.302] (" + pointer + ") type must be number, but is string";
+        int i = 0;
+        CHECK_THROWS_WITH_AS(i = inner->get<int>(), expected.c_str(), nlohmann::ordered_json::type_error);
+        CHECK(i == 0);
+    }
+
     SECTION("Regression test for issue #5668 - wrong path for std::map/unordered_map with non-string keys")
     {
         // a map with non-string keys is read from an array of [key, value] arrays;

tests/src/unit-disable-tuple-reference-conversion.cpp

@@ -0,0 +1,93 @@
+//     __ _____ _____ _____
+//  __|  |   __|     |   | |  JSON for Modern C++ (supporting code)
+// |  |  |__   |  |  | | | |  version 3.12.0
+// |_____|_____|_____|_|___|  https://github.com/nlohmann/json
+//
+// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
+// SPDX-License-Identifier: MIT
+
+#include "doctest_compatibility.h"
+
+// This file tests the opt-in JSON_DISABLE_TUPLE_REFERENCE_CONVERSION, so it
+// defines the macro itself rather than relying on a -D flag, and runs in every
+// build.
+#ifdef JSON_DISABLE_TUPLE_REFERENCE_CONVERSION
+    #undef JSON_DISABLE_TUPLE_REFERENCE_CONVERSION
+#endif
+
+#define JSON_DISABLE_TUPLE_REFERENCE_CONVERSION 1
+
+#include <nlohmann/json.hpp>
+using nlohmann::json;
+using nlohmann::ordered_json;
+
+#include <string>
+#include <tuple>
+#include <type_traits>
+#include <utility>
+
+// clang before 4 and GCC before 5 cannot create a std::tuple of basic_json
+// references at all, with or without JSON_DISABLE_TUPLE_REFERENCE_CONVERSION:
+// the tuple constructors make them instantiate basic_json's conversion operator
+// for libstdc++'s internal tuple bases, which fails hard
+#if (defined(__clang__) && __clang_major__ < 4) || (!defined(__clang__) && defined(__GNUC__) && __GNUC__ < 5)
+    #define SKIP_TESTS_FOR_JSON_REFERENCE_TUPLES
+#endif
+
+TEST_CASE("JSON_DISABLE_TUPLE_REFERENCE_CONVERSION")
+{
+    SECTION("json is not constructible from a one-element tuple of a json reference")
+    {
+        CHECK_FALSE(std::is_constructible<json, std::tuple<json&>>::value);
+        CHECK_FALSE(std::is_constructible<json, std::tuple<const json&>>::value);
+        CHECK_FALSE(std::is_constructible < json, std::tuple < json && >>::value);
+        CHECK_FALSE(std::is_constructible<json, const std::tuple<json&>&>::value);
+        CHECK_FALSE(std::is_constructible<ordered_json, std::tuple<ordered_json&>>::value);
+    }
+
+#ifndef SKIP_TESTS_FOR_JSON_REFERENCE_TUPLES
+    SECTION("issue #2226 - tuple<const json&> from tuple<json&> keeps the reference")
+    {
+        json j = true;
+        const std::tuple<const json&> tup(std::forward_as_tuple(j));
+        CHECK(&std::get<0>(tup) == &j);
+    }
+
+    SECTION("tuple<json> from tuple<json&> copies the element")
+    {
+        const json j = {{"key", "value"}};
+        const std::tuple<json> t1(std::forward_as_tuple(j));
+        CHECK(std::get<0>(t1) == j);
+
+        json j2 = "text";
+        const std::tuple<json> t2(std::forward_as_tuple(std::move(j2)));
+        CHECK(std::get<0>(t2) == "text");
+    }
+#endif
+
+    SECTION("other tuple conversions are not affected")
+    {
+        const json j = true;
+
+        // one-element tuple holding a json value
+        CHECK(json(std::make_tuple(j)) == json::array({true}));
+
+        // tuples with more than one element, even when holding references
+        int i = 1;
+#ifndef SKIP_TESTS_FOR_JSON_REFERENCE_TUPLES
+        CHECK(json(std::forward_as_tuple(i, j)) == json::array({1, true}));
+        CHECK(json(std::forward_as_tuple(j, j)) == json::array({true, true}));
+#endif
+
+        // one-element tuples holding references to other types
+        std::string s = "text";
+        CHECK(json(std::forward_as_tuple(s)) == json::array({"text"}));
+        CHECK(json(std::forward_as_tuple(i)) == json::array({1}));
+
+#ifndef SKIP_TESTS_FOR_JSON_REFERENCE_TUPLES
+        // a reference to a different basic_json specialization
+        ordered_json oj = true;
+        CHECK(json(std::forward_as_tuple(oj)) == json::array({true}));
+#endif
+    }
+}

tests/src/unit-large_json.cpp

@@ -13,6 +13,7 @@ using nlohmann::json;
 
 #include <algorithm>
 #include <string>
+#include <vector>
 
 TEST_CASE("tests on very large JSONs")
 {
@@ -53,6 +54,24 @@ const json* innermost_value(const json& j, std::size_t& depth)
     return current;
 }
 
+// The text of a value nested depth levels deep around the number 0. Level i is
+// an array if pattern[i % pattern.size()] is '[', and otherwise an object with
+// the single member "a", which every object type enumerates in the same order.
+std::string nested_text(std::size_t depth, const std::string& pattern)
+{
+    std::string text;
+    std::string closing;
+    for (std::size_t i = 0; i < depth; ++i)
+    {
+        const bool array = pattern[i % pattern.size()] == '[';
+        text += array ? "[" : "{\"a\":";
+        closing += array ? ']' : '}';
+    }
+    text += '0';
+    text.append(closing.rbegin(), closing.rend());
+    return text;
+}
+
 } // namespace
 
 TEST_CASE("tests on deeply nested JSONs")
@@ -224,5 +243,114 @@ TEST_CASE("tests on deeply nested JSONs")
             CHECK(*innermost_value(j, unused) == 0);
         }
     }
+
+    SECTION("issue #5650 - stack overflow converting between specializations")
+    {
+        const std::vector<std::string> patterns = {"[", "{", "[{"};
+
+        SECTION("json to ordered_json")
+        {
+            for (const auto& pattern : patterns)
+            {
+                CAPTURE(pattern);
+                const std::string text = nested_text(depth, pattern);
+                const json j = json::parse(text);
+
+                const nlohmann::ordered_json converted = j;
+                CHECK(converted.dump() == text);
+            }
+        }
+
+        SECTION("ordered_json to json")
+        {
+            for (const auto& pattern : patterns)
+            {
+                CAPTURE(pattern);
+                const std::string text = nested_text(depth, pattern);
+                const nlohmann::ordered_json o = nlohmann::ordered_json::parse(text);
+
+                const json converted = o;
+                CHECK(converted.dump() == text);
+            }
+        }
+
+        SECTION("get<ordered_json>()")
+        {
+            for (const auto& pattern : patterns)
+            {
+                CAPTURE(pattern);
+                const std::string text = nested_text(depth, pattern);
+                const json j = json::parse(text);
+
+                CHECK(j.get<nlohmann::ordered_json>().dump() == text);
+            }
+        }
+
+        SECTION("depths around the bound of the recursive descent")
+        {
+            for (std::size_t d = 1; d <= 300; ++d)
+            {
+                CAPTURE(d);
+                for (const auto& pattern : patterns)
+                {
+                    CAPTURE(pattern);
+                    const std::string text = nested_text(d, pattern);
+                    const json j = json::parse(text);
+
+                    const nlohmann::ordered_json converted = j;
+                    CHECK(converted.dump() == text);
+                    const json back = converted;
+                    CHECK(back.dump() == text);
+                }
+            }
+        }
+
+        SECTION("values below the bound are converted as values above it")
+        {
+            // Bury a value below the bound, where it is converted without the
+            // call stack, and compare it with the same value converted on its
+            // own by the containers' range constructors. Its objects have
+            // members that the two object types enumerate in different orders.
+            const auto bury = [](nlohmann::ordered_json value)
+            {
+                for (std::size_t i = 0; i < 200; ++i)
+                {
+                    value = nlohmann::ordered_json::array({std::move(value)});
+                }
+                return value;
+            };
+            const auto dig = [](const json & value)
+            {
+                const json* current = &value;
+                for (std::size_t i = 0; i < 200; ++i)
+                {
+                    current = &current->at(0);
+                }
+                return current;
+            };
+
+            nlohmann::ordered_json value = nlohmann::ordered_json::object();
+            value["z"] = {1, -2, 3U, 4.5, true, nullptr, "six", nlohmann::ordered_json::binary({7, 8}, 9),
+                          nlohmann::ordered_json::binary({10}), nlohmann::ordered_json::array(), nlohmann::ordered_json::object()
+                         };
+            value["y"] = {{"x", {{"w", 1}, {"v", 2}}}, {"u", {3, {{"t", 4}, {"s", 5}}}}};
+            value["r"] = nlohmann::ordered_json::array({nlohmann::ordered_json(nlohmann::ordered_json::value_t::discarded)});
+
+            const json converted_above = value;
+            const json buried = bury(value);
+            const json& converted_below = *dig(buried);
+
+            CHECK(converted_below.dump() == converted_above.dump());
+            CHECK(converted_below.at("z").at(7).get_binary().subtype() == 9);
+            CHECK_FALSE(converted_below.at("z").at(8).get_binary().has_subtype());
+            CHECK(converted_below.at("r").at(0).is_discarded());
+
+            // a discarded value is never equal to anything, so compare the rest
+            value.erase("r");
+            const json without_discarded_above = value;
+            const json without_discarded_buried = bury(value);
+            CHECK(*dig(without_discarded_buried) == without_discarded_above);
+        }
+    }
 }
 

tests/src/unit-locale-cpp.cpp

@@ -257,11 +257,10 @@ struct LocaleSwitchingSax final: public nlohmann::json_sax<json>
 
 TEST_CASE("locale changes between lexer construction and number conversion (#5198)")
 {
-    // float and double are converted without the locale. A long double that
-    // is not binary64 can take the strtold fallback, which honors the locale
-    // that is current at conversion time. The numbers are chosen so that it
-    // does: too many significant digits for Clinger's fast path, an underflow
-    // that std::from_chars rejects, and a plain value.
+    // The numbers are chosen so that the conversion also takes the strtod
+    // fallback, which honors the locale that is current at conversion time:
+    // too many significant digits for Clinger's fast path, an underflow that
+    // std::from_chars rejects, and a plain value.
     const std::vector<std::string> numbers = {"3.14159265358979323846", "1.5e-400", "12.34", "-0.000123456789012345678"};
     std::string text = "[";
     for (const auto& n : numbers)
@@ -325,8 +324,7 @@ TEST_CASE("locale changes between lexer construction and number conversion (#519
             }
         }
 
-        // a long double goes through std::strtold unless it is binary64 or
-        // std::from_chars supports it
+        // a long double goes through std::strtold unless std::from_chars supports it
         {
             bool switched = false;
             const auto cb = [&](int /*depth*/, long_double_json::parse_event_t event, long_double_json& /*parsed*/) noexcept
@@ -352,15 +350,8 @@ TEST_CASE("locale with a multi-byte decimal point")
 {
     // Some locales use a decimal point that is not a single character, e.g.
     // U+066B ARABIC DECIMAL SEPARATOR (two bytes in UTF-8). It cannot be
-    // substituted in place for '.', so the strtold fallback (only for long
-    // double formats other than binary64) converts a copy of the token with
-    // the whole decimal point instead (#5660). The values must be those of the
-    // "C" locale.
-    using long_double_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int64_t, std::uint64_t, long double>;
-    const char* const long_double_numbers = "[3.14159265358979323846, 1.5e-400, -0.000123456789012345678]";
-    REQUIRE(std::setlocale(LC_NUMERIC, "C") != nullptr);
-    const long_double_json expected_long_double = long_double_json::parse(long_double_numbers);
-
+    // substituted in place for '.', so the strtod fallback stops early. The
+    // conversion must still terminate rather than retry forever.
     const std::array<const char*, 6> names = {{"ar_EG.UTF-8", "ar_SA.UTF-8", "fa_IR.UTF-8", "ps_AF.UTF-8", "ar_EG", "fa_IR"}};
     bool tested = false;
     for (const char* name : names)
@@ -378,20 +369,12 @@ TEST_CASE("locale with a multi-byte decimal point")
         tested = true;
 
         // too many significant digits for Clinger's fast path, and an underflow
-        // that std::from_chars rejects: double does not depend on the locale
+        // that std::from_chars rejects: both reach the strtod fallback
         json j;
         CHECK_NOTHROW(j = json::parse("[3.14159265358979323846, 1.5e-400, -0.000123456789012345678]"));
         CHECK(j.is_array());
-        CHECK(j[0] == 3.14159265358979323846);
-        CHECK(j[1] == 0.0);
-        CHECK(j[2] == -0.000123456789012345678);
         CHECK(json::accept("3.14159265358979323846"));
 
-        // a long double that reaches the strtold fallback is not truncated
-        long_double_json ld;
-        CHECK_NOTHROW(ld = long_double_json::parse(long_double_numbers));
-        CHECK(ld == expected_long_double);
-
         // a value the locale-independent paths convert is not affected
         CHECK(json::parse("12.5") == 12.5);
     }

tests/src/unit-msgpack.cpp

@@ -1614,19 +1614,39 @@ TEST_CASE("MessagePack")
             CHECK_THROWS_WITH_AS(_ = json::from_msgpack(std::vector<uint8_t>({0x81})), "[json.exception.parse_error.110] parse error at byte 2: syntax error while parsing MessagePack string: unexpected end of input", json::parse_error&);
         }
 
-        SECTION("invalid UTF-8 in string (see #5529)")
+        SECTION("ill-formed UTF-8 in string (see #5529, #5651)")
         {
+            // the MessagePack specification explicitly allows a str object to
+            // contain a byte sequence that is not valid UTF-8 and expects a
+            // deserializer to hand the original bytes back unchanged; this
+            // library follows that, unlike CBOR/UBJSON/BJData/BSON, whose
+            // specifications require text strings to be valid UTF-8
+
             // a fixstr of length 2 (0xA0 | 2) whose bytes are not valid UTF-8
-            // (0xC0 0xAE is an overlong encoding of '.') must be rejected at
-            // decode time, matching every other kind of malformed binary
-            // input, rather than only failing later when the resulting
-            // value is dumped
-            json _;
-            CHECK_THROWS_WITH_AS(_ = json::from_msgpack(std::vector<uint8_t>({0xa2, 0xc0, 0xae})), "[json.exception.parse_error.113] parse error at byte 3: syntax error while parsing MessagePack string: invalid string: ill-formed UTF-8 byte", json::parse_error&);
-            CHECK(json::from_msgpack(std::vector<uint8_t>({0xa2, 0xc0, 0xae}), true, false).is_discarded());
+            // (0xC0 0xAE is an overlong encoding of '.') round-trips byte for
+            // byte as a string value
+            const std::vector<uint8_t> ill_formed_value = {0xa2, 0xc0, 0xae};
+            json j_value;
+            CHECK_NOTHROW(j_value = json::from_msgpack(ill_formed_value));
+            REQUIRE(j_value.is_string());
+            CHECK(j_value.get_ref<const json::string_t&>() == std::string("\xc0\xae"));
+            CHECK(json::from_msgpack(json::to_msgpack(j_value)) == j_value);
+            // dump() still requires valid UTF-8 and throws for such a value,
+            // unless an error handler that replaces or ignores the bytes is
+            // passed
+            CHECK_THROWS_AS(j_value.dump(), json::type_error&);
+
+            // the same bytes as an object key round-trip as well
+            const std::vector<uint8_t> ill_formed_key = {0x81, 0xa2, 0xc0, 0xae, 0x01};
+            json j_key;
+            CHECK_NOTHROW(j_key = json::from_msgpack(ill_formed_key));
+            REQUIRE(j_key.is_object());
+            CHECK(j_key.contains(std::string("\xc0\xae")));
+            CHECK(json::from_msgpack(json::to_msgpack(j_key)) == j_key);
 
             // a MessagePack bin8 blob with the very same bytes is NOT text
             // and must still be accepted as-is
+            json _;
             CHECK_NOTHROW(_ = json::from_msgpack(std::vector<uint8_t>({0xc4, 0x02, 0xc0, 0xae})));
             CHECK(_ == json::binary(std::vector<std::uint8_t>({0xc0, 0xae})));
 

tests/src/unit-regression2.cpp

@@ -18,6 +18,19 @@
 // for some reason including this after the json header leads to linker errors with VS 2017...
 #include <locale>
 
+// skip tests if JSON_DISABLE_TUPLE_REFERENCE_CONVERSION=1 (#2226)
+#if defined(JSON_DISABLE_TUPLE_REFERENCE_CONVERSION) && (JSON_DISABLE_TUPLE_REFERENCE_CONVERSION == 1)
+    #define SKIP_TESTS_FOR_TUPLE_REFERENCE_CONVERSION
+#endif
+
+// clang before 4 and GCC before 5 cannot create a std::tuple of basic_json
+// references at all, with or without JSON_DISABLE_TUPLE_REFERENCE_CONVERSION:
+// the tuple constructors make them instantiate basic_json's conversion operator
+// for libstdc++'s internal tuple bases, which fails hard
+#if (defined(__clang__) && __clang_major__ < 4) || (!defined(__clang__) && defined(__GNUC__) && __GNUC__ < 5)
+    #define SKIP_TESTS_FOR_JSON_REFERENCE_TUPLES
+#endif
+
 #define JSON_TESTS_PRIVATE
 #include <nlohmann/json.hpp>
 using json = nlohmann::json;
@@ -28,6 +41,7 @@ using ordered_json = nlohmann::ordered_json;
 
 #include <cstdio>
 #include <list>
+#include <tuple>
 #include <type_traits>
 #include <utility>
 
@@ -542,6 +556,20 @@ TEST_CASE("regression tests 2")
                       )));
     }
 
+#ifndef SKIP_TESTS_FOR_TUPLE_REFERENCE_CONVERSION
+    SECTION("issue #2226 - std::tuple dangling reference - implicit conversion")
+    {
+        // by default, a one-element tuple holding a json reference converts to
+        // a one-element array; JSON_DISABLE_TUPLE_REFERENCE_CONVERSION removes
+        // this conversion (see unit-disable-tuple-reference-conversion.cpp)
+        const json j = true;
+        CHECK(std::is_constructible<json, std::tuple<const json&>>::value);
+#ifndef SKIP_TESTS_FOR_JSON_REFERENCE_TUPLES
+        CHECK(json(std::forward_as_tuple(j)) == json::array({true}));
+#endif
+    }
+#endif
+
     SECTION("PR #2181 - regression bug with lvalue")
     {
         // see https://github.com/nlohmann/json/pull/2181#issuecomment-653326060

tests/src/unit-ubjson.cpp

@@ -2580,6 +2580,30 @@ TEST_CASE("Universal Binary JSON Specification Examples 1")
             CHECK(json::to_ubjson(j) == v);
             CHECK(json::from_ubjson(v) == j);
         }
+
+        SECTION("ill-formed UTF-8 (see #5651)")
+        {
+            // a string value whose bytes are not valid UTF-8 (0xC0 0xAE is an
+            // overlong encoding of '.') is rejected at decode time, matching
+            // every other kind of malformed binary input, and to_ubjson()
+            // rejects it as well, so a value it accepts can always be read
+            // back
+            const std::vector<uint8_t> v = {'S', 'i', 2, 0xc0, 0xae};
+            json _;
+            CHECK_THROWS_WITH_AS(_ = json::from_ubjson(v), "[json.exception.parse_error.113] parse error at byte 5: syntax error while parsing UBJSON string: invalid string: ill-formed UTF-8 byte", json::parse_error&);
+            CHECK(json::from_ubjson(v, true, false).is_discarded());
+
+            CHECK_THROWS_WITH_AS(json::to_ubjson(json("\xFF")), "[json.exception.type_error.316] invalid UTF-8 byte at index 0: 0xFF", json::type_error&);
+            // a truncated multi-byte sequence
+            CHECK_THROWS_WITH_AS(json::to_ubjson(json("\xC3")), "[json.exception.type_error.316] invalid UTF-8 byte at index 0: 0xC3", json::type_error&);
+            // an encoded surrogate half (U+D800)
+            CHECK_THROWS_WITH_AS(json::to_ubjson(json("\xED\xA0\x80")), "[json.exception.type_error.316] invalid UTF-8 byte at index 0: 0xED", json::type_error&);
+            // an overlong encoding of '.'
+            CHECK_THROWS_WITH_AS(json::to_ubjson(json("\xC0\xAF")), "[json.exception.type_error.316] invalid UTF-8 byte at index 0: 0xC0", json::type_error&);
+
+            // an object key with ill-formed UTF-8 is rejected the same way
+            CHECK_THROWS_WITH_AS(json::to_ubjson(json{{"\xFF", 1}}), "[json.exception.type_error.316] invalid UTF-8 byte at index 0: 0xFF", json::type_error&);
+        }
     }
 
     SECTION("Array Type")