Move instead of deep-copy ordered_json values when an object grows

ordered_map keeps its elements in a std::vector<std::pair<const Key, T>>.
With a std::string key, that pair is not nothrow move constructible (the
const key has to be copied), so std::vector copies every element when it
reallocates. For ordered_json, this deep-copies every member value an
object already holds, including whole nested subtrees, on each growth
step.

Grow the storage in ordered_map instead, copying the keys and moving the
values. This happens in two phases, so the strong exception guarantee is
kept without try/catch. The first phase may throw, but only touches a
temporary buffer: it copies the keys, value-initializes the values, and
constructs the new element. The second phase moves the values (noexcept)
and swaps the buffers. Because the new element is constructed before any
value is moved, arguments that refer to elements of the container stay
valid, as with std::vector. Types that cannot take this path keep the
std::vector behavior.

Parsing into ordered_json (ParseStringOrdered, Apple M1 Max, clang -O3):
twitter 3.20 -> 1.70 ms, citm_catalog 7.73 -> 3.67 ms, jeopardy 219 ->
177 ms, canada unchanged. The number of allocations for twitter and
citm_catalog drops by two thirds.

Also add ParseStringOrdered rows to the benchmarks, and document the
growth behavior and the exception safety of ordered_map.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

docs/mkdocs/docs/api/basic_json/from_cbor.md

@@ -80,8 +80,8 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
   the end of the file was not reached when `strict` was set to true
 - Throws [parse_error.112](../../home/exceptions.md#jsonexceptionparse_error112) if unsupported features from CBOR were
   used in the given input or if the input is not valid CBOR
-- Throws [parse_error.113](../../home/exceptions.md#jsonexceptionparse_error113) if a string was expected as a map key,
-  but not found
+- Throws [parse_error.113](../../home/exceptions.md#jsonexceptionparse_error113) if a map key is not a string (keys of other
+  types are not supported, as JSON object keys are always strings) or a string is malformed
 
 ## Complexity
 

docs/mkdocs/docs/api/basic_json/from_msgpack.md

@@ -73,8 +73,8 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
   the end of the file was not reached when `strict` was set to true
 - Throws [parse_error.112](../../home/exceptions.md#jsonexceptionparse_error112) if unsupported features from
   MessagePack were used in the given input or if the input is not valid MessagePack
-- Throws [parse_error.113](../../home/exceptions.md#jsonexceptionparse_error113) if a string was expected as a map key,
-  but not found
+- Throws [parse_error.113](../../home/exceptions.md#jsonexceptionparse_error113) if a map key is not a string (keys of other
+  types are not supported, as JSON object keys are always strings) or a string is malformed
 
 ## Complexity
 

docs/mkdocs/docs/api/ordered_map.md

@@ -28,6 +28,11 @@ A minimal map-like container that preserves insertion order for use within [`nlo
 The type uses a `std::vector` to store object elements. Therefore, adding elements can yield a reallocation in which
 case all iterators (including the `end()` iterator) and all references to the elements are invalidated.
 
+When the storage grows, the keys are copied and the mapped values are moved to the new storage. A plain `std::vector`
+would copy the whole elements instead, because their `#!cpp const` keys make them not nothrow move constructible; for
+[`ordered_json`](ordered_json.md), this would be a deep copy of every nested value. The values are only copied if
+`T` is not default constructible or not nothrow move assignable.
+
 ## Member types
 
 - **key_type** - key type (`Key`)
@@ -56,6 +61,11 @@ std::equal_to<>     // since C++14
 - **find**
 - **insert**
 
+## Exception safety
+
+**emplace**, **operator\[\]**, and **insert(value)** have the strong exception guarantee: if an exception is thrown (for
+instance, because copying a key or allocating memory fails), the contents of the container are unchanged.
+
 ## Complexity
 
 Because the elements are stored in a `std::vector` in insertion order, there is no index to look a key up by. Every
@@ -122,3 +132,4 @@ This differs from `#!cpp std::map`, where the same operations are O(log n).
 
 - Added in version 3.9.0 to implement [`nlohmann::ordered_json`](ordered_json.md).
 - Added **key_compare** member in version 3.11.0.
+- Changed in version 3.13.0: growing the storage moves the mapped values instead of copying them.

docs/mkdocs/docs/features/binary_formats/cbor.md

@@ -174,7 +174,20 @@ The library maps CBOR types to JSON value types as follows:
 
 !!! warning "Object keys"
 
-    CBOR allows map keys of any type, whereas JSON only allows strings as keys in object values. Therefore, CBOR maps with keys other than UTF-8 strings are rejected.
+    CBOR allows map keys of any type, whereas JSON only allows strings as keys in object values. Therefore, CBOR maps
+    with keys other than text strings (major type 3) are rejected with a
+    [`parse_error.113`](../../home/exceptions.md#jsonexceptionparse_error113) exception (or, with `allow_exceptions` set
+    to `false`, a discarded value) naming the type of the key that was found, for instance:
+
+    ```
+    [json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR object key: only string keys are supported, but found an unsigned integer; last byte: 0x01
+    ```
+
+    This applies to the [SAX interface](../parsing/sax_interface.md) as well, as the key is read before it is passed
+    on. This is a deliberate restriction of the library's JSON value model, not an oversight: formats built on CBOR
+    maps with integer keys, such as COSE ([RFC 9052](https://www.rfc-editor.org/rfc/rfc9052.html)) or CWT
+    ([RFC 8392](https://www.rfc-editor.org/rfc/rfc8392.html)), cannot be read with this library and need a
+    general-purpose CBOR library instead.
 
 !!! warning "UTF-8 validation of text strings"
 

docs/mkdocs/docs/features/binary_formats/messagepack.md

@@ -138,6 +138,21 @@ The library maps MessagePack types to JSON value types as follows:
 
     Any MessagePack output created by `to_msgpack` can be successfully parsed by `from_msgpack`.
 
+!!! warning "Object keys"
+
+    MessagePack allows map keys of any type, whereas JSON only allows strings as keys in object values. Like the
+    JSON-compatible [profile](https://github.com/msgpack/msgpack/blob/master/spec.md#profile) sketched in the
+    MessagePack specification, this library restricts map keys to `str` values. Maps with keys of any other type are
+    rejected with a [`parse_error.113`](../../home/exceptions.md#jsonexceptionparse_error113) exception (or, with
+    `allow_exceptions` set to `false`, a discarded value) naming the type of the key that was found, for instance:
+
+    ```
+    [json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack object key: only string keys are supported, but found nil; last byte: 0xC0
+    ```
+
+    This applies to the [SAX interface](../parsing/sax_interface.md) as well, as the key is read before it is passed
+    on. Such input needs a general-purpose MessagePack library instead.
+
 !!! warning "UTF-8 validation of string values"
 
     The MessagePack specification requires `str` values (`fixstr`, `str 8`, `str 16`, `str 32`) to be valid UTF-8.

docs/mkdocs/docs/home/exceptions.md

@@ -343,13 +343,20 @@ A string could not be read from a [binary format](../features/binary_formats/ind
 string was read where one was required (for instance as a map key), the string's length specification is invalid, or
 the string's bytes are not valid UTF-8.
 
+CBOR and MessagePack allow map keys of any type, but JSON object keys are always strings. Maps with keys of any other
+type (for instance integers or `null`) are therefore not supported; see the notes on
+[CBOR](../features/binary_formats/cbor.md) and [MessagePack](../features/binary_formats/messagepack.md).
+
 !!! failure "Example messages"
 
     ```
-    [json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0xFF
+    [json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR object key: only string keys are supported, but found an unsigned integer; last byte: 0x01
     ```
     ```
-    [json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack string: expected length specification (0xA0-0xBF, 0xD9-0xDB); last byte: 0xFF
+    [json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack object key: only string keys are supported, but found nil; last byte: 0xC0
+    ```
+    ```
+    [json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0x7C
     ```
     ```
     [json.exception.parse_error.113] parse error at byte 2: syntax error while parsing UBJSON char: byte after 'C' must be in range 0x00..0x7F; last byte: 0x82

include/nlohmann/detail/input/binary_reader.hpp

@@ -1324,6 +1324,80 @@ class binary_reader
         }
     }
 
+    /*!
+    @brief reads a CBOR object key
+
+    RFC 8949 allows any data item as a map key, but only strings have a
+    counterpart in JSON. A key of any other type is rejected with a message
+    naming that type, rather than the one @ref get_cbor_string gives for a
+    malformed string.
+
+    @param[out] result  created key
+
+    @return whether key creation completed
+    */
+    bool get_cbor_object_key(string_t& result)
+    {
+        // EOF and major type 3 (text string) are left to get_cbor_string
+        if (current == char_traits<char_type>::eof() || (static_cast<unsigned int>(current) & 0xE0u) == 0x60u)
+        {
+            return get_cbor_string(result);
+        }
+
+        const char* found = nullptr;
+        switch (static_cast<unsigned int>(current) >> 5u)
+        {
+            case 0:
+                found = "an unsigned integer";
+                break;
+            case 1:
+                found = "a negative integer";
+                break;
+            case 2:
+                found = "a byte string";
+                break;
+            case 4:
+                found = "an array";
+                break;
+            case 5:
+                found = "a map";
+                break;
+            case 6:
+                found = "a tag";
+                break;
+            default: // major type 7
+                switch (current)
+                {
+                    case 0xF4:
+                    case 0xF5:
+                        found = "a boolean";
+                        break;
+                    case 0xF6:
+                        found = "null";
+                        break;
+                    case 0xF7:
+                        found = "undefined";
+                        break;
+                    case 0xF9:
+                    case 0xFA:
+                    case 0xFB:
+                        found = "a floating-point number";
+                        break;
+                    case 0xFF:
+                        found = "a break stop code";
+                        break;
+                    default:
+                        found = "a simple value";
+                        break;
+                }
+                break;
+        }
+
+        auto last_token = get_token_string();
+        return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read,
+                                exception_message(input_format_t::cbor, concat("only string keys are supported, but found ", found, "; last byte: 0x", last_token), "object key"), nullptr));
+    }
+
     /*!
     @brief reads a definite-length CBOR byte array
 
@@ -1568,7 +1642,7 @@ class binary_reader
                 if (top.is_object)
                 {
                     key.clear();
-                    if (JSON_HEDLEY_UNLIKELY(!get_cbor_string(key) || !sax->key(key)))
+                    if (JSON_HEDLEY_UNLIKELY(!get_cbor_object_key(key) || !sax->key(key)))
                     {
                         return false;
                     }
@@ -2069,6 +2143,98 @@ class binary_reader
         }
     }
 
+    /*!
+    @brief reads a MessagePack object key
+
+    The MessagePack specification allows any type as a map key, but only
+    strings have a counterpart in JSON. A key of any other type is rejected
+    with a message naming that type, rather than the one @ref
+    get_msgpack_string gives for a malformed string.
+
+    @param[out] result  created key
+
+    @return whether key creation completed
+    */
+    bool get_msgpack_object_key(string_t& result)
+    {
+        const char* found = nullptr;
+        switch (current)
+        {
+            case 0xC0:
+                found = "nil";
+                break;
+            case 0xC2:
+            case 0xC3:
+                found = "a boolean";
+                break;
+            case 0xCA:
+            case 0xCB:
+                found = "a float";
+                break;
+            case 0xC4:
+            case 0xC5:
+            case 0xC6:
+                found = "a bin";
+                break;
+            case 0xC7:
+            case 0xC8:
+            case 0xC9:
+            case 0xD4:
+            case 0xD5:
+            case 0xD6:
+            case 0xD7:
+            case 0xD8:
+                found = "an ext";
+                break;
+            case 0xCC:
+            case 0xCD:
+            case 0xCE:
+            case 0xCF:
+            case 0xD0:
+            case 0xD1:
+            case 0xD2:
+            case 0xD3:
+                found = "an integer";
+                break;
+            case 0xDC:
+            case 0xDD:
+                found = "an array";
+                break;
+            case 0xDE:
+            case 0xDF:
+                found = "a map";
+                break;
+            default:
+                // fixint, fixmap, and fixarray; strings, EOF, and the unused
+                // byte 0xC1 are left to get_msgpack_string
+                if (current == char_traits<char_type>::eof())
+                {
+                    return get_msgpack_string(result);
+                }
+                if (current <= 0x7F || current >= 0xE0)
+                {
+                    found = "an integer";
+                }
+                else if (current <= 0x8F)
+                {
+                    found = "a map";
+                }
+                else if (current <= 0x9F)
+                {
+                    found = "an array";
+                }
+                else
+                {
+                    return get_msgpack_string(result);
+                }
+                break;
+        }
+
+        auto last_token = get_token_string();
+        return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read,
+                                exception_message(input_format_t::msgpack, concat("only string keys are supported, but found ", found, "; last byte: 0x", last_token), "object key"), nullptr));
+    }
+
     /*!
     @brief reads a MessagePack byte array
 
@@ -2231,7 +2397,7 @@ class binary_reader
                 {
                     get();
                     key.clear();
-                    if (JSON_HEDLEY_UNLIKELY(!get_msgpack_string(key) || !sax->key(key)))
+                    if (JSON_HEDLEY_UNLIKELY(!get_msgpack_object_key(key) || !sax->key(key)))
                     {
                         return false;
                     }

include/nlohmann/detail/input/lexer.hpp

@@ -206,7 +206,6 @@ class lexer : public lexer_base<BasicJsonType>
     explicit lexer(InputAdapterType&& adapter, bool ignore_comments_ = false, bool discard_number_values_ = false) noexcept
         : ia(std::move(adapter))
         , ignore_comments(ignore_comments_)
-        , decimal_point_char(static_cast<char_int_type>(get_decimal_point()))
         , discard_number_values(discard_number_values_)
     {}
 
@@ -222,8 +221,7 @@ class lexer : public lexer_base<BasicJsonType>
     // locales
     /////////////////////
 
-    /// return the locale-dependent decimal point
-    JSON_HEDLEY_PURE
+    /// return the decimal point of the current locale
     static char get_decimal_point() noexcept
     {
         const auto* loc = localeconv();
@@ -1092,9 +1090,10 @@ class lexer : public lexer_base<BasicJsonType>
             token_type::value_float if number could be successfully scanned,
             token_type::parse_error otherwise
 
-    @note The scanner is independent of the current locale. Internally, the
-          locale's decimal point is used instead of `.` to work with the
-          locale-dependent converters.
+    @note The scanner is independent of the current locale: token_buffer
+          always holds `.`. Only the std::strtod fallback of convert_number()
+          depends on the locale, and it looks up the decimal point right
+          before converting (see convert_float_locale_aware()).
     */
     token_type scan_number()  // lgtm [cpp/use-of-goto] `goto` is used in this function to implement the number-parsing state machine described above. By design, any finite input will eventually reach the "done" state or return token_type::parse_error. In each intermediate state, 1 byte of the input is appended to the token_buffer vector, and only the already initialized variables token_buffer, number_type, and error_message are manipulated.
     {
@@ -1183,7 +1182,7 @@ scan_number_zero:
         {
             case '.':
             {
-                add(decimal_point_char);
+                add(current);
                 decimal_point_position = token_buffer.size() - 1;
                 goto scan_number_decimal1;
             }
@@ -1220,7 +1219,7 @@ scan_number_any1:
 
             case '.':
             {
-                add(decimal_point_char);
+                add(current);
                 decimal_point_position = token_buffer.size() - 1;
                 goto scan_number_decimal1;
             }
@@ -1462,9 +1461,9 @@ scan_number_done:
 
         // Only a number below 1 can carry further insignificant zeros, and only
         // while the count stays at the limit does removing them change the
-        // answer - so this loop is skipped for all but a few tokens. Note
-        // token_buffer holds the locale's decimal point, so the fraction is
-        // located through decimal_point_position rather than by searching '.'.
+        // answer - so this loop is skipped for all but a few tokens. The
+        // fraction is located through decimal_point_position rather than by
+        // searching '.'.
         if (lead_zero != 0)
         {
             JSON_ASSERT(has_dot != 0); // an integer "0" cannot reach the limit
@@ -1482,8 +1481,8 @@ scan_number_done:
     @brief convert the number text in token_buffer to its value and token type
 
     The digit sequence in token_buffer has already been validated (by the
-    scan_number() state machine or by the contiguous fast path) and holds the
-    locale decimal point in place of '.'. Integers are parsed first and fall
+    scan_number() state machine or by the contiguous fast path) and holds '.'
+    as decimal point, independent of the locale. Integers are parsed first and fall
     back to floating point on overflow. This is shared so both scanners produce
     identical results.
 
@@ -1563,7 +1562,7 @@ scan_number_done:
         // integer conversion above overflowed. Prefer std::from_chars
         // (Eisel-Lemire, locale-independent, correctly rounded) when available;
         // otherwise the exact Clinger fast path (double only); otherwise the
-        // locale-aware strtof/strtod.
+        // locale-aware strtof/strtod/strtold.
         if (parse_float_from_chars(num_begin, num_end, value_float))
         {
             return token_type::value_float;
@@ -1572,26 +1571,75 @@ scan_number_done:
         // extra pass over the token's bytes, which otherwise shows up on
         // high-precision inputs such as canada.json
         if (mantissa_fits_clinger(mantissa_end)
-                && parse_float_fast(num_begin, num_end, decimal_point_char, value_float))
+                && parse_float_fast(num_begin, num_end, value_float))
         {
             return token_type::value_float;
         }
 
-        char* endptr = nullptr; // NOLINT(misc-const-correctness,cppcoreguidelines-pro-type-vararg,hicpp-vararg)
-        strtof(value_float, token_buffer.data(), &endptr);
-
-        // we checked the number format before
-        JSON_ASSERT(endptr == token_buffer.data() + token_buffer.size());
-
+        convert_float_locale_aware();
         return token_type::value_float;
     }
 
+    /*!
+    @brief convert the float in token_buffer with strtof/strtod/strtold
+
+    These functions expect the decimal point of the *current* locale, so it is
+    looked up right before the conversion instead of once when the lexer is
+    constructed: a locale change in between (by a parser callback, a SAX
+    handler, or another thread) must not truncate the value (#5198). The
+    token has been validated before, so if the conversion stops early and the
+    decimal point changed in the meantime, the locale changed between the
+    lookup and the call, and the conversion is repeated with the new decimal
+    point. If the decimal point did not change, a retry cannot succeed: the
+    locale's decimal point is not a single character (e.g., the two-byte
+    U+066B of ar_EG.UTF-8 or fa_IR.UTF-8) and cannot be substituted in place.
+    The value strtod parsed up to that point is kept, as before this change.
+
+    Note that changing the locale in another thread *while* strtod runs is
+    undefined behavior of the C library, which this function cannot prevent.
+    */
+    void convert_float_locale_aware()
+    {
+        const bool has_dot = decimal_point_position != std::string::npos;
+        char decimal_point = get_decimal_point();
+        for (;;)
+        {
+            const bool substitute = has_dot && decimal_point != '.';
+            if (substitute)
+            {
+                token_buffer[decimal_point_position] = static_cast<typename string_t::value_type>(decimal_point);
+            }
+
+            char* endptr = nullptr; // NOLINT(misc-const-correctness,cppcoreguidelines-pro-type-vararg,hicpp-vararg)
+            strtof(value_float, token_buffer.data(), &endptr);
+
+            if (substitute)
+            {
+                // get_string() hands the token to the SAX interface with '.'
+                token_buffer[decimal_point_position] = '.';
+            }
+
+            if (JSON_HEDLEY_LIKELY(endptr == token_buffer.data() + token_buffer.size()))
+            {
+                return;
+            }
+
+            // retry only if the locale changed; otherwise, this would loop forever
+            const char current_decimal_point = get_decimal_point();
+            if (current_decimal_point == decimal_point)
+            {
+                return;
+            }
+            decimal_point = current_decimal_point;
+        }
+    }
+
     /*!
     @brief contiguous fast path for scanning a number
 
     Parses the whole number token straight from the input buffer, avoiding the
     per-character get()/add() of scan_number(). On success it fills token_buffer
-    (with the locale decimal point substituted, as scan_number() does) and
+    (as scan_number() does) and
     returns the token type. On anything it does not fully recognize as a
     well-formed number it makes no state change and returns
     token_type::uninitialized, so the caller falls back to scan_number(), which
@@ -1707,16 +1755,11 @@ scan_number_done:
         }
 #endif
 
-        // materialize the token exactly as scan_number() would, substituting the
-        // locale decimal point so convert_number()'s strtof fallback stays valid.
-        // reset() already cleared token_buffer, so append() fills it (assign() is
-        // avoided because custom string_t types need not provide it)
+        // materialize the token exactly as scan_number() would. reset() already
+        // cleared token_buffer, so append() fills it (assign() is avoided
+        // because custom string_t types need not provide it)
         token_buffer.append(reinterpret_cast<const typename string_t::value_type*>(data), len);
-        if (dot_index != std::string::npos)
-        {
-            token_buffer[dot_index] = static_cast<typename string_t::value_type>(decimal_point_char);
-            decimal_point_position = dot_index;
-        }
+        decimal_point_position = dot_index;
 
         ia.bulk_skip(len - 1);
         position.chars_read_total += (len - 1);
@@ -1983,11 +2026,7 @@ scan_number_done:
     /// return current string value (implicitly resets the token; useful only once)
     string_t& get_string()
     {
-        // translate decimal points from locale back to '.' (#4084)
-        if (decimal_point_char != '.' && decimal_point_position != std::string::npos)
-        {
-            token_buffer[decimal_point_position] = '.';
-        }
+        // a number token holds '.' regardless of the locale (#4084)
         return token_buffer;
     }
 
@@ -2283,9 +2322,7 @@ scan_number_done:
     number_unsigned_t value_unsigned = 0;
     number_float_t value_float = 0;
 
-    /// the decimal point
-    const char_int_type decimal_point_char = '.';
-    /// the position of the decimal point in the input
+    /// the position of the decimal point in token_buffer
     std::size_t decimal_point_position = std::string::npos;
 
     /// whether the caller (e.g. accept()/json_sax_acceptor) only needs the

include/nlohmann/detail/input/number_parse.hpp

@@ -118,14 +118,12 @@ std::strtod. The parser only activates for number_float_t == double; float and
 long double keep the std::strtof/std::strtold paths (see the templated overload
 below).
 
-@param[in]  first          pointer to the first character of the number
-@param[in]  last           pointer past the last character
-@param[in]  decimal_point  the (locale-dependent) decimal point character
-@param[out] out            the parsed value on success
+@param[in]  first  pointer to the first character of the number
+@param[in]  last   pointer past the last character
+@param[out] out    the parsed value on success
 @return true if the value was parsed exactly; false to fall back to strtod
 */
-template<typename DecimalPointType>
-bool parse_float_fast(const char* first, const char* last, DecimalPointType decimal_point, double& out) noexcept
+inline bool parse_float_fast(const char* first, const char* last, double& out) noexcept
 {
 #if defined(FLT_EVAL_METHOD) && FLT_EVAL_METHOD != 0
     // Clinger's fast path is only exact when double operations are evaluated in
@@ -136,7 +134,6 @@ bool parse_float_fast(const char* first, const char* last, DecimalPointType deci
     // std::from_chars / std::strtod path.
     static_cast<void>(first);
     static_cast<void>(last);
-    static_cast<void>(decimal_point);
     static_cast<void>(out);
     return false;
 #else
@@ -175,7 +172,7 @@ bool parse_float_fast(const char* first, const char* last, DecimalPointType deci
             ++num_digits;
             fractional_digits += static_cast<int>(seen_dot);
         }
-        else if (static_cast<DecimalPointType>(c) == decimal_point)
+        else if (c == '.')
         {
             if (JSON_HEDLEY_UNLIKELY(seen_dot))
             {
@@ -260,8 +257,8 @@ bool parse_float_fast(const char* first, const char* last, DecimalPointType deci
 }
 
 /// fast float path is only exact for `double`; decline for float/long double
-template<typename DecimalPointType, typename FloatType>
-bool parse_float_fast(const char* /*first*/, const char* /*last*/, DecimalPointType /*decimal_point*/, FloatType& /*out*/) noexcept
+template<typename FloatType>
+bool parse_float_fast(const char* /*first*/, const char* /*last*/, FloatType& /*out*/) noexcept
 {
     return false;
 }
@@ -273,9 +270,7 @@ std::from_chars is locale-independent, correctly rounded, and - via the
 Eisel-Lemire algorithm in modern standard libraries - much faster than strtod
 over the whole value range (not just the Clinger subset). It is used only when
 __cpp_lib_to_chars indicates full floating-point support and only when it
-consumes the entire token ([first, last)); a partial parse means the buffer
-uses a non-'.' locale decimal point, in which case the caller falls back to the
-locale-aware path. An under-/overflow (result_out_of_range) also declines, so
+consumes the entire token ([first, last)). An under-/overflow (result_out_of_range) also declines, so
 the caller's strtod fallback supplies the well-defined ±inf/0 result the parser
 expects (side-stepping the P4168 divergence between implementations).
 

include/nlohmann/json.hpp

@@ -6061,240 +6061,21 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
     {
         // the patch
         basic_json result(value_t::array);
-        diff_recursively(result, source, target, path, 0);
-        return result;
-    }
 
-  private:
-    /// @brief two arrays or two objects @ref diff_iteratively is diffing
-    struct diff_frame
-    {
-        diff_frame(const basic_json* source_, const basic_json* target_, const std::size_t path_length_) noexcept
-            : source(source_), target(target_), path_length(path_length_)
-        {}
-
-        // declared for GCC's -Weffc++, which asks for them in a class with
-        // pointer members and a non-trivial destructor; the exception
-        // specifications are left implicit, as GCC 4.8 rejects explicit ones
-        // that differ from them
-        diff_frame(const diff_frame&) = default;
-        diff_frame(diff_frame&&) = default;
-        diff_frame& operator=(const diff_frame&) = default;
-        diff_frame& operator=(diff_frame&&) = default;
-        ~diff_frame() = default;
-
-        /// the values being diffed, both arrays or both objects
-        const basic_json* source;
-        const basic_json* target;
-        /// the length of their path in `current_path`
-        std::size_t path_length;
-        /// arrays: the next index to diff
-        std::size_t index = 0;
-        /// objects: the next member of source to look at
-        const_iterator member{}; // NOLINT(readability-redundant-member-init)
-        /// objects: the keys common to both, in source's order
-        std::vector<typename object_t::key_type> common_keys{}; // NOLINT(readability-redundant-member-init)
-        /// objects: the next entry of common_keys
-        std::size_t next_common = 0;
-        /// objects: the "add" operations for keys only target has
-        basic_json added_ops{}; // NOLINT(readability-redundant-member-init)
-    };
-
-    // The operations of a diff are built by the functions below rather than
-    // where they are needed: building one takes several temporaries, and
-    // unoptimized builds give each temporary a stack slot of its own in the
-    // function it appears in. In diff_recursively, which is on the call stack
-    // once per nesting level, that made every level cost kilobytes of stack.
-
-    /// @brief append a "replace" operation for @a path with @a value to @a result
-    static void diff_replace(basic_json& result, const string_t& path, const basic_json& value)
-    {
-        result.push_back(
-        {
-            {"op", "replace"}, {"path", path}, {"value", value}
-        });
-    }
-
-    /// @brief append a "remove" operation for @a path to @a result
-    static void diff_remove(basic_json& result, const string_t& path)
-    {
-        result.push_back(object(
-        {
-            {"op", "remove"}, {"path", path}
-        }));
-    }
-
-    /// @brief append an "add" operation for @a path with @a value to @a result
-    static void diff_add(basic_json& result, const string_t& path, const basic_json& value)
-    {
-        result.push_back(
-        {
-            {"op", "add"}, {"path", path}, {"value", value}
-        });
-    }
-
-    /// @brief append the "remove" operations for the elements of array
-    ///        @a source from @a index on, and the "add" operations for the
-    ///        elements of array @a target from source's size on, to @a result
-    static void diff_array_tails(basic_json& result, const basic_json& source, const basic_json& target,
-                                 const string_t& path, const std::size_t index)
-    {
-        // remove my remaining elements, highest index first; appending
-        // in that order avoids the quadratic reinsertion done before
-        for (std::size_t j = source.size(); j > index; --j)
-        {
-            diff_remove(result, detail::concat<string_t>(path, '/', detail::to_string<string_t>(j - 1)));
-        }
-
-        // add other remaining elements
-        for (std::size_t i = source.size(); i < target.size(); ++i)
-        {
-            diff_add(result, detail::concat<string_t>(path, "/-"), target[i]);
-        }
-    }
-
-    /*!
-    @brief compare the keys of objects @a source and @a target
-
-    If the keys both objects have are in the same order in both, and the keys
-    only @a target has come after them, stores the keys common to both in
-    source's order in @a common_keys, stores the "add" operations for the keys
-    only @a target has in @a added_ops, and returns true: the caller then diffs
-    the objects member by member. Otherwise, appends operations that remove
-    every member of @a source and add every member of @a target to @a result,
-    and returns false.
-    */
-    static bool diff_object_keys(basic_json& result, const basic_json& source, const basic_json& target,
-                                 const string_t& path, std::vector<typename object_t::key_type>& common_keys,
-                                 basic_json& added_ops)
-    {
-        // first pass: record, for every source key, whether it is
-        // common to both objects (in source's iteration order) or
-        // was deleted (i.e., in source but not in target) -- this is
-        // a by-product of the target.find() call already needed to
-        // tell the two cases apart, so it adds no extra lookups. The
-        // "remove" ops themselves are emitted later, interleaved
-        // with the per-key diffs in the caller's fast path, to match
-        // source's original iteration order (as the original,
-        // pre-reordering-aware implementation did) instead of
-        // grouping all removes before all per-key diffs.
-        std::vector<typename object_t::key_type> common_keys_source_order;
-        for (auto it = source.cbegin(); it != source.cend(); ++it)
-        {
-            if (target.find(it.key()) != target.end())
-            {
-                common_keys_source_order.push_back(it.key());
-            }
-        }
-
-        // second pass: find keys that were added (i.e., in target but
-        // not in source), and record the keys common to both, in
-        // target's iteration order -- again a by-product of the
-        // source.find() call already needed to detect added keys. At
-        // the same time, determine whether every added key comes
-        // after every common key in target's order (a precondition
-        // for the fast path, which only ever appends new keys
-        // at the very end): for an object_t whose iteration order is
-        // a pure function of the key set (e.g. the default std::map,
-        // which always iterates in sorted key order), the order
-        // check further below is always true and this whole
-        // mechanism is effectively a no-op; it only matters for a
-        // reorderable object_t such as the one backing `ordered_json`.
-        // The patch ops for keys that were added (i.e., in target but not
-        // in source) are built here so the fast path can reuse
-        // them without a second source.find() per target key. Only
-        // used by the fast path -- the slow (reordering) path
-        // rebuilds "add" ops for every key itself.
-        std::vector<typename object_t::key_type> common_keys_target_order;
-        bool new_keys_form_suffix = true;
-        bool seen_new_key = false;
-        for (auto it = target.cbegin(); it != target.cend(); ++it)
-        {
-            if (source.find(it.key()) == source.end())
-            {
-                seen_new_key = true;
-                diff_add(added_ops, detail::concat<string_t>(path, '/', detail::escape(it.key())), it.value());
-            }
-            else
-            {
-                common_keys_target_order.push_back(it.key());
-                if (seen_new_key)
-                {
-                    new_keys_form_suffix = false;
-                }
-            }
-        }
-
-        if (common_keys_source_order == common_keys_target_order && new_keys_form_suffix)
-        {
-            // fast path: order of common keys already matches (or the
-            // object_t's iteration order does not depend on
-            // insertion history), so a plain per-key diff is correct
-            // and minimal, as before
-            common_keys = std::move(common_keys_source_order);
-            return true;
-        }
-
-        // slow path: the common keys are in a different relative
-        // order in source and target (only possible for a
-        // reorderable object_t like ordered_map). Building a
-        // minimal reordering patch is a nontrivial (LCS-like)
-        // problem; instead, remove every source key -- both
-        // deleted keys (which must be removed regardless) and
-        // common keys (removed so they can be re-added in
-        // target's order) -- and re-add every key that should
-        // remain, with its final target value, in target's
-        // order. basic_json::patch()'s "add" operation on an
-        // object uses operator[], which appends at the end for a
-        // vector-backed insertion-ordered map when the key does
-        // not already exist -- so removing a key and then adding
-        // it moves it to the end, fixing its position.
-        for (auto it = source.cbegin(); it != source.cend(); ++it)
-        {
-            diff_remove(result, detail::concat<string_t>(path, '/', detail::escape(it.key())));
-        }
-
-        // add every key that is either common (just removed
-        // above) or brand new, in target's iteration order, so
-        // that the final order after applying the patch matches
-        // target exactly
-        for (auto it = target.cbegin(); it != target.cend(); ++it)
-        {
-            diff_add(result, detail::concat<string_t>(path, '/', detail::escape(it.key())), it.value());
-        }
-        return false;
-    }
-
-    /*!
-    @brief @ref diff, for values at nesting level @a depth, appending the
-           operations to @a result
-
-    Diffing two arrays or objects calls this function again, once per nesting
-    level, so values nested deeply enough used to exhaust the call stack and
-    terminate the process. The descent is bounded here: once @ref
-    detail::recursion_depth_limit levels have been entered, @ref
-    diff_iteratively diffs what is left without the call stack.
-    */
-    static void diff_recursively(basic_json& result, const basic_json& source, const basic_json& target,
-                                 const string_t& path, const std::size_t depth)
-    {
-        // if the values are the same, there is nothing to do
+        // if the values are the same, return an empty patch
         if (source == target)
         {
-            return;
-        }
-
-        if (JSON_HEDLEY_UNLIKELY(depth >= detail::recursion_depth_limit()))
-        {
-            diff_iteratively(result, source, target, path);
-            return;
+            return result;
         }
 
         if (source.type() != target.type())
         {
             // different types: replace value
-            diff_replace(result, path, target);
-            return;
+            result.push_back(
+            {
+                {"op", "replace"}, {"path", path}, {"value", target}
+            });
+            return result;
         }
 
         switch (source.type())
@@ -6306,50 +6087,185 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
                 while (i < source.size() && i < target.size())
                 {
                     // recursive call to compare array values at index i
-                    diff_recursively(result, source[i], target[i], detail::concat<string_t>(path, '/', detail::to_string<string_t>(i)), depth + 1);
+                    auto temp_diff = diff(source[i], target[i], detail::concat<string_t>(path, '/', detail::to_string<string_t>(i)));
+                    result.insert(result.end(), temp_diff.begin(), temp_diff.end());
                     ++i;
                 }
 
                 // We now reached the end of at least one array
                 // in a second pass, traverse the remaining elements
-                diff_array_tails(result, source, target, path, i);
+
+                // remove my remaining elements, highest index first; appending
+                // in that order avoids the quadratic reinsertion done before
+                for (std::size_t j = source.size(); j > i; --j)
+                {
+                    result.push_back(object(
+                    {
+                        {"op", "remove"},
+                        {"path", detail::concat<string_t>(path, '/', detail::to_string<string_t>(j - 1))}
+                    }));
+                }
+                i = source.size();
+
+                // add other remaining elements
+                while (i < target.size())
+                {
+                    result.push_back(
+                    {
+                        {"op", "add"},
+                        {"path", detail::concat<string_t>(path, "/-")},
+                        {"value", target[i]}
+                    });
+                    ++i;
+                }
+
                 break;
             }
 
             case value_t::object:
             {
-                std::vector<typename object_t::key_type> common_keys;
-                basic_json added_ops(value_t::array);
-                if (diff_object_keys(result, source, target, path, common_keys, added_ops))
+                // first pass: record, for every source key, whether it is
+                // common to both objects (in source's iteration order) or
+                // was deleted (i.e., in source but not in target) -- this is
+                // a by-product of the target.find() call already needed to
+                // tell the two cases apart, so it adds no extra lookups. The
+                // "remove" ops themselves are emitted later, interleaved
+                // with the recursive per-key diffs in the fast path below,
+                // to match source's original iteration order (as the
+                // original, pre-reordering-aware implementation did) instead
+                // of grouping all removes before all recursive diffs.
+                std::vector<typename object_t::key_type> common_keys_source_order;
+                for (auto it = source.cbegin(); it != source.cend(); ++it)
                 {
-                    // fast path: common_keys is, by construction, the
-                    // subsequence of source's keys that are common to both
-                    // objects, in source's iteration order -- so it can be
-                    // walked in lockstep with `source` using a cheap key
-                    // comparison instead of another lookup. Deleted keys
-                    // (those source keys not in common_keys) are interleaved
-                    // here too, in source's original order, to match the
-                    // historical (pre-reordering-aware) output order.
-                    auto common_it = common_keys.cbegin();
+                    if (target.find(it.key()) != target.end())
+                    {
+                        common_keys_source_order.push_back(it.key());
+                    }
+                }
+
+                // second pass: find keys that were added (i.e., in target but
+                // not in source), and record the keys common to both, in
+                // target's iteration order -- again a by-product of the
+                // source.find() call already needed to detect added keys. At
+                // the same time, determine whether every added key comes
+                // after every common key in target's order (a precondition
+                // for the fast path below, which only ever appends new keys
+                // at the very end): for an object_t whose iteration order is
+                // a pure function of the key set (e.g. the default std::map,
+                // which always iterates in sorted key order), the order
+                // check further below is always true and this whole
+                // mechanism is effectively a no-op; it only matters for a
+                // reorderable object_t such as the one backing `ordered_json`.
+                // patch ops for keys that were added (i.e., in target but not
+                // in source); built here so the fast path below can reuse
+                // them without a second source.find() per target key. Only
+                // used by the fast path -- the slow (reordering) path
+                // rebuilds "add" ops for every key itself.
+                std::vector<typename object_t::key_type> common_keys_target_order;
+                basic_json added_ops(value_t::array);
+                bool new_keys_form_suffix = true;
+                bool seen_new_key = false;
+                for (auto it = target.cbegin(); it != target.cend(); ++it)
+                {
+                    if (source.find(it.key()) == source.end())
+                    {
+                        seen_new_key = true;
+                        const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
+                        added_ops.push_back(
+                        {
+                            {"op", "add"}, {"path", path_key},
+                            {"value", it.value()}
+                        });
+                    }
+                    else
+                    {
+                        common_keys_target_order.push_back(it.key());
+                        if (seen_new_key)
+                        {
+                            new_keys_form_suffix = false;
+                        }
+                    }
+                }
+
+                if (common_keys_source_order == common_keys_target_order && new_keys_form_suffix)
+                {
+                    // fast path: order of common keys already matches (or the
+                    // object_t's iteration order does not depend on
+                    // insertion history), so a plain per-key recursive diff
+                    // is correct and minimal, as before. common_keys_source_order
+                    // is, by construction, the subsequence of source's keys
+                    // that are common to both objects, in source's iteration
+                    // order -- so it can be walked in lockstep with `source`
+                    // using a cheap key comparison instead of another lookup.
+                    // Deleted keys (those source keys not in common_keys_source_order)
+                    // are interleaved here too, in source's original order, to
+                    // match the historical (pre-reordering-aware) output order.
+                    auto common_it = common_keys_source_order.cbegin();
                     for (auto it = source.cbegin(); it != source.cend(); ++it)
                     {
-                        if (common_it != common_keys.cend() && it.key() == *common_it)
+                        if (common_it != common_keys_source_order.cend() && it.key() == *common_it)
                         {
-                            diff_recursively(result, it.value(), target[it.key()], detail::concat<string_t>(path, '/', detail::escape(it.key())), depth + 1);
+                            const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
+                            auto temp_diff = diff(it.value(), target[it.key()], path_key);
+                            result.insert(result.end(), temp_diff.begin(), temp_diff.end());
                             ++common_it;
                         }
                         else
                         {
                             // found a key that is not in target -> remove it
-                            diff_remove(result, detail::concat<string_t>(path, '/', detail::escape(it.key())));
+                            const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
+                            result.push_back(object(
+                            {
+                                {"op", "remove"}, {"path", path_key}
+                            }));
                         }
                     }
 
-                    // append the "add" ops for brand-new keys collected by
-                    // diff_object_keys -- no second source.find() per target
-                    // key needed
+                    // append the "add" ops for brand-new keys collected above
+                    // during the pass over target -- no second source.find()
+                    // per target key needed
                     result.insert(result.end(), added_ops.begin(), added_ops.end());
                 }
+                else
+                {
+                    // slow path: the common keys are in a different relative
+                    // order in source and target (only possible for a
+                    // reorderable object_t like ordered_map). Building a
+                    // minimal reordering patch is a nontrivial (LCS-like)
+                    // problem; instead, remove every source key -- both
+                    // deleted keys (which must be removed regardless) and
+                    // common keys (removed so they can be re-added in
+                    // target's order) -- and re-add every key that should
+                    // remain, with its final target value, in target's
+                    // order. basic_json::patch()'s "add" operation on an
+                    // object uses operator[], which appends at the end for a
+                    // vector-backed insertion-ordered map when the key does
+                    // not already exist -- so removing a key and then adding
+                    // it moves it to the end, fixing its position.
+                    for (auto it = source.cbegin(); it != source.cend(); ++it)
+                    {
+                        const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
+                        result.push_back(object(
+                        {
+                            {"op", "remove"}, {"path", path_key}
+                        }));
+                    }
+
+                    // add every key that is either common (just removed
+                    // above) or brand new, in target's iteration order, so
+                    // that the final order after applying the patch matches
+                    // target exactly
+                    for (auto it = target.cbegin(); it != target.cend(); ++it)
+                    {
+                        const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
+                        result.push_back(
+                        {
+                            {"op", "add"}, {"path", path_key},
+                            {"value", it.value()}
+                        });
+                    }
+                }
+
                 break;
             }
 
@@ -6364,170 +6280,16 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
             default:
             {
                 // both primitive types: replace value
-                diff_replace(result, path, target);
+                result.push_back(
+                {
+                    {"op", "replace"}, {"path", path}, {"value", target}
+                });
                 break;
             }
         }
+
+        return result;
     }
-
-    /*!
-    @brief @ref diff without the call stack, appending the operations to
-           @a result
-
-    Produces the same operations as @ref diff_recursively. Only reached for
-    values nested more deeply than @ref detail::recursion_depth_limit.
-    */
-    static void diff_iteratively(basic_json& result, const basic_json& source, const basic_json& target,
-                                 const string_t& path)
-    {
-        // The arrays and objects being diffed are kept on an explicit stack,
-        // and every pair of elements is still diffed completely before the
-        // next one, so the operations come out in the same order as in
-        // diff_recursively. The path of the values being diffed is kept in
-        // one buffer that grows and shrinks with the stack, rather than in a
-        // new string per level.
-        std::vector<diff_frame> stack;
-        string_t current_path = path;
-
-        // diff `s` against `t`, whose path is current_path: primitives,
-        // values of different types, and objects whose members were reordered
-        // are handled right away; arrays and other objects get a frame
-        const auto enter = [&result, &stack, &current_path](const basic_json & s, const basic_json & t)
-        {
-            // if the values are the same, there is nothing to do. Arrays and
-            // objects are not compared up front: comparing them visits
-            // everything below them, so doing that at every level would take
-            // quadratic time in the nesting depth - equal ones yield no
-            // operations anyway.
-            if ((!s.is_structured() || !t.is_structured()) && s == t)
-            {
-                return;
-            }
-
-            if (s.type() != t.type())
-            {
-                // different types: replace value
-                diff_replace(result, current_path, t);
-                return;
-            }
-
-            switch (s.type())
-            {
-                case value_t::array:
-                {
-                    stack.emplace_back(&s, &t, current_path.size());
-                    return;
-                }
-
-                case value_t::object:
-                {
-                    std::vector<typename object_t::key_type> common_keys;
-                    basic_json added_ops(value_t::array);
-                    if (diff_object_keys(result, s, t, current_path, common_keys, added_ops))
-                    {
-                        // fast path: the frame walks source in lockstep with
-                        // common_keys, as diff_recursively does, and appends
-                        // added_ops once all members are done
-                        stack.emplace_back(&s, &t, current_path.size());
-                        stack.back().member = s.cbegin();
-                        stack.back().common_keys = std::move(common_keys);
-                        stack.back().added_ops = std::move(added_ops);
-                    }
-                    return;
-                }
-
-                case value_t::null:
-                case value_t::string:
-                case value_t::boolean:
-                case value_t::number_integer:
-                case value_t::number_unsigned:
-                case value_t::number_float:
-                case value_t::binary:
-                case value_t::discarded:
-                default:
-                {
-                    // both primitive types: replace value
-                    diff_replace(result, current_path, t);
-                    return;
-                }
-            }
-        };
-
-        enter(source, target);
-        while (!stack.empty())
-        {
-            // the frame is copied out member by member and changed through
-            // stack.back(): enter() may push a frame and the end of the loop
-            // pops it, either of which would invalidate a reference to it
-            const basic_json* const s = stack.back().source;
-            const basic_json* const t = stack.back().target;
-            const std::size_t path_length = stack.back().path_length;
-            const std::size_t depth = stack.size();
-
-            if (s->is_array())
-            {
-                const auto& source_array = *s->m_data.m_value.array;
-                const auto& target_array = *t->m_data.m_value.array;
-
-                // first pass: traverse common elements
-                const std::size_t i = stack.back().index;
-                if (i < source_array.size() && i < target_array.size())
-                {
-                    ++stack.back().index;
-                    detail::concat_into(current_path, '/', detail::to_string<string_t>(i));
-                    enter(source_array[i], target_array[i]);
-                    if (stack.size() == depth)
-                    {
-                        current_path.resize(path_length);
-                    }
-                    continue;
-                }
-
-                // We now reached the end of at least one array
-                // in a second pass, traverse the remaining elements
-                diff_array_tails(result, *s, *t, current_path, i);
-            }
-            else
-            {
-                const const_iterator it = stack.back().member;
-                if (it != s->cend())
-                {
-                    ++stack.back().member;
-                    const std::size_t next_common = stack.back().next_common;
-                    if (next_common < stack.back().common_keys.size() && it.key() == stack.back().common_keys[next_common])
-                    {
-                        ++stack.back().next_common;
-                        const basic_json& target_value = (*t)[it.key()];
-                        detail::concat_into(current_path, '/', detail::escape(it.key()));
-                        enter(it.value(), target_value);
-                        if (stack.size() == depth)
-                        {
-                            current_path.resize(path_length);
-                        }
-                    }
-                    else
-                    {
-                        // found a key that is not in target -> remove it
-                        diff_remove(result, detail::concat<string_t>(current_path, '/', detail::escape(it.key())));
-                    }
-                    continue;
-                }
-
-                // append the "add" ops for brand-new keys collected when the
-                // object was entered
-                result.insert(result.end(), stack.back().added_ops.begin(), stack.back().added_ops.end());
-            }
-
-            // this array or object is done: continue with the one it is in
-            stack.pop_back();
-            if (!stack.empty())
-            {
-                current_path.resize(stack.back().path_length);
-            }
-        }
-    }
-
-  public:
     /// @}
 
     ////////////////////////////////

include/nlohmann/ordered_map.hpp

@@ -8,13 +8,15 @@
 
 #pragma once
 
+#include <algorithm> // max, min
 #include <functional> // equal_to, less
 #include <initializer_list> // initializer_list
 #include <iterator> // input_iterator_tag, iterator_traits
 #include <memory> // allocator
 #include <stdexcept> // for out_of_range
-#include <type_traits> // enable_if, is_convertible
-#include <utility> // pair
+#include <tuple> // forward_as_tuple
+#include <type_traits> // enable_if, integral_constant, is_convertible, is_nothrow_move_constructible
+#include <utility> // forward, move, pair, piecewise_construct
 #include <vector> // vector
 
 #include <nlohmann/detail/macro_scope.hpp>
@@ -79,7 +81,7 @@ template <class Key, class T, class IgnoredLess = std::less<Key>,
                 return {it, false};
             }
         }
-        Container::emplace_back(key, std::forward<T>(t));
+        append(key, std::forward<T>(t));
         return {std::prev(this->end()), true};
     }
 
@@ -94,7 +96,7 @@ template <class Key, class T, class IgnoredLess = std::less<Key>,
                 return {it, false};
             }
         }
-        Container::emplace_back(std::forward<KeyType>(key), std::forward<T>(t));
+        append(std::forward<KeyType>(key), std::forward<T>(t));
         return {std::prev(this->end()), true};
     }
 
@@ -368,7 +370,7 @@ template <class Key, class T, class IgnoredLess = std::less<Key>,
                 return {it, false};
             }
         }
-        Container::push_back(value);
+        append(value);
         return {--this->end(), true};
     }
 
@@ -386,6 +388,64 @@ template <class Key, class T, class IgnoredLess = std::less<Key>,
     }
 
 private:
+    /*!
+    @brief add an element whose key is not yet contained at the end
+
+    A std::vector copies all elements when it grows, because their const keys
+    make them not nothrow move constructible. For ordered_json, this is a deep
+    copy of every value. Where the strong exception guarantee can be kept, grow
+    the storage here instead, copying only the keys and moving the values.
+    */
+    template<typename... Args>
+    void append(Args&& ... args)
+    {
+        // evaluated here rather than at class scope, because T is still
+        // incomplete when basic_json instantiates its object_t
+        using move_values = std::integral_constant<bool, detail::conjunction<
+                detail::negation<std::is_nothrow_move_constructible<value_type>>,
+                std::is_copy_constructible<key_type>,
+                detail::is_default_constructible<mapped_type>,
+                std::is_nothrow_move_assignable<mapped_type>>::value>;
+        append_impl(move_values{}, std::forward<Args>(args)...);
+    }
+
+    template<typename... Args>
+    void append_impl(std::true_type /*unused*/, Args&& ... args)
+    {
+        if (this->size() < this->capacity())
+        {
+            Container::emplace_back(std::forward<Args>(args)...);
+            return;
+        }
+
+        // 1. May throw, but only changes tmp: copy the keys, value-initialize
+        //    the values, and add the new element. The arguments may refer to
+        //    elements of this container, so they are used before any value is
+        //    moved out of it.
+        Container tmp(this->get_allocator()); // equal allocators, so swap() is valid
+        tmp.reserve((std::min)(this->max_size(), (std::max)(size_type{1}, 2 * this->size())));
+        for (const auto& element : *this)
+        {
+            tmp.emplace_back(std::piecewise_construct, std::forward_as_tuple(element.first), std::forward_as_tuple());
+        }
+        tmp.emplace_back(std::forward<Args>(args)...);
+
+        // 2. Cannot throw: move the values over and adopt the new storage.
+        auto it = tmp.begin();
+        for (auto& element : *this)
+        {
+            it->second = std::move(element.second);
+            ++it;
+        }
+        Container::swap(tmp);
+    }
+
+    template<typename... Args>
+    void append_impl(std::false_type /*unused*/, Args&& ... args)
+    {
+        Container::emplace_back(std::forward<Args>(args)...);
+    }
+
     JSON_NO_UNIQUE_ADDRESS key_compare m_compare = key_compare();
 };
 

tests/benchmarks/src/benchmarks.cpp

@@ -119,6 +119,39 @@ BENCHMARK_CAPTURE(ParseIndented, canada / 4,       TEST_DATA_DIRECTORY "/nativej
 BENCHMARK_CAPTURE(ParseIndented, citm_catalog / 4, TEST_DATA_DIRECTORY "/nativejson-benchmark/citm_catalog.json", 4);
 BENCHMARK_CAPTURE(ParseIndented, twitter / 4,      TEST_DATA_DIRECTORY "/nativejson-benchmark/twitter.json",      4);
 
+//////////////////////////////////////////////////////////////////////////////
+// parse JSON from string into an ordered_json
+//
+// Same as ParseString above, but with nlohmann::ordered_json, whose objects
+// keep their members in a vector: the pair of rows shows what preserving the
+// insertion order costs.
+//////////////////////////////////////////////////////////////////////////////
+
+static void ParseStringOrdered(benchmark::State& state, const char* filename)
+{
+    std::ifstream f(filename);
+    std::string str((std::istreambuf_iterator<char>(f)), std::istreambuf_iterator<char>());
+
+    while (state.KeepRunning())
+    {
+        state.PauseTiming();
+        auto* j = new nlohmann::ordered_json();
+        state.ResumeTiming();
+
+        *j = nlohmann::ordered_json::parse(str);
+
+        state.PauseTiming();
+        delete j;
+        state.ResumeTiming();
+    }
+
+    state.SetBytesProcessed(state.iterations() * str.size());
+}
+BENCHMARK_CAPTURE(ParseStringOrdered, jeopardy,     TEST_DATA_DIRECTORY "/jeopardy/jeopardy.json");
+BENCHMARK_CAPTURE(ParseStringOrdered, canada,       TEST_DATA_DIRECTORY "/nativejson-benchmark/canada.json");
+BENCHMARK_CAPTURE(ParseStringOrdered, citm_catalog, TEST_DATA_DIRECTORY "/nativejson-benchmark/citm_catalog.json");
+BENCHMARK_CAPTURE(ParseStringOrdered, twitter,      TEST_DATA_DIRECTORY "/nativejson-benchmark/twitter.json");
+
 //////////////////////////////////////////////////////////////////////////////
 // serialize JSON
 //////////////////////////////////////////////////////////////////////////////

tests/src/unit-cbor.cpp

@@ -1830,10 +1830,51 @@ TEST_CASE("CBOR")
         SECTION("invalid string in map")
         {
             json _;
-            CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector<uint8_t>({0xa1, 0xff, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0xFF", json::parse_error&);
+            CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector<uint8_t>({0xa1, 0xff, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR object key: only string keys are supported, but found a break stop code; last byte: 0xFF", json::parse_error&);
             CHECK(json::from_cbor(std::vector<uint8_t>({0xa1, 0xff, 0x01}), true, false).is_discarded());
         }
 
+        SECTION("non-string key (see #2766 and #3381)")
+        {
+            // only text strings map to JSON object keys; any other key is
+            // rejected with a message naming its type
+            const std::vector<std::pair<std::vector<std::uint8_t>, std::string>> cases =
+            {
+                {{0xA1, 0x01, 0x01}, "an unsigned integer; last byte: 0x01"},
+                {{0xA1, 0x20, 0x01}, "a negative integer; last byte: 0x20"},
+                {{0xA1, 0x41, 0x61, 0x01}, "a byte string; last byte: 0x41"},
+                {{0xA1, 0x80, 0x01}, "an array; last byte: 0x80"},
+                {{0xA1, 0xA0, 0x01}, "a map; last byte: 0xA0"},
+                {{0xA1, 0xC0, 0x61, 0x61, 0x01}, "a tag; last byte: 0xC0"},
+                {{0xA1, 0xF4, 0x01}, "a boolean; last byte: 0xF4"},
+                {{0xA1, 0xF5, 0x01}, "a boolean; last byte: 0xF5"},
+                {{0xA1, 0xF6, 0x01}, "null; last byte: 0xF6"},
+                {{0xA1, 0xF7, 0x01}, "undefined; last byte: 0xF7"},
+                {{0xA1, 0xF9, 0x3C, 0x00, 0x01}, "a floating-point number; last byte: 0xF9"},
+                {{0xA1, 0xFA, 0x3F, 0x80, 0x00, 0x00, 0x01}, "a floating-point number; last byte: 0xFA"},
+                {{0xA1, 0xFB, 0x3F, 0xF0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01}, "a floating-point number; last byte: 0xFB"},
+                {{0xA1, 0xE0, 0x01}, "a simple value; last byte: 0xE0"},
+                {{0xA1, 0xF8, 0x20, 0x01}, "a simple value; last byte: 0xF8"},
+                // indefinite-length map
+                {{0xBF, 0x01, 0x01, 0xFF}, "an unsigned integer; last byte: 0x01"},
+            };
+
+            for (const auto& c : cases)
+            {
+                CAPTURE(c.first)
+                const std::string expected = "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR object key: only string keys are supported, but found " + c.second;
+                json _;
+                CHECK_THROWS_WITH_AS(_ = json::from_cbor(c.first), expected.c_str(), json::parse_error&);
+                CHECK(json::from_cbor(c.first, true, false).is_discarded());
+            }
+
+            // a key of major type 3 with a reserved length is still reported as
+            // a malformed string, and a missing key as the end of input
+            json _;
+            CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector<uint8_t>({0xA1})), "[json.exception.parse_error.110] parse error at byte 2: syntax error while parsing CBOR string: unexpected end of input", json::parse_error&);
+            CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector<uint8_t>({0xA1, 0x7C, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0x7C", json::parse_error&);
+        }
+
         SECTION("invalid UTF-8 in string (see #5529)")
         {
             // a two-character text string (major type 3) whose bytes are not
@@ -2284,7 +2325,7 @@ TEST_CASE("CBOR indefinite-length strings do not recurse per chunk")
     SECTION("a break marker outside an indefinite-length string is not a string")
     {
         // 0xFF only closes a string that was opened; on its own it is not one
-        CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector<uint8_t>({0xA1, 0xFF, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0xFF", json::parse_error&);
+        CHECK_THROWS_WITH_AS(_ = json::from_cbor(std::vector<uint8_t>({0xA1, 0xFF, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR object key: only string keys are supported, but found a break stop code; last byte: 0xFF", json::parse_error&);
     }
 }
 

tests/src/unit-class_lexer.cpp

@@ -666,7 +666,7 @@ TEST_CASE("parse_float_fast declines what it cannot convert exactly")
     // always safe: the caller then falls back to a slower, exact conversion.
     const auto fast = [](const std::string & s, double & out)
     {
-        return nlohmann::detail::parse_float_fast(s.data(), s.data() + s.size(), '.', out);
+        return nlohmann::detail::parse_float_fast(s.data(), s.data() + s.size(), out);
     };
     double out = 0;
 

tests/src/unit-disabled_exceptions.cpp

@@ -46,6 +46,24 @@ TEST_CASE("Tests with disabled exceptions")
         CHECK(*sax_no_exception::error_string == "[json.exception.parse_error.101] parse error at line 1, column 1: syntax error while parsing value - invalid literal; last read: 'x'");
         delete sax_no_exception::error_string; // NOLINT(cppcoreguidelines-owning-memory)
     }
+
+    SECTION("growing an ordered_json object")
+    {
+        auto j = nlohmann::ordered_json::object();
+        for (int i = 0; i < 100; ++i)
+        {
+            j[std::to_string(i)] = {{"nested", i}};
+        }
+
+        CHECK(j.size() == 100);
+        int i = 0;
+        for (const auto& element : j.items())
+        {
+            CHECK(element.key() == std::to_string(i));
+            CHECK(element.value()["nested"] == i);
+            ++i;
+        }
+    }
 }
 
 DOCTEST_GCC_SUPPRESS_WARNING_POP

tests/src/unit-json_patch.cpp

@@ -15,65 +15,8 @@ using nlohmann::json;
 #endif
 
 #include <fstream>
-#include <string>
-#include <vector>
 #include "make_test_data_available.hpp"
 
-namespace
-{
-// alternating objects and arrays nested `depth` levels deep, with members that
-// depend on `variant` at some levels, so diffing two variants yields
-// operations on many levels: replacing the innermost value, adding, removing,
-// and (for ordered_json) reordering members, and changing array lengths
-template<typename BasicJsonType>
-BasicJsonType nested(const std::size_t depth, const int variant)
-{
-    BasicJsonType value = variant;
-    for (std::size_t i = 0; i < depth; ++i)
-    {
-        if (i % 2 == 0)
-        {
-            BasicJsonType object = BasicJsonType::object();
-            if ((i + static_cast<std::size_t>(variant)) % 7 == 0)
-            {
-                object["x"] = i;
-            }
-            if (variant == 2 && i % 11 == 0)
-            {
-                object["z"] = "z";
-            }
-            object["a"] = std::move(value);
-            if (variant == 1 && i % 5 == 0)
-            {
-                object["y"] = 1;
-            }
-            value = std::move(object);
-        }
-        else
-        {
-            BasicJsonType array = BasicJsonType::array({std::move(value)});
-            if ((i + static_cast<std::size_t>(variant)) % 3 == 0)
-            {
-                array.push_back(i);
-            }
-            value = std::move(array);
-        }
-    }
-    return value;
-}
-
-// a path of `depth` reference tokens, as nested() nests its values
-std::string nested_path(const std::size_t depth)
-{
-    std::string path;
-    for (std::size_t i = depth; i > 0; --i)
-    {
-        path += (i - 1) % 2 == 0 ? "/a" : "/0";
-    }
-    return path;
-}
-} // namespace
-
 TEST_CASE("JSON patch")
 {
     SECTION("examples from RFC 6902")
@@ -1809,102 +1752,6 @@ TEST_CASE("JSON patch - diff emits array removals in descending index order")
     }
 }
 
-TEST_CASE("JSON patch: diff of deeply nested values")
-{
-    SECTION("the diff reproduces the target at every depth")
-    {
-        // depths on either side of the nesting depth up to which diff()
-        // recurses (detail::recursion_depth_limit(), 128); not every depth up
-        // to 300, as the test would then time out under Valgrind
-        std::vector<std::size_t> depths;
-        for (std::size_t depth = 0; depth <= 16; ++depth)
-        {
-            depths.push_back(depth);
-        }
-        for (std::size_t depth = 120; depth <= 136; ++depth)
-        {
-            depths.push_back(depth);
-        }
-        depths.push_back(300);
-
-        for (const auto depth : depths)
-        {
-            CAPTURE(depth);
-            for (int from = 0; from < 3; ++from)
-            {
-                for (int to = 0; to < 3; ++to)
-                {
-                    CAPTURE(from);
-                    CAPTURE(to);
-                    const auto source = nested<json>(depth, from);
-                    const auto target = nested<json>(depth, to);
-                    const auto patch = json::diff(source, target);
-                    CHECK(source.patch(patch) == target);
-                    CHECK(patch.empty() == (from == to));
-
-                    const auto ordered_source = nested<nlohmann::ordered_json>(depth, from);
-                    const auto ordered_target = nested<nlohmann::ordered_json>(depth, to);
-                    CHECK(ordered_source.patch(nlohmann::ordered_json::diff(ordered_source, ordered_target)) == ordered_target);
-                }
-            }
-        }
-    }
-
-    SECTION("a difference only in the innermost value is one replace operation")
-    {
-        for (std::size_t depth = 0; depth <= 300; ++depth)
-        {
-            CAPTURE(depth);
-            json source = 1;
-            json target = 2;
-            for (std::size_t i = 0; i < depth; ++i)
-            {
-                source = i % 2 == 0 ? json::object({{"a", std::move(source)}}) : json::array({std::move(source)});
-                target = i % 2 == 0 ? json::object({{"a", std::move(target)}}) : json::array({std::move(target)});
-            }
-            CHECK(json::diff(source, target, "/root") == json::array({{{"op", "replace"}, {"path", "/root" + nested_path(depth)}, {"value", 2}}}));
-        }
-    }
-
-    SECTION("values nested too deeply for the call stack (#5393)")
-    {
-        // diff() used to recurse once per nesting level, and compared the
-        // values with operator== on every level. The values are only
-        // parsed and diffed, never copied or compared, since those recurse
-        // too.
-        const std::size_t depth = 100000;
-        for (const bool objects :
-                {
-                    false, true
-                })
-        {
-            CAPTURE(objects);
-            std::string source_text;
-            std::string target_text;
-            std::string equal_text;
-            std::string path;
-            for (std::size_t i = 0; i < depth; ++i)
-            {
-                source_text += objects ? "{\"a\":" : "[";
-                path += objects ? "/a" : "/0";
-            }
-            target_text = source_text + "2";
-            equal_text = source_text + "1";
-            source_text += "1";
-            const std::string closing(depth, objects ? '}' : ']');
-            const auto source = json::parse(source_text + closing);
-
-            const auto patch = json::diff(source, json::parse(target_text + closing));
-            REQUIRE(patch.size() == 1);
-            CHECK(patch[0]["op"] == "replace");
-            CHECK(patch[0]["path"] == path);
-            CHECK(patch[0]["value"] == 2);
-
-            CHECK(json::diff(source, json::parse(equal_text + closing)).empty());
-        }
-    }
-}
-
 TEST_CASE("JSON patch - every operation on ordered_json")
 {
     using nlohmann::ordered_json;

tests/src/unit-locale-cpp.cpp

@@ -12,7 +12,12 @@
 #include <nlohmann/json.hpp>
 using nlohmann::json;
 
+#include <array>
 #include <clocale>
+#include <map>
+#include <string>
+#include <utility>
+#include <vector>
 
 struct ParserImpl final: public nlohmann::json_sax<json>
 {
@@ -175,3 +180,208 @@ TEST_CASE("locale-dependent test (LC_NUMERIC=de_DE)")
         MESSAGE("locale de_DE is not usable");
     }
 }
+
+namespace
+{
+// records the numbers of a flat array and switches LC_NUMERIC to the given
+// locale once the array opens - after the lexer was constructed, but before
+// any number in the array is lexed
+struct LocaleSwitchingSax final: public nlohmann::json_sax<json>
+{
+    explicit LocaleSwitchingSax(const char* switch_to)
+        : locale_after_open(switch_to)
+    {}
+
+    bool null() override
+    {
+        return true;
+    }
+    bool boolean(bool /*val*/) override
+    {
+        return true;
+    }
+    bool number_integer(json::number_integer_t /*val*/) override
+    {
+        return true;
+    }
+    bool number_unsigned(json::number_unsigned_t /*val*/) override
+    {
+        return true;
+    }
+    bool number_float(json::number_float_t val, const json::string_t& s) override
+    {
+        values.push_back(val);
+        strings.push_back(s);
+        return true;
+    }
+    bool string(json::string_t& /*val*/) override
+    {
+        return true;
+    }
+    bool binary(json::binary_t& /*val*/) override
+    {
+        return true;
+    }
+    bool start_object(std::size_t /*val*/) override
+    {
+        return true;
+    }
+    bool key(json::string_t& /*val*/) override
+    {
+        return true;
+    }
+    bool end_object() override
+    {
+        return true;
+    }
+    bool start_array(std::size_t /*val*/) override
+    {
+        switched = std::setlocale(LC_NUMERIC, locale_after_open.c_str()) != nullptr;
+        return true;
+    }
+    bool end_array() override
+    {
+        return true;
+    }
+    bool parse_error(std::size_t /*val*/, const std::string& /*val*/, const nlohmann::detail::exception& /*val*/) override
+    {
+        return false;
+    }
+
+    std::string locale_after_open;
+    bool switched = false;
+    std::vector<json::number_float_t> values {}; // NOLINT(readability-redundant-member-init)
+    std::vector<json::string_t> strings {}; // NOLINT(readability-redundant-member-init)
+};
+} // namespace
+
+TEST_CASE("locale changes between lexer construction and number conversion (#5198)")
+{
+    // The numbers are chosen so that the conversion also takes the strtod
+    // fallback, which honors the locale that is current at conversion time:
+    // too many significant digits for Clinger's fast path, an underflow that
+    // std::from_chars rejects, and a plain value.
+    const std::vector<std::string> numbers = {"3.14159265358979323846", "1.5e-400", "12.34", "-0.000123456789012345678"};
+    std::string text = "[";
+    for (const auto& n : numbers)
+    {
+        text += (text.size() == 1 ? "" : ",") + n;
+    }
+    text += "]";
+
+    using long_double_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int64_t, std::uint64_t, long double>;
+
+    // reference values, parsed without a locale switch
+    REQUIRE(std::setlocale(LC_NUMERIC, "C") != nullptr);
+    const json expected = json::parse(text);
+    const long_double_json expected_ld = long_double_json::parse(text);
+
+    const std::array<std::pair<const char*, const char*>, 2> transitions =
+    {
+        {
+            {"C", "de_DE"},
+            {"de_DE", "C"}
+        }
+    };
+
+    for (const auto& transition : transitions)
+    {
+        CAPTURE(transition.first);
+        CAPTURE(transition.second);
+
+        if (std::setlocale(LC_NUMERIC, transition.first) == nullptr)
+        {
+            MESSAGE("locale is not usable");
+            continue;
+        }
+
+        // SAX parsing
+        {
+            LocaleSwitchingSax sax(transition.second);
+            CHECK(json::sax_parse(text, &sax));
+            if (sax.switched)
+            {
+                CHECK(sax.values == expected.get<std::vector<json::number_float_t>>());
+                CHECK(sax.strings == numbers);
+            }
+        }
+
+        // DOM parsing with a callback
+        {
+            bool switched = false;
+            const auto cb = [&](int /*depth*/, json::parse_event_t event, json& /*parsed*/)
+            {
+                if (event == json::parse_event_t::array_start)
+                {
+                    switched = std::setlocale(LC_NUMERIC, transition.second) != nullptr;
+                }
+                return true;
+            };
+            const json j = json::parse(text, cb);
+            if (switched)
+            {
+                CHECK(j == expected);
+            }
+        }
+
+        // a long double goes through std::strtold unless std::from_chars supports it
+        {
+            bool switched = false;
+            const auto cb = [&](int /*depth*/, long_double_json::parse_event_t event, long_double_json& /*parsed*/)
+            {
+                if (event == long_double_json::parse_event_t::array_start)
+                {
+                    switched = std::setlocale(LC_NUMERIC, transition.second) != nullptr;
+                }
+                return true;
+            };
+            const long_double_json j = long_double_json::parse(text, cb);
+            if (switched)
+            {
+                CHECK(j == expected_ld);
+            }
+        }
+    }
+
+    std::setlocale(LC_NUMERIC, "C");
+}
+
+TEST_CASE("locale with a multi-byte decimal point")
+{
+    // Some locales use a decimal point that is not a single character, e.g.
+    // U+066B ARABIC DECIMAL SEPARATOR (two bytes in UTF-8). It cannot be
+    // substituted in place for '.', so the strtod fallback stops early. The
+    // conversion must still terminate rather than retry forever.
+    const std::array<const char*, 6> names = {{"ar_EG.UTF-8", "ar_SA.UTF-8", "fa_IR.UTF-8", "ps_AF.UTF-8", "ar_EG", "fa_IR"}};
+    bool tested = false;
+    for (const char* name : names)
+    {
+        if (std::setlocale(LC_NUMERIC, name) == nullptr)
+        {
+            continue;
+        }
+        const std::string decimal_point = std::localeconv()->decimal_point;
+        if (decimal_point.size() < 2)
+        {
+            continue;
+        }
+        CAPTURE(name);
+        tested = true;
+
+        // too many significant digits for Clinger's fast path, and an underflow
+        // that std::from_chars rejects: both reach the strtod fallback
+        json j;
+        CHECK_NOTHROW(j = json::parse("[3.14159265358979323846, 1.5e-400, -0.000123456789012345678]"));
+        CHECK(j.is_array());
+        CHECK(json::accept("3.14159265358979323846"));
+
+        // a value the locale-independent paths convert is not affected
+        CHECK(json::parse("12.5") == 12.5);
+    }
+    if (!tested)
+    {
+        MESSAGE("no locale with a multi-byte decimal point is usable");
+    }
+
+    std::setlocale(LC_NUMERIC, "C");
+}

tests/src/unit-msgpack.cpp

@@ -1551,10 +1551,69 @@ TEST_CASE("MessagePack")
         SECTION("invalid string in map")
         {
             json _;
-            CHECK_THROWS_WITH_AS(_ = json::from_msgpack(std::vector<uint8_t>({0x81, 0xff, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack string: expected length specification (0xA0-0xBF, 0xD9-0xDB); last byte: 0xFF", json::parse_error&);
+            CHECK_THROWS_WITH_AS(_ = json::from_msgpack(std::vector<uint8_t>({0x81, 0xff, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack object key: only string keys are supported, but found an integer; last byte: 0xFF", json::parse_error&);
             CHECK(json::from_msgpack(std::vector<uint8_t>({0x81, 0xff, 0x01}), true, false).is_discarded());
         }
 
+        SECTION("non-string key (see #3381)")
+        {
+            // only strings map to JSON object keys; any other key is rejected
+            // with a message naming its type
+            const std::vector<std::pair<std::vector<std::uint8_t>, std::string>> cases =
+            {
+                {{0x81, 0xC0, 0x01}, "nil; last byte: 0xC0"},
+                {{0x81, 0xC2, 0x01}, "a boolean; last byte: 0xC2"},
+                {{0x81, 0xC3, 0x01}, "a boolean; last byte: 0xC3"},
+                {{0x81, 0xCA, 0x3F, 0x80, 0x00, 0x00, 0x01}, "a float; last byte: 0xCA"},
+                {{0x81, 0xCB, 0x3F, 0xF0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01}, "a float; last byte: 0xCB"},
+                {{0x81, 0xC4, 0x00, 0x01}, "a bin; last byte: 0xC4"},
+                {{0x81, 0xC5, 0x00, 0x00, 0x01}, "a bin; last byte: 0xC5"},
+                {{0x81, 0xC6, 0x00, 0x00, 0x00, 0x00, 0x01}, "a bin; last byte: 0xC6"},
+                {{0x81, 0xC7, 0x00, 0x01, 0x01}, "an ext; last byte: 0xC7"},
+                {{0x81, 0xC8, 0x00, 0x00, 0x01, 0x01}, "an ext; last byte: 0xC8"},
+                {{0x81, 0xC9, 0x00, 0x00, 0x00, 0x00, 0x01, 0x01}, "an ext; last byte: 0xC9"},
+                {{0x81, 0xD4, 0x01, 0x00, 0x01}, "an ext; last byte: 0xD4"},
+                {{0x81, 0xD5, 0x01, 0x00, 0x00, 0x01}, "an ext; last byte: 0xD5"},
+                {{0x81, 0xD6, 0x01, 0x00, 0x00, 0x00, 0x00, 0x01}, "an ext; last byte: 0xD6"},
+                {{0x81, 0xD7, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01}, "an ext; last byte: 0xD7"},
+                {{0x81, 0xD8, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01}, "an ext; last byte: 0xD8"},
+                {{0x81, 0xCC, 0x01, 0x01}, "an integer; last byte: 0xCC"},
+                {{0x81, 0xCD, 0x00, 0x01, 0x01}, "an integer; last byte: 0xCD"},
+                {{0x81, 0xCE, 0x00, 0x00, 0x00, 0x01, 0x01}, "an integer; last byte: 0xCE"},
+                {{0x81, 0xCF, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x01}, "an integer; last byte: 0xCF"},
+                {{0x81, 0xD0, 0x01, 0x01}, "an integer; last byte: 0xD0"},
+                {{0x81, 0xD1, 0x00, 0x01, 0x01}, "an integer; last byte: 0xD1"},
+                {{0x81, 0xD2, 0x00, 0x00, 0x00, 0x01, 0x01}, "an integer; last byte: 0xD2"},
+                {{0x81, 0xD3, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x01}, "an integer; last byte: 0xD3"},
+                {{0x81, 0x00, 0x01}, "an integer; last byte: 0x00"},
+                {{0x81, 0x7F, 0x01}, "an integer; last byte: 0x7F"},
+                {{0x81, 0xE0, 0x01}, "an integer; last byte: 0xE0"},
+                {{0x81, 0x80, 0x01}, "a map; last byte: 0x80"},
+                {{0x81, 0x8F, 0x01}, "a map; last byte: 0x8F"},
+                {{0x81, 0xDE, 0x00, 0x00, 0x01}, "a map; last byte: 0xDE"},
+                {{0x81, 0xDF, 0x00, 0x00, 0x00, 0x00, 0x01}, "a map; last byte: 0xDF"},
+                {{0x81, 0x90, 0x01}, "an array; last byte: 0x90"},
+                {{0x81, 0x9F, 0x01}, "an array; last byte: 0x9F"},
+                {{0x81, 0xDC, 0x00, 0x00, 0x01}, "an array; last byte: 0xDC"},
+                {{0x81, 0xDD, 0x00, 0x00, 0x00, 0x00, 0x01}, "an array; last byte: 0xDD"},
+            };
+
+            for (const auto& c : cases)
+            {
+                CAPTURE(c.first)
+                const std::string expected = "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack object key: only string keys are supported, but found " + c.second;
+                json _;
+                CHECK_THROWS_WITH_AS(_ = json::from_msgpack(c.first), expected.c_str(), json::parse_error&);
+                CHECK(json::from_msgpack(c.first, true, false).is_discarded());
+            }
+
+            json _;
+            // the unused byte 0xC1 is still reported as a malformed string
+            CHECK_THROWS_WITH_AS(_ = json::from_msgpack(std::vector<uint8_t>({0x81, 0xC1, 0x01})), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing MessagePack string: expected length specification (0xA0-0xBF, 0xD9-0xDB); last byte: 0xC1", json::parse_error&);
+            // a missing key is still reported as the end of input
+            CHECK_THROWS_WITH_AS(_ = json::from_msgpack(std::vector<uint8_t>({0x81})), "[json.exception.parse_error.110] parse error at byte 2: syntax error while parsing MessagePack string: unexpected end of input", json::parse_error&);
+        }
+
         SECTION("invalid UTF-8 in string (see #5529)")
         {
             // a fixstr of length 2 (0xA0 | 2) whose bytes are not valid UTF-8

tests/src/unit-ordered_map.cpp

@@ -11,6 +11,87 @@
 #include <nlohmann/json.hpp>
 using nlohmann::ordered_map;
 
+#include <stdexcept>
+#include <string>
+#include <type_traits>
+#include <utility>
+#include <vector>
+
+namespace
+{
+// number of copies made of counted values
+int value_copies = 0;
+
+// a mapped type that counts its copies; moving from it leaves -1 behind
+struct counted // NOLINT(cppcoreguidelines-special-member-functions,hicpp-special-member-functions)
+{
+    int payload = 0;
+
+    counted() = default;
+    explicit counted(int p) noexcept : payload(p) {}
+    counted(const counted& other) : payload(other.payload)
+    {
+        ++value_copies;
+    }
+    counted(counted&& other) noexcept : payload(other.payload)
+    {
+        other.payload = -1;
+    }
+    counted& operator=(const counted&) = delete;
+    counted& operator=(counted&& other) noexcept
+    {
+        payload = other.payload;
+        other.payload = -1;
+        return *this;
+    }
+};
+
+#if !defined(JSON_NOEXCEPTION)
+// number of throwing_key copies that still succeed; the next one throws
+// (a negative value means that copies never throw)
+int key_copies_until_throw = -1;
+
+// a key type whose copy constructor can be made to throw
+struct throwing_key // NOLINT(cppcoreguidelines-special-member-functions,hicpp-special-member-functions)
+{
+    int id = 0;
+
+    explicit throwing_key(int i) noexcept : id(i) {}
+    throwing_key(const throwing_key& other) : id(other.id)
+    {
+        if (key_copies_until_throw == 0)
+        {
+            throw std::runtime_error("key copy failed");
+        }
+        if (key_copies_until_throw > 0)
+        {
+            --key_copies_until_throw;
+        }
+    }
+    throwing_key& operator=(const throwing_key&) = delete;
+
+    friend bool operator==(const throwing_key& lhs, const throwing_key& rhs) noexcept
+    {
+        return lhs.id == rhs.id;
+    }
+};
+#endif
+
+// a mapped type that cannot be default-constructed
+struct no_default
+{
+    explicit no_default(int v) noexcept : value(v) {}
+    int value;
+};
+
+// ordered_json must keep moving its values when an object grows
+using ordered_object_t = nlohmann::ordered_json::object_t;
+static_assert(!std::is_nothrow_move_constructible<ordered_object_t::value_type>::value, "std::vector would move the elements itself");
+static_assert(std::is_copy_constructible<ordered_object_t::key_type>::value, "keys must be copyable");
+static_assert(std::is_default_constructible<ordered_object_t::mapped_type>::value, "values must be default-constructible");
+static_assert(std::is_nothrow_move_assignable<ordered_object_t::mapped_type>::value, "values must be nothrow move-assignable");
+} // namespace
+
 TEST_CASE("ordered_map")
 {
     SECTION("constructor")
@@ -313,3 +394,270 @@ TEST_CASE("ordered_map")
         }
     }
 }
+
+TEST_CASE("ordered_map growth")
+{
+    SECTION("values are moved, not copied, when the storage grows")
+    {
+        ordered_map<std::string, counted> om;
+        std::size_t growths = 0;
+        value_copies = 0;
+
+        // inserts 100 elements with the given function and counts the growths
+        const auto fill = [&om, &growths](void (*insert)(ordered_map<std::string, counted>&, int))
+        {
+            for (int i = 0; i < 100; ++i)
+            {
+                const auto old_capacity = om.capacity();
+                insert(om, i);
+                if (om.capacity() > old_capacity)
+                {
+                    ++growths;
+                }
+            }
+        };
+
+        // checks that the elements are in insertion order with their values
+        const auto check_contents = [&om]
+        {
+            CHECK(om.size() == 100);
+            int i = 0;
+            for (const auto& element : om)
+            {
+                CHECK(element.first == std::to_string(i));
+                CHECK(element.second.payload == i);
+                ++i;
+            }
+        };
+
+        SECTION("emplace")
+        {
+            fill([](ordered_map<std::string, counted>& m, int i)
+            {
+                m.emplace(std::to_string(i), counted(i));
+            });
+            CHECK(growths >= 3);
+            CHECK(value_copies == 0);
+            check_contents();
+        }
+
+        SECTION("operator[]")
+        {
+            fill([](ordered_map<std::string, counted>& m, int i)
+            {
+                m[std::to_string(i)] = counted(i);
+            });
+            CHECK(growths >= 3);
+            CHECK(value_copies == 0);
+            check_contents();
+        }
+
+        SECTION("insert(value_type&&)")
+        {
+            fill([](ordered_map<std::string, counted>& m, int i)
+            {
+                m.insert({std::to_string(i), counted(i)});
+            });
+            CHECK(growths >= 3);
+            CHECK(value_copies == 0);
+            check_contents();
+        }
+
+        SECTION("insert(const value_type&)")
+        {
+            fill([](ordered_map<std::string, counted>& m, int i)
+            {
+                const std::pair<const std::string, counted> value(std::to_string(i), counted(i));
+                m.insert(value);
+            });
+            CHECK(growths >= 3);
+            // only the inserted values are copied
+            CHECK(value_copies == 100);
+            check_contents();
+        }
+
+        SECTION("insert(first, last)")
+        {
+            std::vector<std::pair<const std::string, counted>> values;
+            values.reserve(100);
+            for (int i = 0; i < 100; ++i)
+            {
+                values.emplace_back(std::to_string(i), counted(i));
+            }
+            value_copies = 0;
+
+            om.insert(values.cbegin(), values.cend());
+            // only the inserted values are copied
+            CHECK(value_copies == 100);
+            check_contents();
+        }
+    }
+
+    SECTION("elements keep their order and values over many growths")
+    {
+        ordered_map<std::string, counted> om;
+        for (int i = 0; i < 1000; ++i)
+        {
+            om.emplace(std::to_string(i), counted(i));
+        }
+
+        CHECK(om.size() == 1000);
+        int i = 0;
+        for (const auto& element : om)
+        {
+            CHECK(element.first == std::to_string(i));
+            CHECK(element.second.payload == i);
+            ++i;
+        }
+    }
+
+    SECTION("arguments may refer to elements of the full container")
+    {
+        SECTION("moving a value out of the container")
+        {
+            ordered_map<std::string, counted> om;
+            om.reserve(4);
+            while (om.size() < om.capacity())
+            {
+                const auto i = static_cast<int>(om.size());
+                om.emplace(std::to_string(i), counted(i));
+            }
+            const auto size = om.size();
+
+            om.emplace("new", std::move(om.at("0")));
+            CHECK(om.size() == size + 1);
+            CHECK(om.at("new").payload == 0);
+            CHECK(om.at("0").payload == -1);
+        }
+
+        SECTION("using a value as key")
+        {
+            ordered_map<std::string, std::string> om;
+            om.reserve(4);
+            while (om.size() < om.capacity())
+            {
+                const auto i = std::to_string(om.size());
+                om.emplace("k" + i, "v" + i);
+            }
+            const auto size = om.size();
+
+            om.emplace(om.at("k0"), std::string("x"));
+            CHECK(om.size() == size + 1);
+            CHECK(om.at("k0") == "v0");
+            CHECK(om.at("v0") == "x");
+        }
+
+        SECTION("ordered_json")
+        {
+            auto j = nlohmann::ordered_json::object();
+            auto& object = j.get_ref<nlohmann::ordered_json::object_t&>();
+            object.reserve(4);
+            while (object.size() < object.capacity())
+            {
+                const auto i = std::to_string(object.size());
+                j[i] = "a value that is too long for the small string optimization " + i;
+            }
+            const auto size = j.size();
+
+            j.emplace("new", std::move(j["0"]));
+            CHECK(j.size() == size + 1);
+            CHECK(j["new"] == "a value that is too long for the small string optimization 0");
+            CHECK(j["0"].is_null());
+        }
+    }
+
+#if !defined(JSON_NOEXCEPTION)
+    SECTION("the container is unchanged if growing it throws")
+    {
+        ordered_map<throwing_key, counted> om;
+        om.reserve(4);
+        while (om.size() < om.capacity())
+        {
+            const auto i = static_cast<int>(om.size());
+            om.emplace(throwing_key(i), counted(i));
+        }
+        const auto size = om.size();
+        const auto capacity = om.capacity();
+
+        // checks that the elements are unchanged
+        const auto check_unchanged = [&om, size, capacity]
+        {
+            CHECK(om.size() == size);
+            CHECK(om.capacity() == capacity);
+            int i = 0;
+            for (const auto& element : om)
+            {
+                CHECK(element.first.id == i);
+                CHECK(element.second.payload == i);
+                ++i;
+            }
+        };
+
+        SECTION("emplace")
+        {
+            // growing copies the existing keys and then the new one; let each of these copies throw
+            for (std::size_t k = 0; k <= size; ++k)
+            {
+                counted value(100);
+                key_copies_until_throw = static_cast<int>(k);
+                CHECK_THROWS_AS(om.emplace(throwing_key(100), std::move(value)), std::runtime_error);
+                key_copies_until_throw = -1;
+
+                check_unchanged();
+                CHECK(value.payload == 100); // NOLINT(bugprone-use-after-move,hicpp-invalid-access-moved)
+            }
+
+            om.emplace(throwing_key(100), counted(100));
+            CHECK(om.size() == size + 1);
+            CHECK(om.capacity() > capacity);
+            CHECK(om.at(throwing_key(100)).payload == 100);
+        }
+
+        SECTION("insert(const value_type&)")
+        {
+            const std::pair<const throwing_key, counted> value(throwing_key(100), counted(100));
+            value_copies = 0;
+
+            key_copies_until_throw = static_cast<int>(size / 2);
+            CHECK_THROWS_AS(om.insert(value), std::runtime_error);
+            key_copies_until_throw = -1;
+
+            check_unchanged();
+            CHECK(value_copies == 0);
+        }
+    }
+#endif
+
+    SECTION("elements that std::vector moves, or that cannot be moved back")
+    {
+        SECTION("nothrow move-constructible elements")
+        {
+            ordered_map<int, counted> om;
+            value_copies = 0;
+            for (int i = 0; i < 100; ++i)
+            {
+                om.emplace(i, counted(i));
+            }
+            CHECK(om.size() == 100);
+            CHECK(value_copies == 0);
+        }
+
+        SECTION("mapped type without default constructor")
+        {
+            ordered_map<std::string, no_default> om;
+            for (int i = 0; i < 100; ++i)
+            {
+                om.emplace(std::to_string(i), no_default(i));
+            }
+
+            CHECK(om.size() == 100);
+            int i = 0;
+            for (const auto& element : om)
+            {
+                CHECK(element.first == std::to_string(i));
+                CHECK(element.second.value == i);
+                ++i;
+            }
+        }
+    }
+}

tests/src/unit-regression1.cpp

@@ -1018,7 +1018,7 @@ TEST_CASE("regression tests 1")
         };
 
         json _;
-        CHECK_THROWS_WITH_AS(_ = json::from_cbor(vec), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0x98", json::parse_error&);
+        CHECK_THROWS_WITH_AS(_ = json::from_cbor(vec), "[json.exception.parse_error.113] parse error at byte 2: syntax error while parsing CBOR object key: only string keys are supported, but found an array; last byte: 0x98", json::parse_error&);
 
         // related test case: nonempty UTF-8 string (indefinite length)
         std::vector<uint8_t> const vec1 {0x7f, 0x61, 0x61};
@@ -1065,7 +1065,7 @@ TEST_CASE("regression tests 1")
         };
 
         json _;
-        CHECK_THROWS_WITH_AS(_ = json::from_cbor(vec1), "[json.exception.parse_error.113] parse error at byte 13: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0xB4", json::parse_error&);
+        CHECK_THROWS_WITH_AS(_ = json::from_cbor(vec1), "[json.exception.parse_error.113] parse error at byte 13: syntax error while parsing CBOR object key: only string keys are supported, but found a map; last byte: 0xB4", json::parse_error&);
 
         // related test case: double-precision
         std::vector<uint8_t> const vec2
@@ -1077,7 +1077,7 @@ TEST_CASE("regression tests 1")
             0x96, 0x96, 0xb4, 0xb4, 0xfa, 0x94, 0x94, 0x61,
             0x61, 0x61, 0x61, 0x61, 0x61, 0x61, 0x61, 0xfb
         };
-        CHECK_THROWS_WITH_AS(_ = json::from_cbor(vec2), "[json.exception.parse_error.113] parse error at byte 13: syntax error while parsing CBOR string: expected length specification (0x60-0x7B) or indefinite string type (0x7F); last byte: 0xB4", json::parse_error&);
+        CHECK_THROWS_WITH_AS(_ = json::from_cbor(vec2), "[json.exception.parse_error.113] parse error at byte 13: syntax error while parsing CBOR object key: only string keys are supported, but found a map; last byte: 0xB4", json::parse_error&);
     }
 
     SECTION("issue #452 - Heap-buffer-overflow (OSS-Fuzz issue 585)")