Handle numbers that do not fit narrow number types in the binary readers

With custom number types narrower than the values in a binary document,
for example basic_json<..., std::int32_t, std::uint32_t, float>, every
binary reader (CBOR, MessagePack, UBJSON, BJData, BSON, BON8) passed the
decoded number to the SAX interface with an implicit conversion: the
integer 5000000000 silently became 705032704, and a finite double such as
1e300 became infinity. The lexer handles the same values in JSON text: an
integer that fits neither integer type is stored as number_float_t, and a
finite number that overflows number_float_t is rejected with
out_of_range.406.

Pass every number read from binary input through three helpers that
apply the lexer's rules:
- emit_signed(): number_integer_t, else number_unsigned_t for a
  non-negative value, else number_float_t
- emit_unsigned(): number_unsigned_t, else number_float_t
- emit_float(): out_of_range.406 if a finite value overflows
  number_float_t; infinity and NaN are passed on

For consistency, a CBOR negative integer below the range of
number_integer_t is now stored as number_float_t, like a too small
integer in JSON text, instead of being rejected with parse_error.112.
With the default number types, this is the only change in behavior.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

docs/mkdocs/docs/api/basic_json/number_float_t.md

@@ -55,6 +55,10 @@ This implementation does exactly follow this approach, as it uses double precisi
 smaller than `-1.79769313486232e+308` and values greater than `1.79769313486232e+308` will be stored as NaN internally
 and be serialized to `null`.
 
+During deserialization (from JSON text or any of the binary formats), a finite number that does not fit into
+`number_float_t` is rejected with [`out_of_range.406`](../../home/exceptions.md#jsonexceptionout_of_range406), for
+example a double-precision number in a binary format when `number_float_t` is `#!cpp float`.
+
 #### Storage
 
 Floating-point number values are stored directly inside a `basic_json` type.

docs/mkdocs/docs/api/basic_json/number_integer_t.md

@@ -47,8 +47,9 @@ With the default values for `NumberIntegerType` (`std::int64_t`), the default va
 
 When the default type is used, the maximal integer number that can be stored is `9223372036854775807` (INT64_MAX) and
 the minimal integer number that can be stored is `-9223372036854775808` (INT64_MIN). Integer numbers that are out of
-range will yield over/underflow when used in a constructor. During deserialization, too large or small integer numbers
-will automatically be stored as [`number_unsigned_t`](number_unsigned_t.md) or [`number_float_t`](number_float_t.md).
+range will yield over/underflow when used in a constructor. During deserialization (from JSON text or any of the binary
+formats), too large or small integer numbers will automatically be stored as [`number_unsigned_t`](number_unsigned_t.md)
+or [`number_float_t`](number_float_t.md).
 
 [RFC 8259](https://tools.ietf.org/html/rfc8259) further states:
 > Note that when such software is used, numbers that are integers and are in the range $[-2^{53}+1, 2^{53}-1]$ are

docs/mkdocs/docs/api/basic_json/number_unsigned_t.md

@@ -48,8 +48,9 @@ With the default values for `NumberUnsignedType` (`std::uint64_t`), the default
 
 When the default type is used, the maximal integer number that can be stored is `18446744073709551615` (UINT64_MAX) and
 the minimal integer number that can be stored is `0`. Integer numbers that are out of range will yield over/underflow
-when used in a constructor. During deserialization, too large or small integer numbers will automatically be stored
-as [`number_integer_t`](number_integer_t.md) or [`number_float_t`](number_float_t.md).
+when used in a constructor. During deserialization (from JSON text or any of the binary formats), too large or small
+integer numbers will automatically be stored as [`number_integer_t`](number_integer_t.md) or
+[`number_float_t`](number_float_t.md).
 
 [RFC 8259](https://tools.ietf.org/html/rfc8259) further states:
 > Note that when such software is used, numbers that are integers and are in the range $[-2^{53}+1, 2^{53}-1]$ are

docs/mkdocs/docs/features/binary_formats/cbor.md

@@ -168,9 +168,9 @@ The library maps CBOR types to JSON value types as follows:
 !!! warning "Negative integer overflow"
 
     CBOR negative integers (major type 1) are decoded as `-1 - n`. If the encoded magnitude `n` is too large for the
-    result to fit into `number_integer_t` (`std::int64_t` by default), parsing fails with a
-    [`parse_error.112`](../../home/exceptions.md#jsonexceptionparse_error112) exception rather than overflowing
-    silently.
+    result to fit into `number_integer_t` (`std::int64_t` by default), the result is stored as `number_float_t`, like
+    a too small integer in JSON text. For example, `-18446744073709551616` (`0x3B` followed by eight `0xFF` bytes) is
+    stored as `-1.8446744073709552e+19`.
 
 !!! warning "Object keys"
 

docs/mkdocs/docs/home/exceptions.md

@@ -331,9 +331,6 @@ An unexpected byte was read in a [binary format](../features/binary_formats/inde
     [json.exception.parse_error.112] parse error at byte 15: syntax error while parsing BSON binary: byte array length cannot be negative, is -1
     ```
     ```
-    [json.exception.parse_error.112] parse error at byte 9: syntax error while parsing CBOR value: negative integer overflow
-    ```
-    ```
     [json.exception.parse_error.112] parse error at byte 5: syntax error while parsing BSON document: document size 6 does not match the number of bytes read (5)
     ```
 
@@ -847,13 +844,18 @@ The JSON Patch operations 'remove' and 'add' cannot be applied to the root eleme
 
 ### json.exception.out_of_range.406
 
-A parsed number could not be stored as without changing it to NaN or INF.
+A parsed number could not be stored without changing it to NaN or INF. For the binary formats, this happens when a
+finite floating-point number does not fit into [`number_float_t`](../api/basic_json/number_float_t.md), for example a
+double-precision number when `number_float_t` is `#!cpp float`.
 
-!!! failure "Example message"
+!!! failure "Example messages"
 
     ```
     number overflow parsing '10E1000'
     ```
+    ```
+    [json.exception.out_of_range.406] syntax error while parsing CBOR value: number overflow
+    ```
 
 ### json.exception.out_of_range.407
 

include/nlohmann/detail/input/binary_reader.hpp

@@ -559,7 +559,7 @@ class binary_reader
             case 0x01: // double
             {
                 double number{};
-                return get_number<double, true>(input_format_t::bson, number) && sax->number_float(static_cast<number_float_t>(number), "");
+                return get_number<double, true>(input_format_t::bson, number) && emit_float(input_format_t::bson, number);
             }
 
             case 0x02: // string
@@ -600,19 +600,19 @@ class binary_reader
             case 0x10: // int32
             {
                 std::int32_t value{};
-                return get_number<std::int32_t, true>(input_format_t::bson, value) && sax->number_integer(value);
+                return get_number<std::int32_t, true>(input_format_t::bson, value) && emit_signed(value);
             }
 
             case 0x12: // int64
             {
                 std::int64_t value{};
-                return get_number<std::int64_t, true>(input_format_t::bson, value) && sax->number_integer(value);
+                return get_number<std::int64_t, true>(input_format_t::bson, value) && emit_signed(value);
             }
 
             case 0x11: // uint64
             {
                 std::uint64_t value{};
-                return get_number<std::uint64_t, true>(input_format_t::bson, value) && sax->number_unsigned(value);
+                return get_number<std::uint64_t, true>(input_format_t::bson, value) && emit_unsigned(value);
             }
 
             default: // anything else is not supported (yet)
@@ -638,14 +638,17 @@ class binary_reader
         {
             return false;
         }
-        const auto max_val = static_cast<NumberType>((std::numeric_limits<number_integer_t>::max)());
-        if (number > max_val)
+
+        // the value is -1 - number, which fits into number_integer_t
+        // whenever number does
+        if (JSON_HEDLEY_LIKELY(value_in_range_of<number_integer_t>(number)))
         {
-            return sax->parse_error(chars_read, get_token_string(),
-                                    parse_error::create(112, chars_read,
-                                            exception_message(input_format_t::cbor, "negative integer overflow", "value"), nullptr));
+            return sax->number_integer(static_cast<number_integer_t>(-1) - static_cast<number_integer_t>(number));
         }
-        return sax->number_integer(static_cast<number_integer_t>(-1) - static_cast<number_integer_t>(number));
+
+        // like the lexer does for JSON text, store a value too small for
+        // number_integer_t as number_float_t
+        return sax->number_float(static_cast<number_float_t>(-1) - static_cast<number_float_t>(number), "");
     }
 
     /*!
@@ -702,25 +705,25 @@ class binary_reader
             case 0x18: // Unsigned integer (one-byte uint8_t follows)
             {
                 std::uint8_t number{};
-                return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
+                return get_number(input_format_t::cbor, number) && emit_unsigned(number);
             }
 
             case 0x19: // Unsigned integer (two-byte uint16_t follows)
             {
                 std::uint16_t number{};
-                return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
+                return get_number(input_format_t::cbor, number) && emit_unsigned(number);
             }
 
             case 0x1A: // Unsigned integer (four-byte uint32_t follows)
             {
                 std::uint32_t number{};
-                return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
+                return get_number(input_format_t::cbor, number) && emit_unsigned(number);
             }
 
             case 0x1B: // Unsigned integer (eight-byte uint64_t follows)
             {
                 std::uint64_t number{};
-                return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
+                return get_number(input_format_t::cbor, number) && emit_unsigned(number);
             }
 
             // Negative integer -1-0x00..-1-0x17 (-1..-24)
@@ -1165,13 +1168,13 @@ class binary_reader
             case 0xFA: // Single-Precision Float (four-byte IEEE 754)
             {
                 float number{};
-                return get_number(input_format_t::cbor, number) && sax->number_float(static_cast<number_float_t>(number), "");
+                return get_number(input_format_t::cbor, number) && emit_float(input_format_t::cbor, number);
             }
 
             case 0xFB: // Double-Precision Float (eight-byte IEEE 754)
             {
                 double number{};
-                return get_number(input_format_t::cbor, number) && sax->number_float(static_cast<number_float_t>(number), "");
+                return get_number(input_format_t::cbor, number) && emit_float(input_format_t::cbor, number);
             }
 
             default: // anything else (0xFF is handled inside the other types)
@@ -1861,61 +1864,61 @@ class binary_reader
             case 0xCA: // float 32
             {
                 float number{};
-                return get_number(input_format_t::msgpack, number) && sax->number_float(static_cast<number_float_t>(number), "");
+                return get_number(input_format_t::msgpack, number) && emit_float(input_format_t::msgpack, number);
             }
 
             case 0xCB: // float 64
             {
                 double number{};
-                return get_number(input_format_t::msgpack, number) && sax->number_float(static_cast<number_float_t>(number), "");
+                return get_number(input_format_t::msgpack, number) && emit_float(input_format_t::msgpack, number);
             }
 
             case 0xCC: // uint 8
             {
                 std::uint8_t number{};
-                return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
+                return get_number(input_format_t::msgpack, number) && emit_unsigned(number);
             }
 
             case 0xCD: // uint 16
             {
                 std::uint16_t number{};
-                return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
+                return get_number(input_format_t::msgpack, number) && emit_unsigned(number);
             }
 
             case 0xCE: // uint 32
             {
                 std::uint32_t number{};
-                return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
+                return get_number(input_format_t::msgpack, number) && emit_unsigned(number);
             }
 
             case 0xCF: // uint 64
             {
                 std::uint64_t number{};
-                return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
+                return get_number(input_format_t::msgpack, number) && emit_unsigned(number);
             }
 
             case 0xD0: // int 8
             {
                 std::int8_t number{};
-                return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
+                return get_number(input_format_t::msgpack, number) && emit_signed(number);
             }
 
             case 0xD1: // int 16
             {
                 std::int16_t number{};
-                return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
+                return get_number(input_format_t::msgpack, number) && emit_signed(number);
             }
 
             case 0xD2: // int 32
             {
                 std::int32_t number{};
-                return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
+                return get_number(input_format_t::msgpack, number) && emit_signed(number);
             }
 
             case 0xD3: // int 64
             {
                 std::int64_t number{};
-                return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
+                return get_number(input_format_t::msgpack, number) && emit_signed(number);
             }
 
             case 0xDC: // array 16
@@ -2756,7 +2759,7 @@ class binary_reader
                         {
                             return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, exception_message(input_format, "excessive ndarray size caused overflow", "size"), nullptr));
                         }
-                        if (JSON_HEDLEY_UNLIKELY(!sax->number_unsigned(static_cast<number_unsigned_t>(i))))
+                        if (JSON_HEDLEY_UNLIKELY(!emit_unsigned(i)))
                         {
                             return false;
                         }
@@ -2888,37 +2891,37 @@ class binary_reader
                     break;
                 }
                 std::uint8_t number{};
-                return get_number(input_format, number) && sax->number_unsigned(number);
+                return get_number(input_format, number) && emit_unsigned(number);
             }
 
             case 'U':
             {
                 std::uint8_t number{};
-                return get_number(input_format, number) && sax->number_unsigned(number);
+                return get_number(input_format, number) && emit_unsigned(number);
             }
 
             case 'i':
             {
                 std::int8_t number{};
-                return get_number(input_format, number) && sax->number_integer(number);
+                return get_number(input_format, number) && emit_signed(number);
             }
 
             case 'I':
             {
                 std::int16_t number{};
-                return get_number(input_format, number) && sax->number_integer(number);
+                return get_number(input_format, number) && emit_signed(number);
             }
 
             case 'l':
             {
                 std::int32_t number{};
-                return get_number(input_format, number) && sax->number_integer(number);
+                return get_number(input_format, number) && emit_signed(number);
             }
 
             case 'L':
             {
                 std::int64_t number{};
-                return get_number(input_format, number) && sax->number_integer(number);
+                return get_number(input_format, number) && emit_signed(number);
             }
 
             case 'u':
@@ -2928,7 +2931,7 @@ class binary_reader
                     break;
                 }
                 std::uint16_t number{};
-                return get_number(input_format, number) && sax->number_unsigned(number);
+                return get_number(input_format, number) && emit_unsigned(number);
             }
 
             case 'm':
@@ -2938,7 +2941,7 @@ class binary_reader
                     break;
                 }
                 std::uint32_t number{};
-                return get_number(input_format, number) && sax->number_unsigned(number);
+                return get_number(input_format, number) && emit_unsigned(number);
             }
 
             case 'M':
@@ -2948,7 +2951,7 @@ class binary_reader
                     break;
                 }
                 std::uint64_t number{};
-                return get_number(input_format, number) && sax->number_unsigned(number);
+                return get_number(input_format, number) && emit_unsigned(number);
             }
 
             case 'h':
@@ -3006,13 +3009,13 @@ class binary_reader
             case 'd':
             {
                 float number{};
-                return get_number(input_format, number) && sax->number_float(static_cast<number_float_t>(number), "");
+                return get_number(input_format, number) && emit_float(input_format, number);
             }
 
             case 'D':
             {
                 double number{};
-                return get_number(input_format, number) && sax->number_float(static_cast<number_float_t>(number), "");
+                return get_number(input_format, number) && emit_float(input_format, number);
             }
 
             case 'H':
@@ -3479,13 +3482,13 @@ class binary_reader
             case 0x8E: // binary32
             {
                 float number{};
-                return get_number(input_format_t::bon8, number) && sax->number_float(static_cast<number_float_t>(number), "");
+                return get_number(input_format_t::bon8, number) && emit_float(input_format_t::bon8, number);
             }
 
             case 0x8F: // binary64
             {
                 double number{};
-                return get_number(input_format_t::bon8, number) && sax->number_float(static_cast<number_float_t>(number), "");
+                return get_number(input_format_t::bon8, number) && emit_float(input_format_t::bon8, number);
             }
 
             case 0xF8:
@@ -3551,7 +3554,9 @@ class binary_reader
     @brief pass an integer to the SAX parser
 
     Non-negative integers are passed as unsigned, negative integers as signed
-    numbers, like the other binary formats do.
+    numbers, like the other binary formats do. A value that does not fit the
+    number type is passed as described for @ref emit_unsigned and
+    @ref emit_signed.
 
     @param[in] number  the integer
     @return whether the SAX parser accepted the value
@@ -3560,9 +3565,9 @@ class binary_reader
     {
         if (number >= 0)
         {
-            return sax->number_unsigned(static_cast<number_unsigned_t>(number));
+            return emit_unsigned(static_cast<std::uint64_t>(number));
         }
-        return sax->number_integer(static_cast<number_integer_t>(number));
+        return emit_signed(number);
     }
 
     /*!
@@ -3619,8 +3624,7 @@ class binary_reader
             value = (value << 8) | static_cast<std::int64_t>(current);
         }
 
-        return negative ? sax->number_integer(static_cast<number_integer_t>(-(value + offset)))
-               : sax->number_unsigned(static_cast<number_unsigned_t>(value + offset));
+        return emit_bon8_integer(negative ? -(value + offset) : value + offset);
     }
 
     /*!
@@ -3917,6 +3921,79 @@ class binary_reader
         return true;
     }
 
+    /*!
+    @brief pass a signed integer read from the input to the SAX parser
+
+    Like the lexer does for JSON text, a value that does not fit into
+    number_integer_t is passed as number_unsigned_t if it is non-negative and
+    fits there, and as number_float_t otherwise. With the default number
+    types, every integer the binary formats can encode fits, so this only
+    matters for narrower custom number types.
+
+    @tparam NumberType a signed integer type
+    @param[in] number  the integer
+    @return whether the SAX parser accepted the value
+    */
+    template<typename NumberType>
+    bool emit_signed(const NumberType number)
+    {
+        if (JSON_HEDLEY_LIKELY(value_in_range_of<number_integer_t>(number)))
+        {
+            return sax->number_integer(static_cast<number_integer_t>(number));
+        }
+        if (value_in_range_of<number_unsigned_t>(number))
+        {
+            return sax->number_unsigned(static_cast<number_unsigned_t>(number));
+        }
+        return sax->number_float(static_cast<number_float_t>(number), "");
+    }
+
+    /*!
+    @brief pass an unsigned integer read from the input to the SAX parser
+
+    Like the lexer does for JSON text, a value that does not fit into
+    number_unsigned_t is passed as number_float_t.
+
+    @tparam NumberType an unsigned integer type
+    @param[in] number  the integer
+    @return whether the SAX parser accepted the value
+    */
+    template<typename NumberType>
+    bool emit_unsigned(const NumberType number)
+    {
+        if (JSON_HEDLEY_LIKELY(value_in_range_of<number_unsigned_t>(number)))
+        {
+            return sax->number_unsigned(static_cast<number_unsigned_t>(number));
+        }
+        return sax->number_float(static_cast<number_float_t>(number), "");
+    }
+
+    /*!
+    @brief pass a floating-point number read from the input to the SAX parser
+
+    Like the lexer does for JSON text, a finite value that overflows
+    number_float_t is rejected instead of silently becoming infinity. Infinity
+    and NaN in the input are passed on unchanged.
+
+    @tparam NumberType a floating-point type
+    @param[in] format  the current format (for diagnostics)
+    @param[in] number  the number
+    @return whether the SAX parser accepted the value
+
+    @throw out_of_range.406 if a finite @a number overflows number_float_t
+    */
+    template<typename NumberType>
+    bool emit_float(const input_format_t format, const NumberType number)
+    {
+        const auto result = static_cast<number_float_t>(number);
+        if (JSON_HEDLEY_UNLIKELY(std::isfinite(number) && !std::isfinite(result)))
+        {
+            return sax->parse_error(chars_read, get_token_string(),
+                                    out_of_range::create(406, exception_message(format, "number overflow", "value"), nullptr));
+        }
+        return sax->number_float(result, "");
+    }
+
     /*!
     @brief create a string by reading characters from the input
 

include/nlohmann/json.hpp

@@ -6061,240 +6061,21 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
     {
         // the patch
         basic_json result(value_t::array);
-        diff_recursively(result, source, target, path, 0);
-        return result;
-    }
 
-  private:
-    /// @brief two arrays or two objects @ref diff_iteratively is diffing
-    struct diff_frame
-    {
-        diff_frame(const basic_json* source_, const basic_json* target_, const std::size_t path_length_) noexcept
-            : source(source_), target(target_), path_length(path_length_)
-        {}
-
-        // declared for GCC's -Weffc++, which asks for them in a class with
-        // pointer members and a non-trivial destructor; the exception
-        // specifications are left implicit, as GCC 4.8 rejects explicit ones
-        // that differ from them
-        diff_frame(const diff_frame&) = default;
-        diff_frame(diff_frame&&) = default;
-        diff_frame& operator=(const diff_frame&) = default;
-        diff_frame& operator=(diff_frame&&) = default;
-        ~diff_frame() = default;
-
-        /// the values being diffed, both arrays or both objects
-        const basic_json* source;
-        const basic_json* target;
-        /// the length of their path in `current_path`
-        std::size_t path_length;
-        /// arrays: the next index to diff
-        std::size_t index = 0;
-        /// objects: the next member of source to look at
-        const_iterator member{}; // NOLINT(readability-redundant-member-init)
-        /// objects: the keys common to both, in source's order
-        std::vector<typename object_t::key_type> common_keys{}; // NOLINT(readability-redundant-member-init)
-        /// objects: the next entry of common_keys
-        std::size_t next_common = 0;
-        /// objects: the "add" operations for keys only target has
-        basic_json added_ops{}; // NOLINT(readability-redundant-member-init)
-    };
-
-    // The operations of a diff are built by the functions below rather than
-    // where they are needed: building one takes several temporaries, and
-    // unoptimized builds give each temporary a stack slot of its own in the
-    // function it appears in. In diff_recursively, which is on the call stack
-    // once per nesting level, that made every level cost kilobytes of stack.
-
-    /// @brief append a "replace" operation for @a path with @a value to @a result
-    static void diff_replace(basic_json& result, const string_t& path, const basic_json& value)
-    {
-        result.push_back(
-        {
-            {"op", "replace"}, {"path", path}, {"value", value}
-        });
-    }
-
-    /// @brief append a "remove" operation for @a path to @a result
-    static void diff_remove(basic_json& result, const string_t& path)
-    {
-        result.push_back(object(
-        {
-            {"op", "remove"}, {"path", path}
-        }));
-    }
-
-    /// @brief append an "add" operation for @a path with @a value to @a result
-    static void diff_add(basic_json& result, const string_t& path, const basic_json& value)
-    {
-        result.push_back(
-        {
-            {"op", "add"}, {"path", path}, {"value", value}
-        });
-    }
-
-    /// @brief append the "remove" operations for the elements of array
-    ///        @a source from @a index on, and the "add" operations for the
-    ///        elements of array @a target from source's size on, to @a result
-    static void diff_array_tails(basic_json& result, const basic_json& source, const basic_json& target,
-                                 const string_t& path, const std::size_t index)
-    {
-        // remove my remaining elements, highest index first; appending
-        // in that order avoids the quadratic reinsertion done before
-        for (std::size_t j = source.size(); j > index; --j)
-        {
-            diff_remove(result, detail::concat<string_t>(path, '/', detail::to_string<string_t>(j - 1)));
-        }
-
-        // add other remaining elements
-        for (std::size_t i = source.size(); i < target.size(); ++i)
-        {
-            diff_add(result, detail::concat<string_t>(path, "/-"), target[i]);
-        }
-    }
-
-    /*!
-    @brief compare the keys of objects @a source and @a target
-
-    If the keys both objects have are in the same order in both, and the keys
-    only @a target has come after them, stores the keys common to both in
-    source's order in @a common_keys, stores the "add" operations for the keys
-    only @a target has in @a added_ops, and returns true: the caller then diffs
-    the objects member by member. Otherwise, appends operations that remove
-    every member of @a source and add every member of @a target to @a result,
-    and returns false.
-    */
-    static bool diff_object_keys(basic_json& result, const basic_json& source, const basic_json& target,
-                                 const string_t& path, std::vector<typename object_t::key_type>& common_keys,
-                                 basic_json& added_ops)
-    {
-        // first pass: record, for every source key, whether it is
-        // common to both objects (in source's iteration order) or
-        // was deleted (i.e., in source but not in target) -- this is
-        // a by-product of the target.find() call already needed to
-        // tell the two cases apart, so it adds no extra lookups. The
-        // "remove" ops themselves are emitted later, interleaved
-        // with the per-key diffs in the caller's fast path, to match
-        // source's original iteration order (as the original,
-        // pre-reordering-aware implementation did) instead of
-        // grouping all removes before all per-key diffs.
-        std::vector<typename object_t::key_type> common_keys_source_order;
-        for (auto it = source.cbegin(); it != source.cend(); ++it)
-        {
-            if (target.find(it.key()) != target.end())
-            {
-                common_keys_source_order.push_back(it.key());
-            }
-        }
-
-        // second pass: find keys that were added (i.e., in target but
-        // not in source), and record the keys common to both, in
-        // target's iteration order -- again a by-product of the
-        // source.find() call already needed to detect added keys. At
-        // the same time, determine whether every added key comes
-        // after every common key in target's order (a precondition
-        // for the fast path, which only ever appends new keys
-        // at the very end): for an object_t whose iteration order is
-        // a pure function of the key set (e.g. the default std::map,
-        // which always iterates in sorted key order), the order
-        // check further below is always true and this whole
-        // mechanism is effectively a no-op; it only matters for a
-        // reorderable object_t such as the one backing `ordered_json`.
-        // The patch ops for keys that were added (i.e., in target but not
-        // in source) are built here so the fast path can reuse
-        // them without a second source.find() per target key. Only
-        // used by the fast path -- the slow (reordering) path
-        // rebuilds "add" ops for every key itself.
-        std::vector<typename object_t::key_type> common_keys_target_order;
-        bool new_keys_form_suffix = true;
-        bool seen_new_key = false;
-        for (auto it = target.cbegin(); it != target.cend(); ++it)
-        {
-            if (source.find(it.key()) == source.end())
-            {
-                seen_new_key = true;
-                diff_add(added_ops, detail::concat<string_t>(path, '/', detail::escape(it.key())), it.value());
-            }
-            else
-            {
-                common_keys_target_order.push_back(it.key());
-                if (seen_new_key)
-                {
-                    new_keys_form_suffix = false;
-                }
-            }
-        }
-
-        if (common_keys_source_order == common_keys_target_order && new_keys_form_suffix)
-        {
-            // fast path: order of common keys already matches (or the
-            // object_t's iteration order does not depend on
-            // insertion history), so a plain per-key diff is correct
-            // and minimal, as before
-            common_keys = std::move(common_keys_source_order);
-            return true;
-        }
-
-        // slow path: the common keys are in a different relative
-        // order in source and target (only possible for a
-        // reorderable object_t like ordered_map). Building a
-        // minimal reordering patch is a nontrivial (LCS-like)
-        // problem; instead, remove every source key -- both
-        // deleted keys (which must be removed regardless) and
-        // common keys (removed so they can be re-added in
-        // target's order) -- and re-add every key that should
-        // remain, with its final target value, in target's
-        // order. basic_json::patch()'s "add" operation on an
-        // object uses operator[], which appends at the end for a
-        // vector-backed insertion-ordered map when the key does
-        // not already exist -- so removing a key and then adding
-        // it moves it to the end, fixing its position.
-        for (auto it = source.cbegin(); it != source.cend(); ++it)
-        {
-            diff_remove(result, detail::concat<string_t>(path, '/', detail::escape(it.key())));
-        }
-
-        // add every key that is either common (just removed
-        // above) or brand new, in target's iteration order, so
-        // that the final order after applying the patch matches
-        // target exactly
-        for (auto it = target.cbegin(); it != target.cend(); ++it)
-        {
-            diff_add(result, detail::concat<string_t>(path, '/', detail::escape(it.key())), it.value());
-        }
-        return false;
-    }
-
-    /*!
-    @brief @ref diff, for values at nesting level @a depth, appending the
-           operations to @a result
-
-    Diffing two arrays or objects calls this function again, once per nesting
-    level, so values nested deeply enough used to exhaust the call stack and
-    terminate the process. The descent is bounded here: once @ref
-    detail::recursion_depth_limit levels have been entered, @ref
-    diff_iteratively diffs what is left without the call stack.
-    */
-    static void diff_recursively(basic_json& result, const basic_json& source, const basic_json& target,
-                                 const string_t& path, const std::size_t depth)
-    {
-        // if the values are the same, there is nothing to do
+        // if the values are the same, return an empty patch
         if (source == target)
         {
-            return;
-        }
-
-        if (JSON_HEDLEY_UNLIKELY(depth >= detail::recursion_depth_limit()))
-        {
-            diff_iteratively(result, source, target, path);
-            return;
+            return result;
         }
 
         if (source.type() != target.type())
         {
             // different types: replace value
-            diff_replace(result, path, target);
-            return;
+            result.push_back(
+            {
+                {"op", "replace"}, {"path", path}, {"value", target}
+            });
+            return result;
         }
 
         switch (source.type())
@@ -6306,50 +6087,185 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
                 while (i < source.size() && i < target.size())
                 {
                     // recursive call to compare array values at index i
-                    diff_recursively(result, source[i], target[i], detail::concat<string_t>(path, '/', detail::to_string<string_t>(i)), depth + 1);
+                    auto temp_diff = diff(source[i], target[i], detail::concat<string_t>(path, '/', detail::to_string<string_t>(i)));
+                    result.insert(result.end(), temp_diff.begin(), temp_diff.end());
                     ++i;
                 }
 
                 // We now reached the end of at least one array
                 // in a second pass, traverse the remaining elements
-                diff_array_tails(result, source, target, path, i);
+
+                // remove my remaining elements, highest index first; appending
+                // in that order avoids the quadratic reinsertion done before
+                for (std::size_t j = source.size(); j > i; --j)
+                {
+                    result.push_back(object(
+                    {
+                        {"op", "remove"},
+                        {"path", detail::concat<string_t>(path, '/', detail::to_string<string_t>(j - 1))}
+                    }));
+                }
+                i = source.size();
+
+                // add other remaining elements
+                while (i < target.size())
+                {
+                    result.push_back(
+                    {
+                        {"op", "add"},
+                        {"path", detail::concat<string_t>(path, "/-")},
+                        {"value", target[i]}
+                    });
+                    ++i;
+                }
+
                 break;
             }
 
             case value_t::object:
             {
-                std::vector<typename object_t::key_type> common_keys;
-                basic_json added_ops(value_t::array);
-                if (diff_object_keys(result, source, target, path, common_keys, added_ops))
+                // first pass: record, for every source key, whether it is
+                // common to both objects (in source's iteration order) or
+                // was deleted (i.e., in source but not in target) -- this is
+                // a by-product of the target.find() call already needed to
+                // tell the two cases apart, so it adds no extra lookups. The
+                // "remove" ops themselves are emitted later, interleaved
+                // with the recursive per-key diffs in the fast path below,
+                // to match source's original iteration order (as the
+                // original, pre-reordering-aware implementation did) instead
+                // of grouping all removes before all recursive diffs.
+                std::vector<typename object_t::key_type> common_keys_source_order;
+                for (auto it = source.cbegin(); it != source.cend(); ++it)
                 {
-                    // fast path: common_keys is, by construction, the
-                    // subsequence of source's keys that are common to both
-                    // objects, in source's iteration order -- so it can be
-                    // walked in lockstep with `source` using a cheap key
-                    // comparison instead of another lookup. Deleted keys
-                    // (those source keys not in common_keys) are interleaved
-                    // here too, in source's original order, to match the
-                    // historical (pre-reordering-aware) output order.
-                    auto common_it = common_keys.cbegin();
+                    if (target.find(it.key()) != target.end())
+                    {
+                        common_keys_source_order.push_back(it.key());
+                    }
+                }
+
+                // second pass: find keys that were added (i.e., in target but
+                // not in source), and record the keys common to both, in
+                // target's iteration order -- again a by-product of the
+                // source.find() call already needed to detect added keys. At
+                // the same time, determine whether every added key comes
+                // after every common key in target's order (a precondition
+                // for the fast path below, which only ever appends new keys
+                // at the very end): for an object_t whose iteration order is
+                // a pure function of the key set (e.g. the default std::map,
+                // which always iterates in sorted key order), the order
+                // check further below is always true and this whole
+                // mechanism is effectively a no-op; it only matters for a
+                // reorderable object_t such as the one backing `ordered_json`.
+                // patch ops for keys that were added (i.e., in target but not
+                // in source); built here so the fast path below can reuse
+                // them without a second source.find() per target key. Only
+                // used by the fast path -- the slow (reordering) path
+                // rebuilds "add" ops for every key itself.
+                std::vector<typename object_t::key_type> common_keys_target_order;
+                basic_json added_ops(value_t::array);
+                bool new_keys_form_suffix = true;
+                bool seen_new_key = false;
+                for (auto it = target.cbegin(); it != target.cend(); ++it)
+                {
+                    if (source.find(it.key()) == source.end())
+                    {
+                        seen_new_key = true;
+                        const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
+                        added_ops.push_back(
+                        {
+                            {"op", "add"}, {"path", path_key},
+                            {"value", it.value()}
+                        });
+                    }
+                    else
+                    {
+                        common_keys_target_order.push_back(it.key());
+                        if (seen_new_key)
+                        {
+                            new_keys_form_suffix = false;
+                        }
+                    }
+                }
+
+                if (common_keys_source_order == common_keys_target_order && new_keys_form_suffix)
+                {
+                    // fast path: order of common keys already matches (or the
+                    // object_t's iteration order does not depend on
+                    // insertion history), so a plain per-key recursive diff
+                    // is correct and minimal, as before. common_keys_source_order
+                    // is, by construction, the subsequence of source's keys
+                    // that are common to both objects, in source's iteration
+                    // order -- so it can be walked in lockstep with `source`
+                    // using a cheap key comparison instead of another lookup.
+                    // Deleted keys (those source keys not in common_keys_source_order)
+                    // are interleaved here too, in source's original order, to
+                    // match the historical (pre-reordering-aware) output order.
+                    auto common_it = common_keys_source_order.cbegin();
                     for (auto it = source.cbegin(); it != source.cend(); ++it)
                     {
-                        if (common_it != common_keys.cend() && it.key() == *common_it)
+                        if (common_it != common_keys_source_order.cend() && it.key() == *common_it)
                         {
-                            diff_recursively(result, it.value(), target[it.key()], detail::concat<string_t>(path, '/', detail::escape(it.key())), depth + 1);
+                            const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
+                            auto temp_diff = diff(it.value(), target[it.key()], path_key);
+                            result.insert(result.end(), temp_diff.begin(), temp_diff.end());
                             ++common_it;
                         }
                         else
                         {
                             // found a key that is not in target -> remove it
-                            diff_remove(result, detail::concat<string_t>(path, '/', detail::escape(it.key())));
+                            const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
+                            result.push_back(object(
+                            {
+                                {"op", "remove"}, {"path", path_key}
+                            }));
                         }
                     }
 
-                    // append the "add" ops for brand-new keys collected by
-                    // diff_object_keys -- no second source.find() per target
-                    // key needed
+                    // append the "add" ops for brand-new keys collected above
+                    // during the pass over target -- no second source.find()
+                    // per target key needed
                     result.insert(result.end(), added_ops.begin(), added_ops.end());
                 }
+                else
+                {
+                    // slow path: the common keys are in a different relative
+                    // order in source and target (only possible for a
+                    // reorderable object_t like ordered_map). Building a
+                    // minimal reordering patch is a nontrivial (LCS-like)
+                    // problem; instead, remove every source key -- both
+                    // deleted keys (which must be removed regardless) and
+                    // common keys (removed so they can be re-added in
+                    // target's order) -- and re-add every key that should
+                    // remain, with its final target value, in target's
+                    // order. basic_json::patch()'s "add" operation on an
+                    // object uses operator[], which appends at the end for a
+                    // vector-backed insertion-ordered map when the key does
+                    // not already exist -- so removing a key and then adding
+                    // it moves it to the end, fixing its position.
+                    for (auto it = source.cbegin(); it != source.cend(); ++it)
+                    {
+                        const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
+                        result.push_back(object(
+                        {
+                            {"op", "remove"}, {"path", path_key}
+                        }));
+                    }
+
+                    // add every key that is either common (just removed
+                    // above) or brand new, in target's iteration order, so
+                    // that the final order after applying the patch matches
+                    // target exactly
+                    for (auto it = target.cbegin(); it != target.cend(); ++it)
+                    {
+                        const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
+                        result.push_back(
+                        {
+                            {"op", "add"}, {"path", path_key},
+                            {"value", it.value()}
+                        });
+                    }
+                }
+
                 break;
             }
 
@@ -6364,170 +6280,16 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
             default:
             {
                 // both primitive types: replace value
-                diff_replace(result, path, target);
+                result.push_back(
+                {
+                    {"op", "replace"}, {"path", path}, {"value", target}
+                });
                 break;
             }
         }
+
+        return result;
     }
-
-    /*!
-    @brief @ref diff without the call stack, appending the operations to
-           @a result
-
-    Produces the same operations as @ref diff_recursively. Only reached for
-    values nested more deeply than @ref detail::recursion_depth_limit.
-    */
-    static void diff_iteratively(basic_json& result, const basic_json& source, const basic_json& target,
-                                 const string_t& path)
-    {
-        // The arrays and objects being diffed are kept on an explicit stack,
-        // and every pair of elements is still diffed completely before the
-        // next one, so the operations come out in the same order as in
-        // diff_recursively. The path of the values being diffed is kept in
-        // one buffer that grows and shrinks with the stack, rather than in a
-        // new string per level.
-        std::vector<diff_frame> stack;
-        string_t current_path = path;
-
-        // diff `s` against `t`, whose path is current_path: primitives,
-        // values of different types, and objects whose members were reordered
-        // are handled right away; arrays and other objects get a frame
-        const auto enter = [&result, &stack, &current_path](const basic_json & s, const basic_json & t)
-        {
-            // if the values are the same, there is nothing to do. Arrays and
-            // objects are not compared up front: comparing them visits
-            // everything below them, so doing that at every level would take
-            // quadratic time in the nesting depth - equal ones yield no
-            // operations anyway.
-            if ((!s.is_structured() || !t.is_structured()) && s == t)
-            {
-                return;
-            }
-
-            if (s.type() != t.type())
-            {
-                // different types: replace value
-                diff_replace(result, current_path, t);
-                return;
-            }
-
-            switch (s.type())
-            {
-                case value_t::array:
-                {
-                    stack.emplace_back(&s, &t, current_path.size());
-                    return;
-                }
-
-                case value_t::object:
-                {
-                    std::vector<typename object_t::key_type> common_keys;
-                    basic_json added_ops(value_t::array);
-                    if (diff_object_keys(result, s, t, current_path, common_keys, added_ops))
-                    {
-                        // fast path: the frame walks source in lockstep with
-                        // common_keys, as diff_recursively does, and appends
-                        // added_ops once all members are done
-                        stack.emplace_back(&s, &t, current_path.size());
-                        stack.back().member = s.cbegin();
-                        stack.back().common_keys = std::move(common_keys);
-                        stack.back().added_ops = std::move(added_ops);
-                    }
-                    return;
-                }
-
-                case value_t::null:
-                case value_t::string:
-                case value_t::boolean:
-                case value_t::number_integer:
-                case value_t::number_unsigned:
-                case value_t::number_float:
-                case value_t::binary:
-                case value_t::discarded:
-                default:
-                {
-                    // both primitive types: replace value
-                    diff_replace(result, current_path, t);
-                    return;
-                }
-            }
-        };
-
-        enter(source, target);
-        while (!stack.empty())
-        {
-            // the frame is copied out member by member and changed through
-            // stack.back(): enter() may push a frame and the end of the loop
-            // pops it, either of which would invalidate a reference to it
-            const basic_json* const s = stack.back().source;
-            const basic_json* const t = stack.back().target;
-            const std::size_t path_length = stack.back().path_length;
-            const std::size_t depth = stack.size();
-
-            if (s->is_array())
-            {
-                const auto& source_array = *s->m_data.m_value.array;
-                const auto& target_array = *t->m_data.m_value.array;
-
-                // first pass: traverse common elements
-                const std::size_t i = stack.back().index;
-                if (i < source_array.size() && i < target_array.size())
-                {
-                    ++stack.back().index;
-                    detail::concat_into(current_path, '/', detail::to_string<string_t>(i));
-                    enter(source_array[i], target_array[i]);
-                    if (stack.size() == depth)
-                    {
-                        current_path.resize(path_length);
-                    }
-                    continue;
-                }
-
-                // We now reached the end of at least one array
-                // in a second pass, traverse the remaining elements
-                diff_array_tails(result, *s, *t, current_path, i);
-            }
-            else
-            {
-                const const_iterator it = stack.back().member;
-                if (it != s->cend())
-                {
-                    ++stack.back().member;
-                    const std::size_t next_common = stack.back().next_common;
-                    if (next_common < stack.back().common_keys.size() && it.key() == stack.back().common_keys[next_common])
-                    {
-                        ++stack.back().next_common;
-                        const basic_json& target_value = (*t)[it.key()];
-                        detail::concat_into(current_path, '/', detail::escape(it.key()));
-                        enter(it.value(), target_value);
-                        if (stack.size() == depth)
-                        {
-                            current_path.resize(path_length);
-                        }
-                    }
-                    else
-                    {
-                        // found a key that is not in target -> remove it
-                        diff_remove(result, detail::concat<string_t>(current_path, '/', detail::escape(it.key())));
-                    }
-                    continue;
-                }
-
-                // append the "add" ops for brand-new keys collected when the
-                // object was entered
-                result.insert(result.end(), stack.back().added_ops.begin(), stack.back().added_ops.end());
-            }
-
-            // this array or object is done: continue with the one it is in
-            stack.pop_back();
-            if (!stack.empty())
-            {
-                current_path.resize(stack.back().path_length);
-            }
-        }
-    }
-
-  public:
     /// @}
 
     ////////////////////////////////

tests/src/unit-binary_formats.cpp

@@ -11,7 +11,12 @@
 #include <nlohmann/json.hpp>
 using nlohmann::json;
 
+#include <cmath>
 #include <fstream>
+#include <limits>
+#include <map>
+#include <string>
+#include <vector>
 #include "make_test_data_available.hpp"
 
 TEST_CASE("Binary Formats" * doctest::skip())
@@ -224,3 +229,114 @@ TEST_CASE("Binary Formats" * doctest::skip())
         CHECK((100.0 * double(ubjson_3_size) / double(json_size)) == Approx(89.450));
     }
 }
+
+TEST_CASE("Binary formats with narrow number types")
+{
+    // Numbers that do not fit the number types are handled like the lexer
+    // handles them in JSON text: an integer that fits neither integer type is
+    // stored as a floating-point number, and a finite floating-point number
+    // that overflows number_float_t is rejected with out_of_range.406.
+    using narrow_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int32_t, std::uint32_t, float>;
+    using bytes = std::vector<std::uint8_t>;
+
+    struct binary_format
+    {
+        const char* name;
+        bytes (*encode)(const json&);
+        narrow_json (*decode)(const bytes&, bool);
+    };
+
+    const std::vector<binary_format> formats =
+    {
+        {
+            "CBOR", [](const json & j) { return json::to_cbor(j); },
+            [](const bytes & v, bool allow_exceptions)
+            {
+                return narrow_json::from_cbor(v, true, allow_exceptions);
+            }
+        },
+        {
+            "MessagePack", [](const json & j) { return json::to_msgpack(j); },
+            [](const bytes & v, bool allow_exceptions)
+            {
+                return narrow_json::from_msgpack(v, true, allow_exceptions);
+            }
+        },
+        {
+            "UBJSON", [](const json & j) { return json::to_ubjson(j); },
+            [](const bytes & v, bool allow_exceptions)
+            {
+                return narrow_json::from_ubjson(v, true, allow_exceptions);
+            }
+        },
+        {
+            "BJData", [](const json & j) { return json::to_bjdata(j); },
+            [](const bytes & v, bool allow_exceptions)
+            {
+                return narrow_json::from_bjdata(v, true, allow_exceptions);
+            }
+        },
+        {
+            // BSON can only store numbers as object members
+            "BSON", [](const json & j) { return json::to_bson(json{{"a", j}}); },
+            [](const bytes & v, bool allow_exceptions)
+            {
+                const auto result = narrow_json::from_bson(v, true, allow_exceptions);
+                return result.is_discarded() ? result : result.at("a");
+            }
+        },
+        {
+            "BON8", [](const json & j) { return json::to_bon8(j); },
+            [](const bytes & v, bool allow_exceptions)
+            {
+                return narrow_json::from_bon8(v, true, allow_exceptions);
+            }
+        },
+    };
+
+    for (const auto& format : formats)
+    {
+        const std::string name = format.name;
+        INFO("format := ", name);
+        const auto roundtrip = [&format](const json & j)
+        {
+            return format.decode(format.encode(j), true);
+        };
+
+        // integers that fit keep their type
+        CHECK(roundtrip(json(-5)).is_number_integer());
+        CHECK(roundtrip(json(-5)).get<std::int32_t>() == -5);
+        CHECK(roundtrip(json(3000000000u)).is_number_unsigned());
+        CHECK(roundtrip(json(3000000000u)).get<std::uint32_t>() == 3000000000u);
+
+        // integers that fit neither integer type are stored as float
+        CHECK(roundtrip(json(5000000000u)).is_number_float());
+        CHECK(roundtrip(json(5000000000u)).get<float>() == 5000000000.0f);
+        if (name != "BON8") // BON8 cannot encode integers above INT64_MAX
+        {
+            CHECK(roundtrip(json(10000000000000000000u)).is_number_float());
+            CHECK(roundtrip(json(10000000000000000000u)).get<float>() == 10000000000000000000.0f);
+        }
+        CHECK(roundtrip(json(-3000000000)).is_number_float());
+        CHECK(roundtrip(json(-3000000000)).get<float>() == -3000000000.0f);
+        CHECK(roundtrip(json(-5000000000)).is_number_float());
+        CHECK(roundtrip(json(-5000000000)).get<float>() == -5000000000.0f);
+
+        // floating-point numbers that fit
+        CHECK(roundtrip(json(1.5)).get<float>() == 1.5f);
+        const auto just_above_max = std::nextafter(static_cast<double>((std::numeric_limits<float>::max)()),
+                                    std::numeric_limits<double>::infinity());
+        CHECK(roundtrip(json(just_above_max)).get<float>() == (std::numeric_limits<float>::max)());
+
+        // infinity and NaN are passed on
+        CHECK(std::isinf(roundtrip(json(std::numeric_limits<double>::infinity())).get<float>()));
+        CHECK(std::isnan(roundtrip(json(std::numeric_limits<double>::quiet_NaN())).get<float>()));
+
+        // finite floating-point numbers that overflow number_float_t are rejected
+        const std::string message = "[json.exception.out_of_range.406] syntax error while parsing " + name
+                                    + " value: number overflow";
+        CHECK_THROWS_WITH_AS(roundtrip(json(1e300)), message.c_str(), narrow_json::out_of_range&);
+        CHECK_THROWS_WITH_AS(roundtrip(json(-1e300)), message.c_str(), narrow_json::out_of_range&);
+        CHECK(format.decode(format.encode(json(1e300)), false).is_discarded());
+    }
+}

tests/src/unit-cbor.cpp

@@ -3146,7 +3146,8 @@ TEST_CASE("Tagged values")
         // CBOR encodes negative integers as: result = -1 - n
         // For type 0x3B, n is an 8-byte uint64_t. Valid range for n with
         // the default int64_t is [0, INT64_MAX], producing results in [INT64_MIN, -1].
-        // When n > INT64_MAX, the result exceeds int64_t range and is rejected.
+        // When n > INT64_MAX, the result exceeds int64_t range and is stored
+        // as a floating-point number, as the lexer does for JSON text.
 
         SECTION("n = 0 is valid (result = -1)")
         {
@@ -3167,33 +3168,34 @@ TEST_CASE("Tagged values")
             CHECK(result.get<int64_t>() == (std::numeric_limits<int64_t>::min)());
         }
 
-        SECTION("n = INT64_MAX + 1 is rejected (overflow)")
+        SECTION("n = INT64_MAX + 1 is stored as float")
         {
             // n = INT64_MAX + 1 (0x8000000000000000)
-            // result = -1 - n = -9223372036854775809, which exceeds int64_t range
+            // result = -1 - n = -9223372036854775809, which exceeds int64_t range;
+            // the nearest double is -9223372036854775808.0
             const std::vector<uint8_t> input = {0x3B, 0x80, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00};
-            json _;
-            CHECK_THROWS_WITH_AS(_ = json::from_cbor(input),
-                                 "[json.exception.parse_error.112] parse error at byte 9: syntax error while parsing CBOR value: negative integer overflow",
-                                 json::parse_error);
+            const auto result = json::from_cbor(input);
+            CHECK(result.is_number_float());
+            CHECK(result.get<double>() == -9223372036854775808.0);
+            CHECK(result == json::parse("-9223372036854775809"));
         }
 
-        SECTION("n = UINT64_MAX is rejected (overflow)")
+        SECTION("n = UINT64_MAX is stored as float")
         {
             // n = UINT64_MAX (0xFFFFFFFFFFFFFFFF)
             // result = -1 - n = -18446744073709551616, which exceeds int64_t range
             const std::vector<uint8_t> input = {0x3B, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF};
-            json _;
-            CHECK_THROWS_WITH_AS(_ = json::from_cbor(input),
-                                 "[json.exception.parse_error.112] parse error at byte 9: syntax error while parsing CBOR value: negative integer overflow",
-                                 json::parse_error);
+            const auto result = json::from_cbor(input);
+            CHECK(result.is_number_float());
+            CHECK(result.get<double>() == -18446744073709551616.0);
+            CHECK(result == json::parse("-18446744073709551616"));
         }
 
-        SECTION("overflow with allow_exceptions=false returns discarded")
+        SECTION("overflow with allow_exceptions=false is not an error")
         {
             const std::vector<uint8_t> input = {0x3B, 0x80, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00};
             const auto result = json::from_cbor(input, true, false);
-            CHECK(result.is_discarded());
+            CHECK(result.is_number_float());
         }
     }
 

tests/src/unit-json_patch.cpp

@@ -15,65 +15,8 @@ using nlohmann::json;
 #endif
 
 #include <fstream>
-#include <string>
-#include <vector>
 #include "make_test_data_available.hpp"
 
-namespace
-{
-// alternating objects and arrays nested `depth` levels deep, with members that
-// depend on `variant` at some levels, so diffing two variants yields
-// operations on many levels: replacing the innermost value, adding, removing,
-// and (for ordered_json) reordering members, and changing array lengths
-template<typename BasicJsonType>
-BasicJsonType nested(const std::size_t depth, const int variant)
-{
-    BasicJsonType value = variant;
-    for (std::size_t i = 0; i < depth; ++i)
-    {
-        if (i % 2 == 0)
-        {
-            BasicJsonType object = BasicJsonType::object();
-            if ((i + static_cast<std::size_t>(variant)) % 7 == 0)
-            {
-                object["x"] = i;
-            }
-            if (variant == 2 && i % 11 == 0)
-            {
-                object["z"] = "z";
-            }
-            object["a"] = std::move(value);
-            if (variant == 1 && i % 5 == 0)
-            {
-                object["y"] = 1;
-            }
-            value = std::move(object);
-        }
-        else
-        {
-            BasicJsonType array = BasicJsonType::array({std::move(value)});
-            if ((i + static_cast<std::size_t>(variant)) % 3 == 0)
-            {
-                array.push_back(i);
-            }
-            value = std::move(array);
-        }
-    }
-    return value;
-}
-
-// a path of `depth` reference tokens, as nested() nests its values
-std::string nested_path(const std::size_t depth)
-{
-    std::string path;
-    for (std::size_t i = depth; i > 0; --i)
-    {
-        path += (i - 1) % 2 == 0 ? "/a" : "/0";
-    }
-    return path;
-}
-} // namespace
-
 TEST_CASE("JSON patch")
 {
     SECTION("examples from RFC 6902")
@@ -1809,102 +1752,6 @@ TEST_CASE("JSON patch - diff emits array removals in descending index order")
     }
 }
 
-TEST_CASE("JSON patch: diff of deeply nested values")
-{
-    SECTION("the diff reproduces the target at every depth")
-    {
-        // depths on either side of the nesting depth up to which diff()
-        // recurses (detail::recursion_depth_limit(), 128); not every depth up
-        // to 300, as the test would then time out under Valgrind
-        std::vector<std::size_t> depths;
-        for (std::size_t depth = 0; depth <= 16; ++depth)
-        {
-            depths.push_back(depth);
-        }
-        for (std::size_t depth = 120; depth <= 136; ++depth)
-        {
-            depths.push_back(depth);
-        }
-        depths.push_back(300);
-
-        for (const auto depth : depths)
-        {
-            CAPTURE(depth);
-            for (int from = 0; from < 3; ++from)
-            {
-                for (int to = 0; to < 3; ++to)
-                {
-                    CAPTURE(from);
-                    CAPTURE(to);
-                    const auto source = nested<json>(depth, from);
-                    const auto target = nested<json>(depth, to);
-                    const auto patch = json::diff(source, target);
-                    CHECK(source.patch(patch) == target);
-                    CHECK(patch.empty() == (from == to));
-
-                    const auto ordered_source = nested<nlohmann::ordered_json>(depth, from);
-                    const auto ordered_target = nested<nlohmann::ordered_json>(depth, to);
-                    CHECK(ordered_source.patch(nlohmann::ordered_json::diff(ordered_source, ordered_target)) == ordered_target);
-                }
-            }
-        }
-    }
-
-    SECTION("a difference only in the innermost value is one replace operation")
-    {
-        for (std::size_t depth = 0; depth <= 300; ++depth)
-        {
-            CAPTURE(depth);
-            json source = 1;
-            json target = 2;
-            for (std::size_t i = 0; i < depth; ++i)
-            {
-                source = i % 2 == 0 ? json::object({{"a", std::move(source)}}) : json::array({std::move(source)});
-                target = i % 2 == 0 ? json::object({{"a", std::move(target)}}) : json::array({std::move(target)});
-            }
-            CHECK(json::diff(source, target, "/root") == json::array({{{"op", "replace"}, {"path", "/root" + nested_path(depth)}, {"value", 2}}}));
-        }
-    }
-
-    SECTION("values nested too deeply for the call stack (#5393)")
-    {
-        // diff() used to recurse once per nesting level, and compared the
-        // values with operator== on every level. The values are only
-        // parsed and diffed, never copied or compared, since those recurse
-        // too.
-        const std::size_t depth = 100000;
-        for (const bool objects :
-                {
-                    false, true
-                })
-        {
-            CAPTURE(objects);
-            std::string source_text;
-            std::string target_text;
-            std::string equal_text;
-            std::string path;
-            for (std::size_t i = 0; i < depth; ++i)
-            {
-                source_text += objects ? "{\"a\":" : "[";
-                path += objects ? "/a" : "/0";
-            }
-            target_text = source_text + "2";
-            equal_text = source_text + "1";
-            source_text += "1";
-            const std::string closing(depth, objects ? '}' : ']');
-            const auto source = json::parse(source_text + closing);
-
-            const auto patch = json::diff(source, json::parse(target_text + closing));
-            REQUIRE(patch.size() == 1);
-            CHECK(patch[0]["op"] == "replace");
-            CHECK(patch[0]["path"] == path);
-            CHECK(patch[0]["value"] == 2);
-
-            CHECK(json::diff(source, json::parse(equal_text + closing)).empty());
-        }
-    }
-}
-
 TEST_CASE("JSON patch - every operation on ordered_json")
 {
     using nlohmann::ordered_json;