Merge branch 'develop' into claude/binary-reader-narrow-numbers

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

.clang-tidy

@@ -1,9 +1,11 @@
-# TODO: portability-template-virtual-member-function is only removed to get the CI going. It has to be addressed at some point.
+# TODO: The first three checks are only removed to get the CI going. They have to be addressed at some point.
 # TODO: portability-avoid-pragma-once: should be fixed eventually
 
 Checks: '*,
 
          -portability-template-virtual-member-function,
+         -bugprone-use-after-move,
+         -hicpp-invalid-access-moved,
 
          -altera-id-dependent-backward-branch,
          -altera-struct-pack-align,

docs/mkdocs/docs/api/basic_json/meta.md

@@ -13,7 +13,7 @@ JSON object holding version information
 
 | key         | description                                                                                                                                                                                                                                                                    |
 |-------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `compiler`  | Information on the used compiler. It is an object with the following keys: `c++` (the used C++ standard), `family` (the compiler family; possible values are `clang`, `icc`, `gcc`, `hp`, `ilecpp`, `msvc`, `pgcpp`, `sunpro`, and `unknown`), and `version` (the compiler version). |
+| `compiler`  | Information on the used compiler. It is an object with the following keys: `c++` (the used C++ standard), `family` (the compiler family; possible values are `clang`, `icc`, `gcc`, `ilecpp`, `msvc`, `pgcpp`, `sunpro`, and `unknown`), and `version` (the compiler version). On HP aCC compilers, `compiler` is instead the plain string `hp`. |
 | `copyright` | The copyright line for the library as string.                                                                                                                                                                                                                                  |
 | `name`      | The name of the library as string.                                                                                                                                                                                                                                             |
 | `platform`  | The used platform as string. Possible values are `win32`, `linux`, `apple`, `unix`, and `unknown`.                                                                                                                                                                             |

docs/mkdocs/docs/api/basic_json/number_float_t.md

@@ -55,6 +55,10 @@ This implementation does exactly follow this approach, as it uses double precisi
 smaller than `-1.79769313486232e+308` and values greater than `1.79769313486232e+308` will be stored as NaN internally
 and be serialized to `null`.
 
+During deserialization (from JSON text or any of the binary formats), a finite number that does not fit into
+`number_float_t` is rejected with [`out_of_range.406`](../../home/exceptions.md#jsonexceptionout_of_range406), for
+example a double-precision number in a binary format when `number_float_t` is `#!cpp float`.
+
 #### Storage
 
 Floating-point number values are stored directly inside a `basic_json` type.

docs/mkdocs/docs/api/basic_json/number_integer_t.md

@@ -47,8 +47,9 @@ With the default values for `NumberIntegerType` (`std::int64_t`), the default va
 
 When the default type is used, the maximal integer number that can be stored is `9223372036854775807` (INT64_MAX) and
 the minimal integer number that can be stored is `-9223372036854775808` (INT64_MIN). Integer numbers that are out of
-range will yield over/underflow when used in a constructor. During deserialization, too large or small integer numbers
-will automatically be stored as [`number_unsigned_t`](number_unsigned_t.md) or [`number_float_t`](number_float_t.md).
+range will yield over/underflow when used in a constructor. During deserialization (from JSON text or any of the binary
+formats), too large or small integer numbers will automatically be stored as [`number_unsigned_t`](number_unsigned_t.md)
+or [`number_float_t`](number_float_t.md).
 
 [RFC 8259](https://tools.ietf.org/html/rfc8259) further states:
 > Note that when such software is used, numbers that are integers and are in the range $[-2^{53}+1, 2^{53}-1]$ are

docs/mkdocs/docs/api/basic_json/number_unsigned_t.md

@@ -48,8 +48,9 @@ With the default values for `NumberUnsignedType` (`std::uint64_t`), the default
 
 When the default type is used, the maximal integer number that can be stored is `18446744073709551615` (UINT64_MAX) and
 the minimal integer number that can be stored is `0`. Integer numbers that are out of range will yield over/underflow
-when used in a constructor. During deserialization, too large or small integer numbers will automatically be stored
-as [`number_integer_t`](number_integer_t.md) or [`number_float_t`](number_float_t.md).
+when used in a constructor. During deserialization (from JSON text or any of the binary formats), too large or small
+integer numbers will automatically be stored as [`number_integer_t`](number_integer_t.md) or
+[`number_float_t`](number_float_t.md).
 
 [RFC 8259](https://tools.ietf.org/html/rfc8259) further states:
 > Note that when such software is used, numbers that are integers and are in the range $[-2^{53}+1, 2^{53}-1]$ are

docs/mkdocs/docs/features/binary_formats/cbor.md

@@ -168,9 +168,9 @@ The library maps CBOR types to JSON value types as follows:
 !!! warning "Negative integer overflow"
 
     CBOR negative integers (major type 1) are decoded as `-1 - n`. If the encoded magnitude `n` is too large for the
-    result to fit into `number_integer_t` (`std::int64_t` by default), parsing fails with a
-    [`parse_error.112`](../../home/exceptions.md#jsonexceptionparse_error112) exception rather than overflowing
-    silently.
+    result to fit into `number_integer_t` (`std::int64_t` by default), the result is stored as `number_float_t`, like
+    a too small integer in JSON text. For example, `-18446744073709551616` (`0x3B` followed by eight `0xFF` bytes) is
+    stored as `-1.8446744073709552e+19`.
 
 !!! warning "Object keys"
 

docs/mkdocs/docs/home/exceptions.md

@@ -331,9 +331,6 @@ An unexpected byte was read in a [binary format](../features/binary_formats/inde
     [json.exception.parse_error.112] parse error at byte 15: syntax error while parsing BSON binary: byte array length cannot be negative, is -1
     ```
     ```
-    [json.exception.parse_error.112] parse error at byte 9: syntax error while parsing CBOR value: negative integer overflow
-    ```
-    ```
     [json.exception.parse_error.112] parse error at byte 5: syntax error while parsing BSON document: document size 6 does not match the number of bytes read (5)
     ```
 
@@ -854,13 +851,18 @@ The JSON Patch operations 'remove' and 'add' cannot be applied to the root eleme
 
 ### json.exception.out_of_range.406
 
-A parsed number could not be stored as without changing it to NaN or INF.
+A parsed number could not be stored without changing it to NaN or INF. For the binary formats, this happens when a
+finite floating-point number does not fit into [`number_float_t`](../api/basic_json/number_float_t.md), for example a
+double-precision number when `number_float_t` is `#!cpp float`.
 
-!!! failure "Example message"
+!!! failure "Example messages"
 
     ```
     number overflow parsing '10E1000'
     ```
+    ```
+    [json.exception.out_of_range.406] syntax error while parsing CBOR value: number overflow
+    ```
 
 ### json.exception.out_of_range.407
 

include/nlohmann/detail/conversions/from_json.hpp

@@ -353,7 +353,7 @@ template < typename BasicJsonType, typename T, std::size_t... Idx >
 std::array<T, sizeof...(Idx)> from_json_inplace_array_impl(BasicJsonType&& j,
                      identity_tag<std::array<T, sizeof...(Idx)>> /*unused*/, index_sequence<Idx...> /*unused*/)
 {
-    return { { j.at(Idx).template get<T>()... } };
+    return { { std::forward<BasicJsonType>(j).at(Idx).template get<T>()... } };
 }
 
 template < typename BasicJsonType, typename T, std::size_t N >
@@ -502,7 +502,7 @@ using tuple_type = std::tuple < decltype(from_json_tuple_get_impl(std::declval<B
 template<std::size_t PTagValue, typename... Args, typename BasicJsonType, std::size_t... Idx>
 tuple_type<PTagValue, BasicJsonType, Args...> from_json_tuple_impl_base(BasicJsonType&& j, index_sequence<Idx...> /*unused*/)
 {
-    return tuple_type<PTagValue, BasicJsonType, Args...>(from_json_tuple_get_impl(j.at(Idx), detail::identity_tag<Args> {}, detail::priority_tag<PTagValue> {})...);
+    return tuple_type<PTagValue, BasicJsonType, Args...>(from_json_tuple_get_impl(std::forward<BasicJsonType>(j).at(Idx), detail::identity_tag<Args> {}, detail::priority_tag<PTagValue> {})...);
 }
 
 template<std::size_t PTagValue, typename BasicJsonType>
@@ -514,8 +514,8 @@ std::tuple<> from_json_tuple_impl_base(BasicJsonType& /*unused*/, index_sequence
 template < typename BasicJsonType, class A1, class A2 >
 std::pair<A1, A2> from_json_tuple_impl(BasicJsonType&& j, identity_tag<std::pair<A1, A2>> /*unused*/, priority_tag<0> /*unused*/)
 {
-    return {j.at(0).template get<A1>(),
-            j.at(1).template get<A2>()};
+    return {std::forward<BasicJsonType>(j).at(0).template get<A1>(),
+            std::forward<BasicJsonType>(j).at(1).template get<A2>()};
 }
 
 template<typename BasicJsonType, typename A1, typename A2>

include/nlohmann/detail/input/binary_reader.hpp

@@ -559,7 +559,7 @@ class binary_reader
             case 0x01: // double
             {
                 double number{};
-                return get_number<double, true>(input_format_t::bson, number) && sax->number_float(static_cast<number_float_t>(number), "");
+                return get_number<double, true>(input_format_t::bson, number) && emit_float(input_format_t::bson, number);
             }
 
             case 0x02: // string
@@ -600,19 +600,19 @@ class binary_reader
             case 0x10: // int32
             {
                 std::int32_t value{};
-                return get_number<std::int32_t, true>(input_format_t::bson, value) && sax->number_integer(value);
+                return get_number<std::int32_t, true>(input_format_t::bson, value) && emit_signed(input_format_t::bson, value);
             }
 
             case 0x12: // int64
             {
                 std::int64_t value{};
-                return get_number<std::int64_t, true>(input_format_t::bson, value) && sax->number_integer(value);
+                return get_number<std::int64_t, true>(input_format_t::bson, value) && emit_signed(input_format_t::bson, value);
             }
 
             case 0x11: // uint64
             {
                 std::uint64_t value{};
-                return get_number<std::uint64_t, true>(input_format_t::bson, value) && sax->number_unsigned(value);
+                return get_number<std::uint64_t, true>(input_format_t::bson, value) && emit_unsigned(input_format_t::bson, value);
             }
 
             default: // anything else is not supported (yet)
@@ -638,14 +638,19 @@ class binary_reader
         {
             return false;
         }
-        const auto max_val = static_cast<NumberType>((std::numeric_limits<number_integer_t>::max)());
-        if (number > max_val)
+
+        // the value is -1 - number, which fits into number_integer_t
+        // whenever number does
+        if (JSON_HEDLEY_LIKELY(value_in_range_of<number_integer_t>(number)))
         {
-            return sax->parse_error(chars_read, get_token_string(),
-                                    parse_error::create(112, chars_read,
-                                            exception_message(input_format_t::cbor, "negative integer overflow", "value"), nullptr));
+            return sax->number_integer(static_cast<number_integer_t>(-1) - static_cast<number_integer_t>(number));
         }
-        return sax->number_integer(static_cast<number_integer_t>(-1) - static_cast<number_integer_t>(number));
+
+        // like the lexer does for JSON text, store a value too small for
+        // number_integer_t as number_float_t; compute it as long double so
+        // that emit_float sees a finite value and can detect an overflow of
+        // number_float_t
+        return emit_float(input_format_t::cbor, static_cast<long double>(-1) - static_cast<long double>(number));
     }
 
     /*!
@@ -702,25 +707,25 @@ class binary_reader
             case 0x18: // Unsigned integer (one-byte uint8_t follows)
             {
                 std::uint8_t number{};
-                return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
+                return get_number(input_format_t::cbor, number) && emit_unsigned(input_format_t::cbor, number);
             }
 
             case 0x19: // Unsigned integer (two-byte uint16_t follows)
             {
                 std::uint16_t number{};
-                return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
+                return get_number(input_format_t::cbor, number) && emit_unsigned(input_format_t::cbor, number);
             }
 
             case 0x1A: // Unsigned integer (four-byte uint32_t follows)
             {
                 std::uint32_t number{};
-                return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
+                return get_number(input_format_t::cbor, number) && emit_unsigned(input_format_t::cbor, number);
             }
 
             case 0x1B: // Unsigned integer (eight-byte uint64_t follows)
             {
                 std::uint64_t number{};
-                return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
+                return get_number(input_format_t::cbor, number) && emit_unsigned(input_format_t::cbor, number);
             }
 
             // Negative integer -1-0x00..-1-0x17 (-1..-24)
@@ -1165,13 +1170,13 @@ class binary_reader
             case 0xFA: // Single-Precision Float (four-byte IEEE 754)
             {
                 float number{};
-                return get_number(input_format_t::cbor, number) && sax->number_float(static_cast<number_float_t>(number), "");
+                return get_number(input_format_t::cbor, number) && emit_float(input_format_t::cbor, number);
             }
 
             case 0xFB: // Double-Precision Float (eight-byte IEEE 754)
             {
                 double number{};
-                return get_number(input_format_t::cbor, number) && sax->number_float(static_cast<number_float_t>(number), "");
+                return get_number(input_format_t::cbor, number) && emit_float(input_format_t::cbor, number);
             }
 
             default: // anything else (0xFF is handled inside the other types)
@@ -1935,61 +1940,61 @@ class binary_reader
             case 0xCA: // float 32
             {
                 float number{};
-                return get_number(input_format_t::msgpack, number) && sax->number_float(static_cast<number_float_t>(number), "");
+                return get_number(input_format_t::msgpack, number) && emit_float(input_format_t::msgpack, number);
             }
 
             case 0xCB: // float 64
             {
                 double number{};
-                return get_number(input_format_t::msgpack, number) && sax->number_float(static_cast<number_float_t>(number), "");
+                return get_number(input_format_t::msgpack, number) && emit_float(input_format_t::msgpack, number);
             }
 
             case 0xCC: // uint 8
             {
                 std::uint8_t number{};
-                return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
+                return get_number(input_format_t::msgpack, number) && emit_unsigned(input_format_t::msgpack, number);
             }
 
             case 0xCD: // uint 16
             {
                 std::uint16_t number{};
-                return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
+                return get_number(input_format_t::msgpack, number) && emit_unsigned(input_format_t::msgpack, number);
             }
 
             case 0xCE: // uint 32
             {
                 std::uint32_t number{};
-                return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
+                return get_number(input_format_t::msgpack, number) && emit_unsigned(input_format_t::msgpack, number);
             }
 
             case 0xCF: // uint 64
             {
                 std::uint64_t number{};
-                return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
+                return get_number(input_format_t::msgpack, number) && emit_unsigned(input_format_t::msgpack, number);
             }
 
             case 0xD0: // int 8
             {
                 std::int8_t number{};
-                return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
+                return get_number(input_format_t::msgpack, number) && emit_signed(input_format_t::msgpack, number);
             }
 
             case 0xD1: // int 16
             {
                 std::int16_t number{};
-                return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
+                return get_number(input_format_t::msgpack, number) && emit_signed(input_format_t::msgpack, number);
             }
 
             case 0xD2: // int 32
             {
                 std::int32_t number{};
-                return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
+                return get_number(input_format_t::msgpack, number) && emit_signed(input_format_t::msgpack, number);
             }
 
             case 0xD3: // int 64
             {
                 std::int64_t number{};
-                return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
+                return get_number(input_format_t::msgpack, number) && emit_signed(input_format_t::msgpack, number);
             }
 
             case 0xDC: // array 16
@@ -2922,7 +2927,7 @@ class binary_reader
                         {
                             return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, exception_message(input_format, "excessive ndarray size caused overflow", "size"), nullptr));
                         }
-                        if (JSON_HEDLEY_UNLIKELY(!sax->number_unsigned(static_cast<number_unsigned_t>(i))))
+                        if (JSON_HEDLEY_UNLIKELY(!emit_unsigned(input_format, i)))
                         {
                             return false;
                         }
@@ -3054,37 +3059,37 @@ class binary_reader
                     break;
                 }
                 std::uint8_t number{};
-                return get_number(input_format, number) && sax->number_unsigned(number);
+                return get_number(input_format, number) && emit_unsigned(input_format, number);
             }
 
             case 'U':
             {
                 std::uint8_t number{};
-                return get_number(input_format, number) && sax->number_unsigned(number);
+                return get_number(input_format, number) && emit_unsigned(input_format, number);
             }
 
             case 'i':
             {
                 std::int8_t number{};
-                return get_number(input_format, number) && sax->number_integer(number);
+                return get_number(input_format, number) && emit_signed(input_format, number);
             }
 
             case 'I':
             {
                 std::int16_t number{};
-                return get_number(input_format, number) && sax->number_integer(number);
+                return get_number(input_format, number) && emit_signed(input_format, number);
             }
 
             case 'l':
             {
                 std::int32_t number{};
-                return get_number(input_format, number) && sax->number_integer(number);
+                return get_number(input_format, number) && emit_signed(input_format, number);
             }
 
             case 'L':
             {
                 std::int64_t number{};
-                return get_number(input_format, number) && sax->number_integer(number);
+                return get_number(input_format, number) && emit_signed(input_format, number);
             }
 
             case 'u':
@@ -3094,7 +3099,7 @@ class binary_reader
                     break;
                 }
                 std::uint16_t number{};
-                return get_number(input_format, number) && sax->number_unsigned(number);
+                return get_number(input_format, number) && emit_unsigned(input_format, number);
             }
 
             case 'm':
@@ -3104,7 +3109,7 @@ class binary_reader
                     break;
                 }
                 std::uint32_t number{};
-                return get_number(input_format, number) && sax->number_unsigned(number);
+                return get_number(input_format, number) && emit_unsigned(input_format, number);
             }
 
             case 'M':
@@ -3114,7 +3119,7 @@ class binary_reader
                     break;
                 }
                 std::uint64_t number{};
-                return get_number(input_format, number) && sax->number_unsigned(number);
+                return get_number(input_format, number) && emit_unsigned(input_format, number);
             }
 
             case 'h':
@@ -3172,13 +3177,13 @@ class binary_reader
             case 'd':
             {
                 float number{};
-                return get_number(input_format, number) && sax->number_float(static_cast<number_float_t>(number), "");
+                return get_number(input_format, number) && emit_float(input_format, number);
             }
 
             case 'D':
             {
                 double number{};
-                return get_number(input_format, number) && sax->number_float(static_cast<number_float_t>(number), "");
+                return get_number(input_format, number) && emit_float(input_format, number);
             }
 
             case 'H':
@@ -3645,13 +3650,13 @@ class binary_reader
             case 0x8E: // binary32
             {
                 float number{};
-                return get_number(input_format_t::bon8, number) && sax->number_float(static_cast<number_float_t>(number), "");
+                return get_number(input_format_t::bon8, number) && emit_float(input_format_t::bon8, number);
             }
 
             case 0x8F: // binary64
             {
                 double number{};
-                return get_number(input_format_t::bon8, number) && sax->number_float(static_cast<number_float_t>(number), "");
+                return get_number(input_format_t::bon8, number) && emit_float(input_format_t::bon8, number);
             }
 
             case 0xF8:
@@ -3717,7 +3722,9 @@ class binary_reader
     @brief pass an integer to the SAX parser
 
     Non-negative integers are passed as unsigned, negative integers as signed
-    numbers, like the other binary formats do.
+    numbers, like the other binary formats do. A value that does not fit the
+    number type is passed as described for @ref emit_unsigned and
+    @ref emit_signed.
 
     @param[in] number  the integer
     @return whether the SAX parser accepted the value
@@ -3726,9 +3733,9 @@ class binary_reader
     {
         if (number >= 0)
         {
-            return sax->number_unsigned(static_cast<number_unsigned_t>(number));
+            return emit_unsigned(input_format_t::bon8, static_cast<std::uint64_t>(number));
         }
-        return sax->number_integer(static_cast<number_integer_t>(number));
+        return emit_signed(input_format_t::bon8, number);
     }
 
     /*!
@@ -3785,8 +3792,7 @@ class binary_reader
             value = (value << 8) | static_cast<std::int64_t>(current);
         }
 
-        return negative ? sax->number_integer(static_cast<number_integer_t>(-(value + offset)))
-               : sax->number_unsigned(static_cast<number_unsigned_t>(value + offset));
+        return emit_bon8_integer(negative ? -(value + offset) : value + offset);
     }
 
     /*!
@@ -4083,6 +4089,88 @@ class binary_reader
         return true;
     }
 
+    /*!
+    @brief pass a signed integer read from the input to the SAX parser
+
+    Like the lexer does for JSON text, a value that does not fit into
+    number_integer_t is passed as number_unsigned_t if it is non-negative and
+    fits there, and as number_float_t otherwise. With the default number
+    types, every integer the binary formats can encode fits, so this only
+    matters for narrower custom number types.
+
+    @tparam NumberType a signed integer type
+    @param[in] format  the current format (for diagnostics)
+    @param[in] number  the integer
+    @return whether the SAX parser accepted the value
+
+    @throw out_of_range.406 if @a number overflows number_float_t (see
+           @ref emit_float)
+    */
+    template<typename NumberType>
+    bool emit_signed(const input_format_t format, const NumberType number)
+    {
+        if (JSON_HEDLEY_LIKELY(value_in_range_of<number_integer_t>(number)))
+        {
+            return sax->number_integer(static_cast<number_integer_t>(number));
+        }
+        if (value_in_range_of<number_unsigned_t>(number))
+        {
+            return sax->number_unsigned(static_cast<number_unsigned_t>(number));
+        }
+        return emit_float(format, number);
+    }
+
+    /*!
+    @brief pass an unsigned integer read from the input to the SAX parser
+
+    Like the lexer does for JSON text, a value that does not fit into
+    number_unsigned_t is passed as number_float_t.
+
+    @tparam NumberType an unsigned integer type
+    @param[in] format  the current format (for diagnostics)
+    @param[in] number  the integer
+    @return whether the SAX parser accepted the value
+
+    @throw out_of_range.406 if @a number overflows number_float_t (see
+           @ref emit_float)
+    */
+    template<typename NumberType>
+    bool emit_unsigned(const input_format_t format, const NumberType number)
+    {
+        if (JSON_HEDLEY_LIKELY(value_in_range_of<number_unsigned_t>(number)))
+        {
+            return sax->number_unsigned(static_cast<number_unsigned_t>(number));
+        }
+        return emit_float(format, number);
+    }
+
+    /*!
+    @brief pass a floating-point number read from the input to the SAX parser
+
+    Like the lexer does for JSON text, a finite value that overflows
+    number_float_t is rejected instead of silently becoming infinity. Infinity
+    and NaN in the input are passed on unchanged. Integers only overflow if
+    number_float_t cannot represent 2^64, e.g., a half-precision type.
+
+    @tparam NumberType a floating-point or integer type
+    @param[in] format  the current format (for diagnostics)
+    @param[in] number  the number
+    @return whether the SAX parser accepted the value
+
+    @throw out_of_range.406 if a finite @a number overflows number_float_t
+    */
+    template<typename NumberType>
+    bool emit_float(const input_format_t format, const NumberType number)
+    {
+        const auto result = static_cast<number_float_t>(number);
+        if (JSON_HEDLEY_UNLIKELY(std::isfinite(number) && !std::isfinite(result)))
+        {
+            return sax->parse_error(chars_read, get_token_string(),
+                                    out_of_range::create(406, exception_message(format, "number overflow", "value"), nullptr));
+        }
+        return sax->number_float(result, "");
+    }
+
     /*!
     @brief create a string by reading characters from the input
 

include/nlohmann/detail/input/input_adapters.hpp

@@ -763,7 +763,6 @@ struct container_input_adapter_factory< ContainerType,
 
            static adapter_type create(ContainerType&& container)
 {
-    // NOLINTNEXTLINE(bugprone-use-after-move,hicpp-invalid-access-moved) forwarded twice on purpose, so begin() and end() see the same value category and yield matching iterator types
     return input_adapter(begin(std::forward<ContainerType>(container)), end(std::forward<ContainerType>(container)));
 }
        };

include/nlohmann/detail/string_concat.hpp

@@ -39,6 +39,7 @@ inline std::size_t concat_length(const char /*c*/, const Args& ... rest)
 template<typename... Args>
 inline std::size_t concat_length(const char* cstr, const Args& ... rest)
 {
+    // cppcheck-suppress ignoredReturnValue
     return ::strlen(cstr) + concat_length(rest...);
 }
 

include/nlohmann/ordered_map.hpp

@@ -70,41 +70,14 @@ template <class Key, class T, class IgnoredLess = std::less<Key>,
         return *this;
     }
 
-private:
-    /// @brief find the entry for @a key, for either constness of @a self
-    /// @note the single place that performs the linear key search
-    template<typename Self, typename KeyType>
-    static auto find_impl(Self& self, KeyType&& key) -> decltype(self.begin())
-    {
-        for (auto it = self.begin(); it != self.end(); ++it)
-        {
-            if (self.m_compare(it->first, key))
-            {
-                return it;
-            }
-        }
-        return self.end();
-    }
-
-    /// @brief remove the entry @a it points to, preserving order
-    /// @note keys are not movable, so the tail is destroyed and re-constructed in place
-    void erase_at(iterator it)
-    {
-        for (auto next = it; ++next != this->end(); ++it)
-        {
-            it->~value_type(); // Destroy but keep allocation
-            new (&*it) value_type{std::move(*next)};
-        }
-        Container::pop_back();
-    }
-
-public:
     std::pair<iterator, bool> emplace(const key_type& key, T&& t)
     {
-        const auto it = find_impl(*this, key);
-        if (it != this->end())
+        for (auto it = this->begin(); it != this->end(); ++it)
         {
-            return {it, false};
+            if (m_compare(it->first, key))
+            {
+                return {it, false};
+            }
         }
         Container::emplace_back(key, std::forward<T>(t));
         return {std::prev(this->end()), true};
@@ -114,10 +87,12 @@ public:
                  detail::is_usable_as_key_type<key_compare, key_type, KeyType>::value, int> = 0>
     std::pair<iterator, bool> emplace(KeyType && key, T && t)
     {
-        const auto it = find_impl(*this, key);
-        if (it != this->end())
+        for (auto it = this->begin(); it != this->end(); ++it)
         {
-            return {it, false};
+            if (m_compare(it->first, key))
+            {
+                return {it, false};
+            }
         }
         Container::emplace_back(std::forward<KeyType>(key), std::forward<T>(t));
         return {std::prev(this->end()), true};
@@ -149,55 +124,75 @@ public:
 
     T& at(const key_type& key)
     {
-        const auto it = find_impl(*this, key);
-        if (it == this->end())
+        for (auto it = this->begin(); it != this->end(); ++it)
         {
-            JSON_THROW(std::out_of_range("key not found"));
+            if (m_compare(it->first, key))
+            {
+                return it->second;
+            }
         }
-        return it->second;
+
+        JSON_THROW(std::out_of_range("key not found"));
     }
 
     template<class KeyType, detail::enable_if_t<
                  detail::is_usable_as_key_type<key_compare, key_type, KeyType>::value, int> = 0>
     T & at(KeyType && key) // NOLINT(cppcoreguidelines-missing-std-forward)
     {
-        const auto it = find_impl(*this, key);
-        if (it == this->end())
+        for (auto it = this->begin(); it != this->end(); ++it)
         {
-            JSON_THROW(std::out_of_range("key not found"));
+            if (m_compare(it->first, key))
+            {
+                return it->second;
+            }
         }
-        return it->second;
+
+        JSON_THROW(std::out_of_range("key not found"));
     }
 
     const T& at(const key_type& key) const
     {
-        const auto it = find_impl(*this, key);
-        if (it == this->end())
+        for (auto it = this->begin(); it != this->end(); ++it)
         {
-            JSON_THROW(std::out_of_range("key not found"));
+            if (m_compare(it->first, key))
+            {
+                return it->second;
+            }
         }
-        return it->second;
+
+        JSON_THROW(std::out_of_range("key not found"));
     }
 
     template<class KeyType, detail::enable_if_t<
                  detail::is_usable_as_key_type<key_compare, key_type, KeyType>::value, int> = 0>
     const T & at(KeyType && key) const // NOLINT(cppcoreguidelines-missing-std-forward)
     {
-        const auto it = find_impl(*this, key);
-        if (it == this->end())
+        for (auto it = this->begin(); it != this->end(); ++it)
         {
-            JSON_THROW(std::out_of_range("key not found"));
+            if (m_compare(it->first, key))
+            {
+                return it->second;
+            }
         }
-        return it->second;
+
+        JSON_THROW(std::out_of_range("key not found"));
     }
 
     size_type erase(const key_type& key)
     {
-        const auto it = find_impl(*this, key);
-        if (it != this->end())
+        for (auto it = this->begin(); it != this->end(); ++it)
         {
-            erase_at(it);
-            return 1;
+            if (m_compare(it->first, key))
+            {
+                // Since we cannot move const Keys, re-construct them in place
+                for (auto next = it; ++next != this->end(); ++it)
+                {
+                    it->~value_type(); // Destroy but keep allocation
+                    new (&*it) value_type{std::move(*next)};
+                }
+                Container::pop_back();
+                return 1;
+            }
         }
         return 0;
     }
@@ -206,11 +201,19 @@ public:
                  detail::is_usable_as_key_type<key_compare, key_type, KeyType>::value, int> = 0>
     size_type erase(KeyType && key) // NOLINT(cppcoreguidelines-missing-std-forward)
     {
-        const auto it = find_impl(*this, key);
-        if (it != this->end())
+        for (auto it = this->begin(); it != this->end(); ++it)
         {
-            erase_at(it);
-            return 1;
+            if (m_compare(it->first, key))
+            {
+                // Since we cannot move const Keys, re-construct them in place
+                for (auto next = it; ++next != this->end(); ++it)
+                {
+                    it->~value_type(); // Destroy but keep allocation
+                    new (&*it) value_type{std::move(*next)};
+                }
+                Container::pop_back();
+                return 1;
+            }
         }
         return 0;
     }
@@ -275,38 +278,80 @@ public:
 
     size_type count(const key_type& key) const
     {
-        return find_impl(*this, key) != this->end() ? 1 : 0;
+        for (auto it = this->begin(); it != this->end(); ++it)
+        {
+            if (m_compare(it->first, key))
+            {
+                return 1;
+            }
+        }
+        return 0;
     }
 
     template<class KeyType, detail::enable_if_t<
                  detail::is_usable_as_key_type<key_compare, key_type, KeyType>::value, int> = 0>
     size_type count(KeyType && key) const // NOLINT(cppcoreguidelines-missing-std-forward)
     {
-        return find_impl(*this, key) != this->end() ? 1 : 0;
+        for (auto it = this->begin(); it != this->end(); ++it)
+        {
+            if (m_compare(it->first, key))
+            {
+                return 1;
+            }
+        }
+        return 0;
     }
 
     iterator find(const key_type& key)
     {
-        return find_impl(*this, key);
+        for (auto it = this->begin(); it != this->end(); ++it)
+        {
+            if (m_compare(it->first, key))
+            {
+                return it;
+            }
+        }
+        return Container::end();
     }
 
     template<class KeyType, detail::enable_if_t<
                  detail::is_usable_as_key_type<key_compare, key_type, KeyType>::value, int> = 0>
     iterator find(KeyType && key) // NOLINT(cppcoreguidelines-missing-std-forward)
     {
-        return find_impl(*this, key);
+        for (auto it = this->begin(); it != this->end(); ++it)
+        {
+            if (m_compare(it->first, key))
+            {
+                return it;
+            }
+        }
+        return Container::end();
     }
 
     const_iterator find(const key_type& key) const
     {
-        return find_impl(*this, key);
+        for (auto it = this->begin(); it != this->end(); ++it)
+        {
+            if (m_compare(it->first, key))
+            {
+                return it;
+            }
+        }
+        return Container::end();
     }
 
     template<class KeyType, detail::enable_if_t<
                  detail::is_usable_as_key_type<key_compare, key_type, KeyType>::value, int> = 0>
     const_iterator find(KeyType && key) const // NOLINT(cppcoreguidelines-missing-std-forward)
     {
-        return find_impl(*this, key);
+        for (auto it = this->begin(); it != this->end(); ++it)
+        {
+            if (m_compare(it->first, key))
+            {
+                return it;
+            }
+        }
+        return Container::end();
     }
 
     std::pair<iterator, bool> insert( value_type&& value )
@@ -316,10 +361,12 @@ public:
 
     std::pair<iterator, bool> insert( const value_type& value )
     {
-        const auto it = find_impl(*this, value.first);
-        if (it != this->end())
+        for (auto it = this->begin(); it != this->end(); ++it)
         {
-            return {it, false};
+            if (m_compare(it->first, value.first))
+            {
+                return {it, false};
+            }
         }
         Container::push_back(value);
         return {--this->end(), true};

tests/src/unit-binary_formats.cpp

@@ -11,7 +11,12 @@
 #include <nlohmann/json.hpp>
 using nlohmann::json;
 
+#include <cmath>
 #include <fstream>
+#include <limits>
+#include <map>
+#include <string>
+#include <vector>
 #include "make_test_data_available.hpp"
 
 TEST_CASE("Binary Formats" * doctest::skip())
@@ -224,3 +229,139 @@ TEST_CASE("Binary Formats" * doctest::skip())
         CHECK((100.0 * double(ubjson_3_size) / double(json_size)) == Approx(89.450));
     }
 }
+
+namespace
+{
+// the binary formats as function pointers for "Binary formats with narrow number types";
+// named functions rather than lambdas, because clang 3.5 cannot convert a lambda
+// to a function pointer in the braced initializer of the format table
+using narrow_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int32_t, std::uint32_t, float>;
+using bytes = std::vector<std::uint8_t>;
+
+bytes encode_cbor(const json& j)
+{
+    return json::to_cbor(j);
+}
+narrow_json decode_cbor(const bytes& v, bool allow_exceptions)
+{
+    return narrow_json::from_cbor(v, true, allow_exceptions);
+}
+
+bytes encode_msgpack(const json& j)
+{
+    return json::to_msgpack(j);
+}
+narrow_json decode_msgpack(const bytes& v, bool allow_exceptions)
+{
+    return narrow_json::from_msgpack(v, true, allow_exceptions);
+}
+
+bytes encode_ubjson(const json& j)
+{
+    return json::to_ubjson(j);
+}
+narrow_json decode_ubjson(const bytes& v, bool allow_exceptions)
+{
+    return narrow_json::from_ubjson(v, true, allow_exceptions);
+}
+
+bytes encode_bjdata(const json& j)
+{
+    return json::to_bjdata(j);
+}
+narrow_json decode_bjdata(const bytes& v, bool allow_exceptions)
+{
+    return narrow_json::from_bjdata(v, true, allow_exceptions);
+}
+
+// BSON can only store numbers as object members
+bytes encode_bson(const json& j)
+{
+    return json::to_bson(json{{"a", j}});
+}
+narrow_json decode_bson(const bytes& v, bool allow_exceptions)
+{
+    const auto result = narrow_json::from_bson(v, true, allow_exceptions);
+    return result.is_discarded() ? result : result.at("a");
+}
+
+bytes encode_bon8(const json& j)
+{
+    return json::to_bon8(j);
+}
+narrow_json decode_bon8(const bytes& v, bool allow_exceptions)
+{
+    return narrow_json::from_bon8(v, true, allow_exceptions);
+}
+
+} // namespace
+
+TEST_CASE("Binary formats with narrow number types")
+{
+    // Numbers that do not fit the number types are handled like the lexer
+    // handles them in JSON text: an integer that fits neither integer type is
+    // stored as a floating-point number, and a finite floating-point number
+    // that overflows number_float_t is rejected with out_of_range.406.
+    struct binary_format
+    {
+        const char* name;
+        bytes (*encode)(const json&);
+        narrow_json (*decode)(const bytes&, bool);
+    };
+
+    const std::vector<binary_format> formats =
+    {
+        {"CBOR", encode_cbor, decode_cbor},
+        {"MessagePack", encode_msgpack, decode_msgpack},
+        {"UBJSON", encode_ubjson, decode_ubjson},
+        {"BJData", encode_bjdata, decode_bjdata},
+        {"BSON", encode_bson, decode_bson},
+        {"BON8", encode_bon8, decode_bon8},
+    };
+
+    for (const auto& format : formats)
+    {
+        const std::string name = format.name;
+        INFO("format := ", name);
+        const auto roundtrip = [&format](const json & j)
+        {
+            return format.decode(format.encode(j), true);
+        };
+
+        // integers that fit keep their type
+        CHECK(roundtrip(json(-5)).is_number_integer());
+        CHECK(roundtrip(json(-5)).get<std::int32_t>() == -5);
+        CHECK(roundtrip(json(3000000000u)).is_number_unsigned());
+        CHECK(roundtrip(json(3000000000u)).get<std::uint32_t>() == 3000000000u);
+
+        // integers that fit neither integer type are stored as float
+        CHECK(roundtrip(json(5000000000u)).is_number_float());
+        CHECK(roundtrip(json(5000000000u)).get<float>() == 5000000000.0f);
+        if (name != "BON8") // BON8 cannot encode integers above INT64_MAX
+        {
+            CHECK(roundtrip(json(10000000000000000000u)).is_number_float());
+            CHECK(roundtrip(json(10000000000000000000u)).get<float>() == 10000000000000000000.0f);
+        }
+        CHECK(roundtrip(json(-3000000000LL)).is_number_float());
+        CHECK(roundtrip(json(-3000000000LL)).get<float>() == -3000000000.0f);
+        CHECK(roundtrip(json(-5000000000LL)).is_number_float());
+        CHECK(roundtrip(json(-5000000000LL)).get<float>() == -5000000000.0f);
+
+        // floating-point numbers that fit
+        CHECK(roundtrip(json(1.5)).get<float>() == 1.5f);
+        const auto just_above_max = std::nextafter(static_cast<double>((std::numeric_limits<float>::max)()),
+                                    std::numeric_limits<double>::infinity());
+        CHECK(roundtrip(json(just_above_max)).get<float>() == (std::numeric_limits<float>::max)());
+
+        // infinity and NaN are passed on
+        CHECK(std::isinf(roundtrip(json(std::numeric_limits<double>::infinity())).get<float>()));
+        CHECK(std::isnan(roundtrip(json(std::numeric_limits<double>::quiet_NaN())).get<float>()));
+
+        // finite floating-point numbers that overflow number_float_t are rejected
+        const std::string message = "[json.exception.out_of_range.406] syntax error while parsing " + name
+                                    + " value: number overflow";
+        CHECK_THROWS_WITH_AS(roundtrip(json(1e300)), message.c_str(), narrow_json::out_of_range&);
+        CHECK_THROWS_WITH_AS(roundtrip(json(-1e300)), message.c_str(), narrow_json::out_of_range&);
+        CHECK(format.decode(format.encode(json(1e300)), false).is_discarded());
+    }
+}

tests/src/unit-cbor.cpp

@@ -3187,7 +3187,8 @@ TEST_CASE("Tagged values")
         // CBOR encodes negative integers as: result = -1 - n
         // For type 0x3B, n is an 8-byte uint64_t. Valid range for n with
         // the default int64_t is [0, INT64_MAX], producing results in [INT64_MIN, -1].
-        // When n > INT64_MAX, the result exceeds int64_t range and is rejected.
+        // When n > INT64_MAX, the result exceeds int64_t range and is stored
+        // as a floating-point number, as the lexer does for JSON text.
 
         SECTION("n = 0 is valid (result = -1)")
         {
@@ -3208,33 +3209,34 @@ TEST_CASE("Tagged values")
             CHECK(result.get<int64_t>() == (std::numeric_limits<int64_t>::min)());
         }
 
-        SECTION("n = INT64_MAX + 1 is rejected (overflow)")
+        SECTION("n = INT64_MAX + 1 is stored as float")
         {
             // n = INT64_MAX + 1 (0x8000000000000000)
-            // result = -1 - n = -9223372036854775809, which exceeds int64_t range
+            // result = -1 - n = -9223372036854775809, which exceeds int64_t range;
+            // the nearest double is -9223372036854775808.0
             const std::vector<uint8_t> input = {0x3B, 0x80, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00};
-            json _;
-            CHECK_THROWS_WITH_AS(_ = json::from_cbor(input),
-                                 "[json.exception.parse_error.112] parse error at byte 9: syntax error while parsing CBOR value: negative integer overflow",
-                                 json::parse_error);
+            const auto result = json::from_cbor(input);
+            CHECK(result.is_number_float());
+            CHECK(result.get<double>() == -9223372036854775808.0);
+            CHECK(result == json::parse("-9223372036854775809"));
         }
 
-        SECTION("n = UINT64_MAX is rejected (overflow)")
+        SECTION("n = UINT64_MAX is stored as float")
         {
             // n = UINT64_MAX (0xFFFFFFFFFFFFFFFF)
             // result = -1 - n = -18446744073709551616, which exceeds int64_t range
             const std::vector<uint8_t> input = {0x3B, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF};
-            json _;
-            CHECK_THROWS_WITH_AS(_ = json::from_cbor(input),
-                                 "[json.exception.parse_error.112] parse error at byte 9: syntax error while parsing CBOR value: negative integer overflow",
-                                 json::parse_error);
+            const auto result = json::from_cbor(input);
+            CHECK(result.is_number_float());
+            CHECK(result.get<double>() == -18446744073709551616.0);
+            CHECK(result == json::parse("-18446744073709551616"));
         }
 
-        SECTION("overflow with allow_exceptions=false returns discarded")
+        SECTION("overflow with allow_exceptions=false is not an error")
         {
             const std::vector<uint8_t> input = {0x3B, 0x80, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00};
             const auto result = json::from_cbor(input, true, false);
-            CHECK(result.is_discarded());
+            CHECK(result.is_number_float());
         }
     }
 

tests/src/unit-class_parser.cpp

@@ -2525,10 +2525,12 @@ TEST_CASE("diagnostic positions: value lifetime, input adapters, and SAX")
 
         SECTION("move constructor resets the moved-from value to npos")
         {
-            // basic_json(basic_json&&) (json.hpp, around line 1944) copies
+            // basic_json(basic_json&&) (json.hpp, around line 1265) copies
             // other's start_position/end_position into *this and then resets
-            // other's to npos. Only the top-level moved-from value is
-            // affected; its (moved-away) children are gone along with it.
+            // other's to npos (see the cppcheck-suppress[accessForwarded]
+            // annotation there, which flags this reset as worth a second
+            // look). Only the top-level moved-from value is affected; its
+            // (moved-away) children are gone along with it.
             const std::string s = R"({"a":1,"b":[1,2,3]})";
             json a = json::parse(s);
             const auto a_start = a.start_pos();

tests/src/unit-modifiers.cpp

@@ -618,49 +618,6 @@ TEST_CASE("modifiers")
             }
         }
 
-        SECTION("rvalue at position moves rather than copies")
-        {
-            // regression test: insert(pos, basic_json&&) used to forward to
-            // insert(pos, const basic_json&) because the named rvalue
-            // reference parameter is itself an lvalue, so it always
-            // deep-copied its argument instead of moving it
-            json j_big = std::string(1000, 'x');
-            const auto* const original_buffer = j_big.get_ref<const std::string&>().data();
-
-            auto it = j_array.insert(j_array.begin(), std::move(j_big));
-            CHECK(j_array.size() == 5);
-            CHECK(*it == json(std::string(1000, 'x')));
-            CHECK((*it).get_ref<const std::string&>().data() == original_buffer);
-
-            // the moved-from value is null, the same as after push_back(&&)
-            CHECK(j_big.is_null()); // NOLINT(bugprone-use-after-move,hicpp-invalid-access-moved)
-        }
-
-        SECTION("self-aliasing insertion")
-        {
-            SECTION("without reallocation")
-            {
-                json j_self = {1, 2, 3, 4};
-                j_self.get_ref<json::array_t&>().reserve(j_self.size() + 1);
-
-                auto it = j_self.insert(j_self.begin(), std::move(j_self[1]));
-                CHECK(j_self.size() == 5);
-                CHECK(*it == json(2));
-                CHECK(j_self == json({2, 1, nullptr, 3, 4}));
-            }
-
-            SECTION("with reallocation")
-            {
-                json j_self = {1, 2, 3, 4};
-                j_self.get_ref<json::array_t&>().shrink_to_fit();
-
-                auto it = j_self.insert(j_self.begin(), std::move(j_self[1]));
-                CHECK(j_self.size() == 5);
-                CHECK(*it == json(2));
-                CHECK(j_self == json({2, 1, nullptr, 3, 4}));
-            }
-        }
-
         SECTION("copies at position")
         {
             SECTION("insert before begin()")