Fix -Weffc++ errors in the #5198 locale test

GCC's -Weffc++ (an error in ci_test_gcc and ci_test_standards_gcc)
rejected LocaleSwitchingSax: it has a pointer data member but does not
declare its copy operations, and its vectors are not initialized in the
member initializer list. Store the locale name as a std::string and give
the vectors brace initializers, like SaxEventLogger in
unit-deserialization.cpp.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

docs/mkdocs/docs/api/basic_json/number_float_t.md

@@ -55,10 +55,6 @@ This implementation does exactly follow this approach, as it uses double precisi
 smaller than `-1.79769313486232e+308` and values greater than `1.79769313486232e+308` will be stored as NaN internally
 and be serialized to `null`.
 
-During deserialization (from JSON text or any of the binary formats), a finite number that does not fit into
-`number_float_t` is rejected with [`out_of_range.406`](../../home/exceptions.md#jsonexceptionout_of_range406), for
-example a double-precision number in a binary format when `number_float_t` is `#!cpp float`.
-
 #### Storage
 
 Floating-point number values are stored directly inside a `basic_json` type.

docs/mkdocs/docs/api/basic_json/number_integer_t.md

@@ -47,9 +47,8 @@ With the default values for `NumberIntegerType` (`std::int64_t`), the default va
 
 When the default type is used, the maximal integer number that can be stored is `9223372036854775807` (INT64_MAX) and
 the minimal integer number that can be stored is `-9223372036854775808` (INT64_MIN). Integer numbers that are out of
-range will yield over/underflow when used in a constructor. During deserialization (from JSON text or any of the binary
-formats), too large or small integer numbers will automatically be stored as [`number_unsigned_t`](number_unsigned_t.md)
-or [`number_float_t`](number_float_t.md).
+range will yield over/underflow when used in a constructor. During deserialization, too large or small integer numbers
+will automatically be stored as [`number_unsigned_t`](number_unsigned_t.md) or [`number_float_t`](number_float_t.md).
 
 [RFC 8259](https://tools.ietf.org/html/rfc8259) further states:
 > Note that when such software is used, numbers that are integers and are in the range $[-2^{53}+1, 2^{53}-1]$ are

docs/mkdocs/docs/api/basic_json/number_unsigned_t.md

@@ -48,9 +48,8 @@ With the default values for `NumberUnsignedType` (`std::uint64_t`), the default
 
 When the default type is used, the maximal integer number that can be stored is `18446744073709551615` (UINT64_MAX) and
 the minimal integer number that can be stored is `0`. Integer numbers that are out of range will yield over/underflow
-when used in a constructor. During deserialization (from JSON text or any of the binary formats), too large or small
-integer numbers will automatically be stored as [`number_integer_t`](number_integer_t.md) or
-[`number_float_t`](number_float_t.md).
+when used in a constructor. During deserialization, too large or small integer numbers will automatically be stored
+as [`number_integer_t`](number_integer_t.md) or [`number_float_t`](number_float_t.md).
 
 [RFC 8259](https://tools.ietf.org/html/rfc8259) further states:
 > Note that when such software is used, numbers that are integers and are in the range $[-2^{53}+1, 2^{53}-1]$ are

docs/mkdocs/docs/features/binary_formats/cbor.md

@@ -168,9 +168,9 @@ The library maps CBOR types to JSON value types as follows:
 !!! warning "Negative integer overflow"
 
     CBOR negative integers (major type 1) are decoded as `-1 - n`. If the encoded magnitude `n` is too large for the
-    result to fit into `number_integer_t` (`std::int64_t` by default), the result is stored as `number_float_t`, like
-    a too small integer in JSON text. For example, `-18446744073709551616` (`0x3B` followed by eight `0xFF` bytes) is
-    stored as `-1.8446744073709552e+19`.
+    result to fit into `number_integer_t` (`std::int64_t` by default), parsing fails with a
+    [`parse_error.112`](../../home/exceptions.md#jsonexceptionparse_error112) exception rather than overflowing
+    silently.
 
 !!! warning "Object keys"
 

docs/mkdocs/docs/home/exceptions.md

@@ -331,6 +331,9 @@ An unexpected byte was read in a [binary format](../features/binary_formats/inde
     [json.exception.parse_error.112] parse error at byte 15: syntax error while parsing BSON binary: byte array length cannot be negative, is -1
     ```
     ```
+    [json.exception.parse_error.112] parse error at byte 9: syntax error while parsing CBOR value: negative integer overflow
+    ```
+    ```
     [json.exception.parse_error.112] parse error at byte 5: syntax error while parsing BSON document: document size 6 does not match the number of bytes read (5)
     ```
 
@@ -844,18 +847,13 @@ The JSON Patch operations 'remove' and 'add' cannot be applied to the root eleme
 
 ### json.exception.out_of_range.406
 
-A parsed number could not be stored without changing it to NaN or INF. For the binary formats, this happens when a
-finite floating-point number does not fit into [`number_float_t`](../api/basic_json/number_float_t.md), for example a
-double-precision number when `number_float_t` is `#!cpp float`.
+A parsed number could not be stored as without changing it to NaN or INF.
 
-!!! failure "Example messages"
+!!! failure "Example message"
 
     ```
     number overflow parsing '10E1000'
     ```
-    ```
-    [json.exception.out_of_range.406] syntax error while parsing CBOR value: number overflow
-    ```
 
 ### json.exception.out_of_range.407
 

include/nlohmann/detail/input/binary_reader.hpp

@@ -559,7 +559,7 @@ class binary_reader
             case 0x01: // double
             {
                 double number{};
-                return get_number<double, true>(input_format_t::bson, number) && emit_float(input_format_t::bson, number);
+                return get_number<double, true>(input_format_t::bson, number) && sax->number_float(static_cast<number_float_t>(number), "");
             }
 
             case 0x02: // string
@@ -600,19 +600,19 @@ class binary_reader
             case 0x10: // int32
             {
                 std::int32_t value{};
-                return get_number<std::int32_t, true>(input_format_t::bson, value) && emit_signed(value);
+                return get_number<std::int32_t, true>(input_format_t::bson, value) && sax->number_integer(value);
             }
 
             case 0x12: // int64
             {
                 std::int64_t value{};
-                return get_number<std::int64_t, true>(input_format_t::bson, value) && emit_signed(value);
+                return get_number<std::int64_t, true>(input_format_t::bson, value) && sax->number_integer(value);
             }
 
             case 0x11: // uint64
             {
                 std::uint64_t value{};
-                return get_number<std::uint64_t, true>(input_format_t::bson, value) && emit_unsigned(value);
+                return get_number<std::uint64_t, true>(input_format_t::bson, value) && sax->number_unsigned(value);
             }
 
             default: // anything else is not supported (yet)
@@ -638,17 +638,14 @@ class binary_reader
         {
             return false;
         }
-
-        // the value is -1 - number, which fits into number_integer_t
-        // whenever number does
-        if (JSON_HEDLEY_LIKELY(value_in_range_of<number_integer_t>(number)))
+        const auto max_val = static_cast<NumberType>((std::numeric_limits<number_integer_t>::max)());
+        if (number > max_val)
         {
-            return sax->number_integer(static_cast<number_integer_t>(-1) - static_cast<number_integer_t>(number));
+            return sax->parse_error(chars_read, get_token_string(),
+                                    parse_error::create(112, chars_read,
+                                            exception_message(input_format_t::cbor, "negative integer overflow", "value"), nullptr));
         }
-
-        // like the lexer does for JSON text, store a value too small for
-        // number_integer_t as number_float_t
-        return sax->number_float(static_cast<number_float_t>(-1) - static_cast<number_float_t>(number), "");
+        return sax->number_integer(static_cast<number_integer_t>(-1) - static_cast<number_integer_t>(number));
     }
 
     /*!
@@ -705,25 +702,25 @@ class binary_reader
             case 0x18: // Unsigned integer (one-byte uint8_t follows)
             {
                 std::uint8_t number{};
-                return get_number(input_format_t::cbor, number) && emit_unsigned(number);
+                return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
             }
 
             case 0x19: // Unsigned integer (two-byte uint16_t follows)
             {
                 std::uint16_t number{};
-                return get_number(input_format_t::cbor, number) && emit_unsigned(number);
+                return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
             }
 
             case 0x1A: // Unsigned integer (four-byte uint32_t follows)
             {
                 std::uint32_t number{};
-                return get_number(input_format_t::cbor, number) && emit_unsigned(number);
+                return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
             }
 
             case 0x1B: // Unsigned integer (eight-byte uint64_t follows)
             {
                 std::uint64_t number{};
-                return get_number(input_format_t::cbor, number) && emit_unsigned(number);
+                return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
             }
 
             // Negative integer -1-0x00..-1-0x17 (-1..-24)
@@ -1168,13 +1165,13 @@ class binary_reader
             case 0xFA: // Single-Precision Float (four-byte IEEE 754)
             {
                 float number{};
-                return get_number(input_format_t::cbor, number) && emit_float(input_format_t::cbor, number);
+                return get_number(input_format_t::cbor, number) && sax->number_float(static_cast<number_float_t>(number), "");
             }
 
             case 0xFB: // Double-Precision Float (eight-byte IEEE 754)
             {
                 double number{};
-                return get_number(input_format_t::cbor, number) && emit_float(input_format_t::cbor, number);
+                return get_number(input_format_t::cbor, number) && sax->number_float(static_cast<number_float_t>(number), "");
             }
 
             default: // anything else (0xFF is handled inside the other types)
@@ -1864,61 +1861,61 @@ class binary_reader
             case 0xCA: // float 32
             {
                 float number{};
-                return get_number(input_format_t::msgpack, number) && emit_float(input_format_t::msgpack, number);
+                return get_number(input_format_t::msgpack, number) && sax->number_float(static_cast<number_float_t>(number), "");
             }
 
             case 0xCB: // float 64
             {
                 double number{};
-                return get_number(input_format_t::msgpack, number) && emit_float(input_format_t::msgpack, number);
+                return get_number(input_format_t::msgpack, number) && sax->number_float(static_cast<number_float_t>(number), "");
             }
 
             case 0xCC: // uint 8
             {
                 std::uint8_t number{};
-                return get_number(input_format_t::msgpack, number) && emit_unsigned(number);
+                return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
             }
 
             case 0xCD: // uint 16
             {
                 std::uint16_t number{};
-                return get_number(input_format_t::msgpack, number) && emit_unsigned(number);
+                return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
             }
 
             case 0xCE: // uint 32
             {
                 std::uint32_t number{};
-                return get_number(input_format_t::msgpack, number) && emit_unsigned(number);
+                return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
             }
 
             case 0xCF: // uint 64
             {
                 std::uint64_t number{};
-                return get_number(input_format_t::msgpack, number) && emit_unsigned(number);
+                return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
             }
 
             case 0xD0: // int 8
             {
                 std::int8_t number{};
-                return get_number(input_format_t::msgpack, number) && emit_signed(number);
+                return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
             }
 
             case 0xD1: // int 16
             {
                 std::int16_t number{};
-                return get_number(input_format_t::msgpack, number) && emit_signed(number);
+                return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
             }
 
             case 0xD2: // int 32
             {
                 std::int32_t number{};
-                return get_number(input_format_t::msgpack, number) && emit_signed(number);
+                return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
             }
 
             case 0xD3: // int 64
             {
                 std::int64_t number{};
-                return get_number(input_format_t::msgpack, number) && emit_signed(number);
+                return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
             }
 
             case 0xDC: // array 16
@@ -2759,7 +2756,7 @@ class binary_reader
                         {
                             return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, exception_message(input_format, "excessive ndarray size caused overflow", "size"), nullptr));
                         }
-                        if (JSON_HEDLEY_UNLIKELY(!emit_unsigned(i)))
+                        if (JSON_HEDLEY_UNLIKELY(!sax->number_unsigned(static_cast<number_unsigned_t>(i))))
                         {
                             return false;
                         }
@@ -2891,37 +2888,37 @@ class binary_reader
                     break;
                 }
                 std::uint8_t number{};
-                return get_number(input_format, number) && emit_unsigned(number);
+                return get_number(input_format, number) && sax->number_unsigned(number);
             }
 
             case 'U':
             {
                 std::uint8_t number{};
-                return get_number(input_format, number) && emit_unsigned(number);
+                return get_number(input_format, number) && sax->number_unsigned(number);
             }
 
             case 'i':
             {
                 std::int8_t number{};
-                return get_number(input_format, number) && emit_signed(number);
+                return get_number(input_format, number) && sax->number_integer(number);
             }
 
             case 'I':
             {
                 std::int16_t number{};
-                return get_number(input_format, number) && emit_signed(number);
+                return get_number(input_format, number) && sax->number_integer(number);
             }
 
             case 'l':
             {
                 std::int32_t number{};
-                return get_number(input_format, number) && emit_signed(number);
+                return get_number(input_format, number) && sax->number_integer(number);
             }
 
             case 'L':
             {
                 std::int64_t number{};
-                return get_number(input_format, number) && emit_signed(number);
+                return get_number(input_format, number) && sax->number_integer(number);
             }
 
             case 'u':
@@ -2931,7 +2928,7 @@ class binary_reader
                     break;
                 }
                 std::uint16_t number{};
-                return get_number(input_format, number) && emit_unsigned(number);
+                return get_number(input_format, number) && sax->number_unsigned(number);
             }
 
             case 'm':
@@ -2941,7 +2938,7 @@ class binary_reader
                     break;
                 }
                 std::uint32_t number{};
-                return get_number(input_format, number) && emit_unsigned(number);
+                return get_number(input_format, number) && sax->number_unsigned(number);
             }
 
             case 'M':
@@ -2951,7 +2948,7 @@ class binary_reader
                     break;
                 }
                 std::uint64_t number{};
-                return get_number(input_format, number) && emit_unsigned(number);
+                return get_number(input_format, number) && sax->number_unsigned(number);
             }
 
             case 'h':
@@ -3009,13 +3006,13 @@ class binary_reader
             case 'd':
             {
                 float number{};
-                return get_number(input_format, number) && emit_float(input_format, number);
+                return get_number(input_format, number) && sax->number_float(static_cast<number_float_t>(number), "");
             }
 
             case 'D':
             {
                 double number{};
-                return get_number(input_format, number) && emit_float(input_format, number);
+                return get_number(input_format, number) && sax->number_float(static_cast<number_float_t>(number), "");
             }
 
             case 'H':
@@ -3482,13 +3479,13 @@ class binary_reader
             case 0x8E: // binary32
             {
                 float number{};
-                return get_number(input_format_t::bon8, number) && emit_float(input_format_t::bon8, number);
+                return get_number(input_format_t::bon8, number) && sax->number_float(static_cast<number_float_t>(number), "");
             }
 
             case 0x8F: // binary64
             {
                 double number{};
-                return get_number(input_format_t::bon8, number) && emit_float(input_format_t::bon8, number);
+                return get_number(input_format_t::bon8, number) && sax->number_float(static_cast<number_float_t>(number), "");
             }
 
             case 0xF8:
@@ -3554,9 +3551,7 @@ class binary_reader
     @brief pass an integer to the SAX parser
 
     Non-negative integers are passed as unsigned, negative integers as signed
-    numbers, like the other binary formats do. A value that does not fit the
-    number type is passed as described for @ref emit_unsigned and
-    @ref emit_signed.
+    numbers, like the other binary formats do.
 
     @param[in] number  the integer
     @return whether the SAX parser accepted the value
@@ -3565,9 +3560,9 @@ class binary_reader
     {
         if (number >= 0)
         {
-            return emit_unsigned(static_cast<std::uint64_t>(number));
+            return sax->number_unsigned(static_cast<number_unsigned_t>(number));
         }
-        return emit_signed(number);
+        return sax->number_integer(static_cast<number_integer_t>(number));
     }
 
     /*!
@@ -3624,7 +3619,8 @@ class binary_reader
             value = (value << 8) | static_cast<std::int64_t>(current);
         }
 
-        return emit_bon8_integer(negative ? -(value + offset) : value + offset);
+        return negative ? sax->number_integer(static_cast<number_integer_t>(-(value + offset)))
+               : sax->number_unsigned(static_cast<number_unsigned_t>(value + offset));
     }
 
     /*!
@@ -3921,79 +3917,6 @@ class binary_reader
         return true;
     }
 
-    /*!
-    @brief pass a signed integer read from the input to the SAX parser
-
-    Like the lexer does for JSON text, a value that does not fit into
-    number_integer_t is passed as number_unsigned_t if it is non-negative and
-    fits there, and as number_float_t otherwise. With the default number
-    types, every integer the binary formats can encode fits, so this only
-    matters for narrower custom number types.
-
-    @tparam NumberType a signed integer type
-    @param[in] number  the integer
-    @return whether the SAX parser accepted the value
-    */
-    template<typename NumberType>
-    bool emit_signed(const NumberType number)
-    {
-        if (JSON_HEDLEY_LIKELY(value_in_range_of<number_integer_t>(number)))
-        {
-            return sax->number_integer(static_cast<number_integer_t>(number));
-        }
-        if (value_in_range_of<number_unsigned_t>(number))
-        {
-            return sax->number_unsigned(static_cast<number_unsigned_t>(number));
-        }
-        return sax->number_float(static_cast<number_float_t>(number), "");
-    }
-
-    /*!
-    @brief pass an unsigned integer read from the input to the SAX parser
-
-    Like the lexer does for JSON text, a value that does not fit into
-    number_unsigned_t is passed as number_float_t.
-
-    @tparam NumberType an unsigned integer type
-    @param[in] number  the integer
-    @return whether the SAX parser accepted the value
-    */
-    template<typename NumberType>
-    bool emit_unsigned(const NumberType number)
-    {
-        if (JSON_HEDLEY_LIKELY(value_in_range_of<number_unsigned_t>(number)))
-        {
-            return sax->number_unsigned(static_cast<number_unsigned_t>(number));
-        }
-        return sax->number_float(static_cast<number_float_t>(number), "");
-    }
-
-    /*!
-    @brief pass a floating-point number read from the input to the SAX parser
-
-    Like the lexer does for JSON text, a finite value that overflows
-    number_float_t is rejected instead of silently becoming infinity. Infinity
-    and NaN in the input are passed on unchanged.
-
-    @tparam NumberType a floating-point type
-    @param[in] format  the current format (for diagnostics)
-    @param[in] number  the number
-    @return whether the SAX parser accepted the value
-
-    @throw out_of_range.406 if a finite @a number overflows number_float_t
-    */
-    template<typename NumberType>
-    bool emit_float(const input_format_t format, const NumberType number)
-    {
-        const auto result = static_cast<number_float_t>(number);
-        if (JSON_HEDLEY_UNLIKELY(std::isfinite(number) && !std::isfinite(result)))
-        {
-            return sax->parse_error(chars_read, get_token_string(),
-                                    out_of_range::create(406, exception_message(format, "number overflow", "value"), nullptr));
-        }
-        return sax->number_float(result, "");
-    }
-
     /*!
     @brief create a string by reading characters from the input
 

include/nlohmann/detail/input/lexer.hpp

@@ -206,7 +206,6 @@ class lexer : public lexer_base<BasicJsonType>
     explicit lexer(InputAdapterType&& adapter, bool ignore_comments_ = false, bool discard_number_values_ = false) noexcept
         : ia(std::move(adapter))
         , ignore_comments(ignore_comments_)
-        , decimal_point_char(static_cast<char_int_type>(get_decimal_point()))
         , discard_number_values(discard_number_values_)
     {}
 
@@ -222,8 +221,7 @@ class lexer : public lexer_base<BasicJsonType>
     // locales
     /////////////////////
 
-    /// return the locale-dependent decimal point
-    JSON_HEDLEY_PURE
+    /// return the decimal point of the current locale
     static char get_decimal_point() noexcept
     {
         const auto* loc = localeconv();
@@ -1092,9 +1090,10 @@ class lexer : public lexer_base<BasicJsonType>
             token_type::value_float if number could be successfully scanned,
             token_type::parse_error otherwise
 
-    @note The scanner is independent of the current locale. Internally, the
-          locale's decimal point is used instead of `.` to work with the
-          locale-dependent converters.
+    @note The scanner is independent of the current locale: token_buffer
+          always holds `.`. Only the std::strtod fallback of convert_number()
+          depends on the locale, and it looks up the decimal point right
+          before converting (see convert_float_locale_aware()).
     */
     token_type scan_number()  // lgtm [cpp/use-of-goto] `goto` is used in this function to implement the number-parsing state machine described above. By design, any finite input will eventually reach the "done" state or return token_type::parse_error. In each intermediate state, 1 byte of the input is appended to the token_buffer vector, and only the already initialized variables token_buffer, number_type, and error_message are manipulated.
     {
@@ -1183,7 +1182,7 @@ scan_number_zero:
         {
             case '.':
             {
-                add(decimal_point_char);
+                add(current);
                 decimal_point_position = token_buffer.size() - 1;
                 goto scan_number_decimal1;
             }
@@ -1220,7 +1219,7 @@ scan_number_any1:
 
             case '.':
             {
-                add(decimal_point_char);
+                add(current);
                 decimal_point_position = token_buffer.size() - 1;
                 goto scan_number_decimal1;
             }
@@ -1462,9 +1461,9 @@ scan_number_done:
 
         // Only a number below 1 can carry further insignificant zeros, and only
         // while the count stays at the limit does removing them change the
-        // answer - so this loop is skipped for all but a few tokens. Note
-        // token_buffer holds the locale's decimal point, so the fraction is
-        // located through decimal_point_position rather than by searching '.'.
+        // answer - so this loop is skipped for all but a few tokens. The
+        // fraction is located through decimal_point_position rather than by
+        // searching '.'.
         if (lead_zero != 0)
         {
             JSON_ASSERT(has_dot != 0); // an integer "0" cannot reach the limit
@@ -1482,8 +1481,8 @@ scan_number_done:
     @brief convert the number text in token_buffer to its value and token type
 
     The digit sequence in token_buffer has already been validated (by the
-    scan_number() state machine or by the contiguous fast path) and holds the
-    locale decimal point in place of '.'. Integers are parsed first and fall
+    scan_number() state machine or by the contiguous fast path) and holds '.'
+    as decimal point, independent of the locale. Integers are parsed first and fall
     back to floating point on overflow. This is shared so both scanners produce
     identical results.
 
@@ -1563,7 +1562,7 @@ scan_number_done:
         // integer conversion above overflowed. Prefer std::from_chars
         // (Eisel-Lemire, locale-independent, correctly rounded) when available;
         // otherwise the exact Clinger fast path (double only); otherwise the
-        // locale-aware strtof/strtod.
+        // locale-aware strtof/strtod/strtold.
         if (parse_float_from_chars(num_begin, num_end, value_float))
         {
             return token_type::value_float;
@@ -1572,26 +1571,75 @@ scan_number_done:
         // extra pass over the token's bytes, which otherwise shows up on
         // high-precision inputs such as canada.json
         if (mantissa_fits_clinger(mantissa_end)
-                && parse_float_fast(num_begin, num_end, decimal_point_char, value_float))
+                && parse_float_fast(num_begin, num_end, value_float))
         {
             return token_type::value_float;
         }
 
-        char* endptr = nullptr; // NOLINT(misc-const-correctness,cppcoreguidelines-pro-type-vararg,hicpp-vararg)
-        strtof(value_float, token_buffer.data(), &endptr);
-
-        // we checked the number format before
-        JSON_ASSERT(endptr == token_buffer.data() + token_buffer.size());
-
+        convert_float_locale_aware();
         return token_type::value_float;
     }
 
+    /*!
+    @brief convert the float in token_buffer with strtof/strtod/strtold
+
+    These functions expect the decimal point of the *current* locale, so it is
+    looked up right before the conversion instead of once when the lexer is
+    constructed: a locale change in between (by a parser callback, a SAX
+    handler, or another thread) must not truncate the value (#5198). The
+    token has been validated before, so if the conversion stops early and the
+    decimal point changed in the meantime, the locale changed between the
+    lookup and the call, and the conversion is repeated with the new decimal
+    point. If the decimal point did not change, a retry cannot succeed: the
+    locale's decimal point is not a single character (e.g., the two-byte
+    U+066B of ar_EG.UTF-8 or fa_IR.UTF-8) and cannot be substituted in place.
+    The value strtod parsed up to that point is kept, as before this change.
+
+    Note that changing the locale in another thread *while* strtod runs is
+    undefined behavior of the C library, which this function cannot prevent.
+    */
+    void convert_float_locale_aware()
+    {
+        const bool has_dot = decimal_point_position != std::string::npos;
+        char decimal_point = get_decimal_point();
+        for (;;)
+        {
+            const bool substitute = has_dot && decimal_point != '.';
+            if (substitute)
+            {
+                token_buffer[decimal_point_position] = static_cast<typename string_t::value_type>(decimal_point);
+            }
+
+            char* endptr = nullptr; // NOLINT(misc-const-correctness,cppcoreguidelines-pro-type-vararg,hicpp-vararg)
+            strtof(value_float, token_buffer.data(), &endptr);
+
+            if (substitute)
+            {
+                // get_string() hands the token to the SAX interface with '.'
+                token_buffer[decimal_point_position] = '.';
+            }
+
+            if (JSON_HEDLEY_LIKELY(endptr == token_buffer.data() + token_buffer.size()))
+            {
+                return;
+            }
+
+            // retry only if the locale changed; otherwise, this would loop forever
+            const char current_decimal_point = get_decimal_point();
+            if (current_decimal_point == decimal_point)
+            {
+                return;
+            }
+            decimal_point = current_decimal_point;
+        }
+    }
+
     /*!
     @brief contiguous fast path for scanning a number
 
     Parses the whole number token straight from the input buffer, avoiding the
     per-character get()/add() of scan_number(). On success it fills token_buffer
-    (with the locale decimal point substituted, as scan_number() does) and
+    (as scan_number() does) and
     returns the token type. On anything it does not fully recognize as a
     well-formed number it makes no state change and returns
     token_type::uninitialized, so the caller falls back to scan_number(), which
@@ -1707,16 +1755,11 @@ scan_number_done:
         }
 #endif
 
-        // materialize the token exactly as scan_number() would, substituting the
-        // locale decimal point so convert_number()'s strtof fallback stays valid.
-        // reset() already cleared token_buffer, so append() fills it (assign() is
-        // avoided because custom string_t types need not provide it)
+        // materialize the token exactly as scan_number() would. reset() already
+        // cleared token_buffer, so append() fills it (assign() is avoided
+        // because custom string_t types need not provide it)
         token_buffer.append(reinterpret_cast<const typename string_t::value_type*>(data), len);
-        if (dot_index != std::string::npos)
-        {
-            token_buffer[dot_index] = static_cast<typename string_t::value_type>(decimal_point_char);
-            decimal_point_position = dot_index;
-        }
+        decimal_point_position = dot_index;
 
         ia.bulk_skip(len - 1);
         position.chars_read_total += (len - 1);
@@ -1983,11 +2026,7 @@ scan_number_done:
     /// return current string value (implicitly resets the token; useful only once)
     string_t& get_string()
     {
-        // translate decimal points from locale back to '.' (#4084)
-        if (decimal_point_char != '.' && decimal_point_position != std::string::npos)
-        {
-            token_buffer[decimal_point_position] = '.';
-        }
+        // a number token holds '.' regardless of the locale (#4084)
         return token_buffer;
     }
 
@@ -2283,9 +2322,7 @@ scan_number_done:
     number_unsigned_t value_unsigned = 0;
     number_float_t value_float = 0;
 
-    /// the decimal point
-    const char_int_type decimal_point_char = '.';
-    /// the position of the decimal point in the input
+    /// the position of the decimal point in token_buffer
     std::size_t decimal_point_position = std::string::npos;
 
     /// whether the caller (e.g. accept()/json_sax_acceptor) only needs the

include/nlohmann/detail/input/number_parse.hpp

@@ -118,14 +118,12 @@ std::strtod. The parser only activates for number_float_t == double; float and
 long double keep the std::strtof/std::strtold paths (see the templated overload
 below).
 
-@param[in]  first          pointer to the first character of the number
-@param[in]  last           pointer past the last character
-@param[in]  decimal_point  the (locale-dependent) decimal point character
-@param[out] out            the parsed value on success
+@param[in]  first  pointer to the first character of the number
+@param[in]  last   pointer past the last character
+@param[out] out    the parsed value on success
 @return true if the value was parsed exactly; false to fall back to strtod
 */
-template<typename DecimalPointType>
-bool parse_float_fast(const char* first, const char* last, DecimalPointType decimal_point, double& out) noexcept
+inline bool parse_float_fast(const char* first, const char* last, double& out) noexcept
 {
 #if defined(FLT_EVAL_METHOD) && FLT_EVAL_METHOD != 0
     // Clinger's fast path is only exact when double operations are evaluated in
@@ -136,7 +134,6 @@ bool parse_float_fast(const char* first, const char* last, DecimalPointType deci
     // std::from_chars / std::strtod path.
     static_cast<void>(first);
     static_cast<void>(last);
-    static_cast<void>(decimal_point);
     static_cast<void>(out);
     return false;
 #else
@@ -175,7 +172,7 @@ bool parse_float_fast(const char* first, const char* last, DecimalPointType deci
             ++num_digits;
             fractional_digits += static_cast<int>(seen_dot);
         }
-        else if (static_cast<DecimalPointType>(c) == decimal_point)
+        else if (c == '.')
         {
             if (JSON_HEDLEY_UNLIKELY(seen_dot))
             {
@@ -260,8 +257,8 @@ bool parse_float_fast(const char* first, const char* last, DecimalPointType deci
 }
 
 /// fast float path is only exact for `double`; decline for float/long double
-template<typename DecimalPointType, typename FloatType>
-bool parse_float_fast(const char* /*first*/, const char* /*last*/, DecimalPointType /*decimal_point*/, FloatType& /*out*/) noexcept
+template<typename FloatType>
+bool parse_float_fast(const char* /*first*/, const char* /*last*/, FloatType& /*out*/) noexcept
 {
     return false;
 }
@@ -273,9 +270,7 @@ std::from_chars is locale-independent, correctly rounded, and - via the
 Eisel-Lemire algorithm in modern standard libraries - much faster than strtod
 over the whole value range (not just the Clinger subset). It is used only when
 __cpp_lib_to_chars indicates full floating-point support and only when it
-consumes the entire token ([first, last)); a partial parse means the buffer
-uses a non-'.' locale decimal point, in which case the caller falls back to the
-locale-aware path. An under-/overflow (result_out_of_range) also declines, so
+consumes the entire token ([first, last)). An under-/overflow (result_out_of_range) also declines, so
 the caller's strtod fallback supplies the well-defined ±inf/0 result the parser
 expects (side-stepping the P4168 divergence between implementations).
 

single_include/nlohmann/json.hpp

@@ -8605,14 +8605,12 @@ std::strtod. The parser only activates for number_float_t == double; float and
 long double keep the std::strtof/std::strtold paths (see the templated overload
 below).
 
-@param[in]  first          pointer to the first character of the number
-@param[in]  last           pointer past the last character
-@param[in]  decimal_point  the (locale-dependent) decimal point character
-@param[out] out            the parsed value on success
+@param[in]  first  pointer to the first character of the number
+@param[in]  last   pointer past the last character
+@param[out] out    the parsed value on success
 @return true if the value was parsed exactly; false to fall back to strtod
 */
-template<typename DecimalPointType>
-bool parse_float_fast(const char* first, const char* last, DecimalPointType decimal_point, double& out) noexcept
+inline bool parse_float_fast(const char* first, const char* last, double& out) noexcept
 {
 #if defined(FLT_EVAL_METHOD) && FLT_EVAL_METHOD != 0
     // Clinger's fast path is only exact when double operations are evaluated in
@@ -8623,7 +8621,6 @@ bool parse_float_fast(const char* first, const char* last, DecimalPointType deci
     // std::from_chars / std::strtod path.
     static_cast<void>(first);
     static_cast<void>(last);
-    static_cast<void>(decimal_point);
     static_cast<void>(out);
     return false;
 #else
@@ -8662,7 +8659,7 @@ bool parse_float_fast(const char* first, const char* last, DecimalPointType deci
             ++num_digits;
             fractional_digits += static_cast<int>(seen_dot);
         }
-        else if (static_cast<DecimalPointType>(c) == decimal_point)
+        else if (c == '.')
         {
             if (JSON_HEDLEY_UNLIKELY(seen_dot))
             {
@@ -8747,8 +8744,8 @@ bool parse_float_fast(const char* first, const char* last, DecimalPointType deci
 }
 
 /// fast float path is only exact for `double`; decline for float/long double
-template<typename DecimalPointType, typename FloatType>
-bool parse_float_fast(const char* /*first*/, const char* /*last*/, DecimalPointType /*decimal_point*/, FloatType& /*out*/) noexcept
+template<typename FloatType>
+bool parse_float_fast(const char* /*first*/, const char* /*last*/, FloatType& /*out*/) noexcept
 {
     return false;
 }
@@ -8760,9 +8757,7 @@ std::from_chars is locale-independent, correctly rounded, and - via the
 Eisel-Lemire algorithm in modern standard libraries - much faster than strtod
 over the whole value range (not just the Clinger subset). It is used only when
 __cpp_lib_to_chars indicates full floating-point support and only when it
-consumes the entire token ([first, last)); a partial parse means the buffer
-uses a non-'.' locale decimal point, in which case the caller falls back to the
-locale-aware path. An under-/overflow (result_out_of_range) also declines, so
+consumes the entire token ([first, last)). An under-/overflow (result_out_of_range) also declines, so
 the caller's strtod fallback supplies the well-defined ±inf/0 result the parser
 expects (side-stepping the P4168 divergence between implementations).
 
@@ -9303,7 +9298,6 @@ class lexer : public lexer_base<BasicJsonType>
     explicit lexer(InputAdapterType&& adapter, bool ignore_comments_ = false, bool discard_number_values_ = false) noexcept
         : ia(std::move(adapter))
         , ignore_comments(ignore_comments_)
-        , decimal_point_char(static_cast<char_int_type>(get_decimal_point()))
         , discard_number_values(discard_number_values_)
     {}
 
@@ -9319,8 +9313,7 @@ class lexer : public lexer_base<BasicJsonType>
     // locales
     /////////////////////
 
-    /// return the locale-dependent decimal point
-    JSON_HEDLEY_PURE
+    /// return the decimal point of the current locale
     static char get_decimal_point() noexcept
     {
         const auto* loc = localeconv();
@@ -10189,9 +10182,10 @@ class lexer : public lexer_base<BasicJsonType>
             token_type::value_float if number could be successfully scanned,
             token_type::parse_error otherwise
 
-    @note The scanner is independent of the current locale. Internally, the
-          locale's decimal point is used instead of `.` to work with the
-          locale-dependent converters.
+    @note The scanner is independent of the current locale: token_buffer
+          always holds `.`. Only the std::strtod fallback of convert_number()
+          depends on the locale, and it looks up the decimal point right
+          before converting (see convert_float_locale_aware()).
     */
     token_type scan_number()  // lgtm [cpp/use-of-goto] `goto` is used in this function to implement the number-parsing state machine described above. By design, any finite input will eventually reach the "done" state or return token_type::parse_error. In each intermediate state, 1 byte of the input is appended to the token_buffer vector, and only the already initialized variables token_buffer, number_type, and error_message are manipulated.
     {
@@ -10280,7 +10274,7 @@ scan_number_zero:
         {
             case '.':
             {
-                add(decimal_point_char);
+                add(current);
                 decimal_point_position = token_buffer.size() - 1;
                 goto scan_number_decimal1;
             }
@@ -10317,7 +10311,7 @@ scan_number_any1:
 
             case '.':
             {
-                add(decimal_point_char);
+                add(current);
                 decimal_point_position = token_buffer.size() - 1;
                 goto scan_number_decimal1;
             }
@@ -10559,9 +10553,9 @@ scan_number_done:
 
         // Only a number below 1 can carry further insignificant zeros, and only
         // while the count stays at the limit does removing them change the
-        // answer - so this loop is skipped for all but a few tokens. Note
-        // token_buffer holds the locale's decimal point, so the fraction is
-        // located through decimal_point_position rather than by searching '.'.
+        // answer - so this loop is skipped for all but a few tokens. The
+        // fraction is located through decimal_point_position rather than by
+        // searching '.'.
         if (lead_zero != 0)
         {
             JSON_ASSERT(has_dot != 0); // an integer "0" cannot reach the limit
@@ -10579,8 +10573,8 @@ scan_number_done:
     @brief convert the number text in token_buffer to its value and token type
 
     The digit sequence in token_buffer has already been validated (by the
-    scan_number() state machine or by the contiguous fast path) and holds the
-    locale decimal point in place of '.'. Integers are parsed first and fall
+    scan_number() state machine or by the contiguous fast path) and holds '.'
+    as decimal point, independent of the locale. Integers are parsed first and fall
     back to floating point on overflow. This is shared so both scanners produce
     identical results.
 
@@ -10660,7 +10654,7 @@ scan_number_done:
         // integer conversion above overflowed. Prefer std::from_chars
         // (Eisel-Lemire, locale-independent, correctly rounded) when available;
         // otherwise the exact Clinger fast path (double only); otherwise the
-        // locale-aware strtof/strtod.
+        // locale-aware strtof/strtod/strtold.
         if (parse_float_from_chars(num_begin, num_end, value_float))
         {
             return token_type::value_float;
@@ -10669,26 +10663,75 @@ scan_number_done:
         // extra pass over the token's bytes, which otherwise shows up on
         // high-precision inputs such as canada.json
         if (mantissa_fits_clinger(mantissa_end)
-                && parse_float_fast(num_begin, num_end, decimal_point_char, value_float))
+                && parse_float_fast(num_begin, num_end, value_float))
         {
             return token_type::value_float;
         }
 
-        char* endptr = nullptr; // NOLINT(misc-const-correctness,cppcoreguidelines-pro-type-vararg,hicpp-vararg)
-        strtof(value_float, token_buffer.data(), &endptr);
-
-        // we checked the number format before
-        JSON_ASSERT(endptr == token_buffer.data() + token_buffer.size());
-
+        convert_float_locale_aware();
         return token_type::value_float;
     }
 
+    /*!
+    @brief convert the float in token_buffer with strtof/strtod/strtold
+
+    These functions expect the decimal point of the *current* locale, so it is
+    looked up right before the conversion instead of once when the lexer is
+    constructed: a locale change in between (by a parser callback, a SAX
+    handler, or another thread) must not truncate the value (#5198). The
+    token has been validated before, so if the conversion stops early and the
+    decimal point changed in the meantime, the locale changed between the
+    lookup and the call, and the conversion is repeated with the new decimal
+    point. If the decimal point did not change, a retry cannot succeed: the
+    locale's decimal point is not a single character (e.g., the two-byte
+    U+066B of ar_EG.UTF-8 or fa_IR.UTF-8) and cannot be substituted in place.
+    The value strtod parsed up to that point is kept, as before this change.
+
+    Note that changing the locale in another thread *while* strtod runs is
+    undefined behavior of the C library, which this function cannot prevent.
+    */
+    void convert_float_locale_aware()
+    {
+        const bool has_dot = decimal_point_position != std::string::npos;
+        char decimal_point = get_decimal_point();
+        for (;;)
+        {
+            const bool substitute = has_dot && decimal_point != '.';
+            if (substitute)
+            {
+                token_buffer[decimal_point_position] = static_cast<typename string_t::value_type>(decimal_point);
+            }
+
+            char* endptr = nullptr; // NOLINT(misc-const-correctness,cppcoreguidelines-pro-type-vararg,hicpp-vararg)
+            strtof(value_float, token_buffer.data(), &endptr);
+
+            if (substitute)
+            {
+                // get_string() hands the token to the SAX interface with '.'
+                token_buffer[decimal_point_position] = '.';
+            }
+
+            if (JSON_HEDLEY_LIKELY(endptr == token_buffer.data() + token_buffer.size()))
+            {
+                return;
+            }
+
+            // retry only if the locale changed; otherwise, this would loop forever
+            const char current_decimal_point = get_decimal_point();
+            if (current_decimal_point == decimal_point)
+            {
+                return;
+            }
+            decimal_point = current_decimal_point;
+        }
+    }
+
     /*!
     @brief contiguous fast path for scanning a number
 
     Parses the whole number token straight from the input buffer, avoiding the
     per-character get()/add() of scan_number(). On success it fills token_buffer
-    (with the locale decimal point substituted, as scan_number() does) and
+    (as scan_number() does) and
     returns the token type. On anything it does not fully recognize as a
     well-formed number it makes no state change and returns
     token_type::uninitialized, so the caller falls back to scan_number(), which
@@ -10804,16 +10847,11 @@ scan_number_done:
         }
 #endif
 
-        // materialize the token exactly as scan_number() would, substituting the
-        // locale decimal point so convert_number()'s strtof fallback stays valid.
-        // reset() already cleared token_buffer, so append() fills it (assign() is
-        // avoided because custom string_t types need not provide it)
+        // materialize the token exactly as scan_number() would. reset() already
+        // cleared token_buffer, so append() fills it (assign() is avoided
+        // because custom string_t types need not provide it)
         token_buffer.append(reinterpret_cast<const typename string_t::value_type*>(data), len);
-        if (dot_index != std::string::npos)
-        {
-            token_buffer[dot_index] = static_cast<typename string_t::value_type>(decimal_point_char);
-            decimal_point_position = dot_index;
-        }
+        decimal_point_position = dot_index;
 
         ia.bulk_skip(len - 1);
         position.chars_read_total += (len - 1);
@@ -11080,11 +11118,7 @@ scan_number_done:
     /// return current string value (implicitly resets the token; useful only once)
     string_t& get_string()
     {
-        // translate decimal points from locale back to '.' (#4084)
-        if (decimal_point_char != '.' && decimal_point_position != std::string::npos)
-        {
-            token_buffer[decimal_point_position] = '.';
-        }
+        // a number token holds '.' regardless of the locale (#4084)
         return token_buffer;
     }
 
@@ -11380,9 +11414,7 @@ scan_number_done:
     number_unsigned_t value_unsigned = 0;
     number_float_t value_float = 0;
 
-    /// the decimal point
-    const char_int_type decimal_point_char = '.';
-    /// the position of the decimal point in the input
+    /// the position of the decimal point in token_buffer
     std::size_t decimal_point_position = std::string::npos;
 
     /// whether the caller (e.g. accept()/json_sax_acceptor) only needs the
@@ -13294,7 +13326,7 @@ class binary_reader
             case 0x01: // double
             {
                 double number{};
-                return get_number<double, true>(input_format_t::bson, number) && emit_float(input_format_t::bson, number);
+                return get_number<double, true>(input_format_t::bson, number) && sax->number_float(static_cast<number_float_t>(number), "");
             }
 
             case 0x02: // string
@@ -13335,19 +13367,19 @@ class binary_reader
             case 0x10: // int32
             {
                 std::int32_t value{};
-                return get_number<std::int32_t, true>(input_format_t::bson, value) && emit_signed(value);
+                return get_number<std::int32_t, true>(input_format_t::bson, value) && sax->number_integer(value);
             }
 
             case 0x12: // int64
             {
                 std::int64_t value{};
-                return get_number<std::int64_t, true>(input_format_t::bson, value) && emit_signed(value);
+                return get_number<std::int64_t, true>(input_format_t::bson, value) && sax->number_integer(value);
             }
 
             case 0x11: // uint64
             {
                 std::uint64_t value{};
-                return get_number<std::uint64_t, true>(input_format_t::bson, value) && emit_unsigned(value);
+                return get_number<std::uint64_t, true>(input_format_t::bson, value) && sax->number_unsigned(value);
             }
 
             default: // anything else is not supported (yet)
@@ -13373,17 +13405,14 @@ class binary_reader
         {
             return false;
         }
-
-        // the value is -1 - number, which fits into number_integer_t
-        // whenever number does
-        if (JSON_HEDLEY_LIKELY(value_in_range_of<number_integer_t>(number)))
+        const auto max_val = static_cast<NumberType>((std::numeric_limits<number_integer_t>::max)());
+        if (number > max_val)
         {
-            return sax->number_integer(static_cast<number_integer_t>(-1) - static_cast<number_integer_t>(number));
+            return sax->parse_error(chars_read, get_token_string(),
+                                    parse_error::create(112, chars_read,
+                                            exception_message(input_format_t::cbor, "negative integer overflow", "value"), nullptr));
         }
-
-        // like the lexer does for JSON text, store a value too small for
-        // number_integer_t as number_float_t
-        return sax->number_float(static_cast<number_float_t>(-1) - static_cast<number_float_t>(number), "");
+        return sax->number_integer(static_cast<number_integer_t>(-1) - static_cast<number_integer_t>(number));
     }
 
     /*!
@@ -13440,25 +13469,25 @@ class binary_reader
             case 0x18: // Unsigned integer (one-byte uint8_t follows)
             {
                 std::uint8_t number{};
-                return get_number(input_format_t::cbor, number) && emit_unsigned(number);
+                return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
             }
 
             case 0x19: // Unsigned integer (two-byte uint16_t follows)
             {
                 std::uint16_t number{};
-                return get_number(input_format_t::cbor, number) && emit_unsigned(number);
+                return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
             }
 
             case 0x1A: // Unsigned integer (four-byte uint32_t follows)
             {
                 std::uint32_t number{};
-                return get_number(input_format_t::cbor, number) && emit_unsigned(number);
+                return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
             }
 
             case 0x1B: // Unsigned integer (eight-byte uint64_t follows)
             {
                 std::uint64_t number{};
-                return get_number(input_format_t::cbor, number) && emit_unsigned(number);
+                return get_number(input_format_t::cbor, number) && sax->number_unsigned(number);
             }
 
             // Negative integer -1-0x00..-1-0x17 (-1..-24)
@@ -13903,13 +13932,13 @@ class binary_reader
             case 0xFA: // Single-Precision Float (four-byte IEEE 754)
             {
                 float number{};
-                return get_number(input_format_t::cbor, number) && emit_float(input_format_t::cbor, number);
+                return get_number(input_format_t::cbor, number) && sax->number_float(static_cast<number_float_t>(number), "");
             }
 
             case 0xFB: // Double-Precision Float (eight-byte IEEE 754)
             {
                 double number{};
-                return get_number(input_format_t::cbor, number) && emit_float(input_format_t::cbor, number);
+                return get_number(input_format_t::cbor, number) && sax->number_float(static_cast<number_float_t>(number), "");
             }
 
             default: // anything else (0xFF is handled inside the other types)
@@ -14599,61 +14628,61 @@ class binary_reader
             case 0xCA: // float 32
             {
                 float number{};
-                return get_number(input_format_t::msgpack, number) && emit_float(input_format_t::msgpack, number);
+                return get_number(input_format_t::msgpack, number) && sax->number_float(static_cast<number_float_t>(number), "");
             }
 
             case 0xCB: // float 64
             {
                 double number{};
-                return get_number(input_format_t::msgpack, number) && emit_float(input_format_t::msgpack, number);
+                return get_number(input_format_t::msgpack, number) && sax->number_float(static_cast<number_float_t>(number), "");
             }
 
             case 0xCC: // uint 8
             {
                 std::uint8_t number{};
-                return get_number(input_format_t::msgpack, number) && emit_unsigned(number);
+                return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
             }
 
             case 0xCD: // uint 16
             {
                 std::uint16_t number{};
-                return get_number(input_format_t::msgpack, number) && emit_unsigned(number);
+                return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
             }
 
             case 0xCE: // uint 32
             {
                 std::uint32_t number{};
-                return get_number(input_format_t::msgpack, number) && emit_unsigned(number);
+                return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
             }
 
             case 0xCF: // uint 64
             {
                 std::uint64_t number{};
-                return get_number(input_format_t::msgpack, number) && emit_unsigned(number);
+                return get_number(input_format_t::msgpack, number) && sax->number_unsigned(number);
             }
 
             case 0xD0: // int 8
             {
                 std::int8_t number{};
-                return get_number(input_format_t::msgpack, number) && emit_signed(number);
+                return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
             }
 
             case 0xD1: // int 16
             {
                 std::int16_t number{};
-                return get_number(input_format_t::msgpack, number) && emit_signed(number);
+                return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
             }
 
             case 0xD2: // int 32
             {
                 std::int32_t number{};
-                return get_number(input_format_t::msgpack, number) && emit_signed(number);
+                return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
             }
 
             case 0xD3: // int 64
             {
                 std::int64_t number{};
-                return get_number(input_format_t::msgpack, number) && emit_signed(number);
+                return get_number(input_format_t::msgpack, number) && sax->number_integer(number);
             }
 
             case 0xDC: // array 16
@@ -15494,7 +15523,7 @@ class binary_reader
                         {
                             return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, exception_message(input_format, "excessive ndarray size caused overflow", "size"), nullptr));
                         }
-                        if (JSON_HEDLEY_UNLIKELY(!emit_unsigned(i)))
+                        if (JSON_HEDLEY_UNLIKELY(!sax->number_unsigned(static_cast<number_unsigned_t>(i))))
                         {
                             return false;
                         }
@@ -15626,37 +15655,37 @@ class binary_reader
                     break;
                 }
                 std::uint8_t number{};
-                return get_number(input_format, number) && emit_unsigned(number);
+                return get_number(input_format, number) && sax->number_unsigned(number);
             }
 
             case 'U':
             {
                 std::uint8_t number{};
-                return get_number(input_format, number) && emit_unsigned(number);
+                return get_number(input_format, number) && sax->number_unsigned(number);
             }
 
             case 'i':
             {
                 std::int8_t number{};
-                return get_number(input_format, number) && emit_signed(number);
+                return get_number(input_format, number) && sax->number_integer(number);
             }
 
             case 'I':
             {
                 std::int16_t number{};
-                return get_number(input_format, number) && emit_signed(number);
+                return get_number(input_format, number) && sax->number_integer(number);
             }
 
             case 'l':
             {
                 std::int32_t number{};
-                return get_number(input_format, number) && emit_signed(number);
+                return get_number(input_format, number) && sax->number_integer(number);
             }
 
             case 'L':
             {
                 std::int64_t number{};
-                return get_number(input_format, number) && emit_signed(number);
+                return get_number(input_format, number) && sax->number_integer(number);
             }
 
             case 'u':
@@ -15666,7 +15695,7 @@ class binary_reader
                     break;
                 }
                 std::uint16_t number{};
-                return get_number(input_format, number) && emit_unsigned(number);
+                return get_number(input_format, number) && sax->number_unsigned(number);
             }
 
             case 'm':
@@ -15676,7 +15705,7 @@ class binary_reader
                     break;
                 }
                 std::uint32_t number{};
-                return get_number(input_format, number) && emit_unsigned(number);
+                return get_number(input_format, number) && sax->number_unsigned(number);
             }
 
             case 'M':
@@ -15686,7 +15715,7 @@ class binary_reader
                     break;
                 }
                 std::uint64_t number{};
-                return get_number(input_format, number) && emit_unsigned(number);
+                return get_number(input_format, number) && sax->number_unsigned(number);
             }
 
             case 'h':
@@ -15744,13 +15773,13 @@ class binary_reader
             case 'd':
             {
                 float number{};
-                return get_number(input_format, number) && emit_float(input_format, number);
+                return get_number(input_format, number) && sax->number_float(static_cast<number_float_t>(number), "");
             }
 
             case 'D':
             {
                 double number{};
-                return get_number(input_format, number) && emit_float(input_format, number);
+                return get_number(input_format, number) && sax->number_float(static_cast<number_float_t>(number), "");
             }
 
             case 'H':
@@ -16217,13 +16246,13 @@ class binary_reader
             case 0x8E: // binary32
             {
                 float number{};
-                return get_number(input_format_t::bon8, number) && emit_float(input_format_t::bon8, number);
+                return get_number(input_format_t::bon8, number) && sax->number_float(static_cast<number_float_t>(number), "");
             }
 
             case 0x8F: // binary64
             {
                 double number{};
-                return get_number(input_format_t::bon8, number) && emit_float(input_format_t::bon8, number);
+                return get_number(input_format_t::bon8, number) && sax->number_float(static_cast<number_float_t>(number), "");
             }
 
             case 0xF8:
@@ -16289,9 +16318,7 @@ class binary_reader
     @brief pass an integer to the SAX parser
 
     Non-negative integers are passed as unsigned, negative integers as signed
-    numbers, like the other binary formats do. A value that does not fit the
-    number type is passed as described for @ref emit_unsigned and
-    @ref emit_signed.
+    numbers, like the other binary formats do.
 
     @param[in] number  the integer
     @return whether the SAX parser accepted the value
@@ -16300,9 +16327,9 @@ class binary_reader
     {
         if (number >= 0)
         {
-            return emit_unsigned(static_cast<std::uint64_t>(number));
+            return sax->number_unsigned(static_cast<number_unsigned_t>(number));
         }
-        return emit_signed(number);
+        return sax->number_integer(static_cast<number_integer_t>(number));
     }
 
     /*!
@@ -16359,7 +16386,8 @@ class binary_reader
             value = (value << 8) | static_cast<std::int64_t>(current);
         }
 
-        return emit_bon8_integer(negative ? -(value + offset) : value + offset);
+        return negative ? sax->number_integer(static_cast<number_integer_t>(-(value + offset)))
+               : sax->number_unsigned(static_cast<number_unsigned_t>(value + offset));
     }
 
     /*!
@@ -16656,79 +16684,6 @@ class binary_reader
         return true;
     }
 
-    /*!
-    @brief pass a signed integer read from the input to the SAX parser
-
-    Like the lexer does for JSON text, a value that does not fit into
-    number_integer_t is passed as number_unsigned_t if it is non-negative and
-    fits there, and as number_float_t otherwise. With the default number
-    types, every integer the binary formats can encode fits, so this only
-    matters for narrower custom number types.
-
-    @tparam NumberType a signed integer type
-    @param[in] number  the integer
-    @return whether the SAX parser accepted the value
-    */
-    template<typename NumberType>
-    bool emit_signed(const NumberType number)
-    {
-        if (JSON_HEDLEY_LIKELY(value_in_range_of<number_integer_t>(number)))
-        {
-            return sax->number_integer(static_cast<number_integer_t>(number));
-        }
-        if (value_in_range_of<number_unsigned_t>(number))
-        {
-            return sax->number_unsigned(static_cast<number_unsigned_t>(number));
-        }
-        return sax->number_float(static_cast<number_float_t>(number), "");
-    }
-
-    /*!
-    @brief pass an unsigned integer read from the input to the SAX parser
-
-    Like the lexer does for JSON text, a value that does not fit into
-    number_unsigned_t is passed as number_float_t.
-
-    @tparam NumberType an unsigned integer type
-    @param[in] number  the integer
-    @return whether the SAX parser accepted the value
-    */
-    template<typename NumberType>
-    bool emit_unsigned(const NumberType number)
-    {
-        if (JSON_HEDLEY_LIKELY(value_in_range_of<number_unsigned_t>(number)))
-        {
-            return sax->number_unsigned(static_cast<number_unsigned_t>(number));
-        }
-        return sax->number_float(static_cast<number_float_t>(number), "");
-    }
-
-    /*!
-    @brief pass a floating-point number read from the input to the SAX parser
-
-    Like the lexer does for JSON text, a finite value that overflows
-    number_float_t is rejected instead of silently becoming infinity. Infinity
-    and NaN in the input are passed on unchanged.
-
-    @tparam NumberType a floating-point type
-    @param[in] format  the current format (for diagnostics)
-    @param[in] number  the number
-    @return whether the SAX parser accepted the value
-
-    @throw out_of_range.406 if a finite @a number overflows number_float_t
-    */
-    template<typename NumberType>
-    bool emit_float(const input_format_t format, const NumberType number)
-    {
-        const auto result = static_cast<number_float_t>(number);
-        if (JSON_HEDLEY_UNLIKELY(std::isfinite(number) && !std::isfinite(result)))
-        {
-            return sax->parse_error(chars_read, get_token_string(),
-                                    out_of_range::create(406, exception_message(format, "number overflow", "value"), nullptr));
-        }
-        return sax->number_float(result, "");
-    }
-
     /*!
     @brief create a string by reading characters from the input
 

tests/src/unit-binary_formats.cpp

@@ -11,12 +11,7 @@
 #include <nlohmann/json.hpp>
 using nlohmann::json;
 
-#include <cmath>
 #include <fstream>
-#include <limits>
-#include <map>
-#include <string>
-#include <vector>
 #include "make_test_data_available.hpp"
 
 TEST_CASE("Binary Formats" * doctest::skip())
@@ -229,114 +224,3 @@ TEST_CASE("Binary Formats" * doctest::skip())
         CHECK((100.0 * double(ubjson_3_size) / double(json_size)) == Approx(89.450));
     }
 }
-
-TEST_CASE("Binary formats with narrow number types")
-{
-    // Numbers that do not fit the number types are handled like the lexer
-    // handles them in JSON text: an integer that fits neither integer type is
-    // stored as a floating-point number, and a finite floating-point number
-    // that overflows number_float_t is rejected with out_of_range.406.
-    using narrow_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int32_t, std::uint32_t, float>;
-    using bytes = std::vector<std::uint8_t>;
-
-    struct binary_format
-    {
-        const char* name;
-        bytes (*encode)(const json&);
-        narrow_json (*decode)(const bytes&, bool);
-    };
-
-    const std::vector<binary_format> formats =
-    {
-        {
-            "CBOR", [](const json & j) { return json::to_cbor(j); },
-            [](const bytes & v, bool allow_exceptions)
-            {
-                return narrow_json::from_cbor(v, true, allow_exceptions);
-            }
-        },
-        {
-            "MessagePack", [](const json & j) { return json::to_msgpack(j); },
-            [](const bytes & v, bool allow_exceptions)
-            {
-                return narrow_json::from_msgpack(v, true, allow_exceptions);
-            }
-        },
-        {
-            "UBJSON", [](const json & j) { return json::to_ubjson(j); },
-            [](const bytes & v, bool allow_exceptions)
-            {
-                return narrow_json::from_ubjson(v, true, allow_exceptions);
-            }
-        },
-        {
-            "BJData", [](const json & j) { return json::to_bjdata(j); },
-            [](const bytes & v, bool allow_exceptions)
-            {
-                return narrow_json::from_bjdata(v, true, allow_exceptions);
-            }
-        },
-        {
-            // BSON can only store numbers as object members
-            "BSON", [](const json & j) { return json::to_bson(json{{"a", j}}); },
-            [](const bytes & v, bool allow_exceptions)
-            {
-                const auto result = narrow_json::from_bson(v, true, allow_exceptions);
-                return result.is_discarded() ? result : result.at("a");
-            }
-        },
-        {
-            "BON8", [](const json & j) { return json::to_bon8(j); },
-            [](const bytes & v, bool allow_exceptions)
-            {
-                return narrow_json::from_bon8(v, true, allow_exceptions);
-            }
-        },
-    };
-
-    for (const auto& format : formats)
-    {
-        const std::string name = format.name;
-        INFO("format := ", name);
-        const auto roundtrip = [&format](const json & j)
-        {
-            return format.decode(format.encode(j), true);
-        };
-
-        // integers that fit keep their type
-        CHECK(roundtrip(json(-5)).is_number_integer());
-        CHECK(roundtrip(json(-5)).get<std::int32_t>() == -5);
-        CHECK(roundtrip(json(3000000000u)).is_number_unsigned());
-        CHECK(roundtrip(json(3000000000u)).get<std::uint32_t>() == 3000000000u);
-
-        // integers that fit neither integer type are stored as float
-        CHECK(roundtrip(json(5000000000u)).is_number_float());
-        CHECK(roundtrip(json(5000000000u)).get<float>() == 5000000000.0f);
-        if (name != "BON8") // BON8 cannot encode integers above INT64_MAX
-        {
-            CHECK(roundtrip(json(10000000000000000000u)).is_number_float());
-            CHECK(roundtrip(json(10000000000000000000u)).get<float>() == 10000000000000000000.0f);
-        }
-        CHECK(roundtrip(json(-3000000000)).is_number_float());
-        CHECK(roundtrip(json(-3000000000)).get<float>() == -3000000000.0f);
-        CHECK(roundtrip(json(-5000000000)).is_number_float());
-        CHECK(roundtrip(json(-5000000000)).get<float>() == -5000000000.0f);
-
-        // floating-point numbers that fit
-        CHECK(roundtrip(json(1.5)).get<float>() == 1.5f);
-        const auto just_above_max = std::nextafter(static_cast<double>((std::numeric_limits<float>::max)()),
-                                    std::numeric_limits<double>::infinity());
-        CHECK(roundtrip(json(just_above_max)).get<float>() == (std::numeric_limits<float>::max)());
-
-        // infinity and NaN are passed on
-        CHECK(std::isinf(roundtrip(json(std::numeric_limits<double>::infinity())).get<float>()));
-        CHECK(std::isnan(roundtrip(json(std::numeric_limits<double>::quiet_NaN())).get<float>()));
-
-        // finite floating-point numbers that overflow number_float_t are rejected
-        const std::string message = "[json.exception.out_of_range.406] syntax error while parsing " + name
-                                    + " value: number overflow";
-        CHECK_THROWS_WITH_AS(roundtrip(json(1e300)), message.c_str(), narrow_json::out_of_range&);
-        CHECK_THROWS_WITH_AS(roundtrip(json(-1e300)), message.c_str(), narrow_json::out_of_range&);
-        CHECK(format.decode(format.encode(json(1e300)), false).is_discarded());
-    }
-}

tests/src/unit-cbor.cpp

@@ -3146,8 +3146,7 @@ TEST_CASE("Tagged values")
         // CBOR encodes negative integers as: result = -1 - n
         // For type 0x3B, n is an 8-byte uint64_t. Valid range for n with
         // the default int64_t is [0, INT64_MAX], producing results in [INT64_MIN, -1].
-        // When n > INT64_MAX, the result exceeds int64_t range and is stored
-        // as a floating-point number, as the lexer does for JSON text.
+        // When n > INT64_MAX, the result exceeds int64_t range and is rejected.
 
         SECTION("n = 0 is valid (result = -1)")
         {
@@ -3168,34 +3167,33 @@ TEST_CASE("Tagged values")
             CHECK(result.get<int64_t>() == (std::numeric_limits<int64_t>::min)());
         }
 
-        SECTION("n = INT64_MAX + 1 is stored as float")
+        SECTION("n = INT64_MAX + 1 is rejected (overflow)")
         {
             // n = INT64_MAX + 1 (0x8000000000000000)
-            // result = -1 - n = -9223372036854775809, which exceeds int64_t range;
-            // the nearest double is -9223372036854775808.0
+            // result = -1 - n = -9223372036854775809, which exceeds int64_t range
             const std::vector<uint8_t> input = {0x3B, 0x80, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00};
-            const auto result = json::from_cbor(input);
-            CHECK(result.is_number_float());
-            CHECK(result.get<double>() == -9223372036854775808.0);
-            CHECK(result == json::parse("-9223372036854775809"));
+            json _;
+            CHECK_THROWS_WITH_AS(_ = json::from_cbor(input),
+                                 "[json.exception.parse_error.112] parse error at byte 9: syntax error while parsing CBOR value: negative integer overflow",
+                                 json::parse_error);
         }
 
-        SECTION("n = UINT64_MAX is stored as float")
+        SECTION("n = UINT64_MAX is rejected (overflow)")
         {
             // n = UINT64_MAX (0xFFFFFFFFFFFFFFFF)
             // result = -1 - n = -18446744073709551616, which exceeds int64_t range
             const std::vector<uint8_t> input = {0x3B, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF};
-            const auto result = json::from_cbor(input);
-            CHECK(result.is_number_float());
-            CHECK(result.get<double>() == -18446744073709551616.0);
-            CHECK(result == json::parse("-18446744073709551616"));
+            json _;
+            CHECK_THROWS_WITH_AS(_ = json::from_cbor(input),
+                                 "[json.exception.parse_error.112] parse error at byte 9: syntax error while parsing CBOR value: negative integer overflow",
+                                 json::parse_error);
         }
 
-        SECTION("overflow with allow_exceptions=false is not an error")
+        SECTION("overflow with allow_exceptions=false returns discarded")
         {
             const std::vector<uint8_t> input = {0x3B, 0x80, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00};
             const auto result = json::from_cbor(input, true, false);
-            CHECK(result.is_number_float());
+            CHECK(result.is_discarded());
         }
     }
 

tests/src/unit-class_lexer.cpp

@@ -666,7 +666,7 @@ TEST_CASE("parse_float_fast declines what it cannot convert exactly")
     // always safe: the caller then falls back to a slower, exact conversion.
     const auto fast = [](const std::string & s, double & out)
     {
-        return nlohmann::detail::parse_float_fast(s.data(), s.data() + s.size(), '.', out);
+        return nlohmann::detail::parse_float_fast(s.data(), s.data() + s.size(), out);
     };
     double out = 0;
 

tests/src/unit-locale-cpp.cpp

@@ -12,7 +12,12 @@
 #include <nlohmann/json.hpp>
 using nlohmann::json;
 
+#include <array>
 #include <clocale>
+#include <map>
+#include <string>
+#include <utility>
+#include <vector>
 
 struct ParserImpl final: public nlohmann::json_sax<json>
 {
@@ -175,3 +180,208 @@ TEST_CASE("locale-dependent test (LC_NUMERIC=de_DE)")
         MESSAGE("locale de_DE is not usable");
     }
 }
+
+namespace
+{
+// records the numbers of a flat array and switches LC_NUMERIC to the given
+// locale once the array opens - after the lexer was constructed, but before
+// any number in the array is lexed
+struct LocaleSwitchingSax final: public nlohmann::json_sax<json>
+{
+    explicit LocaleSwitchingSax(const char* switch_to)
+        : locale_after_open(switch_to)
+    {}
+
+    bool null() override
+    {
+        return true;
+    }
+    bool boolean(bool /*val*/) override
+    {
+        return true;
+    }
+    bool number_integer(json::number_integer_t /*val*/) override
+    {
+        return true;
+    }
+    bool number_unsigned(json::number_unsigned_t /*val*/) override
+    {
+        return true;
+    }
+    bool number_float(json::number_float_t val, const json::string_t& s) override
+    {
+        values.push_back(val);
+        strings.push_back(s);
+        return true;
+    }
+    bool string(json::string_t& /*val*/) override
+    {
+        return true;
+    }
+    bool binary(json::binary_t& /*val*/) override
+    {
+        return true;
+    }
+    bool start_object(std::size_t /*val*/) override
+    {
+        return true;
+    }
+    bool key(json::string_t& /*val*/) override
+    {
+        return true;
+    }
+    bool end_object() override
+    {
+        return true;
+    }
+    bool start_array(std::size_t /*val*/) override
+    {
+        switched = std::setlocale(LC_NUMERIC, locale_after_open.c_str()) != nullptr;
+        return true;
+    }
+    bool end_array() override
+    {
+        return true;
+    }
+    bool parse_error(std::size_t /*val*/, const std::string& /*val*/, const nlohmann::detail::exception& /*val*/) override
+    {
+        return false;
+    }
+
+    std::string locale_after_open;
+    bool switched = false;
+    std::vector<json::number_float_t> values {}; // NOLINT(readability-redundant-member-init)
+    std::vector<json::string_t> strings {}; // NOLINT(readability-redundant-member-init)
+};
+} // namespace
+
+TEST_CASE("locale changes between lexer construction and number conversion (#5198)")
+{
+    // The numbers are chosen so that the conversion also takes the strtod
+    // fallback, which honors the locale that is current at conversion time:
+    // too many significant digits for Clinger's fast path, an underflow that
+    // std::from_chars rejects, and a plain value.
+    const std::vector<std::string> numbers = {"3.14159265358979323846", "1.5e-400", "12.34", "-0.000123456789012345678"};
+    std::string text = "[";
+    for (const auto& n : numbers)
+    {
+        text += (text.size() == 1 ? "" : ",") + n;
+    }
+    text += "]";
+
+    using long_double_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int64_t, std::uint64_t, long double>;
+
+    // reference values, parsed without a locale switch
+    REQUIRE(std::setlocale(LC_NUMERIC, "C") != nullptr);
+    const json expected = json::parse(text);
+    const long_double_json expected_ld = long_double_json::parse(text);
+
+    const std::array<std::pair<const char*, const char*>, 2> transitions =
+    {
+        {
+            {"C", "de_DE"},
+            {"de_DE", "C"}
+        }
+    };
+
+    for (const auto& transition : transitions)
+    {
+        CAPTURE(transition.first);
+        CAPTURE(transition.second);
+
+        if (std::setlocale(LC_NUMERIC, transition.first) == nullptr)
+        {
+            MESSAGE("locale is not usable");
+            continue;
+        }
+
+        // SAX parsing
+        {
+            LocaleSwitchingSax sax(transition.second);
+            CHECK(json::sax_parse(text, &sax));
+            if (sax.switched)
+            {
+                CHECK(sax.values == expected.get<std::vector<json::number_float_t>>());
+                CHECK(sax.strings == numbers);
+            }
+        }
+
+        // DOM parsing with a callback
+        {
+            bool switched = false;
+            const auto cb = [&](int /*depth*/, json::parse_event_t event, json& /*parsed*/)
+            {
+                if (event == json::parse_event_t::array_start)
+                {
+                    switched = std::setlocale(LC_NUMERIC, transition.second) != nullptr;
+                }
+                return true;
+            };
+            const json j = json::parse(text, cb);
+            if (switched)
+            {
+                CHECK(j == expected);
+            }
+        }
+
+        // a long double goes through std::strtold unless std::from_chars supports it
+        {
+            bool switched = false;
+            const auto cb = [&](int /*depth*/, long_double_json::parse_event_t event, long_double_json& /*parsed*/)
+            {
+                if (event == long_double_json::parse_event_t::array_start)
+                {
+                    switched = std::setlocale(LC_NUMERIC, transition.second) != nullptr;
+                }
+                return true;
+            };
+            const long_double_json j = long_double_json::parse(text, cb);
+            if (switched)
+            {
+                CHECK(j == expected_ld);
+            }
+        }
+    }
+
+    std::setlocale(LC_NUMERIC, "C");
+}
+
+TEST_CASE("locale with a multi-byte decimal point")
+{
+    // Some locales use a decimal point that is not a single character, e.g.
+    // U+066B ARABIC DECIMAL SEPARATOR (two bytes in UTF-8). It cannot be
+    // substituted in place for '.', so the strtod fallback stops early. The
+    // conversion must still terminate rather than retry forever.
+    const std::array<const char*, 6> names = {{"ar_EG.UTF-8", "ar_SA.UTF-8", "fa_IR.UTF-8", "ps_AF.UTF-8", "ar_EG", "fa_IR"}};
+    bool tested = false;
+    for (const char* name : names)
+    {
+        if (std::setlocale(LC_NUMERIC, name) == nullptr)
+        {
+            continue;
+        }
+        const std::string decimal_point = std::localeconv()->decimal_point;
+        if (decimal_point.size() < 2)
+        {
+            continue;
+        }
+        CAPTURE(name);
+        tested = true;
+
+        // too many significant digits for Clinger's fast path, and an underflow
+        // that std::from_chars rejects: both reach the strtod fallback
+        json j;
+        CHECK_NOTHROW(j = json::parse("[3.14159265358979323846, 1.5e-400, -0.000123456789012345678]"));
+        CHECK(j.is_array());
+        CHECK(json::accept("3.14159265358979323846"));
+
+        // a value the locale-independent paths convert is not affected
+        CHECK(json::parse("12.5") == 12.5);
+    }
+    if (!tested)
+    {
+        MESSAGE("no locale with a multi-byte decimal point is usable");
+    }
+
+    std::setlocale(LC_NUMERIC, "C");
+}