Compare commits

..
Author SHA1 Message Date
Niels Lohmann 4c43e03d40 Merge CI fixes into json-view/23-zmij
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-08 16:39:25 +02:00
Niels Lohmann d22193a8b6 Fix CI findings in the Zmij writer and its tests
- bit_ops.hpp: include macro_scope.hpp (JSON_HEDLEY_ALWAYS_INLINE); fixes IWYU
- to_chars.hpp: C4100 for the unused parameter in release builds, clang-tidy
  sign comparison, cpplint runtime/int, GCC -Wstrict-overflow (unsigned abs)
- unit-to_chars.cpp: parse with the library instead of strtod (MinGW's strtod
  rounds some 16/17 digit inputs wrongly); no floating-point std::to_chars
  with icpc

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-08 16:33:32 +02:00
Niels Lohmann 22b82b487b Merge branch 'json-view/02b-float-parser' into json-view/23-zmij
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-08 16:25:42 +02:00
Niels Lohmann 43c75bef51 Merge branch 'develop' into json-view/02b-float-parser
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

# Conflicts:
#	tests/src/unit-class_lexer.cpp
2026-10-08 16:19:06 +02:00
Niels Lohmann cb3c0177ed Merge branch 'json-view/02b-float-parser' into json-view/23-zmij
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-07 20:26:39 +02:00
Niels Lohmann 39d34f30ba Merge branch 'develop' into json-view/02b-float-parser
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-07 20:26:13 +02:00
Niels Lohmann 88ddacb84b Fix CI warnings in own float parser
- pow5_table.hpp: pow5_128_largest_power was unused in this branch's
  own code (GCC -Werror=unused-const-variable); tie it to the table
  size with a static_assert instead of removing it, since a later
  branch in the stack (json-view/23-zmij) uses it.
- number_parse.hpp: rename the local variable `copy` to `buffer` to
  satisfy cpplint's build/include_what_you_use check.
- unit-class_lexer.cpp: extend the NOLINT list on the seeded mt19937
  with bugprone-random-generator-seed, and parenthesize
  `8 * sizeof(Bits) - 1` for clang-tidy.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-07 20:26:12 +02:00
Niels Lohmann bfa4886f0f Write doubles with the shortest digits (Żmij), digits in registers
Write doubles with the conversion of Zmij by Victor Zverovich (MIT),
ported to C++11 (detail/conversions/zmij.hpp). It finds the
shortest decimal that reads back as the same double, and the
closest one if there are several. Grisu2, used until now, is fast
but not always shortest: it sometimes writes a 17th digit where 16
suffice, or a last digit that is not the closest. The layout is
unchanged (1.5, 100.0, 1e+100, -0.0); float keeps Grisu2.

Digits are converted eight at a time with the BCD conversion of
Xiang JunBo, as in Zmij, and written with one byte swap per eight
digits and fixed-size moves instead of per-digit loops. Leading and
trailing zeros are counted from those bytes. to_chars() uses a
local buffer when the caller's is shorter than the 41 bytes this
may write. The powers of ten come from the number-parsing table,
adjusted where it holds values rounded up, and extended with Zmij's
compressed tables beyond 10^308.

write_shortest() converts its 16 digits in one vector register
(SSE2 on x86-64, NEON on 64-bit Arm, both baseline) and inserts the
decimal point inside the register, avoiding a store-forwarding
stall that cost about 25% of the time to write a double. dump()
writes floats and integers straight into the serializer's write
buffer instead of copying them from a member buffer, and small
integers eight digits at a time. read_eight_bytes() and
parse_eight_digits() are marked always-inline, which GCC had been
calling out of line in the number-parsing loops.

Of one million random doubles, about 0.14% are now written with
different digits, always to a value that still reads back as the
same double.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-07 16:41:47 +02:00
Niels Lohmann 68d61b6aef Speed up the lexer: own float parser, string scan, and \u table
Give the library its own correctly rounded float converter for
binary32 and binary64 (IEEE 754), and speed up the lexer's string
and escape scanning.

The converter splits a number token into sign, significand, and
decimal exponent, then tries Clinger's fast path, then a templated
Eisel-Lemire step, and falls back to an exact big-integer digit
comparison for tokens with more than 19 significant digits whose two
candidate values round differently. This replaces std::from_chars
and strtod/strtof for both formats, so parsed values no longer
depend on the C/C++ library or the current locale. The strtold
fallback kept for other long double formats (x87, binary128) now
also copies a multi-byte decimal point correctly, fixing #5660.
eisel_lemire() and decimal_to_float() are always inlined so callers
keep the whole conversion in their hot loop.

The string-scanning kernels in string_scan.hpp find a stop byte with
the trailing-zero count of the SWAR mask instead of a byte loop, and
scalar_string_bulk_run() validates a run of multi-byte UTF-8
sequences one after another instead of re-searching after each one.

get_codepoint() decodes a contiguous \uXXXX escape with one table
lookup per byte instead of four range-checked get() calls; the
streaming path and all error positions are unchanged.

Adds 508 generated hard float-parsing cases with expected binary32
and binary64 bits, and kernel-comparison tests for the string scans
and the escape table against byte-by-byte references.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-07 16:41:43 +02:00
46 changed files with 6159 additions and 2431 deletions

No files matched your search

+1
View File
@@ -25,6 +25,7 @@ cc_library(
"include/nlohmann/detail/conversions/from_json.hpp",
"include/nlohmann/detail/conversions/to_chars.hpp",
"include/nlohmann/detail/conversions/to_json.hpp",
"include/nlohmann/detail/conversions/zmij.hpp",
"include/nlohmann/detail/exceptions.hpp",
"include/nlohmann/detail/hash.hpp",
"include/nlohmann/detail/input/binary_reader.hpp",
+2 -1
View File
@@ -1394,9 +1394,10 @@ THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR I
- The class contains the UTF-8 Decoder from Bjoern Hoehrmann which is licensed under the [MIT License](https://opensource.org/licenses/MIT) (see above). Copyright &copy; 2008-2009 [Björn Hoehrmann](https://bjoern.hoehrmann.de/) <bjoern@hoehrmann.de>
- The class contains a slightly modified version of the Grisu2 algorithm from Florian Loitsch which is licensed under the [MIT License](https://opensource.org/licenses/MIT) (see above). Copyright &copy; 2009 [Florian Loitsch](https://florian.loitsch.com/)
- The class contains a port of the shortest double-to-decimal conversion of [Żmij](https://github.com/vitaut/zmij) by Victor Zverovich, which is licensed under the [MIT License](https://opensource.org/licenses/MIT) (see above). Copyright &copy; 2025 [Victor Zverovich](https://github.com/vitaut)
- The class contains a copy of [Hedley](https://nemequ.github.io/hedley/) from Evan Nemerson which is licensed as [CC0-1.0](https://creativecommons.org/publicdomain/zero/1.0/).
- The class contains parts of [Google Abseil](https://github.com/abseil/abseil-cpp) which is licensed under the [Apache 2.0 License](https://opensource.org/licenses/Apache-2.0).
- The class contains an adapted version of the Eisel-Lemire algorithm and its table of powers of five from [fast_float](https://github.com/fastfloat/fast_float) by Daniel Lemire and contributors, which is available under the [MIT License](https://opensource.org/licenses/MIT) (used here), the Apache 2.0 License, and the Boost Software License. Copyright &copy; 2021 The fast_float authors
- The class contains an adapted version of the Eisel-Lemire algorithm, its table of powers of five, and its digit comparison for long numbers from [fast_float](https://github.com/fastfloat/fast_float) by Daniel Lemire and contributors, which is available under the [MIT License](https://opensource.org/licenses/MIT) (used here), the Apache 2.0 License, and the Boost Software License. Copyright &copy; 2021 The fast_float authors
<img align="right" src="https://git.fsfe.org/reuse/reuse-ci/raw/branch/master/reuse-horizontal.png" alt="REUSE Software">
+5
View File
@@ -62,6 +62,9 @@ Linear.
## Notes
Floating-point numbers are written with the fewest digits that read back as the same value (for `#!cpp double`; see
[number handling](../../features/types/number_handling.md#number-serialization)).
Binary values are serialized as an object containing two keys:
- "bytes": an array of bytes as integers
@@ -97,3 +100,5 @@ Binary values are serialized as an object containing two keys:
- Error handlers added in version 3.4.0.
- Serialization of binary values added in version 3.8.0.
- Error handler `keep` added in version 3.13.0.
- Doubles are written with the shortest digits (Żmij instead of Grisu2) since version 3.13.0; about 0.1% of doubles are
written differently, most of them with fewer digits.
@@ -23,9 +23,10 @@ type to use.
## Template parameters
`NumberFloatType`
: the type to store floating-point numbers. Parsing and serialization are implemented in terms of
`#!cpp std::strtof`/`#!cpp std::strtod`/`#!cpp std::strtold` and `#!cpp std::snprintf`, so the type must be
`#!cpp float`, `#!cpp double`, or `#!cpp long double`. The
: the type to store floating-point numbers. The parser converts `#!cpp float`, `#!cpp double`, and a
`#!cpp long double` that is IEEE 754 binary64 itself and other `#!cpp long double` formats with
`#!cpp std::from_chars` or `#!cpp std::strtold`, and serialization falls back to `#!cpp std::snprintf`, so the
type must be `#!cpp float`, `#!cpp double`, or `#!cpp long double`. The
[binary formats](../../features/binary_formats/index.md) additionally require `#!cpp float` or `#!cpp double`,
because they have no encoding for `#!cpp long double`. See
[Template Parameter Requirements](../../features/types/template_parameters.md#numberfloattype).
@@ -82,12 +82,13 @@ flowchart TD
- Numbers with a decimal digit or scientific notation are always stored as `#!c double`.
- The number types can be changed, see [Template number types](#template-number-types).
- Integers are converted by the library's own digit parser. Floating-point numbers are converted with
[`std::from_chars`](https://en.cppreference.com/w/cpp/utility/from_chars) if the library is compiled with C++17
and the standard library supports it, then with an exact fast path for `#!c double` values with few significant
digits, and otherwise with the locale-aware
[`std::strtod`](https://en.cppreference.com/w/cpp/string/byte/strtof) (`std::strtof`/`std::strtold` for the
other floating-point types). Before version 3.13.0, the conversion was realized by
- The library converts integers and floating-point numbers itself, independent of the locale. Floating-point
numbers are correctly rounded (to nearest, ties to even). Only a `#!c long double` that is not IEEE 754 binary64
(e.g., the 80-bit x87 format) is converted with `#!cpp std::from_chars` where available, or else with
[`std::strtold`](https://en.cppreference.com/w/cpp/string/byte/strtof). For that call, the library temporarily
replaces the `.` with the decimal point of the current locale (which may be longer than one byte, e.g., in
`fa_IR.UTF-8`), so the result does not depend on the locale either. Changing the locale in another thread during
parsing is undefined behavior of the C library, though. Before version 3.13.0, the conversion was realized by
[`std::strtoull`](https://en.cppreference.com/w/cpp/string/byte/strtoul),
[`std::strtoll`](https://en.cppreference.com/w/cpp/string/byte/strtol), and `std::strtod`, respectively.
@@ -100,10 +101,10 @@ flowchart TD
### Number limits
- Any 64-bit signed or unsigned integer can be stored without loss of precision.
- Numbers exceeding the limits of `#!c double` (i.e., numbers that after conversion via
[`std::strtod`](https://en.cppreference.com/w/cpp/string/byte/strtof) are not satisfying
- Numbers exceeding the limits of `#!c double` (i.e., numbers whose rounded value is not satisfying
[`std::isfinite`](https://en.cppreference.com/w/cpp/numeric/math/isfinite) such as `#!c 1E400`) will throw exception
[`json.exception.out_of_range.406`](../../home/exceptions.md#jsonexceptionout_of_range406) during parsing.
[`json.exception.out_of_range.406`](../../home/exceptions.md#jsonexceptionout_of_range406) during parsing. Numbers too
small for `#!c double` (such as `#!c 1E-400`) become zero, with the sign of the number.
- Floating-point numbers are rounded to the next number representable as `double`. For instance
`#!c 3.141592653589793238462643383279` is stored as [`0x400921fb54442d18`](https://float.exposed/0x400921fb54442d18).
This is the same behavior as the code `#!c double x = 3.141592653589793238462643383279;`.
@@ -133,9 +134,10 @@ That is, `-0` is stored as a signed integer, but the serialization does not repr
### Number serialization
- Integer numbers are serialized as is; that is, no scientific notation is used.
- Floating-point numbers are serialized as specified by the `#!c %g` printf modifier with
[`std::numeric_limits<double>::max_digits10`](https://en.cppreference.com/w/cpp/types/numeric_limits/max_digits10)
significant digits. The rationale is to use the shortest representation while still allowing round-tripping.
- Floating-point numbers are serialized with the fewest digits that read back as the same value (the closest such
digits if there are several), in the layout of the `#!c %g` printf modifier: `#!c 1.5`, `#!c 100.0`, `#!c 1e+100`.
Doubles are converted with the algorithm of [Żmij](https://github.com/vitaut/zmij), floats with Grisu2, which
can write more digits than necessary.
!!! hint "Notes regarding precision of floating-point numbers"
@@ -30,9 +30,9 @@ Requirements are split into two groups:
diagnosed with dedicated error messages, and violating most of them results in a compiler error somewhere inside
the library. Four violations are not caught at compile time at all:
- A [`StringType`](#stringtype) whose `data()` is not null-terminated compiles and can silently misparse
floating-point numbers, because the lexer may hand the buffer to `#!cpp std::strtod`, which reads up to the
terminating null character.
- A [`StringType`](#stringtype) whose `data()` is not null-terminated compiles and silently misparses numbers
stored as a `#!cpp long double` that is not IEEE 754 binary64 (e.g., the 80-bit x87 format), because the lexer
hands the buffer to `#!cpp std::strtold`.
- A stateful [`AllocatorType`](#allocatortype) compiles and silently ignores its state: allocation, deallocation,
and [`get_allocator()`](../../api/basic_json/get_allocator.md) each use a different default-constructed instance.
- The two [cross-specialization conversions](#cross-specialization-conversions) below. These abort on an assertion
@@ -541,16 +541,18 @@ therefore silently changes parse results rather than raising an error. See
`NumberFloatType` must be one of `#!cpp float`, `#!cpp double`, or `#!cpp long double`:
- The [parser](../parsing/index.md) converts number literals with `#!cpp std::from_chars` or, as a fallback, with
`#!cpp std::strtof`, `#!cpp std::strtod`, or `#!cpp std::strtold`; the library provides overloads for exactly these
three types.
- The [parser](../parsing/index.md) converts number literals to `#!cpp float`, `#!cpp double`, and a
`#!cpp long double` that is IEEE 754 binary64 itself; other `#!cpp long double` formats are converted with
`#!cpp std::from_chars` where available, or with `#!cpp std::strtold`. The library provides overloads for exactly
these three types.
- [`dump`](../../api/basic_json/dump.md) falls back to `#!cpp std::snprintf` with the `%g` and `%Lg` conversion
specifiers, for which the library likewise provides only `#!cpp double` and `#!cpp long double` overloads
(`#!cpp float` is promoted to `#!cpp double`).
If `#!cpp std::numeric_limits<NumberFloatType>` describes an IEEE 754 binary32 or binary64 number, `dump` uses the
Grisu2 algorithm, which produces the shortest representation that round-trips. Otherwise the `snprintf` fallback with
`max_digits10` digits is used.
If `#!cpp std::numeric_limits<NumberFloatType>` describes an IEEE 754 binary64 number, `dump` uses the algorithm of
Żmij, which produces the shortest representation that round-trips. For IEEE 754 binary32 numbers, it uses Grisu2,
which produces a short representation that round-trips. Otherwise the `snprintf` fallback with `max_digits10` digits is
used.
### Required for the binary formats
@@ -562,7 +564,7 @@ binary32 or binary64 field and have no encoding for `#!cpp long double`.
| Type | Support |
|--------------------------|-----------------------------------------------------------------------------------------------------------------------|
| `#!cpp double` (default) | full; short round-trip output through Grisu2 |
| `#!cpp double` (default) | full; shortest round-trip output through Żmij |
| `#!cpp float` | full; short round-trip output through Grisu2 |
| `#!cpp long double` | `dump` and `parse` only; the binary format writers do not compile, as they only handle IEEE 754 binary32 and binary64 |
| any other type | not usable |
+3 -1
View File
@@ -18,6 +18,8 @@ The class contains the UTF-8 Decoder from Bjoern Hoehrmann which is licensed und
The class contains a slightly modified version of the Grisu2 algorithm from Florian Loitsch which is licensed under the [MIT License](https://opensource.org/licenses/MIT) (see above). Copyright &copy; 2009 [Florian Loitsch](https://florian.loitsch.com/)
The class contains a port of the shortest double-to-decimal conversion of [Żmij](https://github.com/vitaut/zmij) by Victor Zverovich, which is licensed under the [MIT License](https://opensource.org/licenses/MIT) (see above). Copyright &copy; 2025 [Victor Zverovich](https://github.com/vitaut)
The class contains a copy of [Hedley](https://nemequ.github.io/hedley/) from Evan Nemerson which is licensed as [CC0-1.0](https://creativecommons.org/publicdomain/zero/1.0/).
The class contains an adapted version of the Eisel-Lemire algorithm and its table of powers of five from [fast_float](https://github.com/fastfloat/fast_float) by Daniel Lemire and contributors, which is available under the [MIT License](https://opensource.org/licenses/MIT) (used here), the Apache 2.0 License, and the Boost Software License. Copyright &copy; 2021 The fast_float authors
The class contains an adapted version of the Eisel-Lemire algorithm, its table of powers of five, and its digit comparison for long numbers from [fast_float](https://github.com/fastfloat/fast_float) by Daniel Lemire and contributors, which is available under the [MIT License](https://opensource.org/licenses/MIT) (used here), the Apache 2.0 License, and the Boost Software License. Copyright &copy; 2021 The fast_float authors
+29 -4
View File
@@ -13,7 +13,7 @@
#include <intrin0.h> // __umulh, _umul128
#endif
#include <nlohmann/detail/abi_macros.hpp>
#include <nlohmann/detail/macro_scope.hpp> // JSON_HEDLEY_ALWAYS_INLINE, NLOHMANN_JSON_NAMESPACE_BEGIN
// Portable bit-level helpers for the number and string scanners. They use
// compiler builtins or platform-specific intrinsics where available and plain
@@ -42,6 +42,25 @@ inline int count_leading_zeros(std::uint64_t x) noexcept
#endif
}
/// number of trailing zero bits of x (x != 0)
inline int count_trailing_zeros(std::uint64_t x) noexcept
{
#if defined(__GNUC__) || defined(__clang__)
return __builtin_ctzll(x);
#else
int n = 0;
for (int shift = 32; shift != 0; shift >>= 1)
{
if ((x << (64 - shift)) == 0)
{
n += shift;
x >>= shift;
}
}
return n;
#endif
}
/// the 128-bit product of two 64-bit numbers
struct uint128_parts
{
@@ -76,15 +95,21 @@ inline uint128_parts full_multiplication(std::uint64_t a, std::uint64_t b) noexc
}
/// eight bytes as a little-endian word (compilers fold this into one load on
/// little-endian targets)
inline std::uint64_t read_eight_bytes(const char* p) noexcept
/// little-endian targets; always inlined, as GCC otherwise calls it in the
/// number loops)
JSON_HEDLEY_ALWAYS_INLINE std::uint64_t read_eight_bytes(const unsigned char* b) noexcept
{
const auto* b = reinterpret_cast<const unsigned char*>(p); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
return static_cast<std::uint64_t>(b[0]) | (static_cast<std::uint64_t>(b[1]) << 8u)
| (static_cast<std::uint64_t>(b[2]) << 16u) | (static_cast<std::uint64_t>(b[3]) << 24u)
| (static_cast<std::uint64_t>(b[4]) << 32u) | (static_cast<std::uint64_t>(b[5]) << 40u)
| (static_cast<std::uint64_t>(b[6]) << 48u) | (static_cast<std::uint64_t>(b[7]) << 56u);
}
/// eight bytes as a little-endian word
JSON_HEDLEY_ALWAYS_INLINE std::uint64_t read_eight_bytes(const char* p) noexcept
{
return read_eight_bytes(reinterpret_cast<const unsigned char*>(p)); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
}
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
@@ -48,7 +48,7 @@ inline void from_json(const BasicJsonType& j, typename std::nullptr_t& n)
{
if (JSON_HEDLEY_UNLIKELY(!j.is_null()))
{
throw_type_must_be("null", j);
JSON_THROW(type_error::create(302, concat("type must be null, but is ", j.type_name()), &j));
}
n = nullptr;
}
@@ -102,7 +102,7 @@ void get_arithmetic_value(const BasicJsonType& j, ArithmeticType& val)
case value_t::binary:
case value_t::discarded:
default:
throw_type_must_be("number", j);
JSON_THROW(type_error::create(302, concat("type must be number, but is ", j.type_name()), &j));
}
}
@@ -111,7 +111,7 @@ inline void from_json(const BasicJsonType& j, typename BasicJsonType::boolean_t&
{
if (JSON_HEDLEY_UNLIKELY(!j.is_boolean()))
{
throw_type_must_be("boolean", j);
JSON_THROW(type_error::create(302, concat("type must be boolean, but is ", j.type_name()), &j));
}
b = *j.template get_ptr<const typename BasicJsonType::boolean_t*>();
}
@@ -121,7 +121,7 @@ inline void from_json(const BasicJsonType& j, typename BasicJsonType::string_t&
{
if (JSON_HEDLEY_UNLIKELY(!j.is_string()))
{
throw_type_must_be("string", j);
JSON_THROW(type_error::create(302, concat("type must be string, but is ", j.type_name()), &j));
}
s = *j.template get_ptr<const typename BasicJsonType::string_t*>();
}
@@ -137,7 +137,7 @@ inline void from_json(const BasicJsonType& j, StringType& s)
{
if (JSON_HEDLEY_UNLIKELY(!j.is_string()))
{
throw_type_must_be("string", j);
JSON_THROW(type_error::create(302, concat("type must be string, but is ", j.type_name()), &j));
}
s = *j.template get_ptr<const typename BasicJsonType::string_t*>();
@@ -183,7 +183,7 @@ inline void from_json(const BasicJsonType& j, std::forward_list<T, Allocator>& l
{
if (JSON_HEDLEY_UNLIKELY(!j.is_array()))
{
throw_type_must_be("array", j);
JSON_THROW(type_error::create(302, concat("type must be array, but is ", j.type_name()), &j));
}
l.clear();
std::transform(j.rbegin(), j.rend(),
@@ -200,7 +200,7 @@ inline void from_json(const BasicJsonType& j, std::valarray<T>& l)
{
if (JSON_HEDLEY_UNLIKELY(!j.is_array()))
{
throw_type_must_be("array", j);
JSON_THROW(type_error::create(302, concat("type must be array, but is ", j.type_name()), &j));
}
l.resize(j.size());
std::transform(j.begin(), j.end(), std::begin(l),
@@ -305,7 +305,7 @@ void())
{
if (JSON_HEDLEY_UNLIKELY(!j.is_array()))
{
throw_type_must_be("array", j);
JSON_THROW(type_error::create(302, concat("type must be array, but is ", j.type_name()), &j));
}
from_json_array_impl(j, arr, priority_tag<3> {});
@@ -324,7 +324,7 @@ auto from_json(const BasicJsonType& j, identity_tag<std::array<T, N>> tag)
{
if (JSON_HEDLEY_UNLIKELY(!j.is_array()))
{
throw_type_must_be("array", j);
JSON_THROW(type_error::create(302, concat("type must be array, but is ", j.type_name()), &j));
}
return from_json_inplace_array_impl(j, tag, make_index_sequence<N> {});
@@ -335,7 +335,7 @@ inline void from_json(const BasicJsonType& j, typename BasicJsonType::binary_t&
{
if (JSON_HEDLEY_UNLIKELY(!j.is_binary()))
{
throw_type_must_be("binary", j);
JSON_THROW(type_error::create(302, concat("type must be binary, but is ", j.type_name()), &j));
}
bin = *j.template get_ptr<const typename BasicJsonType::binary_t*>();
@@ -356,7 +356,7 @@ inline void from_json(const BasicJsonType& j, CompatibleArrayType& bin)
}
else
{
throw_type_must_be("binary or array", j);
JSON_THROW(type_error::create(302, concat("type must be binary or array, but is ", j.type_name()), &j));
}
}
@@ -377,7 +377,7 @@ inline void from_json(const BasicJsonType& j, ConstructibleObjectType& obj)
{
if (JSON_HEDLEY_UNLIKELY(!j.is_object()))
{
throw_type_must_be("object", j);
JSON_THROW(type_error::create(302, concat("type must be object, but is ", j.type_name()), &j));
}
ConstructibleObjectType ret;
@@ -432,7 +432,7 @@ inline void from_json(const BasicJsonType& j, ArithmeticType& val)
case value_t::binary:
case value_t::discarded:
default:
throw_type_must_be("number", j);
JSON_THROW(type_error::create(302, concat("type must be number, but is ", j.type_name()), &j));
}
}
@@ -504,7 +504,7 @@ auto from_json(const BasicJsonType& j, TupleRelated&& t)
{
if (JSON_HEDLEY_UNLIKELY(!j.is_array()))
{
throw_type_must_be("array", j);
JSON_THROW(type_error::create(302, concat("type must be array, but is ", j.type_name()), &j));
}
return from_json_tuple_impl(j, std::forward<TupleRelated>(t), priority_tag<3> {});
@@ -517,14 +517,14 @@ void from_json_pair_array_to_map(const BasicJsonType& j, MapType& m)
{
if (JSON_HEDLEY_UNLIKELY(!j.is_array()))
{
throw_type_must_be("array", j);
JSON_THROW(type_error::create(302, concat("type must be array, but is ", j.type_name()), &j));
}
m.clear();
for (const auto& p : j)
{
if (JSON_HEDLEY_UNLIKELY(!p.is_array()))
{
throw_type_must_be("array", p);
JSON_THROW(type_error::create(302, concat("type must be array, but is ", p.type_name()), &p));
}
m.emplace(p.at(0).template get<typename MapType::key_type>(), p.at(1).template get<typename MapType::mapped_type>());
}
@@ -591,7 +591,7 @@ inline void from_json(const BasicJsonType& j, std_fs::path& p)
{
if (JSON_HEDLEY_UNLIKELY(!j.is_string()))
{
throw_type_must_be("string", j);
JSON_THROW(type_error::create(302, concat("type must be string, but is ", j.type_name()), &j));
}
const auto& s = *j.template get_ptr<const typename BasicJsonType::string_t*>();
// Checking for C++20 standard or later can be insufficient in case the
+473 -23
View File
@@ -11,11 +11,32 @@
#include <array> // array
#include <cmath> // signbit, isfinite
#include <cstddef> // size_t
#include <cstdint> // intN_t, uintN_t
#include <cstring> // memcpy, memmove
#include <limits> // numeric_limits
#include <type_traits> // conditional
#ifdef _MSC_VER
#include <cstdlib> // _byteswap_uint64
#endif
// SSE2 (every x86-64 CPU) and NEON (every 64-bit Arm CPU) convert the 16
// digits of a double at once
#if defined(__x86_64__) || (defined(_M_X64) && !defined(_M_ARM64EC))
#include <emmintrin.h>
#define JSON_DTOA_SSE2 1
#define JSON_DTOA_NEON 0
#elif (defined(__aarch64__) || defined(_M_ARM64)) && !defined(_M_ARM64EC) && !defined(__ARM_BIG_ENDIAN)
#include <arm_neon.h>
#define JSON_DTOA_SSE2 0
#define JSON_DTOA_NEON 1
#else
#define JSON_DTOA_SSE2 0
#define JSON_DTOA_NEON 0
#endif
#include <nlohmann/detail/conversions/zmij.hpp>
#include <nlohmann/detail/macro_scope.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN
@@ -918,6 +939,88 @@ void grisu2(char* buf, int& len, int& decimal_exponent, FloatType value)
grisu2(buf, len, decimal_exponent, w.minus, w.w, w.plus);
}
/*!
@brief the shortest digits of a positive finite float (other than double): Grisu2
*/
template<typename FloatType>
JSON_HEDLEY_NON_NULL(1)
void shortest_digits(char* buf, int& len, int& decimal_exponent, FloatType value)
{
grisu2(buf, len, decimal_exponent, value);
}
/*!
@brief the shortest digits of a positive finite double: the conversion of
Zmij (see zmij.hpp), which always finds the shortest digits that read back as
the same value (Grisu2 does not for about one double in a thousand), and the
closest of them if there are several
v = buf * 10^decimal_exponent, as for grisu2()
*/
JSON_HEDLEY_NON_NULL(1)
inline void shortest_digits(char* buf, int& len, int& decimal_exponent, double value)
{
static_assert(std::numeric_limits<double>::is_iec559 && std::numeric_limits<double>::digits == 53,
"internal error: the conversion of Zmij needs IEEE 754 binary64 doubles");
JSON_ASSERT(std::isfinite(value));
JSON_ASSERT(value > 0);
std::uint64_t bits = 0;
std::memcpy(&bits, &value, sizeof(bits));
zmij::decimal d = zmij::to_decimal(bits);
// without trailing zeros (up to 16): 8, 4, 2, 1 at a time
while (d.significand % 100000000 == 0)
{
d.significand /= 100000000;
d.exponent += 8;
}
if (d.significand % 10000 == 0)
{
d.significand /= 10000;
d.exponent += 4;
}
if (d.significand % 100 == 0)
{
d.significand /= 100;
d.exponent += 2;
}
if (d.significand % 10 == 0)
{
d.significand /= 10;
d.exponent += 1;
}
// at most 17 digits, written from the back two at a time
static constexpr const char* pairs =
"00010203040506070809101112131415161718192021222324252627282930313233343536373839"
"40414243444546474849505152535455565758596061626364656667686970717273747576777879"
"8081828384858687888990919293949596979899";
std::array<char, 20> digits{};
std::size_t n = digits.size();
while (d.significand >= 100)
{
const std::uint64_t two_digits = d.significand % 100; // a variable: GCC calls a cast of the remainder useless where std::uint64_t is std::size_t
const auto i = static_cast<std::size_t>(two_digits) * 2;
d.significand /= 100;
n -= 2;
digits[n] = pairs[i];
digits[n + 1] = pairs[i + 1];
}
if (d.significand >= 10)
{
const auto i = static_cast<std::size_t>(d.significand) * 2;
n -= 2;
digits[n] = pairs[i];
digits[n + 1] = pairs[i + 1];
}
else
{
digits[--n] = static_cast<char>('0' + d.significand);
}
len = static_cast<int>(digits.size() - n);
std::memcpy(buf, digits.data() + n, static_cast<std::size_t>(len));
decimal_exponent = d.exponent;
}
/*!
@brief appends a decimal representation of e to buf
@return a pointer to the element following the exponent.
@@ -1047,6 +1150,375 @@ inline char* format_buffer(char* buf, int len, int decimal_exponent,
return append_exponent(buf, n - 1);
}
/// eight decimal digits (a value below 10^8) as bytes 0..9, the first digit
/// in the most significant byte: three steps that divide all lanes at once
/// by a multiplication (the conversion of Xiang JunBo, as in Zmij)
inline std::uint64_t eight_digit_bytes(std::uint64_t abcdefgh) noexcept
{
const std::uint64_t abcd_efgh = abcdefgh + (((std::uint64_t{1} << 32u) - 10000u) * ((abcdefgh * (((std::uint64_t{1} << 40u) / 10000u) + 1u)) >> 40u));
const std::uint64_t ab_cd_ef_gh = abcd_efgh + (((std::uint64_t{1} << 16u) - 100u) * (((abcd_efgh * (((std::uint64_t{1} << 19u) / 100u) + 1u)) >> 19u) & 0x7F0000007Fu));
return ab_cd_ef_gh + (((std::uint64_t{1} << 8u) - 10u) * (((ab_cd_ef_gh * (((std::uint64_t{1} << 10u) / 10u) + 1u)) >> 10u) & 0x000F000F000F000Fu));
}
/// store the bytes of v, the most significant one first (one byte swap and
/// one store where the byte order is known: compilers do not reliably merge
/// the byte stores once this is inlined)
inline void store_msb_first(char* p, std::uint64_t v) noexcept
{
#if defined(__BYTE_ORDER__) && defined(__ORDER_LITTLE_ENDIAN__) && __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
v = __builtin_bswap64(v);
std::memcpy(p, &v, sizeof(v));
#elif defined(__BYTE_ORDER__) && defined(__ORDER_BIG_ENDIAN__) && __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__
std::memcpy(p, &v, sizeof(v));
#elif defined(_MSC_VER) // (little-endian on all its targets)
v = _byteswap_uint64(v);
std::memcpy(p, &v, sizeof(v));
#else
for (unsigned i = 0; i < 8; ++i)
{
p[i] = static_cast<char>(v >> (56u - (8u * i)));
}
#endif
}
/*!
@brief digits * 10^exp for a double, in the layout of format_buffer()
The layout is that of format_buffer() with min_exp -4 and max_exp 15 (the
digits10 of double). The digits are converted eight at a time and placed
with fixed-size moves instead of per-digit loops and moves of the buffer.
@param[in] digits the digits (not 0, at most 17 digits; trailing zeros allowed)
@param[in] exp the decimal exponent of the last digit
@return a pointer past the text; up to 41 bytes at @a first are written
(some beyond the returned end)
*/
JSON_HEDLEY_NON_NULL(1)
JSON_HEDLEY_RETURNS_NON_NULL
inline char* write_decimal(char* first, std::uint64_t digits, int exp) noexcept
{
JSON_ASSERT(digits != 0 && digits < 100000000000000000u);
const std::uint64_t upper = digits / 100000000u;
const std::uint64_t b0 = upper / 100000000u; // (one digit: it is its own byte)
const std::uint64_t b1 = eight_digit_bytes(upper % 100000000u);
const std::uint64_t b2 = eight_digit_bytes(digits % 100000000u);
// leading and trailing zero digits: zero bytes, counted without division
int leading = 16;
int zeros = 16;
if (b0 != 0)
{
leading = count_leading_zeros(b0) / 8;
}
else if (b1 != 0)
{
leading = 8 + (count_leading_zeros(b1) / 8);
}
else
{
leading += count_leading_zeros(b2) / 8;
}
if (b2 != 0)
{
zeros = count_trailing_zeros(b2) / 8;
}
else if (b1 != 0)
{
zeros = 8 + (count_trailing_zeros(b1) / 8);
}
// (else: 16, b0 is the one digit that is not 0)
// the digits as text at text + leading, then '0's, so that fixed-size
// moves need not check how many digits there are
std::array<char, 64> text; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
store_msb_first(text.data(), b0 + 0x3030303030303030u);
store_msb_first(text.data() + 8, b1 + 0x3030303030303030u);
store_msb_first(text.data() + 16, b2 + 0x3030303030303030u);
std::memset(text.data() + 24, '0', 40);
const int k = 24 - leading - zeros; // significant digits
const int n = k + exp + zeros; // position of the decimal point after the first digit
const char* const s0 = text.data() + leading;
if (-4 < n && n <= 15)
{
// "0.[000]digits" (n <= 0) is the digits after 1 - n leading '0's
// with the point after the first; "digits[000].0" (n >= k) and
// "dig.its" put the point after n characters
const int pad = n <= 0 ? 1 - n : 0;
const char* const s = s0 - pad;
const int len = k + pad;
const int point = n + pad;
std::memcpy(first, s, 16);
std::memcpy(first + point + 1, s + point, 24);
first[point] = '.';
return first + (point >= len ? point + 2 : len + 1);
}
// d.igitse+XX, with at least two exponent digits (as append_exponent())
std::memcpy(first, s0, 16);
std::memcpy(first + 2, s0 + 1, 16);
first[1] = '.';
char* const end = first + (k == 1 ? 1 : k + 1);
const int e = n - 1;
const auto ea = e < 0 ? 0u - static_cast<unsigned>(e) : static_cast<unsigned>(e); // (unsigned: no signed overflow to assume)
const bool three = ea >= 100;
end[0] = 'e';
end[1] = e < 0 ? '-' : '+';
end[2] = static_cast<char>('0' + (three ? ea / 100 : (ea / 10) % 10));
end[3] = static_cast<char>('0' + (three ? (ea / 10) % 10 : ea % 10));
end[4] = static_cast<char>('0' + (ea % 10));
return end + (three ? 5 : 4);
}
/*!
@brief the shortest decimal of a positive double (Zmij), as write_decimal()
writes it
For a normal double, the shorter candidate has 15 or 16 digits: they are
converted at once (two halves of eight digits) and followed by the digit
after them, if there is one, without the multiplication and division by 10
that counting the digits of one number would take. The fixed layouts move
the digits after the point by one byte.
@return a pointer past the text; up to 41 bytes at @a first are written
(some beyond the returned end)
*/
JSON_HEDLEY_NON_NULL(1)
JSON_HEDLEY_RETURNS_NON_NULL
inline char* write_shortest(char* first, const zmij::shortest_decimal d) noexcept
{
const std::uint64_t sig = d.integral;
if (JSON_HEDLEY_UNLIKELY(sig < 100000000000000u || sig >= 10000000000000000u))
{
// (subnormals)
return d.has_digit ? write_decimal(first, (sig * 10) + d.digit, d.exponent) : write_decimal(first, sig, d.exponent + 1);
}
const bool sixteen = sig >= 1000000000000000u; // (else 15 digits)
const int last = d.has_digit ? d.digit : 0;
const std::uint64_t upper = sig / 100000000u;
#if JSON_DTOA_SSE2
// NOLINTBEGIN(portability-simd-intrinsics)
// the two halves in the 64-bit lanes, each as abcd * 2^32 + efgh, then as
// bytes (as eight_digit_bytes(), one lane each)
const __m128i x = _mm_set_epi64x(static_cast<long long>(sig - (upper * 100000000u)), static_cast<long long>(upper)); // NOLINT(runtime/int)
const __m128i abcd = _mm_srli_epi64(_mm_mul_epu32(x, _mm_set1_epi64x(109951163)), 40); // 2^40 / 10000 + 1
const __m128i abcd_efgh = _mm_add_epi64(x, _mm_mul_epu32(abcd, _mm_set1_epi64x(4294957296))); // 2^32 - 10000
// 32-bit lanes in the order of the text: abcd, efgh of both halves
const __m128i fours = _mm_shuffle_epi32(abcd_efgh, _MM_SHUFFLE(2, 3, 0, 1));
const __m128i ab = _mm_srli_epi16(_mm_mulhi_epu16(fours, _mm_set1_epi32(5243)), 3);
const __m128i ab_cd = _mm_or_si128(_mm_slli_epi32(_mm_sub_epi16(fours, _mm_mullo_epi16(ab, _mm_set1_epi32(100))), 16), ab);
// 16-bit lanes ab (< 100) -> bytes a, b: 256 * ab - 2559 * (ab / 10)
const __m128i bytes = _mm_sub_epi16(_mm_slli_epi16(ab_cd, 8), _mm_mullo_epi16(_mm_set1_epi16(2559), _mm_mulhi_epu16(ab_cd, _mm_set1_epi16(6554))));
// the last digit that is not 0 (sig is not 0)
const auto nonzero = static_cast<std::uint64_t>(_mm_movemask_epi8(_mm_cmpgt_epi8(bytes, _mm_setzero_si128())));
const int digits = 63 - count_leading_zeros(nonzero) + (sixteen ? 1 : 0); // without trailing zeros
const __m128i chars = _mm_add_epi8(bytes, _mm_set1_epi8('0'));
// the 16 characters from the first digit
const __m128i s = sixteen ? chars : _mm_or_si128(_mm_srli_si128(chars, 1), _mm_slli_si128(_mm_cvtsi32_si128('0' + last), 15));
const char s16 = static_cast<char>(sixteen ? '0' + last : '0'); // the 17th
const auto store_16 = [&s](char* p) noexcept
{
std::memcpy(p, &s, 16);
};
const char first_digit = static_cast<char>(_mm_cvtsi128_si32(s));
// NOLINTEND(portability-simd-intrinsics)
#elif JSON_DTOA_NEON
// as with SSE2: the halves in 32-bit lanes, then abcd, efgh of both
const uint32x2_t halves = vcreate_u32(upper | ((sig - (upper * 100000000u)) << 32u));
const uint32x2_t abcd = vmovn_u64(vshrq_n_u64(vmull_n_u32(halves, static_cast<std::uint32_t>(((std::uint64_t{1} << 40u) / 10000u) + 1u)), 40));
const uint32x2_t efgh = vmls_n_u32(halves, abcd, 10000u);
const uint32x4_t fours = vcombine_u32(vzip1_u32(abcd, efgh), vzip2_u32(abcd, efgh));
const uint32x4_t ab = vshrq_n_u32(vmulq_n_u32(fours, 5243u), 19);
const uint16x8_t ab_cd = vreinterpretq_u16_u32(vorrq_u32(ab, vshlq_n_u32(vmlsq_n_u32(fours, ab, 100u), 16)));
const uint16x8_t tens = vshrq_n_u16(vmulq_n_u16(ab_cd, 103u), 10);
const uint8x16_t bytes = vreinterpretq_u8_u16(vorrq_u16(tens, vshlq_n_u16(vmlsq_n_u16(ab_cd, tens, 10u), 8)));
// the last digit that is not 0 (sig is not 0): a nibble per byte
const std::uint64_t nonzero = vget_lane_u64(vreinterpret_u64_u8(vshrn_n_u16(vreinterpretq_u16_u8(vtstq_u8(bytes, bytes)), 4)), 0);
const int digits = ((63 - count_leading_zeros(nonzero)) / 4) + (sixteen ? 1 : 0); // without trailing zeros
const uint8x16_t chars = vaddq_u8(bytes, vdupq_n_u8('0'));
// the 16 characters from the first digit
const uint8x16_t s = sixteen ? chars : vextq_u8(chars, vdupq_n_u8(static_cast<std::uint8_t>('0' + last)), 1);
const char s16 = static_cast<char>(sixteen ? '0' + last : '0'); // the 17th
const auto store_16 = [&s](char* p) noexcept
{
vst1q_u8(reinterpret_cast<std::uint8_t*>(p), s); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
};
const auto first_digit = static_cast<char>(vgetq_lane_u8(s, 0));
#else
const std::uint64_t hi = eight_digit_bytes(upper);
const std::uint64_t lo = eight_digit_bytes(sig - (upper * 100000000u));
// trailing zero digits: zero bytes (sig is not 0)
const int zeros = lo != 0 ? count_trailing_zeros(lo) / 8 : 8 + (count_trailing_zeros(hi) / 8);
const int digits = 15 - zeros + (sixteen ? 1 : 0); // without trailing zeros
// the 16 characters from the first digit
const std::uint64_t s_hi = (sixteen ? hi : (hi << 8u) | (lo >> 56u)) + 0x3030303030303030u;
const std::uint64_t s_lo = (sixteen ? lo : (lo << 8u) | static_cast<std::uint64_t>(last)) + 0x3030303030303030u;
const char s16 = static_cast<char>(sixteen ? '0' + last : '0'); // the 17th
const auto store_16 = [s_hi, s_lo](char* p) noexcept
{
store_msb_first(p, s_hi);
store_msb_first(p + 8, s_lo);
};
const auto first_digit = static_cast<char>(s_hi >> 56u);
#endif
const int len = d.has_digit ? 16 + (sixteen ? 1 : 0) : digits; // significant digits
const int n = 16 + (sixteen ? 1 : 0) + d.exponent; // digits before the point
if (JSON_HEDLEY_LIKELY(n >= 1 && n <= 15))
{
// "dig.its" and "digits[000].0": the digits after the point move by
// one byte ('0's follow the digits)
#if JSON_DTOA_SSE2
// NOLINTBEGIN(portability-simd-intrinsics)
// (in the register: reading the digits back from memory right after
// storing them waits until the stores are done)
const __m128i at = _mm_set1_epi8(static_cast<char>(n));
const __m128i index = _mm_setr_epi8(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15);
const __m128i before = _mm_cmpgt_epi8(at, index);
const __m128i after = _mm_cmpgt_epi8(index, at);
const __m128i text = _mm_or_si128(_mm_or_si128(_mm_and_si128(s, before), _mm_and_si128(_mm_slli_si128(s, 1), after)),
_mm_andnot_si128(_mm_or_si128(before, after), _mm_set1_epi8('.')));
std::memcpy(first, &text, 16);
first[16] = static_cast<char>(_mm_extract_epi16(s, 7) >> 8);
first[17] = s16;
// NOLINTEND(portability-simd-intrinsics)
#elif JSON_DTOA_NEON
const uint8x16_t index = vcombine_u8(vcreate_u8(0x0706050403020100u), vcreate_u8(0x0F0E0D0C0B0A0908u));
const uint8x16_t at = vdupq_n_u8(static_cast<std::uint8_t>(n));
const uint8x16_t after_point = vbslq_u8(vcgtq_u8(index, at), vextq_u8(vdupq_n_u8(0), s, 15), vdupq_n_u8('.'));
vst1q_u8(reinterpret_cast<std::uint8_t*>(first), vbslq_u8(vcltq_u8(index, at), s, after_point)); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
first[16] = static_cast<char>(vgetq_lane_u8(s, 15));
first[17] = s16;
#else
store_16(first);
first[16] = s16;
std::uint64_t after_point[2]; // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays,cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
std::memcpy(after_point, first + n, 16);
std::memcpy(first + n + 1, after_point, 16);
first[n] = '.';
#endif
return first + (n >= len ? n + 2 : len + 1);
}
if (n <= 0 && n > -4)
{
// "0.[000]digits"
std::memset(first, '0', 8);
first[1] = '.';
store_16(first + 2 - n);
first[18 - n] = s16;
return first + 2 - n + len;
}
// d.igitse+XX, with at least two exponent digits (as append_exponent())
store_16(first + 1);
first[17] = s16;
first[0] = first_digit;
first[1] = '.';
char* const end = first + (len == 1 ? 1 : len + 1);
const int e = n - 1;
const auto ea = e < 0 ? 0u - static_cast<unsigned>(e) : static_cast<unsigned>(e); // (unsigned: no signed overflow to assume)
const bool three = ea >= 100;
end[0] = 'e';
end[1] = e < 0 ? '-' : '+';
end[2] = static_cast<char>('0' + (three ? ea / 100 : (ea / 10) % 10));
end[3] = static_cast<char>('0' + (three ? (ea / 10) % 10 : ea % 10));
end[4] = static_cast<char>('0' + (ea % 10));
return end + (three ? 5 : 4);
}
/// the powers of ten up to 10^16
inline const std::array<std::uint64_t, 17>& powers_of_ten_16() noexcept
{
static const std::array<std::uint64_t, 17> powers =
{
{
1u, 10u, 100u, 1000u, 10000u, 100000u, 1000000u, 10000000u, 100000000u, 1000000000u, 10000000000u,
100000000000u, 1000000000000u, 10000000000000u, 100000000000000u, 1000000000000000u, 10000000000000000u
}
};
return powers;
}
/*!
@brief digits * 10^exp, as write_decimal() writes it, for the digits of a
double that need no conversion (count digits, at most 15, the first not 0;
trailing zeros allowed): extended to 16 digits and written by write_shortest()
@return a pointer past the text; up to 41 bytes at @a first are written
(some beyond the returned end)
*/
JSON_HEDLEY_NON_NULL(1)
JSON_HEDLEY_RETURNS_NON_NULL
inline char* write_short_decimal(char* first, std::uint64_t digits, int count, int exp) noexcept
{
JSON_ASSERT(digits >= powers_of_ten_16()[static_cast<std::size_t>(count - 1)] && count <= 15);
const int scale = 16 - count;
return write_shortest(first, zmij::shortest_decimal{digits * powers_of_ten_16()[static_cast<std::size_t>(scale)], exp - scale - 1, 0, false});
}
/// as write_short_decimal(), counting the digits (not 0, less than 10^15)
JSON_HEDLEY_NON_NULL(1)
JSON_HEDLEY_RETURNS_NON_NULL
inline char* write_short_decimal(char* first, std::uint64_t digits, int exp) noexcept
{
JSON_ASSERT(digits != 0 && digits < 1000000000000000u);
// floor(log10(2^bits)) + 1 digits, or one less
const int log2_bound = ((64 - count_leading_zeros(digits)) * 1233) >> 12;
const int count = log2_bound + (digits >= powers_of_ten_16()[static_cast<std::size_t>(log2_bound)] ? 1 : 0);
return write_short_decimal(first, digits, count, exp);
}
/// a positive finite float (other than double): Grisu2 and format_buffer()
template<typename FloatType>
JSON_HEDLEY_NON_NULL(1, 2)
JSON_HEDLEY_RETURNS_NON_NULL
char* write_positive(char* first, const char* last, FloatType value)
{
JSON_ASSERT(last - first >= std::numeric_limits<FloatType>::max_digits10);
static_cast<void>(last); // (only used in the assertion)
// Compute v = buffer * 10^decimal_exponent.
// The decimal digits are stored in the buffer, which needs to be interpreted
// as an unsigned decimal integer.
// len is the length of the buffer, i.e., the number of decimal digits.
int len = 0;
int decimal_exponent = 0;
shortest_digits(first, len, decimal_exponent, value);
JSON_ASSERT(len <= std::numeric_limits<FloatType>::max_digits10);
// Format the buffer like printf("%.*g", prec, value)
constexpr int kMinExp = -4;
// Use digits10 here to increase compatibility with version 2.
constexpr int kMaxExp = std::numeric_limits<FloatType>::digits10;
JSON_ASSERT(last - first >= kMaxExp + 2);
JSON_ASSERT(last - first >= 2 + (-kMinExp - 1) + std::numeric_limits<FloatType>::max_digits10);
JSON_ASSERT(last - first >= std::numeric_limits<FloatType>::max_digits10 + 6);
return format_buffer(first, len, decimal_exponent, kMinExp, kMaxExp);
}
/// a positive finite double: the shortest digits (Zmij), laid out by
/// write_shortest() (through a local buffer if [first, last) is shorter than
/// the 41 bytes it may write)
JSON_HEDLEY_NON_NULL(1, 2)
JSON_HEDLEY_RETURNS_NON_NULL
inline char* write_positive(char* first, const char* last, double value)
{
static_assert(std::numeric_limits<double>::is_iec559 && std::numeric_limits<double>::digits == 53,
"internal error: the conversion of Zmij needs IEEE 754 binary64 doubles");
std::uint64_t bits = 0;
std::memcpy(&bits, &value, sizeof(bits));
const zmij::shortest_decimal d = zmij::to_shortest(bits);
if (JSON_HEDLEY_LIKELY(last - first >= 41))
{
return write_shortest(first, d);
}
std::array<char, 64> buf; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
const auto len = static_cast<std::size_t>(write_shortest(buf.data(), d) - buf.data());
JSON_ASSERT(last - first >= static_cast<std::ptrdiff_t>(len));
std::memcpy(first, buf.data(), len);
return first + len;
}
} // namespace dtoa_impl
/*!
@@ -1064,7 +1536,6 @@ JSON_HEDLEY_NON_NULL(1, 2)
JSON_HEDLEY_RETURNS_NON_NULL
char* to_chars(char* first, const char* last, FloatType value)
{
static_cast<void>(last); // maybe unused - fix warning
JSON_ASSERT(std::isfinite(value));
// Use signbit(value) instead of (value < 0) since signbit works for -0.
@@ -1090,28 +1561,7 @@ char* to_chars(char* first, const char* last, FloatType value)
JSON_HEDLEY_DIAGNOSTIC_POP
#endif
JSON_ASSERT(last - first >= std::numeric_limits<FloatType>::max_digits10);
// Compute v = buffer * 10^decimal_exponent.
// The decimal digits are stored in the buffer, which needs to be interpreted
// as an unsigned decimal integer.
// len is the length of the buffer, i.e., the number of decimal digits.
int len = 0;
int decimal_exponent = 0;
dtoa_impl::grisu2(first, len, decimal_exponent, value);
JSON_ASSERT(len <= std::numeric_limits<FloatType>::max_digits10);
// Format the buffer like printf("%.*g", prec, value)
constexpr int kMinExp = -4;
// Use digits10 here to increase compatibility with version 2.
constexpr int kMaxExp = std::numeric_limits<FloatType>::digits10;
JSON_ASSERT(last - first >= kMaxExp + 2);
JSON_ASSERT(last - first >= 2 + (-kMinExp - 1) + std::numeric_limits<FloatType>::max_digits10);
JSON_ASSERT(last - first >= std::numeric_limits<FloatType>::max_digits10 + 6);
return dtoa_impl::format_buffer(first, len, decimal_exponent, kMinExp, kMaxExp);
return dtoa_impl::write_positive(first, last, value);
}
} // namespace detail
@@ -414,13 +414,13 @@ inline void to_json(BasicJsonType& j, const EnumKeyedMap& map)
BasicJsonType key = p.first;
if (JSON_HEDLEY_UNLIKELY(!key.is_string()))
{
throw_type_must_be("string", key);
JSON_THROW(type_error::create(302, concat("type must be string, but is ", key.type_name()), &key));
}
auto& key_string = *key.template get_ptr<typename BasicJsonType::string_t*>();
if (JSON_HEDLEY_UNLIKELY(!obj.emplace(key_string, BasicJsonType(p.second)).second))
{
JSON_THROW(type_error::create(exception_id::enum_key_duplicate, concat("duplicate object key '", key_string, "'"), &key));
JSON_THROW(type_error::create(318, concat("duplicate object key '", key_string, "'"), &key));
}
}
external_constructor<value_t::object>::construct(j, std::move(obj));
@@ -0,0 +1,238 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2025 Victor Zverovich <https://github.com/vitaut/zmij>
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#pragma once
#include <array> // array
#include <cstddef> // size_t
#include <cstdint> // uint32_t, uint64_t
#include <nlohmann/detail/abi_macros.hpp>
#include <nlohmann/detail/bit_ops.hpp>
#include <nlohmann/detail/input/pow5_table.hpp>
#include <nlohmann/detail/macro_scope.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
{
/*!
@brief the shortest decimal representation of a double
A C++11 port of the conversion of Zmij by Victor Zverovich
(https://github.com/vitaut/zmij, MIT license): the shortest decimal in the
rounding interval of a double, the closest one if there are several. Zmij
credits Xiang JunBo (producing the shorter candidate without a division) and
Dougall Johnson (the compressed powers of ten). The powers of ten are taken
from the table for number parsing (pow5_table.hpp) where it holds them, and
computed from the compressed tables of Zmij beyond it.
*/
namespace zmij
{
/// significand * 10^exponent
struct decimal
{
std::uint64_t significand;
int exponent;
};
/// the compressed powers of ten of Zmij
inline const std::array<std::uint64_t, 28>& pow10_minor() noexcept
{
static const std::array<std::uint64_t, 28> table =
{
{
0x8000000000000000u, 0xa000000000000000u, 0xc800000000000000u, 0xfa00000000000000u, 0x9c40000000000000u,
0xc350000000000000u, 0xf424000000000000u, 0x9896800000000000u, 0xbebc200000000000u, 0xee6b280000000000u,
0x9502f90000000000u, 0xba43b74000000000u, 0xe8d4a51000000000u, 0x9184e72a00000000u, 0xb5e620f480000000u,
0xe35fa931a0000000u, 0x8e1bc9bf04000000u, 0xb1a2bc2ec5000000u, 0xde0b6b3a76400000u, 0x8ac7230489e80000u,
0xad78ebc5ac620000u, 0xd8d726b7177a8000u, 0x878678326eac9000u, 0xa968163f0a57b400u, 0xd3c21bcecceda100u,
0x84595161401484a0u, 0xa56fa5b99019a5c8u, 0xcecb8f27f4200f3au
}
};
return table;
}
/// (high, low) pairs
inline const std::array<std::uint64_t, 50>& pow10_major() noexcept
{
static const std::array<std::uint64_t, 50> table =
{
{
0xaddcb9e83c6b1793u, 0xdf4abe242a1bbf3eu, 0xaf8e5410288e1b6fu, 0x07ecf0ae5ee44ddau, 0xb1442798f49ffb4au, 0x99cd11cfdf41779du,
0xb2fe3f0b8599ef07u, 0x861fa7e6dcb4aa15u, 0xb4bca50b065abe63u, 0x0fed077a756b53aau, 0xb67f6455292cbf08u, 0x1a3bc84c17b1d543u,
0xb84687c269ef3bfbu, 0x3d5d514f40eea742u, 0xba121a4650e4ddebu, 0x92f34d62616ce413u, 0xbbe226efb628afeau, 0x890489f70a55368cu,
0xbdb6b8e905cb600fu, 0x5400e987bbc1c921u, 0xbf8fdb78849a5f96u, 0xde98520472bdd034u, 0xc16d9a0095928a27u, 0x75b7053c0f178294u,
0xc350000000000000u, 0x0000000000000000u, 0xc5371912364ce305u, 0x6c28000000000000u, 0xc722f0ef9d80aad6u, 0x424d3ad2b7b97ef6u,
0xc913936dd571c84cu, 0x03bc3a19cd1e38eau, 0xcb090c8001ab551cu, 0x5cadf5bfd3072cc6u, 0xcd036837130890a1u, 0x36dba887c37a8c10u,
0xcf02b2c21207ef2eu, 0x94f967e45e03f4bcu, 0xd106f86e69d785c7u, 0xe13336d701beba52u, 0xd31045a8341ca07cu, 0x1ede48111209a051u,
0xd51ea6fa85785631u, 0x552a74227f3ea566u, 0xd732290fbacaf133u, 0xa97c177947ad4096u, 0xd94ad8b1c7380874u, 0x18375281ae7822bdu,
0xdb68c2ca82ed2a05u, 0xa67398db9f6820e1u
}
};
return table;
}
/// one bit per power: whether the computed value is one unit too large
inline const std::array<std::uint32_t, 21>& pow10_fixups() noexcept
{
static const std::array<std::uint32_t, 21> table =
{
{
0x8d8fc810u, 0x06100293u, 0x19000000u, 0x00100000u, 0x00000908u, 0x00000000u, 0x04e00300u, 0x3807e0b2u, 0x3d83d793u, 0x0006f5ccu,
0x00000000u, 0xffff0000u, 0x8076337du, 0x4ff45ba0u, 0x09405033u, 0x034376d9u, 0x09000000u, 0x4e100501u, 0x076d14dcu, 0xf964f45eu,
0x0000003du
}
};
return table;
}
/// the 128-bit significand of 10^k, rounded down, for k in [-307, 341]
/// (compute_pow10 of Zmij)
inline uint128_parts compute_pow10(int k) noexcept
{
const auto i = static_cast<unsigned>(k + 307);
const std::uint64_t m = pow10_minor()[(i + 24) % 28];
const std::size_t j = 2 * static_cast<std::size_t>((i + 24) / 28);
const std::uint64_t h_hi = pow10_major()[j];
const std::uint64_t h_lo = pow10_major()[j + 1];
const std::uint64_t h1 = full_multiplication(h_lo, m).high;
const std::uint64_t c0 = h_lo * m;
const std::uint64_t c1 = h1 + (h_hi * m);
const std::uint64_t c2 = (c1 < h1 ? 1u : 0u) + full_multiplication(h_hi, m).high;
uint128_parts r{};
if ((c2 >> 63u) != 0)
{
r.high = c2;
r.low = c1;
}
else
{
r.high = (c2 << 1u) | (c1 >> 63u);
r.low = (c1 << 1u) | (c0 >> 63u);
}
r.low -= (pow10_fixups()[i >> 5u] >> (i & 31u)) & 1u;
return r;
}
/// The 128-bit significand of 10^k, rounded down, for k in [-342, 341].
/// Up to 10^308, the table for number parsing holds the same significands
/// (those of 5^k), except for k in [-27, -1], where it holds them one unit
/// larger (as the Eisel-Lemire algorithm needs them).
inline uint128_parts pow10(int k) noexcept
{
if (k > pow5_128_largest_power)
{
return compute_pow10(k); // (only for the smallest doubles)
}
const auto i = 2 * static_cast<std::size_t>(k - pow5_128_smallest_power);
uint128_parts r{pow5_128()[i + 1], pow5_128()[i]};
const std::uint64_t adjust = static_cast<unsigned>(k + 27) < 27u ? 1u : 0u;
r.high -= r.low < adjust ? 1u : 0u;
r.low -= adjust;
return r;
}
/// (x_hi * 2^64 + x_lo) * y >> 64, as 128 bits
inline uint128_parts umul192_hi128(std::uint64_t x_hi, std::uint64_t x_lo, std::uint64_t y) noexcept
{
const uint128_parts p = full_multiplication(x_hi, y);
uint128_parts r{};
r.low = p.low + full_multiplication(x_lo, y).high;
r.high = p.high + (r.low < p.low ? 1u : 0u);
return r;
}
/// (x * y + c) >> 64
inline std::uint64_t umul128_add_hi64(std::uint64_t x, std::uint64_t y, std::uint64_t c) noexcept
{
const uint128_parts p = full_multiplication(x, y);
return p.high + (p.low + c < p.low ? 1u : 0u);
}
/// the result of Zmij: the shorter candidate and, if that is outside the
/// rounding interval, the digit after it (16 bytes: returned in registers)
struct shortest_decimal
{
std::uint64_t integral; ///< the shorter candidate (15 or 16 digits for normal doubles)
int exponent; ///< the decimal exponent of the digit after it
unsigned char digit; ///< the digit after it (if has_digit)
bool has_digit; ///< whether the shortest decimal is integral * 10 + digit
};
/// The shortest decimal in the rounding interval of a positive finite double
/// given by its bits, the closest one if there are several (to_decimal of
/// Zmij, which keeps the last digit apart: the 15 or 16 digits before it can be
/// converted without a multiplication by 10 first). Always inlined: GCC
/// otherwise calls it, and its result goes through memory.
JSON_HEDLEY_ALWAYS_INLINE shortest_decimal to_shortest(std::uint64_t bits) noexcept
{
constexpr int extra_shift = 9;
const auto raw_exp = static_cast<int>((bits >> 52u) & 0x7FFu);
std::uint64_t bin_sig = bits & ((std::uint64_t{1} << 52u) - 1);
// a power of two has a narrower interval below (except the smallest normal)
const bool regular = bin_sig != 0 || raw_exp <= 1;
const int bin_exp = (raw_exp == 0 ? 1 : raw_exp) - 1075;
if (raw_exp != 0)
{
bin_sig |= std::uint64_t{1} << 52u;
}
// floor(log10(2^bin_exp)), or floor(log10(3/4 * 2^bin_exp)) for the irregular case
const int dec_exp = ((bin_exp * 315653) - (regular ? 0 : 131072)) >> 20;
// scaled by 10^(-dec_exp - 1): the integral part is the shorter candidate
const int shift = bin_exp + ((-(dec_exp + 1) * 217707) >> 16) + 1 + extra_shift;
const uint128_parts p10 = pow10(-dec_exp - 1);
const uint128_parts p = umul192_hi128(p10.high, p10.low, bin_sig << static_cast<unsigned>(shift));
std::uint64_t integral = p.high >> static_cast<unsigned>(extra_shift);
const std::uint64_t fractional = (p.high << static_cast<unsigned>(64 - extra_shift)) | (p.low >> static_cast<unsigned>(extra_shift));
std::uint64_t digit = 0;
bool round_up = false;
bool round_down = false;
if (JSON_HEDLEY_LIKELY(regular))
{
const std::uint64_t half_ulp = (p10.high >> static_cast<unsigned>(extra_shift + 1 - shift)) + (1 - (bin_sig & 1u));
round_up = fractional + half_ulp < fractional;
round_down = half_ulp > fractional;
// the last digit of the longer candidate, rounded to nearest
digit = umul128_add_hi64(fractional, 10, (std::uint64_t{1} << 63u) + 6);
if (fractional == (std::uint64_t{1} << 62u))
{
digit = 2; // 2.5 rounds to 2
}
}
else
{
const std::uint64_t half_ulp = p10.high >> static_cast<unsigned>(extra_shift + 1 - shift);
round_up = half_ulp > ~std::uint64_t{0} - fractional;
round_down = (half_ulp >> 1u) > fractional;
digit = umul128_add_hi64(fractional, 10, (std::uint64_t{1} << 63u) - 1);
const std::uint64_t lowest = umul128_add_hi64(fractional - (half_ulp >> 1u), 10, ~std::uint64_t{0});
digit = digit < lowest ? lowest : digit;
}
integral += round_up ? 1u : 0u;
// if the shorter candidate is outside the rounding interval: one digit more
return shortest_decimal{integral, dec_exp, static_cast<unsigned char>(digit), !round_up && !round_down};
}
/// The shortest decimal in the rounding interval of a positive finite double
/// given by its bits, as one number. The significand can end in zeros.
inline decimal to_decimal(std::uint64_t bits) noexcept
{
const shortest_decimal d = to_shortest(bits);
if (d.has_digit)
{
return decimal{(d.integral * 10) + d.digit, d.exponent};
}
return decimal{d.integral, d.exponent + 1};
}
} // namespace zmij
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
-144
View File
@@ -45,84 +45,6 @@ namespace detail
// exceptions //
////////////////
/*!
@brief the ids of the exceptions thrown by the library
@note The values are part of the public API: they are the `id` member of the
exceptions and appear in their `what()` messages.
@sa https://json.nlohmann.me/home/exceptions/
*/
enum class exception_id : int
{
// parse_error
syntax_error = 101, ///< unexpected token or invalid literal while parsing JSON
patch_not_an_array = 104, ///< a JSON Patch document is not an array of objects
patch_invalid_operation = 105, ///< a JSON Patch operation is malformed
pointer_index_leading_zero = 106, ///< a JSON Pointer array index has a leading zero
pointer_missing_slash = 107, ///< a JSON Pointer does not start with '/'
pointer_invalid_escape = 108, ///< a JSON Pointer contains an escape other than ~0 and ~1
pointer_index_not_a_number = 109, ///< a JSON Pointer array index is not a number
unexpected_end_of_input = 110, ///< a binary input ends early, or has bytes left at its end
unexpected_byte = 112, ///< a binary input contains an unexpected byte or invalid length
invalid_string_or_size = 113, ///< a binary input contains an invalid string or size specification
bson_unsupported_type = 114, ///< a BSON record type is not supported
invalid_high_precision_number = 115, ///< a UBJSON/BJData high-precision number cannot be parsed
// invalid_iterator
iterators_incompatible = 201, ///< the iterators of a range belong to different values
iterator_from_other_value = 202, ///< an iterator does not belong to the value it is used with
iterator_range_from_other_value = 203, ///< an iterator range does not belong to the value it is used with
iterator_range_out_of_range = 204, ///< an iterator range of a primitive value is not [begin, end)
iterator_out_of_range = 205, ///< an iterator of a primitive value is not begin()
iterator_range_of_null = 206, ///< an iterator range belongs to a null value
iterator_key_not_object = 207, ///< key() is called on an iterator of a non-object
iterator_subscript_on_object = 208, ///< operator[] is called on an iterator of an object
iterator_arithmetic_on_object = 209, ///< an offset operator is used on an iterator of an object
insert_range_incompatible = 210, ///< the iterators of an inserted range belong to different values
insert_range_into_itself = 211, ///< an inserted range belongs to the value it is inserted into
iterators_compare_different_values = 212, ///< iterators of different values are compared
iterator_order_on_object = 213, ///< iterators of an object are compared by order
iterator_value_unavailable = 214, ///< an iterator does not refer to a value
// type_error
object_from_non_pairs = 301, ///< an object is created from an initializer list that is not a list of pairs
type_mismatch = 302, ///< a value has the wrong type for a conversion
incompatible_reference_type = 303, ///< get_ref() is called with a reference type that does not match the value
at_wrong_type = 304, ///< at() is called on a value of the wrong type
subscript_wrong_type = 305, ///< operator[] is called on a value of the wrong type
value_wrong_type = 306, ///< value() is called on a value of the wrong type
erase_wrong_type = 307, ///< erase() is called on a value of the wrong type
push_back_wrong_type = 308, ///< push_back() or operator+= is called on a value of the wrong type
insert_wrong_type = 309, ///< insert() is called on a value of the wrong type
swap_wrong_type = 310, ///< swap() is called on a value of the wrong type
emplace_wrong_type = 311, ///< emplace() or emplace_back() is called on a value of the wrong type
update_wrong_type = 312, ///< update() is called on a value of the wrong type
unflatten_invalid_value = 313, ///< unflatten() finds conflicting paths
unflatten_not_object = 314, ///< unflatten() is called on a non-object
unflatten_value_not_primitive = 315, ///< unflatten() is called on an object with non-primitive values
invalid_utf8 = 316, ///< dump() finds a string that is not valid UTF-8
type_not_serializable = 317, ///< a value cannot be serialized to the requested binary format
enum_key_duplicate = 318, ///< an enum-keyed map has two keys that serialize to the same string
discarded_value_used = 321, ///< a discarded value is used
// out_of_range
array_index_out_of_range = 401, ///< an array index is out of range
pointer_past_the_end_index = 402, ///< a JSON Pointer uses the array index '-'
key_not_found = 403, ///< an object key is not found
pointer_unresolved = 404, ///< a JSON Pointer reference token cannot be resolved
patch_on_root = 405, ///< the root of a value has no parent, e.g. for a JSON Patch 'remove' or 'add' at the root
number_overflow = 406, ///< a number cannot be stored without overflowing to NaN or INF
integer_too_large = 407, ///< an integer cannot be represented in the binary format
container_too_large = 408, ///< the size of a container in a binary input exceeds the maximal capacity
bson_key_with_null = 409, ///< a BSON key contains U+0000
value_out_of_range = 410, ///< an enum value is undefined, or a JSON Pointer array index exceeds size_type
patch_add_parent_not_container = 411, ///< the parent of a JSON Patch 'add' target is not a container
length_too_large = 412, ///< a length does not fit into the length field of a binary format
patch_remove_parent_not_container = 413, ///< the parent of a JSON Patch 'remove' target is not a container
patch_move_into_child = 414, ///< a JSON Patch 'move' moves a value into one of its children
subtype_out_of_range = 415, ///< a binary subtype does not fit into one byte
// other_error
internal_error = 500, ///< unreachable code was reached
patch_test_failed = 501, ///< a JSON Patch 'test' operation failed
size_marker_required = 502 ///< UBJSON/BJData output with type markers requires size markers
};
/// @brief general exception of the @ref basic_json class
/// @sa https://json.nlohmann.me/api/basic_json/exception/
class exception : public std::exception
@@ -273,18 +195,6 @@ class parse_error : public exception
return {id_, byte_, w.c_str()};
}
template<typename BasicJsonContext, enable_if_t<is_basic_json_context<BasicJsonContext>::value, int> = 0>
static parse_error create(exception_id id_, const position_t& pos, const std::string& what_arg, BasicJsonContext context)
{
return create(static_cast<int>(id_), pos, what_arg, context);
}
template<typename BasicJsonContext, enable_if_t<is_basic_json_context<BasicJsonContext>::value, int> = 0>
static parse_error create(exception_id id_, std::size_t byte_, const std::string& what_arg, BasicJsonContext context)
{
return create(static_cast<int>(id_), byte_, what_arg, context);
}
/*!
@brief byte index of the parse error
@@ -319,12 +229,6 @@ class invalid_iterator : public exception
return {id_, w.c_str()};
}
template<typename BasicJsonContext, enable_if_t<is_basic_json_context<BasicJsonContext>::value, int> = 0>
static invalid_iterator create(exception_id id_, const std::string& what_arg, BasicJsonContext context)
{
return create(static_cast<int>(id_), what_arg, context);
}
private:
JSON_HEDLEY_NON_NULL(3)
invalid_iterator(int id_, const char* what_arg)
@@ -343,12 +247,6 @@ class type_error : public exception
return {id_, w.c_str()};
}
template<typename BasicJsonContext, enable_if_t<is_basic_json_context<BasicJsonContext>::value, int> = 0>
static type_error create(exception_id id_, const std::string& what_arg, BasicJsonContext context)
{
return create(static_cast<int>(id_), what_arg, context);
}
private:
JSON_HEDLEY_NON_NULL(3)
type_error(int id_, const char* what_arg) : exception(id_, what_arg) {}
@@ -366,12 +264,6 @@ class out_of_range : public exception
return {id_, w.c_str()};
}
template<typename BasicJsonContext, enable_if_t<is_basic_json_context<BasicJsonContext>::value, int> = 0>
static out_of_range create(exception_id id_, const std::string& what_arg, BasicJsonContext context)
{
return create(static_cast<int>(id_), what_arg, context);
}
private:
JSON_HEDLEY_NON_NULL(3)
out_of_range(int id_, const char* what_arg) : exception(id_, what_arg) {}
@@ -389,12 +281,6 @@ class other_error : public exception
return {id_, w.c_str()};
}
template<typename BasicJsonContext, enable_if_t<is_basic_json_context<BasicJsonContext>::value, int> = 0>
static other_error create(exception_id id_, const std::string& what_arg, BasicJsonContext context)
{
return create(static_cast<int>(id_), what_arg, context);
}
private:
JSON_HEDLEY_NON_NULL(3)
other_error(int id_, const char* what_arg) : exception(id_, what_arg) {}
@@ -421,36 +307,6 @@ void templated_json_throw(ExceptionType exception)
(void)exception;
}
/*!
@brief throws because @a j does not have the type a conversion expects
@param[in] expected the expected type(s), e.g. "array" or "binary or array"
@param[in] j the value with the wrong type
@throw type_error.302 always
*/
template<typename BasicJsonType>
JSON_HEDLEY_NO_RETURN inline void throw_type_must_be(const char* expected, const BasicJsonType& j)
{
static_cast<void>(expected); // unused when JSON_NOEXCEPTION is defined
static_cast<void>(j);
JSON_THROW(type_error::create(exception_id::type_mismatch, concat("type must be ", expected, ", but is ", j.type_name()), &j));
}
/*!
@brief throws because an operation is not supported for the type of @a j
@param[in] id_ the id of the type_error exception (at_wrong_type..update_wrong_type)
@param[in] operation the operation, e.g. "erase()"
@param[in] j the value the operation was called on
@throw type_error always
*/
template<typename BasicJsonType>
JSON_HEDLEY_NO_RETURN inline void throw_cannot_use_with(const exception_id id_, const char* operation, const BasicJsonType& j)
{
static_cast<void>(id_); // unused when JSON_NOEXCEPTION is defined
static_cast<void>(operation);
static_cast<void>(j);
JSON_THROW(type_error::create(id_, concat("cannot use ", operation, " with ", j.type_name()), &j));
}
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
+135 -94
View File
@@ -12,6 +12,7 @@
#include <cmath> // ldexp
#include <cstddef> // size_t
#include <cstdint> // uint8_t, uint16_t, uint32_t, uint64_t, uintmax_t
#include <cstdio> // snprintf
#include <cstring> // memcpy
#include <iterator> // back_inserter
#include <limits> // numeric_limits
@@ -195,7 +196,8 @@ class binary_reader
if (JSON_HEDLEY_UNLIKELY(current != char_traits<char_type>::eof()))
{
return last_byte_error(exception_id::unexpected_end_of_input, concat("expected end of input; last byte: 0x", get_token_string()), "value");
return sax->parse_error(chars_read, get_token_string(), parse_error::create(110, chars_read,
exception_message(concat("expected end of input; last byte: 0x", get_token_string()), "value"), nullptr));
}
}
@@ -317,7 +319,8 @@ class binary_reader
{
if (JSON_HEDLEY_UNLIKELY(document_size < 0 || static_cast<std::size_t>(document_size) != chars_read - document_start))
{
return last_byte_error(exception_id::unexpected_byte, concat("document size ", std::to_string(document_size), " does not match the number of bytes read (", std::to_string(chars_read - document_start), ")"), "document");
return sax->parse_error(chars_read, get_token_string(), parse_error::create(112, chars_read,
exception_message(concat("document size ", std::to_string(document_size), " does not match the number of bytes read (", std::to_string(chars_read - document_start), ")"), "document"), nullptr));
}
return true;
}
@@ -509,7 +512,9 @@ class binary_reader
{
if (JSON_HEDLEY_UNLIKELY(len < 1))
{
return last_byte_error(exception_id::unexpected_byte, concat("string length must be at least 1, is ", std::to_string(len)), "string");
auto last_token = get_token_string();
return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read,
exception_message(concat("string length must be at least 1, is ", std::to_string(len)), "string"), nullptr));
}
if (JSON_HEDLEY_UNLIKELY(!get_string(len - static_cast<NumberType>(1), result)))
@@ -519,7 +524,10 @@ class binary_reader
if (JSON_HEDLEY_UNLIKELY(get() != 0x00))
{
return last_byte_error(exception_id::unexpected_byte, "BSON string is not null-terminated", "string");
auto last_token = get_token_string();
return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read,
exception_message("BSON string is not null-terminated",
"string"), nullptr));
}
return check_string_utf8(result, "string");
@@ -539,7 +547,9 @@ class binary_reader
{
if (JSON_HEDLEY_UNLIKELY(len < 0))
{
return last_byte_error(exception_id::unexpected_byte, concat("byte array length cannot be negative, is ", std::to_string(len)), "binary");
auto last_token = get_token_string();
return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read,
exception_message(concat("byte array length cannot be negative, is ", std::to_string(len)), "binary"), nullptr));
}
// All BSON binary values have a subtype
@@ -629,9 +639,11 @@ class binary_reader
default: // anything else is not supported (yet)
{
const std::string cr_str = hex_byte(static_cast<std::uint8_t>(element_type));
std::array<char, 3> cr{{}};
static_cast<void>((std::snprintf)(cr.data(), cr.size(), "%.2hhX", static_cast<unsigned char>(element_type))); // NOLINT(cppcoreguidelines-pro-type-vararg,hicpp-vararg)
const std::string cr_str{cr.data()};
return sax->parse_error(element_type_parse_position, cr_str,
parse_error::create(exception_id::bson_unsupported_type, element_type_parse_position, concat("Unsupported BSON record type 0x", cr_str), nullptr));
parse_error::create(114, element_type_parse_position, concat("Unsupported BSON record type 0x", cr_str), nullptr));
}
}
}
@@ -955,7 +967,9 @@ class binary_reader
{
if (tag_handler == cbor_tag_handler_t::error)
{
return invalid_byte("value");
auto last_token = get_token_string();
return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read,
exception_message(concat("invalid byte: 0x", last_token), "value"), nullptr));
}
// ignore and store: the tag value is already in the head, so
@@ -975,7 +989,9 @@ class binary_reader
{
case cbor_tag_handler_t::error:
{
return invalid_byte("value");
auto last_token = get_token_string();
return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read,
exception_message(concat("invalid byte: 0x", last_token), "value"), nullptr));
}
case cbor_tag_handler_t::ignore:
@@ -1049,7 +1065,9 @@ class binary_reader
default: // anything else (0xFF is handled inside the other types)
{
return invalid_byte("value");
auto last_token = get_token_string();
return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read,
exception_message(concat("invalid byte: 0x", last_token), "value"), nullptr));
}
}
}
@@ -1062,7 +1080,10 @@ class binary_reader
*/
bool cbor_indefinite_string_error(const char* type_name, const char* context)
{
return last_byte_error(exception_id::invalid_string_or_size, concat("indefinite-length ", type_name, " is not allowed inside indefinite-length ", type_name, "; last byte: 0x", get_token_string()), context);
auto last_token = get_token_string();
return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read,
exception_message(concat("indefinite-length ", type_name,
" is not allowed inside indefinite-length ", type_name, "; last byte: 0x", last_token), context), nullptr));
}
/*!
@@ -1139,7 +1160,9 @@ class binary_reader
default:
{
return last_byte_error(exception_id::invalid_string_or_size, concat("expected length specification (0x60-0x7B)", inside_indefinite ? "" : " or indefinite string type (0x7F)", "; last byte: 0x", get_token_string()), "string");
auto last_token = get_token_string();
return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read,
exception_message(concat("expected length specification (0x60-0x7B)", inside_indefinite ? "" : " or indefinite string type (0x7F)", "; last byte: 0x", last_token), "string"), nullptr));
}
}
}
@@ -1269,7 +1292,9 @@ class binary_reader
break;
}
return last_byte_error(exception_id::invalid_string_or_size, concat("only string keys are supported, but found ", found, "; last byte: 0x", get_token_string()), "object key");
auto last_token = get_token_string();
return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read,
exception_message(concat("only string keys are supported, but found ", found, "; last byte: 0x", last_token), "object key"), nullptr));
}
/*!
@@ -1350,7 +1375,9 @@ class binary_reader
default:
{
return last_byte_error(exception_id::invalid_string_or_size, concat("expected length specification (0x40-0x5B)", inside_indefinite ? "" : " or indefinite binary array type (0x5F)", "; last byte: 0x", get_token_string()), "binary");
auto last_token = get_token_string();
return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read,
exception_message(concat("expected length specification (0x40-0x5B)", inside_indefinite ? "" : " or indefinite binary array type (0x5F)", "; last byte: 0x", last_token), "binary"), nullptr));
}
}
}
@@ -1496,7 +1523,7 @@ class binary_reader
{
if (JSON_HEDLEY_UNLIKELY(!value_in_range_of<std::size_t>(len) || len == detail::unknown_size()))
{
return sax->parse_error(chars_read, get_token_string(), out_of_range::create(exception_id::container_too_large,
return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408,
exception_message(concat("excessive ", context, " size"), "size"), nullptr));
}
result = conditional_static_cast<std::size_t>(len);
@@ -1984,7 +2011,9 @@ class binary_reader
default: // anything else
{
return invalid_byte("value");
auto last_token = get_token_string();
return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read,
exception_message(concat("invalid byte: 0x", last_token), "value"), nullptr));
}
}
}
@@ -2065,7 +2094,9 @@ class binary_reader
default:
{
return last_byte_error(exception_id::invalid_string_or_size, concat("expected length specification (0xA0-0xBF, 0xD9-0xDB); last byte: 0x", get_token_string()), "string");
auto last_token = get_token_string();
return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read,
exception_message(concat("expected length specification (0xA0-0xBF, 0xD9-0xDB); last byte: 0x", last_token), "string"), nullptr));
}
}
}
@@ -2157,7 +2188,9 @@ class binary_reader
break;
}
return last_byte_error(exception_id::invalid_string_or_size, concat("only string keys are supported, but found ", found, "; last byte: 0x", get_token_string()), "object key");
auto last_token = get_token_string();
return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read,
exception_message(concat("only string keys are supported, but found ", found, "; last byte: 0x", last_token), "object key"), nullptr));
}
/*!
@@ -2466,7 +2499,8 @@ class binary_reader
{
if (JSON_HEDLEY_UNLIKELY(len < 0))
{
return last_byte_error(exception_id::invalid_string_or_size, "string length must not be negative", "string");
return sax->parse_error(chars_read, get_token_string(), parse_error::create(113, chars_read,
exception_message("string length must not be negative", "string"), nullptr));
}
return true;
}
@@ -2566,7 +2600,18 @@ class binary_reader
default:
break;
}
return length_type_error("", "string");
auto last_token = get_token_string();
std::string message;
if (input_format != input_format_t::bjdata)
{
message = "expected length type specification (U, i, I, l, L); last byte: 0x" + last_token;
}
else
{
message = "expected length type specification (U, i, u, I, m, l, M, L); last byte: 0x" + last_token;
}
return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, exception_message(message, "string"), nullptr));
}
/*!
@@ -2651,11 +2696,12 @@ class binary_reader
}
if (JSON_HEDLEY_UNLIKELY(number < 0))
{
return last_byte_error(exception_id::invalid_string_or_size, "count in an optimized container must be positive", "size");
return sax->parse_error(chars_read, get_token_string(), parse_error::create(113, chars_read,
exception_message("count in an optimized container must be positive", "size"), nullptr));
}
if (JSON_HEDLEY_UNLIKELY(!value_in_range_of<std::size_t>(number)))
{
return sax->parse_error(chars_read, get_token_string(), out_of_range::create(exception_id::container_too_large,
return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408,
exception_message("integer value overflow", "size"), nullptr));
}
result = static_cast<std::size_t>(number); // NOLINT(bugprone-signed-char-misuse,cert-str34-c): number is not a char
@@ -2753,7 +2799,7 @@ class binary_reader
}
if (!value_in_range_of<std::size_t>(number))
{
return sax->parse_error(chars_read, get_token_string(), out_of_range::create(exception_id::container_too_large,
return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408,
exception_message("integer value overflow", "size"), nullptr));
}
result = detail::conditional_static_cast<std::size_t>(number);
@@ -2768,7 +2814,7 @@ class binary_reader
}
if (is_ndarray) // ndarray dimensional vector can only contain integers and cannot embed another array
{
return last_byte_error(exception_id::invalid_string_or_size, "ndarray dimensional vector is not allowed", "size");
return sax->parse_error(chars_read, get_token_string(), parse_error::create(113, chars_read, exception_message("ndarray dimensional vector is not allowed", "size"), nullptr));
}
std::vector<size_t> dim;
if (JSON_HEDLEY_UNLIKELY(!get_ubjson_ndarray_size(dim)))
@@ -2804,7 +2850,9 @@ class binary_reader
const char* type_name = bjd_type_name(ndarray_dtype);
if (JSON_HEDLEY_UNLIKELY(type_name == nullptr))
{
return invalid_byte("type");
auto last_token = get_token_string();
return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read,
exception_message("invalid byte: 0x" + last_token, "type"), nullptr));
}
string_t type_key = "_ArrayType_";
@@ -2832,7 +2880,7 @@ class binary_reader
// or SIZE_MAX.
if (JSON_HEDLEY_UNLIKELY(result > (std::numeric_limits<std::size_t>::max)() / i))
{
return sax->parse_error(chars_read, get_token_string(), out_of_range::create(exception_id::container_too_large, exception_message("excessive ndarray size caused overflow", "size"), nullptr));
return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, exception_message("excessive ndarray size caused overflow", "size"), nullptr));
}
result *= i;
// the pre-check above already rules out result becoming 0
@@ -2841,7 +2889,7 @@ class binary_reader
// unknown-size container (see get_ubjson_size_type())
if (result == npos)
{
return sax->parse_error(chars_read, get_token_string(), out_of_range::create(exception_id::container_too_large, exception_message("excessive ndarray size caused overflow", "size"), nullptr));
return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408, exception_message("excessive ndarray size caused overflow", "size"), nullptr));
}
if (JSON_HEDLEY_UNLIKELY(!emit_unsigned(i)))
{
@@ -2858,7 +2906,18 @@ class binary_reader
default:
break;
}
return length_type_error(" after '#'", "size");
auto last_token = get_token_string();
std::string message;
if (input_format != input_format_t::bjdata)
{
message = "expected length type specification (U, i, I, l, L) after '#'; last byte: 0x" + last_token;
}
else
{
message = "expected length type specification (U, i, u, I, m, l, M, L) after '#'; last byte: 0x" + last_token;
}
return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read, exception_message(message, "size"), nullptr));
}
/*!
@@ -2891,7 +2950,9 @@ class binary_reader
if (input_format == input_format_t::bjdata
&& JSON_HEDLEY_UNLIKELY(is_bjd_excluded_optimized_type(result.second)))
{
return last_byte_error(exception_id::unexpected_byte, concat("marker 0x", get_token_string(), " is not a permitted optimized array type"), "type");
auto last_token = get_token_string();
return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read,
exception_message(concat("marker 0x", last_token, " is not a permitted optimized array type"), "type"), nullptr));
}
if (JSON_HEDLEY_UNLIKELY(!unexpect_eof("type")))
@@ -2906,7 +2967,9 @@ class binary_reader
{
return false;
}
return last_byte_error(exception_id::unexpected_byte, concat("expected '#' after type information; last byte: 0x", get_token_string()), "size");
auto last_token = get_token_string();
return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read,
exception_message(concat("expected '#' after type information; last byte: 0x", last_token), "size"), nullptr));
}
const bool is_error = get_ubjson_size_value(result.first, is_ndarray, 0, result.second);
@@ -2925,7 +2988,8 @@ class binary_reader
const bool is_error = get_ubjson_size_value(result.first, is_ndarray);
if (input_format == input_format_t::bjdata && is_ndarray && !inside_ndarray)
{
return last_byte_error(exception_id::unexpected_byte, "ndarray requires both type and size", "size");
return sax->parse_error(chars_read, get_token_string(), parse_error::create(112, chars_read,
exception_message("ndarray requires both type and size", "size"), nullptr));
}
return is_error;
}
@@ -3057,7 +3121,9 @@ class binary_reader
}
if (JSON_HEDLEY_UNLIKELY(current > 127))
{
return last_byte_error(exception_id::invalid_string_or_size, concat("byte after 'C' must be in range 0x00..0x7F; last byte: 0x", get_token_string()), "char");
auto last_token = get_token_string();
return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read,
exception_message(concat("byte after 'C' must be in range 0x00..0x7F; last byte: 0x", last_token), "char"), nullptr));
}
string_t s(1, static_cast<typename string_t::value_type>(current));
return sax->string(s);
@@ -3078,7 +3144,8 @@ class binary_reader
default: // anything else
break;
}
return invalid_byte("value");
auto last_token = get_token_string();
return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read, exception_message("invalid byte: 0x" + last_token, "value"), nullptr));
}
/*!
@@ -3141,7 +3208,7 @@ class binary_reader
if (JSON_HEDLEY_UNLIKELY((size_and_type.second == 'Z' || size_and_type.second == 'T' || size_and_type.second == 'F')
&& size_and_type.first > max_valueless_container_size))
{
return sax->parse_error(chars_read, get_token_string(), out_of_range::create(exception_id::container_too_large,
return sax->parse_error(chars_read, get_token_string(), out_of_range::create(408,
exception_message("excessive array size", "size"), nullptr));
}
@@ -3177,7 +3244,9 @@ class binary_reader
// do not accept ND-array size in objects in BJData
if (input_format == input_format_t::bjdata && size_and_type.first != npos && (size_and_type.second & (1 << 8)) != 0)
{
return last_byte_error(exception_id::unexpected_byte, "BJData object does not support ND-array size in optimized format", "object");
auto last_token = get_token_string();
return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read,
exception_message("BJData object does not support ND-array size in optimized format", "object"), nullptr));
}
if (size_and_type.first != npos)
@@ -3214,7 +3283,7 @@ class binary_reader
// the lexer would stop at a NUL and accept the digits before it
if (JSON_HEDLEY_UNLIKELY(current == '\0'))
{
return sax->parse_error(chars_read, "00", parse_error::create(exception_id::invalid_high_precision_number, chars_read,
return sax->parse_error(chars_read, "00", parse_error::create(115, chars_read,
exception_message("invalid number text; last byte: 0x00", "high-precision number"), nullptr));
}
number_vector.push_back(static_cast<char>(current));
@@ -3231,7 +3300,7 @@ class binary_reader
if (JSON_HEDLEY_UNLIKELY(result_remainder != token_type::end_of_input))
{
return sax->parse_error(chars_read, number_string, parse_error::create(exception_id::invalid_high_precision_number, chars_read,
return sax->parse_error(chars_read, number_string, parse_error::create(115, chars_read,
exception_message(concat("invalid number text: ", number_lexer.get_token_string()), "high-precision number"), nullptr));
}
@@ -3249,7 +3318,7 @@ class binary_reader
return sax->parse_error(
chars_read,
number_string,
out_of_range::create(exception_id::number_overflow, concat("number overflow parsing '", number_string, '\''), nullptr));
out_of_range::create(406, concat("number overflow parsing '", number_string, '\''), nullptr));
}
// number_string is a std::string, while the SAX interface takes a
// string_t; convert explicitly, as the two are only implicitly
@@ -3271,7 +3340,7 @@ class binary_reader
case token_type::end_of_input:
case token_type::literal_or_value:
default:
return sax->parse_error(chars_read, number_string, parse_error::create(exception_id::invalid_high_precision_number, chars_read,
return sax->parse_error(chars_read, number_string, parse_error::create(115, chars_read,
exception_message(concat("invalid number text: ", number_lexer.get_token_string()), "high-precision number"), nullptr));
}
}
@@ -3330,6 +3399,20 @@ class binary_reader
return 0x80 <= c && c <= 0xBF;
}
/*!
@brief report a parse error at the last read byte
@param[in] detail a detailed error message
@param[in] context further context information
@return false
*/
bool bon8_error(const std::string& detail, const char* context)
{
auto last_token = get_token_string();
return sax->parse_error(chars_read, last_token, parse_error::create(112, chars_read,
exception_message(concat(detail, ": 0x", last_token), context), nullptr));
}
/*!
@brief read a BON8 value and everything nested inside it
@@ -3545,7 +3628,7 @@ class binary_reader
}
// 0xFE: end of container where a value is expected
return invalid_byte("value");
return bon8_error("invalid byte", "value");
}
/*!
@@ -3672,7 +3755,7 @@ class binary_reader
current = byte;
}
return unexpected_byte("expected a string; last byte", "key");
return bon8_error("expected a string; last byte", "key");
}
/*!
@@ -3795,7 +3878,7 @@ class binary_reader
if (JSON_HEDLEY_UNLIKELY(!valid_second))
{
return unexpected_byte("invalid UTF-8 byte", "string");
return bon8_error("invalid UTF-8 byte", "string");
}
result.push_back(static_cast<typename string_t::value_type>(byte));
@@ -3809,7 +3892,7 @@ class binary_reader
}
if (JSON_HEDLEY_UNLIKELY(!is_bon8_continuation(current)))
{
return unexpected_byte("invalid UTF-8 byte", "string");
return bon8_error("invalid UTF-8 byte", "string");
}
result.push_back(static_cast<typename string_t::value_type>(current));
}
@@ -3854,7 +3937,7 @@ class binary_reader
{
// in case of failure, advance position by 1 to report the failing location
++chars_read;
sax->parse_error(chars_read, "<end of file>", parse_error::create(exception_id::unexpected_end_of_input, chars_read, exception_message("unexpected end of input", context), nullptr));
sax->parse_error(chars_read, "<end of file>", parse_error::create(110, chars_read, exception_message("unexpected end of input", context), nullptr));
return false;
}
return true;
@@ -4008,7 +4091,7 @@ class binary_reader
if (JSON_HEDLEY_UNLIKELY(std::isfinite(number) && !std::isfinite(result)))
{
return sax->parse_error(chars_read, get_token_string(),
out_of_range::create(exception_id::number_overflow, exception_message("number overflow", "value"), nullptr));
out_of_range::create(406, exception_message("number overflow", "value"), nullptr));
}
return sax->number_float(result, "");
}
@@ -4128,7 +4211,9 @@ class binary_reader
if (error_handler == error_handler_t::strict)
{
return last_byte_error(exception_id::invalid_string_or_size, "invalid string: ill-formed UTF-8 byte", context);
auto last_token = get_token_string();
return sax->parse_error(chars_read, last_token, parse_error::create(113, chars_read,
exception_message("invalid string: ill-formed UTF-8 byte", context), nullptr));
}
result = sanitize_utf8(result, error_handler);
@@ -4224,63 +4309,19 @@ class binary_reader
if (JSON_HEDLEY_UNLIKELY(current == char_traits<char_type>::eof()))
{
return sax->parse_error(chars_read, "<end of file>",
parse_error::create(exception_id::unexpected_end_of_input, chars_read, exception_message("unexpected end of input", context), nullptr));
parse_error::create(110, chars_read, exception_message("unexpected end of input", context), nullptr));
}
return true;
}
/*!
@brief reports a parse error at the last read byte
@param[in] id_ the id of the parse_error exception
@param[in] detail a detailed error message
@param[in] context further context information
@return the result of the SAX parser's parse_error()
*/
bool last_byte_error(const exception_id id_, const std::string& detail, const std::string& context) const
{
return sax->parse_error(chars_read, get_token_string(), parse_error::create(id_, chars_read, exception_message(detail, context), nullptr));
}
/*!
@brief reports the last read byte as unexpected (parse_error.112)
@param[in] detail what is wrong with the byte, e.g. "invalid byte"
@param[in] context further context information
@return the result of the SAX parser's parse_error()
*/
bool unexpected_byte(const char* detail, const std::string& context) const
{
return last_byte_error(exception_id::unexpected_byte, concat(detail, ": 0x", get_token_string()), context);
}
/*!
@brief reports the last read byte as invalid (parse_error.112)
@param[in] context further context information
@return the result of the SAX parser's parse_error()
*/
bool invalid_byte(const char* context) const
{
return unexpected_byte("invalid byte", context);
}
/*!
@brief reports that the last read byte is not a UBJSON/BJData length type (parse_error.113)
@param[in] position where the length type was expected, e.g. " after '#'"
@param[in] context further context information
@return the result of the SAX parser's parse_error()
*/
bool length_type_error(const char* position, const char* context) const
{
const char* types = input_format == input_format_t::bjdata ? "U, i, u, I, m, l, M, L" : "U, i, I, l, L";
return last_byte_error(exception_id::invalid_string_or_size,
concat("expected length type specification (", types, ")", position, "; last byte: 0x", get_token_string()), context);
}
/*!
@return a string representation of the last read byte
*/
std::string get_token_string() const
{
return hex_byte(static_cast<std::uint8_t>(current));
std::array<char, 3> cr{{}};
static_cast<void>((std::snprintf)(cr.data(), cr.size(), "%.2hhX", static_cast<unsigned char>(current))); // NOLINT(cppcoreguidelines-pro-type-vararg,hicpp-vararg)
return std::string{cr.data()};
}
/*!
@@ -581,7 +581,7 @@ class wide_string_input_adapter
template<class T>
JSON_HEDLEY_NO_RETURN std::size_t get_elements(T* /*dest*/, std::size_t /*count*/ = 1)
{
JSON_THROW(parse_error::create(exception_id::unexpected_byte, 1, "wide string type cannot be interpreted as binary data", nullptr));
JSON_THROW(parse_error::create(112, 1, "wide string type cannot be interpreted as binary data", nullptr));
}
private:
@@ -793,7 +793,7 @@ inline file_input_adapter input_adapter(std::FILE* file)
{
if (file == nullptr)
{
JSON_THROW(parse_error::create(exception_id::syntax_error, 0, "attempting to parse an empty input; check that your input string or stream contains the expected JSON", nullptr));
JSON_THROW(parse_error::create(101, 0, "attempting to parse an empty input; check that your input string or stream contains the expected JSON", nullptr));
}
return file_input_adapter(file);
}
@@ -802,7 +802,7 @@ inline input_stream_adapter input_adapter(std::istream& stream)
{
if (stream.rdbuf() == nullptr)
{
JSON_THROW(parse_error::create(exception_id::syntax_error, 0, "attempting to parse an empty input; check that your input string or stream contains the expected JSON", nullptr));
JSON_THROW(parse_error::create(101, 0, "attempting to parse an empty input; check that your input string or stream contains the expected JSON", nullptr));
}
return input_stream_adapter(stream);
}
@@ -827,7 +827,7 @@ contiguous_bytes_input_adapter input_adapter(CharT b)
{
if (b == nullptr)
{
JSON_THROW(parse_error::create(exception_id::syntax_error, 0, "attempting to parse an empty input; check that your input string or stream contains the expected JSON", nullptr));
JSON_THROW(parse_error::create(101, 0, "attempting to parse an empty input; check that your input string or stream contains the expected JSON", nullptr));
}
auto length = std::strlen(reinterpret_cast<const char*>(b));
const auto* ptr = reinterpret_cast<const char*>(b);
+76 -83
View File
@@ -176,27 +176,6 @@ template<typename ArrayType>
inline void reserve_array(ArrayType& /*arr*/, std::size_t /*len*/, priority_tag<0> /*unused*/)
{}
/*!
@brief reports an object or array whose announced size exceeds max_size()
Shared by json_sax_dom_parser and json_sax_dom_callback_parser.
@param[in] sax the SAX parser to report the error to
@param[in] len the number of elements announced by the input, or unknown_size()
@param[in] kind "object" or "array"
@param[in] ref the object or array that was just created
@return whether parsing should continue (false after reporting out_of_range.408)
*/
template<typename SAX, typename BasicJsonType>
bool check_container_size(SAX& sax, std::size_t len, const char* kind, BasicJsonType* ref)
{
if (JSON_HEDLEY_UNLIKELY(len != detail::unknown_size() && len > ref->max_size()))
{
return sax.parse_error(0, "", out_of_range::create(exception_id::container_too_large, concat("excessive ", kind, " size: ", std::to_string(len)), ref));
}
return true;
}
#if JSON_DIAGNOSTIC_POSITIONS
/*!
@brief set the diagnostic positions of a value the DOM SAX parsers just stored
@@ -206,35 +185,6 @@ befriends this struct, as the position members are private.
*/
struct diagnostic_positions
{
/*!
@param[in,out] v the object or array whose opening brace or bracket was just read
@param[in] lexer the lexer that read it, or nullptr to leave @a v alone
*/
template<typename BasicJsonType, typename LexerType>
static void set_container_start(BasicJsonType& v, LexerType* lexer)
{
if (lexer)
{
// Lexer has read the first character of the container, so
// subtract 1 from the position to get the correct start position.
v.start_position = lexer->get_position() - 1;
}
}
/*!
@param[in,out] v the object or array whose closing brace or bracket was just read
@param[in] lexer the lexer that read it, or nullptr to leave @a v alone
*/
template<typename BasicJsonType, typename LexerType>
static void set_container_end(BasicJsonType& v, LexerType* lexer)
{
if (lexer)
{
// Lexer's position is past the closing brace or bracket, so set that as the end position.
v.end_position = lexer->get_position();
}
}
/*!
@param[in,out] v the value that was just parsed
@param[in] lexer the lexer that read it, or nullptr to leave @a v alone
@@ -399,11 +349,22 @@ class json_sax_dom_parser
ref_stack.push_back(handle_value(BasicJsonType::value_t::object));
#if JSON_DIAGNOSTIC_POSITIONS
// Manually set the start position of the object here.
// Ensure this is after the call to handle_value to ensure correct start position.
diagnostic_positions::set_container_start(*ref_stack.back(), m_lexer_ref);
if (m_lexer_ref)
{
// Lexer has read the first character of the object, so
// subtract 1 from the position to get the correct start position.
ref_stack.back()->start_position = m_lexer_ref->get_position() - 1;
}
#endif
return check_container_size(*this, len, "object", ref_stack.back());
if (JSON_HEDLEY_UNLIKELY(len != detail::unknown_size() && len > ref_stack.back()->max_size()))
{
return parse_error(0, "", out_of_range::create(408, concat("excessive object size: ", std::to_string(len)), ref_stack.back()));
}
return true;
}
bool key(string_t& val)
@@ -422,7 +383,11 @@ class json_sax_dom_parser
JSON_ASSERT(ref_stack.back()->is_object());
#if JSON_DIAGNOSTIC_POSITIONS
diagnostic_positions::set_container_end(*ref_stack.back(), m_lexer_ref);
if (m_lexer_ref)
{
// Lexer's position is past the closing brace, so set that as the end position.
ref_stack.back()->end_position = m_lexer_ref->get_position();
}
#endif
ref_stack.back()->set_parents();
@@ -435,13 +400,17 @@ class json_sax_dom_parser
ref_stack.push_back(handle_value(BasicJsonType::value_t::array));
#if JSON_DIAGNOSTIC_POSITIONS
// Manually set the start position of the array here.
// Ensure this is after the call to handle_value to ensure correct start position.
diagnostic_positions::set_container_start(*ref_stack.back(), m_lexer_ref);
if (m_lexer_ref)
{
ref_stack.back()->start_position = m_lexer_ref->get_position() - 1;
}
#endif
if (JSON_HEDLEY_UNLIKELY(!check_container_size(*this, len, "array", ref_stack.back())))
if (JSON_HEDLEY_UNLIKELY(len != detail::unknown_size() && len > ref_stack.back()->max_size()))
{
return false;
return parse_error(0, "", out_of_range::create(408, concat("excessive array size: ", std::to_string(len)), ref_stack.back()));
}
if (len != detail::unknown_size())
@@ -458,7 +427,11 @@ class json_sax_dom_parser
JSON_ASSERT(ref_stack.back()->is_array());
#if JSON_DIAGNOSTIC_POSITIONS
diagnostic_positions::set_container_end(*ref_stack.back(), m_lexer_ref);
if (m_lexer_ref)
{
// Lexer's position is past the closing bracket, so set that as the end position.
ref_stack.back()->end_position = m_lexer_ref->get_position();
}
#endif
ref_stack.back()->set_parents();
@@ -638,11 +611,21 @@ class json_sax_dom_callback_parser
{
#if JSON_DIAGNOSTIC_POSITIONS
// Manually set the start position of the object here.
// Ensure this is after the call to handle_value to ensure correct start position.
diagnostic_positions::set_container_start(*ref_stack.back(), m_lexer_ref);
if (m_lexer_ref)
{
// Lexer has read the first character of the object, so
// subtract 1 from the position to get the correct start position.
ref_stack.back()->start_position = m_lexer_ref->get_position() - 1;
}
#endif
return check_container_size(*this, len, "object", ref_stack.back());
// check object limit
if (JSON_HEDLEY_UNLIKELY(len != detail::unknown_size() && len > ref_stack.back()->max_size()))
{
return parse_error(0, "", out_of_range::create(408, concat("excessive object size: ", std::to_string(len)), ref_stack.back()));
}
}
return true;
}
@@ -712,7 +695,11 @@ class json_sax_dom_callback_parser
{
#if JSON_DIAGNOSTIC_POSITIONS
diagnostic_positions::set_container_end(*ref_stack.back(), m_lexer_ref);
if (m_lexer_ref)
{
// Lexer's position is past the closing brace, so set that as the end position.
ref_stack.back()->end_position = m_lexer_ref->get_position();
}
#endif
ref_stack.back()->set_parents();
@@ -723,7 +710,13 @@ class json_sax_dom_callback_parser
}
}
const string_t object_key = pop_container();
JSON_ASSERT(!ref_stack.empty());
JSON_ASSERT(!keep_stack.empty());
JSON_ASSERT(!container_key_stack.empty());
ref_stack.pop_back();
keep_stack.pop_back();
const string_t object_key = std::move(container_key_stack.back());
container_key_stack.pop_back();
if (!ref_stack.empty() && ref_stack.back() && ref_stack.back()->is_structured())
{
@@ -749,13 +742,20 @@ class json_sax_dom_callback_parser
{
#if JSON_DIAGNOSTIC_POSITIONS
// Manually set the start position of the array here.
// Ensure this is after the call to handle_value to ensure correct start position.
diagnostic_positions::set_container_start(*ref_stack.back(), m_lexer_ref);
if (m_lexer_ref)
{
// Lexer has read the first character of the array, so
// subtract 1 from the position to get the correct start position.
ref_stack.back()->start_position = m_lexer_ref->get_position() - 1;
}
#endif
if (JSON_HEDLEY_UNLIKELY(!check_container_size(*this, len, "array", ref_stack.back())))
// check array limit
if (JSON_HEDLEY_UNLIKELY(len != detail::unknown_size() && len > ref_stack.back()->max_size()))
{
return false;
return parse_error(0, "", out_of_range::create(408, concat("excessive array size: ", std::to_string(len)), ref_stack.back()));
}
if (len != detail::unknown_size())
@@ -779,7 +779,11 @@ class json_sax_dom_callback_parser
{
#if JSON_DIAGNOSTIC_POSITIONS
diagnostic_positions::set_container_end(*ref_stack.back(), m_lexer_ref);
if (m_lexer_ref)
{
// Lexer's position is past the closing bracket, so set that as the end position.
ref_stack.back()->end_position = m_lexer_ref->get_position();
}
#endif
ref_stack.back()->set_parents();
@@ -805,7 +809,13 @@ class json_sax_dom_callback_parser
}
}
const string_t object_key = pop_container();
JSON_ASSERT(!ref_stack.empty());
JSON_ASSERT(!keep_stack.empty());
JSON_ASSERT(!container_key_stack.empty());
ref_stack.pop_back();
keep_stack.pop_back();
const string_t object_key = std::move(container_key_stack.back());
container_key_stack.pop_back();
// remove discarded value
if (!ref_stack.empty() && ref_stack.back())
@@ -892,23 +902,6 @@ class json_sax_dom_callback_parser
return string_t{};
}
/*!
@brief leave the object or array that end_object()/end_array() just closed
@return the key the container is stored under in its parent (see
container_key_stack)
*/
string_t pop_container()
{
JSON_ASSERT(!ref_stack.empty());
JSON_ASSERT(!keep_stack.empty());
JSON_ASSERT(!container_key_stack.empty());
ref_stack.pop_back();
keep_stack.pop_back();
string_t object_key = std::move(container_key_stack.back());
container_key_stack.pop_back();
return object_key;
}
/*!
@brief remove the discarded value the callback rejected from its parent,
unless it is a duplicate key's slot with a stashed previous value, in
+59 -10
View File
@@ -221,6 +221,44 @@ class lexer : public lexer_base<BasicJsonType>
// scan functions
/////////////////////
/// contiguous input: try to decode the 4 hex digits following `\\u`
/// directly from the input buffer via hex_codepoint(), instead of 4 calls
/// to get(). On success, advances the adapter and the position counters
/// exactly as those 4 get() calls would (a hex digit is never '\n', so
/// only the flat counters move) and leaves @a current holding the last of
/// the 4 digits, just as the last such get() would; the codepoint is
/// written to @a out. Makes no state change and returns false - for a
/// pending unget, fewer than 4 remaining bytes, or any of the 4 bytes not
/// being a hex digit - so the caller falls back unchanged to the
/// per-character loop, which then reports the same diagnostic (stopping
/// at the first invalid digit) as before this optimization.
bool get_codepoint_bulk(std::true_type /*bulk*/, int& out)
{
if (next_unget || ia.bulk_remaining() < 4)
{
return false;
}
const char_type* const raw = ia.bulk_data();
const int codepoint = hex_codepoint(reinterpret_cast<const unsigned char*>(raw));
if (codepoint < 0)
{
return false;
}
ia.bulk_skip(4);
// a hex digit is never a newline, so only the flat counters advance
position.chars_read_total += 4;
position.chars_read_current_line += 4;
current = char_traits<char_type>::to_int_type(raw[3]);
out = codepoint;
return true;
}
/// streaming input: no bulk fast path
bool get_codepoint_bulk(std::false_type /*bulk*/, int& /*out*/) const noexcept
{
return false;
}
/*!
@brief get codepoint from 4 hex characters following `\\u`
@@ -240,6 +278,14 @@ class lexer : public lexer_base<BasicJsonType>
{
// this function only makes sense after reading `\u`
JSON_ASSERT(current == 'u');
// contiguous input: decode all 4 hex digits directly from the buffer
int fast_codepoint = 0;
if (get_codepoint_bulk(std::integral_constant<bool, bulk_scan> {}, fast_codepoint))
{
return fast_codepoint;
}
int codepoint = 0;
const auto factors = { 12u, 8u, 4u, 0u };
@@ -1044,9 +1090,11 @@ class lexer : public lexer_base<BasicJsonType>
token_type::parse_error otherwise
@note The scanner is independent of the current locale: token_buffer
always holds `.`. Only the std::strtod fallback of convert_number()
depends on the locale, and it looks up the decimal point right
before converting (see detail::convert_float_locale_aware()).
always holds `.`. The conversion of float and double does not use
the locale either. Only the std::strtold fallback of
convert_number() for long double formats other than binary64
depends on it, and it looks up the decimal point right before
converting (see detail::convert_float_locale_aware()).
*/
token_type scan_number() // lgtm [cpp/use-of-goto] `goto` is used in this function to implement the number-parsing state machine described above. By design, any finite input will eventually reach the "done" state or return token_type::parse_error. In each intermediate state, 1 byte of the input is appended to the token_buffer vector, and only the already initialized variables token_buffer, number_type, and error_message are manipulated.
{
@@ -1059,7 +1107,7 @@ class lexer : public lexer_base<BasicJsonType>
// offset just past the last mantissa byte in token_buffer (i.e. the
// index of 'e'/'E', or the whole token when there is no exponent).
// convert_number() uses it to count significant digits; npos means
// convert_number() uses it to split the token; npos means
// "not seen an exponent yet" and is resolved at scan_number_done
std::size_t mantissa_end = std::string::npos;
@@ -1389,8 +1437,8 @@ scan_number_done:
@param[in] mantissa_end offset just past the last mantissa byte in
token_buffer (the index of 'e'/'E', or
token_buffer.size() when there is no exponent);
used to skip Clinger's fast path when it cannot
possibly succeed - see detail::mantissa_fits_clinger()
with decimal_point_position, it locates the parts
of a float token without scanning it again
*/
token_type convert_number(token_type number_type, std::size_t mantissa_end)
{
@@ -1444,10 +1492,11 @@ scan_number_done:
}
// this code is reached if we parse a floating-point number or if an
// integer conversion above overflowed. Prefer std::from_chars
// (Eisel-Lemire, locale-independent, correctly rounded) when available;
// otherwise the exact Clinger fast path (double only); otherwise the
// locale-aware strtof/strtod/strtold.
// integer conversion above overflowed. float and double (and long
// double where it is binary64) are converted by the library itself,
// correctly rounded and independent of the locale; other long double
// formats use std::from_chars when available, otherwise the
// locale-aware strtold.
if (convert_float_fast(num_begin, num_end, decimal_point_position, mantissa_end, value_float))
{
return token_type::value_float;
File diff suppressed because it is too large. Load diff
+39 -26
View File
@@ -156,7 +156,9 @@ class parser
// strict mode: next byte must be EOF
if (get_token() != token_type::end_of_input)
{
return syntax_error(*sax, exception_message(token_type::end_of_input, "value"));
return sax->parse_error(m_lexer.get_position(),
m_lexer.get_token_string(),
parse_error::create(101, m_lexer.get_position(), exception_message(token_type::end_of_input, "value"), nullptr));
}
}
else
@@ -195,7 +197,10 @@ class parser
// in strict mode, input must be completely read
if (get_token() != token_type::end_of_input)
{
syntax_error(sdp, exception_message(token_type::end_of_input, "value"));
sdp.parse_error(m_lexer.get_position(),
m_lexer.get_token_string(),
parse_error::create(101, m_lexer.get_position(),
exception_message(token_type::end_of_input, "value"), nullptr));
}
}
else
@@ -245,7 +250,9 @@ class parser
// parse key
if (JSON_HEDLEY_UNLIKELY(last_token != token_type::value_string))
{
return syntax_error(*sax, exception_message(token_type::value_string, "object key"));
return sax->parse_error(m_lexer.get_position(),
m_lexer.get_token_string(),
parse_error::create(101, m_lexer.get_position(), exception_message(token_type::value_string, "object key"), nullptr));
}
if (JSON_HEDLEY_UNLIKELY(!sax->key(m_lexer.get_string())))
{
@@ -255,7 +262,9 @@ class parser
// parse separator (:)
if (JSON_HEDLEY_UNLIKELY(!get_token_expecting(token_type::name_separator)))
{
return syntax_error(*sax, exception_message(token_type::name_separator, "object separator"));
return sax->parse_error(m_lexer.get_position(),
m_lexer.get_token_string(),
parse_error::create(101, m_lexer.get_position(), exception_message(token_type::name_separator, "object separator"), nullptr));
}
// remember we are now inside an object
@@ -298,7 +307,7 @@ class parser
{
return sax->parse_error(m_lexer.get_position(),
m_lexer.get_token_string(),
out_of_range::create(exception_id::number_overflow, concat("number overflow parsing '", m_lexer.get_token_string(), '\''), nullptr));
out_of_range::create(406, concat("number overflow parsing '", m_lexer.get_token_string(), '\''), nullptr));
}
if (JSON_HEDLEY_UNLIKELY(!sax->number_float(res, m_lexer.get_string())))
@@ -366,16 +375,23 @@ class parser
case token_type::parse_error:
{
// using "uninitialized" to avoid an "expected" message
return syntax_error(*sax, exception_message(token_type::uninitialized, "value"));
return sax->parse_error(m_lexer.get_position(),
m_lexer.get_token_string(),
parse_error::create(101, m_lexer.get_position(), exception_message(token_type::uninitialized, "value"), nullptr));
}
case token_type::end_of_input:
{
if (JSON_HEDLEY_UNLIKELY(m_lexer.get_position().chars_read_total == 1))
{
return syntax_error(*sax, "attempting to parse an empty input; check that your input string or stream contains the expected JSON");
return sax->parse_error(m_lexer.get_position(),
m_lexer.get_token_string(),
parse_error::create(101, m_lexer.get_position(),
"attempting to parse an empty input; check that your input string or stream contains the expected JSON", nullptr));
}
return syntax_error(*sax, exception_message(token_type::literal_or_value, "value"));
return sax->parse_error(m_lexer.get_position(),
m_lexer.get_token_string(),
parse_error::create(101, m_lexer.get_position(), exception_message(token_type::literal_or_value, "value"), nullptr));
}
case token_type::uninitialized:
case token_type::end_array:
@@ -385,7 +401,9 @@ class parser
case token_type::literal_or_value:
default: // the last token was unexpected
{
return syntax_error(*sax, exception_message(token_type::literal_or_value, "value"));
return sax->parse_error(m_lexer.get_position(),
m_lexer.get_token_string(),
parse_error::create(101, m_lexer.get_position(), exception_message(token_type::literal_or_value, "value"), nullptr));
}
}
}
@@ -436,7 +454,9 @@ class parser
continue;
}
return syntax_error(*sax, exception_message(token_type::end_array, "array"));
return sax->parse_error(m_lexer.get_position(),
m_lexer.get_token_string(),
parse_error::create(101, m_lexer.get_position(), exception_message(token_type::end_array, "array"), nullptr));
}
// states.back() is false -> object
@@ -453,7 +473,9 @@ class parser
// parse key
if (JSON_HEDLEY_UNLIKELY(last_token != token_type::value_string))
{
return syntax_error(*sax, exception_message(token_type::value_string, "object key"));
return sax->parse_error(m_lexer.get_position(),
m_lexer.get_token_string(),
parse_error::create(101, m_lexer.get_position(), exception_message(token_type::value_string, "object key"), nullptr));
}
if (JSON_HEDLEY_UNLIKELY(!sax->key(m_lexer.get_string())))
@@ -464,7 +486,9 @@ class parser
// parse separator (:)
if (JSON_HEDLEY_UNLIKELY(!get_token_expecting(token_type::name_separator)))
{
return syntax_error(*sax, exception_message(token_type::name_separator, "object separator"));
return sax->parse_error(m_lexer.get_position(),
m_lexer.get_token_string(),
parse_error::create(101, m_lexer.get_position(), exception_message(token_type::name_separator, "object separator"), nullptr));
}
// parse values
@@ -492,7 +516,9 @@ class parser
continue;
}
return syntax_error(*sax, exception_message(token_type::end_object, "object"));
return sax->parse_error(m_lexer.get_position(),
m_lexer.get_token_string(),
parse_error::create(101, m_lexer.get_position(), exception_message(token_type::end_object, "object"), nullptr));
}
}
@@ -509,19 +535,6 @@ class parser
return (last_token = m_lexer.scan_expecting(expected_type)) == expected_type;
}
/*!
@brief reports a syntax error at the current token (parse_error.101)
@param[in] sax the SAX parser to report the error to
@param[in] message the error message
@return the result of the SAX parser's parse_error()
*/
template<typename SAX>
bool syntax_error(SAX& sax, const std::string& message)
{
return sax.parse_error(m_lexer.get_position(), m_lexer.get_token_string(),
parse_error::create(exception_id::syntax_error, m_lexer.get_position(), message, nullptr));
}
std::string exception_message(const token_type expected, const std::string& context)
{
std::string error_msg = "syntax error ";
@@ -22,6 +22,10 @@ namespace detail
constexpr std::int64_t pow5_128_smallest_power = -342;
constexpr std::int64_t pow5_128_largest_power = 308;
// every entry of pow5_128() holds two 64-bit halves of 5^q, one per covered power of 5
static_assert((pow5_128_largest_power - pow5_128_smallest_power + 1) * 2 == 1302,
"pow5_128_smallest_power/pow5_128_largest_power must match the size of the pow5_128() table");
/*!
@brief 128-bit approximations of 5^q for q in [-342, 308]
+70 -29
View File
@@ -8,10 +8,12 @@
#pragma once
#include <array> // array
#include <cstddef> // size_t
#include <cstdint> // uint64_t
#include <cstdint> // uint64_t, uint8_t
#include <cstring> // memcpy
#include <nlohmann/detail/bit_ops.hpp>
#include <nlohmann/detail/macro_scope.hpp>
// Optional SIMD backend for bulk UTF-8 validation. This is an opt-in external
@@ -69,18 +71,12 @@ inline std::size_t find_string_special(const unsigned char* data, std::size_t n)
std::size_t i = 0;
for (; i + 8 <= n; i += 8)
{
std::uint64_t word = 0;
std::memcpy(&word, data + i, sizeof(word));
if (swar_string_special(word) != 0)
const std::uint64_t special = swar_string_special(read_eight_bytes(data + i));
if (special != 0)
{
// a special byte is in this word; locate it (endian-agnostic)
for (std::size_t j = 0; j < 8; ++j)
{
if (is_string_special(data[i + j]))
{
return i + j;
}
}
// the lowest flagged byte is the first special one: the borrows of
// the subtractions can only flag bytes above a true hit
return i + (static_cast<std::size_t>(count_trailing_zeros(special)) / 8);
}
}
for (; i < n; ++i)
@@ -114,8 +110,7 @@ inline std::size_t find_ascii_copyable_run(const unsigned char* data, std::size_
std::size_t i = 0;
for (; i + 8 <= n; i += 8)
{
std::uint64_t v = 0;
std::memcpy(&v, data + i, sizeof(v));
const std::uint64_t v = read_eight_bytes(data + i);
const std::uint64_t q = v ^ 0x2222222222222222ull; // '"' (0x22)
const std::uint64_t b = v ^ 0x5C5C5C5C5C5C5C5Cull; // '\\' (0x5C)
const std::uint64_t d = v ^ 0x7F7F7F7F7F7F7F7Full; // DEL (0x7F)
@@ -126,7 +121,9 @@ inline std::size_t find_ascii_copyable_run(const unsigned char* data, std::size_
| (v & high); // >= 0x80
if (stop != 0)
{
break;
// the lowest flagged byte is the first one to stop at (see
// find_string_special())
return i + (static_cast<std::size_t>(count_trailing_zeros(stop)) / 8);
}
}
for (; i < n; ++i)
@@ -253,12 +250,18 @@ inline std::size_t scalar_string_bulk_run(const unsigned char* data, std::size_t
{
break; // end of buffer, or a quote/escape/control byte
}
const std::size_t seq = validate_one_utf8(data + pos, n - pos);
if (seq == 0)
// a run of multi-byte sequences (e.g. CJK text) is validated sequence
// by sequence without searching for the next special byte in between
do
{
break; // ill-formed or truncated: let the byte path diagnose it
const std::size_t seq = validate_one_utf8(data + pos, n - pos);
if (seq == 0)
{
return pos; // ill-formed or truncated: let the byte path diagnose it
}
pos += seq;
}
pos += seq;
while (pos < n && data[pos] >= 0x80u);
}
return pos;
}
@@ -273,8 +276,7 @@ inline std::size_t find_string_delimiter(const unsigned char* data, std::size_t
std::size_t i = 0;
for (; i + 8 <= n; i += 8)
{
std::uint64_t v = 0;
std::memcpy(&v, data + i, sizeof(v));
const std::uint64_t v = read_eight_bytes(data + i);
const std::uint64_t q = v ^ 0x2222222222222222ull;
const std::uint64_t b = v ^ 0x5C5C5C5C5C5C5C5Cull;
const std::uint64_t hit = ((q - ones) & ~q & high)
@@ -282,14 +284,8 @@ inline std::size_t find_string_delimiter(const unsigned char* data, std::size_t
| ((v - 0x2020202020202020ull) & ~v & high);
if (hit != 0)
{
for (std::size_t j = 0; j < 8; ++j)
{
const unsigned char c = data[i + j];
if (c == '\"' || c == '\\' || c < 0x20u)
{
return i + j;
}
}
// the lowest flagged byte is the first delimiter (see find_string_special())
return i + (static_cast<std::size_t>(count_trailing_zeros(hit)) / 8);
}
}
for (; i < n; ++i)
@@ -320,5 +316,50 @@ inline std::size_t string_bulk_run(const unsigned char* data, std::size_t n) noe
return scalar_string_bulk_run(data, n);
}
// Decode the 4 hex digits at [data, data+4) - the digits following a `\u`
// escape - into a codepoint 0x0000..0xFFFF via one table lookup per byte
// (after yyjson's read_hex_u16), or return -1 if any of the 4 bytes is not a
// hex digit ('0'..'9', 'A'..'F', 'a'..'f'). The caller must already have
// checked that 4 bytes are available; used by lexer::get_codepoint()'s
// contiguous fast path. On -1 it falls back to the byte-at-a-time loop, which
// stops at the first invalid digit, so the reported error and position are
// unaffected by this fast path.
inline int hex_codepoint(const unsigned char* data) noexcept
{
static const std::array<std::uint8_t, 256> hex_digit_table = // NOLINT(cppcoreguidelines-avoid-non-const-global-variables)
{
{
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // 00..0F
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // 10..1F
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // 20..2F
0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // 30..3F ('0'..'9')
0xFF, 0x0A, 0x0B, 0x0C, 0x0D, 0x0E, 0x0F, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // 40..4F ('A'..'F')
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // 50..5F
0xFF, 0x0A, 0x0B, 0x0C, 0x0D, 0x0E, 0x0F, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // 60..6F ('a'..'f')
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // 70..7F
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // 80..8F
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // 90..9F
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // A0..AF
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // B0..BF
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // C0..CF
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // D0..DF
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // E0..EF
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF // F0..FF
}
};
const std::uint8_t d0 = hex_digit_table[data[0]];
const std::uint8_t d1 = hex_digit_table[data[1]];
const std::uint8_t d2 = hex_digit_table[data[2]];
const std::uint8_t d3 = hex_digit_table[data[3]];
// every valid digit is <= 0xF; the combined OR only exceeds it if at
// least one of the four bytes was not a hex digit (looked up as 0xFF)
if ((d0 | d1 | d2 | d3) > 0x0F)
{
return -1;
}
return (d0 << 12) | (d1 << 8) | (d2 << 4) | d3;
}
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
+12 -18
View File
@@ -279,12 +279,6 @@ class iter_impl // NOLINT(cppcoreguidelines-special-member-functions,hicpp-speci
}
}
/// @throw invalid_iterator.214 always
JSON_HEDLEY_NO_RETURN void throw_cannot_get_value() const
{
JSON_THROW(invalid_iterator::create(exception_id::iterator_value_unavailable, "cannot get value", m_object));
}
public:
/*!
@brief return a reference to the value pointed to by the iterator
@@ -309,7 +303,7 @@ class iter_impl // NOLINT(cppcoreguidelines-special-member-functions,hicpp-speci
}
case value_t::null:
throw_cannot_get_value();
JSON_THROW(invalid_iterator::create(214, "cannot get value", m_object));
case value_t::string:
case value_t::boolean:
@@ -325,7 +319,7 @@ class iter_impl // NOLINT(cppcoreguidelines-special-member-functions,hicpp-speci
return *m_object;
}
throw_cannot_get_value();
JSON_THROW(invalid_iterator::create(214, "cannot get value", m_object));
}
}
}
@@ -367,7 +361,7 @@ class iter_impl // NOLINT(cppcoreguidelines-special-member-functions,hicpp-speci
return m_object;
}
throw_cannot_get_value();
JSON_THROW(invalid_iterator::create(214, "cannot get value", m_object));
}
}
}
@@ -484,7 +478,7 @@ class iter_impl // NOLINT(cppcoreguidelines-special-member-functions,hicpp-speci
// if objects are not the same, the comparison is undefined
if (JSON_HEDLEY_UNLIKELY(m_object != other.m_object))
{
JSON_THROW(invalid_iterator::create(exception_id::iterators_compare_different_values, "cannot compare iterators of different containers", m_object));
JSON_THROW(invalid_iterator::create(212, "cannot compare iterators of different containers", m_object));
}
// value-initialized forward iterators can be compared, and must compare equal to other value-initialized iterators of the same type #4493
@@ -533,7 +527,7 @@ class iter_impl // NOLINT(cppcoreguidelines-special-member-functions,hicpp-speci
// if objects are not the same, the comparison is undefined
if (JSON_HEDLEY_UNLIKELY(m_object != other.m_object))
{
JSON_THROW(invalid_iterator::create(exception_id::iterators_compare_different_values, "cannot compare iterators of different containers", m_object));
JSON_THROW(invalid_iterator::create(212, "cannot compare iterators of different containers", m_object));
}
// value-initialized forward iterators can be compared, and must compare equal to other value-initialized iterators of the same type #4493
@@ -546,7 +540,7 @@ class iter_impl // NOLINT(cppcoreguidelines-special-member-functions,hicpp-speci
switch (m_object->m_data.m_type)
{
case value_t::object:
JSON_THROW(invalid_iterator::create(exception_id::iterator_order_on_object, "cannot compare order of object iterators", m_object));
JSON_THROW(invalid_iterator::create(213, "cannot compare order of object iterators", m_object));
case value_t::array:
return (m_it.array_iterator < other.m_it.array_iterator);
@@ -602,7 +596,7 @@ class iter_impl // NOLINT(cppcoreguidelines-special-member-functions,hicpp-speci
switch (m_object->m_data.m_type)
{
case value_t::object:
JSON_THROW(invalid_iterator::create(exception_id::iterator_arithmetic_on_object, "cannot use offsets with object iterators", m_object));
JSON_THROW(invalid_iterator::create(209, "cannot use offsets with object iterators", m_object));
case value_t::array:
{
@@ -681,7 +675,7 @@ class iter_impl // NOLINT(cppcoreguidelines-special-member-functions,hicpp-speci
switch (m_object->m_data.m_type)
{
case value_t::object:
JSON_THROW(invalid_iterator::create(exception_id::iterator_arithmetic_on_object, "cannot use offsets with object iterators", m_object));
JSON_THROW(invalid_iterator::create(209, "cannot use offsets with object iterators", m_object));
case value_t::array:
return m_it.array_iterator - other.m_it.array_iterator;
@@ -710,13 +704,13 @@ class iter_impl // NOLINT(cppcoreguidelines-special-member-functions,hicpp-speci
switch (m_object->m_data.m_type)
{
case value_t::object:
JSON_THROW(invalid_iterator::create(exception_id::iterator_subscript_on_object, "cannot use operator[] for object iterators", m_object));
JSON_THROW(invalid_iterator::create(208, "cannot use operator[] for object iterators", m_object));
case value_t::array:
return *std::next(m_it.array_iterator, n);
case value_t::null:
throw_cannot_get_value();
JSON_THROW(invalid_iterator::create(214, "cannot get value", m_object));
case value_t::string:
case value_t::boolean:
@@ -732,7 +726,7 @@ class iter_impl // NOLINT(cppcoreguidelines-special-member-functions,hicpp-speci
return *m_object;
}
throw_cannot_get_value();
JSON_THROW(invalid_iterator::create(214, "cannot get value", m_object));
}
}
}
@@ -750,7 +744,7 @@ class iter_impl // NOLINT(cppcoreguidelines-special-member-functions,hicpp-speci
return m_it.object_iterator->first;
}
JSON_THROW(invalid_iterator::create(exception_id::iterator_key_not_object, "cannot use key() for non-object iterators", m_object));
JSON_THROW(invalid_iterator::create(207, "cannot use key() for non-object iterators", m_object));
}
/*!
+22 -37
View File
@@ -166,7 +166,7 @@ class json_pointer
{
if (JSON_HEDLEY_UNLIKELY(empty()))
{
throw_no_parent();
JSON_THROW(detail::out_of_range::create(405, "JSON pointer has no parent", nullptr));
}
reference_tokens.erase(reference_tokens.begin());
@@ -178,7 +178,7 @@ class json_pointer
{
if (JSON_HEDLEY_UNLIKELY(empty()))
{
throw_no_parent();
JSON_THROW(detail::out_of_range::create(405, "JSON pointer has no parent", nullptr));
}
return reference_tokens.front();
@@ -204,7 +204,7 @@ class json_pointer
{
if (JSON_HEDLEY_UNLIKELY(empty()))
{
throw_no_parent();
JSON_THROW(detail::out_of_range::create(405, "JSON pointer has no parent", nullptr));
}
reference_tokens.pop_back();
@@ -216,7 +216,7 @@ class json_pointer
{
if (JSON_HEDLEY_UNLIKELY(empty()))
{
throw_no_parent();
JSON_THROW(detail::out_of_range::create(405, "JSON pointer has no parent", nullptr));
}
return reference_tokens.back();
@@ -244,21 +244,6 @@ class json_pointer
}
private:
/// @throw out_of_range.405 always
JSON_HEDLEY_NO_RETURN static void throw_no_parent()
{
JSON_THROW(detail::out_of_range::create(detail::exception_id::patch_on_root, "JSON pointer has no parent", nullptr));
}
/// @throw out_of_range.404 always
template<typename BasicJsonContext>
JSON_HEDLEY_NO_RETURN static void throw_unresolved(const string_t& reference_token, BasicJsonContext context)
{
static_cast<void>(reference_token); // unused when JSON_NOEXCEPTION is defined
static_cast<void>(context);
JSON_THROW(detail::out_of_range::create(detail::exception_id::pointer_unresolved, detail::concat("unresolved reference token '", reference_token, "'"), context));
}
/*!
@brief result of @ref parse_array_index
@@ -344,13 +329,13 @@ class json_pointer
// the branches differ in their messages, not after JSON_THROW's expansion
// NOLINTNEXTLINE(bugprone-branch-clone)
case array_index_status::leading_zero:
JSON_THROW(detail::parse_error::create(detail::exception_id::pointer_index_leading_zero, 0, detail::concat("array index '", s, "' must not begin with '0'"), nullptr));
JSON_THROW(detail::parse_error::create(106, 0, detail::concat("array index '", s, "' must not begin with '0'"), nullptr));
case array_index_status::not_a_number:
JSON_THROW(detail::parse_error::create(detail::exception_id::pointer_index_not_a_number, 0, detail::concat("array index '", s, "' is not a number"), nullptr));
JSON_THROW(detail::parse_error::create(109, 0, detail::concat("array index '", s, "' is not a number"), nullptr));
case array_index_status::unresolved:
throw_unresolved(s, nullptr);
JSON_THROW(detail::out_of_range::create(404, detail::concat("unresolved reference token '", s, "'"), nullptr));
case array_index_status::exceeds_size_type:
JSON_THROW(detail::out_of_range::create(detail::exception_id::value_out_of_range, detail::concat("array index ", s, " exceeds size_type"), nullptr));
JSON_THROW(detail::out_of_range::create(410, detail::concat("array index ", s, " exceeds size_type"), nullptr));
case array_index_status::ok:
default:
break;
@@ -364,7 +349,7 @@ class json_pointer
{
if (JSON_HEDLEY_UNLIKELY(empty()))
{
throw_no_parent();
JSON_THROW(detail::out_of_range::create(405, "JSON pointer has no parent", nullptr));
}
json_pointer result = *this;
@@ -452,7 +437,7 @@ class json_pointer
case detail::value_t::binary:
case detail::value_t::discarded:
default:
JSON_THROW(detail::type_error::create(detail::exception_id::unflatten_invalid_value, "invalid value to unflatten", &j));
JSON_THROW(detail::type_error::create(313, "invalid value to unflatten", &j));
}
prefix.push_back(reference_token);
@@ -535,7 +520,7 @@ class json_pointer
case detail::value_t::binary:
case detail::value_t::discarded:
default:
throw_unresolved(reference_token, ptr);
JSON_THROW(detail::out_of_range::create(404, detail::concat("unresolved reference token '", reference_token, "'"), ptr));
}
}
@@ -567,7 +552,7 @@ class json_pointer
if (JSON_HEDLEY_UNLIKELY(reference_token == "-"))
{
// "-" always fails the range check
JSON_THROW(detail::out_of_range::create(detail::exception_id::pointer_past_the_end_index, detail::concat(
JSON_THROW(detail::out_of_range::create(402, detail::concat(
"array index '-' (", std::to_string(ptr->m_data.m_value.array->size()),
") is out of range"), ptr));
}
@@ -576,7 +561,7 @@ class json_pointer
// Bounds check before access to avoid exception with JSON_NOEXCEPTION
if (JSON_HEDLEY_UNLIKELY(idx >= ptr->m_data.m_value.array->size()))
{
JSON_THROW(detail::out_of_range::create(detail::exception_id::array_index_out_of_range, detail::concat(
JSON_THROW(detail::out_of_range::create(401, detail::concat(
"array index ", std::to_string(idx), " is out of range"), ptr));
}
ptr = &ptr->operator[](idx);
@@ -592,7 +577,7 @@ class json_pointer
case detail::value_t::binary:
case detail::value_t::discarded:
default:
throw_unresolved(reference_token, ptr);
JSON_THROW(detail::out_of_range::create(404, detail::concat("unresolved reference token '", reference_token, "'"), ptr));
}
}
@@ -636,7 +621,7 @@ class json_pointer
if (JSON_HEDLEY_UNLIKELY(reference_token == "-"))
{
// "-" cannot be used for const access
JSON_THROW(detail::out_of_range::create(detail::exception_id::pointer_past_the_end_index, detail::concat("array index '-' (", std::to_string(ptr->m_data.m_value.array->size()), ") is out of range"), ptr));
JSON_THROW(detail::out_of_range::create(402, detail::concat("array index '-' (", std::to_string(ptr->m_data.m_value.array->size()), ") is out of range"), ptr));
}
// use unchecked array access; the const operator[]
@@ -654,7 +639,7 @@ class json_pointer
case detail::value_t::binary:
case detail::value_t::discarded:
default:
throw_unresolved(reference_token, ptr);
JSON_THROW(detail::out_of_range::create(404, detail::concat("unresolved reference token '", reference_token, "'"), ptr));
}
}
@@ -709,9 +694,9 @@ class json_pointer
// the branches differ in their messages, not after JSON_THROW's expansion
// NOLINTNEXTLINE(bugprone-branch-clone)
case array_index_status::leading_zero:
JSON_THROW(detail::parse_error::create(detail::exception_id::pointer_index_leading_zero, 0, detail::concat("array index '", reference_token, "' must not begin with '0'"), nullptr));
JSON_THROW(detail::parse_error::create(106, 0, detail::concat("array index '", reference_token, "' must not begin with '0'"), nullptr));
case array_index_status::not_a_number:
JSON_THROW(detail::parse_error::create(detail::exception_id::pointer_index_not_a_number, 0, detail::concat("array index '", reference_token, "' is not a number"), nullptr));
JSON_THROW(detail::parse_error::create(109, 0, detail::concat("array index '", reference_token, "' is not a number"), nullptr));
case array_index_status::unresolved:
case array_index_status::exceeds_size_type:
return nullptr;
@@ -838,7 +823,7 @@ class json_pointer
// check if a nonempty reference string begins with slash
if (JSON_HEDLEY_UNLIKELY(reference_string[0] != '/'))
{
JSON_THROW(detail::parse_error::create(detail::exception_id::pointer_missing_slash, 1, detail::concat("JSON pointer must be empty or begin with '/' - was: '", reference_string, "'"), nullptr));
JSON_THROW(detail::parse_error::create(107, 1, detail::concat("JSON pointer must be empty or begin with '/' - was: '", reference_string, "'"), nullptr));
}
// extract the reference tokens:
@@ -874,7 +859,7 @@ class json_pointer
(reference_token[pos + 1] != '0' &&
reference_token[pos + 1] != '1')))
{
JSON_THROW(detail::parse_error::create(detail::exception_id::pointer_invalid_escape, 0, "escape character '~' must be followed with '0' or '1'", nullptr));
JSON_THROW(detail::parse_error::create(108, 0, "escape character '~' must be followed with '0' or '1'", nullptr));
}
}
@@ -971,7 +956,7 @@ class json_pointer
{
if (JSON_HEDLEY_UNLIKELY(!value.is_object()))
{
JSON_THROW(detail::type_error::create(detail::exception_id::unflatten_not_object, "only objects can be unflattened", &value));
JSON_THROW(detail::type_error::create(314, "only objects can be unflattened", &value));
}
BasicJsonType result;
@@ -999,7 +984,7 @@ class json_pointer
{
if (JSON_HEDLEY_UNLIKELY(!element.second.is_primitive()))
{
JSON_THROW(detail::type_error::create(detail::exception_id::unflatten_value_not_primitive, "values in object must be primitive", &element.second));
JSON_THROW(detail::type_error::create(315, "values in object must be primitive", &element.second));
}
// Assign the value to the reference pointed to by JSON pointer. Note
+2 -2
View File
@@ -312,7 +312,7 @@
return ej_pair.first == e; \
}); \
if (it != std::end(m)) j = it->second; \
else ::nlohmann::detail::templated_json_throw<nlohmann::detail::out_of_range>(nlohmann::detail::out_of_range::create(nlohmann::detail::exception_id::value_out_of_range,"enum value out of range for " #ENUM_TYPE, nullptr)); \
else ::nlohmann::detail::templated_json_throw<nlohmann::detail::out_of_range>(nlohmann::detail::out_of_range::create(410,"enum value out of range for " #ENUM_TYPE, nullptr)); \
} \
template<typename BasicJsonType> \
inline void from_json(const BasicJsonType& j, ENUM_TYPE& e) \
@@ -327,7 +327,7 @@
return ej_pair.second == j; \
}); \
if (it != std::end(m)) e = it->first; \
else ::nlohmann::detail::templated_json_throw<nlohmann::detail::out_of_range>(nlohmann::detail::out_of_range::create(nlohmann::detail::exception_id::value_out_of_range, nlohmann::detail::concat("enum value out of range for " #ENUM_TYPE ": ", j.dump(-1, ' ', false, nlohmann::detail::error_handler_t::replace)), &j)); \
else ::nlohmann::detail::templated_json_throw<nlohmann::detail::out_of_range>(nlohmann::detail::out_of_range::create(410, nlohmann::detail::concat("enum value out of range for " #ENUM_TYPE ": ", j.dump(-1, ' ', false, nlohmann::detail::error_handler_t::replace)), &j)); \
}
// Ugly macros to avoid uglier copy-paste when specializing basic_json. They
@@ -21,6 +21,8 @@
#undef JSON_NO_UNIQUE_ADDRESS
#undef JSON_DISABLE_ENUM_SERIALIZATION
#undef JSON_DISABLE_TUPLE_REFERENCE_CONVERSION
#undef JSON_DTOA_SSE2
#undef JSON_DTOA_NEON
#ifndef JSON_TEST_KEEP_MACROS
#undef JSON_CATCH
@@ -151,7 +151,7 @@ class binary_writer
case value_t::discarded:
default:
{
JSON_THROW(type_error::create(exception_id::type_not_serializable, concat("to serialize to BSON, top-level type must be object, but is ", j.type_name()), &j));
JSON_THROW(type_error::create(317, concat("to serialize to BSON, top-level type must be object, but is ", j.type_name()), &j));
}
}
}
@@ -354,7 +354,7 @@ class binary_writer
{
if (JSON_HEDLEY_UNLIKELY(!value_in_range_of<std::uint32_t>(length)))
{
JSON_THROW(out_of_range::create(exception_id::length_too_large, concat("MessagePack length ", std::to_string(length), " exceeds maximum of ", std::to_string((std::numeric_limits<std::uint32_t>::max)())), &j));
JSON_THROW(out_of_range::create(412, concat("MessagePack length ", std::to_string(length), " exceeds maximum of ", std::to_string((std::numeric_limits<std::uint32_t>::max)())), &j));
}
static_cast<void>(j);
@@ -613,7 +613,7 @@ class binary_writer
{
if (JSON_HEDLEY_UNLIKELY(j.m_data.m_value.binary->subtype() > (std::numeric_limits<std::uint8_t>::max)()))
{
JSON_THROW(out_of_range::create(exception_id::subtype_out_of_range, concat("subtype ", std::to_string(j.m_data.m_value.binary->subtype()), " is too large for the MessagePack ext type (max 255)"), &j));
JSON_THROW(out_of_range::create(415, concat("subtype ", std::to_string(j.m_data.m_value.binary->subtype()), " is too large for the MessagePack ext type (max 255)"), &j));
}
write_number(static_cast<std::int8_t>(j.m_data.m_value.binary->subtype()));
@@ -767,7 +767,7 @@ class binary_writer
{
if (!use_count)
{
JSON_THROW(other_error::create(exception_id::size_marker_required, "use_type requires use_size = true", &j));
JSON_THROW(other_error::create(502, "use_type requires use_size = true", &j));
}
oa.write_character(to_char_type('$'));
oa.write_character(bjdata_draft3 ? 'B' : 'U');
@@ -866,7 +866,7 @@ class binary_writer
{
static_cast<void>(j); // unused when JSON_NOEXCEPTION is defined
static_cast<void>(format_name);
JSON_THROW(type_error::create(exception_id::discarded_value_used, concat("cannot serialize discarded value to ", format_name), &j));
JSON_THROW(type_error::create(321, concat("cannot serialize discarded value to ", format_name), &j));
}
void write_msgpack_array_prefix(const std::size_t N, const BasicJsonType& j)
@@ -1132,7 +1132,7 @@ class binary_writer
{
if (!use_count)
{
JSON_THROW(other_error::create(exception_id::size_marker_required, "use_type requires use_size = true", &j));
JSON_THROW(other_error::create(502, "use_type requires use_size = true", &j));
}
const CharType first_prefix = ubjson_prefix(j.front(), use_bjdata);
const bool same_prefix = std::all_of(j.begin() + 1, j.end(),
@@ -1182,7 +1182,7 @@ class binary_writer
{
if (!use_count)
{
JSON_THROW(other_error::create(exception_id::size_marker_required, "use_type requires use_size = true", &j));
JSON_THROW(other_error::create(502, "use_type requires use_size = true", &j));
}
const CharType first_prefix = ubjson_prefix(j.front(), use_bjdata);
const bool same_prefix = std::all_of(j.begin(), j.end(),
@@ -1397,7 +1397,7 @@ class binary_writer
const auto it = name.find(static_cast<typename string_t::value_type>(0));
if (JSON_HEDLEY_UNLIKELY(it != BasicJsonType::string_t::npos))
{
JSON_THROW(out_of_range::create(exception_id::bson_key_with_null, concat("BSON key cannot contain code point U+0000 (at byte ", std::to_string(it), ")"), &j));
JSON_THROW(out_of_range::create(409, concat("BSON key cannot contain code point U+0000 (at byte ", std::to_string(it), ")"), &j));
}
string_t storage;
@@ -1415,7 +1415,7 @@ class binary_writer
{
if (JSON_HEDLEY_UNLIKELY(!value_in_range_of<std::int32_t>(size)))
{
JSON_THROW(out_of_range::create(exception_id::length_too_large, concat("BSON length ", std::to_string(size), " exceeds maximum of ", std::to_string((std::numeric_limits<std::int32_t>::max)())), nullptr));
JSON_THROW(out_of_range::create(412, concat("BSON length ", std::to_string(size), " exceeds maximum of ", std::to_string((std::numeric_limits<std::int32_t>::max)())), nullptr));
}
return static_cast<std::int32_t>(size);
@@ -1601,7 +1601,7 @@ class binary_writer
if (value.has_subtype() && JSON_HEDLEY_UNLIKELY(value.subtype() > (std::numeric_limits<std::uint8_t>::max)()))
{
JSON_THROW(out_of_range::create(exception_id::subtype_out_of_range, concat("subtype ", std::to_string(value.subtype()), " is too large for the BSON binary subtype (max 255)"), &j));
JSON_THROW(out_of_range::create(415, concat("subtype ", std::to_string(value.subtype()), " is too large for the BSON binary subtype (max 255)"), &j));
}
return sizeof(std::int32_t) + value.size() + 1ul;
@@ -2551,7 +2551,7 @@ class binary_writer
{
if (j.m_data.m_value.number_unsigned > static_cast<typename BasicJsonType::number_unsigned_t>((std::numeric_limits<std::int64_t>::max)()))
{
JSON_THROW(out_of_range::create(exception_id::integer_too_large, concat("integer number ", std::to_string(j.m_data.m_value.number_unsigned), " cannot be represented by BON8 as it does not fit int64"), &j));
JSON_THROW(out_of_range::create(407, concat("integer number ", std::to_string(j.m_data.m_value.number_unsigned), " cannot be represented by BON8 as it does not fit int64"), &j));
}
write_bon8_integer(static_cast<std::int64_t>(j.m_data.m_value.number_unsigned));
string_open = false;
@@ -2704,7 +2704,7 @@ class binary_writer
const std::size_t valid = valid_utf8_prefix(data, s.size());
if (JSON_HEDLEY_UNLIKELY(valid != s.size()))
{
JSON_THROW(type_error::create(exception_id::invalid_utf8, concat("invalid UTF-8 byte at index ", std::to_string(valid), ": 0x", detail::hex_byte(data[valid])), &context));
JSON_THROW(type_error::create(316, concat("invalid UTF-8 byte at index ", std::to_string(valid), ": 0x", detail::hex_byte(data[valid])), &context));
}
}
+47 -18
View File
@@ -888,7 +888,7 @@ class serializer
{
case error_handler_t::strict:
{
JSON_THROW(type_error::create(exception_id::invalid_utf8, concat("invalid UTF-8 byte at index ", std::to_string(i), ": 0x", detail::hex_byte(byte)), nullptr));
JSON_THROW(type_error::create(316, concat("invalid UTF-8 byte at index ", std::to_string(i), ": 0x", detail::hex_byte(byte)), nullptr));
}
case error_handler_t::ignore:
@@ -1021,7 +1021,7 @@ class serializer
{
case error_handler_t::strict:
{
JSON_THROW(type_error::create(exception_id::invalid_utf8, concat("incomplete UTF-8 string; last byte: 0x", detail::hex_byte(static_cast<std::uint8_t>(s[s.size() - 1]))), nullptr));
JSON_THROW(type_error::create(316, concat("incomplete UTF-8 string; last byte: 0x", detail::hex_byte(static_cast<std::uint8_t>(s[s.size() - 1]))), nullptr));
}
case error_handler_t::ignore:
@@ -1366,8 +1366,9 @@ class serializer
/*!
@brief dump an integer
Dump a given integer, appending it to @ref write_buffer. Works internally with
@a number_buffer.
Dump a given integer, appending it to @ref write_buffer (directly: copying
the digits from another buffer right after writing them waits until the
stores are done).
@param[in] x integer number (signed or unsigned) to dump
@tparam NumberType either @a number_integer_t or @a number_unsigned_t
@@ -1402,33 +1403,57 @@ class serializer
return;
}
// use a pointer to fill the buffer
auto buffer_ptr = number_buffer.begin(); // NOLINT(llvm-qualified-auto,readability-qualified-auto)
// use a pointer to fill the buffer (room for as much as number_buffer holds)
if (JSON_HEDLEY_UNLIKELY(write_buffer_pos + number_buffer.size() > write_buffer.size()))
{
flush();
}
auto* buffer_ptr = write_buffer.data() + write_buffer_pos;
number_unsigned_t abs_value;
unsigned int n_chars{};
// one byte for the minus sign
unsigned int n_chars = 0;
if (is_negative_number(x))
{
*buffer_ptr = '-';
abs_value = remove_sign(static_cast<number_integer_t>(x));
// account one more byte for the minus sign
n_chars = 1 + count_digits(abs_value);
n_chars = 1;
}
else
{
abs_value = static_cast<number_unsigned_t>(x);
n_chars = count_digits(abs_value);
}
// up to 16 digits: eight at a time (as the digits of floats), written
// without leading zeros
if (abs_value < 10000000000000000u)
{
const std::uint64_t value = abs_value;
const std::uint64_t upper = value / 100000000u;
const std::uint64_t first = dtoa_impl::eight_digit_bytes(upper != 0 ? upper : value);
const auto leading = static_cast<unsigned>(count_leading_zeros(first) / 8); // (first is not 0)
char* const p = buffer_ptr + n_chars;
dtoa_impl::store_msb_first(p, (first << (8 * leading)) + 0x3030303030303030u);
n_chars += 8 - leading;
if (upper != 0)
{
dtoa_impl::store_msb_first(p + 8 - leading, dtoa_impl::eight_digit_bytes(value - (upper * 100000000u)) + 0x3030303030303030u);
n_chars += 8;
}
write_buffer_pos += n_chars;
return;
}
n_chars += count_digits(abs_value);
// spare 1 byte for '\0'
JSON_ASSERT(n_chars < number_buffer.size() - 1);
// jump to the end to generate the string from backward,
// so we later avoid reversing the result
buffer_ptr += static_cast<typename decltype(number_buffer)::difference_type>(n_chars);
buffer_ptr += n_chars;
// Fast int2ascii implementation inspired by "Fastware" talk by Andrei Alexandrescu
// See: https://www.youtube.com/watch?v=o4-CwDo2zpg
@@ -1451,14 +1476,13 @@ class serializer
*(--buffer_ptr) = static_cast<char>('0' + abs_value);
}
put_buffer(number_buffer, n_chars);
write_buffer_pos += n_chars;
}
/*!
@brief dump a floating-point number
Dump a given floating-point number, appending it to @ref write_buffer. Works internally
with @a number_buffer.
Dump a given floating-point number, appending it to @ref write_buffer.
@param[in] x floating-point number to dump
*/
@@ -1485,10 +1509,15 @@ class serializer
void dump_float(number_float_t x, std::true_type /*is_ieee_single_or_double*/)
{
auto* begin = number_buffer.data();
// directly into the write buffer: copying the text from number_buffer
// right after to_chars() wrote it waits until its stores are done
if (JSON_HEDLEY_UNLIKELY(write_buffer_pos + number_buffer.size() > write_buffer.size()))
{
flush();
}
auto* begin = write_buffer.data() + write_buffer_pos;
auto* end = ::nlohmann::detail::to_chars(begin, begin + number_buffer.size(), x);
put_buffer(number_buffer, static_cast<std::size_t>(end - begin));
write_buffer_pos += static_cast<std::size_t>(end - begin);
}
JSON_HEDLEY_NON_NULL(1)
+71 -80
View File
@@ -639,7 +639,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
object = nullptr; // silence warning, see #821
if (JSON_HEDLEY_UNLIKELY(t == value_t::null))
{
JSON_THROW(other_error::create(detail::exception_id::internal_error, "961c151d2e87f2686a955a9be24d316f1362bf21 3.12.0", nullptr)); // LCOV_EXCL_LINE
JSON_THROW(other_error::create(500, "961c151d2e87f2686a955a9be24d316f1362bf21 3.12.0", nullptr)); // LCOV_EXCL_LINE
}
break;
}
@@ -2287,7 +2287,7 @@ public:
// if an object is wanted but impossible, throw an exception
if (JSON_HEDLEY_UNLIKELY(manual_type == value_t::object && !is_an_object))
{
JSON_THROW(type_error::create(detail::exception_id::object_from_non_pairs, "cannot create object from initializer list", nullptr));
JSON_THROW(type_error::create(301, "cannot create object from initializer list", nullptr));
}
}
@@ -2407,7 +2407,7 @@ public:
// make sure the iterator fits the current value
if (JSON_HEDLEY_UNLIKELY(first.m_object != last.m_object))
{
JSON_THROW(invalid_iterator::create(detail::exception_id::iterators_incompatible, "iterators are not compatible", nullptr));
JSON_THROW(invalid_iterator::create(201, "iterators are not compatible", nullptr));
}
// copy type from the first iterator
@@ -2426,7 +2426,7 @@ public:
if (JSON_HEDLEY_UNLIKELY(!first.m_it.primitive_iterator.is_begin()
|| !last.m_it.primitive_iterator.is_end()))
{
JSON_THROW(invalid_iterator::create(detail::exception_id::iterator_range_out_of_range, "iterators out of range", first.m_object));
JSON_THROW(invalid_iterator::create(204, "iterators out of range", first.m_object));
}
break;
}
@@ -2494,7 +2494,7 @@ public:
case value_t::null:
case value_t::discarded:
default:
JSON_THROW(invalid_iterator::create(detail::exception_id::iterator_range_of_null, detail::concat("cannot construct with iterators from ", first.m_object->type_name()), first.m_object));
JSON_THROW(invalid_iterator::create(206, detail::concat("cannot construct with iterators from ", first.m_object->type_name()), first.m_object));
}
set_parents();
@@ -2882,7 +2882,7 @@ public:
return *ptr;
}
JSON_THROW(type_error::create(detail::exception_id::incompatible_reference_type, detail::concat("incompatible ReferenceType for get_ref, actual type is ", obj.type_name()), &obj));
JSON_THROW(type_error::create(303, detail::concat("incompatible ReferenceType for get_ref, actual type is ", obj.type_name()), &obj));
}
public:
@@ -3266,7 +3266,7 @@ public:
{
if (!is_binary())
{
detail::throw_type_must_be("binary", *this);
JSON_THROW(type_error::create(302, detail::concat("type must be binary, but is ", type_name()), this));
}
return *get_ptr<binary_t*>();
@@ -3278,7 +3278,7 @@ public:
{
if (!is_binary())
{
detail::throw_type_must_be("binary", *this);
JSON_THROW(type_error::create(302, detail::concat("type must be binary, but is ", type_name()), this));
}
return *get_ptr<const binary_t*>();
@@ -3320,7 +3320,7 @@ public:
// at only works for objects
if (JSON_HEDLEY_UNLIKELY(!j.is_object()))
{
detail::throw_cannot_use_with(detail::exception_id::at_wrong_type, "at()", j);
JSON_THROW(type_error::create(304, detail::concat("cannot use at() with ", j.type_name()), &j));
}
auto it = object_lookup(j, std::forward<KeyType>(key));
@@ -3330,7 +3330,7 @@ public:
// std::map or ordered_map) never moves from its argument, so key is still
// valid here regardless of whether KeyType was deduced as an rvalue reference
// NOLINTNEXTLINE(bugprone-use-after-move,hicpp-invalid-access-moved)
JSON_THROW(out_of_range::create(detail::exception_id::key_not_found, detail::concat("key '", string_t(key), "' not found"), &j));
JSON_THROW(out_of_range::create(403, detail::concat("key '", string_t(key), "' not found"), &j));
}
return it->second;
}
@@ -3345,12 +3345,12 @@ public:
// at only works for arrays
if (JSON_HEDLEY_UNLIKELY(!j.is_array()))
{
detail::throw_cannot_use_with(detail::exception_id::at_wrong_type, "at()", j);
JSON_THROW(type_error::create(304, detail::concat("cannot use at() with ", j.type_name()), &j));
}
if (JSON_HEDLEY_UNLIKELY(idx >= j.m_data.m_value.array->size()))
{
JSON_THROW(out_of_range::create(detail::exception_id::array_index_out_of_range, detail::concat("array index ", std::to_string(idx), " is out of range"), &j));
JSON_THROW(out_of_range::create(401, detail::concat("array index ", std::to_string(idx), " is out of range"), &j));
}
return (*j.m_data.m_value.array)[idx];
@@ -3383,15 +3383,6 @@ public:
m_data.m_type = value_t::object;
}
/// @brief throws because operator[] is not supported for the type of this value
/// @param[in] argument the kind of the operator's argument, "numeric" or "string"
/// @throw type_error.305 always
JSON_HEDLEY_NO_RETURN void throw_subscript_wrong_type(const char* argument) const
{
detail::throw_cannot_use_with(detail::exception_id::subscript_wrong_type,
detail::concat("operator[] with a ", argument, " argument").c_str(), *this);
}
public:
////////////////////
// element access //
@@ -3496,7 +3487,7 @@ public:
return m_data.m_value.array->operator[](idx);
}
throw_subscript_wrong_type("numeric");
JSON_THROW(type_error::create(305, detail::concat("cannot use operator[] with a numeric argument with ", type_name()), this));
}
/// @brief access specified array element
@@ -3510,7 +3501,7 @@ public:
return m_data.m_value.array->operator[](idx);
}
throw_subscript_wrong_type("numeric");
JSON_THROW(type_error::create(305, detail::concat("cannot use operator[] with a numeric argument with ", type_name()), this));
}
/// @brief access specified object element
@@ -3530,7 +3521,7 @@ public:
return set_parent(result.first->second);
}
throw_subscript_wrong_type("string");
JSON_THROW(type_error::create(305, detail::concat("cannot use operator[] with a string argument with ", type_name()), this));
}
/// @brief access specified object element
@@ -3545,7 +3536,7 @@ public:
return it->second;
}
throw_subscript_wrong_type("string");
JSON_THROW(type_error::create(305, detail::concat("cannot use operator[] with a string argument with ", type_name()), this));
}
// these two functions resolve a (const) char * ambiguity affecting Clang and MSVC
@@ -3581,7 +3572,7 @@ public:
return set_parent(result.first->second);
}
throw_subscript_wrong_type("string");
JSON_THROW(type_error::create(305, detail::concat("cannot use operator[] with a string argument with ", type_name()), this));
}
/// @brief access specified object element
@@ -3598,7 +3589,7 @@ public:
return it->second;
}
throw_subscript_wrong_type("string");
JSON_THROW(type_error::create(305, detail::concat("cannot use operator[] with a string argument with ", type_name()), this));
}
private:
@@ -3622,7 +3613,7 @@ public:
// value only works for objects
if (JSON_HEDLEY_UNLIKELY(!is_object()))
{
detail::throw_cannot_use_with(detail::exception_id::value_wrong_type, "value()", *this);
JSON_THROW(type_error::create(306, detail::concat("cannot use value() with ", type_name()), this));
}
const auto it = find(std::forward<KeyType>(key));
@@ -3637,7 +3628,7 @@ public:
// value only works for arrays and objects
if (JSON_HEDLEY_UNLIKELY(!is_structured()))
{
detail::throw_cannot_use_with(detail::exception_id::value_wrong_type, "value()", *this);
JSON_THROW(type_error::create(306, detail::concat("cannot use value() with ", type_name()), this));
}
return ptr.get_checked_or_null(this);
@@ -3804,7 +3795,7 @@ public:
// make sure the iterator fits the current value
if (JSON_HEDLEY_UNLIKELY(this != pos.m_object))
{
JSON_THROW(invalid_iterator::create(detail::exception_id::iterator_from_other_value, "iterator does not fit current value", this));
JSON_THROW(invalid_iterator::create(202, "iterator does not fit current value", this));
}
IteratorType result = end();
@@ -3820,7 +3811,7 @@ public:
{
if (JSON_HEDLEY_UNLIKELY(!pos.m_it.primitive_iterator.is_begin()))
{
JSON_THROW(invalid_iterator::create(detail::exception_id::iterator_out_of_range, "iterator out of range", this));
JSON_THROW(invalid_iterator::create(205, "iterator out of range", this));
}
m_data.m_value.destroy(m_data.m_type);
@@ -3846,7 +3837,7 @@ public:
case value_t::null:
case value_t::discarded:
default:
detail::throw_cannot_use_with(detail::exception_id::erase_wrong_type, "erase()", *this);
JSON_THROW(type_error::create(307, detail::concat("cannot use erase() with ", type_name()), this));
}
return result;
@@ -3862,7 +3853,7 @@ public:
// make sure the iterator fits the current value
if (JSON_HEDLEY_UNLIKELY(this != first.m_object || this != last.m_object))
{
JSON_THROW(invalid_iterator::create(detail::exception_id::iterator_range_from_other_value, "iterators do not fit current value", this));
JSON_THROW(invalid_iterator::create(203, "iterators do not fit current value", this));
}
IteratorType result = end();
@@ -3879,7 +3870,7 @@ public:
if (JSON_HEDLEY_UNLIKELY(!first.m_it.primitive_iterator.is_begin()
|| !last.m_it.primitive_iterator.is_end()))
{
JSON_THROW(invalid_iterator::create(detail::exception_id::iterator_range_out_of_range, "iterators out of range", this));
JSON_THROW(invalid_iterator::create(204, "iterators out of range", this));
}
m_data.m_value.destroy(m_data.m_type);
@@ -3907,7 +3898,7 @@ public:
case value_t::null:
case value_t::discarded:
default:
detail::throw_cannot_use_with(detail::exception_id::erase_wrong_type, "erase()", *this);
JSON_THROW(type_error::create(307, detail::concat("cannot use erase() with ", type_name()), this));
}
return result;
@@ -3921,7 +3912,7 @@ public:
// this erase only works for objects
if (JSON_HEDLEY_UNLIKELY(!is_object()))
{
detail::throw_cannot_use_with(detail::exception_id::erase_wrong_type, "erase()", *this);
JSON_THROW(type_error::create(307, detail::concat("cannot use erase() with ", type_name()), this));
}
const auto erased = m_data.m_value.object->erase(std::forward<KeyType>(key));
@@ -3936,7 +3927,7 @@ public:
// this erase only works for objects
if (JSON_HEDLEY_UNLIKELY(!is_object()))
{
detail::throw_cannot_use_with(detail::exception_id::erase_wrong_type, "erase()", *this);
JSON_THROW(type_error::create(307, detail::concat("cannot use erase() with ", type_name()), this));
}
const auto it = object_lookup(*this, std::forward<KeyType>(key));
@@ -3978,14 +3969,14 @@ public:
{
if (JSON_HEDLEY_UNLIKELY(idx >= size()))
{
JSON_THROW(out_of_range::create(detail::exception_id::array_index_out_of_range, detail::concat("array index ", std::to_string(idx), " is out of range"), this));
JSON_THROW(out_of_range::create(401, detail::concat("array index ", std::to_string(idx), " is out of range"), this));
}
m_data.m_value.array->erase(m_data.m_value.array->begin() + static_cast<difference_type>(idx));
}
else
{
detail::throw_cannot_use_with(detail::exception_id::erase_wrong_type, "erase()", *this);
JSON_THROW(type_error::create(307, detail::concat("cannot use erase() with ", type_name()), this));
}
}
@@ -4474,7 +4465,7 @@ public:
// push_back only works for null objects or arrays
if (JSON_HEDLEY_UNLIKELY(!(is_null() || is_array())))
{
detail::throw_cannot_use_with(detail::exception_id::push_back_wrong_type, "push_back()", *this);
JSON_THROW(type_error::create(308, detail::concat("cannot use push_back() with ", type_name()), this));
}
// transform a null object into an array
@@ -4505,7 +4496,7 @@ public:
// push_back only works for null objects or arrays
if (JSON_HEDLEY_UNLIKELY(!(is_null() || is_array())))
{
detail::throw_cannot_use_with(detail::exception_id::push_back_wrong_type, "push_back()", *this);
JSON_THROW(type_error::create(308, detail::concat("cannot use push_back() with ", type_name()), this));
}
// transform a null object into an array
@@ -4535,7 +4526,7 @@ public:
// push_back only works for null objects or objects
if (JSON_HEDLEY_UNLIKELY(!(is_null() || is_object())))
{
detail::throw_cannot_use_with(detail::exception_id::push_back_wrong_type, "push_back()", *this);
JSON_THROW(type_error::create(308, detail::concat("cannot use push_back() with ", type_name()), this));
}
// transform a null object into an object
@@ -4589,7 +4580,7 @@ public:
// emplace_back only works for null objects or arrays
if (JSON_HEDLEY_UNLIKELY(!(is_null() || is_array())))
{
detail::throw_cannot_use_with(detail::exception_id::emplace_wrong_type, "emplace_back()", *this);
JSON_THROW(type_error::create(311, detail::concat("cannot use emplace_back() with ", type_name()), this));
}
// transform a null object into an array
@@ -4612,7 +4603,7 @@ public:
// emplace only works for null objects or arrays
if (JSON_HEDLEY_UNLIKELY(!(is_null() || is_object())))
{
detail::throw_cannot_use_with(detail::exception_id::emplace_wrong_type, "emplace()", *this);
JSON_THROW(type_error::create(311, detail::concat("cannot use emplace() with ", type_name()), this));
}
// transform a null object into an object
@@ -4664,14 +4655,14 @@ public:
// check if iterator pos fits to this JSON value
if (JSON_HEDLEY_UNLIKELY(pos.m_object != this))
{
JSON_THROW(invalid_iterator::create(detail::exception_id::iterator_from_other_value, "iterator does not fit current value", this));
JSON_THROW(invalid_iterator::create(202, "iterator does not fit current value", this));
}
// insert to array and return iterator
return insert_iterator(pos, val);
}
detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
JSON_THROW(type_error::create(309, detail::concat("cannot use insert() with ", type_name()), this));
}
/// @brief inserts element into array
@@ -4684,7 +4675,7 @@ public:
// check if iterator pos fits to this JSON value
if (JSON_HEDLEY_UNLIKELY(pos.m_object != this))
{
JSON_THROW(invalid_iterator::create(detail::exception_id::iterator_from_other_value, "iterator does not fit current value", this));
JSON_THROW(invalid_iterator::create(202, "iterator does not fit current value", this));
}
// moving into a local first keeps this safe even if val aliases
@@ -4693,7 +4684,7 @@ public:
return insert_iterator(pos, std::move(tmp));
}
detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
JSON_THROW(type_error::create(309, detail::concat("cannot use insert() with ", type_name()), this));
}
/// @brief inserts copies of element into array
@@ -4706,14 +4697,14 @@ public:
// check if iterator pos fits to this JSON value
if (JSON_HEDLEY_UNLIKELY(pos.m_object != this))
{
JSON_THROW(invalid_iterator::create(detail::exception_id::iterator_from_other_value, "iterator does not fit current value", this));
JSON_THROW(invalid_iterator::create(202, "iterator does not fit current value", this));
}
// insert to array and return iterator
return insert_iterator(pos, cnt, val);
}
detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
JSON_THROW(type_error::create(309, detail::concat("cannot use insert() with ", type_name()), this));
}
/// @brief inserts range of elements into array
@@ -4723,30 +4714,30 @@ public:
// insert only works for arrays
if (JSON_HEDLEY_UNLIKELY(!is_array()))
{
detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
JSON_THROW(type_error::create(309, detail::concat("cannot use insert() with ", type_name()), this));
}
// check if iterator pos fits to this JSON value
if (JSON_HEDLEY_UNLIKELY(pos.m_object != this))
{
JSON_THROW(invalid_iterator::create(detail::exception_id::iterator_from_other_value, "iterator does not fit current value", this));
JSON_THROW(invalid_iterator::create(202, "iterator does not fit current value", this));
}
// check if range iterators belong to the same JSON object
if (JSON_HEDLEY_UNLIKELY(first.m_object != last.m_object))
{
JSON_THROW(invalid_iterator::create(detail::exception_id::insert_range_incompatible, "iterators do not fit", this));
JSON_THROW(invalid_iterator::create(210, "iterators do not fit", this));
}
if (JSON_HEDLEY_UNLIKELY(first.m_object == this))
{
JSON_THROW(invalid_iterator::create(detail::exception_id::insert_range_into_itself, "passed iterators may not belong to container", this));
JSON_THROW(invalid_iterator::create(211, "passed iterators may not belong to container", this));
}
// passed iterators must belong to arrays
if (JSON_HEDLEY_UNLIKELY(!first.m_object->is_array()))
{
JSON_THROW(invalid_iterator::create(detail::exception_id::iterator_from_other_value, "iterators first and last must point to arrays", this));
JSON_THROW(invalid_iterator::create(202, "iterators first and last must point to arrays", this));
}
// insert to array and return iterator
@@ -4760,13 +4751,13 @@ public:
// insert only works for arrays
if (JSON_HEDLEY_UNLIKELY(!is_array()))
{
detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
JSON_THROW(type_error::create(309, detail::concat("cannot use insert() with ", type_name()), this));
}
// check if iterator pos fits to this JSON value
if (JSON_HEDLEY_UNLIKELY(pos.m_object != this))
{
JSON_THROW(invalid_iterator::create(detail::exception_id::iterator_from_other_value, "iterator does not fit current value", this));
JSON_THROW(invalid_iterator::create(202, "iterator does not fit current value", this));
}
// copy the values first: ilist may refer to elements of this array
@@ -4788,19 +4779,19 @@ public:
// insert only works for objects
if (JSON_HEDLEY_UNLIKELY(!is_object()))
{
detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
JSON_THROW(type_error::create(309, detail::concat("cannot use insert() with ", type_name()), this));
}
// check if range iterators belong to the same JSON object
if (JSON_HEDLEY_UNLIKELY(first.m_object != last.m_object))
{
JSON_THROW(invalid_iterator::create(detail::exception_id::insert_range_incompatible, "iterators do not fit", this));
JSON_THROW(invalid_iterator::create(210, "iterators do not fit", this));
}
// passed iterators must belong to objects
if (JSON_HEDLEY_UNLIKELY(!first.m_object->is_object()))
{
JSON_THROW(invalid_iterator::create(detail::exception_id::iterator_from_other_value, "iterators first and last must point to objects", this));
JSON_THROW(invalid_iterator::create(202, "iterators first and last must point to objects", this));
}
m_data.m_value.object->insert(first.m_it.object_iterator, last.m_it.object_iterator);
@@ -4817,7 +4808,7 @@ public:
// j, not the copy made below)
if (JSON_HEDLEY_UNLIKELY(!j.is_object()))
{
detail::throw_cannot_use_with(detail::exception_id::update_wrong_type, "update()", j);
JSON_THROW(type_error::create(312, detail::concat("cannot use update() with ", j.type_name()), &j));
}
// copy first: j may be *this or one of its descendants, and is
@@ -4835,13 +4826,13 @@ public:
// check if range iterators belong to the same JSON object
if (JSON_HEDLEY_UNLIKELY(first.m_object != last.m_object))
{
JSON_THROW(invalid_iterator::create(detail::exception_id::insert_range_incompatible, "iterators do not fit", this));
JSON_THROW(invalid_iterator::create(210, "iterators do not fit", this));
}
// passed iterators must belong to objects
if (JSON_HEDLEY_UNLIKELY(!first.m_object->is_object()))
{
detail::throw_cannot_use_with(detail::exception_id::update_wrong_type, "update()", *first.m_object);
JSON_THROW(type_error::create(312, detail::concat("cannot use update() with ", first.m_object->type_name()), first.m_object));
}
// copy first: the range may belong to *this or one of its
@@ -4877,7 +4868,7 @@ public:
if (JSON_HEDLEY_UNLIKELY(!is_object()))
{
detail::throw_cannot_use_with(detail::exception_id::update_wrong_type, "update()", *this);
JSON_THROW(type_error::create(312, detail::concat("cannot use update() with ", type_name()), this));
}
}
@@ -5040,7 +5031,7 @@ public:
}
else
{
detail::throw_cannot_use_with(detail::exception_id::swap_wrong_type, "swap(array_t&)", *this);
JSON_THROW(type_error::create(310, detail::concat("cannot use swap(array_t&) with ", type_name()), this));
}
}
@@ -5057,7 +5048,7 @@ public:
}
else
{
detail::throw_cannot_use_with(detail::exception_id::swap_wrong_type, "swap(object_t&)", *this);
JSON_THROW(type_error::create(310, detail::concat("cannot use swap(object_t&) with ", type_name()), this));
}
}
@@ -5073,7 +5064,7 @@ public:
}
else
{
detail::throw_cannot_use_with(detail::exception_id::swap_wrong_type, "swap(string_t&)", *this);
JSON_THROW(type_error::create(310, detail::concat("cannot use swap(string_t&) with ", type_name()), this));
}
}
@@ -5089,7 +5080,7 @@ public:
}
else
{
detail::throw_cannot_use_with(detail::exception_id::swap_wrong_type, "swap(binary_t&)", *this);
JSON_THROW(type_error::create(310, detail::concat("cannot use swap(binary_t&) with ", type_name()), this));
}
}
@@ -5105,7 +5096,7 @@ public:
}
else
{
detail::throw_cannot_use_with(detail::exception_id::swap_wrong_type, "swap(binary_t::container_type&)", *this);
JSON_THROW(type_error::create(310, detail::concat("cannot use swap(binary_t::container_type&) with ", type_name()), this));
}
}
@@ -6567,7 +6558,7 @@ public:
if (JSON_HEDLEY_UNLIKELY(idx > parent.size()))
{
// avoid undefined behavior
JSON_THROW(out_of_range::create(detail::exception_id::array_index_out_of_range, detail::concat("array index ", std::to_string(idx), " is out of range"), &parent));
JSON_THROW(out_of_range::create(401, detail::concat("array index ", std::to_string(idx), " is out of range"), &parent));
}
// default case: insert add offset
@@ -6586,7 +6577,7 @@ public:
case value_t::binary:
case value_t::discarded:
default:
JSON_THROW(out_of_range::create(detail::exception_id::patch_add_parent_not_container, detail::concat("cannot add value: the JSON Patch 'add' target's parent is of type ", parent.type_name(), ", but must be an object or array"), &parent));
JSON_THROW(out_of_range::create(411, detail::concat("cannot add value: the JSON Patch 'add' target's parent is of type ", parent.type_name(), ", but must be an object or array"), &parent));
}
};
@@ -6609,7 +6600,7 @@ public:
}
else
{
JSON_THROW(out_of_range::create(detail::exception_id::key_not_found, detail::concat("key '", last_path, "' not found"), this));
JSON_THROW(out_of_range::create(403, detail::concat("key '", last_path, "' not found"), this));
}
}
else if (parent.is_array())
@@ -6621,7 +6612,7 @@ public:
{
// the parent of a "remove" target must be an object or array
// (see #5396)
JSON_THROW(out_of_range::create(detail::exception_id::patch_remove_parent_not_container, detail::concat("cannot remove value: the JSON Patch 'remove' target's parent is of type ", parent.type_name(), ", but must be an object or array"), &parent));
JSON_THROW(out_of_range::create(413, detail::concat("cannot remove value: the JSON Patch 'remove' target's parent is of type ", parent.type_name(), ", but must be an object or array"), &parent));
}
};
@@ -6652,7 +6643,7 @@ public:
// type check: top level value must be an array
if (JSON_HEDLEY_UNLIKELY(!json_patch.is_array()))
{
JSON_THROW(parse_error::create(detail::exception_id::patch_not_an_array, 0, "JSON patch must be an array of objects", &json_patch));
JSON_THROW(parse_error::create(104, 0, "JSON patch must be an array of objects", &json_patch));
}
// iterate and apply the operations
@@ -6673,14 +6664,14 @@ public:
if (JSON_HEDLEY_UNLIKELY(it == val.m_data.m_value.object->end()))
{
// NOLINTNEXTLINE(performance-inefficient-string-concatenation)
JSON_THROW(parse_error::create(detail::exception_id::patch_invalid_operation, 0, detail::concat(error_msg, " must have member '", member, "'"), &val));
JSON_THROW(parse_error::create(105, 0, detail::concat(error_msg, " must have member '", member, "'"), &val));
}
// check if the result is of type string
if (JSON_HEDLEY_UNLIKELY(string_type && !it->second.is_string()))
{
// NOLINTNEXTLINE(performance-inefficient-string-concatenation)
JSON_THROW(parse_error::create(detail::exception_id::patch_invalid_operation, 0, detail::concat(error_msg, " must have string member '", member, "'"), &val));
JSON_THROW(parse_error::create(105, 0, detail::concat(error_msg, " must have string member '", member, "'"), &val));
}
// no error: return value
@@ -6690,7 +6681,7 @@ public:
// type check: every element of the array must be an object
if (JSON_HEDLEY_UNLIKELY(!val.is_object()))
{
JSON_THROW(parse_error::create(detail::exception_id::patch_not_an_array, 0, "JSON patch must be an array of objects", &val));
JSON_THROW(parse_error::create(104, 0, "JSON patch must be an array of objects", &val));
}
// collect mandatory members
@@ -6726,7 +6717,7 @@ public:
if (JSON_HEDLEY_UNLIKELY(is_proper_prefix(from_ptr, ptr)))
{
JSON_THROW(out_of_range::create(detail::exception_id::patch_move_into_child, detail::concat("cannot move value: 'from' path '", from_path, "' is a proper prefix of 'path' '", path, "'"), &result));
JSON_THROW(out_of_range::create(414, detail::concat("cannot move value: 'from' path '", from_path, "' is a proper prefix of 'path' '", path, "'"), &result));
}
// the "from" location must exist - use at()
@@ -6773,7 +6764,7 @@ public:
// throw an exception if the test fails
if (JSON_HEDLEY_UNLIKELY(!success))
{
JSON_THROW(other_error::create(detail::exception_id::patch_test_failed, detail::concat("unsuccessful: ", val.dump()), &val));
JSON_THROW(other_error::create(501, detail::concat("unsuccessful: ", val.dump()), &val));
}
break;
@@ -6784,7 +6775,7 @@ public:
{
// op must be "add", "remove", "replace", "move", "copy", or
// "test"
JSON_THROW(parse_error::create(detail::exception_id::patch_invalid_operation, 0, detail::concat("operation value '", op, "' is invalid"), &val));
JSON_THROW(parse_error::create(105, 0, detail::concat("operation value '", op, "' is invalid"), &val));
}
}
}
+24 -17
View File
@@ -90,19 +90,6 @@ private:
return self.end();
}
/// @brief shared implementation of the const and non-const at() overloads
/// @throw std::out_of_range if @a key is not found
template<typename Self, typename KeyType>
static auto at_impl(Self& self, const KeyType& key) -> decltype((self.begin()->second))
{
const auto it = find_impl(self, key);
if (it == self.end())
{
JSON_THROW(std::out_of_range("key not found"));
}
return it->second;
}
/// @brief remove the entry @a it points to, preserving order
/// @note keys are not movable, so the tail is destroyed and re-constructed in place
void erase_at(iterator it)
@@ -169,26 +156,46 @@ public:
T& at(const key_type& key)
{
return at_impl(*this, key);
const auto it = find_impl(*this, key);
if (it == this->end())
{
JSON_THROW(std::out_of_range("key not found"));
}
return it->second;
}
template<class KeyType, detail::enable_if_t<
detail::is_usable_as_key_type<key_compare, key_type, KeyType>::value, int> = 0>
T & at(KeyType && key) // NOLINT(cppcoreguidelines-missing-std-forward)
{
return at_impl(*this, key);
const auto it = find_impl(*this, key);
if (it == this->end())
{
JSON_THROW(std::out_of_range("key not found"));
}
return it->second;
}
const T& at(const key_type& key) const
{
return at_impl(*this, key);
const auto it = find_impl(*this, key);
if (it == this->end())
{
JSON_THROW(std::out_of_range("key not found"));
}
return it->second;
}
template<class KeyType, detail::enable_if_t<
detail::is_usable_as_key_type<key_compare, key_type, KeyType>::value, int> = 0>
const T & at(KeyType && key) const // NOLINT(cppcoreguidelines-missing-std-forward)
{
return at_impl(*this, key);
const auto it = find_impl(*this, key);
if (it == this->end())
{
JSON_THROW(std::out_of_range("key not found"));
}
return it->second;
}
size_type erase(const key_type& key)
File diff suppressed because it is too large. Load diff
+599
View File
@@ -0,0 +1,599 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#pragma once
#include <array> // array
#include <cstdint> // uint32_t, uint64_t
// Number tokens that are hard to round correctly, with the IEEE-754 binary64
// and binary32 bits of their correctly rounded values (ties to even; infinity
// for an overflow, a signed zero for an underflow).
//
// For doubles and floats around 0, the smallest normal number, 1, 2^24, 2^53,
// 0.1, and the largest finite number, and for random ones, the exact midpoint
// m to the next number gives: m, m with one unit more and less in the last
// digit, m with "01" and "0...01" appended, m with trailing zeros, and m cut
// after 17 to 30 digits (rounded down and up, so that the rounding is decided
// after the 19th digit), in fixed and exponent notation, 30% of them negative.
// Tokens longer than 80 characters are left out, except for four of 700 digits
// and more. Zeros, underflow, overflow, huge exponents, and integers beyond 64
// bits complete the set. Of the 508 tokens, 134 (as double) and 150 (as
// float) need the exact comparison with the midpoint (detail::digit_comparison()).
//
// The expected bits were computed with exact rational arithmetic in Python
// (fractions.Fraction) and cross-checked with Python's float(); strtod_l and
// strtof_l of Apple's libc and of glibc agree. Generated by
// compact_hard_cases.py 5 (with hard_cases.py), see the pull request that
// added this file.
namespace float_hard_cases
{
struct hard_case
{
const char* token;
std::uint64_t bits64;
std::uint32_t bits32;
};
inline const std::array<hard_case, 508>& cases()
{
static const std::array<hard_case, 508> table =
{
{
{"-2.4703282292062327e-324", 0x8000000000000000u, 0x80000000u},
{"24703282292062328e-340", 0x0000000000000001u, 0x00000000u},
{"247032822920623272e-341", 0x0000000000000000u, 0x00000000u},
{"-0.2470328229206232721e-323", 0x8000000000000001u, 0x80000000u},
{"-0.24703282292062327208e-323", 0x8000000000000000u, 0x80000000u},
{"-2.4703282292062327209e-324", 0x8000000000000001u, 0x80000000u},
{"2.47032822920623272088e-324", 0x0000000000000000u, 0x00000000u},
{"247032822920623272089e-344", 0x0000000000000001u, 0x00000000u},
{"-247032822920623272088284396434e-353", 0x8000000000000000u, 0x80000000u},
{"0.247032822920623272088284396435e-323", 0x0000000000000001u, 0x00000000u},
{"-74109846876186981e-340", 0x8000000000000001u, 0x80000000u},
{"0.74109846876186982e-323", 0x0000000000000002u, 0x00000000u},
{"-0.7410984687618698162e-323", 0x8000000000000001u, 0x80000000u},
{"-7.410984687618698163e-324", 0x8000000000000002u, 0x80000000u},
{"7.4109846876186981626e-324", 0x0000000000000001u, 0x00000000u},
{"-74109846876186981627e-343", 0x8000000000000002u, 0x80000000u},
{"-741098468761869816264e-344", 0x8000000000000001u, 0x80000000u},
{"0.741098468761869816265e-323", 0x0000000000000002u, 0x00000000u},
{"0.741098468761869816264853189302e-323", 0x0000000000000001u, 0x00000000u},
{"-7.41098468761869816264853189303e-324", 0x8000000000000002u, 0x80000000u},
{"0.22250738585072006e-307", 0x000FFFFFFFFFFFFEu, 0x00000000u},
{"2.2250738585072007e-308", 0x000FFFFFFFFFFFFFu, 0x00000000u},
{"2.225073858507200641e-308", 0x000FFFFFFFFFFFFEu, 0x00000000u},
{"-2225073858507200642e-326", 0x800FFFFFFFFFFFFFu, 0x80000000u},
{"22250738585072006419e-327", 0x000FFFFFFFFFFFFEu, 0x00000000u},
{"0.2225073858507200642e-307", 0x000FFFFFFFFFFFFFu, 0x00000000u},
{"0.222507385850720064199e-307", 0x000FFFFFFFFFFFFEu, 0x00000000u},
{"2.225073858507200642e-308", 0x000FFFFFFFFFFFFFu, 0x00000000u},
{"-2.22507385850720064199176395546e-308", 0x800FFFFFFFFFFFFEu, 0x80000000u},
{"222507385850720064199176395547e-337", 0x000FFFFFFFFFFFFFu, 0x00000000u},
{"-2.2250738585072011e-308", 0x800FFFFFFFFFFFFFu, 0x80000000u},
{"-22250738585072012e-324", 0x8010000000000000u, 0x80000000u},
{"-2225073858507201136e-326", 0x800FFFFFFFFFFFFFu, 0x80000000u},
{"0.2225073858507201137e-307", 0x0010000000000000u, 0x00000000u},
{"0.2225073858507201136e-307", 0x000FFFFFFFFFFFFFu, 0x00000000u},
{"-2.2250738585072011361e-308", 0x8010000000000000u, 0x80000000u},
{"2.22507385850720113605e-308", 0x000FFFFFFFFFFFFFu, 0x00000000u},
{"222507385850720113606e-328", 0x0010000000000000u, 0x00000000u},
{"22250738585072011360574097967e-336", 0x000FFFFFFFFFFFFFu, 0x00000000u},
{"0.222507385850720113605740979671e-307", 0x0010000000000000u, 0x00000000u},
{"22250738585072016e-324", 0x0010000000000000u, 0x00000000u},
{"0.22250738585072017e-307", 0x0010000000000001u, 0x00000000u},
{"0.222507385850720163e-307", 0x0010000000000000u, 0x00000000u},
{"2.225073858507201631e-308", 0x0010000000000001u, 0x00000000u},
{"-2.2250738585072016301e-308", 0x8010000000000000u, 0x80000000u},
{"22250738585072016302e-327", 0x0010000000000001u, 0x00000000u},
{"-222507385850720163012e-328", 0x8010000000000000u, 0x80000000u},
{"0.222507385850720163013e-307", 0x0010000000000001u, 0x00000000u},
{"0.222507385850720163012305563795e-307", 0x0010000000000000u, 0x00000000u},
{"-2.22507385850720163012305563796e-308", 0x8010000000000001u, 0x80000000u},
{"0.17976931348623156E+309", 0x7FEFFFFFFFFFFFFEu, 0x7F800000u},
{"1.7976931348623157e308", 0x7FEFFFFFFFFFFFFFu, 0x7F800000u},
{"1.797693134862315608e308", 0x7FEFFFFFFFFFFFFEu, 0x7F800000u},
{"-1797693134862315609e290", 0xFFEFFFFFFFFFFFFFu, 0xFF800000u},
{"-17976931348623156083e289", 0xFFEFFFFFFFFFFFFEu, 0xFF800000u},
{"-0.17976931348623156084E+309", 0xFFEFFFFFFFFFFFFFu, 0xFF800000u},
{"0.179769313486231560835E+309", 0x7FEFFFFFFFFFFFFEu, 0x7F800000u},
{"-1.79769313486231560836e308", 0xFFEFFFFFFFFFFFFFu, 0xFF800000u},
{"1.79769313486231560835325876058e308", 0x7FEFFFFFFFFFFFFEu, 0x7F800000u},
{"179769313486231560835325876059e279", 0x7FEFFFFFFFFFFFFFu, 0x7F800000u},
{"1.7976931348623158e308", 0x7FEFFFFFFFFFFFFFu, 0x7F800000u},
{"17976931348623159e292", 0x7FF0000000000000u, 0x7F800000u},
{"1797693134862315807e290", 0x7FEFFFFFFFFFFFFFu, 0x7F800000u},
{"0.1797693134862315808E+309", 0x7FF0000000000000u, 0x7F800000u},
{"0.17976931348623158079E+309", 0x7FEFFFFFFFFFFFFFu, 0x7F800000u},
{"-1.797693134862315808e308", 0xFFF0000000000000u, 0xFF800000u},
{"1.79769313486231580793e308", 0x7FEFFFFFFFFFFFFFu, 0x7F800000u},
{"179769313486231580794e288", 0x7FF0000000000000u, 0x7F800000u},
{"179769313486231580793728971405e279", 0x7FEFFFFFFFFFFFFFu, 0x7F800000u},
{"-0.179769313486231580793728971406E+309", 0xFFF0000000000000u, 0xFF800000u},
{"100000000000000011102230246251565404236316680908203125e-53", 0x3FF0000000000000u, 0x3F800000u},
{"-1.00000000000000011102230246251565404236316680908203126", 0xBFF0000000000001u, 0xBF800000u},
{"1.00000000000000011102230246251565404236316680908203124e0", 0x3FF0000000000000u, 0x3F800000u},
{"10000000000000001110223024625156540423631668090820312501e-55", 0x3FF0000000000001u, 0x3F800000u},
{"1.00000000000000011102230246251565404236316680908203125000000000000000000001", 0x3FF0000000000001u, 0x3F800000u},
{"10000000000000001e-16", 0x3FF0000000000000u, 0x3F800000u},
{"1.0000000000000002", 0x3FF0000000000001u, 0x3F800000u},
{"1.000000000000000111", 0x3FF0000000000000u, 0x3F800000u},
{"1.000000000000000112e0", 0x3FF0000000000001u, 0x3F800000u},
{"1.000000000000000111e0", 0x3FF0000000000000u, 0x3F800000u},
{"-10000000000000001111e-19", 0xBFF0000000000001u, 0xBF800000u},
{"-100000000000000011102e-20", 0xBFF0000000000000u, 0xBF800000u},
{"-1.00000000000000011103", 0xBFF0000000000001u, 0xBF800000u},
{"1.00000000000000011102230246251", 0x3FF0000000000000u, 0x3F800000u},
{"1.00000000000000011102230246252e0", 0x3FF0000000000001u, 0x3F800000u},
{"-0.999999999999999944488848768742172978818416595458984375", 0xBFF0000000000000u, 0xBF800000u},
{"-9.99999999999999944488848768742172978818416595458984376e-1", 0xBFF0000000000000u, 0xBF800000u},
{"999999999999999944488848768742172978818416595458984374e-54", 0x3FEFFFFFFFFFFFFFu, 0x3F800000u},
{"0.99999999999999994448884876874217297881841659545898437501", 0x3FF0000000000000u, 0x3F800000u},
{"9.99999999999999944488848768742172978818416595458984375000000000000000000001e-1", 0x3FF0000000000000u, 0x3F800000u},
{"-0.99999999999999994", 0xBFEFFFFFFFFFFFFFu, 0xBF800000u},
{"9.9999999999999995e-1", 0x3FF0000000000000u, 0x3F800000u},
{"9.999999999999999444e-1", 0x3FEFFFFFFFFFFFFFu, 0x3F800000u},
{"9999999999999999445e-19", 0x3FF0000000000000u, 0x3F800000u},
{"99999999999999994448e-20", 0x3FEFFFFFFFFFFFFFu, 0x3F800000u},
{"-0.99999999999999994449", 0xBFF0000000000000u, 0xBF800000u},
{"0.999999999999999944488", 0x3FEFFFFFFFFFFFFFu, 0x3F800000u},
{"-9.99999999999999944489e-1", 0xBFF0000000000000u, 0xBF800000u},
{"9.99999999999999944488848768742e-1", 0x3FEFFFFFFFFFFFFFu, 0x3F800000u},
{"999999999999999944488848768743e-30", 0x3FF0000000000000u, 0x3F800000u},
{"-9.007199254740993e15", 0xC340000000000000u, 0xDA000000u},
{"9007199254740994e0", 0x4340000000000001u, 0x5A000000u},
{"9007199254740992", 0x4340000000000000u, 0x5A000000u},
{"9.00719925474099301e15", 0x4340000000000001u, 0x5A000000u},
{"9007199254740993000000000000000000001e-21", 0x4340000000000001u, 0x5A000000u},
{"-9007199254740993.000000000000000000000000000000", 0xC340000000000000u, 0xDA000000u},
{"90071992547409915e-1", 0x4340000000000000u, 0x5A000000u},
{"-9007199254740991.6", 0xC340000000000000u, 0xDA000000u},
{"9.0071992547409914e15", 0x433FFFFFFFFFFFFFu, 0x5A000000u},
{"-9007199254740991501e-3", 0xC340000000000000u, 0xDA000000u},
{"9007199254740991.5000000000000000000001", 0x4340000000000000u, 0x5A000000u},
{"9.0071992547409915000000000000000000000000000000e15", 0x4340000000000000u, 0x5A000000u},
{"0.100000000000000012490009027033011079765856266021728515625", 0x3FB999999999999Au, 0x3DCCCCCDu},
{"1.00000000000000012490009027033011079765856266021728515626e-1", 0x3FB999999999999Bu, 0x3DCCCCCDu},
{"100000000000000012490009027033011079765856266021728515624e-57", 0x3FB999999999999Au, 0x3DCCCCCDu},
{"0.10000000000000001249000902703301107976585626602172851562501", 0x3FB999999999999Bu, 0x3DCCCCCDu},
{"0.10000000000000001", 0x3FB999999999999Au, 0x3DCCCCCDu},
{"1.0000000000000002e-1", 0x3FB999999999999Bu, 0x3DCCCCCDu},
{"-1.000000000000000124e-1", 0xBFB999999999999Au, 0xBDCCCCCDu},
{"1000000000000000125e-19", 0x3FB999999999999Bu, 0x3DCCCCCDu},
{"-10000000000000001249e-20", 0xBFB999999999999Au, 0xBDCCCCCDu},
{"0.1000000000000000125", 0x3FB999999999999Bu, 0x3DCCCCCDu},
{"0.10000000000000001249", 0x3FB999999999999Au, 0x3DCCCCCDu},
{"1.00000000000000012491e-1", 0x3FB999999999999Bu, 0x3DCCCCCDu},
{"1.00000000000000012490009027033e-1", 0x3FB999999999999Au, 0x3DCCCCCDu},
{"100000000000000012490009027034e-30", 0x3FB999999999999Bu, 0x3DCCCCCDu},
{"2.45134755833537796875e14", 0x42EBDE5C4164D83Au, 0x575EF2E2u},
{"-245134755833537796876e-6", 0xC2EBDE5C4164D83Au, 0xD75EF2E2u},
{"-245134755833537.796874", 0xC2EBDE5C4164D839u, 0xD75EF2E2u},
{"2.4513475583353779687501e14", 0x42EBDE5C4164D83Au, 0x575EF2E2u},
{"245134755833537796875000000000000000000001e-27", 0x42EBDE5C4164D83Au, 0x575EF2E2u},
{"245134755833537.796875000000000000000000000000000000", 0x42EBDE5C4164D83Au, 0x575EF2E2u},
{"2.4513475583353779e14", 0x42EBDE5C4164D839u, 0x575EF2E2u},
{"2451347558335378e-1", 0x42EBDE5C4164D83Au, 0x575EF2E2u},
{"2451347558335377968e-4", 0x42EBDE5C4164D839u, 0x575EF2E2u},
{"245134755833537.7969", 0x42EBDE5C4164D83Au, 0x575EF2E2u},
{"245134755833537.79687", 0x42EBDE5C4164D839u, 0x575EF2E2u},
{"2.4513475583353779688e14", 0x42EBDE5C4164D83Au, 0x575EF2E2u},
{"181510327827821147441864013671875e-23", 0x41DB0C11CB91CE38u, 0x4ED8608Eu},
{"-1815103278.27821147441864013671876", 0xC1DB0C11CB91CE38u, 0xCED8608Eu},
{"1.81510327827821147441864013671874e9", 0x41DB0C11CB91CE37u, 0x4ED8608Eu},
{"18151032782782114744186401367187501e-25", 0x41DB0C11CB91CE38u, 0x4ED8608Eu},
{"1815103278.27821147441864013671875000000000000000000001", 0x41DB0C11CB91CE38u, 0x4ED8608Eu},
{"-1.81510327827821147441864013671875000000000000000000000000000000e9", 0xC1DB0C11CB91CE38u, 0xCED8608Eu},
{"18151032782782114e-7", 0x41DB0C11CB91CE37u, 0x4ED8608Eu},
{"1815103278.2782115", 0x41DB0C11CB91CE38u, 0x4ED8608Eu},
{"1815103278.278211474", 0x41DB0C11CB91CE37u, 0x4ED8608Eu},
{"-1.815103278278211475e9", 0xC1DB0C11CB91CE38u, 0xCED8608Eu},
{"1.8151032782782114744e9", 0x41DB0C11CB91CE37u, 0x4ED8608Eu},
{"-18151032782782114745e-10", 0xC1DB0C11CB91CE38u, 0xCED8608Eu},
{"181510327827821147441e-11", 0x41DB0C11CB91CE37u, 0x4ED8608Eu},
{"1815103278.27821147442", 0x41DB0C11CB91CE38u, 0x4ED8608Eu},
{"1815103278.27821147441864013671", 0x41DB0C11CB91CE37u, 0x4ED8608Eu},
{"1.81510327827821147441864013672e9", 0x41DB0C11CB91CE38u, 0x4ED8608Eu},
{"3809325632181785344", 0x43CA6EB8BD69FE2Au, 0x5E5375C6u},
{"3.809325632181785345e18", 0x43CA6EB8BD69FE2Au, 0x5E5375C6u},
{"3809325632181785343e0", 0x43CA6EB8BD69FE29u, 0x5E5375C6u},
{"3809325632181785344.01", 0x43CA6EB8BD69FE2Au, 0x5E5375C6u},
{"3.809325632181785344000000000000000000001e18", 0x43CA6EB8BD69FE2Au, 0x5E5375C6u},
{"3809325632181785344000000000000000000000000000000e-30", 0x43CA6EB8BD69FE2Au, 0x5E5375C6u},
{"3809325632181785300", 0x43CA6EB8BD69FE29u, 0x5E5375C6u},
{"3.8093256321817854e18", 0x43CA6EB8BD69FE2Au, 0x5E5375C6u},
{"4.046966549916366943359375e12", 0x428D7210076CE2F0u, 0x546B9080u},
{"4046966549916366943359376e-12", 0x428D7210076CE2F0u, 0x546B9080u},
{"4046966549916.366943359374", 0x428D7210076CE2EFu, 0x546B9080u},
{"4.04696654991636694335937501e12", 0x428D7210076CE2F0u, 0x546B9080u},
{"4046966549916366943359375000000000000000000001e-33", 0x428D7210076CE2F0u, 0x546B9080u},
{"-4046966549916.366943359375000000000000000000000000000000", 0xC28D7210076CE2F0u, 0xD46B9080u},
{"4.0469665499163669e12", 0x428D7210076CE2EFu, 0x546B9080u},
{"4046966549916367e-3", 0x428D7210076CE2F0u, 0x546B9080u},
{"4046966549916366943e-6", 0x428D7210076CE2EFu, 0x546B9080u},
{"4046966549916.366944", 0x428D7210076CE2F0u, 0x546B9080u},
{"4046966549916.3669433", 0x428D7210076CE2EFu, 0x546B9080u},
{"4.0469665499163669434e12", 0x428D7210076CE2F0u, 0x546B9080u},
{"-4.04696654991636694335e12", 0xC28D7210076CE2EFu, 0xD46B9080u},
{"404696654991636694336e-8", 0x428D7210076CE2F0u, 0x546B9080u},
{"28093802557000874e154", 0x63529C3B77330BDBu, 0x7F800000u},
{"0.28093802557000875E+171", 0x63529C3B77330BDCu, 0x7F800000u},
{"0.2809380255700087447E+171", 0x63529C3B77330BDBu, 0x7F800000u},
{"2.809380255700087448e170", 0x63529C3B77330BDCu, 0x7F800000u},
{"2.8093802557000874472e170", 0x63529C3B77330BDBu, 0x7F800000u},
{"28093802557000874473e151", 0x63529C3B77330BDCu, 0x7F800000u},
{"280938025570008744728e150", 0x63529C3B77330BDBu, 0x7F800000u},
{"-0.280938025570008744729E+171", 0xE3529C3B77330BDCu, 0xFF800000u},
{"-0.280938025570008744728403667979E+171", 0xE3529C3B77330BDBu, 0xFF800000u},
{"2.8093802557000874472840366798e170", 0x63529C3B77330BDCu, 0x7F800000u},
{"0.39523280297734525e-154", 0x1FE0F51BF17FD374u, 0x00000000u},
{"-3.9523280297734526e-155", 0x9FE0F51BF17FD375u, 0x80000000u},
{"3.952328029773452547e-155", 0x1FE0F51BF17FD374u, 0x00000000u},
{"3952328029773452548e-173", 0x1FE0F51BF17FD375u, 0x00000000u},
{"-39523280297734525478e-174", 0x9FE0F51BF17FD374u, 0x80000000u},
{"0.39523280297734525479e-154", 0x1FE0F51BF17FD375u, 0x00000000u},
{"-0.395232802977345254787e-154", 0x9FE0F51BF17FD374u, 0x80000000u},
{"-3.95232802977345254788e-155", 0x9FE0F51BF17FD375u, 0x80000000u},
{"-3.95232802977345254787245825501e-155", 0x9FE0F51BF17FD374u, 0x80000000u},
{"395232802977345254787245825502e-184", 0x1FE0F51BF17FD375u, 0x00000000u},
{"-1.0790205420931879e-276", 0x86A3209CA6233255u, 0x80000000u},
{"-1079020542093188e-291", 0x86A3209CA6233256u, 0x80000000u},
{"-1079020542093187947e-294", 0x86A3209CA6233255u, 0x80000000u},
{"0.1079020542093187948e-275", 0x06A3209CA6233256u, 0x00000000u},
{"0.1079020542093187947e-275", 0x06A3209CA6233255u, 0x00000000u},
{"1.0790205420931879471e-276", 0x06A3209CA6233256u, 0x00000000u},
{"1.07902054209318794701e-276", 0x06A3209CA6233255u, 0x00000000u},
{"-107902054209318794702e-296", 0x86A3209CA6233256u, 0x80000000u},
{"107902054209318794701153285302e-305", 0x06A3209CA6233255u, 0x00000000u},
{"-0.107902054209318794701153285303e-275", 0x86A3209CA6233256u, 0x80000000u},
{"58530471071351308e-228", 0x1413B446E6A16A3Bu, 0x00000000u},
{"-0.58530471071351309e-211", 0x9413B446E6A16A3Cu, 0x80000000u},
{"0.5853047107135130893e-211", 0x1413B446E6A16A3Bu, 0x00000000u},
{"-5.853047107135130894e-212", 0x9413B446E6A16A3Cu, 0x80000000u},
{"5.853047107135130893e-212", 0x1413B446E6A16A3Bu, 0x00000000u},
{"58530471071351308931e-231", 0x1413B446E6A16A3Cu, 0x00000000u},
{"-585304710713513089304e-232", 0x9413B446E6A16A3Bu, 0x80000000u},
{"0.585304710713513089305e-211", 0x1413B446E6A16A3Cu, 0x00000000u},
{"-0.585304710713513089304248824438e-211", 0x9413B446E6A16A3Bu, 0x80000000u},
{"5.85304710713513089304248824439e-212", 0x1413B446E6A16A3Cu, 0x00000000u},
{"0.19334214893983531e-78", 0x2F96ECBF1CFB10F6u, 0x00000000u},
{"1.9334214893983532e-79", 0x2F96ECBF1CFB10F7u, 0x00000000u},
{"1.933421489398353102e-79", 0x2F96ECBF1CFB10F6u, 0x00000000u},
{"1933421489398353103e-97", 0x2F96ECBF1CFB10F7u, 0x00000000u},
{"19334214893983531023e-98", 0x2F96ECBF1CFB10F6u, 0x00000000u},
{"0.19334214893983531024e-78", 0x2F96ECBF1CFB10F7u, 0x00000000u},
{"0.193342148939835310231e-78", 0x2F96ECBF1CFB10F6u, 0x00000000u},
{"1.93342148939835310232e-79", 0x2F96ECBF1CFB10F7u, 0x00000000u},
{"-1.93342148939835310231359014704e-79", 0xAF96ECBF1CFB10F6u, 0x80000000u},
{"193342148939835310231359014705e-108", 0x2F96ECBF1CFB10F7u, 0x00000000u},
{"2.9873358928024455e227", 0x6F2938807814E8A2u, 0x7F800000u},
{"29873358928024456e211", 0x6F2938807814E8A3u, 0x7F800000u},
{"298733589280244551e210", 0x6F2938807814E8A2u, 0x7F800000u},
{"0.2987335892802445511E+228", 0x6F2938807814E8A3u, 0x7F800000u},
{"0.29873358928024455109E+228", 0x6F2938807814E8A2u, 0x7F800000u},
{"2.987335892802445511e227", 0x6F2938807814E8A3u, 0x7F800000u},
{"-2.98733589280244551098e227", 0xEF2938807814E8A2u, 0xFF800000u},
{"-298733589280244551099e207", 0xEF2938807814E8A3u, 0xFF800000u},
{"298733589280244551098081559931e198", 0x6F2938807814E8A2u, 0x7F800000u},
{"0.298733589280244551098081559932E+228", 0x6F2938807814E8A3u, 0x7F800000u},
{"7.0064923216240853e-46", 0x3690000000000000u, 0x00000000u},
{"70064923216240854e-62", 0x3690000000000000u, 0x00000001u},
{"7006492321624085354e-64", 0x3690000000000000u, 0x00000000u},
{"-0.7006492321624085355e-45", 0xB690000000000000u, 0x80000001u},
{"0.70064923216240853546e-45", 0x3690000000000000u, 0x00000000u},
{"7.0064923216240853547e-46", 0x3690000000000000u, 0x00000001u},
{"7.00649232162408535461e-46", 0x3690000000000000u, 0x00000000u},
{"-700649232162408535462e-66", 0xB690000000000000u, 0x80000001u},
{"-700649232162408535461864791644e-75", 0xB690000000000000u, 0x80000000u},
{"0.700649232162408535461864791645e-45", 0x3690000000000000u, 0x00000001u},
{"21019476964872256e-61", 0x36A8000000000000u, 0x00000001u},
{"0.21019476964872257e-44", 0x36A8000000000000u, 0x00000002u},
{"-0.2101947696487225606e-44", 0xB6A8000000000000u, 0x80000001u},
{"-2.101947696487225607e-45", 0xB6A8000000000000u, 0x80000002u},
{"2.1019476964872256063e-45", 0x36A8000000000000u, 0x00000001u},
{"21019476964872256064e-64", 0x36A8000000000000u, 0x00000002u},
{"-210194769648722560638e-65", 0xB6A8000000000000u, 0x80000001u},
{"-0.210194769648722560639e-44", 0xB6A8000000000000u, 0x80000002u},
{"-0.210194769648722560638559437493e-44", 0xB6A8000000000000u, 0x80000001u},
{"2.10194769648722560638559437494e-45", 0x36A8000000000000u, 0x00000002u},
{"0.11754941406275178e-37", 0x380FFFFFA0000000u, 0x007FFFFEu},
{"1.1754941406275179e-38", 0x380FFFFFA0000000u, 0x007FFFFFu},
{"-1.175494140627517859e-38", 0xB80FFFFFA0000000u, 0x807FFFFEu},
{"117549414062751786e-55", 0x380FFFFFA0000000u, 0x007FFFFFu},
{"-11754941406275178592e-57", 0xB80FFFFFA0000000u, 0x807FFFFEu},
{"0.11754941406275178593e-37", 0x380FFFFFA0000000u, 0x007FFFFFu},
{"0.117549414062751785924e-37", 0x380FFFFFA0000000u, 0x007FFFFEu},
{"1.17549414062751785925e-38", 0x380FFFFFA0000000u, 0x007FFFFFu},
{"-1.17549414062751785924617589866e-38", 0xB80FFFFFA0000000u, 0x807FFFFEu},
{"117549414062751785924617589867e-67", 0x380FFFFFA0000000u, 0x007FFFFFu},
{"1.1754942807573642e-38", 0x380FFFFFDFFFFFFFu, 0x007FFFFFu},
{"11754942807573643e-54", 0x380FFFFFE0000000u, 0x00800000u},
{"1175494280757364291e-56", 0x380FFFFFE0000000u, 0x007FFFFFu},
{"0.1175494280757364292e-37", 0x380FFFFFE0000000u, 0x00800000u},
{"0.11754942807573642917e-37", 0x380FFFFFE0000000u, 0x007FFFFFu},
{"-1.1754942807573642918e-38", 0xB80FFFFFE0000000u, 0x80800000u},
{"-1.17549428075736429172e-38", 0xB80FFFFFE0000000u, 0x807FFFFFu},
{"-117549428075736429173e-58", 0xB80FFFFFE0000000u, 0x80800000u},
{"117549428075736429172788299103e-67", 0x380FFFFFE0000000u, 0x007FFFFFu},
{"0.117549428075736429172788299104e-37", 0x380FFFFFE0000000u, 0x00800000u},
{"11754944208872107e-54", 0x3810000010000000u, 0x00800000u},
{"-0.11754944208872108e-37", 0xB810000010000000u, 0x80800001u},
{"0.1175494420887210724e-37", 0x3810000010000000u, 0x00800000u},
{"1.175494420887210725e-38", 0x3810000010000000u, 0x00800001u},
{"1.1754944208872107242e-38", 0x3810000010000000u, 0x00800000u},
{"-11754944208872107243e-57", 0xB810000010000000u, 0x80800001u},
{"-11754944208872107242e-57", 0xB810000010000000u, 0x80800000u},
{"0.117549442088721072421e-37", 0x3810000010000000u, 0x00800001u},
{"0.11754944208872107242095900834e-37", 0x3810000010000000u, 0x00800000u},
{"-1.17549442088721072420959008341e-38", 0xB810000010000000u, 0x80800001u},
{"340282336497324057985868971510891282432", 0x47EFFFFFD0000000u, 0x7F7FFFFEu},
{"-3.40282336497324057985868971510891282433e38", 0xC7EFFFFFD0000000u, 0xFF7FFFFFu},
{"340282336497324057985868971510891282431e0", 0x47EFFFFFD0000000u, 0x7F7FFFFEu},
{"340282336497324057985868971510891282432.01", 0x47EFFFFFD0000000u, 0x7F7FFFFFu},
{"3.40282336497324057985868971510891282432000000000000000000001e38", 0x47EFFFFFD0000000u, 0x7F7FFFFFu},
{"340282336497324057985868971510891282432000000000000000000000000000000e-30", 0x47EFFFFFD0000000u, 0x7F7FFFFEu},
{"340282336497324050000000000000000000000", 0x47EFFFFFD0000000u, 0x7F7FFFFEu},
{"3.4028233649732406e38", 0x47EFFFFFD0000000u, 0x7F7FFFFFu},
{"3.402823364973240579e38", 0x47EFFFFFD0000000u, 0x7F7FFFFEu},
{"340282336497324058e21", 0x47EFFFFFD0000000u, 0x7F7FFFFFu},
{"34028233649732405798e19", 0x47EFFFFFD0000000u, 0x7F7FFFFEu},
{"340282336497324057990000000000000000000", 0x47EFFFFFD0000000u, 0x7F7FFFFFu},
{"340282336497324057985000000000000000000", 0x47EFFFFFD0000000u, 0x7F7FFFFEu},
{"3.40282336497324057986e38", 0x47EFFFFFD0000000u, 0x7F7FFFFFu},
{"3.4028233649732405798586897151e38", 0x47EFFFFFD0000000u, 0x7F7FFFFEu},
{"340282336497324057985868971511e9", 0x47EFFFFFD0000000u, 0x7F7FFFFFu},
{"3.40282356779733661637539395458142568448e38", 0x47EFFFFFF0000000u, 0x7F800000u},
{"-340282356779733661637539395458142568449e0", 0xC7EFFFFFF0000000u, 0xFF800000u},
{"340282356779733661637539395458142568447", 0x47EFFFFFF0000000u, 0x7F7FFFFFu},
{"3.4028235677973366163753939545814256844801e38", 0x47EFFFFFF0000000u, 0x7F800000u},
{"340282356779733661637539395458142568448000000000000000000001e-21", 0x47EFFFFFF0000000u, 0x7F800000u},
{"340282356779733661637539395458142568448.000000000000000000000000000000", 0x47EFFFFFF0000000u, 0x7F800000u},
{"3.4028235677973366e38", 0x47EFFFFFF0000000u, 0x7F7FFFFFu},
{"-34028235677973367e22", 0xC7EFFFFFF0000000u, 0xFF800000u},
{"3402823567797336616e20", 0x47EFFFFFF0000000u, 0x7F7FFFFFu},
{"340282356779733661700000000000000000000", 0x47EFFFFFF0000000u, 0x7F800000u},
{"340282356779733661630000000000000000000", 0x47EFFFFFF0000000u, 0x7F7FFFFFu},
{"3.4028235677973366164e38", 0x47EFFFFFF0000000u, 0x7F800000u},
{"3.40282356779733661637e38", 0x47EFFFFFF0000000u, 0x7F7FFFFFu},
{"340282356779733661638e18", 0x47EFFFFFF0000000u, 0x7F800000u},
{"-340282356779733661637539395458e9", 0xC7EFFFFFF0000000u, 0xFF7FFFFFu},
{"340282356779733661637539395459000000000", 0x47EFFFFFF0000000u, 0x7F800000u},
{"-1000000059604644775390625e-24", 0xBFF0000010000000u, 0xBF800000u},
{"-1.000000059604644775390626", 0xBFF0000010000000u, 0xBF800001u},
{"-1.000000059604644775390624e0", 0xBFF0000010000000u, 0xBF800000u},
{"100000005960464477539062501e-26", 0x3FF0000010000000u, 0x3F800001u},
{"1.000000059604644775390625000000000000000000001", 0x3FF0000010000000u, 0x3F800001u},
{"1.000000059604644775390625000000000000000000000000000000e0", 0x3FF0000010000000u, 0x3F800000u},
{"-10000000596046447e-16", 0xBFF0000010000000u, 0xBF800000u},
{"1.0000000596046448", 0x3FF0000010000000u, 0x3F800001u},
{"1.000000059604644775", 0x3FF0000010000000u, 0x3F800000u},
{"-1.000000059604644776e0", 0xBFF0000010000000u, 0xBF800001u},
{"-1.0000000596046447753e0", 0xBFF0000010000000u, 0xBF800000u},
{"10000000596046447754e-19", 0x3FF0000010000000u, 0x3F800001u},
{"100000005960464477539e-20", 0x3FF0000010000000u, 0x3F800000u},
{"-1.0000000596046447754", 0xBFF0000010000000u, 0xBF800001u},
{"0.9999999701976776123046875", 0x3FEFFFFFF0000000u, 0x3F800000u},
{"9.999999701976776123046876e-1", 0x3FEFFFFFF0000000u, 0x3F800000u},
{"9999999701976776123046874e-25", 0x3FEFFFFFF0000000u, 0x3F7FFFFFu},
{"0.999999970197677612304687501", 0x3FEFFFFFF0000000u, 0x3F800000u},
{"-9.999999701976776123046875000000000000000000001e-1", 0xBFEFFFFFF0000000u, 0xBF800000u},
{"-9999999701976776123046875000000000000000000000000000000e-55", 0xBFEFFFFFF0000000u, 0xBF800000u},
{"0.99999997019767761", 0x3FEFFFFFF0000000u, 0x3F7FFFFFu},
{"-9.9999997019767762e-1", 0xBFEFFFFFF0000000u, 0xBF800000u},
{"-9.999999701976776123e-1", 0xBFEFFFFFF0000000u, 0xBF7FFFFFu},
{"9999999701976776124e-19", 0x3FEFFFFFF0000000u, 0x3F800000u},
{"9999999701976776123e-19", 0x3FEFFFFFF0000000u, 0x3F7FFFFFu},
{"0.99999997019767761231", 0x3FEFFFFFF0000000u, 0x3F800000u},
{"-0.999999970197677612304", 0xBFEFFFFFF0000000u, 0xBF7FFFFFu},
{"9.99999970197677612305e-1", 0x3FEFFFFFF0000000u, 0x3F800000u},
{"-1.6777217e7", 0xC170000010000000u, 0xCB800000u},
{"16777218e0", 0x4170000020000000u, 0x4B800001u},
{"16777216", 0x4170000000000000u, 0x4B800000u},
{"-1.677721701e7", 0xC17000001028F5C3u, 0xCB800001u},
{"-16777217000000000000000000001e-21", 0xC170000010000000u, 0xCB800001u},
{"16777217.000000000000000000000000000000", 0x4170000010000000u, 0x4B800000u},
{"167772155e-1", 0x416FFFFFF0000000u, 0x4B800000u},
{"16777215.6", 0x416FFFFFF3333333u, 0x4B800000u},
{"1.67772154e7", 0x416FFFFFECCCCCCDu, 0x4B7FFFFFu},
{"16777215501e-3", 0x416FFFFFF0083127u, 0x4B800000u},
{"-16777215.5000000000000000000001", 0xC16FFFFFF0000000u, 0xCB800000u},
{"-1.67772155000000000000000000000000000000e7", 0xC16FFFFFF0000000u, 0xCB800000u},
{"0.1000000052154064178466796875", 0x3FB99999B0000000u, 0x3DCCCCCEu},
{"1.000000052154064178466796876e-1", 0x3FB99999B0000000u, 0x3DCCCCCEu},
{"-1000000052154064178466796874e-28", 0xBFB99999B0000000u, 0xBDCCCCCDu},
{"0.100000005215406417846679687501", 0x3FB99999B0000000u, 0x3DCCCCCEu},
{"1.000000052154064178466796875000000000000000000001e-1", 0x3FB99999B0000000u, 0x3DCCCCCEu},
{"-1000000052154064178466796875000000000000000000000000000000e-58", 0xBFB99999B0000000u, 0xBDCCCCCEu},
{"-0.10000000521540641", 0xBFB99999AFFFFFFFu, 0xBDCCCCCDu},
{"-1.0000000521540642e-1", 0xBFB99999B0000000u, 0xBDCCCCCEu},
{"1.000000052154064178e-1", 0x3FB99999B0000000u, 0x3DCCCCCDu},
{"-1000000052154064179e-19", 0xBFB99999B0000000u, 0xBDCCCCCEu},
{"10000000521540641784e-20", 0x3FB99999B0000000u, 0x3DCCCCCDu},
{"0.10000000521540641785", 0x3FB99999B0000000u, 0x3DCCCCCEu},
{"0.100000005215406417846", 0x3FB99999B0000000u, 0x3DCCCCCDu},
{"1.00000005215406417847e-1", 0x3FB99999B0000000u, 0x3DCCCCCEu},
{"5.429001220703125e3", 0x40B5350050000000u, 0x45A9A802u},
{"-5429001220703126e-12", 0xC0B5350050000001u, 0xC5A9A803u},
{"-5429.001220703124", 0xC0B535004FFFFFFFu, 0xC5A9A802u},
{"5.42900122070312501e3", 0x40B5350050000000u, 0x45A9A803u},
{"5429001220703125000000000000000000001e-33", 0x40B5350050000000u, 0x45A9A803u},
{"-5429.001220703125000000000000000000000000000000", 0xC0B5350050000000u, 0xC5A9A802u},
{"503719056e0", 0x41BE062490000000u, 0x4DF03124u},
{"503719057", 0x41BE062491000000u, 0x4DF03125u},
{"5.03719055e8", 0x41BE06248F000000u, 0x4DF03124u},
{"50371905601e-2", 0x41BE062490028F5Cu, 0x4DF03125u},
{"503719056.000000000000000000001", 0x41BE062490000000u, 0x4DF03125u},
{"5.03719056000000000000000000000000000000e8", 0x41BE062490000000u, 0x4DF03124u},
{"-92331620", 0xC196037990000000u, 0xCCB01BCCu},
{"9.233163e7", 0x41960379B8000000u, 0x4CB01BCEu},
{"9233161e1", 0x4196037968000000u, 0x4CB01BCBu},
{"92331620.1", 0x4196037990666666u, 0x4CB01BCDu},
{"9.233162000000000000000000001e7", 0x4196037990000000u, 0x4CB01BCDu},
{"9233162000000000000000000000000000000e-29", 0x4196037990000000u, 0x4CB01BCCu},
{"3.002458625e6", 0x4146E82D50000000u, 0x4A37416Au},
{"3002458626e-3", 0x4146E82D5020C49Cu, 0x4A37416Bu},
{"3002458.624", 0x4146E82D4FDF3B64u, 0x4A37416Au},
{"-3.00245862501e6", 0xC146E82D500053E3u, 0xCA37416Bu},
{"3002458625000000000000000000001e-24", 0x4146E82D50000000u, 0x4A37416Bu},
{"3002458.625000000000000000000000000000000", 0x4146E82D50000000u, 0x4A37416Au},
{"-1095485584696182596504479582065262592e1", 0xC7A07BA830000000u, 0xFD03DD42u},
{"10954855846961825965044795820652625930", 0x47A07BA830000000u, 0x7D03DD42u},
{"1.095485584696182596504479582065262591e37", 0x47A07BA830000000u, 0x7D03DD41u},
{"109548558469618259650447958206526259201e-1", 0x47A07BA830000000u, 0x7D03DD42u},
{"-10954855846961825965044795820652625920.00000000000000000001", 0xC7A07BA830000000u, 0xFD03DD42u},
{"1.095485584696182596504479582065262592000000000000000000000000000000e37", 0x47A07BA830000000u, 0x7D03DD42u},
{"10954855846961825e21", 0x47A07BA830000000u, 0x7D03DD41u},
{"10954855846961826000000000000000000000", 0x47A07BA830000000u, 0x7D03DD42u},
{"10954855846961825960000000000000000000", 0x47A07BA830000000u, 0x7D03DD41u},
{"1.095485584696182597e37", 0x47A07BA830000000u, 0x7D03DD42u},
{"1.0954855846961825965e37", 0x47A07BA830000000u, 0x7D03DD41u},
{"-10954855846961825966e18", 0xC7A07BA830000000u, 0xFD03DD42u},
{"10954855846961825965e18", 0x47A07BA830000000u, 0x7D03DD41u},
{"10954855846961825965100000000000000000", 0x47A07BA830000000u, 0x7D03DD42u},
{"10954855846961825965044795820600000000", 0x47A07BA830000000u, 0x7D03DD41u},
{"-1.09548558469618259650447958207e37", 0xC7A07BA830000000u, 0xFD03DD42u},
{"1.6449216019182103706535606608388384863861375606575165875256061553955078126e-21", 0x3B9F125A50000000u, 0x1CF892D3u},
{"-16449216019182103706535606608388384863861375606575165875256061553955078124e-94", 0xBB9F125A50000000u, 0x9CF892D2u},
{"-0.0000000000000000000016449216019182103", 0xBB9F125A50000000u, 0x9CF892D2u},
{"-1.6449216019182104e-21", 0xBB9F125A50000000u, 0x9CF892D3u},
{"-1.64492160191821037e-21", 0xBB9F125A50000000u, 0x9CF892D2u},
{"-1644921601918210371e-39", 0xBB9F125A50000000u, 0x9CF892D3u},
{"-16449216019182103706e-40", 0xBB9F125A50000000u, 0x9CF892D2u},
{"0.0000000000000000000016449216019182103707", 0x3B9F125A50000000u, 0x1CF892D3u},
{"0.00000000000000000000164492160191821037065", 0x3B9F125A50000000u, 0x1CF892D2u},
{"1.64492160191821037066e-21", 0x3B9F125A50000000u, 0x1CF892D3u},
{"1.64492160191821037065356066083e-21", 0x3B9F125A50000000u, 0x1CF892D2u},
{"164492160191821037065356066084e-50", 0x3B9F125A50000000u, 0x1CF892D3u},
{"6.565061509609222412109375e-1", 0x3FE5021930000000u, 0x3F2810CAu},
{"6565061509609222412109376e-25", 0x3FE5021930000000u, 0x3F2810CAu},
{"0.6565061509609222412109374", 0x3FE5021930000000u, 0x3F2810C9u},
{"6.56506150960922241210937501e-1", 0x3FE5021930000000u, 0x3F2810CAu},
{"6565061509609222412109375000000000000000000001e-46", 0x3FE5021930000000u, 0x3F2810CAu},
{"-0.6565061509609222412109375000000000000000000000000000000", 0xBFE5021930000000u, 0xBF2810CAu},
{"6.5650615096092224e-1", 0x3FE5021930000000u, 0x3F2810C9u},
{"-65650615096092225e-17", 0xBFE5021930000000u, 0xBF2810CAu},
{"6565061509609222412e-19", 0x3FE5021930000000u, 0x3F2810C9u},
{"0.6565061509609222413", 0x3FE5021930000000u, 0x3F2810CAu},
{"0.65650615096092224121", 0x3FE5021930000000u, 0x3F2810C9u},
{"6.5650615096092224122e-1", 0x3FE5021930000000u, 0x3F2810CAu},
{"-6.5650615096092224121e-1", 0xBFE5021930000000u, 0xBF2810C9u},
{"656506150960922241211e-21", 0x3FE5021930000000u, 0x3F2810CAu},
{"18014627239033005156980393746124491372029297053813934326171875e-77", 0x3CA9F63970000000u, 0x254FB1CCu},
{"0.00000000000000018014627239033005156980393746124491372029297053813934326171876", 0x3CA9F63970000000u, 0x254FB1CCu},
{"-1.8014627239033005156980393746124491372029297053813934326171874e-16", 0xBCA9F63970000000u, 0xA54FB1CBu},
{"1801462723903300515698039374612449137202929705381393432617187501e-79", 0x3CA9F63970000000u, 0x254FB1CCu},
{"18014627239033005e-32", 0x3CA9F63970000000u, 0x254FB1CBu},
{"0.00000000000000018014627239033006", 0x3CA9F63970000000u, 0x254FB1CCu},
{"0.0000000000000001801462723903300515", 0x3CA9F63970000000u, 0x254FB1CBu},
{"-1.801462723903300516e-16", 0xBCA9F63970000000u, 0xA54FB1CCu},
{"-1.8014627239033005156e-16", 0xBCA9F63970000000u, 0xA54FB1CBu},
{"18014627239033005157e-35", 0x3CA9F63970000000u, 0x254FB1CCu},
{"180146272390330051569e-36", 0x3CA9F63970000000u, 0x254FB1CBu},
{"-0.00000000000000018014627239033005157", 0xBCA9F63970000000u, 0xA54FB1CCu},
{"0.000000000000000180146272390330051569803937461", 0x3CA9F63970000000u, 0x254FB1CBu},
{"1.80146272390330051569803937462e-16", 0x3CA9F63970000000u, 0x254FB1CCu},
{"0.05534819327294826507568359375", 0x3FAC569930000000u, 0x3D62B4CAu},
{"-5.534819327294826507568359376e-2", 0xBFAC569930000000u, 0xBD62B4CAu},
{"5534819327294826507568359374e-29", 0x3FAC569930000000u, 0x3D62B4C9u},
{"-0.0553481932729482650756835937501", 0xBFAC569930000000u, 0xBD62B4CAu},
{"5.534819327294826507568359375000000000000000000001e-2", 0x3FAC569930000000u, 0x3D62B4CAu},
{"5534819327294826507568359375000000000000000000000000000000e-59", 0x3FAC569930000000u, 0x3D62B4CAu},
{"0.055348193272948265", 0x3FAC569930000000u, 0x3D62B4C9u},
{"5.5348193272948266e-2", 0x3FAC569930000000u, 0x3D62B4CAu},
{"5.534819327294826507e-2", 0x3FAC569930000000u, 0x3D62B4C9u},
{"5534819327294826508e-20", 0x3FAC569930000000u, 0x3D62B4CAu},
{"55348193272948265075e-21", 0x3FAC569930000000u, 0x3D62B4C9u},
{"-0.055348193272948265076", 0xBFAC569930000000u, 0xBD62B4CAu},
{"0.0553481932729482650756", 0x3FAC569930000000u, 0x3D62B4C9u},
{"-5.53481932729482650757e-2", 0xBFAC569930000000u, 0xBD62B4CAu},
{"5.179692133247783258005389047985340416e36", 0x478F2C9450000000u, 0x7C7964A2u},
{"5179692133247783258005389047985340417e0", 0x478F2C9450000000u, 0x7C7964A3u},
{"5179692133247783258005389047985340415", 0x478F2C9450000000u, 0x7C7964A2u},
{"-5.17969213324778325800538904798534041601e36", 0xC78F2C9450000000u, 0xFC7964A3u},
{"-5179692133247783258005389047985340416000000000000000000001e-21", 0xC78F2C9450000000u, 0xFC7964A3u},
{"-5179692133247783258005389047985340416.000000000000000000000000000000", 0xC78F2C9450000000u, 0xFC7964A2u},
{"5.1796921332477832e36", 0x478F2C9450000000u, 0x7C7964A2u},
{"51796921332477833e20", 0x478F2C9450000000u, 0x7C7964A3u},
{"-5179692133247783258e18", 0xC78F2C9450000000u, 0xFC7964A2u},
{"5179692133247783259000000000000000000", 0x478F2C9450000000u, 0x7C7964A3u},
{"5179692133247783258000000000000000000", 0x478F2C9450000000u, 0x7C7964A2u},
{"5.1796921332477832581e36", 0x478F2C9450000000u, 0x7C7964A3u},
{"-5.179692133247783258e36", 0xC78F2C9450000000u, 0xFC7964A2u},
{"-517969213324778325801e16", 0xC78F2C9450000000u, 0xFC7964A3u},
{"517969213324778325800538904798e7", 0x478F2C9450000000u, 0x7C7964A2u},
{"5179692133247783258005389047990000000", 0x478F2C9450000000u, 0x7C7964A3u},
{
"0.22250738585072011360574097967091319759348195463516456480234261097248222220210769455165295239081350"
"8791414915891303962110687008643869459464552765720740782062174337998814106326732925355228688137214901"
"2981122451451889849057222307285255133155755015914397476397983411801999323962548289017107081850690630"
"6666559949382757725720157630626906633326475653000092458883164330377797918696120494973903778297049050"
"5108060994073026293712895895000358379996720725430436028407889577179615094551674824347103070260914462"
"1572289880258182545180325707018860872113128079512233426288368622321503775666622503982534335974568884"
"4239002654981983854879482922068947216898310996983658468140228542433306603398508864458040010349339704"
"2756718644338377048603786162277173854562306587467901408672332763671875e-307", 0x0010000000000000u, 0x00000000u
},
{
"2.22507385850720113605740979670913197593481954635164564802342610972482222202107694551652952390813508"
"7914149158913039621106870086438694594645527657207407820621743379988141063267329253552286881372149012"
"9811224514518898490572223072852551331557550159143974763979834118019993239625482890171070818506906306"
"6665599493827577257201576306269066333264756530000924588831643303777979186961204949739037782970490505"
"1080609940730262937128958950003583799967207254304360284078895771796150945516748243471030702609144621"
"5722898802581825451803257070188608721131280795122334262883686223215037756666225039825343359745688844"
"2390026549819838548794829220689472168983109969836584681402285424333066033985088644580400103493397042"
"756718644338377048603786162277173854562306587467901408672332763671875000000000000000000001e-308", 0x0010000000000000u, 0x00000000u
},
{
"0.11754942807573642917278829910357665133228589927589904276829631184250030649651730385585324256680905"
"8189392089843750000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
"0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
"0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
"0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
"0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
"0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
"00000000000000000000000000000000000000000000000000000000000000000e-37", 0x380FFFFFE0000000u, 0x00800000u
},
{
"1175494280757364291727882991035766513322858992758990427682963118425003064965173038558532425668090581"
"8939208984375000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
"0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
"0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
"0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
"0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
"0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
"00000000000001e-751", 0x380FFFFFE0000000u, 0x00800000u
},
{"0", 0x0000000000000000u, 0x00000000u},
{"-0", 0x8000000000000000u, 0x80000000u},
{"0.0", 0x0000000000000000u, 0x00000000u},
{"-0.0", 0x8000000000000000u, 0x80000000u},
{"0e999999999999999999999", 0x0000000000000000u, 0x00000000u},
{"-0.000e-99999", 0x8000000000000000u, 0x80000000u},
{"1e-400", 0x0000000000000000u, 0x00000000u},
{"-1e-400", 0x8000000000000000u, 0x80000000u},
{"1e400", 0x7FF0000000000000u, 0x7F800000u},
{"-1e400", 0xFFF0000000000000u, 0xFF800000u},
{"1e-50", 0x358DEE7A4AD4B81Fu, 0x00000000u},
{"-1e-50", 0xB58DEE7A4AD4B81Fu, 0x80000000u},
{"1e39", 0x48078287F49C4A1Du, 0x7F800000u},
{"-1e39", 0xC8078287F49C4A1Du, 0xFF800000u},
{"1e99999999999999999999999999", 0x7FF0000000000000u, 0x7F800000u},
{"1e-99999999999999999999999999", 0x0000000000000000u, 0x00000000u},
{"1e0000000000000000000000000000000000000000308", 0x7FE1CCF385EBC8A0u, 0x7F800000u},
{"123456789012345678901234567890e-30", 0x3FBF9ADD3746F65Fu, 0x3DFCD6EAu},
{"18446744073709551615", 0x43F0000000000000u, 0x5F800000u},
{"18446744073709551616", 0x43F0000000000000u, 0x5F800000u},
{"-9223372036854775808", 0xC3E0000000000000u, 0xDF000000u},
{"-9223372036854775809", 0xC3E0000000000000u, 0xDF000000u},
}
};
return table;
}
} // namespace float_hard_cases
+32 -3
View File
@@ -51,7 +51,22 @@ The provided function `LLVMFuzzerTestOneInput` can be used in different fuzzer
drivers.
*/
#include "fuzzer_common.hpp"
#include <cassert>
#include <nlohmann/json.hpp>
// the round-trip checks below are assertions; NDEBUG would compile them away
#ifdef NDEBUG
#error "the fuzzer drivers must be built without NDEBUG"
#endif
using json = nlohmann::json;
// compares dumps rather than values, because NaN != NaN; keep writes strings
// byte for byte, so ill-formed UTF-8 that a binary reader accepts cannot throw
static bool same_value(const json& lhs, const json& rhs)
{
return lhs.dump(-1, ' ', false, json::error_handler_t::keep) == rhs.dump(-1, ' ', false, json::error_handler_t::keep);
}
// value-stable comparison for the round-trip checks below; see the note
// above on why this compares dump()s rather than the json values directly
@@ -65,9 +80,23 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
{
std::vector<uint8_t> const vec1(data, data + size);
// step 0: parse input without exceptions
// step 0: parse input without exceptions; a parse error must then be
// reported as a discarded value, never thrown
json j_noexcept;
bool noexcept_threw = false;
json const j_noexcept = parse_without_exceptions([&] { return json::from_bjdata(vec1, true, false); }, noexcept_threw);
try
{
j_noexcept = json::from_bjdata(vec1, true, false);
}
catch (const json::parse_error&)
{
assert(false);
}
catch (const json::exception&)
{
// type and out-of-range errors are not parse errors and still throw
noexcept_threw = true;
}
// whether step 1 succeeded; if not, the catch blocks below check that
// step 0 failed, too
bool parsed = false;
+32 -3
View File
@@ -25,8 +25,23 @@ The provided function `LLVMFuzzerTestOneInput` can be used in different fuzzer
drivers.
*/
#include <cassert>
#include <sstream>
#include "fuzzer_common.hpp"
#include <nlohmann/json.hpp>
// the round-trip checks below are assertions; NDEBUG would compile them away
#ifdef NDEBUG
#error "the fuzzer drivers must be built without NDEBUG"
#endif
using json = nlohmann::json;
// compares dumps rather than values, because NaN != NaN; keep writes strings
// byte for byte, so ill-formed UTF-8 that a binary reader accepts cannot throw
static bool same_value(const json& lhs, const json& rhs)
{
return lhs.dump(-1, ' ', false, json::error_handler_t::keep) == rhs.dump(-1, ' ', false, json::error_handler_t::keep);
}
namespace
{
@@ -57,9 +72,23 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
std::vector<uint8_t> const vec1(data, data + size);
// step 0: parse input without exceptions
// step 0: parse input without exceptions; a parse error must then be
// reported as a discarded value, never thrown
json j_noexcept;
bool noexcept_threw = false;
json const j_noexcept = parse_without_exceptions([&] { return json::from_bon8(vec1, true, false); }, noexcept_threw);
try
{
j_noexcept = json::from_bon8(vec1, true, false);
}
catch (const json::parse_error&)
{
assert(false);
}
catch (const json::exception&)
{
// type and out-of-range errors are not parse errors and still throw
noexcept_threw = true;
}
// whether step 1 succeeded; if not, the catch blocks below check that
// step 0 failed, too
bool parsed = false;
+32 -3
View File
@@ -21,16 +21,45 @@ The provided function `LLVMFuzzerTestOneInput` can be used in different fuzzer
drivers.
*/
#include "fuzzer_common.hpp"
#include <cassert>
#include <nlohmann/json.hpp>
// the round-trip checks below are assertions; NDEBUG would compile them away
#ifdef NDEBUG
#error "the fuzzer drivers must be built without NDEBUG"
#endif
using json = nlohmann::json;
// compares dumps rather than values, because NaN != NaN; keep writes strings
// byte for byte, so ill-formed UTF-8 that a binary reader accepts cannot throw
static bool same_value(const json& lhs, const json& rhs)
{
return lhs.dump(-1, ' ', false, json::error_handler_t::keep) == rhs.dump(-1, ' ', false, json::error_handler_t::keep);
}
// see http://llvm.org/docs/LibFuzzer.html
extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
{
std::vector<uint8_t> const vec1(data, data + size);
// step 0: parse input without exceptions
// step 0: parse input without exceptions; a parse error must then be
// reported as a discarded value, never thrown
json j_noexcept;
bool noexcept_threw = false;
json const j_noexcept = parse_without_exceptions([&] { return json::from_bson(vec1, true, false); }, noexcept_threw);
try
{
j_noexcept = json::from_bson(vec1, true, false);
}
catch (const json::parse_error&)
{
assert(false);
}
catch (const json::exception&)
{
// type and out-of-range errors are not parse errors and still throw
noexcept_threw = true;
}
// whether step 1 succeeded; if not, the catch blocks below check that
// step 0 failed, too
bool parsed = false;
+32 -3
View File
@@ -21,16 +21,45 @@ The provided function `LLVMFuzzerTestOneInput` can be used in different fuzzer
drivers.
*/
#include "fuzzer_common.hpp"
#include <cassert>
#include <nlohmann/json.hpp>
// the round-trip checks below are assertions; NDEBUG would compile them away
#ifdef NDEBUG
#error "the fuzzer drivers must be built without NDEBUG"
#endif
using json = nlohmann::json;
// compares dumps rather than values, because NaN != NaN; keep writes strings
// byte for byte, so ill-formed UTF-8 that a binary reader accepts cannot throw
static bool same_value(const json& lhs, const json& rhs)
{
return lhs.dump(-1, ' ', false, json::error_handler_t::keep) == rhs.dump(-1, ' ', false, json::error_handler_t::keep);
}
// see http://llvm.org/docs/LibFuzzer.html
extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
{
std::vector<uint8_t> const vec1(data, data + size);
// step 0: parse input without exceptions
// step 0: parse input without exceptions; a parse error must then be
// reported as a discarded value, never thrown
json j_noexcept;
bool noexcept_threw = false;
json const j_noexcept = parse_without_exceptions([&] { return json::from_cbor(vec1, true, false); }, noexcept_threw);
try
{
j_noexcept = json::from_cbor(vec1, true, false);
}
catch (const json::parse_error&)
{
assert(false);
}
catch (const json::exception&)
{
// type and out-of-range errors are not parse errors and still throw
noexcept_threw = true;
}
// whether step 1 succeeded; if not, the catch blocks below check that
// step 0 failed, too
bool parsed = false;
+32 -3
View File
@@ -22,14 +22,43 @@ The provided function `LLVMFuzzerTestOneInput` can be used in different fuzzer
drivers.
*/
#include "fuzzer_common.hpp"
#include <cassert>
#include <nlohmann/json.hpp>
// the round-trip checks below are assertions; NDEBUG would compile them away
#ifdef NDEBUG
#error "the fuzzer drivers must be built without NDEBUG"
#endif
using json = nlohmann::json;
// compares dumps rather than values, because NaN != NaN; keep writes strings
// byte for byte, so ill-formed UTF-8 that a binary reader accepts cannot throw
static bool same_value(const json& lhs, const json& rhs)
{
return lhs.dump(-1, ' ', false, json::error_handler_t::keep) == rhs.dump(-1, ' ', false, json::error_handler_t::keep);
}
// see http://llvm.org/docs/LibFuzzer.html
extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
{
// step 0: parse input without exceptions
// step 0: parse input without exceptions; a parse error must then be
// reported as a discarded value, never thrown
json j_noexcept;
bool noexcept_threw = false;
json const j_noexcept = parse_without_exceptions([&] { return json::parse(data, data + size, nullptr, false); }, noexcept_threw);
try
{
j_noexcept = json::parse(data, data + size, nullptr, false);
}
catch (const json::parse_error&)
{
assert(false);
}
catch (const json::exception&)
{
// type and out-of-range errors are not parse errors and still throw
noexcept_threw = true;
}
// whether step 1 succeeded; if not, the catch blocks below check that
// step 0 failed, too
bool parsed = false;
+32 -3
View File
@@ -21,16 +21,45 @@ The provided function `LLVMFuzzerTestOneInput` can be used in different fuzzer
drivers.
*/
#include "fuzzer_common.hpp"
#include <cassert>
#include <nlohmann/json.hpp>
// the round-trip checks below are assertions; NDEBUG would compile them away
#ifdef NDEBUG
#error "the fuzzer drivers must be built without NDEBUG"
#endif
using json = nlohmann::json;
// compares dumps rather than values, because NaN != NaN; keep writes strings
// byte for byte, so ill-formed UTF-8 that a binary reader accepts cannot throw
static bool same_value(const json& lhs, const json& rhs)
{
return lhs.dump(-1, ' ', false, json::error_handler_t::keep) == rhs.dump(-1, ' ', false, json::error_handler_t::keep);
}
// see http://llvm.org/docs/LibFuzzer.html
extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
{
std::vector<uint8_t> const vec1(data, data + size);
// step 0: parse input without exceptions
// step 0: parse input without exceptions; a parse error must then be
// reported as a discarded value, never thrown
json j_noexcept;
bool noexcept_threw = false;
json const j_noexcept = parse_without_exceptions([&] { return json::from_msgpack(vec1, true, false); }, noexcept_threw);
try
{
j_noexcept = json::from_msgpack(vec1, true, false);
}
catch (const json::parse_error&)
{
assert(false);
}
catch (const json::exception&)
{
// type and out-of-range errors are not parse errors and still throw
noexcept_threw = true;
}
// whether step 1 succeeded; if not, the catch blocks below check that
// step 0 failed, too
bool parsed = false;
+32 -3
View File
@@ -30,16 +30,45 @@ The provided function `LLVMFuzzerTestOneInput` can be used in different fuzzer
drivers.
*/
#include "fuzzer_common.hpp"
#include <cassert>
#include <nlohmann/json.hpp>
// the round-trip checks below are assertions; NDEBUG would compile them away
#ifdef NDEBUG
#error "the fuzzer drivers must be built without NDEBUG"
#endif
using json = nlohmann::json;
// compares dumps rather than values, because NaN != NaN; keep writes strings
// byte for byte, so ill-formed UTF-8 that a binary reader accepts cannot throw
static bool same_value(const json& lhs, const json& rhs)
{
return lhs.dump(-1, ' ', false, json::error_handler_t::keep) == rhs.dump(-1, ' ', false, json::error_handler_t::keep);
}
// see http://llvm.org/docs/LibFuzzer.html
extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
{
std::vector<uint8_t> const vec1(data, data + size);
// step 0: parse input without exceptions
// step 0: parse input without exceptions; a parse error must then be
// reported as a discarded value, never thrown
json j_noexcept;
bool noexcept_threw = false;
json const j_noexcept = parse_without_exceptions([&] { return json::from_ubjson(vec1, true, false); }, noexcept_threw);
try
{
j_noexcept = json::from_ubjson(vec1, true, false);
}
catch (const json::parse_error&)
{
assert(false);
}
catch (const json::exception&)
{
// type and out-of-range errors are not parse errors and still throw
noexcept_threw = true;
}
// whether step 1 succeeded; if not, the catch blocks below check that
// step 0 failed, too
bool parsed = false;
-51
View File
@@ -1,51 +0,0 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
// code shared by the fuzzer drivers tests/src/fuzzer-parse_*.cpp
#pragma once
#include <cassert> // assert
#include <nlohmann/json.hpp>
using nlohmann::json; // NOLINT(google-global-names-in-headers): shared by all fuzzer drivers
// the round-trip checks of the drivers are assertions; NDEBUG would compile them away
#ifdef NDEBUG
#error "the fuzzer drivers must be built without NDEBUG"
#endif
// compares dumps rather than values, because NaN != NaN; keep writes strings
// byte for byte, so ill-formed UTF-8 that a binary reader accepts cannot throw
inline bool same_value(const json& lhs, const json& rhs)
{
return lhs.dump(-1, ' ', false, json::error_handler_t::keep) == rhs.dump(-1, ' ', false, json::error_handler_t::keep);
}
// step 0 of each driver: parse the input without exceptions; a parse error
// must then be reported as a discarded value, never thrown. Type and
// out-of-range errors are not parse errors and still throw; then @a threw is
// set and null is returned.
template<typename Parse>
json parse_without_exceptions(Parse parse, bool& threw)
{
threw = false;
try
{
return parse();
}
catch (const json::parse_error&)
{
assert(false);
}
catch (const json::exception&)
{
threw = true;
}
return {};
}
-185
View File
@@ -1,185 +0,0 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#pragma once
#include <cstddef> // size_t
#include <cstdint> // uint8_t (via json::binary_t)
#include <limits> // numeric_limits
#include <string> // string, to_string
#include <vector> // vector
#include <nlohmann/json.hpp>
// the loggers are defined in this header only, so their vtables are emitted in
// every test that includes it
#if defined(__clang__)
#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Wweak-vtables"
#endif
namespace utils
{
/// a SAX event consumer that records every event it receives as a
/// human-readable string, used by the deserialization tests to check the
/// exact sequence of SAX events a parse run produces
struct SaxEventLogger : public nlohmann::json_sax<nlohmann::json>
{
using json = nlohmann::json;
bool null() override
{
events.emplace_back("null()");
return true;
}
bool boolean(bool val) override
{
events.emplace_back(val ? "boolean(true)" : "boolean(false)");
return true;
}
bool number_integer(json::number_integer_t val) override
{
events.push_back("number_integer(" + std::to_string(val) + ")");
return true;
}
bool number_unsigned(json::number_unsigned_t val) override
{
events.push_back("number_unsigned(" + std::to_string(val) + ")");
return true;
}
bool number_float(json::number_float_t /*val*/, const std::string& s) override
{
events.push_back("number_float(" + s + ")");
return true;
}
bool string(std::string& val) override
{
events.push_back("string(" + val + ")");
return true;
}
bool binary(json::binary_t& val) override
{
std::string binary_contents = "binary(";
std::string comma_space;
for (const auto b : val)
{
binary_contents.append(comma_space);
binary_contents.append(std::to_string(static_cast<int>(b)));
comma_space = ", ";
}
binary_contents.append(")");
events.push_back(binary_contents);
return true;
}
bool start_object(std::size_t elements) override
{
if (elements == (std::numeric_limits<std::size_t>::max)())
{
events.emplace_back("start_object()");
}
else
{
events.push_back("start_object(" + std::to_string(elements) + ")");
}
return true;
}
bool key(std::string& val) override
{
events.push_back("key(" + val + ")");
return true;
}
bool end_object() override
{
events.emplace_back("end_object()");
return true;
}
bool start_array(std::size_t elements) override
{
if (elements == (std::numeric_limits<std::size_t>::max)())
{
events.emplace_back("start_array()");
}
else
{
events.push_back("start_array(" + std::to_string(elements) + ")");
}
return true;
}
bool end_array() override
{
events.emplace_back("end_array()");
return true;
}
bool parse_error(std::size_t position, const std::string& /*last_token*/, const json::exception& /*ex*/) override
{
errored = true;
events.push_back("parse_error(" + std::to_string(position) + ")");
return false;
}
std::vector<std::string> events {}; // NOLINT(readability-redundant-member-init)
bool errored = false;
};
struct SaxEventLoggerExitAfterStartObject : public SaxEventLogger
{
bool start_object(std::size_t elements) override
{
if (elements == (std::numeric_limits<std::size_t>::max)())
{
events.emplace_back("start_object()");
}
else
{
events.push_back("start_object(" + std::to_string(elements) + ")");
}
return false;
}
};
struct SaxEventLoggerExitAfterKey : public SaxEventLogger
{
bool key(std::string& val) override
{
events.push_back("key(" + val + ")");
return false;
}
};
struct SaxEventLoggerExitAfterStartArray : public SaxEventLogger
{
bool start_array(std::size_t elements) override
{
if (elements == (std::numeric_limits<std::size_t>::max)())
{
events.emplace_back("start_array()");
}
else
{
events.push_back("start_array(" + std::to_string(elements) + ")");
}
return false;
}
};
} // namespace utils
#if defined(__clang__)
#pragma clang diagnostic pop
#endif
+11 -11
View File
@@ -38,7 +38,7 @@ TEST_CASE("Binary Formats")
const auto ubjson_2_size = json::to_ubjson(j, true).size();
const auto ubjson_3_size = json::to_ubjson(j, true, true).size();
CHECK(json_size == 2090303);
CHECK(json_size == 2090234);
CHECK(bjdata_1_size == 1112030);
CHECK(bjdata_2_size == 1224148);
CHECK(bjdata_3_size == 1224148);
@@ -51,16 +51,16 @@ TEST_CASE("Binary Formats")
CHECK(ubjson_3_size == 1169069);
CHECK((100.0 * double(json_size) / double(json_size)) == Approx(100.0));
CHECK((100.0 * double(bjdata_1_size) / double(json_size)) == Approx(53.199));
CHECK((100.0 * double(bjdata_2_size) / double(json_size)) == Approx(58.563));
CHECK((100.0 * double(bjdata_3_size) / double(json_size)) == Approx(58.563));
CHECK((100.0 * double(bon8_size) / double(json_size)) == Approx(50.509));
CHECK((100.0 * double(bson_size) / double(json_size)) == Approx(85.849));
CHECK((100.0 * double(cbor_size) / double(json_size)) == Approx(50.497));
CHECK((100.0 * double(msgpack_size) / double(json_size)) == Approx(50.526));
CHECK((100.0 * double(ubjson_1_size) / double(json_size)) == Approx(53.199));
CHECK((100.0 * double(ubjson_2_size) / double(json_size)) == Approx(58.563));
CHECK((100.0 * double(ubjson_3_size) / double(json_size)) == Approx(55.928));
CHECK((100.0 * double(bjdata_1_size) / double(json_size)) == Approx(53.201));
CHECK((100.0 * double(bjdata_2_size) / double(json_size)) == Approx(58.565));
CHECK((100.0 * double(bjdata_3_size) / double(json_size)) == Approx(58.565));
CHECK((100.0 * double(bon8_size) / double(json_size)) == Approx(50.511));
CHECK((100.0 * double(bson_size) / double(json_size)) == Approx(85.853));
CHECK((100.0 * double(cbor_size) / double(json_size)) == Approx(50.499));
CHECK((100.0 * double(msgpack_size) / double(json_size)) == Approx(50.528));
CHECK((100.0 * double(ubjson_1_size) / double(json_size)) == Approx(53.201));
CHECK((100.0 * double(ubjson_2_size) / double(json_size)) == Approx(58.565));
CHECK((100.0 * double(ubjson_3_size) / double(json_size)) == Approx(55.930));
}
SECTION("twitter.json")
+553 -106
View File
@@ -13,16 +13,20 @@
using nlohmann::json;
#include <array> // array
#include <cfloat> // FLT_EVAL_METHOD
#include <cstdint> // uint32_t, uint64_t
#include <cstdio> // snprintf
#include <cstdlib> // strtod
#include <cstring> // memcpy
#include <limits> // numeric_limits
#include <map> // map
#include <random> // mt19937
#include <sstream> // stringstream
#include <string> // string
#include <utility> // pair
#include <vector> // vector
#include "float_hard_cases.hpp"
namespace
{
// shortcut to scan a string literal
@@ -258,7 +262,7 @@ TEST_CASE("lexer number fast path")
"123456789012345678901234567890", // huge -> float
"0.30000000000000004", "2.2250738585072014e-308", "1e308",
// high-precision / wide-exponent values that exercise the
// std::from_chars (Eisel-Lemire) path beyond the Clinger subset
// Eisel-Lemire path beyond the Clinger subset
"1.7976931348623157e308", "1.2345678901234567e-250",
"9007199254740993", "5e-324", "1e-320"
};
@@ -280,20 +284,18 @@ TEST_CASE("lexer number fast path")
}
}
SECTION("significant-digit gate for the Clinger fast path")
SECTION("significant digits around Clinger's fast path")
{
// Clinger's fast path needs a significand below 2^53, so it cannot
// succeed once the mantissa has 17 or more significant digits (the
// significand would be at least 10^16). The lexer skips the attempt
// there. That is only allowed to save work: every value must still come
// out bit-exactly, and both scanners must agree. In particular the gate
// must not fire for tokens whose leading zeros merely look like extra
// digits - "0.1234567890123456" has 16 significant digits, not 17.
// Clinger's fast path needs a significand of at most 2^53, which
// tokens with 17 or more significant digits exceed. The conversion
// splits the token at the positions the scanners recorded, so leading
// zeros must not count as digits - "0.1234567890123456" has 16
// significant digits, not 17 - and both scanners must agree.
const std::vector<std::string> numbers =
{
"1234567890123456", // 16 significant digits
"12345678901234567", // 17 -> attempt skipped
"123456789012345678", // 18 -> attempt skipped
"12345678901234567", // 17
"123456789012345678", // 18
"0.1234567890123456", // 16: the leading "0" is not significant
"0.12345678901234567", // 17
"0.00000000000000001", // 1, in a long token
@@ -664,46 +666,323 @@ TEST_CASE("lexer string fast path")
}
}
TEST_CASE("parse_float_fast declines what it cannot convert exactly")
TEST_CASE("lexer escape fast path")
{
// The lexer only hands well-formed numbers to parse_float_fast, so the
// malformed ones below can only be passed to it directly. Declining is
// always safe: the caller then falls back to a slower, exact conversion.
const auto fast = [](const std::string & s, double & out)
// json::accept() never throws, so this section stays covered without
// exceptions; it pins which of the cases below are valid/invalid and
// checks the contiguous and streaming paths agree on that classification.
SECTION("accept() parity")
{
return nlohmann::detail::parse_float_fast(s.data(), s.data() + s.size(), out);
const std::vector<std::pair<std::string, bool>> cases =
{
{"\\u0041", true}, {"\\u00e4", true}, {"\\u00E4", true},
{"\\uD83D\\uDE00", true},
{"\\u12", false}, {"\\u12G4", false}, {"\\uXYZW", false},
{"\\uD800", false}, {"\\uD800A", false}, {"\\uD800\\u0041", false},
{"\\uDC00", false}, {"\\u", false}
};
for (const auto& c : cases)
{
for (const std::size_t offset :
{
std::size_t{0}, std::size_t{9}
})
{
const std::string doc = "[\"" + std::string(offset, 'a') + c.first + "\"]";
CAPTURE(doc)
CHECK(json::accept(doc) == c.second);
std::stringstream ss(doc);
CHECK(json::accept(ss) == c.second);
}
}
}
#if !defined(JSON_NOEXCEPTION)
// the full outcome of parsing @a doc: the parsed value, or the exact
// error message, so a mismatch in either is caught
const auto outcome = [](const std::string & doc, bool streaming) -> std::string
{
try
{
if (streaming)
{
std::stringstream ss(doc);
const json j = json::parse(ss);
return j.dump();
}
const json j = json::parse(doc);
return j.dump();
}
catch (const json::exception& e)
{
return {e.what()};
}
};
double out = 0;
#if defined(FLT_EVAL_METHOD) && FLT_EVAL_METHOD != 0
// without true double precision, the fast path declines everything
CHECK_FALSE(fast("1.5", out));
#else
CHECK(fast("1.5", out));
CHECK(out == 1.5);
CHECK(fast("+2.5e1", out));
CHECK(out == 25.0);
CHECK(fast("-25E-1", out));
CHECK(out == -2.5);
CHECK(fast("1e", out));
CHECK(out == 1.0);
SECTION("contiguous vs streaming parity")
{
const std::vector<std::string> escapes =
{
"\\u0041", // "A"
"\\u00e4", // "ä" (lowercase hex)
"\\u00E4", // "ä" (uppercase hex)
"\\uD83D\\uDE00", // valid surrogate pair (an emoji)
"\\u12", // truncated: only 2 hex digits before the closing quote
"\\u12G4", // invalid hex digit at the 3rd position
"\\uXYZW", // all 4 bytes invalid
"\\uD800", // lone high surrogate, string ends right after
"\\uD800A", // high surrogate not followed by another \u escape
"\\uD800\\u0041", // high surrogate followed by \u, but not a low surrogate
"\\uDC00", // lone low surrogate
"\\u", // '\u' with nothing after (closing quote right away)
};
// once at the start of the string and once past the first 8-byte SWAR
// word of the outer string_bulk_run, so the escape is reached both
// right after the opening quote and mid-run
for (const auto& escape : escapes)
{
for (const std::size_t offset :
{
std::size_t{0}, std::size_t{9}
})
{
const std::string doc = "[\"" + std::string(offset, 'a') + escape + "\"]";
CAPTURE(doc)
CHECK(outcome(doc, false) == outcome(doc, true));
}
// the escape is the last thing before end of input: no closing
// quote at all
const std::string truncated_doc = "[\"" + escape;
CAPTURE(truncated_doc)
CHECK(outcome(truncated_doc, false) == outcome(truncated_doc, true));
}
}
SECTION("truncated \\u escape at every distance from the end of input")
{
// ia.bulk_remaining() must correctly report fewer than 4 bytes for
// every possible count of trailing hex-looking bytes (0, 1, 2, or 3)
// before end of input, so the fast path declines and the byte path
// alone reports the "must be followed by 4 hex digits" error, at the
// same position, in every case
for (const std::string& tail :
{
std::string{}, std::string("1"), std::string("12"), std::string("123")
})
{
const std::string doc = "[\"\\u" + tail;
CAPTURE(doc)
CHECK(outcome(doc, false) == outcome(doc, true));
CHECK(outcome(doc, false).find("must be followed by 4 hex digits") != std::string::npos);
}
}
SECTION("invalid hex digit at every position of the 4")
{
// the fast path must decline for *any* invalid byte among the 4, not
// just the first, and the byte path must then stop at exactly that
// position - same as it always has
for (std::size_t bad_pos = 0; bad_pos < 4; ++bad_pos)
{
std::string digits = "1234";
digits[bad_pos] = 'g'; // not a hex digit
const std::string doc = "[\"\\u" + digits + "\"]";
CAPTURE(doc)
CHECK(outcome(doc, false) == outcome(doc, true));
CHECK(outcome(doc, false).find("must be followed by 4 hex digits") != std::string::npos);
}
}
SECTION("random escapes")
{
// A seeded PRNG builds the 4 bytes following `\u` from a mix of hex
// digits and non-hex bytes, at varying distances from the start of
// the string, to compare the two scanners on many more shapes than
// are practical to enumerate by hand.
std::mt19937 gen(7654321); // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed)
const std::string hex_alphabet = "0123456789AaBbCcDdEeFf";
std::uniform_int_distribution<std::size_t> pick_hex(0, hex_alphabet.size() - 1);
std::uniform_int_distribution<int> pick_byte(1, 255); // never NUL
std::uniform_int_distribution<int> pick_is_hex(0, 4); // 4-in-5 chance of a hex digit
std::uniform_int_distribution<std::size_t> pick_offset(0, 12);
std::vector<std::string> mismatches;
for (int iter = 0; iter < 3000; ++iter)
{
std::string digits;
for (int i = 0; i < 4; ++i)
{
if (pick_is_hex(gen) != 0)
{
digits += hex_alphabet[pick_hex(gen)];
}
else
{
char c = static_cast<char>(pick_byte(gen));
if (c == '"' || c == '\\')
{
// keep the string well-formed apart from the escape
// itself, so any mismatch is attributable to the \u
// handling and not to an unrelated quote/escape
c = 'z';
}
digits += c;
}
}
const std::string doc = "[\"" + std::string(pick_offset(gen), 'a') + "\\u" + digits + "\"]";
if (outcome(doc, false) != outcome(doc, true))
{
mismatches.push_back(doc);
}
}
CAPTURE(mismatches)
CHECK(mismatches.empty());
}
#endif
}
// not a number
CHECK_FALSE(fast("", out));
CHECK_FALSE(fast("-", out));
CHECK_FALSE(fast(".", out));
CHECK_FALSE(fast("1.2.3", out));
CHECK_FALSE(fast("1x", out));
CHECK_FALSE(fast("1e+", out));
CHECK_FALSE(fast("1e1x", out));
namespace
{
// the index of the decimal point (or npos) and of the end of the mantissa of a
// number token, which the lexer records while scanning it
std::pair<std::size_t, std::size_t> float_token_layout(const std::string& s)
{
std::size_t dot = std::string::npos;
std::size_t mantissa_end = s.size();
for (std::size_t i = 0; i < s.size(); ++i)
{
if (s[i] == '.')
{
dot = i;
}
else if (s[i] == 'e' || s[i] == 'E')
{
mantissa_end = i;
break;
}
}
return {dot, mantissa_end};
}
// numbers that are not represented exactly on the fast path
CHECK_FALSE(fast("12345678901234567890", out));
CHECK_FALSE(fast("1e10000", out));
CHECK_FALSE(fast("9007199254740993", out));
CHECK_FALSE(fast("1e23", out));
CHECK_FALSE(fast("1e-23", out));
template<typename FloatType>
FloatType parse_native(const std::string& s)
{
const auto layout = float_token_layout(s);
return nlohmann::detail::parse_float_native<FloatType>(s.data(), s.data() + s.size(), layout.first, layout.second);
}
std::uint64_t bits_of(double d)
{
std::uint64_t b = 0;
std::memcpy(&b, &d, sizeof(b));
return b;
}
std::uint32_t bits_of(float f)
{
std::uint32_t b = 0;
std::memcpy(&b, &f, sizeof(b));
return b;
}
std::uint64_t native_bits64(const std::string& s)
{
return bits_of(parse_native<double>(s));
}
std::uint32_t native_bits32(const std::string& s)
{
return bits_of(parse_native<float>(s));
}
} // namespace
TEST_CASE("parse_float_native rounds correctly")
{
SECTION("double")
{
CHECK(native_bits64("1.5") == 0x3FF8000000000000u);
CHECK(native_bits64("0.1") == 0x3FB999999999999Au);
CHECK(native_bits64("-0.0") == 0x8000000000000000u);
CHECK(native_bits64("0e999999999999999999999") == 0u);
// 2^53 + 1 is exactly between two doubles: ties to even, unless more digits follow
CHECK(native_bits64("9007199254740993") == 0x4340000000000000u);
CHECK(native_bits64("9007199254740993.0000000000000000001") == 0x4340000000000001u);
CHECK(native_bits64("9007199254740992.9999999999999999999") == 0x4340000000000000u);
// 1 + 2^-53 exactly (a tie), and one unit in the 55th digit around it
CHECK(native_bits64("1.00000000000000011102230246251565404236316680908203125") == 0x3FF0000000000000u);
CHECK(native_bits64("1.00000000000000011102230246251565404236316680908203126") == 0x3FF0000000000001u);
CHECK(native_bits64("1.00000000000000011102230246251565404236316680908203124") == 0x3FF0000000000000u);
// subnormal and overflow boundaries
CHECK(native_bits64("2.4703282292062327e-324") == 0u);
CHECK(native_bits64("2.4703282292062328e-324") == 1u);
CHECK(native_bits64("2.2250738585072011e-308") == 0x000FFFFFFFFFFFFFu);
CHECK(native_bits64("2.2250738585072012e-308") == 0x0010000000000000u);
CHECK(native_bits64("1.7976931348623157e308") == 0x7FEFFFFFFFFFFFFFu);
CHECK(native_bits64("1.7976931348623159e308") == 0x7FF0000000000000u);
CHECK(native_bits64("-1e400") == 0xFFF0000000000000u);
CHECK(native_bits64("-1e-400") == 0x8000000000000000u);
// exponents and zeros far beyond the range cancel out
CHECK(native_bits64("0." + std::string(1000, '0') + "1e1001") == 0x3FF0000000000000u);
CHECK(native_bits64("1" + std::string(1000, '0') + "e-1000") == 0x3FF0000000000000u);
CHECK(native_bits64("1e-99999999999999999999999") == 0u);
CHECK(native_bits64("1E+99999999999999999999999") == 0x7FF0000000000000u);
// more digits than any midpoint has (769): only whether a nonzero digit follows matters
const std::string tie = "1.00000000000000011102230246251565404236316680908203125";
CHECK(native_bits64(tie + std::string(800, '0')) == 0x3FF0000000000000u);
CHECK(native_bits64(tie + std::string(800, '0') + "1") == 0x3FF0000000000001u);
}
SECTION("float")
{
CHECK(native_bits32("1.5") == 0x3FC00000u);
CHECK(native_bits32("0.1") == 0x3DCCCCCDu);
CHECK(native_bits32("-0.0") == 0x80000000u);
// 2^24 + 1 is exactly between two floats
CHECK(native_bits32("16777217") == 0x4B800000u);
CHECK(native_bits32("16777217.000000000000000000001") == 0x4B800001u);
CHECK(native_bits32("16777218.999999999999999999999") == 0x4B800001u);
CHECK(native_bits32("16777219") == 0x4B800002u);
// subnormal and overflow boundaries
CHECK(native_bits32("3.4028235677973366e38") == 0x7F7FFFFFu);
CHECK(native_bits32("3.4028235677973367e38") == 0x7F800000u);
CHECK(native_bits32("7.006492321624085e-46") == 0u);
CHECK(native_bits32("7.006492321624086e-46") == 1u);
CHECK(native_bits32("1.1754942e-38") == 0x007FFFFFu);
CHECK(native_bits32("-1.17549435e-38") == 0x80800000u);
CHECK(native_bits32("1e39") == 0x7F800000u);
CHECK(native_bits32("-1e-50") == 0x80000000u);
// not rounded through double: its double would round to another float
CHECK(native_bits32("1.00000005960464477539062500000000001") == 0x3F800001u);
CHECK(native_bits32("9007199254740993") == 0x5A000000u);
}
SECTION("the conversion shared with other parsers")
{
// convert_float() gives the lexer's results, for every type
const std::vector<std::string> tokens =
{
"0", "-0.0", "1.5", "0.1", "1e-400", "-2.5E+3", "123456789012345678901234567890",
"9007199254740993.0000000000000000001", "4.9406564584124654e-324"
};
using float_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int64_t, std::uint64_t, float>;
using long_double_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int64_t, std::uint64_t, long double>;
for (const auto& t : tokens)
{
CAPTURE(t)
const auto layout = float_token_layout(t);
const char* const first = t.data();
const char* const last = first + t.size();
const auto d = nlohmann::detail::convert_float<double>(first, last, layout.first, layout.second);
const auto f = nlohmann::detail::convert_float<float>(first, last, layout.first, layout.second);
const auto ld = nlohmann::detail::convert_float<long double>(first, last, layout.first, layout.second);
CHECK(bits_of(d) == bits_of(json::parse(t).get<double>()));
CHECK(bits_of(f) == bits_of(float_json::parse(t).get<float>()));
CHECK(ld == long_double_json::parse(t).get<long double>());
}
}
}
namespace
@@ -807,40 +1086,6 @@ std::size_t big_bit_length(const big_uint& a)
}
return n;
}
std::uint64_t bits_of(double d)
{
std::uint64_t b = 0;
std::memcpy(&b, &d, sizeof(b));
return b;
}
bool eisel_lemire(const std::string& s, double& out)
{
return nlohmann::detail::parse_float_eisel_lemire(s.data(), s.data() + s.size(), out);
}
// significant digits of a token, without trailing zeros
std::size_t significant_digits(const std::string& s)
{
std::string digits;
for (const char c : s)
{
if (c == 'e' || c == 'E')
{
break;
}
if (c >= '0' && c <= '9' && !(digits.empty() && c == '0'))
{
digits += c;
}
}
while (!digits.empty() && digits.back() == '0')
{
digits.pop_back();
}
return digits.size();
}
} // namespace
TEST_CASE("Eisel-Lemire float conversion")
@@ -1268,26 +1513,33 @@ TEST_CASE("Eisel-Lemire float conversion")
for (const auto& c : known)
{
CAPTURE(c.first)
double out = 0;
if (eisel_lemire(c.first, out))
{
CHECK(bits_of(out) == c.second);
}
else
{
// only tokens with more than 19 significant digits are left to
// strtod: those whose value lies too close to a tie
CHECK(significant_digits(c.first) > 19);
}
CHECK(native_bits64(c.first) == c.second);
}
}
SECTION("binary32")
{
using binary32 = nlohmann::detail::ieee_binary_format<24>;
CHECK(nlohmann::detail::eisel_lemire<binary32>(0, 1) == 0x3F800000u);
CHECK(nlohmann::detail::eisel_lemire<binary32>(-1, 1) == 0x3DCCCCCDu);
CHECK(nlohmann::detail::eisel_lemire<binary32>(-1, 15) == 0x3FC00000u);
CHECK(nlohmann::detail::eisel_lemire<binary32>(0, 16777217) == 0x4B800000u); // tie, to even
CHECK(nlohmann::detail::eisel_lemire<binary32>(0, 16777219) == 0x4B800002u); // tie, to even
CHECK(nlohmann::detail::eisel_lemire<binary32>(-45, 1) == 0x00000001u);
CHECK(nlohmann::detail::eisel_lemire<binary32>(-46, 7) == 0x00000000u);
CHECK(nlohmann::detail::eisel_lemire<binary32>(-46, 8) == 0x00000001u);
CHECK(nlohmann::detail::eisel_lemire<binary32>(-65, 9999999999999999999u) == 0x00000000u);
CHECK(nlohmann::detail::eisel_lemire<binary32>(20, 3402823466385288598u) == 0x7F7FFFFFu);
CHECK(nlohmann::detail::eisel_lemire<binary32>(20, 3402823669209384635u) == 0x7F800000u);
CHECK(nlohmann::detail::eisel_lemire<binary32>(39, 1) == 0x7F800000u);
CHECK(nlohmann::detail::eisel_lemire<binary32>(-5, 0) == 0x00000000u);
}
SECTION("round trip")
{
// every double written by to_chars and read back, also with trailing
// digits that make the token longer than 19 digits
// every double written by to_chars and read back, and its 17-digit
// form with trailing digits that make the token longer than 19 digits
std::uint64_t state = 5295;
std::size_t declined = 0;
for (int i = 0; i < 200000; ++i)
{
state ^= state << 13u;
@@ -1309,30 +1561,51 @@ TEST_CASE("Eisel-Lemire float conversion")
const char* end = nlohmann::detail::to_chars(buffer.data(), buffer.data() + buffer.size(), d);
const std::string token(buffer.data(), static_cast<std::size_t>(end - buffer.data()));
CAPTURE(token)
double out = 0;
REQUIRE(eisel_lemire(token, out));
CHECK(bits_of(out) == b);
CHECK(native_bits64(token) == b);
// insert digits before the exponent: the value moves by far less
// than the distance to the rounding boundary, so it must not change
std::string longer = token;
// insert digits before the exponent of the 17-digit form: that
// form lies strictly inside the rounding interval of the double
// (the shortest one may lie on its boundary), and the digits move
// it by far less than the distance to the boundary, so the value
// must not change
std::array<char, 64> digits17{};
static_cast<void>(std::snprintf(digits17.data(), digits17.size(), "%.17g", d)); // NOLINT(cppcoreguidelines-pro-type-vararg,hicpp-vararg)
std::string longer = digits17.data();
const std::size_t e = longer.find('e');
const std::size_t dot = longer.find('.');
const std::string extra = dot == std::string::npos ? ".000000000000000000001" : "000000000000000000001";
longer.insert(e == std::string::npos ? longer.size() : e, extra);
CAPTURE(longer)
if (eisel_lemire(longer, out))
CHECK(native_bits64(longer) == b);
}
}
SECTION("round trip, binary32")
{
std::uint32_t state = 5295;
for (int i = 0; i < 100000; ++i)
{
state ^= state << 13u;
state ^= state >> 17u;
state ^= state << 5u;
std::uint32_t b = state;
if ((b & 0x7F800000u) == 0x7F800000u)
{
CHECK(bits_of(out) == b);
continue; // infinity or NaN
}
else
if (i % 4 == 0)
{
// w and w + 1 round differently: only when the value is very
// close to a rounding boundary
++declined;
b &= 0x807FFFFFu; // subnormals
}
float f = 0;
std::memcpy(&f, &b, sizeof(f));
std::array<char, 64> buffer{};
const char* end = nlohmann::detail::to_chars(buffer.data(), buffer.data() + buffer.size(), f);
const std::string token(buffer.data(), static_cast<std::size_t>(end - buffer.data()));
CAPTURE(token)
CHECK(native_bits32(token) == b);
}
CHECK(declined < 1000); // 107 of the 200,000
}
SECTION("used by the lexer")
@@ -1346,3 +1619,177 @@ TEST_CASE("Eisel-Lemire float conversion")
"[json.exception.out_of_range.406] number overflow parsing '1.7976931348623159e308'", json::out_of_range&);
}
}
namespace
{
using float_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int64_t, std::uint64_t, float>;
// the bits of the float that parse() gives for a token, via both scanners;
// the value must be the same for both
template<typename Json, typename Bits>
void check_parse(const std::string& token, Bits expected, Bits infinity)
{
std::stringstream stream(token);
if ((expected & ~(Bits{1} << ((8 * sizeof(Bits)) - 1))) == infinity)
{
Json _;
CHECK_THROWS_WITH_AS(_ = Json::parse(token), ("[json.exception.out_of_range.406] number overflow parsing '" + token + "'").c_str(), typename Json::out_of_range&);
CHECK_THROWS_WITH_AS(_ = Json::parse(stream), ("[json.exception.out_of_range.406] number overflow parsing '" + token + "'").c_str(), typename Json::out_of_range&);
return;
}
const Json contiguous = Json::parse(token);
const Json streamed = Json::parse(stream);
if (contiguous.is_number_float()) // not an integer that fits
{
CHECK(bits_of(contiguous.template get<typename Json::number_float_t>()) == expected);
CHECK(bits_of(streamed.template get<typename Json::number_float_t>()) == expected);
}
else
{
CHECK(streamed.is_number_integer());
}
}
} // namespace
TEST_CASE("float conversion of hard cases")
{
// see float_hard_cases.hpp
for (const auto& c : float_hard_cases::cases())
{
const std::string token = c.token;
CAPTURE(token)
CHECK(native_bits64(token) == c.bits64);
CHECK(native_bits32(token) == c.bits32);
check_parse<json>(token, c.bits64, std::uint64_t{0x7FF0000000000000u});
check_parse<float_json>(token, c.bits32, std::uint32_t{0x7F800000u});
}
}
TEST_CASE("float overflow and underflow in the parser")
{
SECTION("double")
{
check_parse<json>("1.7976931348623157e308", std::uint64_t{0x7FEFFFFFFFFFFFFFu}, std::uint64_t{0x7FF0000000000000u});
check_parse<json>("1.7976931348623159e308", std::uint64_t{0x7FF0000000000000u}, std::uint64_t{0x7FF0000000000000u});
check_parse<json>("-1e309", std::uint64_t{0xFFF0000000000000u}, std::uint64_t{0x7FF0000000000000u});
check_parse<json>("1" + std::string(400, '0'), std::uint64_t{0x7FF0000000000000u}, std::uint64_t{0x7FF0000000000000u});
check_parse<json>("1e99999999999999999999", std::uint64_t{0x7FF0000000000000u}, std::uint64_t{0x7FF0000000000000u});
// an underflow gives a zero with the sign of the token
check_parse<json>("1e-400", std::uint64_t{0}, std::uint64_t{0x7FF0000000000000u});
check_parse<json>("-1e-400", std::uint64_t{0x8000000000000000u}, std::uint64_t{0x7FF0000000000000u});
check_parse<json>("-2.4703282292062327e-324", std::uint64_t{0x8000000000000000u}, std::uint64_t{0x7FF0000000000000u});
check_parse<json>("0." + std::string(400, '0') + "1", std::uint64_t{0}, std::uint64_t{0x7FF0000000000000u});
}
SECTION("float")
{
check_parse<float_json>("3.4028234e38", std::uint32_t{0x7F7FFFFFu}, std::uint32_t{0x7F800000u});
check_parse<float_json>("3.4028236e38", std::uint32_t{0x7F800000u}, std::uint32_t{0x7F800000u});
check_parse<float_json>("-1e39", std::uint32_t{0xFF800000u}, std::uint32_t{0x7F800000u});
check_parse<float_json>("1e-46", std::uint32_t{0}, std::uint32_t{0x7F800000u});
check_parse<float_json>("-1e-46", std::uint32_t{0x80000000u}, std::uint32_t{0x7F800000u});
check_parse<float_json>("-7.006492321624085e-46", std::uint32_t{0x80000000u}, std::uint32_t{0x7F800000u});
check_parse<float_json>("-7.006492321624086e-46", std::uint32_t{0x80000001u}, std::uint32_t{0x7F800000u});
}
}
TEST_CASE("string scanning kernels")
{
// the word-at-a-time kernels must stop exactly where a byte-by-byte scan
// stops, for any content, length, and alignment
const auto reference_special = [](const unsigned char* data, std::size_t n)
{
std::size_t i = 0;
while (i < n && !nlohmann::detail::is_string_special(data[i]))
{
++i;
}
return i;
};
const auto reference_copyable = [](const unsigned char* data, std::size_t n)
{
std::size_t i = 0;
while (i < n && nlohmann::detail::is_ascii_copyable(data[i]))
{
++i;
}
return i;
};
const auto reference_bulk_run = [](const unsigned char* data, std::size_t n)
{
std::size_t i = 0;
while (i < n)
{
if (data[i] < 0x80u)
{
if (nlohmann::detail::is_string_special(data[i]))
{
break;
}
++i;
continue;
}
const std::size_t seq = nlohmann::detail::validate_one_utf8(data + i, n - i);
if (seq == 0)
{
break;
}
i += seq;
}
return i;
};
// pieces: ordinary ASCII, stops, DEL, well-formed sequences of every
// length, and ill-formed or truncated ones
const std::vector<std::string> pieces =
{
"a", "Z", " ", "~", "0123456789", "\"", "\\", std::string(1, '\0'), "\n", "\x1F", "\x7F",
"\xC3\xA4", "\xE2\x82\xAC", "\xE6\x97\xA5\xE6\x9C\xAC", "\xF0\x9F\x98\x80", "\xED\x9F\xBF",
"\x80", "\xC0\x80", "\xC3", "\xE2\x82", "\xED\xA0\x80", "\xF4\x90\x80\x80", "\xFF",
};
std::uint64_t state = 5295;
const auto next = [&state]()
{
state ^= state << 13u;
state ^= state >> 7u;
state ^= state << 17u;
return state;
};
// the upper half as a 32-bit value: converts to std::size_t implicitly on
// every platform (a cast of std::uint64_t is useless where both are the
// same type, and required where std::size_t is 32 bits wide)
const auto next_small = [&next]()
{
return static_cast<std::uint32_t>(next() >> 32u);
};
for (int round = 0; round < 100000; ++round)
{
// mostly ordinary text, so that runs span several words
std::string text(next_small() % 8u, '.');
const std::size_t count = next_small() % 12u;
for (std::size_t k = 0; k < count; ++k)
{
const std::size_t p = (next() % 4 == 0) ? next_small() % pieces.size() : 0;
text += pieces[p];
text += std::string(next_small() % 10u, 'x');
}
const auto* data = reinterpret_cast<const unsigned char*>(text.data()); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
for (std::size_t offset = 0; offset < 3 && offset <= text.size(); ++offset)
{
const std::size_t n = text.size() - offset;
CAPTURE(text)
CAPTURE(offset)
CHECK(nlohmann::detail::find_string_special(data + offset, n) == reference_special(data + offset, n));
CHECK(nlohmann::detail::find_ascii_copyable_run(data + offset, n) == reference_copyable(data + offset, n));
CHECK(nlohmann::detail::scalar_string_bulk_run(data + offset, n) == reference_bulk_run(data + offset, n));
}
}
// the trailing-zero count, whichever implementation the compiler gets
for (int k = 0; k < 64; ++k)
{
const std::uint64_t bit = std::uint64_t{1} << k;
CHECK(nlohmann::detail::count_trailing_zeros(bit) == k);
CHECK(nlohmann::detail::count_trailing_zeros(bit | (bit << 1u) | 0x8000000000000000u) == k);
}
}
+184 -5
View File
@@ -37,15 +37,194 @@ using nlohmann::json;
#include <utility>
#include <vector>
#include "sax_countdown.hpp"
#include "sax_event_loggers.hpp"
#include "test_utils.hpp"
using utils::SaxCountdown;
using utils::SaxEventLogger;
namespace
{
class SaxEventLogger
{
public:
bool null()
{
events.emplace_back("null()");
return true;
}
bool boolean(bool val)
{
events.emplace_back(val ? "boolean(true)" : "boolean(false)");
return true;
}
bool number_integer(json::number_integer_t val)
{
events.push_back("number_integer(" + std::to_string(val) + ")");
return true;
}
bool number_unsigned(json::number_unsigned_t val)
{
events.push_back("number_unsigned(" + std::to_string(val) + ")");
return true;
}
bool number_float(json::number_float_t /*unused*/, const std::string& s)
{
events.push_back("number_float(" + s + ")");
return true;
}
bool string(std::string& val)
{
events.push_back("string(" + val + ")");
return true;
}
bool binary(json::binary_t& val)
{
std::string binary_contents = "binary(";
std::string comma_space;
for (auto b : val)
{
binary_contents.append(comma_space);
binary_contents.append(std::to_string(static_cast<int>(b)));
comma_space = ", ";
}
binary_contents.append(")");
events.push_back(binary_contents);
return true;
}
bool start_object(std::size_t elements)
{
if (elements == (std::numeric_limits<std::size_t>::max)())
{
events.emplace_back("start_object()");
}
else
{
events.push_back("start_object(" + std::to_string(elements) + ")");
}
return true;
}
bool key(std::string& val)
{
events.push_back("key(" + val + ")");
return true;
}
bool end_object()
{
events.emplace_back("end_object()");
return true;
}
bool start_array(std::size_t elements)
{
if (elements == (std::numeric_limits<std::size_t>::max)())
{
events.emplace_back("start_array()");
}
else
{
events.push_back("start_array(" + std::to_string(elements) + ")");
}
return true;
}
bool end_array()
{
events.emplace_back("end_array()");
return true;
}
bool parse_error(std::size_t position, const std::string& /*unused*/, const json::exception& /*unused*/)
{
errored = true;
events.push_back("parse_error(" + std::to_string(position) + ")");
return false;
}
std::vector<std::string> events {}; // NOLINT(readability-redundant-member-init)
bool errored = false;
};
class SaxCountdown : public nlohmann::json::json_sax_t
{
public:
explicit SaxCountdown(const int count) : events_left(count)
{}
bool null() override
{
return events_left-- > 0;
}
bool boolean(bool /*val*/) override
{
return events_left-- > 0;
}
bool number_integer(json::number_integer_t /*val*/) override
{
return events_left-- > 0;
}
bool number_unsigned(json::number_unsigned_t /*val*/) override
{
return events_left-- > 0;
}
bool number_float(json::number_float_t /*val*/, const std::string& /*s*/) override
{
return events_left-- > 0;
}
bool string(std::string& /*val*/) override
{
return events_left-- > 0;
}
bool binary(json::binary_t& /*val*/) override
{
return events_left-- > 0;
}
bool start_object(std::size_t /*elements*/) override
{
return events_left-- > 0;
}
bool key(std::string& /*val*/) override
{
return events_left-- > 0;
}
bool end_object() override
{
return events_left-- > 0;
}
bool start_array(std::size_t /*elements*/) override
{
return events_left-- > 0;
}
bool end_array() override
{
return events_left-- > 0;
}
bool parse_error(std::size_t /*position*/, const std::string& /*last_token*/, const json::exception& /*ex*/) override
{
return false;
}
private:
int events_left = 0;
};
json parser_helper(const std::string& s);
bool accept_helper(const std::string& s);
void comments_helper(const std::string& s);
+147 -7
View File
@@ -36,15 +36,155 @@ using nlohmann::json;
#include <string>
#include <valarray>
#include "sax_event_loggers.hpp"
using utils::SaxEventLogger;
using utils::SaxEventLoggerExitAfterKey;
using utils::SaxEventLoggerExitAfterStartArray;
using utils::SaxEventLoggerExitAfterStartObject;
namespace
{
struct SaxEventLogger : public nlohmann::json_sax<json>
{
bool null() override
{
events.emplace_back("null()");
return true;
}
bool boolean(bool val) override
{
events.emplace_back(val ? "boolean(true)" : "boolean(false)");
return true;
}
bool number_integer(json::number_integer_t val) override
{
events.push_back("number_integer(" + std::to_string(val) + ")");
return true;
}
bool number_unsigned(json::number_unsigned_t val) override
{
events.push_back("number_unsigned(" + std::to_string(val) + ")");
return true;
}
bool number_float(json::number_float_t /*val*/, const std::string& s) override
{
events.push_back("number_float(" + s + ")");
return true;
}
bool string(std::string& val) override
{
events.push_back("string(" + val + ")");
return true;
}
bool binary(json::binary_t& val) override
{
std::string binary_contents = "binary(";
std::string comma_space;
for (auto b : val)
{
binary_contents.append(comma_space);
binary_contents.append(std::to_string(static_cast<int>(b)));
comma_space = ", ";
}
binary_contents.append(")");
events.push_back(binary_contents);
return true;
}
bool start_object(std::size_t elements) override
{
if (elements == (std::numeric_limits<std::size_t>::max)())
{
events.emplace_back("start_object()");
}
else
{
events.push_back("start_object(" + std::to_string(elements) + ")");
}
return true;
}
bool key(std::string& val) override
{
events.push_back("key(" + val + ")");
return true;
}
bool end_object() override
{
events.emplace_back("end_object()");
return true;
}
bool start_array(std::size_t elements) override
{
if (elements == (std::numeric_limits<std::size_t>::max)())
{
events.emplace_back("start_array()");
}
else
{
events.push_back("start_array(" + std::to_string(elements) + ")");
}
return true;
}
bool end_array() override
{
events.emplace_back("end_array()");
return true;
}
bool parse_error(std::size_t position, const std::string& /*last_token*/, const json::exception& /*ex*/) override
{
events.push_back("parse_error(" + std::to_string(position) + ")");
return false;
}
std::vector<std::string> events {}; // NOLINT(readability-redundant-member-init)
};
struct SaxEventLoggerExitAfterStartObject : public SaxEventLogger
{
bool start_object(std::size_t elements) override
{
if (elements == (std::numeric_limits<std::size_t>::max)())
{
events.emplace_back("start_object()");
}
else
{
events.push_back("start_object(" + std::to_string(elements) + ")");
}
return false;
}
};
struct SaxEventLoggerExitAfterKey : public SaxEventLogger
{
bool key(std::string& val) override
{
events.push_back("key(" + val + ")");
return false;
}
};
struct SaxEventLoggerExitAfterStartArray : public SaxEventLogger
{
bool start_array(std::size_t elements) override
{
if (elements == (std::numeric_limits<std::size_t>::max)())
{
events.emplace_back("start_array()");
}
else
{
events.push_back("start_array(" + std::to_string(elements) + ")");
}
return false;
}
};
template <typename T>
class proxy_iterator
{
+25 -8
View File
@@ -260,10 +260,11 @@ struct LocaleSwitchingSax final: public nlohmann::json_sax<json>
TEST_CASE("locale changes between lexer construction and number conversion (#5198)")
{
// The numbers are chosen so that the conversion also takes the strtod
// fallback, which honors the locale that is current at conversion time:
// too many significant digits for Clinger's fast path, an underflow that
// std::from_chars rejects, and a plain value.
// float and double are converted without the locale. A long double that
// is not binary64 can take the strtold fallback, which honors the locale
// that is current at conversion time. The numbers are chosen so that it
// does: too many significant digits for Clinger's fast path, an underflow
// that std::from_chars rejects, and a plain value.
const std::vector<std::string> numbers = {"3.14159265358979323846", "1.5e-400", "12.34", "-0.000123456789012345678"};
std::string text = "[";
for (const auto& n : numbers)
@@ -327,7 +328,8 @@ TEST_CASE("locale changes between lexer construction and number conversion (#519
}
}
// a long double goes through std::strtold unless std::from_chars supports it
// a long double goes through std::strtold unless it is binary64 or
// std::from_chars supports it
{
bool switched = false;
const auto cb = [&](int /*depth*/, long_double_json::parse_event_t event, long_double_json& /*parsed*/) noexcept
@@ -353,8 +355,15 @@ TEST_CASE("locale with a multi-byte decimal point")
{
// Some locales use a decimal point that is not a single character, e.g.
// U+066B ARABIC DECIMAL SEPARATOR (two bytes in UTF-8). It cannot be
// substituted in place for '.', so the strtod fallback stops early. The
// conversion must still terminate rather than retry forever.
// substituted in place for '.', so the strtold fallback (only for long
// double formats other than binary64) converts a copy of the token with
// the whole decimal point instead (#5660). The values must be those of the
// "C" locale.
using long_double_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int64_t, std::uint64_t, long double>;
const char* const long_double_numbers = "[3.14159265358979323846, 1.5e-400, -0.000123456789012345678]";
REQUIRE(std::setlocale(LC_NUMERIC, "C") != nullptr);
const long_double_json expected_long_double = long_double_json::parse(long_double_numbers);
const std::array<const char*, 6> names = {{"ar_EG.UTF-8", "ar_SA.UTF-8", "fa_IR.UTF-8", "ps_AF.UTF-8", "ar_EG", "fa_IR"}};
bool tested = false;
for (const char* name : names)
@@ -372,12 +381,20 @@ TEST_CASE("locale with a multi-byte decimal point")
tested = true;
// too many significant digits for Clinger's fast path, and an underflow
// that std::from_chars rejects: both reach the strtod fallback
// that std::from_chars rejects: double does not depend on the locale
json j;
CHECK_NOTHROW(j = json::parse("[3.14159265358979323846, 1.5e-400, -0.000123456789012345678]"));
CHECK(j.is_array());
CHECK(j[0] == 3.14159265358979323846);
CHECK(j[1] == 0.0);
CHECK(j[2] == -0.000123456789012345678);
CHECK(json::accept("3.14159265358979323846"));
// a long double that reaches the strtold fallback is not truncated
long_double_json ld;
CHECK_NOTHROW(ld = long_double_json::parse(long_double_numbers));
CHECK(ld == expected_long_double);
// a value the locale-independent paths convert is not affected
CHECK(json::parse("12.5") == 12.5);
}
+272 -1
View File
@@ -15,6 +15,23 @@
#include <nlohmann/json.hpp>
using nlohmann::detail::dtoa_impl::reinterpret_bits;
#include <array>
#include <cmath>
#include <cstdint>
#include <cstdio>
#include <cstdlib>
#include <iomanip>
#include <limits>
#include <locale>
#include <random>
#include <sstream>
#include <string>
#include <utility>
#include <vector>
#if defined(JSON_HAS_CPP_17)
#include <charconv>
#endif
namespace
{
float make_float(uint32_t sign_bit, uint32_t biased_exponent, uint32_t significand)
@@ -450,7 +467,7 @@ TEST_CASE("formatting")
check_double( 1.2345e+18, "1.2345e+18" ); // 1.2345e+18 1.2345e+18 1.2345e18
check_double( 1.2345e+19, "1.2345e+19" ); // 1.2345e+19 1.2345e+19 1.2345e19
check_double( 1.2345e+20, "1.2345e+20" ); // 1.2345e+20 1.2345e+20 1.2345e20
check_double( 1.2345e+21, "1.2344999999999999e+21" ); // 1.2345e+21 1.2344999999999999e+21 1.2345e21
check_double( 1.2345e+21, "1.2345e+21" ); // 1.2345e+21 1.2344999999999999e+21 1.2345e21
check_double( 1.2345e+22, "1.2345e+22" ); // 1.2345e+22 1.2345e+22 1.2345e22
}
@@ -514,3 +531,257 @@ TEST_CASE("formatting")
check_integer(1000000000000000000LL, "1000000000000000000");
}
}
namespace
{
// a small unsigned big integer (32-bit limbs, least significant first), to
// recompute the powers of ten of the shortest double conversion
using big = std::vector<std::uint32_t>;
void big_mul_small(big& x, std::uint32_t m)
{
std::uint64_t carry = 0;
for (auto& limb : x)
{
const std::uint64_t v = (static_cast<std::uint64_t>(limb) * m) + carry;
limb = static_cast<std::uint32_t>(v);
carry = v >> 32u;
}
if (carry != 0)
{
x.push_back(static_cast<std::uint32_t>(carry));
}
}
void big_div_small(big& x, std::uint32_t d)
{
std::uint64_t rest = 0;
for (std::size_t i = x.size(); i-- > 0;)
{
const std::uint64_t v = (rest << 32u) | x[i];
x[i] = static_cast<std::uint32_t>(v / d);
rest = v % d;
}
while (!x.empty() && x.back() == 0)
{
x.pop_back();
}
}
std::size_t big_bit_length(const big& x)
{
std::size_t n = 32 * x.size();
for (std::uint32_t top = x.back(); (top & 0x80000000u) == 0; top <<= 1u)
{
--n;
}
return n;
}
bool big_bit(const big& x, std::size_t i)
{
return ((x[i / 32] >> (i % 32)) & 1u) != 0;
}
/// the 128 most significant bits of x (floor), shifted left if x has fewer bits
std::pair<std::uint64_t, std::uint64_t> big_top128(const big& x)
{
const std::size_t n = big_bit_length(x);
std::uint64_t high = 0;
std::uint64_t low = 0;
for (std::size_t k = 0; k < 128; ++k)
{
const bool bit = k < n && big_bit(x, n - 1 - k);
if (k < 64)
{
high = (high << 1u) | (bit ? 1u : 0u);
}
else
{
low = (low << 1u) | (bit ? 1u : 0u);
}
}
return {high, low};
}
/// the digits (without trailing zeros) and the decimal exponent of a
/// representation "[-]d[.ddd][e[+-]x]"
std::pair<std::string, int> digits_and_exponent(const std::string& s)
{
std::string digits;
int point = -1;
int exponent = 0;
for (std::size_t i = 0; i < s.size(); ++i)
{
const char c = s[i];
if (c >= '0' && c <= '9')
{
digits += c;
}
else if (c == '.')
{
point = static_cast<int>(digits.size());
}
else if (c == 'e' || c == 'E')
{
exponent = std::stoi(s.substr(i + 1));
break;
}
}
int e = exponent + (point < 0 ? static_cast<int>(digits.size()) : point) - static_cast<int>(digits.size());
const std::size_t first = digits.find_first_not_of('0');
digits = first == std::string::npos ? "0" : digits.substr(first);
while (digits.size() > 1 && digits.back() == '0')
{
digits.pop_back();
++e;
}
return {digits, e};
}
/// the correctly rounded double of a decimal text
/// (not std::strtod: the C runtimes of some platforms, e.g. MinGW's, round
/// some 16 and 17 digit inputs wrongly)
double parse_double(const std::string& text)
{
const nlohmann::json j = nlohmann::json::parse(text, nullptr, false);
// (a discarded value: out of range, as strtod's HUGE_VAL)
return j.is_discarded() ? std::numeric_limits<double>::infinity() : j.get<double>();
}
/// whether the decimal digits * 10^e reads back as v
bool reads_back(const std::string& digits, int e, double v)
{
const std::string text = digits + "e" + std::to_string(e);
// (compared bit for bit: v is positive and finite, and -Wfloat-equal)
return reinterpret_bits<std::uint64_t>(parse_double(text)) == reinterpret_bits<std::uint64_t>(v);
}
/// Check the representation of a positive finite double: it reads back as
/// the same value, and no representation with fewer digits does.
void check_shortest(double v)
{
std::array<char, 33> buf{};
char* end = nlohmann::detail::to_chars(buf.data(), buf.data() + 32, v);
const std::string text(buf.data(), end);
CAPTURE(text)
CHECK(parse_double(text) == v);
// the layout is that of format_buffer() for the same digits
std::array<char, 64> reference{};
int len = 0;
int exponent = 0;
nlohmann::detail::dtoa_impl::shortest_digits(reference.data(), len, exponent, v);
const char* const reference_end = nlohmann::detail::dtoa_impl::format_buffer(reference.data(), len, exponent, -4, 15);
CHECK(text == std::string(reference.data(), static_cast<std::size_t>(reference_end - reference.data())));
const auto de = digits_and_exponent(text);
const std::string& digits = de.first;
if (digits.size() > 1)
{
// the decimals of one digit fewer next to the value
// (a stream rather than snprintf("%.*e"), whose output GCC cannot bound)
std::ostringstream shorter;
shorter.imbue(std::locale::classic());
shorter << std::scientific << std::setprecision(static_cast<int>(digits.size()) - 2) << v;
const auto near = digits_and_exponent(shorter.str());
// as an integer with digits.size() - 1 digits
std::string m = near.first;
int e = near.second;
while (m.size() < digits.size() - 1)
{
m += '0';
--e;
}
const std::uint64_t mid = std::stoull(m);
for (const std::uint64_t candidate :
{
mid - 1, mid, mid + 1
})
{
CAPTURE(candidate)
CHECK(!reads_back(std::to_string(candidate), e, v));
}
}
// (icpc with libstdc++ 11 defines __cpp_lib_to_chars, but has no floating-point std::to_chars)
#if defined(JSON_HAS_CPP_17) && defined(__cpp_lib_to_chars) && !defined(__INTEL_COMPILER)
// the closest of the shortest representations, as std::to_chars finds it
std::array<char, 64> std_text{};
const auto r = std::to_chars(std_text.data(), std_text.data() + std_text.size(), v, std::chars_format::scientific);
CHECK(digits_and_exponent(std::string(std_text.data(), r.ptr)) == de);
#endif
}
} // namespace
TEST_CASE("shortest digits of doubles")
{
SECTION("powers of ten")
{
// the 128-bit significands of 10^k, rounded down, recomputed
for (int k = -342; k <= 341; ++k)
{
CAPTURE(k)
big x{1};
if (k >= 0)
{
for (int i = 0; i < k; ++i)
{
big_mul_small(x, 10);
}
}
else
{
// floor(2^b / 10^-k) for a b that leaves more than 128 bits
const int b = 128 + 64 + (4 * -k);
x.assign(static_cast<std::size_t>(b / 32) + 1, 0);
x.back() = 1u << (b % 32);
for (int i = 0; i < -k; ++i)
{
big_div_small(x, 10);
}
}
const auto expected = big_top128(x);
const auto actual = nlohmann::detail::zmij::pow10(k);
CHECK(actual.high == expected.first);
CHECK(actual.low == expected.second);
}
}
SECTION("boundary values")
{
for (const double v :
{
std::numeric_limits<double>::min(), std::numeric_limits<double>::max(), std::numeric_limits<double>::denorm_min(),
std::nextafter(std::numeric_limits<double>::min(), 0.0), 1.0, 2.0, 0.1, 0.3, 1e21, 1e22, 1e23, 5e-324, 9007199254740993.0,
1.2345e+21, 2.2250738585072014e-308, 1.7976931348623157e308, 4.9406564584124654e-324, 123456789012345680.0
})
{
check_shortest(v);
}
// all powers of two (their rounding interval is narrower below)
for (int e = -1074; e <= 1023; ++e)
{
check_shortest(std::ldexp(1.0, e));
}
// powers of ten and their neighbors
for (int e = -323; e <= 308; ++e)
{
const double p = std::strtod(("1e" + std::to_string(e)).c_str(), nullptr);
check_shortest(p);
check_shortest(std::nextafter(p, 0.0));
check_shortest(std::nextafter(p, std::numeric_limits<double>::infinity()));
}
}
SECTION("random doubles")
{
std::mt19937_64 rng(5295); // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed): reproducible
for (int i = 0; i < 100000; ++i)
{
const std::uint64_t bits = rng() & 0x7FFFFFFFFFFFFFFFu;
const auto v = reinterpret_bits<double>(bits);
if (std::isfinite(v) && bits != 0)
{
check_shortest(v);
}
}
}
}