Compare commits

..
Author SHA1 Message Date
Niels Lohmann e96e2982a5 Fix CI: useless cast in the growth of the view's node index
GCC -Werror=useless-cast (ci_test_gcc on Linux x86-64) rejected
static_cast<std::size_t>(guess + (guess / 4) + 64): the sum is a
std::uint64_t prvalue, the same type as std::size_t there, while the cast
is needed where std::size_t is 32 bits wide. Cast a named variable
instead, which GCC does not report. The build stopped at an earlier error
before, so the previous CI run did not show this one.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-30 20:19:13 +02:00
Niels Lohmann 0a64e6c99b Fix CI: MSVC C4127 in the view builder and the single-header test build
- msvc (Win32, /W4 /WX) reported C4127 (conditional expression is
  constant) for `TrailingCommas && cur() == ']'` and the like when the
  option is off. Route the template arguments through a static enabled()
  function, as json.hpp's nesting_depth_exhausted() does.
- ci_test_single_header compiled unit-json_view_builder.cpp against
  single_include/, which does not contain the internal
  nlohmann/detail/view headers. Build that test only with the multiple
  headers.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-30 20:19:13 +02:00
Niels Lohmann d8b8c2498f Test the C++11 stand-in for std::string_view of json_view
Six members of string_ref (length, begin, end, operator[], operator!=,
and operator<<) were not reached before C++17, where string_ref is
std::string_view.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-30 20:19:13 +02:00
Niels Lohmann 2fb66ba859 Remove the eight-digit case of the view's digit parser
Since whole blocks of eight digits are read directly, parse_upto8() only
gets fewer than eight digits; its eight-digit case was dead code.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-30 20:19:13 +02:00
Niels Lohmann ab49a7b1bf Address the cpplint findings of the view's parser
The exponent of the overflow check is an std::int64_t instead of a long
(runtime/int).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-30 20:19:13 +02:00
Niels Lohmann 154f240022 Read whole blocks of eight digits of json_view directly
A block of eight digits of a number token lies inside the input (the
digits were counted while scanning, or are recorded in the digit
layout), so parse_upto19() reads it without the bounds check of the
last, partial block. Traversing canada.json: -11% instructions, -6%
cycles.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-30 20:19:13 +02:00
Niels Lohmann 8d3280de1e Classify a NUL inside a string of json_view as a control character
json::parse ends the input at a NUL only between values (where it does
at all); inside a string, a NUL is a control character that must be
escaped. The view reported it as a missing closing quote. (Only the
error code differed: the exception comes from the library parser.)

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-30 20:19:13 +02:00
Niels Lohmann 118af5015e Test the overflow check of json_view at the largest double
Numbers of more than 19 digits at the boundary of the largest double are
decided by the exact comparison with the midpoint in the overflow check.
The check uses the floating-point type of the document: with float, the
view rejects what parse() rejects (1e39, 3.4028236e38, the midpoint between
the largest float and 2^128), and double documents are not affected.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-30 20:19:13 +02:00
Niels Lohmann 9b0b4ff26d Address the clang-tidy findings of the view's parser
- tables as std::array; the frames of the first 64 levels stay a C array
  (not initialized on purpose, NOLINT)
- \u escapes are decoded with the library's hex_codepoint() instead of a
  second table
- the parse failure is private, with an accessor; the special member
  functions of the builder are all declared
- no nested conditional operators; explicit parentheses; a repeated
  branch body merged; auto for casts
- the test's C arrays, fixed seed, and escaped literals are marked, as in
  the other tests

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-30 20:19:13 +02:00
Niels Lohmann 694bfd1b4e Keep the arguments of the view's throw helpers used without exceptions
With JSON_NOEXCEPTION, NLOHMANN_VIEW_THROW(e) was std::abort() alone, so
the parameters of the functions that build the exceptions were unused, a
warning that the builds with -Werror turn into an error. The exception is
now evaluated before std::abort(); the program ends anyway.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-30 20:19:13 +02:00
Niels Lohmann ad82821c89 Add the one-pass parser of json_view (internal)
The builder parses JSON text in one pass into the node index: strings and
numbers stay in the source (escaped strings are decoded into an arena),
integers are converted while their digits are in cache, and floats keep
their digit layout for a later conversion. It accepts exactly what
json::parse accepts, for every combination of comments and trailing commas,
with and without a terminating NUL, and with JSON_STRICT_NUL_HANDLING.

Parse state lives in a local cursor whose address never escapes, so that it
stays in registers; out-of-line helpers (errors, regrowth, escapes,
comments) are members of the builder and get the positions they need. The
value dispatch is expanded once for array elements and once for member
values. Literals are compared with memcmp and words read in a fixed byte
order, so nothing depends on the platform's byte order. Error messages come
with the public classes.

Tests (unit-json_view_builder.cpp): accept/reject and values against
json::parse for handwritten, generated, and damaged documents under all
option combinations, from std::string and from exact-size buffers (no read
past the input under AddressSanitizer), deep nesting up to 100,000 levels,
NUL/BOM/whitespace cases, and the test-suite files.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-30 20:19:13 +02:00
Niels Lohmann 1bf0b1b6c2 Add the node index and scanning primitives of json_view (internal)
Internal parts of the zero-copy view (#5295), under detail/view and not
included by json.hpp, so that users of json.hpp compile nothing of it:

- macro_scope.hpp/macro_unscope.hpp: the few macros the view needs, under
  its own prefix (json.hpp undefines its own at its end); the throw macro
  honors JSON_NOEXCEPTION and JSON_THROW_USER like JSON_THROW
- string_ref.hpp: std::string_view from C++17 on, else a small stand-in
- node.hpp: the 16-byte node of the index; its kinds are value_t values
  (checked by a static_assert)
- document_data.hpp: the storage of a parsed document (node array, decode
  arena, owned input)
- scan.hpp: string and digit scanning with unrolled checks at fixed offsets
  (after yyjson) and the library's SWAR and UTF-8 checks, independent of the
  byte order

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-30 20:19:13 +02:00
Niels Lohmann b88e5f9107 Keep the behavior-changing configuration readable after json.hpp
json.hpp undefines JSON_STRICT_NUL_HANDLING and
JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON at its end. Code that builds on
the library after it, such as the planned json_view.hpp, reads them from
detail::abi_config instead. The constants live in the ABI namespace, which
already encodes both settings, so they always match the basic_json in use.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-30 20:19:13 +02:00
Niels Lohmann d23803fd32 Decode \u escapes with a table in the lexer
get_codepoint() read the four hex digits of a \u escape with four calls
to get(), each classified by a chain of range comparisons. For contiguous
input, get_codepoint_bulk() now decodes them with one lookup per byte
(hex_codepoint() in string_scan.hpp, after yyjson's read_hex_u16): a
256-entry table maps a byte to its value, or 0xFF for anything else, and
an invalid digit shows in the OR of the four values. It then skips the
four bytes and updates the position counters as four get() calls would.
If a digit is invalid or fewer than four bytes are left, it changes
nothing and the existing loop runs, so errors are reported with the same
message and position as before.

json::parse, best of 5 runs in separate processes (M1 Max): the escaped
twitter.json (every non-ASCII character as \u) -13.6%, all other files
within 0.3%.

Tests compare the contiguous and the streaming path (value or exception
message) for valid escapes, surrogate pairs, truncated and invalid digits
at every position, and 3,000 seeded random escapes.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-30 20:19:12 +02:00
Niels Lohmann f1014c938a Fix CI: useless casts to std::size_t in the string-scan tests
GCC -Werror=useless-cast rejected static_cast<std::size_t>(next() % n):
on 64-bit Linux std::uint64_t and std::size_t are the same type, while
the cast is needed where std::size_t is 32 bits wide. Draw the sizes from
a 32-bit value instead, which converts to std::size_t implicitly on every
platform.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-30 20:19:12 +02:00
Niels Lohmann e91fdad877 Find the stop byte of a string run without a byte loop
find_string_special() and find_ascii_copyable_run() test eight bytes at a
time, but located the stopping byte inside a word with a byte loop. The
lowest flagged byte of the SWAR tests is always a true hit (the borrows of
the subtractions can only flag bytes above one), so its index is now the
trailing-zero count of the mask; words are read in little-endian order on
every platform, so this does not depend on the byte order.
scalar_string_bulk_run() validates a run of multi-byte UTF-8 sequences one
after another instead of searching for the next special byte in between,
which helps text in non-Latin scripts.

The kernels serve the lexer's contiguous fast path, the serializer, and the
binary formats. New tests compare all three with byte-by-byte reference
scans on 100,000 generated buffers at three alignments; the portable
fallback of count_trailing_zeros() was checked against the builtin.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-30 20:19:12 +02:00
Niels Lohmann 9c71689715 Convert long doubles under a multi-byte decimal point completely
The strtold fallback, which is left only for long double formats that
are not binary64 (x87, binary128), substituted the first byte of the
locale's decimal point for '.'. Under a locale whose decimal point is
longer than one byte, such as fa_IR.UTF-8 or ar_EG.UTF-8 (U+066B),
strtold stopped there and the value was truncated at the decimal point.
A longer decimal point is now put into a copy of the token.

The test "locale with a multi-byte decimal point" now compares the long
double values with those of the "C" locale; with x87 long doubles it
failed before.

Fixes #5660.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-30 20:19:11 +02:00
Niels Lohmann 44ec53c77b Convert float and double with the library's own correctly rounded parser
float, double, and long double where it is IEEE-754 binary64 (MSVC, Apple
arm64) are now converted by the library itself, correctly rounded and
independent of the locale and of the C and C++ libraries:

- The token is split into sign, significand w (at most 19 digits), and
  decimal exponent q, using the positions of the decimal point and the
  exponent that the scanners already recorded, so no character is
  classified again.
- Clinger's fast path where w and 10^|q| are exact.
- Eisel-Lemire otherwise, now templated for binary32 and binary64.
- For tokens with more than 19 digits whose w and w + 1 round differently,
  an exact big-integer comparison with the midpoint between the two
  candidates (the digit comparison of fast_float, simplified).

This replaces the separate token walks of Clinger's fast path and of
Eisel-Lemire, the significant-digit gate that avoided the former, and, for
float and double, std::from_chars and the locale-aware strtod. std::from_chars
and strtold remain only for other long double formats (x87, binary128,
double-double) and for types that are not IEEE-754. Values are bit-identical
to before wherever the previous conversion was correctly rounded; tokens
converted in a locale with a multi-byte decimal point are now also exact.
Overflow still gives out_of_range.406, underflow a signed zero.

convert_float() is the entry point for other parsers of JSON text: it
converts like the lexer, without allocation for binary32/binary64.

Tests: exact-bit tests for double and float (ties, subnormal and overflow
boundaries, huge exponents, more digits than any midpoint), Eisel-Lemire for
binary32, the round trips of 200,000 doubles and 100,000 floats without
declines, 508 generated hard cases with the expected bits of both formats
(float_hard_cases.hpp) through the converter and both scanners, and
JSON-level overflow/underflow checks for double and float. The locale tests
now check the values in a locale with a multi-byte decimal point.

Docs: the statements that parsing uses strtod/strtof/strtold; the fast_float
credit now names the digit comparison.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-30 20:19:11 +02:00
26 changed files with 5419 additions and 1973 deletions
+8
View File
@@ -20,6 +20,7 @@ cc_library(
hdrs = [
"include/nlohmann/adl_serializer.hpp",
"include/nlohmann/byte_container_with_subtype.hpp",
"include/nlohmann/detail/abi_config.hpp",
"include/nlohmann/detail/abi_macros.hpp",
"include/nlohmann/detail/bit_ops.hpp",
"include/nlohmann/detail/conversions/from_json.hpp",
@@ -65,6 +66,13 @@ cc_library(
"include/nlohmann/detail/string_escape.hpp",
"include/nlohmann/detail/string_utils.hpp",
"include/nlohmann/detail/value_t.hpp",
"include/nlohmann/detail/view/builder.hpp",
"include/nlohmann/detail/view/document_data.hpp",
"include/nlohmann/detail/view/macro_scope.hpp",
"include/nlohmann/detail/view/macro_unscope.hpp",
"include/nlohmann/detail/view/node.hpp",
"include/nlohmann/detail/view/scan.hpp",
"include/nlohmann/detail/view/string_ref.hpp",
"include/nlohmann/json.hpp",
"include/nlohmann/json_fwd.hpp",
"include/nlohmann/json_literals.hpp",
+1 -1
View File
@@ -1395,7 +1395,7 @@ THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR I
- The class contains a slightly modified version of the Grisu2 algorithm from Florian Loitsch which is licensed under the [MIT License](https://opensource.org/licenses/MIT) (see above). Copyright &copy; 2009 [Florian Loitsch](https://florian.loitsch.com/)
- The class contains a copy of [Hedley](https://nemequ.github.io/hedley/) from Evan Nemerson which is licensed as [CC0-1.0](https://creativecommons.org/publicdomain/zero/1.0/).
- The class contains parts of [Google Abseil](https://github.com/abseil/abseil-cpp) which is licensed under the [Apache 2.0 License](https://opensource.org/licenses/Apache-2.0).
- The class contains an adapted version of the Eisel-Lemire algorithm and its table of powers of five from [fast_float](https://github.com/fastfloat/fast_float) by Daniel Lemire and contributors, which is available under the [MIT License](https://opensource.org/licenses/MIT) (used here), the Apache 2.0 License, and the Boost Software License. Copyright &copy; 2021 The fast_float authors
- The class contains an adapted version of the Eisel-Lemire algorithm, its table of powers of five, and its digit comparison for long numbers from [fast_float](https://github.com/fastfloat/fast_float) by Daniel Lemire and contributors, which is available under the [MIT License](https://opensource.org/licenses/MIT) (used here), the Apache 2.0 License, and the Boost Software License. Copyright &copy; 2021 The fast_float authors
<img align="right" src="https://git.fsfe.org/reuse/reuse-ci/raw/branch/master/reuse-horizontal.png" alt="REUSE Software">
@@ -23,9 +23,10 @@ type to use.
## Template parameters
`NumberFloatType`
: the type to store floating-point numbers. Parsing and serialization are implemented in terms of
`#!cpp std::strtof`/`#!cpp std::strtod`/`#!cpp std::strtold` and `#!cpp std::snprintf`, so the type must be
`#!cpp float`, `#!cpp double`, or `#!cpp long double`. The
: the type to store floating-point numbers. The parser converts `#!cpp float`, `#!cpp double`, and a
`#!cpp long double` that is IEEE 754 binary64 itself and other `#!cpp long double` formats with
`#!cpp std::from_chars` or `#!cpp std::strtold`, and serialization falls back to `#!cpp std::snprintf`, so the
type must be `#!cpp float`, `#!cpp double`, or `#!cpp long double`. The
[binary formats](../../features/binary_formats/index.md) additionally require `#!cpp float` or `#!cpp double`,
because they have no encoding for `#!cpp long double`. See
[Template Parameter Requirements](../../features/types/template_parameters.md#numberfloattype).
@@ -71,10 +71,11 @@ otherwise, it uses unsigned integer storage.
- Numbers with a decimal digit or scientific notation are always stored as `#!c double`.
- The number types can be changed, see [Template number types](#template-number-types).
- As of version 3.9.1, the conversion is realized by
[`std::strtoull`](https://en.cppreference.com/w/cpp/string/byte/strtoul),
[`std::strtoll`](https://en.cppreference.com/w/cpp/string/byte/strtol), and
[`std::strtod`](https://en.cppreference.com/w/cpp/string/byte/strtof), respectively.
- The library converts integers and floating-point numbers itself, independent of the locale. Floating-point
numbers are correctly rounded (to nearest, ties to even). Only a `#!c long double` that is not IEEE 754 binary64
(e.g., the 80-bit x87 format) is converted with `#!cpp std::from_chars` where available, or with
[`std::strtold`](https://en.cppreference.com/w/cpp/string/byte/strtof), which gets the decimal point of the
current locale, also one longer than one byte (e.g., in `fa_IR.UTF-8`).
!!! example "Examples"
@@ -85,10 +86,10 @@ otherwise, it uses unsigned integer storage.
### Number limits
- Any 64-bit signed or unsigned integer can be stored without loss of precision.
- Numbers exceeding the limits of `#!c double` (i.e., numbers that after conversion via
[`std::strtod`](https://en.cppreference.com/w/cpp/string/byte/strtof) are not satisfying
- Numbers exceeding the limits of `#!c double` (i.e., numbers whose rounded value is not satisfying
[`std::isfinite`](https://en.cppreference.com/w/cpp/numeric/math/isfinite) such as `#!c 1E400`) will throw exception
[`json.exception.out_of_range.406`](../../home/exceptions.md#jsonexceptionout_of_range406) during parsing.
[`json.exception.out_of_range.406`](../../home/exceptions.md#jsonexceptionout_of_range406) during parsing. Numbers too
small for `#!c double` (such as `#!c 1E-400`) become zero, with the sign of the number.
- Floating-point numbers are rounded to the next number representable as `double`. For instance
`#!c 3.141592653589793238462643383279` is stored as [`0x400921fb54442d18`](https://float.exposed/0x400921fb54442d18).
This is the same behavior as the code `#!c double x = 3.141592653589793238462643383279;`.
@@ -26,8 +26,9 @@ Requirements are split into two groups:
diagnosed with dedicated error messages, and violating most of them results in a compiler error somewhere inside
the library. Four violations are not caught at compile time at all:
- A [`StringType`](#stringtype) whose `data()` is not null-terminated compiles and silently misparses numbers,
because the lexer hands the buffer to `#!cpp std::strtoull`/`#!cpp std::strtoll`/`#!cpp std::strtod`.
- A [`StringType`](#stringtype) whose `data()` is not null-terminated compiles and silently misparses numbers
stored as a `#!cpp long double` that is not IEEE 754 binary64 (e.g., the 80-bit x87 format), because the lexer
hands the buffer to `#!cpp std::strtold`.
- A stateful [`AllocatorType`](#allocatortype) compiles and silently ignores its state: allocation, deallocation,
and [`get_allocator()`](../../api/basic_json/get_allocator.md) each use a different default-constructed instance.
- The two [cross-specialization conversions](#cross-specialization-conversions) below. These abort on an assertion
@@ -535,8 +536,10 @@ therefore silently changes parse results rather than raising an error. See
`NumberFloatType` must be one of `#!cpp float`, `#!cpp double`, or `#!cpp long double`:
- The [parser](../parsing/index.md) converts number literals with `#!cpp std::strtof`, `#!cpp std::strtod`, or
`#!cpp std::strtold`; the library provides overloads for exactly these three types.
- The [parser](../parsing/index.md) converts number literals to `#!cpp float`, `#!cpp double`, and a
`#!cpp long double` that is IEEE 754 binary64 itself; other `#!cpp long double` formats are converted with
`#!cpp std::from_chars` where available, or with `#!cpp std::strtold`. The library provides overloads for exactly
these three types.
- [`dump`](../../api/basic_json/dump.md) falls back to `#!cpp std::snprintf` with the `%g` and `%Lg` conversion
specifiers, for which the library likewise provides only `#!cpp double` and `#!cpp long double` overloads
(`#!cpp float` is promoted to `#!cpp double`).
+1 -1
View File
@@ -20,4 +20,4 @@ The class contains a slightly modified version of the Grisu2 algorithm from Flor
The class contains a copy of [Hedley](https://nemequ.github.io/hedley/) from Evan Nemerson which is licensed as [CC0-1.0](https://creativecommons.org/publicdomain/zero/1.0/).
The class contains an adapted version of the Eisel-Lemire algorithm and its table of powers of five from [fast_float](https://github.com/fastfloat/fast_float) by Daniel Lemire and contributors, which is available under the [MIT License](https://opensource.org/licenses/MIT) (used here), the Apache 2.0 License, and the Boost Software License. Copyright &copy; 2021 The fast_float authors
The class contains an adapted version of the Eisel-Lemire algorithm, its table of powers of five, and its digit comparison for long numbers from [fast_float](https://github.com/fastfloat/fast_float) by Daniel Lemire and contributors, which is available under the [MIT License](https://opensource.org/licenses/MIT) (used here), the Apache 2.0 License, and the Boost Software License. Copyright &copy; 2021 The fast_float authors
+34
View File
@@ -0,0 +1,34 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#pragma once
#include <nlohmann/detail/abi_macros.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
{
/*!
@brief the configuration macros that change the library's behavior
json.hpp undefines these macros at its end (see macro_unscope.hpp), so code
that builds on the library after it (json_view.hpp) reads them here. Like the
macros, they are part of the ABI namespace, so they always match the
basic_json they are used with.
*/
struct abi_config
{
/// JSON_STRICT_NUL_HANDLING: a null byte is an error, not the end of input
static constexpr bool strict_nul_handling = JSON_STRICT_NUL_HANDLING != 0;
/// JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON
static constexpr bool legacy_discarded_value_comparison = JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON != 0;
};
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
+26 -2
View File
@@ -39,6 +39,25 @@ inline int count_leading_zeros(std::uint64_t x) noexcept
#endif
}
/// number of trailing zero bits of x (x != 0)
inline int count_trailing_zeros(std::uint64_t x) noexcept
{
#if defined(__GNUC__) || defined(__clang__)
return __builtin_ctzll(x);
#else
int n = 0;
for (int shift = 32; shift != 0; shift >>= 1)
{
if ((x << (64 - shift)) == 0)
{
n += shift;
x >>= shift;
}
}
return n;
#endif
}
/// the 128-bit product of two 64-bit numbers
struct uint128_parts
{
@@ -68,14 +87,19 @@ inline uint128_parts full_multiplication(std::uint64_t a, std::uint64_t b) noexc
/// eight bytes as a little-endian word (compilers fold this into one load on
/// little-endian targets)
inline std::uint64_t read_eight_bytes(const char* p) noexcept
inline std::uint64_t read_eight_bytes(const unsigned char* b) noexcept
{
const auto* b = reinterpret_cast<const unsigned char*>(p); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
return static_cast<std::uint64_t>(b[0]) | (static_cast<std::uint64_t>(b[1]) << 8u)
| (static_cast<std::uint64_t>(b[2]) << 16u) | (static_cast<std::uint64_t>(b[3]) << 24u)
| (static_cast<std::uint64_t>(b[4]) << 32u) | (static_cast<std::uint64_t>(b[5]) << 40u)
| (static_cast<std::uint64_t>(b[6]) << 48u) | (static_cast<std::uint64_t>(b[7]) << 56u);
}
/// eight bytes as a little-endian word
inline std::uint64_t read_eight_bytes(const char* p) noexcept
{
return read_eight_bytes(reinterpret_cast<const unsigned char*>(p)); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
}
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
+59 -10
View File
@@ -219,6 +219,44 @@ class lexer : public lexer_base<BasicJsonType>
// scan functions
/////////////////////
/// contiguous input: try to decode the 4 hex digits following `\u`
/// directly from the input buffer via hex_codepoint(), instead of 4 calls
/// to get(). On success, advances the adapter and the position counters
/// exactly as those 4 get() calls would (a hex digit is never '\n', so
/// only the flat counters move) and leaves @a current holding the last of
/// the 4 digits, just as the last such get() would; the codepoint is
/// written to @a out. Makes no state change and returns false - for a
/// pending unget, fewer than 4 remaining bytes, or any of the 4 bytes not
/// being a hex digit - so the caller falls back unchanged to the
/// per-character loop, which then reports the same diagnostic (stopping
/// at the first invalid digit) as before this optimization.
bool get_codepoint_bulk(std::true_type /*bulk*/, int& out)
{
if (next_unget || ia.bulk_remaining() < 4)
{
return false;
}
const char_type* const raw = ia.bulk_data();
const int codepoint = hex_codepoint(reinterpret_cast<const unsigned char*>(raw));
if (codepoint < 0)
{
return false;
}
ia.bulk_skip(4);
// a hex digit is never a newline, so only the flat counters advance
position.chars_read_total += 4;
position.chars_read_current_line += 4;
current = char_traits<char_type>::to_int_type(raw[3]);
out = codepoint;
return true;
}
/// streaming input: no bulk fast path
bool get_codepoint_bulk(std::false_type /*bulk*/, int& /*out*/) const noexcept
{
return false;
}
/*!
@brief get codepoint from 4 hex characters following `\u`
@@ -238,6 +276,14 @@ class lexer : public lexer_base<BasicJsonType>
{
// this function only makes sense after reading `\u`
JSON_ASSERT(current == 'u');
// contiguous input: decode all 4 hex digits directly from the buffer
int fast_codepoint = 0;
if (get_codepoint_bulk(std::integral_constant<bool, bulk_scan> {}, fast_codepoint))
{
return fast_codepoint;
}
int codepoint = 0;
const auto factors = { 12u, 8u, 4u, 0u };
@@ -1059,9 +1105,11 @@ class lexer : public lexer_base<BasicJsonType>
token_type::parse_error otherwise
@note The scanner is independent of the current locale: token_buffer
always holds `.`. Only the std::strtod fallback of convert_number()
depends on the locale, and it looks up the decimal point right
before converting (see detail::convert_float_locale_aware()).
always holds `.`. The conversion of float and double does not use
the locale either. Only the std::strtold fallback of
convert_number() for long double formats other than binary64
depends on it, and it looks up the decimal point right before
converting (see detail::convert_float_locale_aware()).
*/
token_type scan_number() // lgtm [cpp/use-of-goto] `goto` is used in this function to implement the number-parsing state machine described above. By design, any finite input will eventually reach the "done" state or return token_type::parse_error. In each intermediate state, 1 byte of the input is appended to the token_buffer vector, and only the already initialized variables token_buffer, number_type, and error_message are manipulated.
{
@@ -1074,7 +1122,7 @@ class lexer : public lexer_base<BasicJsonType>
// offset just past the last mantissa byte in token_buffer (i.e. the
// index of 'e'/'E', or the whole token when there is no exponent).
// convert_number() uses it to count significant digits; npos means
// convert_number() uses it to split the token; npos means
// "not seen an exponent yet" and is resolved at scan_number_done
std::size_t mantissa_end = std::string::npos;
@@ -1404,8 +1452,8 @@ scan_number_done:
@param[in] mantissa_end offset just past the last mantissa byte in
token_buffer (the index of 'e'/'E', or
token_buffer.size() when there is no exponent);
used to skip Clinger's fast path when it cannot
possibly succeed - see detail::mantissa_fits_clinger()
with decimal_point_position, it locates the parts
of a float token without scanning it again
*/
token_type convert_number(token_type number_type, std::size_t mantissa_end)
{
@@ -1474,10 +1522,11 @@ scan_number_done:
}
// this code is reached if we parse a floating-point number or if an
// integer conversion above overflowed. Prefer std::from_chars
// (Eisel-Lemire, locale-independent, correctly rounded) when available;
// otherwise the exact Clinger fast path (double only); otherwise the
// locale-aware strtof/strtod/strtold.
// integer conversion above overflowed. float and double (and long
// double where it is binary64) are converted by the library itself,
// correctly rounded and independent of the locale; other long double
// formats use std::from_chars when available, otherwise the
// locale-aware strtold.
if (convert_float_fast(num_begin, num_end, decimal_point_position, mantissa_end, value_float))
{
return token_type::value_float;
File diff suppressed because it is too large Load Diff
+70 -29
View File
@@ -8,10 +8,12 @@
#pragma once
#include <array> // array
#include <cstddef> // size_t
#include <cstdint> // uint64_t
#include <cstdint> // uint64_t, uint8_t
#include <cstring> // memcpy
#include <nlohmann/detail/bit_ops.hpp>
#include <nlohmann/detail/macro_scope.hpp>
// Optional SIMD backend for bulk UTF-8 validation. This is an opt-in external
@@ -69,18 +71,12 @@ inline std::size_t find_string_special(const unsigned char* data, std::size_t n)
std::size_t i = 0;
for (; i + 8 <= n; i += 8)
{
std::uint64_t word = 0;
std::memcpy(&word, data + i, sizeof(word));
if (swar_string_special(word) != 0)
const std::uint64_t special = swar_string_special(read_eight_bytes(data + i));
if (special != 0)
{
// a special byte is in this word; locate it (endian-agnostic)
for (std::size_t j = 0; j < 8; ++j)
{
if (is_string_special(data[i + j]))
{
return i + j;
}
}
// the lowest flagged byte is the first special one: the borrows of
// the subtractions can only flag bytes above a true hit
return i + (static_cast<std::size_t>(count_trailing_zeros(special)) / 8);
}
}
for (; i < n; ++i)
@@ -114,8 +110,7 @@ inline std::size_t find_ascii_copyable_run(const unsigned char* data, std::size_
std::size_t i = 0;
for (; i + 8 <= n; i += 8)
{
std::uint64_t v = 0;
std::memcpy(&v, data + i, sizeof(v));
const std::uint64_t v = read_eight_bytes(data + i);
const std::uint64_t q = v ^ 0x2222222222222222ull; // '"' (0x22)
const std::uint64_t b = v ^ 0x5C5C5C5C5C5C5C5Cull; // '\\' (0x5C)
const std::uint64_t d = v ^ 0x7F7F7F7F7F7F7F7Full; // DEL (0x7F)
@@ -126,7 +121,9 @@ inline std::size_t find_ascii_copyable_run(const unsigned char* data, std::size_
| (v & high); // >= 0x80
if (stop != 0)
{
break;
// the lowest flagged byte is the first one to stop at (see
// find_string_special())
return i + (static_cast<std::size_t>(count_trailing_zeros(stop)) / 8);
}
}
for (; i < n; ++i)
@@ -253,12 +250,18 @@ inline std::size_t scalar_string_bulk_run(const unsigned char* data, std::size_t
{
break; // end of buffer, or a quote/escape/control byte
}
const std::size_t seq = validate_one_utf8(data + pos, n - pos);
if (seq == 0)
// a run of multi-byte sequences (e.g. CJK text) is validated sequence
// by sequence without searching for the next special byte in between
do
{
break; // ill-formed or truncated: let the byte path diagnose it
const std::size_t seq = validate_one_utf8(data + pos, n - pos);
if (seq == 0)
{
return pos; // ill-formed or truncated: let the byte path diagnose it
}
pos += seq;
}
pos += seq;
while (pos < n && data[pos] >= 0x80u);
}
return pos;
}
@@ -273,8 +276,7 @@ inline std::size_t find_string_delimiter(const unsigned char* data, std::size_t
std::size_t i = 0;
for (; i + 8 <= n; i += 8)
{
std::uint64_t v = 0;
std::memcpy(&v, data + i, sizeof(v));
const std::uint64_t v = read_eight_bytes(data + i);
const std::uint64_t q = v ^ 0x2222222222222222ull;
const std::uint64_t b = v ^ 0x5C5C5C5C5C5C5C5Cull;
const std::uint64_t hit = ((q - ones) & ~q & high)
@@ -282,14 +284,8 @@ inline std::size_t find_string_delimiter(const unsigned char* data, std::size_t
| ((v - 0x2020202020202020ull) & ~v & high);
if (hit != 0)
{
for (std::size_t j = 0; j < 8; ++j)
{
const unsigned char c = data[i + j];
if (c == '\"' || c == '\\' || c < 0x20u)
{
return i + j;
}
}
// the lowest flagged byte is the first delimiter (see find_string_special())
return i + (static_cast<std::size_t>(count_trailing_zeros(hit)) / 8);
}
}
for (; i < n; ++i)
@@ -320,5 +316,50 @@ inline std::size_t string_bulk_run(const unsigned char* data, std::size_t n) noe
return scalar_string_bulk_run(data, n);
}
// Decode the 4 hex digits at [data, data+4) - the digits following a `\u`
// escape - into a codepoint 0x0000..0xFFFF via one table lookup per byte
// (after yyjson's read_hex_u16), or return -1 if any of the 4 bytes is not a
// hex digit ('0'..'9', 'A'..'F', 'a'..'f'). The caller must already have
// checked that 4 bytes are available; used by lexer::get_codepoint()'s
// contiguous fast path. On -1 it falls back to the byte-at-a-time loop, which
// stops at the first invalid digit, so the reported error and position are
// unaffected by this fast path.
inline int hex_codepoint(const unsigned char* data) noexcept
{
static const std::array<std::uint8_t, 256> hex_digit_table = // NOLINT(cppcoreguidelines-avoid-non-const-global-variables)
{
{
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // 00..0F
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // 10..1F
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // 20..2F
0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // 30..3F ('0'..'9')
0xFF, 0x0A, 0x0B, 0x0C, 0x0D, 0x0E, 0x0F, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // 40..4F ('A'..'F')
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // 50..5F
0xFF, 0x0A, 0x0B, 0x0C, 0x0D, 0x0E, 0x0F, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // 60..6F ('a'..'f')
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // 70..7F
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // 80..8F
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // 90..9F
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // A0..AF
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // B0..BF
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // C0..CF
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // D0..DF
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, // E0..EF
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF // F0..FF
}
};
const std::uint8_t d0 = hex_digit_table[data[0]];
const std::uint8_t d1 = hex_digit_table[data[1]];
const std::uint8_t d2 = hex_digit_table[data[2]];
const std::uint8_t d3 = hex_digit_table[data[3]];
// every valid digit is <= 0xF; the combined OR only exceeds it if at
// least one of the four bytes was not a hex digit (looked up as 0xFF)
if ((d0 | d1 | d2 | d3) > 0x0F)
{
return -1;
}
return (d0 << 12) | (d1 << 8) | (d2 << 4) | d3;
}
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,130 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#pragma once
#include <array> // array
#include <cstddef> // size_t
#include <cstring> // memcpy
#include <new> // operator new, placement new
#include <string> // string
#include <nlohmann/json.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
#include <nlohmann/detail/view/node.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
{
namespace view
{
/// storage of a parsed document; heap-allocated (header and an initial node
/// array in one block) so that views survive moves of the owning document
struct document_data
{
const char* src = nullptr;
std::size_t size = 0;
node* tape = nullptr;
std::size_t tape_size = 0;
std::size_t tape_cap = 0;
node* inline_tape = nullptr; ///< node array allocated together with this header
std::size_t inline_cap = 0;
std::string arena{}; ///< decoded strings that contained escapes // NOLINT(readability-redundant-member-init)
std::string owned{}; ///< owned copy of the input, if any // NOLINT(readability-redundant-member-init)
std::array<const char*, 4> base = {{nullptr, nullptr, nullptr, nullptr}}; ///< string bases: source, arena (indexed by flags & node_flags::storage)
bool discarded = true;
/// one allocation for the header and room for `nodes` nodes; large
/// documents get a separate node array instead (so it can be trimmed)
static document_data* create(std::size_t nodes)
{
nodes = nodes <= 256 ? nodes : 0;
void* mem = ::operator new (sizeof(document_data) + (nodes * sizeof(node)));
auto* d = new (mem) document_data(); // NOLINT(cppcoreguidelines-owning-memory): owned by the returned pointer, freed by deleter
// (aligned: sizeof is a multiple of the alignment; through void*, as GCC's -Wcast-align wants)
d->inline_tape = static_cast<node*>(static_cast<void*>(static_cast<char*>(mem) + sizeof(document_data))); // NOLINT(bugprone-casting-through-void)
d->inline_cap = nodes;
d->tape = d->inline_tape;
d->tape_cap = nodes;
return d;
}
struct deleter
{
void operator()(document_data* d) const noexcept
{
d->~document_data();
::operator delete (d);
}
};
document_data() noexcept = default;
document_data(const document_data&) = delete;
document_data(document_data&&) = delete;
document_data& operator=(const document_data&) = delete;
document_data& operator=(document_data&&) = delete;
~document_data()
{
release();
}
void release() noexcept
{
if (tape != inline_tape)
{
::operator delete (tape);
}
tape = inline_tape;
tape_cap = inline_cap;
}
/// make room for n nodes; keeps the first tape_size nodes
void reserve(std::size_t n)
{
if (n <= tape_cap)
{
return;
}
node* fresh = static_cast<node*>(::operator new (n * sizeof(node)));
if (tape_size != 0)
{
std::memcpy(fresh, tape, tape_size * sizeof(node));
}
release();
tape = fresh;
tape_cap = n;
}
const char* str(const node& n) const noexcept
{
return base[n.flags & node_flags::storage] + n.off;
}
/// the node after n's subtree (containers span `next` nodes, scalars one)
static NLOHMANN_VIEW_ALWAYS_INLINE const node* after(const node* n) noexcept
{
return n + (is_container(*n) ? n->next : 1u);
}
/// first element (array) or first key (object) of a container
static NLOHMANN_VIEW_ALWAYS_INLINE const node* first_child(const node* n) noexcept
{
return n + 1;
}
/// end of the elements of a container
static NLOHMANN_VIEW_ALWAYS_INLINE const node* child_end(const node* n) noexcept
{
return n + n->next;
}
};
} // namespace view
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
@@ -0,0 +1,63 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#pragma once
// Macros of json_view.hpp and its detail headers. json.hpp undefines its own
// macros at its end (macro_unscope.hpp), so the view defines the few it needs
// under its own prefix; json_view.hpp undefines them all at its end
// (detail/view/macro_unscope.hpp). Configuration that json.hpp undefines is
// read from detail::abi_config instead.
#if (defined(__cplusplus) && __cplusplus >= 201703L) || (defined(_MSVC_LANG) && _MSVC_LANG >= 201703L)
#define NLOHMANN_VIEW_HAS_CPP_17 1
#else
#define NLOHMANN_VIEW_HAS_CPP_17 0
#endif
#if defined(__GNUC__) || defined(__clang__)
#define NLOHMANN_VIEW_LIKELY(x) __builtin_expect(!!(x), 1)
#define NLOHMANN_VIEW_UNLIKELY(x) __builtin_expect(!!(x), 0)
#define NLOHMANN_VIEW_ALWAYS_INLINE inline __attribute__((always_inline))
#define NLOHMANN_VIEW_NOINLINE __attribute__((noinline))
#elif defined(_MSC_VER)
#define NLOHMANN_VIEW_LIKELY(x) (x)
#define NLOHMANN_VIEW_UNLIKELY(x) (x)
#define NLOHMANN_VIEW_ALWAYS_INLINE __forceinline
#define NLOHMANN_VIEW_NOINLINE __declspec(noinline)
#else
#define NLOHMANN_VIEW_LIKELY(x) (x)
#define NLOHMANN_VIEW_UNLIKELY(x) (x)
#define NLOHMANN_VIEW_ALWAYS_INLINE inline
#define NLOHMANN_VIEW_NOINLINE
#endif
// exceptions as in json.hpp (JSON_NOEXCEPTION, JSON_THROW_USER)
#if (defined(__cpp_exceptions) || defined(__EXCEPTIONS) || defined(_CPPUNWIND)) && !defined(JSON_NOEXCEPTION)
#define NLOHMANN_VIEW_THROW(exception) throw exception
#else
#include <cstdlib>
// (the exception is built first, so that the arguments of the throwing
// helpers count as used; the program ends anyway)
#define NLOHMANN_VIEW_THROW(exception) (static_cast<void>(exception), std::abort())
#endif
#if defined(JSON_THROW_USER)
#undef NLOHMANN_VIEW_THROW
#define NLOHMANN_VIEW_THROW JSON_THROW_USER
#endif
// the parser stores a node's first word at once where the layout of `node` is
// known to be little-endian (MSVC targets are); elsewhere field by field
#if (defined(__BYTE_ORDER__) && defined(__ORDER_LITTLE_ENDIAN__) && __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__) || defined(_MSC_VER)
#define NLOHMANN_VIEW_LITTLE_ENDIAN 1
#else
#define NLOHMANN_VIEW_LITTLE_ENDIAN 0
#endif
/// sixteen checks at fixed offsets 0..15
#define NLOHMANN_VIEW_REPEAT16(X) X(0) X(1) X(2) X(3) X(4) X(5) X(6) X(7) X(8) X(9) X(10) X(11) X(12) X(13) X(14) X(15)
@@ -0,0 +1,20 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#pragma once
// undefine the macros of detail/view/macro_scope.hpp (at the end of json_view.hpp)
#undef NLOHMANN_VIEW_HAS_CPP_17
#undef NLOHMANN_VIEW_LIKELY
#undef NLOHMANN_VIEW_UNLIKELY
#undef NLOHMANN_VIEW_ALWAYS_INLINE
#undef NLOHMANN_VIEW_NOINLINE
#undef NLOHMANN_VIEW_THROW
#undef NLOHMANN_VIEW_LITTLE_ENDIAN
#undef NLOHMANN_VIEW_REPEAT16
+97
View File
@@ -0,0 +1,97 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#pragma once
#include <cstddef> // size_t
#include <cstdint> // uint8_t, uint16_t, uint32_t, uint64_t
#include <cstring> // memcpy
#include <nlohmann/json.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
{
namespace view
{
// the node kinds are value_t values; the tests of is_container() and of the
// number kinds depend on this numbering
static_assert(static_cast<std::uint8_t>(value_t::null) == 0 && static_cast<std::uint8_t>(value_t::object) == 1
&& static_cast<std::uint8_t>(value_t::array) == 2 && static_cast<std::uint8_t>(value_t::string) == 3
&& static_cast<std::uint8_t>(value_t::boolean) == 4 && static_cast<std::uint8_t>(value_t::number_integer) == 5
&& static_cast<std::uint8_t>(value_t::number_unsigned) == 6 && static_cast<std::uint8_t>(value_t::number_float) == 7,
"the node format depends on the numbering of value_t");
/// node flags
struct node_flags
{
static constexpr std::uint8_t escaped = 1; ///< string payload lives in the decode arena, not the source
static constexpr std::uint8_t storage = 3; ///< mask: where a string or number token lives (index into document_data::base)
static constexpr std::uint8_t is_true = 4; ///< boolean value
};
/// One entry of the flat index, in document order. An object's members are
/// stored as key node followed by the value's subtree. Integers keep their
/// converted 64-bit value in the len/next bytes (the node after a scalar is
/// always the next one, and the token length follows from `extra`).
struct node
{
std::uint8_t kind; ///< value_t
std::uint8_t flags; ///< node_flags
std::uint16_t extra; ///< numbers: integer digits (low byte) and fraction digits (high byte), 255 = "many"; otherwise 0
std::uint32_t off; ///< source offset (string content, number token, literal, bracket); arena offset if node_flags::escaped
std::uint32_t len; ///< string: decoded bytes; float: token bytes; array/object: element count
std::uint32_t next; ///< array/object: number of nodes of the subtree (its extent in the enclosing sequence)
};
static_assert(sizeof(node) == 16, "node must stay 16 bytes");
NLOHMANN_VIEW_ALWAYS_INLINE bool is_container(const node& n) noexcept
{
return static_cast<unsigned>(n.kind) - 1u <= 1u;
}
/// the converted value of an integer node (stored in len/next)
NLOHMANN_VIEW_ALWAYS_INLINE std::uint64_t integer_bits(const node& n) noexcept
{
std::uint64_t v = 0;
std::memcpy(&v, reinterpret_cast<const unsigned char*>(&n) + 8, 8); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
return v;
}
NLOHMANN_VIEW_ALWAYS_INLINE void set_integer_bits(node& n, std::uint64_t v) noexcept
{
std::memcpy(reinterpret_cast<unsigned char*>(&n) + 8, &v, 8); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
}
/// token length of a number node
NLOHMANN_VIEW_ALWAYS_INLINE std::uint32_t number_length(const node& n) noexcept
{
return n.kind == static_cast<std::uint8_t>(value_t::number_float) ? n.len
: (n.extra & 0xFFu) + (n.kind == static_cast<std::uint8_t>(value_t::number_integer) ? 1u : 0u);
}
/// estimated number of nodes for an input of `size` bytes (one node per ~12
/// bytes covers typical documents without regrowth)
inline std::size_t estimate_nodes(std::size_t size) noexcept
{
return (size / 12) + 16;
}
/// estimated number of nodes for the input [src, src + size): pretty-printed
/// input (whitespace after the first byte) needs about a node per 12 bytes,
/// minified input up to one per 4 (yyjson tells the two apart the same way)
inline std::size_t estimate_nodes(const char* src, std::size_t size) noexcept
{
return size >= 2 && (src[1] == ' ' || src[1] == '\n' || src[1] == '\r' || src[1] == '\t') ? estimate_nodes(size) : (size / 4) + 16;
}
} // namespace view
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
+189
View File
@@ -0,0 +1,189 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-FileCopyrightText: 2020 YaoYuan <https://github.com/ibireme/yyjson>
// SPDX-License-Identifier: MIT
#pragma once
#include <array> // array
#include <cstddef> // size_t
#include <cstdint> // uint8_t, uint16_t, uint64_t
#include <cstring> // memcpy
#include <nlohmann/json.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
// Scanning primitives of the view's parser. The unrolled checks at fixed
// offsets follow yyjson (https://github.com/ibireme/yyjson, MIT license): the
// loads do not depend on each other, so the CPU can run ahead. Words are read
// with read_eight_bytes(), so nothing here depends on the byte order.
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
{
namespace view
{
/// 1 for bytes that may appear verbatim in a string: 0x20..0x7F except '"' and '\\'
inline const std::uint8_t* string_plain() noexcept
{
static const std::array<std::uint8_t, 256> table =
{
{
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // 0x00..0x1F
1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, // 0x20..0x3F ('"')
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, // 0x40..0x5F ('\\')
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, // 0x60..0x7F
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // 0x80..0x9F
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // 0xA0..0xBF
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // 0xC0..0xDF
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // 0xE0..0xFF
}
};
return table.data();
}
NLOHMANN_VIEW_ALWAYS_INLINE bool is_digit(unsigned char c) noexcept
{
return static_cast<unsigned char>(c - '0') <= 9;
}
/// two bytes as they are in memory (only compared with byte-symmetric patterns)
NLOHMANN_VIEW_ALWAYS_INLINE std::uint16_t load16(const unsigned char* p) noexcept
{
std::uint16_t w = 0;
std::memcpy(&w, p, 2);
return w;
}
/// Advance over plain string bytes and well-formed UTF-8. Stops at a quote,
/// a backslash, a control character, ill-formed UTF-8, or the end. The first
/// 16 bytes are checked one by one, so that the position advances by
/// constants in predicted branches (most strings are short); longer runs
/// continue eight bytes at a time.
NLOHMANN_VIEW_ALWAYS_INLINE const unsigned char* scan_string_run(const unsigned char* p, const unsigned char* e) noexcept
{
const std::uint8_t* plain = string_plain();
for (;;)
{
if (e - p >= 16)
{
#define NLOHMANN_VIEW_STEP(i) if (NLOHMANN_VIEW_LIKELY(plain[p[i]] != 0)) {} else { p += (i); goto stop; }
NLOHMANN_VIEW_REPEAT16(NLOHMANN_VIEW_STEP)
#undef NLOHMANN_VIEW_STEP
p += 16;
while (e - p >= 8)
{
const std::uint64_t special = swar_string_special(read_eight_bytes(p));
if (special != 0)
{
p += count_trailing_zeros(special) / 8;
goto stop;
}
p += 8;
}
continue;
}
while (p != e && plain[*p] != 0)
{
++p;
}
if (p == e)
{
return p;
}
stop:
if (*p < 0x80)
{
return p; // quote, backslash, or control character
}
// non-ASCII: a run of well-formed sequences (the library's check, so
// that exactly what json::parse accepts is accepted)
do
{
const std::size_t n = validate_one_utf8(p, static_cast<std::size_t>(e - p));
if (n == 0)
{
return p;
}
p += n;
}
while (p != e && *p >= 0x80);
}
}
/// advance over ASCII digits
NLOHMANN_VIEW_ALWAYS_INLINE const unsigned char* skip_digits(const unsigned char* p, const unsigned char* e) noexcept
{
while (e - p >= 16)
{
#define NLOHMANN_VIEW_STEP(i) if (NLOHMANN_VIEW_LIKELY(is_digit(p[i]))) {} else { return p + (i); }
NLOHMANN_VIEW_REPEAT16(NLOHMANN_VIEW_STEP)
#undef NLOHMANN_VIEW_STEP
p += 16;
}
while (p != e && is_digit(*p))
{
++p;
}
return p;
}
/// powers of ten up to 10^19 as integers
inline std::uint64_t int_pow10(unsigned k) noexcept
{
static const std::array<std::uint64_t, 20> table =
{
{
1u, 10u, 100u, 1000u, 10000u, 100000u, 1000000u, 10000000u, 100000000u, 1000000000u,
10000000000u, 100000000000u, 1000000000000u, 10000000000000u, 100000000000000u, 1000000000000000u,
10000000000000000u, 100000000000000000u, 1000000000000000000u, 10000000000000000000u
}
};
return table[k];
}
/// value of 0 < k < 8 digits at p in one step if [p, p + 8) lies below
/// limit, else one digit at a time (whole blocks of eight digits are read by
/// parse_upto19() directly)
NLOHMANN_VIEW_ALWAYS_INLINE std::uint64_t parse_upto8(const unsigned char* p, unsigned k, const unsigned char* limit) noexcept
{
if (NLOHMANN_VIEW_LIKELY(limit - p >= 8))
{
// move the k digits to the top and pad the vacated low bytes with '0'
const unsigned shift = 8 * (8 - k);
return parse_eight_digits((read_eight_bytes(p) << shift) | (0x3030303030303030u >> (8 * k)));
}
std::uint64_t v = 0;
for (unsigned i = 0; i < k; ++i)
{
v = (v * 10) + static_cast<std::uint64_t>(p[i] - '0');
}
return v;
}
/// value of k <= 19 digits at p
NLOHMANN_VIEW_ALWAYS_INLINE std::uint64_t parse_upto19(const unsigned char* p, unsigned k, const unsigned char* limit) noexcept
{
std::uint64_t w = 0;
while (k >= 8)
{
// (eight digits of the token: they lie below limit)
w = (w * 100000000u) + parse_eight_digits(read_eight_bytes(p));
p += 8;
k -= 8;
}
if (k != 0)
{
w = (w * int_pow10(k)) + parse_upto8(p, k, limit);
}
return w;
}
} // namespace view
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
+111
View File
@@ -0,0 +1,111 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#pragma once
#include <algorithm> // min
#include <cstddef> // size_t
#include <cstring> // memcmp, strlen
#include <string> // basic_string
#include <nlohmann/json.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
#if NLOHMANN_VIEW_HAS_CPP_17
#include <string_view> // string_view
#endif
#ifndef JSON_NO_IO
#include <ostream> // ostream
#endif
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
{
namespace view
{
#if NLOHMANN_VIEW_HAS_CPP_17
using string_ref = std::string_view;
#else
/// minimal C++11 stand-in for std::string_view
class string_ref
{
public:
using size_type = std::size_t;
using const_iterator = const char*;
string_ref() noexcept = default;
string_ref(const char* s) : m_data(s), m_size(std::strlen(s)) {} // NOLINT(google-explicit-constructor,hicpp-explicit-conversions)
string_ref(const char* s, std::size_t n) noexcept : m_data(s), m_size(n) {}
template<typename Traits, typename Alloc>
string_ref(const std::basic_string<char, Traits, Alloc>& s) noexcept : m_data(s.data()), m_size(s.size()) {} // NOLINT(google-explicit-constructor,hicpp-explicit-conversions)
const char* data() const noexcept
{
return m_data;
}
std::size_t size() const noexcept
{
return m_size;
}
std::size_t length() const noexcept
{
return m_size;
}
bool empty() const noexcept
{
return m_size == 0;
}
const char* begin() const noexcept
{
return m_data;
}
const char* end() const noexcept
{
return m_data + m_size;
}
char operator[](std::size_t i) const noexcept
{
return m_data[i];
}
template<typename Traits, typename Alloc>
explicit operator std::basic_string<char, Traits, Alloc>() const
{
return std::basic_string<char, Traits, Alloc>(m_data, m_size);
}
friend bool operator==(string_ref a, string_ref b) noexcept
{
return a.m_size == b.m_size && (a.m_size == 0 || std::memcmp(a.m_data, b.m_data, a.m_size) == 0);
}
friend bool operator!=(string_ref a, string_ref b) noexcept
{
return !(a == b);
}
friend bool operator<(string_ref a, string_ref b) noexcept
{
const int c = std::memcmp(a.m_data, b.m_data, (std::min)(a.m_size, b.m_size));
return c != 0 ? c < 0 : a.m_size < b.m_size;
}
#ifndef JSON_NO_IO
friend std::ostream& operator<<(std::ostream& o, string_ref s)
{
return o.write(s.m_data, static_cast<std::streamsize>(s.m_size));
}
#endif
private:
const char* m_data = "";
std::size_t m_size = 0;
};
#endif
} // namespace view
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
+184 -422
View File
@@ -45,6 +45,7 @@
#include <nlohmann/adl_serializer.hpp>
#include <nlohmann/byte_container_with_subtype.hpp>
#include <nlohmann/detail/abi_config.hpp>
#include <nlohmann/detail/conversions/from_json.hpp>
#include <nlohmann/detail/conversions/to_json.hpp>
#include <nlohmann/detail/exceptions.hpp>
@@ -6106,256 +6107,21 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
{
// the patch
basic_json result(value_t::array);
diff_recursively(result, source, target, path, 0);
return result;
}
private:
/// @brief two arrays or two objects @ref diff_iteratively is diffing
struct diff_frame
{
diff_frame(const basic_json* source_, const basic_json* target_, const std::size_t path_length_) noexcept
: source(source_), target(target_), path_length(path_length_)
{}
// declared for GCC's -Weffc++, which asks for them in a class with
// pointer members and a non-trivial destructor; the exception
// specifications are left implicit, as GCC 4.8 rejects explicit ones
// that differ from them
diff_frame(const diff_frame&) = default;
diff_frame(diff_frame&&) = default;
diff_frame& operator=(const diff_frame&) = default;
diff_frame& operator=(diff_frame&&) = default;
~diff_frame() = default;
/// the values being diffed, both arrays or both objects
const basic_json* source;
const basic_json* target;
/// the length of their path in `current_path`
std::size_t path_length;
/// arrays: the next index to diff
std::size_t index = 0;
/// objects: the next member of source to look at
const_iterator member{}; // NOLINT(readability-redundant-member-init)
/// objects: the keys common to both, in source's order
std::vector<typename object_t::key_type> common_keys{}; // NOLINT(readability-redundant-member-init)
/// objects: the next entry of common_keys
std::size_t next_common = 0;
/// objects: the "add" operations for keys only target has
basic_json added_ops{}; // NOLINT(readability-redundant-member-init)
};
// The operations of a diff are built by the functions below rather than
// where they are needed: building one takes several temporaries, and
// unoptimized builds give each temporary a stack slot of its own in the
// function it appears in. In diff_recursively, which is on the call stack
// once per nesting level, that made every level cost kilobytes of stack.
/// @brief append a "replace" operation for @a path with @a value to @a result
static void diff_replace(basic_json& result, const string_t& path, const basic_json& value)
{
result.push_back(
{
{"op", "replace"}, {"path", path}, {"value", value}
});
}
/// @brief append a "remove" operation for @a path to @a result
static void diff_remove(basic_json& result, const string_t& path)
{
result.push_back(object(
{
{"op", "remove"}, {"path", path}
}));
}
/// @brief append an "add" operation for @a path with @a value to @a result
static void diff_add(basic_json& result, const string_t& path, const basic_json& value)
{
result.push_back(
{
{"op", "add"}, {"path", path}, {"value", value}
});
}
/// @brief append the "remove" operations for the elements of array
/// @a source from @a index on, and the "add" operations for the
/// elements of array @a target from source's size on, to @a result
static void diff_array_tails(basic_json& result, const basic_json& source, const basic_json& target,
const string_t& path, const std::size_t index)
{
// remove my remaining elements, highest index first; appending
// in that order avoids the quadratic reinsertion done before
for (std::size_t j = source.size(); j > index; --j)
{
diff_remove(result, detail::concat<string_t>(path, '/', detail::to_string<string_t>(j - 1)));
}
// add other remaining elements
for (std::size_t i = source.size(); i < target.size(); ++i)
{
diff_add(result, detail::concat<string_t>(path, "/-"), target[i]);
}
}
/*!
@brief compare the keys of objects @a source and @a target
If object_t does not keep its members in insertion order, or if the keys
both objects have are in the same order in both, and the keys only
@a target has come after them, stores the keys common to both in
source's order in @a common_keys, stores the "add" operations for the keys
only @a target has in @a added_ops, and returns true: the caller then diffs
the objects member by member. Otherwise, appends operations that remove
every member of @a source and add every member of @a target to @a result,
and returns false.
*/
static bool diff_object_keys(basic_json& result, const basic_json& source, const basic_json& target,
const string_t& path, std::vector<typename object_t::key_type>& common_keys,
basic_json& added_ops)
{
// first pass: record, for every source key, whether it is
// common to both objects (in source's iteration order) or
// was deleted (i.e., in source but not in target) -- this is
// a by-product of the target.find() call already needed to
// tell the two cases apart, so it adds no extra lookups. The
// "remove" ops themselves are emitted later, interleaved
// with the per-key diffs in the caller's fast path, to match
// source's original iteration order (as the original,
// pre-reordering-aware implementation did) instead of
// grouping all removes before all per-key diffs.
std::vector<typename object_t::key_type> common_keys_source_order;
for (auto it = source.cbegin(); it != source.cend(); ++it)
{
if (target.find(it.key()) != target.end())
{
common_keys_source_order.push_back(it.key());
}
}
// second pass: find keys that were added (i.e., in target but
// not in source), and record the keys common to both, in
// target's iteration order -- again a by-product of the
// source.find() call already needed to detect added keys. At
// the same time, determine whether every added key comes
// after every common key in target's order (a precondition
// for the fast path, which only ever appends new keys
// at the very end). Both are only needed for an object_t that
// keeps its members in insertion order, such as the one
// backing `ordered_json`; for any other object_t, the fast
// path is always taken and they are not computed.
// The patch ops for keys that were added (i.e., in target but not
// in source) are built here so the fast path can reuse
// them without a second source.find() per target key. Only
// used by the fast path -- the slow (reordering) path
// rebuilds "add" ops for every key itself.
std::vector<typename object_t::key_type> common_keys_target_order;
bool new_keys_form_suffix = true;
bool seen_new_key = false;
for (auto it = target.cbegin(); it != target.cend(); ++it)
{
if (source.find(it.key()) == source.end())
{
seen_new_key = true;
diff_add(added_ops, detail::concat<string_t>(path, '/', detail::escape(it.key())), it.value());
}
else
{
#ifdef JSON_HEDLEY_MSVC_VERSION
#pragma warning(push )
#pragma warning(disable : 4127) // ignore warning to replace if with if constexpr
#endif
if (detail::is_ordered_map<object_t>::value)
{
common_keys_target_order.push_back(it.key());
if (seen_new_key)
{
new_keys_form_suffix = false;
}
}
#ifdef JSON_HEDLEY_MSVC_VERSION
#pragma warning( pop )
#endif
}
}
// Only an object type that keeps its members in insertion
// order, such as nlohmann::ordered_map, can need reordering:
// patch() appends a new member at the end of such an object.
// Any other object type places its members itself - std::map
// in key order, a hash map in an order its operator== ignores -
// so a member-by-member diff always reproduces target there.
if (!detail::is_ordered_map<object_t>::value
|| (common_keys_source_order == common_keys_target_order && new_keys_form_suffix))
{
// fast path: order of common keys already matches (or the
// object_t's iteration order does not depend on
// insertion history), so a plain per-key diff is correct
// and minimal, as before
common_keys = std::move(common_keys_source_order);
return true;
}
// slow path: the common keys are in a different relative
// order in source and target (only possible for a
// reorderable object_t like ordered_map). Building a
// minimal reordering patch is a nontrivial (LCS-like)
// problem; instead, remove every source key -- both
// deleted keys (which must be removed regardless) and
// common keys (removed so they can be re-added in
// target's order) -- and re-add every key that should
// remain, with its final target value, in target's
// order. basic_json::patch()'s "add" operation on an
// object uses operator[], which appends at the end for a
// vector-backed insertion-ordered map when the key does
// not already exist -- so removing a key and then adding
// it moves it to the end, fixing its position.
for (auto it = source.cbegin(); it != source.cend(); ++it)
{
diff_remove(result, detail::concat<string_t>(path, '/', detail::escape(it.key())));
}
// add every key that is either common (just removed
// above) or brand new, in target's iteration order, so
// that the final order after applying the patch matches
// target exactly
for (auto it = target.cbegin(); it != target.cend(); ++it)
{
diff_add(result, detail::concat<string_t>(path, '/', detail::escape(it.key())), it.value());
}
return false;
}
/*!
@brief @ref diff, for values at nesting level @a depth, appending the
operations to @a result
Diffing two arrays or objects calls this function again, once per nesting
level, so values nested deeply enough used to exhaust the call stack and
terminate the process. The descent is bounded here: once @ref
detail::recursion_depth_limit levels have been entered, @ref
diff_iteratively diffs what is left without the call stack.
*/
static void diff_recursively(basic_json& result, const basic_json& source, const basic_json& target,
const string_t& path, const std::size_t depth)
{
// if the values are the same, there is nothing to do
// if the values are the same, return an empty patch
if (source == target)
{
return;
}
if (JSON_HEDLEY_UNLIKELY(depth >= detail::recursion_depth_limit()))
{
diff_iteratively(result, source, target, path);
return;
return result;
}
if (source.type() != target.type())
{
// different types: replace value
diff_replace(result, path, target);
return;
result.push_back(
{
{"op", "replace"}, {"path", path}, {"value", target}
});
return result;
}
switch (source.type())
@@ -6367,50 +6133,200 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
while (i < source.size() && i < target.size())
{
// recursive call to compare array values at index i
diff_recursively(result, source[i], target[i], detail::concat<string_t>(path, '/', detail::to_string<string_t>(i)), depth + 1);
auto temp_diff = diff(source[i], target[i], detail::concat<string_t>(path, '/', detail::to_string<string_t>(i)));
result.insert(result.end(), temp_diff.begin(), temp_diff.end());
++i;
}
// We now reached the end of at least one array
// in a second pass, traverse the remaining elements
diff_array_tails(result, source, target, path, i);
// remove my remaining elements, highest index first; appending
// in that order avoids the quadratic reinsertion done before
for (std::size_t j = source.size(); j > i; --j)
{
result.push_back(object(
{
{"op", "remove"},
{"path", detail::concat<string_t>(path, '/', detail::to_string<string_t>(j - 1))}
}));
}
i = source.size();
// add other remaining elements
while (i < target.size())
{
result.push_back(
{
{"op", "add"},
{"path", detail::concat<string_t>(path, "/-")},
{"value", target[i]}
});
++i;
}
break;
}
case value_t::object:
{
std::vector<typename object_t::key_type> common_keys;
basic_json added_ops(value_t::array);
if (diff_object_keys(result, source, target, path, common_keys, added_ops))
// first pass: record, for every source key, whether it is
// common to both objects (in source's iteration order) or
// was deleted (i.e., in source but not in target) -- this is
// a by-product of the target.find() call already needed to
// tell the two cases apart, so it adds no extra lookups. The
// "remove" ops themselves are emitted later, interleaved
// with the recursive per-key diffs in the fast path below,
// to match source's original iteration order (as the
// original, pre-reordering-aware implementation did) instead
// of grouping all removes before all recursive diffs.
std::vector<typename object_t::key_type> common_keys_source_order;
for (auto it = source.cbegin(); it != source.cend(); ++it)
{
// fast path: common_keys is, by construction, the
// subsequence of source's keys that are common to both
// objects, in source's iteration order -- so it can be
// walked in lockstep with `source` using a cheap key
// comparison instead of another lookup. Deleted keys
// (those source keys not in common_keys) are interleaved
// here too, in source's original order, to match the
// historical (pre-reordering-aware) output order.
auto common_it = common_keys.cbegin();
if (target.find(it.key()) != target.end())
{
common_keys_source_order.push_back(it.key());
}
}
// second pass: find keys that were added (i.e., in target but
// not in source), and record the keys common to both, in
// target's iteration order -- again a by-product of the
// source.find() call already needed to detect added keys. At
// the same time, determine whether every added key comes
// after every common key in target's order (a precondition
// for the fast path below, which only ever appends new keys
// at the very end). Both are only needed for an object_t that
// keeps its members in insertion order, such as the one
// backing `ordered_json`; for any other object_t, the fast
// path is always taken and they are not computed.
// patch ops for keys that were added (i.e., in target but not
// in source); built here so the fast path below can reuse
// them without a second source.find() per target key. Only
// used by the fast path -- the slow (reordering) path
// rebuilds "add" ops for every key itself.
std::vector<typename object_t::key_type> common_keys_target_order;
basic_json added_ops(value_t::array);
bool new_keys_form_suffix = true;
bool seen_new_key = false;
for (auto it = target.cbegin(); it != target.cend(); ++it)
{
if (source.find(it.key()) == source.end())
{
seen_new_key = true;
const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
added_ops.push_back(
{
{"op", "add"}, {"path", path_key},
{"value", it.value()}
});
}
else
{
#ifdef JSON_HEDLEY_MSVC_VERSION
#pragma warning(push )
#pragma warning(disable : 4127) // ignore warning to replace if with if constexpr
#endif
if (detail::is_ordered_map<object_t>::value)
{
common_keys_target_order.push_back(it.key());
if (seen_new_key)
{
new_keys_form_suffix = false;
}
}
#ifdef JSON_HEDLEY_MSVC_VERSION
#pragma warning( pop )
#endif
}
}
// Only an object type that keeps its members in insertion
// order, such as nlohmann::ordered_map, can need reordering:
// patch() appends a new member at the end of such an object.
// Any other object type places its members itself - std::map
// in key order, a hash map in an order its operator== ignores -
// so a member-by-member diff always reproduces target there.
if (!detail::is_ordered_map<object_t>::value
|| (common_keys_source_order == common_keys_target_order && new_keys_form_suffix))
{
// fast path: order of common keys already matches (or the
// object_t's iteration order does not depend on
// insertion history), so a plain per-key recursive diff
// is correct and minimal, as before. common_keys_source_order
// is, by construction, the subsequence of source's keys
// that are common to both objects, in source's iteration
// order -- so it can be walked in lockstep with `source`
// using a cheap key comparison instead of another lookup.
// Deleted keys (those source keys not in common_keys_source_order)
// are interleaved here too, in source's original order, to
// match the historical (pre-reordering-aware) output order.
auto common_it = common_keys_source_order.cbegin();
for (auto it = source.cbegin(); it != source.cend(); ++it)
{
if (common_it != common_keys.cend() && it.key() == *common_it)
if (common_it != common_keys_source_order.cend() && it.key() == *common_it)
{
diff_recursively(result, it.value(), target[it.key()], detail::concat<string_t>(path, '/', detail::escape(it.key())), depth + 1);
const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
auto temp_diff = diff(it.value(), target[it.key()], path_key);
result.insert(result.end(), temp_diff.begin(), temp_diff.end());
++common_it;
}
else
{
// found a key that is not in target -> remove it
diff_remove(result, detail::concat<string_t>(path, '/', detail::escape(it.key())));
const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
result.push_back(object(
{
{"op", "remove"}, {"path", path_key}
}));
}
}
// append the "add" ops for brand-new keys collected by
// diff_object_keys -- no second source.find() per target
// key needed
// append the "add" ops for brand-new keys collected above
// during the pass over target -- no second source.find()
// per target key needed
result.insert(result.end(), added_ops.begin(), added_ops.end());
}
else
{
// slow path: the common keys are in a different relative
// order in source and target (only possible for a
// reorderable object_t like ordered_map). Building a
// minimal reordering patch is a nontrivial (LCS-like)
// problem; instead, remove every source key -- both
// deleted keys (which must be removed regardless) and
// common keys (removed so they can be re-added in
// target's order) -- and re-add every key that should
// remain, with its final target value, in target's
// order. basic_json::patch()'s "add" operation on an
// object uses operator[], which appends at the end for a
// vector-backed insertion-ordered map when the key does
// not already exist -- so removing a key and then adding
// it moves it to the end, fixing its position.
for (auto it = source.cbegin(); it != source.cend(); ++it)
{
const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
result.push_back(object(
{
{"op", "remove"}, {"path", path_key}
}));
}
// add every key that is either common (just removed
// above) or brand new, in target's iteration order, so
// that the final order after applying the patch matches
// target exactly
for (auto it = target.cbegin(); it != target.cend(); ++it)
{
const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
result.push_back(
{
{"op", "add"}, {"path", path_key},
{"value", it.value()}
});
}
}
break;
}
@@ -6425,170 +6341,16 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
default:
{
// both primitive types: replace value
diff_replace(result, path, target);
result.push_back(
{
{"op", "replace"}, {"path", path}, {"value", target}
});
break;
}
}
return result;
}
/*!
@brief @ref diff without the call stack, appending the operations to
@a result
Produces the same operations as @ref diff_recursively. Only reached for
values nested more deeply than @ref detail::recursion_depth_limit.
*/
static void diff_iteratively(basic_json& result, const basic_json& source, const basic_json& target,
const string_t& path)
{
// The arrays and objects being diffed are kept on an explicit stack,
// and every pair of elements is still diffed completely before the
// next one, so the operations come out in the same order as in
// diff_recursively. The path of the values being diffed is kept in
// one buffer that grows and shrinks with the stack, rather than in a
// new string per level.
std::vector<diff_frame> stack;
string_t current_path = path;
// diff `s` against `t`, whose path is current_path: primitives,
// values of different types, and objects whose members were reordered
// are handled right away; arrays and other objects get a frame
const auto enter = [&result, &stack, &current_path](const basic_json & s, const basic_json & t)
{
// if the values are the same, there is nothing to do. Arrays and
// objects are not compared up front: comparing them visits
// everything below them, so doing that at every level would take
// quadratic time in the nesting depth - equal ones yield no
// operations anyway.
if ((!s.is_structured() || !t.is_structured()) && s == t)
{
return;
}
if (s.type() != t.type())
{
// different types: replace value
diff_replace(result, current_path, t);
return;
}
switch (s.type())
{
case value_t::array:
{
stack.emplace_back(&s, &t, current_path.size());
return;
}
case value_t::object:
{
std::vector<typename object_t::key_type> common_keys;
basic_json added_ops(value_t::array);
if (diff_object_keys(result, s, t, current_path, common_keys, added_ops))
{
// fast path: the frame walks source in lockstep with
// common_keys, as diff_recursively does, and appends
// added_ops once all members are done
stack.emplace_back(&s, &t, current_path.size());
stack.back().member = s.cbegin();
stack.back().common_keys = std::move(common_keys);
stack.back().added_ops = std::move(added_ops);
}
return;
}
case value_t::null:
case value_t::string:
case value_t::boolean:
case value_t::number_integer:
case value_t::number_unsigned:
case value_t::number_float:
case value_t::binary:
case value_t::discarded:
default:
{
// both primitive types: replace value
diff_replace(result, current_path, t);
return;
}
}
};
enter(source, target);
while (!stack.empty())
{
// the frame is copied out member by member and changed through
// stack.back(): enter() may push a frame and the end of the loop
// pops it, either of which would invalidate a reference to it
const basic_json* const s = stack.back().source;
const basic_json* const t = stack.back().target;
const std::size_t path_length = stack.back().path_length;
const std::size_t depth = stack.size();
if (s->is_array())
{
const auto& source_array = *s->m_data.m_value.array;
const auto& target_array = *t->m_data.m_value.array;
// first pass: traverse common elements
const std::size_t i = stack.back().index;
if (i < source_array.size() && i < target_array.size())
{
++stack.back().index;
detail::concat_into(current_path, '/', detail::to_string<string_t>(i));
enter(source_array[i], target_array[i]);
if (stack.size() == depth)
{
current_path.resize(path_length);
}
continue;
}
// We now reached the end of at least one array
// in a second pass, traverse the remaining elements
diff_array_tails(result, *s, *t, current_path, i);
}
else
{
const const_iterator it = stack.back().member;
if (it != s->cend())
{
++stack.back().member;
const std::size_t next_common = stack.back().next_common;
if (next_common < stack.back().common_keys.size() && it.key() == stack.back().common_keys[next_common])
{
++stack.back().next_common;
const basic_json& target_value = (*t)[it.key()];
detail::concat_into(current_path, '/', detail::escape(it.key()));
enter(it.value(), target_value);
if (stack.size() == depth)
{
current_path.resize(path_length);
}
}
else
{
// found a key that is not in target -> remove it
diff_remove(result, detail::concat<string_t>(current_path, '/', detail::escape(it.key())));
}
continue;
}
// append the "add" ops for brand-new keys collected when the
// object was entered
result.insert(result.end(), stack.back().added_ops.begin(), stack.back().added_ops.end());
}
// this array or object is done: continue with the one it is in
stack.pop_back();
if (!stack.empty())
{
current_path.resize(stack.back().path_length);
}
}
}
public:
/// @}
////////////////////////////////
File diff suppressed because it is too large Load Diff
+4
View File
@@ -279,6 +279,10 @@ if(json_32bit_test_only)
elseif(NOT json_32bit_test)
list(FILTER files EXCLUDE REGEX src/unit-32bit.cpp)
endif()
if(NOT JSON_MultipleHeaders)
# the internal headers of json_view are not part of a single header yet
list(FILTER files EXCLUDE REGEX src/unit-json_view_builder.cpp)
endif()
foreach(file ${files})
json_test_add_test_for(${file} MAIN test_main CXX_STANDARDS ${test_cxx_standards} ${test_force})
+599
View File
@@ -0,0 +1,599 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#pragma once
#include <array> // array
#include <cstdint> // uint32_t, uint64_t
// Number tokens that are hard to round correctly, with the IEEE-754 binary64
// and binary32 bits of their correctly rounded values (ties to even; infinity
// for an overflow, a signed zero for an underflow).
//
// For doubles and floats around 0, the smallest normal number, 1, 2^24, 2^53,
// 0.1, and the largest finite number, and for random ones, the exact midpoint
// m to the next number gives: m, m with one unit more and less in the last
// digit, m with "01" and "0...01" appended, m with trailing zeros, and m cut
// after 17 to 30 digits (rounded down and up, so that the rounding is decided
// after the 19th digit), in fixed and exponent notation, 30% of them negative.
// Tokens longer than 80 characters are left out, except for four of 700 digits
// and more. Zeros, underflow, overflow, huge exponents, and integers beyond 64
// bits complete the set. Of the 508 tokens, 134 (as double) and 150 (as
// float) need the exact comparison with the midpoint (detail::digit_comparison()).
//
// The expected bits were computed with exact rational arithmetic in Python
// (fractions.Fraction) and cross-checked with Python's float(); strtod_l and
// strtof_l of Apple's libc and of glibc agree. Generated by
// compact_hard_cases.py 5 (with hard_cases.py), see the pull request that
// added this file.
namespace float_hard_cases
{
struct hard_case
{
const char* token;
std::uint64_t bits64;
std::uint32_t bits32;
};
inline const std::array<hard_case, 508>& cases()
{
static const std::array<hard_case, 508> table =
{
{
{"-2.4703282292062327e-324", 0x8000000000000000u, 0x80000000u},
{"24703282292062328e-340", 0x0000000000000001u, 0x00000000u},
{"247032822920623272e-341", 0x0000000000000000u, 0x00000000u},
{"-0.2470328229206232721e-323", 0x8000000000000001u, 0x80000000u},
{"-0.24703282292062327208e-323", 0x8000000000000000u, 0x80000000u},
{"-2.4703282292062327209e-324", 0x8000000000000001u, 0x80000000u},
{"2.47032822920623272088e-324", 0x0000000000000000u, 0x00000000u},
{"247032822920623272089e-344", 0x0000000000000001u, 0x00000000u},
{"-247032822920623272088284396434e-353", 0x8000000000000000u, 0x80000000u},
{"0.247032822920623272088284396435e-323", 0x0000000000000001u, 0x00000000u},
{"-74109846876186981e-340", 0x8000000000000001u, 0x80000000u},
{"0.74109846876186982e-323", 0x0000000000000002u, 0x00000000u},
{"-0.7410984687618698162e-323", 0x8000000000000001u, 0x80000000u},
{"-7.410984687618698163e-324", 0x8000000000000002u, 0x80000000u},
{"7.4109846876186981626e-324", 0x0000000000000001u, 0x00000000u},
{"-74109846876186981627e-343", 0x8000000000000002u, 0x80000000u},
{"-741098468761869816264e-344", 0x8000000000000001u, 0x80000000u},
{"0.741098468761869816265e-323", 0x0000000000000002u, 0x00000000u},
{"0.741098468761869816264853189302e-323", 0x0000000000000001u, 0x00000000u},
{"-7.41098468761869816264853189303e-324", 0x8000000000000002u, 0x80000000u},
{"0.22250738585072006e-307", 0x000FFFFFFFFFFFFEu, 0x00000000u},
{"2.2250738585072007e-308", 0x000FFFFFFFFFFFFFu, 0x00000000u},
{"2.225073858507200641e-308", 0x000FFFFFFFFFFFFEu, 0x00000000u},
{"-2225073858507200642e-326", 0x800FFFFFFFFFFFFFu, 0x80000000u},
{"22250738585072006419e-327", 0x000FFFFFFFFFFFFEu, 0x00000000u},
{"0.2225073858507200642e-307", 0x000FFFFFFFFFFFFFu, 0x00000000u},
{"0.222507385850720064199e-307", 0x000FFFFFFFFFFFFEu, 0x00000000u},
{"2.225073858507200642e-308", 0x000FFFFFFFFFFFFFu, 0x00000000u},
{"-2.22507385850720064199176395546e-308", 0x800FFFFFFFFFFFFEu, 0x80000000u},
{"222507385850720064199176395547e-337", 0x000FFFFFFFFFFFFFu, 0x00000000u},
{"-2.2250738585072011e-308", 0x800FFFFFFFFFFFFFu, 0x80000000u},
{"-22250738585072012e-324", 0x8010000000000000u, 0x80000000u},
{"-2225073858507201136e-326", 0x800FFFFFFFFFFFFFu, 0x80000000u},
{"0.2225073858507201137e-307", 0x0010000000000000u, 0x00000000u},
{"0.2225073858507201136e-307", 0x000FFFFFFFFFFFFFu, 0x00000000u},
{"-2.2250738585072011361e-308", 0x8010000000000000u, 0x80000000u},
{"2.22507385850720113605e-308", 0x000FFFFFFFFFFFFFu, 0x00000000u},
{"222507385850720113606e-328", 0x0010000000000000u, 0x00000000u},
{"22250738585072011360574097967e-336", 0x000FFFFFFFFFFFFFu, 0x00000000u},
{"0.222507385850720113605740979671e-307", 0x0010000000000000u, 0x00000000u},
{"22250738585072016e-324", 0x0010000000000000u, 0x00000000u},
{"0.22250738585072017e-307", 0x0010000000000001u, 0x00000000u},
{"0.222507385850720163e-307", 0x0010000000000000u, 0x00000000u},
{"2.225073858507201631e-308", 0x0010000000000001u, 0x00000000u},
{"-2.2250738585072016301e-308", 0x8010000000000000u, 0x80000000u},
{"22250738585072016302e-327", 0x0010000000000001u, 0x00000000u},
{"-222507385850720163012e-328", 0x8010000000000000u, 0x80000000u},
{"0.222507385850720163013e-307", 0x0010000000000001u, 0x00000000u},
{"0.222507385850720163012305563795e-307", 0x0010000000000000u, 0x00000000u},
{"-2.22507385850720163012305563796e-308", 0x8010000000000001u, 0x80000000u},
{"0.17976931348623156E+309", 0x7FEFFFFFFFFFFFFEu, 0x7F800000u},
{"1.7976931348623157e308", 0x7FEFFFFFFFFFFFFFu, 0x7F800000u},
{"1.797693134862315608e308", 0x7FEFFFFFFFFFFFFEu, 0x7F800000u},
{"-1797693134862315609e290", 0xFFEFFFFFFFFFFFFFu, 0xFF800000u},
{"-17976931348623156083e289", 0xFFEFFFFFFFFFFFFEu, 0xFF800000u},
{"-0.17976931348623156084E+309", 0xFFEFFFFFFFFFFFFFu, 0xFF800000u},
{"0.179769313486231560835E+309", 0x7FEFFFFFFFFFFFFEu, 0x7F800000u},
{"-1.79769313486231560836e308", 0xFFEFFFFFFFFFFFFFu, 0xFF800000u},
{"1.79769313486231560835325876058e308", 0x7FEFFFFFFFFFFFFEu, 0x7F800000u},
{"179769313486231560835325876059e279", 0x7FEFFFFFFFFFFFFFu, 0x7F800000u},
{"1.7976931348623158e308", 0x7FEFFFFFFFFFFFFFu, 0x7F800000u},
{"17976931348623159e292", 0x7FF0000000000000u, 0x7F800000u},
{"1797693134862315807e290", 0x7FEFFFFFFFFFFFFFu, 0x7F800000u},
{"0.1797693134862315808E+309", 0x7FF0000000000000u, 0x7F800000u},
{"0.17976931348623158079E+309", 0x7FEFFFFFFFFFFFFFu, 0x7F800000u},
{"-1.797693134862315808e308", 0xFFF0000000000000u, 0xFF800000u},
{"1.79769313486231580793e308", 0x7FEFFFFFFFFFFFFFu, 0x7F800000u},
{"179769313486231580794e288", 0x7FF0000000000000u, 0x7F800000u},
{"179769313486231580793728971405e279", 0x7FEFFFFFFFFFFFFFu, 0x7F800000u},
{"-0.179769313486231580793728971406E+309", 0xFFF0000000000000u, 0xFF800000u},
{"100000000000000011102230246251565404236316680908203125e-53", 0x3FF0000000000000u, 0x3F800000u},
{"-1.00000000000000011102230246251565404236316680908203126", 0xBFF0000000000001u, 0xBF800000u},
{"1.00000000000000011102230246251565404236316680908203124e0", 0x3FF0000000000000u, 0x3F800000u},
{"10000000000000001110223024625156540423631668090820312501e-55", 0x3FF0000000000001u, 0x3F800000u},
{"1.00000000000000011102230246251565404236316680908203125000000000000000000001", 0x3FF0000000000001u, 0x3F800000u},
{"10000000000000001e-16", 0x3FF0000000000000u, 0x3F800000u},
{"1.0000000000000002", 0x3FF0000000000001u, 0x3F800000u},
{"1.000000000000000111", 0x3FF0000000000000u, 0x3F800000u},
{"1.000000000000000112e0", 0x3FF0000000000001u, 0x3F800000u},
{"1.000000000000000111e0", 0x3FF0000000000000u, 0x3F800000u},
{"-10000000000000001111e-19", 0xBFF0000000000001u, 0xBF800000u},
{"-100000000000000011102e-20", 0xBFF0000000000000u, 0xBF800000u},
{"-1.00000000000000011103", 0xBFF0000000000001u, 0xBF800000u},
{"1.00000000000000011102230246251", 0x3FF0000000000000u, 0x3F800000u},
{"1.00000000000000011102230246252e0", 0x3FF0000000000001u, 0x3F800000u},
{"-0.999999999999999944488848768742172978818416595458984375", 0xBFF0000000000000u, 0xBF800000u},
{"-9.99999999999999944488848768742172978818416595458984376e-1", 0xBFF0000000000000u, 0xBF800000u},
{"999999999999999944488848768742172978818416595458984374e-54", 0x3FEFFFFFFFFFFFFFu, 0x3F800000u},
{"0.99999999999999994448884876874217297881841659545898437501", 0x3FF0000000000000u, 0x3F800000u},
{"9.99999999999999944488848768742172978818416595458984375000000000000000000001e-1", 0x3FF0000000000000u, 0x3F800000u},
{"-0.99999999999999994", 0xBFEFFFFFFFFFFFFFu, 0xBF800000u},
{"9.9999999999999995e-1", 0x3FF0000000000000u, 0x3F800000u},
{"9.999999999999999444e-1", 0x3FEFFFFFFFFFFFFFu, 0x3F800000u},
{"9999999999999999445e-19", 0x3FF0000000000000u, 0x3F800000u},
{"99999999999999994448e-20", 0x3FEFFFFFFFFFFFFFu, 0x3F800000u},
{"-0.99999999999999994449", 0xBFF0000000000000u, 0xBF800000u},
{"0.999999999999999944488", 0x3FEFFFFFFFFFFFFFu, 0x3F800000u},
{"-9.99999999999999944489e-1", 0xBFF0000000000000u, 0xBF800000u},
{"9.99999999999999944488848768742e-1", 0x3FEFFFFFFFFFFFFFu, 0x3F800000u},
{"999999999999999944488848768743e-30", 0x3FF0000000000000u, 0x3F800000u},
{"-9.007199254740993e15", 0xC340000000000000u, 0xDA000000u},
{"9007199254740994e0", 0x4340000000000001u, 0x5A000000u},
{"9007199254740992", 0x4340000000000000u, 0x5A000000u},
{"9.00719925474099301e15", 0x4340000000000001u, 0x5A000000u},
{"9007199254740993000000000000000000001e-21", 0x4340000000000001u, 0x5A000000u},
{"-9007199254740993.000000000000000000000000000000", 0xC340000000000000u, 0xDA000000u},
{"90071992547409915e-1", 0x4340000000000000u, 0x5A000000u},
{"-9007199254740991.6", 0xC340000000000000u, 0xDA000000u},
{"9.0071992547409914e15", 0x433FFFFFFFFFFFFFu, 0x5A000000u},
{"-9007199254740991501e-3", 0xC340000000000000u, 0xDA000000u},
{"9007199254740991.5000000000000000000001", 0x4340000000000000u, 0x5A000000u},
{"9.0071992547409915000000000000000000000000000000e15", 0x4340000000000000u, 0x5A000000u},
{"0.100000000000000012490009027033011079765856266021728515625", 0x3FB999999999999Au, 0x3DCCCCCDu},
{"1.00000000000000012490009027033011079765856266021728515626e-1", 0x3FB999999999999Bu, 0x3DCCCCCDu},
{"100000000000000012490009027033011079765856266021728515624e-57", 0x3FB999999999999Au, 0x3DCCCCCDu},
{"0.10000000000000001249000902703301107976585626602172851562501", 0x3FB999999999999Bu, 0x3DCCCCCDu},
{"0.10000000000000001", 0x3FB999999999999Au, 0x3DCCCCCDu},
{"1.0000000000000002e-1", 0x3FB999999999999Bu, 0x3DCCCCCDu},
{"-1.000000000000000124e-1", 0xBFB999999999999Au, 0xBDCCCCCDu},
{"1000000000000000125e-19", 0x3FB999999999999Bu, 0x3DCCCCCDu},
{"-10000000000000001249e-20", 0xBFB999999999999Au, 0xBDCCCCCDu},
{"0.1000000000000000125", 0x3FB999999999999Bu, 0x3DCCCCCDu},
{"0.10000000000000001249", 0x3FB999999999999Au, 0x3DCCCCCDu},
{"1.00000000000000012491e-1", 0x3FB999999999999Bu, 0x3DCCCCCDu},
{"1.00000000000000012490009027033e-1", 0x3FB999999999999Au, 0x3DCCCCCDu},
{"100000000000000012490009027034e-30", 0x3FB999999999999Bu, 0x3DCCCCCDu},
{"2.45134755833537796875e14", 0x42EBDE5C4164D83Au, 0x575EF2E2u},
{"-245134755833537796876e-6", 0xC2EBDE5C4164D83Au, 0xD75EF2E2u},
{"-245134755833537.796874", 0xC2EBDE5C4164D839u, 0xD75EF2E2u},
{"2.4513475583353779687501e14", 0x42EBDE5C4164D83Au, 0x575EF2E2u},
{"245134755833537796875000000000000000000001e-27", 0x42EBDE5C4164D83Au, 0x575EF2E2u},
{"245134755833537.796875000000000000000000000000000000", 0x42EBDE5C4164D83Au, 0x575EF2E2u},
{"2.4513475583353779e14", 0x42EBDE5C4164D839u, 0x575EF2E2u},
{"2451347558335378e-1", 0x42EBDE5C4164D83Au, 0x575EF2E2u},
{"2451347558335377968e-4", 0x42EBDE5C4164D839u, 0x575EF2E2u},
{"245134755833537.7969", 0x42EBDE5C4164D83Au, 0x575EF2E2u},
{"245134755833537.79687", 0x42EBDE5C4164D839u, 0x575EF2E2u},
{"2.4513475583353779688e14", 0x42EBDE5C4164D83Au, 0x575EF2E2u},
{"181510327827821147441864013671875e-23", 0x41DB0C11CB91CE38u, 0x4ED8608Eu},
{"-1815103278.27821147441864013671876", 0xC1DB0C11CB91CE38u, 0xCED8608Eu},
{"1.81510327827821147441864013671874e9", 0x41DB0C11CB91CE37u, 0x4ED8608Eu},
{"18151032782782114744186401367187501e-25", 0x41DB0C11CB91CE38u, 0x4ED8608Eu},
{"1815103278.27821147441864013671875000000000000000000001", 0x41DB0C11CB91CE38u, 0x4ED8608Eu},
{"-1.81510327827821147441864013671875000000000000000000000000000000e9", 0xC1DB0C11CB91CE38u, 0xCED8608Eu},
{"18151032782782114e-7", 0x41DB0C11CB91CE37u, 0x4ED8608Eu},
{"1815103278.2782115", 0x41DB0C11CB91CE38u, 0x4ED8608Eu},
{"1815103278.278211474", 0x41DB0C11CB91CE37u, 0x4ED8608Eu},
{"-1.815103278278211475e9", 0xC1DB0C11CB91CE38u, 0xCED8608Eu},
{"1.8151032782782114744e9", 0x41DB0C11CB91CE37u, 0x4ED8608Eu},
{"-18151032782782114745e-10", 0xC1DB0C11CB91CE38u, 0xCED8608Eu},
{"181510327827821147441e-11", 0x41DB0C11CB91CE37u, 0x4ED8608Eu},
{"1815103278.27821147442", 0x41DB0C11CB91CE38u, 0x4ED8608Eu},
{"1815103278.27821147441864013671", 0x41DB0C11CB91CE37u, 0x4ED8608Eu},
{"1.81510327827821147441864013672e9", 0x41DB0C11CB91CE38u, 0x4ED8608Eu},
{"3809325632181785344", 0x43CA6EB8BD69FE2Au, 0x5E5375C6u},
{"3.809325632181785345e18", 0x43CA6EB8BD69FE2Au, 0x5E5375C6u},
{"3809325632181785343e0", 0x43CA6EB8BD69FE29u, 0x5E5375C6u},
{"3809325632181785344.01", 0x43CA6EB8BD69FE2Au, 0x5E5375C6u},
{"3.809325632181785344000000000000000000001e18", 0x43CA6EB8BD69FE2Au, 0x5E5375C6u},
{"3809325632181785344000000000000000000000000000000e-30", 0x43CA6EB8BD69FE2Au, 0x5E5375C6u},
{"3809325632181785300", 0x43CA6EB8BD69FE29u, 0x5E5375C6u},
{"3.8093256321817854e18", 0x43CA6EB8BD69FE2Au, 0x5E5375C6u},
{"4.046966549916366943359375e12", 0x428D7210076CE2F0u, 0x546B9080u},
{"4046966549916366943359376e-12", 0x428D7210076CE2F0u, 0x546B9080u},
{"4046966549916.366943359374", 0x428D7210076CE2EFu, 0x546B9080u},
{"4.04696654991636694335937501e12", 0x428D7210076CE2F0u, 0x546B9080u},
{"4046966549916366943359375000000000000000000001e-33", 0x428D7210076CE2F0u, 0x546B9080u},
{"-4046966549916.366943359375000000000000000000000000000000", 0xC28D7210076CE2F0u, 0xD46B9080u},
{"4.0469665499163669e12", 0x428D7210076CE2EFu, 0x546B9080u},
{"4046966549916367e-3", 0x428D7210076CE2F0u, 0x546B9080u},
{"4046966549916366943e-6", 0x428D7210076CE2EFu, 0x546B9080u},
{"4046966549916.366944", 0x428D7210076CE2F0u, 0x546B9080u},
{"4046966549916.3669433", 0x428D7210076CE2EFu, 0x546B9080u},
{"4.0469665499163669434e12", 0x428D7210076CE2F0u, 0x546B9080u},
{"-4.04696654991636694335e12", 0xC28D7210076CE2EFu, 0xD46B9080u},
{"404696654991636694336e-8", 0x428D7210076CE2F0u, 0x546B9080u},
{"28093802557000874e154", 0x63529C3B77330BDBu, 0x7F800000u},
{"0.28093802557000875E+171", 0x63529C3B77330BDCu, 0x7F800000u},
{"0.2809380255700087447E+171", 0x63529C3B77330BDBu, 0x7F800000u},
{"2.809380255700087448e170", 0x63529C3B77330BDCu, 0x7F800000u},
{"2.8093802557000874472e170", 0x63529C3B77330BDBu, 0x7F800000u},
{"28093802557000874473e151", 0x63529C3B77330BDCu, 0x7F800000u},
{"280938025570008744728e150", 0x63529C3B77330BDBu, 0x7F800000u},
{"-0.280938025570008744729E+171", 0xE3529C3B77330BDCu, 0xFF800000u},
{"-0.280938025570008744728403667979E+171", 0xE3529C3B77330BDBu, 0xFF800000u},
{"2.8093802557000874472840366798e170", 0x63529C3B77330BDCu, 0x7F800000u},
{"0.39523280297734525e-154", 0x1FE0F51BF17FD374u, 0x00000000u},
{"-3.9523280297734526e-155", 0x9FE0F51BF17FD375u, 0x80000000u},
{"3.952328029773452547e-155", 0x1FE0F51BF17FD374u, 0x00000000u},
{"3952328029773452548e-173", 0x1FE0F51BF17FD375u, 0x00000000u},
{"-39523280297734525478e-174", 0x9FE0F51BF17FD374u, 0x80000000u},
{"0.39523280297734525479e-154", 0x1FE0F51BF17FD375u, 0x00000000u},
{"-0.395232802977345254787e-154", 0x9FE0F51BF17FD374u, 0x80000000u},
{"-3.95232802977345254788e-155", 0x9FE0F51BF17FD375u, 0x80000000u},
{"-3.95232802977345254787245825501e-155", 0x9FE0F51BF17FD374u, 0x80000000u},
{"395232802977345254787245825502e-184", 0x1FE0F51BF17FD375u, 0x00000000u},
{"-1.0790205420931879e-276", 0x86A3209CA6233255u, 0x80000000u},
{"-1079020542093188e-291", 0x86A3209CA6233256u, 0x80000000u},
{"-1079020542093187947e-294", 0x86A3209CA6233255u, 0x80000000u},
{"0.1079020542093187948e-275", 0x06A3209CA6233256u, 0x00000000u},
{"0.1079020542093187947e-275", 0x06A3209CA6233255u, 0x00000000u},
{"1.0790205420931879471e-276", 0x06A3209CA6233256u, 0x00000000u},
{"1.07902054209318794701e-276", 0x06A3209CA6233255u, 0x00000000u},
{"-107902054209318794702e-296", 0x86A3209CA6233256u, 0x80000000u},
{"107902054209318794701153285302e-305", 0x06A3209CA6233255u, 0x00000000u},
{"-0.107902054209318794701153285303e-275", 0x86A3209CA6233256u, 0x80000000u},
{"58530471071351308e-228", 0x1413B446E6A16A3Bu, 0x00000000u},
{"-0.58530471071351309e-211", 0x9413B446E6A16A3Cu, 0x80000000u},
{"0.5853047107135130893e-211", 0x1413B446E6A16A3Bu, 0x00000000u},
{"-5.853047107135130894e-212", 0x9413B446E6A16A3Cu, 0x80000000u},
{"5.853047107135130893e-212", 0x1413B446E6A16A3Bu, 0x00000000u},
{"58530471071351308931e-231", 0x1413B446E6A16A3Cu, 0x00000000u},
{"-585304710713513089304e-232", 0x9413B446E6A16A3Bu, 0x80000000u},
{"0.585304710713513089305e-211", 0x1413B446E6A16A3Cu, 0x00000000u},
{"-0.585304710713513089304248824438e-211", 0x9413B446E6A16A3Bu, 0x80000000u},
{"5.85304710713513089304248824439e-212", 0x1413B446E6A16A3Cu, 0x00000000u},
{"0.19334214893983531e-78", 0x2F96ECBF1CFB10F6u, 0x00000000u},
{"1.9334214893983532e-79", 0x2F96ECBF1CFB10F7u, 0x00000000u},
{"1.933421489398353102e-79", 0x2F96ECBF1CFB10F6u, 0x00000000u},
{"1933421489398353103e-97", 0x2F96ECBF1CFB10F7u, 0x00000000u},
{"19334214893983531023e-98", 0x2F96ECBF1CFB10F6u, 0x00000000u},
{"0.19334214893983531024e-78", 0x2F96ECBF1CFB10F7u, 0x00000000u},
{"0.193342148939835310231e-78", 0x2F96ECBF1CFB10F6u, 0x00000000u},
{"1.93342148939835310232e-79", 0x2F96ECBF1CFB10F7u, 0x00000000u},
{"-1.93342148939835310231359014704e-79", 0xAF96ECBF1CFB10F6u, 0x80000000u},
{"193342148939835310231359014705e-108", 0x2F96ECBF1CFB10F7u, 0x00000000u},
{"2.9873358928024455e227", 0x6F2938807814E8A2u, 0x7F800000u},
{"29873358928024456e211", 0x6F2938807814E8A3u, 0x7F800000u},
{"298733589280244551e210", 0x6F2938807814E8A2u, 0x7F800000u},
{"0.2987335892802445511E+228", 0x6F2938807814E8A3u, 0x7F800000u},
{"0.29873358928024455109E+228", 0x6F2938807814E8A2u, 0x7F800000u},
{"2.987335892802445511e227", 0x6F2938807814E8A3u, 0x7F800000u},
{"-2.98733589280244551098e227", 0xEF2938807814E8A2u, 0xFF800000u},
{"-298733589280244551099e207", 0xEF2938807814E8A3u, 0xFF800000u},
{"298733589280244551098081559931e198", 0x6F2938807814E8A2u, 0x7F800000u},
{"0.298733589280244551098081559932E+228", 0x6F2938807814E8A3u, 0x7F800000u},
{"7.0064923216240853e-46", 0x3690000000000000u, 0x00000000u},
{"70064923216240854e-62", 0x3690000000000000u, 0x00000001u},
{"7006492321624085354e-64", 0x3690000000000000u, 0x00000000u},
{"-0.7006492321624085355e-45", 0xB690000000000000u, 0x80000001u},
{"0.70064923216240853546e-45", 0x3690000000000000u, 0x00000000u},
{"7.0064923216240853547e-46", 0x3690000000000000u, 0x00000001u},
{"7.00649232162408535461e-46", 0x3690000000000000u, 0x00000000u},
{"-700649232162408535462e-66", 0xB690000000000000u, 0x80000001u},
{"-700649232162408535461864791644e-75", 0xB690000000000000u, 0x80000000u},
{"0.700649232162408535461864791645e-45", 0x3690000000000000u, 0x00000001u},
{"21019476964872256e-61", 0x36A8000000000000u, 0x00000001u},
{"0.21019476964872257e-44", 0x36A8000000000000u, 0x00000002u},
{"-0.2101947696487225606e-44", 0xB6A8000000000000u, 0x80000001u},
{"-2.101947696487225607e-45", 0xB6A8000000000000u, 0x80000002u},
{"2.1019476964872256063e-45", 0x36A8000000000000u, 0x00000001u},
{"21019476964872256064e-64", 0x36A8000000000000u, 0x00000002u},
{"-210194769648722560638e-65", 0xB6A8000000000000u, 0x80000001u},
{"-0.210194769648722560639e-44", 0xB6A8000000000000u, 0x80000002u},
{"-0.210194769648722560638559437493e-44", 0xB6A8000000000000u, 0x80000001u},
{"2.10194769648722560638559437494e-45", 0x36A8000000000000u, 0x00000002u},
{"0.11754941406275178e-37", 0x380FFFFFA0000000u, 0x007FFFFEu},
{"1.1754941406275179e-38", 0x380FFFFFA0000000u, 0x007FFFFFu},
{"-1.175494140627517859e-38", 0xB80FFFFFA0000000u, 0x807FFFFEu},
{"117549414062751786e-55", 0x380FFFFFA0000000u, 0x007FFFFFu},
{"-11754941406275178592e-57", 0xB80FFFFFA0000000u, 0x807FFFFEu},
{"0.11754941406275178593e-37", 0x380FFFFFA0000000u, 0x007FFFFFu},
{"0.117549414062751785924e-37", 0x380FFFFFA0000000u, 0x007FFFFEu},
{"1.17549414062751785925e-38", 0x380FFFFFA0000000u, 0x007FFFFFu},
{"-1.17549414062751785924617589866e-38", 0xB80FFFFFA0000000u, 0x807FFFFEu},
{"117549414062751785924617589867e-67", 0x380FFFFFA0000000u, 0x007FFFFFu},
{"1.1754942807573642e-38", 0x380FFFFFDFFFFFFFu, 0x007FFFFFu},
{"11754942807573643e-54", 0x380FFFFFE0000000u, 0x00800000u},
{"1175494280757364291e-56", 0x380FFFFFE0000000u, 0x007FFFFFu},
{"0.1175494280757364292e-37", 0x380FFFFFE0000000u, 0x00800000u},
{"0.11754942807573642917e-37", 0x380FFFFFE0000000u, 0x007FFFFFu},
{"-1.1754942807573642918e-38", 0xB80FFFFFE0000000u, 0x80800000u},
{"-1.17549428075736429172e-38", 0xB80FFFFFE0000000u, 0x807FFFFFu},
{"-117549428075736429173e-58", 0xB80FFFFFE0000000u, 0x80800000u},
{"117549428075736429172788299103e-67", 0x380FFFFFE0000000u, 0x007FFFFFu},
{"0.117549428075736429172788299104e-37", 0x380FFFFFE0000000u, 0x00800000u},
{"11754944208872107e-54", 0x3810000010000000u, 0x00800000u},
{"-0.11754944208872108e-37", 0xB810000010000000u, 0x80800001u},
{"0.1175494420887210724e-37", 0x3810000010000000u, 0x00800000u},
{"1.175494420887210725e-38", 0x3810000010000000u, 0x00800001u},
{"1.1754944208872107242e-38", 0x3810000010000000u, 0x00800000u},
{"-11754944208872107243e-57", 0xB810000010000000u, 0x80800001u},
{"-11754944208872107242e-57", 0xB810000010000000u, 0x80800000u},
{"0.117549442088721072421e-37", 0x3810000010000000u, 0x00800001u},
{"0.11754944208872107242095900834e-37", 0x3810000010000000u, 0x00800000u},
{"-1.17549442088721072420959008341e-38", 0xB810000010000000u, 0x80800001u},
{"340282336497324057985868971510891282432", 0x47EFFFFFD0000000u, 0x7F7FFFFEu},
{"-3.40282336497324057985868971510891282433e38", 0xC7EFFFFFD0000000u, 0xFF7FFFFFu},
{"340282336497324057985868971510891282431e0", 0x47EFFFFFD0000000u, 0x7F7FFFFEu},
{"340282336497324057985868971510891282432.01", 0x47EFFFFFD0000000u, 0x7F7FFFFFu},
{"3.40282336497324057985868971510891282432000000000000000000001e38", 0x47EFFFFFD0000000u, 0x7F7FFFFFu},
{"340282336497324057985868971510891282432000000000000000000000000000000e-30", 0x47EFFFFFD0000000u, 0x7F7FFFFEu},
{"340282336497324050000000000000000000000", 0x47EFFFFFD0000000u, 0x7F7FFFFEu},
{"3.4028233649732406e38", 0x47EFFFFFD0000000u, 0x7F7FFFFFu},
{"3.402823364973240579e38", 0x47EFFFFFD0000000u, 0x7F7FFFFEu},
{"340282336497324058e21", 0x47EFFFFFD0000000u, 0x7F7FFFFFu},
{"34028233649732405798e19", 0x47EFFFFFD0000000u, 0x7F7FFFFEu},
{"340282336497324057990000000000000000000", 0x47EFFFFFD0000000u, 0x7F7FFFFFu},
{"340282336497324057985000000000000000000", 0x47EFFFFFD0000000u, 0x7F7FFFFEu},
{"3.40282336497324057986e38", 0x47EFFFFFD0000000u, 0x7F7FFFFFu},
{"3.4028233649732405798586897151e38", 0x47EFFFFFD0000000u, 0x7F7FFFFEu},
{"340282336497324057985868971511e9", 0x47EFFFFFD0000000u, 0x7F7FFFFFu},
{"3.40282356779733661637539395458142568448e38", 0x47EFFFFFF0000000u, 0x7F800000u},
{"-340282356779733661637539395458142568449e0", 0xC7EFFFFFF0000000u, 0xFF800000u},
{"340282356779733661637539395458142568447", 0x47EFFFFFF0000000u, 0x7F7FFFFFu},
{"3.4028235677973366163753939545814256844801e38", 0x47EFFFFFF0000000u, 0x7F800000u},
{"340282356779733661637539395458142568448000000000000000000001e-21", 0x47EFFFFFF0000000u, 0x7F800000u},
{"340282356779733661637539395458142568448.000000000000000000000000000000", 0x47EFFFFFF0000000u, 0x7F800000u},
{"3.4028235677973366e38", 0x47EFFFFFF0000000u, 0x7F7FFFFFu},
{"-34028235677973367e22", 0xC7EFFFFFF0000000u, 0xFF800000u},
{"3402823567797336616e20", 0x47EFFFFFF0000000u, 0x7F7FFFFFu},
{"340282356779733661700000000000000000000", 0x47EFFFFFF0000000u, 0x7F800000u},
{"340282356779733661630000000000000000000", 0x47EFFFFFF0000000u, 0x7F7FFFFFu},
{"3.4028235677973366164e38", 0x47EFFFFFF0000000u, 0x7F800000u},
{"3.40282356779733661637e38", 0x47EFFFFFF0000000u, 0x7F7FFFFFu},
{"340282356779733661638e18", 0x47EFFFFFF0000000u, 0x7F800000u},
{"-340282356779733661637539395458e9", 0xC7EFFFFFF0000000u, 0xFF7FFFFFu},
{"340282356779733661637539395459000000000", 0x47EFFFFFF0000000u, 0x7F800000u},
{"-1000000059604644775390625e-24", 0xBFF0000010000000u, 0xBF800000u},
{"-1.000000059604644775390626", 0xBFF0000010000000u, 0xBF800001u},
{"-1.000000059604644775390624e0", 0xBFF0000010000000u, 0xBF800000u},
{"100000005960464477539062501e-26", 0x3FF0000010000000u, 0x3F800001u},
{"1.000000059604644775390625000000000000000000001", 0x3FF0000010000000u, 0x3F800001u},
{"1.000000059604644775390625000000000000000000000000000000e0", 0x3FF0000010000000u, 0x3F800000u},
{"-10000000596046447e-16", 0xBFF0000010000000u, 0xBF800000u},
{"1.0000000596046448", 0x3FF0000010000000u, 0x3F800001u},
{"1.000000059604644775", 0x3FF0000010000000u, 0x3F800000u},
{"-1.000000059604644776e0", 0xBFF0000010000000u, 0xBF800001u},
{"-1.0000000596046447753e0", 0xBFF0000010000000u, 0xBF800000u},
{"10000000596046447754e-19", 0x3FF0000010000000u, 0x3F800001u},
{"100000005960464477539e-20", 0x3FF0000010000000u, 0x3F800000u},
{"-1.0000000596046447754", 0xBFF0000010000000u, 0xBF800001u},
{"0.9999999701976776123046875", 0x3FEFFFFFF0000000u, 0x3F800000u},
{"9.999999701976776123046876e-1", 0x3FEFFFFFF0000000u, 0x3F800000u},
{"9999999701976776123046874e-25", 0x3FEFFFFFF0000000u, 0x3F7FFFFFu},
{"0.999999970197677612304687501", 0x3FEFFFFFF0000000u, 0x3F800000u},
{"-9.999999701976776123046875000000000000000000001e-1", 0xBFEFFFFFF0000000u, 0xBF800000u},
{"-9999999701976776123046875000000000000000000000000000000e-55", 0xBFEFFFFFF0000000u, 0xBF800000u},
{"0.99999997019767761", 0x3FEFFFFFF0000000u, 0x3F7FFFFFu},
{"-9.9999997019767762e-1", 0xBFEFFFFFF0000000u, 0xBF800000u},
{"-9.999999701976776123e-1", 0xBFEFFFFFF0000000u, 0xBF7FFFFFu},
{"9999999701976776124e-19", 0x3FEFFFFFF0000000u, 0x3F800000u},
{"9999999701976776123e-19", 0x3FEFFFFFF0000000u, 0x3F7FFFFFu},
{"0.99999997019767761231", 0x3FEFFFFFF0000000u, 0x3F800000u},
{"-0.999999970197677612304", 0xBFEFFFFFF0000000u, 0xBF7FFFFFu},
{"9.99999970197677612305e-1", 0x3FEFFFFFF0000000u, 0x3F800000u},
{"-1.6777217e7", 0xC170000010000000u, 0xCB800000u},
{"16777218e0", 0x4170000020000000u, 0x4B800001u},
{"16777216", 0x4170000000000000u, 0x4B800000u},
{"-1.677721701e7", 0xC17000001028F5C3u, 0xCB800001u},
{"-16777217000000000000000000001e-21", 0xC170000010000000u, 0xCB800001u},
{"16777217.000000000000000000000000000000", 0x4170000010000000u, 0x4B800000u},
{"167772155e-1", 0x416FFFFFF0000000u, 0x4B800000u},
{"16777215.6", 0x416FFFFFF3333333u, 0x4B800000u},
{"1.67772154e7", 0x416FFFFFECCCCCCDu, 0x4B7FFFFFu},
{"16777215501e-3", 0x416FFFFFF0083127u, 0x4B800000u},
{"-16777215.5000000000000000000001", 0xC16FFFFFF0000000u, 0xCB800000u},
{"-1.67772155000000000000000000000000000000e7", 0xC16FFFFFF0000000u, 0xCB800000u},
{"0.1000000052154064178466796875", 0x3FB99999B0000000u, 0x3DCCCCCEu},
{"1.000000052154064178466796876e-1", 0x3FB99999B0000000u, 0x3DCCCCCEu},
{"-1000000052154064178466796874e-28", 0xBFB99999B0000000u, 0xBDCCCCCDu},
{"0.100000005215406417846679687501", 0x3FB99999B0000000u, 0x3DCCCCCEu},
{"1.000000052154064178466796875000000000000000000001e-1", 0x3FB99999B0000000u, 0x3DCCCCCEu},
{"-1000000052154064178466796875000000000000000000000000000000e-58", 0xBFB99999B0000000u, 0xBDCCCCCEu},
{"-0.10000000521540641", 0xBFB99999AFFFFFFFu, 0xBDCCCCCDu},
{"-1.0000000521540642e-1", 0xBFB99999B0000000u, 0xBDCCCCCEu},
{"1.000000052154064178e-1", 0x3FB99999B0000000u, 0x3DCCCCCDu},
{"-1000000052154064179e-19", 0xBFB99999B0000000u, 0xBDCCCCCEu},
{"10000000521540641784e-20", 0x3FB99999B0000000u, 0x3DCCCCCDu},
{"0.10000000521540641785", 0x3FB99999B0000000u, 0x3DCCCCCEu},
{"0.100000005215406417846", 0x3FB99999B0000000u, 0x3DCCCCCDu},
{"1.00000005215406417847e-1", 0x3FB99999B0000000u, 0x3DCCCCCEu},
{"5.429001220703125e3", 0x40B5350050000000u, 0x45A9A802u},
{"-5429001220703126e-12", 0xC0B5350050000001u, 0xC5A9A803u},
{"-5429.001220703124", 0xC0B535004FFFFFFFu, 0xC5A9A802u},
{"5.42900122070312501e3", 0x40B5350050000000u, 0x45A9A803u},
{"5429001220703125000000000000000000001e-33", 0x40B5350050000000u, 0x45A9A803u},
{"-5429.001220703125000000000000000000000000000000", 0xC0B5350050000000u, 0xC5A9A802u},
{"503719056e0", 0x41BE062490000000u, 0x4DF03124u},
{"503719057", 0x41BE062491000000u, 0x4DF03125u},
{"5.03719055e8", 0x41BE06248F000000u, 0x4DF03124u},
{"50371905601e-2", 0x41BE062490028F5Cu, 0x4DF03125u},
{"503719056.000000000000000000001", 0x41BE062490000000u, 0x4DF03125u},
{"5.03719056000000000000000000000000000000e8", 0x41BE062490000000u, 0x4DF03124u},
{"-92331620", 0xC196037990000000u, 0xCCB01BCCu},
{"9.233163e7", 0x41960379B8000000u, 0x4CB01BCEu},
{"9233161e1", 0x4196037968000000u, 0x4CB01BCBu},
{"92331620.1", 0x4196037990666666u, 0x4CB01BCDu},
{"9.233162000000000000000000001e7", 0x4196037990000000u, 0x4CB01BCDu},
{"9233162000000000000000000000000000000e-29", 0x4196037990000000u, 0x4CB01BCCu},
{"3.002458625e6", 0x4146E82D50000000u, 0x4A37416Au},
{"3002458626e-3", 0x4146E82D5020C49Cu, 0x4A37416Bu},
{"3002458.624", 0x4146E82D4FDF3B64u, 0x4A37416Au},
{"-3.00245862501e6", 0xC146E82D500053E3u, 0xCA37416Bu},
{"3002458625000000000000000000001e-24", 0x4146E82D50000000u, 0x4A37416Bu},
{"3002458.625000000000000000000000000000000", 0x4146E82D50000000u, 0x4A37416Au},
{"-1095485584696182596504479582065262592e1", 0xC7A07BA830000000u, 0xFD03DD42u},
{"10954855846961825965044795820652625930", 0x47A07BA830000000u, 0x7D03DD42u},
{"1.095485584696182596504479582065262591e37", 0x47A07BA830000000u, 0x7D03DD41u},
{"109548558469618259650447958206526259201e-1", 0x47A07BA830000000u, 0x7D03DD42u},
{"-10954855846961825965044795820652625920.00000000000000000001", 0xC7A07BA830000000u, 0xFD03DD42u},
{"1.095485584696182596504479582065262592000000000000000000000000000000e37", 0x47A07BA830000000u, 0x7D03DD42u},
{"10954855846961825e21", 0x47A07BA830000000u, 0x7D03DD41u},
{"10954855846961826000000000000000000000", 0x47A07BA830000000u, 0x7D03DD42u},
{"10954855846961825960000000000000000000", 0x47A07BA830000000u, 0x7D03DD41u},
{"1.095485584696182597e37", 0x47A07BA830000000u, 0x7D03DD42u},
{"1.0954855846961825965e37", 0x47A07BA830000000u, 0x7D03DD41u},
{"-10954855846961825966e18", 0xC7A07BA830000000u, 0xFD03DD42u},
{"10954855846961825965e18", 0x47A07BA830000000u, 0x7D03DD41u},
{"10954855846961825965100000000000000000", 0x47A07BA830000000u, 0x7D03DD42u},
{"10954855846961825965044795820600000000", 0x47A07BA830000000u, 0x7D03DD41u},
{"-1.09548558469618259650447958207e37", 0xC7A07BA830000000u, 0xFD03DD42u},
{"1.6449216019182103706535606608388384863861375606575165875256061553955078126e-21", 0x3B9F125A50000000u, 0x1CF892D3u},
{"-16449216019182103706535606608388384863861375606575165875256061553955078124e-94", 0xBB9F125A50000000u, 0x9CF892D2u},
{"-0.0000000000000000000016449216019182103", 0xBB9F125A50000000u, 0x9CF892D2u},
{"-1.6449216019182104e-21", 0xBB9F125A50000000u, 0x9CF892D3u},
{"-1.64492160191821037e-21", 0xBB9F125A50000000u, 0x9CF892D2u},
{"-1644921601918210371e-39", 0xBB9F125A50000000u, 0x9CF892D3u},
{"-16449216019182103706e-40", 0xBB9F125A50000000u, 0x9CF892D2u},
{"0.0000000000000000000016449216019182103707", 0x3B9F125A50000000u, 0x1CF892D3u},
{"0.00000000000000000000164492160191821037065", 0x3B9F125A50000000u, 0x1CF892D2u},
{"1.64492160191821037066e-21", 0x3B9F125A50000000u, 0x1CF892D3u},
{"1.64492160191821037065356066083e-21", 0x3B9F125A50000000u, 0x1CF892D2u},
{"164492160191821037065356066084e-50", 0x3B9F125A50000000u, 0x1CF892D3u},
{"6.565061509609222412109375e-1", 0x3FE5021930000000u, 0x3F2810CAu},
{"6565061509609222412109376e-25", 0x3FE5021930000000u, 0x3F2810CAu},
{"0.6565061509609222412109374", 0x3FE5021930000000u, 0x3F2810C9u},
{"6.56506150960922241210937501e-1", 0x3FE5021930000000u, 0x3F2810CAu},
{"6565061509609222412109375000000000000000000001e-46", 0x3FE5021930000000u, 0x3F2810CAu},
{"-0.6565061509609222412109375000000000000000000000000000000", 0xBFE5021930000000u, 0xBF2810CAu},
{"6.5650615096092224e-1", 0x3FE5021930000000u, 0x3F2810C9u},
{"-65650615096092225e-17", 0xBFE5021930000000u, 0xBF2810CAu},
{"6565061509609222412e-19", 0x3FE5021930000000u, 0x3F2810C9u},
{"0.6565061509609222413", 0x3FE5021930000000u, 0x3F2810CAu},
{"0.65650615096092224121", 0x3FE5021930000000u, 0x3F2810C9u},
{"6.5650615096092224122e-1", 0x3FE5021930000000u, 0x3F2810CAu},
{"-6.5650615096092224121e-1", 0xBFE5021930000000u, 0xBF2810C9u},
{"656506150960922241211e-21", 0x3FE5021930000000u, 0x3F2810CAu},
{"18014627239033005156980393746124491372029297053813934326171875e-77", 0x3CA9F63970000000u, 0x254FB1CCu},
{"0.00000000000000018014627239033005156980393746124491372029297053813934326171876", 0x3CA9F63970000000u, 0x254FB1CCu},
{"-1.8014627239033005156980393746124491372029297053813934326171874e-16", 0xBCA9F63970000000u, 0xA54FB1CBu},
{"1801462723903300515698039374612449137202929705381393432617187501e-79", 0x3CA9F63970000000u, 0x254FB1CCu},
{"18014627239033005e-32", 0x3CA9F63970000000u, 0x254FB1CBu},
{"0.00000000000000018014627239033006", 0x3CA9F63970000000u, 0x254FB1CCu},
{"0.0000000000000001801462723903300515", 0x3CA9F63970000000u, 0x254FB1CBu},
{"-1.801462723903300516e-16", 0xBCA9F63970000000u, 0xA54FB1CCu},
{"-1.8014627239033005156e-16", 0xBCA9F63970000000u, 0xA54FB1CBu},
{"18014627239033005157e-35", 0x3CA9F63970000000u, 0x254FB1CCu},
{"180146272390330051569e-36", 0x3CA9F63970000000u, 0x254FB1CBu},
{"-0.00000000000000018014627239033005157", 0xBCA9F63970000000u, 0xA54FB1CCu},
{"0.000000000000000180146272390330051569803937461", 0x3CA9F63970000000u, 0x254FB1CBu},
{"1.80146272390330051569803937462e-16", 0x3CA9F63970000000u, 0x254FB1CCu},
{"0.05534819327294826507568359375", 0x3FAC569930000000u, 0x3D62B4CAu},
{"-5.534819327294826507568359376e-2", 0xBFAC569930000000u, 0xBD62B4CAu},
{"5534819327294826507568359374e-29", 0x3FAC569930000000u, 0x3D62B4C9u},
{"-0.0553481932729482650756835937501", 0xBFAC569930000000u, 0xBD62B4CAu},
{"5.534819327294826507568359375000000000000000000001e-2", 0x3FAC569930000000u, 0x3D62B4CAu},
{"5534819327294826507568359375000000000000000000000000000000e-59", 0x3FAC569930000000u, 0x3D62B4CAu},
{"0.055348193272948265", 0x3FAC569930000000u, 0x3D62B4C9u},
{"5.5348193272948266e-2", 0x3FAC569930000000u, 0x3D62B4CAu},
{"5.534819327294826507e-2", 0x3FAC569930000000u, 0x3D62B4C9u},
{"5534819327294826508e-20", 0x3FAC569930000000u, 0x3D62B4CAu},
{"55348193272948265075e-21", 0x3FAC569930000000u, 0x3D62B4C9u},
{"-0.055348193272948265076", 0xBFAC569930000000u, 0xBD62B4CAu},
{"0.0553481932729482650756", 0x3FAC569930000000u, 0x3D62B4C9u},
{"-5.53481932729482650757e-2", 0xBFAC569930000000u, 0xBD62B4CAu},
{"5.179692133247783258005389047985340416e36", 0x478F2C9450000000u, 0x7C7964A2u},
{"5179692133247783258005389047985340417e0", 0x478F2C9450000000u, 0x7C7964A3u},
{"5179692133247783258005389047985340415", 0x478F2C9450000000u, 0x7C7964A2u},
{"-5.17969213324778325800538904798534041601e36", 0xC78F2C9450000000u, 0xFC7964A3u},
{"-5179692133247783258005389047985340416000000000000000000001e-21", 0xC78F2C9450000000u, 0xFC7964A3u},
{"-5179692133247783258005389047985340416.000000000000000000000000000000", 0xC78F2C9450000000u, 0xFC7964A2u},
{"5.1796921332477832e36", 0x478F2C9450000000u, 0x7C7964A2u},
{"51796921332477833e20", 0x478F2C9450000000u, 0x7C7964A3u},
{"-5179692133247783258e18", 0xC78F2C9450000000u, 0xFC7964A2u},
{"5179692133247783259000000000000000000", 0x478F2C9450000000u, 0x7C7964A3u},
{"5179692133247783258000000000000000000", 0x478F2C9450000000u, 0x7C7964A2u},
{"5.1796921332477832581e36", 0x478F2C9450000000u, 0x7C7964A3u},
{"-5.179692133247783258e36", 0xC78F2C9450000000u, 0xFC7964A2u},
{"-517969213324778325801e16", 0xC78F2C9450000000u, 0xFC7964A3u},
{"517969213324778325800538904798e7", 0x478F2C9450000000u, 0x7C7964A2u},
{"5179692133247783258005389047990000000", 0x478F2C9450000000u, 0x7C7964A3u},
{
"0.22250738585072011360574097967091319759348195463516456480234261097248222220210769455165295239081350"
"8791414915891303962110687008643869459464552765720740782062174337998814106326732925355228688137214901"
"2981122451451889849057222307285255133155755015914397476397983411801999323962548289017107081850690630"
"6666559949382757725720157630626906633326475653000092458883164330377797918696120494973903778297049050"
"5108060994073026293712895895000358379996720725430436028407889577179615094551674824347103070260914462"
"1572289880258182545180325707018860872113128079512233426288368622321503775666622503982534335974568884"
"4239002654981983854879482922068947216898310996983658468140228542433306603398508864458040010349339704"
"2756718644338377048603786162277173854562306587467901408672332763671875e-307", 0x0010000000000000u, 0x00000000u
},
{
"2.22507385850720113605740979670913197593481954635164564802342610972482222202107694551652952390813508"
"7914149158913039621106870086438694594645527657207407820621743379988141063267329253552286881372149012"
"9811224514518898490572223072852551331557550159143974763979834118019993239625482890171070818506906306"
"6665599493827577257201576306269066333264756530000924588831643303777979186961204949739037782970490505"
"1080609940730262937128958950003583799967207254304360284078895771796150945516748243471030702609144621"
"5722898802581825451803257070188608721131280795122334262883686223215037756666225039825343359745688844"
"2390026549819838548794829220689472168983109969836584681402285424333066033985088644580400103493397042"
"756718644338377048603786162277173854562306587467901408672332763671875000000000000000000001e-308", 0x0010000000000000u, 0x00000000u
},
{
"0.11754942807573642917278829910357665133228589927589904276829631184250030649651730385585324256680905"
"8189392089843750000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
"0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
"0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
"0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
"0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
"0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
"00000000000000000000000000000000000000000000000000000000000000000e-37", 0x380FFFFFE0000000u, 0x00800000u
},
{
"1175494280757364291727882991035766513322858992758990427682963118425003064965173038558532425668090581"
"8939208984375000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
"0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
"0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
"0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
"0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
"0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
"00000000000001e-751", 0x380FFFFFE0000000u, 0x00800000u
},
{"0", 0x0000000000000000u, 0x00000000u},
{"-0", 0x8000000000000000u, 0x80000000u},
{"0.0", 0x0000000000000000u, 0x00000000u},
{"-0.0", 0x8000000000000000u, 0x80000000u},
{"0e999999999999999999999", 0x0000000000000000u, 0x00000000u},
{"-0.000e-99999", 0x8000000000000000u, 0x80000000u},
{"1e-400", 0x0000000000000000u, 0x00000000u},
{"-1e-400", 0x8000000000000000u, 0x80000000u},
{"1e400", 0x7FF0000000000000u, 0x7F800000u},
{"-1e400", 0xFFF0000000000000u, 0xFF800000u},
{"1e-50", 0x358DEE7A4AD4B81Fu, 0x00000000u},
{"-1e-50", 0xB58DEE7A4AD4B81Fu, 0x80000000u},
{"1e39", 0x48078287F49C4A1Du, 0x7F800000u},
{"-1e39", 0xC8078287F49C4A1Du, 0xFF800000u},
{"1e99999999999999999999999999", 0x7FF0000000000000u, 0x7F800000u},
{"1e-99999999999999999999999999", 0x0000000000000000u, 0x00000000u},
{"1e0000000000000000000000000000000000000000308", 0x7FE1CCF385EBC8A0u, 0x7F800000u},
{"123456789012345678901234567890e-30", 0x3FBF9ADD3746F65Fu, 0x3DFCD6EAu},
{"18446744073709551615", 0x43F0000000000000u, 0x5F800000u},
{"18446744073709551616", 0x43F0000000000000u, 0x5F800000u},
{"-9223372036854775808", 0xC3E0000000000000u, 0xDF000000u},
{"-9223372036854775809", 0xC3E0000000000000u, 0xDF000000u},
}
};
return table;
}
} // namespace float_hard_cases
+553 -106
View File
@@ -13,15 +13,19 @@
using nlohmann::json;
#include <array> // array
#include <cfloat> // FLT_EVAL_METHOD
#include <cstdint> // uint32_t, uint64_t
#include <cstdio> // snprintf
#include <cstdlib> // strtod
#include <cstring> // memcpy
#include <map> // map
#include <random> // mt19937
#include <sstream> // stringstream
#include <string> // string
#include <utility> // pair
#include <vector> // vector
#include "float_hard_cases.hpp"
namespace
{
// shortcut to scan a string literal
@@ -257,7 +261,7 @@ TEST_CASE("lexer number fast path")
"123456789012345678901234567890", // huge -> float
"0.30000000000000004", "2.2250738585072014e-308", "1e308",
// high-precision / wide-exponent values that exercise the
// std::from_chars (Eisel-Lemire) path beyond the Clinger subset
// Eisel-Lemire path beyond the Clinger subset
"1.7976931348623157e308", "1.2345678901234567e-250",
"9007199254740993", "5e-324", "1e-320"
};
@@ -279,20 +283,18 @@ TEST_CASE("lexer number fast path")
}
}
SECTION("significant-digit gate for the Clinger fast path")
SECTION("significant digits around Clinger's fast path")
{
// Clinger's fast path needs a significand below 2^53, so it cannot
// succeed once the mantissa has 17 or more significant digits (the
// significand would be at least 10^16). The lexer skips the attempt
// there. That is only allowed to save work: every value must still come
// out bit-exactly, and both scanners must agree. In particular the gate
// must not fire for tokens whose leading zeros merely look like extra
// digits - "0.1234567890123456" has 16 significant digits, not 17.
// Clinger's fast path needs a significand of at most 2^53, which
// tokens with 17 or more significant digits exceed. The conversion
// splits the token at the positions the scanners recorded, so leading
// zeros must not count as digits - "0.1234567890123456" has 16
// significant digits, not 17 - and both scanners must agree.
const std::vector<std::string> numbers =
{
"1234567890123456", // 16 significant digits
"12345678901234567", // 17 -> attempt skipped
"123456789012345678", // 18 -> attempt skipped
"12345678901234567", // 17
"123456789012345678", // 18
"0.1234567890123456", // 16: the leading "0" is not significant
"0.12345678901234567", // 17
"0.00000000000000001", // 1, in a long token
@@ -663,46 +665,323 @@ TEST_CASE("lexer string fast path")
}
}
TEST_CASE("parse_float_fast declines what it cannot convert exactly")
TEST_CASE("lexer escape fast path")
{
// The lexer only hands well-formed numbers to parse_float_fast, so the
// malformed ones below can only be passed to it directly. Declining is
// always safe: the caller then falls back to a slower, exact conversion.
const auto fast = [](const std::string & s, double & out)
// json::accept() never throws, so this section stays covered without
// exceptions; it pins which of the cases below are valid/invalid and
// checks the contiguous and streaming paths agree on that classification.
SECTION("accept() parity")
{
return nlohmann::detail::parse_float_fast(s.data(), s.data() + s.size(), out);
const std::vector<std::pair<std::string, bool>> cases =
{
{"\\u0041", true}, {"\\u00e4", true}, {"\\u00E4", true},
{"\\uD83D\\uDE00", true},
{"\\u12", false}, {"\\u12G4", false}, {"\\uXYZW", false},
{"\\uD800", false}, {"\\uD800A", false}, {"\\uD800\\u0041", false},
{"\\uDC00", false}, {"\\u", false}
};
for (const auto& c : cases)
{
for (const std::size_t offset :
{
std::size_t{0}, std::size_t{9}
})
{
const std::string doc = "[\"" + std::string(offset, 'a') + c.first + "\"]";
CAPTURE(doc);
CHECK(json::accept(doc) == c.second);
std::stringstream ss(doc);
CHECK(json::accept(ss) == c.second);
}
}
}
#if !defined(JSON_NOEXCEPTION)
// the full outcome of parsing @a doc: the parsed value, or the exact
// error message, so a mismatch in either is caught
const auto outcome = [](const std::string & doc, bool streaming) -> std::string
{
try
{
if (streaming)
{
std::stringstream ss(doc);
const json j = json::parse(ss);
return j.dump();
}
const json j = json::parse(doc);
return j.dump();
}
catch (const json::exception& e)
{
return {e.what()};
}
};
double out = 0;
#if defined(FLT_EVAL_METHOD) && FLT_EVAL_METHOD != 0
// without true double precision, the fast path declines everything
CHECK_FALSE(fast("1.5", out));
#else
CHECK(fast("1.5", out));
CHECK(out == 1.5);
CHECK(fast("+2.5e1", out));
CHECK(out == 25.0);
CHECK(fast("-25E-1", out));
CHECK(out == -2.5);
CHECK(fast("1e", out));
CHECK(out == 1.0);
SECTION("contiguous vs streaming parity")
{
const std::vector<std::string> escapes =
{
"\\u0041", // "A"
"\\u00e4", // "ä" (lowercase hex)
"\\u00E4", // "ä" (uppercase hex)
"\\uD83D\\uDE00", // valid surrogate pair (an emoji)
"\\u12", // truncated: only 2 hex digits before the closing quote
"\\u12G4", // invalid hex digit at the 3rd position
"\\uXYZW", // all 4 bytes invalid
"\\uD800", // lone high surrogate, string ends right after
"\\uD800A", // high surrogate not followed by another \u escape
"\\uD800\\u0041", // high surrogate followed by \u, but not a low surrogate
"\\uDC00", // lone low surrogate
"\\u", // '\u' with nothing after (closing quote right away)
};
// once at the start of the string and once past the first 8-byte SWAR
// word of the outer string_bulk_run, so the escape is reached both
// right after the opening quote and mid-run
for (const auto& escape : escapes)
{
for (const std::size_t offset :
{
std::size_t{0}, std::size_t{9}
})
{
const std::string doc = "[\"" + std::string(offset, 'a') + escape + "\"]";
CAPTURE(doc);
CHECK(outcome(doc, false) == outcome(doc, true));
}
// the escape is the last thing before end of input: no closing
// quote at all
const std::string truncated_doc = "[\"" + escape;
CAPTURE(truncated_doc);
CHECK(outcome(truncated_doc, false) == outcome(truncated_doc, true));
}
}
SECTION("truncated \\u escape at every distance from the end of input")
{
// ia.bulk_remaining() must correctly report fewer than 4 bytes for
// every possible count of trailing hex-looking bytes (0, 1, 2, or 3)
// before end of input, so the fast path declines and the byte path
// alone reports the "must be followed by 4 hex digits" error, at the
// same position, in every case
for (const std::string& tail :
{
std::string{}, std::string("1"), std::string("12"), std::string("123")
})
{
const std::string doc = "[\"\\u" + tail;
CAPTURE(doc);
CHECK(outcome(doc, false) == outcome(doc, true));
CHECK(outcome(doc, false).find("must be followed by 4 hex digits") != std::string::npos);
}
}
SECTION("invalid hex digit at every position of the 4")
{
// the fast path must decline for *any* invalid byte among the 4, not
// just the first, and the byte path must then stop at exactly that
// position - same as it always has
for (std::size_t bad_pos = 0; bad_pos < 4; ++bad_pos)
{
std::string digits = "1234";
digits[bad_pos] = 'g'; // not a hex digit
const std::string doc = "[\"\\u" + digits + "\"]";
CAPTURE(doc);
CHECK(outcome(doc, false) == outcome(doc, true));
CHECK(outcome(doc, false).find("must be followed by 4 hex digits") != std::string::npos);
}
}
SECTION("random escapes")
{
// A seeded PRNG builds the 4 bytes following `\u` from a mix of hex
// digits and non-hex bytes, at varying distances from the start of
// the string, to compare the two scanners on many more shapes than
// are practical to enumerate by hand.
std::mt19937 gen(7654321); // NOLINT(cert-msc32-c,cert-msc51-cpp)
const std::string hex_alphabet = "0123456789AaBbCcDdEeFf";
std::uniform_int_distribution<std::size_t> pick_hex(0, hex_alphabet.size() - 1);
std::uniform_int_distribution<int> pick_byte(1, 255); // never NUL
std::uniform_int_distribution<int> pick_is_hex(0, 4); // 4-in-5 chance of a hex digit
std::uniform_int_distribution<std::size_t> pick_offset(0, 12);
std::vector<std::string> mismatches;
for (int iter = 0; iter < 3000; ++iter)
{
std::string digits;
for (int i = 0; i < 4; ++i)
{
if (pick_is_hex(gen) != 0)
{
digits += hex_alphabet[pick_hex(gen)];
}
else
{
char c = static_cast<char>(pick_byte(gen));
if (c == '"' || c == '\\')
{
// keep the string well-formed apart from the escape
// itself, so any mismatch is attributable to the \u
// handling and not to an unrelated quote/escape
c = 'z';
}
digits += c;
}
}
const std::string doc = "[\"" + std::string(pick_offset(gen), 'a') + "\\u" + digits + "\"]";
if (outcome(doc, false) != outcome(doc, true))
{
mismatches.push_back(doc);
}
}
CAPTURE(mismatches);
CHECK(mismatches.empty());
}
#endif
}
// not a number
CHECK_FALSE(fast("", out));
CHECK_FALSE(fast("-", out));
CHECK_FALSE(fast(".", out));
CHECK_FALSE(fast("1.2.3", out));
CHECK_FALSE(fast("1x", out));
CHECK_FALSE(fast("1e+", out));
CHECK_FALSE(fast("1e1x", out));
namespace
{
// the index of the decimal point (or npos) and of the end of the mantissa of a
// number token, which the lexer records while scanning it
std::pair<std::size_t, std::size_t> float_token_layout(const std::string& s)
{
std::size_t dot = std::string::npos;
std::size_t mantissa_end = s.size();
for (std::size_t i = 0; i < s.size(); ++i)
{
if (s[i] == '.')
{
dot = i;
}
else if (s[i] == 'e' || s[i] == 'E')
{
mantissa_end = i;
break;
}
}
return {dot, mantissa_end};
}
// numbers that are not represented exactly on the fast path
CHECK_FALSE(fast("12345678901234567890", out));
CHECK_FALSE(fast("1e10000", out));
CHECK_FALSE(fast("9007199254740993", out));
CHECK_FALSE(fast("1e23", out));
CHECK_FALSE(fast("1e-23", out));
template<typename FloatType>
FloatType parse_native(const std::string& s)
{
const auto layout = float_token_layout(s);
return nlohmann::detail::parse_float_native<FloatType>(s.data(), s.data() + s.size(), layout.first, layout.second);
}
std::uint64_t bits_of(double d)
{
std::uint64_t b = 0;
std::memcpy(&b, &d, sizeof(b));
return b;
}
std::uint32_t bits_of(float f)
{
std::uint32_t b = 0;
std::memcpy(&b, &f, sizeof(b));
return b;
}
std::uint64_t native_bits64(const std::string& s)
{
return bits_of(parse_native<double>(s));
}
std::uint32_t native_bits32(const std::string& s)
{
return bits_of(parse_native<float>(s));
}
} // namespace
TEST_CASE("parse_float_native rounds correctly")
{
SECTION("double")
{
CHECK(native_bits64("1.5") == 0x3FF8000000000000u);
CHECK(native_bits64("0.1") == 0x3FB999999999999Au);
CHECK(native_bits64("-0.0") == 0x8000000000000000u);
CHECK(native_bits64("0e999999999999999999999") == 0u);
// 2^53 + 1 is exactly between two doubles: ties to even, unless more digits follow
CHECK(native_bits64("9007199254740993") == 0x4340000000000000u);
CHECK(native_bits64("9007199254740993.0000000000000000001") == 0x4340000000000001u);
CHECK(native_bits64("9007199254740992.9999999999999999999") == 0x4340000000000000u);
// 1 + 2^-53 exactly (a tie), and one unit in the 55th digit around it
CHECK(native_bits64("1.00000000000000011102230246251565404236316680908203125") == 0x3FF0000000000000u);
CHECK(native_bits64("1.00000000000000011102230246251565404236316680908203126") == 0x3FF0000000000001u);
CHECK(native_bits64("1.00000000000000011102230246251565404236316680908203124") == 0x3FF0000000000000u);
// subnormal and overflow boundaries
CHECK(native_bits64("2.4703282292062327e-324") == 0u);
CHECK(native_bits64("2.4703282292062328e-324") == 1u);
CHECK(native_bits64("2.2250738585072011e-308") == 0x000FFFFFFFFFFFFFu);
CHECK(native_bits64("2.2250738585072012e-308") == 0x0010000000000000u);
CHECK(native_bits64("1.7976931348623157e308") == 0x7FEFFFFFFFFFFFFFu);
CHECK(native_bits64("1.7976931348623159e308") == 0x7FF0000000000000u);
CHECK(native_bits64("-1e400") == 0xFFF0000000000000u);
CHECK(native_bits64("-1e-400") == 0x8000000000000000u);
// exponents and zeros far beyond the range cancel out
CHECK(native_bits64("0." + std::string(1000, '0') + "1e1001") == 0x3FF0000000000000u);
CHECK(native_bits64("1" + std::string(1000, '0') + "e-1000") == 0x3FF0000000000000u);
CHECK(native_bits64("1e-99999999999999999999999") == 0u);
CHECK(native_bits64("1E+99999999999999999999999") == 0x7FF0000000000000u);
// more digits than any midpoint has (769): only whether a nonzero digit follows matters
const std::string tie = "1.00000000000000011102230246251565404236316680908203125";
CHECK(native_bits64(tie + std::string(800, '0')) == 0x3FF0000000000000u);
CHECK(native_bits64(tie + std::string(800, '0') + "1") == 0x3FF0000000000001u);
}
SECTION("float")
{
CHECK(native_bits32("1.5") == 0x3FC00000u);
CHECK(native_bits32("0.1") == 0x3DCCCCCDu);
CHECK(native_bits32("-0.0") == 0x80000000u);
// 2^24 + 1 is exactly between two floats
CHECK(native_bits32("16777217") == 0x4B800000u);
CHECK(native_bits32("16777217.000000000000000000001") == 0x4B800001u);
CHECK(native_bits32("16777218.999999999999999999999") == 0x4B800001u);
CHECK(native_bits32("16777219") == 0x4B800002u);
// subnormal and overflow boundaries
CHECK(native_bits32("3.4028235677973366e38") == 0x7F7FFFFFu);
CHECK(native_bits32("3.4028235677973367e38") == 0x7F800000u);
CHECK(native_bits32("7.006492321624085e-46") == 0u);
CHECK(native_bits32("7.006492321624086e-46") == 1u);
CHECK(native_bits32("1.1754942e-38") == 0x007FFFFFu);
CHECK(native_bits32("-1.17549435e-38") == 0x80800000u);
CHECK(native_bits32("1e39") == 0x7F800000u);
CHECK(native_bits32("-1e-50") == 0x80000000u);
// not rounded through double: its double would round to another float
CHECK(native_bits32("1.00000005960464477539062500000000001") == 0x3F800001u);
CHECK(native_bits32("9007199254740993") == 0x5A000000u);
}
SECTION("the conversion shared with other parsers")
{
// convert_float() gives the lexer's results, for every type
const std::vector<std::string> tokens =
{
"0", "-0.0", "1.5", "0.1", "1e-400", "-2.5E+3", "123456789012345678901234567890",
"9007199254740993.0000000000000000001", "4.9406564584124654e-324"
};
using float_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int64_t, std::uint64_t, float>;
using long_double_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int64_t, std::uint64_t, long double>;
for (const auto& t : tokens)
{
CAPTURE(t);
const auto layout = float_token_layout(t);
const char* const first = t.data();
const char* const last = first + t.size();
const auto d = nlohmann::detail::convert_float<double>(first, last, layout.first, layout.second);
const auto f = nlohmann::detail::convert_float<float>(first, last, layout.first, layout.second);
const auto ld = nlohmann::detail::convert_float<long double>(first, last, layout.first, layout.second);
CHECK(bits_of(d) == bits_of(json::parse(t).get<double>()));
CHECK(bits_of(f) == bits_of(float_json::parse(t).get<float>()));
CHECK(ld == long_double_json::parse(t).get<long double>());
}
}
}
namespace
@@ -806,40 +1085,6 @@ std::size_t big_bit_length(const big_uint& a)
}
return n;
}
std::uint64_t bits_of(double d)
{
std::uint64_t b = 0;
std::memcpy(&b, &d, sizeof(b));
return b;
}
bool eisel_lemire(const std::string& s, double& out)
{
return nlohmann::detail::parse_float_eisel_lemire(s.data(), s.data() + s.size(), out);
}
// significant digits of a token, without trailing zeros
std::size_t significant_digits(const std::string& s)
{
std::string digits;
for (const char c : s)
{
if (c == 'e' || c == 'E')
{
break;
}
if (c >= '0' && c <= '9' && !(digits.empty() && c == '0'))
{
digits += c;
}
}
while (!digits.empty() && digits.back() == '0')
{
digits.pop_back();
}
return digits.size();
}
} // namespace
TEST_CASE("Eisel-Lemire float conversion")
@@ -1237,26 +1482,33 @@ TEST_CASE("Eisel-Lemire float conversion")
for (const auto& c : known)
{
CAPTURE(c.first);
double out = 0;
if (eisel_lemire(c.first, out))
{
CHECK(bits_of(out) == c.second);
}
else
{
// only tokens with more than 19 significant digits are left to
// strtod: those whose value lies too close to a tie
CHECK(significant_digits(c.first) > 19);
}
CHECK(native_bits64(c.first) == c.second);
}
}
SECTION("binary32")
{
using binary32 = nlohmann::detail::ieee_binary_format<24>;
CHECK(nlohmann::detail::eisel_lemire<binary32>(0, 1) == 0x3F800000u);
CHECK(nlohmann::detail::eisel_lemire<binary32>(-1, 1) == 0x3DCCCCCDu);
CHECK(nlohmann::detail::eisel_lemire<binary32>(-1, 15) == 0x3FC00000u);
CHECK(nlohmann::detail::eisel_lemire<binary32>(0, 16777217) == 0x4B800000u); // tie, to even
CHECK(nlohmann::detail::eisel_lemire<binary32>(0, 16777219) == 0x4B800002u); // tie, to even
CHECK(nlohmann::detail::eisel_lemire<binary32>(-45, 1) == 0x00000001u);
CHECK(nlohmann::detail::eisel_lemire<binary32>(-46, 7) == 0x00000000u);
CHECK(nlohmann::detail::eisel_lemire<binary32>(-46, 8) == 0x00000001u);
CHECK(nlohmann::detail::eisel_lemire<binary32>(-65, 9999999999999999999u) == 0x00000000u);
CHECK(nlohmann::detail::eisel_lemire<binary32>(20, 3402823466385288598u) == 0x7F7FFFFFu);
CHECK(nlohmann::detail::eisel_lemire<binary32>(20, 3402823669209384635u) == 0x7F800000u);
CHECK(nlohmann::detail::eisel_lemire<binary32>(39, 1) == 0x7F800000u);
CHECK(nlohmann::detail::eisel_lemire<binary32>(-5, 0) == 0x00000000u);
}
SECTION("round trip")
{
// every double written by to_chars and read back, also with trailing
// digits that make the token longer than 19 digits
// every double written by to_chars and read back, and its 17-digit
// form with trailing digits that make the token longer than 19 digits
std::uint64_t state = 5295;
std::size_t declined = 0;
for (int i = 0; i < 200000; ++i)
{
state ^= state << 13u;
@@ -1278,30 +1530,51 @@ TEST_CASE("Eisel-Lemire float conversion")
const char* end = nlohmann::detail::to_chars(buffer.data(), buffer.data() + buffer.size(), d);
const std::string token(buffer.data(), static_cast<std::size_t>(end - buffer.data()));
CAPTURE(token);
double out = 0;
REQUIRE(eisel_lemire(token, out));
CHECK(bits_of(out) == b);
CHECK(native_bits64(token) == b);
// insert digits before the exponent: the value moves by far less
// than the distance to the rounding boundary, so it must not change
std::string longer = token;
// insert digits before the exponent of the 17-digit form: that
// form lies strictly inside the rounding interval of the double
// (the shortest one may lie on its boundary), and the digits move
// it by far less than the distance to the boundary, so the value
// must not change
std::array<char, 64> digits17{};
static_cast<void>(std::snprintf(digits17.data(), digits17.size(), "%.17g", d)); // NOLINT(cppcoreguidelines-pro-type-vararg,hicpp-vararg)
std::string longer = digits17.data();
const std::size_t e = longer.find('e');
const std::size_t dot = longer.find('.');
const std::string extra = dot == std::string::npos ? ".000000000000000000001" : "000000000000000000001";
longer.insert(e == std::string::npos ? longer.size() : e, extra);
CAPTURE(longer);
if (eisel_lemire(longer, out))
CHECK(native_bits64(longer) == b);
}
}
SECTION("round trip, binary32")
{
std::uint32_t state = 5295;
for (int i = 0; i < 100000; ++i)
{
state ^= state << 13u;
state ^= state >> 17u;
state ^= state << 5u;
std::uint32_t b = state;
if ((b & 0x7F800000u) == 0x7F800000u)
{
CHECK(bits_of(out) == b);
continue; // infinity or NaN
}
else
if (i % 4 == 0)
{
// w and w + 1 round differently: only when the value is very
// close to a rounding boundary
++declined;
b &= 0x807FFFFFu; // subnormals
}
float f = 0;
std::memcpy(&f, &b, sizeof(f));
std::array<char, 64> buffer{};
const char* end = nlohmann::detail::to_chars(buffer.data(), buffer.data() + buffer.size(), f);
const std::string token(buffer.data(), static_cast<std::size_t>(end - buffer.data()));
CAPTURE(token);
CHECK(native_bits32(token) == b);
}
CHECK(declined < 1000); // 107 of the 200,000
}
SECTION("used by the lexer")
@@ -1315,3 +1588,177 @@ TEST_CASE("Eisel-Lemire float conversion")
"[json.exception.out_of_range.406] number overflow parsing '1.7976931348623159e308'", json::out_of_range&);
}
}
namespace
{
using float_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int64_t, std::uint64_t, float>;
// the bits of the float that parse() gives for a token, via both scanners;
// the value must be the same for both
template<typename Json, typename Bits>
void check_parse(const std::string& token, Bits expected, Bits infinity)
{
std::stringstream stream(token);
if ((expected & ~(Bits{1} << (8 * sizeof(Bits) - 1))) == infinity)
{
Json _;
CHECK_THROWS_WITH_AS(_ = Json::parse(token), ("[json.exception.out_of_range.406] number overflow parsing '" + token + "'").c_str(), typename Json::out_of_range&);
CHECK_THROWS_WITH_AS(_ = Json::parse(stream), ("[json.exception.out_of_range.406] number overflow parsing '" + token + "'").c_str(), typename Json::out_of_range&);
return;
}
const Json contiguous = Json::parse(token);
const Json streamed = Json::parse(stream);
if (contiguous.is_number_float()) // not an integer that fits
{
CHECK(bits_of(contiguous.template get<typename Json::number_float_t>()) == expected);
CHECK(bits_of(streamed.template get<typename Json::number_float_t>()) == expected);
}
else
{
CHECK(streamed.is_number_integer());
}
}
} // namespace
TEST_CASE("float conversion of hard cases")
{
// see float_hard_cases.hpp
for (const auto& c : float_hard_cases::cases())
{
const std::string token = c.token;
CAPTURE(token);
CHECK(native_bits64(token) == c.bits64);
CHECK(native_bits32(token) == c.bits32);
check_parse<json>(token, c.bits64, std::uint64_t{0x7FF0000000000000u});
check_parse<float_json>(token, c.bits32, std::uint32_t{0x7F800000u});
}
}
TEST_CASE("float overflow and underflow in the parser")
{
SECTION("double")
{
check_parse<json>("1.7976931348623157e308", std::uint64_t{0x7FEFFFFFFFFFFFFFu}, std::uint64_t{0x7FF0000000000000u});
check_parse<json>("1.7976931348623159e308", std::uint64_t{0x7FF0000000000000u}, std::uint64_t{0x7FF0000000000000u});
check_parse<json>("-1e309", std::uint64_t{0xFFF0000000000000u}, std::uint64_t{0x7FF0000000000000u});
check_parse<json>("1" + std::string(400, '0'), std::uint64_t{0x7FF0000000000000u}, std::uint64_t{0x7FF0000000000000u});
check_parse<json>("1e99999999999999999999", std::uint64_t{0x7FF0000000000000u}, std::uint64_t{0x7FF0000000000000u});
// an underflow gives a zero with the sign of the token
check_parse<json>("1e-400", std::uint64_t{0}, std::uint64_t{0x7FF0000000000000u});
check_parse<json>("-1e-400", std::uint64_t{0x8000000000000000u}, std::uint64_t{0x7FF0000000000000u});
check_parse<json>("-2.4703282292062327e-324", std::uint64_t{0x8000000000000000u}, std::uint64_t{0x7FF0000000000000u});
check_parse<json>("0." + std::string(400, '0') + "1", std::uint64_t{0}, std::uint64_t{0x7FF0000000000000u});
}
SECTION("float")
{
check_parse<float_json>("3.4028234e38", std::uint32_t{0x7F7FFFFFu}, std::uint32_t{0x7F800000u});
check_parse<float_json>("3.4028236e38", std::uint32_t{0x7F800000u}, std::uint32_t{0x7F800000u});
check_parse<float_json>("-1e39", std::uint32_t{0xFF800000u}, std::uint32_t{0x7F800000u});
check_parse<float_json>("1e-46", std::uint32_t{0}, std::uint32_t{0x7F800000u});
check_parse<float_json>("-1e-46", std::uint32_t{0x80000000u}, std::uint32_t{0x7F800000u});
check_parse<float_json>("-7.006492321624085e-46", std::uint32_t{0x80000000u}, std::uint32_t{0x7F800000u});
check_parse<float_json>("-7.006492321624086e-46", std::uint32_t{0x80000001u}, std::uint32_t{0x7F800000u});
}
}
TEST_CASE("string scanning kernels")
{
// the word-at-a-time kernels must stop exactly where a byte-by-byte scan
// stops, for any content, length, and alignment
const auto reference_special = [](const unsigned char* data, std::size_t n)
{
std::size_t i = 0;
while (i < n && !nlohmann::detail::is_string_special(data[i]))
{
++i;
}
return i;
};
const auto reference_copyable = [](const unsigned char* data, std::size_t n)
{
std::size_t i = 0;
while (i < n && nlohmann::detail::is_ascii_copyable(data[i]))
{
++i;
}
return i;
};
const auto reference_bulk_run = [](const unsigned char* data, std::size_t n)
{
std::size_t i = 0;
while (i < n)
{
if (data[i] < 0x80u)
{
if (nlohmann::detail::is_string_special(data[i]))
{
break;
}
++i;
continue;
}
const std::size_t seq = nlohmann::detail::validate_one_utf8(data + i, n - i);
if (seq == 0)
{
break;
}
i += seq;
}
return i;
};
// pieces: ordinary ASCII, stops, DEL, well-formed sequences of every
// length, and ill-formed or truncated ones
const std::vector<std::string> pieces =
{
"a", "Z", " ", "~", "0123456789", "\"", "\\", std::string(1, '\0'), "\n", "\x1F", "\x7F",
"\xC3\xA4", "\xE2\x82\xAC", "\xE6\x97\xA5\xE6\x9C\xAC", "\xF0\x9F\x98\x80", "\xED\x9F\xBF",
"\x80", "\xC0\x80", "\xC3", "\xE2\x82", "\xED\xA0\x80", "\xF4\x90\x80\x80", "\xFF",
};
std::uint64_t state = 5295;
const auto next = [&state]()
{
state ^= state << 13u;
state ^= state >> 7u;
state ^= state << 17u;
return state;
};
// the upper half as a 32-bit value: converts to std::size_t implicitly on
// every platform (a cast of std::uint64_t is useless where both are the
// same type, and required where std::size_t is 32 bits wide)
const auto next_small = [&next]()
{
return static_cast<std::uint32_t>(next() >> 32u);
};
for (int round = 0; round < 100000; ++round)
{
// mostly ordinary text, so that runs span several words
std::string text(next_small() % 8u, '.');
const std::size_t count = next_small() % 12u;
for (std::size_t k = 0; k < count; ++k)
{
const std::size_t p = (next() % 4 == 0) ? next_small() % pieces.size() : 0;
text += pieces[p];
text += std::string(next_small() % 10u, 'x');
}
const auto* data = reinterpret_cast<const unsigned char*>(text.data()); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
for (std::size_t offset = 0; offset < 3 && offset <= text.size(); ++offset)
{
const std::size_t n = text.size() - offset;
CAPTURE(text);
CAPTURE(offset);
CHECK(nlohmann::detail::find_string_special(data + offset, n) == reference_special(data + offset, n));
CHECK(nlohmann::detail::find_ascii_copyable_run(data + offset, n) == reference_copyable(data + offset, n));
CHECK(nlohmann::detail::scalar_string_bulk_run(data + offset, n) == reference_bulk_run(data + offset, n));
}
}
// the trailing-zero count, whichever implementation the compiler gets
for (int k = 0; k < 64; ++k)
{
const std::uint64_t bit = std::uint64_t{1} << k;
CHECK(nlohmann::detail::count_trailing_zeros(bit) == k);
CHECK(nlohmann::detail::count_trailing_zeros(bit | (bit << 1u) | 0x8000000000000000u) == k);
}
}
-153
View File
@@ -15,65 +15,8 @@ using nlohmann::json;
#endif
#include <fstream>
#include <string>
#include <vector>
#include "make_test_data_available.hpp"
namespace
{
// alternating objects and arrays nested `depth` levels deep, with members that
// depend on `variant` at some levels, so diffing two variants yields
// operations on many levels: replacing the innermost value, adding, removing,
// and (for ordered_json) reordering members, and changing array lengths
template<typename BasicJsonType>
BasicJsonType nested(const std::size_t depth, const int variant)
{
BasicJsonType value = variant;
for (std::size_t i = 0; i < depth; ++i)
{
if (i % 2 == 0)
{
BasicJsonType object = BasicJsonType::object();
if ((i + static_cast<std::size_t>(variant)) % 7 == 0)
{
object["x"] = i;
}
if (variant == 2 && i % 11 == 0)
{
object["z"] = "z";
}
object["a"] = std::move(value);
if (variant == 1 && i % 5 == 0)
{
object["y"] = 1;
}
value = std::move(object);
}
else
{
BasicJsonType array = BasicJsonType::array({std::move(value)});
if ((i + static_cast<std::size_t>(variant)) % 3 == 0)
{
array.push_back(i);
}
value = std::move(array);
}
}
return value;
}
// a path of `depth` reference tokens, as nested() nests its values
std::string nested_path(const std::size_t depth)
{
std::string path;
for (std::size_t i = depth; i > 0; --i)
{
path += (i - 1) % 2 == 0 ? "/a" : "/0";
}
return path;
}
} // namespace
TEST_CASE("JSON patch")
{
SECTION("examples from RFC 6902")
@@ -1809,102 +1752,6 @@ TEST_CASE("JSON patch - diff emits array removals in descending index order")
}
}
TEST_CASE("JSON patch: diff of deeply nested values")
{
SECTION("the diff reproduces the target at every depth")
{
// depths on either side of the nesting depth up to which diff()
// recurses (detail::recursion_depth_limit(), 128); not every depth up
// to 300, as the test would then time out under Valgrind
std::vector<std::size_t> depths;
for (std::size_t depth = 0; depth <= 16; ++depth)
{
depths.push_back(depth);
}
for (std::size_t depth = 120; depth <= 136; ++depth)
{
depths.push_back(depth);
}
depths.push_back(300);
for (const auto depth : depths)
{
CAPTURE(depth);
for (int from = 0; from < 3; ++from)
{
for (int to = 0; to < 3; ++to)
{
CAPTURE(from);
CAPTURE(to);
const auto source = nested<json>(depth, from);
const auto target = nested<json>(depth, to);
const auto patch = json::diff(source, target);
CHECK(source.patch(patch) == target);
CHECK(patch.empty() == (from == to));
const auto ordered_source = nested<nlohmann::ordered_json>(depth, from);
const auto ordered_target = nested<nlohmann::ordered_json>(depth, to);
CHECK(ordered_source.patch(nlohmann::ordered_json::diff(ordered_source, ordered_target)) == ordered_target);
}
}
}
}
SECTION("a difference only in the innermost value is one replace operation")
{
for (std::size_t depth = 0; depth <= 300; ++depth)
{
CAPTURE(depth);
json source = 1;
json target = 2;
for (std::size_t i = 0; i < depth; ++i)
{
source = i % 2 == 0 ? json::object({{"a", std::move(source)}}) : json::array({std::move(source)});
target = i % 2 == 0 ? json::object({{"a", std::move(target)}}) : json::array({std::move(target)});
}
CHECK(json::diff(source, target, "/root") == json::array({{{"op", "replace"}, {"path", "/root" + nested_path(depth)}, {"value", 2}}}));
}
}
SECTION("values nested too deeply for the call stack (#5393)")
{
// diff() used to recurse once per nesting level, and compared the
// values with operator== on every level. The values are only
// parsed and diffed, never copied or compared, since those recurse
// too.
const std::size_t depth = 100000;
for (const bool objects :
{
false, true
})
{
CAPTURE(objects);
std::string source_text;
std::string target_text;
std::string equal_text;
std::string path;
for (std::size_t i = 0; i < depth; ++i)
{
source_text += objects ? "{\"a\":" : "[";
path += objects ? "/a" : "/0";
}
target_text = source_text + "2";
equal_text = source_text + "1";
source_text += "1";
const std::string closing(depth, objects ? '}' : ']');
const auto source = json::parse(source_text + closing);
const auto patch = json::diff(source, json::parse(target_text + closing));
REQUIRE(patch.size() == 1);
CHECK(patch[0]["op"] == "replace");
CHECK(patch[0]["path"] == path);
CHECK(patch[0]["value"] == 2);
CHECK(json::diff(source, json::parse(equal_text + closing)).empty());
}
}
}
TEST_CASE("JSON patch - diff() takes the fast path for non-reorderable object types (regression #5639)")
{
// #5465 added an order check to diff()'s object handling so a
+399
View File
@@ -0,0 +1,399 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#include "doctest_compatibility.h"
#include <nlohmann/json.hpp>
#include <nlohmann/detail/view/builder.hpp>
#include <nlohmann/detail/view/string_ref.hpp>
using nlohmann::json;
#include <cstdint>
#include <fstream>
#include <map>
#include <memory>
#include <random>
#include <sstream>
#include <string>
#include <utility>
#include <vector>
#include <test_data.hpp>
namespace
{
using nlohmann::detail::view::document_data;
using nlohmann::detail::view::node;
// the node index of a text; a vector input has no terminating NUL, so that
// AddressSanitizer catches any read past the last byte
struct built
{
std::unique_ptr<document_data, document_data::deleter> data{}; // NOLINT(readability-redundant-member-init)
std::vector<char> copy{}; // NOLINT(readability-redundant-member-init)
bool ok = false;
nlohmann::detail::view::parse_failure failure{};
};
template<typename FloatType = double>
built build(const std::string& text, bool comments, bool trailing_commas, bool sentinel)
{
built r;
r.data.reset(document_data::create(nlohmann::detail::view::estimate_nodes(text.data(), text.size())));
const char* src = text.c_str();
if (!sentinel)
{
r.copy.assign(text.begin(), text.end());
src = r.copy.data();
}
r.ok = nlohmann::detail::view::build < FloatType, !nlohmann::detail::abi_config::strict_nul_handling > (*r.data, src, text.size(), comments, trailing_commas, sentinel, r.failure);
r.data->src = src;
r.data->base[0] = src;
r.data->base[1] = r.data->arena.data();
return r;
}
// the value of a subtree, as json::parse would build it
json value_of(const document_data& d, const node*& n)
{
const node& x = *n;
++n;
switch (static_cast<json::value_t>(x.kind))
{
case json::value_t::object:
{
json o = json::object();
const node* const end = &x + x.next;
while (n != end)
{
const std::string key(d.str(*n), n->len);
++n;
o[key] = value_of(d, n);
}
return o;
}
case json::value_t::array:
{
json a = json::array();
const node* const end = &x + x.next;
while (n != end)
{
a.push_back(value_of(d, n));
}
return a;
}
case json::value_t::string:
return std::string(d.str(x), x.len);
case json::value_t::boolean:
return (x.flags & nlohmann::detail::view::node_flags::is_true) != 0;
case json::value_t::number_integer:
return static_cast<std::int64_t>(nlohmann::detail::view::integer_bits(x));
case json::value_t::number_unsigned:
return nlohmann::detail::view::integer_bits(x);
case json::value_t::number_float:
return json::parse(std::string(d.src + x.off, x.len)).get<double>();
case json::value_t::null:
case json::value_t::binary:
case json::value_t::discarded:
default:
return nullptr;
}
}
json value_of(const built& b)
{
const node* n = b.data->tape;
json v = value_of(*b.data, n);
CHECK(n == b.data->tape + b.data->tape_size);
return v;
}
// accept/reject and the value must match json::parse, for all options and
// with and without a NUL after the text
void check_same(const std::string& text)
{
CAPTURE(text);
for (int options = 0; options < 4; ++options)
{
const bool comments = (options & 1) != 0;
const bool trailing_commas = (options & 2) != 0;
const bool accepted = json::accept(text, comments, trailing_commas);
for (const bool sentinel :
{
true, false
})
{
const built b = build(text, comments, trailing_commas, sentinel);
CHECK(b.ok == accepted);
if (b.ok && accepted)
{
CHECK(value_of(b) == json::parse(text, nullptr, true, comments, trailing_commas));
}
}
}
}
// a small deterministic generator of documents
struct generator
{
std::mt19937 rng{5295}; // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed)
int r(int n)
{
return static_cast<int>(rng() % static_cast<unsigned>(n));
}
void ws(std::string& o)
{
for (int n = r(4) == 0 ? r(12) : r(2); n > 0; --n)
{
o += " \n\t\r "[r(6)];
}
}
void str(std::string& o)
{
static const char* const pieces[] = {"a", "Z", " ", "~", "\\n", "\\\"", "\\\\", "\\/", "\\u00e9", "\\ud83d\\ude00", "\xc3\xa9", "\xe3\x81\x82", "\xf0\x9f\x98\x80", "\x7f", "\\u001f", "long enough text to leave the first 16 bytes"}; // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
o += '"';
for (int n = r(3) == 0 ? r(20) : r(6); n > 0; --n)
{
o += pieces[r(16)];
}
o += '"';
}
void num(std::string& o)
{
static const char* const numbers[] = {"0", "-0", "1", "-1", "12", "123456789", "1234567890123456789", "9223372036854775807", "-9223372036854775808", // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
"9223372036854775808", "18446744073709551615", "18446744073709551616", "-9223372036854775809",
"1.5", "-2.25e-3", "1e10", "1E+2", "0.000001", "3.141592653589793238462643", "1e308", "-1e-400", "123.456e7"
};
o += numbers[r(22)];
}
void value(std::string& o, int depth)
{
ws(o);
const int k = depth > 5 ? 2 + r(6) : r(8);
if (k == 0 || k == 1)
{
const bool object = k == 0;
o += object ? '{' : '[';
for (int i = r(5); i > 0; --i)
{
ws(o);
if (object)
{
str(o);
ws(o);
o += ':';
}
value(o, depth + 1);
o += i > 1 ? "," : "";
}
ws(o);
o += object ? '}' : ']';
}
else if (k < 4)
{
str(o);
}
else if (k < 6)
{
num(o);
}
else
{
static const char* const literals[] = {"true", "false", "null"}; // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
o += literals[r(3)];
}
ws(o);
}
};
} // namespace
TEST_CASE("json_view string_ref")
{
// std::string_view in C++17, a stand-in with the same members before
using nlohmann::detail::view::string_ref;
const std::string text = "abc";
const string_ref r(text);
CHECK(r.length() == 3);
CHECK(std::string(r.begin(), r.end()) == "abc");
CHECK(r[1] == 'b');
CHECK(r != string_ref("abd"));
CHECK_FALSE(r != string_ref("abcd", 3));
std::ostringstream o;
o << r;
CHECK(o.str() == "abc");
}
TEST_CASE("json_view builder")
{
SECTION("scalars and containers")
{
for (const char* text :
{
"null", "true", "false", "0", "-0", "42", "-42", "1.5", "\"\"", "\"abc\"", "[]", "{}", "[1,2,3]", "{\"a\":1,\"b\":[true,null]}", // NOLINT(modernize-raw-string-literal)
" [ 1 , 2 ] ", "{\"a\" : {\"b\" : {}}}", "[[[]]]", "\"\\u00e4\\n\\ud83d\\ude00\"", "{\"a\":1,\"a\":2}", "18446744073709551616", // NOLINT(modernize-raw-string-literal)
"-9223372036854775809", "123456789012345678901234567890", "1e400", "-1e400", "1.7976931348623157e308"
})
{
check_same(text);
}
// the midpoint between the largest double and 2^1024 rounds to
// infinity (an overflow), one less to the largest double: with more
// than 19 digits, Eisel-Lemire cannot decide these, and the overflow
// check needs the exact comparison with the midpoint
const std::string midpoint = "179769313486231580793728971405303415079934132710037826936173778980444968292764750946649017977587207096330286416692887910946555547851940402630657488671505820681908902000708383676273854845817711531764475730270069855571366959622842914819860834936475292719074168444365510704342711559699508093042880177904174497792";
const std::string below = "179769313486231580793728971405303415079934132710037826936173778980444968292764750946649017977587207096330286416692887910946555547851940402630657488671505820681908902000708383676273854845817711531764475730270069855571366959622842914819860834936475292719074168444365510704342711559699508093042880177904174497791";
check_same(midpoint);
check_same("-" + midpoint);
check_same(below);
check_same("[" + below + "," + midpoint + "]");
// the check uses the floating-point type of the document: with float,
// the view rejects what parse() rejects (out_of_range.406), and a
// double document is not affected
using float_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int64_t, std::uint64_t, float>;
CHECK_FALSE(float_json::accept("1e39"));
CHECK(float_json::accept("3.4028235e38"));
CHECK_FALSE(float_json::accept("3.4028236e38"));
for (const char* text :
{
"1e39", "-1e39", "3.4028235e38", "-3.4028235e38", "3.4028236e38", "-3.4028236e38", "3.4028234663852886e38", "1e38",
"340282356779733661637539395458142568448", "340282356779733661637539395458142568447.99", "0.00034028236e42",
"[1.5e38, 3.5e38]", "{\"a\": 1e-50, \"b\": 1e39}"
})
{
CAPTURE(text);
const bool float_accepted = float_json::accept(text);
for (const bool sentinel :
{
true, false
})
{
const built f = build<float>(text, false, false, sentinel);
CHECK(f.ok == float_accepted);
if (!f.ok)
{
CHECK(f.failure.code == nlohmann::detail::view::error_code::number_overflow);
}
CHECK(build<double>(text, false, false, sentinel).ok == json::accept(text));
}
}
}
SECTION("malformed input")
{
for (const char* text :
{
"", " ", "[", "]", "{", "}", "[1,]", "{\"a\":1,}", "[1 2]", "{\"a\" 1}", "{1:2}", "tru", "nul", "fals", "truex", "-", "01", "1.", ".5", "1e", "1e+",
"\"", "\"abc", "\"\\x\"", "\"\\u12\"", "\"\\u12G4\"", "\"\\ud800\"", "\"\\udc00\"", "\"\\ud800\\u0041\"", "\"\x01\"", "\"\xff\"", "\"\xc3\"", // NOLINT(modernize-raw-string-literal)
"\"\xe0\x80\x80\"", "\"\xed\xa0\x80\"", "[1]x", "[1] [2]", "/", "/*", "/* */ 1", "// c\n1", "1 // c", "[1,/*c*/2]", "[1,2,]"
})
{
check_same(text);
}
}
SECTION("NUL, BOM, and whitespace")
{
// a NUL inside a string is a control character, as for json::parse
// (where a NUL ends the input, it does so only between values)
for (const bool sentinel :
{
true, false
})
{
const built b = build(std::string("[\"ab\0cd\"]", 9), false, false, sentinel);
CHECK(!b.ok);
CHECK(b.failure.code == nlohmann::detail::view::error_code::string_control_character);
CHECK(b.failure.offset == 4);
}
check_same(std::string("[1]\0garbage", 11));
check_same(std::string("[1\0]", 4));
check_same(std::string("[1, // c\0\n2]", 12));
check_same(std::string("[1, /* c\0 */ 2]", 15));
check_same("\xEF\xBB\xBF[1]");
check_same("\xEF\xBB[1]");
check_same(" \t\r\n 7 \n");
for (const char* text :
{"[1]\r", "[1]\n", "[1]\r\n", "[1,\r2]", "[1,\r\n2]", "7\r", "\"x\"\r", "{\"a\":\r\n1}\r", "[\n 1,\n 2\n]", "{\n \"a\": [\n 1\n ]\n}"
})
{
check_same(text);
}
}
SECTION("deep nesting")
{
// the open containers beyond 64 levels live on the heap
for (const std::size_t depth :
{
63u, 64u, 65u, 1000u, 100000u
})
{
const std::string arrays = std::string(depth, '[') + std::string(depth, ']');
const built b = build(arrays, false, false, false);
REQUIRE(b.ok);
CHECK(b.data->tape_size == depth);
CHECK(b.data->tape[0].next == depth);
std::string objects;
for (std::size_t i = 0; i < depth; ++i)
{
objects += "{\"a\":";
}
objects += '1' + std::string(depth, '}');
const built o = build(objects, false, false, false);
REQUIRE(o.ok);
CHECK(o.data->tape_size == (2 * depth) + 1);
CHECK(!build(std::string(depth, '[') + std::string(depth - 1, ']'), false, false, false).ok);
}
}
SECTION("generated documents and damaged copies")
{
generator g;
for (int i = 0; i < 3000; ++i)
{
std::string text;
g.value(text, 0);
check_same(text);
// damage: flip one byte, or cut the text
std::string damaged = text;
const auto at = static_cast<std::size_t>(g.r(static_cast<int>(damaged.size())));
static const char replacements[] = {'x', '"', '\\', ',', ':', ']', '}', '[', '{', '1', '-', '.', 'e', '\0', '\n', '/'}; // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
damaged[at] = replacements[g.r(16)];
check_same(damaged);
check_same(text.substr(0, at));
}
}
SECTION("test files")
{
for (const char* name :
{
"/json.org/1.json", "/json.org/2.json", "/json.org/3.json", "/json.org/4.json", "/json.org/5.json",
"/json_testsuite/sample.json", "/nativejson-benchmark/canada.json", "/nativejson-benchmark/citm_catalog.json",
"/nativejson-benchmark/twitter.json", "/json_tests/pass1.json", "/json_tests/pass2.json", "/json_tests/pass3.json"
})
{
CAPTURE(name);
std::ifstream f(std::string(TEST_DATA_DIRECTORY) + name, std::ios::binary);
std::stringstream ss;
ss << f.rdbuf();
const std::string text = ss.str();
REQUIRE(!text.empty());
const built b = build(text, false, false, true);
REQUIRE(b.ok);
CHECK(value_of(b) == json::parse(text));
}
}
}
+25 -8
View File
@@ -257,10 +257,11 @@ struct LocaleSwitchingSax final: public nlohmann::json_sax<json>
TEST_CASE("locale changes between lexer construction and number conversion (#5198)")
{
// The numbers are chosen so that the conversion also takes the strtod
// fallback, which honors the locale that is current at conversion time:
// too many significant digits for Clinger's fast path, an underflow that
// std::from_chars rejects, and a plain value.
// float and double are converted without the locale. A long double that
// is not binary64 can take the strtold fallback, which honors the locale
// that is current at conversion time. The numbers are chosen so that it
// does: too many significant digits for Clinger's fast path, an underflow
// that std::from_chars rejects, and a plain value.
const std::vector<std::string> numbers = {"3.14159265358979323846", "1.5e-400", "12.34", "-0.000123456789012345678"};
std::string text = "[";
for (const auto& n : numbers)
@@ -324,7 +325,8 @@ TEST_CASE("locale changes between lexer construction and number conversion (#519
}
}
// a long double goes through std::strtold unless std::from_chars supports it
// a long double goes through std::strtold unless it is binary64 or
// std::from_chars supports it
{
bool switched = false;
const auto cb = [&](int /*depth*/, long_double_json::parse_event_t event, long_double_json& /*parsed*/) noexcept
@@ -350,8 +352,15 @@ TEST_CASE("locale with a multi-byte decimal point")
{
// Some locales use a decimal point that is not a single character, e.g.
// U+066B ARABIC DECIMAL SEPARATOR (two bytes in UTF-8). It cannot be
// substituted in place for '.', so the strtod fallback stops early. The
// conversion must still terminate rather than retry forever.
// substituted in place for '.', so the strtold fallback (only for long
// double formats other than binary64) converts a copy of the token with
// the whole decimal point instead (#5660). The values must be those of the
// "C" locale.
using long_double_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int64_t, std::uint64_t, long double>;
const char* const long_double_numbers = "[3.14159265358979323846, 1.5e-400, -0.000123456789012345678]";
REQUIRE(std::setlocale(LC_NUMERIC, "C") != nullptr);
const long_double_json expected_long_double = long_double_json::parse(long_double_numbers);
const std::array<const char*, 6> names = {{"ar_EG.UTF-8", "ar_SA.UTF-8", "fa_IR.UTF-8", "ps_AF.UTF-8", "ar_EG", "fa_IR"}};
bool tested = false;
for (const char* name : names)
@@ -369,12 +378,20 @@ TEST_CASE("locale with a multi-byte decimal point")
tested = true;
// too many significant digits for Clinger's fast path, and an underflow
// that std::from_chars rejects: both reach the strtod fallback
// that std::from_chars rejects: double does not depend on the locale
json j;
CHECK_NOTHROW(j = json::parse("[3.14159265358979323846, 1.5e-400, -0.000123456789012345678]"));
CHECK(j.is_array());
CHECK(j[0] == 3.14159265358979323846);
CHECK(j[1] == 0.0);
CHECK(j[2] == -0.000123456789012345678);
CHECK(json::accept("3.14159265358979323846"));
// a long double that reaches the strtold fallback is not truncated
long_double_json ld;
CHECK_NOTHROW(ld = long_double_json::parse(long_double_numbers));
CHECK(ld == expected_long_double);
// a value the locale-independent paths convert is not affected
CHECK(json::parse("12.5") == 12.5);
}