Compare commits

..
Author SHA1 Message Date
Niels Lohmann 649961c8fe Merge branch 'perf/lexer-number-scan' into perf/lexer-lazy-float-text
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-15 22:46:59 +02:00
Niels Lohmann fe8900bd92 Merge branch 'claude/parser-stream-performance-8cebe9' into perf/lexer-number-scan
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-15 22:46:54 +02:00
Niels Lohmann 39565f62d7 Merge branch 'develop' into claude/parser-stream-performance-8cebe9
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-15 22:46:29 +02:00
Niels Lohmann 48d5e59adf Merge branch 'perf/lexer-number-scan' into perf/lexer-lazy-float-text
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-15 20:23:21 +02:00
Niels Lohmann 547e745289 Merge branch 'claude/parser-stream-performance-8cebe9' into perf/lexer-number-scan
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-15 20:23:17 +02:00
Niels Lohmann 1457bfb948 Merge branch 'develop' into claude/parser-stream-performance-8cebe9
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-15 20:22:53 +02:00
Niels Lohmann 5423dd8de3 Copy the text of float tokens only when a SAX handler asks for it
After the digits are converted while scanning, copying the token text into
token_buffer (an out-of-line std::string::append) was about 10% of parsing
canada.json. The only reader is the string argument of
json_sax::number_float(), and the library's own handlers
(json_sax_dom_parser, json_sax_dom_callback_parser, json_sax_acceptor)
ignore it.

For contiguous input, the fast path now keeps a pointer to the token in the
input, and get_string() copies it on first use. The parser passes an empty
string to the handlers marked by the new internal trait
sax_ignores_number_text instead of calling get_string(), so json::parse()
and json::accept() never copy float text. User handlers, and
JSON_DIAGNOSTIC_POSITIONS (which reads get_string().size()), still see the
exact text. Keys and strings, which never have pending text, use the
unchecked get_string_token(). Input read through a window into a stream
(std::istream, FILE*) copies the text right away, because the next read may
refill the window.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-15 18:37:02 +02:00
Niels Lohmann 32ca00cdfa Accumulate number digits while scanning them
The contiguous number fast path validated a token byte by byte, copied it
into token_buffer, and then converted it in out-of-line functions that
scanned the digits a second time: parse_float_native() called
parse_float_significand(), which returned its result through memory and
called std::any_of (out of line with GCC) twice per number, and
parse_integer_signed() divided by ten on every digit for its overflow check.

scan_number_bulk_contiguous() now accumulates the significand while it
validates the digits, eight at a time where at least eight bytes remain
(is_eight_digits() and parse_eight_digits(), after fast_float). Floats with
at most 19 digits go straight to decimal_to_float(); integers with at most
18 digits are stored without an overflow check (a narrower number type
still rejects values that do not round-trip). Longer tokens take the
existing path. has_nonzero_digit() is a plain loop and
parse_float_significand() is always inlined, which helps that path.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-15 18:37:01 +02:00
Niels Lohmann 598fbd0417 Avoid implicit unsigned char to char conversion in stream tests
Building a std::string from the iterators of a CBOR byte vector narrows
each byte, which -fsanitize=implicit-conversion reports for bytes above
0x7F. Construct the string from the raw bytes instead.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 21:30:07 +02:00
Niels Lohmann 898084b386 Merge branch 'develop' into claude/parser-stream-performance-8cebe9
Resolve the macro_unscope.hpp conflict by keeping both sets of #undefs
and regenerate the amalgamation.

The \u fast path from #5738 (get_codepoint_bulk) assumed that bulk
scanning implies lazy position tracking and advanced the adapter with a
bare bulk_skip(4). Stream adapters now scan in bulk but track position
and token eagerly, so a lone-surrogate error was reported 4 columns too
early. Advance through bulk_consume() instead.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 21:30:06 +02:00
Niels Lohmann f13fbd2577 Add benchmarks for parsing from a FILE* and for many values from a stream
ParseFILE/* parses the nativejson-benchmark files from a FILE*, and
ParseStreamValues/* reads 10,000 small values from one std::istringstream
with operator>> (NDJSON objects and plain numbers), which measures the
per-value cost of stream input.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:34:01 +02:00
Niels Lohmann c391abccb1 Lock a FILE once per parse instead of once per character
Parsing from a FILE* reads one character at a time with std::fgetc, which
locks and unlocks the FILE object for every character. file_input_adapter
now locks the file once when it is created and unlocks it when it is
destroyed - also when parsing throws - and reads with the unlocked stdio
functions in between: flockfile/getc_unlocked/funlockfile with glibc,
Apple, Android, and the BSDs, and _lock_file/_fgetc_nolock/_unlock_file
with the Microsoft C runtime. Other C libraries keep std::fgetc. The lock
is recursive, so fread() in the binary readers and stdio calls of a SAX
handler or parser callback on the same thread keep working.

On an Apple M1 Max, parsing from a FILE* is 2.9 to 4.0 times as fast
(twitter.json 14.3 -> 4.1 ms, citm_catalog.json 35.4 -> 8.8 ms); with
glibc, whose locking is cheaper, 1.1 to 1.3 times. The values and the
position of the file after parsing are unchanged.

Other threads that use the same FILE object now wait until parsing is
done instead of interleaving their reads with the parser's.
JSON_DISABLE_UNLOCKED_FILE_IO restores std::fgetc, for programs in which a
SAX handler waits for another thread that uses the same FILE. The new
tests check from a second thread that the lock is released after success
and after a parse error, and run again with the macro.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:34:00 +02:00
Niels Lohmann d5631dccbd Scan the get area of an input stream in bulk
Parsing from a std::istream read one character at a time through the
streambuf, so none of the parser's fast paths applied: it was 1.8 to 2.5
times slower than reading the file into a string and parsing that.

input_stream_adapter now exposes the streambuf's get area (gptr() to
egptr()) to the lexer as a window. The protected get-area members are
reached through member pointers formed in a class derived from
std::streambuf, which is ordinary, standard-conforming member access. The
lexer scans the window with the same bulk paths as a string, and asks
the streambuf for the next character as before when the window is used
up. Nothing is read ahead, copied, or put back, so the stream is left
exactly where it was left before, with and without
JSON_PRECISE_STREAM_POSITION, and a streambuf without a get area is still
read one character at a time.

The adapter also keeps the current token for error messages as a pointer
into the get area, and copies it out only before the get area is
refilled, instead of the lexer copying every character.

On an Apple M1 Max (Apple Clang, -O3), parsing from an std::ifstream is
1.6 to 2.1 times as fast (twitter.json 4.08 -> 2.30 ms, citm_catalog.json
8.75 -> 4.08 ms), and reading 100,000 small values with operator>> 1.4
(objects) to 4.0 (integers) times as fast. With GCC 13 and libstdc++ on
x86-64 Linux, std::ifstream is 1.5 to 2.2 times as fast. Parsing from a
stream is now within 11 to 18 % of reading the file into a string first.

JSON_DISABLE_STREAM_BULK_SCAN restores reading one character at a time,
for streambufs that do not follow the usual get-area contract. The new
tests compare every observable result (values, exception messages, stream
state, remaining characters) for get areas of 1 to 4096 bytes against a
streambuf without a get area, and run again with the macro and with
JSON_PRECISE_STREAM_POSITION.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:26:38 +02:00
Niels Lohmann 80d00eda50 Let the lexer's bulk fast paths run on a window of the input
The SWAR string scan, the bulk number scan, and the indentation skip were
only enabled for adapters that support seeking (contiguous random-access
input), because they rely on the lazy token and position tracking that
reconstructs both from the consumed input on demand.

This decouples the bulk paths from seeking, so that an adapter can expose
a contiguous block that is only a window into its input (bulk_window):

- bulk_consume() does the bookkeeping that get() would have done for the
  skipped characters when token and position are tracked eagerly.
- A number that reaches the end of a window is finished character by
  character, because it may continue after the window.
- After a bulk-scanned number, a window adapter reads the terminating
  character and ungets it, as scan_number() does, so that the input
  handed back to the user is consumed exactly as before.
- An adapter can keep the current token itself (supports_token_tracking,
  begin_token(), mark_token_start(), copy_token()) so the lexer need not
  copy every character for its error messages.

No adapter declares any of this yet, so the behavior is unchanged.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-14 17:13:58 +02:00
22 changed files with 697 additions and 271 deletions

No files matched your search

+8 -10
View File
@@ -166,21 +166,19 @@ jobs:
# link, but the binaries clang 11.0.1 and clang 18.1.8 then produce crash # link, but the binaries clang 11.0.1 and clang 18.1.8 then produce crash
# before doctest prints its first line - 39 of 102 tests on clang 18. # before doctest prints its first line - 39 of 102 tests on clang 18.
# Keep the objects small by splitting the test files instead. # Keep the objects small by splitting the test files instead.
# The tests link with LLD: GNU ld fails on objects with more than 32767 # Objects with more than 65535 sections fail to link with "relocation
# sections with "relocation truncated to fit: IMAGE_REL_AMD64_REL32 # truncated to fit: IMAGE_REL_AMD64_REL32 against `.rdata'": the MinGW
# against `.rdata'", as it stores the section an associative COMDAT # linker stores the section an associative COMDAT section belongs to in
# section belongs to in 16 bits (x_associated in binutils' # 16 bits (x_associated in binutils' include/coff/internal.h), so it
# include/coff/internal.h) and so discards the jump tables of inline # discards the jump tables of inline functions together with the wrong
# functions together with the wrong function. Each basic_json # function. Each basic_json specialization a test file instantiates adds
# specialization and input type a test file instantiates adds many # many sections, so test a second one in a file of its own.
# sections, so unit-msgpack.cpp and unit-regression2.cpp kept crossing
# that limit as the library grew.
- name: Run CMake - name: Run CMake
run: cmake -S . -B build ^ run: cmake -S . -B build ^
-DCMAKE_CXX_COMPILER="C:/Program Files/LLVM/bin/clang++.exe" ^ -DCMAKE_CXX_COMPILER="C:/Program Files/LLVM/bin/clang++.exe" ^
-DCMAKE_CXX_FLAGS="--target=x86_64-w64-mingw32 -stdlib=libstdc++ -pthread" ^ -DCMAKE_CXX_FLAGS="--target=x86_64-w64-mingw32 -stdlib=libstdc++ -pthread" ^
-DCMAKE_CXX_FLAGS_DEBUG="-g0" ^ -DCMAKE_CXX_FLAGS_DEBUG="-g0" ^
-DCMAKE_EXE_LINKER_FLAGS="-fuse-ld=lld -lwinpthread" ^ -DCMAKE_EXE_LINKER_FLAGS="-lwinpthread" ^
-G"MinGW Makefiles" ^ -G"MinGW Makefiles" ^
-DCMAKE_BUILD_TYPE=Debug ^ -DCMAKE_BUILD_TYPE=Debug ^
-DJSON_BuildTests=On -DJSON_BuildTests=On
@@ -13,7 +13,7 @@
#include <optional> // optional #include <optional> // optional
#endif #endif
#include <iterator> // begin, end, make_move_iterator, move_iterator #include <iterator> // begin, end
#include <memory> // allocator_traits #include <memory> // allocator_traits
#include <string> // basic_string, char_traits #include <string> // basic_string, char_traits
#include <tuple> // tuple, get #include <tuple> // tuple, get
@@ -48,32 +48,6 @@ namespace detail
* leave j with the type of the new value, but the pointer to the destroyed old one. * leave j with the type of the new value, but the pointer to the destroyed old one.
*/ */
// a move iterator over the elements of an rvalue container, unless they are
// const (e.g., std::set): libstdc++ before GCC 5 defines the reference of a
// std::move_iterator as value_type&&, which cannot bind to a const element
template<typename Iterator>
std::move_iterator<Iterator> make_move_iterator_if_mutable_impl(const Iterator& it, std::true_type /*mutable*/)
{
return std::make_move_iterator(it);
}
template<typename Iterator>
Iterator make_move_iterator_if_mutable_impl(const Iterator& it, std::false_type /*mutable*/)
{
return it;
}
template<typename Iterator>
using is_mutable_iterator = std::integral_constant < bool,
!std::is_const<typename std::remove_reference<decltype(*std::declval<Iterator&>())>::type>::value >;
template<typename Iterator>
auto make_move_iterator_if_mutable(const Iterator& it)
-> decltype(make_move_iterator_if_mutable_impl(it, is_mutable_iterator<Iterator> {}))
{
return make_move_iterator_if_mutable_impl(it, is_mutable_iterator<Iterator> {});
}
template<value_t> struct external_constructor; template<value_t> struct external_constructor;
template<> template<>
@@ -200,7 +174,7 @@ struct external_constructor<value_t::array>
using std::begin; using std::begin;
using std::end; using std::end;
const typename BasicJsonType::json_value value(j.template create<typename BasicJsonType::array_t>(make_move_iterator_if_mutable(begin(arr)), make_move_iterator_if_mutable(end(arr)))); const typename BasicJsonType::json_value value(j.template create<typename BasicJsonType::array_t>(std::make_move_iterator(begin(arr)), std::make_move_iterator(end(arr))));
j.replace_value(value_t::array, value); j.replace_value(value_t::array, value);
} }
@@ -261,7 +235,7 @@ struct external_constructor<value_t::object>
using std::begin; using std::begin;
using std::end; using std::end;
const typename BasicJsonType::json_value value(j.template create<typename BasicJsonType::object_t>(make_move_iterator_if_mutable(begin(obj)), make_move_iterator_if_mutable(end(obj)))); const typename BasicJsonType::json_value value(j.template create<typename BasicJsonType::object_t>(std::make_move_iterator(begin(obj)), std::make_move_iterator(end(obj))));
j.replace_value(value_t::object, value); j.replace_value(value_t::object, value);
} }
@@ -439,27 +413,25 @@ using is_movable_container_rvalue = negation<disjunction<
std::is_const<T>, std::is_const<T>,
has_dedicated_to_json<T>, has_dedicated_to_json<T>,
is_basic_json<T>, is_basic_json<T>,
is_range_view_type<T>,
is_enum_keyed_map_type<T>, is_enum_keyed_map_type<T>,
std::is_same<T, std::vector<bool>>>>; std::is_same<T, std::vector<bool>>>>;
// rvalue array-like containers; object types are ruled out before views are // rvalue array-like containers
// checked, as std::ranges::view<T> for the object_t of an ordered_json would
// need that ordered_json to be complete (GCC 11, see #3312)
template<typename BasicJsonType, typename T> template<typename BasicJsonType, typename T>
using is_rvalue_array_to_json = conjunction<is_movable_container_rvalue<T>, using is_rvalue_array_to_json = conjunction<is_movable_container_rvalue<T>,
negation<is_compatible_object_type<BasicJsonType, T>>,
negation<is_range_view_type<T>>,
is_compatible_array_type<BasicJsonType, T>, is_compatible_array_type<BasicJsonType, T>,
negation<disjunction< negation<disjunction<
std::is_same<T, typename BasicJsonType::array_t>, std::is_same<T, typename BasicJsonType::array_t>,
std::is_same<T, typename BasicJsonType::binary_t>, std::is_same<T, typename BasicJsonType::binary_t>,
is_compatible_object_type<BasicJsonType, T>,
is_compatible_string_type<BasicJsonType, T>, is_compatible_string_type<BasicJsonType, T>,
is_compatible_binary_type<BasicJsonType, T>>>>; is_compatible_binary_type<BasicJsonType, T>>>>;
// rvalue containers: their elements are moved instead of copied // rvalue containers: their elements are moved instead of copied
template<typename BasicJsonType, typename CompatibleArrayType, template<typename BasicJsonType, typename CompatibleArrayType,
enable_if_t<is_rvalue_array_to_json<BasicJsonType, CompatibleArrayType>::value, int> = 0> enable_if_t<is_rvalue_array_to_json<BasicJsonType, CompatibleArrayType>::value, int> = 0>
inline void to_json(BasicJsonType& j, CompatibleArrayType && arr) // NOLINT(cppcoreguidelines-missing-std-forward) inline void to_json(BasicJsonType& j, CompatibleArrayType && arr)
{ {
external_constructor<value_t::array>::construct_moving(j, arr); external_constructor<value_t::array>::construct_moving(j, arr);
} }
@@ -537,8 +509,7 @@ inline void to_json(BasicJsonType& j, const EnumKeyedMap& map)
} }
#endif #endif
// rvalue object-like containers (views have no key_type and mapped_type, so // rvalue object-like containers
// they are no compatible object type)
template<typename BasicJsonType, typename T> template<typename BasicJsonType, typename T>
using is_rvalue_object_to_json = conjunction<is_movable_container_rvalue<T>, using is_rvalue_object_to_json = conjunction<is_movable_container_rvalue<T>,
negation<std::is_same<T, typename BasicJsonType::object_t>>, negation<std::is_same<T, typename BasicJsonType::object_t>>,
@@ -546,7 +517,7 @@ using is_rvalue_object_to_json = conjunction<is_movable_container_rvalue<T>,
template<typename BasicJsonType, typename CompatibleObjectType, template<typename BasicJsonType, typename CompatibleObjectType,
enable_if_t<is_rvalue_object_to_json<BasicJsonType, CompatibleObjectType>::value, int> = 0> enable_if_t<is_rvalue_object_to_json<BasicJsonType, CompatibleObjectType>::value, int> = 0>
inline void to_json(BasicJsonType& j, CompatibleObjectType && obj) // NOLINT(cppcoreguidelines-missing-std-forward) inline void to_json(BasicJsonType& j, CompatibleObjectType && obj)
{ {
external_constructor<value_t::object>::construct_moving(j, obj); external_constructor<value_t::object>::construct_moving(j, obj);
} }
+1 -1
View File
@@ -185,7 +185,7 @@ JSON_HEDLEY_ALWAYS_INLINE shortest_decimal to_shortest(std::uint64_t bits) noexc
const uint128_parts p = umul192_hi128(p10.high, p10.low, bin_sig << static_cast<unsigned>(shift)); const uint128_parts p = umul192_hi128(p10.high, p10.low, bin_sig << static_cast<unsigned>(shift));
std::uint64_t integral = p.high >> static_cast<unsigned>(extra_shift); std::uint64_t integral = p.high >> static_cast<unsigned>(extra_shift);
// (no casts in the shifts: MSVC 2015 warns C4554 on them despite the parentheses) // (no casts in the shifts: MSVC 2015 warns C4554 on them despite the parentheses)
constexpr auto fractional_low_shift = static_cast<unsigned>(extra_shift); constexpr unsigned fractional_low_shift = static_cast<unsigned>(extra_shift);
constexpr unsigned fractional_high_shift = 64u - fractional_low_shift; constexpr unsigned fractional_high_shift = 64u - fractional_low_shift;
const std::uint64_t fractional = (p.high << fractional_high_shift) | (p.low >> fractional_low_shift); const std::uint64_t fractional = (p.high << fractional_high_shift) | (p.low >> fractional_low_shift);
std::uint64_t digit = 0; std::uint64_t digit = 0;
+2 -1
View File
@@ -494,7 +494,8 @@ JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE void throw_patch_missing_member(c
static_cast<void>(string_type); static_cast<void>(string_type);
static_cast<void>(val); static_cast<void>(val);
// context-sensitive error message // context-sensitive error message
JSON_THROW(parse_error::create(exception_id::patch_invalid_operation, 0, concat((op == "op") ? std::string("operation") : concat("operation '", op, '\''), string_type ? " must have string member '" : " must have member '", member, "'"), &val)); const std::string error_msg = (op == "op") ? std::string("operation") : concat("operation '", op, '\'');
JSON_THROW(parse_error::create(exception_id::patch_invalid_operation, 0, concat(error_msg, string_type ? " must have string member '" : " must have member '", member, "'"), &val));
} }
template<typename StringType, typename BasicJsonType> template<typename StringType, typename BasicJsonType>
@@ -90,9 +90,12 @@ class file_input_adapter
file_input_adapter& operator=(const file_input_adapter&) = delete; file_input_adapter& operator=(const file_input_adapter&) = delete;
file_input_adapter& operator=(file_input_adapter&&) = delete; file_input_adapter& operator=(file_input_adapter&&) = delete;
~file_input_adapter() ~file_input_adapter()
{
if (m_file != nullptr)
{ {
unlock(); unlock();
} }
}
std::char_traits<char>::int_type get_character() noexcept std::char_traits<char>::int_type get_character() noexcept
{ {
@@ -129,11 +132,6 @@ class file_input_adapter
void unlock() noexcept void unlock() noexcept
{ {
// a moved-from adapter has no file and holds no lock
if (m_file == nullptr)
{
return;
}
#if JSON_UNLOCKED_FILE_IO == 1 #if JSON_UNLOCKED_FILE_IO == 1
funlockfile(m_file); funlockfile(m_file);
#elif JSON_UNLOCKED_FILE_IO == 2 #elif JSON_UNLOCKED_FILE_IO == 2
+111 -14
View File
@@ -1625,9 +1625,12 @@ scan_number_done:
const std::size_t avail = rem + 1; const std::size_t avail = rem + 1;
// validate + classify the number extent (mirrors scan_number()'s grammar) // validate + classify the number extent (mirrors scan_number()'s grammar)
// and accumulate the mantissa digits on the way, so that tokens with up
// to 19 digits need no second pass for their value
std::size_t i = 0; std::size_t i = 0;
std::size_t dot_index = std::string::npos; std::size_t dot_index = std::string::npos;
token_type number_type = token_type::value_unsigned; token_type number_type = token_type::value_unsigned;
std::uint64_t w = 0;
if (data[0] == '-') if (data[0] == '-')
{ {
number_type = token_type::value_integer; number_type = token_type::value_integer;
@@ -1637,22 +1640,21 @@ scan_number_done:
return token_type::uninitialized; return token_type::uninitialized;
} }
} }
const std::size_t integer_begin = i;
if (data[i] == '0') if (data[i] == '0')
{ {
++i; ++i;
} }
else if (data[i] >= '1' && data[i] <= '9') else if (data[i] >= '1' && data[i] <= '9')
{ {
++i; i = accumulate_digits(data, i, avail, w);
while (i < avail && data[i] >= '0' && data[i] <= '9')
{
++i;
}
} }
else else
{ {
return token_type::uninitialized; return token_type::uninitialized;
} }
std::size_t digit_count = i - integer_begin;
std::size_t fraction_digits = 0;
if (i < avail && data[i] == '.') if (i < avail && data[i] == '.')
{ {
number_type = token_type::value_float; number_type = token_type::value_float;
@@ -1662,29 +1664,39 @@ scan_number_done:
{ {
return token_type::uninitialized; return token_type::uninitialized;
} }
while (i < avail && data[i] >= '0' && data[i] <= '9') const std::size_t fraction_begin = i;
{ i = accumulate_digits(data, i, avail, w);
++i; fraction_digits = i - fraction_begin;
} digit_count += fraction_digits;
} }
// the mantissa ends here, whether or not an exponent part follows // the mantissa ends here, whether or not an exponent part follows
const std::size_t mantissa_end = i; const std::size_t mantissa_end = i;
std::int64_t exponent = 0;
if (i < avail && (data[i] == 'e' || data[i] == 'E')) if (i < avail && (data[i] == 'e' || data[i] == 'E'))
{ {
number_type = token_type::value_float; number_type = token_type::value_float;
++i; ++i;
bool negative_exponent = false;
if (i < avail && (data[i] == '+' || data[i] == '-')) if (i < avail && (data[i] == '+' || data[i] == '-'))
{ {
negative_exponent = data[i] == '-';
++i; ++i;
} }
if (i >= avail || !(data[i] >= '0' && data[i] <= '9')) if (i >= avail || !(data[i] >= '0' && data[i] <= '9'))
{ {
return token_type::uninitialized; return token_type::uninitialized;
} }
// saturated far beyond every range, see parse_float_exponent()
constexpr std::int64_t saturation = 100000000000000000; // 10^17
while (i < avail && data[i] >= '0' && data[i] <= '9') while (i < avail && data[i] >= '0' && data[i] <= '9')
{ {
if (exponent < saturation)
{
exponent = (exponent * 10) + (data[i] - '0');
}
++i; ++i;
} }
exponent = negative_exponent ? -exponent : exponent;
} }
const std::size_t len = i; const std::size_t len = i;
if (number_may_continue(len == avail, std::integral_constant<bool, bulk_window> {})) if (number_may_continue(len == avail, std::integral_constant<bool, bulk_window> {}))
@@ -1705,7 +1717,10 @@ scan_number_done:
#if !JSON_DIAGNOSTIC_POSITIONS #if !JSON_DIAGNOSTIC_POSITIONS
if (number_type != token_type::value_float) if (number_type != token_type::value_float)
{ {
const token_type integer_result = convert_integer(number_type, data, data + len); // up to 18 digits always fit into std::uint64_t and std::int64_t
const token_type integer_result = (digit_count <= 18)
? store_integer(number_type, w)
: convert_integer(number_type, data, data + len);
if (JSON_HEDLEY_LIKELY(integer_result != token_type::uninitialized)) if (JSON_HEDLEY_LIKELY(integer_result != token_type::uninitialized))
{ {
bulk_consume(data + 1, len - 1); bulk_consume(data + 1, len - 1);
@@ -1719,15 +1734,42 @@ scan_number_done:
} }
#endif #endif
// materialize the token exactly as scan_number() would. reset() already
// cleared token_buffer, so append() fills it (assign() is avoided
// because custom string_t types need not provide it)
token_buffer.append(reinterpret_cast<const typename string_t::value_type*>(data), len);
decimal_point_position = dot_index; decimal_point_position = dot_index;
bulk_consume(data + 1, len - 1); bulk_consume(data + 1, len - 1);
read_number_terminator(std::integral_constant<bool, bulk_window> {}); read_number_terminator(std::integral_constant<bool, bulk_window> {});
// a float whose digits all fit into w converts from the accumulated
// value; longer ones (and other float formats) take the general path
if (number_type == token_type::value_float && digit_count <= 19)
{
float_significand s;
s.w = w;
s.exponent = exponent - static_cast<std::int64_t>(fraction_digits);
s.negative = data[0] == '-';
if (convert_accumulated_float(s, std::integral_constant<bool, has_native_float_format<number_float_t>::value> {}))
{
// the token text is only copied into token_buffer when
// get_string() asks for it; the library's own SAX handlers
// ignore it. A window into a stream may be refilled by the
// next read, so there the text is copied right away.
if (bulk_window)
{
token_buffer.append(reinterpret_cast<const typename string_t::value_type*>(data), len);
}
else
{
pending_number_text = data;
pending_number_length = len;
}
return token_type::value_float;
}
}
// materialize the token exactly as scan_number() would. reset() already
// cleared token_buffer, so append() fills it (assign() is avoided
// because custom string_t types need not provide it)
token_buffer.append(reinterpret_cast<const typename string_t::value_type*>(data), len);
return convert_number(number_type, mantissa_end); return convert_number(number_type, mantissa_end);
} }
@@ -1756,6 +1798,35 @@ scan_number_done:
/// the input is not handed back, so the terminator can stay unread /// the input is not handed back, so the terminator can stay unread
void read_number_terminator(std::false_type /*window*/) const noexcept {} void read_number_terminator(std::false_type /*window*/) const noexcept {}
/// binary32/binary64: the value of a float token from its accumulated
/// significand (see scan_number_bulk_contiguous())
bool convert_accumulated_float(const float_significand& s, std::true_type /*native*/) noexcept
{
value_float = decimal_to_float<number_float_t>(s);
return true;
}
/// other formats (e.g. x87 long double): not instantiated for them, as
/// decimal_to_float() only supports binary32/binary64
static bool convert_accumulated_float(const float_significand& /*s*/, std::false_type /*native*/) noexcept
{
return false;
}
/// the value of an integer token with at most 18 digits, accumulated in @a w
token_type store_integer(token_type number_type, std::uint64_t w) noexcept
{
if (number_type == token_type::value_unsigned)
{
value_unsigned = static_cast<number_unsigned_t>(w);
// reject values that do not round-trip into a narrower type
return (static_cast<std::uint64_t>(value_unsigned) == w) ? token_type::value_unsigned : token_type::uninitialized;
}
const std::int64_t x = -static_cast<std::int64_t>(w);
value_integer = static_cast<number_integer_t>(x);
return (static_cast<std::int64_t>(value_integer) == x) ? token_type::value_integer : token_type::uninitialized;
}
/// contiguous input: try the number fast path, else the byte-path scanner /// contiguous input: try the number fast path, else the byte-path scanner
token_type scan_number_dispatch(std::true_type /*bulk*/) token_type scan_number_dispatch(std::true_type /*bulk*/)
{ {
@@ -1798,6 +1869,7 @@ scan_number_done:
void reset() noexcept void reset() noexcept
{ {
token_buffer.clear(); token_buffer.clear();
pending_number_text = nullptr;
decimal_point_position = std::string::npos; decimal_point_position = std::string::npos;
#if JSON_DIAGNOSTIC_POSITIONS #if JSON_DIAGNOSTIC_POSITIONS
@@ -2057,10 +2129,30 @@ scan_number_done:
/// return current string value /// return current string value
string_t& get_string() string_t& get_string()
{ {
if (pending_number_text != nullptr)
{
token_buffer.append(reinterpret_cast<const typename string_t::value_type*>(pending_number_text), pending_number_length);
pending_number_text = nullptr;
}
// a number token holds '.' regardless of the locale (#4084) // a number token holds '.' regardless of the locale (#4084)
return token_buffer; return token_buffer;
} }
/// the value of a string token (key or string), which never has pending
/// number text, so get_string()'s check is not needed
string_t& get_string_token() noexcept
{
JSON_ASSERT(pending_number_text == nullptr);
return token_buffer;
}
/// the text of a float token for a SAX handler that ignores it (the
/// library's own handlers): empty, without copying the token
const string_t& get_ignored_number_text() const noexcept
{
return token_buffer;
}
///////////////////// /////////////////////
// diagnostics // diagnostics
///////////////////// /////////////////////
@@ -2456,6 +2548,11 @@ scan_number_done:
/// whether the next get() call should just return current /// whether the next get() call should just return current
bool next_unget = false; bool next_unget = false;
/// the text of the current float token in the input, not yet copied into
/// token_buffer (see get_string()), or nullptr
const char* pending_number_text = nullptr;
std::size_t pending_number_length = 0;
/// the start position of the current token /// the start position of the current token
position_t position {}; position_t position {};
+46 -6
View File
@@ -275,13 +275,53 @@ JSON_HEDLEY_ALWAYS_INLINE std::uint32_t parse_eight_digits(std::uint64_t v) noex
return static_cast<std::uint32_t>(((v & 0x0000FFFF0000FFFFu) * 42949672960001u) >> 32u); return static_cast<std::uint32_t>(((v & 0x0000FFFF0000FFFFu) * 42949672960001u) >> 32u);
} }
/// whether [first, last) contains a digit other than '0' /// whether the eight bytes in @a v (see read_eight_bytes()) are all ASCII
inline bool has_nonzero_digit(const char* first, const char* last) noexcept /// digits (after fast_float)
JSON_HEDLEY_ALWAYS_INLINE bool is_eight_digits(std::uint64_t v) noexcept
{ {
return std::any_of(first, last, [](char c) noexcept return ((v & 0xF0F0F0F0F0F0F0F0u) | (((v + 0x0606060606060606u) & 0xF0F0F0F0F0F0F0F0u) >> 4u)) == 0x3333333333333333u;
}
/*!
@brief skip the ASCII digits starting at data[i] and accumulate their value
Eight digits are taken at a time while at least eight bytes remain. @a w wraps
around after 19 digits; the caller only uses it for shorter digit runs.
@return the index of the first byte at or after @a i that is not a digit
*/
JSON_HEDLEY_ALWAYS_INLINE std::size_t accumulate_digits(const char* data, std::size_t i, std::size_t avail, std::uint64_t& w) noexcept
{
while (avail - i >= 8)
{ {
return c != '0'; const std::uint64_t v = read_eight_bytes(data + i);
}); if (!is_eight_digits(v))
{
break;
}
w = (w * 100000000u) + parse_eight_digits(v);
i += 8;
}
while (i < avail && data[i] >= '0' && data[i] <= '9')
{
w = (w * 10u) + static_cast<std::uint64_t>(data[i] - '0');
++i;
}
return i;
}
/// whether [first, last) contains a digit other than '0'; a plain loop, as
/// GCC calls std::any_of out of line even for the usual empty range
JSON_HEDLEY_ALWAYS_INLINE bool has_nonzero_digit(const char* first, const char* last) noexcept
{
for (; first != last; ++first)
{
if (*first != '0')
{
return true;
}
}
return false;
} }
/// the value of the validated exponent digits [+-]?[0-9]+ in [first, last), /// the value of the validated exponent digits [+-]?[0-9]+ in [first, last),
@@ -327,7 +367,7 @@ truncated is set.
std::string::npos if there is none std::string::npos if there is none
@param[in] mantissa_end index of the 'e'/'E', or the token length @param[in] mantissa_end index of the 'e'/'E', or the token length
*/ */
inline float_significand parse_float_significand(const char* first, const char* last, JSON_HEDLEY_ALWAYS_INLINE float_significand parse_float_significand(const char* first, const char* last,
std::size_t decimal_point_position, std::size_t mantissa_end) noexcept std::size_t decimal_point_position, std::size_t mantissa_end) noexcept
{ {
float_significand s; float_significand s;
+35 -4
View File
@@ -51,6 +51,20 @@ template<typename BasicJsonType>
using parser_callback_t = using parser_callback_t =
std::function<bool(int /*depth*/, parse_event_t /*event*/, BasicJsonType& /*parsed*/)>; std::function<bool(int /*depth*/, parse_event_t /*event*/, BasicJsonType& /*parsed*/)>;
/// whether a SAX handler ignores the text of float tokens (the library's own
/// handlers do), so that the parser need not materialize it
template<typename SAX>
struct sax_ignores_number_text : std::false_type {};
template<typename BasicJsonType, typename InputAdapterType>
struct sax_ignores_number_text<json_sax_dom_parser<BasicJsonType, InputAdapterType>> : std::true_type {};
template<typename BasicJsonType, typename InputAdapterType>
struct sax_ignores_number_text<json_sax_dom_callback_parser<BasicJsonType, InputAdapterType>> : std::true_type {};
template<typename BasicJsonType>
struct sax_ignores_number_text<json_sax_acceptor<BasicJsonType>> : std::true_type {};
/*! /*!
@brief syntax analysis @brief syntax analysis
@@ -171,6 +185,23 @@ class parser
} }
private: private:
/// the text of the current float token for @a sax (see sax_ignores_number_text)
template<typename SAX>
const string_t& number_float_text(SAX* /*sax*/)
{
return number_float_text(std::integral_constant<bool, sax_ignores_number_text<SAX>::value> {});
}
const string_t& number_float_text(std::true_type /*ignored*/) const noexcept
{
return m_lexer.get_ignored_number_text();
}
string_t& number_float_text(std::false_type /*ignored*/)
{
return m_lexer.get_string();
}
/*! /*!
@brief run a DOM SAX parser to completion and position the lexer @brief run a DOM SAX parser to completion and position the lexer
@@ -247,7 +278,7 @@ class parser
{ {
return syntax_error(*sax, exception_message(token_type::value_string, "object key")); return syntax_error(*sax, exception_message(token_type::value_string, "object key"));
} }
if (JSON_HEDLEY_UNLIKELY(!sax->key(m_lexer.get_string()))) if (JSON_HEDLEY_UNLIKELY(!sax->key(m_lexer.get_string_token())))
{ {
return false; return false;
} }
@@ -301,7 +332,7 @@ class parser
out_of_range::create(exception_id::number_overflow, concat("number overflow parsing '", m_lexer.get_token_string(), '\''), nullptr)); out_of_range::create(exception_id::number_overflow, concat("number overflow parsing '", m_lexer.get_token_string(), '\''), nullptr));
} }
if (JSON_HEDLEY_UNLIKELY(!sax->number_float(res, m_lexer.get_string()))) if (JSON_HEDLEY_UNLIKELY(!sax->number_float(res, number_float_text(sax))))
{ {
return false; return false;
} }
@@ -347,7 +378,7 @@ class parser
case token_type::value_string: case token_type::value_string:
{ {
if (JSON_HEDLEY_UNLIKELY(!sax->string(m_lexer.get_string()))) if (JSON_HEDLEY_UNLIKELY(!sax->string(m_lexer.get_string_token())))
{ {
return false; return false;
} }
@@ -456,7 +487,7 @@ class parser
return syntax_error(*sax, exception_message(token_type::value_string, "object key")); return syntax_error(*sax, exception_message(token_type::value_string, "object key"));
} }
if (JSON_HEDLEY_UNLIKELY(!sax->key(m_lexer.get_string()))) if (JSON_HEDLEY_UNLIKELY(!sax->key(m_lexer.get_string_token())))
{ {
return false; return false;
} }
@@ -680,12 +680,12 @@ class iter_impl // NOLINT(cppcoreguidelines-special-member-functions,hicpp-speci
{ {
JSON_ASSERT(m_object != nullptr); JSON_ASSERT(m_object != nullptr);
if (JSON_HEDLEY_UNLIKELY(!m_object->is_object())) if (JSON_HEDLEY_LIKELY(m_object->is_object()))
{ {
throw_literal<invalid_iterator>(exception_id::iterator_key_not_object, "cannot use key() for non-object iterators", m_object); return m_it.object_iterator->first;
} }
return m_it.object_iterator->first; throw_literal<invalid_iterator>(exception_id::iterator_key_not_object, "cannot use key() for non-object iterators", m_object);
} }
/*! /*!
+3 -4
View File
@@ -11,7 +11,6 @@
#include <algorithm> // all_of #include <algorithm> // all_of
#include <cctype> // isdigit #include <cctype> // isdigit
#include <cerrno> // errno, ERANGE #include <cerrno> // errno, ERANGE
#include <cstddef> // size_t
#include <cstdlib> // strtoull #include <cstdlib> // strtoull
#ifndef JSON_NO_IO #ifndef JSON_NO_IO
#include <iosfwd> // ostream #include <iosfwd> // ostream
@@ -335,7 +334,7 @@ class json_pointer
{ {
break; // not a plain number: let strtoull decide break; // not a plain number: let strtoull decide
} }
acc = static_cast<size_type>((acc * 10) + static_cast<size_type>(c - '0')); acc = static_cast<size_type>(acc * 10 + static_cast<size_type>(c - '0'));
if (JSON_HEDLEY_UNLIKELY(i == s.size() - 1)) if (JSON_HEDLEY_UNLIKELY(i == s.size() - 1))
{ {
idx = acc; idx = acc;
@@ -530,10 +529,10 @@ class json_pointer
static std::size_t hash_of(std::size_t parent, const string_t& token) noexcept static std::size_t hash_of(std::size_t parent, const string_t& token) noexcept
{ {
std::size_t h = parent * static_cast<std::size_t>(0x9E3779B97F4A7C15ULL & (std::numeric_limits<std::size_t>::max)()); std::size_t h = parent * static_cast<std::size_t>(0x9E3779B97F4A7C15ULL);
for (std::size_t i = 0; i < token.size(); ++i) for (std::size_t i = 0; i < token.size(); ++i)
{ {
h = (h ^ static_cast<unsigned char>(token[i])) * static_cast<std::size_t>(0x100000001B3ULL & (std::numeric_limits<std::size_t>::max)()); h = (h ^ static_cast<unsigned char>(token[i])) * static_cast<std::size_t>(0x100000001B3ULL);
} }
return h ^ (h >> 29u); return h ^ (h >> 29u);
} }
+29 -31
View File
@@ -29,7 +29,6 @@
#endif // JSON_NO_IO #endif // JSON_NO_IO
#include <iterator> // make_move_iterator, random_access_iterator_tag #include <iterator> // make_move_iterator, random_access_iterator_tag
#include <limits> // numeric_limits #include <limits> // numeric_limits
#include <map> // map
#include <memory> // unique_ptr #include <memory> // unique_ptr
#include <set> // swap, operator!= #include <set> // swap, operator!=
#include <stdexcept> // length_error #include <stdexcept> // length_error
@@ -755,20 +754,26 @@ private:
// one level deep); stops at the first structured child it meets // one level deep); stops at the first structured child it meets
static bool all_children_leaves(const basic_json& v) noexcept static bool all_children_leaves(const basic_json& v) noexcept
{ {
// array_t only needs forward iterators (see custom_array_type)
if (v.m_data.m_type == value_t::array) if (v.m_data.m_type == value_t::array)
{ {
const auto& a = *v.m_data.m_value.array; const auto& a = *v.m_data.m_value.array;
return std::all_of(a.begin(), a.end(), [](const basic_json & el) for (auto it = a.rbegin(); it != a.rend(); ++it)
{ {
return has_no_children(el); if (!has_no_children(*it))
}); {
return false;
} }
const auto& o = *v.m_data.m_value.object; }
return std::all_of(o.begin(), o.end(), [](const typename object_t::value_type & el) return true;
}
for (const auto& el : *v.m_data.m_value.object)
{ {
return has_no_children(el.second); if (!has_no_children(el.second))
}); {
return false;
}
}
return true;
} }
// The walk in destroy_container() may take the children of a // The walk in destroy_container() may take the children of a
@@ -1225,7 +1230,7 @@ public:
} }
template < typename O = object_t, detail::enable_if_t < detail::has_stable_elements<O>::value, int > = 0 > template < typename O = object_t, detail::enable_if_t < detail::has_stable_elements<O>::value, int > = 0 >
std::size_t object_capacity() const noexcept static constexpr std::size_t object_capacity() noexcept
{ {
return 0; return 0;
} }
@@ -1237,7 +1242,7 @@ public:
} }
template < typename O = object_t, detail::enable_if_t < !detail::has_stable_elements<O>::value && !detail::is_nlohmann_ordered_map<O>::value, int > = 0 > template < typename O = object_t, detail::enable_if_t < !detail::has_stable_elements<O>::value && !detail::is_nlohmann_ordered_map<O>::value, int > = 0 >
std::size_t object_capacity() const noexcept static constexpr std::size_t object_capacity() noexcept
{ {
return detail::unknown_size(); return detail::unknown_size();
} }
@@ -3038,21 +3043,18 @@ public:
JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE
static void throw_iterator_range_of_null(const basic_json* obj) static void throw_iterator_range_of_null(const basic_json* obj)
{ {
static_cast<void>(obj); // unused when JSON_NOEXCEPTION is defined
JSON_THROW(invalid_iterator::create(detail::exception_id::iterator_range_of_null, detail::concat("cannot construct with iterators from ", obj->type_name()), obj)); JSON_THROW(invalid_iterator::create(detail::exception_id::iterator_range_of_null, detail::concat("cannot construct with iterators from ", obj->type_name()), obj));
} }
JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE
static void throw_incompatible_reference_type(const basic_json& obj) static void throw_incompatible_reference_type(const basic_json& obj)
{ {
static_cast<void>(obj); // unused when JSON_NOEXCEPTION is defined
JSON_THROW(type_error::create(detail::exception_id::incompatible_reference_type, detail::concat("incompatible ReferenceType for get_ref, actual type is ", obj.type_name()), &obj)); JSON_THROW(type_error::create(detail::exception_id::incompatible_reference_type, detail::concat("incompatible ReferenceType for get_ref, actual type is ", obj.type_name()), &obj));
} }
JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE
static void throw_array_index_exceeds_size_type(size_type idx) static void throw_array_index_exceeds_size_type(size_type idx)
{ {
static_cast<void>(idx); // unused when JSON_NOEXCEPTION is defined
JSON_THROW(std::length_error(detail::concat("array index ", std::to_string(idx), " exceeds size_type"))); JSON_THROW(std::length_error(detail::concat("array index ", std::to_string(idx), " exceeds size_type")));
} }
@@ -3060,8 +3062,6 @@ public:
JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE
static void throw_array_index_out_of_range(size_type idx, const basic_json& j) static void throw_array_index_out_of_range(size_type idx, const basic_json& j)
{ {
static_cast<void>(idx); // unused when JSON_NOEXCEPTION is defined
static_cast<void>(j);
JSON_THROW(out_of_range::create(detail::exception_id::array_index_out_of_range, detail::concat("array index ", std::to_string(idx), " is out of range"), &j)); JSON_THROW(out_of_range::create(detail::exception_id::array_index_out_of_range, detail::concat("array index ", std::to_string(idx), " is out of range"), &j));
} }
@@ -3070,8 +3070,6 @@ public:
JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE
static void throw_key_not_found(const KeyType& key, const basic_json& j) static void throw_key_not_found(const KeyType& key, const basic_json& j)
{ {
static_cast<void>(key); // unused when JSON_NOEXCEPTION is defined
static_cast<void>(j);
JSON_THROW(out_of_range::create(detail::exception_id::key_not_found, detail::concat("key '", key, "' not found"), &j)); JSON_THROW(out_of_range::create(detail::exception_id::key_not_found, detail::concat("key '", key, "' not found"), &j));
} }
@@ -4895,11 +4893,8 @@ public:
iterator insert(const_iterator pos, const basic_json& val) // NOLINT(performance-unnecessary-value-param) iterator insert(const_iterator pos, const basic_json& val) // NOLINT(performance-unnecessary-value-param)
{ {
// insert only works for arrays // insert only works for arrays
if (JSON_HEDLEY_UNLIKELY(!is_array())) if (JSON_HEDLEY_LIKELY(is_array()))
{ {
detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
}
// check if iterator pos fits to this JSON value // check if iterator pos fits to this JSON value
if (JSON_HEDLEY_UNLIKELY(pos.m_object != this)) if (JSON_HEDLEY_UNLIKELY(pos.m_object != this))
{ {
@@ -4910,16 +4905,16 @@ public:
return insert_iterator(pos, val); return insert_iterator(pos, val);
} }
detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
}
/// @brief inserts element into array /// @brief inserts element into array
/// @sa https://json.nlohmann.me/api/basic_json/insert/ /// @sa https://json.nlohmann.me/api/basic_json/insert/
iterator insert(const_iterator pos, basic_json&& val) // NOLINT(performance-unnecessary-value-param) iterator insert(const_iterator pos, basic_json&& val) // NOLINT(performance-unnecessary-value-param)
{ {
// insert only works for arrays // insert only works for arrays
if (JSON_HEDLEY_UNLIKELY(!is_array())) if (JSON_HEDLEY_LIKELY(is_array()))
{ {
detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
}
// check if iterator pos fits to this JSON value // check if iterator pos fits to this JSON value
if (JSON_HEDLEY_UNLIKELY(pos.m_object != this)) if (JSON_HEDLEY_UNLIKELY(pos.m_object != this))
{ {
@@ -4932,16 +4927,16 @@ public:
return insert_iterator(pos, std::move(tmp)); return insert_iterator(pos, std::move(tmp));
} }
detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
}
/// @brief inserts copies of element into array /// @brief inserts copies of element into array
/// @sa https://json.nlohmann.me/api/basic_json/insert/ /// @sa https://json.nlohmann.me/api/basic_json/insert/
iterator insert(const_iterator pos, size_type cnt, const basic_json& val) // NOLINT(performance-unnecessary-value-param) iterator insert(const_iterator pos, size_type cnt, const basic_json& val) // NOLINT(performance-unnecessary-value-param)
{ {
// insert only works for arrays // insert only works for arrays
if (JSON_HEDLEY_UNLIKELY(!is_array())) if (JSON_HEDLEY_LIKELY(is_array()))
{ {
detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
}
// check if iterator pos fits to this JSON value // check if iterator pos fits to this JSON value
if (JSON_HEDLEY_UNLIKELY(pos.m_object != this)) if (JSON_HEDLEY_UNLIKELY(pos.m_object != this))
{ {
@@ -4952,6 +4947,9 @@ public:
return insert_iterator(pos, cnt, val); return insert_iterator(pos, cnt, val);
} }
detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
}
/// @brief inserts range of elements into array /// @brief inserts range of elements into array
/// @sa https://json.nlohmann.me/api/basic_json/insert/ /// @sa https://json.nlohmann.me/api/basic_json/insert/
iterator insert(const_iterator pos, const_iterator first, const_iterator last) // NOLINT(performance-unnecessary-value-param) iterator insert(const_iterator pos, const_iterator first, const_iterator last) // NOLINT(performance-unnecessary-value-param)
@@ -5455,7 +5453,7 @@ public:
// inline fast path for two values of the same non-owning scalar type; anything // inline fast path for two values of the same non-owning scalar type; anything
// else falls through to the out-of-line general comparison // else falls through to the out-of-line general comparison
#define JSON_IMPLEMENT_SCALAR_FAST(op, null_result) \ #define JSON_IMPLEMENT_SCALAR_FAST(op, null_result) \
if (lhs.m_data.m_type == rhs.m_data.m_type) /* NOLINT(readability/braces) */ \ if (lhs.m_data.m_type == rhs.m_data.m_type) \
{ \ { \
switch (lhs.m_data.m_type) \ switch (lhs.m_data.m_type) \
{ \ { \
-1
View File
@@ -23,7 +23,6 @@
#include <nlohmann/detail/abi_macros.hpp> #include <nlohmann/detail/abi_macros.hpp>
#include <nlohmann/detail/macro_scope.hpp> #include <nlohmann/detail/macro_scope.hpp>
#include <nlohmann/detail/meta/type_traits.hpp> #include <nlohmann/detail/meta/type_traits.hpp>
#include <nlohmann/thirdparty/hedley/hedley.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN NLOHMANN_JSON_NAMESPACE_BEGIN
+242 -109
View File
@@ -29,7 +29,6 @@
#endif // JSON_NO_IO #endif // JSON_NO_IO
#include <iterator> // make_move_iterator, random_access_iterator_tag #include <iterator> // make_move_iterator, random_access_iterator_tag
#include <limits> // numeric_limits #include <limits> // numeric_limits
#include <map> // map
#include <memory> // unique_ptr #include <memory> // unique_ptr
#include <set> // swap, operator!= #include <set> // swap, operator!=
#include <stdexcept> // length_error #include <stdexcept> // length_error
@@ -5803,7 +5802,8 @@ JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE void throw_patch_missing_member(c
static_cast<void>(string_type); static_cast<void>(string_type);
static_cast<void>(val); static_cast<void>(val);
// context-sensitive error message // context-sensitive error message
JSON_THROW(parse_error::create(exception_id::patch_invalid_operation, 0, concat((op == "op") ? std::string("operation") : concat("operation '", op, '\''), string_type ? " must have string member '" : " must have member '", member, "'"), &val)); const std::string error_msg = (op == "op") ? std::string("operation") : concat("operation '", op, '\'');
JSON_THROW(parse_error::create(exception_id::patch_invalid_operation, 0, concat(error_msg, string_type ? " must have string member '" : " must have member '", member, "'"), &val));
} }
template<typename StringType, typename BasicJsonType> template<typename StringType, typename BasicJsonType>
@@ -6563,7 +6563,7 @@ NLOHMANN_JSON_NAMESPACE_END
#include <optional> // optional #include <optional> // optional
#endif #endif
#include <iterator> // begin, end, make_move_iterator, move_iterator #include <iterator> // begin, end
#include <memory> // allocator_traits #include <memory> // allocator_traits
#include <string> // basic_string, char_traits #include <string> // basic_string, char_traits
#include <tuple> // tuple, get #include <tuple> // tuple, get
@@ -7212,32 +7212,6 @@ namespace detail
* leave j with the type of the new value, but the pointer to the destroyed old one. * leave j with the type of the new value, but the pointer to the destroyed old one.
*/ */
// a move iterator over the elements of an rvalue container, unless they are
// const (e.g., std::set): libstdc++ before GCC 5 defines the reference of a
// std::move_iterator as value_type&&, which cannot bind to a const element
template<typename Iterator>
std::move_iterator<Iterator> make_move_iterator_if_mutable_impl(const Iterator& it, std::true_type /*mutable*/)
{
return std::make_move_iterator(it);
}
template<typename Iterator>
Iterator make_move_iterator_if_mutable_impl(const Iterator& it, std::false_type /*mutable*/)
{
return it;
}
template<typename Iterator>
using is_mutable_iterator = std::integral_constant < bool,
!std::is_const<typename std::remove_reference<decltype(*std::declval<Iterator&>())>::type>::value >;
template<typename Iterator>
auto make_move_iterator_if_mutable(const Iterator& it)
-> decltype(make_move_iterator_if_mutable_impl(it, is_mutable_iterator<Iterator> {}))
{
return make_move_iterator_if_mutable_impl(it, is_mutable_iterator<Iterator> {});
}
template<value_t> struct external_constructor; template<value_t> struct external_constructor;
template<> template<>
@@ -7364,7 +7338,7 @@ struct external_constructor<value_t::array>
using std::begin; using std::begin;
using std::end; using std::end;
const typename BasicJsonType::json_value value(j.template create<typename BasicJsonType::array_t>(make_move_iterator_if_mutable(begin(arr)), make_move_iterator_if_mutable(end(arr)))); const typename BasicJsonType::json_value value(j.template create<typename BasicJsonType::array_t>(std::make_move_iterator(begin(arr)), std::make_move_iterator(end(arr))));
j.replace_value(value_t::array, value); j.replace_value(value_t::array, value);
} }
@@ -7425,7 +7399,7 @@ struct external_constructor<value_t::object>
using std::begin; using std::begin;
using std::end; using std::end;
const typename BasicJsonType::json_value value(j.template create<typename BasicJsonType::object_t>(make_move_iterator_if_mutable(begin(obj)), make_move_iterator_if_mutable(end(obj)))); const typename BasicJsonType::json_value value(j.template create<typename BasicJsonType::object_t>(std::make_move_iterator(begin(obj)), std::make_move_iterator(end(obj))));
j.replace_value(value_t::object, value); j.replace_value(value_t::object, value);
} }
@@ -7603,27 +7577,25 @@ using is_movable_container_rvalue = negation<disjunction<
std::is_const<T>, std::is_const<T>,
has_dedicated_to_json<T>, has_dedicated_to_json<T>,
is_basic_json<T>, is_basic_json<T>,
is_range_view_type<T>,
is_enum_keyed_map_type<T>, is_enum_keyed_map_type<T>,
std::is_same<T, std::vector<bool>>>>; std::is_same<T, std::vector<bool>>>>;
// rvalue array-like containers; object types are ruled out before views are // rvalue array-like containers
// checked, as std::ranges::view<T> for the object_t of an ordered_json would
// need that ordered_json to be complete (GCC 11, see #3312)
template<typename BasicJsonType, typename T> template<typename BasicJsonType, typename T>
using is_rvalue_array_to_json = conjunction<is_movable_container_rvalue<T>, using is_rvalue_array_to_json = conjunction<is_movable_container_rvalue<T>,
negation<is_compatible_object_type<BasicJsonType, T>>,
negation<is_range_view_type<T>>,
is_compatible_array_type<BasicJsonType, T>, is_compatible_array_type<BasicJsonType, T>,
negation<disjunction< negation<disjunction<
std::is_same<T, typename BasicJsonType::array_t>, std::is_same<T, typename BasicJsonType::array_t>,
std::is_same<T, typename BasicJsonType::binary_t>, std::is_same<T, typename BasicJsonType::binary_t>,
is_compatible_object_type<BasicJsonType, T>,
is_compatible_string_type<BasicJsonType, T>, is_compatible_string_type<BasicJsonType, T>,
is_compatible_binary_type<BasicJsonType, T>>>>; is_compatible_binary_type<BasicJsonType, T>>>>;
// rvalue containers: their elements are moved instead of copied // rvalue containers: their elements are moved instead of copied
template<typename BasicJsonType, typename CompatibleArrayType, template<typename BasicJsonType, typename CompatibleArrayType,
enable_if_t<is_rvalue_array_to_json<BasicJsonType, CompatibleArrayType>::value, int> = 0> enable_if_t<is_rvalue_array_to_json<BasicJsonType, CompatibleArrayType>::value, int> = 0>
inline void to_json(BasicJsonType& j, CompatibleArrayType && arr) // NOLINT(cppcoreguidelines-missing-std-forward) inline void to_json(BasicJsonType& j, CompatibleArrayType && arr)
{ {
external_constructor<value_t::array>::construct_moving(j, arr); external_constructor<value_t::array>::construct_moving(j, arr);
} }
@@ -7701,8 +7673,7 @@ inline void to_json(BasicJsonType& j, const EnumKeyedMap& map)
} }
#endif #endif
// rvalue object-like containers (views have no key_type and mapped_type, so // rvalue object-like containers
// they are no compatible object type)
template<typename BasicJsonType, typename T> template<typename BasicJsonType, typename T>
using is_rvalue_object_to_json = conjunction<is_movable_container_rvalue<T>, using is_rvalue_object_to_json = conjunction<is_movable_container_rvalue<T>,
negation<std::is_same<T, typename BasicJsonType::object_t>>, negation<std::is_same<T, typename BasicJsonType::object_t>>,
@@ -7710,7 +7681,7 @@ using is_rvalue_object_to_json = conjunction<is_movable_container_rvalue<T>,
template<typename BasicJsonType, typename CompatibleObjectType, template<typename BasicJsonType, typename CompatibleObjectType,
enable_if_t<is_rvalue_object_to_json<BasicJsonType, CompatibleObjectType>::value, int> = 0> enable_if_t<is_rvalue_object_to_json<BasicJsonType, CompatibleObjectType>::value, int> = 0>
inline void to_json(BasicJsonType& j, CompatibleObjectType && obj) // NOLINT(cppcoreguidelines-missing-std-forward) inline void to_json(BasicJsonType& j, CompatibleObjectType && obj)
{ {
external_constructor<value_t::object>::construct_moving(j, obj); external_constructor<value_t::object>::construct_moving(j, obj);
} }
@@ -8396,9 +8367,12 @@ class file_input_adapter
file_input_adapter& operator=(const file_input_adapter&) = delete; file_input_adapter& operator=(const file_input_adapter&) = delete;
file_input_adapter& operator=(file_input_adapter&&) = delete; file_input_adapter& operator=(file_input_adapter&&) = delete;
~file_input_adapter() ~file_input_adapter()
{
if (m_file != nullptr)
{ {
unlock(); unlock();
} }
}
std::char_traits<char>::int_type get_character() noexcept std::char_traits<char>::int_type get_character() noexcept
{ {
@@ -8435,11 +8409,6 @@ class file_input_adapter
void unlock() noexcept void unlock() noexcept
{ {
// a moved-from adapter has no file and holds no lock
if (m_file == nullptr)
{
return;
}
#if JSON_UNLOCKED_FILE_IO == 1 #if JSON_UNLOCKED_FILE_IO == 1
funlockfile(m_file); funlockfile(m_file);
#elif JSON_UNLOCKED_FILE_IO == 2 #elif JSON_UNLOCKED_FILE_IO == 2
@@ -10347,13 +10316,53 @@ JSON_HEDLEY_ALWAYS_INLINE std::uint32_t parse_eight_digits(std::uint64_t v) noex
return static_cast<std::uint32_t>(((v & 0x0000FFFF0000FFFFu) * 42949672960001u) >> 32u); return static_cast<std::uint32_t>(((v & 0x0000FFFF0000FFFFu) * 42949672960001u) >> 32u);
} }
/// whether [first, last) contains a digit other than '0' /// whether the eight bytes in @a v (see read_eight_bytes()) are all ASCII
inline bool has_nonzero_digit(const char* first, const char* last) noexcept /// digits (after fast_float)
JSON_HEDLEY_ALWAYS_INLINE bool is_eight_digits(std::uint64_t v) noexcept
{ {
return std::any_of(first, last, [](char c) noexcept return ((v & 0xF0F0F0F0F0F0F0F0u) | (((v + 0x0606060606060606u) & 0xF0F0F0F0F0F0F0F0u) >> 4u)) == 0x3333333333333333u;
}
/*!
@brief skip the ASCII digits starting at data[i] and accumulate their value
Eight digits are taken at a time while at least eight bytes remain. @a w wraps
around after 19 digits; the caller only uses it for shorter digit runs.
@return the index of the first byte at or after @a i that is not a digit
*/
JSON_HEDLEY_ALWAYS_INLINE std::size_t accumulate_digits(const char* data, std::size_t i, std::size_t avail, std::uint64_t& w) noexcept
{
while (avail - i >= 8)
{ {
return c != '0'; const std::uint64_t v = read_eight_bytes(data + i);
}); if (!is_eight_digits(v))
{
break;
}
w = (w * 100000000u) + parse_eight_digits(v);
i += 8;
}
while (i < avail && data[i] >= '0' && data[i] <= '9')
{
w = (w * 10u) + static_cast<std::uint64_t>(data[i] - '0');
++i;
}
return i;
}
/// whether [first, last) contains a digit other than '0'; a plain loop, as
/// GCC calls std::any_of out of line even for the usual empty range
JSON_HEDLEY_ALWAYS_INLINE bool has_nonzero_digit(const char* first, const char* last) noexcept
{
for (; first != last; ++first)
{
if (*first != '0')
{
return true;
}
}
return false;
} }
/// the value of the validated exponent digits [+-]?[0-9]+ in [first, last), /// the value of the validated exponent digits [+-]?[0-9]+ in [first, last),
@@ -10399,7 +10408,7 @@ truncated is set.
std::string::npos if there is none std::string::npos if there is none
@param[in] mantissa_end index of the 'e'/'E', or the token length @param[in] mantissa_end index of the 'e'/'E', or the token length
*/ */
inline float_significand parse_float_significand(const char* first, const char* last, JSON_HEDLEY_ALWAYS_INLINE float_significand parse_float_significand(const char* first, const char* last,
std::size_t decimal_point_position, std::size_t mantissa_end) noexcept std::size_t decimal_point_position, std::size_t mantissa_end) noexcept
{ {
float_significand s; float_significand s;
@@ -13102,9 +13111,12 @@ scan_number_done:
const std::size_t avail = rem + 1; const std::size_t avail = rem + 1;
// validate + classify the number extent (mirrors scan_number()'s grammar) // validate + classify the number extent (mirrors scan_number()'s grammar)
// and accumulate the mantissa digits on the way, so that tokens with up
// to 19 digits need no second pass for their value
std::size_t i = 0; std::size_t i = 0;
std::size_t dot_index = std::string::npos; std::size_t dot_index = std::string::npos;
token_type number_type = token_type::value_unsigned; token_type number_type = token_type::value_unsigned;
std::uint64_t w = 0;
if (data[0] == '-') if (data[0] == '-')
{ {
number_type = token_type::value_integer; number_type = token_type::value_integer;
@@ -13114,22 +13126,21 @@ scan_number_done:
return token_type::uninitialized; return token_type::uninitialized;
} }
} }
const std::size_t integer_begin = i;
if (data[i] == '0') if (data[i] == '0')
{ {
++i; ++i;
} }
else if (data[i] >= '1' && data[i] <= '9') else if (data[i] >= '1' && data[i] <= '9')
{ {
++i; i = accumulate_digits(data, i, avail, w);
while (i < avail && data[i] >= '0' && data[i] <= '9')
{
++i;
}
} }
else else
{ {
return token_type::uninitialized; return token_type::uninitialized;
} }
std::size_t digit_count = i - integer_begin;
std::size_t fraction_digits = 0;
if (i < avail && data[i] == '.') if (i < avail && data[i] == '.')
{ {
number_type = token_type::value_float; number_type = token_type::value_float;
@@ -13139,29 +13150,39 @@ scan_number_done:
{ {
return token_type::uninitialized; return token_type::uninitialized;
} }
while (i < avail && data[i] >= '0' && data[i] <= '9') const std::size_t fraction_begin = i;
{ i = accumulate_digits(data, i, avail, w);
++i; fraction_digits = i - fraction_begin;
} digit_count += fraction_digits;
} }
// the mantissa ends here, whether or not an exponent part follows // the mantissa ends here, whether or not an exponent part follows
const std::size_t mantissa_end = i; const std::size_t mantissa_end = i;
std::int64_t exponent = 0;
if (i < avail && (data[i] == 'e' || data[i] == 'E')) if (i < avail && (data[i] == 'e' || data[i] == 'E'))
{ {
number_type = token_type::value_float; number_type = token_type::value_float;
++i; ++i;
bool negative_exponent = false;
if (i < avail && (data[i] == '+' || data[i] == '-')) if (i < avail && (data[i] == '+' || data[i] == '-'))
{ {
negative_exponent = data[i] == '-';
++i; ++i;
} }
if (i >= avail || !(data[i] >= '0' && data[i] <= '9')) if (i >= avail || !(data[i] >= '0' && data[i] <= '9'))
{ {
return token_type::uninitialized; return token_type::uninitialized;
} }
// saturated far beyond every range, see parse_float_exponent()
constexpr std::int64_t saturation = 100000000000000000; // 10^17
while (i < avail && data[i] >= '0' && data[i] <= '9') while (i < avail && data[i] >= '0' && data[i] <= '9')
{ {
if (exponent < saturation)
{
exponent = (exponent * 10) + (data[i] - '0');
}
++i; ++i;
} }
exponent = negative_exponent ? -exponent : exponent;
} }
const std::size_t len = i; const std::size_t len = i;
if (number_may_continue(len == avail, std::integral_constant<bool, bulk_window> {})) if (number_may_continue(len == avail, std::integral_constant<bool, bulk_window> {}))
@@ -13182,7 +13203,10 @@ scan_number_done:
#if !JSON_DIAGNOSTIC_POSITIONS #if !JSON_DIAGNOSTIC_POSITIONS
if (number_type != token_type::value_float) if (number_type != token_type::value_float)
{ {
const token_type integer_result = convert_integer(number_type, data, data + len); // up to 18 digits always fit into std::uint64_t and std::int64_t
const token_type integer_result = (digit_count <= 18)
? store_integer(number_type, w)
: convert_integer(number_type, data, data + len);
if (JSON_HEDLEY_LIKELY(integer_result != token_type::uninitialized)) if (JSON_HEDLEY_LIKELY(integer_result != token_type::uninitialized))
{ {
bulk_consume(data + 1, len - 1); bulk_consume(data + 1, len - 1);
@@ -13196,15 +13220,42 @@ scan_number_done:
} }
#endif #endif
// materialize the token exactly as scan_number() would. reset() already
// cleared token_buffer, so append() fills it (assign() is avoided
// because custom string_t types need not provide it)
token_buffer.append(reinterpret_cast<const typename string_t::value_type*>(data), len);
decimal_point_position = dot_index; decimal_point_position = dot_index;
bulk_consume(data + 1, len - 1); bulk_consume(data + 1, len - 1);
read_number_terminator(std::integral_constant<bool, bulk_window> {}); read_number_terminator(std::integral_constant<bool, bulk_window> {});
// a float whose digits all fit into w converts from the accumulated
// value; longer ones (and other float formats) take the general path
if (number_type == token_type::value_float && digit_count <= 19)
{
float_significand s;
s.w = w;
s.exponent = exponent - static_cast<std::int64_t>(fraction_digits);
s.negative = data[0] == '-';
if (convert_accumulated_float(s, std::integral_constant<bool, has_native_float_format<number_float_t>::value> {}))
{
// the token text is only copied into token_buffer when
// get_string() asks for it; the library's own SAX handlers
// ignore it. A window into a stream may be refilled by the
// next read, so there the text is copied right away.
if (bulk_window)
{
token_buffer.append(reinterpret_cast<const typename string_t::value_type*>(data), len);
}
else
{
pending_number_text = data;
pending_number_length = len;
}
return token_type::value_float;
}
}
// materialize the token exactly as scan_number() would. reset() already
// cleared token_buffer, so append() fills it (assign() is avoided
// because custom string_t types need not provide it)
token_buffer.append(reinterpret_cast<const typename string_t::value_type*>(data), len);
return convert_number(number_type, mantissa_end); return convert_number(number_type, mantissa_end);
} }
@@ -13233,6 +13284,35 @@ scan_number_done:
/// the input is not handed back, so the terminator can stay unread /// the input is not handed back, so the terminator can stay unread
void read_number_terminator(std::false_type /*window*/) const noexcept {} void read_number_terminator(std::false_type /*window*/) const noexcept {}
/// binary32/binary64: the value of a float token from its accumulated
/// significand (see scan_number_bulk_contiguous())
bool convert_accumulated_float(const float_significand& s, std::true_type /*native*/) noexcept
{
value_float = decimal_to_float<number_float_t>(s);
return true;
}
/// other formats (e.g. x87 long double): not instantiated for them, as
/// decimal_to_float() only supports binary32/binary64
static bool convert_accumulated_float(const float_significand& /*s*/, std::false_type /*native*/) noexcept
{
return false;
}
/// the value of an integer token with at most 18 digits, accumulated in @a w
token_type store_integer(token_type number_type, std::uint64_t w) noexcept
{
if (number_type == token_type::value_unsigned)
{
value_unsigned = static_cast<number_unsigned_t>(w);
// reject values that do not round-trip into a narrower type
return (static_cast<std::uint64_t>(value_unsigned) == w) ? token_type::value_unsigned : token_type::uninitialized;
}
const std::int64_t x = -static_cast<std::int64_t>(w);
value_integer = static_cast<number_integer_t>(x);
return (static_cast<std::int64_t>(value_integer) == x) ? token_type::value_integer : token_type::uninitialized;
}
/// contiguous input: try the number fast path, else the byte-path scanner /// contiguous input: try the number fast path, else the byte-path scanner
token_type scan_number_dispatch(std::true_type /*bulk*/) token_type scan_number_dispatch(std::true_type /*bulk*/)
{ {
@@ -13275,6 +13355,7 @@ scan_number_done:
void reset() noexcept void reset() noexcept
{ {
token_buffer.clear(); token_buffer.clear();
pending_number_text = nullptr;
decimal_point_position = std::string::npos; decimal_point_position = std::string::npos;
#if JSON_DIAGNOSTIC_POSITIONS #if JSON_DIAGNOSTIC_POSITIONS
@@ -13534,10 +13615,30 @@ scan_number_done:
/// return current string value /// return current string value
string_t& get_string() string_t& get_string()
{ {
if (pending_number_text != nullptr)
{
token_buffer.append(reinterpret_cast<const typename string_t::value_type*>(pending_number_text), pending_number_length);
pending_number_text = nullptr;
}
// a number token holds '.' regardless of the locale (#4084) // a number token holds '.' regardless of the locale (#4084)
return token_buffer; return token_buffer;
} }
/// the value of a string token (key or string), which never has pending
/// number text, so get_string()'s check is not needed
string_t& get_string_token() noexcept
{
JSON_ASSERT(pending_number_text == nullptr);
return token_buffer;
}
/// the text of a float token for a SAX handler that ignores it (the
/// library's own handlers): empty, without copying the token
const string_t& get_ignored_number_text() const noexcept
{
return token_buffer;
}
///////////////////// /////////////////////
// diagnostics // diagnostics
///////////////////// /////////////////////
@@ -13933,6 +14034,11 @@ scan_number_done:
/// whether the next get() call should just return current /// whether the next get() call should just return current
bool next_unget = false; bool next_unget = false;
/// the text of the current float token in the input, not yet copied into
/// token_buffer (see get_string()), or nullptr
const char* pending_number_text = nullptr;
std::size_t pending_number_length = 0;
/// the start position of the current token /// the start position of the current token
position_t position {}; position_t position {};
@@ -19801,6 +19907,20 @@ template<typename BasicJsonType>
using parser_callback_t = using parser_callback_t =
std::function<bool(int /*depth*/, parse_event_t /*event*/, BasicJsonType& /*parsed*/)>; std::function<bool(int /*depth*/, parse_event_t /*event*/, BasicJsonType& /*parsed*/)>;
/// whether a SAX handler ignores the text of float tokens (the library's own
/// handlers do), so that the parser need not materialize it
template<typename SAX>
struct sax_ignores_number_text : std::false_type {};
template<typename BasicJsonType, typename InputAdapterType>
struct sax_ignores_number_text<json_sax_dom_parser<BasicJsonType, InputAdapterType>> : std::true_type {};
template<typename BasicJsonType, typename InputAdapterType>
struct sax_ignores_number_text<json_sax_dom_callback_parser<BasicJsonType, InputAdapterType>> : std::true_type {};
template<typename BasicJsonType>
struct sax_ignores_number_text<json_sax_acceptor<BasicJsonType>> : std::true_type {};
/*! /*!
@brief syntax analysis @brief syntax analysis
@@ -19921,6 +20041,23 @@ class parser
} }
private: private:
/// the text of the current float token for @a sax (see sax_ignores_number_text)
template<typename SAX>
const string_t& number_float_text(SAX* /*sax*/)
{
return number_float_text(std::integral_constant<bool, sax_ignores_number_text<SAX>::value> {});
}
const string_t& number_float_text(std::true_type /*ignored*/) const noexcept
{
return m_lexer.get_ignored_number_text();
}
string_t& number_float_text(std::false_type /*ignored*/)
{
return m_lexer.get_string();
}
/*! /*!
@brief run a DOM SAX parser to completion and position the lexer @brief run a DOM SAX parser to completion and position the lexer
@@ -19997,7 +20134,7 @@ class parser
{ {
return syntax_error(*sax, exception_message(token_type::value_string, "object key")); return syntax_error(*sax, exception_message(token_type::value_string, "object key"));
} }
if (JSON_HEDLEY_UNLIKELY(!sax->key(m_lexer.get_string()))) if (JSON_HEDLEY_UNLIKELY(!sax->key(m_lexer.get_string_token())))
{ {
return false; return false;
} }
@@ -20051,7 +20188,7 @@ class parser
out_of_range::create(exception_id::number_overflow, concat("number overflow parsing '", m_lexer.get_token_string(), '\''), nullptr)); out_of_range::create(exception_id::number_overflow, concat("number overflow parsing '", m_lexer.get_token_string(), '\''), nullptr));
} }
if (JSON_HEDLEY_UNLIKELY(!sax->number_float(res, m_lexer.get_string()))) if (JSON_HEDLEY_UNLIKELY(!sax->number_float(res, number_float_text(sax))))
{ {
return false; return false;
} }
@@ -20097,7 +20234,7 @@ class parser
case token_type::value_string: case token_type::value_string:
{ {
if (JSON_HEDLEY_UNLIKELY(!sax->string(m_lexer.get_string()))) if (JSON_HEDLEY_UNLIKELY(!sax->string(m_lexer.get_string_token())))
{ {
return false; return false;
} }
@@ -20206,7 +20343,7 @@ class parser
return syntax_error(*sax, exception_message(token_type::value_string, "object key")); return syntax_error(*sax, exception_message(token_type::value_string, "object key"));
} }
if (JSON_HEDLEY_UNLIKELY(!sax->key(m_lexer.get_string()))) if (JSON_HEDLEY_UNLIKELY(!sax->key(m_lexer.get_string_token())))
{ {
return false; return false;
} }
@@ -21183,12 +21320,12 @@ class iter_impl // NOLINT(cppcoreguidelines-special-member-functions,hicpp-speci
{ {
JSON_ASSERT(m_object != nullptr); JSON_ASSERT(m_object != nullptr);
if (JSON_HEDLEY_UNLIKELY(!m_object->is_object())) if (JSON_HEDLEY_LIKELY(m_object->is_object()))
{ {
throw_literal<invalid_iterator>(exception_id::iterator_key_not_object, "cannot use key() for non-object iterators", m_object); return m_it.object_iterator->first;
} }
return m_it.object_iterator->first; throw_literal<invalid_iterator>(exception_id::iterator_key_not_object, "cannot use key() for non-object iterators", m_object);
} }
/*! /*!
@@ -21415,7 +21552,6 @@ NLOHMANN_JSON_NAMESPACE_END
#include <algorithm> // all_of #include <algorithm> // all_of
#include <cctype> // isdigit #include <cctype> // isdigit
#include <cerrno> // errno, ERANGE #include <cerrno> // errno, ERANGE
#include <cstddef> // size_t
#include <cstdlib> // strtoull #include <cstdlib> // strtoull
#ifndef JSON_NO_IO #ifndef JSON_NO_IO
#include <iosfwd> // ostream #include <iosfwd> // ostream
@@ -21746,7 +21882,7 @@ class json_pointer
{ {
break; // not a plain number: let strtoull decide break; // not a plain number: let strtoull decide
} }
acc = static_cast<size_type>((acc * 10) + static_cast<size_type>(c - '0')); acc = static_cast<size_type>(acc * 10 + static_cast<size_type>(c - '0'));
if (JSON_HEDLEY_UNLIKELY(i == s.size() - 1)) if (JSON_HEDLEY_UNLIKELY(i == s.size() - 1))
{ {
idx = acc; idx = acc;
@@ -21941,10 +22077,10 @@ class json_pointer
static std::size_t hash_of(std::size_t parent, const string_t& token) noexcept static std::size_t hash_of(std::size_t parent, const string_t& token) noexcept
{ {
std::size_t h = parent * static_cast<std::size_t>(0x9E3779B97F4A7C15ULL & (std::numeric_limits<std::size_t>::max)()); std::size_t h = parent * static_cast<std::size_t>(0x9E3779B97F4A7C15ULL);
for (std::size_t i = 0; i < token.size(); ++i) for (std::size_t i = 0; i < token.size(); ++i)
{ {
h = (h ^ static_cast<unsigned char>(token[i])) * static_cast<std::size_t>(0x100000001B3ULL & (std::numeric_limits<std::size_t>::max)()); h = (h ^ static_cast<unsigned char>(token[i])) * static_cast<std::size_t>(0x100000001B3ULL);
} }
return h ^ (h >> 29u); return h ^ (h >> 29u);
} }
@@ -26923,7 +27059,7 @@ JSON_HEDLEY_ALWAYS_INLINE shortest_decimal to_shortest(std::uint64_t bits) noexc
const uint128_parts p = umul192_hi128(p10.high, p10.low, bin_sig << static_cast<unsigned>(shift)); const uint128_parts p = umul192_hi128(p10.high, p10.low, bin_sig << static_cast<unsigned>(shift));
std::uint64_t integral = p.high >> static_cast<unsigned>(extra_shift); std::uint64_t integral = p.high >> static_cast<unsigned>(extra_shift);
// (no casts in the shifts: MSVC 2015 warns C4554 on them despite the parentheses) // (no casts in the shifts: MSVC 2015 warns C4554 on them despite the parentheses)
constexpr auto fractional_low_shift = static_cast<unsigned>(extra_shift); constexpr unsigned fractional_low_shift = static_cast<unsigned>(extra_shift);
constexpr unsigned fractional_high_shift = 64u - fractional_low_shift; constexpr unsigned fractional_high_shift = 64u - fractional_low_shift;
const std::uint64_t fractional = (p.high << fractional_high_shift) | (p.low >> fractional_low_shift); const std::uint64_t fractional = (p.high << fractional_high_shift) | (p.low >> fractional_low_shift);
std::uint64_t digit = 0; std::uint64_t digit = 0;
@@ -30173,8 +30309,6 @@ NLOHMANN_JSON_NAMESPACE_END
// #include <nlohmann/detail/meta/type_traits.hpp> // #include <nlohmann/detail/meta/type_traits.hpp>
// #include <nlohmann/thirdparty/hedley/hedley.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN NLOHMANN_JSON_NAMESPACE_BEGIN
@@ -31259,20 +31393,26 @@ private:
// one level deep); stops at the first structured child it meets // one level deep); stops at the first structured child it meets
static bool all_children_leaves(const basic_json& v) noexcept static bool all_children_leaves(const basic_json& v) noexcept
{ {
// array_t only needs forward iterators (see custom_array_type)
if (v.m_data.m_type == value_t::array) if (v.m_data.m_type == value_t::array)
{ {
const auto& a = *v.m_data.m_value.array; const auto& a = *v.m_data.m_value.array;
return std::all_of(a.begin(), a.end(), [](const basic_json & el) for (auto it = a.rbegin(); it != a.rend(); ++it)
{ {
return has_no_children(el); if (!has_no_children(*it))
}); {
return false;
} }
const auto& o = *v.m_data.m_value.object; }
return std::all_of(o.begin(), o.end(), [](const typename object_t::value_type & el) return true;
}
for (const auto& el : *v.m_data.m_value.object)
{ {
return has_no_children(el.second); if (!has_no_children(el.second))
}); {
return false;
}
}
return true;
} }
// The walk in destroy_container() may take the children of a // The walk in destroy_container() may take the children of a
@@ -31729,7 +31869,7 @@ public:
} }
template < typename O = object_t, detail::enable_if_t < detail::has_stable_elements<O>::value, int > = 0 > template < typename O = object_t, detail::enable_if_t < detail::has_stable_elements<O>::value, int > = 0 >
std::size_t object_capacity() const noexcept static constexpr std::size_t object_capacity() noexcept
{ {
return 0; return 0;
} }
@@ -31741,7 +31881,7 @@ public:
} }
template < typename O = object_t, detail::enable_if_t < !detail::has_stable_elements<O>::value && !detail::is_nlohmann_ordered_map<O>::value, int > = 0 > template < typename O = object_t, detail::enable_if_t < !detail::has_stable_elements<O>::value && !detail::is_nlohmann_ordered_map<O>::value, int > = 0 >
std::size_t object_capacity() const noexcept static constexpr std::size_t object_capacity() noexcept
{ {
return detail::unknown_size(); return detail::unknown_size();
} }
@@ -33542,21 +33682,18 @@ public:
JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE
static void throw_iterator_range_of_null(const basic_json* obj) static void throw_iterator_range_of_null(const basic_json* obj)
{ {
static_cast<void>(obj); // unused when JSON_NOEXCEPTION is defined
JSON_THROW(invalid_iterator::create(detail::exception_id::iterator_range_of_null, detail::concat("cannot construct with iterators from ", obj->type_name()), obj)); JSON_THROW(invalid_iterator::create(detail::exception_id::iterator_range_of_null, detail::concat("cannot construct with iterators from ", obj->type_name()), obj));
} }
JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE
static void throw_incompatible_reference_type(const basic_json& obj) static void throw_incompatible_reference_type(const basic_json& obj)
{ {
static_cast<void>(obj); // unused when JSON_NOEXCEPTION is defined
JSON_THROW(type_error::create(detail::exception_id::incompatible_reference_type, detail::concat("incompatible ReferenceType for get_ref, actual type is ", obj.type_name()), &obj)); JSON_THROW(type_error::create(detail::exception_id::incompatible_reference_type, detail::concat("incompatible ReferenceType for get_ref, actual type is ", obj.type_name()), &obj));
} }
JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE
static void throw_array_index_exceeds_size_type(size_type idx) static void throw_array_index_exceeds_size_type(size_type idx)
{ {
static_cast<void>(idx); // unused when JSON_NOEXCEPTION is defined
JSON_THROW(std::length_error(detail::concat("array index ", std::to_string(idx), " exceeds size_type"))); JSON_THROW(std::length_error(detail::concat("array index ", std::to_string(idx), " exceeds size_type")));
} }
@@ -33564,8 +33701,6 @@ public:
JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE
static void throw_array_index_out_of_range(size_type idx, const basic_json& j) static void throw_array_index_out_of_range(size_type idx, const basic_json& j)
{ {
static_cast<void>(idx); // unused when JSON_NOEXCEPTION is defined
static_cast<void>(j);
JSON_THROW(out_of_range::create(detail::exception_id::array_index_out_of_range, detail::concat("array index ", std::to_string(idx), " is out of range"), &j)); JSON_THROW(out_of_range::create(detail::exception_id::array_index_out_of_range, detail::concat("array index ", std::to_string(idx), " is out of range"), &j));
} }
@@ -33574,8 +33709,6 @@ public:
JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE JSON_HEDLEY_NO_RETURN JSON_HEDLEY_NEVER_INLINE
static void throw_key_not_found(const KeyType& key, const basic_json& j) static void throw_key_not_found(const KeyType& key, const basic_json& j)
{ {
static_cast<void>(key); // unused when JSON_NOEXCEPTION is defined
static_cast<void>(j);
JSON_THROW(out_of_range::create(detail::exception_id::key_not_found, detail::concat("key '", key, "' not found"), &j)); JSON_THROW(out_of_range::create(detail::exception_id::key_not_found, detail::concat("key '", key, "' not found"), &j));
} }
@@ -35399,11 +35532,8 @@ public:
iterator insert(const_iterator pos, const basic_json& val) // NOLINT(performance-unnecessary-value-param) iterator insert(const_iterator pos, const basic_json& val) // NOLINT(performance-unnecessary-value-param)
{ {
// insert only works for arrays // insert only works for arrays
if (JSON_HEDLEY_UNLIKELY(!is_array())) if (JSON_HEDLEY_LIKELY(is_array()))
{ {
detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
}
// check if iterator pos fits to this JSON value // check if iterator pos fits to this JSON value
if (JSON_HEDLEY_UNLIKELY(pos.m_object != this)) if (JSON_HEDLEY_UNLIKELY(pos.m_object != this))
{ {
@@ -35414,16 +35544,16 @@ public:
return insert_iterator(pos, val); return insert_iterator(pos, val);
} }
detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
}
/// @brief inserts element into array /// @brief inserts element into array
/// @sa https://json.nlohmann.me/api/basic_json/insert/ /// @sa https://json.nlohmann.me/api/basic_json/insert/
iterator insert(const_iterator pos, basic_json&& val) // NOLINT(performance-unnecessary-value-param) iterator insert(const_iterator pos, basic_json&& val) // NOLINT(performance-unnecessary-value-param)
{ {
// insert only works for arrays // insert only works for arrays
if (JSON_HEDLEY_UNLIKELY(!is_array())) if (JSON_HEDLEY_LIKELY(is_array()))
{ {
detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
}
// check if iterator pos fits to this JSON value // check if iterator pos fits to this JSON value
if (JSON_HEDLEY_UNLIKELY(pos.m_object != this)) if (JSON_HEDLEY_UNLIKELY(pos.m_object != this))
{ {
@@ -35436,16 +35566,16 @@ public:
return insert_iterator(pos, std::move(tmp)); return insert_iterator(pos, std::move(tmp));
} }
detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
}
/// @brief inserts copies of element into array /// @brief inserts copies of element into array
/// @sa https://json.nlohmann.me/api/basic_json/insert/ /// @sa https://json.nlohmann.me/api/basic_json/insert/
iterator insert(const_iterator pos, size_type cnt, const basic_json& val) // NOLINT(performance-unnecessary-value-param) iterator insert(const_iterator pos, size_type cnt, const basic_json& val) // NOLINT(performance-unnecessary-value-param)
{ {
// insert only works for arrays // insert only works for arrays
if (JSON_HEDLEY_UNLIKELY(!is_array())) if (JSON_HEDLEY_LIKELY(is_array()))
{ {
detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
}
// check if iterator pos fits to this JSON value // check if iterator pos fits to this JSON value
if (JSON_HEDLEY_UNLIKELY(pos.m_object != this)) if (JSON_HEDLEY_UNLIKELY(pos.m_object != this))
{ {
@@ -35456,6 +35586,9 @@ public:
return insert_iterator(pos, cnt, val); return insert_iterator(pos, cnt, val);
} }
detail::throw_cannot_use_with(detail::exception_id::insert_wrong_type, "insert()", *this);
}
/// @brief inserts range of elements into array /// @brief inserts range of elements into array
/// @sa https://json.nlohmann.me/api/basic_json/insert/ /// @sa https://json.nlohmann.me/api/basic_json/insert/
iterator insert(const_iterator pos, const_iterator first, const_iterator last) // NOLINT(performance-unnecessary-value-param) iterator insert(const_iterator pos, const_iterator first, const_iterator last) // NOLINT(performance-unnecessary-value-param)
@@ -35959,7 +36092,7 @@ public:
// inline fast path for two values of the same non-owning scalar type; anything // inline fast path for two values of the same non-owning scalar type; anything
// else falls through to the out-of-line general comparison // else falls through to the out-of-line general comparison
#define JSON_IMPLEMENT_SCALAR_FAST(op, null_result) \ #define JSON_IMPLEMENT_SCALAR_FAST(op, null_result) \
if (lhs.m_data.m_type == rhs.m_data.m_type) /* NOLINT(readability/braces) */ \ if (lhs.m_data.m_type == rhs.m_data.m_type) \
{ \ { \
switch (lhs.m_data.m_type) \ switch (lhs.m_data.m_type) \
{ \ { \
+106
View File
@@ -15,9 +15,12 @@ using nlohmann::json;
#include <array> // array #include <array> // array
#include <cstdint> // uint32_t, uint64_t #include <cstdint> // uint32_t, uint64_t
#include <cstdio> // snprintf #include <cstdio> // snprintf
#include <cmath> // isfinite
#include <cstdlib> // strtod #include <cstdlib> // strtod
#include <cstring> // memcpy #include <cstring> // memcpy
#include <limits> // numeric_limits #include <limits> // numeric_limits
#include <initializer_list> // initializer_list
#include <list> // list
#include <map> // map #include <map> // map
#include <random> // mt19937 #include <random> // mt19937
#include <sstream> // stringstream #include <sstream> // stringstream
@@ -332,6 +335,109 @@ TEST_CASE("lexer number fast path")
} }
} }
SECTION("digits accumulated while scanning")
{
// The fast path accumulates the digits while it validates them, eight
// at a time where possible, and converts tokens with at most 19
// digits (floats) or 18 digits (integers) from that value. Every digit
// count from 1 to 25, in the integer and in the fraction part, with
// and without sign and exponent, must give the byte path's result. A
// std::list has no contiguous storage, so parsing it takes the byte
// path.
// type and value, or the error (values beyond the range of double
// are rejected with out_of_range.406 on both paths)
const auto outcome = [](const std::string & doc, bool byte_path) -> std::string
{
JSON_TRY
{
const std::list<char> chars(doc.begin(), doc.end());
const json j = byte_path ? json::parse(chars.begin(), chars.end()) : json::parse(doc);
return std::string(j[0].type_name()) + '|' + j.dump();
}
JSON_CATCH (const json::exception & e)
{
return e.what();
}
};
// the concatenation of @a parts
const auto join = [](std::initializer_list<std::string> parts)
{
std::string result;
for (const auto& part : parts)
{
result += part;
}
return result;
};
std::vector<std::string> numbers;
for (std::size_t digits = 1; digits <= 25; ++digits)
{
std::string run;
for (std::size_t i = 0; i < digits; ++i)
{
run += static_cast<char>('1' + static_cast<char>((i * 7) % 9));
}
const std::string nines(digits, '9');
for (const char* sign_text :
{"", "-"
})
{
const std::string sign = sign_text;
numbers.push_back(join({sign, run}));
numbers.push_back(join({sign, nines}));
numbers.push_back(join({sign, "1", std::string(digits - 1, '0')}));
numbers.push_back(join({sign, "0.", run}));
numbers.push_back(join({sign, "0.", std::string(digits, '0'), "1"}));
numbers.push_back(join({sign, run, ".", run}));
numbers.push_back(join({sign, run, "e-", std::to_string(digits)}));
numbers.push_back(join({sign, "1.", run, "E+", std::to_string(digits * 13)}));
}
}
// exponents beyond every range, and zeros
numbers.insert(numbers.end(), {"1e400", "1e-400", "-1e99999999999999999999", "1e-99999999999999999999",
"0e99999999999999999999", "0.0e-5", "-0.000", "123456789e-0000000000000000000000330"
});
for (const auto& n : numbers)
{
CAPTURE(n)
const std::string doc = "[" + n + "]";
const std::string a = outcome(doc, false);
CHECK(a == outcome(doc, true));
const double expected = std::strtod(n.c_str(), nullptr);
if (a.compare(0, 6, "number") == 0 && std::isfinite(expected) && n.find_first_of(".eE") != std::string::npos)
{
CHECK(json::parse(doc)[0].get<double>() == expected);
}
}
}
SECTION("integers that do not fit a narrower type become floats")
{
// up to 18 digits, the fast path stores the accumulated value without
// an overflow check; a narrower number_integer_t/number_unsigned_t
// must still see the value rejected and converted as a float
using json32 = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int32_t, std::uint32_t, float>;
for (const char* n :
{"4294967295", "4294967296", "2147483647", "-2147483648", "-2147483649", "99999999999", "-99999999999"
})
{
CAPTURE(n)
const std::string doc = std::string("[") + n + "]";
const std::list<char> chars(doc.begin(), doc.end());
const json32 a = json32::parse(doc);
const json32 b = json32::parse(chars.begin(), chars.end());
CHECK(a[0].type() == b[0].type());
CHECK(a == b);
}
CHECK(json32::parse("[4294967295]")[0].is_number_unsigned());
CHECK(json32::parse("[4294967296]")[0].is_number_float());
CHECK(json32::parse("[-2147483648]")[0].is_number_integer());
CHECK(json32::parse("[-2147483649]")[0].is_number_float());
}
SECTION("token type classification") SECTION("token type classification")
{ {
CHECK((scan_string("0") == json::lexer::token_type::value_unsigned)); CHECK((scan_string("0") == json::lexer::token_type::value_unsigned));
+75
View File
@@ -30,7 +30,9 @@ using nlohmann::json;
#include <valarray> #include <valarray>
#include <algorithm> #include <algorithm>
#include <cstdio> #include <cstdio>
#include <cstdint>
#include <fstream> #include <fstream>
#include <iterator>
#include <list> #include <list>
#include <sstream> #include <sstream>
#include <string> #include <string>
@@ -2450,6 +2452,79 @@ std::string parse_error_message_range(IteratorType first, IteratorType last)
} }
} // namespace } // namespace
TEST_CASE("text of float tokens")
{
// For contiguous input, the lexer copies the text of a float token into
// its buffer only when a SAX handler asks for it (the library's own
// handlers do not). A user handler must still see the exact text, for
// every input adapter and in every position of a document.
const std::string doc = R"([1.5, -0.0, 0.1, 1e5, -2.5E-4, 12345678901234567890.5, 0.12345678901234567890123,
{"a": 3.25, "b": [6.02e23, 1]}, "x", true, 7e-1])";
const std::vector<std::string> expected_floats =
{
"number_float(1.5)", "number_float(-0.0)", "number_float(0.1)", "number_float(1e5)", "number_float(-2.5E-4)",
"number_float(12345678901234567890.5)", "number_float(0.12345678901234567890123)", "number_float(3.25)",
"number_float(6.02e23)", "number_float(7e-1)"
};
const auto floats = [](const std::vector<std::string>& events)
{
std::vector<std::string> result;
std::copy_if(events.begin(), events.end(), std::back_inserter(result), [](const std::string & e)
{
return e.compare(0, 13, "number_float(") == 0;
});
return result;
};
SECTION("contiguous input")
{
SaxEventLogger l;
CHECK(json::sax_parse(doc, &l));
CHECK(floats(l.events) == expected_floats);
}
SECTION("byte input")
{
const std::vector<std::uint8_t> bytes(doc.begin(), doc.end());
SaxEventLogger l;
CHECK(json::sax_parse(bytes, &l));
CHECK(floats(l.events) == expected_floats);
}
SECTION("stream input")
{
std::istringstream is(doc);
SaxEventLogger l;
CHECK(json::sax_parse(is, &l));
CHECK(floats(l.events) == expected_floats);
}
SECTION("non-contiguous iterators")
{
const std::list<char> chars(doc.begin(), doc.end());
SaxEventLogger l;
CHECK(json::sax_parse(chars.begin(), chars.end(), &l));
CHECK(floats(l.events) == expected_floats);
}
SECTION("the library's own handlers do not need the text")
{
using lexer_adapter = nlohmann::detail::iterator_input_adapter<const char*>;
CHECK(nlohmann::detail::sax_ignores_number_text<nlohmann::detail::json_sax_dom_parser<json, lexer_adapter>>::value);
CHECK(nlohmann::detail::sax_ignores_number_text<nlohmann::detail::json_sax_dom_callback_parser<json, lexer_adapter>>::value);
CHECK(nlohmann::detail::sax_ignores_number_text<nlohmann::detail::json_sax_acceptor<json>>::value);
CHECK_FALSE(nlohmann::detail::sax_ignores_number_text<SaxEventLogger>::value);
// and the values are the same as when the text is read
const json j = json::parse(doc);
CHECK(j[0] == 1.5);
CHECK(j[5].get<double>() == 12345678901234567890.5);
CHECK(j[7]["b"][0] == 6.02e23);
CHECK(json::accept(doc));
}
}
TEST_CASE("last-read diagnostics are identical across input adapters") TEST_CASE("last-read diagnostics are identical across input adapters")
{ {
// The lexer reconstructs the "last read" token lazily for seekable adapters // The lexer reconstructs the "last read" token lazily for seekable adapters
+7 -7
View File
@@ -198,11 +198,11 @@ TEST_CASE("copy constructor and destructor across value types")
const json src_uint = 7u; const json src_uint = 7u;
const json src_float = 2.5; const json src_float = 2.5;
json c_null(src_null); // NOLINT(performance-unnecessary-copy-initialization): tests the copy constructor json c_null(src_null);
json c_bool(src_bool); // NOLINT(performance-unnecessary-copy-initialization): tests the copy constructor json c_bool(src_bool);
json c_int(src_int); // NOLINT(performance-unnecessary-copy-initialization): tests the copy constructor json c_int(src_int);
json c_uint(src_uint); // NOLINT(performance-unnecessary-copy-initialization): tests the copy constructor json c_uint(src_uint);
json c_float(src_float); // NOLINT(performance-unnecessary-copy-initialization): tests the copy constructor json c_float(src_float);
CHECK(c_null.is_null()); CHECK(c_null.is_null());
CHECK(c_bool == true); CHECK(c_bool == true);
@@ -300,12 +300,12 @@ TEST_CASE("copy constructor and destructor across value types")
REQUIRE(v.size() == 70); REQUIRE(v.size() == 70);
std::vector<json> copy(v); std::vector<json> copy(v);
CHECK((copy == v)); CHECK(copy == v);
// mutating the copy leaves the original alone // mutating the copy leaves the original alone
copy[5] = "changed"; copy[5] = "changed";
copy[6].push_back(1); copy[6].push_back(1);
CHECK((copy != v)); CHECK(copy != v);
CHECK(v[5] == std::string(5, 'x')); CHECK(v[5] == std::string(5, 'x'));
CHECK(v[6].size() == 3); CHECK(v[6].size() == 3);
+2 -2
View File
@@ -817,8 +817,8 @@ template<class Key, class T, class IgnoredLess = std::less<Key>,
struct relocating_map : nlohmann::ordered_map<Key, T, IgnoredLess, Allocator> struct relocating_map : nlohmann::ordered_map<Key, T, IgnoredLess, Allocator>
{ {
using base_type = nlohmann::ordered_map<Key, T, IgnoredLess, Allocator>; using base_type = nlohmann::ordered_map<Key, T, IgnoredLess, Allocator>;
using iterator = typename base_type::iterator; using typename base_type::iterator;
using Container = typename base_type::Container; using typename base_type::Container;
using base_type::base_type; using base_type::base_type;
template<typename... Args> template<typename... Args>
+4 -5
View File
@@ -1637,7 +1637,6 @@ TEST_CASE_TEMPLATE("reverse iteration and member access", Json, nlohmann::json,
{ {
Json j = {1, 2, 3, 4}; Json j = {1, 2, 3, 4};
const std::vector<int> reversed = {4, 3, 2, 1};
auto it = j.end(); auto it = j.end();
std::vector<int> seen; std::vector<int> seen;
while (it != j.begin()) while (it != j.begin())
@@ -1645,7 +1644,7 @@ TEST_CASE_TEMPLATE("reverse iteration and member access", Json, nlohmann::json,
--it; --it;
seen.push_back(it->template get<int>()); seen.push_back(it->template get<int>());
} }
CHECK(seen == reversed); CHECK(seen == std::vector<int> {4, 3, 2, 1});
auto it2 = j.end(); auto it2 = j.end();
auto old = it2--; auto old = it2--;
@@ -1657,14 +1656,14 @@ TEST_CASE_TEMPLATE("reverse iteration and member access", Json, nlohmann::json,
{ {
seen.push_back(r->template get<int>()); seen.push_back(r->template get<int>());
} }
CHECK(seen == reversed); CHECK(seen == std::vector<int> {4, 3, 2, 1});
seen.clear(); seen.clear();
for (auto r = j.crbegin(); r != j.crend(); ++r) for (auto r = j.crbegin(); r != j.crend(); ++r)
{ {
seen.push_back(r->template get<int>()); seen.push_back(*r);
} }
CHECK(seen == reversed); CHECK(seen == std::vector<int> {4, 3, 2, 1});
} }
SECTION("object: -- and rbegin/rend") SECTION("object: -- and rbegin/rend")
+2 -2
View File
@@ -1230,11 +1230,11 @@ TEST_CASE("JSON pointer array index tokens match strtoull semantics")
SECTION("random tokens") SECTION("random tokens")
{ {
std::mt19937 gen(42); // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed): reproducible std::mt19937 gen(42);
const std::string alphabet = "0123456789012345678901234567890123456789-+ ex"; const std::string alphabet = "0123456789012345678901234567890123456789-+ ex";
for (int i = 0; i < 5000; ++i) for (int i = 0; i < 5000; ++i)
{ {
std::string token(gen() % 23, '0'); std::string token(static_cast<std::size_t>(gen() % 23), '0');
for (auto& c : token) for (auto& c : token)
{ {
c = alphabet[gen() % alphabet.size()]; c = alphabet[gen() % alphabet.size()];
+2 -8
View File
@@ -1374,13 +1374,7 @@ void check_dump_against_reference(const std::string& s, const bool with_strict =
}; };
for (const auto error_handler : error_handlers) for (const auto error_handler : error_handlers)
{ {
#if defined(JSON_NOEXCEPTION)
// a strict dump of invalid UTF-8 calls std::abort() instead of throwing
static_cast<void>(with_strict);
if (error_handler == json::error_handler_t::strict)
#else
if (!with_strict && error_handler == json::error_handler_t::strict) if (!with_strict && error_handler == json::error_handler_t::strict)
#endif
{ {
continue; continue;
} }
@@ -1435,7 +1429,7 @@ TEST_CASE("invalid UTF-8 at every position is handled like the reference")
"\xF4\x90\x80\x80", "\xF4\x90\x80\x80",
// sequences cut short by ASCII, a control character, a quote, a // sequences cut short by ASCII, a control character, a quote, a
// backslash, DEL, or the lead byte of a well-formed character // backslash, DEL, or the lead byte of a well-formed character
"\xC3" "a", "\xE2\x82" "\n", "\xE2\x82\"", "\xF0\x9F\x98\\", "\xC3\x7F", "\xC3\xC3\xA4", "\xE2\xF0\x9F\x98\x80", // NOLINT(bugprone-suspicious-missing-comma): "\xC3" "a" ends the hex escape "\xC3" "a", "\xE2\x82" "\n", "\xE2\x82\"", "\xF0\x9F\x98\\", "\xC3\x7F", "\xC3\xC3\xA4", "\xE2\xF0\x9F\x98\x80",
// truncated sequences // truncated sequences
"\xC3", "\xE2\x82", "\xF0\x9F\x98" "\xC3", "\xE2\x82", "\xF0\x9F\x98"
}; };
@@ -1495,7 +1489,7 @@ TEST_CASE("invalid UTF-8 at every position is handled like the reference")
{ {
for (const auto& pattern : patterns) for (const auto& pattern : patterns)
{ {
check_dump_against_reference(make_string(filler, pos, pattern, length - pos), bytes <= (filler.size() == 1 ? 17u : 8u)); check_dump_against_reference(make_string(filler, pos, pattern, length - pos), bytes <= (filler.size() == 1 ? 17 : 8));
} }
} }
} }
-4
View File
@@ -218,11 +218,7 @@ TEST_CASE("compliance tests from nativejson-benchmark")
n1e308[0] = '['; n1e308[0] = '[';
n1e308[1] = '1'; n1e308[1] = '1';
n1e308[310] = ']'; n1e308[310] = ']';
#if JSON_STRICT_NUL_HANDLING
n1e308.pop_back(); // a NUL byte after the value is a syntax error in strict mode
#else
n1e308[311] = '\0'; n1e308[311] = '\0';
#endif
TEST_DOUBLE(n1e308, 1E308); TEST_DOUBLE(n1e308, 1E308);
} }
+7 -16
View File
@@ -54,8 +54,8 @@ TEST_CASE("exception messages of out-of-line throw sites")
SECTION("at") SECTION("at")
{ {
const json& carr = arr; const json carr = arr;
const json& cobj = obj; const json cobj = obj;
CHECK_MSG((void)arr.at(static_cast<json::size_type>(10)), "[json.exception.out_of_range.401] array index 10 is out of range"); CHECK_MSG((void)arr.at(static_cast<json::size_type>(10)), "[json.exception.out_of_range.401] array index 10 is out of range");
CHECK_MSG((void)carr.at(static_cast<json::size_type>(3)), "[json.exception.out_of_range.401] array index 3 is out of range"); CHECK_MSG((void)carr.at(static_cast<json::size_type>(3)), "[json.exception.out_of_range.401] array index 3 is out of range");
CHECK_MSG((void)num.at(static_cast<json::size_type>(0)), "[json.exception.type_error.304] cannot use at() with number"); CHECK_MSG((void)num.at(static_cast<json::size_type>(0)), "[json.exception.type_error.304] cannot use at() with number");
@@ -65,7 +65,7 @@ TEST_CASE("exception messages of out-of-line throw sites")
CHECK_MSG((void)num.at("a"), "[json.exception.type_error.304] cannot use at() with number"); CHECK_MSG((void)num.at("a"), "[json.exception.type_error.304] cannot use at() with number");
nlohmann::ordered_json oobj = {{"a", 1}}; nlohmann::ordered_json oobj = {{"a", 1}};
const nlohmann::ordered_json& coobj = oobj; const nlohmann::ordered_json coobj = oobj;
CHECK_MSG((void)oobj.at("z"), "[json.exception.out_of_range.403] key 'z' not found"); CHECK_MSG((void)oobj.at("z"), "[json.exception.out_of_range.403] key 'z' not found");
CHECK_MSG((void)coobj.at("z"), "[json.exception.out_of_range.403] key 'z' not found"); CHECK_MSG((void)coobj.at("z"), "[json.exception.out_of_range.403] key 'z' not found");
CHECK_MSG((void)coobj.at(std::string("zz")), "[json.exception.out_of_range.403] key 'zz' not found"); CHECK_MSG((void)coobj.at(std::string("zz")), "[json.exception.out_of_range.403] key 'zz' not found");
@@ -94,10 +94,9 @@ TEST_CASE("exception messages of out-of-line throw sites")
CHECK_MSG(obj.update(arr.begin(), arr2.end()), "[json.exception.invalid_iterator.210] iterators do not fit"); CHECK_MSG(obj.update(arr.begin(), arr2.end()), "[json.exception.invalid_iterator.210] iterators do not fit");
} }
#if !defined(JSON_NOEXCEPTION) // patch_msg() catches the exceptions, which JSON_NOEXCEPTION turns into std::abort()
SECTION("patch") SECTION("patch")
{ {
const auto patch_msg = [](const char* doc, const char* patch) -> std::string const auto patch_msg = [](const char* doc, const char* patch)
{ {
try try
{ {
@@ -106,23 +105,16 @@ TEST_CASE("exception messages of out-of-line throw sites")
} }
catch (const json::exception& e) catch (const json::exception& e)
{ {
// with JSON_DIAGNOSTICS, what() names the offending value like "(/0) ", // with JSON_DIAGNOSTICS, what() names the offending value like "(/0) "; drop it
// and with JSON_DIAGNOSTIC_POSITIONS its position like "(bytes 0-2) "; drop both
std::string msg = e.what(); std::string msg = e.what();
for (const char* marker : const auto pos = msg.find(" (/");
{
" (/", " (bytes "
})
{
const auto pos = msg.find(marker);
if (pos != std::string::npos) if (pos != std::string::npos)
{ {
msg.erase(pos + 1, msg.find(") ", pos) - pos + 1); msg.erase(pos + 1, msg.find(") ", pos) - pos + 1);
} }
}
return msg; return msg;
} }
return {}; return std::string();
}; };
CHECK(patch_msg("{}", "{}") == "[json.exception.parse_error.104] parse error: JSON patch must be an array of objects"); CHECK(patch_msg("{}", "{}") == "[json.exception.parse_error.104] parse error: JSON patch must be an array of objects");
@@ -140,5 +132,4 @@ TEST_CASE("exception messages of out-of-line throw sites")
CHECK(patch_msg("{\"a\":1}", "[{\"op\":\"test\",\"path\":\"/a\",\"value\":2}]") == "[json.exception.other_error.501] unsuccessful: {\"op\":\"test\",\"path\":\"/a\",\"value\":2}"); CHECK(patch_msg("{\"a\":1}", "[{\"op\":\"test\",\"path\":\"/a\",\"value\":2}]") == "[json.exception.other_error.501] unsuccessful: {\"op\":\"test\",\"path\":\"/a\",\"value\":2}");
CHECK(patch_msg("{\"a\":1}", "[{\"op\":\"move\",\"path\":\"/a\"}]") == "[json.exception.parse_error.105] parse error: operation 'move' must have member 'from'"); CHECK(patch_msg("{\"a\":1}", "[{\"op\":\"move\",\"path\":\"/a\"}]") == "[json.exception.parse_error.105] parse error: operation 'move' must have member 'from'");
} }
#endif
} }