mirror of
https://github.com/nlohmann/json.git
synced 2026-10-04 13:40:33 +00:00
73e9eae3c135e262dac3fe3e7978b46694e531d3
1015
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Niels Lohmann
|
73e9eae3c1 |
Add an error_handler parameter for UTF-8 to the binary readers and writers (#5746)
Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
f56b418c56 |
Follow each binary format's UTF-8 rule: strict writers (CBOR/UBJSON/BJData/BSON), lenient readers (#5741)
Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
 Niels LohmannandRaphael Grimm
|
40021f38fb |
Add basic_json::as_base_class and document name conflicts with custom base classes (#5589)
* Add basic_json::as_base_class and document name conflicts with custom base classes Members of basic_json hide members of a custom base class with the same name, and future releases may add members that hide ones accessible today. Document this in json_base_class_t and add as_base_class() to reach hidden members without spelling out the cast. Also make json_base_class_t a public member type. It was documented since 3.12.0, but declared private, so users could not name it. Supersedes #3899. Co-authored-by: Raphael Grimm <1005058+barcode@users.noreply.github.com> Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Add as_base_class to the docset search index New public members get an entry in docs/docset/docSet.sql (as done for to_bon8/from_bon8 in #2998). Without it, the Dash/Zeal docset built from the documentation cannot find basic_json::as_base_class. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Silence clang-tidy for the hidden type_name() in the base class test ci_clang_tidy failed with readability-convert-member-functions-to-static on base_class_with_hidden_members::type_name(). It must stay a non-static member: the test shows that it is hidden by the non-static basic_json::type_name() and reachable through as_base_class(). Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> Co-authored-by: Raphael Grimm <1005058+barcode@users.noreply.github.com> |
||
|
Niels Lohmann
|
c5650eaa3c |
Reject malformed UTF-16/UTF-32 units in wide-string input (#5704)
The wide-string input adapter (used for std::u16string, std::u32string, std::wstring, and iterators over 2- or 4-byte character types) passed some malformed code units on to the lexer as values that are neither a byte (0x00..0xFF) nor char_traits<char>::eof(). As a result: - A lone UTF-16 surrogate inside true/false/null was accepted if its low byte matched the expected letter, or ended the input silently if it was the last unit. - A high surrogate followed by a unit that is not its low surrogate swallowed that unit; if the swallowed unit was the newline ending a // comment, the comment silently extended over the next line. - Where wint_t is a signed int (macOS, the BSDs), a negative wchar_t collided with char_traits<char>::eof() (ending the input early) or was truncated to its low byte, depending on its value. The UTF-32 helper now converts the code unit to std::uint32_t before the range checks, so a negative unit reaches the same "emit 0xFF" branch already used for code points above U+10FFFF. The UTF-16 helper now peeks at the next unit before consuming it, and emits 0xFF instead of the raw surrogate when no valid pair is found, matching how ill-formed UTF-8 bytes are rejected elsewhere in the lexer. Fixes #5645. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
38a2db260c |
Remove dead metaprogramming and duplicated code in traits and pointers (#5728)
* Remove unused is_sax and is_detected_convertible detail::is_sax had no user: the parser and the binary reader only use is_sax_static_asserts, so is_sax was a second, unchecked copy of the SAX event list. is_sax_static_asserts asserted boolean(bool) twice in a row, and detail::is_detected_convertible was never used anywhere. Remove all three and include <cstddef> for size_t instead of <cstdint>. Only names in nlohmann::detail are removed; behavior, public API and ABI are unchanged. The diagnostics for an incomplete SAX handler are the same, apart from the duplicated boolean() message. Part of #5708 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Replace meta/logic.hpp with a disjunction trait meta/logic.hpp added a second set of type-level boolean helpers (cxpr_and, cxpr_or, cxpr_not, ...) next to the existing conjunction and negation in type_traits.hpp. It was used only by one static_assert in from_json_tuple_impl, two of its templates were never used, and it was the only header without the license banner and relied on transitive includes for <type_traits>. Add the missing disjunction next to conjunction and negation, use the three in the static_assert, and delete logic.hpp together with its BUILD.bazel entry. same_sign now uses disjunction as well, which resolves the 2022 TODO waiting for such a trait. The static_assert accepts and rejects the same types as before. Only names in nlohmann::detail change; behavior, public API and ABI are unchanged. Part of #5708 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Remove unused would_call_std_* from NLOHMANN_CAN_CALL_STD_FUNC_IMPL Besides detail::result_of_begin/end, which is_range and iterator_t use, the macro defined a namespace detail2 with a tag type, a catch-all overload and would_call_std_begin/end, plus would_call_std_begin/end structs directly in namespace nlohmann. Nothing has used them since they were added in #3020. Reduce the macro to its detail part. Without the trailing struct the ';' after the two invocations would be an empty declaration that -Wextra-semi flags, so drop it. macro_scope.hpp included meta/detected.hpp only for this macro; all users of detected.hpp include it (or type_traits.hpp) themselves, so remove the include. Behavior and ABI are unchanged. The undocumented, untested and unused names nlohmann::would_call_std_begin, nlohmann::would_call_std_end and namespace nlohmann::detail2 are no longer declared. Part of #5708 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Simplify is_ordered_map to reuse has_capacity is_ordered_map re-detected capacity() with a C++03 sizeof/vararg trick right after has_capacity did the same detection through is_detected. For ordered_map, the old trick took the address of std::vector::capacity, which [namespace.std]/6 makes unspecified. Reuse has_capacity instead, which removes the unspecified-behavior pointer-to-std-member and two NOLINT suppressions. Part of #5708 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Remove duplicate const overload of json_pointer::get_checked The const and non-const get_checked() overloads had byte-identical 50-line bodies, differing only in the signature. The remaining template deduces a const-qualified BasicJsonType for const callers, so at(), the out_of_range::create() calls and the bounds check all still work. Part of #5708 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix tautological clause in iter_impl's iterator category assertion The static_assert meant to check the LegacyBidirectionalIterator named requirement had a first clause comparing std::bidirectional_iterator_tag to itself, which is always true and checks nothing; only array_t::iterator was actually being checked, despite the message claiming object iterators were checked too. Drop the tautological clause, reword the message to describe what is actually checked, and note that object_t may use a forward-only iterator as long as reverse iteration and operator-- are unused. The check is intentionally not extended to object_t::iterator, since that would reject object types with forward-only iterators that compile and work correctly today. Part of #5708 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix misplaced and stale comments in JSON_HAS_RANGES and conversions The JSON_HAS_RANGES feature-detection block had its libc++ comment sitting above the clang+libstdc++ branch it does not describe, leaving the libc++ branch uncommented and the clang+libstdc++ branch without its own rationale. Move each comment to sit under its own branch, and give the clang+libstdc++ branch (added in issue 5161) its own one-line reason referencing that issue instead of reusing the libc++ branch's comment. Also fix a duplicated-word typo ("in large in large cpp files") in from_json.hpp, drop two unanswered 2017 design questions left as comments in type_traits.hpp and from_json.hpp that no longer reflect open questions, and correct NLOHMANN_JSON_SERIALIZE_ENUM_STRICT's @since tag from 3.12.0 to 3.13.0, the release it was actually introduced in. Part of #5708 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Support any-rank C arrays in from_json, not just rank 1-4 from_json() for C arrays had four hand-unrolled overloads (rank 1-4, added incrementally in #4262), each with its own nested loops. to_json() already handles any rank recursively, so a rank-5+ C array could be serialized but not read back with get_to()/get<>(). Replace the four overloads with one from_json() SFINAE-constrained on get<remove_all_extents<T>::type>() existing, forwarding to a pair of mutually recursive from_json_c_array_element() helpers: one assigns a non-array element via get<T>(), the other loops over a array element and recurses one dimension at a time. Each dimension still goes through at(), so type_error.304/out_of_range.401 stay unchanged; ranks 1-4 keep their existing behavior and semantics. Adds rank-5 round-trip and mismatched-shape tests to unit-conversions.cpp. Public API: additive only (rank 5+ C arrays become readable). Signed-off-by: Niels Lohmann <mail@nlohmann.me> #5708 item 1 * Move templated_json_throw into nlohmann::detail templated_json_throw() was defined in macro_scope.hpp, which is included outside NLOHMANN_JSON_NAMESPACE_BEGIN, so the helper leaked into the global namespace as ::templated_json_throw with no ABI tag. Unqualified lookup in NLOHMANN_JSON_SERIALIZE_ENUM_STRICT could then bind to a same-named function declared in the user's own namespace instead, which fails to compile with Clang ("does not name a template"). Move the helper next to the exception classes in exceptions.hpp, inside nlohmann::detail, and call it qualified as ::nlohmann::detail::templated_json_throw<...>(...) from both macro expansion sites. Rewrite the doc comment to give the real reason for the helper (JSON_THROW may expand to code that discards its argument, e.g. when exceptions are disabled) and fix the "supress" typo. templated_json_throw was never released (added by #5151 after v3.12.0), so it can be moved freely. Adds a regression test that expands NLOHMANN_JSON_SERIALIZE_ENUM_STRICT inside a namespace declaring its own templated_json_throw. Public API: no change (::templated_json_throw was an unreleased, unintentional global-namespace leak with no callers relying on its location). Overlaps #5698, which rewrites the same two macro call lines; the overlapping hunks are small and should be trivial to reconcile on rebase. Signed-off-by: Niels Lohmann <mail@nlohmann.me> #5708 item 2 * Factor the repeated JSON_HAS_RANGES/MinGW guard into one macro The std::ranges view conversion (excluded on MinGW because of its incomplete C++20 ranges support, #4916) was gated by the same #if JSON_HAS_RANGES && !defined(__MINGW32__) condition at seven independent sites in to_json.hpp and type_traits.hpp, with the MinGW rationale duplicated in two of them and missing from the rest. Since the sites come in matching pairs (one enables is_compatible_range_view and a view-based overload, the other adds the exclusion to the plain-array-type overload), a drift between any pair would produce an ambiguous or missing overload on exactly one platform. Add JSON_HAS_RANGE_VIEW_CONVERSION next to JSON_HAS_RANGES in macro_scope.hpp, combining both conditions with the #4916 reasoning in one place, #undef it in macro_unscope.hpp, and use it at all seven sites. This does not fold the MinGW check into JSON_HAS_RANGES itself: JSON_HAS_RANGES is user-overridable and also gates the enable_borrowed_range specialization in iteration_proxy.hpp, which is not excluded on MinGW. No behavior or public API change: JSON_HAS_RANGE_VIEW_CONVERSION expands to exactly the condition that was previously written out at each site. Overlaps #5585, #5600 and #3575, which touch the same to_json.hpp and type_traits.hpp lines; the change here is a mechanical search-and-replace of the guard condition and should rebase cleanly. Signed-off-by: Niels Lohmann <mail@nlohmann.me> #5708 item 11 * De-duplicate from_json.hpp's map and array-fallback bodies Several from_json() overload pairs in from_json.hpp were copies of each other, so a fix has to be applied twice (as #5681 already does): - from_json(..., std::map&) and from_json(..., std::unordered_map&) for non-string keys had identical 16-line bodies: array check, m.clear(), pair check loop, m.emplace(...). Route both through a new from_json_pair_array_to_map(j, m) helper. - The from_json_array_impl priority_tag<1> and priority_tag<0> fallbacks ran the same std::transform/std::inserter loop, differing only in ret.reserve(j.size()). Merge them into one body and, modeled on the existing from_json_object_reserve, add a from_json_array_reserve pair so the reserve() call is only made for ConstructibleArrayType that support it. Error ids (type_error.302), messages, diagnostic paths ((at(0)/at(1)) and behavior for types with/without reserve() are unchanged; only the duplication is removed. Public API: no change. Overlaps #5681, which changes the "&j" to "&p" line in both map bodies; the shared helper here should make that a one-line change instead of two on rebase. Signed-off-by: Niels Lohmann <mail@nlohmann.me> #5708 item 5 * Unify json_pointer's three array-index parsers array_index(), contains() and get_checked_or_null() each re-implemented the RFC 6901 array-index rules and the size_type range check: array_index() does the canonical parse and throws; contains() (which must not throw, #5395) re-validates every digit by hand and runs its own strtoull/ERANGE check before calling array_index() anyway, parsing every array token twice; get_checked_or_null() wraps array_index() in JSON_TRY/ JSON_INTERNAL_CATCH (detail::out_of_range&) to turn an unrepresentable index into "not found". Add a single private, noexcept parse_array_index(s, idx) returning an array_index_status (ok / leading_zero / not_a_number / unresolved / exceeds_size_type). array_index() becomes a thin wrapper mapping each status to the existing parse_error.106/109 or out_of_range.404/410; contains() and get_checked_or_null() switch on the status directly. This removes contains()'s digit-validation loop and its second strtoull call, and get_checked_or_null()'s JSON_TRY/JSON_INTERNAL_CATCH. Bugfix as a consequence: get_checked_or_null()'s JSON_TRY/ JSON_INTERNAL_CATCH was dead code under JSON_NOEXCEPTION (JSON_TRY expands to "if(true)" and the catch to "if(false)", so JSON_THROW's std::abort() ran unconditionally), meaning value() and contains() would abort instead of returning the default/false for an out-of-range-sized or oversized array index when exceptions are disabled (#5672). Switching on parse_array_index()'s return value instead of relying on an actual throw/catch fixes this: get_checked_or_null() now returns nullptr for array_index_status::unresolved/exceeds_size_type in every build configuration, and still calls JSON_THROW (aborting under JSON_NOEXCEPTION, as before) only for a malformed index (leading_zero/not_a_number), matching its documented @throw list. All existing error ids, messages and diagnostic paths are unchanged; a few reference tokens that used to fail contains()'s manual per-character validation (e.g. "1a") now fail via array_index_status::unresolved instead, with no observable difference since contains() only returns bool. Adds regression tests to unit-element_access2.cpp's "access on array type" section covering value() with an index that exceeds size_type and one with a trailing non-digit, both of which must yield the default value rather than abort/throw. Public API: no change. Overlaps #5700, #5614 and #5692, which touch the contains() and get_checked_or_null() array hunks; this change replaces those hunks with calls into the new shared parser, so a rebase will need to re-apply their token-handling changes (e.g. the empty-token case) on top of the switch statements here. Signed-off-by: Niels Lohmann <mail@nlohmann.me> #5708 item 4 * Regenerate single_include after merging develop The merge commit kept develop's single_include/nlohmann/json.hpp because make amalgamate saw it as up to date. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Address review: switch in array_index, drop redundant inline - json_pointer::array_index() dispatches on array_index_status with a switch, matching the other parse_array_index() caller - drop `inline` from the function templates this PR adds or moves in from_json.hpp - reword a comment that described the change rather than the code Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
1df1e8a845 |
Make cross-string-type basic_json conversion explicit without implicit conversions (#5591)
* Make cross-string-type basic_json conversion explicit without implicit conversions The converting constructor from another basic_json specialization was always implicit, so a value with a different string_t (std::wstring, a string with a custom allocator, ...) silently converted into a temporary, e.g. when passed to a function taking const nlohmann::json&. Such conversions do not produce correct values (#3425), and JSON_USE_IMPLICIT_CONVERSIONS=0 did not catch them. When JSON_USE_IMPLICIT_CONVERSIONS is 0, the constructor is now explicit if the string types differ. Specializations sharing a string type (json and ordered_json, different serializers or object maps) stay implicitly convertible, so the NLOHMANN_DEFINE_TYPE_* macros keep working with nested json members. get<BasicJsonType>() constructs explicitly and works in both modes. Fixes #2649. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Construct explicitly in get_to() and to_json(std::optional) With JSON_USE_IMPLICIT_CONVERSIONS=0 the conversion from a basic_json with a different string type is now explicit, but two library paths still assigned such a value implicitly and failed to compile inside the library: - get_to() with a basic_json target (the #2175 overload) did `v = *this`, so json(42).get_to(alt_json&) broke although get<alt_json>() works. - to_json(BasicJsonType&, const std::optional<T>&) is constrained on std::is_constructible (which accepts the explicit constructor) but did `j = *opt`, so converting a std::optional<alt_json> into a json broke. Both now construct the value explicitly, as get_impl() already does. Also replace static_cast<bool>(JSON_USE_IMPLICIT_CONVERSIONS) with a comparison: clang-tidy's modernize-use-bool-literals rejected the cast of the integer literal the macro expands to, failing ci_clang_tidy. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
a212d3b2e4 |
Preserve the object comparator's state in a deep copy past the nesting bound (#5722)
* Preserve the object comparator's state in a deep copy past the nesting bound copy_object_level(), used by the copy constructor and copy assignment once a value is nested deeper than the iterative deep copy's bound (128 levels, or every copy under JSON_NO_THREAD_LOCAL), built each object's copy with the object type's plain range constructor. That default-constructs the object's comparator instead of copying the original's. For an object type whose comparator carries state, such as a std::map that compares keys case-sensitively only when constructed that way, the copy then ordered - and could even deduplicate - its keys differently from the original. Add detail::is_comparator_constructible_object_type, a detection trait for object types that provide a key_comp() and a constructor taking a range and a comparator, the way std::map does. copy_object_level now dispatches on it: an object type that qualifies gets its copy built with src_object.key_comp() passed along; other object types, such as nlohmann::ordered_map (which has a key_compare for its std::map-like interface, but no key_comp()), keep using the plain range constructor exactly as before. merge_patch and update() were checked for the same pattern; neither is affected, since both only ever add members one at a time to an object that already has its own comparator (or start a brand new default-constructed one), rather than rebuilding an object_t from a range copied out of an existing, possibly custom-comparator object. Fixes #5649. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Keep astyle from padding the create_object_with_comparator templates Spell the negated condition as detail::negation<...> instead of a leading '!', which made astyle spread the template header out. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
1edf0ef041 |
Fix value(json_pointer, default) aborting under JSON_NOEXCEPTION (#5700)
With exceptions disabled (JSON_NOEXCEPTION or -fno-exceptions),
value(const json_pointer&, default) called std::abort() for array
reference tokens that array_index() rejects with out_of_range.404/410:
indices too large to fit size_type, the empty token ("/"), and tokens
like "/1a". With exceptions enabled, the same tokens correctly yielded
the default value, because get_checked_or_null() relied on
JSON_TRY/JSON_INTERNAL_CATCH (detail::out_of_range&) to turn the
exception into nullptr; under JSON_NOEXCEPTION, JSON_THROW aborts
before that catch is ever reached.
get_checked_or_null() now detects those out-of-range tokens itself,
the same way contains(json_pointer) already does (#5495), and only
calls array_index() for tokens that must still raise parse_error.106
or parse_error.109 (e.g. "/01", "/+1"), matching the documented
behavior of value().
Added regression tests to tests/src/unit-disabled_exceptions.cpp
(built with JSON_NOEXCEPTION and -fno-exceptions) and the matching
checks to tests/src/unit-element_access2.cpp for normal exception
mode.
Fixes #5672.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
|
||
|
Niels Lohmann
|
756b28c2b8 |
Keep a NUL byte ending a // comment as the end of input (#5696)
With the default NUL handling (JSON_STRICT_NUL_HANDLING not set), a NUL byte in the input is treated as the real end of input everywhere - except when it immediately ends a `//` comment: scan_comment() matched '\0' as a comment terminator like '\n', so the NUL was consumed as part of the comment and scan() never saw it as end of input; the next get() then kept reading past it. Multi-line comments and JSON_STRICT_NUL_HANDLING=1 were unaffected, since there the NUL is just part of the comment text. Fix scan_comment() to leave the NUL unconsumed (unget()) instead of returning it as part of the comment, so the following scan() reports it as end of input, exactly as for a NUL anywhere else. Fixes #5659. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
49cd427196 |
Copy-construct the base class of a deep copy's elements, not assign it (#5690)
The bounded-descent copy added by #5389 built the elements of a deep copy (nested past the 128-level bound) by default-constructing them and then having copy_metadata() assign their base class afterwards. That assignment is only instantiated for values nested past the bound, but being called from copy_structured() at all meant it was compiled for every copy, so a CustomBaseClass that is copy-constructible but not move-assignable (for example one with a const data member) no longer let its basic_json be copy-constructed, at any depth. copy_array_level() and copy_object_level() now build each element with a private-tag-selected constructor that copy-constructs the base class (and, under JSON_DIAGNOSTIC_POSITIONS, copies the positions) directly, the same way the copy constructor already builds elements within the 128-level bound. Copying a basic_json is therefore back to requiring only a copy-constructible base class, as documented and as it was before #5389; copy assignment is unchanged and still requires an assignable one. Fixes #5674. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
b730946432 |
Fix key types convertible to std::string_view breaking lookups (#5689)
Since #4958, a key type implicitly convertible to std::string_view was accepted by is_usable_as_basic_json_key_type without checking that the object's comparator can actually compare object_t::key_type with that key type. The key was then forwarded unchanged to the underlying map, so const operator[], at, find, count, contains, erase and value failed to compile (a hard error inside <map>) for a key convertible only to std::string_view, and value() rejected such keys outright. For keys convertible to both std::string and std::string_view, the KeyType&& templates now won overload resolution over the object_t::key_type overloads and then failed the same way, a regression from 3.12.0. Only the non-const operator[] worked, because it uses emplace(), which constructs a std::string from the key explicitly. ordered_json was not affected, since ordered_map checks comparability itself. Add a trait, is_string_view_convertible_key_type, that recognizes a key type that is convertible to std::string_view but not directly comparable with the object's key type, provided std::string_view itself is comparable with it. at(), operator[], find(), count(), contains(), erase() and value() now route such keys through a new lookup_key() helper that converts them to std::string_view before they reach the object, matching how the object's transparent comparator already supports std::string_view lookups. Fixes #5663. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
0d01d6ae90 |
Classify leaves with operator<=> itself past the nesting bound (#5686)
* Classify leaves with operator<=> itself past the nesting bound In C++20, an ordered comparison past the nesting bound classified a pair of leaves by asking == first and then order_leaves(), which calls < and > - both derived from <=>. For a pair of binary values with the same bytes but a different subtype, == reports them unequal, while <=> (through std::vector<std::uint8_t>::operator<=>) reports them equivalent, so the pair ended the comparison as unordered instead of letting the next element decide - unlike an array or object within the bound, which compares such a pair with its own operator<=> and gets equivalent. So operator<=>, and the <, <=, >, >= derived from it, could give a different result for the same two values depending on how deeply the values were nested, or unordered at every depth with JSON_NO_THREAD_LOCAL defined. compare_leaves() now classifies such a pair in C++20 with operator<=> itself instead, matching how a value within the bound is compared; the equality-only and pre-C++20 ordered cases are unchanged. Which of the three runs is chosen by overloading on std::integral_constant<bool, Ordered>, the same tag dispatch order_leaves() already uses, rather than a runtime "if (Ordered)" on a template parameter, which MSVC would flag as a constant condition (C4127). Added a regression test to unit-comparison.cpp that nests such a pair 0, 127, 128 and 200 levels deep (127 stays within the 128-level bound, 128 and 200 do not) and checks that operator<=> and operator< agree at every depth. Fixes #5654. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Drop the version history note for a bug that was never released The regression came from #5390, which is not in any release. Addresses review comment by @gregmarr. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
6f2048cd5d |
Accept lvalues in ordered_map::emplace's value parameter (#5685)
* Accept lvalues in ordered_map::emplace's value parameter
ordered_map::emplace(key, value) took the mapped value only by T&&, an
rvalue reference rather than a forwarding reference, so
ordered_json::emplace("a", value) failed to compile whenever value was
an lvalue or a const lvalue, even though the same call compiles for
json (whose object_t is std::map, with a variadic emplace). Turn the
value parameter into a separately-deduced forwarding reference,
constrained with std::is_constructible so the overloads still only
accept something convertible to the mapped type. std::map-compatible
semantics are unchanged: emplace still does nothing if the key already
exists.
Open PR #5609 also touches ordered_map.hpp (moving values on vector
growth); this change only touches the two emplace() overloads and
should not conflict.
Fixes #5673.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Avoid astyle's padding in ordered_map::emplace's template headers
Use detail::conjunction instead of && and drop the redundant V&& in detail::is_constructible, so astyle keeps the usual template formatting. Addresses review comment by @gregmarr.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
|
||
|
Niels Lohmann
|
df27cc3d4c |
Add scalar-on-left overloads for legacy discarded comparisons in C++20 (#5682)
With JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON=1 and C++20, a scalar on the left-hand side of <= or >= (e.g., `1 <= discarded`) yielded false instead of the documented true. The C++20 legacy block only had member operators, which are only candidates when the basic_json is the left operand; for a scalar on the left, overload resolution picked the candidate rewritten from operator<=>, which does not emulate the legacy behavior. The C++17 branch already has scalar-on-the-left friend overloads for <= and >=; add the equivalent pair to the C++20 legacy block. Added a regression test to tests/src/unit-comparison.cpp covering all four operand orders for both operators. Fixes #5665. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
d11e89471b |
Assert on missing array indices in const operator[] and document the JSON pointer case (#5606)
The const operator[] overloads are unchecked by design, and a missing key or index is undefined behavior. The key overload guards this with a runtime assertion, but the index overload did not, although the element access documentation says an assertion fires in both cases. The const JSON pointer overload inherits both through json_pointer::get_unchecked(), so a pointer to a missing array index read out of bounds even in debug builds, and its documentation promised out_of_range.404 for any pointer that cannot be resolved. Add JSON_ASSERT(idx < size()) to const operator[](size_type), which also covers the index leg of the const JSON pointer overload. Document the undefined behavior for the const JSON pointer overload in operator[].md and in the runtime assertions page. Release builds are unchanged. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
6d0867a85f |
Make comparisons with scalars noexcept only when the conversion is (#5751)
The comparison operators taking a scalar (==, !=, <, <=, >, >=, and C++20's <=>) convert the scalar to a basic_json and compare, but were unconditionally noexcept. When that conversion throws, the program called std::terminate instead of propagating the exception, e.g. when comparing a json with a string literal under memory pressure (std::bad_alloc) or with an enum value not mapped by NLOHMANN_JSON_SERIALIZE_ENUM_STRICT (out_of_range.410). clang-tidy 22.1 reports the latter as bugprone-exception-escape. Declare the 16 scalar overloads noexcept(std::is_nothrow_constructible<basic_json, ScalarType>::value): they stay noexcept for numbers, Booleans, nullptr, and plain enums, and are noexcept(false) for strings and enums whose to_json may throw. The comparisons of two basic_json values are unchanged. Restore the strict-enum comparisons removed from unit-conversions.cpp in the previous PR, check that comparing an unmapped strict enum now throws, and pin the new exception specifications in unit-noexcept.cpp. Document the exception safety of overload (2) on all seven operator pages. Ran make amalgamate. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
63c10a51fc |
Review and extend the documentation, and check it in CI (#5638)
* Review and extend the documentation, and check it in CI A review of all documentation pages found factual errors, dead links, missing cross-references, and gaps in examples. This fixes them and adds checks so the same problems are caught automatically. Fixes: - wrong signatures and version histories (operator!= C++20 member, binary() subtype type, get<PointerType>(), JSON_NO_THREAD_LOCAL, ...) - stale descriptions (number parsing since #5283, UBJSON table, SAX example that no longer compiled, tsl::ordered_map advice) - dead internal and external links; repology.org badges (the domain is suspended) replaced by badges that query the registries directly - deprecation notes link the migration guide; the guide itself fixed Additions: - "See also" sections, cross-references, 25 runnable examples, 12 Mermaid diagrams, new API pages for json_pointer::operator<=> and byte_container_with_subtype::operator==/!= - landing page, guides for untrusted input and performance - "unreleased" badge after versions newer than the latest release Checks: - strict documentation build (broken links/anchors fail it); CI and the publish workflow fetch the full history the build needs - weekly external link check, Mermaid syntax check in CI - check_structure.py: example titles, heading levels, alt texts, header links, docset index coverage; its unused-example check works again - all examples produce the same output on every platform Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Keep the customer links that could not be fixed A dead link on the customers page is still the evidence of where the use of the library was documented. Keep the original URLs of the entries without a working replacement (Marne, Cisco Webex Desk Camera, Philips Hue, CyberArk) and exclude exactly these URLs from the link check. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Correct the duplicate-key recipe's claim about SAX positions The SAX interface's key() receives no position either; only parse_error() does. Also note that the recipe does not report the path to the repeated key (see discussion #5085). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Say the library is available as a single header and mention json_fwd.hpp Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Correct documentation errors found while hunting for bugs - patch/patch_inplace: list the JSON pointer errors parse_error.106-109 and out_of_range.402/404, and quote the actual parse_error.105 message. - unflatten: list parse_error.106/107/108 and out_of_range.404. - to_bson: list out_of_range.415 (binary subtype above 255) and note that 412 and 415 are new in 3.13.0. - to_string: state that string_t must be convertible to std::string, also in the StringType requirements table. - JSON Lines: a `while (input >> j)` loop also throws after the last value for concatenated JSON values; show a loop that works for both. - BON8: a string gets 0xFF only if nothing follows it in the message; a string at the end of an array or object is ended by 0xFE. - custom_string_type.hpp: add operator+=(char), which the "Always required" list asks for (json_pointer::to_string, flatten, unflatten, and diff did not compile), and an ADL int_to_string for diff and items. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Cache the release headers with functools.lru_cache Codacy (Pylint) flagged the mutable default argument that header() used as its cache. functools.lru_cache keeps the same memoization without it. The script's output is unchanged. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
4d46bce4e1 |
Fix CI on develop (#5750)
* Fix CI configuration broken by recent merges and tool updates - gcc_flags.cmake: drop -Wexperimental-fmv-target, which GCC 16 accepts only on aarch64; amd64 rejects it, so every GCC job failed while checking the compiler. - ci_get_cmake: add VERBATIM so the checksum pipeline is passed to the shell intact (the unescaped `$'` broke the generated Makefile and build.ninja, failing ci_cmake_flags and ci_module_cpp20); match the SHA-256 entry case-insensitively, as CMake 3.5.0 lists the archive as "Linux-x86_64"; and unpack with --strip-components, as that archive's top-level directory is spelled "Linux" too. - ci_single_binaries: compile json.hpp's TU without IWYU's --error, as the comment above the gate already intends. - tests: restore -Wno-deprecated-declarations for all non-MSVC compilers (#5737 kept it for GCC only), as several tests call deprecated functions on purpose; include thirdparty/fifo_map as SYSTEM. - .clang-tidy: set misc-use-internal-linkage.AnalyzeTypes to false; clang-tidy 22.1 extended the check to classes and enums and flagged 100 test helper types. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix library warnings and a JSON_DIAGNOSTICS parent bug - binary_reader: pass integers to sax->number_integer() through conditional_static_cast<number_integer_t>, making the existing narrowing for a narrow number_integer_t explicit (MSVC C4244 and GCC -Wconversion/-Warith-conversion with the int16_t test from #5694); mark two Infer DEAD_STORE false positives with @infer-ignore. - to_json: set the parents of an array built from a C++20 range view after all elements are in place; a reallocating push_back moved the earlier elements and left their parent pointers stale, failing the JSON_DIAGNOSTICS invariant assertion. - json.hpp: suppress MSVC C4127 for the new is_ordered_map check in diff(), like the three existing ones; spell out std::formatter::parse's return and iterator types for clang-tidy 22.1. - number_parse: make the Eisel-Lemire digit counter unsigned (GCC -Wstrict-overflow). - string_utils: take encode_utf8's callable by const reference (cppcoreguidelines-missing-std-forward) and drop a \u from its doc comment (-Wdocumentation-unknown-command). - ordered_map: include <memory> for std::allocator (cpplint). Ran make amalgamate. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix tests failing in CI on develop - unit-allocator: skip the #5640 test under MSVC STL iterator debugging, where containers allocate a debug proxy in noexcept move constructors and a failing allocation terminates; move a decrement out of an if condition (bugprone-inc-dec-in-conditions). - unit-conversions: expect the "(/0)" path with JSON_DIAGNOSTICS; compare strict enums via get<>() rather than through the noexcept operator==(ScalarType, json), which bugprone-exception-escape flags. - unit-alt-string: suppress -Wexit-time-destructors for the strict enum macro and misc-use-internal-linkage for its enum. - unit-bjdata: call the static lookup functions through the type and pass unsigned char (-Wsign-conversion on amd64). - Mark Infer false positives with @infer-ignore in unit-diagnostics, unit-pointer_access, unit-udt, and unit-conversions. - Smaller clang-tidy 22.1 findings in unit-class_parser, unit-constructor2, unit-custom-base-class, unit-locale-cpp, and unit-noexcept. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
6aec1a085f |
Report BON8 input that ends after a UTF-8 lead byte as truncated (#5677)
A lead byte (0xC2..0xF7) inside a string begins either another character (if a continuation byte follows) or an integer (otherwise). When the input ended right after the lead byte, the reader took the missing byte as "not a continuation byte", ended the string before the lead byte, and treated the lead byte as the start of the next value. With strict=false, a message cut off there was therefore read as a shorter value: the 11 bytes of "😀😀é" cut after 9 bytes gave "😀😀", and ["aé"] cut after 3 of its 5 bytes gave ["a"]. With strict=true, the input was rejected with a misleading message ("expected end of input"), or, for a key, with parse_error.112 instead of 110. Either reading of the lead byte leaves the message incomplete: a string at the end of a message must be terminated by 0xFF, so the lead byte cannot belong to a following message. Report parse_error.110 (unexpected end of input) for strings and keys, as the comment on get_bon8_string() already requires and as the reference decoder (HikoGUI) does. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
e400780533 |
Re-amalgamate single_include (#5745)
* Re-amalgamate single_include #5737 changed 13 headers under include/ but merged without the matching single_include/nlohmann/json.hpp update, so the amalgamated header still had, among others, the GCC C++20 -Wignored-attributes pragma block and the clang -Wdocumentation push/pop that #5737 removed, the forwarding from_json tuple/array helpers it replaced with const references, and lacked the output_adapter char_traits changes it added. Regenerated with `make amalgamate` (astyle 3.4.13). The diff is exactly `git diff |
||
|
Niels Lohmann
|
cff0a61369 |
Share DOM SAX position handling; fix stale parser and lexer comments (#5731)
* Share the diagnostic-position setter of the DOM SAX parsers json_sax_dom_parser and json_sax_dom_callback_parser each had a private copy of handle_diagnostic_positions_for_json_value(), identical except for comments. Move the body into one static member function, detail::diagnostic_positions::set_from_lexer(value, lexer), which both classes call with their lexer pointer. basic_json befriends the new struct (only when JSON_DIAGNOSTIC_POSITIONS is enabled), as the position members are private. The discarded case is reached through the callback parser, so the LCOV_EXCL markers that only the dom parser's copy had are gone. The NOLINT on the unreachable default case loses the stray "-warnings-as-errors", which is not a check name. The start-position setup in start_object()/start_array() is left alone, as #5706 is editing the callback parser's versions. Behavior, the public API and the ABI are unchanged. Part of #5712 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Correct the parser comments on recursion and skip_to_state_evaluation The class documentation called the parser a recursive descent parser, but sax_parse_internal() is a loop that keeps the open containers on an explicit stack. The comment at the end of an array and of an object said the flag is set to false while the code below it sets it to true. Describe what the code does instead. Comments only; behavior, the public API and the ABI are unchanged. Part of #5712 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Update the discard_number_values comments to the current number path The comments explaining the accept() shortcut in convert_number() and the member documentation still argued in terms of strtoull()/strtoll() and errno, which #5283 replaced with convert_integer(), and pointed at scan_number() instead of convert_number(). They also did not say that scan_number_bulk_contiguous() converts integers itself, so the shortcut is only reached for input without bulk access, with JSON_DIAGNOSTIC_POSITIONS, or when the bulk scanner falls back. Rewrite both comments to describe the digit-count check in front of convert_integer(), keeping the 18-digit bound and the json_sax_acceptor argument. The stale <cstdlib> comment is left for after #5616, which edits that include block. Comments only; behavior, the public API and the ABI are unchanged. Part of #5712 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * List the UTF-8 validators instead of calling the DFA the only one The documentation of decode() called the Hoehrmann DFA the single source of truth for UTF-8 validation. It is used only by the serializer and by is_valid_utf8() (CBOR/MessagePack/BSON/UBJSON/BJData text strings). The lexer's scan_string() switch, validate_one_utf8() / valid_utf8_prefix() (bulk string scan, BON8 bulk path and BON8 writer) and the BON8 byte path in get_bon8_string() check the RFC 3629 ranges on their own. Replace the sentence with a list of the four validators, what each is used for, and a note that they must accept the same sequences. Sharing code between them was considered and dropped: it would save a few lines in a validator that is entangled with BON8 pushback, and #5677 is editing the BON8 byte path. Comments only; behavior, the public API and the ABI are unchanged. Part of #5712 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix stale doc comments and include lists in the input headers input_adapters.hpp included <memory> and <numeric> for the removed shared_ptr-based adapter design but used neither; it called (std::min) without including <algorithm>. json_sax.hpp used std::numeric_limits without including <limits>. Also corrected comments that no longer matched the code: input_stream_adapter does not skip the input's BOM (the lexer's skip_bom() does), the span_input_adapter comment named the no-longer-existing input_buffer_adapter type, lexer::get_string() does not reset the token, binary_reader's get_number() doc opened with /* instead of /*! (so Doxygen skipped it) and omitted BON8 from its endianness note, and the UBJSON-binary-types note did not mention that BJData 'B' arrays are read as binary. Left out: the lgtm suppression on lexer.hpp's scan_number() (in #5616's hunk) and the "-1 if unknown" wording in json_sax.hpp's start_object/start_array docs (in draft #5267's hunk), per the verdict's conflict list. Part of #5712 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Deduplicate the strict-EOF/release_lookahead/error block in parser::parse() json_sax_dom_callback_parser and json_sax_dom_parser branches of parser::parse() ran the same ~25 lines after sax_parse_internal(): the strict-mode EOF check (raising parse_error.101 through the SAX parser), release_lookahead() in non-strict mode, and mapping an errored SAX parser to a discarded result. The two copies had already drifted apart in formatting and in the second copy's "see above" comment. Add a private parse_dom(DomSax&, strict) member that runs this shared sequence once and returns whether the SAX parser did not error; both branches of parse() now only construct their DOM SAX parser, call parse_dom(), and (for the callback parser) map a discarded top-level value to null. sax_parse() is left untouched, since it only runs the EOF check and release_lookahead() when sax_parse_internal() succeeded, unlike parse(), which runs them unconditionally. Behavior-preserving: same operations in the same order for both SAX parser kinds. Verified with unit-class_parser (strict/non-strict, callback and non-callback), unit-deserialization and unit-disabled_exceptions (JSON_NOEXCEPTION), plus a clean make amalgamate / make check-amalgamation diff. Overlaps #5601, which touches the same lines. Signed-off-by: Niels Lohmann <mail@nlohmann.me> #5712 item 2 * Share the code point to UTF-8 encoding between the wide-string helpers and the lexer The 1/2/3/4-byte UTF-8 encoding ladder was written out by hand three times: in wide_string_input_helper<..., 4>::fill_buffer() for a UTF-32 code point, in the UTF-16 helper for both a BMP code unit and a valid surrogate pair, and in the lexer's \uXXXX/\uXXXX\uYYYY handling. The copies had drifted: the UTF-32 helper masked the leading bits of each byte (& 0x1Fu, & 0x0Fu, & 0x07u) where the others relied on the shift alone, even though both give the same result for a code point that is already known to be in range. Add detail::encode_utf8(cp, out) in string_utils.hpp, a single encoder that invokes a callable once per output byte, most significant byte first. Use it in the three valid-code-point branches (UTF-32 code points up to U+10FFFF, UTF-16 code units outside the surrogate range, and valid UTF-16 surrogate pairs) and in the lexer's \u handling, where out forwards to add(). The UTF-16 helper's deliberate pass-through of malformed surrogate units and the UTF-32 helper's 0xFF sentinel for code points above U+10FFFF are untouched, since neither reaches the new helper. Behavior-preserving: same bytes in the same order for every valid code point, verified with unit-class_lexer, unit-class_parser, unit-deserialization, unit-wstring and the non-test-data parts of unit-unicode1..5 (ASan/UBSan, C++11/17/20), and an escape-heavy parse microbenchmark that shows no change (about 73 ms either way, median of 3, 1M escape sequences). single_include/ regenerated with make amalgamate; make check-amalgamation leaves a clean tree. Overlaps #5704, which rewrites the wide_string_input_helper specializations touched here. Signed-off-by: Niels Lohmann <mail@nlohmann.me> #5712 item 6 --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
3926fcaac3 |
Deduplicate serializer dump code; fix stale includes, docs, and lint (#5729)
* Share scalar serialization between dump_internal and dump_value dump_value()'s cases for string, binary, boolean, number_integer, number_unsigned, number_float, discarded and null were a byte-for-byte copy of dump_internal()'s (added together in #5285 for the iterative fallback path). Any future change to scalar output had to be made in both places, or the recursive and depth-limited paths would silently start producing different bytes. Extract the shared cases into a private dump_scalar() and have both dump_internal() and dump_value() call it. Output is unchanged: dump(), dump(4), dump(-1,' ',true) and the replace/ignore error_handler_t variants are byte-identical over the json_test_data corpus before and after, and dump() throughput on a scalar-heavy document is unaffected. Part of #5709 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Drop serializer.hpp's dependency on binary_writer.hpp The only use of binary_writer in serializer.hpp was binary_writer<BasicJsonType, char>::to_char_type() to write the U+FFFD replacement character's three bytes. With CharType=char this is an identity conversion, so the include of binary_writer.hpp (and transitively binary_reader.hpp) pulled in a large, unrelated header for a no-op call. Write the three bytes directly instead. serializer.hpp compiles standalone with -Wall -Wextra -Werror, with and without -funsigned-char, and unit-serialization's error_handler_t::replace cases (with and without ensure_ascii) still pass. Moving binary_writer's to_char_type/to_msgpack_length to its private section is left as an optional follow-up. Part of #5709 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix stale #include lines in the output headers output_adapters.hpp included <algorithm> and <iterator> for std::copy and std::back_inserter, which have not been used there since #3569 (2022). serializer.hpp included <algorithm> for std::reverse (also unused), <cmath> for labs/isnan/signbit (only std::isfinite is used) and <utility> for std::move (nothing from <utility> is used there), while using std::next without including <iterator> at all, relying on getting it transitively through output_adapters.hpp's own stale <iterator>. Drop the unused includes, add <iterator> for std::next, and correct the remaining include comments. Both headers still compile standalone with -Wall -Wextra -Werror. Part of #5709 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Remove JSON_HEDLEY_NON_NULL(2) from write_characters() overrides output_vector_adapter, output_stream_adapter and output_string_adapter declared their write_characters(const CharType*, std::size_t) override JSON_HEDLEY_NON_NULL(2), but binary_writer legitimately calls it with a null pointer and length 0 for an empty string or binary value; the type-erased call path only stayed silent under UBSan because the static callee at those call sites is the unattributed virtual base. A nonnull attribute on a definition lets GCC and Clang assume the parameter is non-null inside the function body even when the call is virtual, so this was latent undefined behavior, not just style. Drop the attribute from the three overrides and document the (nullptr, 0) contract on output_adapter_protocol::write_characters. unit-cbor, unit-msgpack, unit-bson and unit-bon8 (which all exercise empty binary/string payloads through the stream and vector/string adapters) pass under -fsanitize=address,undefined,nonnull-attribute. Part of #5709 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Update stale serializer doc comments to match the current implementation dump_internal()'s doc block still described the pre-#5285/#5449 implementation: an escape_string() function that does not exist (the function is dump_escaped), integer conversion "implicitly via operator<<" (dump_integer actually uses a digit-pair lookup table), and floating-point conversion via "%g" (IEEE-754 types go through to_chars, others through snprintf). dump_value()'s comment said elements are pushed for dump_internal to walk, but it is dump_iteratively() that walks the stack. dump_escaped(), dump_integer() and dump_float() each said they write "to output stream @a o", which has not been true since the writer moved to write_buffer. Doc-only change; no behavior, API or ABI impact. Part of #5709 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Merge duplicate byte-to-hex helper and drop stale '| 0' promotions serializer::hex_bytes() and binary_writer::hex_byte() had identical bodies. Keep one, detail::hex_byte() in string_utils.hpp, and use it from both. Also drop the `| 0` at the two serializer call sites (hex_bytes(byte | 0) and hex_bytes(s.back() | 0)): #3088 ( |
||
|
Niels Lohmann
|
18dd5663b0 |
Diff deeply nested values without recursing per nesting level (#5548)
* Diff deeply nested values without recursing per nesting level diff() descended into both values once per nesting level, and compared them with operator== on every level on the way, which recurses as well. Values nested deeply enough - 25,000 levels on an 8 MiB stack - exhausted the call stack and terminated the process, although parse() accepts them without complaint. On such a chain the per-level comparisons and path strings also made diff() quadratic in time and memory. Both the recursion and operator== only descend as far as the source is nested. So diff() first checks, recursing at most diff_depth_limit() (128) levels, whether the source is nested more deeply than that. If not - all but a vanishing minority of values - the recursive algorithm diffs it exactly as before, now as diff_recursively(). Otherwise diff_iteratively() walks the two values on an explicit stack, emitting the same operations in the same order. It does not compare arrays and objects with operator== up front (equal ones yield no operations anyway), keeps the path in one buffer instead of a new string per level, and hands every subtree that is not nested too deeply back to diff_recursively(), so equal parts are still skipped quickly. The check costs one pass over the source. On a 3,000-object document that is about 30% of diffing two equal values (which is just an operator== call), about 10% of diffing values that differ in a few places, and noise when arrays change length. Once operator== no longer recurses (#5390), the check can go. Tests check that the patch reproduces the target at every depth up to 300, for json and ordered_json, including reordered members. They also check the exact operation for a difference deep inside, and diff values nested 100,000 levels deep. Fixes #5393 for diff(). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Make diff_frame a member struct that declares its special members GCC's -Weffc++ (an error in CI) asks a class with pointer members, a user constructor and a non-trivial destructor to declare its copy constructor and copy assignment; diff_frame's vector and basic_json members make its destructor non-trivial. Declare all five as defaulted, which also satisfies clang-tidy's special-member-functions check. Leave their exception specifications implicit: GCC 4.8 rejects an explicit one that differs from the implicit one, as it does for flatten_task in #5517. The converting constructor cannot throw, and is now declared noexcept for GCC's -Wnoexcept, which flags the emplace_back() under C++26 otherwise. The struct also moves from diff_iteratively() into the class, like dump_frame in the serializer. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Use the shared recursion limit in diff() diff_depth_limit() is gone in favor of detail::recursion_depth_limit(). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Diff fewer nesting depths so the test does not time out under Valgrind Checking every depth up to 300 made test-json_patch exceed the 1500 s ctest timeout in ci_test_valgrind. Check the depths up to 16, those around the recursion limit of 128, and 300 instead. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Mark the diff frame's value-initialized members for clang-tidy The braces are kept for GCC's -Weffc++, as in json_sax.hpp. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Bound diff()'s descent with a depth count instead of scanning the source Now that operator== no longer recurses (#5390), diff() can keep its per-level equality shortcut all the way down. It diffs recursively for the first detail::recursion_depth_limit() levels, as merge_patch() does, and hands anything deeper to diff_iteratively(). The nesting_exceeds() scan, which cost about 30% on equal documents, is gone, and diff() is on par with develop again. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Note that the diff frame reference is invalidated by pop_back() too Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Keep diff()'s recursive levels small and its result elided diff_recursively built every patch operation in place from initializer lists. Unoptimized builds give each of those temporaries its own stack slot, so every level of the bounded descent cost kilobytes of stack (about 6 KB with clang -O0), and the 128 recursive levels overflowed the 1 MB stack of MSVC Debug in the "deeply nested values" test. The operations and the key comparison of two objects are now built by separate functions, which diff_iteratively shares, and both diff functions append to one result instead of returning a patch per level that the caller copies. With clang -O0, diffing values nested 300 levels deep now peaks at about 190 KB of stack instead of 880 KB. Since diff() now owns the only returned value, clang's -Wnrvo no longer reports the returns of diff_recursively, which alternated between the local patch and diff_iteratively's result. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Copy the diff frame's members instead of holding a reference to it The loop in diff_iteratively held a reference to the top frame, which enter() invalidates when it pushes and the end of the loop invalidates when it pops. Nothing used it afterwards, but a later change could. As in the other iterative walks, the members the loop reads are now copied out as constants and the ones it advances are changed through stack.back(). The frame as a whole is not copied: it holds the common keys and the "add" operations of an object. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
a32f61eb98 | Fix update() and merge_patch() when the argument is *this or one of its members (#5678) | ||
|
Niels Lohmann
|
1d675cdb46 |
Fix CI jobs that check less than they claim; move arm64 to GitHub (#5733)
* Fix the ci_cmake_flags wiring so every option is checked
The CMake 3.31.6 flag list referred to itself before it was defined,
so only JSON_BuildTests was checked with that version. The targets for
the CMake running the build ("_2") were created but never added to
ci_cmake_flags, and the three versions shared one build directory.
JSON_StrictNulHandling was not in the list at all.
Use the 3.5.0 list for 3.31.6, add JSON_StrictNulHandling, and create
one ci_cmake_flag_<flag> target per option for the running CMake with
its own build directory. Also use the function parameter in the
COMMENT, refresh the stale version comment, and let ci_clean remove
the downloaded cmake-<version> directories instead of the long-gone
cmake-3.5.0-Darwin64.
Part of #5715
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix ci_test_clang_libcxx_cxx* jobs silently building without warnings
CMake only seeds CMAKE_CXX_FLAGS from the CXXFLAGS environment variable
when the cache entry is unset, so the explicit -DCMAKE_CXX_FLAGS="-stdlib=libc++"
argument made it ignore CXXFLAGS="${CLANG_CXXFLAGS}" entirely. The six
ci_test_standards_clang (..., libcxx) jobs therefore compiled without
-Weverything/-Werror while their libstdc++ siblings did use them.
Pass -stdlib=libc++ through the same CXXFLAGS value instead of a separate
-D argument, and give the target its own build directory
(build_clang_libcxx_cxx${CXX_STANDARD}) so it no longer shares a CMake
cache with the libstdc++ variant. Suppress the resulting
-Wthread-safety-negative finding from libc++'s std::mutex annotations,
which fires on doctest's reporters in this translation unit only.
Part of #5715
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Remove the no-op AppVeyor with_win_header job
The with_win_header matrix entry patched Windows.h into
single_include/nlohmann/json.hpp before building, but JSON_MultipleHeaders
has defaulted to ON since #3532 (2022-06), so CMakeLists.txt points the
tests at include/ and the patched single header is never compiled. The
job has been a no-op VS2015 build since then.
Windows.h coverage already exists through tests/src/unit-windows_h.cpp
(#3631), which runs in every MSVC job. Delete the dead matrix entry and
its before_build steps, and cite unit-windows_h.cpp from the QA page.
Part of #5715
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Remove unused ci_oclint and ci_pvs_studio targets
No workflow invokes ci_oclint, ci_pvs_studio, or their tool discovery.
ci_oclint also had a side effect on every JSON_CI configure: it copied
the single header into src_single/all.cpp and added an add_executable()
for it without EXCLUDE_FROM_ALL, so a plain build compiled a 1.2 MB
translation unit that only that unused target consumed. ci_pvs_studio
duplicates the Makefile's pvs_studio target, which is kept.
Also drop the duplicate --check-level=exhaustive flag passed twice to
the same ci_cppcheck invocation.
Part of #5715
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Stop Dependabot from proposing astyle bumps
astyle is deliberately pinned at 3.4.13 because newer versions reformat
unrelated lines and this version defines the formatting that
check_amalgamation.yml enforces. Without an ignore rule, Dependabot
keeps opening PRs for every new astyle release (most recently #4580,
#4942, #5445, #5448), each of which fails the amalgamation check and
gets closed unmerged.
Part of #5715
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Move Linux arm64 CI from dead Cirrus CI to ubuntu-24.04-arm
Cirrus CI stopped reporting check runs on develop sometime after
|
||
|
Niels Lohmann
|
c261578431 |
Deduplicate binary reader/writer helpers and fix stale comments (#5730)
* Fix stale and missing comments in binary_writer The doc block of write_number() ended up above the byte_swap() helpers added in #5286, about 80 lines from the function. It was also a plain comment that Doxygen skips, said "write a number to output input", and left BON8 out of the big-endian formats. Move it back onto write_number() as a /*! block and fix the text. write_bson() documented "@pre j.type() == value_t::object", but it throws type_error.317 for every other type, and to_bson() relies on that. Document the exception instead. Explain why the CBOR binary subtype is always written with a 0xD8..0xDB head and never in the one-byte tag form: binary_reader with cbor_tag_handler_t::store only keeps those heads as a subtype, so switching to write_cbor_head() would break round trips for subtypes 0..23. Also fix the grammar of the to_char_type comment. Comments only; no change in behavior, API or ABI. Part of #5710 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Merge the duplicated UBJSON/BJData integer marker ladders write_number_with_ubjson_prefix() (unsigned and signed overloads) and ubjson_prefix() (number_integer and number_unsigned cases) each picked the UBJSON/BJData integer marker (i, U, I, u, l, m, L, M, H) with their own independent if/else ladder, and the values beyond 64 bits were handled by a second, tag-dispatched pair of ladders. An optimized container announces the marker of its first element via ubjson_prefix() and then writes every element through write_number_with_ubjson_prefix(), so the two had to be kept in lockstep by hand across four call sites. Replace all of that with one ubjson_integer_prefix() built on value_in_range_of<T>, and one write_ubjson_integer_payload() that writes the value (or, for 'H', the decimal digits) for a given marker. write_number_with_ubjson_prefix() and ubjson_prefix() keep their signatures and now just call these two helpers. Behavior, the public API and the ABI are unchanged. Verified with a new regression test covering scalars and $-optimized arrays/objects at every int8/uint8/int16/uint16/int32/uint32/int64/uint64 boundary for to_ubjson/to_bjdata (both use_size/use_type settings), and by diffing to_ubjson/to_bjdata output before and after over the json_test_data corpus (bit-identical). Part of #5710 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Remove dead get_char parameters in binary_reader The non-recursive rewrite of the binary readers (#5505, #5506, #5507) left parse_cbor_internal()'s and parse_ubjson_internal()'s get_char parameters dead: parse_cbor_internal() has one caller and it always passes true, and parse_ubjson_internal() has one caller and it always uses the true default. Both parameters, and the @param docs describing the "reuse the last character" mode they used to select, no longer correspond to anything. Drop both parameters, initialise fetch/prefix unconditionally, and update the two call sites in sax_parse(). parse_cbor_value()'s and get_ubjson_string()'s own get_char parameters are unrelated and are left alone; both still have a false caller. Also delete a stray `@return whether a valid MessagePack value was passed to the SAX parser` doxygen block that sits directly above parse_msgpack_value()'s real doc comment, a leftover of the same rewrite. Behavior, the public API and the ABI are unchanged; these are private members of detail::binary_reader. Verified by compiling with -Wunused-parameter and running unit-cbor, unit-ubjson, unit-bjdata and unit-msgpack (offline, against the stubbed test_data.hpp). Part of #5711 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Share the IEEE half-precision decoder between CBOR and BJData binary_reader had two ~45-line copies of the IEEE 754 half-precision decoder: CBOR's case 0xF9 and BJData's case 'h'. Once formatting is normalised, the two blocks were identical except for the byte order used to assemble the 16-bit half (CBOR is big endian, BJData is little endian). Any future change to half-float decoding had to be made and kept in sync in both places. Add one get_half_float(format, little_endian) helper that does the two get()/unexpect_eof() reads, assembles the half in the requested byte order, decodes it per RFC 8949 Appendix D, and calls sax->number_float. Both cases now just call it with their byte order; the BJData case keeps its bjdata-only guard. Behavior, the public API and the ABI are unchanged. Verified with a scratch probe comparing the old and new decoders bit-for-bit (NaN by isnan()) over all 65536 wire byte pairs, in both formats, and by running unit-cbor and unit-bjdata (offline, against the stubbed test_data.hpp). Part of #5711 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Deduplicate the MessagePack unsigned-integer writer ladder The number_integer (non-negative branch) and number_unsigned cases in write_msgpack() each held their own copy of the fixint/uint8/16/32/64 ladder, kept in lockstep only by a comment ("we used the code from the value_t::number_unsigned case here"). Both copies mixed union members: the signed copy compared number_unsigned but wrote number_integer, and vice versa. Extract write_msgpack_unsigned(std::uint64_t), mirroring how write_cbor_head() already avoids the same duplication for CBOR, and call it from both cases. Each case now reads only its own active union member. Output bytes are unchanged for the default 64-bit number types. #5710 item 3 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Unify float marker selection and fix the long double compile error Four formats picked between a float32 and float64 marker through four different helper styles: dummy-argument overloads for CBOR and MessagePack, an std::is_same template for BON8, and a runtime if-chain on input_format_t for write_compact_float(). With number_float_t set to long double, to_cbor, to_msgpack and to_ubjson failed inside the library with "call to 'get_cbor_float_prefix' is ambiguous", while to_bson kept working because write_bson_double() takes a plain double. Change write_compact_float() to take the two marker bytes directly (each of its three callers already knows them at compile time) instead of an input_format_t it only forwarded, and delete the now-unused get_cbor_float_prefix(), get_msgpack_float_prefix(), get_bon8_float_prefix() and get_compact_float_prefix() helpers. Turn the two get_ubjson_float_prefix() overloads into one template. Both write_compact_float() and get_ubjson_float_prefix() now report an unsupported number_float_t with a static_assert naming the requirement, rather than an ambiguous-overload error; the assert lives in the function body, not the class scope, so to_bson with long double is unaffected. Verified with a probe basic_json<..., long double>: to_bson still compiles and round-trips, while to_cbor/to_msgpack/to_ubjson now fail to compile with the new static_assert message. This changes the text of an existing compile error for users with an unsupported number_float_t (documented as a public-API-visible change in #5710). #5710 item 1 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Deduplicate the BJData ndarray writer's dtype dispatch and drop <map> write_bjdata_ndarray() built a 12-entry std::map<string_t, CharType> on every call just to translate the _ArrayType_ name to a dtype marker (the only reason binary_writer.hpp included <map>), then mapped dtype to C++ type twice more: once as a switch for the range-check pass and once as a separate if/else chain for the write pass, with nothing checking that the two agreed. The caller also ran three at() lookups, and the callee called value.at(key) about ten more times for the same three members. Replace the map with bjdata_ndarray_type_marker(), a plain string comparison chain (a C++11 constexpr function cannot contain a switch, so this mirrors binary_reader's own static table style). Replace the switch/if-chain pair with one write_bjdata_ndarray_elements() that switches on dtype once and calls a per-type helper - write_bjdata_ndarray_element<T>() for the eight integer dtypes and write_bjdata_ndarray_float_element() for 'd' - with a dry_run flag selecting the range check or the actual write, so the two passes can no longer disagree on the type. _ArrayType_, _ArraySize_ and _ArrayData_ are now looked up once into references, and the four header marker bytes ('[', '$', '#') are written through to_char_type() like the rest of the UBJSON/BJData writer. The 'd' (single-precision) rule is left exactly as before, since #5707 is expected to change it separately. Verified byte-for-byte identical output before/after for every dtype (including the Draft 2/Draft 3 'byte' fallback and the use_count/ use_type combinations) via a standalone probe, plus round-tripping through from_bjdata(). Overlaps #5707, which is expected to touch the 'd' dtype case, and #5518, which is expected to move the write_bjdata_ndarray() call site. #5710 item 4 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Assert that write_bson_document() consumes every calc_bson_sizes() entry calc_bson_sizes() and write_bson_document() are a hand-synchronized pair of passes over the same object/array tree, introduced by #5553: the size pass appends to nested_sizes in visiting order, and the write pass consumes the table by position with nested_sizes[next_size++]. Nothing checked that the write pass consumed the whole table. If a future change touched only one of the two passes - for example to skip or reject an entry - every later size prefix in the document would be silently wrong. Add JSON_ASSERT(next_size == nested_sizes.size()) where write_bson_document() returns, so such a future drift between the two passes is caught immediately (JSON_ASSERT expands to nothing in release builds using assert(), and the fuzzers/tests already build with it enabled). The two passes agree today, so this changes nothing observable; it only guards against the risk described in #5710 item 5. Extracting a shared stepper for the two passes (the second half of the proposed change) is left for a follow-up: it only saves ~30 lines and the issue asks for it only if the result reads clearly, which needs more room to get right than a mechanical cleanup pass allows. #5710 item 5 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Make the BJData lookup tables static functions instead of members binary_reader held bjd_optimized_type_markers and bjd_types_map as non-static const members (12 string_t objects for the type-name table), built and destroyed on every from_cbor/from_msgpack/from_bson/ from_ubjson/from_bon8/from_bjdata call even though only from_bjdata ever reads them. They also needed the #define/decltype/#undef workaround from #3637 and two NOLINTNEXTLINE suppressions, and binary_writer already carries the same two lists in another form (is_bjdata_excluded_type_marker() and a local std::map in write_bjdata_ndarray(), the latter removed by the item-4 commit), so the excluded-marker lists could drift apart. Replace bjd_optimized_type_markers with static constexpr is_bjd_excluded_optimized_type(char_int_type), using the same ||-chain as binary_writer's is_bjdata_excluded_type_marker(). Replace bjd_types_map with a non-constexpr static bjd_type_name(char_int_type) switch returning nullptr for an unknown marker (a C++11 constexpr function cannot contain a switch). Delete both JSON_BINARY_READER_MAKE_* macros, the bjd_type pair alias, the NOLINTNEXTLINE suppressions, detail::make_array() (no longer used anywhere), and the now-unused <algorithm> and <array> includes. Update the two call sites (the ND-array excluded-type check and the _ArrayType_ lookup) accordingly, and replace unit-bjdata.cpp's "LUT arrays are sorted" section, which only checked the two tables' internal ordering, with a check of all 12 type names and all 8 excluded markers against both new functions. #5711 item 1 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Read CBOR's 1/2/4/8-byte argument through one helper parse_cbor_internal() hand-wrote the same "read a 1/2/4/8-byte big-endian unsigned integer" ladder four times over: - twice for tag numbers 0xD8-0xDB, once in the tag_handler::ignore branch and once, nearly identically, in the ::store branch (~90 lines to read one integer); - twice more for container lengths, once for array heads 0x98-0x9B and once for map heads 0xB8-0xBB, where the 1/2-byte forms called enter_array()/enter_object() directly and the 4/8-byte forms additionally went through get_cbor_container_size(). Add get_cbor_argument(std::uint64_t&), reading the width selected by current & 0x1F via the same get_number() calls as before (so EOF is reported exactly as before), and route all four sites through it: - 0xD8-0xDB now read the argument once per branch instead of switching on `current` a second time; behavior split cleanly from embedded tags 0xC0-0xD7 (tag value in the head, no argument to read), which is now its own case block that no longer has to fall into the ::store switch's "default" case to reach the same tag_pending = true; return true; outcome. - 0x98-0x9B and 0xB8-0xBB collapse into one case block each, always going through get_cbor_container_size() (harmless for 1/2-byte lengths, which already always fit). Verified byte-for-byte identical behavior before/after with a standalone probe covering embedded and multi-byte tags under all three tag_handler_t settings, a tag over a byte string (subtype path), truncated tag/length arguments of every width, and array/map lengths of every width, including the out_of_range.408 "excessive size" case: same exceptions, same messages, same chars_read, same successful results. Left the string/byte-string length ladders in get_cbor_string()/ get_cbor_binary() untouched, as noted in #5711 item 2, since #5325 is expected to touch them separately. Overlaps #5601 (adds a branch right above the embedded-tag case) and #5607 (touches the integer cases 0x18-0x1B, which share this ladder's shape in separate hunks). #5711 item 2 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Add leave_container() to match enter_container() Every container is opened through enter_container(), whose docs promise that a check placed there runs before every start event. The close side had no equivalent: the same "container_stack.pop_back(); dispatch to end_object() or end_array()" sequence was written out separately in BSON, CBOR, MessagePack, UBJSON/BJData and BON8, each copying the pattern of keeping an is_object flag around the pop_back() that would otherwise invalidate a reference to it. A check needed on close would have had to be added in five places, and a sixth copy could go unnoticed. Add leave_container() next to enter_container(), doing the same pop-then-dispatch, and replace the five sites with it. Each site keeps its own surrounding logic (BSON's check_bson_document_size() call before popping, MessagePack's is_object copy used again below, UBJSON/BJData's remaining-container handling after popping, BON8's top used again below); only the repeated pop/dispatch line pair is now shared. Verified all six binary-format unit suites and unit-regression2's deep-nesting tests (dependent count/reuse count and the bjdata ndarray depth cases) still pass, compiled with -Wall -Wextra and ASan/UBSan. Overlaps #5601, which is expected to add a sixth close site in its own skip loop; that site can route through leave_container() too once it lands. #5711 item 4 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Stop passing the input format to sax_parse() when the reader already has it binary_reader's constructor stores the format in the input_format member, and sax_parse(format, sax_, strict, tag_handler) took the same value again purely to dispatch on it. Every in-tree caller passed the same value both times (all 16 from_cbor/from_msgpack/from_ubjson/ from_bjdata/from_bon8/from_bson call sites in json.hpp, and the three public basic_json::sax_parse() overloads), so nothing was broken today, but a caller of the detail class directly (only reachable via JSON_PRIVATE_UNLESS_TESTED, as unit-bjdata.cpp already does) could pass a mismatched pair - say bjdata to the constructor and ubjson to sax_parse - and dispatch on one format while applying the other format's rules; the default-constructed input_format_t::json reader would additionally hit JSON_ASSERT(false) in exception_message() on its first error. Add sax_parse(json_sax_t*, bool, cbor_tag_handler_t) forwarding to the existing overload with the stored input_format, and switch every caller to it: the 16 from_*() sites (keeping their `// cppcheck-suppress[accessMoved]` comments) and the three basic_json::sax_parse() overloads, all of which already had the format available from their own `format` parameter. The four-argument overload is kept for anyone still calling it, now with JSON_ASSERT(format == input_format) so a mismatch fails immediately in a debug build (assert-enabled binaries, including the fuzzers and test suite) instead of misbehaving; verified with a probe that constructs a reader for one format and calls the explicit overload with another, which aborts on that assertion as expected. Removing or asserting against the constructor's input_format_t::json default, which would affect direct detail users, is left as a separate decision per #5711 item 5. Overlaps #5601, which is expected to add an AllowRecovery template parameter to sax_parse() and touch these same call sites in json.hpp. #5711 item 5 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Deduplicate UBJSON/BJData signed-count handling, drop dead ndarray checks get_ubjson_size_value()'s 'i'/'I'/'l'/'L' cases each read a differently sized signed integer and then repeated the same "reject negative with error 113" check; only 'L' additionally checked value_in_range_of for the out_of_range.408 case. Any change to that error path had to be made four times. Add get_ubjson_signed_count<SignedType>(std::size_t&), doing the read, the negative check and the range check once, and route all four markers through it. The range check is a no-op for 'i'/'I'/'l' (their values always fit std::size_t) and only live for 'L' on a 32-bit std::size_t target, matching today's behavior exactly. In the ndarray dimension-product loop, the preceding loop already returns early on any zero dimension and result starts at 1, so `i > 0` in the pre-multiplication overflow check was always true, and `result == 0` in the post-multiplication check could not be reached either: two positive factors whose product does not overflow (as the pre-check already guarantees) cannot be zero. Drop the dead `i > 0 &&` and narrow the post-check to `result == npos`, the one case the pre-check cannot rule out (an exact, non-overflowing match with the sentinel reserved for unknown-size containers), with a comment explaining why. Verified byte-for-byte identical behavior before/after with a standalone probe covering negative counts for every marker, a matching positive count, and ndarray inputs, plus the full unit-ubjson and unit-bjdata suites (same assertion counts as before this change). Overlaps #5601 (rewrites the four parse_error calls and the overflow checks touched here) and #5607/#5707 (touch neighboring lines in the same functions). #5711 item 6 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Drop redundant format parameter and dummy float argument (review) binary_reader::sax_parse(format, ...) only ever had to equal the format given to the constructor, which it asserted. With every caller already on the format-less overload, remove the four-argument overload and dispatch on the stored input_format directly. binary_reader is a detail class, so this is not a public API change. get_ubjson_float_prefix() took a value only to deduce its type; make the type an explicit template argument instead. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
35802e78d6 |
Remove dead Makefile targets; document macro_builder; tidy serve_header (#5735)
* Remove dead doctest help entry and pretty_format target from Makefile The top-level Makefile still carried three leftovers: - The help text listed a "doctest" target that was removed in #4560, so "make doctest" fails with "No rule to make target". The example check now runs as "make check_output -C docs". - "pretty_format" ran clang-format on all sources, but .clang-format was deleted in #4573, so the target reformatted everything in the default LLVM style, against the Artistic Style formatting that "make pretty" applies and CI enforces. - "clean" removed benchmarks/files/numbers/*.json, a directory that no longer exists since the benchmarks moved to tests/benchmarks (#3462). Only maintainer tooling changes; the library is not affected. Part of #5717 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Document tools/macro_builder and tidy up serve_header.py tools/macro_builder generates the NLOHMANN_JSON_EXPAND, NLOHMANN_JSON_GET_MACRO and NLOHMANN_JSON_PASTE* macros in macro_scope.hpp, but nothing referred to it. Add a README that explains what it generates, how to run it and where the output goes, and which dependent tables (NLOHMANN_JSON_DOUBLE_PASTE, NLOHMANN_JSON_TYPE_BODY) are maintained by hand. Point to it from a comment above NLOHMANN_JSON_EXPAND. The generator itself is unchanged; following the README reproduces the header byte for byte. In serve_header.py, drop the LGTM suppression (LGTM.com shut down in 2022), replace the """.""" placeholder docstrings with real ones, and import socket and ssl at module level. DualStackServer.server_bind uses socket, which was only imported under __main__; when the module was imported instead, the NameError was swallowed and IPV6_V6ONLY was not cleared. The header change is a comment only; behavior, API and ABI are unchanged. Part of #5717 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Hash every release_files artifact, not a hardcoded subset The `release` target signed and copied json_fwd.hpp into release_files alongside json.hpp, but the shasum line that writes hashes.txt only listed json.hpp, include.zip and json.tar.xz. Users could not verify the published json_fwd.hpp against hashes.txt. Hash every file in release_files except the .asc signatures instead of naming files by hand, so a newly shipped header (such as the json_literals.hpp that #5610 adds to this target) cannot be missed again. Only affects the generated hashes.txt release artifact; the library itself is unaffected. Overlaps #5610, which touches the same lines to add json_literals.hpp to the release target. #5717 item 1 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Remove the broken fuzz_testing* Makefile targets fuzz_testing and fuzz_testing_{bon8,bson,cbor,msgpack,ubjson} seeded fuzz-testing/testcases from tests/data, which was removed in |
||
|
Niels Lohmann
|
7d7055ec50 |
Fix stack overflow converting deep values between specializations (#5723)
* Fix stack overflow converting deep values between specializations Constructing a basic_json from another specialization (json to ordered_json or back, also via get<ordered_json>()) converted every container with its range constructor, which calls the converting constructor for each element. The call stack therefore grew with every nesting level, and a value nested some 30,000 levels deep overflowed it. The conversion now bounds its descent the way the copy constructor does since #5387: the first 128 levels are converted exactly as before, and below that convert_iteratively() finishes the value with an explicit stack. It builds each container bottom-up from its converted elements with the container's range constructor, so member order and keys that become equal are handled as before, and it gives a value its type only once its container exists, so an exception leaves nothing behind that cannot be destroyed. Parents (JSON_DIAGNOSTICS) and positions (JSON_DIAGNOSTIC_POSITIONS) are set for every value. Converting a null value no longer resets its positions: the constructor assigned null to a value that already was null, which swapped in the positions of the temporary. Fixes #5650. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Explain why converting null keeps positions and why next is a reference Review feedback on #5723 (gregmarr): clarify in comments that the converting constructor has already copied the positions of val, which the null case keeps like every other case, and that next must be a reference into pending so that ++next advances the stored iterator. Comments only; no code change. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Refer to recursion_depth_limit() in the convert_structured() docs The comment still named nesting_depth_limit, which #5637 removed on develop in favor of detail::recursion_depth_limit(). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Advance the pending iterator through pending.back() and shorten the null comment Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
3b6ae43c53 |
Add JSON_DISABLE_TUPLE_REFERENCE_CONVERSION to fix std::tuple conversions (#5598)
* Add JSON_DISABLE_TUPLE_REFERENCE_CONVERSION to fix std::tuple conversions basic_json can be constructed from std::tuple<json&>, which it turns into a one-element array. Because of this, std::tuple picks its converting constructor that converts the whole source tuple instead of the element-wise one. As a result, std::tuple<const json&> built from std::forward_as_tuple(j) binds to a temporary (a compile error with libc++, a dangling reference with other standard libraries), and std::tuple<json> built the same way holds [j] instead of a copy of j. The new opt-in macro JSON_DISABLE_TUPLE_REFERENCE_CONVERSION (CMake option JSON_DisableTupleReferenceConversion) removes the conversion from a one-element tuple holding a reference to the same basic_json type, so std::tuple converts element-wise. It is off by default, so existing behavior is unchanged. It does not change any function body and therefore is not part of the ABI tag. Fixes #2226 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Convert one-element tuples to arrays on every compiler to_json for std::tuple assigns a braced list, j = { std::get<Idx>(t)... }. With a single element that is itself a basic_json, Apple clang 15 and 16 treat j = {x} as a copy of x, so std::tuple<json>{true} became true instead of [true]. The macOS jobs (Xcode 15.1, 16.1) failed the new checks in unit-disable-tuple-reference-conversion and unit-regression2. The one-element overload that already handles JSON_BRACE_INIT_COPY_SEMANTICS builds the array (or object, for a [string, value] element) explicitly, the same way the initializer-list constructor does. Use it unconditionally. The output is unchanged on compilers that already wrapped the element. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Skip json reference tuple tests on clang < 4 and GCC < 5 ci_test_compilers_gcc_old (4.8) and ci_test_compilers_clang (3.4) could not compile the new tuple tests. Creating a std::tuple of basic_json references, e.g. std::forward_as_tuple(j), makes these compilers instantiate basic_json's conversion operator for libstdc++'s internal tuple bases, which fails hard. This happens with and without JSON_DISABLE_TUPLE_REFERENCE_CONVERSION, so it is a limitation of these compilers, not of the new option. Tested with the CI images: clang 3.4 to 3.9 and GCC 4.8 and 4.9 fail, clang 4, 5, and 6 and GCC 5 and 6 compile all cases. Skip only the checks that create such tuples; the is_constructible checks still run. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
6a073dbae4 |
Give operator>> a strong exception-safety guarantee (#5695)
operator>> parsed directly into its basic_json& target, so a parse error left the target holding whatever was parsed before the error instead of its previous value. With JSON_DIAGNOSTICS=1, that partial value also violated the class invariant, because the parent pointers of an array or object's elements are only set when the container is closed, which a failed parse never reaches; copying such a value then aborted in assert_invariant(). Fix it the way basic_json::parse() already handles this: parse into a temporary and move it into the target only once parsing succeeds, so the target is left unchanged if an exception is thrown. Fixes #5652. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
fdcc569eee |
Use only documented StringType members in json_pointer (#5692)
contains(const json_pointer&) and operator/=(std::size_t) (and hence
operator/(std::size_t)) used string_t operations that the StringType
template parameter documentation explicitly does not require:
comparing string_t with a const char* literal, c_str(), and
constructibility from std::string. This made both functions fail to
compile for a conforming custom StringType, even though the
documentation's own reference StringType satisfies the requirements.
Fix contains() to compare individual chars ('0'..'9') instead of
comparing string_t with const char* literals, and to call data()
(documented to be null-terminated) instead of c_str(). Fix
operator/=(std::size_t) to build the array-index token via the
existing detail::to_string<StringType> helper (ADL int_to_string() or
assignment from std::to_string()) instead of via std::to_string()
directly, matching how diff(), items(), and std::hash already convert
a std::size_t to a StringType.
Add regression tests to tests/src/unit-alt-string.cpp: contains() for
present/missing keys and indices, "-", a leading zero, and a
non-numeric token on an array, plus json_pointer::operator/(std::size_t).
Fixes #5666.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
|
||
|
Niels Lohmann
|
6fc0d501f3 |
Move the user-defined string literals to <nlohmann/json_literals.hpp> and add JSON_NO_AUTOMATIC_UDLS (#5610)
* Add JSON_NO_UDLS to leave out the user-defined string literals The bodies of operator""_json and operator""_json_pointer call the parser, so every translation unit including the library instantiates it, even if it never parses anything. Defining JSON_NO_UDLS leaves the literals out entirely, which saves 15-35% compile time for such translation units (#5294). Nothing changes if the macro is not defined. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Mention JSON_NO_UDLS in the list of exported module symbols Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Move the user-defined string literals to <nlohmann/json_literals.hpp> Following the review in #5294, the literals now live in their own header instead of being removed entirely: <nlohmann/json.hpp> includes it at the end unless JSON_NO_AUTOMATIC_UDLS (renamed from JSON_NO_UDLS) is defined, so a project can opt out globally and include the header only where the literals are used. The header only uses public and standard macros, because the library's internal macros are undefined at the end of json.hpp and the amalgamation inlines macro_scope.hpp only once. For the same reason, the library no longer defines and undefines JSON_USE_GLOBAL_UDLS, so a user's definition is still visible to the header. The single-header copy is identical to the multi-header one, as it only includes <nlohmann/json.hpp>. The module always exports the literals. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix CI: include cycle, GCC 4.8 literal operator spacing, and global UDLs off in the JSON_NO_AUTOMATIC_UDLS test - Suppress clang-tidy misc-header-include-cycle on the intentional mutual include of json.hpp and json_literals.hpp. - Use operator"" _json with a space for GCC 4.8 in the test's detection aliases, as the header does. - Only test the global literal operators when JSON_USE_GLOBAL_UDLS is on. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Declare the literal operators through a local macro The GCC 4.8 spacing condition was repeated for both operator definitions and the global using-declarations. NLOHMANN_JSON_LITERAL_OPERATOR(suffix) now selects operator""##suffix or operator"" suffix in one place and is undefined at the end of json_literals.hpp. Suggested by gregmarr in review. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
67435c9c7e |
Give a deep copy its type only after its container exists (#5721)
When copying a value nested deeper than 128 levels, and an allocation fails while an inner array or object is being copied, the partially built copy ended up with an element typed array/object but holding a null pointer. That element was already a fully constructed member of its parent's container, so destroying the parent during stack unwinding dereferenced the null pointer (release builds) or failed assert_invariant() (debug builds), instead of letting std::bad_alloc reach the caller. copy_iteratively() set a pending worklist element's type right after popping it, before the next loop iteration created its container in copy_array_level()/copy_object_level(). Move that type assignment into those two functions, right after the container is successfully created, and drop the premature one in copy_iteratively(), so a half-built element stays a null value - as copy_shallow()'s comment already promised - until it can safely hold one. Add a regression test to tests/src/unit-allocator.cpp that copies a value nested 130 levels deep (both arrays and objects, with a std::map- and an ordered_map-backed object_t) and fails every allocation of the copy in turn: each attempt must throw std::bad_alloc without crashing, and the source must stay unchanged. Fixes #5640. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
2ea6d8c127 |
Require the found key to equal the looked-up key when comparing objects (#5720)
For an object type whose comparator treats unequal keys as equivalent (for example a std::map with a case-insensitive comparator), compare_iteratively() looked up a mismatched left key in the right object with find(), which uses the object's own comparator, and accepted whatever entry it found without checking that the keys are actually equal. A case-insensitive comparator then found "KEY" for "key", so two objects nested past the recursion bound (or at every depth with JSON_NO_THREAD_LOCAL) could compare equal even though the object type's own operator== - and basic_json itself, below the bound - consider them different. Accept the found entry only if its key equals (not just compares equivalent to) the looked-up key. Fixes #5655. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
66877675b1 |
Check the iterator range for binary values in basic_json(first, last) (#5719)
basic_json(first, last) treated value_t::binary like the structured types (array, object) in the range check, so it always copied the whole binary value regardless of the iterators, even for an empty range such as (b.end(), b.end()). The other primitive types (number, boolean, string) already reject such a range with invalid_iterator.204, and erase(first, last) already does the same for binary values, so this made the constructor inconsistent with both. Move case value_t::binary into the group of checked primitive types. Also update the two matching passages in basic_json.md that describe overload 7, and add a version-history note. Fixes #5670. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
7fd6895788 |
Hide a discarded container's content from the parser callback (#5706)
When a parser callback rejects an object's or array's start event,
json_sax_dom_callback_parser kept calling it for everything inside
that container anyway: nested keys, values, and the start/end events
of containers below it. This contradicts parser_callback_t's own
documentation, which promises that discarding a container at its
start event also hides its content from the callback.
The same code path also kept a full copy of every key inside such a
discarded container in key_stack until the whole parse finished,
because the early return for values that are not stored skipped the
matching pop. Filtering out a large subtree is the main reason to use
a callback, so this made peak memory during the parse scale with the
size of the very subtree the callback was trying to skip.
Fix start_object(), start_array(), and key() so that a container
whose own start event was discarded, or that is nested inside one, is
never handed to the callback, and no longer pushes onto the key
stacks. A container whose start event was accepted but whose key was
rejected still gets its content reported, as documented ("the
callback is still called for the associated value, but its return
value has no further effect"); only its own bookkeeping is skipped
since it will not be stored.
Fixes #5643.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
|
||
|
Niels Lohmann
|
6ae17630a4 |
Reject integral keys for contains(), find(), and count() at compile time (#5705)
j.contains(0), j.find(0), and j.count(0) used to compile: the literal 0 is a null pointer constant, so it converts to a null const char*, and the overloads taking const typename object_t::key_type& accepted it by constructing a std::string from that null pointer, which is undefined behavior (a crash with both libc++ and libstdc++). value(0, default_value) had the same problem in C++11, where the object comparator is not transparent. Add deleted overloads for integral arguments to contains(), find() (const and non-const), count(), and value() so that these calls are compile errors in every supported language mode instead of crashing. Calls with string, string_view, json_pointer, and size-typed element access (at(), operator[](), erase()) are unaffected. Fixes #5657. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
5bd766aa50 |
Move to_bson's binary subtype check into calc_bson_sizes (#5703)
to_bson() rejected a binary value's subtype above 255 (out_of_range.415) in write_bson_binary(), which only has the binary_t, not the basic_json value that holds it, so the exception was created with no JSON_DIAGNOSTICS context even though the equivalent to_msgpack() check names the value's path. The check also ran after the document size, all preceding elements, and this element's header and length had already reached the output adapter, so a caller-provided std::vector or std::string ended up holding a truncated document. calc_bson_sizes() already walks every value before anything is written, to size embedded documents and arrays and to reject invalid keys (out_of_range.409) up front. The subtype check now runs there instead, in calc_bson_binary_size(), which is given the basic_json value so the exception can use it as context. The now-redundant check in write_bson_binary() is removed, since calc_bson_sizes() always throws first if any binary value in the document has an oversized subtype. Fixes #5675. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
4bb1b14b06 |
Trim the compiler-appended NUL from wide/UTF string literals too (#5702)
With JSON_STRICT_NUL_HANDLING defined to 1, parsing a wide, UTF-16, UTF-32, or (C++20) UTF-8 string literal (e.g. json::parse(L"[1]")) failed with parse_error.101 at the terminating NUL of the literal, and accept() returned false. The array overload of input_adapter() only dropped the compiler-added trailing '\0' for arrays of char, so for wchar_t, char16_t, char32_t, and char8_t arrays that terminator was passed to the parser as data, which the macro then rejected. Broaden the trimming to every character type that a string literal can use (char, wchar_t, char16_t, char32_t, and, since C++20, char8_t). Arrays of any other element type (unsigned char, std::uint8_t, ...), as used for CBOR/MessagePack, are unaffected: a trailing zero byte there is still read as genuine data. Fixes #5658. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
 Niels LohmannandClaude Sonnet 5
|
bfe0f32d71 |
Fix std::terminate and null pointer access in input_stream_adapter (#5699)
Parsing from a std::istream crashed in two unusual but valid stream states, both in input_stream_adapter: - With eofbit in the stream's exceptions() mask, get_character() sets eofbit via is->clear(), which throws std::ios_base::failure. While that exception unwinds, ~input_stream_adapter() called clear() again to reset eofbit, which is still set and still in the exception mask, so it throws a second time out of the (implicitly noexcept) destructor and std::terminate() is called. The destructor now only calls clear() if a bit other than eofbit remains set, so the first exception can propagate normally. - For an std::istream without a stream buffer (rdbuf() == nullptr, e.g. std::istream(nullptr)), the constructor stored the null pointer without checking it, and get_character() dereferenced it. input_adapter(std::istream&) now throws parse_error.101 for such a stream, the same as it already does for a null FILE* or char*. Added regression tests to unit-deserialization.cpp and, for the JSON_PRECISE_STREAM_POSITION variant of get_character(), to unit-precise-stream-position.cpp; both crashed before this fix. Documented the two exceptions in parse.md and operator_gtgt.md. Fixes #5646. Signed-off-by: Niels Lohmann <mail@nlohmann.me> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Niels Lohmann
|
44a88d85be |
Fix NLOHMANN_JSON_SERIALIZE_ENUM_STRICT's from_json message (#5698)
from_json built its out_of_range.410 message with "..." + j.dump(). If the unmatched value is (or contains) a string with invalid UTF-8, that dump() itself throws type_error.316, so the caller got type_error.316 instead of the documented out_of_range.410; such strings can reach get<Enum>() unvalidated, e.g. from from_cbor()/from_msgpack(). With a custom string_t, j.dump() returns that type, and "const char*" + string_t does not compile unless the type happens to provide operator+, so the macro failed to compile for such types. Build the message with detail::concat(), which appends any type exposing data()/size() and always yields a std::string, and dump with error_handler_t::replace so building the message itself cannot throw. Added regression tests: an invalid-UTF-8 case in the existing strict-enum test in unit-conversions.cpp, and a strict-enum use with alt_string (the custom string_t from unit-alt-string.cpp) to cover the compile failure. Fixes #5667. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
5b11a0282c |
Exchange the CustomBaseClass subobject in basic_json::swap() (#5697)
basic_json::swap() (and the friend swap() and the pre-C++20 std::swap overload that forward to it) only exchanged m_data.m_type/m_data.m_value, leaving each value's json_base_class_t subobject in place. This is inconsistent with the copy and move constructors and copy assignment, which all carry the base class along with the value, so after a.swap(b) any metadata stored in a CustomBaseClass ended up attached to the wrong value. Algorithms that mix swap() with moves, such as std::sort, scrambled the metadata across the whole container. Fix the member swap() to also exchange the json_base_class_t subobject and extend the noexcept specifications of swap() and the friend swap() accordingly. Fixes #5653. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
bfea6f36d3 |
Fix to_msgpack() reading the inactive number union member (#5694)
* Fix to_msgpack() reading the inactive number union member basic_json stores number_integer and number_unsigned in a union, and number_unsigned_t only has to be at least as wide as number_integer_t (with the default types, both are 64-bit and have the same representation). When number_integer_t is narrower, write_msgpack() read the wrong union member in two places: - The number_unsigned case wrote number_integer's bits instead of number_unsigned's, silently writing the wrong value whenever it did not fit in number_integer_t. - The number_integer case (non-negative branch) picked the encoded width by comparing number_unsigned's bits, which is undefined behavior, though the value written was still number_integer's, so at worst a too-wide encoding was chosen. Read the active member in both cases, like the other binary writers (CBOR, UBJSON, BJData, BSON, BON8) already do. Fixes #5644. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Cast number_integer to number_unsigned_t only once in to_msgpack() Addresses review comment by @gregmarr. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
e444a66276 |
Copy values before inserting an initializer list into an array (#5693)
* Copy values before inserting an initializer list into an array
insert(pos, {...}) inserted wrong values when the initializer list
contained const references to elements of the array being inserted
into. json_ref stores only a pointer for a const lvalue, so the
initializer_list_t range passed straight to the array's range insert
aliased the array's own storage; std::vector::insert(pos, first, last)
may move or shift elements before copying from that range, so the
source elements were already stale by the time they were read
(different wrong results on libc++ and libstdc++).
Copy the referenced values into a temporary array_t first, then move
that temporary into place, so the source range never aliases the
array being modified.
Fixes #5656.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Use the reserve_array helper in the initializer_list insert fix
The previous commit called array_t::reserve() directly on the
temporary buffer used to copy an ilist's values before inserting.
std::deque, a documented ArrayType (tests/src/unit-custom-array-type.cpp),
has no reserve(), so insert(pos, initializer_list) no longer compiled
for it. Use the existing detail::reserve_array() SFINAE helper (already
used by the SAX DOM parser) instead, which leaves array types without
reserve() untouched.
Added a regression check that deque_json::insert(pos, {...}) compiles
and handles the aliasing case from #5656.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
|
||
|
Niels Lohmann
|
1151826508 |
Take diff()'s fast path unless the object type reorders members (#5691)
* Take diff()'s fast path unless the object type reorders members For every object type except an insertion-ordered one like ordered_map, diff() no longer produced a member-by-member patch when target had a key that sorts before a key the two objects share: it fell through to the slow path, which removes every member of source and re-adds every member of target, instead of just adding the new key. #5465 added an order check to require the fast path to also reproduce target's member order, needed because ordered_map's patch()-driven "add" appends a new member at the end. The check compared the common keys' order between source and target and also required that every added key come after every common key in target's order ("new_keys_form_suffix"). The comment above it argued this check is always true for std::map, and that reasoning is correct for the order of the common keys themselves, but not for new_keys_form_suffix: a std::map iterates in sorted key order, so a new key that sorts before an existing common key is enumerated between common keys, making new_keys_form_suffix false even though std::map's own key order does not need reordering at all - it places every member itself, regardless of insertion history, so a member-by-member diff already reproduces target's iteration order. Only require the order check for an object type that keeps insertion order, using the same detail::is_ordered_map trait the library already uses to recognize such an object type in set_parent(). Every other object type - std::map in key order, a hash map in an order its operator== ignores - always takes the fast path. Added a regression test to unit-json_patch.cpp: the issue's example now yields a single "add" op for json, while ordered_json still takes the slow path to reproduce target's member order. Fixes #5639. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Only track the target key order in diff() for insertion-ordered objects common_keys_target_order and new_keys_form_suffix are only read when object_t keeps its members in insertion order; skip building them otherwise. Addresses review comment by @gregmarr. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
6218aa212b |
Throw std::length_error for operator[](SIZE_MAX) instead of corrupting the array (#5687)
For idx == SIZE_MAX, the non-const array operator[] computed the new size as idx + 1, which wraps to 0. resize(0) then emptied the array, and the subsequent operator[](idx) on the now-empty vector wrote one element before its buffer. Every other too-large index (e.g. SIZE_MAX - 1) already went through resize(), which throws std::length_error and leaves the array unchanged; SIZE_MAX was the one value for which the overflow bypassed that safety net. Add a guard that throws std::length_error before computing idx + 1 when idx is the largest representable size_type value, so the array is left unchanged, matching the exception vector::resize() already throws for smaller (but still too large) indices. Fixes #5647. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
42f89e3130 |
Let to_json(std::optional<T>) propagate exceptions from T's to_json (#5684)
* Let to_json(std::optional<T>) propagate exceptions from T's to_json The overload was marked noexcept even though its body assigns *opt to the JSON value, which calls T's to_json (or allocates for std::string, std::vector, or json). Any exception from there -- a user-defined to_json reporting an error, or std::bad_alloc -- called std::terminate() instead of propagating. The noexcept also made basic_json's converting constructor noexcept(true) for std::optional<T>, so json j = opt; could not report the error either. Fixes #5642. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix CI: make to_json(std::optional<T>) conditionally noexcept GCC's -Wnoexcept (an error in ci_test_gcc) fired at to_json_fn's noexcept(noexcept(to_json(j, val))): after dropping the unconditional noexcept, to_json(std::optional<int>) had no exception specification although GCC could prove its body cannot throw. It also made json(std::optional<int>) lose its noexcept. Declare the overload noexcept exactly when assigning the contained value to the JSON value is (std::is_nothrow_assignable<BasicJsonType&, const T&>), which is what the body does. std::optional<int> is noexcept again; a T whose to_json may throw still propagates the exception. Static assertions in the test check both cases. The test's throwing to_json triggered -Wmissing-prototypes and -Wmissing-noreturn (clang) and -Wmissing-declarations and -Wsuggest-attribute=noreturn (GCC). Move the type and its to_json into an anonymous namespace, mark the function [[noreturn]], and compile them only without JSON_NOEXCEPTION, like the test that uses them. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
6d7845d207 |
Fix deprecated json_pointer/string operator== warning in value() (#5683)
value(KeyType&&, default) is constrained on is_comparable_with_object_key, which passes KeyType as a reference. is_comparable's dispatch on is_json_pointer_of<A, B> only matches a json_pointer as a plain type or a plain reference, so a const-qualified reference (as produced when KeyType is deduced from a json_pointer argument) fell through to is_comparable_no_json_pointer, which instantiates the deprecated json_pointer/string comparison operators. This made ordered_json's transparent comparator (and any transparent comparator on a custom string type) warn under -Wdeprecated-declarations when calling value(json_pointer, default), even though no such comparison is ever performed. at() was already fixed for this in #5289, which does not use is_comparable_with_object_key. Strip references and cv-qualifiers with uncvref_t before the is_json_pointer_of dispatch, so any reference-to-json_pointer is recognized regardless of qualifiers. Fixes #5664. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
678fd3017b |
Fix element path for map/unordered_map JSON_DIAGNOSTICS errors (#5681)
When converting a JSON array to std::map or std::unordered_map with a non-string key, each element must itself be a [key, value] array. If an element is not an array, from_json() threw type_error 302 with the outer array's value (&j) as the exception context, so with JSON_DIAGNOSTICS enabled the message pointed at the whole array instead of the offending element (e.g. "(/outer/m)" instead of "(/outer/m/2)"), even though the message text already described the element's type. Both from_json() overloads now pass the element (&p) as the context, so the reported JSON Pointer matches the type named in the message, the same way std::vector<std::vector<T>> and similar conversions already do. Fixes #5668. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
fc4c9c3446 |
Fix clear() to also reset the subtype of a binary value (#5680)
* Fix clear() to also reset the subtype of a binary value clear() on a binary value cleared the bytes but left the subtype untouched, so the result was not equal to a default-constructed binary value even though the documentation says clear() has the same effect as *this = basic_json(type()). The fix calls byte_container_with_subtype::clear_subtype() alongside the existing clear() call. Extended the "filled binary" clear() test in unit-modifiers.cpp with a case that uses a subtype, since the existing cases only covered binary values without one. Fixes #5669. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix the table alignment in clear.md Addresses review comment by @gregmarr. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |