mirror of
https://github.com/nlohmann/json.git
synced 2026-10-24 23:08:56 +00:00
d9c0fda82913ca39a0e803b04eb6603b312b7093
20
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Niels Lohmann
|
9c50bdd140 |
Make the DOM parser cheaper per container and per member (#5882)
* Reserve the parser's nesting stack once The stack of open arrays and objects was a std::vector<std::uint8_t> that allocated at the first push_back and then reallocated at 1, 2, 4, 8 bytes, so a document nested four levels deep made four allocations before it parsed a value. A document that starts with a container now reserves 32 levels in one allocation; deeper documents grow as before. The loop that reads the tokens is the code it was, so large documents are not affected (a stack object with its own storage, and a reserve at every container, both cost the token loop instructions or cycles). Fixed cost of parsing a 124-byte object (macOS instructions): accept -7.6 %, DOM -2.5 %; no change for a single number. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Build arrays of the DOM parser in one exact-size allocation Reading an array of unknown length appended its elements to a std::vector one at a time, so a small array reallocated (and moved its elements) at sizes 1, 2, 4, ... The DOM SAX parser now collects the elements of arrays whose length is unknown on one stack shared by all open arrays and moves them into an array of exactly the right size when the closing bracket arrives. An array that grows past 16 elements switches to appending to its own vector, so large arrays pay no second copy. Binary formats, which announce the length, and builds with JSON_DIAGNOSTICS or JSON_DIAGNOSTIC_POSITIONS (whose values need their final place from the start) keep storing into the array directly. The stacks of open containers are reserved on first use instead of growing from one element. The binary readers get a DOM parser of their own, json_sax_dom_binary_parser: it is the plain parser that stores every value directly in its container (the code the readers used before). Their containers announce their size, so the bookkeeping of the collecting parser (frame entries, the destination of a finished container) was pure overhead for them: with the collecting parser FromBinaryShape nested was 3-12 % and containers up to 5 % more instructions (GCC) than before. With the plain parser Linux Ir of FromBinaryShape and FromBinaryBuffer is within 0.8 % of develop (Clang: identical). canada.json has 56 000 two-element arrays that took three allocations each and now take two. DOM parse Ir on Linux against develop (GCC / Clang): canada -7.0 % / -7.7 %, citm -0.8 % / -1.6 %, 124-byte object -5.4 % / -5.5 %; accept and SAX are untouched. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Reuse one value for the key events of the callback parser parse() with a callback built a new string value for every key it passed to the callback (a heap allocation and release per key), and copied the key a second time into the object. The callback parser now keeps one string value for key events, which it assigns, and moves the key into the object. Callback parse, Linux Ir (GCC / Clang): twitter -13.3 % / -11.7 %, citm -9.5 % / -7.6 %. (Earlier macOS measurement: twitter -19.5 % instructions.) Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Remember where the DOM parser stores the next value The DOM SAX parser looked up the innermost open container for every value: the end of its frame stack, then the object or array pointer in the frame. Compilers have to reload all of that after each value is stored, since a store of a value may alias the frame. The parser now keeps the innermost container's kind and, for an array that stores directly, the array itself in members, updated when a container opens or closes. Flat arrays of numbers are the case that mattered: parsing floats.json (1 000 000 doubles) into a DOM took 2 % more cycles than before the previous commit on macOS and now takes 0.4 % fewer than the code before it (signed_ints.json -0.8 %). Linux Ir is unchanged (canada -6.1 % GCC / -6.9 % Clang, citm -0.3 % / -0.8 %, twitter +0.2 % / 0.0 %). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Look up the keys of big ordered_json objects in a hash table ordered_map keeps its elements in a vector and searches it linearly for every key it adds, so parsing an object with n keys into ordered_json took O(n^2) key comparisons: 33 billion instructions for one object with 30 000 keys, where json needs 245 million. From 64 keys on, the DOM parser now keeps a hash table of the keys of the open object (hashes and positions only; the elements stay in the vector) and finds duplicate keys there. The result is the same, including the position of a key that occurs twice. Smaller objects, std::map based objects and key types without std::hash are untouched. ordered_json parse, macOS instructions: object with 1 000 keys 64.8 M -> 14.6 M, 10 000 keys 5 104 M -> 38.7 M, 30 000 keys 33 358 M -> 89 M. Linux Ir of an object with 2 000 keys: 43.0 M -> 1.9 M (GCC), 37.9 M -> 1.8 M (Clang). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Trim the work the DOM parser does per container and per member Opening and closing an array or object cost the DOM parser more than before the exact-size array change, which showed on documents of many small objects (jeopardy.json: +1.4 % Ir with GCC). This takes it back: - the stack of open containers keeps its first eight entries in the parser object instead of in a vector, so it needs no capacity check beyond the push itself and no allocation for typical documents; - a container is created in the place it ends up in, instead of as a temporary that is then moved there; - a new object member, which is null, is constructed over the null instead of being swapped with it (only with the default base class of basic_json); - the key index that big ordered_json objects use is only a part of the stack entries of objects that can have one; - an array that outgrows the stack reserves twice what it has so far, as a vector that grew by itself would have. DOM parse Ir of the series up to here against develop, Linux GCC / Clang: canada -6.6 % / -7.3 %, citm +1.2 % / +0.2 %, jeopardy +0.4 % / -0.1 %, 124-byte object -6.5 % / -6.2 %. (twitter measures +6.8 % / +6.6 % at this point in steady state, almost all of it in glibc's malloc/consolidate for the new allocation sizes; with the last commit of the series it is -0.1 % / 0.0 %.) Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Test parsing containers of the sizes where the DOM parser changes storage The DOM parser collects the elements of an array on a stack (up to 16) and the members of an ordered_json object (up to 32) before it moves them into the container, and uses a hash table for keys from 64 members on. Parse arrays and objects of the sizes around these limits, nested into each other, with repeated keys, and as random trees that are built next to their text, and compare json and ordered_json against the value built through the API. The tests pass on the code from before these changes as well. The random trees are what finds the mistakes that are easy to make here: while the collecting change was being written, a collected array that outgrew the stack at the moment its parent had just reached the limit left its elements in the parent, and only these tests (142 failed assertions) showed it. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Do not convert floats that cannot overflow in accept() accept() only has to find out whether a float token is finite: json_sax_acceptor ignores the value, and the parser rejects only a float that overflows. A float whose digits fit into 19 digits and whose decimal exponent is at most max_exponent10 - 19 is below 10^max_exponent10 and cannot overflow, so the lexer, told by accept() that the value is not needed, skips the conversion for those. A token that may overflow, with more digits, or in another float format is converted as before, so 1e309 and its neighbours are rejected as they were (the new test compares accept() with parse() on tokens around the limit). json::accept(): canada.json Ir -8.6 % (GCC) / -10.9 % (Clang), 124-byte object -3.2 %; cycles on the M1 (release benchmarks, LTO): canada -7.8 %, floats.json -9.0 %. The check tests the accept() flag first, so the parser that needs the value pays one load and branch: SaxParse canada +0.35 % Ir (GCC and Clang), DOM canada +0.4 %. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Collect the members of ordered_json objects on a stack until they are known An ordered_json object keeps its members in a vector and appends every new member to it, so reading an object of unknown size reallocated the vector at 1, 2, 4, 8, ... members, and ordered_map copies all keys on every reallocation. The DOM SAX parser now collects the members of such objects on one stack shared by all open objects, as it does the elements of arrays, and moves them into a vector of exactly the right size when the closing brace arrives. An object that gets past 32 members switches to appending to its own vector (and, from 64, to the key index of the earlier commit). The members are the same, in the same order, and a key that occurs twice keeps the place of its first occurrence. std::map based objects, objects whose size the input announces (binary formats), and builds with JSON_DIAGNOSTICS or JSON_DIAGNOSTIC_POSITIONS are untouched. What only ordered objects need (the collected members, the key indexes) is a member of the parser only for them, and the code that stores a value tries the array that takes its elements directly first. An array that outgrows the stack now reserves 32 elements, the capacity a vector that grew by itself has at that point; with the larger reserve before, parsing floats.json (1 000 000 numbers) was 2 % slower on Apple M1 than before this series. ordered_json parse, Linux Ir against develop (GCC / Clang): citm -11.6 % / -11.7 %, twitter -8.4 % / -8.4 %, jeopardy (12 000 objects of 7 keys) -10.5 % / -8.0 %, canada -3.9 % / -5.2 %, 124-byte object -20.9 % / -16.5 %. macOS time (M1, earlier base): citm -19 %, twitter -13 %, jeopardy -13 %, canada -14 %, 124-byte object -26 %. json parse: steady-state twitter -0.1 % (see the previous commits), citm -1.2 %. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Rename the key index's match callback so Flawfinder doesn't read it as std::equal Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Bind the parser test's strings by reference in the range-for loop GCC and Clang 14 (-Wrange-loop-construct) reported the copy. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Collect container values only in basic_json's own parse functions A json_sax_dom_parser that a user constructs (as in the sax_no_exception example) must hold the values read before a parse error, as on develop: its value is used after the error. The values of containers of unknown size are now collected on a stack only with the new template parameter Collect (default false), which json::parse() and the json_view materialization set; without it, every value is stored in place as it arrives. A test checks the values kept after an error for json and ordered_json. Also fixes the clang-tidy, cpplint and Infer findings: the frame stack keeps its first frames in a std::array (left uninitialized) and the others in a std::unique_ptr, auto for create<>(), NOLINT for member initializers -Weffc++ needs, and an @infer-ignore for the nullptr that place_container() does not use. json::parse() and ordered_json::parse() instructions are unchanged (Apple clang; twitter, citm, canada, jeopardy: -0.03 % to +0.25 %). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Clarify the DOM parser's collection stacks and rename reenter_enclosing_container() Address review comments: explain why a closed container erases only the top of the shared stack, set the payload before the type, rename the helper that makes the enclosing container current again, and declare the parser's state stack right before reserving it. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Export json_sax_dom_parser from the C++20 module for MSVC MSVC needs the detail classes that basic_json befriends to be visible in the module (#3970, as json_sax_dom_callback_parser); with its new template parameter, json_sax_dom_parser failed in ci_module_cpp20 (C2039, not a member of nlohmann::detail). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Keep the overflow-limit numbers of the parser test in a std::vector clang-tidy (cppcoreguidelines-avoid-c-arrays) rejected the C array. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix the clang-tidy findings of the ordered_json parser tests The container sizes are a std::array, the expected text uses raw string literals, the generator's arithmetic is in parentheses, auto for the result of make<>(), and single characters are appended as characters. clang-tidy now reports nothing on the lines the branch adds to unit-ordered_json.cpp and unit-class_parser.cpp (apart from checks CI's clang-tidy does not have). Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
6f2048cd5d |
Accept lvalues in ordered_map::emplace's value parameter (#5685)
* Accept lvalues in ordered_map::emplace's value parameter
ordered_map::emplace(key, value) took the mapped value only by T&&, an
rvalue reference rather than a forwarding reference, so
ordered_json::emplace("a", value) failed to compile whenever value was
an lvalue or a const lvalue, even though the same call compiles for
json (whose object_t is std::map, with a variadic emplace). Turn the
value parameter into a separately-deduced forwarding reference,
constrained with std::is_constructible so the overloads still only
accept something convertible to the mapped type. std::map-compatible
semantics are unchanged: emplace still does nothing if the key already
exists.
Open PR #5609 also touches ordered_map.hpp (moving values on vector
growth); this change only touches the two emplace() overloads and
should not conflict.
Fixes #5673.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Avoid astyle's padding in ordered_map::emplace's template headers
Use detail::conjunction instead of && and drop the redundant V&& in detail::is_constructible, so astyle keeps the usual template formatting. Addresses review comment by @gregmarr.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
|
||
 Niels LohmannandClaude Sonnet 5
|
56b3ee566c |
Fix stack overflow when copying a deeply nested value (#5387) (#5389)
* Bound the descent of the copy constructor basic_json's copy constructor copied objects and arrays by handing the container to its own copy constructor, which copy-constructs every element and so reaches this constructor again, once per nesting level. A value nested deeply enough exhausted the call stack and terminated the process with a segmentation fault - no exception, nothing the caller could catch. Parsing such a value works, as the parser is iterative, and so does destroying one, as #1436 made destruction iterative. Bound how far the copy descends rather than take the call stack away from it. The first levels are copied exactly as they were - the containers copy their own elements, which is by far the fastest way to fill them - and only once the copy has descended 128 levels is the value below it finished without the call stack, through an explicit worklist. Copying can therefore no longer exhaust the stack, however deeply a value is nested, while a value nested less deeply than the bound - all but a vanishing minority - is copied by the very same code as before and pays only for one counter. That counter lives in thread_local storage, as one shared between threads would be raced. JSON_NO_THREAD_LOCAL switches it off for toolchains without thread_local; copying then goes through the worklist right away, which yields the same values but is measurably slower. The deferred values are completed before the copy they belong to returns, so a value copied while another copy is going on - by a custom base class, say - is unaffected by the copy it is nested in. operator= takes its argument by value, so copy assignment is fixed as well. Copying is as fast as it was, within measurement noise (medians of 9 interleaved runs, clang -O3): -1.3% for an array of strings, +0.0% for a flat object, +0.1% for a flat array of numbers, +0.3% for nested arrays, +0.6% for nested objects and +1.2% for a twitter-like document. Copying a three-key object costs about ten nanoseconds more, the counter. Deferring every level instead, rather than only those below the bound, measured between 3% and 9% slower depending on the shape of the value. This fixes #5387 for the copy constructor. dump() is still recursive. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Test the copy constructor's iterative path in CI The copy constructor descends into 128 levels before it finishes a value without the call stack, so the iterative path is otherwise only reached by the few tests that nest deeper than that. JSON_NO_THREAD_LOCAL switches the descent off, which sends every value down that path. Running the whole test suite that way covers it with every object type, string type, allocator, and base class the suite already exercises. The new ci_test_no_thread_local target does that; the macro had no build coverage at all before. Copying a nested value also has to carry over what the element-wise copy constructor would have copied: the parents that JSON_DIAGNOSTICS relies on, and the positions that JSON_DIAGNOSTIC_POSITIONS reports. Both are now checked on either side of the descent bound, for objects and arrays. Neither was tested before, and dropping either one makes the new tests fail. Also quantify what JSON_NO_THREAD_LOCAL costs a copy instead of calling it "measurably slower". Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Split the regression tests so that they keep linking Linking test-regression2 fails with "relocation truncated to fit: IMAGE_REL_AMD64_REL32 against `.rdata'" once its object grows past what the MinGW linker copes with, and the copy constructor's helpers push it over: the object grows by 6.3%, from 4,654,128 to 4,944,920 bytes at -O0, and develop links at the smaller of the two. Building the tests optimized shrinks the object enough to link, but the binaries clang 11.0.1 and clang 18.1.8 then produce crash before doctest prints its first line - 39 of 102 tests on clang 18 - so the objects have to become smaller rather than denser. Moving the test cases that follow "regression tests 2" into a file of their own brings that object to 4,687,888 bytes, which is 0.7% above the size that links today rather than 6.3%. Both files still build for C++11, C++17 and C++20, and run the same 9 test cases and 135 assertions as before, now spread over two binaries. New regression tests belong in unit-regression3.cpp from here on, which is what CONTRIBUTING.md now says. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Do not use thread_local storage with Clang targeting MinGW Every test that copies a value segfaults there - 42 of 105 on clang 11.0.1, 39 of 102 on clang 18.1.8 - while the same tests pass with GCC targeting MinGW, with Clang targeting MSVC, and with every other toolchain the library is tested on. The counter that bounds the copy constructor's descent is the library's first use of thread_local, so that job had never exercised it before. JSON_NO_THREAD_LOCAL already covers toolchains without thread_local storage, and copying yields the same values with it, only more slowly. Define it for this one automatically. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Balance the warning suppression the split separated unit-regression2.cpp opens a DOCTEST_CLANG_SUPPRESS_WARNING_PUSH block at the top and closed it at the very bottom, which the split moved into unit-regression3.cpp: one file was left with a push and no pop, the other with a pop and no push, which clang reports as an error. Give each file the pair it needs. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Check both shapes without a C-style array clang-tidy rejects the array the two shapes were iterated over (cppcoreguidelines-avoid-c-arrays). The array only existed because astyle reformats a range-for over a braced initializer list into something unreadable; naming the two cases avoids both. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Split the regression tests far enough to leave room The first split left unit-regression2.cpp 0.7% below the size develop links at, which the comparison change in the follow-up immediately used up: the MinGW linker fails on test-regression2_cpp20 again, naming copy_shallow and to_partial_ordering among the relocations it cannot fit. Move the sections from "issue #2067" on, and the helper types they use, so that the file stops being the one that decides whether the tests can be linked at all. At -O0 and C++20, unit-regression2.cpp is now 2,964,944 bytes against develop's 4,708,248, and 3,070,568 bytes with the follow-up applied - roughly a third smaller either way, rather than a fraction of a percent larger. The 135 assertions are the same ones as before, now spread over three test cases in two files. Also silence the clang-tidy findings the deep-nesting tests draw: the copies they make are what is being tested, and the reserve() computation gets its parentheses. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Move the #4804 alias to the file that uses it The split left the json_4804 alias behind in unit-regression2.cpp while the test case that uses it went to unit-regression3.cpp, which does not build for C++17 and C++20 as a result. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Include <span> where the split moved its only use The #2546 test case guards itself with __has_include(<span>), but the include itself sat in unit-regression2.cpp's preamble and stayed behind, so the section compiled without a declaration wherever the guard passed - which nvhpc reported and libc++ builds do not, as they skip the section altogether. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Keep the descent bookkeeping in one place Copying carried a depth count, a depth limit and a guard of its own, and the comparison in the follow-up added a second set beside them. Neither operation needs its own: they are never nested inside one another by the library - copying a value does not compare one, and comparing two values does not copy them - and where user code nests them anyway, sharing the count only ends a descent sooner than it had to. So there is now one nesting_depth(), one nesting_depth_limit() and one nesting_depth_guard, which the follow-up uses instead of adding its own. Inverting the test in copy_structured leaves the too-deep case and the no-thread-local case as the same code. The guard takes the count rather than looking it up, because the caller has looked it up already to test it against the limit, and reaching thread-local storage twice on the path that is taken almost every time is worth avoiding. The switch that copies the value of anything that is not an object or an array was written twice - once in the copy constructor, once in copy_shallow - so that adding a value_t meant editing both, and missing one would have been silent. It is copy_leaf_value now, and inlined: both callers have already sorted the containers out, and folding that test into the switch is what keeps a value made mostly of numbers copying as fast as it did. Copying canada.json, citm_catalog.json and twitter.json is within 0.6% of what it was before, measured as a paired ratio over 18 interleaved rounds against a run-to-run spread of 0.3%. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Check that an abandoned copy can still be destroyed Copying a value without the call stack builds the copy from the top down, and every value whose own copy has not been made yet stays a null value until it is. That is what lets a copy be abandoned half-built: the destructor finds nothing but complete values and null ones. Nothing tested it. Failing an allocation part-way through a copy of a deeply nested value does, with the allocator the file already has for exactly this kind of test. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Name the test's locals so Flawfinder stops matching them The code scanning job reports CWE-362 - "check when opening files" - for a test that opens no files: Flawfinder matched a local variable called open. Rename it and its partner. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Keep the descent guard's bookkeeping self-contained nesting_depth_limit() and nesting_depth_guard were only used inside the JSON_NO_THREAD_LOCAL-guarded branch of copy_structured(), but were defined unconditionally. Move them inside the #ifndef, and have the guard look up the depth and test it against the limit itself (via okay()) instead of making the caller do it - the caller no longer needs to touch nesting_depth() at all. Also shrink the thread-local counter to std::uint8_t, matching what its own doc comment already argued. Addresses gregmarr's review comments on #5389. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Make nesting_depth_guard usable regardless of JSON_NO_THREAD_LOCAL nesting_depth_limit() and nesting_depth() stay behind #ifndef JSON_NO_THREAD_LOCAL, since a descent cannot be bounded without a per-thread count. But the guard itself now always exists, becoming a no-op that is never okay() under that macro - the same way the bound is already reached on every call without one. copy_structured() no longer needs to know which case it is in. This is what lets #5390 reuse the guard for comparison, which cannot test JSON_NO_THREAD_LOCAL where the macro-based operators use it: the guard now carries that distinction itself instead of requiring every caller to. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Silence VS2015's C4503 for the custom-base-class test The deep-copy support added for #5387 lengthened the mangled name of std::allocator_traits<...>::construct for the test's map type past VS2015's limit, which /WX turns into a build failure even though the name is only used for (now-truncated) debug info. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Remove dead unused-parameter casts from copy_metadata() @gregmarr asked whether the static_cast<void> pair in the JSON_DIAGNOSTIC_POSITIONS-off branch was needed for an empty json_base_class_t. It isn't: src and dst are already referenced unconditionally by the base-class copy above, so no -Wunused-parameter warning fires either way (checked with -Wall -Wextra -Wunused-parameter, JSON_DIAGNOSTIC_POSITIONS 0 and 1). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix CI: build custom array types without a fill constructor, re-amalgamate copy_array_level() built the destination array with the fill constructor array_t(count, value), which is not part of the array container interface the library otherwise assumes (e.g. custom ArrayTypes that only provide a default and an iterator-pair constructor, as covered by unit-custom-array-type.cpp). Default- construct the array and resize() it instead, matching how the rest of the codebase already grows array_t. Also re-run the amalgamation, which had fallen out of sync with include/nlohmann/json.hpp. Signed-off-by: Niels Lohmann <mail@nlohmann.me> Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Niels Lohmann
|
e564136c22 |
Make diff() account for member order in ordered_json objects (#5465)
* Make diff() account for member order in ordered_json objects diff() compared source/target objects purely by key set, ignoring relative member order. For ordered_json (insertion-ordered, vector- backed object_t), two objects that differ only in member order are unequal via operator==, but diff() never emitted any patch operation to fix the order, so source.patch(diff(source, target)) == target could fail to hold. Fix by detecting when common keys appear in a different relative order in source vs. target (or when a new key would need to land somewhere other than the end), and in that case removing and re-adding the affected keys in target's order, which relies on patch()'s "add" op appending new keys at the end of an ordered_map. For plain json (std::map-backed, always key-sorted iteration) this is a no-op and the original minimal per-key diff path is unchanged. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Avoid redundant lookups in diff()'s object-order tracking The previous fix for ordered_json member order re-derived common-key order and suffix information with extra target.find()/source.find() calls layered on top of the pre-existing removed/added-key passes, instead of reusing those same passes. This roughly tripled the number of map lookups per diff() call for every object, including plain `json`, where the reordering path is never taken. Piggyback the order tracking (and the "add" op construction for new keys) onto the two passes the algorithm already needs to detect removed/added keys, and walk the fast path's recursion in lockstep with the precomputed common-key list instead of re-querying `target`. This restores diff() to its pre-existing lookup count; benchmarked at n=1000 keys, ordered_json::diff() was roughly 2x slower than baseline before this change and is back within noise of baseline after it. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Preserve diff()'s original op ordering and fix a slow-path deletion gap Splitting removed-key detection and common-key recursion into separate passes (for the earlier lookup-count fix) changed the emitted patch's op order: all "remove" ops now came before all recursive per-key diffs, instead of interleaved in source's iteration order as the original implementation did. This broke docs/mkdocs/docs/examples/diff.output's exact-match CI check (ci_test_examples) even though the patch was still semantically correct. Defer "remove" emission into the same walk that does the recursive diffs, so common keys and deleted keys are interleaved in source order again, matching historical output. While restructuring that walk, the reordering ("slow path") branch was only emitting "remove" for keys common to both objects, never for keys present in source but genuinely absent from target -- a key deleted alongside an actual reorder would silently survive the patch. Fixed by removing every source key in the slow path (both deleted and common keys need removing there; common keys are then re-added in target's order). Verified with a targeted reorder+deletion case and a fresh 20,000-case round-trip fuzz run (0 failures). Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
5a05627b1f | 🚨 fix warning (#5169) | ||
 Akhilesh Arora
|
93e49decbd |
Fix incomplete-type error in set_parents with ordered_json (#5167)
When iteration_proxy_value<iter_impl<ordered_json>> appears in a context
that requires it to be complete (function or lambda parameter), the
compiler instantiates basic_json<ordered_map> and walks into
set_parents(iterator, typename iterator::difference_type)
while iterator is still incomplete, failing with "invalid use of
incomplete type".
basic_json::difference_type is already std::ptrdiff_t, so just naming
the underlying type directly avoids the dependent lookup. Behavior and
ABI are unchanged. This was the approach suggested in the issue thread.
Added a regression case in unit-ordered_json.cpp using the same trigger
pattern (lambda parameter naming the proxy type).
Fixes #3732
Signed-off-by: Akhilesh Arora <akhildawra@gmail.com>
|
||
|
Niels Lohmann
|
515d994acb |
📄 adjust year (#5044)
Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
54be9b04f0 | 📄 update REUSE (#4960) | ||
|
Niels Lohmann
|
1705bfe914 |
🔖 set version to 3.12.0 (#4727)
Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
f06604fce0 |
Bump the copyright years (#4606)
* 📄 bump the copyright years Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 📄 bump the copyright years Signed-off-by: Niels Lohmann <mail@nlohmann.me> * 📄 bump the copyright years Signed-off-by: Niels Lohmann <niels.lohmann@gmail.com> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> Signed-off-by: Niels Lohmann <niels.lohmann@gmail.com> |
||
|
Niels Lohmann
|
1b9a9d1f21 |
Update licenses (#4521)
* 📄 update licenses * 📄 update licenses |
||
|
Niels Lohmann
|
9cca280a4d | JSON for Modern C++ 3.11.3 (#4222) | ||
|
Niels Lohmann
|
f56c6e2e30 | Update documentation for the next release (#4216) | ||
|
Niels Lohmann
|
836b7beca4 | Fix CI, again (#4083) | ||
 Niels LohmannandFlorian Albrechtskirchinger
|
58bd97e2b1 |
Add clang-tools to required tools for ci_static_analysis_clang (#3724)
* 💚 add clang-tools to required tools for ci_static_analysis_clang * 🚨 update Clang-Tidy warning selection * 🚨 fix Clang-Tidy warnings * 🚨 fix Clang-Tidy warnings * 🚨 fix Clang-Tidy warnings * 🚨 fix Clang-Tidy warnings * 🚨 fix Clang-Tidy warnings * 🚨 fix Clang-Tidy warnings * 🚨 fix Clang-Tidy warnings * 🚨 fix Clang-Tidy warnings * 🚨 fix Clang-Tidy warnings * 🚨 fix Clang-Tidy warnings * 🚨 fix Clang-Tidy warnings (#3738) * ⏪ revert fix * ⏪ revert fix * 🚨 fix Clang-Tidy warnings (#3739) Co-authored-by: Florian Albrechtskirchinger <falbrechtskirchinger@gmail.com> |
||
|
Niels Lohmann
|
9d69186291 | 🔖 set version to 3.11.2 | ||
|
Niels Lohmann
|
f2020da0dd | 🔖 set version to 3.11.1 | ||
|
Niels Lohmann
|
ce0e13ccea | 🔖 set version to 3.11.0 | ||
|
Niels LohmannandFlorian Albrechtskirchinger
|
527da54dcb |
Use REUSE framework (#3546)
* 📄 add licenses * 👷 add REUSE compliance check * 📝 add badge for REUSE Co-authored-by: Florian Albrechtskirchinger <falbrechtskirchinger@gmail.com> |
||
|
Niels Lohmann
|
b21c345179 |
Reorganize directories (#3462)
* 🚚 move files * 🚚 rename doc folder to docs * 🚚 rename test folder to tests |