The MinGW linker of the Windows clang jobs cannot link object files with more
than 32767 sections ("relocation truncated to fit: IMAGE_REL_AMD64_REL32
against `.rdata'"). unit-json_view.cpp reaches that limit as the stack
grows, so its "json_view dump" and "json_view comparison" test cases move
into unit-json_view_dump.cpp. The test generator and has_duplicate_keys()
that both files use move into json_view_test_helpers.hpp.
The new file mentions JSON_HAS_CPP_17, so it is built for C++17 like the file
it was split from.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The custom object key tests from #5328 instantiate a second basic_json
specialization in unit-cbor.cpp and unit-msgpack.cpp. This pushed
unit-msgpack.cpp.obj past 65535 sections in the clang (MinGW) jobs of
the Windows workflow, and linking test-msgpack_cpp11 fails with
"relocation truncated to fit: IMAGE_REL_AMD64_REL32 against `.rdata'".
The GNU linker keeps the section an associative COMDAT section belongs
to in 16 bits (x_associated in include/coff/internal.h), although big
object files store 32 bits. In larger objects it therefore ties the
jump tables of inline functions to the wrong function and discards them
together with that function's duplicate.
Move the four tests unchanged into unit-custom-object-key-type.cpp and
document the limit in windows.yml.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Lookups find the first member of a repeated key again, so set(object, key,
value) assigns that member (keeping the key where it is) and drops the
others, as documented before, and a view taken from object[key] shows the
new value. This undoes the code and documentation changes of 5cbb8e6d6.
Lookups in edited objects (navigation of editable documents) stop at the
first match, like those of parsed objects.
The tests that commit added expect the first member from lookups now. In the
seeded differential test, the documents with repeated keys repeat them after
the real members; the reference holds the first members, which the edits
address and the lookups are compared with, while materialize() is compared
with what parse() makes of the document's dump.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Lookups in objects with 128 members or more return the first member of a
repeated key again, like the linear search of smaller objects. This undoes
the code change of a69542046; its test now expects the first member from
lookups (with and without a table) and the last value from materialize()
and basic_json::parse().
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
dump() writes every member and == resolves duplicate keys as parse()
does, whereas lookups find the first member of a duplicate key.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Looking up the last member of a duplicate key cannot stop at a match, so
every lookup scanned the whole object (1.6 to 3.4 times slower for small
objects). operator[](key), at, find, value, contains, count, and JSON
pointer resolution return the first member again, as yyjson and simdjson
do; materialize() and get<map>() keep the last value, as parse().
The documentation says so in the feature page and on each lookup page,
and explains how to get the value parse() would give. The integer index
templates and the discarded chaining of operator[] stay.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Unflatten in time and memory linear in the pointer depth
#5443 made unflatten() decide between arrays and objects independently
of the iteration order by collecting the pointer prefixes that have a
reference token 0 below them in a std::set<std::vector<string_t>>.
Every such prefix was stored as a copy of all its reference tokens, and
get_and_create() compared whole prefix vectors at every step, so
unflattening a pointer of depth d took time and memory quadratic in d:
a 10,000-level array pointer took 18 s and 1.3 GB, a 100,000-level one
did not finish.
The prefixes are now numbered nodes of a tree, so each is stored once
and get_and_create() follows the tree token by token. The result is
unchanged, including its independence of the iteration order.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Initialize prefix_tree members to satisfy -Weffc++
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Move prefix_tree setup and child insertion into member functions
The constructor now creates the root node, add_child() inserts a
reference token below a prefix and returns the child's number, and
find_child() looks one up for get_and_create().
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
An editable document reads the last member of a repeated key (as the
read-only document does), but set(object, key, value) assigned the first
one: a view taken from object[key] before the call did not show the new
value. It now assigns the last member's value, keeps the key at the
position of its first occurrence (where materialize() puts it), and drops
the other members.
The seeded differential test now also edits documents whose objects
repeat keys and compares every lookup (operator[], at, find, value,
contains, count, and JSON pointers) with the parsed basic_json value.
A targeted test covers duplicates before and after edits that move the
object, in large objects with an index, in moved arrays, and in values
copied from other documents.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Lookups in objects with 128 members or more now return the last member of
a repeated key, like the linear search of smaller objects and like
materialize() and basic_json::parse(). build_object_index let the first
occurrence win, so the same text gave different results depending on the
size of the object.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Flatten deeply nested values without recursing per nesting level
json_pointer::flatten() called itself once per nesting level, so
flatten() on a value nested deeply enough exhausted the call stack.
#5547 and #5548 fixed merge_patch() and diff() from #5393, but flatten()
was left out.
flatten() now walks the value with an explicit stack and keeps the path
in one buffer that grows and shrinks with it. It has a single code path
and no depth limit: the old version built a new path string per child,
so the iterative one is no slower on shallow values and much faster on
deep ones. The output, including the order of an ordered_json result,
is unchanged.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Construct flatten frames in place
Give the frame a constructor so both call sites can use emplace_back, as
suggested in the review; index starts at 0 for every frame.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Keep converted object keys alive while writing UBJSON and BJData
Since #5746, write_ubjson and write_ubjson_iterative pass each object key
to sanitize_utf8_for_write and keep the returned reference. When
object_t::key_type is not string_t but converts to it, the argument is a
temporary that is destroyed at the end of the statement, and the
function returns a reference to it in every case but a sanitized copy,
so the key bytes are read from a dead object (AddressSanitizer:
stack-use-after-scope). Default json and ordered_json are unaffected.
Bind the key to a named object_key_string_t first: a reference when
key_type is string_t, so no copy is added there, and a converted copy
otherwise. A deleted overload of sanitize_utf8_for_write for anything
other than string_t turns a recurrence into a compile error.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Suppress -Wunused-member-function for the converting_key test type
converting_key::data() is only called when JSON_DIAGNOSTICS is enabled.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix clang-tidy findings in the UBJSON/BJData converted-key fix
Suppress hicpp/modernize-use-equals-delete on the deleted
sanitize_utf8_for_write overload: it guards a private helper and must stay
private. Replace the C-style array in the new test with std::array.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
#5787 missed the instantiations spelled as `basic_json <` (astyle's
formatting) or ending in the CustomBaseClass argument. Convert them,
including the binary_t.md example pointed out in review.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
A view is tested with is_discarded(); the lookups in at(), value() and
contains(json_pointer) and the unit tests no longer rely on the explicit
conversion to bool.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Niels Lohmann