The custom object key tests from #5328 instantiate a second basic_json
specialization in unit-cbor.cpp and unit-msgpack.cpp. This pushed
unit-msgpack.cpp.obj past 65535 sections in the clang (MinGW) jobs of
the Windows workflow, and linking test-msgpack_cpp11 fails with
"relocation truncated to fit: IMAGE_REL_AMD64_REL32 against `.rdata'".
The GNU linker keeps the section an associative COMDAT section belongs
to in 16 bits (x_associated in include/coff/internal.h), although big
object files store 32 bits. In larger objects it therefore ties the
jump tables of inline functions to the wrong function and discards them
together with that function's duplicate.
Move the four tests unchanged into unit-custom-object-key-type.cpp and
document the limit in windows.yml.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Unflatten in time and memory linear in the pointer depth
#5443 made unflatten() decide between arrays and objects independently
of the iteration order by collecting the pointer prefixes that have a
reference token 0 below them in a std::set<std::vector<string_t>>.
Every such prefix was stored as a copy of all its reference tokens, and
get_and_create() compared whole prefix vectors at every step, so
unflattening a pointer of depth d took time and memory quadratic in d:
a 10,000-level array pointer took 18 s and 1.3 GB, a 100,000-level one
did not finish.
The prefixes are now numbered nodes of a tree, so each is stored once
and get_and_create() follows the tree token by token. The result is
unchanged, including its independence of the iteration order.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Initialize prefix_tree members to satisfy -Weffc++
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Move prefix_tree setup and child insertion into member functions
The constructor now creates the root node, add_child() inserts a
reference token below a prefix and returns the child's number, and
find_child() looks one up for get_and_create().
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Flatten deeply nested values without recursing per nesting level
json_pointer::flatten() called itself once per nesting level, so
flatten() on a value nested deeply enough exhausted the call stack.
#5547 and #5548 fixed merge_patch() and diff() from #5393, but flatten()
was left out.
flatten() now walks the value with an explicit stack and keeps the path
in one buffer that grows and shrinks with it. It has a single code path
and no depth limit: the old version built a new path string per child,
so the iterative one is no slower on shallow values and much faster on
deep ones. The output, including the order of an ordered_json result,
is unchanged.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Construct flatten frames in place
Give the frame a constructor so both call sites can use emplace_back, as
suggested in the review; index starts at 0 for every frame.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Keep converted object keys alive while writing UBJSON and BJData
Since #5746, write_ubjson and write_ubjson_iterative pass each object key
to sanitize_utf8_for_write and keep the returned reference. When
object_t::key_type is not string_t but converts to it, the argument is a
temporary that is destroyed at the end of the statement, and the
function returns a reference to it in every case but a sanitized copy,
so the key bytes are read from a dead object (AddressSanitizer:
stack-use-after-scope). Default json and ordered_json are unaffected.
Bind the key to a named object_key_string_t first: a reference when
key_type is string_t, so no copy is added there, and a converted copy
otherwise. A deleted overload of sanitize_utf8_for_write for anything
other than string_t turns a recurrence into a compile error.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Suppress -Wunused-member-function for the converting_key test type
converting_key::data() is only called when JSON_DIAGNOSTICS is enabled.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix clang-tidy findings in the UBJSON/BJData converted-key fix
Suppress hicpp/modernize-use-equals-delete on the deleted
sanitize_utf8_for_write overload: it guards a private helper and must stay
private. Replace the C-style array in the new test with std::array.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
#5787 missed the instantiations spelled as `basic_json <` (astyle's
formatting) or ending in the CustomBaseClass argument. Convert them,
including the binary_t.md example pointed out in review.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The fuzzer only parsed a std::string with default options. Also parse an
exact-size byte vector, which has no NUL after its last byte and takes the
bounds-checked path, and derive ignore_comments and ignore_trailing_commas
from the first input byte, comparing against json::parse and json::accept
with the same options.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The check rejects inputs of 0xFFFFFFF0 bytes or more, but the exception
message and the documentation said 4 GiB. Name the limit once
(max_input_size), and state 4 GiB minus 16 bytes in the message and the
documentation. Test the limit with a container that only claims the size.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
reserve(n) and the growth of the index computed n * sizeof(node) without a
check, which wraps around on 32-bit targets for inputs of about 1 GiB and
allocates a too small array. Throw std::bad_alloc for a count beyond the
address space and clamp the growth step to it.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The non-little-endian path of the node writer wrote the integer's native
word over len and next, so len got the high half there. Compose and split
the value explicitly (len is the low half, next the high half); the
little-endian path stays a plain memcpy.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
auto v = json_document::parse(text).root() compiled and left the view
dangling. Delete the overload for rvalue documents, take a named document
in the tests, and document the lifetime rule.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
explicit operator bool meant "refers to a value", which silently differs
from what a basic_json converts to. Use !v.is_discarded() instead.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
json_document::parse(std::move(const_string)) failed to compile with
"no matching read_kind": only a non-const rvalue std::string can be moved
from. Treat a const rvalue as a copied byte container.
parse() does not accept everything BasicJsonType::parse() does: a FILE*
and pointers to or arrays of wide characters are rejected at compile time.
List the supported inputs instead.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
parse(ptr, len) compiled: len converted to allow_exceptions, and ptr was
read as a C string, past the end of a buffer without a terminating NUL.
Delete the overloads of parse, parse_copy, accept, and read that take an
integer other than bool where the flags are expected.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The non-template overloads for double asserted binary64 doubles wherever json.hpp was included, so the library no longer compiled where double is not IEEE 754 binary64 (AVR, -fshort-double). A trait now picks Zmij for any binary64 type, including a long double of that format (MSVC, Apple Arm), and Grisu2 for the others. Remove the unused write_short_decimal, powers_of_ten_16, zmij::decimal, zmij::to_decimal, and shortest_digits(double).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* avoid allocating temporary basic_json for CBOR and MessagePack object keys
Signed-off-by: alexprabhat99 <alexpbara@gmail.com>
* add size() to the custom object key test type
UBJSON and BJData access object keys through size() and c_str()
directly, so the key type now provides both and the comment says why.
Signed-off-by: alexprabhat99 <alexpbara@gmail.com>
* address review: drop key size()/c_str(), test keys below the depth limit
Nothing in the library calls size() or c_str() on an object key, so the
test key type only keeps data(), which JSON_DIAGNOSTICS needs.
The CBOR and MessagePack custom key tests now also nest objects deeper
than detail::recursion_depth_limit(), so keys written by
write_cbor_iterative and write_msgpack_iterative are covered as well.
Signed-off-by: alexprabhat99 <alexpbara@gmail.com>
---------
Signed-off-by: alexprabhat99 <alexpbara@gmail.com>
* Address review comments on #5775 and #5779
Allow new defaulted parameters and new default arguments in the API
stability rules, mention the macro opt-in, and drop the redundant
recompile advice. Describe test-diagnostics-optimized as the regression
test for the fixed#5742.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Document what counts as a breaking change in the API stability rules
Spell out the 3.x compatibility rules in the roadmap: new defaulted
parameters, new default arguments, noexcept/constexpr, template
parameters, parse and dump results, accepted input, key iteration order,
iterator invalidation, implicit conversions, to_json/from_json lookup,
json_sax, value_t enumerators, and documented macros, CMake options and
headers. Also list std::hash values as not part of the public API, and
link the macro overview from the section.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
basic_string_view::_S_compare stores the difference of two lengths in a
signed difference_type on purpose, which -fsanitize=integer reports for
every comparison with a shorter view (test-json_view_cpp17).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Drop the empty braced NSDMIs of the std::string members: old Clang
rejects the defaulted constructor when it is used by a member
initializer before the end of the class definition.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Niels Lohmann
Bas van Dijkandbvandijk
Alex Prabhat Bara