mirror of
https://github.com/nlohmann/json.git
synced 2026-10-07 15:07:13 +00:00
73ceb71c95c93803221c1a4cfa2cec091a23e781
1042
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Niels Lohmann
|
73ceb71c95 |
Add images of json_documents: save() and load()
An image is a document stored so that loading it needs no
parsing: save() writes the node index, the text and the decoded
strings; the static load() reads an image written by save().
load() takes a pointer and size, a borrowed vector, or an owned
rvalue vector; the nodes are copied so they are aligned and can
be edited, while the text and decoded strings stay in the image.
image_check controls how much load() trusts the input: full
checks structure, bounds, strings and numbers, the parser's own
guarantees; bounds checks structure and bounds only; none skips
all checks, for images from a trusted source.
Layout is little-endian only ("NJVI" header, nodes, text, decoded
strings), following the idea of zero-copy formats such as
FlatBuffers and YaFF; the check follows FlatBuffers' Verifier.
New errors: parse_error.116 for a malformed image or a failed
check, type_error.320 for a discarded document or a big-endian
target.
A dedicated fuzzer and 6,000 seeded corruptions, checked under
ASan/UBSan, found and fixed two gaps: unchecked reserved header
fields, and unbounded null/boolean offsets that could make
dump() throw std::length_error.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
|
||
|
Niels Lohmann
|
c6ee5a64be |
Add editable json_documents: set, push_back, insert, and erase
basic_json_document gets a second template parameter, Editable (false by default), plus the aliases json_editable_document, json_editable_view, ordered_json_editable_document and ordered_json_editable_view. Editable documents can change values and structure without rewriting the source text: set()/push_back() on values, keys, array indices and JSON pointers; insert() before an array element; erase() of an object key, array index or JSON pointer. New values and element sequences go into edit storage that the document owns and never moves, so views keep referring to their value across edits and a parsed node never moves. Read-only documents walk the plain node array and are unaffected. Strings are checked for UTF-8 on entry, so dump() of an editable document never throws type_error.316. Binary values cannot be stored (type_error.319). A seeded differential test applies random edits to an editable document and to the equivalent ordered_json and compares both after every step. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
8daec2b596 |
Scan json_view strings with SIMD and index large objects
Speed up json_view's parser with SIMD scanning and a hash table for large objects. Long runs of string bytes are scanned 16 bytes at a time with NEON (AArch64, GCC and Clang) and SSE2 (x86-64), both baseline instruction sets. Keys keep 16 table checks before the vector loop, because their lengths repeat from record to record; string values get 8, because their lengths vary more. Non-ASCII text is validated 16 bytes at a time with simdjson's "lookup4" check (Keiser and Lemire, 2021), with NEON on AArch64 and, on x86-64, with SSSE3. SSSE3 is not part of baseline x86-64, so the check is compiled for SSSE3 with a function attribute and used only where CPUID reports it, which all x86-64 CPUs since about 2011 do; the answer is cached in a statically initialized atomic, so there is no guard of a local static and no global constructor. The same input is accepted either way. JSON_VIEW_NO_SIMD selects the portable code. On x86-64, string runs are now checked vector-first: one SSE2 compare from the first byte finds the end of most keys and short values, instead of a branch per byte for the first 8-16 bytes. AArch64 keeps the byte-wise steps, where a NEON mask costs more and the branches predict well. Entering an object or array no longer stalls: open() stores the parent's frame field by field instead of building it on the stack and reading it back with wider loads, which waited for the narrower stores to retire. Objects with 128 members or more get an open-addressing hash table built when the object closes, so operator[], at(), find(), contains(), count(), value(), and JSON pointers take constant time on average in such objects; of duplicate keys, the first is kept, as for the linear search. The idea comes from Boost.JSON. simdjson is credited in simd.hpp's SPDX block, the README, and license.md. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
da1ca7f9d7 |
Add dump() and comparisons to json_view
Add basic_json_view::dump() and the comparison operators, and read floats from the parser's digit layout instead of rescanning the token. dump(indent, indent_char, ensure_ascii, number_format) writes a value the way ordered_json::parse(text).dump() writes it for the same arguments: members in document order, all of them should a key occur more than once; strings escaped by the same rules, using the library's scanning kernels; floats written with the library's to_chars conversion, so the output equals basic_json's byte for byte; integers copied from the source, where they are already canonical, except -0, which parse() reads as 0. There is no error_handler argument, because the view only holds valid UTF-8. number_format::source copies numbers exactly as they appear in the source (e.g. "1.50", "1E2", "-0"), which basic_json cannot provide. operator<< takes the indentation from the stream width, as for basic_json. The writer walks iteratively, so nesting depth is limited by memory only. operator== and operator!= compare two views, or a view and a basic_json value in either order, by the rules basic_json's operator== uses: numbers compare by value across their types, objects compare by their members with duplicate keys resolved as parse() resolves them, member order matters only where the object type keeps one, and discarded views compare as discarded basic_json values do, including under JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON. Nothing is materialized except single scalars. While parsing, the view now records where the integer digits, the fraction digits, and the exponent of a float token are, so floats and doubles with at most 19 digits are read from that layout with the library's decimal_to_float() instead of rescanning the token. Both round correctly, so the values are those of parse(). get<double>(), materialize(), dump(), and the comparisons all use it. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
77acd4563c |
Add element access, iteration, values, and JSON pointers to json_view
Give basic_json_view the read-only access functions of basic_json: operator[] and at() with keys and indices, front()/back(), find(), contains(), count(), begin()/end() and cbegin()/cend(), items() with structured bindings from C++17 on, and type_name(). Exceptions have the ids and messages of the const functions of basic_json. Where basic_json has undefined behavior the view answers safely: operator[] with a missing key or an out-of-range index returns a discarded view, and front()/back() of an empty container throw invalid_iterator.214. Objects are iterated in document order, and all members are visited; duplicate-key lookups find the first member (as yyjson and simdjson do), while parse(), materialize(), and the map conversions keep the last value, as parse() does. Keys of up to 16 bytes are compared with two overlapping loads. Add value conversions: get<T>()/get_to() for arithmetic types, bool, nullptr_t, strings (std::basic_string copied, string_view_t without a copy), BasicJsonType, views, std::vector, and maps with string keys; get_string() for the string without a copy; number_token() for the number exactly as written in the source; value() with keys and JSON pointers; and operator[]/at()/ contains() with JSON pointers. Everything else, including types with from_json(), goes through materialize() of that subtree. get<T>() of arithmetic types is inlined down to the conversion, so reading an integer needs no call. detail::json_pointer_access exposes a pointer's reference tokens to code outside basic_json. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
759dd2e1b1 |
Add json_document and json_view: node index, parser, and document
Add json_document and json_view, a read-only, zero-copy index of a JSON text, as the first public slice of the zero-copy view (#5295). A parse produces a flat array of 16-byte nodes in document order, one per value and one per object key. Strings stay in the source text; escaped strings are decoded into an arena. Integers are converted while their digits are in the cache; floats keep only their digit layout and are converted on read. Containers store the size of their subtree, so a reader can step over one in constant time. A document makes a handful of allocations, however many values it has. The parser accepts exactly what json::parse accepts, with every combination of ignore_comments and ignore_trailing_commas, with and without a trailing NUL, and under JSON_STRICT_NUL_HANDLING. It is portable C++11 and does not depend on byte order. basic_json_document adds parse, parse_copy, accept, read (reuses a document's memory), root, is_discarded, source, owns_source, node_count, memory_usage, and shrink_to_fit. basic_json_view adds type, the is_* queries, operator bool, size, empty, materialize, and source_offset. A parse error throws the same exception basic_json::parse would throw for the same input, message and position included. detail::abi_config keeps JSON_STRICT_NUL_HANDLING and JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON readable after json.hpp undefines them, in the ABI namespace so they always match the basic_json in use. A NUL byte that ends a // comment is the end of the input, as in parse() since #5696. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
bfa4886f0f |
Write doubles with the shortest digits (Żmij), digits in registers
Write doubles with the conversion of Zmij by Victor Zverovich (MIT), ported to C++11 (detail/conversions/zmij.hpp). It finds the shortest decimal that reads back as the same double, and the closest one if there are several. Grisu2, used until now, is fast but not always shortest: it sometimes writes a 17th digit where 16 suffice, or a last digit that is not the closest. The layout is unchanged (1.5, 100.0, 1e+100, -0.0); float keeps Grisu2. Digits are converted eight at a time with the BCD conversion of Xiang JunBo, as in Zmij, and written with one byte swap per eight digits and fixed-size moves instead of per-digit loops. Leading and trailing zeros are counted from those bytes. to_chars() uses a local buffer when the caller's is shorter than the 41 bytes this may write. The powers of ten come from the number-parsing table, adjusted where it holds values rounded up, and extended with Zmij's compressed tables beyond 10^308. write_shortest() converts its 16 digits in one vector register (SSE2 on x86-64, NEON on 64-bit Arm, both baseline) and inserts the decimal point inside the register, avoiding a store-forwarding stall that cost about 25% of the time to write a double. dump() writes floats and integers straight into the serializer's write buffer instead of copying them from a member buffer, and small integers eight digits at a time. read_eight_bytes() and parse_eight_digits() are marked always-inline, which GCC had been calling out of line in the number-parsing loops. Of one million random doubles, about 0.14% are now written with different digits, always to a value that still reads back as the same double. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
68d61b6aef |
Speed up the lexer: own float parser, string scan, and \u table
Give the library its own correctly rounded float converter for binary32 and binary64 (IEEE 754), and speed up the lexer's string and escape scanning. The converter splits a number token into sign, significand, and decimal exponent, then tries Clinger's fast path, then a templated Eisel-Lemire step, and falls back to an exact big-integer digit comparison for tokens with more than 19 significant digits whose two candidate values round differently. This replaces std::from_chars and strtod/strtof for both formats, so parsed values no longer depend on the C/C++ library or the current locale. The strtold fallback kept for other long double formats (x87, binary128) now also copies a multi-byte decimal point correctly, fixing #5660. eisel_lemire() and decimal_to_float() are always inlined so callers keep the whole conversion in their hot loop. The string-scanning kernels in string_scan.hpp find a stop byte with the trailing-zero count of the SWAR mask instead of a byte loop, and scalar_string_bulk_run() validates a run of multi-byte UTF-8 sequences one after another instead of re-searching after each one. get_codepoint() decodes a contiguous \uXXXX escape with one table lookup per byte instead of four range-checked get() calls; the streaming path and all error positions are unchanged. Adds 508 generated hard float-parsing cases with expected binary32 and binary64 bits, and kernel-comparison tests for the string scans and the escape table against byte-by-byte references. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
367336c83d |
Fix CI on develop after #5585 (#5779)
* Fix CI on develop after #5585 - test-diagnostics-optimized: -O3 makes GCC's -Winline and -Wsuggest-attribute=pure/const warnings fire with the ci_test_gcc flag set; turn them off for this test. - test-diagnostics-optimized: suppress Clang's -Wexit-time-destructors for the static table in to_json. - Infer: raise pulse-max-disjuncts from 20 to 40. With the default, Pulse loses the stored type in basic_json::replace_value() and reports false null dereferences of get_ptr() results in unit-pointer_access.cpp. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Ignore Infer's false USE_AFTER_DELETE in ordered_map::erase Infer's std::string model keeps the buffer of a moved-from string, so the destroy-and-reconstruct loop in erase(first, last) looks like it destroys a buffer twice. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Mark throw_on_discarded()'s parameters as used without exceptions With JSON_NOEXCEPTION, JSON_THROW expands to std::abort(), so Clang's -Wunused-parameter breaks test-disabled_exceptions (since #5761). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Skip the span_input_adapter sax_parse checks with deleted deprecated functions The #5676 regression test (#5740) calls the deprecated sax_parse(span_input_adapter&&, ...), which JSON_DELETE_DEPRECATED_FUNCTIONS deletes, so ci_test_delete_deprecated_functions failed to build. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fall back to the first entry in test-diagnostics-optimized's to_json clang-tidy (clang-analyzer-security.ArrayBound) flagged it->second for a value not in the table. Use the same fallback as NLOHMANN_JSON_SERIALIZE_ENUM; the test still fails with -Werror=array-bounds on the headers from before #5585. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix clang-tidy findings in tests from #5762 and #5774 - unit-regression2.cpp (#5762): const/auto for the destroy() test values; NOLINT the intended copy in check_destroy_edge_case(). - unit-serialization.cpp (#5774): build the expected strings with += instead of chained operator+ (performance-inefficient-string-concatenation). Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
30e542e52a |
Declare the namespace-scope constants in to_chars.hpp inline (#5776)
MSVC warns with C5260 that kAlpha and kGamma have internal linkage when the header is used as a header unit or through a module. JSON_INLINE_VARIABLE makes them inline variables from C++17 on. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
24dd7c63f0 |
Re-amalgamate after #5740 (#5777)
#5740 changed include/nlohmann/json.hpp without regenerating single_include/nlohmann/json.hpp. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
1c754cfe31 |
Copy a pair-shaped array value under JSON_BRACE_INIT_COPY_SEMANTICS (#5701)
With JSON_BRACE_INIT_COPY_SEMANTICS enabled, single-element brace
initialization from a JSON value decided whether to copy the value or
build an object by inspecting the value's runtime shape: a two-element
array whose first element is a string, such as ["key", 42], was turned
into an object instead of being copied. This made the behavior depend
on the element's content, and it did not distinguish an existing value
of this shape from a nested braced pair written in the source, such as
the inner {"key", "value"} of {{"key", "value"}}.
json_ref now records whether it was constructed from a braced list
(true only for the std::initializer_list<json_ref> constructor used
for nested braced lists) or from a value. The initializer-list
constructor uses this to copy or move a single non-braced-list element
before deciding whether the list describes an object, so a JSON value
is always copied regardless of its shape, while a braced pair written
in the source still creates an object.
Fixes #5662.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
|
||
 Niels LohmannandJoseph.Demarest
|
ff6f3d7d4a |
Reject nested indefinite-length CBOR string chunks (#5766)
* fix(cbor): reject nested indefinite string chunks Signed-off-by: Joseph.Demarest <joseph@demarest.dev> Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Address review comments on nested indefinite-length CBOR strings Rename is_chunk to inside_indefinite, update the stale test section names, and use lowercase comments like the surrounding code. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Joseph.Demarest <joseph@demarest.dev> Signed-off-by: Niels Lohmann <mail@nlohmann.me> Co-authored-by: Joseph.Demarest <joseph@demarest.dev> |
||
|
Niels Lohmann
|
21a69230bd |
Create a value before giving it its type (#5585)
* Create a value before giving it its type Squashed onto develop from: - Create a value before giving it its type - Skip the failed-allocation test when exceptions are disabled - Keep the created pointer rather than an uninitialized json_value - Skip the vector<bool> failed-allocation check for VS 2015 with iterator debugging - Test the remaining to_json overloads with a failing allocation - Skip the to_json allocation-failure section on VS 2015 Debug - Fix false GCC -Warray-bounds error with JSON_DIAGNOSTICS at -O3 (#5744) Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Store the new value with a helper in all to_json constructors Every external_constructor<>::construct now creates the new value first and hands it to basic_json::replace_value(), which destroys the old value, stores the new one before setting its type (as elsewhere in this PR), sets the parents, and checks the invariant. The std::vector<bool> and std::valarray overloads use array_t's range constructor, which the other array overloads already rely on. Range views keep their loop, as begin() and end() of a view may have different types. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
 
|
162e13b86f |
Add with_*_t alias templates to create basic_json types with changed template parameters (#5758)
* Add helper types to make it easier to create a basic_json type with modified template parameters Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Rename with_changed_*_t aliases to with_*_t and merge integer/unsigned aliases Per review discussion on #3898 between gregmarr and nlohmann: - rename with_changed_X_t to with_X_t for brevity - replace the separate with_changed_integer_t/with_changed_unsigned_t aliases with a single with_integers_t<NumberIntegerType2, NumberUnsignedType2> - add @sa doc comment links for the upcoming documentation page Co-authored-by: Raphael Grimm <1005058+barcode@users.noreply.github.com> Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Add documentation for the with_*_t member alias templates Add docs/mkdocs/docs/api/basic_json/with_t.md documenting with_object_t, with_array_t, with_string_t, with_boolean_t, with_integers_t, with_float_t, with_allocator_t, with_json_serializer_t, with_binary_t and with_base_class_t, with an accompanying example, and link the page from the basic_json member types list and the mkdocs navigation. Co-authored-by: Raphael Grimm <1005058+barcode@users.noreply.github.com> Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Add tests for the with_*_t member alias templates Check with std::is_same that each with_*_t alias produces the expected basic_json type, and that with_string_t keeps nlohmann::ordered_map as the object type when used on ordered_json. Co-authored-by: Raphael Grimm <1005058+barcode@users.noreply.github.com> Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix with_t nav entry and document chaining of the with_*_t aliases Indent the with_t entry in mkdocs.yml so it is listed under basic_json, explain that the aliases can be chained and work on ordered_json, and test both, including json::with_object_t<ordered_map> == ordered_json. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Add docset entry for basic_json::with_t Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> Co-authored-by: barcode <barcode@example.com> Co-authored-by: Raphael Grimm <1005058+barcode@users.noreply.github.com> |
||
 fhgffy
|
69874e4544 |
Fix NUL bytes in UBJSON/BJData high-precision numbers (#5760)
* Fix NUL-terminated UBJSON and BJData high-precision payloads Signed-off-by: fhgffy <102001626+fhgffy@users.noreply.github.com> * Use English comments for the high-precision NUL fix Signed-off-by: fhgffy <102001626+fhgffy@users.noreply.github.com> * Track NUL bytes while reading high-precision payloads Signed-off-by: fhgffy <102001626+fhgffy@users.noreply.github.com> * Drop dates from code comments. Signed-off-by: fhgffy <102001626+fhgffy@users.noreply.github.com> * Report a NUL in a high-precision number where it is read Return the parse error from the read loop so the byte offset points at the NUL, and drop the separate check after lexing. Signed-off-by: fhgffy <102001626+fhgffy@users.noreply.github.com> * Use fixed text for the high-precision NUL error 2026-10-06: Use the known zero byte directly instead of formatting and concatenating it. Preserve the SAX token, error message, and byte offset. Signed-off-by: fhgffy <102001626+fhgffy@users.noreply.github.com> --------- Signed-off-by: fhgffy <102001626+fhgffy@users.noreply.github.com> |
||
|
fhgffy
|
9b179cee1e |
Handle all value_t enumerators in has_no_children() (#5770)
2026-10-07: add the scalar enum labels already handled by the default branch so -Werror=switch-enum builds succeed. Keep the existing behavior and synchronize the generated single header. Signed-off-by: fhgffy <102001626+fhgffy@users.noreply.github.com> |
||
|
Niels Lohmann
|
6d4543e743 |
Fix destroy() for ObjectTypes without reverse iteration (#5767)
The non-recursive destroy walk from #5762 picked an object's last child via object_t::rbegin() and std::prev(end()). Neither is available for every ObjectType: no_key_compare_map in unit-custom-object-type.cpp has no rbegin(), so develop no longer compiles that test, and hash maps such as std::unordered_map only have forward iterators. The walk can take an object's children in any order, as long as it finds the same child again while the object is not modified in between. So objects with bidirectional iterators keep using their last child (O(1) to remove from vector-based maps like ordered_map), and objects with forward-only iterators use begin() instead. No reverse iteration or rbegin() is needed any more, and the walk stays allocation-free. Adds a forward-only ObjectType to the tests, destroyed both with mixed nesting and 100000 levels deep. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
  Michiel van SlobbeandMichiel van Slobbe
|
5ecb704f6b |
Speedup; check for the expected separator before the lexer's token switch (#5592)
* Check for the expected separator before the lexer's token switch After a key the parser expects ':', after a value usually ','. Test for that character first instead of going through scan()'s switch, which compiles to an indirect jump. Any other character takes the old path, so tokens and error messages are unchanged. Parsing 6.3% faster with GCC 15.2 and 2.7% with Clang 22.1 (geomean of the ParseString, ParseFile and ParseIndented benchmarks). Signed-off-by: Michiel van Slobbe <michiel.van.slobbe@gmail.com> * Improvement: address PR comments Signed-off-by: Michiel van Slobbe <michiel.van.slobbe@gmail.com> * fix: address comments Signed-off-by: Michiel van Slobbe <michiel.van.slobbe@gmail.com> * Fix clang-tidy bugprone-signed-char-misuse in scan_expecting Convert the expected separator through unsigned char before storing it as char_int_type. The generated code is unchanged. Signed-off-by: Michiel van Slobbe <michiel.van.slobbe@gmail.com> * Use raw string literals in the separator comment tests Signed-off-by: Michiel van Slobbe <michiel.van.slobbe@gmail.com> --------- Signed-off-by: Michiel van Slobbe <michiel.van.slobbe@gmail.com> Co-authored-by: Michiel van Slobbe <michiel.van.slobbe@gmail.com> |
||
  
|
0a365865f9 |
Make basic_json destruction allocation-free and non-recursive (#5762)
* Use the provided allocator in destroy() (#4842) Uses the provided allocator to allocate the stack used to avoid recursion in the destroy() implementation used by ~basic_json. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Test that the destructor uses the provided allocator Adds a regression test for #4842: destroying a nested array or object must allocate its temporary stack through the basic_json allocator, not std::allocator. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix allocation failure during JSON destruction Signed-off-by: Michael Sam <michaelsam94@users.noreply.github.com> Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Make json_value::destroy() non-recursive and allocation-free destroy() used to flatten a nested array/object into a heap-allocated std::vector to avoid recursing per nesting level. That vector could itself throw bad_alloc under memory pressure, and since it now used the basic_json's own allocator (#4842), a failing allocator supplied by the caller made this more likely, not less. An exception thrown from inside ~basic_json(), which is noexcept, terminates the program (#5135). Replace the vector-based stack with a pointer-reversal walk that visits the tree without recursing per level and without allocating anything: cur is the array/object currently being emptied, prev is its parent (or null at the top). A parent's last child slot doubles as storage for that parent's own parent link while we are below it, so no extra memory is needed. A child is only ever removed once it is a scalar or an empty array/object, which neither allocates nor recurses more than one level deep. take() moves m_data between these locals directly, bypassing set_parents()/assert_invariant() (the former is O(#children) per call under JSON_DIAGNOSTICS, which would make the walk quadratic otherwise). This also removes the std::vector<basic_json, allocator_type> stack added by #4842, so the extra allocations it introduced disappear along with it. Co-authored-by: Michael Sam <9461037+michaelsam94@users.noreply.github.com> Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Test that destroy() performs no allocation, even under memory pressure Update the #4842 regression test: it used to check that destroying a nested array/object made at least one allocation through the provided allocator (the old flattening stack). Now that destroy() does not allocate at all, assert the opposite: zero allocations, deallocations only. Rework the #5135 regression test to use a dedicated failing/counting allocator instead of overriding the process-wide ::operator new and ::operator delete, which affected every allocation in the whole unit-regression2 binary rather than just the values under test. Keep the original small repro as one case, and add deep (100000 levels) and wide-and-deep nested array/object/ordered_json cases, all destroyed while every further allocation is made to fail: the destructor must complete without allocating, without throwing, and without leaking. Co-authored-by: Michael Sam <9461037+michaelsam94@users.noreply.github.com> Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Refactor destroy() for readability and add edge-case tests Apply review feedback from Greg Marr on the json_value::destroy() non-recursive, allocation-free destruction walk (#5135): - last_child() now uses object->rbegin()->second instead of std::prev(object->end())->second; pop_last_child() keeps std::prev(end()) since erase() needs a forward iterator. - is_empty_container() becomes has_no_children(), a switch that returns true for every non-container type as well as empty array/object, simplifying the "scalar or already-empty child" check at the call site. The local variable `last` is renamed to `cur_last_ref` for clarity. - free_container() asserts the array/object is already empty before freeing it, and the object branches assert the expected type. - destroy(value_t t) is now a thin dispatcher to destroy_string(), destroy_binary(), and destroy_container(t), each handling its own "not initialized" check and sharing the simple cases first in the switch. - destroy_container() moves the top-level container into the local stand-in via a plain swap of the json_value union, instead of a manual copy plus clearing array/object by hand. - The "cur has no children and there is no parent" case now frees cur and returns immediately, so the main loop is a plain while (true) with no trailing code after it. Also adds edge-case tests for both json and ordered_json (mixes of empty/non-empty arrays and objects, container children in first/last position, single-element chains, top-level empty containers, and destruction via erase()/assignment), plus a mixed-tree case in the "destructor performs no allocation" test. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Make the destroy() walk helpers private They modify basic_json internals without maintaining its invariants and are only meant for destroy_container(), so they no longer need to be reachable from the rest of basic_json. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> Signed-off-by: Michael Sam <michaelsam94@users.noreply.github.com> Co-authored-by: Vesko Karaganev <vesko.karaganev@gmail.com> Co-authored-by: Michael Sam <michaelsam94@users.noreply.github.com> Co-authored-by: Michael Sam <9461037+michaelsam94@users.noreply.github.com> |
||
 
|
5379e04ce4 |
Throw type_error.321 when serializing discarded values to binary formats (#5761)
* Throw type_error.321 when serializing discarded values to binary formats The CBOR, MessagePack, UBJSON, BJData, and BSON writers silently skipped the payload of a value_t::discarded value nested in an array or object, while still writing its slot in the element/member count (and, for BSON, its entry header), producing a binary document whose declared size does not match what was actually written. Throw type_error.321 instead, for a discarded value anywhere in the tree, including at the top level. Rewritten from the original PR against the current (non-recursive option aside) binary_writer.hpp, which has changed substantially since this was first proposed; the out_of_range.412 MessagePack size check and unrelated test reformatting from that PR are dropped as out of scope here. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Document and test type_error.321 for discarded binary values Add docs for the new exception (home/exceptions.md and the Exceptions sections of to_cbor/to_msgpack/to_ubjson/to_bjdata/to_bson) and test coverage for a discarded value nested in an array or object, nested deeper, and (for UBJSON/BJData) inside an optimized same-type array, for each of CBOR, MessagePack, UBJSON, BJData, and BSON. Adjust the three pre-existing "discarded" tests that asserted the old silent behavior (empty/short output) to expect type_error.321 instead. Co-authored-by: ameliabarnabyhub <312084480+ameliabarnabyhub@users.noreply.github.com> Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> Co-authored-by: ameliabarnabyhub <ameliabarnabyhub@users.noreply.github.com> Co-authored-by: ameliabarnabyhub <312084480+ameliabarnabyhub@users.noreply.github.com> |
||
|
Niels Lohmann
|
8a142bc436 |
Fix CI on develop after #5600, #5607, and #5755 (#5764)
* Fix CI on develop after #5600, #5607, and #5755 - binary_reader: cast the result of -1 - number back to number_integer_t, because a number_integer_t narrower than int is promoted to int, which GCC's -Warith-conversion rejects (ci_test_gcc, ci_test_standards_gcc) - JSON_DELETE_DEPRECATED_FUNCTIONS: declare the deleted stream operators as function templates at namespace scope, because GCC < 5 rejects deleted friend functions ("can't initialize friend function") and Clang 7-9 report a redefinition when a class template has a deleted friend function - docs: give the examples of JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS "Example:" titles and add the page to the docset (style_check) Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Ignore libstdc++'s <format> sign change in the sanitizer job libstdc++ 14's <format> initializes a size_t parameter with -1 (GCC bug 119429), so every std::format call fails ci_test_clang_sanitizer under -fsanitize=integer (test-std-format_cpp20). Exclude only the implicit-integer-sign-change check and only that header via -fsanitize-ignorelist. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix the AppVeyor (MSVC 2015-2019) build - binary_reader: emit_signed/emit_unsigned pass integers that do not fit the number types to emit_float as long double, because MSVC's <cmath> has no integer overloads of std::isfinite (C2668 'fpclassify'), from #5607 - scalar comparisons: the friend operators take the JSON type for their noexcept from their parameter, because MSVC 2015/2017 take basic_json as the class template there (C3203) and MSVC 2019 16.0 does not see member types or template parameters, from #5751 - unit-conversions2: skip the !is_nothrow_constructible static_assert for std::optional on MSVC 2017, which evaluates the conditional noexcept as true (C2607), from #5754 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Re-amalgamate Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Avoid MSVC 2015's C4800 for enums with underlying type bool MSVC 2015 warns about any conversion to bool (C4800), even with an explicit cast, so the enum conversions from #5754 (#5671) failed the AppVeyor build with /WX. Convert to bool by comparing with zero via the new detail::bool_aware_static_cast, and keep doctest from printing the enum in the test. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Skip the #5650 allocator test on MSVC 2015 debug builds MSVC 2015's debug STL constructs the containers' debug proxies through the allocator in noexcept constructors, so countdown_allocator's failing construction crashes test-allocator (SIGSEGV) instead of throwing std::bad_alloc. Use the guard #5585 uses for the same reason. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Skip deleted-function detection checks on MSVC 2015 MSVC 2015 does not treat selecting a deleted function in decltype as a substitution failure, so the detection traits in unit-delete_deprecated_functions (#5755) and the integral-key checks in unit-element_access2 (#5657) report deleted overloads as callable there. Calling them still fails to compile. Skip those checks for _MSC_VER < 1910, and use the stream operators for real in the runtime section, so MSVC 2015 still compiles them with the macro set. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Split the Visual Studio 2017 AppVeyor jobs in two The VS 2017 jobs hit AppVeyor's 60-minute limit per job while still compiling the tests (77 of about 108 test targets after 58 minutes). They pass /std:c++17 for everything anyway, so build only the C++17 variant of each test (JSON_TestStandards=17), and split the unit test files across two jobs each with the new JSON_TestShard=<index>/<count> option, which keeps every <count>-th test file starting at <index>. The extra variants of single test files are built in shard 0 only. CMAKE_OPTIONS is no longer quoted in appveyor.yml, so that it can hold more than one option. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix std::terminate when converting std::optional with MSVC 2017 MSVC 2017 evaluates std::is_nothrow_assignable<json&, const T&> as true even if T's to_json throws, so to_json(json&, const std::optional<T>&) was noexcept there and the exception from #5642's test called std::terminate instead of propagating. Make that conversion never noexcept on MSVC 2017; all other compilers keep the exact condition. The static_asserts on the condition are skipped for MSVC 2017; the runtime check that the exception propagates still runs there. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
23d3b373e1 |
Add JSON_DELETE_DEPRECATED_FUNCTIONS to delete the deprecated functions (#5755)
* Add JSON_DELETE_DEPRECATED_FUNCTIONS to delete the deprecated functions Defining JSON_DELETE_DEPRECATED_FUNCTIONS to 1 (or the CMake option JSON_DeleteDeprecatedFunctions) declares every deprecated function as deleted instead of deprecated, so that code that is not ready for 4.0.0 no longer compiles. A deleted function still takes part in overload resolution, so from_*(ptr, len) cannot silently bind len to the strict parameter of from_*(InputType&&, bool); the roadmap now plans to keep these overloads deleted in 4.0.0 instead of removing them. The legacy discarded-value comparison is left to its own macro. Also update the 4.0 roadmap: add JSON_DISABLE_TUPLE_REFERENCE_CONVERSION and JSON_DELETE_DEPRECATED_FUNCTIONS to the macro table, add the from_bjdata/from_bon8 (ptr, len) overloads to the deprecated functions, document the macro in the migration guide, and fix the docs style check findings (example titles, missing docset entry for JSON_STRICT_BINARY_UTF8). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Declare each deprecated function once and guard only its body Instead of repeating every deprecated declaration in an #if JSON_DELETE_DEPRECATED_FUNCTIONS branch, keep one declaration (with its deprecation attribute) and switch only between "= delete;" and the function body. Suggested by @gregmarr in the review. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
c5a7a4b46d |
Add deprecated from_bon8/from_bjdata(ptr, len) overloads (#5688)
from_bon8(ptr, len) and from_bjdata(ptr, len) had no overload for a pointer and a length, unlike from_cbor/from_msgpack/from_ubjson/ from_bson. The call instead bound to from_*(InputType&&, bool strict), which read ptr as a NUL-terminated C string via strlen and silently converted len to the strict flag. Data containing a 0x00 byte was cut off there; data without one was read past the end of the buffer. Add a deprecated (ptr, len, strict, allow_exceptions) overload for each function that forwards to (ptr, ptr + len, ...), matching the existing deprecated overloads of the other four binary readers. Since neither function ever had this overload, the deprecation is declared as of version 3.13.0, the next unreleased version, rather than the version each function was originally added in. Fixes #5648. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
 Niels LohmannandMuhammad Amir bin Mohamad Ghazaly
|
8c1f60a45e |
Store maps with enum keys as objects (opt-in) (#5600)
* Store maps with enum keys as objects (opt-in) Maps with enum keys, such as std::map<E, T>, are stored as arrays of [key, value] pairs, because enums are not convertible to the string type of object keys - even if NLOHMANN_JSON_SERIALIZE_ENUM maps them to strings (#4378). The new JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS macro stores them as objects instead, converting each key with the enum's to_json. It applies to any map-like type with enum keys (std::map with any comparator, std::unordered_map, ...). A key that does not convert to a string throws type_error.302, and two keys converting to the same string throw the new type_error.318, rather than losing an entry. The macro changes the output of inline functions, so it is part of the ABI tag (_ekmo). Reading needs no macro: std::map and std::unordered_map with enum keys are now also read from objects, converting each key with the enum's from_json. That input was rejected before, and arrays of pairs are still read, so data written either way can be read. This supersedes #4531, which first proposed storing these maps as objects. Co-authored-by: Muhammad Amir bin Mohamad Ghazaly <amirghaz@umich.edu> Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Keep multimaps with enum keys as arrays of pairs With JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS, is_enum_keyed_map also matched std::multimap and std::unordered_multimap. Storing them as objects throws type_error.318 as soon as a key occurs twice, which is the normal case for a multimap, so such values could no longer be serialized at all once the macro was enabled, although they are stored losslessly as arrays of [key, value] pairs without it. Exclude maps with non-unique keys from is_enum_keyed_map. They are detected by insert(value_type) returning an iterator rather than a pair<iterator, bool>. Map-like types without such an insert() are still treated as before. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Move the default enum-keyed map tests out of unit-conversions.cpp The Windows clang 20.1.8 job (MinGW, Debug) failed to link test-conversions_cpp17 with "relocation truncated to fit: IMAGE_REL_AMD64_REL32 against .rdata": the object file of unit-conversions.cpp was already close to the limit, and the new "maps with enum keys" test case pushed it over. windows.yml asks to keep these objects small by splitting test files. Move the test case unchanged into unit-enum_keyed_maps_default.cpp, with the three enums it needs. It still honors a -D flag for JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS, as before. unit-conversions.cpp is back to its state on develop. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Build the enum-keyed map test object instead of parsing it ci_test_diagnostic_positions failed in unit-enum_keyed_maps_default.cpp: with JSON_DIAGNOSTIC_POSITIONS, a parsed value adds its byte range to the exception message ("(bytes 0-7) type must be array, but is object"), so the exact-message checks did not match. Build the object in memory, like unit-custom-array-type.cpp does. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> Co-authored-by: Muhammad Amir bin Mohamad Ghazaly <amirghaz@umich.edu> |
||
|
Niels Lohmann
|
6b4b825af2 |
Handle numbers that do not fit narrow number types in the binary readers (#5607)
* Handle numbers that do not fit narrow number types in the binary readers With custom number types narrower than the values in a binary document, for example basic_json<..., std::int32_t, std::uint32_t, float>, every binary reader (CBOR, MessagePack, UBJSON, BJData, BSON, BON8) passed the decoded number to the SAX interface with an implicit conversion: the integer 5000000000 silently became 705032704, and a finite double such as 1e300 became infinity. The lexer handles the same values in JSON text: an integer that fits neither integer type is stored as number_float_t, and a finite number that overflows number_float_t is rejected with out_of_range.406. Pass every number read from binary input through three helpers that apply the lexer's rules: - emit_signed(): number_integer_t, else number_unsigned_t for a non-negative value, else number_float_t - emit_unsigned(): number_unsigned_t, else number_float_t - emit_float(): out_of_range.406 if a finite value overflows number_float_t; infinity and NaN are passed on For consistency, a CBOR negative integer below the range of number_integer_t is now stored as number_float_t, like a too small integer in JSON text, instead of being rejected with parse_error.112. With the default number types, this is the only change in behavior. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix MSVC and clang 3.5 in the narrow number type test MSVC types 3000000000 and 5000000000 as unsigned long, so json(-3000000000) triggered C4146 (unary minus on an unsigned type), which /WX turns into an error. Use LL literals, as elsewhere in the tests. clang 3.5 cannot convert the lambdas in the braced initializer of the format table to function pointers. Use named functions instead. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Check integer-to-float fallbacks for overflow in the binary readers emit_signed, emit_unsigned, and the CBOR negative integer fallback now pass their number_float_t fallback through emit_float, so a value that overflows number_float_t is rejected with out_of_range.406 like a floating-point value, instead of silently becoming infinity. This only matters for a number_float_t that cannot represent 2^64, such as a half-precision type. The CBOR value -1 - n is computed as long double so that emit_float sees a finite value. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Use the input_format member instead of passing the format to binary_reader helpers The helpers (get_number, get_to, get_string, get_binary, get_bytes, emit_signed, emit_unsigned, emit_float, unexpect_eof, exception_message) are members of binary_reader, which already stores the format it was constructed with, so the parameter was redundant. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
d8d47be4a5 |
Fix CI on develop after merging the ready-to-merge PRs (#5754)
* Keep the serializer conversion for objects whose keys cannot be converted #5591 added a test converting nlohmann::json into a basic_json whose string type cannot be constructed from std::string. That instantiates convert_iteratively(), whose members.emplace_back(next.key(), ...) needs exactly that key conversion, and broke the build of unit-alt-string. Dispatch on the key's constructibility and leave such conversions to the serializers, as the levels above the nesting bound already do (#3425). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix the remaining CI failures on develop - unit-wstring: with a 16-bit wchar_t (Windows), a lone surrogate is reported as the ill-formed byte 0xFF since #5704; the std::wstring expectations still had the previous <U+0000>. - ci_single_binaries: json_literals.hpp (#5610) and json.hpp include each other on purpose, and IWYU, not following the cycle, asks to replace json.hpp with json_fwd.hpp. Report its findings without failing the build, as already done for json.hpp. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix the library warnings and noexcept specifications from the merged PRs - binary_reader: rename the error_handler constructor parameter, which shadowed the member (-Wshadow, -Wshadow-field-in-constructor; #5746) - basic_json(copy_construct_tag, ...): declare it noexcept when copying the base class is (GCC 16 -Wnoexcept; #5690) - the scalar-on-left legacy comparison operators: noexcept only when converting the scalar is, like their member counterparts (#5682, #5751) - compare_leaves: use std::is_eq/is_lt/is_gt instead of comparing a std::partial_ordering with 0 (-Wzero-as-null-pointer-constant; #5686) - serializer: silence MSVC C4127 for the EnsureAscii template parameter (#5741, #5746) - clang-tidy: return the sanitized reference in binary_writer, take the key of ordered_map::find_impl by const reference (#5727), and mark the switches over parse_array_index (#5728) - ordered_map: keep <memory> for std::allocator (IWYU) Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Split unit-conversions.cpp so MinGW can link it clang 18 with the MinGW linker failed to link test-conversions_cpp17 ("relocation truncated to fit: IMAGE_REL_AMD64_REL32"). As windows.yml recommends, keep the objects small by splitting the test file. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix the tests added by the merged PRs for all CI configurations - discard the results of dump() and from_*() in CHECK_THROWS with utils::ignore_return_value (GCC -Werror=unused-result) - give unit-bson's huge_string_t a default constructor (MSVC C2512, GCC 5, clang 3.5) - unit-disabled_exceptions: use the literals namespace when the global UDLs are off (ci_test_noglobaludls; #5700) - unit-binary_utf8_strict: expect the JSON pointer prefix with JSON_DIAGNOSTICS (#5741) - skip the tests that rely on exceptions under JSON_NOEXCEPTION (#5678, #5732) - clang-tidy and clang -Werror: static test data, CAPTURE(...);, const-correctness, use-after-move alias, unused conversion operator, a missing <iterator> include Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Title the macro examples and add JSON_STRICT_BINARY_UTF8 to the docset The documentation style check requires "Example: ..." titles on pages with several examples (#5741, #5591) and a docset entry for every macro page. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Regenerate BUILD.bazel and nlohmann_json.natvis Signed-off-by: Niels Lohmann <mail@nlohmann.me> #5746 added detail/output/error_handler.hpp and #5741 the json_abi_sbu8 ABI tag. * Install libidn11 for the CMake 3.5.0 binary in ci_cmake_flags Signed-off-by: Niels Lohmann <mail@nlohmann.me> #5733 moved ci_cmake_options from ubuntu:focal to ubuntu:24.04, which no longer ships libidn.so.11; the CMake 3.5.0 release binary links against it, so every ci_cmake_flags run has failed since. Install focal's libidn11 package for that matrix entry only. * Suppress Infer's false STACK_VARIABLE_ADDRESS_ESCAPE in get_impl get_impl() returns its local by value. A test added by the merged PRs instantiates it with a type Infer misreads, so ci_infer reported the 2021 code for the first time. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
73e9eae3c1 |
Add an error_handler parameter for UTF-8 to the binary readers and writers (#5746)
Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
f56b418c56 |
Follow each binary format's UTF-8 rule: strict writers (CBOR/UBJSON/BJData/BSON), lenient readers (#5741)
Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels LohmannandRaphael Grimm
|
40021f38fb |
Add basic_json::as_base_class and document name conflicts with custom base classes (#5589)
* Add basic_json::as_base_class and document name conflicts with custom base classes Members of basic_json hide members of a custom base class with the same name, and future releases may add members that hide ones accessible today. Document this in json_base_class_t and add as_base_class() to reach hidden members without spelling out the cast. Also make json_base_class_t a public member type. It was documented since 3.12.0, but declared private, so users could not name it. Supersedes #3899. Co-authored-by: Raphael Grimm <1005058+barcode@users.noreply.github.com> Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Add as_base_class to the docset search index New public members get an entry in docs/docset/docSet.sql (as done for to_bon8/from_bon8 in #2998). Without it, the Dash/Zeal docset built from the documentation cannot find basic_json::as_base_class. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Silence clang-tidy for the hidden type_name() in the base class test ci_clang_tidy failed with readability-convert-member-functions-to-static on base_class_with_hidden_members::type_name(). It must stay a non-static member: the test shows that it is hidden by the non-static basic_json::type_name() and reachable through as_base_class(). Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> Co-authored-by: Raphael Grimm <1005058+barcode@users.noreply.github.com> |
||
|
Niels Lohmann
|
c5650eaa3c |
Reject malformed UTF-16/UTF-32 units in wide-string input (#5704)
The wide-string input adapter (used for std::u16string, std::u32string, std::wstring, and iterators over 2- or 4-byte character types) passed some malformed code units on to the lexer as values that are neither a byte (0x00..0xFF) nor char_traits<char>::eof(). As a result: - A lone UTF-16 surrogate inside true/false/null was accepted if its low byte matched the expected letter, or ended the input silently if it was the last unit. - A high surrogate followed by a unit that is not its low surrogate swallowed that unit; if the swallowed unit was the newline ending a // comment, the comment silently extended over the next line. - Where wint_t is a signed int (macOS, the BSDs), a negative wchar_t collided with char_traits<char>::eof() (ending the input early) or was truncated to its low byte, depending on its value. The UTF-32 helper now converts the code unit to std::uint32_t before the range checks, so a negative unit reaches the same "emit 0xFF" branch already used for code points above U+10FFFF. The UTF-16 helper now peeks at the next unit before consuming it, and emits 0xFF instead of the raw surrogate when no valid pair is found, matching how ill-formed UTF-8 bytes are rejected elsewhere in the lexer. Fixes #5645. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
38a2db260c |
Remove dead metaprogramming and duplicated code in traits and pointers (#5728)
* Remove unused is_sax and is_detected_convertible detail::is_sax had no user: the parser and the binary reader only use is_sax_static_asserts, so is_sax was a second, unchecked copy of the SAX event list. is_sax_static_asserts asserted boolean(bool) twice in a row, and detail::is_detected_convertible was never used anywhere. Remove all three and include <cstddef> for size_t instead of <cstdint>. Only names in nlohmann::detail are removed; behavior, public API and ABI are unchanged. The diagnostics for an incomplete SAX handler are the same, apart from the duplicated boolean() message. Part of #5708 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Replace meta/logic.hpp with a disjunction trait meta/logic.hpp added a second set of type-level boolean helpers (cxpr_and, cxpr_or, cxpr_not, ...) next to the existing conjunction and negation in type_traits.hpp. It was used only by one static_assert in from_json_tuple_impl, two of its templates were never used, and it was the only header without the license banner and relied on transitive includes for <type_traits>. Add the missing disjunction next to conjunction and negation, use the three in the static_assert, and delete logic.hpp together with its BUILD.bazel entry. same_sign now uses disjunction as well, which resolves the 2022 TODO waiting for such a trait. The static_assert accepts and rejects the same types as before. Only names in nlohmann::detail change; behavior, public API and ABI are unchanged. Part of #5708 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Remove unused would_call_std_* from NLOHMANN_CAN_CALL_STD_FUNC_IMPL Besides detail::result_of_begin/end, which is_range and iterator_t use, the macro defined a namespace detail2 with a tag type, a catch-all overload and would_call_std_begin/end, plus would_call_std_begin/end structs directly in namespace nlohmann. Nothing has used them since they were added in #3020. Reduce the macro to its detail part. Without the trailing struct the ';' after the two invocations would be an empty declaration that -Wextra-semi flags, so drop it. macro_scope.hpp included meta/detected.hpp only for this macro; all users of detected.hpp include it (or type_traits.hpp) themselves, so remove the include. Behavior and ABI are unchanged. The undocumented, untested and unused names nlohmann::would_call_std_begin, nlohmann::would_call_std_end and namespace nlohmann::detail2 are no longer declared. Part of #5708 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Simplify is_ordered_map to reuse has_capacity is_ordered_map re-detected capacity() with a C++03 sizeof/vararg trick right after has_capacity did the same detection through is_detected. For ordered_map, the old trick took the address of std::vector::capacity, which [namespace.std]/6 makes unspecified. Reuse has_capacity instead, which removes the unspecified-behavior pointer-to-std-member and two NOLINT suppressions. Part of #5708 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Remove duplicate const overload of json_pointer::get_checked The const and non-const get_checked() overloads had byte-identical 50-line bodies, differing only in the signature. The remaining template deduces a const-qualified BasicJsonType for const callers, so at(), the out_of_range::create() calls and the bounds check all still work. Part of #5708 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix tautological clause in iter_impl's iterator category assertion The static_assert meant to check the LegacyBidirectionalIterator named requirement had a first clause comparing std::bidirectional_iterator_tag to itself, which is always true and checks nothing; only array_t::iterator was actually being checked, despite the message claiming object iterators were checked too. Drop the tautological clause, reword the message to describe what is actually checked, and note that object_t may use a forward-only iterator as long as reverse iteration and operator-- are unused. The check is intentionally not extended to object_t::iterator, since that would reject object types with forward-only iterators that compile and work correctly today. Part of #5708 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix misplaced and stale comments in JSON_HAS_RANGES and conversions The JSON_HAS_RANGES feature-detection block had its libc++ comment sitting above the clang+libstdc++ branch it does not describe, leaving the libc++ branch uncommented and the clang+libstdc++ branch without its own rationale. Move each comment to sit under its own branch, and give the clang+libstdc++ branch (added in issue 5161) its own one-line reason referencing that issue instead of reusing the libc++ branch's comment. Also fix a duplicated-word typo ("in large in large cpp files") in from_json.hpp, drop two unanswered 2017 design questions left as comments in type_traits.hpp and from_json.hpp that no longer reflect open questions, and correct NLOHMANN_JSON_SERIALIZE_ENUM_STRICT's @since tag from 3.12.0 to 3.13.0, the release it was actually introduced in. Part of #5708 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Support any-rank C arrays in from_json, not just rank 1-4 from_json() for C arrays had four hand-unrolled overloads (rank 1-4, added incrementally in #4262), each with its own nested loops. to_json() already handles any rank recursively, so a rank-5+ C array could be serialized but not read back with get_to()/get<>(). Replace the four overloads with one from_json() SFINAE-constrained on get<remove_all_extents<T>::type>() existing, forwarding to a pair of mutually recursive from_json_c_array_element() helpers: one assigns a non-array element via get<T>(), the other loops over a array element and recurses one dimension at a time. Each dimension still goes through at(), so type_error.304/out_of_range.401 stay unchanged; ranks 1-4 keep their existing behavior and semantics. Adds rank-5 round-trip and mismatched-shape tests to unit-conversions.cpp. Public API: additive only (rank 5+ C arrays become readable). Signed-off-by: Niels Lohmann <mail@nlohmann.me> #5708 item 1 * Move templated_json_throw into nlohmann::detail templated_json_throw() was defined in macro_scope.hpp, which is included outside NLOHMANN_JSON_NAMESPACE_BEGIN, so the helper leaked into the global namespace as ::templated_json_throw with no ABI tag. Unqualified lookup in NLOHMANN_JSON_SERIALIZE_ENUM_STRICT could then bind to a same-named function declared in the user's own namespace instead, which fails to compile with Clang ("does not name a template"). Move the helper next to the exception classes in exceptions.hpp, inside nlohmann::detail, and call it qualified as ::nlohmann::detail::templated_json_throw<...>(...) from both macro expansion sites. Rewrite the doc comment to give the real reason for the helper (JSON_THROW may expand to code that discards its argument, e.g. when exceptions are disabled) and fix the "supress" typo. templated_json_throw was never released (added by #5151 after v3.12.0), so it can be moved freely. Adds a regression test that expands NLOHMANN_JSON_SERIALIZE_ENUM_STRICT inside a namespace declaring its own templated_json_throw. Public API: no change (::templated_json_throw was an unreleased, unintentional global-namespace leak with no callers relying on its location). Overlaps #5698, which rewrites the same two macro call lines; the overlapping hunks are small and should be trivial to reconcile on rebase. Signed-off-by: Niels Lohmann <mail@nlohmann.me> #5708 item 2 * Factor the repeated JSON_HAS_RANGES/MinGW guard into one macro The std::ranges view conversion (excluded on MinGW because of its incomplete C++20 ranges support, #4916) was gated by the same #if JSON_HAS_RANGES && !defined(__MINGW32__) condition at seven independent sites in to_json.hpp and type_traits.hpp, with the MinGW rationale duplicated in two of them and missing from the rest. Since the sites come in matching pairs (one enables is_compatible_range_view and a view-based overload, the other adds the exclusion to the plain-array-type overload), a drift between any pair would produce an ambiguous or missing overload on exactly one platform. Add JSON_HAS_RANGE_VIEW_CONVERSION next to JSON_HAS_RANGES in macro_scope.hpp, combining both conditions with the #4916 reasoning in one place, #undef it in macro_unscope.hpp, and use it at all seven sites. This does not fold the MinGW check into JSON_HAS_RANGES itself: JSON_HAS_RANGES is user-overridable and also gates the enable_borrowed_range specialization in iteration_proxy.hpp, which is not excluded on MinGW. No behavior or public API change: JSON_HAS_RANGE_VIEW_CONVERSION expands to exactly the condition that was previously written out at each site. Overlaps #5585, #5600 and #3575, which touch the same to_json.hpp and type_traits.hpp lines; the change here is a mechanical search-and-replace of the guard condition and should rebase cleanly. Signed-off-by: Niels Lohmann <mail@nlohmann.me> #5708 item 11 * De-duplicate from_json.hpp's map and array-fallback bodies Several from_json() overload pairs in from_json.hpp were copies of each other, so a fix has to be applied twice (as #5681 already does): - from_json(..., std::map&) and from_json(..., std::unordered_map&) for non-string keys had identical 16-line bodies: array check, m.clear(), pair check loop, m.emplace(...). Route both through a new from_json_pair_array_to_map(j, m) helper. - The from_json_array_impl priority_tag<1> and priority_tag<0> fallbacks ran the same std::transform/std::inserter loop, differing only in ret.reserve(j.size()). Merge them into one body and, modeled on the existing from_json_object_reserve, add a from_json_array_reserve pair so the reserve() call is only made for ConstructibleArrayType that support it. Error ids (type_error.302), messages, diagnostic paths ((at(0)/at(1)) and behavior for types with/without reserve() are unchanged; only the duplication is removed. Public API: no change. Overlaps #5681, which changes the "&j" to "&p" line in both map bodies; the shared helper here should make that a one-line change instead of two on rebase. Signed-off-by: Niels Lohmann <mail@nlohmann.me> #5708 item 5 * Unify json_pointer's three array-index parsers array_index(), contains() and get_checked_or_null() each re-implemented the RFC 6901 array-index rules and the size_type range check: array_index() does the canonical parse and throws; contains() (which must not throw, #5395) re-validates every digit by hand and runs its own strtoull/ERANGE check before calling array_index() anyway, parsing every array token twice; get_checked_or_null() wraps array_index() in JSON_TRY/ JSON_INTERNAL_CATCH (detail::out_of_range&) to turn an unrepresentable index into "not found". Add a single private, noexcept parse_array_index(s, idx) returning an array_index_status (ok / leading_zero / not_a_number / unresolved / exceeds_size_type). array_index() becomes a thin wrapper mapping each status to the existing parse_error.106/109 or out_of_range.404/410; contains() and get_checked_or_null() switch on the status directly. This removes contains()'s digit-validation loop and its second strtoull call, and get_checked_or_null()'s JSON_TRY/JSON_INTERNAL_CATCH. Bugfix as a consequence: get_checked_or_null()'s JSON_TRY/ JSON_INTERNAL_CATCH was dead code under JSON_NOEXCEPTION (JSON_TRY expands to "if(true)" and the catch to "if(false)", so JSON_THROW's std::abort() ran unconditionally), meaning value() and contains() would abort instead of returning the default/false for an out-of-range-sized or oversized array index when exceptions are disabled (#5672). Switching on parse_array_index()'s return value instead of relying on an actual throw/catch fixes this: get_checked_or_null() now returns nullptr for array_index_status::unresolved/exceeds_size_type in every build configuration, and still calls JSON_THROW (aborting under JSON_NOEXCEPTION, as before) only for a malformed index (leading_zero/not_a_number), matching its documented @throw list. All existing error ids, messages and diagnostic paths are unchanged; a few reference tokens that used to fail contains()'s manual per-character validation (e.g. "1a") now fail via array_index_status::unresolved instead, with no observable difference since contains() only returns bool. Adds regression tests to unit-element_access2.cpp's "access on array type" section covering value() with an index that exceeds size_type and one with a trailing non-digit, both of which must yield the default value rather than abort/throw. Public API: no change. Overlaps #5700, #5614 and #5692, which touch the contains() and get_checked_or_null() array hunks; this change replaces those hunks with calls into the new shared parser, so a rebase will need to re-apply their token-handling changes (e.g. the empty-token case) on top of the switch statements here. Signed-off-by: Niels Lohmann <mail@nlohmann.me> #5708 item 4 * Regenerate single_include after merging develop The merge commit kept develop's single_include/nlohmann/json.hpp because make amalgamate saw it as up to date. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Address review: switch in array_index, drop redundant inline - json_pointer::array_index() dispatches on array_index_status with a switch, matching the other parse_array_index() caller - drop `inline` from the function templates this PR adds or moves in from_json.hpp - reword a comment that described the change rather than the code Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
1df1e8a845 |
Make cross-string-type basic_json conversion explicit without implicit conversions (#5591)
* Make cross-string-type basic_json conversion explicit without implicit conversions The converting constructor from another basic_json specialization was always implicit, so a value with a different string_t (std::wstring, a string with a custom allocator, ...) silently converted into a temporary, e.g. when passed to a function taking const nlohmann::json&. Such conversions do not produce correct values (#3425), and JSON_USE_IMPLICIT_CONVERSIONS=0 did not catch them. When JSON_USE_IMPLICIT_CONVERSIONS is 0, the constructor is now explicit if the string types differ. Specializations sharing a string type (json and ordered_json, different serializers or object maps) stay implicitly convertible, so the NLOHMANN_DEFINE_TYPE_* macros keep working with nested json members. get<BasicJsonType>() constructs explicitly and works in both modes. Fixes #2649. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Construct explicitly in get_to() and to_json(std::optional) With JSON_USE_IMPLICIT_CONVERSIONS=0 the conversion from a basic_json with a different string type is now explicit, but two library paths still assigned such a value implicitly and failed to compile inside the library: - get_to() with a basic_json target (the #2175 overload) did `v = *this`, so json(42).get_to(alt_json&) broke although get<alt_json>() works. - to_json(BasicJsonType&, const std::optional<T>&) is constrained on std::is_constructible (which accepts the explicit constructor) but did `j = *opt`, so converting a std::optional<alt_json> into a json broke. Both now construct the value explicitly, as get_impl() already does. Also replace static_cast<bool>(JSON_USE_IMPLICIT_CONVERSIONS) with a comparison: clang-tidy's modernize-use-bool-literals rejected the cast of the integer literal the macro expands to, failing ci_clang_tidy. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
a212d3b2e4 |
Preserve the object comparator's state in a deep copy past the nesting bound (#5722)
* Preserve the object comparator's state in a deep copy past the nesting bound copy_object_level(), used by the copy constructor and copy assignment once a value is nested deeper than the iterative deep copy's bound (128 levels, or every copy under JSON_NO_THREAD_LOCAL), built each object's copy with the object type's plain range constructor. That default-constructs the object's comparator instead of copying the original's. For an object type whose comparator carries state, such as a std::map that compares keys case-sensitively only when constructed that way, the copy then ordered - and could even deduplicate - its keys differently from the original. Add detail::is_comparator_constructible_object_type, a detection trait for object types that provide a key_comp() and a constructor taking a range and a comparator, the way std::map does. copy_object_level now dispatches on it: an object type that qualifies gets its copy built with src_object.key_comp() passed along; other object types, such as nlohmann::ordered_map (which has a key_compare for its std::map-like interface, but no key_comp()), keep using the plain range constructor exactly as before. merge_patch and update() were checked for the same pattern; neither is affected, since both only ever add members one at a time to an object that already has its own comparator (or start a brand new default-constructed one), rather than rebuilding an object_t from a range copied out of an existing, possibly custom-comparator object. Fixes #5649. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Keep astyle from padding the create_object_with_comparator templates Spell the negated condition as detail::negation<...> instead of a leading '!', which made astyle spread the template header out. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
1edf0ef041 |
Fix value(json_pointer, default) aborting under JSON_NOEXCEPTION (#5700)
With exceptions disabled (JSON_NOEXCEPTION or -fno-exceptions),
value(const json_pointer&, default) called std::abort() for array
reference tokens that array_index() rejects with out_of_range.404/410:
indices too large to fit size_type, the empty token ("/"), and tokens
like "/1a". With exceptions enabled, the same tokens correctly yielded
the default value, because get_checked_or_null() relied on
JSON_TRY/JSON_INTERNAL_CATCH (detail::out_of_range&) to turn the
exception into nullptr; under JSON_NOEXCEPTION, JSON_THROW aborts
before that catch is ever reached.
get_checked_or_null() now detects those out-of-range tokens itself,
the same way contains(json_pointer) already does (#5495), and only
calls array_index() for tokens that must still raise parse_error.106
or parse_error.109 (e.g. "/01", "/+1"), matching the documented
behavior of value().
Added regression tests to tests/src/unit-disabled_exceptions.cpp
(built with JSON_NOEXCEPTION and -fno-exceptions) and the matching
checks to tests/src/unit-element_access2.cpp for normal exception
mode.
Fixes #5672.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
|
||
|
Niels Lohmann
|
756b28c2b8 |
Keep a NUL byte ending a // comment as the end of input (#5696)
With the default NUL handling (JSON_STRICT_NUL_HANDLING not set), a NUL byte in the input is treated as the real end of input everywhere - except when it immediately ends a `//` comment: scan_comment() matched '\0' as a comment terminator like '\n', so the NUL was consumed as part of the comment and scan() never saw it as end of input; the next get() then kept reading past it. Multi-line comments and JSON_STRICT_NUL_HANDLING=1 were unaffected, since there the NUL is just part of the comment text. Fix scan_comment() to leave the NUL unconsumed (unget()) instead of returning it as part of the comment, so the following scan() reports it as end of input, exactly as for a NUL anywhere else. Fixes #5659. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
49cd427196 |
Copy-construct the base class of a deep copy's elements, not assign it (#5690)
The bounded-descent copy added by #5389 built the elements of a deep copy (nested past the 128-level bound) by default-constructing them and then having copy_metadata() assign their base class afterwards. That assignment is only instantiated for values nested past the bound, but being called from copy_structured() at all meant it was compiled for every copy, so a CustomBaseClass that is copy-constructible but not move-assignable (for example one with a const data member) no longer let its basic_json be copy-constructed, at any depth. copy_array_level() and copy_object_level() now build each element with a private-tag-selected constructor that copy-constructs the base class (and, under JSON_DIAGNOSTIC_POSITIONS, copies the positions) directly, the same way the copy constructor already builds elements within the 128-level bound. Copying a basic_json is therefore back to requiring only a copy-constructible base class, as documented and as it was before #5389; copy assignment is unchanged and still requires an assignable one. Fixes #5674. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
b730946432 |
Fix key types convertible to std::string_view breaking lookups (#5689)
Since #4958, a key type implicitly convertible to std::string_view was accepted by is_usable_as_basic_json_key_type without checking that the object's comparator can actually compare object_t::key_type with that key type. The key was then forwarded unchanged to the underlying map, so const operator[], at, find, count, contains, erase and value failed to compile (a hard error inside <map>) for a key convertible only to std::string_view, and value() rejected such keys outright. For keys convertible to both std::string and std::string_view, the KeyType&& templates now won overload resolution over the object_t::key_type overloads and then failed the same way, a regression from 3.12.0. Only the non-const operator[] worked, because it uses emplace(), which constructs a std::string from the key explicitly. ordered_json was not affected, since ordered_map checks comparability itself. Add a trait, is_string_view_convertible_key_type, that recognizes a key type that is convertible to std::string_view but not directly comparable with the object's key type, provided std::string_view itself is comparable with it. at(), operator[], find(), count(), contains(), erase() and value() now route such keys through a new lookup_key() helper that converts them to std::string_view before they reach the object, matching how the object's transparent comparator already supports std::string_view lookups. Fixes #5663. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
0d01d6ae90 |
Classify leaves with operator<=> itself past the nesting bound (#5686)
* Classify leaves with operator<=> itself past the nesting bound In C++20, an ordered comparison past the nesting bound classified a pair of leaves by asking == first and then order_leaves(), which calls < and > - both derived from <=>. For a pair of binary values with the same bytes but a different subtype, == reports them unequal, while <=> (through std::vector<std::uint8_t>::operator<=>) reports them equivalent, so the pair ended the comparison as unordered instead of letting the next element decide - unlike an array or object within the bound, which compares such a pair with its own operator<=> and gets equivalent. So operator<=>, and the <, <=, >, >= derived from it, could give a different result for the same two values depending on how deeply the values were nested, or unordered at every depth with JSON_NO_THREAD_LOCAL defined. compare_leaves() now classifies such a pair in C++20 with operator<=> itself instead, matching how a value within the bound is compared; the equality-only and pre-C++20 ordered cases are unchanged. Which of the three runs is chosen by overloading on std::integral_constant<bool, Ordered>, the same tag dispatch order_leaves() already uses, rather than a runtime "if (Ordered)" on a template parameter, which MSVC would flag as a constant condition (C4127). Added a regression test to unit-comparison.cpp that nests such a pair 0, 127, 128 and 200 levels deep (127 stays within the 128-level bound, 128 and 200 do not) and checks that operator<=> and operator< agree at every depth. Fixes #5654. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Drop the version history note for a bug that was never released The regression came from #5390, which is not in any release. Addresses review comment by @gregmarr. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
6f2048cd5d |
Accept lvalues in ordered_map::emplace's value parameter (#5685)
* Accept lvalues in ordered_map::emplace's value parameter
ordered_map::emplace(key, value) took the mapped value only by T&&, an
rvalue reference rather than a forwarding reference, so
ordered_json::emplace("a", value) failed to compile whenever value was
an lvalue or a const lvalue, even though the same call compiles for
json (whose object_t is std::map, with a variadic emplace). Turn the
value parameter into a separately-deduced forwarding reference,
constrained with std::is_constructible so the overloads still only
accept something convertible to the mapped type. std::map-compatible
semantics are unchanged: emplace still does nothing if the key already
exists.
Open PR #5609 also touches ordered_map.hpp (moving values on vector
growth); this change only touches the two emplace() overloads and
should not conflict.
Fixes #5673.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Avoid astyle's padding in ordered_map::emplace's template headers
Use detail::conjunction instead of && and drop the redundant V&& in detail::is_constructible, so astyle keeps the usual template formatting. Addresses review comment by @gregmarr.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
|
||
|
Niels Lohmann
|
df27cc3d4c |
Add scalar-on-left overloads for legacy discarded comparisons in C++20 (#5682)
With JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON=1 and C++20, a scalar on the left-hand side of <= or >= (e.g., `1 <= discarded`) yielded false instead of the documented true. The C++20 legacy block only had member operators, which are only candidates when the basic_json is the left operand; for a scalar on the left, overload resolution picked the candidate rewritten from operator<=>, which does not emulate the legacy behavior. The C++17 branch already has scalar-on-the-left friend overloads for <= and >=; add the equivalent pair to the C++20 legacy block. Added a regression test to tests/src/unit-comparison.cpp covering all four operand orders for both operators. Fixes #5665. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
d11e89471b |
Assert on missing array indices in const operator[] and document the JSON pointer case (#5606)
The const operator[] overloads are unchecked by design, and a missing key or index is undefined behavior. The key overload guards this with a runtime assertion, but the index overload did not, although the element access documentation says an assertion fires in both cases. The const JSON pointer overload inherits both through json_pointer::get_unchecked(), so a pointer to a missing array index read out of bounds even in debug builds, and its documentation promised out_of_range.404 for any pointer that cannot be resolved. Add JSON_ASSERT(idx < size()) to const operator[](size_type), which also covers the index leg of the const JSON pointer overload. Document the undefined behavior for the const JSON pointer overload in operator[].md and in the runtime assertions page. Release builds are unchanged. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
6d0867a85f |
Make comparisons with scalars noexcept only when the conversion is (#5751)
The comparison operators taking a scalar (==, !=, <, <=, >, >=, and C++20's <=>) convert the scalar to a basic_json and compare, but were unconditionally noexcept. When that conversion throws, the program called std::terminate instead of propagating the exception, e.g. when comparing a json with a string literal under memory pressure (std::bad_alloc) or with an enum value not mapped by NLOHMANN_JSON_SERIALIZE_ENUM_STRICT (out_of_range.410). clang-tidy 22.1 reports the latter as bugprone-exception-escape. Declare the 16 scalar overloads noexcept(std::is_nothrow_constructible<basic_json, ScalarType>::value): they stay noexcept for numbers, Booleans, nullptr, and plain enums, and are noexcept(false) for strings and enums whose to_json may throw. The comparisons of two basic_json values are unchanged. Restore the strict-enum comparisons removed from unit-conversions.cpp in the previous PR, check that comparing an unmapped strict enum now throws, and pin the new exception specifications in unit-noexcept.cpp. Document the exception safety of overload (2) on all seven operator pages. Ran make amalgamate. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
63c10a51fc |
Review and extend the documentation, and check it in CI (#5638)
* Review and extend the documentation, and check it in CI A review of all documentation pages found factual errors, dead links, missing cross-references, and gaps in examples. This fixes them and adds checks so the same problems are caught automatically. Fixes: - wrong signatures and version histories (operator!= C++20 member, binary() subtype type, get<PointerType>(), JSON_NO_THREAD_LOCAL, ...) - stale descriptions (number parsing since #5283, UBJSON table, SAX example that no longer compiled, tsl::ordered_map advice) - dead internal and external links; repology.org badges (the domain is suspended) replaced by badges that query the registries directly - deprecation notes link the migration guide; the guide itself fixed Additions: - "See also" sections, cross-references, 25 runnable examples, 12 Mermaid diagrams, new API pages for json_pointer::operator<=> and byte_container_with_subtype::operator==/!= - landing page, guides for untrusted input and performance - "unreleased" badge after versions newer than the latest release Checks: - strict documentation build (broken links/anchors fail it); CI and the publish workflow fetch the full history the build needs - weekly external link check, Mermaid syntax check in CI - check_structure.py: example titles, heading levels, alt texts, header links, docset index coverage; its unused-example check works again - all examples produce the same output on every platform Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Keep the customer links that could not be fixed A dead link on the customers page is still the evidence of where the use of the library was documented. Keep the original URLs of the entries without a working replacement (Marne, Cisco Webex Desk Camera, Philips Hue, CyberArk) and exclude exactly these URLs from the link check. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Correct the duplicate-key recipe's claim about SAX positions The SAX interface's key() receives no position either; only parse_error() does. Also note that the recipe does not report the path to the repeated key (see discussion #5085). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Say the library is available as a single header and mention json_fwd.hpp Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Correct documentation errors found while hunting for bugs - patch/patch_inplace: list the JSON pointer errors parse_error.106-109 and out_of_range.402/404, and quote the actual parse_error.105 message. - unflatten: list parse_error.106/107/108 and out_of_range.404. - to_bson: list out_of_range.415 (binary subtype above 255) and note that 412 and 415 are new in 3.13.0. - to_string: state that string_t must be convertible to std::string, also in the StringType requirements table. - JSON Lines: a `while (input >> j)` loop also throws after the last value for concatenated JSON values; show a loop that works for both. - BON8: a string gets 0xFF only if nothing follows it in the message; a string at the end of an array or object is ended by 0xFE. - custom_string_type.hpp: add operator+=(char), which the "Always required" list asks for (json_pointer::to_string, flatten, unflatten, and diff did not compile), and an ADL int_to_string for diff and items. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Cache the release headers with functools.lru_cache Codacy (Pylint) flagged the mutable default argument that header() used as its cache. functools.lru_cache keeps the same memoization without it. The script's output is unchanged. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
4d46bce4e1 |
Fix CI on develop (#5750)
* Fix CI configuration broken by recent merges and tool updates - gcc_flags.cmake: drop -Wexperimental-fmv-target, which GCC 16 accepts only on aarch64; amd64 rejects it, so every GCC job failed while checking the compiler. - ci_get_cmake: add VERBATIM so the checksum pipeline is passed to the shell intact (the unescaped `$'` broke the generated Makefile and build.ninja, failing ci_cmake_flags and ci_module_cpp20); match the SHA-256 entry case-insensitively, as CMake 3.5.0 lists the archive as "Linux-x86_64"; and unpack with --strip-components, as that archive's top-level directory is spelled "Linux" too. - ci_single_binaries: compile json.hpp's TU without IWYU's --error, as the comment above the gate already intends. - tests: restore -Wno-deprecated-declarations for all non-MSVC compilers (#5737 kept it for GCC only), as several tests call deprecated functions on purpose; include thirdparty/fifo_map as SYSTEM. - .clang-tidy: set misc-use-internal-linkage.AnalyzeTypes to false; clang-tidy 22.1 extended the check to classes and enums and flagged 100 test helper types. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix library warnings and a JSON_DIAGNOSTICS parent bug - binary_reader: pass integers to sax->number_integer() through conditional_static_cast<number_integer_t>, making the existing narrowing for a narrow number_integer_t explicit (MSVC C4244 and GCC -Wconversion/-Warith-conversion with the int16_t test from #5694); mark two Infer DEAD_STORE false positives with @infer-ignore. - to_json: set the parents of an array built from a C++20 range view after all elements are in place; a reallocating push_back moved the earlier elements and left their parent pointers stale, failing the JSON_DIAGNOSTICS invariant assertion. - json.hpp: suppress MSVC C4127 for the new is_ordered_map check in diff(), like the three existing ones; spell out std::formatter::parse's return and iterator types for clang-tidy 22.1. - number_parse: make the Eisel-Lemire digit counter unsigned (GCC -Wstrict-overflow). - string_utils: take encode_utf8's callable by const reference (cppcoreguidelines-missing-std-forward) and drop a \u from its doc comment (-Wdocumentation-unknown-command). - ordered_map: include <memory> for std::allocator (cpplint). Ran make amalgamate. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix tests failing in CI on develop - unit-allocator: skip the #5640 test under MSVC STL iterator debugging, where containers allocate a debug proxy in noexcept move constructors and a failing allocation terminates; move a decrement out of an if condition (bugprone-inc-dec-in-conditions). - unit-conversions: expect the "(/0)" path with JSON_DIAGNOSTICS; compare strict enums via get<>() rather than through the noexcept operator==(ScalarType, json), which bugprone-exception-escape flags. - unit-alt-string: suppress -Wexit-time-destructors for the strict enum macro and misc-use-internal-linkage for its enum. - unit-bjdata: call the static lookup functions through the type and pass unsigned char (-Wsign-conversion on amd64). - Mark Infer false positives with @infer-ignore in unit-diagnostics, unit-pointer_access, unit-udt, and unit-conversions. - Smaller clang-tidy 22.1 findings in unit-class_parser, unit-constructor2, unit-custom-base-class, unit-locale-cpp, and unit-noexcept. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
6aec1a085f |
Report BON8 input that ends after a UTF-8 lead byte as truncated (#5677)
A lead byte (0xC2..0xF7) inside a string begins either another character (if a continuation byte follows) or an integer (otherwise). When the input ended right after the lead byte, the reader took the missing byte as "not a continuation byte", ended the string before the lead byte, and treated the lead byte as the start of the next value. With strict=false, a message cut off there was therefore read as a shorter value: the 11 bytes of "😀😀é" cut after 9 bytes gave "😀😀", and ["aé"] cut after 3 of its 5 bytes gave ["a"]. With strict=true, the input was rejected with a misleading message ("expected end of input"), or, for a key, with parse_error.112 instead of 110. Either reading of the lead byte leaves the message incomplete: a string at the end of a message must be terminated by 0xFF, so the lead byte cannot belong to a following message. Report parse_error.110 (unexpected end of input) for strings and keys, as the comment on get_bon8_string() already requires and as the reference decoder (HikoGUI) does. Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
e400780533 |
Re-amalgamate single_include (#5745)
* Re-amalgamate single_include #5737 changed 13 headers under include/ but merged without the matching single_include/nlohmann/json.hpp update, so the amalgamated header still had, among others, the GCC C++20 -Wignored-attributes pragma block and the clang -Wdocumentation push/pop that #5737 removed, the forwarding from_json tuple/array helpers it replaced with const references, and lacked the output_adapter char_traits changes it added. Regenerated with `make amalgamate` (astyle 3.4.13). The diff is exactly `git diff |
||
|
Niels Lohmann
|
cff0a61369 |
Share DOM SAX position handling; fix stale parser and lexer comments (#5731)
* Share the diagnostic-position setter of the DOM SAX parsers json_sax_dom_parser and json_sax_dom_callback_parser each had a private copy of handle_diagnostic_positions_for_json_value(), identical except for comments. Move the body into one static member function, detail::diagnostic_positions::set_from_lexer(value, lexer), which both classes call with their lexer pointer. basic_json befriends the new struct (only when JSON_DIAGNOSTIC_POSITIONS is enabled), as the position members are private. The discarded case is reached through the callback parser, so the LCOV_EXCL markers that only the dom parser's copy had are gone. The NOLINT on the unreachable default case loses the stray "-warnings-as-errors", which is not a check name. The start-position setup in start_object()/start_array() is left alone, as #5706 is editing the callback parser's versions. Behavior, the public API and the ABI are unchanged. Part of #5712 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Correct the parser comments on recursion and skip_to_state_evaluation The class documentation called the parser a recursive descent parser, but sax_parse_internal() is a loop that keeps the open containers on an explicit stack. The comment at the end of an array and of an object said the flag is set to false while the code below it sets it to true. Describe what the code does instead. Comments only; behavior, the public API and the ABI are unchanged. Part of #5712 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Update the discard_number_values comments to the current number path The comments explaining the accept() shortcut in convert_number() and the member documentation still argued in terms of strtoull()/strtoll() and errno, which #5283 replaced with convert_integer(), and pointed at scan_number() instead of convert_number(). They also did not say that scan_number_bulk_contiguous() converts integers itself, so the shortcut is only reached for input without bulk access, with JSON_DIAGNOSTIC_POSITIONS, or when the bulk scanner falls back. Rewrite both comments to describe the digit-count check in front of convert_integer(), keeping the 18-digit bound and the json_sax_acceptor argument. The stale <cstdlib> comment is left for after #5616, which edits that include block. Comments only; behavior, the public API and the ABI are unchanged. Part of #5712 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * List the UTF-8 validators instead of calling the DFA the only one The documentation of decode() called the Hoehrmann DFA the single source of truth for UTF-8 validation. It is used only by the serializer and by is_valid_utf8() (CBOR/MessagePack/BSON/UBJSON/BJData text strings). The lexer's scan_string() switch, validate_one_utf8() / valid_utf8_prefix() (bulk string scan, BON8 bulk path and BON8 writer) and the BON8 byte path in get_bon8_string() check the RFC 3629 ranges on their own. Replace the sentence with a list of the four validators, what each is used for, and a note that they must accept the same sequences. Sharing code between them was considered and dropped: it would save a few lines in a validator that is entangled with BON8 pushback, and #5677 is editing the BON8 byte path. Comments only; behavior, the public API and the ABI are unchanged. Part of #5712 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix stale doc comments and include lists in the input headers input_adapters.hpp included <memory> and <numeric> for the removed shared_ptr-based adapter design but used neither; it called (std::min) without including <algorithm>. json_sax.hpp used std::numeric_limits without including <limits>. Also corrected comments that no longer matched the code: input_stream_adapter does not skip the input's BOM (the lexer's skip_bom() does), the span_input_adapter comment named the no-longer-existing input_buffer_adapter type, lexer::get_string() does not reset the token, binary_reader's get_number() doc opened with /* instead of /*! (so Doxygen skipped it) and omitted BON8 from its endianness note, and the UBJSON-binary-types note did not mention that BJData 'B' arrays are read as binary. Left out: the lgtm suppression on lexer.hpp's scan_number() (in #5616's hunk) and the "-1 if unknown" wording in json_sax.hpp's start_object/start_array docs (in draft #5267's hunk), per the verdict's conflict list. Part of #5712 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Deduplicate the strict-EOF/release_lookahead/error block in parser::parse() json_sax_dom_callback_parser and json_sax_dom_parser branches of parser::parse() ran the same ~25 lines after sax_parse_internal(): the strict-mode EOF check (raising parse_error.101 through the SAX parser), release_lookahead() in non-strict mode, and mapping an errored SAX parser to a discarded result. The two copies had already drifted apart in formatting and in the second copy's "see above" comment. Add a private parse_dom(DomSax&, strict) member that runs this shared sequence once and returns whether the SAX parser did not error; both branches of parse() now only construct their DOM SAX parser, call parse_dom(), and (for the callback parser) map a discarded top-level value to null. sax_parse() is left untouched, since it only runs the EOF check and release_lookahead() when sax_parse_internal() succeeded, unlike parse(), which runs them unconditionally. Behavior-preserving: same operations in the same order for both SAX parser kinds. Verified with unit-class_parser (strict/non-strict, callback and non-callback), unit-deserialization and unit-disabled_exceptions (JSON_NOEXCEPTION), plus a clean make amalgamate / make check-amalgamation diff. Overlaps #5601, which touches the same lines. Signed-off-by: Niels Lohmann <mail@nlohmann.me> #5712 item 2 * Share the code point to UTF-8 encoding between the wide-string helpers and the lexer The 1/2/3/4-byte UTF-8 encoding ladder was written out by hand three times: in wide_string_input_helper<..., 4>::fill_buffer() for a UTF-32 code point, in the UTF-16 helper for both a BMP code unit and a valid surrogate pair, and in the lexer's \uXXXX/\uXXXX\uYYYY handling. The copies had drifted: the UTF-32 helper masked the leading bits of each byte (& 0x1Fu, & 0x0Fu, & 0x07u) where the others relied on the shift alone, even though both give the same result for a code point that is already known to be in range. Add detail::encode_utf8(cp, out) in string_utils.hpp, a single encoder that invokes a callable once per output byte, most significant byte first. Use it in the three valid-code-point branches (UTF-32 code points up to U+10FFFF, UTF-16 code units outside the surrogate range, and valid UTF-16 surrogate pairs) and in the lexer's \u handling, where out forwards to add(). The UTF-16 helper's deliberate pass-through of malformed surrogate units and the UTF-32 helper's 0xFF sentinel for code points above U+10FFFF are untouched, since neither reaches the new helper. Behavior-preserving: same bytes in the same order for every valid code point, verified with unit-class_lexer, unit-class_parser, unit-deserialization, unit-wstring and the non-test-data parts of unit-unicode1..5 (ASan/UBSan, C++11/17/20), and an escape-heavy parse microbenchmark that shows no change (about 73 ms either way, median of 3, 1M escape sequences). single_include/ regenerated with make amalgamate; make check-amalgamation leaves a clean tree. Overlaps #5704, which rewrites the wide_string_input_helper specializations touched here. Signed-off-by: Niels Lohmann <mail@nlohmann.me> #5712 item 6 --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |
||
|
Niels Lohmann
|
3926fcaac3 |
Deduplicate serializer dump code; fix stale includes, docs, and lint (#5729)
* Share scalar serialization between dump_internal and dump_value dump_value()'s cases for string, binary, boolean, number_integer, number_unsigned, number_float, discarded and null were a byte-for-byte copy of dump_internal()'s (added together in #5285 for the iterative fallback path). Any future change to scalar output had to be made in both places, or the recursive and depth-limited paths would silently start producing different bytes. Extract the shared cases into a private dump_scalar() and have both dump_internal() and dump_value() call it. Output is unchanged: dump(), dump(4), dump(-1,' ',true) and the replace/ignore error_handler_t variants are byte-identical over the json_test_data corpus before and after, and dump() throughput on a scalar-heavy document is unaffected. Part of #5709 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Drop serializer.hpp's dependency on binary_writer.hpp The only use of binary_writer in serializer.hpp was binary_writer<BasicJsonType, char>::to_char_type() to write the U+FFFD replacement character's three bytes. With CharType=char this is an identity conversion, so the include of binary_writer.hpp (and transitively binary_reader.hpp) pulled in a large, unrelated header for a no-op call. Write the three bytes directly instead. serializer.hpp compiles standalone with -Wall -Wextra -Werror, with and without -funsigned-char, and unit-serialization's error_handler_t::replace cases (with and without ensure_ascii) still pass. Moving binary_writer's to_char_type/to_msgpack_length to its private section is left as an optional follow-up. Part of #5709 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix stale #include lines in the output headers output_adapters.hpp included <algorithm> and <iterator> for std::copy and std::back_inserter, which have not been used there since #3569 (2022). serializer.hpp included <algorithm> for std::reverse (also unused), <cmath> for labs/isnan/signbit (only std::isfinite is used) and <utility> for std::move (nothing from <utility> is used there), while using std::next without including <iterator> at all, relying on getting it transitively through output_adapters.hpp's own stale <iterator>. Drop the unused includes, add <iterator> for std::next, and correct the remaining include comments. Both headers still compile standalone with -Wall -Wextra -Werror. Part of #5709 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Remove JSON_HEDLEY_NON_NULL(2) from write_characters() overrides output_vector_adapter, output_stream_adapter and output_string_adapter declared their write_characters(const CharType*, std::size_t) override JSON_HEDLEY_NON_NULL(2), but binary_writer legitimately calls it with a null pointer and length 0 for an empty string or binary value; the type-erased call path only stayed silent under UBSan because the static callee at those call sites is the unattributed virtual base. A nonnull attribute on a definition lets GCC and Clang assume the parameter is non-null inside the function body even when the call is virtual, so this was latent undefined behavior, not just style. Drop the attribute from the three overrides and document the (nullptr, 0) contract on output_adapter_protocol::write_characters. unit-cbor, unit-msgpack, unit-bson and unit-bon8 (which all exercise empty binary/string payloads through the stream and vector/string adapters) pass under -fsanitize=address,undefined,nonnull-attribute. Part of #5709 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Update stale serializer doc comments to match the current implementation dump_internal()'s doc block still described the pre-#5285/#5449 implementation: an escape_string() function that does not exist (the function is dump_escaped), integer conversion "implicitly via operator<<" (dump_integer actually uses a digit-pair lookup table), and floating-point conversion via "%g" (IEEE-754 types go through to_chars, others through snprintf). dump_value()'s comment said elements are pushed for dump_internal to walk, but it is dump_iteratively() that walks the stack. dump_escaped(), dump_integer() and dump_float() each said they write "to output stream @a o", which has not been true since the writer moved to write_buffer. Doc-only change; no behavior, API or ABI impact. Part of #5709 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Merge duplicate byte-to-hex helper and drop stale '| 0' promotions serializer::hex_bytes() and binary_writer::hex_byte() had identical bodies. Keep one, detail::hex_byte() in string_utils.hpp, and use it from both. Also drop the `| 0` at the two serializer call sites (hex_bytes(byte | 0) and hex_bytes(s.back() | 0)): #3088 ( |
||
|
Niels Lohmann
|
18dd5663b0 |
Diff deeply nested values without recursing per nesting level (#5548)
* Diff deeply nested values without recursing per nesting level diff() descended into both values once per nesting level, and compared them with operator== on every level on the way, which recurses as well. Values nested deeply enough - 25,000 levels on an 8 MiB stack - exhausted the call stack and terminated the process, although parse() accepts them without complaint. On such a chain the per-level comparisons and path strings also made diff() quadratic in time and memory. Both the recursion and operator== only descend as far as the source is nested. So diff() first checks, recursing at most diff_depth_limit() (128) levels, whether the source is nested more deeply than that. If not - all but a vanishing minority of values - the recursive algorithm diffs it exactly as before, now as diff_recursively(). Otherwise diff_iteratively() walks the two values on an explicit stack, emitting the same operations in the same order. It does not compare arrays and objects with operator== up front (equal ones yield no operations anyway), keeps the path in one buffer instead of a new string per level, and hands every subtree that is not nested too deeply back to diff_recursively(), so equal parts are still skipped quickly. The check costs one pass over the source. On a 3,000-object document that is about 30% of diffing two equal values (which is just an operator== call), about 10% of diffing values that differ in a few places, and noise when arrays change length. Once operator== no longer recurses (#5390), the check can go. Tests check that the patch reproduces the target at every depth up to 300, for json and ordered_json, including reordered members. They also check the exact operation for a difference deep inside, and diff values nested 100,000 levels deep. Fixes #5393 for diff(). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Make diff_frame a member struct that declares its special members GCC's -Weffc++ (an error in CI) asks a class with pointer members, a user constructor and a non-trivial destructor to declare its copy constructor and copy assignment; diff_frame's vector and basic_json members make its destructor non-trivial. Declare all five as defaulted, which also satisfies clang-tidy's special-member-functions check. Leave their exception specifications implicit: GCC 4.8 rejects an explicit one that differs from the implicit one, as it does for flatten_task in #5517. The converting constructor cannot throw, and is now declared noexcept for GCC's -Wnoexcept, which flags the emplace_back() under C++26 otherwise. The struct also moves from diff_iteratively() into the class, like dump_frame in the serializer. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Use the shared recursion limit in diff() diff_depth_limit() is gone in favor of detail::recursion_depth_limit(). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Diff fewer nesting depths so the test does not time out under Valgrind Checking every depth up to 300 made test-json_patch exceed the 1500 s ctest timeout in ci_test_valgrind. Check the depths up to 16, those around the recursion limit of 128, and 300 instead. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Mark the diff frame's value-initialized members for clang-tidy The braces are kept for GCC's -Weffc++, as in json_sax.hpp. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Bound diff()'s descent with a depth count instead of scanning the source Now that operator== no longer recurses (#5390), diff() can keep its per-level equality shortcut all the way down. It diffs recursively for the first detail::recursion_depth_limit() levels, as merge_patch() does, and hands anything deeper to diff_iteratively(). The nesting_exceeds() scan, which cost about 30% on equal documents, is gone, and diff() is on par with develop again. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Note that the diff frame reference is invalidated by pop_back() too Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Keep diff()'s recursive levels small and its result elided diff_recursively built every patch operation in place from initializer lists. Unoptimized builds give each of those temporaries its own stack slot, so every level of the bounded descent cost kilobytes of stack (about 6 KB with clang -O0), and the 128 recursive levels overflowed the 1 MB stack of MSVC Debug in the "deeply nested values" test. The operations and the key comparison of two objects are now built by separate functions, which diff_iteratively shares, and both diff functions append to one result instead of returning a patch per level that the caller copies. With clang -O0, diffing values nested 300 levels deep now peaks at about 190 KB of stack instead of 880 KB. Since diff() now owns the only returned value, clang's -Wnrvo no longer reports the returns of diff_recursively, which alternated between the local patch and diff_iteratively's result. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Copy the diff frame's members instead of holding a reference to it The loop in diff_iteratively held a reference to the top frame, which enter() invalidates when it pushes and the end of the loop invalidates when it pops. Nothing used it afterwards, but a later change could. As in the other iterative walks, the members the loop reads are now copied out as constants and the ones it advances are changed through stack.back(). The frame as a whole is not copied: it holds the common keys and the "add" operations of an object. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> |