* Add BON8 support
Add to_bon8/from_bon8 and input_format_t::bon8 for BON8, a binary format
that uses the byte values that cannot begin a UTF-8 character as type
markers, so strings need no length prefix. It is the most compact of the
supported binary formats on the benchmark files.
The reader is non-recursive like the other binary readers. A string ends
at the first byte that cannot continue it, so the reader hands the one or
two bytes it reads past a string back to the value that follows. The
writer produces the canonical representation of the specification, except
for NFC normalization; its output is identical to that of the reference
implementation (HikoGUI) on all files of the test data.
The round-trip tests need the .bon8 files of json_test_data 3.2.0.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Address review comments
- Reuse detail::validate_one_utf8 to check strings in to_bon8; the error
now names the first byte of the invalid sequence.
- Document that to_bon8 leaves bytes in the output adapter on an
exception, and that string_open is only an output of write_bon8_marker.
- Explain why the pushback buffer of the BON8 reader cannot overflow.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Select the BON8 float prefix by type
get_bon8_float_prefix only depends on the type of its argument, so make
the type a template parameter instead of passing an unused value.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Rename a test variable that Flawfinder mistakes for read()
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix the BON8 CI failures
- compare the float in write_bon8_float with number_float_t constants,
so GCC does not warn about a float-to-double conversion
- mark check_bon8_utf8's context as used when exceptions are disabled
- choose the compact float prefix in a helper rather than with nested
conditional operators (clang-tidy)
- use auto for the cast in the BON8 integer reader (clang-tidy)
- write the int32 minimum test values as long long literals (MSVC C4146)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Amalgamate
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Read BON8 strings in bulk from contiguous input
- copy the valid UTF-8 of a string in one step when the input is
contiguous (twitter.json is read in 1.68 instead of 2.52 ms,
jeopardy.json in 196 instead of 297 ms, close to CBOR and MessagePack)
- share the new valid_utf8_prefix() with the writer's UTF-8 check, which
now skips ASCII 8 bytes at a time
- let the fuzzer check that contiguous and stream input give the same
value or error, and test both paths in the unit tests
- clarify that a second 0xFF after a string is an empty string
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Link the BON8 functions from the other binary format pages
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Name the bulk scan flag after the input, not BON8
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Read BSON keys in bulk from contiguous input
BSON keys (and array indices) are C-style strings, which were read byte
by byte. For contiguous input they are now read up to their \x00-byte in
one step, using the same bulk_scan flag as BON8 strings: twitter.json is
read in 1.46 instead of 2.01 ms, citm_catalog.json in 2.93 instead of
3.33 ms, jeopardy.json in 182 instead of 207 ms. canada.json, whose keys
are almost all one-digit array indices, takes 2 % longer.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix the BON8 CI failures of the bulk-read tests
- skip the contiguous-versus-stream tests of BON8 strings and BSON keys
when exceptions are disabled: they catch the parse errors of invalid
input, and without exceptions the library aborts instead
- use static_cast for the int64 test value (google-readability-casting)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Move the explicit basic_json instantiation into its own test file
Linking test-regression3_cpp20 with clang and MinGW failed with
"relocation truncated to fit: IMAGE_REL_AMD64_REL32 against `.rdata'",
as test-regression2 did before #5511. The explicit instantiation of
basic_json<> for #4825 compiles every member function, including the
BON8 reader and writer, into that object, and it was already close to
the limit (2,226,104 bytes on develop, 2,234,960 with BON8; clang -O1,
C++20).
Give the instantiation a file of its own: unit-regression3 is now
1,594,736 bytes and unit-explicit_instantiation 1,095,064. The new file
mentions JSON_HAS_CPP_17 and JSON_HAS_CPP_20 so it keeps being built
for the C++17 standard the regression was about.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Convert the bytes of the BON8 test strings explicitly
The str() helper constructed a std::string from a byte range, which
converts each unsigned char implicitly; -fsanitize=integer reports that
for bytes of 0x80 and above (ci_test_clang_sanitizer).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The tests fake a container size of UINT32_MAX + 1, which does not fit
into a 32-bit std::size_t: MSVC rejects the truncation (C4305/C4309
with /WX), and clang-cl wraps the size to 0 so nothing throws. Guard
them with SIZE_MAX > UINT32_MAX like the tests from #5584.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
When using cbor_tag_handler_t::store, tags 0xD8-0xDB previously assumed
that the tagged item was a byte string, unconditionally attempting to
parse binary data and failing on valid CBOR documents containing tags
applied to integers, strings, arrays, or objects (such as self-describe
tag 55799).
Check whether the tagged data item is a byte string (0x40-0x5B or 0x5F).
If it is a byte string, store the subtype on the binary value as before.
Otherwise, iteratively process the tagged value in the driver loop using
item_read so that chained tags do not consume native stack space.
Part of #5316.
Signed-off-by: ReturnKartikey <kartikeynegi2000.work@gmail.com>
* Cut test suite runtime in binary roundtrips and integer sweeps
The Linux CI jobs pass --no-skip, so skip() does not help there.
Parse each corpus file once in the binary roundtrip loops instead of
four times. Sample the 16-bit integer ranges with stride 7 (still hits
every low byte) and always keep the endpoints.
Also drop the 5M-node parse test to 500k, which still covers the
non-recursive destructor, and move jeopardy.json into its own skipped
test so the cheaper binary-format size checks actually run.
See #5418.
Signed-off-by: ayush-singh-0601 <singhayush062006@gmail.com>
* Drop useless int32_t casts in the sampled integer loops
ci_test_gcc compiles with -Werror=useless-cast. On that compiler
int32_t is int, so static_cast<int32_t> of the loop bound is an
error. The bounds are already int, and the sampled values do not
change.
Signed-off-by: ayush-singh-0601 <singhayush062006@gmail.com>
* Revert unit-binary_formats.cpp to develop and fix comment
Revert tests/src/unit-binary_formats.cpp to its develop state.
The test-case split made valgrind jobs slower instead of faster,
because the cheaper corpus files (canada/twitter/citm/sample)
now ran under valgrind where they never did before.
Fix the next_integer_sample comment: the function has no 'first'
parameter, so describe what the function actually does.
Signed-off-by: ayush-singh-0601 <singhayush062006@gmail.com>
---------
Signed-off-by: ayush-singh-0601 <singhayush062006@gmail.com>
* Throw instead of writing MessagePack lengths beyond UINT32_MAX
MessagePack stores the length of a string, binary value, array, or
object in at most 32 bits. For a larger value, to_msgpack wrote no length
at all, so the output could not be read back. It now throws
out_of_range.412, which BSON already uses for its 32-bit length fields.
The check lives in one function, so each length is written by an
if/else chain that ends in a plain else, without a condition that can
never be false. It is tested with string and binary types that report a
size beyond UINT32_MAX without allocating it, like the BSON tests do.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix the CI failures of the MessagePack length check
- mark to_msgpack_length's value as used when exceptions are disabled
(-Wunused-parameter, misc-unused-parameters)
- put "Exception safety" before "Exceptions" in to_msgpack.md, as the
documentation style check requires
- create the test's string value from its type: constructing it from a
beyond_uint32_string_t considers the std::filesystem::path conversion,
which libstdc++ 10 reports as ambiguous for a class derived from
std::string (clang 13)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Skip the MessagePack string length test for clang with libstdc++ 10
C++17 builds consider the std::filesystem::path conversion for the
string type, and with clang and libstdc++ 10 that conversion is
ambiguous for a class derived from std::string. Creating the value from
its type did not avoid it, since any basic_json with that string type
instantiates the check. The binary and ext cases are still tested there.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Keep the MessagePack string test type and its alias in one block
astyle indented the alias oddly when it had an #ifdef of its own after
the binary alias; declare it right after the string type, in the same
block.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Compare unordered objects by key below the nesting bound
Values nested deeper than the nesting bound are compared without the
call stack, walking both objects entry by entry. Two equal objects of a
type that enumerates its entries in no fixed order - std::unordered_map,
say - can be walked in different orders, so they compared unequal, and
a deep copy compared unequal to its original. std::unordered_map's own
operator== does not depend on the order, which is what applies above the
bound.
Where the keys differ, equality now finds the entry by its key instead.
An ordering, and ordered_map, whose operator== compares its entries in
sequence, still decide by the key.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Test unordered object equality without std::unordered_map
basic_json<std::unordered_map> instantiates std::pair<const string,
basic_json> while basic_json is still incomplete. The standard does not
require std::unordered_map to support that, and libstdc++ 6 to 9 as well
as the EDG front ends of icpc and nvc++ reject it, which broke the build
of unit-comparison on those CI jobs.
The test now uses an object type derived from std::map (which, as the
default object type, works everywhere) whose comparator orders keys
ascending or descending as chosen at construction, and whose operator==
does not depend on the order of the entries - the property of
std::unordered_map the test is about.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Compare the test object type's entries with std::all_of
clang-tidy (readability-use-anyofallof) asked for std::all_of instead of
the loop in unordered_object_t's operator==. The entry type is spelled
out, as C++11 needs typename for base_type::value_type and C++20
reports it as redundant.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Add tests for uncovered code paths
Cover code the test suite did not reach, found from the Coveralls report
of develop and a local coverage run of HEAD:
- dump() of every kind of value below the bound of the recursive descent
(pretty-printed objects, binary values, discarded values, scalars), and
flushes of the escape and write buffers mid-string and mid-binary
- the iterative comparison: objects with different keys, containers that
are a prefix of each other, and elements that cannot be ordered, each
both at the top level and below the nesting bound
- SAX handlers that stop at any event, including the end of a nested
container, in the BSON, CBOR, MessagePack, UBJSON and BJData readers
- from_bson/cbor/msgpack/ubjson/bjdata returning a discarded value
through the iterator and pointer overloads
- JSON Patch, diff, merge_patch and update(..., true) on ordered_json
- smaller gaps: get_allocator(), to_ubjson/to_bjdata into a string,
value() with an unresolvable JSON pointer, integer/float comparison
below the integer range and with negative fractions, conversion to a
custom binary type, std::formatter::parse on a spec without '}',
unescape() of a lone '~', and the callback parser's start_array()
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Cover more paths that were thought unreachable
- parse_float_fast() declining malformed or inexact input, called
directly since the lexer only passes well-formed numbers to it
- a UTF-16 high surrogate followed by a unit above the low surrogates
- self-assignment of a const_iterator
- a truncated CBOR string read through non-contiguous iterators
- serializing a long double under the de_DE locale, which undoes the
locale's decimal point and thousands separator
- values read from a binary format carrying no diagnostic positions,
with and without a parser callback
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix the CI failures of the new coverage tests
- declare the self-assignment reference const (misc-const-correctness)
- expect the (/path) prefix that JSON_DIAGNOSTICS adds to the messages
of the failing ordered_json patch operations
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Expect the byte range JSON_DIAGNOSTIC_POSITIONS adds to the patch errors
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Build the expected dump of the nested-object test with +=
clang-tidy (performance-inefficient-string-concatenation) reported the
chain of operator+ calls that assembled the expected indented output.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Compare the BJData and UBJSON test outputs byte by byte
Building a std::string from the byte vector converts each byte
implicitly, which -fsanitize=integer reports for bytes of 0x80 and
above (ci_test_clang_sanitizer).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Coverage reported conditions in the binary writer that can never be
false, and marked the code behind them with LCOV_EXCL. Remove them
instead of excluding them:
- CBOR writes the length of a string, binary value, array, or object
exactly like an unsigned integer, only with another major type. One
function, write_cbor_head(), now writes both, so the integer tests
cover every width and the four excluded 64-bit length branches are
gone.
- A last `else if` whose condition holds for every remaining value
(an unsigned value at most UINT64_MAX, a signed one in the range of
int64_t) is now a plain `else`.
- Whether a signed integer fits into an int64 for UBJSON and BJData is
decided by its type at compile time. Only an integer type wider than
64 bits gets a range check and the high-precision fallback.
- The private get_impl(boolean_t*) was never called.
The UBJSON type prefix 'H' of an optimized container of unsigned
integers beyond the range of int64 was reachable although excluded; it
is tested now.
The output is unchanged.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The loop in diff_iteratively held a reference to the top frame, which
enter() invalidates when it pushes and the end of the loop invalidates
when it pops. Nothing used it afterwards, but a later change could. As in
the other iterative walks, the members the loop reads are now copied out
as constants and the ones it advances are changed through stack.back().
The frame as a whole is not copied: it holds the common keys and the
"add" operations of an object.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
diff_recursively built every patch operation in place from initializer
lists. Unoptimized builds give each of those temporaries its own stack
slot, so every level of the bounded descent cost kilobytes of stack
(about 6 KB with clang -O0), and the 128 recursive levels overflowed the
1 MB stack of MSVC Debug in the "deeply nested values" test. The
operations and the key comparison of two objects are now built by
separate functions, which diff_iteratively shares, and both diff
functions append to one result instead of returning a patch per level
that the caller copies. With clang -O0, diffing values nested 300 levels
deep now peaks at about 190 KB of stack instead of 880 KB.
Since diff() now owns the only returned value, clang's -Wnrvo no longer
reports the returns of diff_recursively, which alternated between the
local patch and diff_iteratively's result.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Now that operator== no longer recurses (#5390), diff() can keep its per-level
equality shortcut all the way down. It diffs recursively for the first
detail::recursion_depth_limit() levels, as merge_patch() does, and hands
anything deeper to diff_iteratively(). The nesting_exceeds() scan, which
cost about 30% on equal documents, is gone, and diff() is on par with
develop again.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Write BSON in linear time, without recursing per nesting level
to_bson() had two problems with nested values:
- It recursed once per nesting level, so a value nested deeply enough -
100,000 levels on an 8 MiB stack - exhausted the call stack and
terminated the process, although parse() accepts such values without
complaint.
- BSON prefixes every document and array with its length. The writer
computed that length by walking the entire value below it, again for
every nested document it wrote, which made serializing O(size x depth).
A 200-level document took 30 ms instead of 1.
Both passes are now iterative, and each length is computed exactly once:
- calc_bson_sizes() computes the length of every document and array in
one pass, each from the lengths of its entries, into a table ordered
the way they are written.
- write_bson_document() then writes the document, taking each length from
the table.
Everything observable is unchanged, as a differential test against
develop confirms byte for byte:
- The same bytes are written.
- A key containing U+0000 still throws out_of_range.409 for the same
first key, with the same diagnostics path, before anything is written.
- A document too large for BSON still throws out_of_range.412 before
anything is written.
- A binary subtype above 255 still throws out_of_range.415 after the
same partial output.
Only the enclosing objects and arrays are kept on a stack, so a flat
document allocates nothing for it. Measured against develop (clang -O3,
median of 201 runs): flat objects unchanged, flat arrays 37% faster (the
array length was computed twice), a nested 3,000-object document 2x
faster, a 200-level document 33x faster.
to_bson.md documented the quadratic complexity since #5334; it is linear
again.
Fixes#5392 for BSON, and #5308.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Do not require a default-constructible string_t in the BSON writer
GCC 4.9 and MSVC rejected the test's huge_string_t, which has no default
constructor; develop never default-constructed string_t here either.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Let the BSON index-name helper only fill its output parameter
It returned a reference to the string it filled, so callers held a second
name for index_name. Addresses review feedback.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Make JSON_STRICT_NUL_HANDLING part of the ABI tag
JSON_STRICT_NUL_HANDLING (#5534) changes the bodies of inline functions:
the lexer's handling of '\0' and input_adapter() for char arrays. So
translation units compiled with and without it define the same functions
differently, an ODR violation - the case the ABI tag exists for, as with
JSON_BRACE_INIT_COPY_SEMANTICS (_bics). It now appends _snul to the inline
namespace. The macro is new in 3.13.0, so no existing namespace changes.
Its default moves to abi_macros.hpp, and it is only #undef'd without
JSON_TEST_KEEP_MACROS, as for the other ABI macros. The ABI config tests,
the namespace docs, the macro's docs and the Natvis file cover the new tag.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Amalgamate
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Allocate the deep copy's key scratch space with the provided allocator
The iterative deep copy builds each object's keys in a temporary vector of
key/value pairs before handing them to the object's range constructor. That
vector holds basic_json values, so like the values themselves it now uses
AllocatorType instead of std::allocator.
Also document that AllocatorType covers the JSON values, while most
temporary storage still uses std::allocator.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Count allocate_at_least in the scratch-counting test allocator
From C++23 on, libc++'s containers allocate through allocate_at_least when
the allocator has one. The test allocator inherited it from std::allocator,
so the scratch allocations were not counted and the test failed on Xcode.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 🐛 fix BSON conformance issue
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 🐛 fix BSON conformance issue
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 🐛 reject ill-formed UTF-8 in CBOR/MessagePack/BSON text strings at decode time (#5531)
from_cbor()/from_msgpack()/from_bson() copied the raw bytes of a decoded
text string into the resulting json value without any UTF-8 validation,
even though RFC 8949 §3.1 (CBOR) and the MessagePack/BSON specifications
all require text strings to be valid UTF-8. Malformed input only failed
later, if the value was dump()'d, with a type_error.316 - so the
allow_exceptions=false pattern used specifically to get a discarded
sentinel instead of an exception did not discard this category of
malformed input, unlike every other kind of malformed binary input this
library rejects at decode time (see #5529).
Fix this at the single choke point shared by BSON/CBOR/MessagePack/UBJSON
string reads, binary_reader::get_string(): validate the bytes with the
UTF-8 DFA right after they are read, and report failures the same way as
every other binary_reader error (parse_error.113), so allow_exceptions
and strict discarding behave consistently. get_binary()/binary blob reads
are untouched and still accept arbitrary bytes, since only text strings
are required to be UTF-8.
There were two independent implementations of a UTF-8 validator: the
lexer's streaming scanner, and the serializer's Hoehrmann DFA used by
dump_escaped_impl(). Rather than write a third, the serializer's decode()
function, its utf8d table and the UTF8_ACCEPT/UTF8_REJECT constants are
extracted into detail/string_utils.hpp (a low-level header already
included before both detail/input/ and detail/output/), alongside a new
is_valid_utf8() helper built on the same decode() step. serializer.hpp's
dump_escaped_impl() now calls the shared decode(), so there is exactly
one UTF-8 validator in the codebase; dump()'s exact type_error.316
messages and byte-index reporting are unchanged (see the added
regression-guard test in unit-serialization.cpp).
Claude-Session: https://claude.ai/code/session_01N4RQ1Ahan5YAGbnAQGjZTY
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* ⚡ validate only newly read bytes of binary-format strings
get_string() validated the whole result after each call, but get_bytes()
appends to it and CBOR indefinite-length strings collect all chunks in
the same result, so every chunk re-validated everything read before it.
An input of many small chunks took quadratic time (80000 one-byte chunks,
160 KB of input, took about 7 seconds). Only the newly read bytes are
validated now, which also matches RFC 8949's requirement that every
chunk is valid UTF-8 on its own.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* Support zero-member types in NLOHMANN_DEFINE_TYPE_* macros (#4041)
NLOHMANN_DEFINE_TYPE_INTRUSIVE(Type) and its 11 sibling macros produced
broken code for types with no members to serialize. Invoking a variadic
macro so __VA_ARGS__ is empty is only standard-conforming since C++20,
so a plain __VA_OPT__ fix (as tried in #5142) breaks every pre-C++20
build under -pedantic. Instead, make all 12 macros purely variadic and
dispatch on argument count using a sentinel-padded extension of the
existing NLOHMANN_JSON_GET_MACRO idiom, giving full C++11-C++26 support
with no feature-test gate.
Verified against real GCC 16 and Clang at -std=c++11/14/17/20 with
-pedantic -Werror -Wvariadic-macros: zero regressions in the existing
unit-udt_macro.cpp suite plus 12 new zero-member test cases.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix CI failures in zero-member NLOHMANN_DEFINE_TYPE_* macros
Three issues surfaced on PR #5272's real CI that weren't caught by
local testing against a narrower flag set:
- GCC -Werror=noexcept: the four truly-empty from_json bodies (plain
INTRUSIVE/NON_INTRUSIVE, with and without _WITH_DEFAULT) provably
never throw but weren't declared noexcept; mark them noexcept
explicitly. to_json and the derived-type from_json overloads are
left alone since they genuinely can throw (object assignment /
delegating to the base class's from_json).
- clang-tidy bugprone-macro-parentheses: false positive on the same
8 zero-member bodies (Type/BaseType used purely as declarator
types); suppressed with NOLINTNEXTLINE comments in the same style
already used elsewhere in this file (see NLOHMANN_JSON_SERIALIZE_ENUM).
- MSVC's traditional preprocessor doesn't fully expand
NLOHMANN_JSON_CAT(prefix, NLOHMANN_JSON_TYPE_TAG(...))(...) in one
pass, which broke a pre-existing one-member usage in
unit-regression2.cpp with syntax errors. Wrap all 12 public
dispatcher macros in an extra outer NLOHMANN_JSON_EXPAND(...),
matching the pattern NLOHMANN_JSON_PASTE already uses for the same
MSVC quirk.
Re-verified against real GCC 16 and Clang at -std=c++11/14/17/20 with
-pedantic -Werror -Wvariadic-macros -Wnoexcept, including the exact
files that failed in CI (unit-udt_macro.cpp, unit-regression2.cpp),
against both the modular headers and the re-amalgamated single header.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix clang-tidy misc-const-correctness in unit-udt_macro.cpp
The four zero-member ONLY_SERIALIZE test objects are only ever read
(via to_json), never mutated, so mark them const per clang-tidy.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix derived-type macro dispatch capping members at 62 instead of 63
NLOHMANN_JSON_GET_MACRO resolves 64 positional arguments, with NAME at
position 65. NLOHMANN_JSON_TYPE_TAG dispatches on Type plus the member
list, so it resolves correctly up to the 63 members NLOHMANN_JSON_PASTE
supports. NLOHMANN_JSON_DERIVED_TYPE_TAG dispatched on the two-token
Type,BaseType prefix plus the member list, running out one slot early:
at 63 members, position 65 landed on the last member name instead of a
sentinel and NLOHMANN_JSON_CAT built an undefined identifier such as
NLOHMANN_JSON_DEFINE_DERIVED_TYPE_INTRUSIVE_m63, with the compiler
reporting "unknown type name 'm1'" once per member and nothing pointing
at an argument-count limit.
That silently reduced all six NLOHMANN_DEFINE_DERIVED_TYPE_* macros from
63 members to 62, contradicting the "up to 63 members" contract in
docs/mkdocs/docs/api/macros/nlohmann_define_derived_type.md.
Drop the leading Type and defer to NLOHMANN_JSON_TYPE_TAG so the tag is
computed from BaseType plus the member list, which fits the available
slots. The zero-own-member derived bodies are therefore selected by tag
1 rather than 2, and the sentinel table for the derived tag is no longer
needed.
Add a regression test at the documented maximum for both the plain and
the derived macros; it fails to compile against the previous dispatch.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Name the zero-member macro bodies by intent, not argument count
The dispatch tag was the literal token 1 or N, pasted onto a macro prefix
to select the zero-member or member-carrying body. For the derived-type
macros that reads wrong: their tag is computed after dropping the leading
Type, so the zero-member body was named _1 while taking two parameters
(Type, BaseType).
Emit EMPTY and MEMBERS instead. The mechanism is unchanged -- the tag is
still a token pasted onto the prefix by NLOHMANN_JSON_CAT -- but the body
names now say what they are rather than encoding an argument count that
only lines up for half of the macros.
Collapse the four duplicated zero-member bodies while here: with no
members there is nothing to default, so each _WITH_DEFAULT_EMPTY body was
a byte-for-byte copy of its plain counterpart. They are now one-line
aliases, leaving a single definition of what an empty object serializes
to per intrusive/non-intrusive and base/derived combination.
No functional change: for both zero-member and member-carrying types the
preprocessed to_json/from_json output is token-for-token identical, and
the arity limits are unchanged (63 members, base and derived).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Document zero-member support in the macro API reference
docs/mkdocs/docs/features/arbitrary_types.md already gained a note, but
the three api/macros pages are where the parameter contract is actually
specified and they still described member as a non-empty list.
State that the list may be empty on each page, and add a note showing
what the zero-member case generates: an empty JSON object for the plain
macros, and base-type-only serialization for the derived ones. Both notes
record that the WITH_NAMES variants do not support this.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Keep user macros named EMPTY or MEMBERS out of the member-count dispatch
The dispatch produced the bare token EMPTY or MEMBERS and pasted it onto
the macro prefix afterwards. In between, the token was rescanned, so a
user macro with either name replaced it: with `#define MEMBERS x` in
scope, even NLOHMANN_DEFINE_TYPE_INTRUSIVE(A, member) -- which compiled
before -- expanded to garbage, and `#define EMPTY` broke the zero-member
form.
Paste the suffix onto the prefix directly in the GET_MACRO slot table
instead. Operands of ## are not macro-expanded, so the selected body name
is formed before any user macro can interfere. NLOHMANN_JSON_TYPE_TAG and
NLOHMANN_JSON_DERIVED_TYPE_TAG become NLOHMANN_JSON_TYPE_BODY and
NLOHMANN_JSON_DERIVED_TYPE_BODY, taking the prefix as their first
argument; NLOHMANN_JSON_CAT is no longer needed. The body macro names are
unchanged, and so is the generated code.
Add a regression test that defines EMPTY and MEMBERS around plain and
derived types, with and without members.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Test for EMPTY and MEMBERS so -Wunused-macros accepts them
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Document the benchmarks, and make them build and compare versions again
The benchmark project hasn't configured since #4793: download_test_data.cmake
compiles cmake/detect_libcpp_version.cpp relative to CMAKE_SOURCE_DIR, which
is tests/benchmarks when that is the top-level project, so try_run fails and
so does `make run_benchmarks`. The path is now relative to the module itself,
which is the same file for the main build.
The Dump benchmark discarded dump()'s result, which is [[nodiscard]] by now;
it warned, and left the optimizer free to shorten the loop. The result is
now kept with benchmark::DoNotOptimize.
A new cache variable, JSON_BENCHMARK_INCLUDE_DIR, names the directory holding
the nlohmann/json.hpp to benchmark (single_include by default, as before),
so the same benchmarks can be built against two versions and compared.
tests/benchmarks/README.md documents what is measured, how to build and run
the benchmarks, how to read the output, and how to compare two versions with
Google Benchmark's compare.py; it recommends doing so by hand before a
release rather than in CI.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Point ci_benchmarks at tests/benchmarks
The target has configured ${PROJECT_SOURCE_DIR}/benchmarks since it was
added in #2561, but the benchmarks live in tests/benchmarks.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Pin Google Benchmark to release 1.9.5
The benchmarks fetched Google Benchmark's main branch, so two builds on
different days could measure with different library code, and CMake 3.30
and later warn that the single-argument FetchContent_Populate() is
deprecated. Fetch the 1.9.5 release archive, verified by its SHA-256,
with FetchContent_MakeAvailable() instead. That needs CMake 3.14; Google
Benchmark itself already needed 3.13.
Its -Werror is switched off, so a newer compiler's new warnings cannot
break the pinned release, and its install rules are no longer added.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Ubuntu, Windows, macOS, and CodeQL already cancel an older run of the same
workflow on the same ref. Check amalgamation, CIFuzz, Dependency Review,
Flawfinder, Semgrep, Scorecard, and the labeler did not, so every push
to a pull request left their earlier runs going. Give them the same
concurrency group. The labeler runs on pull_request_target, where
github.ref is the base branch, so it groups by pull request number.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Answer the OpenSSF Best Practices criteria that asked for policies the
project follows but had not written down:
- SECURITY.md: a first response within 14 days, publishing an advisory
with credit once a fix is released, and that only the latest release
receives security fixes.
- Governance: who has access to the project's resources, how write or
admin access is granted, and how CI secrets are stored and rotated.
- Quality assurance: how dependencies of the build, test, and
documentation tooling are pinned, scanned, and kept free of known
vulnerabilities.
Also update the assurance case, since comparison no longer recurses per
nesting level (#5390), and point the best practices badge and links to
bestpractices.dev under the program's current name.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
#5344 added two lines to unit-deserialization.cpp that clang-tidy
reports: modernize-return-braced-init-list for the remaining() helper
and readability-isolate-declaration for "json j1, j2, j3;".
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Improve error message for const fields
* Reject const arguments to get_to() with a clear message
Reword the static_assert, add it to the C array overload of get_to() as well,
and document that v must not be const.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Niels Lohmann <mail@nlohmann.me>
* Compare values without recursing, and without comparing them twice
Comparing two values compared their containers, which compare their elements,
which brought the comparison back once per nesting level. Two values nested
deeply enough exhausted the call stack and terminated the process with a
segmentation fault - the same bug as #5387, in the last operation that still
had it.
Worse, an ordered comparison took exponentially long in the nesting depth
before C++20. std::vector's operator< is a lexicographical comparison, which
asks whether an element is less than its counterpart and then whether the
counterpart is less than it - two full comparisons of everything below that
element, at every level. Comparing two equal values nested 30 levels deep,
which is nothing unusual, took 3.8 seconds; 40 levels would have taken an
hour, and nothing about the value has to be pathological to get there. C++20
is unaffected: std::lexicographical_compare_three_way asks once.
Compare a value that is nested too deeply to descend into on an explicit
stack instead, in a single pass that yields less, equal, greater or unordered
at once. Equality and the three-way comparison descend as they always did for
the first 128 levels, which nothing measurable costs them; an ordered
comparison no longer descends at all, which is what takes the exponent out of
it. Objects and arrays that are not nested deeply are otherwise compared
exactly as before.
The results are unchanged for every pair of values: 68121 comparisons of a
corpus that covers NaN, discarded values, mixed number types, binary values,
empty containers and both object types are identical to develop, in C++11,
C++17 and C++20, with and without thread_local storage and legacy discarded
comparison. Reproducing that meant reproducing two subtleties: a lexicographic
comparison steps over a pair it cannot order, where a three-way comparison
stops at it, and an object compares its keys with < where its entries are
ordered but with == where they are only checked for equality - not with the
object's own comparator, which for nlohmann::ordered_map tells equality.
Equality needs no ordering, so it no longer asks for any: a key or string type
that can only be compared for equality still works.
Measured (medians of 7 interleaved runs, clang -O3, C++11): comparing two
equal values nested 30 levels deep 3778 ms -> 0.002 ms; ordering flat objects
-33.6%; ordering flat arrays of numbers +27.3%, the one shape that pays for
the single pass; equality unchanged throughout.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Describe comparison in the no-thread-local docs and CI target
Comparing two values now bounds its descent with a thread_local counter
just as copying does, so the JSON_NO_THREAD_LOCAL page, the macro
overview and the ci_test_no_thread_local target cover both rather than
copying alone.
Also record what switching the macro on costs a comparison: on the
benchmark documents, comparing two equal values takes 10% to 90% longer.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Take the descent flag as an argument rather than testing it
MSVC reports the test of a constant as C4127 ("conditional expression is
constant"), which the Windows builds treat as an error: may_descend is
false for operator<, so the operand short-circuits the whole condition.
Passing it to compare_descent_exhausted() puts the test where the value
is an ordinary parameter, and leaves the call sites with no condition of
their own.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Note the comparison fallback in the no-thread-local documentation
The macro page describes what the library defines JSON_NO_THREAD_LOCAL for
by itself in terms of copying alone; comparing falls back the same way.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Parenthesise the reserve() computation in the comparison test
clang-tidy reports the mixed * and + as readability-math-missing-
parentheses, as it does for the identical line in the copy test.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Use the shared descent bookkeeping rather than a second set
Comparing kept a thread_local count, a limit and a guard of its own beside
the ones copying already had, all three the same thing under a different
name. They are gone; the shared count, limit and guard do the work.
The guard grows a second constructor here, because the comparison
operators are written as a macro and a macro cannot use the preprocessor:
it cannot look the count up behind an #ifdef the way copy_structured does,
so the guard looks it up for it. nesting_depth_exhausted() arrives for the
same reason - whether an operator descends at all is a constant at every
call site, and testing it there is what MSVC reports as C4127.
Also say in compare_leaves what happens to a pair that is an array on one
side and an object on the other, since the answer is not obvious from the
code: an operator only descends into two values of the same type, so such
a pair is told apart by its types alone - unequal, and ordered the way the
types are - exactly as it is above the bound.
And record what the explicit stack costs: the comparison operators are
noexcept and the container comparison this replaces allocated nothing, so
running out of memory here ends the process instead of throwing. It takes
a value nested past the bound and an exhausted heap to reach, and the same
comparison used to exhaust the call stack, but it is a new way to fail.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Amalgamate
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: qualify the operator>> stream positioning guarantee
operator>>'s notes state that it leaves the stream positioned right
after the parsed value, so that concatenated JSON values can be read
back to back. That does not hold when the value is a number: a number
is only terminated by the character that follows it, and the lexer's
unget() is simulated (it rewinds only the lexer's own bookkeeping),
so that character stays consumed from the stream.
Document the actual behaviour: the guarantee holds for all value types
except numbers, which must be followed by whitespace. Also qualify the
cross-reference on the JSON Lines page, which repeated the unqualified
claim.
Documentation only; the behaviour itself is tracked in #5340.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* fix: restore the character that terminates a number (#5340)
operator>> is documented to leave the stream positioned right after the
parsed value, so that concatenated JSON values can be read back to back.
That did not hold for numbers: a number is only terminated by the
character following it, and lexer::scan_number() reads that character
and calls unget() -- which is simulated and rewinds only the lexer's own
bookkeeping. input_stream_adapter consumes via sbumpc() with no matching
sungetc(), so the terminating character stayed consumed and the next
extraction started one byte too late ('1true' left the stream at 'rue').
Propagating unget() to the adapter directly does not work: next_unget
makes the following get() replay the cached character, so the terminator
would be delivered twice. Instead, restore the still-pending character
once at the end of a non-strict parse, where the input is handed back to
the caller:
- input_stream_adapter gains unget_character() (sungetc()) and advertises
it via supports_unget, detected the same way as supports_seek.
- lexer::restore_pending_unget() turns a pending simulated unget of a
real (non-EOF) character into a real one and clears next_unget so the
character is not also replayed. It is a no-op for adapters that cannot
unget, and reports failure when sungetc() fails, in which case the
input is left as it was before.
- parser calls it on the three non-strict paths, i.e. for operator>> and
sax_parse(strict = false).
Strict parse()/accept() are unaffected: they require the input to end
after the value, so the character is consumed by the end-of-input check
anyway. Parse error messages and reported positions are unchanged.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* tests: fix CI failures in the #5340 test helpers
Four CI failures, all in the new test code:
- GCC (-Werror=useless-cast): drop the `json(...)` wrapper around
`json::parse(...)`, which already returns a `json`.
- GCC (-Werror=unused-result): assign the discarded `json::parse()`
result to a dummy, the idiom used elsewhere in the test suite, and
catch `json::parse_error&` for consistency.
- clang-tidy (google-default-arguments): remove the default argument
from the `pbackfail()` override; `sungetc()` supplies the base
declaration's default.
- MSVC (bad allocation): `no_putback_streambuf::underflow()` set a
one-character get area without advancing `m_pos`, so an implementation
whose `istream::get` peeks before it bumps re-read the same character
forever. Keep no get area at all: `underflow()` peeks, `uflow()`
consumes, and `sungetc()` still always lands in `pbackfail()`, which
is what the test needs.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* fix: leave the character that terminates a number in the input
Read the character following a number without consuming it, instead of
consuming it and putting it back. input_stream_adapter now peeks with
sgetc() and only steps over the character when the next one is requested
or when the adapter is destroyed, so releasing it cannot fail - no
putback position is required from the streambuf.
Suggested by gregmarr in #5344.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: match the version history wording to the peek-based fix
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: drop the whitespace-separator caveat from the parsing pages
The caveat added in #5343 describes the behavior this branch fixes: a
number no longer consumes the character that terminates it, so
concatenated values need no separator.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* refactor: split the strict and non-strict paths in parser
Folding the release_lookahead() call into the existing strict check left
the "in strict mode" comment on an else-if branch, and made the strict
condition in sax_parse() redundant with the branch it followed.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Put the stream position fix behind JSON_PRECISE_STREAM_POSITION
Leaving the character that terminates a number in the stream is observable:
reading "1,2,3" with repeated operator>> works today only because the comma
after each number is swallowed, and std::getline after a number skips the
line break. Both break with the fix, so make it opt-in for 3.x, as suggested
by @gregmarr in the review.
- JSON_PRECISE_STREAM_POSITION (default 0) selects the peek-based
input_stream_adapter. Without it, the adapter is the consuming one from
develop and has no supports_lookahead, so lexer::release_lookahead() and
the parser's calls to it compile to nothing.
- The macro changes input_stream_adapter's layout and member functions, so
it gets the ABI tag _psp, after _bics. The ABI config tests, the natvis
generator, and nlohmann_json.natvis (regenerated) know the tag.
- The tests for the fix move to unit-precise-stream-position.cpp, which
defines the macro itself and runs in every build, and gain the two cases
above. unit-deserialization.cpp pins the default behavior instead.
- The docs describe the default behavior again and point to the new macro
page; version history says "added in 3.13.0, planned default in 4.0.0".
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The windows-11-arm runner now ships MSVC 19.51, which reports doctest's
forward declaration of std::tuple as C5285 ("cannot declare a
specialization for 'std::tuple'"). With /WX this breaks the msvc-arm64
job on develop and on every open pull request. Disable the warning for
the test targets, like the other MSVC warnings already disabled there.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This should have no effect for header only libraries as mentioned.
It was previously removed in e509007d but then accidentally added
again in 26cfec34.
Signed-off-by: Jeremy Nimmer <jeremy.nimmer@tri.global>
* Keep JSON_DIAGNOSTICS parent pointers of ordered_json members after erase() and update()
ordered_json stores its members in a vector, and two operations moved
members without restoring their parent pointers afterwards:
- ordered_map::erase() re-constructs every member after the erased one in
place. The basic_json move constructor leaves m_parent at nullptr, and
none of the object branches of basic_json::erase() (by key, iterator, or
iterator range) called set_parents(). This also affected merge_patch()
with a null member and patch() with a remove operation.
- update() only set the parent pointer of the inserted member. Adding a key
can reallocate the vector, which copies all other members and leaves
their m_parent at nullptr. The set_parents() call added for #4813 only
repaired this for the nested object of a merge, not for the target.
The next assert_invariant() on such an object (for instance, when copying
it) aborted, and diagnostic messages lost the path prefix above the moved
member. std::map-based json was not affected, because its nodes do not
move.
Erasing from an ordered_map object now calls set_parents(), and update()
uses set_parent(), which already refreshes all members for vector-based
objects. This makes the #4813 workaround redundant.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Account for JSON_DIAGNOSTIC_POSITIONS in the ordered_json parent-pointer test
The merge_patch() case parses its input, so with JSON_DIAGNOSTIC_POSITIONS
the exception message also carries the byte range of the parsed value.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Silence clang-tidy for the intentional copy in the ordered_json parent-pointer test
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Keep parent pointers when update() merges past its descent bound
The iterative path of update() only set the parent pointer of the member
it inserted, like the recursive one did before. It now uses set_parent()
too, so ordered_json members that move when a nested object grows keep
their parents, and the set_parents() calls that patched this up after
each nested merge are gone.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Check the fuzzers' UBJSON/BJData round-trip invariants in the unit tests
The strongest correctness checks for the UBJSON and BJData writers lived
only in the OSS-Fuzz drivers: anything from_ubjson()/from_bjdata()
returns must serialize with every option combination, parse back, and
re-serialize stably. Those checks only run at OSS-Fuzz, so regressions
surfaced days later as external reports - the same BJData assert pair
was reported five times over three years, and #5494's harness change
was followed by OSS-Fuzz 563659413 within a day.
Add "UBJSON round-trip invariants" and "BJData round-trip invariants"
test cases that run the drivers' checks on a fixed, deterministic corpus
(tests/src/round_trip_corpus.hpp): integer and float boundaries,
non-finite numbers, strings, binary values, optimized containers, deep
nesting, the JData annotated-array matrix, and seeded random containers.
They also check two properties the drivers do not: the first round trip
preserves the value, and re-serializing reproduces the exact bytes. For
BJData both exclude values containing a binary value, which is read back
as an array of integers unless it was written as a Draft 3 optimized
binary array; this carve-out is now documented in bjdata.md. Run against
the headers before #5542, the BJData test fails, including on the shape
from OSS-Fuzz 563659413.
Also document how OSS-Fuzz reports are handled (reference them as
"OSS-Fuzz: <id>", turn the reproducer into a unit test, keep drivers and
unit tests in sync) in tests/fuzzing.md, and link it from the PR
template and the quality assurance page.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Add the OSS-Fuzz reproducers for 474400817 and 474480402 as unit tests
Following the convention added to tests/fuzzing.md, the reproducers of
the two BJData fuzzer asserts tracked since January are now unit tests:
- 474400817 (assert(false)): an empty object _ArraySize_ was written as
the ND-array header length, which from_bjdata() could not read back.
Fixed by #5455.
- 474480402 (to_bjdata(j2, false, false) == vec2): a one-byte Draft 3
binary array is written in Draft 2 mode as a uint8 array and then
re-serialized with the int8 marker. This is the documented exception to
byte stability, not a library bug; OSS-Fuzz closed it after #5494
relaxed the harness to value stability. The test pins the exact bytes
so the exception stays deliberate.
The 563659413 reproducer is already a unit test (#5542). A comment also
ties the existing UBJSON excessive-count test to the timeout OSS-Fuzz
reported for that shape (testcase 6347769435193344).
OSS-Fuzz: 474400817
OSS-Fuzz: 474480402
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix GCC -Weffc++ and -Wuseless-cast warnings in the round-trip corpus
Initialize the atoms in the member initialization list, and drop the cast of
the generator's result, which already is std::size_t on 64-bit Linux.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Describe the threat model, the trust boundaries, the secure-design
argument, and how common weaknesses are countered, with links to the
quality assurance page as evidence.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Describe what the project will and will not do over the next year,
and point to issue #3453 for the open question of a 4.0 release.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Complete the architecture documentation page
Replace the placeholder bullets and TODOs with a description of the
component pipeline (with a diagram), the source layout, the template
parameters, the value storage (now in struct data), the input and
output adapters, and the SAX interface.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Link sources and basic_json, document full input adapter interface
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Align the default column of the template parameter table
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The windows-11-arm runner image moved to windows-11-vs2026-arm64, which no
longer ships Visual Studio 2022, so the msvc-arm64 job failed at configure
time. Use the "Visual Studio 18 2026" generator like the msvc2026 job.
clang-tidy's modernize-raw-string-literal check flagged two string literals
in the nesting tests added by #5546 and #5547.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Checking every depth up to 300 made test-json_patch exceed the 1500 s ctest
timeout in ci_test_valgrind. Check the depths up to 16, those around the
recursion limit of 128, and 300 instead.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
GCC's -Weffc++ (an error in CI) asks a class with pointer members, a
user constructor and a non-trivial destructor to declare its copy
constructor and copy assignment; diff_frame's vector and basic_json
members make its destructor non-trivial. Declare all five as defaulted,
which also satisfies clang-tidy's special-member-functions check. Leave
their exception specifications implicit: GCC 4.8 rejects an explicit
one that differs from the implicit one, as it does for flatten_task in
#5517.
The converting constructor cannot throw, and is now declared noexcept
for GCC's -Wnoexcept, which flags the emplace_back() under C++26
otherwise. The struct also moves from diff_iteratively() into the class,
like dump_frame in the serializer.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
diff() descended into both values once per nesting level, and compared
them with operator== on every level on the way, which recurses as well.
Values nested deeply enough - 25,000 levels on an 8 MiB stack - exhausted
the call stack and terminated the process, although parse() accepts
them without complaint. On such a chain the per-level comparisons and
path strings also made diff() quadratic in time and memory.
Both the recursion and operator== only descend as far as the source is
nested. So diff() first checks, recursing at most diff_depth_limit()
(128) levels, whether the source is nested more deeply than that. If not
- all but a vanishing minority of values - the recursive algorithm
diffs it exactly as before, now as diff_recursively(). Otherwise
diff_iteratively() walks the two values on an explicit stack, emitting
the same operations in the same order. It does not compare arrays and
objects with operator== up front (equal ones yield no operations
anyway), keeps the path in one buffer instead of a new string per
level, and hands every subtree that is not nested too deeply back to
diff_recursively(), so equal parts are still skipped quickly.
The check costs one pass over the source. On a 3,000-object document
that is about 30% of diffing two equal values (which is just an
operator== call), about 10% of diffing values that differ in a few
places, and noise when arrays change length. Once operator== no longer
recurses (#5390), the check can go.
Tests check that the patch reproduces the target at every depth up to
300, for json and ordered_json, including reordered members. They also
check the exact operation for a difference deep inside, and diff values
nested 100,000 levels deep.
Fixes#5393 for diff().
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Merge deeply nested objects without recursing per nesting level
merge_patch() and update(j, true) merged a nested object by calling
themselves on it, once per nesting level. A value nested deeply enough -
50,000 levels of objects on an 8 MiB stack - exhausted the call stack
and terminated the process, although parse() accepts such values without
complaint.
Bound the descent the same way dump() does. The recursion now carries
the nesting level, and once merge_depth_limit() (128) levels have been
entered, update_members_iteratively() and merge_patch_iteratively()
finish the merge on an explicit stack. They still merge a nested object
completely before the next member, and in the same order, so the results,
including the parents JSON_DIAGNOSTICS reports paths from, are unchanged.
Values nested less deeply than the bound run the same code as before, so
the common case does not pay for the stack: merging only on it cost
10-14% in a first version.
The public signatures are unchanged. The recursive worker behind
merge_patch() has its own name rather than being a private overload, so
that &basic_json::merge_patch stays unambiguous.
Tests check every depth up to 300 against recursive reference
implementations of both operations, check the diagnostic paths past the
bound, and merge objects nested 100,000 levels deep.
Fixes#5545 for update(j, true), and #5393 for merge_patch().
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Use the shared recursion limit in update() and merge_patch()
merge_depth_limit() is gone in favor of detail::recursion_depth_limit().
The two identical function-local frame structs become one member struct,
merge_frame, with a constructor, so both loops emplace_back() their
frames. merge_patch_iteratively() copies the frame it works on out of the
stack and changes it only through stack.back().
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Build the update()/merge_patch() diagnostics test values instead of parsing them
Parsed values carry byte positions under JSON_DIAGNOSTIC_POSITIONS, which
the expected messages do not include.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Hash deeply nested values without recursing per nesting level
std::hash<basic_json> hashed an array or object by hashing each element,
which called detail::hash again once per nesting level. A value nested
deeply enough - 50,000 levels of objects on an 8 MiB stack - exhausted
the call stack and terminated the process. parse() accepts such values
without complaint, since the parser is iterative, and a parsed value is
hashed wherever it is used as a key in an unordered container.
Bound the descent the same way dump() does: detail::hash takes the
nesting level, and once hash_depth_limit() (128) levels have been entered,
hash_iteratively() hashes what is left on an explicit stack. It combines
the seeds in exactly the same order, so hash values are unchanged. A value
nested less deeply than the bound is hashed by the same code as before,
without allocating, and is as fast as before.
Tests check that every depth up to twice the bound hashes exactly like
the recursive definition of the hash, and that values nested 100,000
levels deep hash without crashing.
Fixes#5545 for std::hash.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Declare hash_frame's constructor noexcept
GCC's -Wnoexcept (an error in CI) flags the emplace_back() into the
hash stack under C++26: the constructor cannot throw, since cbegin() is
noexcept, but it did not say so. dump_frame's constructor is noexcept
for the same reason.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Share one recursion depth limit, and copy the hash frame out of the stack
dump() and hash() each defined their own limit on how many nesting levels
they recurse into, and the operations still to come would have added more,
free to diverge over time. They now all use detail::recursion_depth_limit(),
in a header of its own; serializer::dump_depth_limit() and
hash_depth_limit() are gone.
hash_iteratively() now copies the frame it works on out of the stack and
changes the frame only through stack.back(), so nothing can refer into
the stack after entering an element has grown it.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Parenthesize multiplications in the hash test for clang-tidy
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Add missing headers to BUILD.bazel and make its generator reproduce it
The "json" cc_library did not list three headers that the library
includes:
- detail/meta/logic.hpp (added in #5016, included by from_json.hpp)
- detail/input/number_parse.hpp (added in #5283, included by lexer.hpp)
- detail/input/string_scan.hpp (added in #5283, included by lexer.hpp
and serializer.hpp)
Bazel's sandbox only exposes declared headers, so any target depending
on @nlohmann_json//:json and including <nlohmann/json.hpp> failed with
"'nlohmann/detail/meta/logic.hpp' file not found".
The file could not simply be regenerated, because the generator behind
"make BUILD.bazel" was stale: it wrote only the "json" cc_library and
dropped the load() statements, the license block, and the
"singleheader-json" target that were added by hand in #4584. The
generator now emits the complete file, so its output differs from the
previous BUILD.bazel only by the three headers. It also resolves the
glob against the project root instead of the working directory and
sorts the list explicitly.
"make BUILD.bazel" is now phony: in a fresh checkout, BUILD.bazel is
not older than the headers, so make considered it up to date, and a
removed header would never trigger a rebuild. "make check-amalgamation"
also checks that BUILD.bazel is up to date.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Check in CI that BUILD.bazel is up to date
The "Check amalgamation" workflow now also regenerates BUILD.bazel, so a
pull request that adds, renames, or removes a header without updating
the Bazel header list fails, and the attached amalgamation.patch
contains the fix. The failure comment and the contribution guidelines
mention the new check, and the comment now links to the existing
"Amalgamate the source code" section instead of the "Files to change"
anchor that was removed in #4560.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Test JSON_BRACE_INIT_COPY_SEMANTICS for real, and fix one-element tuples under it
The opt-in JSON_BRACE_INIT_COPY_SEMANTICS was never exercised by CI:
- Its only test, in unit-regression3.cpp, was guarded by
`#if defined(JSON_BRACE_INIT_COPY_SEMANTICS)` after the #include. The
header #undefs the macro unconditionally in macro_unscope.hpp, so the
guard was always false and the test compiled to nothing, whatever -D
flag was passed.
- The ci_test_brace_init_copy_semantics target that passes the flag was
not named by any workflow.
Move the test into its own translation unit that defines the macro before
including the header, as unit-diagnostics.cpp does for JSON_DIAGNOSTICS.
It now runs in every CI job and for every standard. Remove the unused
target: it ran the whole suite with the macro, and that suite deliberately
relies on default brace-init semantics in about 90 places
(e.g. `json({1})` meaning `[1]`), so it could never pass.
Running the whole suite with the macro did find one library bug:
to_json for std::tuple builds `j = { std::get<Idx>(t)... }`, so with copy
semantics a one-element tuple became its element. `json(std::tuple<int>{5})`
was `5` instead of `[5]`, and `get<std::tuple<int>>()` threw type_error.302
on the result. Under the macro, a one-element tuple now builds exactly what
the default deduction builds. Without the macro nothing changes.
The new tests also pin that the library's other conversions produce the
same values with and without the macro. The macro page now says that the
macro affects every single-element list (`json j = {1}` is `1`), and that
all translation units must agree on it, since it has no ABI tag.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Make JSON_BRACE_INIT_COPY_SEMANTICS part of the ABI tag
The macro changes the body of the initializer-list constructor and adds a
to_json_tuple_impl overload, both with the same mangled names in either
mode, so mixing translation units silently picked one definition. Encode
it in the inline namespace as `_bics`, as JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON
does with `_ldvcmp`. The macro is new in the unreleased 3.13.0, so no
existing namespace name changes.
- Move the macro's default into abi_macros.hpp so json_fwd.hpp computes
the same namespace, and keep it defined under JSON_TEST_KEEP_MACROS.
- Check the tag in the ABI config tests and in the unit test.
- List `_bics` (and the missing `_dp`) in the namespace docs and in the
natvis generator; regenerate nlohmann_json.natvis.
- Replace the "define it consistently" warning with an ABI note.
Suggested by @gregmarr in the review of #5544.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix the cppcheck, clang-tidy and legacy-comparison CI failures
- to_json_tuple_impl() moved the element in both branches of a ternary;
only one runs, but cppcheck reported accessMoved. Use if/else.
- The ABI tag test looked for "json_abi_bics", which misses when another
tag comes first, as in json_abi_ldvcmp_bics; look for "_bics".
- readability-qualified-auto in the items() test.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
write_ubjson() built a std::vector of the eight markers BJData forbids as
the type of an optimized container - one heap allocation plus a linear
search for every array and object it wrote with use_type, even for plain
UBJSON output, where the list isn't consulted. The list was also spelled
out twice. A constexpr helper, is_bjdata_excluded_type_marker(), replaces
both.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
- "aspect: binary formats" for changes to the binary reader or writer,
their tests, fuzzers and docs, or with a binary format in the title;
- "python" for Python sources and pip requirements files, matching the
label Dependabot sets on its pip updates, so it is never removed there;
- "CI" also for changes to the Dependabot and labeler configurations.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The fuzzer drivers check their round trips with assert(), which NDEBUG
compiles away. The OSS-Fuzz build keeps assertions on today, but nothing
pins that: a build change that adds NDEBUG would silently turn every
round-trip check into a mere "does not crash" check. Each driver now
stops the build with an #error instead, and includes <cassert> itself
rather than relying on json.hpp.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The community-maintained clang-tidy check
modernize-nlohmann-json-explicit-conversions rewrites implicit
conversions into explicit get<T>() calls, which is exactly the
preparation the docs ask for ahead of implicit conversions being
switched off by default. Mention it on the JSON_USE_IMPLICIT_CONVERSIONS
page and in the migration guide, as promised in discussion #4610.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Without a bind address in serve_header.yml, the server listened on all
interfaces, so any machine on the network could fetch the header and
trigger make runs in the working trees. It now listens on localhost
unless configured otherwise; bind: null restores the old behavior.
The header was also sent with Access-Control-Allow-Origin: *, letting
any web page read it. CORS is only needed because Compiler Explorer
downloads #include <https://...> headers in the browser, so the header
now goes only to https://godbolt.org and https://compiler-explorer.com,
configurable with cors_origins.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
write_bjdata_ndarray() encoded a JData-annotated object as a BJData
ND-array whenever its dimensions' product matched _ArrayData_.size(),
which lost information in two ways:
- _ArrayData_ was never required to be an array. null has size 0, any
other scalar has size 1, and iterating an object visits its values, so
e.g. {"_ArraySize_":[1],"_ArrayData_":5} was written as the array [5],
and an object _ArrayData_ came back as an array.
- The reader only restores an annotated object from an ND-array with at
least two non-zero dimensions that is not a 1xN row vector; an empty,
1-D, row-vector, or zero-sized shape is read back as a plain array. The
writer nonetheless emitted ND-array headers for these shapes, so the
annotation was silently dropped.
OSS-Fuzz issue 563659413 hit this in parse_bjdata_fuzzer: an empty binary
_ArraySize_ is written as a plain object and read back as an empty array,
after which {"_ArrayType_":"int16","_ArraySize_":[],"_ArrayData_":null}
was encoded as the ND-array header "[$I#[]" and re-read as [], failing the
harness's value-stability check.
Such objects now fall back to a plain object encoding, which round-trips.
Genuine ND-arrays (two or more positive dimensions, not a 1xN row vector)
are encoded exactly as before. Existing fallback tests that used 1-D
shapes are moved to 2-D shapes so they keep exercising the check they
were written for, and the BJData documentation is updated.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Match ABI tag order in namespace tests to abi_macros.hpp
NLOHMANN_JSON_ABI_TAGS concatenates the tags as _diag, _ldvcmp, _dp,
but the default and noversion ABI tests expected _diag, _dp, _ldvcmp.
The tests therefore failed whenever both JSON_DIAGNOSTIC_POSITIONS and
JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON were enabled, a combination
CI never exercises. Reorder the expectations to match the header.
Also document the _dp tag in the namespace feature page, which listed
only _diag and _ldvcmp.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Test the ABI namespace with all ABI tags enabled
Build the default and noversion ABI config tests a second time with
JSON_DIAGNOSTICS, JSON_DIAGNOSTIC_POSITIONS and
JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON all set, so the expected tag
order is checked on every test run instead of depending on which CMake
options a CI job happens to enable.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Bound the descent of the copy constructor
basic_json's copy constructor copied objects and arrays by handing the
container to its own copy constructor, which copy-constructs every element
and so reaches this constructor again, once per nesting level. A value
nested deeply enough exhausted the call stack and terminated the process
with a segmentation fault - no exception, nothing the caller could catch.
Parsing such a value works, as the parser is iterative, and so does
destroying one, as #1436 made destruction iterative.
Bound how far the copy descends rather than take the call stack away from
it. The first levels are copied exactly as they were - the containers copy
their own elements, which is by far the fastest way to fill them - and only
once the copy has descended 128 levels is the value below it finished
without the call stack, through an explicit worklist. Copying can therefore
no longer exhaust the stack, however deeply a value is nested, while a value
nested less deeply than the bound - all but a vanishing minority - is copied
by the very same code as before and pays only for one counter.
That counter lives in thread_local storage, as one shared between threads
would be raced. JSON_NO_THREAD_LOCAL switches it off for toolchains without
thread_local; copying then goes through the worklist right away, which
yields the same values but is measurably slower.
The deferred values are completed before the copy they belong to returns, so
a value copied while another copy is going on - by a custom base class, say -
is unaffected by the copy it is nested in.
operator= takes its argument by value, so copy assignment is fixed as well.
Copying is as fast as it was, within measurement noise (medians of 9
interleaved runs, clang -O3): -1.3% for an array of strings, +0.0% for a
flat object, +0.1% for a flat array of numbers, +0.3% for nested arrays,
+0.6% for nested objects and +1.2% for a twitter-like document. Copying a
three-key object costs about ten nanoseconds more, the counter. Deferring
every level instead, rather than only those below the bound, measured
between 3% and 9% slower depending on the shape of the value.
This fixes#5387 for the copy constructor. dump() is still recursive.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Test the copy constructor's iterative path in CI
The copy constructor descends into 128 levels before it finishes a value
without the call stack, so the iterative path is otherwise only reached
by the few tests that nest deeper than that.
JSON_NO_THREAD_LOCAL switches the descent off, which sends every value
down that path. Running the whole test suite that way covers it with
every object type, string type, allocator, and base class the suite
already exercises. The new ci_test_no_thread_local target does that; the
macro had no build coverage at all before.
Copying a nested value also has to carry over what the element-wise copy
constructor would have copied: the parents that JSON_DIAGNOSTICS relies
on, and the positions that JSON_DIAGNOSTIC_POSITIONS reports. Both are
now checked on either side of the descent bound, for objects and arrays.
Neither was tested before, and dropping either one makes the new tests
fail.
Also quantify what JSON_NO_THREAD_LOCAL costs a copy instead of calling
it "measurably slower".
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Split the regression tests so that they keep linking
Linking test-regression2 fails with "relocation truncated to fit:
IMAGE_REL_AMD64_REL32 against `.rdata'" once its object grows past what
the MinGW linker copes with, and the copy constructor's helpers push it
over: the object grows by 6.3%, from 4,654,128 to 4,944,920 bytes at -O0,
and develop links at the smaller of the two.
Building the tests optimized shrinks the object enough to link, but the
binaries clang 11.0.1 and clang 18.1.8 then produce crash before doctest
prints its first line - 39 of 102 tests on clang 18 - so the objects have
to become smaller rather than denser.
Moving the test cases that follow "regression tests 2" into a file of
their own brings that object to 4,687,888 bytes, which is 0.7% above the
size that links today rather than 6.3%. Both files still build for C++11,
C++17 and C++20, and run the same 9 test cases and 135 assertions as
before, now spread over two binaries.
New regression tests belong in unit-regression3.cpp from here on, which
is what CONTRIBUTING.md now says.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Do not use thread_local storage with Clang targeting MinGW
Every test that copies a value segfaults there - 42 of 105 on clang
11.0.1, 39 of 102 on clang 18.1.8 - while the same tests pass with GCC
targeting MinGW, with Clang targeting MSVC, and with every other
toolchain the library is tested on. The counter that bounds the copy
constructor's descent is the library's first use of thread_local, so
that job had never exercised it before.
JSON_NO_THREAD_LOCAL already covers toolchains without thread_local
storage, and copying yields the same values with it, only more slowly.
Define it for this one automatically.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Balance the warning suppression the split separated
unit-regression2.cpp opens a DOCTEST_CLANG_SUPPRESS_WARNING_PUSH block at
the top and closed it at the very bottom, which the split moved into
unit-regression3.cpp: one file was left with a push and no pop, the other
with a pop and no push, which clang reports as an error.
Give each file the pair it needs.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Check both shapes without a C-style array
clang-tidy rejects the array the two shapes were iterated over
(cppcoreguidelines-avoid-c-arrays). The array only existed because astyle
reformats a range-for over a braced initializer list into something
unreadable; naming the two cases avoids both.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Split the regression tests far enough to leave room
The first split left unit-regression2.cpp 0.7% below the size develop
links at, which the comparison change in the follow-up immediately used
up: the MinGW linker fails on test-regression2_cpp20 again, naming
copy_shallow and to_partial_ordering among the relocations it cannot fit.
Move the sections from "issue #2067" on, and the helper types they use,
so that the file stops being the one that decides whether the tests can
be linked at all. At -O0 and C++20, unit-regression2.cpp is now 2,964,944
bytes against develop's 4,708,248, and 3,070,568 bytes with the follow-up
applied - roughly a third smaller either way, rather than a fraction of a
percent larger.
The 135 assertions are the same ones as before, now spread over three
test cases in two files.
Also silence the clang-tidy findings the deep-nesting tests draw: the
copies they make are what is being tested, and the reserve() computation
gets its parentheses.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Move the #4804 alias to the file that uses it
The split left the json_4804 alias behind in unit-regression2.cpp while
the test case that uses it went to unit-regression3.cpp, which does not
build for C++17 and C++20 as a result.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Include <span> where the split moved its only use
The #2546 test case guards itself with __has_include(<span>), but the
include itself sat in unit-regression2.cpp's preamble and stayed behind,
so the section compiled without a declaration wherever the guard passed -
which nvhpc reported and libc++ builds do not, as they skip the section
altogether.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Keep the descent bookkeeping in one place
Copying carried a depth count, a depth limit and a guard of its own, and
the comparison in the follow-up added a second set beside them. Neither
operation needs its own: they are never nested inside one another by the
library - copying a value does not compare one, and comparing two values
does not copy them - and where user code nests them anyway, sharing the
count only ends a descent sooner than it had to.
So there is now one nesting_depth(), one nesting_depth_limit() and one
nesting_depth_guard, which the follow-up uses instead of adding its own.
Inverting the test in copy_structured leaves the too-deep case and the
no-thread-local case as the same code.
The guard takes the count rather than looking it up, because the caller
has looked it up already to test it against the limit, and reaching
thread-local storage twice on the path that is taken almost every time is
worth avoiding.
The switch that copies the value of anything that is not an object or an
array was written twice - once in the copy constructor, once in
copy_shallow - so that adding a value_t meant editing both, and missing
one would have been silent. It is copy_leaf_value now, and inlined: both
callers have already sorted the containers out, and folding that test into
the switch is what keeps a value made mostly of numbers copying as fast as
it did.
Copying canada.json, citm_catalog.json and twitter.json is within 0.6% of
what it was before, measured as a paired ratio over 18 interleaved rounds
against a run-to-run spread of 0.3%.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Check that an abandoned copy can still be destroyed
Copying a value without the call stack builds the copy from the top down,
and every value whose own copy has not been made yet stays a null value
until it is. That is what lets a copy be abandoned half-built: the
destructor finds nothing but complete values and null ones.
Nothing tested it. Failing an allocation part-way through a copy of a
deeply nested value does, with the allocator the file already has for
exactly this kind of test.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Name the test's locals so Flawfinder stops matching them
The code scanning job reports CWE-362 - "check when opening files" - for
a test that opens no files: Flawfinder matched a local variable called
open. Rename it and its partner.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Keep the descent guard's bookkeeping self-contained
nesting_depth_limit() and nesting_depth_guard were only used inside
the JSON_NO_THREAD_LOCAL-guarded branch of copy_structured(), but were
defined unconditionally. Move them inside the #ifndef, and have the
guard look up the depth and test it against the limit itself (via
okay()) instead of making the caller do it - the caller no longer
needs to touch nesting_depth() at all. Also shrink the thread-local
counter to std::uint8_t, matching what its own doc comment already
argued.
Addresses gregmarr's review comments on #5389.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Make nesting_depth_guard usable regardless of JSON_NO_THREAD_LOCAL
nesting_depth_limit() and nesting_depth() stay behind #ifndef
JSON_NO_THREAD_LOCAL, since a descent cannot be bounded without a
per-thread count. But the guard itself now always exists, becoming a
no-op that is never okay() under that macro - the same way the bound
is already reached on every call without one. copy_structured() no
longer needs to know which case it is in.
This is what lets #5390 reuse the guard for comparison, which cannot
test JSON_NO_THREAD_LOCAL where the macro-based operators use it: the
guard now carries that distinction itself instead of requiring every
caller to.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Silence VS2015's C4503 for the custom-base-class test
The deep-copy support added for #5387 lengthened the mangled name of
std::allocator_traits<...>::construct for the test's map type past
VS2015's limit, which /WX turns into a build failure even though the
name is only used for (now-truncated) debug info.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Remove dead unused-parameter casts from copy_metadata()
@gregmarr asked whether the static_cast<void> pair in the
JSON_DIAGNOSTIC_POSITIONS-off branch was needed for an empty
json_base_class_t. It isn't: src and dst are already referenced
unconditionally by the base-class copy above, so no -Wunused-parameter
warning fires either way (checked with -Wall -Wextra
-Wunused-parameter, JSON_DIAGNOSTIC_POSITIONS 0 and 1).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix CI: build custom array types without a fill constructor, re-amalgamate
copy_array_level() built the destination array with the fill
constructor array_t(count, value), which is not part of the array
container interface the library otherwise assumes (e.g. custom
ArrayTypes that only provide a default and an iterator-pair
constructor, as covered by unit-custom-array-type.cpp). Default-
construct the array and resize() it instead, matching how the rest
of the codebase already grows array_t.
Also re-run the amalgamation, which had fallen out of sync with
include/nlohmann/json.hpp.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* docs: document that a NUL byte in the input is treated as end of input
A NUL byte anywhere in the input - trailing, or embedded ahead of more
otherwise well-formed JSON - is currently treated the same as genuine
end of input, so parsing silently stops there instead of raising the
parse_error.101 any other unexpected byte triggers. This mirrors the
NUL-terminated-C-string convention already used when no explicit input
length is given (json::parse(const char*) already stops at strlen()),
just applied uniformly rather than only when a length is genuinely
unavailable.
This behavior predates this change and is not being altered here -
changing it would be an observable, backwards-incompatible behavior
change for any caller that (knowingly or not) depends on it, which is
not something to do silently in a patch. Documenting the current,
verified behavior as a new FAQ entry instead, so it's an intentional
and discoverable part of the contract rather than a surprise.
Fixes#5530.
Signed-off-by: Niels Lohmann <niels.lohmann@gmail.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N4RQ1Ahan5YAGbnAQGjZTY
* Add JSON_STRICT_NUL_HANDLING opt-in macro for issue #5530
A NUL byte anywhere in the input is currently treated the same as real
end of input, rather than raising parse_error.101 like any other
unexpected byte (documented in the previous commit's FAQ entry). A full
unconditional fix was tried in PR #5532 but rejected as too risky to
ship by default: any caller could depend on the current behavior, even
unknowingly (e.g. a zero-padded buffer). On PR #5534, gregmarr proposed
a compile-time opt-in flag instead, and the maintainer agreed, wanting
it available now and defaulting to the corrected behavior in 4.0.0.
This mirrors the existing JSON_BRACE_INIT_COPY_SEMANTICS precedent as
closely as sensible:
- JSON_STRICT_NUL_HANDLING defaults to 0 (off); the three lexer sites
that treat '\0' as EOF/comment-terminator are gated with
`#if !JSON_STRICT_NUL_HANDLING` so the default-off behavior is
byte-for-byte identical to today's.
- input_adapters.hpp's `T (&array)[N]` overload additionally trims a
single trailing '\0' from a `char` array (e.g. a string literal like
`json::parse("123")`) when the macro is on, so that case keeps
working; every other element type (unsigned char, std::uint8_t, ...)
always keeps its full extent. This intentionally does *not* reuse the
existing strlen()-based pointer overload via SFINAE-excluding `char`
from the array overload, as originally sketched for this change: that
approach is ambiguous against the newer generic container overload
added since PR #5532, and even where it compiles, strlen()-scanning a
`char` array that is not NUL-terminated within its bounds reads past
the end of the array (confirmed with AddressSanitizer). Trimming only
a single trailing byte, without scanning, avoids both problems.
- Documented via docs/mkdocs/docs/api/macros/json_strict_nul_handling.md,
linked from the macros index/nav/features page, the FAQ entry, and
the parse/accept/operator>> reference pages.
- Tested in unit-class_parser.cpp and unit-deserialization.cpp, default
state unguarded and opt-in state guarded. Since the library itself
#undefs the macro at the end of json.hpp (as JSON_BRACE_INIT_COPY_SEMANTICS
already does), a plain `#if defined(JSON_STRICT_NUL_HANDLING)` guard
after the include never actually triggers; the tests instead capture
the command-line value into a test-local macro before including the
header. A few pre-existing fixtures elsewhere (std::array<uint8_t, N>
sized one larger than their literal, relying on value-initialization
to silently add a trailing zero byte) needed the same one-byte
adjustment to keep passing under the opt-in behavior.
Unlike the precedent, this adds a proper `JSON_StrictNulHandling` CMake
option (rather than a raw -DCMAKE_CXX_FLAGS injection) and wires its
ci_test_strict_nul_handling target into the ci_cmake_options job matrix
in .github/workflows/ubuntu.yml, so the opt-in build is actually
exercised in CI -- closing the one gap in the precedent's own CI setup
(ci_test_brace_init_copy_semantics is defined but never referenced by
any workflow, so it has never actually run).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Clarify where JSON_STRICT_NUL_HANDLING does not reject NUL bytes
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <niels.lohmann@gmail.com>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Add 60 contributors whose work was not yet credited and update seven
links that pointed to renamed or reassigned GitHub accounts.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Devirtualize binary_writer via a value-type output sink
to_cbor/to_msgpack/to_ubjson/to_bjdata/to_bson wrote every byte through
output_adapter_t, a shared_ptr<output_adapter_protocol> whose
write_character/write_characters are virtual. Unlike the lexer (templated
on a concrete InputAdapterType), the binary writer never got that
treatment, so binary output paid a vtable lookup per byte and a
make_shared per call.
Template binary_writer on an OutputSinkType and give it two concrete,
non-virtual sinks:
- output_vector_sink: appends straight into a std::vector (push_back /
insert), used by the vector-returning to_* convenience functions. No
vtable, no shared_ptr; the writes inline.
- output_adapter_sink: forwards to a type-erased output_adapter_t, so the
existing to_*(j, output_adapter) overloads (streams, strings, custom
adapters) keep working exactly as before -- one virtual call each,
unchanged.
binary_writer keeps a convenience constructor taking output_adapter_t
(building the default output_adapter_sink), so the adapter overloads are
untouched; only the convenience functions switch to the vector sink. The
friend declaration and the basic_json binary_writer alias gain the new
(defaulted) template parameter.
Output is byte-for-byte identical: verified across ~3000 randomized
values plus curated edge cases (all scalar widths, strings with invalid
UTF-8, binary, nested arrays/objects) for CBOR, MessagePack, UBJSON (both
size/type settings), BJData, and BSON, plus the output_adapter path, in
C++11/17/20. Warning-clean under clang -Weverything and the gcc pedantic
set; clang-tidy clean on the changed headers; make check-amalgamation
clean.
Throughput (g++ -O3, vs develop): scalar-dense binary output such as
integer arrays ~1.4x; many small to_cbor calls ~1.04x (DOM traversal
bound); string/blob-heavy output unchanged (already bulk-bound). No
workload regressed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix CI failures from binary_writer output-sink change
Four CI jobs failed on the initial commit; all are addressed here without
changing any output (binary encodings remain byte-for-byte identical to
develop across the differential corpus):
1. ci_test_gcc / cuda (-Werror=duplicated-branches): for number_float_t ==
float, static_cast<float>(n) is the identity, so write_compact_float's
two branches are intentionally identical. Once the concrete vector sink
is inlined, GCC constant-folds and diagnoses this (the type-erased path
hid it behind a non-inlined virtual call). Silence -Wduplicated-branches
for GCC (clang has no such warning) alongside the existing -Wfloat-equal
pragma.
2. ci_static_analysis_clang (UBSan nonnull-attribute): binary_writer passes
a null pointer with length 0 for empty strings/binary. output_vector_sink
/ output_adapter_sink declared write_characters JSON_HEDLEY_NON_NULL, so
the sanitizer flagged the (harmless) zero-length call once the sink was
called directly rather than through the attribute-free virtual base. Drop
the attribute from both sinks, matching the pre-existing behavior.
3. ci_cpplint (build/include_what_you_use): output_adapter_sink uses
std::move; add #include <utility>.
4. ci_cuda_example (nvcc 11.8): NVCC's front end rejects the default
template argument on the binary_writer alias template. Revert the alias
to its original single-parameter form (relying on binary_writer's own
defaulted OutputSinkType) and spell out the full type in the vector-sink
convenience functions.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Encode big-endian numbers with a byte swap instead of std::reverse
write_number() reordered multi-byte numbers for the big-endian formats
(CBOR/MessagePack/UBJSON) with std::reverse over the byte array. GCC
lowered only some sizes to a bswap; clang kept a scalar byte shuffle
(0 bswap instructions in the CBOR number path). Replace the reverse with
size-dispatched __builtin_bswap16/32/64 helpers (portable shift fallback
for other compilers; std::reverse retained for exotic sizes such as a
long double number_float_t).
Codegen: the CBOR number path now emits bswap on both compilers
(gcc 2 -> 16, clang 0 -> 4). Output is byte-for-byte identical to the
previous implementation across the binary differential corpus.
Throughput (isolated vs the std::reverse version, best of 9):
CBOR int64 array gcc +7% clang +10%
CBOR uint16 array gcc +27% clang flat
Modest but consistent on number-dense encodings; negligible on
string/blob-heavy output, as expected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Reserve output capacity up front for binary serialization
The vector-returning to_cbor/to_msgpack/to_ubjson/to_bjdata/to_bson grew
the output buffer purely by geometric reallocation. Reserving an estimate
up front avoids the early reallocations, which is the dominant per-byte
cost for array/object-heavy output.
The estimate (binary_reserve_hint) is deliberately conservative and safe
against untrusted input: it consults only the top-level element count
(O(1), no walk of the DOM), guards the multiplication against overflow,
and clamps the result to a fixed 1 MiB ceiling, so a large or hostile DOM
can never force an oversized allocation here. The buffer still grows
geometrically past the hint, so an underestimate only costs a few later
reallocations; scalars/strings/binary are written in one shot and get no
hint. Reserving capacity does not change the bytes produced.
Throughput (g++/clang -O3, vs the previous commit):
cbor int array +10% / +13%
cbor object array +20% / +38%
Output is byte-for-byte identical to develop across the binary
differential corpus.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Address review findings on the binary writer output sinks
- binary_reserve_hint(): the 4-bytes-per-element estimate over-reserved by up
to 4x for arrays of small scalars (CBOR encodes 0..23 in one byte), and the
returned vector kept that capacity. Make the hint a strict lower bound on the
encoded size instead, which also removes the 1 MiB clamp whose branch no test
could reach (the largest container in the suite has 65793 elements).
- Guard the -Wduplicated-branches pragma with __GNUC__ >= 7. The warning does
not exist before GCC 7, so naming it made GCC 4.8/4.9/5/6 - which the CI
matrix still builds - warn under -Wpragmas on every including translation
unit, breaking downstream -Werror builds.
- Constrain the adapter constructor of binary_writer with the enable_if its
documentation already claimed, so a writer over some other sink type is no
longer advertised as constructible from an output adapter.
- Let output_vector_adapter wrap output_vector_sink rather than duplicating the
append logic, so the type-erased and templated paths share one implementation.
- Collapse the three copies of the memcpy/byte_swap/memcpy dance into a single
byte_swap_buffer() helper, and add the MSVC _byteswap_* intrinsics so MSVC no
longer falls back to the scalar shuffle this change exists to eliminate.
- Add a vector_writer() helper for the five vector-returning to_* overloads
instead of spelling out the writer type at each call site, and drop a dead
default member initializer on output_adapter_sink.
- New tests: the vector sink and the adapter sink must produce identical bytes
for every format (the two to_* overloads no longer delegate to each other and
could otherwise drift), and binary_reserve_hint() must never exceed the size
actually written.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Route the -Wduplicated-branches pragma through Hedley
Match #5485, which moved the binary writer's hand-rolled diagnostic
pragmas onto JSON_HEDLEY_PRAGMA (merged into develop while this branch
was open). The devirtualization's -Wduplicated-branches suppression in
write_compact_float was the one raw '#pragma GCC diagnostic' left; it
now uses JSON_HEDLEY_PRAGMA like the adjacent -Wfloat-equal line, still
guarded to GCC >= 7 and non-clang (the warning exists only there).
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* Reject MessagePack/BSON binary subtypes that don't fit their wire format
Both formats store byte_container_with_subtype's subtype (a uint64_t)
in a single byte. The writers cast to std::int8_t/std::uint8_t without
a range check, so subtypes above 255 were silently truncated modulo
256 instead of raising an error. Throw out_of_range.413 instead when
the subtype exceeds the representable range of 0-255.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Move the new binary-subtype regression test out of unit-regression2.cpp
unit-regression2.cpp is already at the edge of what the MinGW linker
can relocate; adding this test's ~26 lines tips test-regression2_cpp20
(clang, Windows) over into "relocation truncated to fit:
IMAGE_REL_AMD64_REL32 against `.rdata'" (see 8ce64b9c1 / b82717c8a for
the same failure mode). Split the test along format lines instead:
MessagePack assertions move to unit-msgpack.cpp, BSON assertions to
unit-bson.cpp. The CBOR round-trip guard is dropped as redundant --
unit-cbor.cpp's "Tagged values" section already round-trips subtypes
up to 8589934590, far past the 70000 checked here.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The array-range insert() overload checked that pos fits the current
value and that first/last share the same owning value, but never
verified that value is itself an array. Passing iterators from an
object, a primitive, or null handed value-initialized (singular)
std::vector iterators straight to array_t::insert(), which is
undefined behavior. Add the missing is_array() check, mirroring the
equivalent check already present in the object-range insert()
overload.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Honor allow_exceptions=false for excessive array/object size (out_of_range.408)
The SAX DOM parsers' start_object()/start_array() threw out_of_range.408
directly via JSON_THROW when a binary format (CBOR/UBJSON/BJData) declared
a container size exceeding max_size(), bypassing the allow_exceptions flag
that every other malformed-input error path in these classes honors via
parse_error(). This meant that json::from_cbor(data, true, false) etc.
could still throw (or abort under JSON_NOEXCEPTION) instead of returning a
discarded value, contrary to the allow_exceptions=false contract.
Route all four call sites (two in json_sax_dom_parser, two in
json_sax_dom_callback_parser) through parse_error() instead, matching the
existing error-handling pattern used elsewhere in this file. Behavior is
unchanged when allow_exceptions is true (the default); the exception
message and type are identical.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Drop a non-portable exact exception message check in the 408 test
The allow_exceptions=false regression test checked the exact message
text produced when allow_exceptions=true (the default). On platforms
where std::size_t is 32-bit (e.g. mingw x86, MSVC Win32 builds), a
declared CBOR length of 2^63 is intercepted earlier, by
get_cbor_container_size()'s own (pre-existing, already correct)
length-narrowing check, with different wording than this fix's
start_array()/start_object() size check -- same error code, same
"still throws when allow_exceptions=true" guarantee, different text.
CHECK_THROWS_AS already verifies the behavior this test cares about
(still throws json::out_of_range, unchanged); drop the exact-message
assertion since it isn't portable across size_t widths and doesn't
add coverage of this fix specifically.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix -Werror=unused-result on json::from_cbor() in the 408 regression test
from_cbor() is [[nodiscard]]; CHECK_THROWS_AS() otherwise discards its
result, which GCC flags under -Werror. Assign to a throwaway json, as
the rest of the suite already does for from_cbor()/from_msgpack().
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Restore a duplicate key's prior value when the callback rejects its new value
json_sax_dom_callback_parser::key() unconditionally overwrote the object
slot for a key with a `discarded` placeholder as soon as the key was
accepted by the parser callback. For a duplicate key (legal JSON), this
destroyed the pre-existing value from an earlier occurrence of the same
key before the new value was even parsed. If the new value was then
rejected by the callback, remove_discarded_value() erased the member
entirely instead of leaving the original value in place, contradicting
the documented behavior that a discarded value behaves as if it was
never read.
Add a small stash of (slot pointer, previous value) pairs so that when
key() overwrites an existing member with the discarded placeholder, the
previous value can be restored later if the corresponding value (scalar,
object, or array) is rejected, instead of being erased. The stash entry
is dropped without restoring once the new value is definitively
accepted (in handle_value() for scalars, end_object()/end_array() for
containers), so a duplicate key whose new value is accepted still keeps
the last value as before. Non-duplicate keys are unaffected: rejecting
their value still removes the member entirely, since there is nothing
to restore.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Mark parser-callback test lambdas noexcept to fix GCC -Wnoexcept -Werror
GCC's libstdc++ std::function move assignment evaluates a noexcept
check that invokes a wrapped callable in an unevaluated context; a
non-noexcept parser_callback_t lambda then trips -Wnoexcept ("noexcept-
expression evaluates to 'false'"), which CI's ci_test_gcc job builds
with -Werror. The pre-existing parser_callback_t test lambdas in this
file already work around this by declaring themselves noexcept; apply
the same fix to the three added lambdas that didn't.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Make diff() account for member order in ordered_json objects
diff() compared source/target objects purely by key set, ignoring
relative member order. For ordered_json (insertion-ordered, vector-
backed object_t), two objects that differ only in member order are
unequal via operator==, but diff() never emitted any patch operation
to fix the order, so source.patch(diff(source, target)) == target
could fail to hold.
Fix by detecting when common keys appear in a different relative
order in source vs. target (or when a new key would need to land
somewhere other than the end), and in that case removing and
re-adding the affected keys in target's order, which relies on
patch()'s "add" op appending new keys at the end of an ordered_map.
For plain json (std::map-backed, always key-sorted iteration) this
is a no-op and the original minimal per-key diff path is unchanged.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Avoid redundant lookups in diff()'s object-order tracking
The previous fix for ordered_json member order re-derived common-key
order and suffix information with extra target.find()/source.find()
calls layered on top of the pre-existing removed/added-key passes,
instead of reusing those same passes. This roughly tripled the number
of map lookups per diff() call for every object, including plain
`json`, where the reordering path is never taken.
Piggyback the order tracking (and the "add" op construction for new
keys) onto the two passes the algorithm already needs to detect
removed/added keys, and walk the fast path's recursion in lockstep
with the precomputed common-key list instead of re-querying `target`.
This restores diff() to its pre-existing lookup count; benchmarked at
n=1000 keys, ordered_json::diff() was roughly 2x slower than baseline
before this change and is back within noise of baseline after it.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Preserve diff()'s original op ordering and fix a slow-path deletion gap
Splitting removed-key detection and common-key recursion into separate
passes (for the earlier lookup-count fix) changed the emitted patch's
op order: all "remove" ops now came before all recursive per-key diffs,
instead of interleaved in source's iteration order as the original
implementation did. This broke docs/mkdocs/docs/examples/diff.output's
exact-match CI check (ci_test_examples) even though the patch was still
semantically correct.
Defer "remove" emission into the same walk that does the recursive
diffs, so common keys and deleted keys are interleaved in source order
again, matching historical output.
While restructuring that walk, the reordering ("slow path") branch was
only emitting "remove" for keys common to both objects, never for keys
present in source but genuinely absent from target -- a key deleted
alongside an actual reorder would silently survive the patch. Fixed by
removing every source key in the slow path (both deleted and common
keys need removing there; common keys are then re-added in target's
order). Verified with a targeted reorder+deletion case and a fresh
20,000-case round-trip fuzz run (0 failures).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The characterization test added in #5482 asserted that basic_json::swap()
does NOT exchange start_position/end_position, based on a misreading of
the code cited for #5420. In fact swap() (json.hpp, around line 3637)
does swap start_position/end_position along with the value, consistent
with copy-assignment. The test's assumption was backwards, so it failed
on every CI job across every branch/PR since the commit landed. Correct
the assertions to match the actual (and correct) behavior: positions are
exchanged together with values.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* Fall back to plain-object encoding when _ArrayType_ is not a string
write_bjdata_ndarray() looked up _ArrayType_ by calling get<string_t>()
directly, which throws type_error.302 when the annotation is not a
string (e.g. a number, null, boolean, array, or object). Per the
documented BJData ndarray contract, an object only qualifies for the
compact ndarray encoding if _ArrayType_ names a known type; anything
else must fall back to plain-object encoding, the same way an unknown
type-name string already does.
Add an is_string() check before the get<string_t>() call so a
non-string _ArrayType_ takes the existing "unrecognized type name"
fallback path instead of throwing.
Fixes#5398.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Relax the BJData fuzzer's round-trip check from byte-exact to value-exact
Fixing #5398 lets to_bjdata() proceed past the object it used to reject,
which exposed a pre-existing, unrelated round-trip quirk to the fuzzer:
a binary_t value serialized through the non-optimized ("$U#"-less)
array encoding is parsed back as a plain array of numbers, since
from_bjdata() has no way to tell "array of uint8 numbers" apart from
"array of bytes" without that optimized header. Re-serializing that
plain array then goes through the generic smallest-type writer, which
- unrelated to this PR, and long predating it - prefers the 'i' (int8)
marker over 'U' (uint8) for values that fit both, so the re-encoded
bytes can differ from the original even though both decode to the same
value.
This is not introduced by the #5398 fix; the same divergence reproduces
from a bare json::binary_t value with no _ArrayType_ annotation
involved at all, on the commit immediately preceding it. A general fix
would mean changing the shared UBJSON/BJData smallest-type selection
that hundreds of existing tests pin to 'i' for small positive
integers, which is out of scope and too risky for this PR.
Update fuzzer-parse_bjdata.cpp's round-trip assertions to check that
re-serializing is value-stable (from_bjdata(to_bjdata(j)) == j) rather
than byte-exact, matching the guarantee BJData actually provides, and
add a regression test in unit-bjdata.cpp using the exact OSS-Fuzz input
that documents the behavior.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Compare dump()s instead of json values in the BJData fuzzer's round-trip check
The value-stability assertion added to fix the earlier OSS-Fuzz crash
(json::from_bjdata(to_bjdata(j2)) == j2) itself broke on a NaN payload:
IEEE 754 NaN is never equal to itself, so operator== reports two
structurally-identical trees containing a non-finite double as
different -- not a round-trip bug, just NaN's ordinary
non-reflexivity. dump() serializes any non-finite double the same
deterministic way (as JSON null, since JSON cannot represent NaN or
Infinity), so comparing dumps is stable under exactly the values that
break operator==.
Verified against both the original OSS-Fuzz crash input and the new
one (0x68 0x68 0x7c, which decodes to a NaN), plus a local 2.5M-case
random-input sweep with no failures.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix to_bjdata() emitting the Draft-3-only 'B' marker in default Draft-2 mode
_ArrayType_ = "byte" mapped unconditionally to the BJData type marker
'B', regardless of the requested bjdata_version. 'B' is defined only by
BJData Draft 3; with the default version (draft2), this produced a
stream that is invalid for Draft 2 and, unlike every other
_ArrayType_, round-tripped back as a binary value instead of the
original annotated object.
Only accept "byte" / emit 'B' when bjdata_version selects Draft 3.
Under Draft 2, fall back to the same plain-object encoding used
elsewhere in this function for other invalid-annotation cases, so the
value round-trips correctly.
Fixes#5404.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Future-proof the Draft-3-only 'B' marker gate
@gregmarr pointed out that dtype == 'B' && bjdata_version != draft3
only future-proofs by accident, since bjdata_version_t currently has
exactly two values. Compare with < instead, so a later draft that
keeps the 'B' marker valid does not need this gate revisited.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix dead ill-formed-fourth-byte UTF-8 test sections (byte3/byte4 typo)
The "ill-formed: wrong fourth byte" SECTIONs in unit-unicode3.cpp,
unit-unicode4.cpp, and unit-unicode5.cpp guarded their loop with a check
on byte3 instead of byte4. Since the enclosing loop already restricts
byte3 to its valid range, the guard was always true and the section's
"continue" fired unconditionally, so check_utf8string()/check_utf8dump()
were never actually invoked for a malformed fourth byte.
Fixing the guard naively (byte3 -> byte4) would also have swept the full
byte2 x byte3 combinatorics for every byte4 value, adding millions of
redundant iterations: the lexer validates continuation bytes strictly in
sequence with early exit (see next_byte_in_range() in lexer.hpp), so once
byte2/byte3 are within their valid range, the byte4 outcome does not
depend on which valid byte2/byte3 values were chosen. Instead, byte2 and
byte3 are now held to a small hedge of representative valid prefixes
(range corners plus a midpoint) while byte4 is still swept exhaustively
over its full 0x00-0xFF range, since that is the actual property under
test. Also fixed the garbled "skip fourth second byte" comment in
unit-unicode3.cpp.
Verified offline: before the fix, the "wrong fourth byte" subcase
executes 0 assertions in all three files (proving it was dead code);
after the fix, it executes 11520 (unicode3), 34560 (unicode4), and 11520
(unicode5) assertions, and a deliberately reintroduced bug in the
lexer's byte4 range check causes it to fail (proving it is now
meaningful). Total per-file assertion counts grow by the same small
amounts, not by millions, and all other sections in these files still
pass unchanged.
Fixes#5416
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Keep full byte2 x byte3 combinatorics in the wrong-fourth-byte sections
The maintainer wants exhaustive coverage of every byte combination here
rather than the representative-prefix reduction, matching the style of
the sibling "wrong second/third byte" sections in the same files.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Add missing diagnostic-positions test coverage (lifetime, input adapters, SAX)
Building on the merged unit-class_parser.cpp from #5417, add
characterization tests (regression protection for existing behavior, not a
behavior change) for JSON_DIAGNOSTIC_POSITIONS:
- value lifetime: copy ctor copies positions recursively, move ctor resets
the moved-from value to npos, and mutating a parsed document (operator[],
push_back, erase) leaves the parent's stale span and siblings' positions
untouched while new values get npos.
- input adapters: wide-string input positions count transcoded UTF-8 bytes
(not wide characters), BOM-prefixed input's start_pos() reflects the
skipped 3-byte BOM, istringstream/ifstream/iterator-pair inputs report
consistent (non-npos) positions, and binary formats (CBOR, MessagePack,
UBJSON, BSON) always report npos.
- a user-constructed json_sax_dom_parser with no lexer (as used when driving
json::sax_parse() directly) reports npos for every value, since it has no
m_lexer_ref to source positions from.
While characterizing swap(), found that basic_json::swap() (and the friend
swap() that forwards to it) does not swap start_position/end_position,
unlike copy-assignment's operator=(basic_json), which does as part of its
copy-and-swap implementation. This looks like a real inconsistency/bug, but
per the scope of this test-only change it is only pinned (not fixed) here;
see the comment at the "swap() does NOT exchange positions" section.
Fixes#5420
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix MSVC source-encoding portability in the wide-string position test
Use é escapes instead of a literal UTF-8-encoded 'é' inside the L""
literal, so the wide string's content does not depend on the compiler's
assumed source character set (MSVC without /utf-8 decodes raw non-ASCII
source bytes using the system code page rather than as UTF-8, which was
producing a wstring of unexpected length/content and failing the
ws.size()/end_pos() assertions on Windows CI).
Also reworded a comment that unintentionally embedded the literal
substring "TODO check", which clang-tidy's google-readability-todo check
flags regardless of quoting context.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* test: cover JSON_NO_IO, JSON_THROW/TRY/CATCH_USER, JSON_SKIP_LIBRARY_VERSION_CHECK, and JSON_DisableEnumSerialization in CI (#5423)
These four supported configuration macros were never actually compiled
anywhere in the test matrix:
- JSON_NO_IO and the JSON_THROW_USER/JSON_TRY_USER/JSON_CATCH_USER trio
are exercised together in a new tests/src/unit-no_io_and_user_exceptions.cpp,
which is automatically picked up by the existing unit-*.cpp test glob and
thus built across the whole standard test matrix.
- JSON_SKIP_LIBRARY_VERSION_CHECK is exercised by a new, dedicated
tests/src/skip_library_version_check.cpp, compiled directly by the new
ci_test_skiplibraryversioncheck target in cmake/ci.cmake: the scenario it
simulates (mixing two differently-versioned inclusions of the library)
unavoidably triggers the compiler's own "macro redefined" warning, which
would fail under the library's own -Weverything/-Werror unit test matrix
for a reason unrelated to the macro under test.
- JSON_DisableEnumSerialization already had #if-guarded tests in several
unit-*.cpp files (from #4384), but no CMake target ever actually set the
JSON_DisableEnumSerialization CMake option, so that guarded code was never
compiled. Add ci_test_disableenumserialization, mirroring the existing
ci_test_noimplicitconversions/ci_test_noglobaludls targets. Building the
full test suite with this option on surfaced one real, narrow gap: get<T>()
on std::vector<std::byte> (used by unit-regression2.cpp's custom BinaryType
tests) relies on std::byte being handled via enum serialization, so add the
same #if-guard convention to the two affected SECTIONs there.
Both new CI targets are added to the ci_cmake_options matrix in
.github/workflows/ubuntu.yml, alongside the existing ci_test_* targets.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* test: cover multi-digit widths and bare alignment in std::formatter<json> (#5423)
Every existing std::formatter spec with a width used a single digit (e.g.
"{:2}"), so the width-parsing loop's accumulation of a second/third digit was
never exercised; add multi-digit width cases. Likewise, every existing spec
with an alignment character also had an explicit fill character, so the
bare-alignment branch (e.g. "{:<}", with no fill) was never exercised; add
cases asserting it keeps the default space indent character.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* test: add coverage for patch_inplace() (#5423)
patch_inplace() had no unit test at all. Add a happy-path case mirroring an
existing patch() example, and -- more importantly -- pin its distinguishing
contract versus patch(): when a multi-operation JSON Patch fails partway
through, patch_inplace() (which mutates the document directly, operation by
operation) leaves whatever operations already succeeded applied, whereas
patch() (which applies the patch to an internal copy that is discarded on
exception) leaves the original completely untouched either way. Verified
empirically against the current implementation before writing the assertions.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* test: fix duplicate TEST_CASE name in unit-no-mem-leak-on-adl-serialize.cpp (#5423)
Two distinct TEST_CASEs were both named "check_for_mem_leak_on_adl_to_json-2".
doctest allows duplicate names, so both still ran, but it makes
--test-case=<name> filtering and reporting ambiguous. Rename the second one
to "-3", continuing the existing "-1"/"-2" sequence.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* test: add direct coverage for the std::u8string to_json overload (#5423)
The ADL to_json overload for std::basic_string<char8_t, ...> was only ever
reached indirectly, via std::filesystem::path::u8string(). Add a test that
constructs a json value directly from a std::u8string, gated the same way as
the overload itself (include/nlohmann/detail/conversions/to_json.hpp): behind
both the std::filesystem::path feature guard and __cpp_lib_char8_t, since the
overload only exists when both are satisfied.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* test: verify move semantics of byte_container_with_subtype's rvalue constructors (#5423)
The two rvalue-reference constructors were never distinguished from their
const-lvalue-reference twins by any test. Add a "move semantics" section that
constructs from an rvalue std::vector, checks the resulting container keeps
the exact same buffer address as the source (a stronger check than just
observing the source ended up empty, since a copy-then-clear could do that
too), and confirms the source vector was left empty.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Guard patch_inplace() partial-application test against JSON_NOEXCEPTION
The "distinguishing contract vs patch(): partial application on
failure" test relies on doc.patch_inplace(patch) actually throwing so
the partially-applied state can be observed right after the throw
point. Under ci_test_noexceptions, JSON_THROW() calls std::abort()
instead of throwing, and doctest's --no-throw test filter (which that
CI job passes) makes CHECK_THROWS_AS() a no-op that never even
evaluates its expression -- so patch_inplace() is never called and the
follow-up assertions fail against the untouched original document.
Guard the whole SECTION with #if !defined(JSON_NOEXCEPTION), following
the same convention already used elsewhere in the test suite (e.g.
unit-class_parser.cpp) for exception-dependent tests.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix MSVC C2220 in the std::u8string conversion test
MSVC's C5321 ("nonstandard extension used: encoding '\xNN' as a
multi-byte utf-8 character") is promoted to a hard error by our MSVC CI
configs. It fires because the test composed a non-ASCII UTF-8 sequence
inside a u8"" literal using raw \x byte escapes; MSVC treats that as
nonstandard and suggests using \u universal-character-names instead,
which every compiler agrees on and which compiles down to the exact
same encoded bytes.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Guard the JSON_THROW_USER test against JSON_NOEXCEPTION and GCC's -Wunused-result
Two independent CI configurations failed to build/run this new test:
- ci_test_noexceptions runs the whole suite with -DJSON_NOEXCEPTION and
doctest's "--no-throw" filter, which compiles CHECK_THROWS_AS() down
to a no-op that never even invokes the guarded expression. Since this
test's whole point is to observe json_throw_user_call_count after
json::parse()/at() actually throw, it can't be meaningfully run under
that filter (our JSON_THROW_USER override still throws real
exceptions regardless of JSON_NOEXCEPTION, but the assertion never
gets a chance to run). Guard the TEST_CASE with
#if !defined(JSON_NOEXCEPTION), mirroring the existing precedent in
unit-json_patch.cpp.
- ci_test_gcc and ci_test_standards_gcc(11) failed with
-Werror=unused-result on the discarded json::parse() return value.
json::parse() is marked warn_unused_result, and unlike a real
[[nodiscard]] attribute, GCC does not consider that satisfied by
doctest's (void)-cast around the expression in C++11 mode. Assign the
result to a discarded local instead, matching the established
`json _ = json::parse(...)` idiom already used throughout
unit-class_parser.cpp.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Suppress a clang-tidy false positive on an intentional defensive copy
performance-unnecessary-copy-initialization suggests copy_for_patch
could be a reference since it's never modified -- but the copy is the
point: it guards against a hypothetical regression where patch()
mutates its receiver, which a reference could never catch (the
follow-up assertion would just compare `original` to itself).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix clang-tidy findings in the JSON_NO_IO/JSON_THROW_USER test
- bugprone-macro-parentheses: wrap the JSON_THROW_USER macro argument in
parentheses at the throw site.
- modernize-raw-string-literal: switch two escaped JSON string literals to
raw string literals.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Add test coverage for ordered_json/alt_json across binary formats and patch/diff/flatten APIs
Closes a test-coverage gap from #5421: ordered_json (and the alt_string-based
basic_json specialization from unit-alt-string.cpp) were never round-tripped
through the binary formats (CBOR/MessagePack/UBJSON/BSON/BJData), nor through
flatten()/unflatten(), diff()/patch()/patch_inplace(), or merge_patch(). Also
adds a std::formatter<ordered_json> spot-check, mirroring the precedent set
by the format_as() ADL-deduction test.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Pass alt_string's std::string constructor argument by value (clang-tidy modernize-pass-by-value)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
#5285's indentation regression test didn't exercise json::binary,
which serializes as an object but always writes its byte array
compactly (dump_byte()). #5186 had covered this case before it was
closed as superseded; port just that coverage here.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: document the implicit requirements on basic_json's template parameters
The requirements that basic_json places on its eleven template parameters
were only implied by how the library uses the resulting object_t, array_t,
string_t, etc. Consumers had to discover them by trial and error.
Add "Template Parameter Requirements" collecting them, split into what is
always required and what is only required when a particular part of the API
is instantiated. Notable findings that were previously undocumented:
- ObjectType must provide a key_compare member type (actual_object_comparator
names object_t::key_compare in both arms of a std::conditional), and its
third template parameter is used as a comparator, so std::unordered_map
cannot be used without a wrapper.
- ArrayType must provide capacity() -- push_back(), emplace_back(),
operator+=(), and operator[](size_type) call it unconditionally -- and
needs random-access iterators, so std::deque and std::list do not work.
- StringType needs contiguous, null-terminated data(), a one-byte value_type,
and either assignability from std::to_string or an ADL int_to_string().
- NumberFloatType must be float, double, or long double for parsing and
serialization; the integer types must satisfy std::is_integral.
- AllocatorType must be stateless, support incomplete types, and use plain
pointers.
- BooleanType and the number types are union members and must be trivial.
Link the new page from the basic_json overview, the types feature page, and
the individual type alias pages, and correct the container examples given for
ObjectType (std::unordered_map) and ArrayType (std::list), which do not work.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix object_comparator_t for object types without key_compare
detail::actual_object_comparator selected between object_t::key_compare and
default_object_comparator_t with std::conditional. Both type arguments of
std::conditional are named eagerly, so object_t::key_compare had to exist
regardless of the condition, and the has_key_compare guard added in 3.11.0
never took effect: any ObjectType without a key_compare member type failed to
compile while instantiating basic_json itself.
Use detected_or_t instead, which resolves through a SFINAE partial
specialization and only names object_t::key_compare when it exists. The
selected type is unchanged for every object type that compiled before, so
object_comparator_t -- a public member type -- keeps its meaning and ABI.
has_key_compare had no other users and is removed.
Add a regression test using an adapter around std::unordered_map, which has no
key_compare; it fails to compile without this change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: list the types that are known to work for each template parameter
Follow up on the template parameter requirements page: state, for every
template parameter, which concrete types work and where they stop working.
Each entry was verified by compiling and running a common workload (DOM
access, dump, parse, CBOR/MessagePack round-trip, flatten, hash) against that
instantiation.
Findings worth calling out:
- ObjectType no longer needs a key_compare member type, so the std::unordered_map
adapter only has to restore the template argument order. A hash-ordered
ObjectType works everywhere except unflatten(), which reconstructs an array
only when it meets the reference token 0 before the other indices.
- ArrayType: std::deque works when wrapped to add capacity(); std::list does not.
- StringType: std::pmr::string and std::basic_string with a custom allocator
compile for the DOM, dump, and parse, but not for the binary readers, flatten,
or diff, because the library assigns std::string values to string_t and
int_to_string cannot be overloaded for a type in namespace std.
- NumberFloatType: long double works for dump and parse but not for the binary
formats, which have no encoding for it.
- BinaryType: std::vector<std::byte> supports assignment, get, and the binary
formats, but neither dump nor std::hash<basic_json>.
Also record the object_comparator_t fix in its version history.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix unflatten and binary dumping for non-default configurations
unflatten() decided between array and object by looking at the first reference
token it happened to see for a node: it started an array only when that token
was 0. With a sorted object type the token 0 always arrives first, so the
result was correct by accident; with an object type whose iteration order is
unspecified, {"/c/2":3,"/c/1":2,"/c/0":1} unflattened to an object with the
keys "0", "1", and "2" instead of an array.
Collect the pointer prefixes that have a reference token 0 among their children
before building the result, and let get_and_create() consult that set. The
outcome is now independent of the iteration order and matches, for every input,
what a sorted object type produced before: a value is restored as an array if
and only if one of its keys is 0. Iterating the flattened object in a different
order would have been simpler, but it would have changed the key order of the
result for insertion-ordered object types.
The serializer, std::hash, and the UBJSON writer converted the elements of a
binary value to an integer implicitly, which does not compile for a BinaryType
whose value type is std::byte, and which made dump() write the bytes of a
signed value type as negative numbers. Convert to std::uint8_t explicitly in
all three places, so every byte type dumps as 0..255. The default
std::vector<std::uint8_t> configuration is unaffected.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: note which Abseil containers can be used as template arguments
Checked against Abseil release 20250127.0 with the same workload as the other
entries on the page (DOM access, dump, parse, CBOR/MessagePack/UBJSON
round-trip, flatten, hash), with and without JSON_DIAGNOSTICS.
absl::flat_hash_map and absl::node_hash_map work as ObjectType through an
adapter that restores the template argument order and makes erase(iterator)
return the following iterator, which Abseil's returns as void. The page now
carries that adapter, and notes that absl::flat_hash_map does not keep
references to the mapped values valid across insertions while
absl::node_hash_map does. Both have a capacity() member, so JSON_DIAGNOSTICS
already refreshes the parent pointers conservatively for them.
absl::btree_map and absl::InlinedVector cannot be used at all: object_t and
array_t are formed while basic_json is still incomplete, and both inspect
their value type at class scope. std::map and std::vector are required by the
standard to tolerate this, third-party containers generally are not, so the
page states the constraint on its own rather than only per container.
absl::InlinedVector does work as BinaryType, where it is instantiated with a
complete type. absl::FixedArray and absl::Cord are not usable.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Relax the ArrayType and ObjectType requirements
Two requirements forced users of otherwise suitable containers to write a
wrapper, and neither was load-bearing.
array_t::capacity() was read in push_back(), emplace_back(), operator+=(), and
operator[](size_type), but set_parent() only looks at the value under
JSON_DIAGNOSTICS; without diagnostics it was computed and discarded. Read it
through array_capacity(), which reports unknown_size() when diagnostics are off
or when the array type has no capacity() at all, and treat an unknown capacity
as "the elements may have moved" so the parent pointers are refreshed
conservatively. std::deque now works as ArrayType, in both builds, and
capacity() is no longer named at all in a default build. Since the capacity is
now only meaningful for array insertions, it moves out of set_parent() into
set_parent_after_array_insert().
basic_json::erase(iterator) assigned the object's erase() return value, which
requires the container to return the following iterator. Abseil's hash maps
return void to avoid computing a successor the caller may not need. Detect that
and compute the successor before erasing; containers that return an iterator,
including the vector-backed ordered_map where a precomputed successor would be
wrong, keep the existing path.
Together these leave an Abseil hash map needing only an alias that restores the
template argument order, and no adapter at all for std::deque.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Do not require string_t to be convertible from std::string
Three places built a std::string and handed it to something expecting a
string_t: the UBJSON high-precision number reader, which every binary reader
instantiates, and the BSON writer's array element size calculation and write.
That silently required string_t to be implicitly convertible from std::string,
which std::string itself and types with a string_view conversion satisfy, but
many string types do not.
Construct the string_t explicitly from the data and size, which the
requirements already cover. This makes boost::container::string, eastl::string,
std::pmr::string, and std::basic_string with a custom allocator work as
StringType, none of which could previously be used with any binary format.
Add binary format coverage to the alt_string test, which had none, including a
UBJSON high-precision number -- the case that goes through the reader path.
BSON stays uncovered there: it additionally needs string_t::find(value_type),
which alt_string does not provide.
Also record which containers from Boost, Abseil, and EASTL work for each
template parameter, and correct two claims: std::pmr::string is usable after
this change, and tsl::ordered_map is not usable at all, because its iterators
expose the mapped value as const while basic_json modifies it in place.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: record compatibility for the common header-only hash maps
ankerl::unordered_dense (map and segmented_map), phmap (flat_hash_map and
node_hash_map), and robin_hood::unordered_flat_map all work as ObjectType
through the same adapter as Abseil's and Boost's hash maps, which only has to
restore the template argument order.
phmap::btree_map and robin_hood::unordered_node_map do not: like the other
btree containers they require a complete value type.
Note that none of these hash maps defines key_compare, so every one of them
depends on object_comparator_t falling back to default_object_comparator_t.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: record Folly and the remaining vector replacements
Folly works, with the caveat that its headers need C++20: folly::fbstring as
StringType, folly::fbvector and folly::small_vector as ArrayType,
folly::fbvector<std::uint8_t> as BinaryType, and folly::F14NodeMap as
ObjectType through the usual argument-order adapter. folly::F14FastMap is the
exception and requires a complete value type.
For ArrayType, boost::container::devector, boost::container::static_vector
(within its fixed capacity), and std::pmr::vector work as well.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: cover fifo_map, gtl, folly::sorted_vector_map, and Qt
nlohmann::fifo_map works through the adapter that has always been documented
for it, and preserves the insertion order. Restore its mention in the object
order page, which was dropped together with the tsl::ordered_map one: unlike
ordered_map it keeps a lookup index, so it is the insertion-ordered option
without the quadratic cost.
gtl::flat_hash_map and folly::sorted_vector_map work as well, the latter
through an alias that drops the allocator, whose value type it disagrees on.
gtl::btree_map does not, for the same reason as the other btree containers.
None of the Qt containers can be used, each for its own reason: QMap has no
value_type, QHash iterators yield the mapped value rather than a pair, QList
has no max_size(), QByteArray spells empty() as isEmpty(), and QString is
UTF-16.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: qualify the std::pmr::string support claim
Listing std::pmr::string as fully supported was an overclaim: it was only ever
checked with the default memory resource, which is not what PMR is for.
basic_json cannot be given an allocator or a memory resource, so a pmr string
inside a value always allocates from std::pmr::get_default_resource(), and
assigning an arena-backed string into a value silently drops its resource,
because polymorphic_allocator does not propagate on copy construction. Passing
polymorphic_allocator as AllocatorType does not compile either. Only the
process-global set_default_resource() redirects these allocations.
Say so, and separate the row from std::basic_string with a custom stateless
allocator, which is unaffected.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: remove a duplicated StringType compatibility section
The StringType section carried two 'Compatible types' tables and two copies of
the reference-implementation tip. The second table was a stale copy from before
the binary format string fixes and still listed std::pmr::string and
std::basic_string with a custom allocator as unusable, contradicting the
corrected table a few lines above it, and it dragged along the old explanation
that blamed int_to_string.
Drop the stale copy and put the surviving table before the notes, so the
'see below' in the std::pmr::string row points forwards.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: correct the template parameter requirements after independent verification
Every claim on the page was re-checked by compiling and running it, including
the rows that say a type cannot be used, which were checked to fail for the
documented reason and not merely to fail. Twenty-four claims were wrong.
The most consequential: the incomplete-type constraint applies to ObjectType
only. object_t is instantiated inside the class definition, because it is
probed for key_compare; array_t is only named there and is not instantiated
until basic_json is complete. So eastl::vector, QList and QVector are not
excluded by incomplete types at all -- they simply have no max_size() -- and
absl::InlinedVector is excluded for a subtler reason of its own.
Further corrections: ObjectType does not need erase(key), which has a fallback,
but does need at(key) for UBJSON output; only == and < are used, or == and <=>
under C++20, not all six; the documented adapter does not fit ankerl or
robin_hood. ArrayType needs no initializer-list insert, and value_type, the
(count, value) constructor and swappability are per-function, not always.
BinaryType needs a range insert for CBOR indefinite-length byte strings and
does not need push_back. StringType needs append(const StringType&)
unconditionally, and does not need operator!= or operator== against const
char*; empty(), resize(n) and reserve(n) are per-subsystem; int_to_string is
needed by diff, items and std::hash rather than by JSON Pointer or flatten.
BooleanType must be implicitly convertible from bool, and JSONSerializer's
second parameter need not carry a default.
std::pmr::string was wrong in the other direction this time: a moved-in string
does keep its memory resource, and later growth allocates from it. Only copies
land on the default resource.
Five requirement violations are not caught at compile time rather than the two
the page claimed; they are now listed together up front. Split every
compatibility table into what works and what does not, as the reasons in the
second half are the useful part.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Reduce the string_t and array_t members the library requires
Several members were required only because of how the library happened to
be written, not because the functionality needs them. Dropping them widens
the set of usable string and array types, and one of them was also a
performance problem.
string_t:
- c_str() is gone. Every call site already knew the length and passed it
along, so data() is enough. The one place that did not, the diagnostics
path in exceptions.hpp, now builds the token from data() and size(),
which also stops it from truncating keys that contain a null byte.
- back() is gone; the serializer indexes the last character instead.
- find(str, pos), replace(), and substr() are gone. escape() and
unescape() rebuilt the string with one replace() per escaped character,
which moves the tail every time: escaping a string of n characters that
all need escaping cost O(n^2). Both now scan with find_first_of() -- a
member the pointer parser already required -- and append whole runs, so
the common case is one search and one copy. Escaping 64000 tildes drops
from 717 ms to 20 ms; a string with nothing to escape gets faster too
(8.4 ms to 5.8 ms), because the scan is still a single memchr per pass.
json_pointer::split() takes its reference tokens with the
(const char*, size_type) constructor rather than substr().
- json_pointer::to_string() accumulates with concat<string_t> instead of
letting concat default to std::string and converting afterwards, so
streaming a json_pointer no longer requires string_t to be assignable
from a std::string.
array_t:
- at(size_type) is gone. basic_json::at(size_type) checked the index by
calling array_t::at() and translating std::out_of_range, which also
required the array type to throw that exact exception. It now compares
against size() and uses operator[]. The thrown exception, its message,
and the behaviour under JSON_NOEXCEPTION are unchanged.
The BSON writer wrote the terminating null byte out of the string's own
buffer (size() + 1). It now writes the byte itself, so string_t::data()
need not be null-terminated for to_bson().
The tests pin the reduced API: alt_string loses the five dropped members
and gains coverage of the escaping paths, and a std::vector whose at() is
hidden is used as an ArrayType.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: record the reduced string_t and array_t requirements
Drop c_str(), back(), find(str, pos), replace(), and substr() from the
StringType requirements and at(size_type) from the ArrayType ones, and
note the string assignment the JSON pointer code performs. Streaming a
json_pointer no longer needs assignability from a std::string.
Add the non-null-terminated data() to the list of violations that are not
diagnosed at compile time -- it was described in the StringType section
but missing from the summary at the top -- and correct the QString row,
which no longer fails for the c_str() it lacks.
JSON_CATCH_USER no longer wraps a catch of std::out_of_range: the last one
went away with array_t::at(). Describe what the library actually catches.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Use character literals for the signed BinaryType test
MSVC rejects char(0xFF) with C4310 (cast truncates constant value),
which the Windows workflow treats as an error. The character literals
carry the same byte values without a narrowing cast.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Do not instantiate a hash map with an incomplete basic_json in the tests
object_t is probed for key_compare inside the definition of basic_json, so
it is instantiated while basic_json is still incomplete. Whether a hash map
survives that depends on the standard library: libstdc++ 9 needs the size of
the mapped type to instantiate std::unordered_map's node type and rejects
the adapter, which broke the GCC 9 builds.
The test now derives its no-key_compare object type from std::map -- which
does cope -- and shadows the inherited key_compare member type with an
entity that is not a type, so the library's probe finds none, exactly as for
a hash map. The unflatten() order-independence checks in unit-json_pointer
already cover the behaviour that the unordered object type was there for.
The limitation is documented for std::unordered_map.
Also address two Clang-Tidy findings the earlier commits introduced:
erase_from_object() declares its iterator with auto, and at(size_type) checks
the type first and then falls through to the return instead of throwing from
an else branch.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Keep diagnostic key paths null-terminated
Building the token from data() and size() kept an embedded null byte in the
key, and since what() hands out a C string, that truncated the whole message
rather than just the key: to_bson() on a key containing U+0000 reported
"[json.exception.out_of_range.409] (/en" instead of the full explanation.
This broke test-bson under JSON_DIAGNOSTICS.
Constructing from data() alone stops at the first null byte, which is what
c_str() did before, so the message is unchanged -- without requiring
string_t to provide c_str().
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Do not parse the value in the array-at() test
JSON_DIAGNOSTIC_POSITIONS adds the byte range of the value to the exception
message, which a parsed value has and an in-memory one does not, so the two
message checks failed in that configuration. Build the array in memory
instead of parsing it; the test is about at(size_type) not needing
array_t::at(), and the byte range is beside the point.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Move the custom BinaryType tests into their own translation unit
The two sections added to unit-regression2.cpp brought a third full
basic_json instantiation into a translation unit that was already large.
With Clang on MinGW that pushed the object over the reach of a 32-bit
relocation and test-regression2_cpp20.exe failed to link:
relocation truncated to fit: IMAGE_REL_AMD64_REL32 against `.rdata'
unit-regression2.cpp is restored to exactly what it was before, and the
coverage moves to unit-custom-binary-type.cpp, next to the object and array
type tests it belongs with. The signed value type is now also covered in
C++11, where std::byte is not available.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Do not require the container iterators to be nothrow move constructible
iter_impl declared its defaulted move operations noexcept. The exception
specification a defaulted function gets implicitly follows from its members,
here internal_iterator, which holds the object and array iterators. libstdc++
gives std::deque's iterator a user-provided copy constructor without noexcept
before version 11, so the implicit specification is noexcept(false) and does
not match the declared one. That deletes the function -- and with g++ 4.8,
which predates CWG 1778, it is an error outright:
error: function 'iter_impl<basic_json<std::map, std::deque> >::iter_impl(
iter_impl&&)' defaulted on its first declaration with an
exception-specification that differs from the implicit declaration
So std::deque, which this branch documents as a usable array type, could not
be used with an older standard library. Leaving the specification to be
computed cannot mismatch; iteration_proxy_value already spells out the same
condition next door.
The default configuration is unaffected: json::iterator, json::const_iterator
and ordered_json::iterator stay nothrow move constructible and move
assignable, which the test now checks so it cannot regress unnoticed.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Address two Clang-Tidy findings the custom container tests exposed
Both come from instantiating basic_json with containers other than the
default ones, and neither shows up with the Clang-Tidy version available
outside CI:
- insert(const_iterator, basic_json&&) forwards its by-value iterator to
the const-reference overload. performance-unnecessary-value-param asks
for the copy to be a move; it only fires for an iterator that is not
trivially copyable, as std::deque's is not. The NOLINT on the function
does not cover it, because the finding is reported where the parameter
is used rather than where it is declared. Move it, which is what the
check asks for and is a (very small) improvement in its own right.
- cppcoreguidelines-use-enum-class rejects the unnamed enum that shadowed
the inherited key_compare member type. An enum class would not do, since
it declares a type of that name and the probe would find it again; a
member function declaration hides the name just as well.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Assert the iterators' exception specification relative to the container
The test pinned that nlohmann::json's iterators stay nothrow movable after
iter_impl's defaulted move operations lost their declared noexcept. That is
not a property of the library, though: the exception specification is now
computed from the container iterators, so it holds only for standard library
implementations whose iterators are themselves nothrow movable.
MSVC's checked iterators before VS2017 are not -- _Iterator_base12 registers
the iterator with the container's debug proxy in a copy constructor that
carries no noexcept -- so the assertions fail on a Visual Studio 2015 debug
build, which is the one debug configuration in the AppVeyor matrix and has no
counterpart in the GitHub Actions matrix.
Assert what the change actually guarantees instead: the iterators are nothrow
movable exactly when the object and array iterators they are built from are.
That still pins the default configuration against a silent regression, and it
is true whatever the standard library provides.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Detect a void-returning erase() through a named trait
erase_from_object() distinguished its two overloads with a decltype of a
member call written inline in a default template argument. Every other
detection in the library goes through the detector machinery in detected.hpp
instead -- has_erase_with_key_type is the same question about the same member
function -- and the inline form is the one shape older compilers are least
reliable about.
Express it the same way: detect_erase_with_iterator plus is_detected_exact,
both of which the library already relies on elsewhere. No behaviour changes.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Give the custom container types only the constructors the library uses
The three container types in the new tests inherited every constructor of
their base with using Base::Base. That asks for more than the test needs: the
library builds an object or an array by default construction, by copy or move,
and -- when converting between two basic_json types or from an initializer
list -- from an iterator range. Declaring those directly makes the requirement
visible in the test, and keeps object types out of a corner where a compiler
has to declare std::map's whole constructor set for a derived class while
basic_json is still incomplete.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Temporarily disable the new custom container tests
AppVeyor is the only CI that builds MSVC 2015 and 2017, and it has now
rejected three heads of this branch. Its build log is not reachable from
where this is being worked on, so the verdict is a single bit and the cause
has to be narrowed down by bisection.
Everything else stays: the library changes, the reduced alt_string, and the
unflatten() tests. If AppVeyor passes with these three translation units
disabled, the cause is one of the six basic_json instantiations they add; if
it fails, it is in the library. Either way this commit is reverted.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Guard the disabled tests with a macro rather than #if 0
Clang-Tidy's readability-avoid-unconditional-preprocessor-if rejects a literal
#if 0. Use a macro that is never defined instead, which the check does not
look at. Still temporary, and reverted together with the previous commit.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Re-enable the array and binary container tests
AppVeyor passed with all three new translation units disabled, so the library
changes, the reduced alt_string, and the unflatten() tests are fine on MSVC
2015 and 2017; the cause is one of the six basic_json instantiations the new
tests add.
Bring back two of the three. If AppVeyor passes again, the cause is in
unit-custom-object-type.cpp, which is the one still disabled; if it fails, it
is in one of these two and needs one more split. Still temporary.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Diagnose two silently violated template parameter requirements
Both were on the list of requirements that are not caught at compile time and
corrupt values rather than failing, and both are a plain size comparison:
- A BinaryType whose value_type is wider than one byte, which the readers and
writers reinterpret as raw bytes anyway.
- A NumberUnsignedType too narrow to hold the absolute value of every
NumberIntegerType value, which makes basic_json(INT64_MIN).dump() yield -0
for std::int64_t with std::uint32_t.
Neither static_assert rejects a configuration that worked before: both only
fire where the result was already wrong. Also add the two comments the review
asked for, in write_bson_string() and calc_bson_array_size(), matching the
ones their counterparts already carry.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Align the template parameter tables and record what is now diagnosed
Every table in the page is reformatted so each column is exactly as wide as
its widest cell, which is what the review asked for in a dozen places: the
separator rows that ran two dashes long, the stray spaces, and the columns
padded well past their content.
The row listing six containers that require a complete mapped type is split
in two so that one cell no longer sets the width of the whole table.
Content changes: NumberUnsignedType is described as any unsigned integer type
at least as wide as NumberIntegerType rather than any unsigned integer type;
the two requirements that are now static_asserts move out of the list of
violations that are not caught at compile time; and the two places that
require a non-const operator[] say why data() will not do (std::string has no
non-const data() before C++17).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Bisect the other way: only the object container tests
The previous head touched only docs/, which AppVeyor's only_commits filter
skips, so it produced no build and no status at all -- the pull request looked
green without ever having been built on MSVC 2015 or 2017.
Swap the guards instead of repeating that step: unit-custom-object-type.cpp is
enabled and the array and binary translation units are disabled. AppVeyor
already passed with all three disabled, so a failure here pins the cause on
no_key_compare_json or void_erase_json, and a pass pins it on the array or
binary file. Still temporary.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Split the two object types apart
AppVeyor failed with only unit-custom-object-type.cpp enabled and passed with
all three new translation units disabled, so the cause is one of the two
object types in this file and not the array or binary ones.
Guard out void_erase_map and leave no_key_compare_map, which separates the two
constructs under suspicion: shadowing the inherited key_compare member type
with an entity that is not a type, and hiding the inherited erase with a
void-returning overload. A failure here points at the first, a pass at the
second. Still temporary.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Build the no-key_compare object type by composition, not inheritance
The "object type without key_compare" test failed on AppVeyor's MSVC
2017 jobs (/std:c++17): its no_key_compare_map derived publicly from
std::map and shadowed the inherited key_compare type with a same-named
member function, relying on ordinary member hiding to make key_compare
unreachable as a type for the library's detection trait. MSVC 2017
does not honor that hiding for a typename-qualified lookup performed
from outside the class and still resolves key_compare to the base's
comparator type, so object_comparator_t incorrectly picked it up
instead of falling back to default_object_comparator_t.
Wrapping a std::map by composition instead removes the base class
entirely, so there is no key_compare to find under any lookup rule,
on any compiler. Also drops the now-unneeded JSON_BISECT_CUSTOM_CONTAINER_TESTS
guard left over from narrowing this down: the void_erase_map test in
the same file was never the cause and is re-enabled unconditionally.
Verified locally with clang++ and g++ under C++17 and C++20.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Re-enable the array and binary custom-container tests
unit-custom-array-type.cpp and unit-custom-binary-type.cpp were still
guarded behind JSON_BISECT_CUSTOM_CONTAINER_TESTS from bisecting the
AppVeyor failure fixed in 7c39f3227, which was unrelated to either
file. The macro was never defined, so none of these tests actually ran
in CI. Verified locally with clang++ and g++ under C++17 and C++20
before removing the guards.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix indentation of custom_object_type per astyle
The one-line function bodies in the composition-based no_key_compare_map
(7c39f3227) do not match the project's Allman brace style, which the
ci_test_amalgamation job enforces with astyle. Reformatted with the
pinned astyle 3.4.13; no functional change.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: add compiled reference implementations for the container template parameters
Each of ObjectType, ArrayType, StringType, and BinaryType now links to a
minimal, self-contained header (docs/mkdocs/docs/examples/custom_*_type.hpp)
that wraps the corresponding standard container by composition and satisfies
every "Always required" member listed on that page. Unlike the prose
requirement lists, these are real code: each header has a companion .cpp that
instantiates a basic_json specialization with it and is compiled and run by
the existing ci_test_examples check (docs/Makefile's check_output_portable),
so the reference implementations cannot silently drift from what the library
actually requires. The .output files were generated with that same target.
StringType's existing pointer to tests/src/unit-alt-string.cpp's alt_string
is kept alongside the new header as a more thorough, battle-tested example.
Verified locally: astyle (pinned 3.4.13, project .astylerc) on the new files;
clang++/g++ under C++11/17/20 for each example against the amalgamated
header; `make check_output_portable` in docs/; `mkdocs build --strict` and
scripts/check_structure.py for the page itself.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Declare no_key_compare_map's accessors noexcept
The GCC C++20 job builds with -Wnoexcept and -Werror, and the standard
library takes noexcept(c.begin()) and noexcept(c.end()) in ranges_base.h and
range_access.h. Forwarding to std::map without repeating its noexcept made
those expressions false, which the warning reports as an error:
error: noexcept-expression evaluates to 'false' because of a call to
no_key_compare_map<...>::begin() [-Werror=noexcept]
note: but ... does not throw; perhaps it should be declared 'noexcept'
Give the accessors the exception specification of what they forward to.
std::map declares begin, end, cbegin, cend, empty, size, max_size, and clear
noexcept, so the wrapper does too. swap is left alone: std::map's is only
conditionally noexcept, and nothing asks for it.
void_erase_map is unaffected because it still derives from std::map and
inherits accessors that already carry the specification.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Write out what the defaulted constructor of no_key_compare_map implied
ci_test_gcc builds with -Weffc++, which asks for data to be initialized in a
member initialization list; a defaulted default constructor does not do that:
error: 'no_key_compare_map<...>::data' should be initialized in the member
initialization list [-Werror=effc++]
Writing the constructor out satisfies that but drops the exception
specification the defaulted one carried, which -Wnoexcept then objects to
where the standard library takes noexcept(construct(...)). Declare it the way
the defaulted constructor was: noexcept when the wrapped map's default
constructor is.
This is the cost of composition -- inheritance carried std::map's exception
specifications and initialization for free, and forwarding by hand has to
restate them.
Checked with the repository's own GCC warning set from cmake/gcc_flags.cmake,
all 346 flags, at C++11, C++17 and C++20: no diagnostics for this file, nor
for the two custom container translation units that were disabled while the
MSVC failure was narrowed down and are built again now.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Mark no_key_compare_map::swap noexcept
Clang-Tidy rejects a swap that is not:
error: swap functions should be marked noexcept
[cppcoreguidelines-noexcept-swap,performance-noexcept-swap]
It was left unmarked on the grounds that std::map::swap is only
conditionally noexcept, so an unconditional promise would be wrong for a
comparator or allocator that can throw while swapping. Both concerns are met
by taking the specification from the wrapped map rather than asserting one:
noexcept(noexcept(data.swap(other.data))). Clang-Tidy accepts that, and no
NOLINT is needed.
Last in the series of specifications that inheritance used to supply and
composition has to write out by hand.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Both overloads swapped the underlying container storage but never called
set_parents(), leaving elements moved into *this with stale m_parent
pointers (typically nullptr from the free-standing array_t/object_t).
This produced wrong JSON Pointer paths in diagnostic messages and could
trip assert_invariant() on subsequent copies. Mirrors the fix already
applied in swap(reference other).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-15 20:46:28 +02:00
208 changed files with 23565 additions and 2209 deletions
Running `make amalgamate` will also apply automatic formatting to the source files using
Running `make amalgamate` will also apply automatic formatting to the source files using
[`Artistic Style`](https://astyle.sourceforge.net/). This formatting may modify your source files in-place. Be certain to review and commit any changes to avoid unintended formatting diffs in commits.
[`Artistic Style`](https://astyle.sourceforge.net/). This formatting may modify your source files in-place. Be certain to review and commit any changes to avoid unintended formatting diffs in commits.
If you add, rename, or remove a header in `include/nlohmann`, also regenerate the header list in
[`BUILD.bazel`](https://github.com/nlohmann/json/blob/develop/BUILD.bazel) (requires CMake) by executing:
```shell
make BUILD.bazel
```
The amalgamation check in CI fails if any of these generated files is out of date.
## Recommended documentation
## Recommended documentation
- The library’s [README file](https://github.com/nlohmann/json/blob/master/README.md) is an excellent starting point to
- The library’s [README file](https://github.com/nlohmann/json/blob/master/README.md) is an excellent starting point to
- [ ] The changes are described in detail, both the what and why.
- [ ] The changes are described in detail, both the what and why.
- [ ] If applicable, an [existing issue](https://github.com/nlohmann/json/issues) is referenced.
- [ ] If applicable, an [existing issue](https://github.com/nlohmann/json/issues) is referenced.
- [ ] If applicable, a fixed [OSS-Fuzz](https://issues.oss-fuzz.com) issue is referenced as `OSS-Fuzz: <id>` (see [fuzz testing](https://github.com/nlohmann/json/blob/develop/tests/fuzzing.md#handling-oss-fuzz-reports)).
- [ ] The [Code coverage](https://coveralls.io/github/nlohmann/json) remained at 100%. A test case for every new line of code.
- [ ] The [Code coverage](https://coveralls.io/github/nlohmann/json) remained at 100%. A test case for every new line of code.
- [ ] If applicable, the [documentation](https://json.nlohmann.me) is updated.
- [ ] If applicable, the [documentation](https://json.nlohmann.me) is updated.
- [ ] The source code is amalgamated by running `make amalgamate`.
- [ ] The source code is amalgamated by running `make amalgamate`.
@diff BUILD.bazel BUILD.bazel~ ||(echo"===================================================================\n BUILD.bazel is out of date! Please run 'make BUILD.bazel'.\n==================================================================="; mv BUILD.bazel~ BUILD.bazel ;false)
@mv BUILD.bazel~ BUILD.bazel
BUILD.bazel:$(SRCS)
# generate the Bazel BUILD file; phony, because a removed header would not trigger a rebuild
[](https://isitmaintained.com/project/nlohmann/json "Average time to resolve an issue")
[](https://isitmaintained.com/project/nlohmann/json "Average time to resolve an issue")
[](https://bestpractices.coreinfrastructure.org/projects/289)
[](https://www.bestpractices.dev/projects/289)
- [Binary formats (BSON, CBOR, MessagePack, UBJSON, and BJData)](#binary-formats-bson-cbor-messagepack-ubjson-and-bjdata)
- [Binary formats (BSON, CBOR, MessagePack, UBJSON, BJData, and BON8)](#binary-formats-bson-cbor-messagepack-ubjson-bjdata-and-bon8)
- [Customers](#customers)
- [Customers](#customers)
- [Ecosystem](#ecosystem)
- [Ecosystem](#ecosystem)
- [Supported compilers](#supported-compilers)
- [Supported compilers](#supported-compilers)
@@ -63,7 +63,7 @@ There are myriads of [JSON](https://json.org) libraries out there, and each may
- **Trivial integration**. Our whole code consists of a single header file [`json.hpp`](https://github.com/nlohmann/json/blob/develop/single_include/nlohmann/json.hpp). That's it. No library, no subproject, no dependencies, no complex build system. The class is written in vanilla C++11. All in all, everything should require no adjustment of your compiler flags or project settings. The library is also included in all popular [package managers](https://json.nlohmann.me/integration/package_managers/).
- **Trivial integration**. Our whole code consists of a single header file [`json.hpp`](https://github.com/nlohmann/json/blob/develop/single_include/nlohmann/json.hpp). That's it. No library, no subproject, no dependencies, no complex build system. The class is written in vanilla C++11. All in all, everything should require no adjustment of your compiler flags or project settings. The library is also included in all popular [package managers](https://json.nlohmann.me/integration/package_managers/).
- **Serious testing**. Our code is heavily [unit-tested](https://github.com/nlohmann/json/tree/develop/tests/src) and covers [100%](https://coveralls.io/r/nlohmann/json) of the code, including all exceptional behavior. Furthermore, we checked with [Valgrind](https://valgrind.org) and the [Clang Sanitizers](https://clang.llvm.org/docs/index.html) that there are no memory leaks. [Google OSS-Fuzz](https://github.com/google/oss-fuzz/tree/master/projects/json) additionally runs fuzz tests against all parsers 24/7, effectively executing billions of tests so far. To maintain high quality, the project is following the [Core Infrastructure Initiative (CII) best practices](https://bestpractices.coreinfrastructure.org/projects/289). See the [quality assurance](https://json.nlohmann.me/community/quality_assurance) overview documentation.
- **Serious testing**. Our code is heavily [unit-tested](https://github.com/nlohmann/json/tree/develop/tests/src) and covers [100%](https://coveralls.io/r/nlohmann/json) of the code, including all exceptional behavior. Furthermore, we checked with [Valgrind](https://valgrind.org) and the [Clang Sanitizers](https://clang.llvm.org/docs/index.html) that there are no memory leaks. [Google OSS-Fuzz](https://github.com/google/oss-fuzz/tree/master/projects/json) additionally runs fuzz tests against all parsers 24/7, effectively executing billions of tests so far. To maintain high quality, the project is following the [OpenSSF Best Practices](https://www.bestpractices.dev/projects/289). See the [quality assurance](https://json.nlohmann.me/community/quality_assurance) overview documentation.
Other aspects were not so important to us:
Other aspects were not so important to us:
@@ -128,7 +128,7 @@ There is also a [**docset**](https://github.com/Kapeli/Dash-User-Contributions/t
- When using `get<ENUM_TYPE>()`, undefined JSON values will default to the first pair specified in your map. Select this default pair carefully. If you desire an exception in this circumstance use `NLOHMANN_JSON_SERIALIZE_ENUM_STRICT()` which behaves identically except for throwing an exception on unrecognized values.
- When using `get<ENUM_TYPE>()`, undefined JSON values will default to the first pair specified in your map. Select this default pair carefully. If you desire an exception in this circumstance use `NLOHMANN_JSON_SERIALIZE_ENUM_STRICT()` which behaves identically except for throwing an exception on unrecognized values.
- If an enum or JSON value is specified more than once in your map, the first matching occurrence from the top of the map will be returned when converting to or from JSON.
- If an enum or JSON value is specified more than once in your map, the first matching occurrence from the top of the map will be returned when converting to or from JSON.
### Binary formats (BSON, CBOR, MessagePack, UBJSON, and BJData)
### Binary formats (BSON, CBOR, MessagePack, UBJSON, BJData, and BON8)
Though JSON is a ubiquitous data format, it is not a very compact format suitable for data exchange, for instance over a network. Hence, the library supports [BSON](https://bsonspec.org) (Binary JSON), [CBOR](https://cbor.io) (Concise Binary Object Representation), [MessagePack](https://msgpack.org), [UBJSON](https://ubjson.org) (Universal Binary JSON Specification) and [BJData](https://neurojson.org/bjdata) (Binary JData) to efficiently encode JSON values to byte vectors and to decode such vectors.
Though JSON is a ubiquitous data format, it is not a very compact format suitable for data exchange, for instance over a network. Hence, the library supports [BSON](https://bsonspec.org) (Binary JSON), [CBOR](https://cbor.io) (Concise Binary Object Representation), [MessagePack](https://msgpack.org), [UBJSON](https://ubjson.org) (Universal Binary JSON Specification), [BJData](https://neurojson.org/bjdata) (Binary JData), and [BON8](https://github.com/hikoworks/hikogui/blob/main/docs/BON8.md) (Binary Object Notation 8) to efficiently encode JSON values to byte vectors and to decode such vectors.
The library also supports binary types from BSON, CBOR (byte strings), and MessagePack (bin, ext, fixext). They are stored by default as `std::vector<std::uint8_t>` to be processed outside the library.
The library also supports binary types from BSON, CBOR (byte strings), and MessagePack (bin, ext, fixext). They are stored by default as `std::vector<std::uint8_t>` to be processed outside the library.
@@ -1421,7 +1429,7 @@ I deeply appreciate the help of the following people.
6. [Joshua C. Randall](https://github.com/jrandall) fixed a bug in the floating-point serialization.
6. [Joshua C. Randall](https://github.com/jrandall) fixed a bug in the floating-point serialization.
7. [Aaron Burghardt](https://github.com/aburgh) implemented code to parse streams incrementally. Furthermore, he greatly improved the parser class by allowing the definition of a filter function to discard undesired elements while parsing.
7. [Aaron Burghardt](https://github.com/aburgh) implemented code to parse streams incrementally. Furthermore, he greatly improved the parser class by allowing the definition of a filter function to discard undesired elements while parsing.
8. [Daniel Kopeček](https://github.com/dkopecek) fixed a bug in the compilation with GCC 5.0.
8. [Daniel Kopeček](https://github.com/dkopecek) fixed a bug in the compilation with GCC 5.0.
9. [Florian Weber](https://github.com/Florianjw) fixed a bug in and improved the performance of the comparison operators.
9. [Fiona Johanna Weber](https://github.com/Fiona-J-W) fixed a bug in and improved the performance of the comparison operators.
10. [Eric Cornelius](https://github.com/EricMCornelius) pointed out a bug in the handling with NaN and infinity values. He also improved the performance of the string escaping.
10. [Eric Cornelius](https://github.com/EricMCornelius) pointed out a bug in the handling with NaN and infinity values. He also improved the performance of the string escaping.
11. [易思龙](https://github.com/likebeta) implemented a conversion from anonymous enums.
11. [易思龙](https://github.com/likebeta) implemented a conversion from anonymous enums.
12. [kepkin](https://github.com/kepkin) patiently pushed forward the support for Microsoft Visual Studio.
12. [kepkin](https://github.com/kepkin) patiently pushed forward the support for Microsoft Visual Studio.
@@ -1523,14 +1531,14 @@ I deeply appreciate the help of the following people.
108. [Kevin Tonon](https://github.com/ktonon) overworked the C++11 compiler checks in CMake.
108. [Kevin Tonon](https://github.com/ktonon) overworked the C++11 compiler checks in CMake.
109. [Axel Huebl](https://github.com/ax3l) simplified a CMake check and added support for the [Spack package manager](https://spack.io).
109. [Axel Huebl](https://github.com/ax3l) simplified a CMake check and added support for the [Spack package manager](https://spack.io).
110. [Carlos O'Ryan](https://github.com/coryan) fixed a typo.
110. [Carlos O'Ryan](https://github.com/coryan) fixed a typo.
111. [James Upjohn](https://github.com/jammehcow) fixed a version number in the compilers section.
111. [James Upjohn](https://github.com/jupjohn) fixed a version number in the compilers section.
112. [Chuck Atkins](https://github.com/chuckatkins) adjusted the CMake files to the CMake packaging guidelines and provided documentation for the CMake integration.
112. [Chuck Atkins](https://github.com/chuckatkins) adjusted the CMake files to the CMake packaging guidelines and provided documentation for the CMake integration.
113. [Jan Schöppach](https://github.com/dns13) fixed a typo.
113. [Jan Schöppach](https://github.com/dns13) fixed a typo.
114. [martin-mfg](https://github.com/martin-mfg) fixed a typo.
114. [martin-mfg](https://github.com/martin-mfg) fixed a typo.
115. [Matthias Möller](https://github.com/TinyTinni) removed the dependency from `std::stringstream`.
115. [Matthias Möller](https://github.com/TinyTinni) removed the dependency from `std::stringstream`.
116. [agrianius](https://github.com/agrianius) added code to use alternative string implementations.
116. [agrianius](https://github.com/agrianius) added code to use alternative string implementations.
117. [Daniel599](https://github.com/Daniel599) allowed to use more algorithms with the `items()` function.
117. [Daniel599](https://github.com/Daniel599) allowed to use more algorithms with the `items()` function.
118. [Julius Rakow](https://github.com/jrakow) fixed the Meson include directory and fixed the links to [cppreference.com](https://cppreference.com).
118. [Julius Rakow](https://github.com/juliusrakow) fixed the Meson include directory and fixed the links to [cppreference.com](https://cppreference.com).
119. [Sonu Lohani](https://github.com/sonulohani) fixed the compilation with MSVC 2015 in debug mode.
119. [Sonu Lohani](https://github.com/sonulohani) fixed the compilation with MSVC 2015 in debug mode.
120. [grembo](https://github.com/grembo) fixed the test suite and re-enabled several test cases.
120. [grembo](https://github.com/grembo) fixed the test suite and re-enabled several test cases.
121. [Hyeon Kim](https://github.com/simnalamburt) introduced the macro `JSON_INTERNAL_CATCH` to control the exception handling inside the library.
121. [Hyeon Kim](https://github.com/simnalamburt) introduced the macro `JSON_INTERNAL_CATCH` to control the exception handling inside the library.
@@ -1581,7 +1589,7 @@ I deeply appreciate the help of the following people.
166. [Mark Beckwith](https://github.com/wythe) fixed a typo.
166. [Mark Beckwith](https://github.com/wythe) fixed a typo.
167. [yann-morin-1998](https://github.com/yann-morin-1998) helped to reduce the CMake requirement to version 3.1.
167. [yann-morin-1998](https://github.com/yann-morin-1998) helped to reduce the CMake requirement to version 3.1.
168. [Konstantin Podsvirov](https://github.com/podsvirov) maintains a package for the MSYS2 software distro.
168. [Konstantin Podsvirov](https://github.com/podsvirov) maintains a package for the MSYS2 software distro.
169. [remyabel](https://github.com/remyabel) added GNUInstallDirs to the CMake files.
169. [remyabel](https://github.com/remyabel2) added GNUInstallDirs to the CMake files.
170. [Taylor Howard](https://github.com/taylorhoward92) fixed a unit test.
170. [Taylor Howard](https://github.com/taylorhoward92) fixed a unit test.
171. [Gabe Ron](https://github.com/Macr0Nerd) implemented the `to_string` method.
171. [Gabe Ron](https://github.com/Macr0Nerd) implemented the `to_string` method.
172. [Watal M. Iwasaki](https://github.com/heavywatal) fixed a Clang warning.
172. [Watal M. Iwasaki](https://github.com/heavywatal) fixed a Clang warning.
@@ -1608,7 +1616,7 @@ I deeply appreciate the help of the following people.
193. [Hubert Chathi](https://github.com/uhoreg) made CMake's version config file architecture-independent.
193. [Hubert Chathi](https://github.com/uhoreg) made CMake's version config file architecture-independent.
194. [OmnipotentEntity](https://github.com/OmnipotentEntity) implemented the binary values for CBOR, MessagePack, BSON, and UBJSON.
194. [OmnipotentEntity](https://github.com/OmnipotentEntity) implemented the binary values for CBOR, MessagePack, BSON, and UBJSON.
195. [ArtemSarmini](https://github.com/ArtemSarmini) fixed a compilation issue with GCC 10 and fixed a leak.
195. [ArtemSarmini](https://github.com/ArtemSarmini) fixed a compilation issue with GCC 10 and fixed a leak.
196. [Evgenii Sopov](https://github.com/sea-kg) integrated the library to the wsjcpp package manager.
196. [Evgenii Sopov](https://github.com/sea5kg) integrated the library to the wsjcpp package manager.
197. [Sergey Linev](https://github.com/linev) fixed a compiler warning.
197. [Sergey Linev](https://github.com/linev) fixed a compiler warning.
198. [Miguel Magalhães](https://github.com/magamig) fixed the year in the copyright.
198. [Miguel Magalhães](https://github.com/magamig) fixed the year in the copyright.
199. [Gareth Sylvester-Bradley](https://github.com/garethsb-sony) fixed a compilation issue with MSVC.
199. [Gareth Sylvester-Bradley](https://github.com/garethsb-sony) fixed a compilation issue with MSVC.
@@ -1702,7 +1710,7 @@ I deeply appreciate the help of the following people.
287. [NN](https://github.com/NN---) added the Visual Studio output directory to `.gitignore`.
287. [NN](https://github.com/NN---) added the Visual Studio output directory to `.gitignore`.
288. [Romain Reignier](https://github.com/romainreignier) improved the performance of the vector output adapter.
288. [Romain Reignier](https://github.com/romainreignier) improved the performance of the vector output adapter.
289. [Mike](https://github.com/Mike-Leo-Smith) fixed the `std::iterator_traits`.
289. [Mike](https://github.com/Mike-Leo-Smith) fixed the `std::iterator_traits`.
291. [vakokako](https://github.com/vakokako) fixed tests when compiling with C++20.
291. [vakokako](https://github.com/vakokako) fixed tests when compiling with C++20.
292. [Alexander “weej” Jones](https://github.com/alexweej) fixed an example in the README.
292. [Alexander “weej” Jones](https://github.com/alexweej) fixed an example in the README.
293. [Eli Schwartz](https://github.com/eli-schwartz) added more files to the `include.zip` archive.
293. [Eli Schwartz](https://github.com/eli-schwartz) added more files to the `include.zip` archive.
@@ -1727,7 +1735,7 @@ I deeply appreciate the help of the following people.
312. [Gareth Sylvester-Bradley](https://github.com/garethsb) added `operator/=` and `operator/` to construct JSON pointers.
312. [Gareth Sylvester-Bradley](https://github.com/garethsb) added `operator/=` and `operator/` to construct JSON pointers.
313. [Michael Macnair](https://github.com/mykter) added support for afl-fuzz testing.
313. [Michael Macnair](https://github.com/mykter) added support for afl-fuzz testing.
314. [Berkus Decker](https://github.com/berkus) fixed a typo in the README.
314. [Berkus Decker](https://github.com/berkus) fixed a typo in the README.
315. [Illia Polishchuk](https://github.com/effolkronium) improved the CMake testing.
315. [Illia Polishchuk](https://github.com/ilqvya) improved the CMake testing.
316. [Ikko Ashimine](https://github.com/eltociear) fixed a typo.
316. [Ikko Ashimine](https://github.com/eltociear) fixed a typo.
317. [Raphael Grimm](https://github.com/barcode) added the possibility to define a custom base class.
317. [Raphael Grimm](https://github.com/barcode) added the possibility to define a custom base class.
318. [tocic](https://github.com/tocic) fixed typos in the documentation.
318. [tocic](https://github.com/tocic) fixed typos in the documentation.
@@ -1797,6 +1805,66 @@ I deeply appreciate the help of the following people.
382. [bitFiedler](https://github.com/bitFiedler) made GDB pretty printer work with Python 3.8.
382. [bitFiedler](https://github.com/bitFiedler) made GDB pretty printer work with Python 3.8.
383. [Gianfranco Costamagna](https://github.com/LocutusOfBorg) fixed a compiler warning.
383. [Gianfranco Costamagna](https://github.com/LocutusOfBorg) fixed a compiler warning.
384. [risa2000](https://github.com/risa2000) made `std::filesystem::path` conversion to/from UTF-8 encoded string explicit.
384. [risa2000](https://github.com/risa2000) made `std::filesystem::path` conversion to/from UTF-8 encoded string explicit.
385. [AM](https://github.com/maqnouch) fixed typos in the README.
386. [dmenendez-gruposantander](https://github.com/dmenendez-gruposantander) fixed typos in the comments of the examples.
387. [Mihai Stan](https://github.com/mstan-xx) fixed comparisons against the literal `0`.
388. [Matt Gumbel](https://github.com/intelmatt) fixed some `-Weffc++` warnings.
389. [vimpunk](https://github.com/vimpunk) moved a lambda out of an unevaluated context to support older compilers.
390. [Chris Harris](https://github.com/cjh1) fixed the compilation with GCC 4.8.
391. [Palmer Dabbelt](https://github.com/palmer-dabbelt) generated and installed a pkg-config file.
392. [Gus Pozuelo](https://github.com/ap-viavi) made `ordered_map` compatible with GCC 5.5, Clang 3.6, and Xcode 9.
393. [AK](https://github.com/Lioncky) fixed an MSVC build error caused by the `min`/`max` macros from `windows.h`.
394. [Sergiu Deitsch](https://github.com/sergiud) provided a fallback for missing `char8_t` support.
395. [Xiaochuan Ye](https://github.com/XueSongTap) fixed `from_msgpack` for `std::byte` input by specializing `std::char_traits`.
396. [Ville Vesilehto](https://github.com/thevilledev) fixed an overflow in the BJData size calculation and rejected overflowing negative integers in CBOR.
397. [NmPassTHFan](https://github.com/nmpassthf) replaced the deprecated `std::is_trivial` for C++26.
398. [Chris Ever](https://github.com/chirsz-ever) added the `ignore_trailing_commas` parser option.
399. [Kuan-Fu Wu](https://github.com/kfwu1999) fixed the example code for `json_pointer` initialization.
400. [David Kilzer](https://github.com/ddkilzer) added a missing header to the input adapters.
401. [Miko](https://github.com/mikomikotaishi) added proper C++20 module support, simplified the module API, and fixed missing exports.
402. [hitgirl](https://github.com/hitgil) fixed the CMake configuration when cross-compiling.
403. [Devon Thomas](https://github.com/ThomaDevOSU) mentioned the Artistic Style formatting in the contribution guidelines.
404. [Erik Hu](https://github.com/Erikhu1) made Coveralls upload errors non-fatal in the CI.
406. [DmitriBogdanov](https://github.com/DmitriBogdanov) fixed broken package manager links in the documentation.
407. [Bander](https://github.com/banderzhm) improved the MSVC compatibility of the C++ modules.
408. [Andy Choi](https://github.com/ccpong) removed an unnecessary `template` keyword before `get` in the README and the documentation.
409. [SamareshSingh](https://github.com/ssam18) fixed single-element brace initialization to copy/move instead of wrapping in an array, fixed the `WITH_DEFAULT` macros for `ordered_map`, and handled moved events in `serve_header.py`.
410. [Aditya](https://github.com/Lumowhisp) improved the documentation of the documentation generation.
411. [cheese1](https://github.com/cheese1) clarified the README.
412. [KhloodElhossiny](https://github.com/khloodelhossiny) enabled `std::string_view` keys in `operator[]`.
413. [Charles Cabergs](https://github.com/cacharle) fixed a `-Wtautological-constant-out-of-range-compare` warning.
414. [EALePain](https://github.com/EALePain) made the `std::tuple` conversion work with reference types such as `std::tie`.
415. [koala_oishi](https://github.com/chibi-dogs) fixed grammatical wording in the README.
416. [riccardoori11](https://github.com/riccardoori11) fixed a typo in the documentation.
417. [Swastik Bose](https://github.com/VasuBhakt) fixed the parent pointers after `update()` with `JSON_DIAGNOSTICS` and fixed the Doxygen autolinking of requirements.
418. [trdesilva](https://github.com/trdesilva) added `front`, `pop_front`, and `push_front` to `json_pointer`.
419. [Akhilesh Arora](https://github.com/akhilesharora) fixed an incomplete-type error with `ordered_json`.
420. [Hariom Phulre](https://github.com/hariomphulre) fixed the C++20 modules compilation with GCC.
422. [George Sedov](https://github.com/radistmorse) added the `NLOHMANN_DEFINE_TYPE_*_WITH_NAMES` macros.
423. [Caillin Nugent](https://github.com/nugentcaillin) added the `NLOHMANN_JSON_SERIALIZE_ENUM_STRICT` macro.
424. [Cosmin D.](https://github.com/drcosmin) fixed `std::filesystem::path` conversions and added an MSVC workaround for `std::unique_ptr`.
425. [Paul Dreik](https://github.com/pauldreik) fixed a test relying on implementation-specific behavior.
426. [Daniel Falk](https://github.com/daniel-falk) added missing copyright notices to the SBOM.
427. [Federico Sfriso](https://github.com/federicosfriso05-dotcom) added support for constructing JSON values from C++20 range views.
428. [Luke Banicevic](https://github.com/banaboi) fixed corrupt BSON output for lengths exceeding `INT32_MAX`, cleaned up the BSON writer, and improved the documentation.
429. [Patrick Armstrong](https://github.com/Patrick10199) updated the CBOR references and the half-precision float assertions.
430. [Yash Bavadiya](https://github.com/xevrion) added checks to all BSON reads.
431. [hum4nBeing](https://github.com/hum4nBeing) fixed the overflow handling of high-precision numbers in UBJSON.
432. [tomatotomata](https://github.com/tomatotomata) added checks for reading CBOR tagged subtypes.
433. [YingqiDuan](https://github.com/YingqiDuan) documented the BSON interoperability.
434. [KBS](https://github.com/youdie006) documented the standards compliance and the strictness of `parse()` and `operator>>`.
435. [Petr Bělohlávek](https://github.com/petrbel) added Clang 21 and 22 to the CI.
436. [Dmitry Rantovov](https://github.com/darkdi) fixed the placement of a CBOR documentation block.
437. [ljcjclljc](https://github.com/ljcjclljc) fixed the comparison of large unsigned integers with signed integers.
438. [Sahil Kamate](https://github.com/sahilkamate03) fixed the handling of CBOR tags 0-5 and 21-23.
439. [Krishnanand G](https://github.com/Krishnanand-G) made the UBJSON writer reject `use_type` without `use_size`.
440. [whn](https://github.com/Whning0513) documented the lenient BSON input handling and corrected the complexity of `to_bson`.
441. [elix3r](https://github.com/22elix3r) fixed `update()` with `merge_objects` when merging a primitive into an object.
442. [Avionic Harshit](https://github.com/avionicharshit-byte) made `diff()` linear when an array shrinks.
443. [Qatadaha Bin Matloob](https://github.com/qatcod) fixed comparisons between integers and floats and fixed unparsable BJData output.
444. [Wu Shuwen](https://github.com/dajiaohuang) removed an unused include.
Thanks a lot for helping out! Please [let me know](mailto:mail@nlohmann.me) if I forgot someone.
Thanks a lot for helping out! Please [let me know](mailto:mail@nlohmann.me) if I forgot someone.
: store tagged values as binary container with subtype (for bytes 0xd8..0xdb)
: store tagged byte strings (for bytes 0xd8..0xdb) as binary values with the tag as subtype; other tagged values are read as if the tag were ignored. If several tags precede a byte string, only the innermost one is stored.
| Out-of-bounds read/write ([CWE-125](https://cwe.mitre.org/data/definitions/125.html), [CWE-787](https://cwe.mitre.org/data/definitions/787.html)) | bounds checks on all reads from the input; AddressSanitizer and Valgrind on the test suite; OSS-Fuzz |
| Use after free, double free ([CWE-416](https://cwe.mitre.org/data/definitions/416.html), [CWE-415](https://cwe.mitre.org/data/definitions/415.html)) | ownership of all memory by values; AddressSanitizer and Valgrind; Clang Static Analyzer |
| Memory leaks ([CWE-401](https://cwe.mitre.org/data/definitions/401.html)) | Valgrind (Memcheck) on the test suite |
| Uncontrolled recursion ([CWE-674](https://cwe.mitre.org/data/definitions/674.html)) | iterative parser, binary readers, and destructor; bounded recursion in value operations; tests with deeply nested inputs |
| Uncontrolled resource consumption ([CWE-400](https://cwe.mitre.org/data/definitions/400.html)) | allocations based on announced sizes are capped; OSS-Fuzz with memory limits |
| Undefined behavior in general ([CWE-758](https://cwe.mitre.org/data/definitions/758.html)) | UndefinedBehaviorSanitizer; runtime assertions; Clang-Tidy, Cppcheck, Clang Static Analyzer, Infer |
In addition, every line of the library is covered by the unit tests, and all parsers are fuzz-tested around the clock
by [OSS-Fuzz](https://github.com/google/oss-fuzz/tree/master/projects/json).
@@ -176,9 +176,19 @@ The library maps CBOR types to JSON value types as follows:
CBOR allows map keys of any type, whereas JSON only allows strings as keys in object values. Therefore, CBOR maps with keys other than UTF-8 strings are rejected.
CBOR allows map keys of any type, whereas JSON only allows strings as keys in object values. Therefore, CBOR maps with keys other than UTF-8 strings are rejected.
!!! warning "UTF-8 validation of text strings"
[RFC 8949, Section 3.1](https://www.rfc-editor.org/rfc/rfc8949.html#section-3.1) requires CBOR text strings
(major type 3) to be valid UTF-8. This library validates the bytes of every text string (object keys included) at
decode time and rejects ill-formed UTF-8 with a
[`parse_error.113`](../../home/exceptions.md#jsonexceptionparse_error113) exception (or, with
`allow_exceptions` set to `false`, a discarded value), rather than only failing later when the resulting value is
dumped. Byte strings (major type 2) are unaffected and are never validated, since they are not required to hold
text.
!!! warning "Tagged items"
!!! warning "Tagged items"
Tagged items (0xC0..0xDB) will throw a parse error by default. They can be ignored by passing `cbor_tag_handler_t::ignore` to function `from_cbor`, in which case the tag is skipped and the enclosed data item is parsed on its own. They can be stored by passing `cbor_tag_handler_t::store` to function `from_cbor`. Note that no tag is ever interpreted: for instance, a text string tagged with tag 0 (date/time) stays a string.
Tagged items (0xC0..0xDB) will throw a parse error by default. They can be ignored by passing `cbor_tag_handler_t::ignore` to function `from_cbor`, in which case the tag is skipped and the enclosed data item is parsed on its own. Passing `cbor_tag_handler_t::store` to function `from_cbor` stores tagged byte strings (for bytes 0xd8..0xdb) as binary values with the tag as subtype; other tagged values are read as if the tag were ignored. If several tags precede a byte string, only the innermost one is stored. Note that no tag is ever interpreted: for instance, a text string tagged with tag 0 (date/time) stays a string.
- [`JSON_DIAGNOSTIC_POSITIONS`](../api/macros/json_diagnostic_positions.md) defined non-zero appends `_dp`.
- [`JSON_BRACE_INIT_COPY_SEMANTICS`](../api/macros/json_brace_init_copy_semantics.md) defined non-zero appends
`_bics`.
- [`JSON_PRECISE_STREAM_POSITION`](../api/macros/json_precise_stream_position.md) defined non-zero appends `_psp`.
- [`JSON_STRICT_NUL_HANDLING`](../api/macros/json_strict_nul_handling.md) defined non-zero appends `_snul`.
- The inline namespace ends with the suffix `_v` followed by the 3 components of the version number separated by
- The inline namespace ends with the suffix `_v` followed by the 3 components of the version number separated by
underscores. To omit the version component, see [Disabling the version component](#disabling-the-version-component)
underscores. To omit the version component, see [Disabling the version component](#disabling-the-version-component)
below.
below.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Niels Lohmann
Kartikey Negi
bucketbase26
Dadi Reddy Sai Praneeth Reddy
dependabot[bot]
Alexander LaninandNiels Lohmann
Jeremy Nimmer