* Allow the same type for number_integer_t and number_unsigned_t
basic_json did not compile with NumberUnsignedType == NumberIntegerType
because overloads on both types collided. Such a basic_json now stores
every integer as number_integer: the colliding overloads take a
placeholder type, unsigned C++ integers are converted to number_integer_t,
and value_t::number_unsigned maps to number_integer.
Also fixes two problems this configuration would hit: the unsigned
integer parser accepted values that wrap around to negative numbers, and
dump() overflowed when computing the absolute value of the minimum.
A repeated NumberFloatType gets a static_assert.
Refs #2573
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Use json::with_integers_t in the test and the docs
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* ✅ skip patch test on GCC 5 to avoid internal compiler error
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix CI: name the unsigned overload type through template parameters
basic_json used number_unsigned_overload_t<basic_json> while the class was
still incomplete. MSVC rejected it as a recursive alias declaration and GCC
4.9, 5 and 7 crashed with an internal compiler error. basic_json now uses
number_unsigned_overload_type<NumberIntegerType, NumberUnsignedType>, which
also makes the GCC 5 workaround in the patch test unnecessary.
Infer reported a null dereference for the number_unsigned case in
from_json, which cannot happen for a basic_json whose number_unsigned_t is
number_integer_t; assert it.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Address review: conjunction/negation trait, astyle-friendly alias, docs table order
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
patch_inplace() built its error messages inline. In particular get_value()
concatenated the context-sensitive "operation 'x'" string on every call, even
when nothing was wrong. The patch exceptions now come from NEVER_INLINE
helpers: the index and key errors reuse basic_json's existing
throw_array_index_out_of_range/throw_key_not_found, and the seven
patch-specific messages get one helper each in detail/exceptions.hpp. Ids
and messages are unchanged, and unit-throw_messages pins them (with and
without JSON_DIAGNOSTICS).
citm_catalog patch: -6.6 % (Apple clang), -9.5 % (GCC 16).
The other remaining throw sites (parser::syntax_error, binary_writer,
serializer, input_adapters, enum_key_duplicate, format_error) were prototyped
and measured: no gain beyond noise, so they stay as they are.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Speed up container destruction, iterator access and diff()
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Destroy the first levels of a value by recursion
The pointer-reversing walk in destroy_container() avoids the call stack but
moves a value around for every container it visits. Recurse for the first
recursion_depth_limit() levels instead - no allocation, no thread-local, a
bounded frame - and hand anything deeper to the walk. Destroy twitter
0.49 -> 0.36 ms, citm 0.80 -> 0.74 ms (3.12.0: 0.47 / 1.00).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Make the value lifecycle inline-friendly
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Compare standard containers without a thread-local depth count
operator== on objects and arrays recursed through the containers' own
operator== and counted the depth in a thread-local variable. Where reading a
thread-local costs a call (macOS), that added about 10% to comparing documents
with many small containers. For the standard containers, walk objects and
arrays directly, compare scalars and strings inline, and pass the depth as an
argument, falling back to compare_iteratively beyond the depth limit. Other
container types keep the previous path.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Address review: owns_memory() helper, leaf compare naming, astyle
- Add detail::owns_memory(value_t), a switch listing every enumerator so
-Wswitch-enum flags it when a type is added; destroy() and the copy
constructor use it instead of hand-written comparisons. Clang emits the
same bitmask test as before.
- Drop the parentheses around the scalar operands in
JSON_IMPLEMENT_SCALAR_CASES; they are only needed for the container
operands (#1530), which the comment now says.
- Build equal_bounded_containers from detail::conjunction so astyle no
longer pads the template brackets.
- Rename equal_scalar() to equal_leaf() and document why it also compares
strings, unlike JSON_IMPLEMENT_SCALAR_FAST.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix quadratic parent updates for ordered_json with JSON_DIAGNOSTICS
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Only trust containers known to keep their elements in place
Address review: decide whether an insertion may have moved the other
members by naming the containers that are known not to, instead of the
ones that are known to.
- detail::has_stable_elements: std::map, std::multimap,
std::unordered_map and std::unordered_multimap; object_capacity()
reports 0 for them.
- detail::is_nlohmann_ordered_map: only the real nlohmann::ordered_map
(which only appends) reports its capacity().
- Any other object type reports unknown_size(), so it gets a full
set_parents() on every insertion. Before, an object type without
capacity() was treated as never moving, and one with capacity() (a
sorted flat map, say) was trusted to move only on reallocation.
- set_parents_after_object_erase() re-parents unless the object type is
stable; the unused set_parent() is removed.
- Comments say "the object type" instead of "ordered_json".
- New test with an object type that moves its members without changing
its capacity; it fails without this change.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Move elements when converting rvalue containers to JSON
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Simplify the rvalue to_json traits
Use negation<disjunction<...>> instead of a conjunction of negations and
keep the template argument lists compact (astyle only keeps the spaced
'< ... >' form when it is written that way).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Speed up array-to-container from_json
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Hint the end when converting an object to a standard container
from_json for map-like containers called emplace() without a hint, which
searches the tree for every key. The keys of the source object are sorted, so
inserting with the end as the hint is constant time, as it was in 3.12.0 via
std::inserter. get<std::map<...>>() of a 25,000 entry object: 4.2 ms -> 3.25 ms
(3.12.0: 3.7 ms).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fall back to emplace() in object from_json for containers without emplace_hint()
ordered_map has no hinted insert, so the insert(end(), ...) call broke
from_json into types holding an ordered_map (regression test #5122).
Use emplace_hint() where available and plain emplace() otherwise.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Speed up JSON pointer parsing, formatting, comparison and lookup
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Restore unflatten() speed after the linear-time rewrite
The prefix tree introduced for linear unflatten() kept a std::map per prefix,
copied each token into it and looked every token up a second time while
creating the values. Replace the per-prefix maps by one open-addressing table
keyed by (parent, token) that refers to the parsed tokens, skip the lookups for
the prefix shared with the previous key (flattened objects are mostly sorted),
and derive the prefix numbers of the second pass from parent links instead of
lookups.
Time and memory stay linear in the pointer depth, and the result is still
independent of the iteration order of the flattened object.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Put [[noreturn]] before __attribute__ on the JSON pointer throw helpers
Clang 18 on Linux rejects a C++11 attribute list after a GNU
__attribute__ ("an attribute list cannot appear here").
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Make flatten() linear for ordered_json
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Use priority_tag instead of a C-style variadic overload in add_flattened
clang-tidy flags the ellipsis fallback (cert-dcl50-cpp,
modernize-avoid-variadic-functions).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Follow-up to review comments on #5825: number_float_t uses double
precision only by default, and the stated limits apply to double.
Also say "default type for X_t" instead of "default value for X_t".
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
- Discarded values: the legacy macro only changes <= and >=; != is true by default
- Out-of-range numbers are rejected while parsing, not stored as NaN
- merge_diff round trip: point to the ordered_json caveat
- ensure_ascii example shows the escaped output
- out_of_range.405 is thrown for "remove"/"move" on the root, not "add"
- CBOR stores binary subtypes as tags
- Integer/float comparison uses exact values, not conversion
- diff, flatten, and the binary writers are depth-bounded
- FAQ: max_digits10 for serialization, trailing commas are supported
- out_of_range.407 is still thrown by to_bon8
- FetchContent_MakeAvailable needs CMake 3.14
- json_fwd.hpp contents, operator< link, RFC 7396 for merge patch
- README: dump() wording, merge_diff, MSYS2/build2, patch comment typos,
operator[] link, embedded CMake example aligned with the docs
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The privacy plugin downloads img.shields.io badges at build time; when
shields.io times out, mkdocs build --strict aborts and the
ci_test_build_documentation job fails until it is restarted. Exclude
the host like repology.org; the version badges stay current this way.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Update Google Benchmark to 1.10.0
Bump the pinned release and its archive checksum. Reword the comment and
README: the version is fixed for reproducible builds and updated like any
other dependency; results stay comparable because both sides of a
comparison are built with the same version.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Do not name the Google Benchmark version in the README
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Write doubles with the shortest digits (Żmij), digits in registers
Write doubles with the conversion of Zmij by Victor Zverovich (MIT),
ported to C++11 (detail/conversions/zmij.hpp). It finds the
shortest decimal that reads back as the same double, and the
closest one if there are several. Grisu2, used until now, is fast
but not always shortest: it sometimes writes a 17th digit where 16
suffice, or a last digit that is not the closest. The layout is
unchanged (1.5, 100.0, 1e+100, -0.0); float keeps Grisu2.
Digits are converted eight at a time with the BCD conversion of
Xiang JunBo, as in Zmij, and written with one byte swap per eight
digits and fixed-size moves instead of per-digit loops. Leading and
trailing zeros are counted from those bytes. to_chars() uses a
local buffer when the caller's is shorter than the 41 bytes this
may write. The powers of ten come from the number-parsing table,
adjusted where it holds values rounded up, and extended with Zmij's
compressed tables beyond 10^308.
write_shortest() converts its 16 digits in one vector register
(SSE2 on x86-64, NEON on 64-bit Arm, both baseline) and inserts the
decimal point inside the register, avoiding a store-forwarding
stall that cost about 25% of the time to write a double. dump()
writes floats and integers straight into the serializer's write
buffer instead of copying them from a member buffer, and small
integers eight digits at a time. read_eight_bytes() and
parse_eight_digits() are marked always-inline, which GCC had been
calling out of line in the number-parsing loops.
Of one million random doubles, about 0.14% are now written with
different digits, always to a value that still reads back as the
same double.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix CI findings in the Zmij writer and its tests
- bit_ops.hpp: include macro_scope.hpp (JSON_HEDLEY_ALWAYS_INLINE); fixes IWYU
- to_chars.hpp: C4100 for the unused parameter in release builds, clang-tidy
sign comparison, cpplint runtime/int, GCC -Wstrict-overflow (unsigned abs)
- unit-to_chars.cpp: parse with the library instead of strtod (MinGW's strtod
rounds some 16/17 digit inputs wrongly); no floating-point std::to_chars
with icpc
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Select the Zmij conversion by a trait, not by the double overload
The non-template overloads for double asserted binary64 doubles wherever json.hpp was included, so the library no longer compiled where double is not IEEE 754 binary64 (AVR, -fshort-double). A trait now picks Zmij for any binary64 type, including a long double of that format (MSVC, Apple Arm), and Grisu2 for the others. Remove the unused write_short_decimal, powers_of_ten_16, zmij::decimal, zmij::to_decimal, and shortest_digits(double).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Credit Zmij's authors in to_chars.hpp, the README, and the license page
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Silence MSVC C4127 for a platform-constant check in the to_chars test
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Regenerate the amalgamated headers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Speed up the lexer: own float parser, string scan, and \u table
Give the library its own correctly rounded float converter for
binary32 and binary64 (IEEE 754), and speed up the lexer's string
and escape scanning.
The converter splits a number token into sign, significand, and
decimal exponent, then tries Clinger's fast path, then a templated
Eisel-Lemire step, and falls back to an exact big-integer digit
comparison for tokens with more than 19 significant digits whose two
candidate values round differently. This replaces std::from_chars
and strtod/strtof for both formats, so parsed values no longer
depend on the C/C++ library or the current locale. The strtold
fallback kept for other long double formats (x87, binary128) now
also copies a multi-byte decimal point correctly, fixing #5660.
eisel_lemire() and decimal_to_float() are always inlined so callers
keep the whole conversion in their hot loop.
The string-scanning kernels in string_scan.hpp find a stop byte with
the trailing-zero count of the SWAR mask instead of a byte loop, and
scalar_string_bulk_run() validates a run of multi-byte UTF-8
sequences one after another instead of re-searching after each one.
get_codepoint() decodes a contiguous \uXXXX escape with one table
lookup per byte instead of four range-checked get() calls; the
streaming path and all error positions are unchanged.
Adds 508 generated hard float-parsing cases with expected binary32
and binary64 bits, and kernel-comparison tests for the string scans
and the escape table against byte-by-byte references.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix CI warnings in own float parser
- pow5_table.hpp: pow5_128_largest_power was unused in this branch's
own code (GCC -Werror=unused-const-variable); tie it to the table
size with a static_assert instead of removing it, since a later
branch in the stack (json-view/23-zmij) uses it.
- number_parse.hpp: rename the local variable `copy` to `buffer` to
satisfy cpplint's build/include_what_you_use check.
- unit-class_lexer.cpp: extend the NOLINT list on the seeded mt19937
with bugprone-random-generator-seed, and parenthesize
`8 * sizeof(Bits) - 1` for clang-tidy.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Load string scan words with memcpy and use MSVC bit-scan intrinsics
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Reword float conversion notes for the string type and number_float_t docs
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix clang-tidy 22 findings in unit-class_lexer.cpp
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Mark the unsigned long of the MSVC bit scans for cpplint
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Regenerate the amalgamated headers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Drop the position updates in the bulk \u path, which #5809 made unused
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Address review: std::any_of in has_nonzero_digit, comment punctuation
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Use data() instead of c_str() in the multi-byte decimal point path
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Drop the class wrapper from the C++20 comparison operator signatures in the docs
The signature boxes of operator<=> and the C++20 half of operator== wrapped
the member declarations in an incomplete "class basic_json { ... };". The
trailing const already marks them as members, and no other page uses such a
wrapper.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Drop the class wrapper from the json_pointer comparison signatures too
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Mark json_fwd.hpp as exported by json.hpp for IWYU / include-cleaner
json_fwd.hpp is the only header declaring the nlohmann::json alias, so
include-what-you-use and clang-tidy's misc-include-cleaner attributed
the name to it and told users of <nlohmann/json.hpp> to include
json_fwd.hpp as well (or instead, which breaks compilation).
Fixes#5812
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Export the remaining client-facing headers from json.hpp and check clients with IWYU
Besides json_fwd.hpp, include analyzers also told clients of
<nlohmann/json.hpp> to include detail/json_ref.hpp (brace
initialization), json_literals.hpp (_json, _json_pointer),
adl_serializer.hpp, detail/macro_scope.hpp (NLOHMANN_DEFINE_*) and
detail/abi_macros.hpp. Mark those includes as exported, too.
ci_single_binaries only checked each header on its own, so it could not
notice this. It now also runs IWYU and clang-tidy's misc-include-cleaner
on small client programs in tests/iwyu_consumer, which fail on develop.
With the exports, json_literals.hpp no longer gets the json_fwd.hpp
suggestion that kept it out of the IWYU gate, so it is gated again.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Pretty-printed JSON indents every line with a run of spaces, which
skip_whitespace() read one character at a time. For contiguous input
(bulk_scan) the lexer now skips the spaces after a newline 8 bytes at a
time directly in the input buffer, including the leading spaces of the
last, partial word. This bypasses get(), which is safe because
bulk_scan implies the lazy position from the previous commit: nothing
is counted per character.
Compared to lazy positions alone, pretty-printed input (twitter,
citm_catalog with indent 2 and 4) parses up to 13% faster and accept()
gets up to 30% faster (GCC 13 and clang 18 on x86-64, Apple clang 21 on
arm64). Minified input is unchanged within noise.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Derive the lexer position lazily for seekable input
The lexer counted the read position on every character: two increments
and a newline check per get(), all on members kept in memory. For
seekable input (pointers, strings, other random-access byte ranges) the
position can be derived instead: chars_read_total is the adapter's read
count (including reads at EOF) minus a pending unget, and line and
column are recomputed from the consumed input only when get_position()
is called, i.e. on parse errors. JSON_DIAGNOSTIC_POSITIONS uses the new
O(1) get_chars_read_total(). Stream, file, wide-string and user-defined
adapters keep the eager counters.
Parsing pretty-printed input gets 8-12% faster and accept() 12-27%
(canada, citm_catalog, twitter; clang 21 arm64 and GCC 13 x86-64).
Error messages, byte offsets, SAX positions and diagnostic positions are
unchanged (differential comparison against develop on 548k inputs).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Move the std::vector<std::byte> binary type tests out of unit-regression3
The #4804/#4209 tests instantiated a second basic_json specialization
(with_binary_t<std::vector<std::byte>>), including its parser and CBOR
reader. With clang 18, the C++20 object of unit-regression3.cpp had
32656 COFF sections on develop; the lazy position helpers pushed it to
32975, past what the MinGW linker of the Windows clang jobs handles
("relocation truncated to fit: IMAGE_REL_AMD64_REL32 against .rdata").
unit-custom-binary-type.cpp already instantiates the same type, so the
tests move there; unit-regression3.cpp drops to 27616 sections.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Report the parse errors of the position test via SAX
The test caught json::parse_error, so it aborted in the
ci_test_noexceptions build, where JSON_NOEXCEPTION turns the throw into
std::abort(). sax_parse() hands the error position and the exception
object to parse_error() without throwing, so the test now records them
there and runs in every configuration.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* support JSON Merge Patch (RFC 7396) diff creation
The JSON merge patch document format describes the set of modifications
to a resource's content, that more closely mimics the syntax of the
resource being modified.
However, and in contrast to JSON Patch (RFC 6902), a JSON Merge Patch
cannot express certain modifications, e.g., changing an array element
at a specific index, or setting a specific object value to null.
The null value in a JSON Merge Patch is used to remove the key from the
object.
The diff algorithm is not part of the RFC 7396, but it was tested
against all examples provided, plus additional cases on how null values
are handled.
Signed-off-by: Luís Murta <luis@murta.dev>
* improve algorithm efficiency
Signed-off-by: Luís Murta <luis@murta.dev>
* fix inconsistent nulls and empty arrays
Also add CHECKs for the patch, documentation and run `make amalgamate`.
Signed-off-by: Luís Murta <luis@murta.dev>
* documentation cleanups
Signed-off-by: Luís Murta <luis@murta.dev>
* add an iterative fallback to merge_diff
Bound the recursion depth like diff() and merge_patch(), and test values
nested past the bound.
Signed-off-by: Luís Murta <luis@murta.dev>
* address the remaining review comments
Signed-off-by: Luís Murta <luis@murta.dev>
* move multi-line raw strings out of CHECK macros
GCC 4.8/5, ICPC and NVHPC mis-tokenize multi-line raw string literals
inside macro arguments, breaking the build or the parsed JSON.
Signed-off-by: Luís Murta <luis@murta.dev>
---------
Signed-off-by: Luís Murta <luis@murta.dev>
#5795 and #5806 changed binary_writer.hpp but were merged without
updated amalgamated files, so the check_amalgamation run on develop
(c59ad9525) fails. This is the output of 'make amalgamate'; no source
files change.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
#5806 added the test while #5802 made to_bon8 reject discarded values; both
merged, so the test failed on develop.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Stop the BON8 writer overflowing the stack on deep values
to_bon8() recursed once per nesting level, so a value the iterative
BON8 reader accepts (e.g. ~24k nested one-element arrays) crashed on
the way back out. #5781 bounded the CBOR, MessagePack, and UBJSON/BJData
writers, but BON8 was merged before it and was not covered.
Apply the same scheme: recurse for the first recursion_depth_limit()
levels, then finish the value with write_bon8_iterative, which keeps
the open containers on a heap stack (reusing binary_container_frame)
and writes the 0xFE closer when it leaves a container with more than
four elements. The output is byte-for-byte unchanged.
Fixes https://issues.oss-fuzz.com/issues/572238015
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Merge the array and object branches of the BON8 iterative writer
write_bon8_iterative and write_bon8_value_or_push handled arrays and
objects in separate branches that repeated the end-of-container check,
the 0xFE closer and the marker computation. Share those parts and branch
only where arrays and objects really differ. The output is unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Hz7VJi1FTKr6gpseLErbbS
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Restore v3.12.0 support for custom object key types
Custom object_t types whose key_type is not string_t compiled with
v3.12.0 for several APIs that unreleased changes broke:
- to_bson failed for every custom key type (#5553 kept a const string_t*
to the key); the nested entry's header is now written where the entry
is found.
- Copying deep values (and parse, merge_patch, update, insert) required
operator== on keys (#5389); keys without one are now paired via find().
- to_cbor/to_msgpack required an implicit conversion to string_t (#5746,
#5328); keys without one go through a temporary basic_json again.
- at() required a conversion to string_t for its error message (#5727);
other keys are passed to concat() unchanged again.
The new unit-custom-object-key-type.cpp covers five key types with
different capabilities.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Use the with_object_t alias for the custom object key test types
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Avoid floating-point equality in custom key type test
GCC with -Werror=float-equal rejects comparing the double value with ==.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Write the head of nested BSON elements in one helper
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Suppress bugprone-return-const-ref-from-parameter in key_for_message
The reference is only passed to concat() within the full-expression that
holds the key, like the similar helpers in binary_writer.hpp.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Make the value of the custom test key types private
clang-tidy (cppcoreguidelines-non-private-member-variables-in-classes)
rejects the protected member; the derived key types use a protected
accessor instead.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Pass keys with data() and size() unchanged into the at() miss message
key_for_message() converted every key that string_t can be constructed
from, so a miss on a string_t or string_view key copied it before
concat() copied it again. Keys that concat() can append through data()
and size() are now passed through; only other keys (string literals,
key types that just convert to string_t) are converted.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
A code search of client code found many hand-written versions of
functionality the library already has: recursive object merges
(update(j, true) since 3.10.5), dotted-path getters (value() and
contains() with a JSON Pointer), get_or helpers (value() throws for a
member that is present but null), and adl_serializer specializations
for std::optional (supported since 3.12.0).
- modifying_values.md: describe both modes of update(), add a
defaults + user settings recipe, and stop recommending Merge Patch
for recursive merges.
- merge_patch.md: note that null deletes keys, so a merge patch is not
a general deep merge; link update().
- default_value.md: add "Nested values" (value/contains with a JSON
Pointer, building a pointer from a dotted path with operator/=) and
a warning that null and mistyped members are not missing.
- value.md: note that a null member is converted, not replaced by the
default.
- json_pointer.md: link value/contains/at with JSON Pointers.
- arbitrary_types.md: note that std::optional needs no serializer.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
write_bon8_value silently skipped discarded values, but the array/object
count marker still counted them, so [1, discarded] produced 82 91: a marker
announcing two elements followed by one. Throw type_error.321 like the other
binary writers do, at any nesting level.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Clean up README, contribution guide, and repository metadata
- REUSE.toml: fix the Hedley path and SPDX id (CC0-1.0), mark the docset
icons as the public-domain JSON logo
- CITATION.cff: v3.12.0 was released on 2025-04-11
- FILES.md: list all workflows, describe the Meson option defaults
- README: ctest -LE, table of contents, typos, stale Android/MinGW advice,
moved links, merge duplicate Thanks entries, list the analysis tools
used in CI
- CONTRIBUTING: iterative parser, json_literals.hpp is generated, links
- .gitignore: backups and release outputs; .gitattributes: mark
generated files
- labeler: label other build systems and .github documentation
- MODULE.bazel: add the module version
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Clean up CI, CMake, Makefile, and Bazel files
- CMakeLists.txt: avoid VERSION_GREATER_EQUAL, which CMake < 3.7 lacks
- ci.cmake: test JSON_DisableTupleReferenceConversion in ci_cmake_flags,
remove unused variables and unreachable per-compiler targets, look up
Clang tools consistently, forward CMAKE_CXX_FLAGS to ci_module_cpp20,
format json_literals.hpp and remove backups in ci_test_amalgamation
- BUILD.bazel: add json_literals.hpp to the single-header target
- workflows: format json_literals.hpp before copying it, drop the obsolete
natvis --version plumbing, name natvis and macro_builder in failure
messages, drop the duplicate amalgamation job, install Valgrind only
where needed, republish docs on version bumps, fix stale names
- Makefile: complete .PHONY and help, check-amalgamation always restores
the checked-in files, natvis uses its own venv, macro_builder_check
installs astyle, clean removes the fuzzer binaries
- remove tools/amalgamate/config_json_view.json (json_view.hpp is not on
develop yet)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix inaccuracies in the documentation
- version history: json_base_class_t (3.11.3), JSON_HAS_CPP_11 (3.10.0),
JSON_HAS_RANGES exclusions, define-type macros, ABI tags
- releases: 3.12.0 raised the minimum CMake version
- from_*: the (ptr, len) overloads are deleted, not removed, in 4.0.0
- contains/count/find: document the deleted integral overloads
- add JSON_HAS_RANGE_VIEW_CONVERSION and list the JSON_HAS_* macros in
the macro overview
- mention BON8 and error_handler where binary formats are listed
- broken links, outdated URLs, warning count, Hunter v0.26.12
- copy_markdown_source hook: expand snippets in the Markdown copies
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Pass /EHsc to the Windows C++20 module build
ci_module_cpp20 now forwards CMAKE_CXX_FLAGS to the module build. The
Windows workflow sets CMAKE_CXX_FLAGS, which replaces CMake's MSVC
defaults including /EHsc, so <chrono> failed with C4530 under /WX.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Report one-character non-numeric array indices like longer ones
A JSON pointer reference token that is not a number but has only one
character (e.g. "/a/x") was reported as out_of_range.404 ("unresolved
reference token"), because the "is not a number" check only ran for
tokens longer than one character; "/a/xy" got parse_error.109. Both now
throw parse_error.109. "-" and the empty token are still reported as
out_of_range.404. As a consequence, value(json_pointer, default) on an
array now throws for "/x" as it already did for "/xy".
Also document why ordered_map::erase's destroy/placement-new loop on
pair<const Key, T> is kept despite [basic.life]/8 before C++20.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Document the one-character array index change
Add 3.13.0 version-history entries to at, operator[], value, patch,
patch_inplace, and unflatten, and describe in exceptions.md which array
indices throw parse_error.109 and which out_of_range.404.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Use and extend the detail helpers to remove duplicated code
Library:
- binary_reader: format bytes with hex_byte() instead of snprintf
- add throw_type_must_be() for the 20 copies of type_error.302
- binary_reader: add last_byte_error()/unexpected_byte() for the 32
"parse error at the last read byte" sites (replaces bon8_error)
- json_sax: add check_container_size() for out_of_range.408 and
diagnostic_positions::set_container_start/_end()
- json_pointer: add throw_no_parent() (405) and throw_unresolved() (404)
Tests:
- unit-class_parser uses the shared utils::SaxCountdown
- move SaxEventLogger (and its ExitAfter* variants) from unit-class_parser
and unit-deserialization into the new tests/src/test_sax.hpp
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Factor out more repeated error paths and test boilerplate
Library:
- add throw_cannot_use_with() for the 34 copies of type_error.304-312
"cannot use X with Y"
- iter_impl: add throw_cannot_get_value() (invalid_iterator.214)
- parser: add syntax_error() for the 12 parse_error.101 sites
- ordered_map: share the four at() bodies via at_impl()
- json_sax_dom_callback_parser: add pop_container() for end_object()
and end_array()
- binary_reader: build the two UBJSON/BJData length-type messages with
concat() and last_byte_error()
Tests:
- move same_value(), the NDEBUG guard, and step 0 (parse without
exceptions) of the seven fuzzer drivers into tests/src/fuzzer_common.hpp
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Name the exception ids and address review comments
Add detail::exception_id, a scoped enum with one named enumerator per
documented exception id, and use it for every id in the library. The
create() functions get an overload for it; the int overloads stay for
user code.
Following the review of #5783: add binary_reader::invalid_byte() and
length_type_error(), basic_json::throw_subscript_wrong_type(), move the
fuzzer includes and the using-declaration into fuzzer_common.hpp, and
rename test_sax.hpp to sax_event_loggers.hpp.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Silence -Wweak-vtables for the SAX event loggers
The loggers moved from anonymous namespaces in the test files into
sax_event_loggers.hpp, so clang now warns that their vtables are
emitted in every translation unit.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Silence MSVC 2015 C4100 in parser::syntax_error for static SAX::parse_error
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Inline invalid_byte() into its call sites
The wrapper only fixed the message string of unexpected_byte(), which is
the same kind of per-argument helper that was declined for
throw_type_must_be(). Call unexpected_byte("invalid byte", ...) directly.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Codacy flagged two Bandit findings in the docset generator added in
#5799: B310 (urlopen with an unchecked scheme) and B506 (yaml.load).
download() now rejects anything but http(s) URLs before opening them,
and the yaml.load call is marked, since its Loader derives from
yaml.SafeLoader. The SHA-1 used to name downloaded files is marked as
not used for security.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Replace the hand-maintained docs/docset/docSet.sql and the bash/sed
Makefile recipe with docs/docset/generate_docset.py:
- The search index is generated from the mkdocs.yml nav and each page's
H1 and declaration. Pages documenting several entities (e.g.
JSON_HAS_CPP_11..26) get one entry per name; all non-API pages become
guides. New API pages no longer need a manual entry.
- Page titles are set from the index names.
- The docset is self-contained: the mermaid loader no longer points to
https://json.nlohmann.me/assets/..., the repository widget no longer
queries api.github.com, remaining remote images are downloaded, and
the build fails if any remote resource load remains.
- The CSS that hides the site navigation now uses Material's classes;
the element selectors lost against them, so the header was shown.
check_structure.py's docset check is replaced by running the generator
in `make style_check`.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The custom object key tests from #5328 instantiate a second basic_json
specialization in unit-cbor.cpp and unit-msgpack.cpp. This pushed
unit-msgpack.cpp.obj past 65535 sections in the clang (MinGW) jobs of
the Windows workflow, and linking test-msgpack_cpp11 fails with
"relocation truncated to fit: IMAGE_REL_AMD64_REL32 against `.rdata'".
The GNU linker keeps the section an associative COMDAT section belongs
to in 16 bits (x_associated in include/coff/internal.h), although big
object files store 32 bits. In larger objects it therefore ties the
jump tables of inline functions to the wrong function and discards them
together with that function's duplicate.
Move the four tests unchanged into unit-custom-object-key-type.cpp and
document the limit in windows.yml.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Unflatten in time and memory linear in the pointer depth
#5443 made unflatten() decide between arrays and objects independently
of the iteration order by collecting the pointer prefixes that have a
reference token 0 below them in a std::set<std::vector<string_t>>.
Every such prefix was stored as a copy of all its reference tokens, and
get_and_create() compared whole prefix vectors at every step, so
unflattening a pointer of depth d took time and memory quadratic in d:
a 10,000-level array pointer took 18 s and 1.3 GB, a 100,000-level one
did not finish.
The prefixes are now numbered nodes of a tree, so each is stored once
and get_and_create() follows the tree token by token. The result is
unchanged, including its independence of the iteration order.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Initialize prefix_tree members to satisfy -Weffc++
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Move prefix_tree setup and child insertion into member functions
The constructor now creates the root node, add_child() inserts a
reference token below a prefix and returns the child's number, and
find_child() looks one up for get_and_create().
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Flatten deeply nested values without recursing per nesting level
json_pointer::flatten() called itself once per nesting level, so
flatten() on a value nested deeply enough exhausted the call stack.
#5547 and #5548 fixed merge_patch() and diff() from #5393, but flatten()
was left out.
flatten() now walks the value with an explicit stack and keeps the path
in one buffer that grows and shrinks with it. It has a single code path
and no depth limit: the old version built a new path string per child,
so the iterative one is no slower on shallow values and much faster on
deep ones. The output, including the order of an ordered_json result,
is unchanged.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Construct flatten frames in place
Give the frame a constructor so both call sites can use emplace_back, as
suggested in the review; index starts at 0 for every frame.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Keep converted object keys alive while writing UBJSON and BJData
Since #5746, write_ubjson and write_ubjson_iterative pass each object key
to sanitize_utf8_for_write and keep the returned reference. When
object_t::key_type is not string_t but converts to it, the argument is a
temporary that is destroyed at the end of the statement, and the
function returns a reference to it in every case but a sanitized copy,
so the key bytes are read from a dead object (AddressSanitizer:
stack-use-after-scope). Default json and ordered_json are unaffected.
Bind the key to a named object_key_string_t first: a reference when
key_type is string_t, so no copy is added there, and a converted copy
otherwise. A deleted overload of sanitize_utf8_for_write for anything
other than string_t turns a recurrence into a compile error.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Suppress -Wunused-member-function for the converting_key test type
converting_key::data() is only called when JSON_DIAGNOSTICS is enabled.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix clang-tidy findings in the UBJSON/BJData converted-key fix
Suppress hicpp/modernize-use-equals-delete on the deleted
sanitize_utf8_for_write overload: it guards a private helper and must stay
private. Replace the C-style array in the new test with std::array.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
#5787 missed the instantiations spelled as `basic_json <` (astyle's
formatting) or ending in the CustomBaseClass argument. Convert them,
including the binary_t.md example pointed out in review.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* avoid allocating temporary basic_json for CBOR and MessagePack object keys
Signed-off-by: alexprabhat99 <alexpbara@gmail.com>
* add size() to the custom object key test type
UBJSON and BJData access object keys through size() and c_str()
directly, so the key type now provides both and the comment says why.
Signed-off-by: alexprabhat99 <alexpbara@gmail.com>
* address review: drop key size()/c_str(), test keys below the depth limit
Nothing in the library calls size() or c_str() on an object key, so the
test key type only keeps data(), which JSON_DIAGNOSTICS needs.
The CBOR and MessagePack custom key tests now also nest objects deeper
than detail::recursion_depth_limit(), so keys written by
write_cbor_iterative and write_msgpack_iterative are covered as well.
Signed-off-by: alexprabhat99 <alexpbara@gmail.com>
---------
Signed-off-by: alexprabhat99 <alexpbara@gmail.com>
* Address review comments on #5775 and #5779
Allow new defaulted parameters and new default arguments in the API
stability rules, mention the macro opt-in, and drop the redundant
recompile advice. Describe test-diagnostics-optimized as the regression
test for the fixed#5742.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Document what counts as a breaking change in the API stability rules
Spell out the 3.x compatibility rules in the roadmap: new defaulted
parameters, new default arguments, noexcept/constexpr, template
parameters, parse and dump results, accepted input, key iteration order,
iterator invalidation, implicit conversions, to_json/from_json lookup,
json_sax, value_t enumerators, and documented macros, CMake options and
headers. Also list std::hash values as not part of the public API, and
link the macro overview from the section.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
contains(json_pointer) is marked JSON_HEDLEY_WARN_UNUSED_RESULT since #5477,
and parse() has been for longer. The contains example ignored the result in
two try blocks waiting for a parse_error that contains() never throws, so
they printed nothing; print the result for those pointers instead.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Use the with_*_t aliases in tests, examples, and docs
Replace spelled-out basic_json<...> instantiations that only change one
or two template parameters with nlohmann::json::with_*_t (or
ordered_json::with_*_t when the object type is ordered_map). Types that
change all three number types chain with_integers_t and with_float_t.
The raw basic_json<...> spelling stays where the template parameter
list itself is the subject: the alias tests in unit-udt.cpp, explicit
instantiations, and the ordered_json/compile-time docs.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix unit-large_json for clang and JSON_DIAGNOSTICS
Two test problems from #5781 broke CI on develop: CAPTURE(depth); trips
clang's -Wextra-semi-stmt, and the type_error.321 messages did not
account for the diagnostics path prefix.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Use static_cast in unit-hash for clang-tidy
#5772 added functional casts that clang-tidy reports as C-style casts
(google-readability-casting). Also append a char instead of a
one-character string in unit-large_json.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Declare the expected message prefix const in unit-large_json
Without JSON_DIAGNOSTICS the prefix was never modified, which
clang-tidy reports (misc-const-correctness).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Install CMake package config files with Meson, and add Meson options
Squashed onto develop from:
- Install CMake package config files with Meson
- meson: Indent code inside an if block
- meson: set a minimum Meson version
- meson: use `override_dependency()` to set dependencies
- meson: use `install_subdir` for headers
- meson: set the C++ standard to C++11
- meson: handle single header and multiheader the same way CMake does
- meson: add support for the GlobalUDLs option
- meson: add support for the ImplictConversions option
- Add meson information to FILES.md
- Complete the Meson options and match CMake's compile definitions
- Add the JSON_* definitions to the CMake pkg-config file
- Document the Meson options and check them in CI
- Fix the Meson CMake target for includedir or datadir outside the prefix
- Avoid //include in Meson-generated CMake target for prefix /
- Add DisableTupleReferenceConversion to Meson and the CMake pkg-config file
- Check in CI that Meson and pkg-config offer the CMake options
- Remove accidentally committed Python bytecode
Co-authored-by: Dylan Baker <dylan@pnwbakers.com>
Signed-off-by: Dylan Baker <dylan@pnwbakers.com>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Add the StrictBinaryUTF8 and DeleteDeprecatedFunctions Meson options
JSON_StrictBinaryUTF8 (#5741) and JSON_DeleteDeprecatedFunctions (#5755)
arrived with develop but were missing from the Meson build and the
pkg-config file, so check_build_options failed. Both Meson options
default to false and add JSON_STRICT_BINARY_UTF8=1 and
JSON_DELETE_DEPRECATED_FUNCTIONS=1 to the dependency, the pkg-config
file, and the generated CMake target; CMake's pkg-config file now
carries both definitions too. The ci_meson_install job sets and checks
them in its non-default install.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Dylan Baker <dylan@pnwbakers.com>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Dylan Baker <dylan@pnwbakers.com>
* Make std::hash<basic_json> consistent with operator== for numbers
operator== converts between number_integer, number_unsigned, and
number_float before comparing, so json(0), json(0U), and json(0.0)
all compare equal. hash() folded the specific value_t into the
result for each of the three numeric cases, giving each a distinct
hash and breaking the standard Hash requirement that a == b implies
hash(a) == hash(b). A std::unordered_set could therefore hold all
three as separate elements even though they compare equal.
hash() now treats all three numeric variants the same way: it
converts the value to number_float_t and combines it with a single
shared type tag, so any two numbers operator== considers equal hash
identically regardless of which internal type actually holds them.
Updated the accompanying test to check this consistency directly
(including via an actual unordered_set) instead of asserting that 0,
0U, and 0.0 hash differently, since that assumption was the bug.
Also corrected the function's own doc comment and the std::hash API
docs, which described the old behavior as intended.
Fixes#5400
Signed-off-by: Afonso Januário <afonso-januario@hotmail.com>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Remove now-unused number_integer_t/number_unsigned_t typedefs in hash()
Merging the three numeric branches into one that only reads
number_float_t left these two aliases unused, which several CI
configurations treat as a build error under -Wunused-local-typedefs.
Signed-off-by: Afonso Januário <afonso-januario@hotmail.com>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Mark the unordered_set in the hash regression test const
clang-tidy's misc-const-correctness check flagged it: the set is
never mutated after construction, only read via size().
Signed-off-by: Afonso Januário <afonso-januario@hotmail.com>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Normalize -0.0 in number hashes and test range ends
operator== compares numbers exactly since #5459, so equal numbers
share one value and convert to the same number_float_t. Update the
comment accordingly, map -0.0 to 0.0 before hashing (std::hash need
not do that), and test -0.0 and the ends of the integer ranges. Show
hash(0.0) in the docs example and note the change in the version
history.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Clarify hash documentation after review
- Say "may hash differently" for null, false, and numbers, since a
collision across types is possible.
- Name the storage types (signed integer, unsigned integer,
floating-point number) instead of example literals.
- Explain that the hash survives converting an integer to
number_float_t but not the lossy conversion back, and that unequal
numbers may share a hash.
- State that the example hash values are illustrative only and vary by
platform, compiler, compiler version, and library version.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Afonso Januário <afonso-januario@hotmail.com>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Afonso Januário <afonso-januario@hotmail.com>
Niels Lohmann
dependabot[bot]
satelliteprogrammer
Bas van Dijkandbvandijk
Alex Prabhat Bara
Suyog Verma
