This should have no effect for header only libraries as mentioned.
It was previously removed in e509007d but then accidentally added
again in 26cfec34.
Signed-off-by: Jeremy Nimmer <jeremy.nimmer@tri.global>
* Keep JSON_DIAGNOSTICS parent pointers of ordered_json members after erase() and update()
ordered_json stores its members in a vector, and two operations moved
members without restoring their parent pointers afterwards:
- ordered_map::erase() re-constructs every member after the erased one in
place. The basic_json move constructor leaves m_parent at nullptr, and
none of the object branches of basic_json::erase() (by key, iterator, or
iterator range) called set_parents(). This also affected merge_patch()
with a null member and patch() with a remove operation.
- update() only set the parent pointer of the inserted member. Adding a key
can reallocate the vector, which copies all other members and leaves
their m_parent at nullptr. The set_parents() call added for #4813 only
repaired this for the nested object of a merge, not for the target.
The next assert_invariant() on such an object (for instance, when copying
it) aborted, and diagnostic messages lost the path prefix above the moved
member. std::map-based json was not affected, because its nodes do not
move.
Erasing from an ordered_map object now calls set_parents(), and update()
uses set_parent(), which already refreshes all members for vector-based
objects. This makes the #4813 workaround redundant.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Account for JSON_DIAGNOSTIC_POSITIONS in the ordered_json parent-pointer test
The merge_patch() case parses its input, so with JSON_DIAGNOSTIC_POSITIONS
the exception message also carries the byte range of the parsed value.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Silence clang-tidy for the intentional copy in the ordered_json parent-pointer test
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Keep parent pointers when update() merges past its descent bound
The iterative path of update() only set the parent pointer of the member
it inserted, like the recursive one did before. It now uses set_parent()
too, so ordered_json members that move when a nested object grows keep
their parents, and the set_parents() calls that patched this up after
each nested merge are gone.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Check the fuzzers' UBJSON/BJData round-trip invariants in the unit tests
The strongest correctness checks for the UBJSON and BJData writers lived
only in the OSS-Fuzz drivers: anything from_ubjson()/from_bjdata()
returns must serialize with every option combination, parse back, and
re-serialize stably. Those checks only run at OSS-Fuzz, so regressions
surfaced days later as external reports - the same BJData assert pair
was reported five times over three years, and #5494's harness change
was followed by OSS-Fuzz 563659413 within a day.
Add "UBJSON round-trip invariants" and "BJData round-trip invariants"
test cases that run the drivers' checks on a fixed, deterministic corpus
(tests/src/round_trip_corpus.hpp): integer and float boundaries,
non-finite numbers, strings, binary values, optimized containers, deep
nesting, the JData annotated-array matrix, and seeded random containers.
They also check two properties the drivers do not: the first round trip
preserves the value, and re-serializing reproduces the exact bytes. For
BJData both exclude values containing a binary value, which is read back
as an array of integers unless it was written as a Draft 3 optimized
binary array; this carve-out is now documented in bjdata.md. Run against
the headers before #5542, the BJData test fails, including on the shape
from OSS-Fuzz 563659413.
Also document how OSS-Fuzz reports are handled (reference them as
"OSS-Fuzz: <id>", turn the reproducer into a unit test, keep drivers and
unit tests in sync) in tests/fuzzing.md, and link it from the PR
template and the quality assurance page.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Add the OSS-Fuzz reproducers for 474400817 and 474480402 as unit tests
Following the convention added to tests/fuzzing.md, the reproducers of
the two BJData fuzzer asserts tracked since January are now unit tests:
- 474400817 (assert(false)): an empty object _ArraySize_ was written as
the ND-array header length, which from_bjdata() could not read back.
Fixed by #5455.
- 474480402 (to_bjdata(j2, false, false) == vec2): a one-byte Draft 3
binary array is written in Draft 2 mode as a uint8 array and then
re-serialized with the int8 marker. This is the documented exception to
byte stability, not a library bug; OSS-Fuzz closed it after #5494
relaxed the harness to value stability. The test pins the exact bytes
so the exception stays deliberate.
The 563659413 reproducer is already a unit test (#5542). A comment also
ties the existing UBJSON excessive-count test to the timeout OSS-Fuzz
reported for that shape (testcase 6347769435193344).
OSS-Fuzz: 474400817
OSS-Fuzz: 474480402
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix GCC -Weffc++ and -Wuseless-cast warnings in the round-trip corpus
Initialize the atoms in the member initialization list, and drop the cast of
the generator's result, which already is std::size_t on 64-bit Linux.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Describe the threat model, the trust boundaries, the secure-design
argument, and how common weaknesses are countered, with links to the
quality assurance page as evidence.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Describe what the project will and will not do over the next year,
and point to issue #3453 for the open question of a 4.0 release.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Complete the architecture documentation page
Replace the placeholder bullets and TODOs with a description of the
component pipeline (with a diagram), the source layout, the template
parameters, the value storage (now in struct data), the input and
output adapters, and the SAX interface.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Link sources and basic_json, document full input adapter interface
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Align the default column of the template parameter table
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The windows-11-arm runner image moved to windows-11-vs2026-arm64, which no
longer ships Visual Studio 2022, so the msvc-arm64 job failed at configure
time. Use the "Visual Studio 18 2026" generator like the msvc2026 job.
clang-tidy's modernize-raw-string-literal check flagged two string literals
in the nesting tests added by #5546 and #5547.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Merge deeply nested objects without recursing per nesting level
merge_patch() and update(j, true) merged a nested object by calling
themselves on it, once per nesting level. A value nested deeply enough -
50,000 levels of objects on an 8 MiB stack - exhausted the call stack
and terminated the process, although parse() accepts such values without
complaint.
Bound the descent the same way dump() does. The recursion now carries
the nesting level, and once merge_depth_limit() (128) levels have been
entered, update_members_iteratively() and merge_patch_iteratively()
finish the merge on an explicit stack. They still merge a nested object
completely before the next member, and in the same order, so the results,
including the parents JSON_DIAGNOSTICS reports paths from, are unchanged.
Values nested less deeply than the bound run the same code as before, so
the common case does not pay for the stack: merging only on it cost
10-14% in a first version.
The public signatures are unchanged. The recursive worker behind
merge_patch() has its own name rather than being a private overload, so
that &basic_json::merge_patch stays unambiguous.
Tests check every depth up to 300 against recursive reference
implementations of both operations, check the diagnostic paths past the
bound, and merge objects nested 100,000 levels deep.
Fixes#5545 for update(j, true), and #5393 for merge_patch().
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Use the shared recursion limit in update() and merge_patch()
merge_depth_limit() is gone in favor of detail::recursion_depth_limit().
The two identical function-local frame structs become one member struct,
merge_frame, with a constructor, so both loops emplace_back() their
frames. merge_patch_iteratively() copies the frame it works on out of the
stack and changes it only through stack.back().
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Build the update()/merge_patch() diagnostics test values instead of parsing them
Parsed values carry byte positions under JSON_DIAGNOSTIC_POSITIONS, which
the expected messages do not include.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Hash deeply nested values without recursing per nesting level
std::hash<basic_json> hashed an array or object by hashing each element,
which called detail::hash again once per nesting level. A value nested
deeply enough - 50,000 levels of objects on an 8 MiB stack - exhausted
the call stack and terminated the process. parse() accepts such values
without complaint, since the parser is iterative, and a parsed value is
hashed wherever it is used as a key in an unordered container.
Bound the descent the same way dump() does: detail::hash takes the
nesting level, and once hash_depth_limit() (128) levels have been entered,
hash_iteratively() hashes what is left on an explicit stack. It combines
the seeds in exactly the same order, so hash values are unchanged. A value
nested less deeply than the bound is hashed by the same code as before,
without allocating, and is as fast as before.
Tests check that every depth up to twice the bound hashes exactly like
the recursive definition of the hash, and that values nested 100,000
levels deep hash without crashing.
Fixes#5545 for std::hash.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Declare hash_frame's constructor noexcept
GCC's -Wnoexcept (an error in CI) flags the emplace_back() into the
hash stack under C++26: the constructor cannot throw, since cbegin() is
noexcept, but it did not say so. dump_frame's constructor is noexcept
for the same reason.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Share one recursion depth limit, and copy the hash frame out of the stack
dump() and hash() each defined their own limit on how many nesting levels
they recurse into, and the operations still to come would have added more,
free to diverge over time. They now all use detail::recursion_depth_limit(),
in a header of its own; serializer::dump_depth_limit() and
hash_depth_limit() are gone.
hash_iteratively() now copies the frame it works on out of the stack and
changes the frame only through stack.back(), so nothing can refer into
the stack after entering an element has grown it.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Parenthesize multiplications in the hash test for clang-tidy
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Add missing headers to BUILD.bazel and make its generator reproduce it
The "json" cc_library did not list three headers that the library
includes:
- detail/meta/logic.hpp (added in #5016, included by from_json.hpp)
- detail/input/number_parse.hpp (added in #5283, included by lexer.hpp)
- detail/input/string_scan.hpp (added in #5283, included by lexer.hpp
and serializer.hpp)
Bazel's sandbox only exposes declared headers, so any target depending
on @nlohmann_json//:json and including <nlohmann/json.hpp> failed with
"'nlohmann/detail/meta/logic.hpp' file not found".
The file could not simply be regenerated, because the generator behind
"make BUILD.bazel" was stale: it wrote only the "json" cc_library and
dropped the load() statements, the license block, and the
"singleheader-json" target that were added by hand in #4584. The
generator now emits the complete file, so its output differs from the
previous BUILD.bazel only by the three headers. It also resolves the
glob against the project root instead of the working directory and
sorts the list explicitly.
"make BUILD.bazel" is now phony: in a fresh checkout, BUILD.bazel is
not older than the headers, so make considered it up to date, and a
removed header would never trigger a rebuild. "make check-amalgamation"
also checks that BUILD.bazel is up to date.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Check in CI that BUILD.bazel is up to date
The "Check amalgamation" workflow now also regenerates BUILD.bazel, so a
pull request that adds, renames, or removes a header without updating
the Bazel header list fails, and the attached amalgamation.patch
contains the fix. The failure comment and the contribution guidelines
mention the new check, and the comment now links to the existing
"Amalgamate the source code" section instead of the "Files to change"
anchor that was removed in #4560.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Test JSON_BRACE_INIT_COPY_SEMANTICS for real, and fix one-element tuples under it
The opt-in JSON_BRACE_INIT_COPY_SEMANTICS was never exercised by CI:
- Its only test, in unit-regression3.cpp, was guarded by
`#if defined(JSON_BRACE_INIT_COPY_SEMANTICS)` after the #include. The
header #undefs the macro unconditionally in macro_unscope.hpp, so the
guard was always false and the test compiled to nothing, whatever -D
flag was passed.
- The ci_test_brace_init_copy_semantics target that passes the flag was
not named by any workflow.
Move the test into its own translation unit that defines the macro before
including the header, as unit-diagnostics.cpp does for JSON_DIAGNOSTICS.
It now runs in every CI job and for every standard. Remove the unused
target: it ran the whole suite with the macro, and that suite deliberately
relies on default brace-init semantics in about 90 places
(e.g. `json({1})` meaning `[1]`), so it could never pass.
Running the whole suite with the macro did find one library bug:
to_json for std::tuple builds `j = { std::get<Idx>(t)... }`, so with copy
semantics a one-element tuple became its element. `json(std::tuple<int>{5})`
was `5` instead of `[5]`, and `get<std::tuple<int>>()` threw type_error.302
on the result. Under the macro, a one-element tuple now builds exactly what
the default deduction builds. Without the macro nothing changes.
The new tests also pin that the library's other conversions produce the
same values with and without the macro. The macro page now says that the
macro affects every single-element list (`json j = {1}` is `1`), and that
all translation units must agree on it, since it has no ABI tag.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Make JSON_BRACE_INIT_COPY_SEMANTICS part of the ABI tag
The macro changes the body of the initializer-list constructor and adds a
to_json_tuple_impl overload, both with the same mangled names in either
mode, so mixing translation units silently picked one definition. Encode
it in the inline namespace as `_bics`, as JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON
does with `_ldvcmp`. The macro is new in the unreleased 3.13.0, so no
existing namespace name changes.
- Move the macro's default into abi_macros.hpp so json_fwd.hpp computes
the same namespace, and keep it defined under JSON_TEST_KEEP_MACROS.
- Check the tag in the ABI config tests and in the unit test.
- List `_bics` (and the missing `_dp`) in the namespace docs and in the
natvis generator; regenerate nlohmann_json.natvis.
- Replace the "define it consistently" warning with an ABI note.
Suggested by @gregmarr in the review of #5544.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix the cppcheck, clang-tidy and legacy-comparison CI failures
- to_json_tuple_impl() moved the element in both branches of a ternary;
only one runs, but cppcheck reported accessMoved. Use if/else.
- The ABI tag test looked for "json_abi_bics", which misses when another
tag comes first, as in json_abi_ldvcmp_bics; look for "_bics".
- readability-qualified-auto in the items() test.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
write_ubjson() built a std::vector of the eight markers BJData forbids as
the type of an optimized container - one heap allocation plus a linear
search for every array and object it wrote with use_type, even for plain
UBJSON output, where the list isn't consulted. The list was also spelled
out twice. A constexpr helper, is_bjdata_excluded_type_marker(), replaces
both.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
- "aspect: binary formats" for changes to the binary reader or writer,
their tests, fuzzers and docs, or with a binary format in the title;
- "python" for Python sources and pip requirements files, matching the
label Dependabot sets on its pip updates, so it is never removed there;
- "CI" also for changes to the Dependabot and labeler configurations.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The fuzzer drivers check their round trips with assert(), which NDEBUG
compiles away. The OSS-Fuzz build keeps assertions on today, but nothing
pins that: a build change that adds NDEBUG would silently turn every
round-trip check into a mere "does not crash" check. Each driver now
stops the build with an #error instead, and includes <cassert> itself
rather than relying on json.hpp.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The community-maintained clang-tidy check
modernize-nlohmann-json-explicit-conversions rewrites implicit
conversions into explicit get<T>() calls, which is exactly the
preparation the docs ask for ahead of implicit conversions being
switched off by default. Mention it on the JSON_USE_IMPLICIT_CONVERSIONS
page and in the migration guide, as promised in discussion #4610.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Without a bind address in serve_header.yml, the server listened on all
interfaces, so any machine on the network could fetch the header and
trigger make runs in the working trees. It now listens on localhost
unless configured otherwise; bind: null restores the old behavior.
The header was also sent with Access-Control-Allow-Origin: *, letting
any web page read it. CORS is only needed because Compiler Explorer
downloads #include <https://...> headers in the browser, so the header
now goes only to https://godbolt.org and https://compiler-explorer.com,
configurable with cors_origins.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
write_bjdata_ndarray() encoded a JData-annotated object as a BJData
ND-array whenever its dimensions' product matched _ArrayData_.size(),
which lost information in two ways:
- _ArrayData_ was never required to be an array. null has size 0, any
other scalar has size 1, and iterating an object visits its values, so
e.g. {"_ArraySize_":[1],"_ArrayData_":5} was written as the array [5],
and an object _ArrayData_ came back as an array.
- The reader only restores an annotated object from an ND-array with at
least two non-zero dimensions that is not a 1xN row vector; an empty,
1-D, row-vector, or zero-sized shape is read back as a plain array. The
writer nonetheless emitted ND-array headers for these shapes, so the
annotation was silently dropped.
OSS-Fuzz issue 563659413 hit this in parse_bjdata_fuzzer: an empty binary
_ArraySize_ is written as a plain object and read back as an empty array,
after which {"_ArrayType_":"int16","_ArraySize_":[],"_ArrayData_":null}
was encoded as the ND-array header "[$I#[]" and re-read as [], failing the
harness's value-stability check.
Such objects now fall back to a plain object encoding, which round-trips.
Genuine ND-arrays (two or more positive dimensions, not a 1xN row vector)
are encoded exactly as before. Existing fallback tests that used 1-D
shapes are moved to 2-D shapes so they keep exercising the check they
were written for, and the BJData documentation is updated.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Match ABI tag order in namespace tests to abi_macros.hpp
NLOHMANN_JSON_ABI_TAGS concatenates the tags as _diag, _ldvcmp, _dp,
but the default and noversion ABI tests expected _diag, _dp, _ldvcmp.
The tests therefore failed whenever both JSON_DIAGNOSTIC_POSITIONS and
JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON were enabled, a combination
CI never exercises. Reorder the expectations to match the header.
Also document the _dp tag in the namespace feature page, which listed
only _diag and _ldvcmp.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Test the ABI namespace with all ABI tags enabled
Build the default and noversion ABI config tests a second time with
JSON_DIAGNOSTICS, JSON_DIAGNOSTIC_POSITIONS and
JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON all set, so the expected tag
order is checked on every test run instead of depending on which CMake
options a CI job happens to enable.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Bound the descent of the copy constructor
basic_json's copy constructor copied objects and arrays by handing the
container to its own copy constructor, which copy-constructs every element
and so reaches this constructor again, once per nesting level. A value
nested deeply enough exhausted the call stack and terminated the process
with a segmentation fault - no exception, nothing the caller could catch.
Parsing such a value works, as the parser is iterative, and so does
destroying one, as #1436 made destruction iterative.
Bound how far the copy descends rather than take the call stack away from
it. The first levels are copied exactly as they were - the containers copy
their own elements, which is by far the fastest way to fill them - and only
once the copy has descended 128 levels is the value below it finished
without the call stack, through an explicit worklist. Copying can therefore
no longer exhaust the stack, however deeply a value is nested, while a value
nested less deeply than the bound - all but a vanishing minority - is copied
by the very same code as before and pays only for one counter.
That counter lives in thread_local storage, as one shared between threads
would be raced. JSON_NO_THREAD_LOCAL switches it off for toolchains without
thread_local; copying then goes through the worklist right away, which
yields the same values but is measurably slower.
The deferred values are completed before the copy they belong to returns, so
a value copied while another copy is going on - by a custom base class, say -
is unaffected by the copy it is nested in.
operator= takes its argument by value, so copy assignment is fixed as well.
Copying is as fast as it was, within measurement noise (medians of 9
interleaved runs, clang -O3): -1.3% for an array of strings, +0.0% for a
flat object, +0.1% for a flat array of numbers, +0.3% for nested arrays,
+0.6% for nested objects and +1.2% for a twitter-like document. Copying a
three-key object costs about ten nanoseconds more, the counter. Deferring
every level instead, rather than only those below the bound, measured
between 3% and 9% slower depending on the shape of the value.
This fixes#5387 for the copy constructor. dump() is still recursive.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Test the copy constructor's iterative path in CI
The copy constructor descends into 128 levels before it finishes a value
without the call stack, so the iterative path is otherwise only reached
by the few tests that nest deeper than that.
JSON_NO_THREAD_LOCAL switches the descent off, which sends every value
down that path. Running the whole test suite that way covers it with
every object type, string type, allocator, and base class the suite
already exercises. The new ci_test_no_thread_local target does that; the
macro had no build coverage at all before.
Copying a nested value also has to carry over what the element-wise copy
constructor would have copied: the parents that JSON_DIAGNOSTICS relies
on, and the positions that JSON_DIAGNOSTIC_POSITIONS reports. Both are
now checked on either side of the descent bound, for objects and arrays.
Neither was tested before, and dropping either one makes the new tests
fail.
Also quantify what JSON_NO_THREAD_LOCAL costs a copy instead of calling
it "measurably slower".
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Split the regression tests so that they keep linking
Linking test-regression2 fails with "relocation truncated to fit:
IMAGE_REL_AMD64_REL32 against `.rdata'" once its object grows past what
the MinGW linker copes with, and the copy constructor's helpers push it
over: the object grows by 6.3%, from 4,654,128 to 4,944,920 bytes at -O0,
and develop links at the smaller of the two.
Building the tests optimized shrinks the object enough to link, but the
binaries clang 11.0.1 and clang 18.1.8 then produce crash before doctest
prints its first line - 39 of 102 tests on clang 18 - so the objects have
to become smaller rather than denser.
Moving the test cases that follow "regression tests 2" into a file of
their own brings that object to 4,687,888 bytes, which is 0.7% above the
size that links today rather than 6.3%. Both files still build for C++11,
C++17 and C++20, and run the same 9 test cases and 135 assertions as
before, now spread over two binaries.
New regression tests belong in unit-regression3.cpp from here on, which
is what CONTRIBUTING.md now says.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Do not use thread_local storage with Clang targeting MinGW
Every test that copies a value segfaults there - 42 of 105 on clang
11.0.1, 39 of 102 on clang 18.1.8 - while the same tests pass with GCC
targeting MinGW, with Clang targeting MSVC, and with every other
toolchain the library is tested on. The counter that bounds the copy
constructor's descent is the library's first use of thread_local, so
that job had never exercised it before.
JSON_NO_THREAD_LOCAL already covers toolchains without thread_local
storage, and copying yields the same values with it, only more slowly.
Define it for this one automatically.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Balance the warning suppression the split separated
unit-regression2.cpp opens a DOCTEST_CLANG_SUPPRESS_WARNING_PUSH block at
the top and closed it at the very bottom, which the split moved into
unit-regression3.cpp: one file was left with a push and no pop, the other
with a pop and no push, which clang reports as an error.
Give each file the pair it needs.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Check both shapes without a C-style array
clang-tidy rejects the array the two shapes were iterated over
(cppcoreguidelines-avoid-c-arrays). The array only existed because astyle
reformats a range-for over a braced initializer list into something
unreadable; naming the two cases avoids both.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Split the regression tests far enough to leave room
The first split left unit-regression2.cpp 0.7% below the size develop
links at, which the comparison change in the follow-up immediately used
up: the MinGW linker fails on test-regression2_cpp20 again, naming
copy_shallow and to_partial_ordering among the relocations it cannot fit.
Move the sections from "issue #2067" on, and the helper types they use,
so that the file stops being the one that decides whether the tests can
be linked at all. At -O0 and C++20, unit-regression2.cpp is now 2,964,944
bytes against develop's 4,708,248, and 3,070,568 bytes with the follow-up
applied - roughly a third smaller either way, rather than a fraction of a
percent larger.
The 135 assertions are the same ones as before, now spread over three
test cases in two files.
Also silence the clang-tidy findings the deep-nesting tests draw: the
copies they make are what is being tested, and the reserve() computation
gets its parentheses.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Move the #4804 alias to the file that uses it
The split left the json_4804 alias behind in unit-regression2.cpp while
the test case that uses it went to unit-regression3.cpp, which does not
build for C++17 and C++20 as a result.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Include <span> where the split moved its only use
The #2546 test case guards itself with __has_include(<span>), but the
include itself sat in unit-regression2.cpp's preamble and stayed behind,
so the section compiled without a declaration wherever the guard passed -
which nvhpc reported and libc++ builds do not, as they skip the section
altogether.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Keep the descent bookkeeping in one place
Copying carried a depth count, a depth limit and a guard of its own, and
the comparison in the follow-up added a second set beside them. Neither
operation needs its own: they are never nested inside one another by the
library - copying a value does not compare one, and comparing two values
does not copy them - and where user code nests them anyway, sharing the
count only ends a descent sooner than it had to.
So there is now one nesting_depth(), one nesting_depth_limit() and one
nesting_depth_guard, which the follow-up uses instead of adding its own.
Inverting the test in copy_structured leaves the too-deep case and the
no-thread-local case as the same code.
The guard takes the count rather than looking it up, because the caller
has looked it up already to test it against the limit, and reaching
thread-local storage twice on the path that is taken almost every time is
worth avoiding.
The switch that copies the value of anything that is not an object or an
array was written twice - once in the copy constructor, once in
copy_shallow - so that adding a value_t meant editing both, and missing
one would have been silent. It is copy_leaf_value now, and inlined: both
callers have already sorted the containers out, and folding that test into
the switch is what keeps a value made mostly of numbers copying as fast as
it did.
Copying canada.json, citm_catalog.json and twitter.json is within 0.6% of
what it was before, measured as a paired ratio over 18 interleaved rounds
against a run-to-run spread of 0.3%.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Check that an abandoned copy can still be destroyed
Copying a value without the call stack builds the copy from the top down,
and every value whose own copy has not been made yet stays a null value
until it is. That is what lets a copy be abandoned half-built: the
destructor finds nothing but complete values and null ones.
Nothing tested it. Failing an allocation part-way through a copy of a
deeply nested value does, with the allocator the file already has for
exactly this kind of test.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Name the test's locals so Flawfinder stops matching them
The code scanning job reports CWE-362 - "check when opening files" - for
a test that opens no files: Flawfinder matched a local variable called
open. Rename it and its partner.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Keep the descent guard's bookkeeping self-contained
nesting_depth_limit() and nesting_depth_guard were only used inside
the JSON_NO_THREAD_LOCAL-guarded branch of copy_structured(), but were
defined unconditionally. Move them inside the #ifndef, and have the
guard look up the depth and test it against the limit itself (via
okay()) instead of making the caller do it - the caller no longer
needs to touch nesting_depth() at all. Also shrink the thread-local
counter to std::uint8_t, matching what its own doc comment already
argued.
Addresses gregmarr's review comments on #5389.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Make nesting_depth_guard usable regardless of JSON_NO_THREAD_LOCAL
nesting_depth_limit() and nesting_depth() stay behind #ifndef
JSON_NO_THREAD_LOCAL, since a descent cannot be bounded without a
per-thread count. But the guard itself now always exists, becoming a
no-op that is never okay() under that macro - the same way the bound
is already reached on every call without one. copy_structured() no
longer needs to know which case it is in.
This is what lets #5390 reuse the guard for comparison, which cannot
test JSON_NO_THREAD_LOCAL where the macro-based operators use it: the
guard now carries that distinction itself instead of requiring every
caller to.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Silence VS2015's C4503 for the custom-base-class test
The deep-copy support added for #5387 lengthened the mangled name of
std::allocator_traits<...>::construct for the test's map type past
VS2015's limit, which /WX turns into a build failure even though the
name is only used for (now-truncated) debug info.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Remove dead unused-parameter casts from copy_metadata()
@gregmarr asked whether the static_cast<void> pair in the
JSON_DIAGNOSTIC_POSITIONS-off branch was needed for an empty
json_base_class_t. It isn't: src and dst are already referenced
unconditionally by the base-class copy above, so no -Wunused-parameter
warning fires either way (checked with -Wall -Wextra
-Wunused-parameter, JSON_DIAGNOSTIC_POSITIONS 0 and 1).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix CI: build custom array types without a fill constructor, re-amalgamate
copy_array_level() built the destination array with the fill
constructor array_t(count, value), which is not part of the array
container interface the library otherwise assumes (e.g. custom
ArrayTypes that only provide a default and an iterator-pair
constructor, as covered by unit-custom-array-type.cpp). Default-
construct the array and resize() it instead, matching how the rest
of the codebase already grows array_t.
Also re-run the amalgamation, which had fallen out of sync with
include/nlohmann/json.hpp.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* docs: document that a NUL byte in the input is treated as end of input
A NUL byte anywhere in the input - trailing, or embedded ahead of more
otherwise well-formed JSON - is currently treated the same as genuine
end of input, so parsing silently stops there instead of raising the
parse_error.101 any other unexpected byte triggers. This mirrors the
NUL-terminated-C-string convention already used when no explicit input
length is given (json::parse(const char*) already stops at strlen()),
just applied uniformly rather than only when a length is genuinely
unavailable.
This behavior predates this change and is not being altered here -
changing it would be an observable, backwards-incompatible behavior
change for any caller that (knowingly or not) depends on it, which is
not something to do silently in a patch. Documenting the current,
verified behavior as a new FAQ entry instead, so it's an intentional
and discoverable part of the contract rather than a surprise.
Fixes#5530.
Signed-off-by: Niels Lohmann <niels.lohmann@gmail.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N4RQ1Ahan5YAGbnAQGjZTY
* Add JSON_STRICT_NUL_HANDLING opt-in macro for issue #5530
A NUL byte anywhere in the input is currently treated the same as real
end of input, rather than raising parse_error.101 like any other
unexpected byte (documented in the previous commit's FAQ entry). A full
unconditional fix was tried in PR #5532 but rejected as too risky to
ship by default: any caller could depend on the current behavior, even
unknowingly (e.g. a zero-padded buffer). On PR #5534, gregmarr proposed
a compile-time opt-in flag instead, and the maintainer agreed, wanting
it available now and defaulting to the corrected behavior in 4.0.0.
This mirrors the existing JSON_BRACE_INIT_COPY_SEMANTICS precedent as
closely as sensible:
- JSON_STRICT_NUL_HANDLING defaults to 0 (off); the three lexer sites
that treat '\0' as EOF/comment-terminator are gated with
`#if !JSON_STRICT_NUL_HANDLING` so the default-off behavior is
byte-for-byte identical to today's.
- input_adapters.hpp's `T (&array)[N]` overload additionally trims a
single trailing '\0' from a `char` array (e.g. a string literal like
`json::parse("123")`) when the macro is on, so that case keeps
working; every other element type (unsigned char, std::uint8_t, ...)
always keeps its full extent. This intentionally does *not* reuse the
existing strlen()-based pointer overload via SFINAE-excluding `char`
from the array overload, as originally sketched for this change: that
approach is ambiguous against the newer generic container overload
added since PR #5532, and even where it compiles, strlen()-scanning a
`char` array that is not NUL-terminated within its bounds reads past
the end of the array (confirmed with AddressSanitizer). Trimming only
a single trailing byte, without scanning, avoids both problems.
- Documented via docs/mkdocs/docs/api/macros/json_strict_nul_handling.md,
linked from the macros index/nav/features page, the FAQ entry, and
the parse/accept/operator>> reference pages.
- Tested in unit-class_parser.cpp and unit-deserialization.cpp, default
state unguarded and opt-in state guarded. Since the library itself
#undefs the macro at the end of json.hpp (as JSON_BRACE_INIT_COPY_SEMANTICS
already does), a plain `#if defined(JSON_STRICT_NUL_HANDLING)` guard
after the include never actually triggers; the tests instead capture
the command-line value into a test-local macro before including the
header. A few pre-existing fixtures elsewhere (std::array<uint8_t, N>
sized one larger than their literal, relying on value-initialization
to silently add a trailing zero byte) needed the same one-byte
adjustment to keep passing under the opt-in behavior.
Unlike the precedent, this adds a proper `JSON_StrictNulHandling` CMake
option (rather than a raw -DCMAKE_CXX_FLAGS injection) and wires its
ci_test_strict_nul_handling target into the ci_cmake_options job matrix
in .github/workflows/ubuntu.yml, so the opt-in build is actually
exercised in CI -- closing the one gap in the precedent's own CI setup
(ci_test_brace_init_copy_semantics is defined but never referenced by
any workflow, so it has never actually run).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Clarify where JSON_STRICT_NUL_HANDLING does not reject NUL bytes
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <niels.lohmann@gmail.com>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Add 60 contributors whose work was not yet credited and update seven
links that pointed to renamed or reassigned GitHub accounts.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Devirtualize binary_writer via a value-type output sink
to_cbor/to_msgpack/to_ubjson/to_bjdata/to_bson wrote every byte through
output_adapter_t, a shared_ptr<output_adapter_protocol> whose
write_character/write_characters are virtual. Unlike the lexer (templated
on a concrete InputAdapterType), the binary writer never got that
treatment, so binary output paid a vtable lookup per byte and a
make_shared per call.
Template binary_writer on an OutputSinkType and give it two concrete,
non-virtual sinks:
- output_vector_sink: appends straight into a std::vector (push_back /
insert), used by the vector-returning to_* convenience functions. No
vtable, no shared_ptr; the writes inline.
- output_adapter_sink: forwards to a type-erased output_adapter_t, so the
existing to_*(j, output_adapter) overloads (streams, strings, custom
adapters) keep working exactly as before -- one virtual call each,
unchanged.
binary_writer keeps a convenience constructor taking output_adapter_t
(building the default output_adapter_sink), so the adapter overloads are
untouched; only the convenience functions switch to the vector sink. The
friend declaration and the basic_json binary_writer alias gain the new
(defaulted) template parameter.
Output is byte-for-byte identical: verified across ~3000 randomized
values plus curated edge cases (all scalar widths, strings with invalid
UTF-8, binary, nested arrays/objects) for CBOR, MessagePack, UBJSON (both
size/type settings), BJData, and BSON, plus the output_adapter path, in
C++11/17/20. Warning-clean under clang -Weverything and the gcc pedantic
set; clang-tidy clean on the changed headers; make check-amalgamation
clean.
Throughput (g++ -O3, vs develop): scalar-dense binary output such as
integer arrays ~1.4x; many small to_cbor calls ~1.04x (DOM traversal
bound); string/blob-heavy output unchanged (already bulk-bound). No
workload regressed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix CI failures from binary_writer output-sink change
Four CI jobs failed on the initial commit; all are addressed here without
changing any output (binary encodings remain byte-for-byte identical to
develop across the differential corpus):
1. ci_test_gcc / cuda (-Werror=duplicated-branches): for number_float_t ==
float, static_cast<float>(n) is the identity, so write_compact_float's
two branches are intentionally identical. Once the concrete vector sink
is inlined, GCC constant-folds and diagnoses this (the type-erased path
hid it behind a non-inlined virtual call). Silence -Wduplicated-branches
for GCC (clang has no such warning) alongside the existing -Wfloat-equal
pragma.
2. ci_static_analysis_clang (UBSan nonnull-attribute): binary_writer passes
a null pointer with length 0 for empty strings/binary. output_vector_sink
/ output_adapter_sink declared write_characters JSON_HEDLEY_NON_NULL, so
the sanitizer flagged the (harmless) zero-length call once the sink was
called directly rather than through the attribute-free virtual base. Drop
the attribute from both sinks, matching the pre-existing behavior.
3. ci_cpplint (build/include_what_you_use): output_adapter_sink uses
std::move; add #include <utility>.
4. ci_cuda_example (nvcc 11.8): NVCC's front end rejects the default
template argument on the binary_writer alias template. Revert the alias
to its original single-parameter form (relying on binary_writer's own
defaulted OutputSinkType) and spell out the full type in the vector-sink
convenience functions.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Encode big-endian numbers with a byte swap instead of std::reverse
write_number() reordered multi-byte numbers for the big-endian formats
(CBOR/MessagePack/UBJSON) with std::reverse over the byte array. GCC
lowered only some sizes to a bswap; clang kept a scalar byte shuffle
(0 bswap instructions in the CBOR number path). Replace the reverse with
size-dispatched __builtin_bswap16/32/64 helpers (portable shift fallback
for other compilers; std::reverse retained for exotic sizes such as a
long double number_float_t).
Codegen: the CBOR number path now emits bswap on both compilers
(gcc 2 -> 16, clang 0 -> 4). Output is byte-for-byte identical to the
previous implementation across the binary differential corpus.
Throughput (isolated vs the std::reverse version, best of 9):
CBOR int64 array gcc +7% clang +10%
CBOR uint16 array gcc +27% clang flat
Modest but consistent on number-dense encodings; negligible on
string/blob-heavy output, as expected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Reserve output capacity up front for binary serialization
The vector-returning to_cbor/to_msgpack/to_ubjson/to_bjdata/to_bson grew
the output buffer purely by geometric reallocation. Reserving an estimate
up front avoids the early reallocations, which is the dominant per-byte
cost for array/object-heavy output.
The estimate (binary_reserve_hint) is deliberately conservative and safe
against untrusted input: it consults only the top-level element count
(O(1), no walk of the DOM), guards the multiplication against overflow,
and clamps the result to a fixed 1 MiB ceiling, so a large or hostile DOM
can never force an oversized allocation here. The buffer still grows
geometrically past the hint, so an underestimate only costs a few later
reallocations; scalars/strings/binary are written in one shot and get no
hint. Reserving capacity does not change the bytes produced.
Throughput (g++/clang -O3, vs the previous commit):
cbor int array +10% / +13%
cbor object array +20% / +38%
Output is byte-for-byte identical to develop across the binary
differential corpus.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Address review findings on the binary writer output sinks
- binary_reserve_hint(): the 4-bytes-per-element estimate over-reserved by up
to 4x for arrays of small scalars (CBOR encodes 0..23 in one byte), and the
returned vector kept that capacity. Make the hint a strict lower bound on the
encoded size instead, which also removes the 1 MiB clamp whose branch no test
could reach (the largest container in the suite has 65793 elements).
- Guard the -Wduplicated-branches pragma with __GNUC__ >= 7. The warning does
not exist before GCC 7, so naming it made GCC 4.8/4.9/5/6 - which the CI
matrix still builds - warn under -Wpragmas on every including translation
unit, breaking downstream -Werror builds.
- Constrain the adapter constructor of binary_writer with the enable_if its
documentation already claimed, so a writer over some other sink type is no
longer advertised as constructible from an output adapter.
- Let output_vector_adapter wrap output_vector_sink rather than duplicating the
append logic, so the type-erased and templated paths share one implementation.
- Collapse the three copies of the memcpy/byte_swap/memcpy dance into a single
byte_swap_buffer() helper, and add the MSVC _byteswap_* intrinsics so MSVC no
longer falls back to the scalar shuffle this change exists to eliminate.
- Add a vector_writer() helper for the five vector-returning to_* overloads
instead of spelling out the writer type at each call site, and drop a dead
default member initializer on output_adapter_sink.
- New tests: the vector sink and the adapter sink must produce identical bytes
for every format (the two to_* overloads no longer delegate to each other and
could otherwise drift), and binary_reserve_hint() must never exceed the size
actually written.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Route the -Wduplicated-branches pragma through Hedley
Match #5485, which moved the binary writer's hand-rolled diagnostic
pragmas onto JSON_HEDLEY_PRAGMA (merged into develop while this branch
was open). The devirtualization's -Wduplicated-branches suppression in
write_compact_float was the one raw '#pragma GCC diagnostic' left; it
now uses JSON_HEDLEY_PRAGMA like the adjacent -Wfloat-equal line, still
guarded to GCC >= 7 and non-clang (the warning exists only there).
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* Reject MessagePack/BSON binary subtypes that don't fit their wire format
Both formats store byte_container_with_subtype's subtype (a uint64_t)
in a single byte. The writers cast to std::int8_t/std::uint8_t without
a range check, so subtypes above 255 were silently truncated modulo
256 instead of raising an error. Throw out_of_range.413 instead when
the subtype exceeds the representable range of 0-255.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Move the new binary-subtype regression test out of unit-regression2.cpp
unit-regression2.cpp is already at the edge of what the MinGW linker
can relocate; adding this test's ~26 lines tips test-regression2_cpp20
(clang, Windows) over into "relocation truncated to fit:
IMAGE_REL_AMD64_REL32 against `.rdata'" (see 8ce64b9c1 / b82717c8a for
the same failure mode). Split the test along format lines instead:
MessagePack assertions move to unit-msgpack.cpp, BSON assertions to
unit-bson.cpp. The CBOR round-trip guard is dropped as redundant --
unit-cbor.cpp's "Tagged values" section already round-trips subtypes
up to 8589934590, far past the 70000 checked here.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The array-range insert() overload checked that pos fits the current
value and that first/last share the same owning value, but never
verified that value is itself an array. Passing iterators from an
object, a primitive, or null handed value-initialized (singular)
std::vector iterators straight to array_t::insert(), which is
undefined behavior. Add the missing is_array() check, mirroring the
equivalent check already present in the object-range insert()
overload.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Honor allow_exceptions=false for excessive array/object size (out_of_range.408)
The SAX DOM parsers' start_object()/start_array() threw out_of_range.408
directly via JSON_THROW when a binary format (CBOR/UBJSON/BJData) declared
a container size exceeding max_size(), bypassing the allow_exceptions flag
that every other malformed-input error path in these classes honors via
parse_error(). This meant that json::from_cbor(data, true, false) etc.
could still throw (or abort under JSON_NOEXCEPTION) instead of returning a
discarded value, contrary to the allow_exceptions=false contract.
Route all four call sites (two in json_sax_dom_parser, two in
json_sax_dom_callback_parser) through parse_error() instead, matching the
existing error-handling pattern used elsewhere in this file. Behavior is
unchanged when allow_exceptions is true (the default); the exception
message and type are identical.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Drop a non-portable exact exception message check in the 408 test
The allow_exceptions=false regression test checked the exact message
text produced when allow_exceptions=true (the default). On platforms
where std::size_t is 32-bit (e.g. mingw x86, MSVC Win32 builds), a
declared CBOR length of 2^63 is intercepted earlier, by
get_cbor_container_size()'s own (pre-existing, already correct)
length-narrowing check, with different wording than this fix's
start_array()/start_object() size check -- same error code, same
"still throws when allow_exceptions=true" guarantee, different text.
CHECK_THROWS_AS already verifies the behavior this test cares about
(still throws json::out_of_range, unchanged); drop the exact-message
assertion since it isn't portable across size_t widths and doesn't
add coverage of this fix specifically.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix -Werror=unused-result on json::from_cbor() in the 408 regression test
from_cbor() is [[nodiscard]]; CHECK_THROWS_AS() otherwise discards its
result, which GCC flags under -Werror. Assign to a throwaway json, as
the rest of the suite already does for from_cbor()/from_msgpack().
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Restore a duplicate key's prior value when the callback rejects its new value
json_sax_dom_callback_parser::key() unconditionally overwrote the object
slot for a key with a `discarded` placeholder as soon as the key was
accepted by the parser callback. For a duplicate key (legal JSON), this
destroyed the pre-existing value from an earlier occurrence of the same
key before the new value was even parsed. If the new value was then
rejected by the callback, remove_discarded_value() erased the member
entirely instead of leaving the original value in place, contradicting
the documented behavior that a discarded value behaves as if it was
never read.
Add a small stash of (slot pointer, previous value) pairs so that when
key() overwrites an existing member with the discarded placeholder, the
previous value can be restored later if the corresponding value (scalar,
object, or array) is rejected, instead of being erased. The stash entry
is dropped without restoring once the new value is definitively
accepted (in handle_value() for scalars, end_object()/end_array() for
containers), so a duplicate key whose new value is accepted still keeps
the last value as before. Non-duplicate keys are unaffected: rejecting
their value still removes the member entirely, since there is nothing
to restore.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Mark parser-callback test lambdas noexcept to fix GCC -Wnoexcept -Werror
GCC's libstdc++ std::function move assignment evaluates a noexcept
check that invokes a wrapped callable in an unevaluated context; a
non-noexcept parser_callback_t lambda then trips -Wnoexcept ("noexcept-
expression evaluates to 'false'"), which CI's ci_test_gcc job builds
with -Werror. The pre-existing parser_callback_t test lambdas in this
file already work around this by declaring themselves noexcept; apply
the same fix to the three added lambdas that didn't.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Make diff() account for member order in ordered_json objects
diff() compared source/target objects purely by key set, ignoring
relative member order. For ordered_json (insertion-ordered, vector-
backed object_t), two objects that differ only in member order are
unequal via operator==, but diff() never emitted any patch operation
to fix the order, so source.patch(diff(source, target)) == target
could fail to hold.
Fix by detecting when common keys appear in a different relative
order in source vs. target (or when a new key would need to land
somewhere other than the end), and in that case removing and
re-adding the affected keys in target's order, which relies on
patch()'s "add" op appending new keys at the end of an ordered_map.
For plain json (std::map-backed, always key-sorted iteration) this
is a no-op and the original minimal per-key diff path is unchanged.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Avoid redundant lookups in diff()'s object-order tracking
The previous fix for ordered_json member order re-derived common-key
order and suffix information with extra target.find()/source.find()
calls layered on top of the pre-existing removed/added-key passes,
instead of reusing those same passes. This roughly tripled the number
of map lookups per diff() call for every object, including plain
`json`, where the reordering path is never taken.
Piggyback the order tracking (and the "add" op construction for new
keys) onto the two passes the algorithm already needs to detect
removed/added keys, and walk the fast path's recursion in lockstep
with the precomputed common-key list instead of re-querying `target`.
This restores diff() to its pre-existing lookup count; benchmarked at
n=1000 keys, ordered_json::diff() was roughly 2x slower than baseline
before this change and is back within noise of baseline after it.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Preserve diff()'s original op ordering and fix a slow-path deletion gap
Splitting removed-key detection and common-key recursion into separate
passes (for the earlier lookup-count fix) changed the emitted patch's
op order: all "remove" ops now came before all recursive per-key diffs,
instead of interleaved in source's iteration order as the original
implementation did. This broke docs/mkdocs/docs/examples/diff.output's
exact-match CI check (ci_test_examples) even though the patch was still
semantically correct.
Defer "remove" emission into the same walk that does the recursive
diffs, so common keys and deleted keys are interleaved in source order
again, matching historical output.
While restructuring that walk, the reordering ("slow path") branch was
only emitting "remove" for keys common to both objects, never for keys
present in source but genuinely absent from target -- a key deleted
alongside an actual reorder would silently survive the patch. Fixed by
removing every source key in the slow path (both deleted and common
keys need removing there; common keys are then re-added in target's
order). Verified with a targeted reorder+deletion case and a fresh
20,000-case round-trip fuzz run (0 failures).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The characterization test added in #5482 asserted that basic_json::swap()
does NOT exchange start_position/end_position, based on a misreading of
the code cited for #5420. In fact swap() (json.hpp, around line 3637)
does swap start_position/end_position along with the value, consistent
with copy-assignment. The test's assumption was backwards, so it failed
on every CI job across every branch/PR since the commit landed. Correct
the assertions to match the actual (and correct) behavior: positions are
exchanged together with values.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* Fall back to plain-object encoding when _ArrayType_ is not a string
write_bjdata_ndarray() looked up _ArrayType_ by calling get<string_t>()
directly, which throws type_error.302 when the annotation is not a
string (e.g. a number, null, boolean, array, or object). Per the
documented BJData ndarray contract, an object only qualifies for the
compact ndarray encoding if _ArrayType_ names a known type; anything
else must fall back to plain-object encoding, the same way an unknown
type-name string already does.
Add an is_string() check before the get<string_t>() call so a
non-string _ArrayType_ takes the existing "unrecognized type name"
fallback path instead of throwing.
Fixes#5398.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Relax the BJData fuzzer's round-trip check from byte-exact to value-exact
Fixing #5398 lets to_bjdata() proceed past the object it used to reject,
which exposed a pre-existing, unrelated round-trip quirk to the fuzzer:
a binary_t value serialized through the non-optimized ("$U#"-less)
array encoding is parsed back as a plain array of numbers, since
from_bjdata() has no way to tell "array of uint8 numbers" apart from
"array of bytes" without that optimized header. Re-serializing that
plain array then goes through the generic smallest-type writer, which
- unrelated to this PR, and long predating it - prefers the 'i' (int8)
marker over 'U' (uint8) for values that fit both, so the re-encoded
bytes can differ from the original even though both decode to the same
value.
This is not introduced by the #5398 fix; the same divergence reproduces
from a bare json::binary_t value with no _ArrayType_ annotation
involved at all, on the commit immediately preceding it. A general fix
would mean changing the shared UBJSON/BJData smallest-type selection
that hundreds of existing tests pin to 'i' for small positive
integers, which is out of scope and too risky for this PR.
Update fuzzer-parse_bjdata.cpp's round-trip assertions to check that
re-serializing is value-stable (from_bjdata(to_bjdata(j)) == j) rather
than byte-exact, matching the guarantee BJData actually provides, and
add a regression test in unit-bjdata.cpp using the exact OSS-Fuzz input
that documents the behavior.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Compare dump()s instead of json values in the BJData fuzzer's round-trip check
The value-stability assertion added to fix the earlier OSS-Fuzz crash
(json::from_bjdata(to_bjdata(j2)) == j2) itself broke on a NaN payload:
IEEE 754 NaN is never equal to itself, so operator== reports two
structurally-identical trees containing a non-finite double as
different -- not a round-trip bug, just NaN's ordinary
non-reflexivity. dump() serializes any non-finite double the same
deterministic way (as JSON null, since JSON cannot represent NaN or
Infinity), so comparing dumps is stable under exactly the values that
break operator==.
Verified against both the original OSS-Fuzz crash input and the new
one (0x68 0x68 0x7c, which decodes to a NaN), plus a local 2.5M-case
random-input sweep with no failures.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix to_bjdata() emitting the Draft-3-only 'B' marker in default Draft-2 mode
_ArrayType_ = "byte" mapped unconditionally to the BJData type marker
'B', regardless of the requested bjdata_version. 'B' is defined only by
BJData Draft 3; with the default version (draft2), this produced a
stream that is invalid for Draft 2 and, unlike every other
_ArrayType_, round-tripped back as a binary value instead of the
original annotated object.
Only accept "byte" / emit 'B' when bjdata_version selects Draft 3.
Under Draft 2, fall back to the same plain-object encoding used
elsewhere in this function for other invalid-annotation cases, so the
value round-trips correctly.
Fixes#5404.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Future-proof the Draft-3-only 'B' marker gate
@gregmarr pointed out that dtype == 'B' && bjdata_version != draft3
only future-proofs by accident, since bjdata_version_t currently has
exactly two values. Compare with < instead, so a later draft that
keeps the 'B' marker valid does not need this gate revisited.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix dead ill-formed-fourth-byte UTF-8 test sections (byte3/byte4 typo)
The "ill-formed: wrong fourth byte" SECTIONs in unit-unicode3.cpp,
unit-unicode4.cpp, and unit-unicode5.cpp guarded their loop with a check
on byte3 instead of byte4. Since the enclosing loop already restricts
byte3 to its valid range, the guard was always true and the section's
"continue" fired unconditionally, so check_utf8string()/check_utf8dump()
were never actually invoked for a malformed fourth byte.
Fixing the guard naively (byte3 -> byte4) would also have swept the full
byte2 x byte3 combinatorics for every byte4 value, adding millions of
redundant iterations: the lexer validates continuation bytes strictly in
sequence with early exit (see next_byte_in_range() in lexer.hpp), so once
byte2/byte3 are within their valid range, the byte4 outcome does not
depend on which valid byte2/byte3 values were chosen. Instead, byte2 and
byte3 are now held to a small hedge of representative valid prefixes
(range corners plus a midpoint) while byte4 is still swept exhaustively
over its full 0x00-0xFF range, since that is the actual property under
test. Also fixed the garbled "skip fourth second byte" comment in
unit-unicode3.cpp.
Verified offline: before the fix, the "wrong fourth byte" subcase
executes 0 assertions in all three files (proving it was dead code);
after the fix, it executes 11520 (unicode3), 34560 (unicode4), and 11520
(unicode5) assertions, and a deliberately reintroduced bug in the
lexer's byte4 range check causes it to fail (proving it is now
meaningful). Total per-file assertion counts grow by the same small
amounts, not by millions, and all other sections in these files still
pass unchanged.
Fixes#5416
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Keep full byte2 x byte3 combinatorics in the wrong-fourth-byte sections
The maintainer wants exhaustive coverage of every byte combination here
rather than the representative-prefix reduction, matching the style of
the sibling "wrong second/third byte" sections in the same files.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Add missing diagnostic-positions test coverage (lifetime, input adapters, SAX)
Building on the merged unit-class_parser.cpp from #5417, add
characterization tests (regression protection for existing behavior, not a
behavior change) for JSON_DIAGNOSTIC_POSITIONS:
- value lifetime: copy ctor copies positions recursively, move ctor resets
the moved-from value to npos, and mutating a parsed document (operator[],
push_back, erase) leaves the parent's stale span and siblings' positions
untouched while new values get npos.
- input adapters: wide-string input positions count transcoded UTF-8 bytes
(not wide characters), BOM-prefixed input's start_pos() reflects the
skipped 3-byte BOM, istringstream/ifstream/iterator-pair inputs report
consistent (non-npos) positions, and binary formats (CBOR, MessagePack,
UBJSON, BSON) always report npos.
- a user-constructed json_sax_dom_parser with no lexer (as used when driving
json::sax_parse() directly) reports npos for every value, since it has no
m_lexer_ref to source positions from.
While characterizing swap(), found that basic_json::swap() (and the friend
swap() that forwards to it) does not swap start_position/end_position,
unlike copy-assignment's operator=(basic_json), which does as part of its
copy-and-swap implementation. This looks like a real inconsistency/bug, but
per the scope of this test-only change it is only pinned (not fixed) here;
see the comment at the "swap() does NOT exchange positions" section.
Fixes#5420
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix MSVC source-encoding portability in the wide-string position test
Use é escapes instead of a literal UTF-8-encoded 'é' inside the L""
literal, so the wide string's content does not depend on the compiler's
assumed source character set (MSVC without /utf-8 decodes raw non-ASCII
source bytes using the system code page rather than as UTF-8, which was
producing a wstring of unexpected length/content and failing the
ws.size()/end_pos() assertions on Windows CI).
Also reworded a comment that unintentionally embedded the literal
substring "TODO check", which clang-tidy's google-readability-todo check
flags regardless of quoting context.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* test: cover JSON_NO_IO, JSON_THROW/TRY/CATCH_USER, JSON_SKIP_LIBRARY_VERSION_CHECK, and JSON_DisableEnumSerialization in CI (#5423)
These four supported configuration macros were never actually compiled
anywhere in the test matrix:
- JSON_NO_IO and the JSON_THROW_USER/JSON_TRY_USER/JSON_CATCH_USER trio
are exercised together in a new tests/src/unit-no_io_and_user_exceptions.cpp,
which is automatically picked up by the existing unit-*.cpp test glob and
thus built across the whole standard test matrix.
- JSON_SKIP_LIBRARY_VERSION_CHECK is exercised by a new, dedicated
tests/src/skip_library_version_check.cpp, compiled directly by the new
ci_test_skiplibraryversioncheck target in cmake/ci.cmake: the scenario it
simulates (mixing two differently-versioned inclusions of the library)
unavoidably triggers the compiler's own "macro redefined" warning, which
would fail under the library's own -Weverything/-Werror unit test matrix
for a reason unrelated to the macro under test.
- JSON_DisableEnumSerialization already had #if-guarded tests in several
unit-*.cpp files (from #4384), but no CMake target ever actually set the
JSON_DisableEnumSerialization CMake option, so that guarded code was never
compiled. Add ci_test_disableenumserialization, mirroring the existing
ci_test_noimplicitconversions/ci_test_noglobaludls targets. Building the
full test suite with this option on surfaced one real, narrow gap: get<T>()
on std::vector<std::byte> (used by unit-regression2.cpp's custom BinaryType
tests) relies on std::byte being handled via enum serialization, so add the
same #if-guard convention to the two affected SECTIONs there.
Both new CI targets are added to the ci_cmake_options matrix in
.github/workflows/ubuntu.yml, alongside the existing ci_test_* targets.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* test: cover multi-digit widths and bare alignment in std::formatter<json> (#5423)
Every existing std::formatter spec with a width used a single digit (e.g.
"{:2}"), so the width-parsing loop's accumulation of a second/third digit was
never exercised; add multi-digit width cases. Likewise, every existing spec
with an alignment character also had an explicit fill character, so the
bare-alignment branch (e.g. "{:<}", with no fill) was never exercised; add
cases asserting it keeps the default space indent character.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* test: add coverage for patch_inplace() (#5423)
patch_inplace() had no unit test at all. Add a happy-path case mirroring an
existing patch() example, and -- more importantly -- pin its distinguishing
contract versus patch(): when a multi-operation JSON Patch fails partway
through, patch_inplace() (which mutates the document directly, operation by
operation) leaves whatever operations already succeeded applied, whereas
patch() (which applies the patch to an internal copy that is discarded on
exception) leaves the original completely untouched either way. Verified
empirically against the current implementation before writing the assertions.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* test: fix duplicate TEST_CASE name in unit-no-mem-leak-on-adl-serialize.cpp (#5423)
Two distinct TEST_CASEs were both named "check_for_mem_leak_on_adl_to_json-2".
doctest allows duplicate names, so both still ran, but it makes
--test-case=<name> filtering and reporting ambiguous. Rename the second one
to "-3", continuing the existing "-1"/"-2" sequence.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* test: add direct coverage for the std::u8string to_json overload (#5423)
The ADL to_json overload for std::basic_string<char8_t, ...> was only ever
reached indirectly, via std::filesystem::path::u8string(). Add a test that
constructs a json value directly from a std::u8string, gated the same way as
the overload itself (include/nlohmann/detail/conversions/to_json.hpp): behind
both the std::filesystem::path feature guard and __cpp_lib_char8_t, since the
overload only exists when both are satisfied.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* test: verify move semantics of byte_container_with_subtype's rvalue constructors (#5423)
The two rvalue-reference constructors were never distinguished from their
const-lvalue-reference twins by any test. Add a "move semantics" section that
constructs from an rvalue std::vector, checks the resulting container keeps
the exact same buffer address as the source (a stronger check than just
observing the source ended up empty, since a copy-then-clear could do that
too), and confirms the source vector was left empty.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Guard patch_inplace() partial-application test against JSON_NOEXCEPTION
The "distinguishing contract vs patch(): partial application on
failure" test relies on doc.patch_inplace(patch) actually throwing so
the partially-applied state can be observed right after the throw
point. Under ci_test_noexceptions, JSON_THROW() calls std::abort()
instead of throwing, and doctest's --no-throw test filter (which that
CI job passes) makes CHECK_THROWS_AS() a no-op that never even
evaluates its expression -- so patch_inplace() is never called and the
follow-up assertions fail against the untouched original document.
Guard the whole SECTION with #if !defined(JSON_NOEXCEPTION), following
the same convention already used elsewhere in the test suite (e.g.
unit-class_parser.cpp) for exception-dependent tests.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix MSVC C2220 in the std::u8string conversion test
MSVC's C5321 ("nonstandard extension used: encoding '\xNN' as a
multi-byte utf-8 character") is promoted to a hard error by our MSVC CI
configs. It fires because the test composed a non-ASCII UTF-8 sequence
inside a u8"" literal using raw \x byte escapes; MSVC treats that as
nonstandard and suggests using \u universal-character-names instead,
which every compiler agrees on and which compiles down to the exact
same encoded bytes.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Guard the JSON_THROW_USER test against JSON_NOEXCEPTION and GCC's -Wunused-result
Two independent CI configurations failed to build/run this new test:
- ci_test_noexceptions runs the whole suite with -DJSON_NOEXCEPTION and
doctest's "--no-throw" filter, which compiles CHECK_THROWS_AS() down
to a no-op that never even invokes the guarded expression. Since this
test's whole point is to observe json_throw_user_call_count after
json::parse()/at() actually throw, it can't be meaningfully run under
that filter (our JSON_THROW_USER override still throws real
exceptions regardless of JSON_NOEXCEPTION, but the assertion never
gets a chance to run). Guard the TEST_CASE with
#if !defined(JSON_NOEXCEPTION), mirroring the existing precedent in
unit-json_patch.cpp.
- ci_test_gcc and ci_test_standards_gcc(11) failed with
-Werror=unused-result on the discarded json::parse() return value.
json::parse() is marked warn_unused_result, and unlike a real
[[nodiscard]] attribute, GCC does not consider that satisfied by
doctest's (void)-cast around the expression in C++11 mode. Assign the
result to a discarded local instead, matching the established
`json _ = json::parse(...)` idiom already used throughout
unit-class_parser.cpp.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Suppress a clang-tidy false positive on an intentional defensive copy
performance-unnecessary-copy-initialization suggests copy_for_patch
could be a reference since it's never modified -- but the copy is the
point: it guards against a hypothetical regression where patch()
mutates its receiver, which a reference could never catch (the
follow-up assertion would just compare `original` to itself).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix clang-tidy findings in the JSON_NO_IO/JSON_THROW_USER test
- bugprone-macro-parentheses: wrap the JSON_THROW_USER macro argument in
parentheses at the throw site.
- modernize-raw-string-literal: switch two escaped JSON string literals to
raw string literals.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Add test coverage for ordered_json/alt_json across binary formats and patch/diff/flatten APIs
Closes a test-coverage gap from #5421: ordered_json (and the alt_string-based
basic_json specialization from unit-alt-string.cpp) were never round-tripped
through the binary formats (CBOR/MessagePack/UBJSON/BSON/BJData), nor through
flatten()/unflatten(), diff()/patch()/patch_inplace(), or merge_patch(). Also
adds a std::formatter<ordered_json> spot-check, mirroring the precedent set
by the format_as() ADL-deduction test.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Pass alt_string's std::string constructor argument by value (clang-tidy modernize-pass-by-value)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
#5285's indentation regression test didn't exercise json::binary,
which serializes as an object but always writes its byte array
compactly (dump_byte()). #5186 had covered this case before it was
closed as superseded; port just that coverage here.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: document the implicit requirements on basic_json's template parameters
The requirements that basic_json places on its eleven template parameters
were only implied by how the library uses the resulting object_t, array_t,
string_t, etc. Consumers had to discover them by trial and error.
Add "Template Parameter Requirements" collecting them, split into what is
always required and what is only required when a particular part of the API
is instantiated. Notable findings that were previously undocumented:
- ObjectType must provide a key_compare member type (actual_object_comparator
names object_t::key_compare in both arms of a std::conditional), and its
third template parameter is used as a comparator, so std::unordered_map
cannot be used without a wrapper.
- ArrayType must provide capacity() -- push_back(), emplace_back(),
operator+=(), and operator[](size_type) call it unconditionally -- and
needs random-access iterators, so std::deque and std::list do not work.
- StringType needs contiguous, null-terminated data(), a one-byte value_type,
and either assignability from std::to_string or an ADL int_to_string().
- NumberFloatType must be float, double, or long double for parsing and
serialization; the integer types must satisfy std::is_integral.
- AllocatorType must be stateless, support incomplete types, and use plain
pointers.
- BooleanType and the number types are union members and must be trivial.
Link the new page from the basic_json overview, the types feature page, and
the individual type alias pages, and correct the container examples given for
ObjectType (std::unordered_map) and ArrayType (std::list), which do not work.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix object_comparator_t for object types without key_compare
detail::actual_object_comparator selected between object_t::key_compare and
default_object_comparator_t with std::conditional. Both type arguments of
std::conditional are named eagerly, so object_t::key_compare had to exist
regardless of the condition, and the has_key_compare guard added in 3.11.0
never took effect: any ObjectType without a key_compare member type failed to
compile while instantiating basic_json itself.
Use detected_or_t instead, which resolves through a SFINAE partial
specialization and only names object_t::key_compare when it exists. The
selected type is unchanged for every object type that compiled before, so
object_comparator_t -- a public member type -- keeps its meaning and ABI.
has_key_compare had no other users and is removed.
Add a regression test using an adapter around std::unordered_map, which has no
key_compare; it fails to compile without this change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: list the types that are known to work for each template parameter
Follow up on the template parameter requirements page: state, for every
template parameter, which concrete types work and where they stop working.
Each entry was verified by compiling and running a common workload (DOM
access, dump, parse, CBOR/MessagePack round-trip, flatten, hash) against that
instantiation.
Findings worth calling out:
- ObjectType no longer needs a key_compare member type, so the std::unordered_map
adapter only has to restore the template argument order. A hash-ordered
ObjectType works everywhere except unflatten(), which reconstructs an array
only when it meets the reference token 0 before the other indices.
- ArrayType: std::deque works when wrapped to add capacity(); std::list does not.
- StringType: std::pmr::string and std::basic_string with a custom allocator
compile for the DOM, dump, and parse, but not for the binary readers, flatten,
or diff, because the library assigns std::string values to string_t and
int_to_string cannot be overloaded for a type in namespace std.
- NumberFloatType: long double works for dump and parse but not for the binary
formats, which have no encoding for it.
- BinaryType: std::vector<std::byte> supports assignment, get, and the binary
formats, but neither dump nor std::hash<basic_json>.
Also record the object_comparator_t fix in its version history.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix unflatten and binary dumping for non-default configurations
unflatten() decided between array and object by looking at the first reference
token it happened to see for a node: it started an array only when that token
was 0. With a sorted object type the token 0 always arrives first, so the
result was correct by accident; with an object type whose iteration order is
unspecified, {"/c/2":3,"/c/1":2,"/c/0":1} unflattened to an object with the
keys "0", "1", and "2" instead of an array.
Collect the pointer prefixes that have a reference token 0 among their children
before building the result, and let get_and_create() consult that set. The
outcome is now independent of the iteration order and matches, for every input,
what a sorted object type produced before: a value is restored as an array if
and only if one of its keys is 0. Iterating the flattened object in a different
order would have been simpler, but it would have changed the key order of the
result for insertion-ordered object types.
The serializer, std::hash, and the UBJSON writer converted the elements of a
binary value to an integer implicitly, which does not compile for a BinaryType
whose value type is std::byte, and which made dump() write the bytes of a
signed value type as negative numbers. Convert to std::uint8_t explicitly in
all three places, so every byte type dumps as 0..255. The default
std::vector<std::uint8_t> configuration is unaffected.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: note which Abseil containers can be used as template arguments
Checked against Abseil release 20250127.0 with the same workload as the other
entries on the page (DOM access, dump, parse, CBOR/MessagePack/UBJSON
round-trip, flatten, hash), with and without JSON_DIAGNOSTICS.
absl::flat_hash_map and absl::node_hash_map work as ObjectType through an
adapter that restores the template argument order and makes erase(iterator)
return the following iterator, which Abseil's returns as void. The page now
carries that adapter, and notes that absl::flat_hash_map does not keep
references to the mapped values valid across insertions while
absl::node_hash_map does. Both have a capacity() member, so JSON_DIAGNOSTICS
already refreshes the parent pointers conservatively for them.
absl::btree_map and absl::InlinedVector cannot be used at all: object_t and
array_t are formed while basic_json is still incomplete, and both inspect
their value type at class scope. std::map and std::vector are required by the
standard to tolerate this, third-party containers generally are not, so the
page states the constraint on its own rather than only per container.
absl::InlinedVector does work as BinaryType, where it is instantiated with a
complete type. absl::FixedArray and absl::Cord are not usable.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Relax the ArrayType and ObjectType requirements
Two requirements forced users of otherwise suitable containers to write a
wrapper, and neither was load-bearing.
array_t::capacity() was read in push_back(), emplace_back(), operator+=(), and
operator[](size_type), but set_parent() only looks at the value under
JSON_DIAGNOSTICS; without diagnostics it was computed and discarded. Read it
through array_capacity(), which reports unknown_size() when diagnostics are off
or when the array type has no capacity() at all, and treat an unknown capacity
as "the elements may have moved" so the parent pointers are refreshed
conservatively. std::deque now works as ArrayType, in both builds, and
capacity() is no longer named at all in a default build. Since the capacity is
now only meaningful for array insertions, it moves out of set_parent() into
set_parent_after_array_insert().
basic_json::erase(iterator) assigned the object's erase() return value, which
requires the container to return the following iterator. Abseil's hash maps
return void to avoid computing a successor the caller may not need. Detect that
and compute the successor before erasing; containers that return an iterator,
including the vector-backed ordered_map where a precomputed successor would be
wrong, keep the existing path.
Together these leave an Abseil hash map needing only an alias that restores the
template argument order, and no adapter at all for std::deque.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Do not require string_t to be convertible from std::string
Three places built a std::string and handed it to something expecting a
string_t: the UBJSON high-precision number reader, which every binary reader
instantiates, and the BSON writer's array element size calculation and write.
That silently required string_t to be implicitly convertible from std::string,
which std::string itself and types with a string_view conversion satisfy, but
many string types do not.
Construct the string_t explicitly from the data and size, which the
requirements already cover. This makes boost::container::string, eastl::string,
std::pmr::string, and std::basic_string with a custom allocator work as
StringType, none of which could previously be used with any binary format.
Add binary format coverage to the alt_string test, which had none, including a
UBJSON high-precision number -- the case that goes through the reader path.
BSON stays uncovered there: it additionally needs string_t::find(value_type),
which alt_string does not provide.
Also record which containers from Boost, Abseil, and EASTL work for each
template parameter, and correct two claims: std::pmr::string is usable after
this change, and tsl::ordered_map is not usable at all, because its iterators
expose the mapped value as const while basic_json modifies it in place.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: record compatibility for the common header-only hash maps
ankerl::unordered_dense (map and segmented_map), phmap (flat_hash_map and
node_hash_map), and robin_hood::unordered_flat_map all work as ObjectType
through the same adapter as Abseil's and Boost's hash maps, which only has to
restore the template argument order.
phmap::btree_map and robin_hood::unordered_node_map do not: like the other
btree containers they require a complete value type.
Note that none of these hash maps defines key_compare, so every one of them
depends on object_comparator_t falling back to default_object_comparator_t.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: record Folly and the remaining vector replacements
Folly works, with the caveat that its headers need C++20: folly::fbstring as
StringType, folly::fbvector and folly::small_vector as ArrayType,
folly::fbvector<std::uint8_t> as BinaryType, and folly::F14NodeMap as
ObjectType through the usual argument-order adapter. folly::F14FastMap is the
exception and requires a complete value type.
For ArrayType, boost::container::devector, boost::container::static_vector
(within its fixed capacity), and std::pmr::vector work as well.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: cover fifo_map, gtl, folly::sorted_vector_map, and Qt
nlohmann::fifo_map works through the adapter that has always been documented
for it, and preserves the insertion order. Restore its mention in the object
order page, which was dropped together with the tsl::ordered_map one: unlike
ordered_map it keeps a lookup index, so it is the insertion-ordered option
without the quadratic cost.
gtl::flat_hash_map and folly::sorted_vector_map work as well, the latter
through an alias that drops the allocator, whose value type it disagrees on.
gtl::btree_map does not, for the same reason as the other btree containers.
None of the Qt containers can be used, each for its own reason: QMap has no
value_type, QHash iterators yield the mapped value rather than a pair, QList
has no max_size(), QByteArray spells empty() as isEmpty(), and QString is
UTF-16.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: qualify the std::pmr::string support claim
Listing std::pmr::string as fully supported was an overclaim: it was only ever
checked with the default memory resource, which is not what PMR is for.
basic_json cannot be given an allocator or a memory resource, so a pmr string
inside a value always allocates from std::pmr::get_default_resource(), and
assigning an arena-backed string into a value silently drops its resource,
because polymorphic_allocator does not propagate on copy construction. Passing
polymorphic_allocator as AllocatorType does not compile either. Only the
process-global set_default_resource() redirects these allocations.
Say so, and separate the row from std::basic_string with a custom stateless
allocator, which is unaffected.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: remove a duplicated StringType compatibility section
The StringType section carried two 'Compatible types' tables and two copies of
the reference-implementation tip. The second table was a stale copy from before
the binary format string fixes and still listed std::pmr::string and
std::basic_string with a custom allocator as unusable, contradicting the
corrected table a few lines above it, and it dragged along the old explanation
that blamed int_to_string.
Drop the stale copy and put the surviving table before the notes, so the
'see below' in the std::pmr::string row points forwards.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: correct the template parameter requirements after independent verification
Every claim on the page was re-checked by compiling and running it, including
the rows that say a type cannot be used, which were checked to fail for the
documented reason and not merely to fail. Twenty-four claims were wrong.
The most consequential: the incomplete-type constraint applies to ObjectType
only. object_t is instantiated inside the class definition, because it is
probed for key_compare; array_t is only named there and is not instantiated
until basic_json is complete. So eastl::vector, QList and QVector are not
excluded by incomplete types at all -- they simply have no max_size() -- and
absl::InlinedVector is excluded for a subtler reason of its own.
Further corrections: ObjectType does not need erase(key), which has a fallback,
but does need at(key) for UBJSON output; only == and < are used, or == and <=>
under C++20, not all six; the documented adapter does not fit ankerl or
robin_hood. ArrayType needs no initializer-list insert, and value_type, the
(count, value) constructor and swappability are per-function, not always.
BinaryType needs a range insert for CBOR indefinite-length byte strings and
does not need push_back. StringType needs append(const StringType&)
unconditionally, and does not need operator!= or operator== against const
char*; empty(), resize(n) and reserve(n) are per-subsystem; int_to_string is
needed by diff, items and std::hash rather than by JSON Pointer or flatten.
BooleanType must be implicitly convertible from bool, and JSONSerializer's
second parameter need not carry a default.
std::pmr::string was wrong in the other direction this time: a moved-in string
does keep its memory resource, and later growth allocates from it. Only copies
land on the default resource.
Five requirement violations are not caught at compile time rather than the two
the page claimed; they are now listed together up front. Split every
compatibility table into what works and what does not, as the reasons in the
second half are the useful part.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Reduce the string_t and array_t members the library requires
Several members were required only because of how the library happened to
be written, not because the functionality needs them. Dropping them widens
the set of usable string and array types, and one of them was also a
performance problem.
string_t:
- c_str() is gone. Every call site already knew the length and passed it
along, so data() is enough. The one place that did not, the diagnostics
path in exceptions.hpp, now builds the token from data() and size(),
which also stops it from truncating keys that contain a null byte.
- back() is gone; the serializer indexes the last character instead.
- find(str, pos), replace(), and substr() are gone. escape() and
unescape() rebuilt the string with one replace() per escaped character,
which moves the tail every time: escaping a string of n characters that
all need escaping cost O(n^2). Both now scan with find_first_of() -- a
member the pointer parser already required -- and append whole runs, so
the common case is one search and one copy. Escaping 64000 tildes drops
from 717 ms to 20 ms; a string with nothing to escape gets faster too
(8.4 ms to 5.8 ms), because the scan is still a single memchr per pass.
json_pointer::split() takes its reference tokens with the
(const char*, size_type) constructor rather than substr().
- json_pointer::to_string() accumulates with concat<string_t> instead of
letting concat default to std::string and converting afterwards, so
streaming a json_pointer no longer requires string_t to be assignable
from a std::string.
array_t:
- at(size_type) is gone. basic_json::at(size_type) checked the index by
calling array_t::at() and translating std::out_of_range, which also
required the array type to throw that exact exception. It now compares
against size() and uses operator[]. The thrown exception, its message,
and the behaviour under JSON_NOEXCEPTION are unchanged.
The BSON writer wrote the terminating null byte out of the string's own
buffer (size() + 1). It now writes the byte itself, so string_t::data()
need not be null-terminated for to_bson().
The tests pin the reduced API: alt_string loses the five dropped members
and gains coverage of the escaping paths, and a std::vector whose at() is
hidden is used as an ArrayType.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: record the reduced string_t and array_t requirements
Drop c_str(), back(), find(str, pos), replace(), and substr() from the
StringType requirements and at(size_type) from the ArrayType ones, and
note the string assignment the JSON pointer code performs. Streaming a
json_pointer no longer needs assignability from a std::string.
Add the non-null-terminated data() to the list of violations that are not
diagnosed at compile time -- it was described in the StringType section
but missing from the summary at the top -- and correct the QString row,
which no longer fails for the c_str() it lacks.
JSON_CATCH_USER no longer wraps a catch of std::out_of_range: the last one
went away with array_t::at(). Describe what the library actually catches.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Use character literals for the signed BinaryType test
MSVC rejects char(0xFF) with C4310 (cast truncates constant value),
which the Windows workflow treats as an error. The character literals
carry the same byte values without a narrowing cast.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Do not instantiate a hash map with an incomplete basic_json in the tests
object_t is probed for key_compare inside the definition of basic_json, so
it is instantiated while basic_json is still incomplete. Whether a hash map
survives that depends on the standard library: libstdc++ 9 needs the size of
the mapped type to instantiate std::unordered_map's node type and rejects
the adapter, which broke the GCC 9 builds.
The test now derives its no-key_compare object type from std::map -- which
does cope -- and shadows the inherited key_compare member type with an
entity that is not a type, so the library's probe finds none, exactly as for
a hash map. The unflatten() order-independence checks in unit-json_pointer
already cover the behaviour that the unordered object type was there for.
The limitation is documented for std::unordered_map.
Also address two Clang-Tidy findings the earlier commits introduced:
erase_from_object() declares its iterator with auto, and at(size_type) checks
the type first and then falls through to the return instead of throwing from
an else branch.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Keep diagnostic key paths null-terminated
Building the token from data() and size() kept an embedded null byte in the
key, and since what() hands out a C string, that truncated the whole message
rather than just the key: to_bson() on a key containing U+0000 reported
"[json.exception.out_of_range.409] (/en" instead of the full explanation.
This broke test-bson under JSON_DIAGNOSTICS.
Constructing from data() alone stops at the first null byte, which is what
c_str() did before, so the message is unchanged -- without requiring
string_t to provide c_str().
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Do not parse the value in the array-at() test
JSON_DIAGNOSTIC_POSITIONS adds the byte range of the value to the exception
message, which a parsed value has and an in-memory one does not, so the two
message checks failed in that configuration. Build the array in memory
instead of parsing it; the test is about at(size_type) not needing
array_t::at(), and the byte range is beside the point.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Move the custom BinaryType tests into their own translation unit
The two sections added to unit-regression2.cpp brought a third full
basic_json instantiation into a translation unit that was already large.
With Clang on MinGW that pushed the object over the reach of a 32-bit
relocation and test-regression2_cpp20.exe failed to link:
relocation truncated to fit: IMAGE_REL_AMD64_REL32 against `.rdata'
unit-regression2.cpp is restored to exactly what it was before, and the
coverage moves to unit-custom-binary-type.cpp, next to the object and array
type tests it belongs with. The signed value type is now also covered in
C++11, where std::byte is not available.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Do not require the container iterators to be nothrow move constructible
iter_impl declared its defaulted move operations noexcept. The exception
specification a defaulted function gets implicitly follows from its members,
here internal_iterator, which holds the object and array iterators. libstdc++
gives std::deque's iterator a user-provided copy constructor without noexcept
before version 11, so the implicit specification is noexcept(false) and does
not match the declared one. That deletes the function -- and with g++ 4.8,
which predates CWG 1778, it is an error outright:
error: function 'iter_impl<basic_json<std::map, std::deque> >::iter_impl(
iter_impl&&)' defaulted on its first declaration with an
exception-specification that differs from the implicit declaration
So std::deque, which this branch documents as a usable array type, could not
be used with an older standard library. Leaving the specification to be
computed cannot mismatch; iteration_proxy_value already spells out the same
condition next door.
The default configuration is unaffected: json::iterator, json::const_iterator
and ordered_json::iterator stay nothrow move constructible and move
assignable, which the test now checks so it cannot regress unnoticed.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Address two Clang-Tidy findings the custom container tests exposed
Both come from instantiating basic_json with containers other than the
default ones, and neither shows up with the Clang-Tidy version available
outside CI:
- insert(const_iterator, basic_json&&) forwards its by-value iterator to
the const-reference overload. performance-unnecessary-value-param asks
for the copy to be a move; it only fires for an iterator that is not
trivially copyable, as std::deque's is not. The NOLINT on the function
does not cover it, because the finding is reported where the parameter
is used rather than where it is declared. Move it, which is what the
check asks for and is a (very small) improvement in its own right.
- cppcoreguidelines-use-enum-class rejects the unnamed enum that shadowed
the inherited key_compare member type. An enum class would not do, since
it declares a type of that name and the probe would find it again; a
member function declaration hides the name just as well.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Assert the iterators' exception specification relative to the container
The test pinned that nlohmann::json's iterators stay nothrow movable after
iter_impl's defaulted move operations lost their declared noexcept. That is
not a property of the library, though: the exception specification is now
computed from the container iterators, so it holds only for standard library
implementations whose iterators are themselves nothrow movable.
MSVC's checked iterators before VS2017 are not -- _Iterator_base12 registers
the iterator with the container's debug proxy in a copy constructor that
carries no noexcept -- so the assertions fail on a Visual Studio 2015 debug
build, which is the one debug configuration in the AppVeyor matrix and has no
counterpart in the GitHub Actions matrix.
Assert what the change actually guarantees instead: the iterators are nothrow
movable exactly when the object and array iterators they are built from are.
That still pins the default configuration against a silent regression, and it
is true whatever the standard library provides.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Detect a void-returning erase() through a named trait
erase_from_object() distinguished its two overloads with a decltype of a
member call written inline in a default template argument. Every other
detection in the library goes through the detector machinery in detected.hpp
instead -- has_erase_with_key_type is the same question about the same member
function -- and the inline form is the one shape older compilers are least
reliable about.
Express it the same way: detect_erase_with_iterator plus is_detected_exact,
both of which the library already relies on elsewhere. No behaviour changes.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Give the custom container types only the constructors the library uses
The three container types in the new tests inherited every constructor of
their base with using Base::Base. That asks for more than the test needs: the
library builds an object or an array by default construction, by copy or move,
and -- when converting between two basic_json types or from an initializer
list -- from an iterator range. Declaring those directly makes the requirement
visible in the test, and keeps object types out of a corner where a compiler
has to declare std::map's whole constructor set for a derived class while
basic_json is still incomplete.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Temporarily disable the new custom container tests
AppVeyor is the only CI that builds MSVC 2015 and 2017, and it has now
rejected three heads of this branch. Its build log is not reachable from
where this is being worked on, so the verdict is a single bit and the cause
has to be narrowed down by bisection.
Everything else stays: the library changes, the reduced alt_string, and the
unflatten() tests. If AppVeyor passes with these three translation units
disabled, the cause is one of the six basic_json instantiations they add; if
it fails, it is in the library. Either way this commit is reverted.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Guard the disabled tests with a macro rather than #if 0
Clang-Tidy's readability-avoid-unconditional-preprocessor-if rejects a literal
#if 0. Use a macro that is never defined instead, which the check does not
look at. Still temporary, and reverted together with the previous commit.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Re-enable the array and binary container tests
AppVeyor passed with all three new translation units disabled, so the library
changes, the reduced alt_string, and the unflatten() tests are fine on MSVC
2015 and 2017; the cause is one of the six basic_json instantiations the new
tests add.
Bring back two of the three. If AppVeyor passes again, the cause is in
unit-custom-object-type.cpp, which is the one still disabled; if it fails, it
is in one of these two and needs one more split. Still temporary.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Diagnose two silently violated template parameter requirements
Both were on the list of requirements that are not caught at compile time and
corrupt values rather than failing, and both are a plain size comparison:
- A BinaryType whose value_type is wider than one byte, which the readers and
writers reinterpret as raw bytes anyway.
- A NumberUnsignedType too narrow to hold the absolute value of every
NumberIntegerType value, which makes basic_json(INT64_MIN).dump() yield -0
for std::int64_t with std::uint32_t.
Neither static_assert rejects a configuration that worked before: both only
fire where the result was already wrong. Also add the two comments the review
asked for, in write_bson_string() and calc_bson_array_size(), matching the
ones their counterparts already carry.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Align the template parameter tables and record what is now diagnosed
Every table in the page is reformatted so each column is exactly as wide as
its widest cell, which is what the review asked for in a dozen places: the
separator rows that ran two dashes long, the stray spaces, and the columns
padded well past their content.
The row listing six containers that require a complete mapped type is split
in two so that one cell no longer sets the width of the whole table.
Content changes: NumberUnsignedType is described as any unsigned integer type
at least as wide as NumberIntegerType rather than any unsigned integer type;
the two requirements that are now static_asserts move out of the list of
violations that are not caught at compile time; and the two places that
require a non-const operator[] say why data() will not do (std::string has no
non-const data() before C++17).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Bisect the other way: only the object container tests
The previous head touched only docs/, which AppVeyor's only_commits filter
skips, so it produced no build and no status at all -- the pull request looked
green without ever having been built on MSVC 2015 or 2017.
Swap the guards instead of repeating that step: unit-custom-object-type.cpp is
enabled and the array and binary translation units are disabled. AppVeyor
already passed with all three disabled, so a failure here pins the cause on
no_key_compare_json or void_erase_json, and a pass pins it on the array or
binary file. Still temporary.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Split the two object types apart
AppVeyor failed with only unit-custom-object-type.cpp enabled and passed with
all three new translation units disabled, so the cause is one of the two
object types in this file and not the array or binary ones.
Guard out void_erase_map and leave no_key_compare_map, which separates the two
constructs under suspicion: shadowing the inherited key_compare member type
with an entity that is not a type, and hiding the inherited erase with a
void-returning overload. A failure here points at the first, a pass at the
second. Still temporary.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Build the no-key_compare object type by composition, not inheritance
The "object type without key_compare" test failed on AppVeyor's MSVC
2017 jobs (/std:c++17): its no_key_compare_map derived publicly from
std::map and shadowed the inherited key_compare type with a same-named
member function, relying on ordinary member hiding to make key_compare
unreachable as a type for the library's detection trait. MSVC 2017
does not honor that hiding for a typename-qualified lookup performed
from outside the class and still resolves key_compare to the base's
comparator type, so object_comparator_t incorrectly picked it up
instead of falling back to default_object_comparator_t.
Wrapping a std::map by composition instead removes the base class
entirely, so there is no key_compare to find under any lookup rule,
on any compiler. Also drops the now-unneeded JSON_BISECT_CUSTOM_CONTAINER_TESTS
guard left over from narrowing this down: the void_erase_map test in
the same file was never the cause and is re-enabled unconditionally.
Verified locally with clang++ and g++ under C++17 and C++20.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Re-enable the array and binary custom-container tests
unit-custom-array-type.cpp and unit-custom-binary-type.cpp were still
guarded behind JSON_BISECT_CUSTOM_CONTAINER_TESTS from bisecting the
AppVeyor failure fixed in 7c39f3227, which was unrelated to either
file. The macro was never defined, so none of these tests actually ran
in CI. Verified locally with clang++ and g++ under C++17 and C++20
before removing the guards.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix indentation of custom_object_type per astyle
The one-line function bodies in the composition-based no_key_compare_map
(7c39f3227) do not match the project's Allman brace style, which the
ci_test_amalgamation job enforces with astyle. Reformatted with the
pinned astyle 3.4.13; no functional change.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: add compiled reference implementations for the container template parameters
Each of ObjectType, ArrayType, StringType, and BinaryType now links to a
minimal, self-contained header (docs/mkdocs/docs/examples/custom_*_type.hpp)
that wraps the corresponding standard container by composition and satisfies
every "Always required" member listed on that page. Unlike the prose
requirement lists, these are real code: each header has a companion .cpp that
instantiates a basic_json specialization with it and is compiled and run by
the existing ci_test_examples check (docs/Makefile's check_output_portable),
so the reference implementations cannot silently drift from what the library
actually requires. The .output files were generated with that same target.
StringType's existing pointer to tests/src/unit-alt-string.cpp's alt_string
is kept alongside the new header as a more thorough, battle-tested example.
Verified locally: astyle (pinned 3.4.13, project .astylerc) on the new files;
clang++/g++ under C++11/17/20 for each example against the amalgamated
header; `make check_output_portable` in docs/; `mkdocs build --strict` and
scripts/check_structure.py for the page itself.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Declare no_key_compare_map's accessors noexcept
The GCC C++20 job builds with -Wnoexcept and -Werror, and the standard
library takes noexcept(c.begin()) and noexcept(c.end()) in ranges_base.h and
range_access.h. Forwarding to std::map without repeating its noexcept made
those expressions false, which the warning reports as an error:
error: noexcept-expression evaluates to 'false' because of a call to
no_key_compare_map<...>::begin() [-Werror=noexcept]
note: but ... does not throw; perhaps it should be declared 'noexcept'
Give the accessors the exception specification of what they forward to.
std::map declares begin, end, cbegin, cend, empty, size, max_size, and clear
noexcept, so the wrapper does too. swap is left alone: std::map's is only
conditionally noexcept, and nothing asks for it.
void_erase_map is unaffected because it still derives from std::map and
inherits accessors that already carry the specification.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Write out what the defaulted constructor of no_key_compare_map implied
ci_test_gcc builds with -Weffc++, which asks for data to be initialized in a
member initialization list; a defaulted default constructor does not do that:
error: 'no_key_compare_map<...>::data' should be initialized in the member
initialization list [-Werror=effc++]
Writing the constructor out satisfies that but drops the exception
specification the defaulted one carried, which -Wnoexcept then objects to
where the standard library takes noexcept(construct(...)). Declare it the way
the defaulted constructor was: noexcept when the wrapped map's default
constructor is.
This is the cost of composition -- inheritance carried std::map's exception
specifications and initialization for free, and forwarding by hand has to
restate them.
Checked with the repository's own GCC warning set from cmake/gcc_flags.cmake,
all 346 flags, at C++11, C++17 and C++20: no diagnostics for this file, nor
for the two custom container translation units that were disabled while the
MSVC failure was narrowed down and are built again now.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Mark no_key_compare_map::swap noexcept
Clang-Tidy rejects a swap that is not:
error: swap functions should be marked noexcept
[cppcoreguidelines-noexcept-swap,performance-noexcept-swap]
It was left unmarked on the grounds that std::map::swap is only
conditionally noexcept, so an unconditional promise would be wrong for a
comparator or allocator that can throw while swapping. Both concerns are met
by taking the specification from the wrapped map rather than asserting one:
noexcept(noexcept(data.swap(other.data))). Clang-Tidy accepts that, and no
NOLINT is needed.
Last in the series of specifications that inheritance used to supply and
composition has to write out by hand.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Both overloads swapped the underlying container storage but never called
set_parents(), leaving elements moved into *this with stale m_parent
pointers (typically nullptr from the free-standing array_t/object_t).
This produced wrong JSON Pointer paths in diagnostic messages and could
trip assert_invariant() on subsequent copies. Mirrors the fix already
applied in swap(reference other).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
repology.org refuses requests coming from GitHub Actions runners. The
Material privacy plugin logs the failed download as a warning, but then
raises FileNotFoundError when it reads back the cache entry it never
wrote, so ci_test_build_documentation aborts the whole documentation
build. develop and every open pull request are currently red because of
this.
The badges are fine for readers -- repology only rejects non-browser
clients -- so exclude them from the plugin's asset self-hosting and keep
them as external references. All other external assets (fonts, shields,
CDN files) are still downloaded and self-hosted as before.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
basic_json::swap() (and the friend swap() that forwards to it) only
exchanged m_data.m_type/m_data.m_value, leaving start_position/end_position
untouched under JSON_DIAGNOSTIC_POSITIONS. This is inconsistent with
copy-assignment's operator=(basic_json), which swaps positions as part of
its copy-and-swap implementation, so after swap(a, b) each value ended up
with the other value's content but its own original position.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Reserve capped array capacity for definite-length binary arrays
CBOR, MessagePack, and the optimized [$type#count UBJSON/BJData form all
pass an exact element count to sax->start_array(len), but
json_sax_dom_parser::start_array() (and the callback variant) only used
len for an overflow check against max_size() and never reserved the
underlying vector, so each element triggered a reallocation cascade via
emplace_back().
Reserve upfront, but cap the reservation at 16384 elements: max_size()
for a std::vector is far larger than any realistic input, so an
unbounded reserve(len) would let a crafted/truncated header (e.g. CBOR
0x9A + a huge uint32 count with no data) trigger a multi-gigabyte
allocation attempt instead of the normal graceful parse_error. With the
cap, a hostile length still fails fast with the existing parse_error,
while realistic arrays get a single up-front allocation.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Make the huge-claimed-length DoS regression tests portable across size_t widths
On a platform where size_t is narrower than 64 bits (e.g. 32-bit mingw/msvc
x86), the previously-hardcoded huge test lengths either collide with that
platform's unknown_size() sentinel (CBOR/MessagePack, both using exactly
SIZE_MAX) or exceed the platform's smaller vector<json>::max_size()
(UBJSON/BJData's 0x7FFFFFFF), so the header is now rejected outright
(out_of_range.408) instead of being accepted and only found short of data
(parse_error.110). Both are safe, bounded rejections of the hostile input;
the property under test -- no attempt to allocate space for billions of
elements -- holds either way. Accept both outcomes instead of pinning the
64-bit-only exact result.
Also fixed an unrelated clang-tidy finding (google-readability-casting) on
the functional-style std::size_t(...) casts in the neighboring "arrays of
various sizes" section.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix remaining CI failures in the huge-claimed-length DoS regression tests
- Apply the same google-readability-casting fix (std::size_t{N} instead
of std::size_t(N)) to the "arrays of various sizes" section in
unit-msgpack.cpp, unit-ubjson.cpp, and unit-bjdata.cpp; only
unit-cbor.cpp had been fixed previously, since clang-tidy's build
didn't get far enough to report the other three in the same pass.
- json_sax_dom_parser::start_array()'s max_size() check calls JSON_THROW
directly rather than going through sax->parse_error(), so unlike the
scanner's own "not enough data" parse_error it is not gated by
allow_exceptions=false. On a platform where a header's claimed count
exceeds max_size() (e.g. 32-bit, for UBJSON/BJData's 0x7FFFFFFF test
value), from_ubjson/from_bjdata(input, true, false) can therefore still
throw instead of returning a discarded value. Make that assertion
tolerant of either outcome, same as the main exception-catching check
above it.
- Guard all four "a huge claimed length..." SECTIONs with
#if !defined(JSON_NOEXCEPTION), matching this test suite's existing
convention for exception-dependent tests: under JSON_NOEXCEPTION,
JSON_THROW never produces a catchable C++ exception at all (it aborts
the process), so a section that relies on try/catch to distinguish
between two acceptable outcomes cannot be expressed under that build
configuration regardless of platform.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Use (std::min)(len, reserve_cap) instead of a ternary in start_array()
Addresses review feedback from @gregmarr on PR #5476.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Discard dump()'s [[nodiscard]] return value in an exception-only check
CHECK_THROWS_WITH_AS(j.dump(), ...) called dump() only to trigger and
catch the exception, but never used the return value. dump() is
warn_unused_result, so GCC's pedantic build (-Werror --all-warnings)
rejected it as -Werror=unused-result, breaking ci_test_gcc. Wrapped in
utils::ignore_return_value(), matching every other such call in this
file.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Mark container_frame top as const in CBOR/UBJSON readers
clang-tidy's misc-const-correctness flagged these on PR #5520's CI: the
BSON sibling copy was already const, but these two were left mutable
even though only container_stack.back().remaining is ever written.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The benchmark suite covered parsing JSON text, dumping, and serializing to
CBOR, but only one binary read: FromMsgpack. Nothing measured from_cbor,
from_ubjson, from_bjdata or from_bson, so a change to binary_reader.hpp had
no baseline to be compared against.
Add read benchmarks for every format, in the two shapes that matter: from a
contiguous buffer, which is what most callers pass, and from a FILE*, which
is what FromMsgpack already measures and which compiles to different code.
FromMsgpack itself is left untouched so its numbers stay comparable across
releases. The input is derived at setup time by serializing a parsed test
file, because the test data repository ships JSON only.
The test files are wide and shallow, but the readers' cost is per container,
so add three value shapes they do not cover -- deeply nested containers, many
sibling containers, and one flat array of numbers -- plus an indefinite-length
CBOR string, a form the writer never emits and which therefore has to be
assembled by hand. UBJSON and BJData are also captured in their size- and
type-annotated form, which the readers handle in a separate code path.
BSON requires an object at the top level, so it cannot reuse the array-rooted
test files; it is captured on the object-rooted ones, and the shapes are
wrapped in an object so every format measures the same value. The setup marks
the benchmark as skipped rather than letting the exception escape if that
requirement is ever violated.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Read BSON documents without recursing per nesting level
An embedded document (record type 0x03) or array (0x04) was read by calling
back into the document reader, which read its element list, which called the
element reader again for the next embedded one. The native call stack
therefore grew with the nesting depth of the input, and about seven bytes buy
a level, so a document of a few hundred kilobytes crashes the process
(#5104). This is the last of the four binary formats to still do that.
Apply the same shape as the other three: open_bson_document() reads the size
prefix and opens the document, parse_bson_element_internal() calls it for both
record types instead of recursing, and parse_bson_internal() loops over the
element list of whichever document is innermost, closing it when its
terminator is reached and resuming the one below.
check_bson_document_size() is unchanged, and so is when it runs: a document is
still measured from the byte before its size prefix to the byte after its
terminator, and still reported before the end event. The frame carries those
two values, which is what a per-document check needs once the reads are
interleaved rather than nested. Nothing else about the element reader changes.
unit-bson passes unchanged. Round trips through to_bson of nested objects,
arrays, arrays of objects and mixed nesting are identical to the previous
commit, as are the errors for a truncated document, an unsupported record
type, a negative size and a size that does not match, including their byte
offsets. A 30,000-level document built by to_bson is now read to completion
where it used to crash.
Note for sequencing: #5185 changes parse_bson_internal(), the element list and
the array reader, which are the functions this commit restructures. It should
land first; this commit then keeps its checks and moves them onto the loop.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Make parse_bson_internal's end-of-document top a copy, not a reference
Same issue as the CBOR and UBJSON/BJData readers: top aliased
container_stack.back() and was read (top.is_object) right after
container_stack.pop_back() ended its lifetime. A copy stays valid
regardless of what happens to the stack; nothing here mutates the live
entry, so no field needs to go through container_stack.back() directly.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Read UBJSON and BJData containers without recursing per nesting level
get_ubjson_array() and get_ubjson_object() read their elements by calling
back into the value reader, which called them again for a nested container,
so the native call stack grew with the nesting depth of the input. '[' alone
opens a container, so half a million of them crashes the process before the
input runs out (#5104). The optimized forms reach the same path through a
size or type annotation, and in plain UBJSON '[' and '{' are permitted as the
type of an optimized container, so "[$[#i\x01" repeated nests just as deeply
at six bytes a level.
Both readers now only open their container, and parse_ubjson_internal() loops:
it closes the containers that have ended, claims the next element of the
innermost one, reads its key when it is an object, and works out the marker
of the value to read next. That last part is where the formats differ, and
the loop follows what the four element loops used to do:
- a sized, typed container gives its elements no marker of their own
- a sized, untyped container reads one for each element
- a container that ends at a marker has the byte already, from the test
against ']' or '}'; for an object it is the first byte of the key
The ND-array wrapper and the 'B' binary shortcut stay as they are. Both read
a complete value rather than opening a container, and their elements are
always scalars: BJData does not permit '[' or '{' as an optimized type, which
is also why only plain UBJSON needed the type-marker case above.
A container of no-ops keeps its behaviour of holding no elements while still
announcing its declared size to the SAX parser, by opening it and then
setting its count to zero.
unit-ubjson and unit-bjdata pass unchanged, 1.39 million assertions between
them, and a behaviour comparison against the previous commit over every
container form -- sized, unsized, typed, untyped, empty, no-op, ND-array,
binary, and the forms nested inside one another -- gives identical values,
error codes, messages and byte offsets. 500,000 levels of each vector now
report a parse error instead of crashing, and a well-formed 100,000-level
value is read to completion.
The driver costs about 3 % on parsing 60,000 small objects and one array of a
million integers, for the reason given in the previous commit; reading the
frame once per element rather than per branch halved what it cost before.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Make the UBJSON/BJData advance loop's top a copy, not a reference
Same issue as the CBOR reader: top aliased container_stack.back() and
was read (top.is_object) right after container_stack.pop_back() ended
its lifetime. A copy stays valid regardless of what happens to the
stack; the one place that mutates the live entry (--top.remaining) now
goes through container_stack.back() directly.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Read CBOR containers and tags without recursing per nesting level
get_cbor_array() and get_cbor_object() read their elements by calling back
into the value reader, which called them again for a nested container, and a
tag was handled by reading the tagged value the same way. All three cost
native stack, and all three cost a single byte to encode: 0x9F opens an
indefinite-length array, 0x81 a one-element array, and 0xC2 is a tag. Half a
million of any of them crashes the process before the input runs out (#5104).
Apply the shape the MessagePack reader already uses: the open containers live
on the heap stack, parse_cbor_value() reads a single value and only opens a
container rather than reading it to its end, and parse_cbor_internal() loops,
resuming the innermost container after each element.
Two things are specific to CBOR. An indefinite-length container ends at a
break marker rather than at a count, and testing for that marker consumes a
byte which is the first byte of the next element when it is not one; the
frame's count is npos for those, and the driver tracks whether the next value
starts at a fresh byte. And a tag is not a value of its own: instead of
reading the tagged value by recursing, the value reader reports that a tag was
read and the driver reads on, so a chain of tags costs no stack at all.
The switch that decodes a value is unchanged apart from the twelve container
cases and the two tag sites. Verified against the previous commit over
definite and indefinite arrays and maps, all four counted forms, empty
containers, nesting of the forms inside each other, truncated inputs, and all
three tag handlers: identical values, error codes, messages and byte offsets.
500,000 levels of each of the three vectors now report parse_error.110 instead
of crashing, and a well-formed 200,000-level value is read to completion.
On performance: the driver does per element what a counted loop used to do
per container, and CBOR pays for it more than MessagePack because the value
reader also has to be told whether to fetch a byte. Parsing 60,000 small
objects and one array of a million integers is 3 to 4 % slower than the
recursive reader, measured over five alternating runs. Against develop the
same two inputs are about 44 % faster, because the entry point no longer
copies the value it parsed; the earlier commit in this series is what pays
for that.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Make parse_cbor_internal's top a copy so it survives pop_back()
top aliased container_stack.back(), and was still read (top.is_object)
right after container_stack.pop_back() destroyed the element it aliased.
Nothing currently reorders those two lines, but the comment claiming the
reference's lifetime was already fine only accounted for reallocation
from a push, not this. A trivially-copyable container_frame makes top a
copy instead, so reads of it stay valid regardless of what happens to the
stack; the one place that mutates the live entry now does so through
container_stack.back() directly rather than through top.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
get_msgpack_array() and get_msgpack_object() read their elements by calling
back into parse_msgpack_internal(), which calls them again for a nested
container. The native call stack therefore grew with the nesting depth of the
input, and each level costs only one byte to encode: 0x91 is a one-element
array, so a few hundred thousand of them crash the process before any of the
input is rejected (#5104).
Keep the open containers on a heap stack instead, the way
parser::sax_parse_internal() has always done for JSON text. A frame records
how many elements are left and whether to close with end_object() or
end_array(); parse_msgpack_value() reads a single value and, for a container,
only opens it; and parse_msgpack_internal() loops, resuming the innermost
container after each element and closing it when its count runs out. Whether
the value that was begun is complete is answered by the stack being empty, so
no separate bookkeeping is needed.
The switch that decodes a value is untouched apart from the six container
cases, which now call enter_container() rather than a reader that loops. That
keeps this diff to the control flow and leaves the decoding of every other
type byte-identical.
enter_container() is the only place a binary reader emits start_object() or
start_array(), so a check that rejects a container can be added there once and
is guaranteed to run before the start event. The frame type and the stack are
shared, ready for the other three formats.
Verified against develop over empty, nested, counted (array 16/32, map 16/32)
and truncated inputs: identical values, error codes, messages and byte
offsets. 300,000 levels now report parse_error.110 instead of crashing, and a
well-formed 300,000-level value is read to completion through the SAX
interface, where develop crashes.
Reading such a value into a basic_json needs the return-by-move change as
well, without which the recursive copy constructor overflows on the way out;
that is the parent commit, and the test for the value path covers the two
together. Timing is unchanged: parsing 60,000 small objects and one array of
a million integers is within run-to-run noise of develop either way.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Linking test-regression2 with clang and MinGW fails with
relocation truncated to fit: IMAGE_REL_AMD64_REL32 against `.rdata'
once the translation unit grows past a certain size: the code can no longer
reach the read-only data it references within the range of a 32-bit
relocation. The file is one of the largest in the test suite and had been
sitting just under that limit, so an unrelated change elsewhere in the
library is enough to tip it over. It is already the second such file --
unit-regression1.cpp was split for size before -- and windows.yml already
carries a workaround for the same limit hitting the debug sections of this
same target, where -g0 was enough because that relocation was against
`.debug_line'. This one is against `.rdata', which no compiler flag avoids.
Move the second half of the regression tests, and the helper types only they
use, into unit-regression3.cpp. The sections are independent -- every
statement in "regression tests 2" was already inside a SECTION -- so they
move unchanged, and the counts confirm nothing was lost: 168 assertions
before the split, 50 plus 118 after.
The result is that both files are comfortably smaller than the one that used
to link, measured with clang at -O1 for C++20:
read-only data text object
before 58,233 1,287,764 3,158,120
unit-regression2.cpp 48,161 1,012,988 2,522,296
unit-regression3.cpp 41,710 772,704 1,878,880
No CMake change is needed: tests/CMakeLists.txt globs src/unit-*.cpp, so the
new file is picked up and built for every standard like its siblings.
CONTRIBUTING.md pointed contributors at unit-regression2.cpp for new bug
tests; it now points at the smaller file and says why the two exist, so the
split does not quietly undo itself.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Bound UBJSON optimized arrays of a valueless type
An element of type 'Z' (null), 'T' (true) or 'F' (false) is encoded by its
type marker alone, so an optimized UBJSON array of one of those has no
payload: reading an element consumes no input at all. Its declared count is
therefore the only thing that decides how much is allocated, and nothing
bounded it. "[$Z#l" and a four-byte count is nine bytes of input describing
two billion values; #2793 reports 35 GB and 150 seconds from ten bytes, and
OSS-Fuzz has an out-of-memory and a timeout report for the same shape.
Every other type costs at least one byte per element, so the end of the input
bounds it. 'N' (no-op) is already skipped rather than stored. Objects are not
affected either: each element is preceded by its key, which costs bytes. And
BJData already refuses these markers as an optimized type, so this is a plain
UBJSON matter.
Reject a count above 1,048,576 elements for those three types with
out_of_range.408, the code this reader already uses for a declared size it
will not honour. The check runs before the SAX start event, so no container
is opened and then abandoned.
Rejecting on the read side alone would break the guarantee that anything
to_ubjson() writes can be read back, and would trip the round-trip assertion
in fuzzer-parse_ubjson.cpp. So the writer falls back to the unoptimized
encoding, one byte per element, for arrays of these types above the same
limit. Its decision depends only on the array's size, which is identical for
a value and for anything parsed back from it, so the round trip is stable.
No existing test changes: the largest such count in the test suite is 65,793.
The excessive-size test that already used this shape still passes, now
rejected a little earlier than by the max_size() check it used to reach.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Note the 1,048,576 valueless-array limit as (1 << 20) in the docs
Addresses review feedback from @gregmarr on PR #5504: spell out the
binary/hex form next to the decimal count so it reads as the round
power-of-two it is, matching how include/nlohmann/detail/input/binary_reader.hpp
defines max_valueless_container_size. Applied in both docs/exceptions.md
and ubjson.md, as requested.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
get_ubjson_size_type() takes an inside_ndarray parameter saying whether it is
being called for an ndarray's dimension vector, where another ndarray is not
allowed. It then seeded the flag it passes down to get_ubjson_size_value()
with `false` rather than with that parameter, and only consulted
inside_ndarray afterwards, on the '$' branch.
So on the '#' branch nothing stopped the descent: every "#[" pair of an input
like "[" followed by "#[#[#[..." opened another dimension vector, several
native stack frames deeper each time, and the recursion was only reported on
the way back out. 100,000 pairs crash the process. This is #5104 again, in a
path that has nothing to do with containers.
Seed the flag with inside_ndarray, which is what get_ubjson_size_value()
documents it wants: "for input, `true` means already inside an ndarray vector
or ndarray dimension is not allowed". The nested '[' is then refused where it
is read, so the length of the chain no longer matters.
Both post-checks gain `&& !inside_ndarray`, because an ndarray was found
*here* only if the flag flipped -- get_ubjson_size_value() only ever returns
`true` when its initial value was `false`, as its documentation says. With
that, the "ndarray can not be recursive" branch is unreachable: a recursive
ndarray is now caught one level earlier, and reported as "ndarray dimensional
vector is not allowed" like every other nested dimension vector.
Three existing expectations move accordingly (vR2, vR4, vR6). All three now
fail earlier, and all three now report the same error that vR1, vR5 and vH
already reported for the same shape, which is the more consistent outcome.
Everything else is unchanged: valid 1D and 2D ndarrays, optimized containers
and plain arrays produce identical results, and unit-ubjson is untouched.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
get_cbor_string() and get_cbor_binary() handled the indefinite-length forms
(0x7F and 0x5F) by calling themselves once per chunk. Each chunk therefore
cost a native stack frame, and since a chunk may itself be an indefinite-
length string, an input of repeated 0x7F bytes reached one frame per input
byte: 200,000 of them crash the process with SIGSEGV before a single byte is
rejected. This is the same defect as #5104, in a path the container-level
work does not touch.
Count the open levels instead of recursing through them. That is enough here
because every chunk is appended to the same result -- get_bytes() writes at
result.size() -- so there is no per-level state to keep. The temporary chunk
string and its copy into the result go away with the recursion.
The definite-length cases move to get_cbor_string_chunk() and
get_cbor_binary_chunk() unchanged, including their error messages, which
still name 0x7F and 0x5F because those are handled one level up.
Behaviour is unchanged. Comparing against develop over the interesting byte
sequences -- empty, single-chunk, nested, over-closed and truncated forms,
both strings and byte arrays, and an indefinite-length map key -- produces
identical values, error codes, messages and byte offsets. The 200,000-level
input now reports parse_error.110 at byte 200001 instead of crashing.
Note that nesting these is not valid CBOR: RFC 8949, Section 3.2.3 forbids
it. This does not change that either way -- it has always been accepted, and
rejecting it is a separate decision (#5317, #5325). Should it be rejected
later, that is now one condition on the level counter rather than a change to
the control flow.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Move the scanned string into the value instead of copying it
The SAX interface documents that the string handed to json_sax::string() may
be moved from, and the DOM handlers already move the one handed to binary().
string() did not, so every string value was copy-constructed out of the
lexer's token buffer, which then kept the buffer alive at its high-water mark
until the next token overwrote it.
Moving hands that buffer to the new value instead. The allocation count is
unchanged - the value needed one either way - but the copy is gone.
jeopardy 247.3 ms -> 240.9 ms (-2.6%)
citm_catalog 4.61 ms -> 4.48 ms (-2.8%)
40k 30-char strings 7.80 ms -> 7.64 ms (-2.1%)
Note this deliberately does not extend to the object key. Moving the key
hands the lexer's buffer - sized for the largest token seen so far - to a key
that is usually short, so the next value has to grow a fresh buffer. Measured,
that costs 11.9% on a document of many small keys with longer values.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Spell out the move rationale at every handle_value(std::move) site
The comment explaining why the value is moved sat only on
json_sax_dom_parser::string(), and the callback parser's string() pointed
at it with "see json_sax_dom_parser::string()". That reference cannot be
searched for - the function is declared as `bool string(string_t& val)`
inside the class, so the qualified name appears nowhere - and the two
binary() overloads, which have always moved, carried no explanation at all.
Put the same comment on all four sites and name json_sax, which is
greppable, instead of a member that is not.
Comment-only; no generated code changes.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
When a parser callback rejects a value, the placeholder stored for it has to
be removed from its parent again. remove_discarded_value() found it by
scanning the parent from the beginning, so filtering a container cost one
scan per rejected member - quadratic in the number of members of a single
container.
A rejected value can only ever be the one most recently added to its parent:
the last element of an array, or the placeholder key() stored under the
current key in an object. Record that key alongside the existing
key_keep_stack, and for a container record it again alongside ref_stack so
end_object()/end_array() can find it in the parent. Removal is then O(1) for
an array and O(log n) for an object, and finding nothing there means nothing
was stored, so there is nothing to remove.
The key for a container is read before handle_value() may consume it, so it
is also correct when the callback rejects the container at its start event
and it never reaches its parent at all.
Discarding half the members of one object, before -> after:
members value rejected container rejected at start
16 000 392 ms -> 3.7 ms 803 ms -> 8.2 ms
64 000 6238 ms -> 14.6 ms 12651 ms -> 32.2 ms
128 000 25339 ms -> 30.6 ms
Results are unchanged: 48 000 randomized documents parsed under 12 different
filtering callbacks - covering duplicate keys, empty keys, rejected keys and
containers rejected at both their start and end events - produce byte
identical output before and after.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Every input in the benchmark corpus is minified or only lightly spaced, so
none of them exercise the lexer's whitespace handling. Real-world JSON is
frequently indented - configuration files, pretty-printed API responses,
anything kept under version control - where the insignificant whitespace can
outweigh the data itself.
Add a ParseIndented family that re-serializes each document with an
indentation and parses that. The content is identical to the matching
ParseString row, so the pair isolates the cost of the whitespace alone.
Measured here, parsing the indented form costs 16-34% more than the minified
form of the same document, which nothing in the suite currently reports.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Stop dump() from heap-allocating its output adapter per call
The serializer held its output sink as output_adapter_t<char>
(a std::shared_ptr<output_adapter_protocol<char>>), which dump() and
operator<< built via make_shared -- one heap allocation per call for a
sink that only wraps a reference to the caller's string or stream.
Hold the sink as a non-owning output_adapter_protocol<char>* instead and
construct the concrete adapter on the stack at the call site. The write
path (o->write_characters) is unchanged, so output is byte-for-byte
identical; a compact dump() of a small object drops from 2 heap
allocations to 1 (only the returned string remains), ~3% faster.
Completes the per-call allocation cleanup on this branch, which already
removed the indent_string buffer (both were reported in #5413).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L1oJ2ggRHS37zeVe94QTA1
Signed-off-by: Claude <noreply@anthropic.com>
* Take the output adapter by reference at the serializer ctor
Per review: the serializer still holds the adapter as a non-owning
pointer, but the constructor now takes output_adapter_protocol<char>&
and takes its address internally, so every call site passes a
reference. A reference cannot be null and reads as a borrow, which
makes the lifetime contract harder to get wrong than handing over a
raw pointer. The stored member and the write path are unchanged.
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Claude <noreply@anthropic.com>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude <noreply@anthropic.com>
* Add SWAR bulk fast path to string serialization (dump_escaped)
When ensure_ascii is false, dump_escaped previously ran every byte of
every string and object key through the UTF-8 DFA decoder, even for the
common case of ordinary text with nothing to escape. This mirrors the
per-byte cost the parser had before the contiguous fast paths.
At a character boundary, bulk-copy the longest run of bytes that need no
escaping using string_bulk_run() - the same SWAR scanner and UTF-8 bulk
validator the lexer's contiguous path uses - and only fall back to the
byte-at-a-time DFA loop for the first byte that needs individual handling
(a quote, backslash, control character, or ill-formed/truncated UTF-8).
Because every "hard" or invalid byte is still processed by the unchanged
byte path, escaping output and error handling (including strict-mode
error 316 position and message) are byte-identical to before.
The ensure_ascii=true path is unchanged: it must escape non-ASCII and
0x7F, which string_bulk_run does not stop on, so a separate predicate
would be needed for it.
Verified byte-for-byte identical dump output against the pre-change
implementation across ~20k randomized byte strings plus curated edge
cases (all escapes, control chars, valid multibyte, surrogates,
overlong, truncated sequences) for both ensure_ascii settings and all
three error handlers, in C++11/17/20 at -O2/-O3.
Throughput (g++ -O3, ensure_ascii=false, vs pre-change):
long ASCII strings 4.2x
twitter-like objects 2.3x
dense CJK 1.4x (further headroom with JSON_USE_SIMDUTF)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Buffer serializer output and add ensure_ascii string fast path
Two further serialization speedups on top of the ensure_ascii=false bulk
copy, both reusing the SWAR primitives in detail/input/string_scan.hpp.
1. Internal write buffer (devirtualization). Every structural character
('{', '"', ',', ...) previously went straight to the output adapter
through a virtual call. Route all writes through put_char/put_chars
into a 1 KiB buffer that flushes in bulk; the public dump() flushes
once the top-level value is done (the recursive worker is split out as
dump_internal). Runs larger than the buffer are written straight
through, so large payloads are not copied twice. This is the dominant
cost for object/array-heavy values.
2. ensure_ascii fast path. dump_escaped previously ran the UTF-8 DFA over
every byte when escaping non-ASCII. Add find_ascii_copyable_run() (a
SWAR scan stopping at '"', '\\', < 0x20, 0x7F, and >= 0x80) so runs of
printable ASCII are bulk-copied, with the byte path handling each
escape/non-ASCII byte exactly as before.
Behavior is unchanged: dump output is byte-for-byte identical to the
previous implementation across ~20k randomized byte strings plus curated
edge cases (all escapes, control chars, 0x7F, valid multibyte,
surrogates, overlong, truncated), for object/array/pretty output, both
ensure_ascii settings, and all three error handlers, in C++11/17/20 at
-O2/-O3. New unit tests cover the buffer flush boundaries, the escape and
0x7F handling, multibyte under both settings, and invalid-UTF-8 handling.
Throughput (g++ -O3, vs the ensure_ascii=false-only baseline):
long ASCII, ensure_ascii=0 4.2x
long ASCII, ensure_ascii=1 4.1x
twitter-like objects 2.7x
dense CJK 1.8x
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Flush serializer buffer in dump_escaped unit test
test-convenience failed (macOS finished first; the failure is
platform-independent) because check_escaped() calls the internal
serializer::dump_escaped() directly and then reads the output stream.
Since dump_escaped() now writes into the serializer's internal write
buffer, the bytes were still buffered and the stream was empty.
Expose flush() under JSON_PRIVATE_UNLESS_TESTED (same visibility as
dump_escaped) and flush in check_escaped() before inspecting the output.
Per-string flushing inside dump_escaped() was rejected on purpose: it
would defeat the buffering that makes object/array-heavy dumps faster.
Library behavior is unchanged (flush()'s body is identical; only its
access label moved).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Avoid deep recursion in serialization write-buffer test
The "many small structural writes exceed the write buffer" subcase built
a 1100-deep nested array and dumped it to force >1024 consecutive
single-character writes through put_char (exercising the write buffer's
flush-when-full branch). dump() recurses per nesting level, so on MSVC
debug builds (smaller default stack, larger frames) this overflowed the
stack and crashed test-serialization; Linux/macOS have enough headroom to
hide it.
Replace the nesting with a flat array of 500 empty strings. Each element
emits '"', '"', ',' via put_char, so the dump is a long run of
single-character writes (1501 bytes > the 1024-byte buffer) at nesting
depth two, hitting the same flush branch without deep recursion. Library
code is unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Split the write-buffer helpers and write indentation directly
Follow-up to @gregmarr's review: put_chars() was doing four unrelated jobs, so
give the two that can be made safe their own entry points.
- put_literal(): takes the literal by reference and deduces the length from the
array bound, so the 27 hand-counted lengths at the call sites can no longer
drift from the literals they describe. A literal is checked at compile time to
fit the buffer, so this path needs no write-through branch.
- put_buffer(): takes the fixed-size buffer itself rather than a bare pointer,
so the length can be checked against the buffer's own bound.
- put_indent(): memsets the indentation into the write buffer, filling and
flushing it as needed. This removes indent_string entirely, and with it both
bugs of #5186: the indentation string was grown by doubling, which is not
enough when indent_step more than doubles it (a heap over-read - dump(2000)
read 2000 bytes out of a 1024-byte string), and the grown part was filled with
a space instead of the configured indent_char. next_indent() keeps that PR's
assertion against the unsigned indentation accumulation wrapping on deep
nesting.
put_chars() keeps the two cases that are genuinely a pointer and a count: the
run-length copies out of the string being escaped, and to_chars() output.
Tests cover an indent_step wider than the write buffer, a non-space indentation
character past the old growth point, and nesting whose accumulated indentation
spans several buffer-fulls. All three fail against develop.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fill the indentation buffer once instead of once per flush
@gregmarr's point on the fill-and-flush loop: flushing does not disturb what
the write buffer holds, so an indentation spanning several buffer-fulls only
has to be written into the buffer once and can then be handed to the adapter
as many times as needed. The loop re-filled it every time, doing work it
already knew was there.
put_indent() now fills the room left in the buffer, and if anything remains,
flushes, fills the buffer once, and re-flushes that same content. It also
returns early for a zero-width indentation, which is what the closing brace of
every outermost value asks for.
Measured over a dump(), counting memset calls and bytes inside put_indent:
indent before after
4 1 call / 4 B 1 call / 4 B
2000 2 calls / 2000 B 2 calls / 2046 B
100000 98 calls / 100000 B 2 calls / 2046 B
The wide case is now constant work rather than proportional to the indentation
width; ordinary widths are unchanged. Tests extended to cover several whole
buffer-fulls and an exact multiple of the buffer size.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Tighten the write-buffer helpers after review
More of @gregmarr's review on the put_* split:
- Reattach the put_chars() doc comment, which the new helpers had been
inserted in front of, leaving it describing put_indent().
- Compute the literal length once in put_literal() instead of spelling N - 1
at each use.
- Add put_string(str, start, end), which keeps the pointer arithmetic and the
bounds assertions inside the function instead of at the call site. With
dump_float()'s to_chars() output moved onto put_buffer() as well, put_chars()
now has no callers outside put_string()/put_buffer(): nothing passes a bare
pointer and a count any more.
- Carry the indentation as std::size_t rather than unsigned int. It is a size,
it is compared and combined with buffer sizes throughout, and the casts in
put_indent() disappear. next_indent() keeps its assertion, which is far
harder to trip on a 64-bit size_t but still reachable where that is 32 bits.
No output change: pretty and compact dumps, binary values included, are
byte-identical to develop.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Silence avoid-c-arrays on put_literal's array reference
clang-tidy flags the reference-to-array parameter under
cppcoreguidelines/hicpp/modernize-avoid-c-arrays, and the CI treats warnings as
errors. Binding to the array is the whole point here - it is what lets the
length be deduced from the literal instead of hand-written at the call site - so
suppress it the same way from_json(), to_json() and get_to() already suppress it
for their own T (&arr)[N] parameters.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Bound the descent of dump()
Serializing a container serializes its elements, so dump() descended into one
call per nesting level. A value nested deeply enough exhausted the call stack
and terminated the process with a segmentation fault - no exception, nothing
the caller could catch. Parsing such a value works, as the parser is
iterative, and so does destroying one, as #1436 made destruction iterative.
Bound how far the descent goes rather than take the call stack away from it.
The first 128 levels are written by exactly the code that always wrote them,
and only below that does dump_iteratively write out what is left, keeping the
containers it has entered on an explicit stack. Serializing can therefore no
longer exhaust the stack, however deeply a value is nested, while a value
nested less deeply than the bound pays only for one comparison per container.
Writing every value that way instead measured between 2% and 20% slower - 20%
on object-heavy documents - which is why the descent is kept for all but the
values that cannot afford it. The bound costs nothing measurable: between
-1.4% and +1.2% across compact and pretty output of number, integer, string,
object-heavy, wide-object and deeply nested documents.
The output is unchanged for every value. Both ways of writing a container
emit the separator in front of every element but the first, rather than
after every element but the last, which puts exactly one between each pair
and none at the end.
This fixes#5387 for dump(). The copy constructor is fixed in #5389.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fold ensure_ascii into the escaper and write bytes without dump_integer
Two hot spots that the write buffer and the bulk scanner left behind.
dump_escaped took ensure_ascii as a runtime flag and tested it inside the
loop, once per character run, although it cannot change while a string is
written. It is now a template parameter, dispatched once per string, which
folds the choice of scanner and lets each of the two be inlined into a loop
of its own. This is the hottest loop in the serializer: it runs over every
string and every object key.
A binary value's bytes went through dump_integer, which counts digits and
does 64-bit arithmetic for a number that is always in [0, 255]. dump_byte
writes the three digits it takes at most straight into the write buffer
instead. Any byte type that is not a plain unsigned byte is still left to
dump_integer, whose representation of it may differ.
Measured against the previous commit (medians of 9 interleaved runs, clang
-O3): binary values -33.8%, dense CJK with ensure_ascii -20.6%, key-heavy
objects -17.8%, deeply nested pretty output -17.9%, dense CJK without
ensure_ascii -11.8%, object-heavy documents -9.3% compact and -9.5% pretty,
a small value dumped in a loop -21.4%, wide objects -2.3%. Arrays of plain
ASCII strings measured 3.5% to 4.2% slower, the one shape that loses; number
and integer arrays are unchanged.
Also tried and dropped: leaving the write and string buffers uninitialized
rather than zeroing 1.5 KB per dump() call. It is worth -30% on small values,
but two nearly identical string workloads moved 18% apart in opposite
directions, so the measurements did not support it.
The output is unchanged for every value: the differential now also covers
every one of the 256 byte values, alone and together, in both binary layouts.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Write a byte without walking a pointer over the buffer
clang-tidy's misc-const-correctness reads the pointer dump_byte advanced over
the write buffer as one whose pointee could be const. Index the buffer
instead, which says the same thing without a raw pointer at all.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Parenthesize the reserve arithmetic in the deep-nesting test
clang-tidy's readability-math-missing-parentheses wants the multiplication
spelled out in reserve(6 * depth + 1), and CI treats its warnings as errors.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Do not scan for a copyable run that cannot exist
Under ensure_ascii, dump_escaped() calls find_ascii_copyable_run() at every
character boundary. When the text is dense non-ASCII - CJK, where every byte
is >= 0x80 - the scanner stops on its first byte and returns zero, so its SWAR
block runs once per character and buys nothing, on top of the escaping that
still has to happen afterwards.
A run can only be non-empty when the first byte is one the scanner may copy,
so test that single byte before calling it. Runs that do exist are found
exactly as before, so the bulk-copy win is unchanged; only the calls that were
always going to return zero are skipped.
Output is unchanged: the dump digest over canada/citm/twitter, in compact,
pretty and ensure_ascii form, matches develop byte for byte.
dump(ensure_ascii=true) develop before after
CJK text 3.54ms 4.25ms 3.36ms
CJK, no ASCII at all 3.09ms 4.02ms 3.02ms
Latin-1-ish text 4.39ms 3.04ms 2.93ms
plain ASCII 3.92ms 0.80ms 0.79ms
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Address review of the write-buffer helpers
Three points from @gregmarr's review:
put_chars() is gone. It was the only entry point taking a bare pointer and
a count, and it existed only so put_string() and put_buffer() had something
to delegate to. Its body now lives in put_string(), and put_buffer() is
put_string(buffer, 0, length) - std::array already carries data() and
size(), so it satisfies the same interface a string does. Nothing appends
characters without a bound any more.
dump_escaped()'s documentation block was duplicated. The dispatcher was
inserted between the original comment and the function it described, and
the comment was copied rather than split. The worker now has its own short
comment saying why ensure_ascii is a template parameter.
The local in dump_byte() is deliberate, and is now documented as such:
writing through write_buffer[] is a char write, which may alias any object,
so with write_buffer_pos updated in place the compiler must reload and
store it around every digit. Measured on a dump of a 4 MiB binary value,
18.0 ms without the local against 7.4 ms with it.
Output is unchanged: byte-identical dumps across 77 files in compact,
pretty, ensure_ascii, pretty+ascii, indent 600 and tab-indent form.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Address review: drop unneeded backslash-escapes and duplicate scan loop
'"' does not need escaping in a char literal, unlike in a string literal.
find_ascii_copyable_run() also duplicated the byte-at-a-time search that
already exists as the loop's own scalar tail; break into it instead of
re-deriving the offset in a second, near-identical loop.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Move pretty_print, ensure_ascii and indent_step into the serializer
None of these change over the life of a serializer, unlike current_indent
and depth, which do change on every recursive call. They are now captured
once in the constructor - matching indent_char and error_handler - instead
of being threaded through dump(), dump_internal(), dump_iteratively(),
dump_value() and dump_escaped() on every call.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Stop the serializer from holding onto std::localeconv()'s pointer
loc was only ever read twice, immediately, to seed thousands_sep and
decimal_point; nothing else in the class used it. A local in the
constructor body serves the same purpose without keeping the pointer
around for the serializer's lifetime.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Keep thousands_sep/decimal_point const via a small locale_chars struct
const members can't be assigned in a constructor body, so seeding them
from std::localeconv() meant either dropping const or holding onto the
lconv* for longer than needed. A sub-object computes both from the
pointer in its own constructor and is itself initialized in serializer's
mem-initializer-list, so the two chars stay const, std::localeconv() is
still called exactly once, and nothing outlives the constructor.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
tests/src/unit-class_parser_diagnostic_positions.cpp and
tests/src/unit-diagnostic-positions-only.cpp were maintained as near-copies
of unit-class_parser.cpp and unit-diagnostic-positions.cpp respectively, and
had drifted: trailing-comma handling, the #5342 filter-array/filter-value
sections, and the cross-input-adapter diagnostics test were never ported to
the positions-enabled copy.
Fold the position-specific assertions into the base files, guarded by
file a second time with the relevant macro set via CMake COMPILE_DEFINITIONS
(mirroring the existing test-comparison_legacy pattern) instead of
maintaining a separate source file. This removes the duplication and, as a
side effect, closes the coverage gaps above since the full test file now
compiles under JSON_DIAGNOSTIC_POSITIONS=1 as well.
Fixes#5417
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
write_bjdata_ndarray() validated that each _ArrayData_ element matched
the number kind (integer vs. float) named by _ArrayType_, but not its
range. An element that did not fit the target C++ type (e.g. 256 for
"uint8") was silently wrapped by the static_cast used to write it, or,
for "single", silently overflowed to infinity.
Range-check each element against the type named by _ArrayType_ before
writing it, reusing the existing fallback path that already encodes
the annotated object as a plain object for other invalid-annotation
cases in this function.
Fixes#5403.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Speed up number parsing in the lexer (fast paths from the fast_float/simdjson world)
The number scanner converted its already-validated digit buffer with
std::strtoull/std::strtoll/std::strtod. Those pull in locale and errno
machinery and dominate number-heavy parsing (strtod runs at ~6 M/s).
Replace them with dedicated parsers over the validated buffer:
- parse_integer_unsigned / parse_integer_signed: accumulate digits with
overflow detection, falling back to the float path on overflow exactly
as the strtoull/strtoll round-trip check did. Overflow behavior is
unchanged for narrower or wider custom number types.
- parse_float_fast: Clinger's exact fast path for `double` (<=19 significant
digits, |exp10| <= 22, significand < 2^53), where significand * 10^exp is
exact under IEEE round-to-nearest. This is the same fast path used by
fast_float/simdjson. It is bit-identical to strtod on this subset and
declines (falling back to strtod) otherwise. Only `double` uses it; float
and long double keep std::strtof/std::strtold via a templated overload.
Measured on representative data (g++ 13, -O3):
- integers: DOM parse +11%, SAX +25-34%
- floats: DOM parse +37%, SAX +70% (clang: float DOM ~1.9x)
No dependencies added; header-only and C++11-clean. Existing parser,
lexer, conversion and deserialization unit tests pass unchanged; a
3M-value random-double fuzz matches strtod bit-for-bit.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AXcDtEma2PjxgmPS9cQGzA
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Add SWAR bulk string scanning for contiguous input (simdjson-style)
scan_string() read the input one character at a time through the input
adapter and classified every byte with a large switch. For contiguous
byte buffers we can instead scan 8 bytes at a time with a SWAR word test
that finds the first byte needing individual handling (the closing quote,
an escape, a control character, or a non-ASCII UTF-8 byte) and bulk-append
the ordinary run in one go.
- input adapters expose supports_bulk_scan / bulk_data / bulk_remaining /
bulk_skip for provably-contiguous, same-type, 1-byte iterator ranges
(raw pointers in every standard; std::string/std::vector/std::array and
friends additionally in C++20 via std::contiguous_iterator).
- the lexer gains a bulk_scan capability (gated on lazy_token_string so
bypassing the per-character capture cannot lose error diagnostics) and a
scan_string_bulk() fast path; streaming/wide/user adapters are unchanged
and keep the byte-at-a-time scanner.
The run contains no newline (all bytes < 0x20 are treated as special), so
position bookkeeping stays exact, and error tokens are still reconstructed
lazily from the consumed byte range. The SWAR special-byte test is pure
uint64_t arithmetic - no intrinsics, no runtime dispatch, C++11-clean.
Measured on representative data, pointer input, g++ 13 -O3
(string values discarded by accept() see the largest gains):
long ASCII strings: DOM +4.5x, SAX +14x, accept +17x (to ~2 GB/s)
short strings: DOM +15%, SAX +62%, accept +85%
escape-heavy: DOM +31%, SAX +26%, accept +28%
Same-input parity verified: 200k randomized documents (escapes, multibyte
UTF-8, surrogate pairs) accept/parse identically via the contiguous SWAR
path and the streaming byte path; unit lexer/parser/diagnostic-position/
deserialization/conversions suites pass unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AXcDtEma2PjxgmPS9cQGzA
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Validate UTF-8 in the bulk string scanner (portable ~2-3x on non-ASCII text)
The SWAR bulk string path stopped at the first non-ASCII byte and handed
every multibyte character to the byte-at-a-time scanner, whose per-byte
get()/next_byte_in_range()/add() machinery runs at roughly half the speed
of validating straight from the buffer. As a result, dense non-ASCII text
(CJK, emoji, accented Latin) parsed ~10-15x slower than ASCII.
Fold well-formed UTF-8 into the bulk run: scan_string_bulk() now, on a
non-ASCII lead byte, validates one sequence with validate_one_utf8() -
which mirrors scan_string()'s per-byte switch ranges exactly (rejecting
overlong forms, surrogates, and out-of-range code points) - and appends it
in place, continuing until the closing quote, an escape, a control byte,
or an ill-formed sequence. All error handling still defers to the byte
path, so error messages and positions are byte-for-byte unchanged.
Because only well-formed content is fast-pathed and every rejection falls
through to the existing scanner, behavior is identical; the win is purely
throughput. Measured on pointer input (accept, string values discarded):
content g++ 13 clang 18
dense CJK 277 -> 648 ~605 MB/s (~2.3x)
dense emoji 299 -> 857 ~702 MB/s (~2.6-2.9x)
mixed 90% ASCII 246 -> 331 ~334 MB/s (~1.35x)
pure ASCII unchanged (~3.2 / 4.1 GB/s)
Verified: 2,000,000 randomized documents built from arbitrary bytes
(overlong, surrogate, truncated, out-of-range sequences) accept/reject and
parse identically via the contiguous path and the streaming byte path;
lexer/parser/diagnostic-position/deserialization/conversions suites pass
unchanged. Pure C++11, no intrinsics.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AXcDtEma2PjxgmPS9cQGzA
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Route contiguous byte containers through the pointer adapter (fast paths in C++11)
json::parse(std::string) - the most common entry point - did not benefit
from the contiguous fast paths (bulk string scanning, UTF-8 bulk
validation, memcpy for binary formats) in C++11..17: std::string::iterator
is a library wrapper, not a raw pointer, and pre-C++20 there is no portable
way to prove it contiguous, so supports_bulk_scan was false. Only raw
pointers, string literals, and C-arrays (and, in C++20, anything modelling
std::contiguous_iterator) took the fast path.
Detect contiguous single-byte containers (std::string, std::vector<char>,
std::vector<std::uint8_t>, std::string_view, ...) via is_contiguous_byte_
container and route them through an iterator_input_adapter built from
data()/data()+size(). The generic iterator-based container overload is
constrained to exclude these, so the two overloads are disjoint and there
is no ambiguity (a plain competing overload loses to the greedy
forwarding-reference container overload on reference binding, and a factory
partial-specialization is ambiguous - both were tried and rejected).
The pointer keeps the container's own element type, so char_type - and
therefore all parsing behavior - is byte-for-byte identical to the iterator
path (const char* for std::string, const std::uint8_t* for
std::vector<std::uint8_t>); only the raw pointer additionally turns on the
fast paths. Lifetimes are unchanged: the container outlives the adapter for
the full parse expression, exactly as the iterators it replaces did.
Measured, C++11, json::parse/accept(std::string), g++ 13:
long ASCII strings: accept 201 -> 3200 MB/s (~16x), parse 174 -> 1444
dense CJK: accept 263 -> 697 MB/s (~2.6x)
short strings: accept 163 -> 243 MB/s (~1.5x)
Verified: char_type preserved for std::string (char) and
std::vector<std::uint8_t> (uint8_t); CBOR/MsgPack round-trips from
std::vector<std::uint8_t> unchanged; 1,000,000 randomized documents accept
and parse identically via std::string and via std::istream;
deserialization/user-defined-input/parser/lexer/conversions/diagnostic-
position suites pass (20,480 assertions); warning-clean on g++ and clang in
C++11/17/20.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AXcDtEma2PjxgmPS9cQGzA
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Add optional simdutf backend for bulk UTF-8 validation (JSON_USE_SIMDUTF)
The bulk string scanner validates UTF-8 straight from a contiguous buffer.
The scalar validator caps at ~0.3-0.7 GB/s on non-ASCII text; a SIMD
validator reaches several GB/s. Rather than hand-rolling SIMD UTF-8
validation (easy to get subtly wrong - a from-scratch SSE attempt rejected
valid CJK), wire in the vetted simdutf library behind an opt-in switch.
simdutf is not header-only (it ships simdutf.cpp and uses runtime CPU
dispatch), so it is not vendored: defining JSON_USE_SIMDUTF includes
<simdutf.h> and routes the bulk validator through simdutf::validate_utf8;
the project supplies and links simdutf. Undefined (the default), nothing
external is included and the portable C++11 scalar path is used, so the
library stays header-only and its baseline behavior is unchanged.
Design keeps behavior identical either way:
- scan_string_bulk() now finds the run up to the next quote/escape/control
byte (non-ASCII allowed) and validates it in one shot; on the rare
validation failure it recomputes the exact valid prefix with the scalar
helper, so ill-formed input still falls through to the byte path and is
reported at the same position with the same message.
- the per-sequence scalar path is factored into scalar_string_bulk_run()
and is the default backend; the refactor is behavior-preserving and does
not change scalar throughput.
Verified: default and JSON_USE_SIMDUTF builds accept/reject/parse
identically across 2,000,000 arbitrary-byte documents and 1,000,000
mixed-escape/UTF-8 documents (differential fuzz vs the streaming byte
path); lexer/parser/diagnostic-position/deserialization suites pass under
both configurations (20,188 assertions with the backend enabled);
warning-clean on g++ and clang, C++11 and C++20, both configurations.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AXcDtEma2PjxgmPS9cQGzA
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Add a contiguous fast path for scanning numbers
scan_number() reads a number one character at a time through the input
adapter (get()) and appends each byte to token_buffer (add()) before
converting. For contiguous input, the per-character get()/add() overhead
dominates: it is roughly two thirds of the time spent on number-heavy
parsing, far more than the value conversion itself.
Add scan_number_bulk_contiguous(), which parses the whole number token
straight from the input buffer: it validates and classifies the extent
with the same grammar as scan_number()'s state machine, materializes
token_buffer in one copy (substituting the locale decimal point exactly as
scan_number() does), advances the adapter, and reuses the shared
convert_number() tail. On anything it does not recognize as a well-formed
number it makes no state change and returns token_type::uninitialized, so
the caller falls back to scan_number(), which then produces the exact
diagnostic. Errors and their positions are therefore unchanged.
The conversion tail is factored out of scan_number() into convert_number()
so both scanners share it; the fast path is selected by tag dispatch on the
existing bulk_scan capability, so streaming/wide/user adapters are
unaffected.
Measured on pointer input, g++ 13 -O3:
- integers: parse +65%, accept +98%
- floats: parse +39%, accept +70%
Verified: 2,000,000 randomized number documents (including overflow-range
integers, long digit strings and %.17g doubles) parse identically via the
contiguous path and the streaming byte path, matching value, type and
round-trip text; the locale suite and existing parser/lexer/conversions/
deserialization tests pass; a new "lexer number fast path" test checks
contiguous-vs-streaming parity, token classification, and that malformed
numbers are rejected identically. Pure C++11, no intrinsics.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AXcDtEma2PjxgmPS9cQGzA
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Move byte-level scan/parse helpers out of lexer.hpp
lexer.hpp had grown by ~600 lines of byte-level helpers that have no
dependency on the lexer's template parameters and clutter the state
machine. Move them, unchanged, into two focused headers as free functions
in namespace detail:
- number_parse.hpp: parse_integer_unsigned/parse_integer_signed (now
templated on the number type) and parse_float_fast (Clinger's exact
double fast path, with the decimal point passed as an argument instead of
read from a lexer member).
- string_scan.hpp: the SWAR string helpers (is_string_special,
swar_string_special, find_string_special, validate_one_utf8,
scalar_string_bulk_run) and the backend-dispatched string_bulk_run,
including the optional simdutf include and find_string_delimiter.
lexer.hpp now includes these and calls the free functions; the methods that
touch lexer state (scan_string, scan_number, scan_string_bulk,
scan_number_bulk_contiguous, convert_number) stay put. This is a pure code
move with no behavior change: lexer.hpp drops from 2357 to 1934 lines, the
now-unused <cstdint>/<cstring>/<limits> includes are removed, and the
free-function form makes the SWAR helpers reusable elsewhere (e.g. the
serializer's string escaping).
Verified: default and JSON_USE_SIMDUTF builds compile; 2,000,000 number and
2,000,000 arbitrary-byte-string differential-fuzz documents parse
identically to before; lexer/parser/conversions/deserialization/locale/
diagnostic-position suites pass (20,576 assertions); warning-clean on g++
and clang in C++11/17/20; the amalgamation regenerates and passes
check-amalgamation.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AXcDtEma2PjxgmPS9cQGzA
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix clang-tidy findings and document JSON_USE_SIMDUTF in the nav
- number_parse.hpp: use std::array for the powers-of-ten table
(avoid-c-arrays) and `auto` for the cast-initialized result
(modernize-use-auto), matching the codebase style (cf. the serializer's
utf8d table). Indexing casts keep the -Wsign-conversion build clean.
- add JSON_USE_SIMDUTF to the mkdocs navigation so the macro page is
reachable.
No behavior change; clang-tidy is clean on the new headers and the
amalgamation is regenerated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AXcDtEma2PjxgmPS9cQGzA
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix number fast path for custom string types without assign()
The contiguous number fast path materialized token_buffer with
token_buffer.assign(data, len), but string_t is only required to provide
the minimal interface the rest of the lexer uses (push_back, append,
clear, operator[], ...). Custom string types such as the test's alt_string
do not implement assign(), so scan_number_bulk_contiguous() failed to
compile for them (unit-alt-string), breaking the gcc/clang standards and
old-compiler CI jobs.
reset() already clears token_buffer, so fill it with append() - which
alt_string and std::string both provide and which the string fast path
already relies on - instead of assign().
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AXcDtEma2PjxgmPS9cQGzA
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Satisfy clang-tidy: parenthesize math and drop unused forwarding reference
The CI clang-tidy (newer than the locally available version) reported two
additional checks on the new code:
- readability-math-missing-parentheses: parenthesize the (a * b) + c digit
accumulations in number_parse.hpp.
- cppcoreguidelines-missing-std-forward: the contiguous-byte-container
input_adapter overload took a forwarding reference but only reads
data()/size() and never forwards it. It is already disjoint from the
generic container overload via SFINAE, so a plain const& is correct and
clearer (and keeps the container alive for the whole parse just as before).
No behavior change; char_type and routing are unchanged (std::string and
std::vector<std::uint8_t> still take the pointer adapter with char/uint8_t
char_type), CBOR/MsgPack round-trips and the 2M number fuzz still pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AXcDtEma2PjxgmPS9cQGzA
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Use std::from_chars (Eisel-Lemire) for float conversion when available
The Clinger fast path is exact only for the "easy" subset (<=19 significant
digits, |exp10| <= 22); high-precision and scientific floats fall through to
strtod, where the failed Clinger attempt actually makes parsing a net loss.
std::from_chars implements the Eisel-Lemire algorithm in modern standard
libraries: locale-independent, correctly rounded, and fast over the whole
value range.
convert_number() now tries parse_float_from_chars() first (guarded by
__cpp_lib_to_chars, so C++11 and libc++-without-float-support keep the
Clinger + strtod path unchanged), then Clinger, then strtof. from_chars is
used only when it consumes the entire token; a partial parse means a non-'.'
locale decimal point, and an under-/overflow (result_out_of_range) also
declines - in both cases the existing strtod fallback supplies the exact
value and the well-defined +/-inf/0 the parser expects, side-stepping the
P4168 divergence between implementations. float and long double now get the
fast path too (Clinger was double-only).
Measured, C++17, g++ 13 -O3, json::parse/accept:
- canada-style floats: ~unchanged (Clinger already covered them)
- high-precision (17 digits): parse 2.1x, accept 2.5x
- scientific (17 digits + exp): parse 3.6x, accept 4.1x
Verified: C++11 (Clinger/strtod) and C++17 (from_chars) parse every value -
including subnormals, boundary values, and 1e9999/1e-9999 over-/underflow -
to bit-identical results; 2M number-fuzz clean; conversions/deserialization/
locale/number-fast-path suites pass in both C++11 and C++17; clang-tidy
clean; warning-clean on g++ and clang in C++11/17/20.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AXcDtEma2PjxgmPS9cQGzA
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Guard from_chars use on JSON_HAS_CPP_17, not just __cpp_lib_to_chars
libstdc++ 15 defines __cpp_lib_to_chars even in C++14 mode (via
bits/version.h pulled in by other headers), but <charconv> is only included
under JSON_HAS_CPP_17. That made parse_float_from_chars() reference
std::from_chars without the header in C++14 builds, breaking gcc-latest,
icpx, and the offline-testdata jobs.
Gate the use on JSON_HAS_CPP_17 && __cpp_lib_to_chars so it matches the
include condition exactly; C++11/14 always take the scalar fallback.
Verified by forcing __cpp_lib_to_chars in a C++14 build: the guard
suppresses std::from_chars and it compiles. C++17 behavior is unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AXcDtEma2PjxgmPS9cQGzA
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Disable Clinger float fast path under extended FP precision (x87)
The contiguous number fast path uses a Clinger-style exact algorithm
(significand * 10^scale in double arithmetic), which is only correctly
rounded when double operations are evaluated in true 53-bit precision.
On the x87 FPU used by 32-bit x86 (FLT_EVAL_METHOD == 2) the single
multiply/divide is computed in 80-bit and then double-rounded to double,
so a small fraction of values land 1 ULP off.
This surfaced as test-cbor_cpp11 and test-msgpack_cpp11 failing on the
mingw (x86) job for regression/floats.json: the C++17 builds pass because
they take the correctly-rounded std::from_chars path, while C++11 falls
back to parse_float_fast(). A 5M-sample check over shortest round-trip
decimals reproduces it: 0 divergences with 53-bit doubles, ~1 in 25 000
with 80-bit intermediates; declining to std::strtod fixes all of them.
Guard parse_float_fast() on FLT_EVAL_METHOD so it declines whenever the
platform evaluates doubles in extended precision, letting the caller use
the correctly-rounded std::from_chars / std::strtod path instead. On
mainstream x86-64/ARM64 (FLT_EVAL_METHOD == 0) the fast path is unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AXcDtEma2PjxgmPS9cQGzA
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Guard <charconv> include with __has_include for GCC 7
GCC 7 sets __cplusplus to the C++17 value under -std=gnu++1z, so
JSON_HAS_CPP_17 is defined, but its libstdc++ ships no <charconv> header
(added in GCC 8; floating-point from_chars in GCC 11). The unconditional
"#if defined(JSON_HAS_CPP_17) #include <charconv>" therefore failed to
compile there: "fatal error: charconv: No such file or directory" in the
ci_test_compilers_gcc (7) job.
Wrap the include in __has_include(<charconv>), mirroring the library's
existing handling of <version> and <filesystem> in macro_scope.hpp. When
the header is absent, __cpp_lib_to_chars stays undefined and
parse_float_from_chars() takes its scalar fallback, so the from_chars use
site (already gated on __cpp_lib_to_chars) is never reached. GCC 8-10,
which have <charconv> but no floating-point from_chars, are unaffected:
they include the header but still take the fallback. GCC 11+ is unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AXcDtEma2PjxgmPS9cQGzA
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Restore the column when ungetting a newline
The contiguous number fast path never reads the character that terminates
a number token, while scan_number() reads it and then ungets it. When that
character is a newline, get() has already cleared chars_read_current_line,
and unget() could only restore lines_read - leaving the column at 0. The
two paths therefore reported different columns for the same document:
json::parse("[01\n]") -> line 1, column 3
json::parse(stringstream) -> line 1, column 0
Remember the column the newline was read at so unget() can restore it.
Both paths now report the position the offending token actually starts at,
which also fixes the pre-existing column-0 artifact for streaming input.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Extend the bulk scan fast paths to sized sentinels
supports_bulk_scan required IteratorType and SentinelType to be the same
type, which excluded std::counted_iterator paired with std::default_sentinel_t
- the combination #5268 had already enabled for the memcpy fast path. Such
input fell back to the byte-at-a-time scanner even though it is contiguous
and its remaining length is computable in O(1).
Factor the "distance is computable in O(1)" test into sentinel_is_sized and
use it for iterator_is_contiguous, supports_seek, and supports_bulk_scan
alike, and share the std::ranges::distance/std::distance dispatch through a
remaining_count() helper.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Document JSON_USE_SIMDUTF on the macro overview page
The macro was only listed in the API macro index; add it to the supported
macros overview alongside the other JSON_USE_* macros, and note that it
selects between two definitions of the same inline function and so must be
defined identically in every translation unit.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Amalgamate source code
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Cover the counted-iterator bulk scan paths
Sized sentinels newly reach the bulk string/number scanners and the
seek-based token reconstruction, so exercise both:
- diagnostics that quote the offending token, which are rebuilt from the
consumed input via copy_consumed_range()
- inputs whose count ends before the underlying buffer does, including a
closing quote that exists only behind the count, a cut inside an 8-byte
SWAR stride, and a cut inside a UTF-8 sequence
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix the SentinelType example on iterator_input_adapter
The comment offered "a C++20 sentinel or counted_iterator" as examples of a
SentinelType, but std::counted_iterator is the IteratorType - the sentinel it
pairs with is std::default_sentinel_t. #5268 corrected the same wording in the
API documentation and left the code comment behind.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Do not discard the parse result in the error position check
json::parse is declared warn_unused_result, and CHECK_THROWS_WITH_AS
evaluates its expression as a discarded statement, so the assertion broke
the -Werror builds (GCC -Werror=unused-result, MSVC C4834 under /WX).
Compare against the helper that already captures the message instead.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Lock the two number grammars together with a parity test
The JSON number grammar is encoded twice: as the scan_number() state machine
and as the contiguous fast path. The fast path declining on anything it does
not recognize keeps most divergence harmless, but if it ever accepted
something the state machine rejects the result would be a silent correctness
bug, and the existing test only pinned a hand-written list of numbers.
Enumerate every string of length 1..4 over "01.eE+-" (2800 tokens) and
require both paths to agree on the parsed value and on the exact error
message. Verified to fail if the fast path's grammar is perturbed.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Skip token_buffer for integers on the contiguous path
An integer token does not need token_buffer: the number_integer and
number_unsigned SAX callbacks take only the value, and the overflow
diagnostic rebuilds the text from the input via get_token_string(). Convert
straight from the input buffer and materialize the token only for the
floating-point tail, which still needs a NUL-terminated buffer for strtod.
JSON_DIAGNOSTIC_POSITIONS derives a number's start position from
get_string().size(), so the copy is kept when that is enabled.
The integer dispatch is factored into convert_integer() and shared with
convert_number(), so both scanners keep using one implementation.
Integer-heavy input, 400k values, -O3:
parse accept
gcc 16 +14% +23%
clang +15% +22%
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Amalgamate source code
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Cover the bulk string and UTF-8 scanners
These paths had no dedicated tests and rested on differential fuzzing only.
Add three sections, all comparing the contiguous scanner against the
byte-at-a-time one on the parsed value and on the exact error message:
- every string of length 1..3 over an alphabet of ordinary ASCII, both
specials, a control byte, escape characters, UTF-8 lead and continuation
bytes, and a byte that is never valid - each at offset 0 and offset 9, so
the bulk scanner sees them with and without a run behind them
- every kind of run-ending byte at each offset across two 8-byte SWAR words,
so multibyte sequences also straddle the word boundary
- the boundaries of every range validate_one_utf8() recognizes: shortest and
longest encodings, overlongs, both ends of the surrogate block, U+10FFFF
and just past it, and truncated sequences
Verified to fail if the bulk validator accepts surrogates, and if the SWAR
word test stops detecting control characters.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Format the new string fast path test with astyle
The pinned astyle expands a braced-init-list used as a range-for range onto
several lines; hoist the two offsets into a named vector instead, which reads
better and leaves nothing for astyle to reformat.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix shadowed locals and guard the exception-dependent tests
Two problems in the tests added for the bulk scanners, both found by CI:
- the inner `const json j` in the counted-iterator diagnostics shadowed the
one declared at test-case scope, which -Wshadow rejects on GCC and clang
and C4456 rejects on MSVC under /WX; rename them
- the new parity checks parse deliberately invalid input, which calls
std::abort() rather than throwing when JSON_NOEXCEPTION is defined, so
they would have crashed the no-exception build; guard them the way the
other tests do
json::accept() does not abort, so the UTF-8 range assertions stay compiled
without exceptions and keep covering validate_one_utf8() there.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Gate the C++20 iterator classification on JSON_HAS_RANGES
Making supports_bulk_scan depend on iterator_is_contiguous meant the trait
is now instantiated for every adapter, not only when get_elements() is
called. On standard libraries with an incomplete <ranges> that is fatal:
libstdc++ 10 evaluates std::contiguous_iterator<std::counted_iterator<T*>>
by calling std::to_address, which needs an operator-> its counted_iterator
does not have, so satisfaction checking is a hard error rather than false.
Reported by clang 14 + libstdc++ 10.
JSON_HAS_RANGES already encodes exactly this ("libstdc++ < 11 has incomplete
C++20 ranges", #4440), so require it for the C++20 branch. Affected
toolchains fall back to the pointer-only test and the byte-at-a-time
scanner, which parses identically, just without the bulk fast paths.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Amalgamate source code
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Satisfy clang-tidy in the new bulk scanner tests
- give the helper lambdas an explicit std::string return type and return
braced initializer lists (modernize-return-braced-init-list)
- replace the C-style array of test cases with a std::vector
(modernize-avoid-c-arrays)
- silence pro-type-member-init on the two brace-initialized aggregates;
default member initializers would stop them being aggregates in C++11
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Avoid escaped literals in the counted-iterator diagnostics list
clang-tidy reads "[\"\\ud834\"]" as a literal better written raw, and the
two literals written next to each other in "[\"a\x01""b\"]" as a missing
comma. The concatenation was there to stop the hex escape swallowing the
following character; build those documents from explicit bytes instead and
use raw strings elsewhere. The byte sequences are unchanged.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Reattach convert_number's documentation
@gregmarr spotted that convert_integer() was inserted between convert_number()
and its doc block, leaving convert_integer() with two stacked blocks and
convert_number() with none. Comment only; no code change.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Address review comments on the simdutf backend
string_bulk_run() had the same `return scalar_string_bulk_run(...)` in both
arms of the `#if`. The simdutf arm already falls through when validation
fails, so a single return after the `#endif` says the same thing.
The JSON_USE_SIMDUTF example showed `#include <simdutf.h>`, which
string_scan.hpp already does under the same guard; users only have to put the
header on the include path and link the library, not include it themselves.
Set the version history entry to 3.13.0, matching the other macro pages
documenting unreleased features.
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MXDi7NTMKAmoArUZSKMc4T
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Pin the number error position against a non-newline terminator
The comment claimed the reported column is the one the offending token starts
at. It is the column reached after the token's last character - which is the
actual point of the unget() change: a number terminated by a newline now
reports what the same number terminated by a space always did.
Assert that equality directly, and add a multi-character token where the start
and end columns differ, so the invariant cannot be read off a single-character
example.
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MXDi7NTMKAmoArUZSKMc4T
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Do not repeat the integer conversion that just failed
scan_number_bulk_contiguous() converts an integer token straight from the input
buffer. When the value does not fit, it materializes token_buffer and calls
convert_number(), which tried the very same integer conversion again before
falling back to floating point.
Recording the outcome in number_type skips the second attempt. The resulting
token type and value are unchanged: convert_number() reached the float tail
either way.
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MXDi7NTMKAmoArUZSKMc4T
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Require a container's value_type to match what data() points at
is_contiguous_byte_container accepted any type with a data() returning a
pointer to a single-byte integral plus a size(). That is duck typing: the two
members say nothing about size() counting the units data() points at.
A type where it does not - fixed-size records, say - was routed to the
pointer-based adapter and parsed as [data(), data() + size()) bytes, silently
truncating input the iterator-based adapter had read in full:
struct record_buffer {
using value_type = std::array<char, 4>;
std::string bytes;
const char* data() const; // raw bytes
std::size_t size() const; // in records
const char* begin() const; const char* end() const;
};
json::parse(record_buffer{"[1,2,3,4,5]"}); // parse error at column 3
Requiring the container's own value_type to be that same element type ties the
two together. Every contiguous standard container satisfies it, so std::string,
std::vector<char>, std::array<char, N> and std::string_view keep the fast path;
anything else falls back to the iterator-based adapter, which is always correct.
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MXDi7NTMKAmoArUZSKMc4T
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Only compile the simdutf backend from C++17 on
simdutf.h rejects anything below C++17 with an #error, so defining
JSON_USE_SIMDUTF in a C++11 or C++14 translation unit did not fail with a
message about simdutf being unavailable - it failed to compile at all, taking
the library's C++11 support with it. Nothing caught this because no build ever
compiled that path.
Gate the include and both uses on JSON_HAS_CPP_17, the same way number_parse.hpp
gates std::from_chars. Below C++17 the macro now has no effect and the scalar
validator runs; it accepts and rejects exactly the same input, so the macro is
safe to set project-wide even when some translation units use an older standard.
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MXDi7NTMKAmoArUZSKMc4T
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Test the simdutf backend in CI
JSON_USE_SIMDUTF was documented and shipped but never compiled by anything in
the repository, so nothing held the backend to the behavior the docs promise.
Add JSON_TestSimdutf (OFF by default), which fetches simdutf and defines
JSON_USE_SIMDUTF for every test target, and a ci_test_simdutf target that runs
the whole suite in that configuration. Because simdutf needs C++17, the suite is
built at C++11 as well, so one job covers both the scalar fallback with the
macro defined and simdutf itself.
The dependency hangs off test_main, whose usage requirements every test target
inherits. The library target and the installed CMake package are deliberately
untouched: making nlohmann_json link simdutf would put a find_dependency() in
the exported package, which is a separate decision.
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MXDi7NTMKAmoArUZSKMc4T
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Warn when JSON_TestSimdutf cannot reach the simdutf backend
simdutf needs C++17: without it the dependency does not even compile, and with
a C++17 compiler but no C++17-or-later standard under test it builds and then
goes unused. Either way the option silently did nothing useful, or broke the
configure step outright.
Resolve the tested standards first, then check them: when none of them can
reach simdutf, skip the dependency and say so, naming which of the two reasons
applies and how to fix it. The tests then run against the scalar validator,
which is what would have happened anyway.
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MXDi7NTMKAmoArUZSKMc4T
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Use record_buffer::data() so clang does not flag it unneeded
The record_buffer test type declares data() and size() so the
is_contiguous_byte_container trait can see both and still reject the
type on its value_type. data() was never called, so clang's
-Wunneeded-member-function (under -Weverything -Werror) failed the
C++20 build. Assert that data() points at the underlying bytes: it
ODR-uses the member and documents the property the type is meant to
demonstrate.
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Skip the float fast path when it cannot succeed
parse_float_fast() (Clinger) needs a significand below 2^53, so it always
declines once the mantissa has 17 or more significant digits. convert_number()
called it unconditionally, so those numbers were walked an extra time before
strtod had to run anyway. On streaming input, where scanning is byte-at-a-time
and there is no compensating win, that made canada.json about 6% slower than
develop.
Derive the significant-digit count from token_buffer indices - the digits are
not scanned again - and skip the call when it is guaranteed to decline. Both
scanners pass the offset where the mantissa ends; the count only has to be
corrected for a leading "0", which the JSON grammar admits nowhere else. The
integer path returns before the check, so integer-heavy input is unaffected.
Values are unchanged: this only avoids an attempt that would have failed.
Verified bit-exact against develop over every number in canada.json,
floats.json, signed_ints.json, unsigned_ints.json, small_signed_ints.json,
citm_catalog.json and twitter.json, for both the contiguous and the streaming
scanner.
parse, streaming develop before after
canada.json 19.4ms 20.5ms 19.3ms
floats.json 135.9ms 131.8ms 128.0ms
parse, contiguous develop before after
canada.json 15.5ms 12.9ms 11.7ms
floats.json 98.6ms 69.8ms 66.7ms
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
The binary entry points end with
return res ? result : basic_json(value_t::discarded);
The condition operator's second operand is an lvalue, so this is not a case
where the return value can be elided or implicitly moved from: every
successful from_cbor(), from_msgpack(), from_ubjson(), from_bjdata() and
from_bson() call deep-copies the value it just parsed, and then destroys the
original.
The copy is not cheap, and it is not incidental: basic_json's copy
constructor walks the whole value. Parsing a 2 MB CBOR document with 60,000
objects, median of 25 runs, clang 17 -O3:
from_cbor 26.99 ms -> 14.65 ms
from_msgpack 26.82 ms -> 14.82 ms
Moving instead of copying is the entire change; the parsed value is not used
again after the return expression is evaluated.
There is a second reason to prefer the move. The copy constructor recurses
once per nesting level, so the copy is also a stack-overflow path on the
return side, on a value the reader has already accepted. That is currently
masked because the readers themselves recurse and overflow first (#5104), but
it has to be fixed for making them iterative to have any effect.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Reserve capacity in from_json() object conversion when supported
The object-to-container from_json() overload filled the target
container one element at a time without reserving capacity, even
when the target type supports reserve() (e.g. std::unordered_map)
and the number of elements is already known. This caused unnecessary
rehashing while parsing large objects into such containers.
Add a reserve-detecting overload (from_json_object_impl), mirroring
the priority_tag-based SFINAE technique already used by the array
conversion path (from_json_array_impl), so that reserve(size()) is
called up front when available and the loop falls back unchanged
otherwise (e.g. for std::map).
Fixes#5406
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Update doc example outputs for new object-conversion iteration order
Reserving capacity in from_json()'s object-conversion path before
inserting elements changes libstdc++'s std::unordered_map bucket
layout, which changes the iteration order used by
get__ValueType_const.cpp, get_to.cpp and operator__ValueType.cpp to
print the elements of a converted std::unordered_map<std::string,
json>. Verified against a clean develop checkout (built with the
same GCC/libstdc++ used in CI) that the old order was produced
without this PR's change and the new order is produced with it, and
that the three affected examples now match their updated expected
output byte-for-byte.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Factor out a reserve-dispatch helper instead of duplicating the object from_json loop
Addresses review feedback from @gregmarr on PR #5472: the emplace loop no
longer needs to exist twice for the reserve/no-reserve cases. A small
from_json_object_reserve() overload pair (SFINAE-dispatched on whether
reserve() exists, mirroring the priority_tag technique used elsewhere)
either calls reserve() or is a no-op; from_json_object_impl() calls it once.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Inline from_json_object_impl into from_json now that it is called only once
Addresses review feedback from @gregmarr on PR #5472: with the reserve
loop de-duplicated, from_json_object_impl no longer needs to be a
separate function that from_json immediately delegates to.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
contains(const json_pointer&) is documented to never throw, but a purely
numeric reference token that is syntactically a valid array index yet
numerically too large to be represented (exceeding size_type's max, or
exceeding ULLONG_MAX and causing strtoull() to set errno to ERANGE) made
it fall through to array_index(), which throws out_of_range.410/404.
Pre-check the token's magnitude the same way array_index() does, but
return false instead of throwing, mirroring how the surrounding code
already rejects other malformed tokens (leading zero, non-digit
characters, "-") without throwing. operator[]/at() are untouched and
keep throwing for these inputs.
Fixes#5395
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Extract C++17-only content from unit-items.cpp into its own test file
unit-items.cpp is a 1433-line file that was being compiled twice per
CI configuration (once for C++11, once for C++17) purely because it
contained a single, small JSON_HAS_CPP_17-gated SECTION ("structured
bindings", 14 lines). Move that SECTION into a new, dedicated file
(tests/src/unit-items-cpp17.cpp) so only that tiny file needs a
second build; unit-items.cpp itself now builds/tests only once. No
tests/CMakeLists.txt changes are needed since the existing
file(GLOB ... src/unit-*.cpp) plus json_test_add_test_for() already
auto-register and standard-gate any new unit-*.cpp file based on
whether it textually contains JSON_HAS_CPP_<N> (the same mechanism
already used for the existing unit-iterators3.cpp file, which follows
the identical pattern).
Verified with plain clang++ under -std=c++11/14/17/20 and via a local
CMake configure+build that:
- unit-items.cpp now only produces a test-items_cpp11 target (the
former test-items_cpp17 target is gone) and its assertion/test-case
counts are unchanged (2 test cases / 222 assertions) for every
standard.
- The new unit-items-cpp17.cpp produces test-items-cpp17_cpp11 (an
intentionally empty translation unit under C++11 that reports 0
tests, 0 assertions, SUCCESS) and test-items-cpp17_cpp17 (1 test
case / 1 assertion, identical to what "structured bindings" ran
as before it was moved).
Separately, unit-regression1.cpp (1530 lines) was also being built
twice per CI configuration because it contained the substring
JSON_HAS_CPP_17 -- but on inspection this was dead code: an orphaned
"#ifdef JSON_HAS_CPP_17 / #include <variant> / #endif" left over from
when the actual std::variant-based regression test (issue #1292) was
relocated to unit-regression2.cpp. Nothing in unit-regression1.cpp
uses <variant>, so there is no SECTION/TEST_CASE to preserve here;
the dead include is simply removed. This was verified by grepping the
file for any other use of "variant" (none) and confirming issue #1292
is still covered by unit-regression2.cpp. Compiled and ran under
-std=c++11/14/17/20 and via CMake: unit-regression1.cpp now only
produces a test-regression1_cpp11 target (test-regression1_cpp17 is
gone) with an unchanged test-case count (3) under every standard.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix astyle indentation of #include inside #ifdef in unit-items-cpp17.cpp
This repo's astyle style keeps preprocessor directives at column 0
even inside #ifdef blocks. The new tests/src/unit-items-cpp17.cpp
had its #include <map>/#include <string> indented, which made the
'check' CI job's amalgamation/formatting diff non-empty and failed
the aggregate check.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Add test coverage for documented lenient BSON input handling
Issue #5333 documented three intentionally-lenient behaviors of the BSON
reader (any non-zero byte accepted as a boolean `true`, BSON array element
keys not validated against the required decimal sequence, and the payload
of binary subtype 0x02 "old binary" returned as-is including its inner
length prefix), but none of them was pinned by a test, so a future change
could silently regress the documented behavior.
Also add coverage for the out_of_range.412 length-overflow check
(shared by binary, string, and (sub-)document BSON length fields) for
the string and document cases; only the binary case was previously
tested.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix 32-bit overflow in huge_string_t BSON length-overflow tests
huge_string_t doubles as basic_json's StringType, so it is used not only
for the JSON string value under test but also for object keys (e.g. "s",
"nested"). Making size() unconditionally lie about being huge therefore
inflated the keys' reported sizes as well, pushing the running totals
computed while walking the BSON document (calc_bson_object_size and
friends in binary_writer.hpp) past what a 32-bit std::size_t can hold.
On 64-bit platforms this happens to still produce a working (if
needlessly large) result, but on 32-bit platforms (e.g. the mingw x86 CI
job) the size_t arithmetic silently wraps around: for the "document" test
this merely surfaces the wrong number in the exception message, but for
the "string" test the wrapped total happens to fall back under
INT32_MAX, so the intended out_of_range.412 guard is skipped entirely and
the code goes on to actually write ~2 GiB worth of characters from the
key's real, tiny buffer - which is what raised the reported
"vector::_M_range_insert" exception instead of a controlled 412.
Make the fake-huge size opt-in via huge_string_t::as_huge() and only
apply it to the string value under test, leaving keys at their real
(small) size. This keeps every intermediate size well within 32-bit
size_t range on any platform, matching how huge_binary_t already avoids
the same trap (it is only ever used as the BSON value type, never as a
key). Expected out_of_range.412 messages are updated accordingly.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Broaden JSON_HEDLEY_WARN_UNUSED_RESULT coverage to pure query functions
Add JSON_HEDLEY_WARN_UNUSED_RESULT to the unambiguous, const,
side-effect-free observer functions whose return value is the entire
purpose of the call:
- dump()
- type(), type_name()
- all is_* predicates (is_primitive, is_structured, is_null,
is_boolean, is_number, is_number_integer, is_number_unsigned,
is_number_float, is_object, is_array, is_string, is_binary,
is_discarded)
- empty(), size(), max_size()
- count(...) (both overloads) and contains(...) (all overloads,
including the deprecated json_pointer<BasicJsonType> overload)
This mirrors the direction the standard library has taken with
[[nodiscard]] on the analogous std::vector/std::map members, and
catches real bugs such as `j.empty();` (meant `j.clear();`) or
`j.contains(k);` with the result thrown away.
Deliberately out of scope (left for a separate, later policy
decision, per the issue): at(), value(), get*(), flatten(),
unflatten(), patch(), merge_patch(), begin()/end(), comparison
operators, erase(), and emplace().
Compiling the full test suite (tests/src/unit-*.cpp) with
-Wunused-result -Werror uncovered one real hit: a regression test in
unit-regression2.cpp called dump() purely to check it does not throw,
discarding the result. Fixed by explicitly casting to void, since the
call is intentionally result-less there.
Fixes#5410
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix discarded nodiscard results across the test suite for GCC's warn_unused_result
A plain (void) cast on a call expression suppresses the C++17 [[nodiscard]]
warning but not GCC's warning for functions annotated via the GNU
__attribute__((warn_unused_result)) form -- which is what
JSON_HEDLEY_WARN_UNUSED_RESULT expands to on GCC. Several existing tests
that call a newly-annotated function (dump(), empty()) purely to check
that it throws/does not throw, discarding the result via (void), newly
warned (and failed -Werror builds) once the annotation was broadened.
Route those discards through a small ignore_return_value() helper
instead, which actually consumes the value and suppresses the warning
on both attribute forms.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Use utils::ignore_return_value() for the issue #1445 dump() discard too
Addresses review feedback from @gregmarr on PR #5477: this call site was
still using the older "capture in a variable, then (void) it" pattern
from before this PR introduced utils::ignore_return_value(), instead of
the helper now used at every other discarded-nodiscard-result call site
this PR touches.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
accept() is a pure query whose only effect is the returned bool; both
parse() overloads and the deprecated accept(span_input_adapter&&, ...)
overload already carry JSON_HEDLEY_WARN_UNUSED_RESULT, but the two
current, recommended accept() overloads were missing it. Add the
annotation to match, so discarding accept()'s result now warns under
-Wunused-result / [[nodiscard]], as it already does for parse().
Fixes#5407
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Reject JSON Patch move when from is a proper prefix of path
RFC 6902 (section 4.4) forbids "from" from being a proper prefix of
"path" for a "move" operation: "a location cannot be moved into one
of its children." "move" is implemented as remove-then-add with no
check for this. For object targets, the subsequent "add" happened to
throw as a side effect of resolving through the now-removed parent,
but for array targets, removing the "from" element shifts subsequent
indices, so "path" silently re-resolves to a different element and
the operation "succeeds" with a silently corrupted document.
Add a check, before performing the remove/add, for whether "from" is
a proper prefix of "path" at the reference-token level. This compares
json_pointer's already-unescaped reference_tokens vectors (basic_json
is a friend of json_pointer) rather than the raw pointer strings, so
that tokens containing escaped '/' or '~' characters are compared
correctly, and a token that merely looks like a string prefix (e.g.
"/ab" vs "/abc/x") is not mistaken for a pointer-token prefix. When
"from" is a proper prefix of "path", throw out_of_range.414.
Fixes#5397.
Stacked on top of the fix for #5396 (branch
issue-5396-patch-remove-primitive-parent), since both touch the same
patch_inplace move/remove handling in include/nlohmann/json.hpp.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Add root-pointer and array-append-token edge case tests for the move prefix check
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Replace std::equal with an explicitly-bounded loop in the move prefix check
The three-iterator std::equal(first1, last1, first2) form has no
explicit end iterator for the second range, which a static analyzer
(Flawfinder, CWE-126) flags as a potential over-read even though the
preceding size comparison already guarantees the second range is long
enough. Rather than argue the point, make the bound visible in the code
itself via an explicit loop -- every access to ptr.reference_tokens is
now guarded by the same index the loop condition bounds against
from_size.
(The C++14 four-iterator std::equal(first1, last1, first2, last2) form
was tried first as a more minimal fix, but this codebase targets C++11
and that overload is not safely usable under -std=c++11 with all
supported standard library implementations.)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Extract the move prefix check into a named helper lambda
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Account for JSON_DIAGNOSTIC_POSITIONS in the move-prefix-check error messages
out_of_range::create() includes a "(bytes X-Y)" position annotation when
JSON_DIAGNOSTIC_POSITIONS is enabled, which the ci_test_diagnostic_positions
CI job builds the whole suite with. The five new out_of_range.414
assertions only checked the annotation-free message. Confirmed
JSON_DIAGNOSTICS produces the same (annotation-free) message as the
default build for this particular throw site (its path-based annotation
is empty at the root, where &result always points here), so only two
message variants are needed, not three.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
RFC 6902 (section 4.2) requires the target location of a "remove"
operation to exist. operation_remove handled parent.is_object() and
parent.is_array(), but had no final else branch: when the resolved
parent was a primitive value or null, neither branch matched and the
operation silently did nothing instead of failing.
Add the missing else branch, throwing out_of_range.413 with wording
that matches the existing out_of_range.411 thrown by the analogous
"add" case (operation_add) for the same kind of invalid parent.
Fixes#5396.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Route hand-rolled diagnostic pragmas through Hedley
Several places in the library hand-roll compiler diagnostic suppression
with raw `#pragma`/`#ifdef __GNUC__`/`#ifdef __clang__` guards instead of
using the Hedley primitives already bundled and used elsewhere
(JSON_HEDLEY_DIAGNOSTIC_PUSH/POP, JSON_HEDLEY_PRAGMA, ...). Converted six
of the seven listed push/pop pairs to use those primitives instead of
raw `#pragma GCC diagnostic`/`#pragma clang diagnostic` text:
- include/nlohmann/json.hpp (~3770, ~3863): -Wfloat-equal
- include/nlohmann/detail/conversions/to_chars.hpp (~1078): -Wfloat-equal
- include/nlohmann/detail/output/binary_writer.hpp (~1844): -Wfloat-equal
- include/nlohmann/detail/iterators/iteration_proxy.hpp (~211): -Wmismatched-tags
- include/nlohmann/detail/exceptions.hpp (~36): -Wweak-vtables
iteration_proxy.hpp did not previously include macro_scope.hpp itself
(it only compiled because some other header included earlier in
json.hpp happened to pull macro_scope.hpp in first); it now includes it
directly like the other detail headers that use Hedley macros, so it is
self-contained.
Each push/pop pair now uses JSON_HEDLEY_DIAGNOSTIC_PUSH/POP
unconditionally (a no-op on compilers that don't need it) and wraps the
actual `#pragma ... diagnostic ignored` text in JSON_HEDLEY_PRAGMA so it
goes through Hedley's _Pragma()-based emission instead of a raw #pragma
line, while keeping the original `#ifdef __GNUC__` / `#if
defined(__clang__)` guard around the ignored-pragma itself.
Deviation from the issue's suggested transformation: the issue's example
replaces the `#ifdef __GNUC__` guard with `#if
JSON_HEDLEY_HAS_WARNING("-Wfloat-equal")`. JSON_HEDLEY_HAS_WARNING is
implemented purely via Clang's `__has_warning` builtin and evaluates to
0 on real GCC (`#define JSON_HEDLEY_HAS_WARNING(warning) (0)` when
`__has_warning` is not defined), so adopting it verbatim would silently
stop suppressing -Wfloat-equal on GCC -- a real regression, not just a
style change. The existing `#ifdef __GNUC__` / `#if defined(__clang__)`
guards were kept for the ignored-pragma to stay behavior-preserving, and
only the push/pop/pragma-emission mechanism was routed through Hedley.
Two of the seven locations from the issue (the -Wignored-attributes
push at the very top of json.hpp and its matching pop after
`#include <nlohmann/detail/macro_unscope.hpp>`) were intentionally left
unconverted:
- The push, at the very top of json.hpp, runs before
`detail/macro_scope.hpp` (and therefore hedley.hpp) has been included
anywhere in the translation unit, so JSON_HEDLEY_DIAGNOSTIC_PUSH is not
yet defined at that point.
- The pop runs after `macro_unscope.hpp`, which -- via hedley_undef.hpp
-- has already #undef'd every JSON_HEDLEY_* macro (by design, see
#5408) precisely so they don't leak to users, so JSON_HEDLEY_DIAGNOSTIC_POP
is no longer defined by the time the pop is reached either.
Making this one pair work would require either hoisting the ~2000
line vendored hedley.hpp to the very top of the amalgamated single
header (a much bigger structural change to single_include than a pure
mechanism swap) or special-casing this one pop ahead of the general
macro cleanup. Both are riskier than the mechanical, behavior-preserving
change requested, so this pair was left as-is.
## Validation
- Compiled include/nlohmann/json.hpp and single_include/nlohmann/json.hpp
with `-Wall -Wextra -Wfloat-equal -Wmismatched-tags -Wweak-vtables`
(clang, which self-identifies as __GNUC__ too): no warnings, same as
before the change.
- Compiled and ran tests/src/unit-to_chars.cpp, unit-conversions.cpp,
unit-iterators1.cpp, unit-iterators2.cpp, and unit-class_parser.cpp
against the fixed include/: all pass.
- Compiled unit-msgpack.cpp, unit-bjdata.cpp, and unit-ubjson.cpp (which
exercise binary_writer.hpp's write_compact_float extensively): all
compile cleanly; the vast majority of assertions pass (the only
failures are pre-existing environment issues unrelated to this change
-- missing generated test-data files, not code correctness).
- Ran `make amalgamate`; the single_include diff is limited to exactly
the lines touched in include/, with no unrelated reordering.
- No real (non-Apple) GCC was available in this environment to test
directly; the `_Pragma("GCC diagnostic ...")` text emitted by
JSON_HEDLEY_PRAGMA is byte-identical to the prior `#pragma GCC
diagnostic ...` text, and the `#ifdef __GNUC__` guard is unchanged, so
GCC's behavior is expected to be identical. CI covers the GCC matrix.
This PR is stacked on top of #5475 (issue-5408-hedley-undef-leak) since
both touch the same files; only the last commit here is new.
Fixes#5409.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Guard JSON_HEDLEY_DIAGNOSTIC_PUSH/POP with the same compiler check as the pragma they bracket
Addresses review feedback from @gregmarr on PR #5485: the push/pop calls
were unconditional, so compilers other than the one the ignored-pragma
targets (e.g. MSVC, or GCC where the pair only applies under __clang__)
now did a needless push/pop with nothing suppressed in between. Move the
existing #ifdef __GNUC__ / #if defined(__clang__) guard to also cover the
push/pop, restoring the original zero-overhead behavior on other compilers
while still emitting the pragma itself through Hedley.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Undefine the four JSON_HEDLEY_* macros that leak after including json.hpp
include/nlohmann/detail/macro_unscope.hpp includes hedley_undef.hpp to
#undef every JSON_HEDLEY_* macro so none of them leak into the including
translation unit. Four macros were missing from that list and therefore
stayed defined after #include <nlohmann/json.hpp>:
- JSON_HEDLEY_PRAGMA
- JSON_HEDLEY_PREDICT_TRUE
- JSON_HEDLEY_PREDICT_FALSE
- JSON_HEDLEY_CLANG_HAS_DECLSPEC_ATTRIBUTE
hedley_undef.hpp is generated (via `make update_hedley`) by grepping
hedley.hpp for its own internal `#undef JSON_HEDLEY_X` redefinition
guards. JSON_HEDLEY_PRAGMA/PREDICT_TRUE/PREDICT_FALSE have no such guard
in upstream Hedley, so they were never picked up. The guard for
JSON_HEDLEY_CLANG_HAS_DECLSPEC_ATTRIBUTE also has an upstream typo
(`JSON_HEDLEY_CLANG_HAS_DECLSPEC_DECLSPEC_ATTRIBUTE`), so hedley_undef.hpp
was undefining the wrong (never-defined) name.
Fixes:
- include/nlohmann/thirdparty/hedley/hedley_undef.hpp: corrected the
DECLSPEC_ATTRIBUTE typo and added the three missing #undef lines,
keeping the file's alphabetical ordering.
- Makefile (update_hedley target): changed hedley_undef.hpp generation to
extract macro names directly from every `#define JSON_HEDLEY_...` in
hedley.hpp instead of from existing `#undef` guards, so a future
`make update_hedley` run undefines every macro Hedley actually defines,
even ones without a pre-existing redefinition guard. This was not run
in this PR (it would also pull in an unrelated upstream Hedley sync);
hedley_undef.hpp was hand-patched instead and single_include was
regenerated with `make amalgamate`.
- tests/src/unit-no-macro-leak.cpp: new regression test (picked up
automatically by tests/CMakeLists.txt's existing unit-*.cpp glob) that
includes json.hpp and then #ifdef/#error-checks every JSON_HEDLEY_*
macro name, so any future leak of any of the 151 vendored macros fails
the build, not just the four fixed here.
Fixes#5408.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Derive the JSON_HEDLEY_* leak-check test from hedley.hpp at build time
tests/src/unit-no-macro-leak.cpp previously hardcoded a static list of
~151 #ifdef/#error checks, one per JSON_HEDLEY_* macro name known at the
time it was written. That list would silently go stale the next time
`make update_hedley` pulls in a vendor update that adds, removes, or
renames a macro, since nothing would force it to be regenerated.
Add cmake/scripts/gen_hedley_undef_check.cmake, which derives the full
list of JSON_HEDLEY_* macro names directly from
include/nlohmann/thirdparty/hedley/hedley.hpp:
- tests/CMakeLists.txt uses it (MODE=checks) to (re)generate
hedley_undef_checks.inc at configure and build time, and wires the
generating custom target as a dependency of the test-no-macro-leak_cpp*
targets so it can never build against a stale copy. unit-no-macro-leak.cpp
now just #include-s the generated file inside its TEST_CASE instead of
carrying the checks itself.
- The Makefile's `update_hedley` target now delegates hedley_undef.hpp
generation to the same script (MODE=undef, new `update_hedley_undef`
target), so the vendored header, the generated #undef list, and the
generated test checks are all derived from the same extraction logic and
cannot drift apart.
This mirrors the approach taken independently in #5415 for the same
issue (#5408), credited there to a self-regenerating mechanism that
"can never drift again" -- ported into this branch instead of the
static list originally proposed here.
Verified with a local CMake configure + build + ctest, both against
include/ (JSON_MultipleHeaders=ON) and against the amalgamated
single_include/nlohmann/json.hpp (JSON_MultipleHeaders=OFF), and by
temporarily deleting a #undef line from hedley_undef.hpp to confirm the
generated test actually fails on a real leak.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix REUSE compliance failure in gen_hedley_undef_check.cmake
The generated file's embedded banner contains the literal text
'SPDX-License-Identifier: MIT' as part of the *content* being written
to hedley_undef.hpp, not as this .cmake script's own REUSE header (it
is already covered by the blanket 'Files: *' rule in .reuse/dep5).
The reuse tool matched that embedded line as an SPDX tag for the
script itself and failed to parse the trailing 'MIT\n")' as a valid
SPDX License Expression, breaking ci_reuse_compliance. Wrap the
embedded banner in REUSE-IgnoreStart/REUSE-IgnoreEnd comments, as
recommended by the tool's own diagnostic output.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Speed up whitespace skipping in the lexer
lexer::skip_whitespace() called get() for every whitespace byte, and
get() checks the (almost always false, once past the first character)
next_unget flag on every call. skip_whitespace() now reads its first
character with get() (needed to honor a pending unget() left over from
finishing the previous token, e.g. scan_number() always ungets the
character that terminated the number) and every further whitespace
character with a new get_ignoring_pending_unget() variant that skips
that branch, since nothing in the loop calls unget().
This is a narrower fix than the full contiguous-buffer bulk-skip
suggested in the issue (scan a run of whitespace directly in the
adapter's buffer and update position counters once per run). That
approach depends on bulk-scan adapter infrastructure
(supports_bulk_scan/bulk_data()/bulk_skip()) introduced by the open,
unmerged parser-performance PR #5283, which this change intentionally
does not depend on or replicate. Building new bulk-scan adapter
infrastructure from scratch was judged out of scope/riskier than
warranted here, so this change is limited to the safe, always-correct
improvement of removing redundant per-character bookkeeping from the
existing byte-at-a-time loop; full bulk-skipping is left as future
work once #5283 (or equivalent adapter support) lands.
Line/column/byte-offset bookkeeping is untouched and verified
bit-for-bit identical before and after this change, including for
pretty-printed (dump(4)) input with embedded newlines.
Fixes#5412
Stacked on top of the PR for #5411 (branch
issue-5411-lexer-skip-conversion).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix codegen regression in skip_whitespace() from #5490
Benchmarking found the get()/get_ignoring_pending_unget() split in
skip_whitespace() made long whitespace runs (e.g. indentation in
pretty-printed JSON) 1.75x-3.2x SLOWER instead of faster, reproducible
with both Apple Clang and GCC.
Root cause: rewriting the loop from a plain do-while into an initial
get() followed by a while-loop defeated the compiler's ability to keep
the input adapter's read/end pointers in registers across iterations;
both compilers instead reloaded them from memory on every character.
The function split itself was not the problem (it still fully
inlines); the loop's control-flow shape was.
The fix keeps the same two-function structure but restores a
do-while shape (guarded by an if for the "first char not whitespace"
case), which lets both compilers hoist the pointers back into
registers, matching or beating pre-#5490 performance.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Share the position-counter bump between get() and get_ignoring_pending_unget()
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Extract current_is_whitespace() to deduplicate skip_whitespace()'s two whitespace checks
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Use a raw string literal for the multi-line error-position test input
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix clang-tidy raw-string-literal finding and guard a new test against JSON_NOEXCEPTION
The issue #5412 whitespace-skipping test added a check_error() helper that
relies on catching json::parse_error to verify the exception message; under
JSON_NOEXCEPTION, JSON_THROW aborts instead of throwing, which crashed
ci_test_noexceptions (and cascaded into the other ci_cmake_options jobs).
Guard the whole section with #if !defined(JSON_NOEXCEPTION), matching the
existing pattern used by sibling tests in this file.
Also switch one escaped string literal to a raw string literal to satisfy
clang-tidy's modernize-raw-string-literal check.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Skip integer conversion in accept()/SAX validation when the value is unused
lexer::scan_number() always converted every numeric token with
strtoull()/strtoll() before returning, even though accept() (and any
consumer using json_sax_acceptor) immediately discards the converted
value. For value_unsigned/value_integer tokens whose digit count
already guarantees the value fits into 64 bits, the conversion cannot
change the accept/reject decision (such tokens are always finite and
unconditionally accepted), so scan_number() can skip strtoull()/
strtoll() entirely in that case when the caller signals it does not
need the value. Numbers with more digits keep using the exact,
unmodified conversion path, so overflow reclassification to
value_float (and the finiteness check on it) is unaffected.
parse() and value_float handling are completely unchanged.
Fixes#5411
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Document why the digit-count fast path is safe regardless of number_unsigned_t/number_integer_t width
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Avoid temporary-string concatenation flagged by clang-tidy in the differential test
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix to_bjdata() emitting unparsable output when _ArraySize_ is not an array
write_bjdata_ndarray() never checked that _ArraySize_ is an array. The shape
is written verbatim as the header length, so a null shape emitted 'Z' and an
object shape emitted '{' after the '#', neither of which from_bjdata()
accepts, and the round-trip guarantee in the BJData docs was broken.
Both slipped through the existing validation: for null, empty() is true so
the element count starts at 0 and the per-dimension loop never runs, and for
an object the loop walks its values, which can satisfy the non-negative
integer check. When _ArrayData_ then matched that count, the writer took the
ndarray path.
Require the shape to be an array, so anything else falls back to a plain
object encoding that round-trips, as the fallback rule in the docs already
specifies.
Signed-off-by: qatcod <79017227+qatcod@users.noreply.github.com>
* Document that _ArraySize_ must be an array in the ndarray requirements
The list at bjdata.md is the exhaustive set of conditions for the ndarray
encoding, but it only implied this one through 'every entry of'.
Signed-off-by: qatcod <79017227+qatcod@users.noreply.github.com>
---------
Signed-off-by: qatcod <79017227+qatcod@users.noreply.github.com>
* docs: add 89 customers and sort all sections
Extend the customers page from 136 to 225 entries. New entries were
found by searching vendor open-source notices and by inspecting
dependency manifests in public repositories.
Every added entry was verified to link either to a page that credits
the library, or to an open source repository where its use is directly
visible (a vendored copy, a build manifest, or an include). Candidates
whose only evidence was a transitive dependency (via ICU or
KDDockWidgets), packaging metadata, or an unused vendored file were
dropped rather than listed.
Also sort every section alphabetically ignoring case, and keep the
Peregrine lunar lander first under "Space Exploration".
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: add 41 customers and rebuild the overview image
The list was assembled by searching for names one at a time, which is why it
had plateaued. These additions come from enumerating instead: a public code
search returning every repository that references the library, and the reverse
build-dependencies of the Debian and Ubuntu source indices.
Every addition cites a first-party call site rather than a repository root,
and every evidence URL was checked to resolve. Several prominent candidates
were rejected on exactly that test:
- Node.js reaches the library only through deps/icu-small, the inherited
attribution ICU carries into everything that ships it.
- Visual Studio Code's only hits are copies of the library's own headers
used as sample C++ in the Copilot extension's test fixtures.
- OSS-Fuzz fuzzes the library rather than calling it; projects/json/ is the
library's own OSS-Fuzz integration.
- libuv matched only its AUTHORS file, which lists a contributor by name.
- simdjson and LLVM matched only benchmarks and a mangled-symbol test.
Two entries were already present under a different name and are merged rather
than duplicated: PrestoDB, and DB Browser for SQLite under its repository name.
GitHub CodeQL's citation moves from the repository root to shared/cpp/Diagnostics.h,
which includes the header directly.
The image is rebuilt with 236 logos over 17 rows. It is smaller than the one it
replaces, 1.02 MB against 1.35 MB, despite carrying 100 more marks.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The mixed number arms of JSON_IMPLEMENT_OPERATOR cast the integer to
number_float_t before comparing. Past the float's mantissa that cast is lossy:
2^63-2 and 2^63-1 both round to 2^63, so each compares equal to that float
while differing from each other. Equality is therefore intransitive and the
ordering is not a strict weak ordering, which makes std::sort over such values,
or using them as keys in std::set or std::map, undefined behavior.
Compare the two exactly instead. The integer's range is a power of two the
float represents exactly, so a float outside it is ordered by magnitude alone;
inside it, truncating the float is exact, and the integer parts and then any
fractional part decide. The helper hands back a pair whose comparison with the
original operator reproduces that ordering, which keeps every operator's return
type as it was, including partial_ordering for the spaceship.
A NaN operand is returned in both members, so NaN stays false for the
relational operators and unordered for <=>. Values a float represents exactly
still compare equal, so json(1) == json(1.0) is unchanged.
Signed-off-by: qatcod <79017227+qatcod@users.noreply.github.com>
When merge_objects is true, recurse only if the existing value is an
object. Otherwise overwrite, matching the documented "all other values
are overwritten as usual" behavior.
Fixes#5402
Signed-off-by: elix3r <157088510+22elix3r@users.noreply.github.com>
A std::pair or std::tuple whose every element is itself a two-element array
with a string first element (e.g. std::pair<std::string, int>) serializes to
a JSON object instead of a JSON array, because to_json builds the value with a
brace initializer and the initializer-list object-detection rule fires. The
resulting object cannot be read back into the original type and collapses
duplicate keys. Document this quirk in the conversions guide, together with the
unaffected cases and the idiom to force an array.
Claude-Session: https://claude.ai/code/session_016cwQq8WQRFzQcGQtbJTtJg
Signed-off-by: Claude <noreply@anthropic.com>
Co-authored-by: Claude <noreply@anthropic.com>
* Throw other_error.502 when UBJSON use_type is set without use_size
Fixes#5321
Signed-off-by: Krishnanand G <118352827+Krishnanand-G@users.noreply.github.com>
* Scope UBJSON use_type check to container branches and expand tests
Signed-off-by: Krishnanand G <118352827+Krishnanand-G@users.noreply.github.com>
* Re-amalgamate single_include/json.hpp
The previous commit updated the split headers but the amalgamated
file didn't go back through astyle before I committed it, so CI's
amalgamation check caught formatting drift in json_fwd.hpp and a
few noexcept clauses in basic_json, plus one doc example. None of
it touches the UBJSON logic. Applied the patch CI generated to
bring single_include back in sync.
Signed-off-by: Krishnanand G <118352827+Krishnanand-G@users.noreply.github.com>
---------
Signed-off-by: Krishnanand G <118352827+Krishnanand-G@users.noreply.github.com>
* Reformat parser_callback_t example with astyle
The file uses "json & /*parsed*/" in three lambda parameter lists, which
astyle rewrites to "json& /*parsed*/" per --align-reference=type. The
drift went unnoticed because CI never format-checked the documentation
examples; "make pretty" does cover them.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Format-check the documentation examples in CI
The examples live in docs/mkdocs/docs/examples, but both format checks
still referenced the long-gone docs/examples path:
- check_amalgamation.yml passed it to find, which printed an error for
the missing path and carried on, so astyle only ever saw include and
tests. The step still exited 0.
- ci.cmake globbed it into INDENT_FILES, and a GLOB_RECURSE over a
missing directory silently yields nothing, so the ci_test_amalgamation
target skipped the examples too.
Either way the 231 example files have never been format-checked. Point
both at the real path, and guard the workflow with an explicit directory
check so a future rename fails the job instead of quietly shrinking the
file list again.
Also drop the dead docs/examples/** path filter from
publish_documentation.yml; docs/mkdocs/** already covers the examples.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix CBOR tag handlers not recognizing tags 0-5 and 21-23
The tagged-item switch in binary_reader::parse_cbor_internal() only handled
head bytes 0xC6-0xD4 and 0xD8-0xDB. Bytes 0xC0-0xC5 (tags 0-5: date/time,
epoch, bignum, decimal, bigfloat) and 0xD5-0xD7 (tags 21-23: base64url,
base64, base16 conversion hints) fell through to the default case and were
reported as invalid bytes, even under cbor_tag_handler_t::ignore and ::store,
despite being valid CBOR major-type-6 tags per RFC 8949.
Add the missing case labels so the full 0xC0-0xDB range is handled
uniformly. Extend the "Tagged values" test in unit-cbor.cpp to cover
0xC0-0xD7, and update the CBOR docs to state the corrected tag range.
Fixes#5315
Signed-off-by: sahilkamate03 <45514385+sahilkamate03@users.noreply.github.com>
* Fix stale CBOR tag docs and add store-mode binary-payload test
The "Incomplete mapping" warning still listed tags 0-5 (date/time,
bignum, decimal fraction, bigfloat) and 21-23 (expected conversions)
as unsupported, even though they now parse correctly under
cbor_tag_handler_t::ignore/store, same as 0xC6..0xD4/0xD8..0xDB.
Remove those five bullets and cross-reference the "Tagged items"
warning below, matching the equivalent docs fix landed independently
in PR #5367.
Also add a cbor_tag_handler_t::store test that wraps a binary
payload (not just a string) for every byte in 0xC0..0xD7, confirming
these tags are unwrapped the same way as 0xC6..0xD4 rather than
mistaken for the 0xD8..0xDB binary-subtype marker syntax, per review
feedback on #5331.
Signed-off-by: sahilkamate03 <45514385+sahilkamate03@users.noreply.github.com>
---------
Signed-off-by: sahilkamate03 <45514385+sahilkamate03@users.noreply.github.com>
The block documenting get_char and tag_handler sat above
get_cbor_negative_integer(), which takes neither, so Doxygen attached it
there and parse_cbor_internal() was left undocumented.
Comment placement only.
Signed-off-by: Dmitry <45711841+darkdi@users.noreply.github.com>
The diagnostic position of a string value was derived by subtracting the
parsed value's length from the end position. Escape sequences make the
source token longer than the value it parses to, so the reported start
position landed inside the string, one byte off per escape sequence:
input: {"a":"\n\n\n\n\n\n"}
start_pos() == 11, so the reported range covered n\n\n\n"
instead of the documented "\n\n\n\n\n\n"
This contradicts the documented behavior of start_pos(), which is the
position of the opening quote, and it also corrupted the "(bytes N-M)"
part of JSON_DIAGNOSTICS exception messages. Strings with multi-byte
UTF-8 but no escapes were unaffected, which is why this went unnoticed.
Record the offset of the token in the lexer when it starts scanning and
use that, instead of reconstructing it from the parsed value. Booleans,
null and numbers already reported correct positions and are unchanged.
The new lexer member and accessor are compiled only when
JSON_DIAGNOSTIC_POSITIONS is enabled, which is already part of the ABI
tag, so the default build is unaffected.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The comment asked whether the no-op marker 'N' may be ignored when a
string is read. It may not: at that point the next byte must be a string
length type specification, and 'N' is not one. No-ops at positions where
a value may start are already consumed by the callers through
get_ignore_noop(), so nothing is lost by not skipping them here.
Replace the TODO with a comment stating that, and add regression tests
pinning both directions: a no-op is accepted at top level (also
repeated), before and after an array element, and before an object key,
between key and value, and before the closing brace of an object of
unknown size; it is rejected where a length type specification is
expected, i.e. after the 'S' marker of a string value and as the key
length of an object of known size.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Do not write BJData ndarrays whose size overflows std::size_t
write_bjdata_ndarray() multiplied the _ArraySize_ dimensions into a
std::size_t without checking for overflow. A product that wraps around
to a value that happens to match the size of _ArrayData_ passed the
length check, and the writer emitted an ndarray header announcing an
element count that cannot be represented:
{"_ArrayType_":"uint8","_ArraySize_":[9223372036854775808,2],"_ArrayData_":[]}
was encoded as 5b 24 55 23 5b 4d 00 00 00 00 00 00 00 80 69 02 5d, an
ndarray of 2^64 elements followed by no data. Reading that back throws
out_of_range.408 ("excessive ndarray size caused overflow"), so to_bjdata
produced output that from_bjdata rejects. This is reachable by parsing
untrusted JSON and re-encoding it as BJData.
Mirror the overflow check the binary reader already performs, and also
reject a single dimension that does not fit into std::size_t, which the
previous cast silently truncated where std::size_t is narrower than 64
bits. Such objects now fall back to a plain object encoding, which is
what the surrounding type and length validation already does for
annotations it cannot represent, and they round-trip unchanged.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Document when to_bjdata converts a JData annotation to an ND-array
The BJData page described the 1-D vector case as the only situation in
which an object carrying _ArrayType_/_ArraySize_/_ArrayData_ is not
written as a compact ND-array. The writer has always had several other
fallbacks -- an unknown _ArrayType_, a dimension that is not a
non-negative integer, an _ArrayData_ whose length does not match the
product of the dimensions, and elements that are not numbers of the
annotated kind -- all of which cause the value to be serialized as a
regular JSON object instead.
Spell out the conditions, including the size-overflow check added in the
preceding commit, so the documented behavior matches the implementation.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Linking test-regression2_cpp20 intermittently fails with
unit-regression2.cpp.obj:(.debug_info+0x16): relocation truncated to
fit: IMAGE_REL_AMD64_SECREL against `.debug_line'
The failure moves between matrix entries from run to run, and the same
commit can pass and fail on consecutive runs, so it is the size of the
debug sections rather than any one Clang version.
The jobs only build and run the tests, so override CMAKE_CXX_FLAGS_DEBUG
to drop the default -g. Everything else about the Debug build is
unchanged: no optimization flag is added and NDEBUG stays undefined, so
JSON_ASSERT remains active.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Conversions whose element count is fixed by the destination C++ type --
`std::pair`, `std::tuple`, `std::array<T, N>`, C arrays, and
`std::map`/`std::unordered_map` with a non-string key -- read exactly the
elements they need via `at` and never compare the JSON array's size to
that number. Excess elements are silently discarded, while a shortfall
throws `out_of_range.401` rather than a `type_error`. Neither direction
was documented in `conversions.md`, `get.md`, or `from_json.md`.
The existing warning covered only `std::array` and stated that a too-short
JSON array leaves the remaining elements default-constructed with no
exception thrown; that is not what happens. Generalize it to all
fixed-size destinations and correct the shortfall direction.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: document the complexity of ordered_map operations
ordered_map stores its elements in a std::vector in insertion order and
has no lookup index, so emplace, operator[], at, find, count, erase, and
insert are all linear scans. The documentation stated no complexity for
any operation, neither in ordered_map.md nor in ordered_json.md.
Add a per-operation complexity table and note the consequence: building
or parsing an ordered_json object of n keys is O(n^2). Measured with
-O2 -DNDEBUG for parsing a flat object of n keys, ordered_json is 5x
slower than json at n=2000 and 54x slower at n=16000, with the timings
quadrupling per doubling of n. Cross-reference the table from
ordered_json.md and from the object order page, which recommends
ordered_json without mentioning the cost.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: move the Complexity section after Member functions
scripts/check_structure.py enforces a fixed section order for pages under
docs/mkdocs/docs/api, in which Complexity comes after Member functions.
The section had been placed right after Iterator invalidation, which made
ci_test_build_documentation fail with structure/section_order.
No content change beyond the move; the table columns are realigned to the
narrower content.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 📝 Document exceptions newly thrown by the binary-format hardening
A round of binary-format input validation (#5274, #5284, #5287, #5332)
added new failure modes without updating exceptions.md, and left two
descriptions factually narrower than the code:
- parse_error.110 said "CBOR or MessagePack"; BSON and UBJSON also
throw it. Generalized, and added the BSON EOF example (#5332).
- parse_error.112: added the BSON document-size mismatch example
(#5287).
- parse_error.113 said "while parsing a map key", but its own existing
UBJSON char example already contradicted that. Broadened to cover
invalid length specifications, and added the negative-string-length
example (#5284).
- out_of_range.408 said "of an UBJSON array or object"; CBOR now throws
it too (#5274). Generalized and added both CBOR examples.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 📝 Correct which types may be referenced in a tuple extraction
The note added in #5271 said a referenced type must be one the library
stores "or an arithmetic type it can convert to/from". The parenthetical
is wrong: is_compatible_reference_type requires an exact match against
the stored types, so std::tuple<int&> is rejected by static_assert even
though int converts fine as a value. Only the value case is permissive.
Spell out the eight admissible types, give the int& counter-example, and
separate the reference restriction from by-value conversion.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Follow-up to #5342, which fixed the parser callback leaving a discarded
member behind when an array or a value under an object key was rejected.
The documentation of parser_callback_t only stated that discarded values
in structured types are skipped, without saying that this covers object
parents and that the key is removed along with the value, so there was no
way to tell the fixed behavior from the buggy one.
Spell out the discarding rules, add an example that exercises the cases
the fix repaired, and correct the return value description: a discarded
top-level value is replaced by null, not by "an empty discarded object".
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: use HTTPS for the astyle and cppcheck links in README
Both links were still `http://`. `astyle.sourceforge.net` serves HTTPS
directly; `cppcheck.sourceforge.net` redirects to
`https://cppcheck.sourceforge.io`, which is also the URL already used in
`docs/mkdocs/docs/community/quality_assurance.md`, so the redirect is
skipped here.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* CI: suppress -Wc2y-extensions for Clang
Clang 22.1 (now shipped by silkeh/clang:latest) diagnoses __COUNTER__ as a
C2y extension, and does so in C++ mode as well. Under -Weverything -Werror
this breaks every ci_test_clang_cxx* / ci_test_clang_libcxx_cxx* target,
independently of the code under test.
The library itself does not use __COUNTER__; all diagnostics originate in
vendored Doctest (DOCTEST_ANONYMOUS, used by TEST_CASE and SECTION).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* docs: document RFC 8259 / JSONTestSuite compliance and parse() vs operator>> strictness
The compliance story lived only in tests/src/unit-testsuites.cpp, so
drive-by comparisons kept claiming the library "does not fully pass
JSONTestSuite". Make it discoverable:
- README: add a "Standards compliance" note stating that both nst
JSONTestSuite revisions run in CI, that all mandatory y_/n_ cases pass
through the strict parse() entry point, and listing the deliberate
implementation-defined i_ choices (unbounded nesting, silent BOM
stripping, noncharacters forwarded, strict rejection of invalid UTF-8
and lone surrogates, out_of_range.406 on numeric overflow).
- features/parsing: add a "Strictness and trailing data" section
documenting that parse() is strict and rejects trailing data while
operator>> follows relaxed iostream semantics (parses one value and
leaves the stream positioned after it) -- the single place a naive
test yields a "non-compliant" result.
Documentation only; no parser behavior change. Closes#5290.
Signed-off-by: manon <youdie006@users.noreply.github.com>
* docs: correct test-data vendoring and parse()/operator>> claims per review
- README: the JSONTestSuite data is downloaded from nlohmann/json_test_data at
configure time, not vendored/committed; say so.
- README: only the updated suite runs y_ and n_ cases through strict parse();
the original suite's y_ cases go through operator>>. Narrow the claim.
- parsing/index.md and operator_gtgt.md: note that operator>> consumes a number's
terminating byte, so concatenated numbers must be whitespace-separated (1 2
works, 1true does not); structural and literal values are unaffected.
Signed-off-by: manon <youdie006@users.noreply.github.com>
---------
Signed-off-by: manon <youdie006@users.noreply.github.com>
Co-authored-by: manon <youdie006@users.noreply.github.com>
is_comparable used a flat && chain to both exclude json_pointer/string
comparisons (added for #4621) and check whether Compare(A, B) is well-formed.
Naming std::is_constructible<decltype(...)> as a later operand of that chain
still causes the decltype to be substituted regardless of the first
operand's value, since the operands aren't lazily deferred like
std::conjunction would defer them. That instantiates the transparent
std::equal_to<>::operator() used by ordered_json, whose noexcept-specifier
evaluates the deprecated json_pointer/string operator==, which Clang (unlike
GCC in this case) warns about even though the result is discarded.
Split is_comparable so the Compare(A, B) checks live in a separate helper
that is only referenced from the specialization selected when
is_json_pointer_of is false, so the decltype is never written when A/B are
a json_pointer/string pair, regardless of compiler.
Signed-off-by: Niels Lohmann <niels.lohmann@gmail.com>
operator>>'s notes state that it leaves the stream positioned right
after the parsed value, so that concatenated JSON values can be read
back to back. That does not hold when the value is a number: a number
is only terminated by the character that follows it, and the lexer's
unget() is simulated (it rewinds only the lexer's own bookkeeping),
so that character stays consumed from the stream.
Document the actual behaviour: the guarantee holds for all value types
except numbers, which must be followed by whitespace. Also qualify the
cross-reference on the JSON Lines page, which repeated the unqualified
claim.
Documentation only; the behaviour itself is tracked in #5340.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* reject CBOR array/map length equal to the indefinite-length marker
Signed-off-by: Angadi Yashaswini <angadi@digiscrypt.com>
* reject CBOR lengths that do not fit in std::size_t via value_in_range_of
Signed-off-by: Angadi Yashaswini <angadi@digiscrypt.com>
---------
Signed-off-by: Angadi Yashaswini <angadi@digiscrypt.com>
This adds a std::isfinite check to the UBJSON floating-point parsing path, throwing out_of_range.406 on overflow. This makes the UBJSON parser's behavior consistent with the normal JSON parser. Fixes#5322.
Signed-off-by: AJ369ninja <abhishek.j@iitg.ac.in>
Co-authored-by: AJ369ninja <abhishek.j@iitg.ac.in>
* validate ndarray element types in write_bjdata_ndarray
Signed-off-by: Angadi Yashaswini <angadi@digiscrypt.com>
* read ndarray elements through get<> instead of a fixed union member
_ArrayType_ names the wire type, not how the value is stored: parsing
keeps a non-negative integer as number_unsigned while the C++ API keeps
an int literal as number_integer. Selecting the union member from the
type marker therefore reads the inactive alternative for one of the two,
so read through get<> instead, which dispatches on the active member.
Also reject a negative _ArraySize_ entry, which is not a usable
dimension, and cover the parse-built path in the tests.
Signed-off-by: Angadi Yashaswini <angadi@digiscrypt.com>
---------
Signed-off-by: Angadi Yashaswini <angadi@digiscrypt.com>
- warn about BSON marker 0x11 interoperability in both directions
- explain subtype-less binary normalization to subtype 0x00
- add a round-trip test for binary values without a subtype
Signed-off-by: YingqiDuan <141370165+YingqiDuan@users.noreply.github.com>
Both had zero documentation anywhere in docs/mkdocs/. The tuple/pair
gap was first spotted in the very first git-log audit pass but never
turned into an actionable todo, so it persisted uncaught across four
subsequent passes.
- Document basic positional std::pair/std::tuple <-> json array
conversion, plus #5016's reference-extraction capability
(get<std::tuple<T&, T&>>() returning references into the stored
array elements).
- Document #5205's new json-from-C++20-range-view constructor
(e.g. nums | std::views::filter(...)).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Closes#3106. set(JSON_Diagnostics ON) before find_package() has no
effect on a package built and installed elsewhere (Homebrew, vcpkg, a
system package, etc.) -- the compile definition is baked into the
exported nlohmann_jsonTargets.cmake at install time and the generated
config script never re-reads that variable. Verified empirically
against the real Homebrew-installed 3.12.0 package: the exported
target carries a fixed $<$<BOOL:OFF>:JSON_DIAGNOSTICS=1>, and the
suggested set(JSON_Diagnostics ON) snippet produces no change in
exception output.
Documents the actual working fix (overriding the imported target's
INTERFACE_COMPILE_DEFINITIONS property after find_package()) and the
multi-target "JSON_DIAGNOSTICS redefined" pitfall reported earlier in
the issue thread.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* Add iterator+sentinel tests and docs for binary deserializers
This commit extends the C++20 ranges support (iterator+sentinel pairs) to the
binary format deserializers from_cbor, from_msgpack, from_ubjson, from_bjdata,
and from_bson, matching what was already done for parse(), accept(), and
sax_parse().
Changes:
- Add istreambuf_sentinel helper to test_utils.hpp for EOF detection in tests
- Add 5 new test cases that read binary files directly via
std::istreambuf_iterator<char> + sentinel, without pre-buffering
- Update documentation for all 5 from_* functions to document overload (3)
with SentinelType parameter
- All tests pass; verified against existing test suite data
- Fix potential buffer over-read warning in heterogeneous iterator test
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Merge iterator+sentinel overloads and fix ambiguity/CI issues
Address PR review feedback and CI failures:
- Merge the separate same-type and sentinel-type iterator overloads of
parse(), accept(), sax_parse(), and the five from_* binary deserializers
into a single overload with SentinelType defaulted to IteratorType,
as suggested in review. Applied the same simplification to the
detail::input_adapter() free functions.
- Fix a latent ambiguity: some compilers (e.g. GCC 4.8) unreliably SFINAE
the operator!= detection for std::nullptr_t against container/string
types, making calls like parse(s, nullptr, ...) ambiguous with the
compatible-input overload. can_compare_ne now explicitly excludes
std::nullptr_t as a SentinelType.
- Use a named enable_if_t template parameter instead of an unnamed
function parameter for the SFINAE guard, fixing a clang-tidy
hicpp-named-parameter/readability-named-parameter failure.
- Update parse.md, accept.md, sax_parse.md, and the five from_*.md pages
to document the merged overload instead of separate (2)/(3) overloads,
also fixing an over-160-char line that broke the documentation
style_check CI job.
- Rework the BSON iterator+sentinel test to parse a BSON file already
present in the test suite instead of writing/deleting a temp file.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix -Wunneeded-internal-declaration for CustomSentinel in test
CustomSentinel lives in an anonymous namespace (internal linkage), and
the library's parse loop only ever evaluates the iterator-first
direction (it != last), so the reversed-order friend operator!= was
never referenced. Clang's -Weverything flags such unused internal
declarations as an error. Drop the unused overload; the used direction
is enough to satisfy can_compare_ne's either-order detection.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix clang-tidy hicpp-named-parameter and misc-const-correctness
- Drop the unused reversed-order operator!= overload from
utils::istreambuf_sentinel (only iterator != sentinel is ever
evaluated) and name the remaining friend's sentinel parameter, fixing
hicpp-named-parameter/readability-named-parameter.
- Mark the istreambuf_iterator first/last helper variable const in the
five binary-format sentinel tests, fixing misc-const-correctness.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix clang-tidy misc-const-correctness in heterogeneous sentinel test
json_str is only read via .data()/.size() and never reassigned, so
clang-tidy correctly flags it as const-able. Verified against the exact
CI job (silkeh/clang:dev, ci_clang_tidy target) by running clang-tidy
directly on this file plus the five binary-format sentinel tests
touched by prior commits; all are now clean.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix discussion #4209: custom BinaryType direct assignment and extraction
When a custom BinaryType is configured (other than the default std::vector<uint8_t>),
users can now:
1. Assign values of that type directly to create binary values (not arrays)
2. Extract binary values back to that type with get<>()
3. Extract arrays to that type (for backward compatibility)
Implementation:
- Add is_compatible_binary_type trait to centralize SFINAE condition
- Update to_json to accept custom BinaryType values directly
- Update from_json to handle both binary and array inputs for custom BinaryType
- Add #include <vector> with IWYU comment to from_json.hpp
- Add comprehensive tests for assignment and array extraction
- Update binary_t documentation with example
This is purely additive and invisible to the default nlohmann::json alias, which
continues to treat std::vector<uint8_t> as arrays.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix CI: missing include and const-correctness
- Add #include <vector> to type_traits.hpp for the new
is_compatible_binary_type trait's std::vector<std::uint8_t> reference
(caught by cpplint's include-what-you-use check)
- Mark test-local json variables const where never reassigned
(caught by clang-tidy's misc-const-correctness check)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Migrate ci_icpc/ci_test_compilers_gcc_old/ci_infer off custom json-ci image
Replaces the last three consumers of ghcr.io/nlohmann/json-ci with official
images: ci_icpc now uses Intel's own intel/oneapi-hpckit:2023.2.1-devel-ubuntu22.04
(the last release with classic icc/icpc before Intel dropped it in oneAPI
2024.0), ci_test_compilers_gcc_old installs old GCCs on official ubuntu:20.04
via the same PPA/archive setup the custom image used (working around
actions/checkout's incompatibility with official gcc:4/5/6 images), and
ci_infer runs directly on ubuntu-latest, fetching Facebook's official Infer
release tarball inline instead of a maintained image. No job in
ubuntu.yml references the custom image anymore.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Update quality-assurance compiler table for CI image migration
Reflects the ci_icpc/ci_test_compilers_gcc_old container migration: the
old-GCC jobs (4.8/4.9/5/6) now compile inside official ubuntu:20.04 rather
than the custom Focal-based json-ci image (same OS, just now attributed to
the official image), and ci_icpc now uses Intel's official
intel/oneapi-hpckit:2023.2.1-devel-ubuntu22.04, bumping the reported ICC
version from 2021.5.0 to 2021.10.0 and the OS from Ubuntu 20.04.3 to 22.04.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix CI failures from the json-ci image migration
- ci_icpc: the official intel/oneapi-hpckit image has no CMake preinstalled
(the custom image bundled one); add the missing lukka/get-cmake step.
- ci_test_compilers_gcc_old: official ubuntu:20.04 has no build tool, so
CMake's default Unix Makefiles generator failed with "CMAKE_MAKE_PROGRAM
is not set"; install make alongside the PPA-provided g++.
- ci_infer: Infer v1.1.0's bundled Clang frontend can't parse GCC 14's
headers (ubuntu-latest's default toolchain), failing with parse errors in
<bits/unicode.h>; bump to the latest release, v1.3.0, whose newer bundled
frontend understands them (release asset also renamed upstream from
infer-linux64-v1.1.0.tar.xz to infer-linux-x86_64-v1.3.0.tar.xz).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix ci_test_compilers_gcc_old: g++-6 missing from xenial-only archives
My inline PPA/archive replication only added the xenial main/universe
suites, but g++-6 isn't available there ("has no installation candidate").
The original custom Dockerfile also pulled from bionic main/universe and
xenial-updates main/universe; add those back to match.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix ci_test_compilers_gcc_old: install git for CMake's FetchContent tests
Official ubuntu:20.04 ships no git at all (actions/checkout only succeeded
via its API-download fallback). The cmake_fetch_content(2) tests invoke
CMake's own ExternalProject_Add, which needs a real git binary and failed
with "could not find git for clone of json-populate". Install git alongside
the other build prerequisites.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix ci_icpc: drop redundant setvars.sh sourcing
Unlike the old custom image, the official intel/oneapi-hpckit image already
has the oneAPI environment (icc/icpc on PATH) baked in at the container
level. Explicitly re-sourcing setvars.sh in the Build step failed with
"setvars.sh has already been run. Skipping re-execution." (exit code 3,
aborting the step under `sh -e`). Drop the now-unnecessary sourcing.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix ci_icpc: exclude classic ICC from the std::span regression test
Bumping to Intel's official intel/oneapi-hpckit:2023.2.1 image (see previous
commit) also bumped classic icc/icpc from 2021.5.0 to 2021.10.0. The newer
version's __has_include(<span>) now returns true, but it still can't
actually compile std::span/std::as_bytes usage:
error: namespace "std" has no member "as_bytes"
error: namespace "std" has no member "span"
Exclude __ICC/__INTEL_COMPILER the same way _LIBCPP_VERSION is already
excluded for issue #4490.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix amalgamation/style check: indent comment per astyle
Verified with the pinned astyle 3.4.13 (make install_astyle) locally;
no further diff.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix ci_icpc: skip UTF-8 u8-literal comparison test on classic ICC
test-deserialization_cpp20 failed:
ERROR: CHECK( j2["emoji"] == "😀" ) is NOT correct!
check_utf8() only guards against MSVC's ANSI-codepage quirk (its docstring
example), but classic ICC has an analogous problem: it doesn't encode a
narrow string literal containing non-ASCII source characters as UTF-8,
so comparing a decoded u8R"(...)" literal against a narrow literal with
the same characters fails. Extend the existing guard.
Verified with the pinned astyle 3.4.13; no diff.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 📝 Fix documentation gaps found in a full GitHub Discussions review
Reviewed all 1008 GitHub Discussions (2020-2026) for recurring questions
that better or more visible documentation would have avoided. Adds/expands
documentation for ~26 distinct gaps, including:
- New "Debugging" page collecting natvis, GDB pretty printer, LLDB status,
and JSON_DIAGNOSTICS pointers (previously scattered/undiscoverable)
- Thread-safety and schema-validation FAQ entries
- StringType's char-based requirement (no wstring/u16string/u32string)
- Brace-initialization-yields-arrays warning directly on the constructor
reference page (previously only in the FAQ, missed by users reading
the constructor docs)
- std::any exclusion from get<T>(), with a manual-dispatch example
- Non-string-keyed std::map serializing as an array of pairs
- ordered_json compatibility with NLOHMANN_DEFINE_TYPE_* macros
(already worked, was undocumented)
- std::array truncation on size-mismatched conversion (no exception)
- static_cast vs. get<std::optional<T>>() divergence
- Recipe for omitting a std::optional field instead of emitting null
- No built-in nesting-depth limit during parsing + a callback-based
workaround recipe
- Recipe for streaming a large homogeneous array via parser callbacks
- operator>> stream-position semantics for concatenated JSON values
- JSON Pointer array-vs-object creation rule for non-existing paths
- CMake target name (nlohmann_json_modules) needed to link C++20 modules
- ESP-IDF/PlatformIO: no official package, link to a community fork
- get(key, default) as the Python dict.get() equivalent
- reserve() recipe for pre-allocating array capacity
- JSONC as an alias for the existing ignore_comments/ignore_trailing_commas
combination (distinct from the unsupported JSON5)
- items() dereferenced-element type: decltype() idiom + detail-namespace
stability caveat
- Various macro/type-conversion limitations (MSGPACK_DEFINE_ARRAY
equivalent, char-array round-tripping, ADL serializer macro gap)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 🎨 fix format
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix container input_adapter SFINAE for lvalue-only ADL begin/end (#111)
The container overload of json::parse(c) / accept(c) / sax_parse(c, ...)
silently dropped from overload resolution for user types whose ADL
begin(T&) / end(T&) accepted only non-const lvalue references
(a legitimate pattern matching std::begin semantics). This was because
the detection code used std::declval<ContainerType>() which synthesized
an rvalue, and the rvalue failed to bind to lvalue-only ADL functions.
Fix by making both the outer input_adapter(ContainerType&&) and the
factory's create(ContainerType&&) forwarding references, preserving the
caller's value category and constness via reference collapsing. This
ensures detection (std::declval) and actual use (std::forward) always
match without needing decay/remove_reference.
- Rewrite input_adapters.hpp container overload with forwarding refs
- Add regression tests for lvalue-only non-const ADL begin/end
- Add regression test for rvalue containers (no breakage)
- Update API docs (parse, accept, sax_parse, from_*) to clarify
that begin/end must match std::begin/std::end semantics
- Add version history notes for 3.13.0
- Regenerate amalgamation
Second-order effect: binary_reader.hpp's internal call to
input_adapter(number_vector) now deduces iterator vs const_iterator
based on the lvalue; functionally harmless (iterator_input_adapter is
iterator-type-agnostic), verified via unit-ubjson/unit-bjdata tests.
Closes remaining limitation from #4354 / PR #5218 (todo 106).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Avoid strlen() in test container to fix Codacy CWE-126 flag
Suppressing the strlen()-based CWE-126 warning with NOLINT/nosec
comments only silenced clang-tidy and the standalone Flawfinder
Action; Codacy's own analysis (which also flags this pattern and
doesn't honor those suppression comments) still reported it as a new
issue, plus flagged the near-duplicate begin/end pair as cloned code.
Store the buffer's size explicitly in MyContainerNonConstADL instead
of computing it via strlen() in end(), which removes the flagged
pattern outright and also de-duplicates the struct from the existing
MyContainer's char*-based begin/end pair.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Avoid trailing return type to satisfy clang-tidy fuchsia-trailing-return
The forwarding-reference input_adapter(ContainerType&&) entry point was
written with an auto/trailing-decltype return type, but this project's
ci_clang_tidy job enables the fuchsia-trailing-return check as an
error, which rejects it. The return type only depends on the template
parameter ContainerType, not on the runtime parameter, so it can be
written as an ordinary leading return type instead - no functional
change.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Avoid C-style array in test to satisfy clang-tidy avoid-c-arrays
clang-tidy's cppcoreguidelines/hicpp/modernize-avoid-c-arrays checks
flagged the char raw_data[] declaration used to reproduce the
lvalue-only non-const ADL begin/end scenario. Use std::string instead
and take a mutable pointer via &raw_data[0], which is the standard
way to get a non-const char* into a string's buffer under C++11
(std::string::data() only returns non-const in C++17 and later).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 📝 Fix stale 3.12.x placeholder in operator_ne.md version history
PR #5253 (removing the hand-written operator!= to fix #3868/P2468R2)
merged after the earlier 3.12.x -> 3.13.0 global sweep, so its new
version-history entries were written with the stale placeholder.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 🐛 Fix stale twitter.com link in docset.json
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 📝 Document a duplicate-object-key rejection recipe
RFC 8259 leaves handling of duplicate object keys to the implementation;
this library silently keeps only the last value for a repeated key.
Discussion #5085 asked for an opt-in rejection mode. Decision: don't
change library behavior, but document the existing parser-callback
workaround instead.
Adds a "Recipe: rejecting duplicate object keys" section to
parser_callbacks.md, adapted from a community-contributed workaround.
Fixed an off-by-one bug in the original snippet: object_start reports
the depth of the object's parent, while key events inside that object
report depth+1, so indexing the per-depth key set with the same depth
in both places caused an out-of-bounds access on nested objects.
Verified the published snippet compiles and behaves correctly for flat
duplicates, nested duplicates, sibling objects sharing key names, and
arrays of objects.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Cross-link the duplicate-key recipe with the existing object_t behavior docs
object_t.md and features/types/index.md already document that duplicate
object keys resolve to an unspecified value (RFC 8259 leaves this to the
implementation). The new recipe's intro overstated this as a guaranteed
"last value wins" rule, which isn't true in general -- parsing text keeps
the last value, but constructing from an initializer list keeps the first.
Reworded the recipe to point at object_t's "unspecified" behavior instead
of asserting a specific rule, and added cross-links from both existing
pages to the new recipe.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Turn the duplicate-key recipe into a standalone, compiled example
Replace the inline code fence in the "rejecting duplicate object keys"
recipe with a proper docs/mkdocs/docs/examples/*.cpp + .output pair,
included via --8<-- like every other example on the site. The .output
file was generated by running it through the project's actual example
build (docs/Makefile: single_include, -std=c++11, -DJSON_USE_GLOBAL_UDLS=0)
and cross-checked with `make check_output`, and the source passes the
pinned astyle 3.4.13 formatting unchanged.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix#3868: Remove operator!= to enable P2468R2 rewritten candidate synthesis
Under C++20 P2468R2, a hand-written operator!= suppresses the compiler's
rewritten-candidate synthesis for operator==, preventing heterogeneous
comparisons like `std::string s; json j; s == j;` from compiling.
Fix by removing the hand-written operator!=, allowing the compiler to
synthesize != as !(a==b) in all language modes (C++20 member functions
and pre-C++20 friend functions).
Behavior change: operator!= now returns !(a==b) unconditionally, including
for special values like NaN and discarded. This means:
- NaN != NaN now returns true (matches IEEE-754 semantics)
- discarded != x now returns true for any x (matches !(discarded == x))
This also fixes underlying defects in previously-working code:
- Restores direct == comparison for views vs json (reverts std::ranges::equal
workaround added in PR #3950 to dodge this bug)
- Re-enables std::string == json comparisons (uncomments check in
unit-constructor1.cpp)
Fixes: #3868, #3979
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 🚨 fix warning
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
PR #5248 added a 5th JSON_HAS_RANGES exclusion branch to
macro_scope.hpp (nvcc CUDA 12.0.x/12.1.x, fixed in 12.2, issue #3907)
shortly after #5252 added the "Known compiler/stdlib exclusions"
list to json_has_ranges.md, so the new branch was missing from the
just-added doc section. Bring the list back to parity with the code
(5 exclusion branches, 5 documented).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix stale Clang -Weverything suppression comments; eliminate -Wno-missing-noreturn
cmake/clang_flags.cmake claimed -Wno-unsafe-buffer-usage was needed only
for Doctest and that -Wno-missing-noreturn had "no way to silence...
otherwise" (PR #4871, which never actually attempted a source fix).
Neither held up under investigation (todo 130):
- -Wno-unsafe-buffer-usage is pervasive (208 distinct sites across 19
files measured with clang trunk in silkeh/clang:dev), spanning the
library's own low-level numeric/buffer code (to_chars, serializer,
lexer, binary reader/writer, input adapters, json_pointer) as well as
vendored Doctest itself (96 of the 208 sites). A source-level fix is
not feasible at this scale; the comment now says so instead of
blaming Doctest alone.
- -Wno-missing-noreturn had exactly two real trigger sites, both
genuinely and unconditionally non-returning: a test-only throwing
allocator (tests/src/unit-allocator.cpp) and, previously undiscovered,
wide_string_input_adapter::get_elements<T>() in
include/nlohmann/detail/input/input_adapters.hpp. Verified this isn't
a wider pattern by checking all 160 JSON_THROW call sites in the
library for functions whose entire body is an unconditional throw.
Annotated both ([[noreturn]] in the test file, since JSON_HEDLEY_NO_RETURN
is #undef'd by the time test code runs; JSON_HEDLEY_NO_RETURN in the
library file, its first real use anywhere in the codebase) and
dropped the suppression entirely.
single_include/nlohmann/json.hpp regenerated via `make amalgamate`;
`make check-amalgamation` passes.
Verified in Docker (silkeh/clang:dev, matching the ci_static_analysis_clang
CI job): baseline builds clean, and the full 194-target test suite builds
with zero warnings under the corrected CLANG_CXXFLAGS (-Wno-missing-noreturn
no longer in the list). Also sanity-compiled and ran unit-allocator.cpp and
unit-wstring.cpp on host Apple Clang to confirm behavior is unchanged.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix MSVC C4702 warning caused by JSON_HEDLEY_NO_RETURN on get_elements()
PR #5250 annotated wide_string_input_adapter::get_elements<T>() with
JSON_HEDLEY_NO_RETURN (it unconditionally throws). On MSVC this expands to
__declspec(noreturn), and MSVC correctly determined that the code following
its call in binary_reader.hpp is unreachable for that instantiation, firing
C4702 under /W4 /WX in the msvc, msvc-vs2026, and msvc-arm64 Debug jobs.
Clang doesn't flag this case, so the Docker verification for #5250 (which
only checked Clang -Weverything) didn't catch it.
This is the same warning class already tolerated for Release builds since
PR #5216, where MSVC's optimizer independently found the same dead code
after /Od was removed. Extend that existing /wd4702 suppression to Debug
builds too, instead of reverting the noreturn annotation.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Document std::optional<T> direct-init/copy-init limitation with null
Add regression test pinning current behavior (CHECK_THROWS_AS) in the null
section of unit-conversions.cpp with detailed comment explaining the C++
language-level cause (std::optional's own converting constructor wins
overload resolution over basic_json::operator T()).
Add a warning callout in conversions.md documenting that direct construction/
assignment of std::optional<T> from JSON null throws type_error 302, with a
clear workaround (use get<std::optional<T>>() or get_to() instead, which
correctly produce std::nullopt).
This is a limitation at the language level: there is no SFINAE path to
distinguish "called from inside std::optional's own constructor" from "direct
call", so fixing it would require breaking changes to operator ValueType().
A permanent fix belongs in the 4.0 type-strictness redesign (#3453).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-Authored-By: Claude Code <noreply@anthropic.com>
* Fix issue reference in std::optional test comment
Update the comment in the null section test to reference #5246 instead of
placeholder #XXXX, clarifying where the direct-init/copy-init limitation is tracked.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Use CHECK_THROWS_AS_WITH for std::optional test assertions
Update the regression tests to use CHECK_THROWS_AS_WITH instead of
CHECK_THROWS_AS to verify both the exception type and the error message.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix CI: use CHECK_THROWS_WITH_AS, the macro that actually exists
CHECK_THROWS_AS_WITH is not a doctest macro; the correct one used throughout
this test suite is CHECK_THROWS_WITH_AS(expr, message, exception_type&), with
the message before the type and the type as a reference. The previous commit
didn't catch this because it only compiled the file standalone with default
settings; this TEST_CASE only compiles under
`#if !JSON_USE_IMPLICIT_CONVERSIONS`, which is why ci_test_noimplicitconversions
was the job that failed. Verified by building and running the test in that
exact configuration (JSON_USE_IMPLICIT_CONVERSIONS=0): 14/14 assertions pass.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Run std::optional test under default implicit-conversions build too
TEST_CASE("std::optional") was guarded by #if !JSON_USE_IMPLICIT_CONVERSIONS,
so it only ever compiled in the non-default build with implicit conversions
disabled. This traces back to commit 1d7688aef (fixes#3859), which changed a
previously dead #ifndef JSON_USE_IMPLICIT_CONVERSIONS guard (the macro is
always defined by that point, so it never held) to #if !JSON_USE_IMPLICIT_CONVERSIONS
-- making the test compile for the first time, but only in the disabled-conversions
build. As a result, std::optional support had zero test coverage in the default
configuration almost every user builds with.
Verified the entire test case (all sections: null, string, bool, number, array,
object) compiles and passes identically with JSON_USE_IMPLICIT_CONVERSIONS both
on (default) and off -- nothing in it actually depends on the setting. Removing
the guard closes the coverage gap with no behavior change: 285 assertions pass
with implicit conversions on, 232 with them off (the difference comes from
other, unrelated conditionally-compiled tests in this file).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 🚨 fix warning
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Code <noreply@anthropic.com>
* Test ci_cuda_example against a CUDA version matrix at C++20 (#3907)
The ci_cuda_example job compiled against the json-ci image's CUDA
11.0 toolkit at cuda_std_11, which cannot exercise #3907 (a c++20
parse error in iteration_proxy.hpp's enable_borrowed_range reported
under nvcc). Switch the job to pull official nvidia/cuda devel images
directly and matrix across CUDA 11.8-12.6 at cuda_std_20 so CI can
empirically confirm which versions are actually affected before any
source-level fix is attempted.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix nvcc CUDA 12.0/12.1 C++20 ranges parse error (#3907)
The diagnostic matrix in this PR confirmed the affected range exactly:
nvcc 12.0.1 and 12.1.1 both fail with "expected initializer before
'<' token" on iteration_proxy.hpp's enable_borrowed_range variable
template specialization at -std=c++20; 12.2.2 and newer already build
cleanly. Guard JSON_HAS_RANGES off for that narrow nvcc version range,
matching the existing GCC-11/libstdc++ carve-outs in the same ifdef
chain, and regenerate single_include accordingly.
Broaden the CUDA smoke test to also exercise comparisons
(operator==/operator<=>, gated independently by
JSON_HAS_THREE_WAY_COMPARISON) and range-based iteration, not just
dump()/erase(), so the fix's actual scope is evidenced by CI rather
than assumed from the single reported symptom.
Have tests/cuda_example/CMakeLists.txt pick the newest C++ standard
the detected nvcc version actually supports (20/17/11) instead of
hard-requiring C++20, so older toolkits build at a lower standard
instead of failing CMake configure outright. This is test-project-local
only; the JSON_HAS_RANGES guard is what protects real client code,
since a header can't control what -std= flag it's compiled with.
Right-size the CI matrix from the 8-version diagnostic sweep down to
11.8.0 (C++17 fallback path) / 12.1.1 (permanent #3907 regression
guard) / 12.6.3 (recent coverage), and update the compiler-version
table in the quality assurance docs to match.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix ci_cuda_example CUDA 11.8 build after C++17 fallback (#3907)
The 11.8.0 leg's graceful C++17 fallback (added in the previous commit)
worked correctly, but the broadened smoke test used the <=> operator
unconditionally, which isn't valid syntax pre-C++20 — nvcc rejected it
with "expected an expression" once the CMake logic picked cuda_std_17
for the older toolkit. Gate those two lines behind
JSON_HAS_THREE_WAY_COMPARISON like the library itself does internally.
Sanity-compiled the file as plain C++ at both -std=c++17 (skips the
guarded block) and -std=c++20 (includes it) locally; the actual nvcc
build is verified via CI on PR #5248.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 📝 Fix documentation gaps for 3.13.0 release (todos 138-142)
- Todo 138: Add "Known issues" section to modules.md with compiler-specific troubleshooting (GCC redefinition, MSVC symbol export). Add pointer note to quality_assurance.md.
- Todo 139: Document CBOR/MessagePack half-precision float encoding for NaN/Infinity (0xF9/0xCA with exact byte sequences). Explain pre-3.13.0 double-precision bug mechanism without issue citations.
- Todo 140: Document CBOR negative-integer-overflow rejection (parse_error.112) for magnitudes exceeding int64_t range (already implemented in rev 1).
- Todo 141: Update version history in value.md and operator[].md with behavior-change details, removing issue citations per citation policy (prose is self-contained).
- Todo 142: Global sed replace of 3.12.x → 3.13.0 placeholder across all 20 documentation files.
Revision 2 incorporates feedback to reduce changelog-like issue citations. Only citations that add unique troubleshooting value are retained (#5103 for GCC workaround, #3970 for MSVC symbol export). "Known issues" section follows PR #5252's visual pattern (info admonition with bold-bullet format).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 📝 Document integer type selection, type_name() invalid value, and std::optional get() fix
- number_handling.md: clarify that positive/negative integers select
unsigned/signed storage based on the leading minus sign (todo 143).
- type_name.md: document the new "invalid" return value for corrupted
JSON values (todo 145).
- get.md: note that get<std::optional<T>>() was unreachable in every
configuration prior to 3.13.0 due to an internal macro-guard bug,
unrelated to JSON_USE_IMPLICIT_CONVERSIONS's actual effect (todo 144).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 📝 Document compiler/stdlib exclusions in macro_scope.hpp
Add "Known compiler/stdlib exclusions" subsections to the public documentation for
JSON_HAS_FILESYSTEM and JSON_HAS_RANGES, listing the exact compiler/stdlib versions
that are silently excluded even when feature-test macros indicate support. Each
exclusion references the originating issue. Also add a pointer note in the compiler
compatibility section linking to these details.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-Authored-By: Claude Code <noreply@anthropic.com>
* 💚 fix build
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Code <noreply@anthropic.com>
When converting objects or strings between different basic_json specializations,
the target's object_t::key_type or string_t must be directly constructible from
the source's corresponding type. If this requirement is not met, the conversion
silently falls back to the array-conversion path, producing incorrect results.
This documents the limitation and provides references to issue #3425, which tracks
this behavior. The comment in unit-alt-string.cpp is clarified to reference the
known limitation with a link to the issue, and suggests the parse() workaround.
Fixes#3425 (documentation; full fix deferred pending type-trait redesign)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Replace snprintf with a branch-free writer for \uXXXX escapes
dump_escaped called std::snprintf(..., "\u%04x", ...) once per escaped
code point in the string serialization hot path. snprintf re-parses
the format string and pulls in locale/printf machinery on every call,
which is far heavier than the fixed 6-/12-byte output warrants. This
is hot for any string containing control characters, and for all
non-ASCII text when ensure_ascii is set.
Replace it with write_u_escape, a small helper that writes the escape
directly into string_buffer via a nibble-to-hex lookup table, mirroring
the existing hand-rolled dump_integer fast path in the same file.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix clang-tidy avoid-c-arrays warning in write_u_escape
Use a const char* rather than a char[] lookup table, matching the
existing hex_bytes helper in the same file.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* ♻️ adjust write_u_escape signature
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Implement the scoped agent-readiness subset for json.nlohmann.me:
- Add the mkdocs-llmstxt plugin to generate llms.txt from the nav
(full_output/llms-full.txt deliberately omitted to avoid dumping
500+ API reference pages into one giant file).
- Add a permissive robots.txt with a Sitemap reference.
- Add a build hook (hooks/copy_markdown_source.py) that copies each
page's Markdown source into the built site as a `<path>.md` sibling
of its HTML output, so agents/tools can fetch raw Markdown directly.
sitemap.xml was already emitted by default and needed no change.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
the standard only specifies that the first elements are sorted.
this caused my experimental C++ standard library implementation to fail.
Signed-off-by: Paul Dreik <github@pauldreik.se>
`lexer::get()` copied every scanned character into `token_string` on the
whole successful-parse hot path, yet that buffer is consumed only by
`get_token_string()` when rendering the "last read" fragment of a parse
error. On well-formed input the per-byte copy (plus the `unget()` pop)
is pure overhead that is always discarded.
For seekable input adapters - random-access, single-byte iterators such
as those backing `std::string`, `const char*`, and `std::vector<char>` -
the offending token is now reconstructed on demand from the input when
an error is reported, using a saved start offset, and the eager copy is
skipped. Streaming adapters (file, istream, wide-string, and user-defined
adapters) keep the eager copy; the strategy is chosen at compile time via
`input_adapter_supports_seek`, so adapters without the capability are
unaffected.
Error messages are byte-for-byte identical across all adapters, verified
by a new parity regression test. Microbenchmark (4 MB mixed JSON, parsed
from a std::string): ~149 -> ~160 MB/s, about +8%.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* Harden CI workflows: validate PR artifact inputs and migrate off deprecated Semgrep action
Address two CI/supply-chain hardening items from the 2026-07-03 security
audit:
- comment_check_amalgamation.yml (todo 117): the privileged `workflow_run`
job consumes an untrusted PR artifact. Validate `author` against a strict
GitHub-username pattern and `number` as a positive integer before use, and
extract the artifact into a dedicated directory (`unzip -o pr.zip -d
./pr_artifact`), reading only the two expected files by fixed path. This
prevents Markdown/mention injection via the attacker-controlled `author`
text and avoids a malicious archive touching the workspace.
- semgrep.yml (todo 118): `returntocorp/semgrep-action` is deprecated (the
org was renamed to `semgrep/*`). Replace it with an explicit `semgrep ci`
invocation via the maintained CLI; the deployment is inferred from
SEMGREP_APP_TOKEN.
Todo 116 (CIFuzz `@master` refs) already carries a comment documenting the
OSS-Fuzz-recommended exception, so no change is needed there.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Fix Semgrep step: use `semgrep scan` instead of token-gated `semgrep ci`
The CI `Scan` job failed with "Path does not exist: semgrep.sarif" because
`semgrep ci` requires a login token (SEMGREP_APP_TOKEN), which this repo does
not have configured, so it bailed without producing a SARIF file. The former
returntocorp/semgrep-action, given no token, fell back to plain
`semgrep scan --sarif`; match that with `semgrep scan --config auto`, which
needs no token and always produces the SARIF for upload.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Bumps init/analyze/autobuild together (previously split across #5226,
#5227, #5228, which each failed CI due to a version mismatch between
the CodeQL config and the running action). Also adds a dependabot
group so future codeql-action bumps land in a single PR.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
When a PR is not amalgamated/formatted, the astyle version friction (see
the recurring blocker across many PRs) means contributors often struggle
to reproduce the exact fix locally. The check now regenerates the
amalgamation and formatting, captures the difference as a patch, and
uploads it as the `amalgamation-patch` artifact. The failure comment
links to that artifact and tells contributors to run
`git apply amalgamation.patch`, so they no longer need to install the
pinned astyle version themselves.
The pass/fail verdict is unchanged: the same PRs fail as before, and a
correctly amalgamated PR uploads nothing and passes.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Add /wd4702 to disable warning C4702: unreachable code in MSVC Release build.
Signed-off-by: Richard Musil <risa2000x@gmail.com>
Co-authored-by: Richard Musil <risa2000x@gmail.com>
* Added NLOHNMANN_JSON_SERIALIZE_ENUM_STRICT
- duplicate of NLOHMANN_JSON_SERIALIZE_ENUM
Signed-off-by: Caillin Nugent <caillinn@student.unimelb.edu.au>
* Added failing tests for NLOHMANN_JSON_SERIALIZE_ENUM_STRICT
Signed-off-by: Caillin Nugent <caillinn@student.unimelb.edu.au>
* modified NLOHMANN_JSON_SERIALIZE_STRICT to throw
Signed-off-by: Caillin Nugent <caillinn@student.unimelb.edu.au>
* added documentation and changed readme to include NLOHMANN_JSON_SERIALIZE_ENUM_STRICT
Signed-off-by: Caillin Nugent <caillinn@student.unimelb.edu.au>
* ran amalgamate
Signed-off-by: Caillin Nugent <caillinn@student.unimelb.edu.au>
* docs(macros): add page for JSON_SERIALIZE_ENUM_STRICT
- added page to nav
- added links to new page where appropriate
Signed-off-by: Caillin Nugent <caillinn@student.unimelb.edu.au>
* refactor(macros): make JSON_SERIALIZE_ENUM_STRICT use JSON_THROW
- added templated wrapper function to fix scope error in calling JSON_THROW
Signed-off-by: Caillin Nugent <caillinn@student.unimelb.edu.au>
* refactor(macros): make NLOHMANN_SERIALIZE_ENUM_STRICT use error code 410
- added error code 410 to docs
Signed-off-by: Caillin Nugent <caillinn@student.unimelb.edu.au>
* tests(macros): add test for to_json with enum value not mentioned
in mapping for NLOHMANN_JSON_SERIALIZE_ENUM_STRICT
Signed-off-by: Caillin Nugent <caillinn@student.unimelb.edu.au>
* Apply suggestions from code review
Co-authored-by: Niels Lohmann <niels.lohmann@gmail.com>
Signed-off-by: Caillin Nugent <nugentcaillin@gmail.com>
* fix(macro): prevent compilation error with -Werror and -Wunused-parameter
with NLOHMANN_JSON_SERIALIZE_ENUM_STRICT
- casted exception to void to avoid warning
Signed-off-by: Caillin Nugent <caillinn@student.unimelb.edu.au>
* fix(docs): add link to NLOHMANN_SERIALIZE_ENUM_STRICT docs to exception page
Signed-off-by: Caillin Nugent <caillinn@student.unimelb.edu.au>
* docs(macros): add example of exception throwing for NLOHMANN_JSON_SERIALIZE_ENUM_STRICT
Signed-off-by: Caillin Nugent <caillinn@student.unimelb.edu.au>
* refactor(macros): add more in-depth error message to NLOHMANN_JSON_SERIALIZE_ENUM_STRICT
- changed error message to follow style of nlohmann/json#4989
- made description of throw wrapper more general
- updated tests and example of exceptions
Signed-off-by: Caillin Nugent <caillinn@student.unimelb.edu.au>
---------
Signed-off-by: Caillin Nugent <caillinn@student.unimelb.edu.au>
Signed-off-by: Caillin Nugent <nugentcaillin@gmail.com>
Co-authored-by: Niels Lohmann <niels.lohmann@gmail.com>
* Fix for printing long doubles bug in dump_float
When you use long double as a floating point type with the current version of this file and try to dump json it prints trash instead of actual number. This if-else fixes the problem. On using long double you just need to add an 'L' modifier before 'g' in format string.
Signed-off-by: Kirill Lokotkov <klokotkov@ya.ru>
* C++11 compatibility
Signed-off-by: Kirill Lokotkov <klokotkov@ya.ru>
* Shorter solution
Signed-off-by: Kirill Lokotkov <klokotkov@ya.ru>
* Applied amalgamate
Signed-off-by: rusloker <klokotkov@ya.ru>
* Add unit tests for `dump()` with `long double` in custom `basic_json`
Signed-off-by: rusloker <klokotkov@ya.ru>
* Fix UB in `snprintf_float` by using `%.*Lg` for `long double`
Signed-off-by: rusloker <klokotkov@ya.ru>
* Use `std::array` for `values` in serialization unit tests to improve type safety
Signed-off-by: rusloker <klokotkov@ya.ru>
* Fix brace initialization for `std::array` in serialization unit tests
Signed-off-by: rusloker <klokotkov@ya.ru>
* Remove comments in `snprintf_float` regarding `%Lg` usage
Signed-off-by: rusloker <klokotkov@ya.ru>
* Skip `long double` infinity dump assertions under Valgrind
Signed-off-by: rusloker <klokotkov@ya.ru>
* Clarify Valgrind bug-tracker reference in `long double` test
Signed-off-by: rusloker <klokotkov@ya.ru>
* Satisfy clang-tidy in `long double` infinity probe
Signed-off-by: rusloker <klokotkov@ya.ru>
---------
Signed-off-by: Kirill Lokotkov <klokotkov@ya.ru>
Signed-off-by: rusloker <klokotkov@ya.ru>
* Add new macros for named conversions
* Unit tests for the named conversion macros
* Update the docs to include the new macros
* Fix the documentation for the macros
the correct maximum number of member variables is 63
* Fix CI tests
* update the named macros
* move the example files
* update the explicit macros expansion
* update documentation
* fix documentation hiccups
* astyle changes
* add static analysis exceptions
* change md header to explicit html to fit the length
* Small corrections to docs
Co-authored-by: Niels Lohmann <niels.lohmann@gmail.com>
Signed-off-by: George Sedov <radist.morse@gmail.com>
---------
Signed-off-by: George Sedov <radist.morse@gmail.com>
Co-authored-by: Niels Lohmann <niels.lohmann@gmail.com>
* Fix compile error when using nlohmann ordered_map with WITH_DEFAULT macros
ordered_map inherits its copy and move assignment from the underlying std vector, which requires value_type to be CopyAssignable. value_type is pair<const Key, T> whose assignment is deleted because of the const Key, so any code that assigns ordered_map (for example the ternary in NLOHMANN_JSON_FROM_WITH_DEFAULT) fails to compile (issue #5122). Provide assignment operators on ordered_map that rebuild via clear plus push_back for copy and transfer the underlying buffer for move, neither of which needs pair assignment. Also switch the map-shaped from_json overload from a transform plus inserter idiom to a range-for plus emplace, which avoids the same hazard.
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
* Update ordered_map.hpp
removed unwanted comments
Signed-off-by: SamareshSingh <97642706+ssam18@users.noreply.github.com>
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
* Update json.hpp
Signed-off-by: SamareshSingh <97642706+ssam18@users.noreply.github.com>
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
* Address CI issues for ordered_map fix
Declare an explicit defaulted destructor on ordered_map so the rule of five is complete (clang-tidy cppcoreguidelines-special-member-functions and hicpp-special-member-functions). Initialize the ordered_map field in the regression test struct so GCC effective-C++ stops flagging Example_5122 with a missing member initializer.
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
* Suppress redundant-member-init lint on Example_5122::c
The empty brace-init on c{} is required by GCC -Weffc++ to mark the member as initialized in the synthesized default constructor, but clang-tidy readability-redundant-member-init flags the same line because ordered_map already has a default constructor. The two checks pull in opposite directions, so add a targeted NOLINT to keep both happy.
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
* Address review: strong exception safety in copy-assign, simplify move-assign noexcept
Copy assignment now constructs a temporary copy before move-assigning the
Container subobject, preserving *this if the copy throws. Move assignment
uses std::is_nothrow_move_assignable<Container> for a cleaner noexcept
specifier, matching the style of the move constructor.
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
* Restore self-assignment check in copy-assign to satisfy cert-oop54-cpp
clang-tidy's cert-oop54-cpp flagged the previous revision because it could
not recognize the implicit self-safety of the copy-then-move pattern.
Restore the explicit `if (this != &other)` guard — strong exception safety
is preserved since the temporary copy is still constructed before the
move-assign of the Container subobject.
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
* Address review: add explicit self-assignment and move-assignment tests for ordered_map
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
* Address review: gate -Wself-assign-overloaded suppression on Clang version
-Wself-assign-overloaded was introduced in Clang 7. Older Clang versions fail the build with "unknown warning group" when the suppression pragma references it unconditionally. Use __has_warning inside an __clang__ branch so the suppression is only emitted on Clang versions that recognize the warning. The inner check stays inside the __clang__ guard because GCC does not provide __has_warning and would tokenize-error on the argument list.
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
* Address CI: drop unused gating macro to silence -Wunused-macros
The previous attempt defined JSON_TEST_5122_SUPPRESS_SELF_ASSIGN_OVERLOADED
as 0 unconditionally and then overrode it to 1 on Clang versions that recognize the warning. On those Clangs the initial define is immediately
undef'd without being read, which trips Clang's -Wunused-macros under -Weverything in the ci_test_clang job. Drop the macro and gate the
DOCTEST_CLANG_SUPPRESS_WARNING_PUSH/POP pragmas directly with __has_warning inside the existing __clang__ branch.
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
---------
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
Signed-off-by: SamareshSingh <97642706+ssam18@users.noreply.github.com>
When iteration_proxy_value<iter_impl<ordered_json>> appears in a context
that requires it to be complete (function or lambda parameter), the
compiler instantiates basic_json<ordered_map> and walks into
set_parents(iterator, typename iterator::difference_type)
while iterator is still incomplete, failing with "invalid use of
incomplete type".
basic_json::difference_type is already std::ptrdiff_t, so just naming
the underlying type directly avoids the dependent lookup. Behavior and
ABI are unchanged. This was the approach suggested in the issue thread.
Added a regression case in unit-ordered_json.cpp using the same trigger
pattern (lambda parameter naming the proxy type).
Fixes#3732
Signed-off-by: Akhilesh Arora <akhildawra@gmail.com>
* fix: treat single-element brace-init as copy/move
When passing a json value using brace initialization with a single element
(e.g., `json j{someObj}` or `foo({someJson})`), C++ always prefers the
initializer_list constructor over the copy/move constructor. This caused
the value to be unexpectedly wrapped in a single-element array.
This bug was previously compiler-dependent (GCC wrapped, Clang did not),
but Clang 20 started matching GCC behavior, making it a universal issue.
Fix: In the initializer_list constructor, when type deduction is enabled
and the list has exactly one element, copy/move it directly instead of
creating a single-element array.
Before:
json obj = {{"key", 1}};
json j{obj}; // -> [{"key":1}] (wrong: array)
foo({obj}); // -> [{"key":1}] (wrong: array)
After:
json j{obj}; // -> {"key":1} (correct: copy)
foo({obj}); // -> {"key":1} (correct: copy)
To explicitly create a single-element array, use json::array({value}).
Fixes the issue #5074
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
* fix: regenerate amalgamated single_include/nlohmann/json.hpp
- Add missing comment from include/nlohmann/json.hpp explaining the
single-element brace-init fix (issue #5074)
- Fix extra 4-space indentation in embedded json_fwd.hpp section
Regenerated by running: make amalgamate
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
* Revert brace-init semantics change and fix amalgamation
The single-element brace-init change was a breaking change that cannot be accepted upstream. Reverted all related source, test, and doc changes, then regenerated single_include with correct indentation to pass the amalgamation CI check.
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
* Fix: add JSON_BRACE_INIT_COPY_SEMANTICS opt-in macro for issue #5074
Single-element brace initialization wrapping in an array cannot be fixed without breaking existing code. Added JSON_BRACE_INIT_COPY_SEMANTICS as an opt-in macro (default 0) so users can enable copy/move semantics for single-element brace init without affecting anyone relying on the current behavior.
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
* docs: add dedicated macro page and CI test target for JSON_BRACE_INIT_COPY_SEMANTICS
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
* fix: remove compiler-dependent assertions from #5074 regression test
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
* fix: use defined() guard for JSON_BRACE_INIT_COPY_SEMANTICS to satisfy -Wundef
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
* docs: fix section name in json_brace_init_copy_semantics.md to pass style check
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
* docs: move Default definition section before Notes to fix style check order
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
---------
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
PR #4873 introduced a safety check in sax_parse functions to catch
nullptr passed as SAX parser object, which had been already annotated by
JSON_HEDLEY_NON_NULL macro.
Compilers (e.g. clang) which respected the non-null annotation tended to
eliminate the safety check completely in optimized builds, while
compilers which did not, compiled the safety check in. This led to
different behaviors accross different compilers/platforms and/or build
types (debug, release).
This commit reverts PR #4873 to remove this discrepancy. Passing null to
non-null annotated parameter is considered to be undefined behavior.
Fixes#5048
Signed-off-by: Richard Musil <risa2000x@gmail.com>
Co-authored-by: Richard Musil <risa2000x@gmail.com>
* Add reference handling to tuples
Signed-off-by: Evelyn LePain <ava.lepain@gmail.com>
* Remove template template type because pair isn't working
Signed-off-by: Evelyn LePain <ava.lepain@gmail.com>
* amalgamate std::tie changes
Signed-off-by: Evelyn LePain <ava.lepain@gmail.com>
* allow the elation of a move by removing the ref requirement
Signed-off-by: Evelyn LePain <ava.lepain@gmail.com>
* force all number_xxx_t to be interchangeable
Signed-off-by: Evelyn LePain <ava.lepain@gmail.com>
* Finally got amalgamate to work correctly
Signed-off-by: Evelyn LePain <ava.lepain@gmail.com>
* remove const version, add a test case for scrambled number representations.
Signed-off-by: Evelyn LePain <ava.lepain@gmail.com>
* Use the logical set of requirements instead of decltype because VS 2015 doesn't like it
Signed-off-by: Evelyn LePain <ava.lepain@gmail.com>
---------
Signed-off-by: Evelyn LePain <ava.lepain@gmail.com>
* fix(cbor): reject negative ints overflowing int64
CBOR encodes negative integers as "-1 - n" where n is uint64_t. When
n > INT64_MAX, casting to int64_t caused undefined behavior and silent
data corruption. Large negative values were incorrectly parsed as
positive integers (e.g., -9223372036854775809 became 9223372036854775807).
Add bounds check for to reject values that exceed int64_t
representable range, returning parse_error instead of silently
corrupting data.
Added regression test cases to verify.
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
* chore: clarify tests
Add test for "n=0" case (result=-1) to cover the smallest magnitude
boundary. Update comments to explain CBOR 0x3B encoding and why
"result=0" is not possible. Clarify that n is an unsigned integer
in the formula "result = -1 - n" to help understanding the tests.
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
* fix(cbor): extend overflow checks for other types
Extend negative integer overflow detection to all CBOR negative
integer cases (0x38, 0x39, 0x3A) for consistency with the existing
0x3B check.
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
---------
Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
Fixes#3659
I was testing serve_header.py with my local development setup and noticed
that when I moved directories into or out of the monitored root, they
weren't being picked up properly. The script would only detect create and
delete events but not move operations.
This was happening because the on_any_event handler only checked for
'created' and 'deleted' events on directories. Move events have a
separate event type 'moved' that includes both the source and destination
paths.
The fix treats a move event like a combination of delete (for the source)
and create (for the destination) - we rescan to remove any trees that were
moved out, and add the destination directory to check for new trees that
were moved in.
This should make the development workflow smoother when reorganizing
project directories while the server is running.
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
* Improve MSVC C++ modules compatibility for nlohmann_json
- Keep complete module interface with all exports
- Export all standard types: json, basic_json, adl_serializer, json_pointer, ordered_json, ordered_map
- Export detail namespace symbols for advanced usage
- Ensure compatibility with MSVC C++20/23 modules
- Maintain full functionality without breaking changes
Signed-off-by: yudaichen <250074249@qq.com>
* Add C++20 module Windows CI test and fix file ending
- Add ci_module_cpp20 job to Windows workflow to verify module support with MSVC
- Add missing newline at end of json.cppm file
Signed-off-by: yudaichen <250074249@qq.com>
---------
Signed-off-by: yudaichen <250074249@qq.com>
* 👷 fix Windows CI
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 👷 fix Windows CI
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 👷 fix Windows CI
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 👷 fix Windows CI
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 👷 fix Windows CI
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 👷 fix Windows CI
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 👷 fix Windows CI
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 👷 fix Windows CI
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 👷 fix Windows CI
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 👷 fix Windows CI
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 👷 fix Windows CI
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 👷 fix Windows CI
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 👷 fix Windows CI
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 👷 fix Windows CI
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
@@ -108,7 +108,9 @@ The tests are located in [`tests/src/unit-*.cpp`](https://github.com/nlohmann/js
are structured along the features of the library or the nature of the tests. Usually, it should be clear from the
context which existing file needs to be extended, and only very few cases require creating new test files.
When fixing a bug, edit `unit-regression2.cpp` and add a section referencing the fixed issue.
When fixing a bug, edit `unit-regression3.cpp` and add a section referencing the fixed issue.
`unit-regression2.cpp` holds the older tests; the two files exist because a single one grew large enough for the
MinGW linker to fail relocating it, so please keep adding to the smaller file rather than growing the larger one.
#### Exceptions
@@ -153,6 +155,18 @@ files directly; instead, modify the include/nlohmann sources and regenerate the
make amalgamate
```
Running `make amalgamate` will also apply automatic formatting to the source files using
[`Artistic Style`](https://astyle.sourceforge.net/). This formatting may modify your source files in-place. Be certain to review and commit any changes to avoid unintended formatting diffs in commits.
If you add, rename, or remove a header in `include/nlohmann`, also regenerate the header list in
[`BUILD.bazel`](https://github.com/nlohmann/json/blob/develop/BUILD.bazel) (requires CMake) by executing:
```shell
make BUILD.bazel
```
The amalgamation check in CI fails if any of these generated files is out of date.
## Recommended documentation
- The library’s [README file](https://github.com/nlohmann/json/blob/master/README.md) is an excellent starting point to
- [ ] The changes are described in detail, both the what and why.
- [ ] If applicable, an [existing issue](https://github.com/nlohmann/json/issues) is referenced.
- [ ] If applicable, a fixed [OSS-Fuzz](https://issues.oss-fuzz.com) issue is referenced as `OSS-Fuzz: <id>` (see [fuzz testing](https://github.com/nlohmann/json/blob/develop/tests/fuzzing.md#handling-oss-fuzz-reports)).
- [ ] The [Code coverage](https://coveralls.io/github/nlohmann/json) remained at 100%. A test case for every new line of code.
- [ ] If applicable, the [documentation](https://json.nlohmann.me) is updated.
- [ ] The source code is amalgamated by running `make amalgamate`.
We would appreciate it if you could provide us with more info about this issue or pull request! Please check the [issue template](https://github.com/nlohmann/json/blob/develop/.github/ISSUE_TEMPLATE.md) and the [pull request template](https://github.com/nlohmann/json/blob/develop/.github/PULL_REQUEST_TEMPLATE.md).
We would appreciate it if you could provide us with more info about this issue or pull request! Please check the [issue template](https://github.com/nlohmann/json/issues/new/choose) and the [pull request template](https://github.com/nlohmann/json/blob/develop/.github/PULL_REQUEST_TEMPLATE.md).
# *OPTIONAL* Label to be added to Issues and Pull Requests with insufficient information given
@@ -9,6 +9,30 @@ This file describes the source for supporting files; that is, files that are not
## Continuous Integration
### `.github/workflows`
The [GitHub Actions](https://docs.github.com/en/actions) workflows that build, test, and analyze the library. Each file in this folder defines one workflow:
-`ubuntu.yml`, `macos.yml`, `windows.yml` — build and run the test suite on Linux, macOS, and Windows.
-`check_amalgamation.yml` — verify that the single-header amalgamation in `single_include` is up to date on pull requests.
-`comment_check_amalgamation.yml` — comment on a pull request when the amalgamation check failed.
-`cifuzz.yml` — run short fuzzing sessions via [OSS-Fuzz CIFuzz](https://google.github.io/oss-fuzz/getting-started/continuous-integration/) on pull requests.
-`codeql-analysis.yml` — run [CodeQL](https://codeql.github.com) code scanning.
-`flawfinder.yml` — run the [Flawfinder](https://dwheeler.com/flawfinder/) static analysis.
-`semgrep.yml` — run [Semgrep](https://semgrep.dev) static analysis.
-`scorecards.yml` — run the [OpenSSF Scorecard](https://securityscorecards.dev) supply-chain security checks.
-`dependency-review.yml` — scan dependency changes in pull requests for known vulnerabilities.
-`labeler.yml` — the "Pull Request Labeler" workflow (see `.github/labeler.yml`).
-`stale.yml` — comment on and close stale issues and pull requests.
-`publish_documentation.yml` — build and publish the documentation on every merge to the `develop` branch.
Further documentation:
- [Workflow syntax for GitHub Actions](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions)
> [!IMPORTANT]
> The folder `.github/workflows` is predetermined by GitHub.
### `.cirrus.yml`
Configuration file for the pipeline at [Cirrus CI](https://cirrus-ci.com/github/nlohmann/json).
@@ -123,7 +147,7 @@ Further documentation:
> [!IMPORTANT]
> The folder `.github/ISSUE_TEMPLATE` is predetermined by GitHub.
### `.github/ISSUE_TEMPLATE/config.yaml`
### `.github/ISSUE_TEMPLATE/config.yml`
Issue template chooser configuration. The file is used to configure the dialog when a new issue is created.
@@ -132,7 +156,7 @@ Further documentation:
- [Configuring issue templates for your repository](https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/configuring-issue-templates-for-your-repository)
> [!IMPORTANT]
> The filename `.github/ISSUE_TEMPLATE/config.yaml` is predetermined by GitHub.
> The filename `.github/ISSUE_TEMPLATE/config.yml` is predetermined by GitHub.
### `.github/labeler.yml`
@@ -165,7 +189,7 @@ Further documentation:
- [Adding a security policy to your repository](https://docs.github.com/en/code-security/getting-started/adding-a-security-policy-to-your-repository)
> [!IMPORTANT]
> The filename `.github/SECURITY.yml` is predetermined by GitHub.
> The filename `.github/SECURITY.md` is predetermined by GitHub.
> [!NOTE]
> The file is part of the documentation and is included in `docs/mkdocs/docs/community/security_policy.md`.
@@ -226,14 +250,28 @@ Further documentation:
### `BUILD.bazel`
The file can be updated by calling
The build definition for [Bazel](https://bazel.build). The file is generated by
`cmake/scripts/gen_bazel_build_file.cmake`, which derives the header list from the files in `include`; change the
script rather than editing the file by hand. The file can be updated by calling
```shell
make BUILD.bazel
```
The "Check amalgamation" workflow fails if the file is out of date.
### `meson.build`
The build definition for the [Meson](https://mesonbuild.com) build system.
### `Package.swift`
### `WORKSPACE.bazel`
The package manifest for the [Swift Package Manager](https://www.swift.org/package-manager/).
### `MODULE.bazel`
The module definition for [Bazel](https://bazel.build)'s [Bzlmod](https://bazel.build/external/module) dependency system. It complements `BUILD.bazel` and replaces the previously used `WORKSPACE.bazel`.
@diff BUILD.bazel BUILD.bazel~ ||(echo"===================================================================\n BUILD.bazel is out of date! Please run 'make BUILD.bazel'.\n==================================================================="; mv BUILD.bazel~ BUILD.bazel ;false)
@mv BUILD.bazel~ BUILD.bazel
BUILD.bazel:$(SRCS)
# generate the Bazel BUILD file; phony, because a removed header would not trigger a rebuild
[](https://isitmaintained.com/project/nlohmann/json "Average time to resolve an issue")
[](https://bestpractices.coreinfrastructure.org/projects/289)
- [Binary formats (BSON, CBOR, MessagePack, UBJSON, and BJData)](#binary-formats-bson-cbor-messagepack-ubjson-and-bjdata)
- [Customers](#customers)
- [Ecosystem](#ecosystem)
- [Supported compilers](#supported-compilers)
- [Integration](#integration)
- [CMake](#cmake)
@@ -69,7 +71,7 @@ Other aspects were not so important to us:
- **Speed**. There are certainly [faster JSON libraries](https://github.com/miloyip/nativejson-benchmark#parsing-time) out there. However, if your goal is to speed up your development by adding JSON support with a single header, then this library is the way to go. If you know how to use a `std::vector` or `std::map`, you are already set.
See the [contribution guidelines](https://github.com/nlohmann/json/blob/master/.github/CONTRIBUTING.md#please-dont) for more information.
See the [contribution guidelines](https://github.com/nlohmann/json/blob/develop/.github/CONTRIBUTING.md#please-dont) for more information.
## Sponsors
@@ -79,6 +81,8 @@ You can sponsor this library at [GitHub Sponsors](https://github.com/sponsors/nl
That's all! When calling the `json` constructor with your type, your custom `to_json` method will be automatically called.
Likewise, when calling `template get<your_type>()` or `get_to(your_type&)`, the `from_json` method will be called.
Likewise, when calling `get<your_type>()` or `get_to(your_type&)`, the `from_json` method will be called.
Some important things:
- Those methods **MUST** be in your type's namespace (which can be the global namespace), or the library will not be able to locate them (in this example, they are in namespace `ns`, where `person` is defined).
- Those methods **MUST** be available (e.g., proper headers must be included) everywhere you use these conversions. Look at [issue 1108](https://github.com/nlohmann/json/issues/1108) for errors that may occur otherwise.
- When using `template get<your_type>()`, `your_type`**MUST** be [DefaultConstructible](https://en.cppreference.com/w/cpp/named_req/DefaultConstructible). (There is a way to bypass this requirement described later.)
- When using `get<your_type>()`, `your_type`**MUST** be [DefaultConstructible](https://en.cppreference.com/w/cpp/named_req/DefaultConstructible). (There is a way to bypass this requirement described later.)
- In function `from_json`, use function [`at()`](https://json.nlohmann.me/api/basic_json/at/) to access the object values rather than `operator[]`. In case a key does not exist, `at` throws an exception that you can handle, whereas `operator[]` exhibits undefined behavior.
- You do not need to add serializers or deserializers for STL types like `std::vector`: the library already implements these.
#### Simplify your life with macros
If you just want to serialize/deserialize some structs, the `to_json`/`from_json` functions can be a lot of boilerplate. There are [**several macros**](https://json.nlohmann.me/features/arbitrary_types/#simplify-your-life-with-macros) to make your life easier as long as you (1) want to use a JSON object as serialization and (2) want to use the member variable names as object keys in that object.
If you just want to serialize/deserialize some structs, the `to_json`/`from_json` functions can be a lot of boilerplate. There are [**several macros**](https://json.nlohmann.me/api/macros/#serializationdeserialization-macros) to make your life easier as long as you want to use a JSON object as serialization.
Which macro to choose depends on whether private member variables need to be accessed, a deserialization is needed, missing values should yield an error or should be replaced by default values, and if derived classes are used. See [this overview to choose the right one for your use case](https://json.nlohmann.me/api/macros/#serializationdeserialization-macros).
Which macro to choose depends on whether private member variables need to be accessed, a deserialization is needed, missing values should yield an error or should be replaced by default values, and if derived classes are used. See [this overview to choose the right one for your use case](https://json.nlohmann.me/features/arbitrary_types/#simplify-your-life-with-macros).
##### Example usage of macros
@@ -853,6 +871,18 @@ namespace ns {
}
```
If you want to inherit the `person` struct and add a field to it, it can be done with:
Here is another example with private members, where [`NLOHMANN_DEFINE_TYPE_INTRUSIVE`](https://json.nlohmann.me/api/macros/nlohmann_define_type_intrusive/) is needed:
```cpp
@@ -869,6 +899,24 @@ namespace ns {
}
```
Or in case if you use some naming convention that you do not want to expose to JSON:
This requires a bit more advanced technique. But first, let's see how this conversion mechanism works:
@@ -913,8 +961,8 @@ namespace nlohmann {
if(j.is_null()){
opt=boost::none;
}else{
opt=j.templateget<T>();// same as above, but with
// adl_serializer<T>::from_json
opt=j.get<T>();// same as above, but with
// adl_serializer<T>::from_json
}
}
};
@@ -941,7 +989,7 @@ namespace nlohmann {
// note: the return type is no longer 'void', and the method only takes
// one argument
staticmove_only_typefrom_json(constjson&j){
return{j.templateget<int>()};
return{j.get<int>()};
}
// Here's the catch! You must provide a to_json method! Otherwise, you
@@ -1005,7 +1053,7 @@ struct bad_serializer
staticvoidto_json(constBasicJsonType&j,T&value){
// this calls BasicJsonType::json_serializer<T>::from_json(j, value)
// if BasicJsonType::json_serializer == bad_serializer ... oops!
value=j.templateget<T>();// oops!
value=j.get<T>();// oops!
}
};
```
@@ -1045,11 +1093,11 @@ assert(j == "stopped");
// json string to enum
jsonj3="running";
assert(j3.templateget<TaskState>()==TS_RUNNING);
assert(j3.get<TaskState>()==TS_RUNNING);
// undefined json value to enum (where the first map entry above is the default)
jsonjPi=3.14;
assert(jPi.templateget<TaskState>()==TS_INVALID);
assert(jPi.get<TaskState>()==TS_INVALID);
```
Just as in [Arbitrary Type Conversions](#arbitrary-types-conversions) above,
@@ -1059,7 +1107,7 @@ Just as in [Arbitrary Type Conversions](#arbitrary-types-conversions) above,
Other Important points:
- When using `template get<ENUM_TYPE>()`, undefined JSON values will default to the first pair specified in your map. Select this default pair carefully.
- When using `get<ENUM_TYPE>()`, undefined JSON values will default to the first pair specified in your map. Select this default pair carefully. If you desire an exception in this circumstance use `NLOHMANN_JSON_SERIALIZE_ENUM_STRICT()` which behaves identically except for throwing an exception on unrecognized values.
- If an enum or JSON value is specified more than once in your map, the first matching occurrence from the top of the map will be returned when converting to or from JSON.
### Binary formats (BSON, CBOR, MessagePack, UBJSON, and BJData)
@@ -1139,9 +1187,14 @@ The library is used in multiple projects, applications, operating systems, etc.
[](https://json.nlohmann.me/home/customers/)
## Ecosystem
Beyond projects that use the library, there are third-party projects that build on top of it - schema validators,
language bindings, format converters, and the like. See the curated [Ecosystem](https://json.nlohmann.me/community/ecosystem/) page.
## Supported compilers
Though it's 2025 already, the support for C++11 is still a bit sparse. Currently, the following compilers are known to work:
Though it's 2026 already, the support for C++11 is still a bit sparse. Currently, the following compilers are known to work:
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
@@ -1368,7 +1421,7 @@ I deeply appreciate the help of the following people.
6. [Joshua C. Randall](https://github.com/jrandall) fixed a bug in the floating-point serialization.
7. [Aaron Burghardt](https://github.com/aburgh) implemented code to parse streams incrementally. Furthermore, he greatly improved the parser class by allowing the definition of a filter function to discard undesired elements while parsing.
8. [Daniel Kopeček](https://github.com/dkopecek) fixed a bug in the compilation with GCC 5.0.
9. [Florian Weber](https://github.com/Florianjw) fixed a bug in and improved the performance of the comparison operators.
9. [Fiona Johanna Weber](https://github.com/Fiona-J-W) fixed a bug in and improved the performance of the comparison operators.
10. [Eric Cornelius](https://github.com/EricMCornelius) pointed out a bug in the handling with NaN and infinity values. He also improved the performance of the string escaping.
11. [易思龙](https://github.com/likebeta) implemented a conversion from anonymous enums.
12. [kepkin](https://github.com/kepkin) patiently pushed forward the support for Microsoft Visual Studio.
@@ -1419,7 +1472,7 @@ I deeply appreciate the help of the following people.
57. [Jared Grubb](https://github.com/jaredgrubb) supported the implementation of user-defined types.
58. [EnricoBilla](https://github.com/EnricoBilla) noted a typo in an example.
59. [Martin Hořeňovský](https://github.com/horenmar) found a way for a 2x speedup for the compilation time of the test suite.
60. [ukhegg](https://github.com/ukhegg) found proposed an improvement for the examples section.
60. [ukhegg](https://github.com/ukhegg) proposed an improvement for the examples section.
61. [rswanson-ihi](https://github.com/rswanson-ihi) noted a typo in the README.
62. [Mihai Stan](https://github.com/stanmihai4) fixed a bug in the comparison with `nullptr`s.
63. [Tushar Maheshwari](https://github.com/tusharpm) added [cotire](https://github.com/sakra/cotire) support to speed up the compilation.
@@ -1470,14 +1523,14 @@ I deeply appreciate the help of the following people.
108. [Kevin Tonon](https://github.com/ktonon) overworked the C++11 compiler checks in CMake.
109. [Axel Huebl](https://github.com/ax3l) simplified a CMake check and added support for the [Spack package manager](https://spack.io).
110. [Carlos O'Ryan](https://github.com/coryan) fixed a typo.
111. [James Upjohn](https://github.com/jammehcow) fixed a version number in the compilers section.
111. [James Upjohn](https://github.com/jupjohn) fixed a version number in the compilers section.
112. [Chuck Atkins](https://github.com/chuckatkins) adjusted the CMake files to the CMake packaging guidelines and provided documentation for the CMake integration.
113. [Jan Schöppach](https://github.com/dns13) fixed a typo.
114. [martin-mfg](https://github.com/martin-mfg) fixed a typo.
115. [Matthias Möller](https://github.com/TinyTinni) removed the dependency from `std::stringstream`.
116. [agrianius](https://github.com/agrianius) added code to use alternative string implementations.
117. [Daniel599](https://github.com/Daniel599) allowed to use more algorithms with the `items()` function.
118. [Julius Rakow](https://github.com/jrakow) fixed the Meson include directory and fixed the links to [cppreference.com](https://cppreference.com).
118. [Julius Rakow](https://github.com/juliusrakow) fixed the Meson include directory and fixed the links to [cppreference.com](https://cppreference.com).
119. [Sonu Lohani](https://github.com/sonulohani) fixed the compilation with MSVC 2015 in debug mode.
120. [grembo](https://github.com/grembo) fixed the test suite and re-enabled several test cases.
121. [Hyeon Kim](https://github.com/simnalamburt) introduced the macro `JSON_INTERNAL_CATCH` to control the exception handling inside the library.
@@ -1528,7 +1581,7 @@ I deeply appreciate the help of the following people.
166. [Mark Beckwith](https://github.com/wythe) fixed a typo.
167. [yann-morin-1998](https://github.com/yann-morin-1998) helped to reduce the CMake requirement to version 3.1.
168. [Konstantin Podsvirov](https://github.com/podsvirov) maintains a package for the MSYS2 software distro.
169. [remyabel](https://github.com/remyabel) added GNUInstallDirs to the CMake files.
169. [remyabel](https://github.com/remyabel2) added GNUInstallDirs to the CMake files.
170. [Taylor Howard](https://github.com/taylorhoward92) fixed a unit test.
171. [Gabe Ron](https://github.com/Macr0Nerd) implemented the `to_string` method.
172. [Watal M. Iwasaki](https://github.com/heavywatal) fixed a Clang warning.
@@ -1555,7 +1608,7 @@ I deeply appreciate the help of the following people.
193. [Hubert Chathi](https://github.com/uhoreg) made CMake's version config file architecture-independent.
194. [OmnipotentEntity](https://github.com/OmnipotentEntity) implemented the binary values for CBOR, MessagePack, BSON, and UBJSON.
195. [ArtemSarmini](https://github.com/ArtemSarmini) fixed a compilation issue with GCC 10 and fixed a leak.
196. [Evgenii Sopov](https://github.com/sea-kg) integrated the library to the wsjcpp package manager.
196. [Evgenii Sopov](https://github.com/sea5kg) integrated the library to the wsjcpp package manager.
197. [Sergey Linev](https://github.com/linev) fixed a compiler warning.
198. [Miguel Magalhães](https://github.com/magamig) fixed the year in the copyright.
199. [Gareth Sylvester-Bradley](https://github.com/garethsb-sony) fixed a compilation issue with MSVC.
@@ -1649,7 +1702,7 @@ I deeply appreciate the help of the following people.
287. [NN](https://github.com/NN---) added the Visual Studio output directory to `.gitignore`.
288. [Romain Reignier](https://github.com/romainreignier) improved the performance of the vector output adapter.
289. [Mike](https://github.com/Mike-Leo-Smith) fixed the `std::iterator_traits`.
291. [vakokako](https://github.com/vakokako) fixed tests when compiling with C++20.
292. [Alexander “weej” Jones](https://github.com/alexweej) fixed an example in the README.
293. [Eli Schwartz](https://github.com/eli-schwartz) added more files to the `include.zip` archive.
@@ -1674,7 +1727,7 @@ I deeply appreciate the help of the following people.
312. [Gareth Sylvester-Bradley](https://github.com/garethsb) added `operator/=` and `operator/` to construct JSON pointers.
313. [Michael Macnair](https://github.com/mykter) added support for afl-fuzz testing.
314. [Berkus Decker](https://github.com/berkus) fixed a typo in the README.
315. [Illia Polishchuk](https://github.com/effolkronium) improved the CMake testing.
315. [Illia Polishchuk](https://github.com/ilqvya) improved the CMake testing.
316. [Ikko Ashimine](https://github.com/eltociear) fixed a typo.
317. [Raphael Grimm](https://github.com/barcode) added the possibility to define a custom base class.
318. [tocic](https://github.com/tocic) fixed typos in the documentation.
@@ -1744,6 +1797,66 @@ I deeply appreciate the help of the following people.
382. [bitFiedler](https://github.com/bitFiedler) made GDB pretty printer work with Python 3.8.
383. [Gianfranco Costamagna](https://github.com/LocutusOfBorg) fixed a compiler warning.
384. [risa2000](https://github.com/risa2000) made `std::filesystem::path` conversion to/from UTF-8 encoded string explicit.
385. [AM](https://github.com/maqnouch) fixed typos in the README.
386. [dmenendez-gruposantander](https://github.com/dmenendez-gruposantander) fixed typos in the comments of the examples.
387. [Mihai Stan](https://github.com/mstan-xx) fixed comparisons against the literal `0`.
388. [Matt Gumbel](https://github.com/intelmatt) fixed some `-Weffc++` warnings.
389. [vimpunk](https://github.com/vimpunk) moved a lambda out of an unevaluated context to support older compilers.
390. [Chris Harris](https://github.com/cjh1) fixed the compilation with GCC 4.8.
391. [Palmer Dabbelt](https://github.com/palmer-dabbelt) generated and installed a pkg-config file.
392. [Gus Pozuelo](https://github.com/ap-viavi) made `ordered_map` compatible with GCC 5.5, Clang 3.6, and Xcode 9.
393. [AK](https://github.com/Lioncky) fixed an MSVC build error caused by the `min`/`max` macros from `windows.h`.
394. [Sergiu Deitsch](https://github.com/sergiud) provided a fallback for missing `char8_t` support.
395. [Xiaochuan Ye](https://github.com/XueSongTap) fixed `from_msgpack` for `std::byte` input by specializing `std::char_traits`.
396. [Ville Vesilehto](https://github.com/thevilledev) fixed an overflow in the BJData size calculation and rejected overflowing negative integers in CBOR.
397. [NmPassTHFan](https://github.com/nmpassthf) replaced the deprecated `std::is_trivial` for C++26.
398. [Chris Ever](https://github.com/chirsz-ever) added the `ignore_trailing_commas` parser option.
399. [Kuan-Fu Wu](https://github.com/kfwu1999) fixed the example code for `json_pointer` initialization.
400. [David Kilzer](https://github.com/ddkilzer) added a missing header to the input adapters.
401. [Miko](https://github.com/mikomikotaishi) added proper C++20 module support, simplified the module API, and fixed missing exports.
402. [hitgirl](https://github.com/hitgil) fixed the CMake configuration when cross-compiling.
403. [Devon Thomas](https://github.com/ThomaDevOSU) mentioned the Artistic Style formatting in the contribution guidelines.
404. [Erik Hu](https://github.com/Erikhu1) made Coveralls upload errors non-fatal in the CI.
406. [DmitriBogdanov](https://github.com/DmitriBogdanov) fixed broken package manager links in the documentation.
407. [Bander](https://github.com/banderzhm) improved the MSVC compatibility of the C++ modules.
408. [Andy Choi](https://github.com/ccpong) removed an unnecessary `template` keyword before `get` in the README and the documentation.
409. [SamareshSingh](https://github.com/ssam18) fixed single-element brace initialization to copy/move instead of wrapping in an array, fixed the `WITH_DEFAULT` macros for `ordered_map`, and handled moved events in `serve_header.py`.
410. [Aditya](https://github.com/Lumowhisp) improved the documentation of the documentation generation.
411. [cheese1](https://github.com/cheese1) clarified the README.
412. [KhloodElhossiny](https://github.com/khloodelhossiny) enabled `std::string_view` keys in `operator[]`.
413. [Charles Cabergs](https://github.com/cacharle) fixed a `-Wtautological-constant-out-of-range-compare` warning.
414. [EALePain](https://github.com/EALePain) made the `std::tuple` conversion work with reference types such as `std::tie`.
415. [koala_oishi](https://github.com/chibi-dogs) fixed grammatical wording in the README.
416. [riccardoori11](https://github.com/riccardoori11) fixed a typo in the documentation.
417. [Swastik Bose](https://github.com/VasuBhakt) fixed the parent pointers after `update()` with `JSON_DIAGNOSTICS` and fixed the Doxygen autolinking of requirements.
418. [trdesilva](https://github.com/trdesilva) added `front`, `pop_front`, and `push_front` to `json_pointer`.
419. [Akhilesh Arora](https://github.com/akhilesharora) fixed an incomplete-type error with `ordered_json`.
420. [Hariom Phulre](https://github.com/hariomphulre) fixed the C++20 modules compilation with GCC.
422. [George Sedov](https://github.com/radistmorse) added the `NLOHMANN_DEFINE_TYPE_*_WITH_NAMES` macros.
423. [Caillin Nugent](https://github.com/nugentcaillin) added the `NLOHMANN_JSON_SERIALIZE_ENUM_STRICT` macro.
424. [Cosmin D.](https://github.com/drcosmin) fixed `std::filesystem::path` conversions and added an MSVC workaround for `std::unique_ptr`.
425. [Paul Dreik](https://github.com/pauldreik) fixed a test relying on implementation-specific behavior.
426. [Daniel Falk](https://github.com/daniel-falk) added missing copyright notices to the SBOM.
427. [Federico Sfriso](https://github.com/federicosfriso05-dotcom) added support for constructing JSON values from C++20 range views.
428. [Luke Banicevic](https://github.com/banaboi) fixed corrupt BSON output for lengths exceeding `INT32_MAX`, cleaned up the BSON writer, and improved the documentation.
429. [Patrick Armstrong](https://github.com/Patrick10199) updated the CBOR references and the half-precision float assertions.
430. [Yash Bavadiya](https://github.com/xevrion) added checks to all BSON reads.
431. [hum4nBeing](https://github.com/hum4nBeing) fixed the overflow handling of high-precision numbers in UBJSON.
432. [tomatotomata](https://github.com/tomatotomata) added checks for reading CBOR tagged subtypes.
433. [YingqiDuan](https://github.com/YingqiDuan) documented the BSON interoperability.
434. [KBS](https://github.com/youdie006) documented the standards compliance and the strictness of `parse()` and `operator>>`.
435. [Petr Bělohlávek](https://github.com/petrbel) added Clang 21 and 22 to the CI.
436. [Dmitry Rantovov](https://github.com/darkdi) fixed the placement of a CBOR documentation block.
437. [ljcjclljc](https://github.com/ljcjclljc) fixed the comparison of large unsigned integers with signed integers.
438. [Sahil Kamate](https://github.com/sahilkamate03) fixed the handling of CBOR tags 0-5 and 21-23.
439. [Krishnanand G](https://github.com/Krishnanand-G) made the UBJSON writer reject `use_type` without `use_size`.
440. [whn](https://github.com/Whning0513) documented the lenient BSON input handling and corrected the complexity of `to_bson`.
441. [elix3r](https://github.com/22elix3r) fixed `update()` with `merge_objects` when merging a primitive into an object.
442. [Avionic Harshit](https://github.com/avionicharshit-byte) made `diff()` linear when an array shrinks.
443. [Qatadaha Bin Matloob](https://github.com/qatcod) fixed comparisons between integers and floats and fixed unparsable BJData output.
444. [Wu Shuwen](https://github.com/dajiaohuang) removed an unused include.
Thanks a lot for helping out! Please [let me know](mailto:mail@nlohmann.me) if I forgot someone.
@@ -1754,13 +1867,13 @@ The library itself consists of a single header file licensed under the MIT licen
- [**amalgamate.py - Amalgamate C source and header files**](https://github.com/edlund/amalgamate) to create a single header file
- [**American fuzzy lop**](https://lcamtuf.coredump.cx/afl/) for fuzz testing
- [**AppVeyor**](https://www.appveyor.com) for [continuous integration](https://ci.appveyor.com/project/nlohmann/json) on Windows
- [**Artistic Style**](http://astyle.sourceforge.net) for automatic source code indentation
- [**Artistic Style**](https://astyle.sourceforge.net) for automatic source code indentation
- [**Clang**](https://clang.llvm.org) for compilation with code sanitizers
- [**CMake**](https://cmake.org) for build automation
- [**Codacy**](https://www.codacy.com) for further [code analysis](https://app.codacy.com/gh/nlohmann/json/dashboard)
- [**Coveralls**](https://coveralls.io) to measure [code coverage](https://coveralls.io/github/nlohmann/json)
- [**Coverity Scan**](https://scan.coverity.com) for [static analysis](https://scan.coverity.com/projects/nlohmann-json)
- [**cppcheck**](http://cppcheck.sourceforge.net) for static analysis
- [**cppcheck**](https://cppcheck.sourceforge.io) for static analysis
- [**doctest**](https://github.com/onqtam/doctest) for the unit tests
- [**GitHub Changelog Generator**](https://github.com/skywinder/github-changelog-generator) to generate the [ChangeLog](https://github.com/nlohmann/json/blob/develop/ChangeLog.md)
- [**Google Benchmark**](https://github.com/google/benchmark) to implement the benchmarks
@@ -1775,6 +1888,15 @@ The library itself consists of a single header file licensed under the MIT licen
## Notes
### Standards compliance
The library targets strict conformance with [RFC 8259](https://tools.ietf.org/html/rfc8259.html). Both the original [JSONTestSuite](https://github.com/nst/JSONTestSuite) and its updated revision are exercised in CI; their test data is downloaded from [`nlohmann/json_test_data`](https://github.com/nlohmann/json_test_data) at configure time rather than committed to this repository (see [`tests/src/unit-testsuites.cpp`](https://github.com/nlohmann/json/blob/develop/tests/src/unit-testsuites.cpp)):
- The updated revision runs all mandatory `y_` (must-accept) and `n_` (must-reject) cases through the strict [`parse()`](https://json.nlohmann.me/api/basic_json/parse/) entry point; the original suite runs its `n_` cases through `parse()` and its `y_` cases through [`operator>>`](https://json.nlohmann.me/api/operator_gtgt/).
- The `i_` (implementation-defined) cases are, by RFC 8259, free to be accepted *or* rejected, so "passing all `i_` cases" is not a meaningful conformance metric. The library makes deliberate, documented choices there: nesting depth is not artificially limited, a leading UTF-8 byte order mark is silently ignored, [Unicode noncharacters](https://www.unicode.org/faq/private_use.html#nonchar1) are forwarded unchanged, invalid UTF-8 and lone/unpaired UTF-16 surrogates are rejected (stricter than required), and a number that cannot be stored without becoming `NaN`/`INF` raises [`out_of_range.406`](https://json.nlohmann.me/home/exceptions/#jsonexceptionout_of_range406).
One behavioral nuance is worth calling out, because a superficial test often misreads it as non-compliance: [`parse()`](https://json.nlohmann.me/api/basic_json/parse/) is strict and rejects trailing data after a value, whereas [`operator>>`](https://json.nlohmann.me/api/operator_gtgt/) follows relaxed iostream semantics — it parses a single value and leaves the stream positioned right after it. Feeding "a valid document followed by trailing bytes" through `operator>>` reports success; the same input through `parse()` is rejected. This is a documented two-API design, not a conformance gap. See [**parsing**](https://json.nlohmann.me/features/parsing/) for details.
### Character encoding
The library supports **Unicode input** as follows:
@@ -1793,7 +1915,7 @@ The library supports **Unicode input** as follows:
This library does not support comments by default. It does so for three reasons:
1. Comments are not part of the [JSON specification](https://tools.ietf.org/html/rfc8259). You may argue that `//` or `/* */` are allowed in JavaScript, but JSON is not JavaScript.
2. This was not an oversight: Douglas Crockford [wrote on this](https://plus.google.com/118095276221607585885/posts/RK8qyGVaGSr) in May 2012:
2. This was not an oversight: Douglas Crockford [wrote on this](https://news.ycombinator.com/item?id=3912149) in May 2012:
> I removed comments from JSON because I saw people were using them to hold parsing directives, a practice which would have destroyed interoperability. I know that the lack of comments makes some people sad, but it shouldn't.
>
@@ -1801,7 +1923,7 @@ This library does not support comments by default. It does so for three reasons:
3. It is dangerous for interoperability if some libraries would add comment support while others don't. Please check [The Harmful Consequences of the Robustness Principle](https://tools.ietf.org/html/draft-iab-protocol-maintenance-01) on this.
However, you can set set parameter `ignore_comments` to true in the `parse` function to ignore `//` or `/* */` comments. Comments will then be treated as whitespace.
However, you can set parameter `ignore_comments` to true in the `parse` function to ignore `//` or `/* */` comments. Comments will then be treated as whitespace.
### Trailing commas
@@ -1851,7 +1973,7 @@ ctest --output-on-failure
Note that during the `ctest` stage, several JSON test files are downloaded from an [external repository](https://github.com/nlohmann/json_test_data). If policies forbid downloading artifacts during testing, you can download the files yourself and pass the directory with the test files via `-DJSON_TestDataDirectory=path` to CMake. Then, no Internet connectivity is required. See [issue #2189](https://github.com/nlohmann/json/issues/2189) for more information.
If the test suite is not found, several test suites will fail like this:
If the testdata is not found, several test suites will fail like this:
In case you have downloaded the library rather than checked out the code via Git, test `cmake_fetch_content_configure` will fail. Please execute `ctest -LE git_required` to skip these tests. See [issue #2189](https://github.com/nlohmann/json/issues/2189) for more information.
Some tests are requiring network to be properly execute. They are labeled as `git_required`. Please execute `ctest -LE git_required` to skip these tests. See [issue #4851](https://github.com/nlohmann/json/issues/4851) for more information.
Some tests change the installed files and hence make the whole process not reproducible. Please execute `ctest -LE not_reproducible` to skip these tests. See [issue #2324](https://github.com/nlohmann/json/issues/2324) for more information. Furthermore, assertions must be switched off to ensure reproducible builds (see [discussion 4494](https://github.com/nlohmann/json/discussions/4494)).
Note you need to call `cmake -LE "not_reproducible|git_required"` to exclude both labels. See [issue #2596](https://github.com/nlohmann/json/issues/2596) for more information.
| `compiler` | Information on the used compiler. It is an object with the following keys: `c++` (the used C++ standard), `family` (the compiler family; possible values are `clang`, `icc`, `gcc`, `ilecpp`, `msvc`, `pgcpp`, `sunpro`, and `unknown`), and `version` (the compiler version). |
| `compiler` | Information on the used compiler. It is an object with the following keys: `c++` (the used C++ standard), `family` (the compiler family; possible values are `clang`, `icc`, `gcc`, `ilecpp`, `msvc`, `pgcpp`, `sunpro`, and `unknown`), and `version` (the compiler version). On HP aCC compilers, `compiler` is instead the plain string `hp`. |
| `copyright` | The copyright line for the library as string. |
| `name` | The name of the library as string. |
| `platform` | The used platform as string. Possible values are `win32`, `linux`, `apple`, `unix`, and `unknown`. |
@@ -32,7 +32,13 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
could not be resolved successfully in the current JSON value; example: `"key baz not found"`.
- Throws [`out_of_range.405`](../../home/exceptions.md#jsonexceptionout_of_range405) if JSON pointer has no parent
("add", "remove", "move")
- Throws [`out_of_range.501`](../../home/exceptions.md#jsonexceptionother_error501) if "test" operation was
- Throws [`out_of_range.411`](../../home/exceptions.md#jsonexceptionout_of_range411) if an "add" operation's target
location has a parent that is neither an object nor an array.
- Throws [`out_of_range.413`](../../home/exceptions.md#jsonexceptionout_of_range413) if a "remove" operation's target
location has a parent that is neither an object nor an array.
- Throws [`out_of_range.414`](../../home/exceptions.md#jsonexceptionout_of_range414) if a "move" operation's "from"
location is a proper prefix of its "path" location.
- Throws [`other_error.501`](../../home/exceptions.md#jsonexceptionother_error501) if "test" operation was
unsuccessful.
## Complexity
@@ -71,3 +77,9 @@ is thrown. In any case, the original value is not changed: the patch is applied
## Version history
- Added in version 2.0.0.
- Added [`out_of_range.411`](../../home/exceptions.md#jsonexceptionout_of_range411) and stopped relying on an internal assertion when an "add" operation's
target location has a non-object/non-array parent in version 3.13.0.
- Added [`out_of_range.413`](../../home/exceptions.md#jsonexceptionout_of_range413) and stopped silently ignoring a "remove" operation whose target
location has a non-object/non-array parent in version 3.13.0.
- Added [`out_of_range.414`](../../home/exceptions.md#jsonexceptionout_of_range414) and rejected a "move" operation whose "from" location is a proper
prefix of its "path" location instead of silently producing a corrupted result in version 3.13.0.
- [merge_patch](merge_patch.md) applies a JSON Merge Patch
## Version history
- Added in version 3.11.0.
- Added [`out_of_range.411`](../../home/exceptions.md#jsonexceptionout_of_range411) and stopped relying on an internal assertion when an "add" operation's
target location has a non-object/non-array parent in version 3.13.0.
- Added [`out_of_range.413`](../../home/exceptions.md#jsonexceptionout_of_range413) and stopped silently ignoring a "remove" operation whose target
location has a non-object/non-array parent in version 3.13.0.
- Added [`out_of_range.414`](../../home/exceptions.md#jsonexceptionout_of_range414) and rejected a "move" operation whose "from" location is a proper
prefix of its "path" location instead of silently producing a corrupted result in version 3.13.0.
@@ -46,9 +46,12 @@ invalidates all iterators and all references.
## Exceptions
All functions can throw the following exception:
- Throws [`type_error.308`](../../home/exceptions.md#jsonexceptiontype_error308) when called on a type other than
JSON array or null; example: `"cannot use push_back() with number"`
1. Throws [`type_error.308`](../../home/exceptions.md#jsonexceptiontype_error308) when called on a type other than
JSON array or null; example: `"cannot use push_back() with number"`
2. Throws [`type_error.308`](../../home/exceptions.md#jsonexceptiontype_error308) when called on a type other than
JSON object or null; example: `"cannot use push_back() with number"`
3. Throws [`type_error.308`](../../home/exceptions.md#jsonexceptiontype_error308) when called on a type other than
JSON array or null; example: `"cannot use push_back() with number"`
## Complexity
@@ -112,6 +115,7 @@ All functions can throw the following exception:
- [emplace_back](emplace_back.md) add a value to an array
- [operator+=](operator+=.md) add a value to an array/object
- [Modifying values](../../features/modifying_values.md) - the article on modifying values
## Version history
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Jeremy Nimmer
Niels Lohmann
dependabot[bot]
Wu Shuwen
Qatadaha Bin Matloob
Avionic Harshit
elix3r
whn
Krishnanand G
Sahil_Kamate
ljcjclljc
Dmitry
Angadi56
Petr Bělohlávek
KBSandmanon
tomatotomata
hum4nBeingandAJ369ninja
YingqiDuan
Yash Bavadiya
Patrick10199
Luke Banicevic
Federico Sfriso
Daniel Falk
Paul Dreik
Cosmin D.
risa2000andRichard Musil
Swastik Bose
Miko
Caillin NugentandNiels Lohmann
Kirill Lokotkov
George SedovandNiels Lohmann
Hariom Phulre
SamareshSingh
Akhilesh Arora
trdesilva
Klaim (Joël Lamotte)
riccardoori11
koala_oishiandBasabdatta Gupta
EALePain
Charles Cabergs
Ville Vesilehto
KhloodElhossinyandkhloodelhossiny
cheese1
Aditya
Gianfranco Costamagna
Andy Choi
Bander
DmitriBogdanov
co63oc
Erik Hu
Nikhil
hitgirlandHitgirl
David KilzerandDavid Kilzer