Copying carried a depth count, a depth limit and a guard of its own, and
the comparison in the follow-up added a second set beside them. Neither
operation needs its own: they are never nested inside one another by the
library - copying a value does not compare one, and comparing two values
does not copy them - and where user code nests them anyway, sharing the
count only ends a descent sooner than it had to.
So there is now one nesting_depth(), one nesting_depth_limit() and one
nesting_depth_guard, which the follow-up uses instead of adding its own.
Inverting the test in copy_structured leaves the too-deep case and the
no-thread-local case as the same code.
The guard takes the count rather than looking it up, because the caller
has looked it up already to test it against the limit, and reaching
thread-local storage twice on the path that is taken almost every time is
worth avoiding.
The switch that copies the value of anything that is not an object or an
array was written twice - once in the copy constructor, once in
copy_shallow - so that adding a value_t meant editing both, and missing
one would have been silent. It is copy_leaf_value now, and inlined: both
callers have already sorted the containers out, and folding that test into
the switch is what keeps a value made mostly of numbers copying as fast as
it did.
Copying canada.json, citm_catalog.json and twitter.json is within 0.6% of
what it was before, measured as a paired ratio over 18 interleaved rounds
against a run-to-run spread of 0.3%.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
basic_json's copy constructor copied objects and arrays by handing the
container to its own copy constructor, which copy-constructs every element
and so reaches this constructor again, once per nesting level. A value
nested deeply enough exhausted the call stack and terminated the process
with a segmentation fault - no exception, nothing the caller could catch.
Parsing such a value works, as the parser is iterative, and so does
destroying one, as #1436 made destruction iterative.
Bound how far the copy descends rather than take the call stack away from
it. The first levels are copied exactly as they were - the containers copy
their own elements, which is by far the fastest way to fill them - and only
once the copy has descended 128 levels is the value below it finished
without the call stack, through an explicit worklist. Copying can therefore
no longer exhaust the stack, however deeply a value is nested, while a value
nested less deeply than the bound - all but a vanishing minority - is copied
by the very same code as before and pays only for one counter.
That counter lives in thread_local storage, as one shared between threads
would be raced. JSON_NO_THREAD_LOCAL switches it off for toolchains without
thread_local; copying then goes through the worklist right away, which
yields the same values but is measurably slower.
The deferred values are completed before the copy they belong to returns, so
a value copied while another copy is going on - by a custom base class, say -
is unaffected by the copy it is nested in.
operator= takes its argument by value, so copy assignment is fixed as well.
Copying is as fast as it was, within measurement noise (medians of 9
interleaved runs, clang -O3): -1.3% for an array of strings, +0.0% for a
flat object, +0.1% for a flat array of numbers, +0.3% for nested arrays,
+0.6% for nested objects and +1.2% for a twitter-like document. Copying a
three-key object costs about ten nanoseconds more, the counter. Deferring
every level instead, rather than only those below the bound, measured
between 3% and 9% slower depending on the shape of the value.
This fixes#5387 for the copy constructor. dump() is still recursive.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Add iterator+sentinel tests and docs for binary deserializers
This commit extends the C++20 ranges support (iterator+sentinel pairs) to the
binary format deserializers from_cbor, from_msgpack, from_ubjson, from_bjdata,
and from_bson, matching what was already done for parse(), accept(), and
sax_parse().
Changes:
- Add istreambuf_sentinel helper to test_utils.hpp for EOF detection in tests
- Add 5 new test cases that read binary files directly via
std::istreambuf_iterator<char> + sentinel, without pre-buffering
- Update documentation for all 5 from_* functions to document overload (3)
with SentinelType parameter
- All tests pass; verified against existing test suite data
- Fix potential buffer over-read warning in heterogeneous iterator test
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Merge iterator+sentinel overloads and fix ambiguity/CI issues
Address PR review feedback and CI failures:
- Merge the separate same-type and sentinel-type iterator overloads of
parse(), accept(), sax_parse(), and the five from_* binary deserializers
into a single overload with SentinelType defaulted to IteratorType,
as suggested in review. Applied the same simplification to the
detail::input_adapter() free functions.
- Fix a latent ambiguity: some compilers (e.g. GCC 4.8) unreliably SFINAE
the operator!= detection for std::nullptr_t against container/string
types, making calls like parse(s, nullptr, ...) ambiguous with the
compatible-input overload. can_compare_ne now explicitly excludes
std::nullptr_t as a SentinelType.
- Use a named enable_if_t template parameter instead of an unnamed
function parameter for the SFINAE guard, fixing a clang-tidy
hicpp-named-parameter/readability-named-parameter failure.
- Update parse.md, accept.md, sax_parse.md, and the five from_*.md pages
to document the merged overload instead of separate (2)/(3) overloads,
also fixing an over-160-char line that broke the documentation
style_check CI job.
- Rework the BSON iterator+sentinel test to parse a BSON file already
present in the test suite instead of writing/deleting a temp file.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix -Wunneeded-internal-declaration for CustomSentinel in test
CustomSentinel lives in an anonymous namespace (internal linkage), and
the library's parse loop only ever evaluates the iterator-first
direction (it != last), so the reversed-order friend operator!= was
never referenced. Clang's -Weverything flags such unused internal
declarations as an error. Drop the unused overload; the used direction
is enough to satisfy can_compare_ne's either-order detection.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix clang-tidy hicpp-named-parameter and misc-const-correctness
- Drop the unused reversed-order operator!= overload from
utils::istreambuf_sentinel (only iterator != sentinel is ever
evaluated) and name the remaining friend's sentinel parameter, fixing
hicpp-named-parameter/readability-named-parameter.
- Mark the istreambuf_iterator first/last helper variable const in the
five binary-format sentinel tests, fixing misc-const-correctness.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix clang-tidy misc-const-correctness in heterogeneous sentinel test
json_str is only read via .data()/.size() and never reassigned, so
clang-tidy correctly flags it as const-able. Verified against the exact
CI job (silkeh/clang:dev, ci_clang_tidy target) by running clang-tidy
directly on this file plus the five binary-format sentinel tests
touched by prior commits; all are now clean.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix#3868: Remove operator!= to enable P2468R2 rewritten candidate synthesis
Under C++20 P2468R2, a hand-written operator!= suppresses the compiler's
rewritten-candidate synthesis for operator==, preventing heterogeneous
comparisons like `std::string s; json j; s == j;` from compiling.
Fix by removing the hand-written operator!=, allowing the compiler to
synthesize != as !(a==b) in all language modes (C++20 member functions
and pre-C++20 friend functions).
Behavior change: operator!= now returns !(a==b) unconditionally, including
for special values like NaN and discarded. This means:
- NaN != NaN now returns true (matches IEEE-754 semantics)
- discarded != x now returns true for any x (matches !(discarded == x))
This also fixes underlying defects in previously-working code:
- Restores direct == comparison for views vs json (reverts std::ranges::equal
workaround added in PR #3950 to dodge this bug)
- Re-enables std::string == json comparisons (uncomments check in
unit-constructor1.cpp)
Fixes: #3868, #3979
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* 🚨 fix warning
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
When iteration_proxy_value<iter_impl<ordered_json>> appears in a context
that requires it to be complete (function or lambda parameter), the
compiler instantiates basic_json<ordered_map> and walks into
set_parents(iterator, typename iterator::difference_type)
while iterator is still incomplete, failing with "invalid use of
incomplete type".
basic_json::difference_type is already std::ptrdiff_t, so just naming
the underlying type directly avoids the dependent lookup. Behavior and
ABI are unchanged. This was the approach suggested in the issue thread.
Added a regression case in unit-ordered_json.cpp using the same trigger
pattern (lambda parameter naming the proxy type).
Fixes#3732
Signed-off-by: Akhilesh Arora <akhildawra@gmail.com>
* fix: treat single-element brace-init as copy/move
When passing a json value using brace initialization with a single element
(e.g., `json j{someObj}` or `foo({someJson})`), C++ always prefers the
initializer_list constructor over the copy/move constructor. This caused
the value to be unexpectedly wrapped in a single-element array.
This bug was previously compiler-dependent (GCC wrapped, Clang did not),
but Clang 20 started matching GCC behavior, making it a universal issue.
Fix: In the initializer_list constructor, when type deduction is enabled
and the list has exactly one element, copy/move it directly instead of
creating a single-element array.
Before:
json obj = {{"key", 1}};
json j{obj}; // -> [{"key":1}] (wrong: array)
foo({obj}); // -> [{"key":1}] (wrong: array)
After:
json j{obj}; // -> {"key":1} (correct: copy)
foo({obj}); // -> {"key":1} (correct: copy)
To explicitly create a single-element array, use json::array({value}).
Fixes the issue #5074
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
* fix: regenerate amalgamated single_include/nlohmann/json.hpp
- Add missing comment from include/nlohmann/json.hpp explaining the
single-element brace-init fix (issue #5074)
- Fix extra 4-space indentation in embedded json_fwd.hpp section
Regenerated by running: make amalgamate
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
* Revert brace-init semantics change and fix amalgamation
The single-element brace-init change was a breaking change that cannot be accepted upstream. Reverted all related source, test, and doc changes, then regenerated single_include with correct indentation to pass the amalgamation CI check.
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
* Fix: add JSON_BRACE_INIT_COPY_SEMANTICS opt-in macro for issue #5074
Single-element brace initialization wrapping in an array cannot be fixed without breaking existing code. Added JSON_BRACE_INIT_COPY_SEMANTICS as an opt-in macro (default 0) so users can enable copy/move semantics for single-element brace init without affecting anyone relying on the current behavior.
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
* docs: add dedicated macro page and CI test target for JSON_BRACE_INIT_COPY_SEMANTICS
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
* fix: remove compiler-dependent assertions from #5074 regression test
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
* fix: use defined() guard for JSON_BRACE_INIT_COPY_SEMANTICS to satisfy -Wundef
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
* docs: fix section name in json_brace_init_copy_semantics.md to pass style check
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
* docs: move Default definition section before Notes to fix style check order
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
---------
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
PR #4873 introduced a safety check in sax_parse functions to catch
nullptr passed as SAX parser object, which had been already annotated by
JSON_HEDLEY_NON_NULL macro.
Compilers (e.g. clang) which respected the non-null annotation tended to
eliminate the safety check completely in optimized builds, while
compilers which did not, compiled the safety check in. This led to
different behaviors accross different compilers/platforms and/or build
types (debug, release).
This commit reverts PR #4873 to remove this discrepancy. Passing null to
non-null annotated parameter is considered to be undefined behavior.
Fixes#5048
Signed-off-by: Richard Musil <risa2000x@gmail.com>
Co-authored-by: Richard Musil <risa2000x@gmail.com>
* Add implementation to retrieve start and end positions of json during parse
* Add more unit tests and add start/stop parsing for arrays
* Add raw value for all types
* Add more tests and fix compiler warning
* Amalgamate
* Fix CLang GCC warnings
* Fix error in build
* Style using astyle 3.1
* Fix whitespace changes
* revert
* more whitespace reverts
* Address PR comments
* Fix failing issues
* More whitespace reverts
* Address remaining PR comments
* Address comments
* Switch to using custom base class instead of default basic_json
* Adding a basic using for a json using the new base class. Also address PR comments and fix CI failures
* Address decltype comments
* Diagnostic positions macro (#4)
Co-authored-by: Sush Shringarputale <sushring@linux.microsoft.com>
* Fix missed include deletion
* Add docs and address other PR comments (#5)
* Add docs and address other PR comments
---------
Co-authored-by: Sush Shringarputale <sushring@linux.microsoft.com>
* Address new PR comments and fix CI tests for documentation
* Update documentation based on feedback (#6)
---------
Co-authored-by: Sush Shringarputale <sushring@linux.microsoft.com>
* Address std::size_t and other comments
* Fix new CI issues
* Fix lcov
* Improve lcov case with update to handle_diagnostic_positions call for discarded values
* Fix indentation of LCOV_EXCL_STOP comments
* fix amalgamation astyle issue
---------
Co-authored-by: Sush Shringarputale <sushring@linux.microsoft.com>
Niels Lohmann
ljcjclljcandGitHub
Cosmin D.andGitHub
Swastik BoseandGitHub
Hariom PhulreandGitHub
Akhilesh AroraandGitHub
SamareshSinghandGitHub
NikhilandGitHub
Nebojša CvetkovićandGitHub
Michael ValladolidandGitHub
Harinath NampallyandGitHub
alferovandGitHub
tomalakgeretkalandGitHub
