Compare commits

...
Author SHA1 Message Date
Niels Lohmann b87a618151 Pin Google Benchmark to release 1.9.5
The benchmarks fetched Google Benchmark's main branch, so two builds on
different days could measure with different library code, and CMake 3.30
and later warn that the single-argument FetchContent_Populate() is
deprecated. Fetch the 1.9.5 release archive, verified by its SHA-256,
with FetchContent_MakeAvailable() instead. That needs CMake 3.14; Google
Benchmark itself already needed 3.13.

Its -Werror is switched off, so a newer compiler's new warnings cannot
break the pinned release, and its install rules are no longer added.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-24 09:48:18 +02:00
Niels Lohmann 67ae3fd85b Point ci_benchmarks at tests/benchmarks
The target has configured ${PROJECT_SOURCE_DIR}/benchmarks since it was
added in #2561, but the benchmarks live in tests/benchmarks.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-24 07:10:37 +02:00
Niels Lohmann 71fe9f0b96 Document the benchmarks, and make them build and compare versions again
The benchmark project hasn't configured since #4793: download_test_data.cmake
compiles cmake/detect_libcpp_version.cpp relative to CMAKE_SOURCE_DIR, which
is tests/benchmarks when that is the top-level project, so try_run fails and
so does `make run_benchmarks`. The path is now relative to the module itself,
which is the same file for the main build.

The Dump benchmark discarded dump()'s result, which is [[nodiscard]] by now;
it warned, and left the optimizer free to shorten the loop. The result is
now kept with benchmark::DoNotOptimize.

A new cache variable, JSON_BENCHMARK_INCLUDE_DIR, names the directory holding
the nlohmann/json.hpp to benchmark (single_include by default, as before),
so the same benchmarks can be built against two versions and compared.

tests/benchmarks/README.md documents what is measured, how to build and run
the benchmarks, how to read the output, and how to compare two versions with
Google Benchmark's compare.py; it recommends doing so by hand before a
release rather than in CI.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-24 07:07:59 +02:00
Niels LohmannandClaude Sonnet 5 ed513715a8 Document that a NUL byte in the input is treated as end of input (#5534)
* docs: document that a NUL byte in the input is treated as end of input

A NUL byte anywhere in the input - trailing, or embedded ahead of more
otherwise well-formed JSON - is currently treated the same as genuine
end of input, so parsing silently stops there instead of raising the
parse_error.101 any other unexpected byte triggers. This mirrors the
NUL-terminated-C-string convention already used when no explicit input
length is given (json::parse(const char*) already stops at strlen()),
just applied uniformly rather than only when a length is genuinely
unavailable.

This behavior predates this change and is not being altered here -
changing it would be an observable, backwards-incompatible behavior
change for any caller that (knowingly or not) depends on it, which is
not something to do silently in a patch. Documenting the current,
verified behavior as a new FAQ entry instead, so it's an intentional
and discoverable part of the contract rather than a surprise.

Fixes #5530.

Signed-off-by: Niels Lohmann <niels.lohmann@gmail.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N4RQ1Ahan5YAGbnAQGjZTY

* Add JSON_STRICT_NUL_HANDLING opt-in macro for issue #5530

A NUL byte anywhere in the input is currently treated the same as real
end of input, rather than raising parse_error.101 like any other
unexpected byte (documented in the previous commit's FAQ entry). A full
unconditional fix was tried in PR #5532 but rejected as too risky to
ship by default: any caller could depend on the current behavior, even
unknowingly (e.g. a zero-padded buffer). On PR #5534, gregmarr proposed
a compile-time opt-in flag instead, and the maintainer agreed, wanting
it available now and defaulting to the corrected behavior in 4.0.0.

This mirrors the existing JSON_BRACE_INIT_COPY_SEMANTICS precedent as
closely as sensible:
- JSON_STRICT_NUL_HANDLING defaults to 0 (off); the three lexer sites
  that treat '\0' as EOF/comment-terminator are gated with
  `#if !JSON_STRICT_NUL_HANDLING` so the default-off behavior is
  byte-for-byte identical to today's.
- input_adapters.hpp's `T (&array)[N]` overload additionally trims a
  single trailing '\0' from a `char` array (e.g. a string literal like
  `json::parse("123")`) when the macro is on, so that case keeps
  working; every other element type (unsigned char, std::uint8_t, ...)
  always keeps its full extent. This intentionally does *not* reuse the
  existing strlen()-based pointer overload via SFINAE-excluding `char`
  from the array overload, as originally sketched for this change: that
  approach is ambiguous against the newer generic container overload
  added since PR #5532, and even where it compiles, strlen()-scanning a
  `char` array that is not NUL-terminated within its bounds reads past
  the end of the array (confirmed with AddressSanitizer). Trimming only
  a single trailing byte, without scanning, avoids both problems.
- Documented via docs/mkdocs/docs/api/macros/json_strict_nul_handling.md,
  linked from the macros index/nav/features page, the FAQ entry, and
  the parse/accept/operator>> reference pages.
- Tested in unit-class_parser.cpp and unit-deserialization.cpp, default
  state unguarded and opt-in state guarded. Since the library itself
  #undefs the macro at the end of json.hpp (as JSON_BRACE_INIT_COPY_SEMANTICS
  already does), a plain `#if defined(JSON_STRICT_NUL_HANDLING)` guard
  after the include never actually triggers; the tests instead capture
  the command-line value into a test-local macro before including the
  header. A few pre-existing fixtures elsewhere (std::array<uint8_t, N>
  sized one larger than their literal, relying on value-initialization
  to silently add a trailing zero byte) needed the same one-byte
  adjustment to keep passing under the opt-in behavior.

Unlike the precedent, this adds a proper `JSON_StrictNulHandling` CMake
option (rather than a raw -DCMAKE_CXX_FLAGS injection) and wires its
ci_test_strict_nul_handling target into the ci_cmake_options job matrix
in .github/workflows/ubuntu.yml, so the opt-in build is actually
exercised in CI -- closing the one gap in the precedent's own CI setup
(ci_test_brace_init_copy_semantics is defined but never referenced by
any workflow, so it has never actually run).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Clarify where JSON_STRICT_NUL_HANDLING does not reject NUL bytes

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <niels.lohmann@gmail.com>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-24 06:55:21 +02:00
Niels Lohmann f751547a81 Add missing contributors to the README thanks list (#5543)
Add 60 contributors whose work was not yet credited and update seven
links that pointed to renamed or reassigned GitHub accounts.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-23 20:43:40 +02:00
dependabot[bot] 06b0452189 Bump mkdocs-git-revision-date-localized-plugin in /docs/mkdocs (#5537)
Bumps [mkdocs-git-revision-date-localized-plugin](https://github.com/timvink/mkdocs-git-revision-date-localized-plugin) from 1.5.4 to 1.6.0.
- [Release notes](https://github.com/timvink/mkdocs-git-revision-date-localized-plugin/releases)
- [Commits](https://github.com/timvink/mkdocs-git-revision-date-localized-plugin/compare/v1.5.4...v1.6.0)

---
updated-dependencies:
- dependency-name: mkdocs-git-revision-date-localized-plugin
  dependency-version: 1.6.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-23 20:32:31 +02:00
Niels LohmannandClaude 1054b2097e Speed up binary writing: value-type output sink + byte-swap number encoding (#5286)
* Devirtualize binary_writer via a value-type output sink

to_cbor/to_msgpack/to_ubjson/to_bjdata/to_bson wrote every byte through
output_adapter_t, a shared_ptr<output_adapter_protocol> whose
write_character/write_characters are virtual. Unlike the lexer (templated
on a concrete InputAdapterType), the binary writer never got that
treatment, so binary output paid a vtable lookup per byte and a
make_shared per call.

Template binary_writer on an OutputSinkType and give it two concrete,
non-virtual sinks:

- output_vector_sink: appends straight into a std::vector (push_back /
  insert), used by the vector-returning to_* convenience functions. No
  vtable, no shared_ptr; the writes inline.
- output_adapter_sink: forwards to a type-erased output_adapter_t, so the
  existing to_*(j, output_adapter) overloads (streams, strings, custom
  adapters) keep working exactly as before -- one virtual call each,
  unchanged.

binary_writer keeps a convenience constructor taking output_adapter_t
(building the default output_adapter_sink), so the adapter overloads are
untouched; only the convenience functions switch to the vector sink. The
friend declaration and the basic_json binary_writer alias gain the new
(defaulted) template parameter.

Output is byte-for-byte identical: verified across ~3000 randomized
values plus curated edge cases (all scalar widths, strings with invalid
UTF-8, binary, nested arrays/objects) for CBOR, MessagePack, UBJSON (both
size/type settings), BJData, and BSON, plus the output_adapter path, in
C++11/17/20. Warning-clean under clang -Weverything and the gcc pedantic
set; clang-tidy clean on the changed headers; make check-amalgamation
clean.

Throughput (g++ -O3, vs develop): scalar-dense binary output such as
integer arrays ~1.4x; many small to_cbor calls ~1.04x (DOM traversal
bound); string/blob-heavy output unchanged (already bulk-bound). No
workload regressed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix CI failures from binary_writer output-sink change

Four CI jobs failed on the initial commit; all are addressed here without
changing any output (binary encodings remain byte-for-byte identical to
develop across the differential corpus):

1. ci_test_gcc / cuda (-Werror=duplicated-branches): for number_float_t ==
   float, static_cast<float>(n) is the identity, so write_compact_float's
   two branches are intentionally identical. Once the concrete vector sink
   is inlined, GCC constant-folds and diagnoses this (the type-erased path
   hid it behind a non-inlined virtual call). Silence -Wduplicated-branches
   for GCC (clang has no such warning) alongside the existing -Wfloat-equal
   pragma.

2. ci_static_analysis_clang (UBSan nonnull-attribute): binary_writer passes
   a null pointer with length 0 for empty strings/binary. output_vector_sink
   / output_adapter_sink declared write_characters JSON_HEDLEY_NON_NULL, so
   the sanitizer flagged the (harmless) zero-length call once the sink was
   called directly rather than through the attribute-free virtual base. Drop
   the attribute from both sinks, matching the pre-existing behavior.

3. ci_cpplint (build/include_what_you_use): output_adapter_sink uses
   std::move; add #include <utility>.

4. ci_cuda_example (nvcc 11.8): NVCC's front end rejects the default
   template argument on the binary_writer alias template. Revert the alias
   to its original single-parameter form (relying on binary_writer's own
   defaulted OutputSinkType) and spell out the full type in the vector-sink
   convenience functions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Encode big-endian numbers with a byte swap instead of std::reverse

write_number() reordered multi-byte numbers for the big-endian formats
(CBOR/MessagePack/UBJSON) with std::reverse over the byte array. GCC
lowered only some sizes to a bswap; clang kept a scalar byte shuffle
(0 bswap instructions in the CBOR number path). Replace the reverse with
size-dispatched __builtin_bswap16/32/64 helpers (portable shift fallback
for other compilers; std::reverse retained for exotic sizes such as a
long double number_float_t).

Codegen: the CBOR number path now emits bswap on both compilers
(gcc 2 -> 16, clang 0 -> 4). Output is byte-for-byte identical to the
previous implementation across the binary differential corpus.

Throughput (isolated vs the std::reverse version, best of 9):
  CBOR int64 array   gcc +7%   clang +10%
  CBOR uint16 array  gcc +27%  clang flat

Modest but consistent on number-dense encodings; negligible on
string/blob-heavy output, as expected.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Reserve output capacity up front for binary serialization

The vector-returning to_cbor/to_msgpack/to_ubjson/to_bjdata/to_bson grew
the output buffer purely by geometric reallocation. Reserving an estimate
up front avoids the early reallocations, which is the dominant per-byte
cost for array/object-heavy output.

The estimate (binary_reserve_hint) is deliberately conservative and safe
against untrusted input: it consults only the top-level element count
(O(1), no walk of the DOM), guards the multiplication against overflow,
and clamps the result to a fixed 1 MiB ceiling, so a large or hostile DOM
can never force an oversized allocation here. The buffer still grows
geometrically past the hint, so an underestimate only costs a few later
reallocations; scalars/strings/binary are written in one shot and get no
hint. Reserving capacity does not change the bytes produced.

Throughput (g++/clang -O3, vs the previous commit):
  cbor int array     +10% / +13%
  cbor object array  +20% / +38%

Output is byte-for-byte identical to develop across the binary
differential corpus.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Address review findings on the binary writer output sinks

- binary_reserve_hint(): the 4-bytes-per-element estimate over-reserved by up
  to 4x for arrays of small scalars (CBOR encodes 0..23 in one byte), and the
  returned vector kept that capacity. Make the hint a strict lower bound on the
  encoded size instead, which also removes the 1 MiB clamp whose branch no test
  could reach (the largest container in the suite has 65793 elements).

- Guard the -Wduplicated-branches pragma with __GNUC__ >= 7. The warning does
  not exist before GCC 7, so naming it made GCC 4.8/4.9/5/6 - which the CI
  matrix still builds - warn under -Wpragmas on every including translation
  unit, breaking downstream -Werror builds.

- Constrain the adapter constructor of binary_writer with the enable_if its
  documentation already claimed, so a writer over some other sink type is no
  longer advertised as constructible from an output adapter.

- Let output_vector_adapter wrap output_vector_sink rather than duplicating the
  append logic, so the type-erased and templated paths share one implementation.

- Collapse the three copies of the memcpy/byte_swap/memcpy dance into a single
  byte_swap_buffer() helper, and add the MSVC _byteswap_* intrinsics so MSVC no
  longer falls back to the scalar shuffle this change exists to eliminate.

- Add a vector_writer() helper for the five vector-returning to_* overloads
  instead of spelling out the writer type at each call site, and drop a dead
  default member initializer on output_adapter_sink.

- New tests: the vector sink and the adapter sink must produce identical bytes
  for every format (the two to_* overloads no longer delegate to each other and
  could otherwise drift), and binary_reserve_hint() must never exceed the size
  actually written.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Route the -Wduplicated-branches pragma through Hedley

Match #5485, which moved the binary writer's hand-rolled diagnostic
pragmas onto JSON_HEDLEY_PRAGMA (merged into develop while this branch
was open). The devirtualization's -Wduplicated-branches suppression in
write_compact_float was the one raw '#pragma GCC diagnostic' left; it
now uses JSON_HEDLEY_PRAGMA like the adjacent -Wfloat-equal line, still
guarded to GCC >= 7 and non-clang (the warning exists only there).

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-23 08:59:46 +02:00
Niels Lohmann f92024b317 De-duplicate the swap() diagnostic-positions characterization test (#5540) 2026-09-23 07:48:21 +02:00
Niels Lohmann 0b20b7e622 Reject MessagePack/BSON binary subtypes that don't fit their wire format (#5469)
* Reject MessagePack/BSON binary subtypes that don't fit their wire format

Both formats store byte_container_with_subtype's subtype (a uint64_t)
in a single byte. The writers cast to std::int8_t/std::uint8_t without
a range check, so subtypes above 255 were silently truncated modulo
256 instead of raising an error. Throw out_of_range.413 instead when
the subtype exceeds the representable range of 0-255.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Move the new binary-subtype regression test out of unit-regression2.cpp

unit-regression2.cpp is already at the edge of what the MinGW linker
can relocate; adding this test's ~26 lines tips test-regression2_cpp20
(clang, Windows) over into "relocation truncated to fit:
IMAGE_REL_AMD64_REL32 against `.rdata'" (see 8ce64b9c1 / b82717c8a for
the same failure mode). Split the test along format lines instead:
MessagePack assertions move to unit-msgpack.cpp, BSON assertions to
unit-bson.cpp. The CBOR round-trip guard is dropped as redundant --
unit-cbor.cpp's "Tagged values" section already round-trips subtypes
up to 8589934590, far past the 70000 checked here.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-22 21:52:32 +02:00
Niels Lohmann d2c1a6a272 Reject array insert(pos, first, last) iterators not pointing into an array (#5468)
The array-range insert() overload checked that pos fits the current
value and that first/last share the same owning value, but never
verified that value is itself an array. Passing iterators from an
object, a primitive, or null handed value-initialized (singular)
std::vector iterators straight to array_t::insert(), which is
undefined behavior. Add the missing is_array() check, mirroring the
equivalent check already present in the object-range insert()
overload.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-22 21:52:31 +02:00
Niels Lohmann a2b19d6158 Honor allow_exceptions=false for excessive array/object size (out_of_range.408) (#5467)
* Honor allow_exceptions=false for excessive array/object size (out_of_range.408)

The SAX DOM parsers' start_object()/start_array() threw out_of_range.408
directly via JSON_THROW when a binary format (CBOR/UBJSON/BJData) declared
a container size exceeding max_size(), bypassing the allow_exceptions flag
that every other malformed-input error path in these classes honors via
parse_error(). This meant that json::from_cbor(data, true, false) etc.
could still throw (or abort under JSON_NOEXCEPTION) instead of returning a
discarded value, contrary to the allow_exceptions=false contract.

Route all four call sites (two in json_sax_dom_parser, two in
json_sax_dom_callback_parser) through parse_error() instead, matching the
existing error-handling pattern used elsewhere in this file. Behavior is
unchanged when allow_exceptions is true (the default); the exception
message and type are identical.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Drop a non-portable exact exception message check in the 408 test

The allow_exceptions=false regression test checked the exact message
text produced when allow_exceptions=true (the default). On platforms
where std::size_t is 32-bit (e.g. mingw x86, MSVC Win32 builds), a
declared CBOR length of 2^63 is intercepted earlier, by
get_cbor_container_size()'s own (pre-existing, already correct)
length-narrowing check, with different wording than this fix's
start_array()/start_object() size check -- same error code, same
"still throws when allow_exceptions=true" guarantee, different text.

CHECK_THROWS_AS already verifies the behavior this test cares about
(still throws json::out_of_range, unchanged); drop the exact-message
assertion since it isn't portable across size_t widths and doesn't
add coverage of this fix specifically.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix -Werror=unused-result on json::from_cbor() in the 408 regression test

from_cbor() is [[nodiscard]]; CHECK_THROWS_AS() otherwise discards its
result, which GCC flags under -Werror. Assign to a throwaway json, as
the rest of the suite already does for from_cbor()/from_msgpack().

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-22 21:52:31 +02:00
Niels Lohmann e485441123 Restore a duplicate key's prior value when the callback rejects its new value (#5466)
* Restore a duplicate key's prior value when the callback rejects its new value

json_sax_dom_callback_parser::key() unconditionally overwrote the object
slot for a key with a `discarded` placeholder as soon as the key was
accepted by the parser callback. For a duplicate key (legal JSON), this
destroyed the pre-existing value from an earlier occurrence of the same
key before the new value was even parsed. If the new value was then
rejected by the callback, remove_discarded_value() erased the member
entirely instead of leaving the original value in place, contradicting
the documented behavior that a discarded value behaves as if it was
never read.

Add a small stash of (slot pointer, previous value) pairs so that when
key() overwrites an existing member with the discarded placeholder, the
previous value can be restored later if the corresponding value (scalar,
object, or array) is rejected, instead of being erased. The stash entry
is dropped without restoring once the new value is definitively
accepted (in handle_value() for scalars, end_object()/end_array() for
containers), so a duplicate key whose new value is accepted still keeps
the last value as before. Non-duplicate keys are unaffected: rejecting
their value still removes the member entirely, since there is nothing
to restore.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Mark parser-callback test lambdas noexcept to fix GCC -Wnoexcept -Werror

GCC's libstdc++ std::function move assignment evaluates a noexcept
check that invokes a wrapped callable in an unevaluated context; a
non-noexcept parser_callback_t lambda then trips -Wnoexcept ("noexcept-
expression evaluates to 'false'"), which CI's ci_test_gcc job builds
with -Werror. The pre-existing parser_callback_t test lambdas in this
file already work around this by declaring themselves noexcept; apply
the same fix to the three added lambdas that didn't.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-22 21:52:30 +02:00
Niels Lohmann e564136c22 Make diff() account for member order in ordered_json objects (#5465)
* Make diff() account for member order in ordered_json objects

diff() compared source/target objects purely by key set, ignoring
relative member order. For ordered_json (insertion-ordered, vector-
backed object_t), two objects that differ only in member order are
unequal via operator==, but diff() never emitted any patch operation
to fix the order, so source.patch(diff(source, target)) == target
could fail to hold.

Fix by detecting when common keys appear in a different relative
order in source vs. target (or when a new key would need to land
somewhere other than the end), and in that case removing and
re-adding the affected keys in target's order, which relies on
patch()'s "add" op appending new keys at the end of an ordered_map.
For plain json (std::map-backed, always key-sorted iteration) this
is a no-op and the original minimal per-key diff path is unchanged.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Avoid redundant lookups in diff()'s object-order tracking

The previous fix for ordered_json member order re-derived common-key
order and suffix information with extra target.find()/source.find()
calls layered on top of the pre-existing removed/added-key passes,
instead of reusing those same passes. This roughly tripled the number
of map lookups per diff() call for every object, including plain
`json`, where the reordering path is never taken.

Piggyback the order tracking (and the "add" op construction for new
keys) onto the two passes the algorithm already needs to detect
removed/added keys, and walk the fast path's recursion in lockstep
with the precomputed common-key list instead of re-querying `target`.
This restores diff() to its pre-existing lookup count; benchmarked at
n=1000 keys, ordered_json::diff() was roughly 2x slower than baseline
before this change and is back within noise of baseline after it.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Preserve diff()'s original op ordering and fix a slow-path deletion gap

Splitting removed-key detection and common-key recursion into separate
passes (for the earlier lookup-count fix) changed the emitted patch's
op order: all "remove" ops now came before all recursive per-key diffs,
instead of interleaved in source's iteration order as the original
implementation did. This broke docs/mkdocs/docs/examples/diff.output's
exact-match CI check (ci_test_examples) even though the patch was still
semantically correct.

Defer "remove" emission into the same walk that does the recursive
diffs, so common keys and deleted keys are interleaved in source order
again, matching historical output.

While restructuring that walk, the reordering ("slow path") branch was
only emitting "remove" for keys common to both objects, never for keys
present in source but genuinely absent from target -- a key deleted
alongside an actual reorder would silently survive the patch. Fixed by
removing every source key in the slow path (both deleted and common
keys need removing there; common keys are then re-added in target's
order). Verified with a targeted reorder+deletion case and a fresh
20,000-case round-trip fuzz run (0 failures).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-22 21:52:30 +02:00
Niels LohmannandClaude Sonnet 5 663013ce64 Fix incorrect diagnostic-positions test assertion for swap() (#5539)
The characterization test added in #5482 asserted that basic_json::swap()
does NOT exchange start_position/end_position, based on a misreading of
the code cited for #5420. In fact swap() (json.hpp, around line 3637)
does swap start_position/end_position along with the value, consistent
with copy-assignment. The test's assumption was backwards, so it failed
on every CI job across every branch/PR since the commit landed. Correct
the assertions to match the actual (and correct) behavior: positions are
exchanged together with values.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-22 21:35:00 +02:00
Niels Lohmann c41152e620 Fall back to plain-object encoding when to_bjdata()'s _ArrayType_ annotation is not a string (#5494)
* Fall back to plain-object encoding when _ArrayType_ is not a string

write_bjdata_ndarray() looked up _ArrayType_ by calling get<string_t>()
directly, which throws type_error.302 when the annotation is not a
string (e.g. a number, null, boolean, array, or object). Per the
documented BJData ndarray contract, an object only qualifies for the
compact ndarray encoding if _ArrayType_ names a known type; anything
else must fall back to plain-object encoding, the same way an unknown
type-name string already does.

Add an is_string() check before the get<string_t>() call so a
non-string _ArrayType_ takes the existing "unrecognized type name"
fallback path instead of throwing.

Fixes #5398.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Relax the BJData fuzzer's round-trip check from byte-exact to value-exact

Fixing #5398 lets to_bjdata() proceed past the object it used to reject,
which exposed a pre-existing, unrelated round-trip quirk to the fuzzer:
a binary_t value serialized through the non-optimized ("$U#"-less)
array encoding is parsed back as a plain array of numbers, since
from_bjdata() has no way to tell "array of uint8 numbers" apart from
"array of bytes" without that optimized header. Re-serializing that
plain array then goes through the generic smallest-type writer, which
- unrelated to this PR, and long predating it - prefers the 'i' (int8)
marker over 'U' (uint8) for values that fit both, so the re-encoded
bytes can differ from the original even though both decode to the same
value.

This is not introduced by the #5398 fix; the same divergence reproduces
from a bare json::binary_t value with no _ArrayType_ annotation
involved at all, on the commit immediately preceding it. A general fix
would mean changing the shared UBJSON/BJData smallest-type selection
that hundreds of existing tests pin to 'i' for small positive
integers, which is out of scope and too risky for this PR.

Update fuzzer-parse_bjdata.cpp's round-trip assertions to check that
re-serializing is value-stable (from_bjdata(to_bjdata(j)) == j) rather
than byte-exact, matching the guarantee BJData actually provides, and
add a regression test in unit-bjdata.cpp using the exact OSS-Fuzz input
that documents the behavior.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Compare dump()s instead of json values in the BJData fuzzer's round-trip check

The value-stability assertion added to fix the earlier OSS-Fuzz crash
(json::from_bjdata(to_bjdata(j2)) == j2) itself broke on a NaN payload:
IEEE 754 NaN is never equal to itself, so operator== reports two
structurally-identical trees containing a non-finite double as
different -- not a round-trip bug, just NaN's ordinary
non-reflexivity. dump() serializes any non-finite double the same
deterministic way (as JSON null, since JSON cannot represent NaN or
Infinity), so comparing dumps is stable under exactly the values that
break operator==.

Verified against both the original OSS-Fuzz crash input and the new
one (0x68 0x68 0x7c, which decodes to a NaN), plus a local 2.5M-case
random-input sweep with no failures.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-16 20:20:22 +02:00
Niels Lohmann 502e9d66f6 Fix to_bjdata() emitting the Draft-3-only 'B' marker in default Draft-2 mode (#5479)
* Fix to_bjdata() emitting the Draft-3-only 'B' marker in default Draft-2 mode

_ArrayType_ = "byte" mapped unconditionally to the BJData type marker
'B', regardless of the requested bjdata_version. 'B' is defined only by
BJData Draft 3; with the default version (draft2), this produced a
stream that is invalid for Draft 2 and, unlike every other
_ArrayType_, round-tripped back as a binary value instead of the
original annotated object.

Only accept "byte" / emit 'B' when bjdata_version selects Draft 3.
Under Draft 2, fall back to the same plain-object encoding used
elsewhere in this function for other invalid-annotation cases, so the
value round-trips correctly.

Fixes #5404.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Future-proof the Draft-3-only 'B' marker gate

@gregmarr pointed out that dtype == 'B' && bjdata_version != draft3
only future-proofs by accident, since bjdata_version_t currently has
exactly two values. Compare with < instead, so a later draft that
keeps the 'B' marker valid does not need this gate revisited.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-16 20:20:21 +02:00
Niels Lohmann e8e1ba0db9 Fix dead ill-formed-fourth-byte UTF-8 test sections (byte3/byte4 typo) (#5499)
* Fix dead ill-formed-fourth-byte UTF-8 test sections (byte3/byte4 typo)

The "ill-formed: wrong fourth byte" SECTIONs in unit-unicode3.cpp,
unit-unicode4.cpp, and unit-unicode5.cpp guarded their loop with a check
on byte3 instead of byte4. Since the enclosing loop already restricts
byte3 to its valid range, the guard was always true and the section's
"continue" fired unconditionally, so check_utf8string()/check_utf8dump()
were never actually invoked for a malformed fourth byte.

Fixing the guard naively (byte3 -> byte4) would also have swept the full
byte2 x byte3 combinatorics for every byte4 value, adding millions of
redundant iterations: the lexer validates continuation bytes strictly in
sequence with early exit (see next_byte_in_range() in lexer.hpp), so once
byte2/byte3 are within their valid range, the byte4 outcome does not
depend on which valid byte2/byte3 values were chosen. Instead, byte2 and
byte3 are now held to a small hedge of representative valid prefixes
(range corners plus a midpoint) while byte4 is still swept exhaustively
over its full 0x00-0xFF range, since that is the actual property under
test. Also fixed the garbled "skip fourth second byte" comment in
unit-unicode3.cpp.

Verified offline: before the fix, the "wrong fourth byte" subcase
executes 0 assertions in all three files (proving it was dead code);
after the fix, it executes 11520 (unicode3), 34560 (unicode4), and 11520
(unicode5) assertions, and a deliberately reintroduced bug in the
lexer's byte4 range check causes it to fail (proving it is now
meaningful). Total per-file assertion counts grow by the same small
amounts, not by millions, and all other sections in these files still
pass unchanged.

Fixes #5416

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Keep full byte2 x byte3 combinatorics in the wrong-fourth-byte sections

The maintainer wants exhaustive coverage of every byte combination here
rather than the representative-prefix reduction, matching the style of
the sibling "wrong second/third byte" sections in the same files.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-16 20:16:43 +02:00
Niels Lohmann 29ba5973b6 Add missing diagnostic-positions test coverage (lifetime, input adapters, SAX) (#5482)
* Add missing diagnostic-positions test coverage (lifetime, input adapters, SAX)

Building on the merged unit-class_parser.cpp from #5417, add
characterization tests (regression protection for existing behavior, not a
behavior change) for JSON_DIAGNOSTIC_POSITIONS:

- value lifetime: copy ctor copies positions recursively, move ctor resets
  the moved-from value to npos, and mutating a parsed document (operator[],
  push_back, erase) leaves the parent's stale span and siblings' positions
  untouched while new values get npos.
- input adapters: wide-string input positions count transcoded UTF-8 bytes
  (not wide characters), BOM-prefixed input's start_pos() reflects the
  skipped 3-byte BOM, istringstream/ifstream/iterator-pair inputs report
  consistent (non-npos) positions, and binary formats (CBOR, MessagePack,
  UBJSON, BSON) always report npos.
- a user-constructed json_sax_dom_parser with no lexer (as used when driving
  json::sax_parse() directly) reports npos for every value, since it has no
  m_lexer_ref to source positions from.

While characterizing swap(), found that basic_json::swap() (and the friend
swap() that forwards to it) does not swap start_position/end_position,
unlike copy-assignment's operator=(basic_json), which does as part of its
copy-and-swap implementation. This looks like a real inconsistency/bug, but
per the scope of this test-only change it is only pinned (not fixed) here;
see the comment at the "swap() does NOT exchange positions" section.

Fixes #5420

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix MSVC source-encoding portability in the wide-string position test

Use é escapes instead of a literal UTF-8-encoded 'é' inside the L""
literal, so the wide string's content does not depend on the compiler's
assumed source character set (MSVC without /utf-8 decodes raw non-ASCII
source bytes using the system code page rather than as UTF-8, which was
producing a wstring of unexpected length/content and failing the
ws.size()/end_pos() assertions on Windows CI).

Also reworded a comment that unintentionally embedded the literal
substring "TODO check", which clang-tidy's google-readability-todo check
flags regardless of quoting context.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-16 20:16:40 +02:00
Niels Lohmann 75efd6b1c3 Test suite: cover untested macro configs, std::formatter branches, patch_inplace, and fix a duplicate TEST_CASE name (#5492)
* test: cover JSON_NO_IO, JSON_THROW/TRY/CATCH_USER, JSON_SKIP_LIBRARY_VERSION_CHECK, and JSON_DisableEnumSerialization in CI (#5423)

These four supported configuration macros were never actually compiled
anywhere in the test matrix:

- JSON_NO_IO and the JSON_THROW_USER/JSON_TRY_USER/JSON_CATCH_USER trio
  are exercised together in a new tests/src/unit-no_io_and_user_exceptions.cpp,
  which is automatically picked up by the existing unit-*.cpp test glob and
  thus built across the whole standard test matrix.
- JSON_SKIP_LIBRARY_VERSION_CHECK is exercised by a new, dedicated
  tests/src/skip_library_version_check.cpp, compiled directly by the new
  ci_test_skiplibraryversioncheck target in cmake/ci.cmake: the scenario it
  simulates (mixing two differently-versioned inclusions of the library)
  unavoidably triggers the compiler's own "macro redefined" warning, which
  would fail under the library's own -Weverything/-Werror unit test matrix
  for a reason unrelated to the macro under test.
- JSON_DisableEnumSerialization already had #if-guarded tests in several
  unit-*.cpp files (from #4384), but no CMake target ever actually set the
  JSON_DisableEnumSerialization CMake option, so that guarded code was never
  compiled. Add ci_test_disableenumserialization, mirroring the existing
  ci_test_noimplicitconversions/ci_test_noglobaludls targets. Building the
  full test suite with this option on surfaced one real, narrow gap: get<T>()
  on std::vector<std::byte> (used by unit-regression2.cpp's custom BinaryType
  tests) relies on std::byte being handled via enum serialization, so add the
  same #if-guard convention to the two affected SECTIONs there.

Both new CI targets are added to the ci_cmake_options matrix in
.github/workflows/ubuntu.yml, alongside the existing ci_test_* targets.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* test: cover multi-digit widths and bare alignment in std::formatter<json> (#5423)

Every existing std::formatter spec with a width used a single digit (e.g.
"{:2}"), so the width-parsing loop's accumulation of a second/third digit was
never exercised; add multi-digit width cases. Likewise, every existing spec
with an alignment character also had an explicit fill character, so the
bare-alignment branch (e.g. "{:<}", with no fill) was never exercised; add
cases asserting it keeps the default space indent character.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* test: add coverage for patch_inplace() (#5423)

patch_inplace() had no unit test at all. Add a happy-path case mirroring an
existing patch() example, and -- more importantly -- pin its distinguishing
contract versus patch(): when a multi-operation JSON Patch fails partway
through, patch_inplace() (which mutates the document directly, operation by
operation) leaves whatever operations already succeeded applied, whereas
patch() (which applies the patch to an internal copy that is discarded on
exception) leaves the original completely untouched either way. Verified
empirically against the current implementation before writing the assertions.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* test: fix duplicate TEST_CASE name in unit-no-mem-leak-on-adl-serialize.cpp (#5423)

Two distinct TEST_CASEs were both named "check_for_mem_leak_on_adl_to_json-2".
doctest allows duplicate names, so both still ran, but it makes
--test-case=<name> filtering and reporting ambiguous. Rename the second one
to "-3", continuing the existing "-1"/"-2" sequence.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* test: add direct coverage for the std::u8string to_json overload (#5423)

The ADL to_json overload for std::basic_string<char8_t, ...> was only ever
reached indirectly, via std::filesystem::path::u8string(). Add a test that
constructs a json value directly from a std::u8string, gated the same way as
the overload itself (include/nlohmann/detail/conversions/to_json.hpp): behind
both the std::filesystem::path feature guard and __cpp_lib_char8_t, since the
overload only exists when both are satisfied.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* test: verify move semantics of byte_container_with_subtype's rvalue constructors (#5423)

The two rvalue-reference constructors were never distinguished from their
const-lvalue-reference twins by any test. Add a "move semantics" section that
constructs from an rvalue std::vector, checks the resulting container keeps
the exact same buffer address as the source (a stronger check than just
observing the source ended up empty, since a copy-then-clear could do that
too), and confirms the source vector was left empty.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Guard patch_inplace() partial-application test against JSON_NOEXCEPTION

The "distinguishing contract vs patch(): partial application on
failure" test relies on doc.patch_inplace(patch) actually throwing so
the partially-applied state can be observed right after the throw
point. Under ci_test_noexceptions, JSON_THROW() calls std::abort()
instead of throwing, and doctest's --no-throw test filter (which that
CI job passes) makes CHECK_THROWS_AS() a no-op that never even
evaluates its expression -- so patch_inplace() is never called and the
follow-up assertions fail against the untouched original document.

Guard the whole SECTION with #if !defined(JSON_NOEXCEPTION), following
the same convention already used elsewhere in the test suite (e.g.
unit-class_parser.cpp) for exception-dependent tests.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix MSVC C2220 in the std::u8string conversion test

MSVC's C5321 ("nonstandard extension used: encoding '\xNN' as a
multi-byte utf-8 character") is promoted to a hard error by our MSVC CI
configs. It fires because the test composed a non-ASCII UTF-8 sequence
inside a u8"" literal using raw \x byte escapes; MSVC treats that as
nonstandard and suggests using \u universal-character-names instead,
which every compiler agrees on and which compiles down to the exact
same encoded bytes.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Guard the JSON_THROW_USER test against JSON_NOEXCEPTION and GCC's -Wunused-result

Two independent CI configurations failed to build/run this new test:

- ci_test_noexceptions runs the whole suite with -DJSON_NOEXCEPTION and
  doctest's "--no-throw" filter, which compiles CHECK_THROWS_AS() down
  to a no-op that never even invokes the guarded expression. Since this
  test's whole point is to observe json_throw_user_call_count after
  json::parse()/at() actually throw, it can't be meaningfully run under
  that filter (our JSON_THROW_USER override still throws real
  exceptions regardless of JSON_NOEXCEPTION, but the assertion never
  gets a chance to run). Guard the TEST_CASE with
  #if !defined(JSON_NOEXCEPTION), mirroring the existing precedent in
  unit-json_patch.cpp.

- ci_test_gcc and ci_test_standards_gcc(11) failed with
  -Werror=unused-result on the discarded json::parse() return value.
  json::parse() is marked warn_unused_result, and unlike a real
  [[nodiscard]] attribute, GCC does not consider that satisfied by
  doctest's (void)-cast around the expression in C++11 mode. Assign the
  result to a discarded local instead, matching the established
  `json _ = json::parse(...)` idiom already used throughout
  unit-class_parser.cpp.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Suppress a clang-tidy false positive on an intentional defensive copy

performance-unnecessary-copy-initialization suggests copy_for_patch
could be a reference since it's never modified -- but the copy is the
point: it guards against a hypothetical regression where patch()
mutates its receiver, which a reference could never catch (the
follow-up assertion would just compare `original` to itself).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix clang-tidy findings in the JSON_NO_IO/JSON_THROW_USER test

- bugprone-macro-parentheses: wrap the JSON_THROW_USER macro argument in
  parentheses at the throw site.
- modernize-raw-string-literal: switch two escaped JSON string literals to
  raw string literals.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-16 20:15:46 +02:00
Niels Lohmann f58db1c9e8 Add test coverage for ordered_json/alt_json across binary formats and patch/diff/flatten APIs (#5480)
* Add test coverage for ordered_json/alt_json across binary formats and patch/diff/flatten APIs

Closes a test-coverage gap from #5421: ordered_json (and the alt_string-based
basic_json specialization from unit-alt-string.cpp) were never round-tripped
through the binary formats (CBOR/MessagePack/UBJSON/BSON/BJData), nor through
flatten()/unflatten(), diff()/patch()/patch_inplace(), or merge_patch(). Also
adds a std::formatter<ordered_json> spot-check, mirroring the precedent set
by the format_as() ADL-deduction test.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Pass alt_string's std::string constructor argument by value (clang-tidy modernize-pass-by-value)

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-16 20:15:20 +02:00
Niels Lohmann ff65f688f7 Cover binary values in the indentation regression tests (#5533)
#5285's indentation regression test didn't exercise json::binary,
which serializes as an object but always writes its byte array
compactly (dump_byte()). #5186 had covered this case before it was
closed as superseded; port just that coverage here.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-16 20:11:19 +02:00
dependabot[bot] af91eee2cc Bump the codeql-action group with 4 updates (#5536)
Bumps the codeql-action group with 4 updates: [github/codeql-action/init](https://github.com/github/codeql-action), [github/codeql-action/autobuild](https://github.com/github/codeql-action), [github/codeql-action/analyze](https://github.com/github/codeql-action) and [github/codeql-action/upload-sarif](https://github.com/github/codeql-action).


Updates `github/codeql-action/init` from 4.37.9 to 4.38.0
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/cdf488f595d80d6e07e03d4674febd5ab45fa938...b96794f015dfd88f77b49b1c93e0fa7110f94c63)

Updates `github/codeql-action/autobuild` from 4.37.9 to 4.38.0
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/cdf488f595d80d6e07e03d4674febd5ab45fa938...b96794f015dfd88f77b49b1c93e0fa7110f94c63)

Updates `github/codeql-action/analyze` from 4.37.9 to 4.38.0
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/cdf488f595d80d6e07e03d4674febd5ab45fa938...b96794f015dfd88f77b49b1c93e0fa7110f94c63)

Updates `github/codeql-action/upload-sarif` from 4.37.9 to 4.38.0
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/cdf488f595d80d6e07e03d4674febd5ab45fa938...b96794f015dfd88f77b49b1c93e0fa7110f94c63)

---
updated-dependencies:
- dependency-name: github/codeql-action/init
  dependency-version: 4.38.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: codeql-action
- dependency-name: github/codeql-action/autobuild
  dependency-version: 4.38.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: codeql-action
- dependency-name: github/codeql-action/analyze
  dependency-version: 4.38.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: codeql-action
- dependency-name: github/codeql-action/upload-sarif
  dependency-version: 4.38.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: codeql-action
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-16 20:11:06 +02:00
Niels LohmannandClaude Opus 5 c72f37a40d Support custom object/array types and improve template parameter handling (#5443)
* docs: document the implicit requirements on basic_json's template parameters

The requirements that basic_json places on its eleven template parameters
were only implied by how the library uses the resulting object_t, array_t,
string_t, etc. Consumers had to discover them by trial and error.

Add "Template Parameter Requirements" collecting them, split into what is
always required and what is only required when a particular part of the API
is instantiated. Notable findings that were previously undocumented:

- ObjectType must provide a key_compare member type (actual_object_comparator
  names object_t::key_compare in both arms of a std::conditional), and its
  third template parameter is used as a comparator, so std::unordered_map
  cannot be used without a wrapper.
- ArrayType must provide capacity() -- push_back(), emplace_back(),
  operator+=(), and operator[](size_type) call it unconditionally -- and
  needs random-access iterators, so std::deque and std::list do not work.
- StringType needs contiguous, null-terminated data(), a one-byte value_type,
  and either assignability from std::to_string or an ADL int_to_string().
- NumberFloatType must be float, double, or long double for parsing and
  serialization; the integer types must satisfy std::is_integral.
- AllocatorType must be stateless, support incomplete types, and use plain
  pointers.
- BooleanType and the number types are union members and must be trivial.

Link the new page from the basic_json overview, the types feature page, and
the individual type alias pages, and correct the container examples given for
ObjectType (std::unordered_map) and ArrayType (std::list), which do not work.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix object_comparator_t for object types without key_compare

detail::actual_object_comparator selected between object_t::key_compare and
default_object_comparator_t with std::conditional. Both type arguments of
std::conditional are named eagerly, so object_t::key_compare had to exist
regardless of the condition, and the has_key_compare guard added in 3.11.0
never took effect: any ObjectType without a key_compare member type failed to
compile while instantiating basic_json itself.

Use detected_or_t instead, which resolves through a SFINAE partial
specialization and only names object_t::key_compare when it exists. The
selected type is unchanged for every object type that compiled before, so
object_comparator_t -- a public member type -- keeps its meaning and ABI.

has_key_compare had no other users and is removed.

Add a regression test using an adapter around std::unordered_map, which has no
key_compare; it fails to compile without this change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: list the types that are known to work for each template parameter

Follow up on the template parameter requirements page: state, for every
template parameter, which concrete types work and where they stop working.
Each entry was verified by compiling and running a common workload (DOM
access, dump, parse, CBOR/MessagePack round-trip, flatten, hash) against that
instantiation.

Findings worth calling out:

- ObjectType no longer needs a key_compare member type, so the std::unordered_map
  adapter only has to restore the template argument order. A hash-ordered
  ObjectType works everywhere except unflatten(), which reconstructs an array
  only when it meets the reference token 0 before the other indices.
- ArrayType: std::deque works when wrapped to add capacity(); std::list does not.
- StringType: std::pmr::string and std::basic_string with a custom allocator
  compile for the DOM, dump, and parse, but not for the binary readers, flatten,
  or diff, because the library assigns std::string values to string_t and
  int_to_string cannot be overloaded for a type in namespace std.
- NumberFloatType: long double works for dump and parse but not for the binary
  formats, which have no encoding for it.
- BinaryType: std::vector<std::byte> supports assignment, get, and the binary
  formats, but neither dump nor std::hash<basic_json>.

Also record the object_comparator_t fix in its version history.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix unflatten and binary dumping for non-default configurations

unflatten() decided between array and object by looking at the first reference
token it happened to see for a node: it started an array only when that token
was 0. With a sorted object type the token 0 always arrives first, so the
result was correct by accident; with an object type whose iteration order is
unspecified, {"/c/2":3,"/c/1":2,"/c/0":1} unflattened to an object with the
keys "0", "1", and "2" instead of an array.

Collect the pointer prefixes that have a reference token 0 among their children
before building the result, and let get_and_create() consult that set. The
outcome is now independent of the iteration order and matches, for every input,
what a sorted object type produced before: a value is restored as an array if
and only if one of its keys is 0. Iterating the flattened object in a different
order would have been simpler, but it would have changed the key order of the
result for insertion-ordered object types.

The serializer, std::hash, and the UBJSON writer converted the elements of a
binary value to an integer implicitly, which does not compile for a BinaryType
whose value type is std::byte, and which made dump() write the bytes of a
signed value type as negative numbers. Convert to std::uint8_t explicitly in
all three places, so every byte type dumps as 0..255. The default
std::vector<std::uint8_t> configuration is unaffected.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: note which Abseil containers can be used as template arguments

Checked against Abseil release 20250127.0 with the same workload as the other
entries on the page (DOM access, dump, parse, CBOR/MessagePack/UBJSON
round-trip, flatten, hash), with and without JSON_DIAGNOSTICS.

absl::flat_hash_map and absl::node_hash_map work as ObjectType through an
adapter that restores the template argument order and makes erase(iterator)
return the following iterator, which Abseil's returns as void. The page now
carries that adapter, and notes that absl::flat_hash_map does not keep
references to the mapped values valid across insertions while
absl::node_hash_map does. Both have a capacity() member, so JSON_DIAGNOSTICS
already refreshes the parent pointers conservatively for them.

absl::btree_map and absl::InlinedVector cannot be used at all: object_t and
array_t are formed while basic_json is still incomplete, and both inspect
their value type at class scope. std::map and std::vector are required by the
standard to tolerate this, third-party containers generally are not, so the
page states the constraint on its own rather than only per container.

absl::InlinedVector does work as BinaryType, where it is instantiated with a
complete type. absl::FixedArray and absl::Cord are not usable.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Relax the ArrayType and ObjectType requirements

Two requirements forced users of otherwise suitable containers to write a
wrapper, and neither was load-bearing.

array_t::capacity() was read in push_back(), emplace_back(), operator+=(), and
operator[](size_type), but set_parent() only looks at the value under
JSON_DIAGNOSTICS; without diagnostics it was computed and discarded. Read it
through array_capacity(), which reports unknown_size() when diagnostics are off
or when the array type has no capacity() at all, and treat an unknown capacity
as "the elements may have moved" so the parent pointers are refreshed
conservatively. std::deque now works as ArrayType, in both builds, and
capacity() is no longer named at all in a default build. Since the capacity is
now only meaningful for array insertions, it moves out of set_parent() into
set_parent_after_array_insert().

basic_json::erase(iterator) assigned the object's erase() return value, which
requires the container to return the following iterator. Abseil's hash maps
return void to avoid computing a successor the caller may not need. Detect that
and compute the successor before erasing; containers that return an iterator,
including the vector-backed ordered_map where a precomputed successor would be
wrong, keep the existing path.

Together these leave an Abseil hash map needing only an alias that restores the
template argument order, and no adapter at all for std::deque.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Do not require string_t to be convertible from std::string

Three places built a std::string and handed it to something expecting a
string_t: the UBJSON high-precision number reader, which every binary reader
instantiates, and the BSON writer's array element size calculation and write.
That silently required string_t to be implicitly convertible from std::string,
which std::string itself and types with a string_view conversion satisfy, but
many string types do not.

Construct the string_t explicitly from the data and size, which the
requirements already cover. This makes boost::container::string, eastl::string,
std::pmr::string, and std::basic_string with a custom allocator work as
StringType, none of which could previously be used with any binary format.

Add binary format coverage to the alt_string test, which had none, including a
UBJSON high-precision number -- the case that goes through the reader path.
BSON stays uncovered there: it additionally needs string_t::find(value_type),
which alt_string does not provide.

Also record which containers from Boost, Abseil, and EASTL work for each
template parameter, and correct two claims: std::pmr::string is usable after
this change, and tsl::ordered_map is not usable at all, because its iterators
expose the mapped value as const while basic_json modifies it in place.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: record compatibility for the common header-only hash maps

ankerl::unordered_dense (map and segmented_map), phmap (flat_hash_map and
node_hash_map), and robin_hood::unordered_flat_map all work as ObjectType
through the same adapter as Abseil's and Boost's hash maps, which only has to
restore the template argument order.

phmap::btree_map and robin_hood::unordered_node_map do not: like the other
btree containers they require a complete value type.

Note that none of these hash maps defines key_compare, so every one of them
depends on object_comparator_t falling back to default_object_comparator_t.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: record Folly and the remaining vector replacements

Folly works, with the caveat that its headers need C++20: folly::fbstring as
StringType, folly::fbvector and folly::small_vector as ArrayType,
folly::fbvector<std::uint8_t> as BinaryType, and folly::F14NodeMap as
ObjectType through the usual argument-order adapter. folly::F14FastMap is the
exception and requires a complete value type.

For ArrayType, boost::container::devector, boost::container::static_vector
(within its fixed capacity), and std::pmr::vector work as well.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: cover fifo_map, gtl, folly::sorted_vector_map, and Qt

nlohmann::fifo_map works through the adapter that has always been documented
for it, and preserves the insertion order. Restore its mention in the object
order page, which was dropped together with the tsl::ordered_map one: unlike
ordered_map it keeps a lookup index, so it is the insertion-ordered option
without the quadratic cost.

gtl::flat_hash_map and folly::sorted_vector_map work as well, the latter
through an alias that drops the allocator, whose value type it disagrees on.
gtl::btree_map does not, for the same reason as the other btree containers.

None of the Qt containers can be used, each for its own reason: QMap has no
value_type, QHash iterators yield the mapped value rather than a pair, QList
has no max_size(), QByteArray spells empty() as isEmpty(), and QString is
UTF-16.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: qualify the std::pmr::string support claim

Listing std::pmr::string as fully supported was an overclaim: it was only ever
checked with the default memory resource, which is not what PMR is for.

basic_json cannot be given an allocator or a memory resource, so a pmr string
inside a value always allocates from std::pmr::get_default_resource(), and
assigning an arena-backed string into a value silently drops its resource,
because polymorphic_allocator does not propagate on copy construction. Passing
polymorphic_allocator as AllocatorType does not compile either. Only the
process-global set_default_resource() redirects these allocations.

Say so, and separate the row from std::basic_string with a custom stateless
allocator, which is unaffected.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: remove a duplicated StringType compatibility section

The StringType section carried two 'Compatible types' tables and two copies of
the reference-implementation tip. The second table was a stale copy from before
the binary format string fixes and still listed std::pmr::string and
std::basic_string with a custom allocator as unusable, contradicting the
corrected table a few lines above it, and it dragged along the old explanation
that blamed int_to_string.

Drop the stale copy and put the surviving table before the notes, so the
'see below' in the std::pmr::string row points forwards.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: correct the template parameter requirements after independent verification

Every claim on the page was re-checked by compiling and running it, including
the rows that say a type cannot be used, which were checked to fail for the
documented reason and not merely to fail. Twenty-four claims were wrong.

The most consequential: the incomplete-type constraint applies to ObjectType
only. object_t is instantiated inside the class definition, because it is
probed for key_compare; array_t is only named there and is not instantiated
until basic_json is complete. So eastl::vector, QList and QVector are not
excluded by incomplete types at all -- they simply have no max_size() -- and
absl::InlinedVector is excluded for a subtler reason of its own.

Further corrections: ObjectType does not need erase(key), which has a fallback,
but does need at(key) for UBJSON output; only == and < are used, or == and <=>
under C++20, not all six; the documented adapter does not fit ankerl or
robin_hood. ArrayType needs no initializer-list insert, and value_type, the
(count, value) constructor and swappability are per-function, not always.
BinaryType needs a range insert for CBOR indefinite-length byte strings and
does not need push_back. StringType needs append(const StringType&)
unconditionally, and does not need operator!= or operator== against const
char*; empty(), resize(n) and reserve(n) are per-subsystem; int_to_string is
needed by diff, items and std::hash rather than by JSON Pointer or flatten.
BooleanType must be implicitly convertible from bool, and JSONSerializer's
second parameter need not carry a default.

std::pmr::string was wrong in the other direction this time: a moved-in string
does keep its memory resource, and later growth allocates from it. Only copies
land on the default resource.

Five requirement violations are not caught at compile time rather than the two
the page claimed; they are now listed together up front. Split every
compatibility table into what works and what does not, as the reasons in the
second half are the useful part.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Reduce the string_t and array_t members the library requires

Several members were required only because of how the library happened to
be written, not because the functionality needs them. Dropping them widens
the set of usable string and array types, and one of them was also a
performance problem.

string_t:

- c_str() is gone. Every call site already knew the length and passed it
  along, so data() is enough. The one place that did not, the diagnostics
  path in exceptions.hpp, now builds the token from data() and size(),
  which also stops it from truncating keys that contain a null byte.
- back() is gone; the serializer indexes the last character instead.
- find(str, pos), replace(), and substr() are gone. escape() and
  unescape() rebuilt the string with one replace() per escaped character,
  which moves the tail every time: escaping a string of n characters that
  all need escaping cost O(n^2). Both now scan with find_first_of() -- a
  member the pointer parser already required -- and append whole runs, so
  the common case is one search and one copy. Escaping 64000 tildes drops
  from 717 ms to 20 ms; a string with nothing to escape gets faster too
  (8.4 ms to 5.8 ms), because the scan is still a single memchr per pass.
  json_pointer::split() takes its reference tokens with the
  (const char*, size_type) constructor rather than substr().
- json_pointer::to_string() accumulates with concat<string_t> instead of
  letting concat default to std::string and converting afterwards, so
  streaming a json_pointer no longer requires string_t to be assignable
  from a std::string.

array_t:

- at(size_type) is gone. basic_json::at(size_type) checked the index by
  calling array_t::at() and translating std::out_of_range, which also
  required the array type to throw that exact exception. It now compares
  against size() and uses operator[]. The thrown exception, its message,
  and the behaviour under JSON_NOEXCEPTION are unchanged.

The BSON writer wrote the terminating null byte out of the string's own
buffer (size() + 1). It now writes the byte itself, so string_t::data()
need not be null-terminated for to_bson().

The tests pin the reduced API: alt_string loses the five dropped members
and gains coverage of the escaping paths, and a std::vector whose at() is
hidden is used as an ArrayType.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: record the reduced string_t and array_t requirements

Drop c_str(), back(), find(str, pos), replace(), and substr() from the
StringType requirements and at(size_type) from the ArrayType ones, and
note the string assignment the JSON pointer code performs. Streaming a
json_pointer no longer needs assignability from a std::string.

Add the non-null-terminated data() to the list of violations that are not
diagnosed at compile time -- it was described in the StringType section
but missing from the summary at the top -- and correct the QString row,
which no longer fails for the c_str() it lacks.

JSON_CATCH_USER no longer wraps a catch of std::out_of_range: the last one
went away with array_t::at(). Describe what the library actually catches.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Use character literals for the signed BinaryType test

MSVC rejects char(0xFF) with C4310 (cast truncates constant value),
which the Windows workflow treats as an error. The character literals
carry the same byte values without a narrowing cast.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Do not instantiate a hash map with an incomplete basic_json in the tests

object_t is probed for key_compare inside the definition of basic_json, so
it is instantiated while basic_json is still incomplete. Whether a hash map
survives that depends on the standard library: libstdc++ 9 needs the size of
the mapped type to instantiate std::unordered_map's node type and rejects
the adapter, which broke the GCC 9 builds.

The test now derives its no-key_compare object type from std::map -- which
does cope -- and shadows the inherited key_compare member type with an
entity that is not a type, so the library's probe finds none, exactly as for
a hash map. The unflatten() order-independence checks in unit-json_pointer
already cover the behaviour that the unordered object type was there for.
The limitation is documented for std::unordered_map.

Also address two Clang-Tidy findings the earlier commits introduced:
erase_from_object() declares its iterator with auto, and at(size_type) checks
the type first and then falls through to the return instead of throwing from
an else branch.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Keep diagnostic key paths null-terminated

Building the token from data() and size() kept an embedded null byte in the
key, and since what() hands out a C string, that truncated the whole message
rather than just the key: to_bson() on a key containing U+0000 reported
"[json.exception.out_of_range.409] (/en" instead of the full explanation.
This broke test-bson under JSON_DIAGNOSTICS.

Constructing from data() alone stops at the first null byte, which is what
c_str() did before, so the message is unchanged -- without requiring
string_t to provide c_str().

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Do not parse the value in the array-at() test

JSON_DIAGNOSTIC_POSITIONS adds the byte range of the value to the exception
message, which a parsed value has and an in-memory one does not, so the two
message checks failed in that configuration. Build the array in memory
instead of parsing it; the test is about at(size_type) not needing
array_t::at(), and the byte range is beside the point.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Move the custom BinaryType tests into their own translation unit

The two sections added to unit-regression2.cpp brought a third full
basic_json instantiation into a translation unit that was already large.
With Clang on MinGW that pushed the object over the reach of a 32-bit
relocation and test-regression2_cpp20.exe failed to link:

    relocation truncated to fit: IMAGE_REL_AMD64_REL32 against `.rdata'

unit-regression2.cpp is restored to exactly what it was before, and the
coverage moves to unit-custom-binary-type.cpp, next to the object and array
type tests it belongs with. The signed value type is now also covered in
C++11, where std::byte is not available.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Do not require the container iterators to be nothrow move constructible

iter_impl declared its defaulted move operations noexcept. The exception
specification a defaulted function gets implicitly follows from its members,
here internal_iterator, which holds the object and array iterators. libstdc++
gives std::deque's iterator a user-provided copy constructor without noexcept
before version 11, so the implicit specification is noexcept(false) and does
not match the declared one. That deletes the function -- and with g++ 4.8,
which predates CWG 1778, it is an error outright:

    error: function 'iter_impl<basic_json<std::map, std::deque> >::iter_impl(
    iter_impl&&)' defaulted on its first declaration with an
    exception-specification that differs from the implicit declaration

So std::deque, which this branch documents as a usable array type, could not
be used with an older standard library. Leaving the specification to be
computed cannot mismatch; iteration_proxy_value already spells out the same
condition next door.

The default configuration is unaffected: json::iterator, json::const_iterator
and ordered_json::iterator stay nothrow move constructible and move
assignable, which the test now checks so it cannot regress unnoticed.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Address two Clang-Tidy findings the custom container tests exposed

Both come from instantiating basic_json with containers other than the
default ones, and neither shows up with the Clang-Tidy version available
outside CI:

- insert(const_iterator, basic_json&&) forwards its by-value iterator to
  the const-reference overload. performance-unnecessary-value-param asks
  for the copy to be a move; it only fires for an iterator that is not
  trivially copyable, as std::deque's is not. The NOLINT on the function
  does not cover it, because the finding is reported where the parameter
  is used rather than where it is declared. Move it, which is what the
  check asks for and is a (very small) improvement in its own right.

- cppcoreguidelines-use-enum-class rejects the unnamed enum that shadowed
  the inherited key_compare member type. An enum class would not do, since
  it declares a type of that name and the probe would find it again; a
  member function declaration hides the name just as well.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Assert the iterators' exception specification relative to the container

The test pinned that nlohmann::json's iterators stay nothrow movable after
iter_impl's defaulted move operations lost their declared noexcept. That is
not a property of the library, though: the exception specification is now
computed from the container iterators, so it holds only for standard library
implementations whose iterators are themselves nothrow movable.

MSVC's checked iterators before VS2017 are not -- _Iterator_base12 registers
the iterator with the container's debug proxy in a copy constructor that
carries no noexcept -- so the assertions fail on a Visual Studio 2015 debug
build, which is the one debug configuration in the AppVeyor matrix and has no
counterpart in the GitHub Actions matrix.

Assert what the change actually guarantees instead: the iterators are nothrow
movable exactly when the object and array iterators they are built from are.
That still pins the default configuration against a silent regression, and it
is true whatever the standard library provides.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Detect a void-returning erase() through a named trait

erase_from_object() distinguished its two overloads with a decltype of a
member call written inline in a default template argument. Every other
detection in the library goes through the detector machinery in detected.hpp
instead -- has_erase_with_key_type is the same question about the same member
function -- and the inline form is the one shape older compilers are least
reliable about.

Express it the same way: detect_erase_with_iterator plus is_detected_exact,
both of which the library already relies on elsewhere. No behaviour changes.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Give the custom container types only the constructors the library uses

The three container types in the new tests inherited every constructor of
their base with using Base::Base. That asks for more than the test needs: the
library builds an object or an array by default construction, by copy or move,
and -- when converting between two basic_json types or from an initializer
list -- from an iterator range. Declaring those directly makes the requirement
visible in the test, and keeps object types out of a corner where a compiler
has to declare std::map's whole constructor set for a derived class while
basic_json is still incomplete.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Temporarily disable the new custom container tests

AppVeyor is the only CI that builds MSVC 2015 and 2017, and it has now
rejected three heads of this branch. Its build log is not reachable from
where this is being worked on, so the verdict is a single bit and the cause
has to be narrowed down by bisection.

Everything else stays: the library changes, the reduced alt_string, and the
unflatten() tests. If AppVeyor passes with these three translation units
disabled, the cause is one of the six basic_json instantiations they add; if
it fails, it is in the library. Either way this commit is reverted.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Guard the disabled tests with a macro rather than #if 0

Clang-Tidy's readability-avoid-unconditional-preprocessor-if rejects a literal
#if 0. Use a macro that is never defined instead, which the check does not
look at. Still temporary, and reverted together with the previous commit.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Re-enable the array and binary container tests

AppVeyor passed with all three new translation units disabled, so the library
changes, the reduced alt_string, and the unflatten() tests are fine on MSVC
2015 and 2017; the cause is one of the six basic_json instantiations the new
tests add.

Bring back two of the three. If AppVeyor passes again, the cause is in
unit-custom-object-type.cpp, which is the one still disabled; if it fails, it
is in one of these two and needs one more split. Still temporary.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Diagnose two silently violated template parameter requirements

Both were on the list of requirements that are not caught at compile time and
corrupt values rather than failing, and both are a plain size comparison:

- A BinaryType whose value_type is wider than one byte, which the readers and
  writers reinterpret as raw bytes anyway.
- A NumberUnsignedType too narrow to hold the absolute value of every
  NumberIntegerType value, which makes basic_json(INT64_MIN).dump() yield -0
  for std::int64_t with std::uint32_t.

Neither static_assert rejects a configuration that worked before: both only
fire where the result was already wrong. Also add the two comments the review
asked for, in write_bson_string() and calc_bson_array_size(), matching the
ones their counterparts already carry.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Align the template parameter tables and record what is now diagnosed

Every table in the page is reformatted so each column is exactly as wide as
its widest cell, which is what the review asked for in a dozen places: the
separator rows that ran two dashes long, the stray spaces, and the columns
padded well past their content.

The row listing six containers that require a complete mapped type is split
in two so that one cell no longer sets the width of the whole table.

Content changes: NumberUnsignedType is described as any unsigned integer type
at least as wide as NumberIntegerType rather than any unsigned integer type;
the two requirements that are now static_asserts move out of the list of
violations that are not caught at compile time; and the two places that
require a non-const operator[] say why data() will not do (std::string has no
non-const data() before C++17).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Bisect the other way: only the object container tests

The previous head touched only docs/, which AppVeyor's only_commits filter
skips, so it produced no build and no status at all -- the pull request looked
green without ever having been built on MSVC 2015 or 2017.

Swap the guards instead of repeating that step: unit-custom-object-type.cpp is
enabled and the array and binary translation units are disabled. AppVeyor
already passed with all three disabled, so a failure here pins the cause on
no_key_compare_json or void_erase_json, and a pass pins it on the array or
binary file. Still temporary.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Split the two object types apart

AppVeyor failed with only unit-custom-object-type.cpp enabled and passed with
all three new translation units disabled, so the cause is one of the two
object types in this file and not the array or binary ones.

Guard out void_erase_map and leave no_key_compare_map, which separates the two
constructs under suspicion: shadowing the inherited key_compare member type
with an entity that is not a type, and hiding the inherited erase with a
void-returning overload. A failure here points at the first, a pass at the
second. Still temporary.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Build the no-key_compare object type by composition, not inheritance

The "object type without key_compare" test failed on AppVeyor's MSVC
2017 jobs (/std:c++17): its no_key_compare_map derived publicly from
std::map and shadowed the inherited key_compare type with a same-named
member function, relying on ordinary member hiding to make key_compare
unreachable as a type for the library's detection trait. MSVC 2017
does not honor that hiding for a typename-qualified lookup performed
from outside the class and still resolves key_compare to the base's
comparator type, so object_comparator_t incorrectly picked it up
instead of falling back to default_object_comparator_t.

Wrapping a std::map by composition instead removes the base class
entirely, so there is no key_compare to find under any lookup rule,
on any compiler. Also drops the now-unneeded JSON_BISECT_CUSTOM_CONTAINER_TESTS
guard left over from narrowing this down: the void_erase_map test in
the same file was never the cause and is re-enabled unconditionally.

Verified locally with clang++ and g++ under C++17 and C++20.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Re-enable the array and binary custom-container tests

unit-custom-array-type.cpp and unit-custom-binary-type.cpp were still
guarded behind JSON_BISECT_CUSTOM_CONTAINER_TESTS from bisecting the
AppVeyor failure fixed in 7c39f3227, which was unrelated to either
file. The macro was never defined, so none of these tests actually ran
in CI. Verified locally with clang++ and g++ under C++17 and C++20
before removing the guards.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix indentation of custom_object_type per astyle

The one-line function bodies in the composition-based no_key_compare_map
(7c39f3227) do not match the project's Allman brace style, which the
ci_test_amalgamation job enforces with astyle. Reformatted with the
pinned astyle 3.4.13; no functional change.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: add compiled reference implementations for the container template parameters

Each of ObjectType, ArrayType, StringType, and BinaryType now links to a
minimal, self-contained header (docs/mkdocs/docs/examples/custom_*_type.hpp)
that wraps the corresponding standard container by composition and satisfies
every "Always required" member listed on that page. Unlike the prose
requirement lists, these are real code: each header has a companion .cpp that
instantiates a basic_json specialization with it and is compiled and run by
the existing ci_test_examples check (docs/Makefile's check_output_portable),
so the reference implementations cannot silently drift from what the library
actually requires. The .output files were generated with that same target.

StringType's existing pointer to tests/src/unit-alt-string.cpp's alt_string
is kept alongside the new header as a more thorough, battle-tested example.

Verified locally: astyle (pinned 3.4.13, project .astylerc) on the new files;
clang++/g++ under C++11/17/20 for each example against the amalgamated
header; `make check_output_portable` in docs/; `mkdocs build --strict` and
scripts/check_structure.py for the page itself.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Declare no_key_compare_map's accessors noexcept

The GCC C++20 job builds with -Wnoexcept and -Werror, and the standard
library takes noexcept(c.begin()) and noexcept(c.end()) in ranges_base.h and
range_access.h. Forwarding to std::map without repeating its noexcept made
those expressions false, which the warning reports as an error:

  error: noexcept-expression evaluates to 'false' because of a call to
         no_key_compare_map<...>::begin()          [-Werror=noexcept]
  note:  but ... does not throw; perhaps it should be declared 'noexcept'

Give the accessors the exception specification of what they forward to.
std::map declares begin, end, cbegin, cend, empty, size, max_size, and clear
noexcept, so the wrapper does too. swap is left alone: std::map's is only
conditionally noexcept, and nothing asks for it.

void_erase_map is unaffected because it still derives from std::map and
inherits accessors that already carry the specification.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Write out what the defaulted constructor of no_key_compare_map implied

ci_test_gcc builds with -Weffc++, which asks for data to be initialized in a
member initialization list; a defaulted default constructor does not do that:

  error: 'no_key_compare_map<...>::data' should be initialized in the member
         initialization list                              [-Werror=effc++]

Writing the constructor out satisfies that but drops the exception
specification the defaulted one carried, which -Wnoexcept then objects to
where the standard library takes noexcept(construct(...)). Declare it the way
the defaulted constructor was: noexcept when the wrapped map's default
constructor is.

This is the cost of composition -- inheritance carried std::map's exception
specifications and initialization for free, and forwarding by hand has to
restate them.

Checked with the repository's own GCC warning set from cmake/gcc_flags.cmake,
all 346 flags, at C++11, C++17 and C++20: no diagnostics for this file, nor
for the two custom container translation units that were disabled while the
MSVC failure was narrowed down and are built again now.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Mark no_key_compare_map::swap noexcept

Clang-Tidy rejects a swap that is not:

  error: swap functions should be marked noexcept
         [cppcoreguidelines-noexcept-swap,performance-noexcept-swap]

It was left unmarked on the grounds that std::map::swap is only
conditionally noexcept, so an unconditional promise would be wrong for a
comparator or allocator that can throw while swapping. Both concerns are met
by taking the specification from the wrapped map rather than asserting one:
noexcept(noexcept(data.swap(other.data))). Clang-Tidy accepts that, and no
NOLINT is needed.

Last in the series of specifications that inheritance used to supply and
composition has to write out by hand.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hxZxz8svM54c6ATEvXp5E
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-15 20:46:37 +02:00
Niels Lohmann 6487678bc5 Fix swap(array_t&)/swap(object_t&) to update parent pointers under JSON_DIAGNOSTICS (#5464)
Both overloads swapped the underlying container storage but never called
set_parents(), leaving elements moved into *this with stale m_parent
pointers (typically nullptr from the free-standing array_t/object_t).
This produced wrong JSON Pointer paths in diagnostic messages and could
trip assert_invariant() on subsequent copies. Mirrors the fix already
applied in swap(reference other).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-15 20:46:28 +02:00
99 changed files with 6558 additions and 786 deletions
+3 -3
View File
@@ -38,14 +38,14 @@ jobs:
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@cdf488f595d80d6e07e03d4674febd5ab45fa938 # v4.37.9
uses: github/codeql-action/init@b96794f015dfd88f77b49b1c93e0fa7110f94c63 # v4.38.0
with:
languages: c-cpp
# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@cdf488f595d80d6e07e03d4674febd5ab45fa938 # v4.37.9
uses: github/codeql-action/autobuild@b96794f015dfd88f77b49b1c93e0fa7110f94c63 # v4.38.0
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@cdf488f595d80d6e07e03d4674febd5ab45fa938 # v4.37.9
uses: github/codeql-action/analyze@b96794f015dfd88f77b49b1c93e0fa7110f94c63 # v4.38.0
+1 -1
View File
@@ -43,6 +43,6 @@ jobs:
output: 'flawfinder_results.sarif'
- name: Upload analysis results to GitHub Security tab
uses: github/codeql-action/upload-sarif@cdf488f595d80d6e07e03d4674febd5ab45fa938 # v4.37.9
uses: github/codeql-action/upload-sarif@b96794f015dfd88f77b49b1c93e0fa7110f94c63 # v4.38.0
with:
sarif_file: ${{github.workspace}}/flawfinder_results.sarif
+1 -1
View File
@@ -76,6 +76,6 @@ jobs:
# Upload the results to GitHub's code scanning dashboard.
- name: "Upload to code-scanning"
uses: github/codeql-action/upload-sarif@cdf488f595d80d6e07e03d4674febd5ab45fa938 # v4.37.9
uses: github/codeql-action/upload-sarif@b96794f015dfd88f77b49b1c93e0fa7110f94c63 # v4.38.0
with:
sarif_file: results.sarif
+1 -1
View File
@@ -61,7 +61,7 @@ jobs:
# Upload SARIF file generated in previous step
- name: Upload SARIF file
uses: github/codeql-action/upload-sarif@cdf488f595d80d6e07e03d4674febd5ab45fa938 # v4.37.9
uses: github/codeql-action/upload-sarif@b96794f015dfd88f77b49b1c93e0fa7110f94c63 # v4.38.0
with:
sarif_file: semgrep.sarif
if: always()
+1 -1
View File
@@ -100,7 +100,7 @@ jobs:
container: ubuntu:focal
strategy:
matrix:
target: [ci_cmake_flags, ci_test_diagnostics, ci_test_diagnostic_positions, ci_test_noexceptions, ci_test_noimplicitconversions, ci_test_legacycomparison, ci_test_noglobaludls, ci_test_simdutf]
target: [ci_cmake_flags, ci_test_diagnostics, ci_test_diagnostic_positions, ci_test_noexceptions, ci_test_noimplicitconversions, ci_test_legacycomparison, ci_test_noglobaludls, ci_test_disableenumserialization, ci_test_skiplibraryversioncheck, ci_test_simdutf, ci_test_strict_nul_handling]
steps:
- name: Install build-essential
run: apt-get update ; apt-get install -y build-essential unzip wget git libssl-dev
+6
View File
@@ -59,6 +59,7 @@ option(JSON_LegacyDiscardedValueComparison "Enable legacy discarded value compar
option(JSON_Install "Install CMake targets during install step." ${MAIN_PROJECT})
option(JSON_MultipleHeaders "Use non-amalgamated version of the library." ON)
option(JSON_SystemInclude "Include as system headers (skip for clang-tidy)." OFF)
option(JSON_StrictNulHandling "Build with strict NUL-byte handling enabled." OFF)
if (JSON_CI)
include(ci)
@@ -108,6 +109,10 @@ if (JSON_Diagnostics)
message(STATUS "Diagnostics enabled (JSON_DIAGNOSTICS=1)")
endif()
if (JSON_StrictNulHandling)
message(STATUS "Strict NUL-byte handling enabled (JSON_STRICT_NUL_HANDLING=1)")
endif()
if (JSON_Diagnostic_Positions)
message(STATUS "Diagnostic positions enabled (JSON_DIAGNOSTIC_POSITIONS=1)")
endif()
@@ -141,6 +146,7 @@ target_compile_definitions(
$<$<BOOL:${JSON_Diagnostics}>:JSON_DIAGNOSTICS=1>
$<$<BOOL:${JSON_Diagnostic_Positions}>:JSON_DIAGNOSTIC_POSITIONS=1>
$<$<BOOL:${JSON_LegacyDiscardedValueComparison}>:JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON=1>
$<$<BOOL:${JSON_StrictNulHandling}>:JSON_STRICT_NUL_HANDLING=1>
)
target_include_directories(
+67 -7
View File
@@ -1421,7 +1421,7 @@ I deeply appreciate the help of the following people.
6. [Joshua C. Randall](https://github.com/jrandall) fixed a bug in the floating-point serialization.
7. [Aaron Burghardt](https://github.com/aburgh) implemented code to parse streams incrementally. Furthermore, he greatly improved the parser class by allowing the definition of a filter function to discard undesired elements while parsing.
8. [Daniel Kopeček](https://github.com/dkopecek) fixed a bug in the compilation with GCC 5.0.
9. [Florian Weber](https://github.com/Florianjw) fixed a bug in and improved the performance of the comparison operators.
9. [Fiona Johanna Weber](https://github.com/Fiona-J-W) fixed a bug in and improved the performance of the comparison operators.
10. [Eric Cornelius](https://github.com/EricMCornelius) pointed out a bug in the handling with NaN and infinity values. He also improved the performance of the string escaping.
11. [易思龙](https://github.com/likebeta) implemented a conversion from anonymous enums.
12. [kepkin](https://github.com/kepkin) patiently pushed forward the support for Microsoft Visual Studio.
@@ -1523,14 +1523,14 @@ I deeply appreciate the help of the following people.
108. [Kevin Tonon](https://github.com/ktonon) overworked the C++11 compiler checks in CMake.
109. [Axel Huebl](https://github.com/ax3l) simplified a CMake check and added support for the [Spack package manager](https://spack.io).
110. [Carlos O'Ryan](https://github.com/coryan) fixed a typo.
111. [James Upjohn](https://github.com/jammehcow) fixed a version number in the compilers section.
111. [James Upjohn](https://github.com/jupjohn) fixed a version number in the compilers section.
112. [Chuck Atkins](https://github.com/chuckatkins) adjusted the CMake files to the CMake packaging guidelines and provided documentation for the CMake integration.
113. [Jan Schöppach](https://github.com/dns13) fixed a typo.
114. [martin-mfg](https://github.com/martin-mfg) fixed a typo.
115. [Matthias Möller](https://github.com/TinyTinni) removed the dependency from `std::stringstream`.
116. [agrianius](https://github.com/agrianius) added code to use alternative string implementations.
117. [Daniel599](https://github.com/Daniel599) allowed to use more algorithms with the `items()` function.
118. [Julius Rakow](https://github.com/jrakow) fixed the Meson include directory and fixed the links to [cppreference.com](https://cppreference.com).
118. [Julius Rakow](https://github.com/juliusrakow) fixed the Meson include directory and fixed the links to [cppreference.com](https://cppreference.com).
119. [Sonu Lohani](https://github.com/sonulohani) fixed the compilation with MSVC 2015 in debug mode.
120. [grembo](https://github.com/grembo) fixed the test suite and re-enabled several test cases.
121. [Hyeon Kim](https://github.com/simnalamburt) introduced the macro `JSON_INTERNAL_CATCH` to control the exception handling inside the library.
@@ -1581,7 +1581,7 @@ I deeply appreciate the help of the following people.
166. [Mark Beckwith](https://github.com/wythe) fixed a typo.
167. [yann-morin-1998](https://github.com/yann-morin-1998) helped to reduce the CMake requirement to version 3.1.
168. [Konstantin Podsvirov](https://github.com/podsvirov) maintains a package for the MSYS2 software distro.
169. [remyabel](https://github.com/remyabel) added GNUInstallDirs to the CMake files.
169. [remyabel](https://github.com/remyabel2) added GNUInstallDirs to the CMake files.
170. [Taylor Howard](https://github.com/taylorhoward92) fixed a unit test.
171. [Gabe Ron](https://github.com/Macr0Nerd) implemented the `to_string` method.
172. [Watal M. Iwasaki](https://github.com/heavywatal) fixed a Clang warning.
@@ -1608,7 +1608,7 @@ I deeply appreciate the help of the following people.
193. [Hubert Chathi](https://github.com/uhoreg) made CMake's version config file architecture-independent.
194. [OmnipotentEntity](https://github.com/OmnipotentEntity) implemented the binary values for CBOR, MessagePack, BSON, and UBJSON.
195. [ArtemSarmini](https://github.com/ArtemSarmini) fixed a compilation issue with GCC 10 and fixed a leak.
196. [Evgenii Sopov](https://github.com/sea-kg) integrated the library to the wsjcpp package manager.
196. [Evgenii Sopov](https://github.com/sea5kg) integrated the library to the wsjcpp package manager.
197. [Sergey Linev](https://github.com/linev) fixed a compiler warning.
198. [Miguel Magalhães](https://github.com/magamig) fixed the year in the copyright.
199. [Gareth Sylvester-Bradley](https://github.com/garethsb-sony) fixed a compilation issue with MSVC.
@@ -1702,7 +1702,7 @@ I deeply appreciate the help of the following people.
287. [NN](https://github.com/NN---) added the Visual Studio output directory to `.gitignore`.
288. [Romain Reignier](https://github.com/romainreignier) improved the performance of the vector output adapter.
289. [Mike](https://github.com/Mike-Leo-Smith) fixed the `std::iterator_traits`.
290. [Richard Hozák](https://github.com/zxey) added macro `JSON_NO_ENUM` to disable default enum conversions.
290. [Richard Hozák](https://github.com/richardhozak) added macro `JSON_NO_ENUM` to disable default enum conversions.
291. [vakokako](https://github.com/vakokako) fixed tests when compiling with C++20.
292. [Alexander “weej” Jones](https://github.com/alexweej) fixed an example in the README.
293. [Eli Schwartz](https://github.com/eli-schwartz) added more files to the `include.zip` archive.
@@ -1727,7 +1727,7 @@ I deeply appreciate the help of the following people.
312. [Gareth Sylvester-Bradley](https://github.com/garethsb) added `operator/=` and `operator/` to construct JSON pointers.
313. [Michael Macnair](https://github.com/mykter) added support for afl-fuzz testing.
314. [Berkus Decker](https://github.com/berkus) fixed a typo in the README.
315. [Illia Polishchuk](https://github.com/effolkronium) improved the CMake testing.
315. [Illia Polishchuk](https://github.com/ilqvya) improved the CMake testing.
316. [Ikko Ashimine](https://github.com/eltociear) fixed a typo.
317. [Raphael Grimm](https://github.com/barcode) added the possibility to define a custom base class.
318. [tocic](https://github.com/tocic) fixed typos in the documentation.
@@ -1797,6 +1797,66 @@ I deeply appreciate the help of the following people.
382. [bitFiedler](https://github.com/bitFiedler) made GDB pretty printer work with Python 3.8.
383. [Gianfranco Costamagna](https://github.com/LocutusOfBorg) fixed a compiler warning.
384. [risa2000](https://github.com/risa2000) made `std::filesystem::path` conversion to/from UTF-8 encoded string explicit.
385. [AM](https://github.com/maqnouch) fixed typos in the README.
386. [dmenendez-gruposantander](https://github.com/dmenendez-gruposantander) fixed typos in the comments of the examples.
387. [Mihai Stan](https://github.com/mstan-xx) fixed comparisons against the literal `0`.
388. [Matt Gumbel](https://github.com/intelmatt) fixed some `-Weffc++` warnings.
389. [vimpunk](https://github.com/vimpunk) moved a lambda out of an unevaluated context to support older compilers.
390. [Chris Harris](https://github.com/cjh1) fixed the compilation with GCC 4.8.
391. [Palmer Dabbelt](https://github.com/palmer-dabbelt) generated and installed a pkg-config file.
392. [Gus Pozuelo](https://github.com/ap-viavi) made `ordered_map` compatible with GCC 5.5, Clang 3.6, and Xcode 9.
393. [AK](https://github.com/Lioncky) fixed an MSVC build error caused by the `min`/`max` macros from `windows.h`.
394. [Sergiu Deitsch](https://github.com/sergiud) provided a fallback for missing `char8_t` support.
395. [Xiaochuan Ye](https://github.com/XueSongTap) fixed `from_msgpack` for `std::byte` input by specializing `std::char_traits`.
396. [Ville Vesilehto](https://github.com/thevilledev) fixed an overflow in the BJData size calculation and rejected overflowing negative integers in CBOR.
397. [NmPassTHFan](https://github.com/nmpassthf) replaced the deprecated `std::is_trivial` for C++26.
398. [Chris Ever](https://github.com/chirsz-ever) added the `ignore_trailing_commas` parser option.
399. [Kuan-Fu Wu](https://github.com/kfwu1999) fixed the example code for `json_pointer` initialization.
400. [David Kilzer](https://github.com/ddkilzer) added a missing header to the input adapters.
401. [Miko](https://github.com/mikomikotaishi) added proper C++20 module support, simplified the module API, and fixed missing exports.
402. [hitgirl](https://github.com/hitgil) fixed the CMake configuration when cross-compiling.
403. [Devon Thomas](https://github.com/ThomaDevOSU) mentioned the Artistic Style formatting in the contribution guidelines.
404. [Erik Hu](https://github.com/Erikhu1) made Coveralls upload errors non-fatal in the CI.
405. [co63oc](https://github.com/co63oc) fixed typos.
406. [DmitriBogdanov](https://github.com/DmitriBogdanov) fixed broken package manager links in the documentation.
407. [Bander](https://github.com/banderzhm) improved the MSVC compatibility of the C++ modules.
408. [Andy Choi](https://github.com/ccpong) removed an unnecessary `template` keyword before `get` in the README and the documentation.
409. [SamareshSingh](https://github.com/ssam18) fixed single-element brace initialization to copy/move instead of wrapping in an array, fixed the `WITH_DEFAULT` macros for `ordered_map`, and handled moved events in `serve_header.py`.
410. [Aditya](https://github.com/Lumowhisp) improved the documentation of the documentation generation.
411. [cheese1](https://github.com/cheese1) clarified the README.
412. [KhloodElhossiny](https://github.com/khloodelhossiny) enabled `std::string_view` keys in `operator[]`.
413. [Charles Cabergs](https://github.com/cacharle) fixed a `-Wtautological-constant-out-of-range-compare` warning.
414. [EALePain](https://github.com/EALePain) made the `std::tuple` conversion work with reference types such as `std::tie`.
415. [koala_oishi](https://github.com/chibi-dogs) fixed grammatical wording in the README.
416. [riccardoori11](https://github.com/riccardoori11) fixed a typo in the documentation.
417. [Swastik Bose](https://github.com/VasuBhakt) fixed the parent pointers after `update()` with `JSON_DIAGNOSTICS` and fixed the Doxygen autolinking of requirements.
418. [trdesilva](https://github.com/trdesilva) added `front`, `pop_front`, and `push_front` to `json_pointer`.
419. [Akhilesh Arora](https://github.com/akhilesharora) fixed an incomplete-type error with `ordered_json`.
420. [Hariom Phulre](https://github.com/hariomphulre) fixed the C++20 modules compilation with GCC.
421. [Kirill Lokotkov](https://github.com/RUSLoker) fixed printing `long double` values.
422. [George Sedov](https://github.com/radistmorse) added the `NLOHMANN_DEFINE_TYPE_*_WITH_NAMES` macros.
423. [Caillin Nugent](https://github.com/nugentcaillin) added the `NLOHMANN_JSON_SERIALIZE_ENUM_STRICT` macro.
424. [Cosmin D.](https://github.com/drcosmin) fixed `std::filesystem::path` conversions and added an MSVC workaround for `std::unique_ptr`.
425. [Paul Dreik](https://github.com/pauldreik) fixed a test relying on implementation-specific behavior.
426. [Daniel Falk](https://github.com/daniel-falk) added missing copyright notices to the SBOM.
427. [Federico Sfriso](https://github.com/federicosfriso05-dotcom) added support for constructing JSON values from C++20 range views.
428. [Luke Banicevic](https://github.com/banaboi) fixed corrupt BSON output for lengths exceeding `INT32_MAX`, cleaned up the BSON writer, and improved the documentation.
429. [Patrick Armstrong](https://github.com/Patrick10199) updated the CBOR references and the half-precision float assertions.
430. [Yash Bavadiya](https://github.com/xevrion) added checks to all BSON reads.
431. [hum4nBeing](https://github.com/hum4nBeing) fixed the overflow handling of high-precision numbers in UBJSON.
432. [tomatotomata](https://github.com/tomatotomata) added checks for reading CBOR tagged subtypes.
433. [YingqiDuan](https://github.com/YingqiDuan) documented the BSON interoperability.
434. [KBS](https://github.com/youdie006) documented the standards compliance and the strictness of `parse()` and `operator>>`.
435. [Petr Bělohlávek](https://github.com/petrbel) added Clang 21 and 22 to the CI.
436. [Dmitry Rantovov](https://github.com/darkdi) fixed the placement of a CBOR documentation block.
437. [ljcjclljc](https://github.com/ljcjclljc) fixed the comparison of large unsigned integers with signed integers.
438. [Sahil Kamate](https://github.com/sahilkamate03) fixed the handling of CBOR tags 0-5 and 21-23.
439. [Krishnanand G](https://github.com/Krishnanand-G) made the UBJSON writer reject `use_type` without `use_size`.
440. [whn](https://github.com/Whning0513) documented the lenient BSON input handling and corrected the complexity of `to_bson`.
441. [elix3r](https://github.com/22elix3r) fixed `update()` with `merge_objects` when merging a primitive into an object.
442. [Avionic Harshit](https://github.com/avionicharshit-byte) made `diff()` linear when an array shrinks.
443. [Qatadaha Bin Matloob](https://github.com/qatcod) fixed comparisons between integers and floats and fixed unparsable BJData output.
444. [Wu Shuwen](https://github.com/dajiaohuang) removed an unused include.
Thanks a lot for helping out! Please [let me know](mailto:mail@nlohmann.me) if I forgot someone.
+52 -1
View File
@@ -245,6 +245,23 @@ add_custom_target(ci_test_brace_init_copy_semantics
COMMENT "Compile and test with brace-init copy semantics enabled"
)
###############################################################################
# Enable strict NUL-byte handling.
###############################################################################
add_custom_target(ci_test_strict_nul_handling
COMMAND ${CMAKE_COMMAND}
-DCMAKE_BUILD_TYPE=Debug -GNinja
-DJSON_BuildTests=ON -DJSON_FastTests=ON -DJSON_StrictNulHandling=ON
-S${PROJECT_SOURCE_DIR} -B${PROJECT_BINARY_DIR}/build_strict_nul_handling
COMMAND ${CMAKE_COMMAND} --build ${PROJECT_BINARY_DIR}/build_strict_nul_handling
# unit-testsuites contains a fixture (a "1e308" test value) that relies on the
# legacy NUL-as-end-of-input behavior this macro disables; exclude it here, as
# it is expected to fail under strict NUL handling and is out of scope for it
COMMAND cd ${PROJECT_BINARY_DIR}/build_strict_nul_handling && ${CMAKE_CTEST_COMMAND} --parallel ${N} --output-on-failure -E "test-testsuites"
COMMENT "Compile and test with strict NUL-byte handling enabled"
)
###############################################################################
# Disable global UDLs.
###############################################################################
@@ -260,6 +277,40 @@ add_custom_target(ci_test_noglobaludls
COMMENT "Compile and test with global UDLs disabled"
)
###############################################################################
# Disable enum serialization.
###############################################################################
add_custom_target(ci_test_disableenumserialization
COMMAND ${CMAKE_COMMAND}
-DCMAKE_BUILD_TYPE=Debug -GNinja
-DJSON_BuildTests=ON -DJSON_FastTests=ON -DJSON_DisableEnumSerialization=ON
-S${PROJECT_SOURCE_DIR} -B${PROJECT_BINARY_DIR}/build_disableenumserialization
COMMAND ${CMAKE_COMMAND} --build ${PROJECT_BINARY_DIR}/build_disableenumserialization
COMMAND cd ${PROJECT_BINARY_DIR}/build_disableenumserialization && ${CMAKE_CTEST_COMMAND} --parallel ${N} --output-on-failure
COMMENT "Compile and test with enum serialization disabled"
)
###############################################################################
# Skip the multiple-inclusion library version check.
###############################################################################
# tests/src/skip_library_version_check.cpp deliberately simulates a scenario
# (mixing two differently-versioned inclusions of the library in one
# translation unit) that unavoidably triggers the compiler's own "macro
# redefined" warning, so -- unlike the ci_test_* targets above -- it is
# compiled directly here, with a modest warning set, instead of being folded
# into the library's own -Weverything/-Werror unit test matrix.
add_custom_target(ci_test_skiplibraryversioncheck
COMMAND ${CMAKE_COMMAND} -E make_directory ${PROJECT_BINARY_DIR}/skip_library_version_check
COMMAND ${CMAKE_CXX_COMPILER} -std=c++11 -Wall -Wextra
-I${PROJECT_SOURCE_DIR}/include
${PROJECT_SOURCE_DIR}/tests/src/skip_library_version_check.cpp
-o ${PROJECT_BINARY_DIR}/skip_library_version_check/skip_library_version_check
COMMAND ${PROJECT_BINARY_DIR}/skip_library_version_check/skip_library_version_check
COMMENT "Compile and run a translation unit simulating a mismatched library version, with JSON_SKIP_LIBRARY_VERSION_CHECK defined"
)
###############################################################################
# Coverage.
###############################################################################
@@ -564,7 +615,7 @@ add_custom_target(ci_single_binaries
add_custom_target(ci_benchmarks
COMMAND ${CMAKE_COMMAND}
-DCMAKE_BUILD_TYPE=Release -GNinja
-S${PROJECT_SOURCE_DIR}/benchmarks -B${PROJECT_BINARY_DIR}/build_benchmarks
-S${PROJECT_SOURCE_DIR}/tests/benchmarks -B${PROJECT_BINARY_DIR}/build_benchmarks
COMMAND ${CMAKE_COMMAND} --build ${PROJECT_BINARY_DIR}/build_benchmarks --target json_benchmarks
COMMAND cd ${PROJECT_BINARY_DIR}/build_benchmarks && ./json_benchmarks
COMMENT "Run benchmarks"
+1 -1
View File
@@ -77,7 +77,7 @@ if(CMAKE_CROSSCOMPILING)
endif()
if(NOT DEFINED LIBCPP_VERSION_OUTPUT_CACHED)
try_run(RUN_RESULT_VAR COMPILE_RESULT_VAR
"${CMAKE_BINARY_DIR}" SOURCES "${CMAKE_SOURCE_DIR}/cmake/detect_libcpp_version.cpp"
"${CMAKE_BINARY_DIR}" SOURCES "${CMAKE_CURRENT_LIST_DIR}/detect_libcpp_version.cpp"
RUN_OUTPUT_VARIABLE LIBCPP_VERSION_OUTPUT
COMPILE_OUTPUT_VARIABLE LIBCPP_VERSION_COMPILE_OUTPUT
)
@@ -90,6 +90,10 @@ Linear in the length of the input. The parser is a predictive LL(1) parser.
A UTF-8 byte order mark is silently ignored.
By default, a `'\0'` (NUL) byte anywhere in the input is treated as end of input, rather than as an ordinary (and,
outside of a string, invalid) byte; see the [FAQ entry](../../home/faq.md#nul-bytes-in-the-input) for details and the
[`JSON_STRICT_NUL_HANDLING`](../macros/json_strict_nul_handling.md) macro to opt into rejecting it instead.
## Examples
??? example
@@ -111,6 +115,8 @@ A UTF-8 byte order mark is silently ignored.
- [parse](parse.md) - deserialize from a compatible input
- [sax_parse](sax_parse.md) - parse input using the SAX interface
- [operator>>](../operator_gtgt.md) - deserialize from stream
- [`JSON_STRICT_NUL_HANDLING`](../macros/json_strict_nul_handling.md) - opt in to rejecting a NUL byte in the input
instead of treating it as end of input
## Version history
@@ -120,6 +126,8 @@ A UTF-8 byte order mark is silently ignored.
- Added `ignore_trailing_commas` in version 3.13.0.
- Extended container support (1) to include types with lvalue-only ADL `begin`/`end` (matching `std::begin`/`std::end` semantics) in version 3.13.0.
- Extended overload (2) to accept heterogeneous iterator+sentinel pairs (C++20 ranges support) in version 3.13.0.
- `JSON_STRICT_NUL_HANDLING` added in version 3.13.0 to optionally reject a NUL byte in the input instead of treating
it as end of input; planned to become the default in version 4.0.0.
!!! warning "Deprecation"
+6 -1
View File
@@ -14,7 +14,11 @@ To store objects in C++, a type is defined by the template parameters explained
## Template parameters
`ArrayType`
: container type to store arrays (e.g., `std::vector` or `std::list`)
: container type to store arrays. It must be a vector-like container: the library uses `operator[]`, `at()`, and
`resize()`, and requires random-access iterators. `#!cpp std::vector` and `#!cpp std::deque` qualify;
`#!cpp std::list` does not. See
[Template Parameter Requirements](../../features/types/template_parameters.md#arraytype) for the full list of
requirements.
`AllocatorType`
: the allocator to use for objects (e.g., `std::allocator`)
@@ -66,3 +70,4 @@ Arrays are stored as pointers in a `basic_json` type. That is, for any access to
## Version history
- Added in version 1.0.0.
- Made `capacity()` optional, so that array types such as `#!cpp std::deque` can be used, in version 3.13.0.
+11 -1
View File
@@ -42,7 +42,9 @@ represent a byte array in modern C++.
`value_type` must additionally be exactly one byte wide (e.g., `std::uint8_t`/`char`/`std::byte`): the binary
serializers (CBOR, MessagePack, BSON, UBJSON) read and write the container's raw bytes via
`reinterpret_cast`, which is only correct for byte-sized elements -- a container like
`#!cpp std::vector<std::intptr_t>` will not work as `BinaryType`.
`#!cpp std::vector<std::intptr_t>` will not work as `BinaryType`. The elements must be stored contiguously, and
the binary readers additionally require `resize()` and `operator[]`. See
[Template Parameter Requirements](../../features/types/template_parameters.md#binarytype) for the full list.
## Notes
@@ -50,6 +52,11 @@ represent a byte array in modern C++.
The default values for `BinaryType` is `#!cpp std::vector<std::uint8_t>`.
#### Supported byte types
`#!cpp std::vector<std::uint8_t>`, `#!cpp std::vector<char>`, and `#!cpp std::vector<std::byte>` are supported.
Regardless of which of them is configured, [`dump`](dump.md) writes the bytes as the numbers 0..255.
#### Custom BinaryType behavior
When a custom `BinaryType` is configured (other than the default `#!cpp std::vector<std::uint8_t>`), you can assign
@@ -126,3 +133,6 @@ type `#!cpp binary_t*` must be dereferenced.
## Version history
- Added in version 3.8.0. Changed the type of subtype to `std::uint64_t` in version 3.10.0.
- Fixed [`dump`](dump.md), [`std::hash`](std_hash.md), and [`to_ubjson`](to_ubjson.md) for byte types that are not
integers (e.g., `#!cpp std::byte`) in version 3.13.0. `dump` now writes the bytes of a signed byte type (e.g.,
`#!cpp char`) as 0..255 rather than as negative numbers.
@@ -11,6 +11,14 @@ literals `#!json true` and `#!json false`.
To store boolean values in C++, a type is defined by the template parameter `BooleanType` which chooses the type to use.
## Template parameters
`BooleanType`
: the type to store booleans. As it is stored directly inside a `basic_json` value (in a union), it must be a
trivially default-constructible, trivially copyable, and trivially destructible type that is convertible to and
from `#!cpp bool`. See
[Template Parameter Requirements](../../features/types/template_parameters.md#booleantype).
## Notes
#### Default type
+4
View File
@@ -35,6 +35,10 @@ class basic_json;
| `BinaryType` | type for binary arrays | [`binary_t`](binary_t.md) |
| `CustomBaseClass` | extension point for user code | [`json_base_class_t`](json_base_class_t.md) |
The library imposes a number of requirements on these types that are not expressed as C++ concepts, such as the
container operations `object_t` and `array_t` must provide, or the fact that `StringType` must be `char`-based. They
are collected in [Template Parameter Requirements](../../features/types/template_parameters.md).
## Specializations
- [**json**](../json.md) - default specialization
@@ -88,6 +88,8 @@ Strong exception safety: if an exception occurs, the original value stays intact
do not belong to the same JSON value; example: `"iterators do not fit"`
- Throws [`invalid_iterator.211`](../../home/exceptions.md#jsonexceptioninvalid_iterator211) if `first` or `last`
are iterators into container for which insert is called; example: `"passed iterators may not belong to container"`
- Throws [`invalid_iterator.202`](../../home/exceptions.md#jsonexceptioninvalid_iterator202) if `first` or `last`
do not point to an array; example: `"iterators first and last must point to arrays"`
4. The function can throw the following exceptions:
- Throws [`type_error.309`](../../home/exceptions.md#jsonexceptiontype_error309) if called on JSON values other than
arrays; example: `"cannot use insert() with string"`
@@ -21,8 +21,11 @@ The default value for `CustomBaseClass` is `void`. In this case, an
#### Limitations
The type `CustomBaseClass` has to be a default-constructible class.
The type `CustomBaseClass` has to be a default-constructible, non-`final` class.
`basic_json` only supports copy/move construction/assignment if `CustomBaseClass` does so as well.
A `CustomBaseClass` with non-static data members forfeits `basic_json`'s
[standard layout](https://en.cppreference.com/w/cpp/named_req/StandardLayoutType) guarantee. See
[Template Parameter Requirements](../../features/types/template_parameters.md#custombaseclass).
## Examples
@@ -19,6 +19,12 @@ using json_serializer = JSONSerializer<T, SFINAE>;
The default values for `json_serializer` is [`adl_serializer`](../adl_serializer/index.md).
#### Requirements
A custom serializer must provide `#!cpp static void to_json(basic_json&, T)` for every type it serializes, and either
`#!cpp static void from_json(const basic_json&, T&)` or `#!cpp static T from_json(const basic_json&)` for every type it
deserializes. See [Template Parameter Requirements](../../features/types/template_parameters.md#jsonserializer).
## Examples
??? example
@@ -20,6 +20,16 @@ used.
To store floating-point numbers in C++, a type is defined by the template parameter `NumberFloatType` which chooses the
type to use.
## Template parameters
`NumberFloatType`
: the type to store floating-point numbers. Parsing and serialization are implemented in terms of
`#!cpp std::strtof`/`#!cpp std::strtod`/`#!cpp std::strtold` and `#!cpp std::snprintf`, so the type must be
`#!cpp float`, `#!cpp double`, or `#!cpp long double`. The
[binary formats](../../features/binary_formats/index.md) additionally require `#!cpp float` or `#!cpp double`,
because they have no encoding for `#!cpp long double`. See
[Template Parameter Requirements](../../features/types/template_parameters.md#numberfloattype).
## Notes
#### Default type
@@ -20,6 +20,13 @@ used.
To store integer numbers in C++, a type is defined by the template parameter `NumberIntegerType` which chooses the type
to use.
## Template parameters
`NumberIntegerType`
: the type to store signed integers. It must be a **signed integral** type (`#!cpp std::is_integral`) with a
`#!cpp std::numeric_limits` specialization, and it is stored directly inside a `basic_json` value. See
[Template Parameter Requirements](../../features/types/template_parameters.md#numberintegertype-and-numberunsignedtype).
## Notes
#### Default type
@@ -20,6 +20,14 @@ used.
To store unsigned integer numbers in C++, a type is defined by the template parameter `NumberUnsignedType` which chooses
the type to use.
## Template parameters
`NumberUnsignedType`
: the type to store unsigned integers. It must be an **unsigned integral** type (`#!cpp std::is_integral`) with a
`#!cpp std::numeric_limits` specialization, and it must be able to represent the absolute value of every
[`number_integer_t`](number_integer_t.md) value. See
[Template Parameter Requirements](../../features/types/template_parameters.md#numberintegertype-and-numberunsignedtype).
## Notes
#### Default type
@@ -30,3 +30,5 @@ and [`default_object_comparator_t`](default_object_comparator_t.md) otherwise.
- Added in version 3.0.0.
- Changed to be conditionally defined as `#!cpp typename object_t::key_compare` or `default_object_comparator_t` in
version 3.11.0.
- Fixed the fallback to `default_object_comparator_t`, which previously failed to compile for object types without a
`key_compare` member type, in version 3.13.0.
+6 -1
View File
@@ -18,7 +18,11 @@ To store objects in C++, a type is defined by the template parameters described
## Template parameters
`ObjectType`
: the container to store objects (e.g., `std::map` or `std::unordered_map`)
: the container to store objects. Its template parameters must have the same order and meaning as those of
`std::map`; in particular, the third parameter is a comparator. `#!cpp std::unordered_map`, whose third parameter
is a hash function, therefore needs an adapter -- see
[Template Parameter Requirements](../../features/types/template_parameters.md#objecttype) for the full list of
requirements, an adapter example, and the containers that are known to work.
`StringType`
: the type of the keys or names (e.g., `std::string`). The comparison function `std::less<StringType>` is used to
@@ -122,3 +126,4 @@ the object is silently converted as an array of key-value pairs, which is incorr
## Version history
- Added in version 1.0.0.
- Allowed object types whose `erase(iterator)` returns `#!cpp void` in version 3.13.0.
+8
View File
@@ -103,6 +103,10 @@ A UTF-8 byte order mark is silently ignored.
Invalid Unicode escapes and unpaired surrogates in the input are reported as
[`parse_error.101`](../../home/exceptions.md#jsonexceptionparse_error101) with a detailed message.
By default, a `'\0'` (NUL) byte anywhere in the input is treated as end of input, rather than as an ordinary (and,
outside of a string, invalid) byte; see the [FAQ entry](../../home/faq.md#nul-bytes-in-the-input) for details and the
[`JSON_STRICT_NUL_HANDLING`](../macros/json_strict_nul_handling.md) macro to opt into rejecting it instead.
## Examples
??? example "Parsing from a character array"
@@ -236,6 +240,8 @@ Invalid Unicode escapes and unpaired surrogates in the input are reported as
- [accept](accept.md) - check if the input is valid JSON
- [sax_parse](sax_parse.md) - parse input using the SAX interface
- [operator>>](../operator_gtgt.md) - deserialize from stream
- [`JSON_STRICT_NUL_HANDLING`](../macros/json_strict_nul_handling.md) - opt in to rejecting a NUL byte in the input
instead of treating it as end of input
## Version history
@@ -246,6 +252,8 @@ Invalid Unicode escapes and unpaired surrogates in the input are reported as
- Added `ignore_trailing_commas` in version 3.13.0.
- Extended container support (1) to include types with lvalue-only ADL `begin`/`end` (matching `std::begin`/`std::end` semantics) in version 3.13.0.
- Extended overload (2) to accept heterogeneous iterator+sentinel pairs (C++20 ranges support) in version 3.13.0.
- `JSON_STRICT_NUL_HANDLING` added in version 3.13.0 to optionally reject a NUL byte in the input instead of treating
it as end of input; planned to become the default in version 4.0.0.
!!! warning "Deprecation"
@@ -23,6 +23,11 @@ JSON class into byte-sized characters during deserialization.
`StringType`. To work with wide-character data, convert it to/from UTF-8 at the boundary instead -- see the
FAQ's [wide string handling](../../home/faq.md#wide-string-handling) section for a conversion recipe.
Beyond the character type, the library expects a substantial part of the `#!cpp std::string` interface (contiguous
null-terminated `data()`, `substr()`, `find()`, `append()`, ...). See
[Template Parameter Requirements](../../features/types/template_parameters.md#stringtype) for the full list and
for the string types that are known to work.
## Notes
#### Default type
@@ -78,3 +83,5 @@ and an example.
## Version history
- Added in version 1.0.0.
- Removed the requirement that `string_t` be implicitly convertible from `#!cpp std::string`, which the BSON writer and
the UBJSON reader relied on, in version 3.13.0.
+9 -1
View File
@@ -37,7 +37,14 @@ Linear in the size of the JSON value.
## Notes
Empty objects and arrays are flattened by [`flatten()`](flatten.md) to `#!json null` values and cannot unflattened to
their original type. Apart from this example, for a JSON value `j`, the following is always true:
their original type.
A flattened array and a flattened object whose keys are array indices are indistinguishable, because both are
described by the same JSON pointers. A value is therefore restored as an array if and only if one of its keys is the
reference token `0`, and as an object otherwise: `#!json {"2": 1}` is restored unchanged, whereas `#!json {"0": 1}` is
restored as `#!json [1]`. This decision does not depend on the order in which the flattened object is iterated.
Apart from these two cases, for a JSON value `j`, the following is always true:
`#!cpp j == j.flatten().unflatten()`.
## Examples
@@ -63,3 +70,4 @@ their original type. Apart from this example, for a JSON value `j`, the followin
## Version history
- Added in version 2.0.0.
- Made the array/object decision independent of the object's iteration order in version 3.13.0.
+5
View File
@@ -14,6 +14,11 @@ header. See also the [macro overview page](../../features/macros.md).
- [**JSON_DIAGNOSTIC_POSITIONS**](json_diagnostic_positions.md) - access positions of elements
- [**JSON_NOEXCEPTION**](json_noexception.md) - switch off exceptions
## Parsing
- [**JSON_STRICT_NUL_HANDLING**](json_strict_nul_handling.md) - opt in to rejecting a NUL byte in the input instead of
treating it as end of input
## Language support
- [**JSON_HAS_CPP_11**<br>**JSON_HAS_CPP_14**<br>**JSON_HAS_CPP_17**<br>**JSON_HAS_CPP_20**](json_has_cpp_11.md) - set supported C++ standard
@@ -0,0 +1,126 @@
# JSON_STRICT_NUL_HANDLING
```cpp
#define JSON_STRICT_NUL_HANDLING /* value */
```
When defined to `1`, a `'\0'` (NUL) byte in JSON text input is rejected with `parse_error.101`, like any other
unexpected byte, instead of being silently treated as end of input.
The macro only affects the JSON text parser ([`parse`](../basic_json/parse.md), [`accept`](../basic_json/accept.md),
[`sax_parse`](../basic_json/sax_parse.md), and [`operator>>`](../operator_gtgt.md)). There are three cases where a NUL
byte is still not rejected:
- The binary formats ([`from_bjdata`](../basic_json/from_bjdata.md), [`from_bson`](../basic_json/from_bson.md),
[`from_cbor`](../basic_json/from_cbor.md), [`from_msgpack`](../basic_json/from_msgpack.md),
[`from_ubjson`](../basic_json/from_ubjson.md)) are never affected: there, `0x00` is ordinary data.
- A bare `const char*` pointer has no length of its own, so its length is still determined with `strlen()`. The first
NUL byte therefore still marks the end of the input, and nothing after it is read.
- One trailing `'\0'` at the end of a `char` array (e.g., a string literal) is trimmed; see the warning below.
## Default definition
The default value is `0` (disabled — existing behavior is preserved).
```cpp
#define JSON_STRICT_NUL_HANDLING 0
```
## Notes
!!! note "Background"
By default, a `'\0'` byte anywhere in the input is treated the same as the real end of the input, rather than as
an ordinary (and, outside of a string, invalid) byte. Everything from that byte onward is silently ignored,
without a parse error - including further, otherwise well-formed JSON:
```cpp
json::parse(std::string("123") + '\0'); // == 123, no error
json::parse(std::string("123") + '\0' + "true"); // == 123, the "true" is silently ignored too
```
This falls out of the same convention used when no explicit input length is given at all: parsing from a
`const char*` already stops at the first NUL byte via `strlen()`, since a bare pointer has no length of its own.
The library applies that same NUL-terminated-C-string convention uniformly, rather than only when a length is
genuinely unavailable - so a `std::string`, iterator range, or container whose content happens to include a NUL
byte is affected the same way a raw `const char*` would be (see the
[FAQ entry](../../home/faq.md#nul-bytes-in-the-input) for a fuller explanation).
This was not fixed unconditionally, because doing so is backwards-incompatible for any caller who happens to
depend on the current behavior - even unknowingly, for instance because their input already contains trailing
padding they never noticed was being discarded (see [#5530](https://github.com/nlohmann/json/issues/5530)).
This macro instead offers an opt-in path to the corrected behavior ahead of version 4.0.0, where it is planned to
become the default.
!!! warning "Opt-in only"
This macro must be defined **before** including `<nlohmann/json.hpp>`. Defining it after the include has no
effect.
Enabling it also changes how a `char` array (including a string literal, e.g. `json::parse("123")`) is read: such
an array normally carries a trailing `'\0'` contributed by the compiler, not by the source text. With this macro
enabled, that one trailing byte is trimmed if present so that parsing a string literal keeps working; every other
byte in the array - including any `'\0'` that is not the very last element - is read as real data and rejected
like any other unexpected byte. Arrays of any other element type (`unsigned char`, `std::uint8_t`, ...), as used
for CBOR or MessagePack, are never affected by this trimming; their full extent - including a genuine trailing
`0x00` - is always preserved, in both states of this macro.
!!! tip "Workaround without the macro"
To reject a NUL byte without enabling this macro, trim your input yourself before calling `parse()`:
```cpp
s.resize(s.find('\0')); // drop everything from the first NUL onward, if any
json::parse(s);
```
## Examples
??? example "Default behavior (macro not defined)"
Without the macro, a NUL byte silently ends parsing at that point:
```cpp
#include <nlohmann/json.hpp>
using json = nlohmann::json;
int main()
{
json j = json::parse(std::string("123") + '\0' + "true");
// j is 123 -- the '\0' and everything after it is silently ignored
}
```
??? example "Opt-in strict handling (macro defined to 1)"
With the macro, a NUL byte is rejected like any other unexpected byte:
```cpp
#define JSON_STRICT_NUL_HANDLING 1
#include <nlohmann/json.hpp>
using json = nlohmann::json;
int main()
{
json j = json::parse(std::string("123") + '\0' + "true");
// throws parse_error.101 -- the NUL byte is now invalid input,
// exactly like any other unexpected trailing byte
json ok = json::parse("123");
// ok is 123 -- parsing from a string literal still works
}
```
## See also
- [FAQ: NUL bytes in the input](../../home/faq.md#nul-bytes-in-the-input)
- [**parse**](../basic_json/parse.md) - deserialize from a compatible input
- [**accept**](../basic_json/accept.md) - check if the input is valid JSON
- [**operator>>**](../operator_gtgt.md) - deserialize from stream
## Version history
- Added in version 3.13.0.
- Planned to become the default (with the macro removed) in version 4.0.0.
@@ -12,9 +12,11 @@
Controls how exceptions are handled by the library.
1. This macro overrides [`#!cpp catch`](https://en.cppreference.com/w/cpp/language/try_catch) calls inside the library.
The argument is the type of the exception to catch. As of version 3.8.0, the library only catches `std::out_of_range`
exceptions internally to rethrow them as [`json::out_of_range`](../../home/exceptions.md#out-of-range) exceptions.
The macro is always followed by a scope.
The argument is the type of the exception to catch. The library uses it in a single place: to swallow any exception
escaping the parent-pointer check that [`JSON_DIAGNOSTICS`](json_diagnostics.md) adds to the class invariant. The
places where the library catches its own [`json::out_of_range`](../../home/exceptions.md#out-of-range) exceptions
use `JSON_INTERNAL_CATCH` instead, which `JSON_CATCH_USER` also overrides unless `JSON_INTERNAL_CATCH_USER` is
defined. The macro is always followed by a scope.
2. This macro overrides `#!cpp throw` calls inside the library. The argument is the exception to be thrown. Note that
`JSON_THROW_USER` should leave the current scope (e.g., by throwing or aborting), as continuing after it may yield
undefined behavior.
+11
View File
@@ -72,6 +72,13 @@ input >> j2; // parses the next value
Note that reading concatenated values does **not** work for [JSON Lines](../features/parsing/json_lines.md)
(newline-delimited JSON) input -- see that page for why and for the recommended alternative.
By default, a `'\0'` (NUL) byte encountered while reading a value is treated as end of input, rather than as an
ordinary (and, outside of a string, invalid) byte; see the [FAQ entry](../home/faq.md#nul-bytes-in-the-input) for
details and the [`JSON_STRICT_NUL_HANDLING`](macros/json_strict_nul_handling.md) macro to opt into rejecting it
instead. Because `operator>>` only parses a single value and does not require the rest of the stream to be consumed,
a NUL byte *after* a complete value has no effect on `operator>>` either way; it only matters while a value is still
being read.
!!! warning "Deprecation"
This function replaces function `#!cpp std::istream& operator<<(basic_json& j, std::istream& i)` which has
@@ -98,7 +105,11 @@ Note that reading concatenated values does **not** work for [JSON Lines](../feat
- [accept](basic_json/accept.md) - check if the input is valid JSON
- [parse](basic_json/parse.md) - deserialize from a compatible input
- [`JSON_STRICT_NUL_HANDLING`](macros/json_strict_nul_handling.md) - opt in to rejecting a NUL byte in the input
instead of treating it as end of input
## Version history
- Added in version 1.0.0.
- `JSON_STRICT_NUL_HANDLING` added in version 3.13.0 to optionally reject a NUL byte in the input instead of treating
it as end of input; planned to become the default in version 4.0.0.
@@ -0,0 +1,19 @@
#include <iostream>
#include <map>
#include <nlohmann/json.hpp>
#include "custom_array_type.hpp"
using custom_json = nlohmann::basic_json<std::map, custom_array_type>;
int main()
{
custom_json j = custom_json::array();
j.push_back(1);
j.push_back(2);
j.push_back(3);
std::cout << j.dump() << std::endl;
std::cout << std::boolalpha << (custom_json::parse(j.dump()) == j) << std::endl;
}
@@ -0,0 +1,152 @@
#pragma once
#include <memory>
#include <utility>
#include <vector>
// A minimal, self-contained ArrayType built around a private std::vector.
// See https://json.nlohmann.me/features/types/template_parameters/#arraytype
template<class T, class Allocator = std::allocator<T>>
class custom_array_type
{
using vector_t = std::vector<T, Allocator>;
vector_t data_;
public:
using value_type = typename vector_t::value_type;
using size_type = typename vector_t::size_type;
using iterator = typename vector_t::iterator;
using const_iterator = typename vector_t::const_iterator;
custom_array_type() = default;
custom_array_type(const custom_array_type&) = default;
custom_array_type(custom_array_type&&) = default;
custom_array_type& operator=(const custom_array_type&) = default;
custom_array_type& operator=(custom_array_type&&) = default;
template<class InputIt>
custom_array_type(InputIt first, InputIt last) : data_(first, last) {}
custom_array_type(size_type count, const T& value) : data_(count, value) {}
iterator begin()
{
return data_.begin();
}
iterator end()
{
return data_.end();
}
const_iterator begin() const
{
return data_.begin();
}
const_iterator end() const
{
return data_.end();
}
const_iterator cbegin() const
{
return data_.cbegin();
}
const_iterator cend() const
{
return data_.cend();
}
bool empty() const
{
return data_.empty();
}
size_type size() const
{
return data_.size();
}
size_type max_size() const
{
return data_.max_size();
}
void clear()
{
data_.clear();
}
void resize(size_type n)
{
data_.resize(n);
}
T& operator[](size_type pos)
{
return data_[pos];
}
const T& operator[](size_type pos) const
{
return data_[pos];
}
T& back()
{
return data_.back();
}
const T& back() const
{
return data_.back();
}
void push_back(const T& value)
{
data_.push_back(value);
}
void push_back(T&& value)
{
data_.push_back(std::move(value));
}
template<class... Args>
void emplace_back(Args&& ... args)
{
data_.emplace_back(std::forward<Args>(args)...);
}
void pop_back()
{
data_.pop_back();
}
iterator insert(const_iterator pos, const T& value)
{
return data_.insert(pos, value);
}
iterator insert(const_iterator pos, size_type count, const T& value)
{
return data_.insert(pos, count, value);
}
template<class InputIt>
iterator insert(const_iterator pos, InputIt first, InputIt last)
{
return data_.insert(pos, first, last);
}
iterator erase(const_iterator pos)
{
return data_.erase(pos);
}
iterator erase(const_iterator first, const_iterator last)
{
return data_.erase(first, last);
}
void swap(custom_array_type& other)
{
data_.swap(other.data_);
}
friend bool operator==(const custom_array_type& lhs, const custom_array_type& rhs)
{
return lhs.data_ == rhs.data_;
}
friend bool operator<(const custom_array_type& lhs, const custom_array_type& rhs)
{
return lhs.data_ < rhs.data_;
}
};
@@ -0,0 +1,2 @@
[1,2,3]
true
@@ -0,0 +1,21 @@
#include <cstdint>
#include <iostream>
#include <map>
#include <string>
#include <vector>
#include <nlohmann/json.hpp>
#include "custom_binary_type.hpp"
using custom_json = nlohmann::basic_json<std::map, std::vector, std::string, bool,
std::int64_t, std::uint64_t, double, std::allocator,
nlohmann::adl_serializer, custom_binary_type>;
int main()
{
const auto j = custom_json::binary({0x01, 0x02, 0x03});
std::cout << j.dump() << std::endl;
std::cout << std::boolalpha << (custom_json::from_cbor(custom_json::to_cbor(j)) == j) << std::endl;
}
@@ -0,0 +1,112 @@
#pragma once
#include <cstdint>
#include <initializer_list>
#include <vector>
// A minimal, self-contained BinaryType built around a private std::vector.
// See https://json.nlohmann.me/features/types/template_parameters/#binarytype
class custom_binary_type
{
using vector_t = std::vector<std::uint8_t>;
vector_t data_;
public:
using value_type = vector_t::value_type;
using size_type = vector_t::size_type;
using iterator = vector_t::iterator;
using const_iterator = vector_t::const_iterator;
custom_binary_type() = default;
custom_binary_type(const custom_binary_type&) = default;
custom_binary_type(custom_binary_type&&) = default;
custom_binary_type& operator=(const custom_binary_type&) = default;
custom_binary_type& operator=(custom_binary_type&&) = default;
template<class InputIt>
custom_binary_type(InputIt first, InputIt last) : data_(first, last) {}
// so basic_json::binary({0x01, 0x02}) can build one directly
custom_binary_type(std::initializer_list<std::uint8_t> init) : data_(init) {}
size_type size() const
{
return data_.size();
}
bool empty() const
{
return data_.empty();
}
void clear()
{
data_.clear();
}
void resize(size_type n)
{
data_.resize(n);
}
// read-only is enough: the writers only ever read from a binary value
const std::uint8_t* data() const
{
return data_.data();
}
std::uint8_t& operator[](size_type pos)
{
return data_[pos];
}
std::uint8_t operator[](size_type pos) const
{
return data_[pos];
}
std::uint8_t& back()
{
return data_.back();
}
std::uint8_t back() const
{
return data_.back();
}
iterator begin()
{
return data_.begin();
}
iterator end()
{
return data_.end();
}
const_iterator begin() const
{
return data_.begin();
}
const_iterator end() const
{
return data_.end();
}
const_iterator cbegin() const
{
return data_.cbegin();
}
const_iterator cend() const
{
return data_.cend();
}
template<class InputIt>
iterator insert(const_iterator pos, InputIt first, InputIt last)
{
return data_.insert(pos, first, last);
}
friend bool operator==(const custom_binary_type& lhs, const custom_binary_type& rhs)
{
return lhs.data_ == rhs.data_;
}
friend bool operator<(const custom_binary_type& lhs, const custom_binary_type& rhs)
{
return lhs.data_ < rhs.data_;
}
};
@@ -0,0 +1,2 @@
{"bytes":[1,2,3],"subtype":null}
true
@@ -0,0 +1,26 @@
#include <iostream>
#include <type_traits>
#include <vector>
#include <nlohmann/json.hpp>
#include "custom_object_type.hpp"
using custom_json = nlohmann::basic_json<custom_object_type, std::vector>;
int main()
{
custom_json j;
j["pi"] = 3.141;
j["happy"] = true;
j["list"] = {1, 2, 3};
std::cout << j.dump(2) << std::endl;
std::cout << std::boolalpha << (custom_json::parse(j.dump()) == j) << std::endl;
// custom_object_type has no key_compare member, so object_comparator_t
// falls back to its default
std::cout << std::boolalpha
<< std::is_same<custom_json::object_comparator_t, custom_json::default_object_comparator_t>::value
<< std::endl;
}
@@ -0,0 +1,144 @@
#pragma once
#include <map>
#include <utility>
// A minimal, self-contained ObjectType built around a private std::map.
// key_compare is deliberately not exposed: when an ObjectType has no
// key_compare member, the library falls back to its own default comparator.
// See https://json.nlohmann.me/features/types/template_parameters/#objecttype
template<class Key, class T, class Compare, class Allocator>
class custom_object_type
{
using map_t = std::map<Key, T, Compare, Allocator>;
map_t data_;
public:
using key_type = typename map_t::key_type;
using mapped_type = typename map_t::mapped_type;
using value_type = typename map_t::value_type;
using size_type = typename map_t::size_type;
using iterator = typename map_t::iterator;
using const_iterator = typename map_t::const_iterator;
custom_object_type() = default;
custom_object_type(const custom_object_type&) = default;
custom_object_type(custom_object_type&&) = default;
custom_object_type& operator=(const custom_object_type&) = default;
custom_object_type& operator=(custom_object_type&&) = default;
template<class InputIt>
custom_object_type(InputIt first, InputIt last) : data_(first, last) {}
iterator begin()
{
return data_.begin();
}
iterator end()
{
return data_.end();
}
const_iterator begin() const
{
return data_.begin();
}
const_iterator end() const
{
return data_.end();
}
const_iterator cbegin() const
{
return data_.cbegin();
}
const_iterator cend() const
{
return data_.cend();
}
bool empty() const
{
return data_.empty();
}
size_type size() const
{
return data_.size();
}
size_type max_size() const
{
return data_.max_size();
}
void clear()
{
data_.clear();
}
iterator find(const key_type& key)
{
return data_.find(key);
}
const_iterator find(const key_type& key) const
{
return data_.find(key);
}
size_type count(const key_type& key) const
{
return data_.count(key);
}
std::pair<iterator, bool> emplace(const key_type& key, const mapped_type& value)
{
return data_.emplace(key, value);
}
std::pair<iterator, bool> insert(const value_type& value)
{
return data_.insert(value);
}
template<class InputIt>
void insert(InputIt first, InputIt last)
{
data_.insert(first, last);
}
mapped_type& operator[](const key_type& key)
{
return data_[key];
}
mapped_type& at(const key_type& key)
{
return data_.at(key);
}
const mapped_type& at(const key_type& key) const
{
return data_.at(key);
}
iterator erase(iterator pos)
{
return data_.erase(pos);
}
iterator erase(iterator first, iterator last)
{
return data_.erase(first, last);
}
size_type erase(const key_type& key)
{
return data_.erase(key);
}
void swap(custom_object_type& other)
{
data_.swap(other.data_);
}
friend bool operator==(const custom_object_type& lhs, const custom_object_type& rhs)
{
return lhs.data_ == rhs.data_;
}
friend bool operator<(const custom_object_type& lhs, const custom_object_type& rhs)
{
return lhs.data_ < rhs.data_;
}
};
@@ -0,0 +1,11 @@
{
"happy": true,
"list": [
1,
2,
3
],
"pi": 3.141
}
true
true
@@ -0,0 +1,20 @@
#include <iostream>
#include <map>
#include <vector>
#include <nlohmann/json.hpp>
#include "custom_string_type.hpp"
using custom_json = nlohmann::basic_json<std::map, std::vector, custom_string_type>;
int main()
{
custom_json j;
j["pi"] = 3.141;
j["happy"] = true;
j["list"] = {1, 2, 3};
std::cout << j.dump(2) << std::endl;
std::cout << std::boolalpha << (custom_json::parse(j.dump()) == j) << std::endl;
}
@@ -0,0 +1,134 @@
#pragma once
#include <ostream>
#include <string>
// A minimal, self-contained StringType built around a private std::string.
// Wraps rather than inherits, so it exposes exactly what the library needs
// and nothing more of std::string's interface.
//
// Covers the "Always required" members, the extras needed for the binary
// formats, and the extras needed for JSON Pointer / flatten / unflatten /
// diff. Extending it further (e.g. for std::hash<basic_json> or to_bson) is
// a matter of adding the extra members listed in the "Required for other
// functionality" table.
//
// See https://json.nlohmann.me/features/types/template_parameters/#stringtype
class custom_string_type
{
std::string data_;
public:
using value_type = char;
using size_type = std::string::size_type;
using iterator = std::string::iterator;
using const_iterator = std::string::const_iterator;
static constexpr size_type npos = std::string::npos;
custom_string_type() = default;
custom_string_type(const custom_string_type&) = default;
custom_string_type(custom_string_type&&) = default;
custom_string_type& operator=(const custom_string_type&) = default;
custom_string_type& operator=(custom_string_type&&) = default;
// not explicit: the library relies on being able to hand it a string literal
custom_string_type(const char* s) : data_(s) {}
custom_string_type(const char* s, size_type count) : data_(s, count) {}
custom_string_type(size_type count, char ch) : data_(count, ch) {}
size_type size() const
{
return data_.size();
}
bool empty() const
{
return data_.empty();
}
void clear()
{
data_.clear();
}
void resize(size_type n)
{
data_.resize(n);
}
void resize(size_type n, char c)
{
data_.resize(n, c);
}
void reserve(size_type n)
{
data_.reserve(n);
}
// must stay null-terminated -- the parser hands this to std::strtoull &
// friends; std::string::data() has guaranteed that since C++11
const char* data() const
{
return data_.data();
}
void push_back(char c)
{
data_.push_back(c);
}
char& operator[](size_type pos)
{
return data_[pos];
}
char operator[](size_type pos) const
{
return data_[pos];
}
custom_string_type& append(const char* s, size_type count)
{
data_.append(s, count);
return *this;
}
custom_string_type& append(const custom_string_type& other)
{
data_.append(other.data_);
return *this;
}
size_type find_first_of(char c, size_type pos = 0) const
{
return data_.find_first_of(c, pos);
}
iterator begin()
{
return data_.begin();
}
iterator end()
{
return data_.end();
}
const_iterator begin() const
{
return data_.begin();
}
const_iterator end() const
{
return data_.end();
}
friend bool operator==(const custom_string_type& lhs, const custom_string_type& rhs)
{
return lhs.data_ == rhs.data_;
}
friend bool operator<(const custom_string_type& lhs, const custom_string_type& rhs)
{
return lhs.data_ < rhs.data_;
}
// not required by the library itself, but dump() returns a custom_string_type
// and this makes `std::cout << j.dump()` work as expected
friend std::ostream& operator<<(std::ostream& os, const custom_string_type& s)
{
return os << s.data_;
}
};
@@ -0,0 +1,10 @@
{
"happy": true,
"list": [
1,
2,
3
],
"pi": 3.141
}
true
+13
View File
@@ -105,6 +105,19 @@ using the library with compilers that do not fully support C++11 and may only wo
See [full documentation of `JSON_SKIP_UNSUPPORTED_COMPILER_CHECK`](../api/macros/json_skip_unsupported_compiler_check.md).
## `JSON_STRICT_NUL_HANDLING`
When defined to `1`, a `'\0'` (NUL) byte anywhere in the input is rejected with `parse_error.101`, like any other
unexpected byte, instead of being silently treated as end of input (see the
[FAQ entry](../home/faq.md#nul-bytes-in-the-input) for background). The default value is `0`, which preserves the
existing behavior; this is planned to become the default in version 4.0.0.
The strict handling can also be enabled with the CMake option
[`JSON_StrictNulHandling`](../integration/cmake.md#json_strictnulhandling) (`OFF` by default) which sets
`JSON_STRICT_NUL_HANDLING` accordingly.
See [full documentation of `JSON_STRICT_NUL_HANDLING`](../api/macros/json_strict_nul_handling.md).
## `JSON_THROW_USER(exception)`
This macro overrides `#!cpp throw` calls inside the library. The argument is the exception to be thrown.
+5 -1
View File
@@ -51,7 +51,11 @@ If you do want to preserve the **insertion order**, you can use the type [`nlohm
--8<-- "examples/ordered_json.output"
```
Alternatively, you can use a more sophisticated ordered map like [`tsl::ordered_map`](https://github.com/Tessil/ordered-map) ([integration](https://github.com/nlohmann/json/issues/546#issuecomment-304447518)) or [`nlohmann::fifo_map`](https://github.com/nlohmann/fifo_map) ([integration](https://github.com/nlohmann/json/issues/485#issuecomment-333652309)).
Alternatively, [`nlohmann::fifo_map`](https://github.com/nlohmann/fifo_map) also preserves the insertion order and, unlike [`ordered_map`](../api/ordered_map.md), keeps a lookup index, so it does not have the quadratic cost described below. It is used through a small adapter ([integration](https://github.com/nlohmann/json/issues/485#issuecomment-333652309)).
If the order does not matter and you only want faster lookup, `boost::unordered_flat_map`, `absl::flat_hash_map`, `absl::node_hash_map`, and several other hash maps work through an adapter that restores the template argument order `basic_json` expects; see [Template Parameter Requirements](types/template_parameters.md#objecttype). Note these are *unordered*, not insertion-ordered.
[`tsl::ordered_map`](https://github.com/Tessil/ordered-map) cannot be used: its iterators expose the mapped value as `const`, while `basic_json` needs to modify it in place.
The [`ordered_map`](../api/ordered_map.md) behind `nlohmann::ordered_json` is deliberately minimal and has no lookup
index, so every key access is a linear scan and building an object of `n` keys costs O(n²). This is unnoticeable at
+6 -1
View File
@@ -79,7 +79,8 @@ template<
class NumberFloatType = double,
template<typename U> class AllocatorType = std::allocator,
template<typename T, typename SFINAE = void> class JSONSerializer = adl_serializer,
class BinaryType = std::vector<std::uint8_t>
class BinaryType = std::vector<std::uint8_t>,
class CustomBaseClass = void
>
class basic_json;
```
@@ -106,6 +107,10 @@ using number_float_t = NumberFloatType;
using binary_t = nlohmann::byte_container_with_subtype<BinaryType>;
```
Not every type can be passed for these template arguments: the library uses the resulting types in ways that imply a
number of requirements, for instance that `StringType` is `char`-based or that `ArrayType` is vector-like. These
requirements are collected in [Template Parameter Requirements](template_parameters.md).
## Objects
@@ -0,0 +1,747 @@
# Template Parameter Requirements
Class [`basic_json`](../../api/basic_json/index.md) is configurable through eleven template parameters. The library
never formally states what a type passed for one of these parameters has to provide -- the requirements are implied by
the way the library uses the resulting [`object_t`](../../api/basic_json/object_t.md),
[`array_t`](../../api/basic_json/array_t.md), [`string_t`](../../api/basic_json/string_t.md), etc. This page collects
these requirements so they do not have to be discovered by trial and error. Each section lists the concrete types
that are known to work for that parameter and the ones that do not, checked against Boost 1.83, Abseil 20250127.0,
Folly, EASTL 3.21, `ankerl::unordered_dense`, `phmap`, `gtl`, `robin_hood`, `tsl::ordered_map`, and Qt 6.
## How to read this page
Requirements are split into two groups:
- **Always required** -- needed to instantiate `basic_json` at all, or needed by functions that virtually every program
uses (construction, element access, [`dump`](../../api/basic_json/dump.md)).
- **Required for ...** -- only needed when a particular part of the API is instantiated. Member function templates are
only instantiated when they are used, so a type may be perfectly usable even though it does not satisfy these
requirements, as long as the corresponding functions are never called.
!!! warning "Requirements are not checked"
Three requirements are checked with a `#!cpp static_assert`: the array iterator category, the width of
[`BinaryType`](#binarytype)'s `value_type`, and [`NumberUnsignedType`](#numberintegertype-and-numberunsignedtype)
being at least as wide as [`NumberIntegerType`](#numberintegertype-and-numberunsignedtype). The rest are not
diagnosed with dedicated error messages, and violating most of them results in a compiler error somewhere inside
the library. Four violations are not caught at compile time at all:
- A [`StringType`](#stringtype) whose `data()` is not null-terminated compiles and silently misparses numbers,
because the lexer hands the buffer to `#!cpp std::strtoull`/`#!cpp std::strtoll`/`#!cpp std::strtod`.
- A stateful [`AllocatorType`](#allocatortype) compiles and silently ignores its state: allocation, deallocation,
and [`get_allocator()`](../../api/basic_json/get_allocator.md) each use a different default-constructed instance.
- The two [cross-specialization conversions](#cross-specialization-conversions) below. These abort on an assertion
in a normal build, and only fail silently under `#!cpp NDEBUG`.
## Overview
| Template parameter | Default | Notable substitutes |
|-------------------------------------------------------------------|-----------------------------------|-----------------------------------------------------------------------|
| [`ObjectType`](#objecttype) | `std::map` | [`nlohmann::ordered_map`](../../api/ordered_map.md), Abseil hash maps |
| [`ArrayType`](#arraytype) | `std::vector` | `#!cpp std::deque` |
| [`StringType`](#stringtype) | `std::string` | `std::string`-like types over `char` |
| [`BooleanType`](#booleantype) | `bool` | none worth using |
| [`NumberIntegerType`](#numberintegertype-and-numberunsignedtype) | `std::int64_t` | any signed integer type |
| [`NumberUnsignedType`](#numberintegertype-and-numberunsignedtype) | `std::uint64_t` | any unsigned integer type at least as wide as `NumberIntegerType` |
| [`NumberFloatType`](#numberfloattype) | `double` | `float` (`long double`: no binary formats) |
| [`AllocatorType`](#allocatortype) | `std::allocator` | stateless allocators |
| [`JSONSerializer`](#jsonserializer) | `adl_serializer` | serializers with the same interface |
| [`BinaryType`](#binarytype) | `#!cpp std::vector<std::uint8_t>` | `#!cpp std::vector<char>` |
| [`CustomBaseClass`](#custombaseclass) | `void` | any default-constructible class |
!!! warning "Third-party containers and incomplete types"
`object_t` is instantiated inside the definition of `basic_json` -- it is probed for a `key_compare` member to
form [`object_comparator_t`](../../api/basic_json/object_comparator_t.md) -- i.e. while `basic_json` is still an
incomplete type. `#!cpp std::map` is required by the standard to support incomplete mapped types; most
third-party maps are not, and inspecting the mapped type at class scope (for instance with
`#!cpp std::is_trivially_move_assignable`) makes them unusable as `ObjectType`, no matter how their template
arguments are adapted. This rules out `absl::btree_map`, `phmap::btree_map`, `gtl::btree_map`,
`robin_hood::unordered_node_map`, `folly::F14FastMap`, and `eastl::hash_map`.
`array_t` is only *named* in the class definition and is not instantiated until `basic_json` is complete, so an
`ArrayType` that inspects its value type at class scope is generally fine -- `boost::container::small_vector` and
`static_vector` both reject incomplete value types yet work here. `absl::InlinedVector` is the exception: the
`#!cpp std::is_trivially_move_assignable<basic_json>` it evaluates while instantiating itself re-enters the
library's own trait machinery mid-instantiation.
!!! note "Folly requires C++20"
Folly's headers use `#!cpp consteval` and `#!cpp std::type_identity`, so any `basic_json` specialization that
names a Folly type has to be compiled as C++20 or later, whatever the rest of the library supports.
## `ObjectType`
`ObjectType` is instantiated as
```cpp
using object_t = ObjectType<StringType, // key_type
basic_json, // mapped_type
default_object_comparator_t, // key_compare
AllocatorType<std::pair<const StringType,
basic_json>>>; // allocator_type
```
i.e., the template arguments follow the order and meaning of `std::map`.
### Always required
- The template must be usable with **four** type arguments in the order shown above. The third argument is a
**comparator**; containers that expect something else in this position (e.g., a hash function) need an alias template
or wrapper -- see [Notes](#notes).
- An optional member type `key_compare`. If it is present it becomes
[`object_comparator_t`](../../api/basic_json/object_comparator_t.md); otherwise
[`default_object_comparator_t`](../../api/basic_json/default_object_comparator_t.md) is used.
- Member types `key_type`, `mapped_type`, `value_type`, and `iterator`.
- `value_type` must behave like `#!cpp std::pair<const key_type, mapped_type>`; the library accesses `.first` and
`.second` on it.
- `iterator` must be default-constructible and satisfy
[LegacyBidirectionalIterator](https://en.cppreference.com/w/cpp/named_req/BidirectionalIterator). The type returned
by `cbegin()`/`cend()` must satisfy the same requirements.
- Constructors: default, copy, move, and from an iterator range `(first, last)`.
- Member functions `begin()`, `end()`, `cbegin()`, `cend()`, `empty()`, `size()`, `max_size()`, `clear()`,
`find(key)`, `count(key)`, `emplace(key, value)`, `insert(value_type)`, `insert(first, last)`, `operator[](key)`,
`erase(iterator)`, and `erase(first, last)`. `erase(iterator)` may return the following iterator or `#!cpp void`;
in the latter case the library computes the successor itself, before erasing.
- `erase(key)` is **optional**: if the container does not provide one, the library falls back to `find(key)` followed
by `erase(iterator)`.
- `at(key)` is required only by [`to_ubjson`](../../api/basic_json/to_ubjson.md) and
[`to_bjdata`](../../api/basic_json/to_bjdata.md), but every container tried here provides it.
- `emplace` and `insert(value_type)` must return `#!cpp std::pair<iterator, bool>` and must have **unique-key**
semantics; multimaps cannot be used.
- The type must be swappable (via `std::swap` or an ADL `swap`).
- The comparison operators `==` and `<`; `!=`, `<=`, `>`, and `>=` are derived from them. Where the library uses
three-way comparison (C++20), `==` and `<=>` are required **instead** -- the six two-way operators do not satisfy
it. They implement [`basic_json`'s comparison operators](../../api/basic_json/operator_eq.md).
### Required for heterogeneous key lookup
The overloads of [`at`](../../api/basic_json/at.md), [`operator[]`](../../api/basic_json/operator%5B%5D.md),
[`find`](../../api/basic_json/find.md), [`contains`](../../api/basic_json/contains.md),
[`count`](../../api/basic_json/count.md), [`erase`](../../api/basic_json/erase.md), and
[`value`](../../api/basic_json/value.md) that accept a key type other than `object_t::key_type` require
- a **transparent** comparator, i.e. [`object_comparator_t`](../../api/basic_json/object_comparator_t.md) has a member
type `is_transparent` (this is why the default comparator is `#!cpp std::less<>` since C++14), and
- corresponding heterogeneous `find`, `count`, `erase`, and `operator[]` overloads on the container.
### Notes
#### `std::unordered_map` needs an adapter
`#!cpp std::unordered_map` cannot be passed directly: its third template parameter is a hash function, but
`basic_json` passes a comparator in that position. An alias template or wrapper that restores the expected argument
order makes it usable:
```cpp
template<class Key, class T, class IgnoredCompare, class Allocator>
struct unordered_map_object
: std::unordered_map<Key, T, std::hash<Key>, std::equal_to<Key>, Allocator>
{
using base_t = std::unordered_map<Key, T, std::hash<Key>, std::equal_to<Key>, Allocator>;
using base_t::base_t;
};
using unordered_json = nlohmann::basic_json<unordered_map_object>;
```
Whether `#!cpp std::unordered_map` can be instantiated at all depends on the standard library: `object_t` is formed
while `basic_json` is still incomplete (see the warning above), and libstdc++ 9 needs the size of the mapped type to
instantiate the hash map's node type, so the adapter does not compile there. Newer libstdc++ versions, and the hash
maps listed below, do not have that problem.
The adapter above works verbatim for Abseil's, Boost's, `phmap`'s and `gtl`'s hash maps, which all place the hash
function third and take a `#!cpp std::pair<const Key, T>` allocator fifth. Two need a different adapter:
- `ankerl::unordered_dense` expects an allocator over `#!cpp std::pair<Key, T>` (non-const key), so the allocator has
to be rebound to that or dropped.
- `robin_hood`'s fifth parameter is the non-type `MaxLoadFactor100`, so its adapter must drop the allocator entirely.
None of these hash maps defines `key_compare`, so all of them additionally rely on `object_comparator_t` falling back
to [`default_object_comparator_t`](../../api/basic_json/default_object_comparator_t.md); see
[`object_comparator_t`](../../api/basic_json/object_comparator_t.md).
#### Abseil hash maps
`absl::flat_hash_map` and `absl::node_hash_map` tolerate an incomplete value type, but they take a hash function as
their third template argument. The same adapter as for `#!cpp std::unordered_map` makes them usable:
```cpp
template<class Key, class T, class IgnoredCompare, class Allocator>
struct flat_hash_object
: absl::flat_hash_map<Key, T, absl::Hash<Key>, std::equal_to<Key>, Allocator>
{
using base_t = absl::flat_hash_map<Key, T, absl::Hash<Key>, std::equal_to<Key>, Allocator>;
using base_t::base_t;
};
using flat_hash_json = nlohmann::basic_json<flat_hash_object>;
```
`absl::node_hash_map` keeps references to the mapped values valid across insertions; `absl::flat_hash_map` does not,
which makes it behave like [`ordered_json`](../../api/ordered_json.md) with respect to
[iterator invalidation](../../api/basic_json/index.md#iterator-invalidation). Both expose a `capacity()` member
function, so [`JSON_DIAGNOSTICS`](../../api/macros/json_diagnostics.md) treats them conservatively and keeps the
parent pointers correct either way.
#### Iteration order
The library never relies on the container's iteration order for correctness; it does determine the order in which
object keys are serialized by [`dump`](../../api/basic_json/dump.md) and visited by
[`items`](../../api/basic_json/items.md). See [Object Order](../object_order.md).
#### `capacity()` marks a container as insertion-ordered
With [`JSON_DIAGNOSTICS`](../../api/macros/json_diagnostics.md) enabled, the library detects insertion-ordered maps by
probing for a `capacity()` member function (`nlohmann::ordered_map` inherits it from `std::vector`) and refreshes all
parent pointers after every insertion. An `ObjectType` that happens to have a `capacity()` member is therefore treated
conservatively -- this is correct, but slower.
#### Key order and duplicate keys
The library does not sort or de-duplicate keys itself; the behavior described in
[`object_t`](../../api/basic_json/object_t.md) is entirely the behavior of the chosen container.
!!! tip "Reference implementation"
`docs/mkdocs/docs/examples/custom_object_type.hpp` wraps a private `#!cpp std::map` and satisfies every
requirement above. It does not define `key_compare`, so `object_comparator_t` falls back to
[`default_object_comparator_t`](../../api/basic_json/default_object_comparator_t.md) -- a good starting point for
a custom `ObjectType`.
```cpp
--8<-- "examples/custom_object_type.hpp"
```
??? example "Compiling and using it"
```cpp
--8<-- "examples/custom_object_type.cpp"
```
Output:
```json
--8<-- "examples/custom_object_type.output"
```
### Compatible containers
| Container | Notes |
|----------------------------------------------------------------------------------|-------------------------------------------------------------------------------|
| `#!cpp std::map` (default) | |
| [`nlohmann::ordered_map`](../../api/ordered_map.md) | used by [`ordered_json`](../../api/ordered_json.md); keeps insertion order |
| [`nlohmann::fifo_map`](https://github.com/nlohmann/fifo_map) | keeps insertion order; adapter puts `fifo_map_compare` in the comparator slot |
| `boost::container::map`, `boost::container::flat_map` | no adapter needed |
| `#!cpp std::unordered_map` | through the adapter above; not with libstdc++ 9, see the note |
| `boost::unordered_map`, `boost::unordered_flat_map`, `boost::unordered_node_map` | through the adapter above |
| `absl::flat_hash_map`, `absl::node_hash_map` | through the adapter above; `flat_hash_map` moves mapped values on rehash |
| `phmap::flat_hash_map`, `phmap::node_hash_map`, `gtl::flat_hash_map` | through the adapter above |
| `ankerl::unordered_dense::map` and `segmented_map` | adapter must rebind or drop the allocator |
| `robin_hood::unordered_flat_map` | adapter must drop the allocator |
| `folly::F14NodeMap` | through the adapter above; requires C++20, see the note above |
| `folly::sorted_vector_map` | alias must drop the allocator, whose value type it disagrees on |
### Containers that cannot be used
| Container | Reason |
|--------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------|
| `absl::btree_map`, `phmap::btree_map`, `gtl::btree_map` | require a complete mapped type |
| `robin_hood::unordered_node_map`, `folly::F14FastMap`, `eastl::hash_map` | require a complete mapped type |
| `eastl::map` | EASTL iterators do not work with `#!cpp std::iterator_traits` |
| `tsl::ordered_map` | its iterators expose the mapped value as `#!cpp const` |
| `QMap` | no `value_type` member type |
| `QHash` | its `value_type` is the mapped type rather than a key/value pair, and its iterators dereference to the mapped value |
| `#!cpp std::multimap`, `#!cpp std::unordered_multimap` | `emplace` does not return `#!cpp std::pair<iterator, bool>` |
## `ArrayType`
`ArrayType` is instantiated as
```cpp
using array_t = ArrayType<basic_json, AllocatorType<basic_json>>;
```
### Always required
- The template must be usable with **two** type arguments (value type and allocator).
- Member types `value_type` and `iterator`.
- Constructors: default, copy, and move; and from an iterator range `(first, last)`.
- Member functions `begin()`, `end()`, `cbegin()`, `cend()`, `empty()`, `size()`, `max_size()`, `clear()`,
`operator[](size_type)`, `back()`, `push_back()`, `emplace_back()`, `pop_back()`, `resize()`,
`insert()` (single element, count, and range), `erase(pos)`, and `erase(first, last)`.
`basic_json::insert(pos, initializer_list)` goes through the range overload, so no initializer-list `insert` is
needed. `at(size_type)` is **not** required: [`basic_json::at(size_type)`](../../api/basic_json/at.md) checks the
index itself and then uses `operator[]`.
- `iterator` must be default-constructible, and it as well as the type returned by `cbegin()`/`cend()` must satisfy
[LegacyRandomAccessIterator](https://en.cppreference.com/w/cpp/named_req/RandomAccessIterator).
A `#!cpp static_assert` only checks for
[LegacyBidirectionalIterator](https://en.cppreference.com/w/cpp/named_req/BidirectionalIterator), but
[`dump`](../../api/basic_json/dump.md) (`cend() - 1`),
[`erase(idx)`](../../api/basic_json/erase.md) (`begin() + idx`), and the random-access operations of
[`basic_json::iterator`](../../api/basic_json/begin.md) require random access.
- The comparison operators, as for [`ObjectType`](#objecttype): `==` and `<`, or `==` and `<=>` under C++20.
### Required for individual functions
- A member type `value_type`, for [`to_bson`](../../api/basic_json/to_bson.md) of an array.
- A constructor from `(count, value)`, for
[`basic_json(size_type, const basic_json&)`](../../api/basic_json/basic_json.md).
- Swappability, via `#!cpp std::swap` or an ADL `swap`, for [`swap(array_t&)`](../../api/basic_json/swap.md).
!!! note "`capacity()` is optional"
With [`JSON_DIAGNOSTICS`](../../api/macros/json_diagnostics.md) enabled, the library reads `array_t::capacity()`
to find out whether adding an element reallocated the array and moved its elements, which would invalidate the
parent pointers. An array type without a `capacity()` member function is handled conservatively: the parent
pointers of all elements are refreshed after every insertion, which makes adding *n* elements cost O(*n*²). Only
diagnostics builds pay this; without them `capacity()` is never called.
!!! tip "Reference implementation"
`docs/mkdocs/docs/examples/custom_array_type.hpp` wraps a private `#!cpp std::vector` and satisfies every
requirement above -- a good starting point for a custom `ArrayType`.
```cpp
--8<-- "examples/custom_array_type.hpp"
```
??? example "Compiling and using it"
```cpp
--8<-- "examples/custom_array_type.cpp"
```
Output:
```json
--8<-- "examples/custom_array_type.output"
```
### Compatible containers
| Container | Notes |
|---------------------------------------------------------|-------------------------------------------------------------------------------------------|
| `#!cpp std::vector` (default) | |
| `#!cpp std::deque` | references survive appends, but not insertions elsewhere; see the `capacity()` note above |
| `#!cpp std::pmr::vector` | through an alias, as the allocator comes from `AllocatorType` instead |
| `boost::container::vector`, `deque`, `devector` | |
| `boost::container::stable_vector` | the only one tried that keeps references valid across *every* insertion |
| `boost::container::small_vector`, `folly::small_vector` | through an alias that fixes the inline capacity |
| `boost::container::static_vector` | through the same kind of alias, for arrays that stay within the fixed capacity |
| `folly::fbvector` | requires C++20, see the note above |
### Containers that cannot be used
| Container | Reason |
|-------------------------------------|-----------------------------------------------------------------------------------------------|
| `#!cpp std::list` | no `operator[]`, and no random-access iterators |
| `eastl::vector`, `QList`, `QVector` | no `max_size()`; they handle the incomplete value type fine |
| `absl::InlinedVector` | requires a complete value type, see the note above |
| `absl::FixedArray` | the size is fixed at construction, so `resize`, `push_back`, `insert` and `erase` are missing |
## `StringType`
`StringType` is used **both** for JSON string values and for the keys of JSON objects
(`string_t` and `object_t::key_type`).
### Always required
- A member type `value_type` that is one byte wide and `char`-compatible. The library stores and processes UTF-8
encoded `char` data and hands `data()` to `#!cpp std::strtoull`/`#!cpp std::strtoll`.
`#!cpp std::wstring`, `#!cpp std::u16string`, and `#!cpp std::u32string` are **not** valid choices; see the FAQ on
[wide string handling](../../home/faq.md#wide-string-handling).
- Constructors: default, copy, move, from `#!cpp const char*` (which must not be `#!cpp explicit`), from
`#!cpp (const char*, size_type)`, and from `#!cpp (size_type, char)`; and copy or move assignment.
- Member functions `size()`, `clear()`, `resize(n, c)`, `data()`, `push_back(char)`, and `operator[]`
(const and non-const, returning references). `c_str()` and `back()` are **not** required.
- `data()` must return a pointer to a contiguous, **null-terminated** buffer -- the parser hands it to
`#!cpp std::strtoull`. A type whose `data()` is not null-terminated does not fail to compile; it silently
misparses numbers.
- `append(const char*, size_type)`, used by [`dump`](../../api/basic_json/dump.md), and `append(const StringType&)`,
used by the CBOR reader for indefinite-length strings. The library's internal string concatenation additionally has
to append a `#!cpp char` and a `#!cpp const char*`; for each it selects between `append(arg)`, `#!cpp operator+=`,
`append(first, last)`, and `append(data, size)`.
- The comparison operator `==` against another `StringType`, and `<` for use as a key of the chosen
[`ObjectType`](#objecttype) (with the default comparator, `#!cpp std::less<>` must be able to compare two
`StringType` values, and a `StringType` with the key types used for lookup). `!=` is never applied to a
`StringType`, and `==` against `#!cpp const char*` is resolved by the implicit `#!cpp const char*` constructor.
### Required for the binary formats
- `resize(n)`, used by the readers to make room for a block of bytes.
- Non-const `operator[]`, into which the readers `#!cpp std::memcpy` those bytes. A non-`#!cpp const` `data()` would
serve just as well, but `#!cpp std::string` has only had one since C++17, and the library still supports C++11.
### Required for JSON Pointer, `flatten`, and `diff`
- A static member `npos` and the member function `find_first_of(char, size_type)` -- together with `data()`,
`reserve(n)`, and `append(const char*, size_type)` they implement the escaping and unescaping of reference tokens
described in RFC 6901. Neither `find(const StringType&, size_type)`, nor `substr(pos, count)`, nor
`replace(pos, count, const StringType&)` is required.
- `empty()`.
- `begin()` and `end()` -- used by
[`operator[](const json_pointer&)`](../../api/basic_json/operator%5B%5D.md) to decide whether a reference token
denotes an array index.
### Required for other functionality
| Functionality | Additional requirement |
|-----------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [`diff`](../../api/basic_json/diff.md), [`items`](../../api/basic_json/items.md), [`std::hash`](../../api/basic_json/std_hash.md) | conversion of a `#!cpp std::size_t` to `StringType`: either assignability from the result of `#!cpp std::to_string`, or an ADL overload `#!cpp void int_to_string(StringType&, std::size_t)` |
| [`std::hash<basic_json>`](../../api/basic_json/std_hash.md) | additionally a specialization of `#!cpp std::hash<StringType>` |
| [`to_bson`](../../api/basic_json/to_bson.md) | `find(value_type)` and `npos` |
| [`parse`](../../api/basic_json/parse.md) from a `string_t` | the input adapters must accept it; otherwise pass a character range |
| `#!cpp operator<<(std::ostream&, const json_pointer&)` | streamability to `#!cpp std::ostream` |
| exception messages | `data()` and `size()`, or `begin()` and `end()` |
### Compatible types
| Type | Notes |
|-----------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `#!cpp std::string` (default) | |
| `#!cpp std::basic_string` with a custom **stateless** allocator | |
| `#!cpp std::pmr::string` | see the warning below before relying on the memory resource |
| `boost::container::string` | needs a user-supplied `#!cpp std::hash` specialization (Boost provides `boost::hash` instead) |
| `folly::fbstring` | requires C++20, see the note above |
| `eastl::string` | needs a user-supplied `#!cpp std::hash` and an ADL `int_to_string` (it is not assignable from a `#!cpp std::string`); [`parse`](../../api/basic_json/parse.md) does not accept it directly -- pass a character range or a `#!cpp std::string` |
| a custom string class in a user-defined namespace | if the requirements above are met |
### Types that cannot be used
| Type | Reason |
|----------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------|
| `#!cpp std::wstring`, `#!cpp std::u16string`, `#!cpp std::u32string` | the character type is not one byte wide |
| `#!cpp std::u8string` | one byte wide, but `#!cpp char8_t` is not `#!cpp char`-compatible |
| `absl::Cord` | no `value_type`, and the storage is not contiguous |
| `QString` | no `append(const char*, size_type)`; its `QChar` is also two bytes wide, though that is never diagnosed |
!!! warning "A `std::pmr::string` mostly does not use the memory resource you choose"
`basic_json` cannot be given an allocator or a memory resource. `AllocatorType` is default-constructed at every
allocation and has to be stateless (see [`AllocatorType`](#allocatortype)), and string values the library creates
are constructed with their own default allocator. So:
- Every string the library itself produces -- from [`parse`](../../api/basic_json/parse.md), from
[`dump`](../../api/basic_json/dump.md), or by default construction -- allocates from
`#!cpp std::pmr::get_default_resource()`.
- **Copying** an arena-backed string into a value silently drops its memory resource: the copy lands on the
default resource, because `#!cpp std::pmr::polymorphic_allocator` does not propagate on copy construction.
Nothing warns about this.
- **Moving** one in does keep it, and later growth still allocates from that arena -- but it does not survive a
copy of the enclosing `basic_json`.
- Passing `#!cpp std::pmr::polymorphic_allocator` as `AllocatorType` does not work around any of this; it does
not compile.
Apart from moving a string in, the only way to redirect these allocations is the process-global
`#!cpp std::pmr::set_default_resource()`.
!!! tip "Reference implementation"
`docs/mkdocs/docs/examples/custom_string_type.hpp` wraps a private `#!cpp std::string` and satisfies every
requirement above -- a good starting point for a custom `StringType`. The unit test
`tests/src/unit-alt-string.cpp` contains a more thorough variant, `alt_string`, exercised against a larger part
of the API.
```cpp
--8<-- "examples/custom_string_type.hpp"
```
??? example "Compiling and using it"
```cpp
--8<-- "examples/custom_string_type.cpp"
```
Output:
```json
--8<-- "examples/custom_string_type.output"
```
## `BooleanType`
`boolean_t` is stored **directly** inside `basic_json`, as a member of an anonymous union.
### Always required
- A literal type that is trivially default-constructible, trivially copyable, and trivially destructible; otherwise the
union's special member functions are deleted.
- **Implicitly** convertible from `#!cpp bool` -- an `#!cpp explicit` constructor is not enough, because the
`to_json` overload for a custom `BooleanType` is constrained on `#!cpp std::is_convertible` -- and contextually
convertible to `#!cpp bool` (here an `#!cpp explicit operator bool` is fine).
- Comparison operators `==`, `!=`, `<`, `<=`, `>`, `>=` (or `<=>`).
- Convertible from and to `#!cpp bool` through the serializer, because
[`get<bool>()`](../../api/basic_json/get.md) is used internally.
There is little reason to use anything other than `#!cpp bool` here.
### Compatible types
`#!cpp bool` is the only usable choice. Another trivially copyable type that is implicitly convertible to and from
`#!cpp bool` -- `#!cpp std::uint8_t`, say -- does compile, and JSON booleans still round-trip, but the type then
serves as both `boolean_t` and an ordinary integer: `basic_json` can no longer be constructed or assigned from a
`#!cpp std::uint8_t` at all (the boolean and unsigned-integer `to_json` overloads become ambiguous), and
[`get<std::uint8_t>()`](../../api/basic_json/get.md) on a number throws
[`type_error.302`](../../home/exceptions.md#jsonexceptiontype_error302) instead of returning the value.
## `NumberIntegerType` and `NumberUnsignedType`
Both types are stored **directly** inside `basic_json`'s union.
### Always required
- `#!cpp std::is_integral` must be satisfied: `NumberIntegerType` must be a **signed** integer type,
`NumberUnsignedType` an **unsigned** integer type. Class types are not supported -- among others, the constructors
taking integer values are constrained on `#!cpp std::is_integral`.
- Trivially default-constructible, trivially copyable, and trivially destructible (union member).
- `#!cpp std::numeric_limits` must be specialized for both types.
- `NumberUnsignedType` must be able to represent the absolute value of every `NumberIntegerType` value; serialization
of negative numbers converts the value to `NumberUnsignedType`. A `#!cpp static_assert` requires it to be at least as
wide as `NumberIntegerType`, which is what that amounts to for the standard integer types.
- Both types must fit into the internal 64-character number buffer used by
[`dump`](../../api/basic_json/dump.md), which is the case for all standard integer types.
- [`std::hash<basic_json>`](../../api/basic_json/std_hash.md) additionally requires `#!cpp std::hash` specializations.
### Notes
The number types influence what the parser accepts: an integer literal that does not round-trip through the chosen type
is stored as [`number_float_t`](../../api/basic_json/number_float_t.md) instead. Choosing types narrower than 64 bits
therefore silently changes parse results rather than raising an error. See
[Number Handling](number_handling.md) for details.
### Compatible types
| Type pair | Support |
|----------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `#!cpp std::int64_t` / `#!cpp std::uint64_t` (default) | full |
| `#!cpp std::int32_t` / `#!cpp std::uint32_t`, `#!cpp long long` / `#!cpp unsigned long long` | full; narrower types change which literals the parser can represent |
| any other pair of standard signed/unsigned integer types | full |
| class types, enumerations | not usable; `#!cpp std::is_integral` must hold |
| `#!cpp bool`, or a type already used for another member of the union | not usable; `#!cpp std::is_integral<bool>` is in fact `#!cpp true`, but the `get_impl_ptr` overloads for `boolean_t`, `number_integer_t`, `number_unsigned_t` and `number_float_t` would collide |
## `NumberFloatType`
`number_float_t` is stored **directly** inside `basic_json`'s union.
### Always required
- Trivially default-constructible, trivially copyable, and trivially destructible (union member).
- `#!cpp std::numeric_limits` must be specialized; `max_digits10` is used to size the conversion.
- `#!cpp std::isfinite` must be applicable to the type.
### Required for parsing and serialization
`NumberFloatType` must be one of `#!cpp float`, `#!cpp double`, or `#!cpp long double`:
- The [parser](../parsing/index.md) converts number literals with `#!cpp std::strtof`, `#!cpp std::strtod`, or
`#!cpp std::strtold`; the library provides overloads for exactly these three types.
- [`dump`](../../api/basic_json/dump.md) falls back to `#!cpp std::snprintf` with the `%g` and `%Lg` conversion
specifiers, for which the library likewise provides only `#!cpp double` and `#!cpp long double` overloads
(`#!cpp float` is promoted to `#!cpp double`).
If `#!cpp std::numeric_limits<NumberFloatType>` describes an IEEE 754 binary32 or binary64 number, `dump` uses the
Grisu2 algorithm, which produces the shortest representation that round-trips. Otherwise the `snprintf` fallback with
`max_digits10` digits is used.
### Required for the binary formats
`NumberFloatType` must be `#!cpp float` or `#!cpp double`. The writers for
[CBOR, MessagePack, UBJSON, BJData, and BSON](../binary_formats/index.md) map a floating-point value onto an IEEE 754
binary32 or binary64 field and have no encoding for `#!cpp long double`.
### Compatible types
| Type | Support |
|--------------------------|-----------------------------------------------------------------------------------------------------------------------|
| `#!cpp double` (default) | full; short round-trip output through Grisu2 |
| `#!cpp float` | full; short round-trip output through Grisu2 |
| `#!cpp long double` | `dump` and `parse` only; the binary format writers do not compile, as they only handle IEEE 754 binary32 and binary64 |
| any other type | not usable |
## `AllocatorType`
`AllocatorType` is instantiated with **one** argument, for each of `object_t`, `array_t`, `string_t`, `binary_t`,
`basic_json`, and `#!cpp std::pair<const StringType, basic_json>`.
### Always required
- The template must be usable with exactly one type argument. The library instantiates `AllocatorType<T>` directly and
never uses `#!cpp std::allocator_traits<...>::rebind_alloc`.
- It must satisfy the [Allocator](https://en.cppreference.com/w/cpp/named_req/Allocator) named requirement so that
`#!cpp std::allocator_traits` can be used with it.
- It must be **default-constructible and stateless**. Objects are allocated with a default-constructed allocator and
deallocated with a *different* default-constructed allocator, and
[`get_allocator()`](../../api/basic_json/get_allocator.md) returns a default-constructed instance. Allocators
carrying state are not supported, so there is no way to tell a `basic_json` where to allocate from; see the note
under [`StringType`](#stringtype) for what that means in practice. A stateful allocator is **not diagnosed**: it
compiles and silently ignores the state.
- It must support **incomplete types**: `AllocatorType<basic_json>` is instantiated inside the definition of
`basic_json` itself.
- `#!cpp std::allocator_traits<AllocatorType<basic_json>>::pointer` becomes
[`basic_json::pointer`](../../api/basic_json/index.md#container-types), and iterators are constructed from raw
`#!cpp basic_json*` values. The `pointer` type must therefore be a plain pointer; fancy pointers are not supported.
### Compatible types
| Type | Support |
|-------------------------------------------------------------------|----------------------------------------|
| `#!cpp std::allocator` (default) | full |
| a custom stateless allocator template | full |
| stateful allocators, e.g. `#!cpp std::pmr::polymorphic_allocator` | not usable; see the requirements above |
## `JSONSerializer`
`JSONSerializer` is instantiated as `JSONSerializer<T, void>` and defaults to
[`adl_serializer`](../../api/adl_serializer/index.md).
### Always required
- The template must accept **two** type arguments. It does not have to give the second one a default -- `basic_json`
declares the parameter as `#!cpp template<typename T, typename SFINAE = void> class JSONSerializer`, so uses such as
`#!cpp JSONSerializer<T>` inside the library supply `#!cpp void` themselves. The second parameter exists so that
partial specializations can be constrained by SFINAE.
- For every type `T` that is converted **to** a JSON value, a static member function
`#!cpp static void to_json(basic_json&, T)` must exist.
- For every type `T` that is converted **from** a JSON value, either
`#!cpp static void from_json(const basic_json&, T&)` or `#!cpp static T from_json(const basic_json&)` must exist.
The latter form is required for types that are not default-constructible; see
[Arbitrary Types Conversions](../arbitrary_types.md).
- To support the [converting constructor](../../api/basic_json/basic_json.md) between different `basic_json`
specializations, `to_json` must be available for `boolean_t`, `number_integer_t`, `number_unsigned_t`,
`number_float_t`, `string_t`, `object_t`, `array_t`, and `binary_t` of the *source* specialization.
### Compatible types
| Type | Support |
|---------------------------------------------------------------------------|-------------------------------------------------------------------|
| [`nlohmann::adl_serializer`](../../api/adl_serializer/index.md) (default) | full |
| a class template deriving from `adl_serializer` | full; the usual way to change behavior while keeping the defaults |
| an unrelated template with the same interface | full, but it has to handle every type the library converts |
## `BinaryType`
`BinaryType` is not a JSON type; it is used for the byte strings of the
[binary formats](../binary_formats/index.md). It is wrapped as
```cpp
using binary_t = nlohmann::byte_container_with_subtype<BinaryType>;
```
### Always required
- A non-`final` class type -- [`byte_container_with_subtype`](../../api/byte_container_with_subtype/index.md) derives
from it publicly.
- A member type `value_type` that is **exactly one byte** wide (e.g., `#!cpp std::uint8_t`, `#!cpp char`, or
`#!cpp std::byte`). Readers and writers reinterpret the container's storage as raw bytes, so a wider `value_type` is
rejected with a `#!cpp static_assert`.
- Contiguous storage: the binary readers `#!cpp std::memcpy` into `#!cpp &binary[n]`, the writers `reinterpret_cast`
`data()`. `#!cpp data() + n` would do for the readers too, but they share one helper with
[`StringType`](#stringtype), whose non-`#!cpp const` `data()` is C++17 and later only.
- Default-constructible, copy-constructible, and move-constructible.
- Member functions `size()`, `empty()`, `data()`, `resize()`, `operator[]`, `back()`, `begin()`, `end()`, `cbegin()`,
and `cend()` with random-access iterators, and `insert(pos, first, last)`, which the CBOR reader uses to join the
chunks of an indefinite-length byte string. `push_back()` is **not** required.
- Comparison operators: `==` is used by
[`byte_container_with_subtype`](../../api/byte_container_with_subtype/index.md), the relational operators by
[`basic_json`'s comparison operators](../../api/basic_json/operator_le.md).
### Required for individual functions
- `clear()`, for [`basic_json::clear()`](../../api/basic_json/clear.md).
`max_size()`, `at()`, `reserve()`, `erase()`, `pop_back()`, and `emplace_back()` are **not** used at all.
See [`binary_t`](../../api/basic_json/binary_t.md) for how a non-default `BinaryType` changes the meaning of assigning
such a container to a `basic_json` value.
!!! tip "Reference implementation"
`docs/mkdocs/docs/examples/custom_binary_type.hpp` wraps a private `#!cpp std::vector<std::uint8_t>` and satisfies
every requirement above -- a good starting point for a custom `BinaryType`.
```cpp
--8<-- "examples/custom_binary_type.hpp"
```
??? example "Compiling and using it"
```cpp
--8<-- "examples/custom_binary_type.cpp"
```
Output:
```json
--8<-- "examples/custom_binary_type.output"
```
### Compatible containers
| Container | Notes |
|---------------------------------------------------------------------------------------------|---------------------------------------------------------------------------|
| `#!cpp std::vector<std::uint8_t>` (default) | |
| `#!cpp std::vector<char>`, `#!cpp std::vector<std::byte>` | `dump()` writes the bytes as 0..255 whichever is used |
| `boost::container::vector<std::uint8_t>`, `boost::container::small_vector<std::uint8_t, N>` | |
| `absl::InlinedVector<std::uint8_t, N>` | usable here, unlike as an `ArrayType`, because the value type is complete |
| `eastl::vector<std::uint8_t>` | usable here, unlike as an `ArrayType`, because `max_size()` is not needed |
| `folly::fbvector<std::uint8_t>` | requires C++20, see the note above |
### Containers that cannot be used
| Container | Reason |
|------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `QByteArray` | no `empty()` (it spells that `isEmpty()`); its `insert` takes an index rather than an iterator; and it converts to `string_t`, which makes `to_json` ambiguous between a string and a binary value |
| `#!cpp std::string` | `binary_t::container_type` and `string_t` would be the same type, so the two [`swap`](../../api/basic_json/swap.md) overloads collide and `basic_json` cannot be instantiated at all |
| `#!cpp std::deque<std::uint8_t>` | storage is not contiguous, so there is no `data()` |
| containers whose `value_type` is wider than one byte | see above -- accepted by the compiler, wrong at runtime |
## `CustomBaseClass`
`CustomBaseClass` is an extension point: unless it is `#!cpp void` (the default, which selects the empty
`nlohmann::json_default_base`), `basic_json` publicly derives from it.
### Always required
- A non-`final`, default-constructible class type.
- `basic_json` is copy-/move-constructible and copy-/move-assignable only if `CustomBaseClass` is.
### Notes
`basic_json` is documented to be a
[StandardLayoutType](https://en.cppreference.com/w/cpp/named_req/StandardLayoutType). Because `basic_json` has
non-static data members of its own, a `CustomBaseClass` with non-static data members forfeits this guarantee.
Note the namespace of `CustomBaseClass` becomes an associated namespace of `basic_json` for the purpose of
argument-dependent lookup.
See [`json_base_class_t`](../../api/basic_json/json_base_class_t.md) for an example.
### Compatible types
| Type | Support |
|----------------------------------------------|----------------------------------------------------------------------------|
| `#!cpp void` (default) | an empty base class is used; no effect on `basic_json` |
| any default-constructible, non-`final` class | full; see [`json_base_class_t`](../../api/basic_json/json_base_class_t.md) |
## Cross-specialization conversions
Converting a value from one `basic_json` specialization into another (see the
[converting constructor](../../api/basic_json/basic_json.md)) imposes two additional requirements that are not
diagnosed at compile time. With assertions enabled they abort on the `#!cpp JSON_ASSERT` at the end of the converting
constructor; under `#!cpp NDEBUG` they fail **silently** at runtime:
- The target `string_t` must be directly constructible from the source `string_t`. Otherwise the string is converted to
an array of character codes.
- The target `object_t::key_type` must be directly constructible from the source object's key type. Otherwise the
object is converted to an array of key/value pairs.
See [issue #3425](https://github.com/nlohmann/json/issues/3425), [`string_t`](../../api/basic_json/string_t.md), and
[`object_t`](../../api/basic_json/object_t.md).
## See also
- [Types](index.md) -- overview of how JSON values are stored
- [Number Handling](number_handling.md) -- how the number types affect parsing and serialization
- [Object Order](../object_order.md) -- using an insertion-ordered `ObjectType`
- [`basic_json`](../../api/basic_json/index.md) -- API documentation of the class template
+15
View File
@@ -970,6 +970,21 @@ A JSON Patch `move` operation's `"from"` location is a proper prefix of its `"pa
This exception was added in version 3.13.0. Before that, this situation could succeed with a corrupted result: for an array target, removing the "from" element before the "add" step shifted subsequent indices, so "path" silently re-resolved to a different element than intended.
### json.exception.out_of_range.415
MessagePack's ext type and BSON's binary subtype are each stored in a single byte. This exception is thrown when serializing a
[`byte_container_with_subtype`](../api/byte_container_with_subtype/index.md) whose subtype exceeds 255.
!!! failure "Example message"
```
[json.exception.out_of_range.415] subtype 70000 is too large for the MessagePack ext type (max 255)
```
!!! note
This exception was added in version 3.13.0. Before that, subtypes above 255 were silently truncated modulo 256 instead of raising an error.
## Further exceptions
This exception is thrown in case of errors that cannot be classified with the
+48
View File
@@ -90,6 +90,54 @@ The library supports **Unicode input** as follows:
In most cases, the parser is right to complain, because the input is not UTF-8 encoded. This is especially true for Microsoft Windows, where Latin-1 or ISO 8859-1 is often the standard encoding.
### NUL bytes in the input
!!! question "Questions"
- Why does `json::parse()` silently ignore part of my input?
- Why does a `std::string`/buffer with extra data after the JSON text parse without error, while a similar-looking string with extra text does not?
A `'\0'` (NUL) byte anywhere in the input is treated the same as the real end of the input, rather than as an ordinary (and, outside of a string, invalid) byte. Everything from that byte onward is silently ignored, without a parse error — including further, otherwise well-formed JSON:
```cpp
json::parse(std::string("123") + '\0'); // == 123, no error
json::parse(std::string("123") + '\0' + "true"); // == 123, the "true" is silently ignored too
```
This is different from any other unexpected trailing byte, which *does* raise [`parse_error.101`](../home/exceptions.md#jsonexceptionparse_error101):
```cpp
json::parse("123x"); // throws parse_error.101: unexpected additional data
```
This falls out of the same convention used when no explicit input length is given at all: `json::parse(const char*)` already stops at the first NUL byte via `strlen()`, since a bare pointer has no length of its own. The library applies that same NUL-terminated-C-string convention uniformly, rather than only when a length is genuinely unavailable — so a `std::string`, iterator range, or container whose content happens to include a NUL byte is affected the same way a raw `const char*` would be.
If your input may contain a trailing or embedded NUL that is **not** meant to signal the end of the JSON text — for instance, a fixed-size, zero-padded buffer — trim it yourself before calling `parse()`, since the library will otherwise silently stop there instead of raising an error:
```cpp
s.resize(s.find('\0')); // drop everything from the first NUL onward, if any
json::parse(s);
```
**Opt-in strict handling (since version 3.13.0)**
Manually trimming every input is easy to forget. If you define [`JSON_STRICT_NUL_HANDLING`](../api/macros/json_strict_nul_handling.md) to `1` before including the library, a `'\0'` byte is instead rejected like any other unexpected byte and raises `parse_error.101`, instead of being treated as end of input:
```cpp
#define JSON_STRICT_NUL_HANDLING 1
#include <nlohmann/json.hpp>
json::parse(std::string("123") + '\0'); // throws parse_error.101 instead of silently returning 123
```
This macro defaults to `0` (disabled, preserving the behavior described above) to avoid breaking existing code that may depend on it, even unknowingly; it is planned to become the default in version 4.0.0. See [its documentation](../api/macros/json_strict_nul_handling.md) for details, including how it also affects `char` arrays such as string literals.
Note that this is unrelated to an *unescaped* NUL byte occurring **inside** a quoted JSON string, which is a different, already-invalid case and is correctly rejected either way:
```cpp
json::parse(std::string("\"") + '\0' + "\""); // throws parse_error.101: control character U+0000 (NUL) must be escaped to \u0000
```
### Wide string handling
!!! question
+5
View File
@@ -198,6 +198,11 @@ Use the non-amalgamated version of the library. This option is `ON` by default.
Treat the library headers like system headers (i.e., adding `SYSTEM` to the [`target_include_directories`](https://cmake.org/cmake/help/latest/command/target_include_directories.html) call) to check for this library by tools like Clang-Tidy. This option is `OFF` by default.
### `JSON_StrictNulHandling`
Reject a `'\0'` (NUL) byte in the input instead of treating it as end of input, by defining the macro
[`JSON_STRICT_NUL_HANDLING`](../api/macros/json_strict_nul_handling.md). This option is `OFF` by default.
### `JSON_Valgrind`
Execute the test suite with [Valgrind](https://valgrind.org). This option is `OFF` by default. Depends on `JSON_BuildTests`.
+2
View File
@@ -98,6 +98,7 @@ nav:
- Types:
- features/types/index.md
- features/types/number_handling.md
- features/types/template_parameters.md
- Integration:
- integration/index.md
- integration/migration_guide.md
@@ -293,6 +294,7 @@ nav:
- 'JSON_NO_IO': api/macros/json_no_io.md
- 'JSON_SKIP_LIBRARY_VERSION_CHECK': api/macros/json_skip_library_version_check.md
- 'JSON_SKIP_UNSUPPORTED_COMPILER_CHECK': api/macros/json_skip_unsupported_compiler_check.md
- 'JSON_STRICT_NUL_HANDLING': api/macros/json_strict_nul_handling.md
- 'JSON_USE_GLOBAL_UDLS': api/macros/json_use_global_udls.md
- 'JSON_USE_IMPLICIT_CONVERSIONS': api/macros/json_use_implicit_conversions.md
- 'JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON': api/macros/json_use_legacy_discarded_value_comparison.md
+1 -1
View File
@@ -1,7 +1,7 @@
wheel==0.48.0
mkdocs==1.6.1 # documentation framework
mkdocs-git-revision-date-localized-plugin==1.5.4 # plugin "git-revision-date-localized"
mkdocs-git-revision-date-localized-plugin==1.6.0 # plugin "git-revision-date-localized"
mkdocs-material==9.7.7 # theme for mkdocs
mkdocs-material-extensions==1.3.1 # extensions
mkdocs-minify-plugin==0.8.0 # plugin "minify"
+4 -1
View File
@@ -101,7 +101,10 @@ class exception : public std::exception
{
if (&element.second == current)
{
tokens.emplace_back(element.first.c_str());
// data() is null-terminated, so a key containing
// a null byte is cut short here rather than
// truncating the whole message at what()
tokens.emplace_back(element.first.data());
break;
}
}
+3 -1
View File
@@ -114,7 +114,9 @@ std::size_t hash(const BasicJsonType& j)
seed = combine(seed, static_cast<std::size_t>(j.get_binary().subtype()));
for (const auto byte : j.get_binary())
{
seed = combine(seed, std::hash<std::uint8_t> {}(byte));
// the cast is needed for binary types whose value type is not
// an integer (e.g., std::byte)
seed = combine(seed, std::hash<std::uint8_t> {}(static_cast<std::uint8_t>(byte)));
}
return seed;
}
@@ -3146,7 +3146,10 @@ class binary_reader
number_string,
out_of_range::create(406, concat("number overflow parsing '", number_string, '\''), nullptr));
}
return sax->number_float(parsed_float, std::move(number_string));
// number_string is a std::string, while the SAX interface takes a
// string_t; convert explicitly, as the two are only implicitly
// convertible for some string types
return sax->number_float(parsed_float, string_t(number_string.data(), number_string.size()));
}
case token_type::uninitialized:
case token_type::literal_true:
@@ -762,6 +762,21 @@ contiguous_bytes_input_adapter input_adapter(CharT b)
template<typename T, std::size_t N>
auto input_adapter(T (&array)[N]) -> decltype(input_adapter(array, array + N)) // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
{
#if JSON_STRICT_NUL_HANDLING
// A `char` array from string-literal initialization (e.g. json::parse("123"))
// carries a trailing '\0' contributed by the compiler, not by the source
// text; drop exactly that one byte so it is not mistaken for real trailing
// data. Every other element type (unsigned char, std::uint8_t, ...) keeps
// the full extent unconditionally, since a trailing zero byte there is
// genuine data (e.g. CBOR/MessagePack). This intentionally does not
// strlen()-scan the array (as the pointer overload above does for a
// null-delimited string): for a `char` array that is not NUL-terminated
// within its bounds, that would read past the end of the array.
if (std::is_same<typename std::remove_cv<T>::type, char>::value && N > 0 && array[N - 1] == 0)
{
return input_adapter(array, array + N - 1);
}
#endif
return input_adapter(array, array + N);
}
+98 -18
View File
@@ -8,11 +8,11 @@
#pragma once
#include <algorithm> // min
#include <algorithm> // find_if, min
#include <cstddef>
#include <string> // string
#include <type_traits> // enable_if_t
#include <utility> // move
#include <utility> // move, pair
#include <vector> // vector
#include <nlohmann/detail/exceptions.hpp>
@@ -278,7 +278,7 @@ class json_sax_dom_parser
if (JSON_HEDLEY_UNLIKELY(len != detail::unknown_size() && len > ref_stack.back()->max_size()))
{
JSON_THROW(out_of_range::create(408, concat("excessive object size: ", std::to_string(len)), ref_stack.back()));
return parse_error(0, "", out_of_range::create(408, concat("excessive object size: ", std::to_string(len)), ref_stack.back()));
}
return true;
@@ -327,7 +327,7 @@ class json_sax_dom_parser
if (JSON_HEDLEY_UNLIKELY(len != detail::unknown_size() && len > ref_stack.back()->max_size()))
{
JSON_THROW(out_of_range::create(408, concat("excessive array size: ", std::to_string(len)), ref_stack.back()));
return parse_error(0, "", out_of_range::create(408, concat("excessive array size: ", std::to_string(len)), ref_stack.back()));
}
if (len != detail::unknown_size())
@@ -611,7 +611,7 @@ class json_sax_dom_callback_parser
// check object limit
if (JSON_HEDLEY_UNLIKELY(len != detail::unknown_size() && len > ref_stack.back()->max_size()))
{
JSON_THROW(out_of_range::create(408, concat("excessive object size: ", std::to_string(len)), ref_stack.back()));
return parse_error(0, "", out_of_range::create(408, concat("excessive object size: ", std::to_string(len)), ref_stack.back()));
}
}
return true;
@@ -631,7 +631,17 @@ class json_sax_dom_callback_parser
// add discarded value at the given key and store the reference for later
if (keep && ref_stack.back())
{
object_element = &(ref_stack.back()->m_data.m_value.object->operator[](val) = discarded);
auto& obj = *ref_stack.back()->m_data.m_value.object;
const auto it = obj.find(val);
if (it != obj.end())
{
// this is a duplicate key (legal in JSON); remember its
// current value so it can be restored later if the new
// value is rejected by the callback, instead of being
// erased together with the discarded placeholder
duplicate_key_stash.emplace_back(&(it->second), it->second);
}
object_element = &(obj[val] = discarded);
}
return true;
@@ -643,13 +653,18 @@ class json_sax_dom_callback_parser
{
if (!callback(static_cast<int>(ref_stack.size()) - 1, parse_event_t::object_end, *ref_stack.back()))
{
// discard object
*ref_stack.back() = discarded;
// discard object, unless this slot holds a duplicate key's
// previous value pending restoration, in which case that
// value is restored instead of being discarded
if (!resolve_duplicate_key_stash(ref_stack.back(), true))
{
*ref_stack.back() = discarded;
#if JSON_DIAGNOSTIC_POSITIONS
// Set start/end positions for discarded object.
handle_diagnostic_positions_for_json_value(*ref_stack.back());
// Set start/end positions for discarded object.
handle_diagnostic_positions_for_json_value(*ref_stack.back());
#endif
}
}
else
{
@@ -663,6 +678,10 @@ class json_sax_dom_callback_parser
#endif
ref_stack.back()->set_parents();
// this object is finally, definitively kept; drop any
// pending duplicate-key stash entry for its slot since it
// can no longer be restored
resolve_duplicate_key_stash(ref_stack.back(), false);
}
}
@@ -711,7 +730,7 @@ class json_sax_dom_callback_parser
// check array limit
if (JSON_HEDLEY_UNLIKELY(len != detail::unknown_size() && len > ref_stack.back()->max_size()))
{
JSON_THROW(out_of_range::create(408, concat("excessive array size: ", std::to_string(len)), ref_stack.back()));
return parse_error(0, "", out_of_range::create(408, concat("excessive array size: ", std::to_string(len)), ref_stack.back()));
}
if (len != detail::unknown_size())
@@ -743,16 +762,25 @@ class json_sax_dom_callback_parser
#endif
ref_stack.back()->set_parents();
// this array is finally, definitively kept; drop any
// pending duplicate-key stash entry for its slot since it
// can no longer be restored
resolve_duplicate_key_stash(ref_stack.back(), false);
}
else
{
// discard array
*ref_stack.back() = discarded;
// discard array, unless this slot holds a duplicate key's
// previous value pending restoration, in which case that
// value is restored instead of being discarded
if (!resolve_duplicate_key_stash(ref_stack.back(), true))
{
*ref_stack.back() = discarded;
#if JSON_DIAGNOSTIC_POSITIONS
// Set start/end positions for discarded array.
handle_diagnostic_positions_for_json_value(*ref_stack.back());
// Set start/end positions for discarded array.
handle_diagnostic_positions_for_json_value(*ref_stack.back());
#endif
}
}
}
@@ -869,6 +897,35 @@ class json_sax_dom_callback_parser
}
#endif
/// if there is a pending duplicate-key stash entry for this exact slot,
/// remove it from the stash; if restore_value is true, the stashed
/// previous value is moved back into the slot first (use this when the
/// new value at that slot was rejected); otherwise the stash entry is
/// simply dropped (use this when the new value was accepted, so it
/// correctly supersedes the old one and no restore should ever happen
/// for this slot again)
/// @return whether a matching stash entry was found (and processed)
bool resolve_duplicate_key_stash(BasicJsonType* slot, bool restore_value)
{
const auto it = std::find_if(duplicate_key_stash.begin(), duplicate_key_stash.end(),
[slot](const std::pair<BasicJsonType*, BasicJsonType>& entry)
{
return entry.first == slot;
});
if (it == duplicate_key_stash.end())
{
return false;
}
if (restore_value)
{
*slot = std::move(it->second);
}
duplicate_key_stash.erase(it);
return true;
}
/*!
@brief the key the value now being handled will be stored under
@@ -887,7 +944,9 @@ class json_sax_dom_callback_parser
}
/*!
@brief remove the discarded value the callback rejected from its parent
@brief remove the discarded value the callback rejected from its parent,
unless it is a duplicate key's slot with a stashed previous value, in
which case that previous value is restored instead
A rejected value can only ever be the one most recently added to @a parent:
the last element of an array, or the placeholder key() stored under @a key
@@ -902,7 +961,7 @@ class json_sax_dom_callback_parser
@param[in,out] parent the container to remove the rejected value from
@param[in] key the key the value was stored under; unused for arrays
*/
static void remove_discarded_value(BasicJsonType& parent, const string_t& key)
void remove_discarded_value(BasicJsonType& parent, const string_t& key)
{
if (parent.is_array())
{
@@ -918,7 +977,12 @@ class json_sax_dom_callback_parser
const auto it = object.find(key);
if (it != object.end() && it->second.is_discarded())
{
object.erase(it);
// a duplicate key's slot has a stashed previous value that
// must be restored instead of being erased
if (!resolve_duplicate_key_stash(&it->second, true))
{
object.erase(it);
}
}
}
}
@@ -1020,6 +1084,16 @@ class json_sax_dom_callback_parser
JSON_ASSERT(object_element);
*object_element = std::move(value);
if (!skip_callback)
{
// this scalar value finally, definitively replaces whatever was
// at this slot; drop any pending duplicate-key stash entry for
// it since it can no longer be restored (a container value at
// this slot is resolved later, in end_object()/end_array(),
// since skip_callback is true for the placeholder handling that
// happens here for those)
resolve_duplicate_key_stash(object_element, false);
}
return {true, object_element};
}
@@ -1039,6 +1113,12 @@ class json_sax_dom_callback_parser
std::vector<string_t> container_key_stack {}; // NOLINT(readability-redundant-member-init)
/// helper to hold the reference for the next object element
BasicJsonType* object_element = nullptr;
/// stash of (slot pointer, previous value) for object members that
/// already existed when key() was called again for the same key
/// (duplicate keys); used to restore the previous value if the new
/// value is later rejected by the callback, instead of erasing the
/// member entirely
std::vector<std::pair<BasicJsonType*, BasicJsonType>> duplicate_key_stash {};
/// whether a syntax error occurred
bool errored = false;
/// callback function
+10 -3
View File
@@ -952,7 +952,9 @@ class lexer : public lexer_base<BasicJsonType>
case '\n':
case '\r':
case char_traits<char_type>::eof():
#if !JSON_STRICT_NUL_HANDLING
case '\0':
#endif
return true;
default:
@@ -970,8 +972,10 @@ class lexer : public lexer_base<BasicJsonType>
{
switch (get())
{
case char_traits<char_type>::eof():
#if !JSON_STRICT_NUL_HANDLING
case '\0':
#endif
case char_traits<char_type>::eof():
{
error_message = "invalid comment; missing closing '*/'";
return false;
@@ -2153,9 +2157,12 @@ scan_number_done:
case '9':
return scan_number_dispatch(std::integral_constant<bool, bulk_scan> {});
// end of input (the null byte is needed when parsing from
// string literals)
#if !JSON_STRICT_NUL_HANDLING
case '\0':
#endif
// end of input; by default, a null byte is also treated as end of
// input for backwards compatibility (see JSON_STRICT_NUL_HANDLING
// to opt into rejecting a null byte in the input instead)
case char_traits<char_type>::eof():
return token_type::end_of_input;
@@ -88,8 +88,13 @@ class iter_impl // NOLINT(cppcoreguidelines-special-member-functions,hicpp-speci
iter_impl() = default;
~iter_impl() = default;
iter_impl(iter_impl&&) noexcept = default;
iter_impl& operator=(iter_impl&&) noexcept = default;
// the exception specification is left to be computed rather than declared:
// an array or object type whose iterator is not nothrow move constructible
// (std::deque's is not before libstdc++ 11) would make a declared noexcept
// differ from the implicit one, which deletes the function -- and is an
// error outright with older compilers
iter_impl(iter_impl&&) = default; // NOLINT(hicpp-noexcept-move,performance-noexcept-move-constructor,cppcoreguidelines-noexcept-move-operations)
iter_impl& operator=(iter_impl&&) = default; // NOLINT(hicpp-noexcept-move,performance-noexcept-move-constructor,cppcoreguidelines-noexcept-move-operations)
/*!
@brief constructor for a given JSON instance
+47 -8
View File
@@ -17,6 +17,7 @@
#endif // JSON_NO_IO
#include <limits> // max
#include <numeric> // accumulate
#include <set> // set
#include <string> // string
#include <utility> // move
#include <vector> // vector
@@ -71,7 +72,7 @@ class json_pointer
string_t{},
[](const string_t& a, const string_t& b)
{
return detail::concat(a, '/', detail::escape(b));
return detail::concat<string_t>(a, '/', detail::escape(b));
});
}
@@ -265,7 +266,7 @@ class json_pointer
JSON_THROW(detail::parse_error::create(109, 0, detail::concat("array index '", s, "' is not a number"), nullptr));
}
const char* p = s.c_str();
const char* p = s.data();
char* p_end = nullptr; // NOLINT(misc-const-correctness)
errno = 0; // strtoull doesn't reset errno
const unsigned long long res = std::strtoull(p, &p_end, 10); // NOLINT(runtime/int)
@@ -300,19 +301,35 @@ class json_pointer
}
private:
/*!
@brief the reference token sequences that denote arrays
@ref unflatten collects the pointer prefixes that have a reference token 0
among their children; @ref get_and_create creates arrays exactly below
those prefixes and objects everywhere else. Deciding this up front keeps
the result independent of the order in which the flattened object is
iterated, which is unspecified for some object types.
*/
using array_parents_t = std::set<std::vector<string_t>>;
/*!
@brief create and return a reference to the pointed to value
@complexity Linear in the number of reference tokens.
@throw parse_error.106 if an array index begins with '0'
@throw parse_error.109 if array index is not a number
@throw type_error.313 if value cannot be unflattened
*/
template<typename BasicJsonType>
BasicJsonType& get_and_create(BasicJsonType& j) const
BasicJsonType& get_and_create(BasicJsonType& j, const array_parents_t& array_parents) const
{
auto* result = &j;
// the reference tokens that have been consumed so far; used to look up
// whether the value to be created below is an array or an object
std::vector<string_t> prefix;
// in case no reference tokens exist, return a reference to the JSON value
// j which will be overwritten by a primitive value
for (const auto& reference_token : reference_tokens)
@@ -321,10 +338,11 @@ class json_pointer
{
case detail::value_t::null:
{
if (reference_token == "0")
if (array_parents.find(prefix) != array_parents.end())
{
// start a new array if the reference token is 0
result = &result->operator[](0);
// some reference token below this position is 0, so the
// value is an array
result = &result->operator[](array_index<BasicJsonType>(reference_token));
}
else
{
@@ -364,6 +382,8 @@ class json_pointer
default:
JSON_THROW(detail::type_error::create(313, "invalid value to unflatten", &j));
}
prefix.push_back(reference_token);
}
return *result;
@@ -837,7 +857,8 @@ class json_pointer
{
// use the text between the beginning of the reference token
// (start) and the last slash (slash).
auto reference_token = reference_string.substr(start, slash - start);
const auto count = (slash == string_t::npos ? reference_string.size() : slash) - start;
auto reference_token = string_t(reference_string.data() + start, count);
// check reference tokens are properly escaped
for (std::size_t pos = reference_token.find_first_of('~');
@@ -953,6 +974,24 @@ class json_pointer
BasicJsonType result;
// collect the pointer prefixes that have a reference token 0 among
// their children; the values below them are arrays, all others are
// objects (see array_parents_t)
array_parents_t array_parents;
for (const auto& element : *value.m_data.m_value.object)
{
json_pointer ptr(element.first);
std::vector<string_t> prefix;
for (auto& reference_token : ptr.reference_tokens)
{
if (reference_token == "0")
{
array_parents.insert(prefix);
}
prefix.push_back(std::move(reference_token));
}
}
// iterate the JSON object values
for (const auto& element : *value.m_data.m_value.object)
{
@@ -965,7 +1004,7 @@ class json_pointer
// that if the JSON pointer is "" (i.e., points to the whole value),
// function get_and_create returns a reference to the result itself.
// An assignment will then create a primitive value.
json_pointer(element.first).get_and_create(result) = element.second;
json_pointer(element.first).get_and_create(result, array_parents) = element.second;
}
return result;
+4
View File
@@ -807,3 +807,7 @@ void templated_json_throw(ExceptionType exception)
#ifndef JSON_BRACE_INIT_COPY_SEMANTICS
#define JSON_BRACE_INIT_COPY_SEMANTICS 0
#endif
#ifndef JSON_STRICT_NUL_HANDLING
#define JSON_STRICT_NUL_HANDLING 0
#endif
@@ -27,6 +27,7 @@
#undef JSON_DISABLE_ENUM_SERIALIZATION
#undef JSON_USE_GLOBAL_UDLS
#undef JSON_BRACE_INIT_COPY_SEMANTICS
#undef JSON_STRICT_NUL_HANDLING
#ifndef JSON_TEST_KEEP_MACROS
#undef JSON_CATCH
+23 -6
View File
@@ -172,17 +172,18 @@ struct has_to_json < BasicJsonType, T, enable_if_t < !is_basic_json<T>::value >>
template<typename T>
using detect_key_compare = typename T::key_compare;
template<typename T>
struct has_key_compare : std::integral_constant<bool, is_detected<detect_key_compare, T>::value> {};
// obtains the actual object key comparator
// obtains the actual object key comparator: object_t::key_compare if the
// object type defines it, and default_object_comparator_t otherwise
//
// note detected_or_t is used rather than std::conditional, because the latter
// names both of its type arguments eagerly; object_t::key_compare would then
// be a hard error for an object type that does not define it
template<typename BasicJsonType>
struct actual_object_comparator
{
using object_t = typename BasicJsonType::object_t;
using object_comparator_t = typename BasicJsonType::default_object_comparator_t;
using type = typename std::conditional < has_key_compare<object_t>::value,
typename object_t::key_compare, object_comparator_t>::type;
using type = detected_or_t<object_comparator_t, detect_key_compare, object_t>;
};
template<typename BasicJsonType>
@@ -778,6 +779,22 @@ using has_erase_with_key_type = typename std::conditional <
std::true_type,
std::false_type >::type;
template<typename ObjectType, typename IteratorType>
using detect_erase_with_iterator = decltype(std::declval<ObjectType&>().erase(std::declval<IteratorType>()));
// type trait to check if erase(iterator) returns void instead of the following
// iterator, as the object types that do not compute a successor the caller may
// not need do
template<typename ObjectType, typename IteratorType>
using erase_returns_void = is_detected_exact<void, detect_erase_with_iterator, ObjectType, IteratorType>;
template<typename T>
using detect_capacity = decltype(std::declval<const T&>().capacity());
// type trait to check if a type has a capacity() member function
template<typename T>
struct has_capacity : std::integral_constant<bool, is_detected<detect_capacity, T>::value> {};
// a naive helper to check if a type is an ordered_map (exploits the fact that
// ordered_map inherits capacity() from std::vector)
template <typename T>
File diff suppressed because it is too large Load Diff
@@ -13,6 +13,7 @@
#include <iterator> // back_inserter
#include <memory> // shared_ptr, make_shared
#include <string> // basic_string
#include <utility> // move
#include <vector> // vector
#ifndef JSON_NO_IO
@@ -44,22 +45,32 @@ template<typename CharType> struct output_adapter_protocol
template<typename CharType>
using output_adapter_t = std::shared_ptr<output_adapter_protocol<CharType>>;
/// output adapter for byte vectors
/// @brief non-virtual output sink writing into a std::vector
///
/// This sink is not part of the virtual output_adapter_protocol hierarchy: it is
/// passed to binary_writer by value as a template parameter, so
/// write_character()/write_characters() are ordinary (inlinable) calls with no
/// vtable lookup and no shared_ptr. It is used for the common
/// `to_cbor`/`to_msgpack`/... into a std::vector. output_vector_adapter below
/// wraps this same sink to provide the virtual interface.
template<typename CharType, typename AllocatorType = std::allocator<CharType>>
class output_vector_adapter : public output_adapter_protocol<CharType>
class output_vector_sink
{
public:
explicit output_vector_adapter(std::vector<CharType, AllocatorType>& vec) noexcept
explicit output_vector_sink(std::vector<CharType, AllocatorType>& vec) noexcept
: v(vec)
{}
void write_character(CharType c) override
void write_character(CharType c)
{
v.push_back(c);
}
JSON_HEDLEY_NON_NULL(2)
void write_characters(const CharType* s, std::size_t length) override
// no JSON_HEDLEY_NON_NULL here: binary_writer legitimately passes a null
// pointer with length 0 for empty strings/binary values. Appending an empty
// range is a no-op; the type-erased path tolerates this via the (unattributed)
// virtual base, and the concrete sink must do the same.
void write_characters(const CharType* s, std::size_t length)
{
v.insert(v.end(), s, s + length);
}
@@ -68,6 +79,34 @@ class output_vector_adapter : public output_adapter_protocol<CharType>
std::vector<CharType, AllocatorType>& v;
};
/// output adapter for byte vectors
///
/// The appending itself lives in output_vector_sink; this class only adds the
/// virtual output_adapter_protocol interface on top of it, so both the
/// type-erased and the templated path share one implementation.
template<typename CharType, typename AllocatorType = std::allocator<CharType>>
class output_vector_adapter : public output_adapter_protocol<CharType>
{
public:
explicit output_vector_adapter(std::vector<CharType, AllocatorType>& vec) noexcept
: sink(vec)
{}
void write_character(CharType c) override
{
sink.write_character(c);
}
JSON_HEDLEY_NON_NULL(2)
void write_characters(const CharType* s, std::size_t length) override
{
sink.write_characters(s, length);
}
private:
output_vector_sink<CharType, AllocatorType> sink;
};
#ifndef JSON_NO_IO
/// output adapter for output streams
template<typename CharType>
@@ -118,6 +157,39 @@ class output_string_adapter : public output_adapter_protocol<CharType>
StringType& str;
};
/// @brief output sink forwarding to a type-erased output adapter
///
/// Wraps the polymorphic output_adapter_t so the same binary_writer template can
/// also target arbitrary adapters (output streams, strings, user-provided
/// adapters) via the `output_adapter`-based overloads. Each write still goes
/// through one virtual call, exactly as before; only the concrete sinks above
/// avoid it.
template<typename CharType>
class output_adapter_sink
{
public:
explicit output_adapter_sink(output_adapter_t<CharType> adapter)
: oa(std::move(adapter))
{
JSON_ASSERT(oa);
}
void write_character(CharType c)
{
oa->write_character(c);
}
// no JSON_HEDLEY_NON_NULL: forwards (null, 0) for empty payloads, exactly as
// the type-erased path already did before this sink existed
void write_characters(const CharType* s, std::size_t length)
{
oa->write_characters(s, length);
}
private:
output_adapter_t<CharType> oa;
};
template<typename CharType, typename StringType = std::basic_string<CharType>>
class output_adapter
{
+20 -6
View File
@@ -1061,7 +1061,7 @@ class serializer
{
case error_handler_t::strict:
{
JSON_THROW(type_error::create(316, concat("incomplete UTF-8 string; last byte: 0x", hex_bytes(static_cast<std::uint8_t>(s.back() | 0))), nullptr));
JSON_THROW(type_error::create(316, concat("incomplete UTF-8 string; last byte: 0x", hex_bytes(static_cast<std::uint8_t>(s[s.size() - 1] | 0))), nullptr));
}
case error_handler_t::ignore:
@@ -1322,6 +1322,19 @@ class serializer
pos += 6;
}
/*!
@brief convert a single element of a binary value to its byte value
The elements of a binary value are dumped as the numbers 0..255, regardless
of the value type of the configured BinaryType: that type may be signed
(`char`), unsigned (`std::uint8_t`), or not an integer at all
(`std::byte`), none of which @ref dump_integer can handle uniformly.
*/
static std::uint8_t to_byte_value(binary_char_t x) noexcept
{
return static_cast<std::uint8_t>(x);
}
// templates to avoid warnings about useless casts
template <typename NumberType, enable_if_t<std::is_signed<NumberType>::value, int> = 0>
bool is_negative_number(NumberType x)
@@ -1343,8 +1356,10 @@ class serializer
an arbitrary number, and the three digits it takes at most are written
straight into the write buffer.
Any byte type that is not a plain unsigned byte is left to @ref dump_integer,
whose representation of it may differ.
Any byte type that is not a plain unsigned byte is converted to its
@ref to_byte_value "byte value" and left to @ref dump_integer, so a signed
or non-integral BinaryType::value_type (`char`, `std::byte`, ...) still
dumps as 0..255.
*/
template<typename ByteType>
void dump_byte(const ByteType value)
@@ -1357,7 +1372,7 @@ class serializer
template<typename ByteType>
void dump_byte(const ByteType value, std::false_type /*is_plain_byte*/)
{
dump_integer(value);
dump_integer(to_byte_value(value));
}
template<typename ByteType>
@@ -1403,8 +1418,7 @@ class serializer
template < typename NumberType, detail::enable_if_t <
std::is_integral<NumberType>::value ||
std::is_same<NumberType, number_unsigned_t>::value ||
std::is_same<NumberType, number_integer_t>::value ||
std::is_same<NumberType, binary_char_t>::value,
std::is_same<NumberType, number_integer_t>::value,
int > = 0 >
void dump_integer(NumberType x)
{
+68 -31
View File
@@ -8,50 +8,56 @@
#pragma once
#include <cstddef> // size_t
#include <nlohmann/detail/abi_macros.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
{
/*!
@brief replace all occurrences of a substring by another string
@param[in,out] s the string to manipulate; changed so that all
occurrences of @a f are replaced with @a t
@param[in] f the substring to replace with @a t
@param[in] t the string to replace @a f
@pre The search string @a f must not be empty. **This precondition is
enforced with an assertion.**
@since version 2.0.0
*/
template<typename StringType>
inline void replace_substring(StringType& s, const StringType& f,
const StringType& t)
{
JSON_ASSERT(!f.empty());
for (auto pos = s.find(f); // find the first occurrence of f
pos != StringType::npos; // make sure f was found
s.replace(pos, f.size(), t), // replace with t, and
pos = s.find(f, pos + t.size())) // find the next occurrence of f
{}
}
/*!
* @brief string escaping as described in RFC 6901 (Sect. 4)
* @param[in] s string to escape
* @return escaped string
*
* Note the order of escaping "~" to "~0" and "/" to "~1" is important.
*
* The string is rebuilt in a single pass, appending whole runs between the
* characters that need escaping. Scanning with find_first_of() keeps the
* common case -- nothing to escape -- as fast as a single search, while
* repeated replace() calls would move the tail of the string once per
* escaped character.
*/
template<typename StringType>
inline StringType escape(StringType s)
inline StringType escape(const StringType& s)
{
replace_substring(s, StringType{"~"}, StringType{"~0"});
replace_substring(s, StringType{"/"}, StringType{"~1"});
return s;
auto next_special = [&s](std::size_t from)
{
const auto tilde = s.find_first_of('~', from);
const auto slash = s.find_first_of('/', from);
return tilde < slash ? tilde : slash; // npos is the largest value
};
auto pos = next_special(0);
if (pos == StringType::npos)
{
return s;
}
StringType result;
result.reserve(s.size() + 2);
std::size_t run = 0;
while (pos != StringType::npos)
{
result.append(s.data() + run, pos - run);
result.append(s[pos] == '~' ? "~0" : "~1", 2);
run = pos + 1;
pos = next_special(run);
}
result.append(s.data() + run, s.size() - run);
return result;
}
/*!
@@ -60,12 +66,43 @@ inline StringType escape(StringType s)
* @return unescaped string
*
* Note the order of escaping "~1" to "/" and "~0" to "~" is important.
*
* Rebuilt in a single pass, see @ref escape. A "~" that is followed by
* neither "0" nor "1" is passed through unchanged; @ref json_pointer rejects
* such input before it gets here.
*/
template<typename StringType>
inline void unescape(StringType& s)
{
replace_substring(s, StringType{"~1"}, StringType{"/"});
replace_substring(s, StringType{"~0"}, StringType{"~"});
auto pos = s.find_first_of('~', 0);
if (pos == StringType::npos)
{
return;
}
StringType result;
result.reserve(s.size());
std::size_t run = 0;
while (pos != StringType::npos)
{
result.append(s.data() + run, pos - run);
const auto next = pos + 1;
if (next < s.size() && (s[next] == '0' || s[next] == '1'))
{
result.append(s[next] == '0' ? "~" : "/", 1);
run = pos + 2;
}
else
{
result.append("~", 1);
run = pos + 1;
}
pos = s.find_first_of('~', run);
}
result.append(s.data() + run, s.size() - run);
s = result;
}
} // namespace detail
+250 -67
View File
@@ -140,7 +140,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
friend ::nlohmann::detail::serializer<basic_json>;
template<typename BasicJsonType>
friend class ::nlohmann::detail::iter_impl;
template<typename BasicJsonType, typename CharType>
template<typename BasicJsonType, typename CharType, typename OutputSinkType>
friend class ::nlohmann::detail::binary_writer;
template<typename BasicJsonType, typename InputType, typename SAX>
friend class ::nlohmann::detail::binary_reader;
@@ -188,6 +188,14 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
template<typename InputType>
using binary_reader = ::nlohmann::detail::binary_reader<basic_json, InputType>;
template<typename CharType> using binary_writer = ::nlohmann::detail::binary_writer<basic_json, CharType>;
// binary_writer over a concrete (non-virtual) sink appending into a std::vector,
// used by the vector-returning to_* overloads
template<typename CharType> using vector_binary_writer =
::nlohmann::detail::binary_writer<basic_json, CharType, ::nlohmann::detail::output_vector_sink<CharType>>;
template<typename CharType> static vector_binary_writer<CharType> vector_writer(std::vector<CharType>& v)
{
return vector_binary_writer<CharType>(::nlohmann::detail::output_vector_sink<CharType>(v));
}
JSON_PRIVATE_UNLESS_TESTED:
using serializer = ::nlohmann::detail::serializer<basic_json>;
@@ -404,6 +412,18 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
/// @}
// Two template parameter requirements that would otherwise be silently
// violated: neither produces a diagnostic of its own, and both corrupt
// values rather than failing.
static_assert(sizeof(typename BinaryType::value_type) == 1,
"BinaryType::value_type must be exactly one byte wide, "
"because the binary readers and writers reinterpret the container's storage as raw bytes");
static_assert(sizeof(NumberUnsignedType) >= sizeof(NumberIntegerType),
"NumberUnsignedType must be at least as wide as NumberIntegerType, "
"because it has to hold the absolute value of every NumberIntegerType value");
private:
/// helper for exception-safe object creation
@@ -784,21 +804,76 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
return it;
}
reference set_parent(reference j, std::size_t old_capacity = detail::unknown_size())
/// @brief erase an element from the object and return the following one
/// Not every map returns an iterator from erase(iterator): some containers
/// (e.g., Abseil's hash maps) return void to avoid computing a successor
/// the caller may not need. Compute it before erasing for those.
template < typename It, detail::enable_if_t <
!detail::erase_returns_void<object_t, It>::value, int > = 0 >
typename object_t::iterator erase_from_object(It pos)
{
return m_data.m_value.object->erase(pos);
}
template < typename It, detail::enable_if_t <
detail::erase_returns_void<object_t, It>::value, int > = 0 >
typename object_t::iterator erase_from_object(It pos)
{
auto next = std::next(pos);
m_data.m_value.object->erase(pos);
return next;
}
/// @brief the capacity of the stored array, or unknown_size()
/// Only JSON_DIAGNOSTICS uses the value, to detect a reallocation that
/// would invalidate the parent pointers. Array types that do not have a
/// capacity() member function report unknown_size(), which is treated as
/// "the elements may have moved".
#if JSON_DIAGNOSTICS
template < typename A = array_t, detail::enable_if_t < detail::has_capacity<A>::value, int > = 0 >
std::size_t array_capacity() const noexcept
{
return m_data.m_value.array->capacity();
}
template < typename A = array_t, detail::enable_if_t < !detail::has_capacity<A>::value, int > = 0 >
std::size_t array_capacity() const noexcept
{
return detail::unknown_size();
}
#else
static constexpr std::size_t array_capacity() noexcept
{
return detail::unknown_size();
}
#endif
/// @brief set the parent of a value that has just been added to an array
/// @param j the added value
/// @param old_capacity the value @ref array_capacity() returned before the
/// insertion
reference set_parent_after_array_insert(reference j, std::size_t old_capacity)
{
#if JSON_DIAGNOSTICS
if (old_capacity != detail::unknown_size())
// see https://github.com/nlohmann/json/issues/2838
JSON_ASSERT(type() == value_t::array);
if (JSON_HEDLEY_UNLIKELY(old_capacity == detail::unknown_size()
|| array_capacity() != old_capacity))
{
// see https://github.com/nlohmann/json/issues/2838
JSON_ASSERT(type() == value_t::array);
if (JSON_HEDLEY_UNLIKELY(m_data.m_value.array->capacity() != old_capacity))
{
// capacity has changed: update all parents
set_parents();
return j;
}
// the capacity has changed, or the array type does not let us tell:
// the elements may have moved, so update all parents
set_parents();
return j;
}
#else
static_cast<void>(old_capacity);
#endif
return set_parent(j);
}
reference set_parent(reference j)
{
#if JSON_DIAGNOSTICS
// ordered_json uses a vector internally, so pointers could have
// been invalidated; see https://github.com/nlohmann/json/issues/2962
#ifdef JSON_HEDLEY_MSVC_VERSION
@@ -817,7 +892,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
j.m_parent = this;
#else
static_cast<void>(j);
static_cast<void>(old_capacity);
#endif
return j;
}
@@ -2029,22 +2103,17 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
reference at(size_type idx)
{
// at only works for arrays
if (JSON_HEDLEY_LIKELY(is_array()))
{
JSON_TRY
{
return set_parent(m_data.m_value.array->at(idx));
}
JSON_CATCH (std::out_of_range&)
{
// create a better exception explanation
JSON_THROW(out_of_range::create(401, detail::concat("array index ", std::to_string(idx), " is out of range"), this));
} // cppcheck-suppress[missingReturn]
}
else
if (JSON_HEDLEY_UNLIKELY(!is_array()))
{
JSON_THROW(type_error::create(304, detail::concat("cannot use at() with ", type_name()), this));
}
if (JSON_HEDLEY_UNLIKELY(idx >= m_data.m_value.array->size()))
{
JSON_THROW(out_of_range::create(401, detail::concat("array index ", std::to_string(idx), " is out of range"), this));
}
return set_parent((*m_data.m_value.array)[idx]);
}
/// @brief access specified array element with bounds checking
@@ -2052,22 +2121,17 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
const_reference at(size_type idx) const
{
// at only works for arrays
if (JSON_HEDLEY_LIKELY(is_array()))
{
JSON_TRY
{
return m_data.m_value.array->at(idx);
}
JSON_CATCH (std::out_of_range&)
{
// create a better exception explanation
JSON_THROW(out_of_range::create(401, detail::concat("array index ", std::to_string(idx), " is out of range"), this));
} // cppcheck-suppress[missingReturn]
}
else
if (JSON_HEDLEY_UNLIKELY(!is_array()))
{
JSON_THROW(type_error::create(304, detail::concat("cannot use at() with ", type_name()), this));
}
if (JSON_HEDLEY_UNLIKELY(idx >= m_data.m_value.array->size()))
{
JSON_THROW(out_of_range::create(401, detail::concat("array index ", std::to_string(idx), " is out of range"), this));
}
return (*m_data.m_value.array)[idx];
}
/// @brief access specified object element with bounds checking
@@ -2167,12 +2231,13 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
#if JSON_DIAGNOSTICS
// remember array size & capacity before resizing
const auto old_size = m_data.m_value.array->size();
const auto old_capacity = m_data.m_value.array->capacity();
const auto old_capacity = array_capacity();
#endif
m_data.m_value.array->resize(idx + 1);
#if JSON_DIAGNOSTICS
if (JSON_HEDLEY_UNLIKELY(m_data.m_value.array->capacity() != old_capacity))
if (JSON_HEDLEY_UNLIKELY(old_capacity == detail::unknown_size()
|| array_capacity() != old_capacity))
{
// capacity has changed: update all parents
set_parents();
@@ -2563,7 +2628,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
case value_t::object:
{
result.m_it.object_iterator = m_data.m_value.object->erase(pos.m_it.object_iterator);
result.m_it.object_iterator = erase_from_object(pos.m_it.object_iterator);
break;
}
@@ -3202,9 +3267,9 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
}
// add the element to the array (move semantics)
const auto old_capacity = m_data.m_value.array->capacity();
const auto old_capacity = array_capacity();
m_data.m_value.array->push_back(std::move(val));
set_parent(m_data.m_value.array->back(), old_capacity);
set_parent_after_array_insert(m_data.m_value.array->back(), old_capacity);
// if val is moved from, basic_json move constructor marks it null, so we do not call the destructor
}
@@ -3235,9 +3300,9 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
}
// add the element to the array
const auto old_capacity = m_data.m_value.array->capacity();
const auto old_capacity = array_capacity();
m_data.m_value.array->push_back(val);
set_parent(m_data.m_value.array->back(), old_capacity);
set_parent_after_array_insert(m_data.m_value.array->back(), old_capacity);
}
/// @brief add an object to an array
@@ -3323,9 +3388,9 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
}
// add the element to the array (perfect forwarding)
const auto old_capacity = m_data.m_value.array->capacity();
const auto old_capacity = array_capacity();
m_data.m_value.array->emplace_back(std::forward<Args>(args)...);
return set_parent(m_data.m_value.array->back(), old_capacity);
return set_parent_after_array_insert(m_data.m_value.array->back(), old_capacity);
}
/// @brief add an object to an object if key does not exist
@@ -3404,7 +3469,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
/// @sa https://json.nlohmann.me/api/basic_json/insert/
iterator insert(const_iterator pos, basic_json&& val) // NOLINT(performance-unnecessary-value-param)
{
return insert(pos, val);
return insert(std::move(pos), val);
}
/// @brief inserts copies of element into array
@@ -3454,6 +3519,12 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
JSON_THROW(invalid_iterator::create(211, "passed iterators may not belong to container", this));
}
// passed iterators must belong to arrays
if (JSON_HEDLEY_UNLIKELY(!first.m_object->is_array()))
{
JSON_THROW(invalid_iterator::create(202, "iterators first and last must point to arrays", this));
}
// insert to array and return iterator
return insert_iterator(pos, first.m_it.array_iterator, last.m_it.array_iterator);
}
@@ -3607,6 +3678,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
{
using std::swap;
swap(*(m_data.m_value.array), other);
set_parents();
}
else
{
@@ -3623,6 +3695,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
{
using std::swap;
swap(*(m_data.m_value.object), other);
set_parents();
}
else
{
@@ -4379,7 +4452,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
static std::vector<std::uint8_t> to_cbor(const basic_json& j)
{
std::vector<std::uint8_t> result;
to_cbor(j, result);
result.reserve(detail::binary_reserve_hint(j));
vector_writer(result).write_cbor(j);
return result;
}
@@ -4402,7 +4476,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
static std::vector<std::uint8_t> to_msgpack(const basic_json& j)
{
std::vector<std::uint8_t> result;
to_msgpack(j, result);
result.reserve(detail::binary_reserve_hint(j));
vector_writer(result).write_msgpack(j);
return result;
}
@@ -4427,7 +4502,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
const bool use_type = false)
{
std::vector<std::uint8_t> result;
to_ubjson(j, result, use_size, use_type);
result.reserve(detail::binary_reserve_hint(j));
vector_writer(result).write_ubjson(j, use_size, use_type);
return result;
}
@@ -4455,7 +4531,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
const bjdata_version_t version = bjdata_version_t::draft2)
{
std::vector<std::uint8_t> result;
to_bjdata(j, result, use_size, use_type, version);
result.reserve(detail::binary_reserve_hint(j));
vector_writer(result).write_ubjson(j, use_size, use_type, true, true, version);
return result;
}
@@ -4482,7 +4559,8 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
static std::vector<std::uint8_t> to_bson(const basic_json& j)
{
std::vector<std::uint8_t> result;
to_bson(j, result);
result.reserve(detail::binary_reserve_hint(j));
vector_writer(result).write_bson(j);
return result;
}
@@ -5273,34 +5351,139 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
case value_t::object:
{
// first pass: traverse this object's elements
// first pass: record, for every source key, whether it is
// common to both objects (in source's iteration order) or
// was deleted (i.e., in source but not in target) -- this is
// a by-product of the target.find() call already needed to
// tell the two cases apart, so it adds no extra lookups. The
// "remove" ops themselves are emitted later, interleaved
// with the recursive per-key diffs in the fast path below,
// to match source's original iteration order (as the
// original, pre-reordering-aware implementation did) instead
// of grouping all removes before all recursive diffs.
std::vector<typename object_t::key_type> common_keys_source_order;
for (auto it = source.cbegin(); it != source.cend(); ++it)
{
// escape the key name to be used in a JSON patch
const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
if (target.find(it.key()) != target.end())
{
// recursive call to compare object values at key it
auto temp_diff = diff(it.value(), target[it.key()], path_key);
result.insert(result.end(), temp_diff.begin(), temp_diff.end());
common_keys_source_order.push_back(it.key());
}
}
// second pass: find keys that were added (i.e., in target but
// not in source), and record the keys common to both, in
// target's iteration order -- again a by-product of the
// source.find() call already needed to detect added keys. At
// the same time, determine whether every added key comes
// after every common key in target's order (a precondition
// for the fast path below, which only ever appends new keys
// at the very end): for an object_t whose iteration order is
// a pure function of the key set (e.g. the default std::map,
// which always iterates in sorted key order), the order
// check further below is always true and this whole
// mechanism is effectively a no-op; it only matters for a
// reorderable object_t such as the one backing `ordered_json`.
// patch ops for keys that were added (i.e., in target but not
// in source); built here so the fast path below can reuse
// them without a second source.find() per target key. Only
// used by the fast path -- the slow (reordering) path
// rebuilds "add" ops for every key itself.
std::vector<typename object_t::key_type> common_keys_target_order;
basic_json added_ops(value_t::array);
bool new_keys_form_suffix = true;
bool seen_new_key = false;
for (auto it = target.cbegin(); it != target.cend(); ++it)
{
if (source.find(it.key()) == source.end())
{
seen_new_key = true;
const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
added_ops.push_back(
{
{"op", "add"}, {"path", path_key},
{"value", it.value()}
});
}
else
{
// found a key that is not in o -> remove it
common_keys_target_order.push_back(it.key());
if (seen_new_key)
{
new_keys_form_suffix = false;
}
}
}
if (common_keys_source_order == common_keys_target_order && new_keys_form_suffix)
{
// fast path: order of common keys already matches (or the
// object_t's iteration order does not depend on
// insertion history), so a plain per-key recursive diff
// is correct and minimal, as before. common_keys_source_order
// is, by construction, the subsequence of source's keys
// that are common to both objects, in source's iteration
// order -- so it can be walked in lockstep with `source`
// using a cheap key comparison instead of another lookup.
// Deleted keys (those source keys not in common_keys_source_order)
// are interleaved here too, in source's original order, to
// match the historical (pre-reordering-aware) output order.
auto common_it = common_keys_source_order.cbegin();
for (auto it = source.cbegin(); it != source.cend(); ++it)
{
if (common_it != common_keys_source_order.cend() && it.key() == *common_it)
{
const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
auto temp_diff = diff(it.value(), target[it.key()], path_key);
result.insert(result.end(), temp_diff.begin(), temp_diff.end());
++common_it;
}
else
{
// found a key that is not in target -> remove it
const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
result.push_back(object(
{
{"op", "remove"}, {"path", path_key}
}));
}
}
// append the "add" ops for brand-new keys collected above
// during the pass over target -- no second source.find()
// per target key needed
result.insert(result.end(), added_ops.begin(), added_ops.end());
}
else
{
// slow path: the common keys are in a different relative
// order in source and target (only possible for a
// reorderable object_t like ordered_map). Building a
// minimal reordering patch is a nontrivial (LCS-like)
// problem; instead, remove every source key -- both
// deleted keys (which must be removed regardless) and
// common keys (removed so they can be re-added in
// target's order) -- and re-add every key that should
// remain, with its final target value, in target's
// order. basic_json::patch()'s "add" operation on an
// object uses operator[], which appends at the end for a
// vector-backed insertion-ordered map when the key does
// not already exist -- so removing a key and then adding
// it moves it to the end, fixing its position.
for (auto it = source.cbegin(); it != source.cend(); ++it)
{
const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
result.push_back(object(
{
{"op", "remove"}, {"path", path_key}
}));
}
}
// second pass: traverse other object's elements
for (auto it = target.cbegin(); it != target.cend(); ++it)
{
if (source.find(it.key()) == source.end())
// add every key that is either common (just removed
// above) or brand new, in target's iteration order, so
// that the final order after applying the patch matches
// target exactly
for (auto it = target.cbegin(); it != target.cend(); ++it)
{
// found a key that is not in this -> add it
const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
result.push_back(
{
File diff suppressed because it is too large Load Diff
+21 -15
View File
@@ -1,4 +1,4 @@
cmake_minimum_required(VERSION 3.11...3.14)
cmake_minimum_required(VERSION 3.14)
project(JSON_Benchmarks LANGUAGES CXX)
# set compiler flags
@@ -6,29 +6,35 @@ if((CMAKE_CXX_COMPILER_ID MATCHES GNU) OR (CMAKE_CXX_COMPILER_ID MATCHES Clang))
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -flto -DNDEBUG -O3")
endif()
# configure Google Benchmarks
# configure Google Benchmark; a fixed release, so that results stay comparable
set(JSON_GOOGLE_BENCHMARK_VERSION 1.9.5)
include(FetchContent)
FetchContent_Declare(
benchmark
GIT_REPOSITORY https://github.com/google/benchmark.git
GIT_TAG origin/main
GIT_SHALLOW TRUE
)
FetchContent_GetProperties(benchmark)
if(NOT benchmark_POPULATED)
FetchContent_Populate(benchmark)
set(BENCHMARK_ENABLE_TESTING OFF CACHE INTERNAL "" FORCE)
add_subdirectory(${benchmark_SOURCE_DIR} ${benchmark_BINARY_DIR})
endif()
# only the library is needed; -Werror would break the pinned release as soon as
# a newer compiler adds a warning
set(BENCHMARK_ENABLE_TESTING OFF CACHE BOOL "" FORCE)
set(BENCHMARK_ENABLE_INSTALL OFF CACHE BOOL "" FORCE)
set(BENCHMARK_ENABLE_WERROR OFF CACHE BOOL "" FORCE)
FetchContent_Declare(benchmark
URL https://github.com/google/benchmark/archive/refs/tags/v${JSON_GOOGLE_BENCHMARK_VERSION}.tar.gz
URL_HASH SHA256=9631341c82bac4a288bef951f8b26b41f69021794184ece969f8473977eaa340
DOWNLOAD_EXTRACT_TIMESTAMP TRUE
)
FetchContent_MakeAvailable(benchmark)
# download test data
set(CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/../../cmake ${CMAKE_MODULE_PATH})
include(download_test_data)
# the header to benchmark; point this at a directory holding another version's
# nlohmann/json.hpp to compare versions (see README.md)
set(JSON_BENCHMARK_INCLUDE_DIR "${CMAKE_CURRENT_SOURCE_DIR}/../../single_include" CACHE PATH
"directory containing the nlohmann/json.hpp to benchmark")
# benchmark binary
add_executable(json_benchmarks src/benchmarks.cpp)
target_compile_features(json_benchmarks PRIVATE cxx_std_11)
target_link_libraries(json_benchmarks benchmark ${CMAKE_THREAD_LIBS_INIT})
add_dependencies(json_benchmarks download_test_data)
target_include_directories(json_benchmarks PRIVATE ${CMAKE_SOURCE_DIR}/../../single_include ${CMAKE_BINARY_DIR}/include)
target_include_directories(json_benchmarks PRIVATE ${JSON_BENCHMARK_INCLUDE_DIR} ${CMAKE_BINARY_DIR}/include)
+130
View File
@@ -0,0 +1,130 @@
# Benchmarks
Micro-benchmarks for parsing, serialization and the binary formats, written with
[Google Benchmark](https://github.com/google/benchmark). They are not run by CI; see
[When to run them](#when-to-run-them).
## What is measured
| benchmark | what it does |
|---|---|
| `ParseFile`, `ParseString` | parse JSON from a file stream or a string |
| `ParseIndented` | parse the large files re-indented by 4 spaces, for the lexer's whitespace handling |
| `Dump` | serialize, compact (`-`) and indented (`4`) |
| `ToCbor`, `BinaryToCbor` | write CBOR; `BinaryToCbor` writes binary values of growing size |
| `FromMsgpack` | read MessagePack; unchanged over the years, so its numbers stay comparable across releases |
| `FromBinaryBuffer`, `FromBinaryFile` | read CBOR, MessagePack, UBJSON, BJData and BSON from a buffer or a `FILE*` |
| `FromBinaryShape` | read deeply nested, container-heavy and scalar-heavy documents in every binary format |
| `FromCborChunkedString` | read CBOR strings split into indefinite-length chunks |
The input files are those of [nativejson-benchmark](https://github.com/miloyip/nativejson-benchmark) (`canada`,
`citm_catalog`, `twitter`), a large `jeopardy` file, and number-heavy files (`floats`, `signed_ints`, ...).
`bytes_per_second` counts the bytes read or written: the JSON text when parsing, the output when serializing.
## Requirements
- CMake 3.14 or later, a C++11 compiler, and Ninja for the `make` target.
- Network access on the first configure: CMake downloads Google Benchmark and the
[test data](https://github.com/nlohmann/json_test_data) into the build directory. To reuse a download of the test
data, pass `-DJSON_TestDataDirectory=<build directory>/test_files`.
- Google Benchmark is pinned to a release (1.9.5), so that results from different days stay comparable. To update it,
change `JSON_GOOGLE_BENCHMARK_VERSION` and the archive's `URL_HASH` in `CMakeLists.txt` together.
- The benchmarks include `single_include/nlohmann/json.hpp`, so run `make amalgamate` after changing anything in
`include/`.
GCC and Clang builds use `-O3 -flto -DNDEBUG`.
## Running them
From the repository root, this builds everything from scratch in `cmake-build-benchmarks` and runs all benchmarks:
```sh
make run_benchmarks
```
To build once and run selectively:
```sh
cmake -S tests/benchmarks -B build-benchmarks -G Ninja -DCMAKE_BUILD_TYPE=Release
cmake --build build-benchmarks
build-benchmarks/json_benchmarks --benchmark_filter='ParseString|Dump'
```
Useful options of `json_benchmarks`:
| option | effect |
|---|---|
| `--benchmark_list_tests` | list the benchmarks instead of running them |
| `--benchmark_filter=<regex>` | run only the benchmarks whose names match |
| `--benchmark_repetitions=<n>` | run every benchmark `n` times and add mean, median, standard deviation and coefficient of variation |
| `--benchmark_enable_random_interleaving=true` | run the repetitions in random order, which spreads out drifts such as thermal throttling |
| `--benchmark_min_time=<seconds>s` | run each benchmark at least this long (e.g. `2s`) |
| `--benchmark_out=<file> --benchmark_out_format=json` | also write the results to a file, e.g. for `compare.py` |
## Reading the output
Each line shows the wall-clock `Time` and the `CPU` time per iteration, the number of `Iterations` Google Benchmark
chose, and the throughput in `bytes_per_second`. With repetitions, the lines ending in `_median` are the ones to
compare. A `_cv` (coefficient of variation) above a few percent means the machine was too noisy for small
differences to mean anything.
## Comparing two versions
To see what a change or a release did, build the same benchmarks twice: once against the header of the version to
compare with, and once against the current one. `JSON_BENCHMARK_INCLUDE_DIR` names the directory holding the
`nlohmann/json.hpp` to benchmark. For example, to compare the current checkout with 3.12.0:
```sh
# the header of the version to compare with
mkdir -p build-baseline-header/nlohmann
git show v3.12.0:single_include/nlohmann/json.hpp > build-baseline-header/nlohmann/json.hpp
# the same benchmarks, built against either header
cmake -S tests/benchmarks -B build-baseline -G Ninja -DCMAKE_BUILD_TYPE=Release \
-DJSON_BENCHMARK_INCLUDE_DIR="$PWD/build-baseline-header"
cmake -S tests/benchmarks -B build-current -G Ninja -DCMAKE_BUILD_TYPE=Release
cmake --build build-baseline
cmake --build build-current
# run both, back to back
build-baseline/json_benchmarks --benchmark_repetitions=10 --benchmark_enable_random_interleaving=true \
--benchmark_out=build-baseline/results.json --benchmark_out_format=json
build-current/json_benchmarks --benchmark_repetitions=10 --benchmark_enable_random_interleaving=true \
--benchmark_out=build-current/results.json --benchmark_out_format=json
```
Google Benchmark ships a tool to compare the two result files. It needs NumPy and SciPy:
```sh
python3 -m venv build-venv
build-venv/bin/pip install numpy scipy
build-venv/bin/python build-current/_deps/benchmark-src/tools/compare.py -a benchmarks build-baseline/results.json build-current/results.json
```
The tool's own `tools/requirements.txt` pins NumPy and SciPy versions that need Python 3.11 or later; with an older
Python, unpinned versions work as well. In its output:
- the `Time` and `CPU` columns are relative changes: `-0.35` means 35% faster, `+0.10` means 10% slower;
- `_pvalue` lines report a Mann-Whitney U test of whether the two versions differ. It needs at least 9
repetitions, and a p-value below 0.05 means the difference is unlikely to be noise;
- `OVERALL_GEOMEAN` summarizes all benchmarks;
- `-a` shows only the aggregates, not every repetition.
The header you compare with must support everything the benchmarks use. The current benchmarks build against 3.12.0.
Only benchmarks present in both result files are compared, so for older releases, either filter the benchmarks or
build that release's own `tests/benchmarks` against its own header.
## Getting stable numbers
- Build and run both versions on the same machine, one right after the other.
- Keep the machine otherwise idle: no builds, no browser, and a laptop plugged in.
- On Linux, set the CPU frequency governor to `performance`, e.g. `sudo cpupower frequency-set --governor performance`.
Google Benchmark prints a warning when frequency scaling is enabled. Pinning the process to a core
(`taskset -c 2 ...`) helps as well.
- Use 10 or more repetitions with random interleaving, compare medians, and treat changes within the `_cv` as noise.
## When to run them
They are a manual step, not part of CI: shared CI runners vary more between runs than most of the effects measured.
Run the comparison above before a release, comparing the previous release tag with `develop`, and for pull requests
that claim to change performance.
+2 -1
View File
@@ -131,7 +131,8 @@ static void Dump(benchmark::State& state, const char* filename, int indent)
while (state.KeepRunning())
{
j.dump(indent);
std::string output = j.dump(indent);
benchmark::DoNotOptimize(output);
}
state.SetBytesProcessed(state.iterations() * j.dump(indent).size());
+34 -4
View File
@@ -21,6 +21,27 @@ array data, it performs the following steps:
- j4 = from_bjdata(vec3)
- assert(j1 == j4)
Re-serializing j2/j3/j4 with the same use_size/use_type settings is checked
for value-stability rather than byte-exact stability: from_bjdata(to_bjdata(j2))
must equal j2 (and likewise for j3, j4). Byte-exact stability does not hold in
general, because a BJData value can lose type fidelity across a round trip
(e.g. a binary_t value serialized without the optimized "$U#" array header is
parsed back as a plain array of numbers, see #5398 and the discussion on
PR #5494) - the numeric value is preserved, but the writer's smallest-type
selection for the now-plain numbers may legitimately pick a different, but
equally valid, single-byte type marker than the dedicated binary-data writer
would have. Both encodings are valid BJData and both decode to the same
value, so this is not treated as a round-trip failure here.
"Value-stable" is checked by comparing dump()s rather than with operator==
directly: a BJData/UBJSON payload can decode to a non-finite double (NaN or
+-Infinity), and IEEE 754 NaN is never equal to itself, so operator== would
report two structurally-identical trees as different whenever a NaN is
involved -- not a round-trip bug, just NaN's ordinary (non-)reflexivity.
dump() serializes any non-finite double the same deterministic way (as JSON
`null`, since JSON itself cannot represent NaN/Infinity), so comparing
dumps is stable under exactly the same values that break operator==.
The provided function `LLVMFuzzerTestOneInput` can be used in different fuzzer
drivers.
*/
@@ -31,6 +52,13 @@ drivers.
using json = nlohmann::json;
// value-stable comparison for the round-trip checks below; see the note
// above on why this compares dump()s rather than the json values directly
static bool is_value_stable(const json& lhs, const json& rhs)
{
return lhs.dump() == rhs.dump();
}
// see http://llvm.org/docs/LibFuzzer.html
extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
{
@@ -56,10 +84,12 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
json const j3 = json::from_bjdata(vec3);
json const j4 = json::from_bjdata(vec4);
// serializations must match
assert(json::to_bjdata(j2, false, false) == vec2);
assert(json::to_bjdata(j3, true, false) == vec3);
assert(json::to_bjdata(j4, true, true) == vec4);
// re-serializing must be value-stable (see the notes above on
// why byte-exact stability is not guaranteed in general, and
// why this compares dump()s rather than the values directly)
assert(is_value_stable(json::from_bjdata(json::to_bjdata(j2, false, false)), j2));
assert(is_value_stable(json::from_bjdata(json::to_bjdata(j3, true, false)), j3));
assert(is_value_stable(json::from_bjdata(json::to_bjdata(j4, true, true)), j4));
}
catch (const json::parse_error&)
{
+61
View File
@@ -0,0 +1,61 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
// Standalone compile-and-run check for the JSON_SKIP_LIBRARY_VERSION_CHECK
// configuration macro, which (per #5423) was never exercised anywhere in the
// test matrix.
//
// include/nlohmann/detail/abi_macros.hpp normally emits a #warning if
// NLOHMANN_JSON_VERSION_MAJOR/MINOR/PATCH are already defined (as they would
// be by an earlier inclusion of a different version of the library) with
// values that mismatch the version about to be defined -- unless
// JSON_SKIP_LIBRARY_VERSION_CHECK is defined, in which case the check (and
// that #warning) is skipped.
//
// This file deliberately is not named tests/src/unit-*.cpp: it is compiled
// directly (with a modest, non-strict warning set) by the dedicated
// ci_test_skiplibraryversioncheck target in cmake/ci.cmake, rather than being
// folded into the library's own -Weverything/-Werror unit test matrix. That
// is because the scenario simulated here -- mixing two different, already
// differently-versioned inclusions of the library in one translation unit --
// unavoidably also triggers the *compiler's own* "macro redefined" warning,
// independent of (and unaffected by) JSON_SKIP_LIBRARY_VERSION_CHECK, which
// only ever silences the library's own #warning. Building this file under
// -Weverything -Werror would therefore fail for a reason unrelated to the
// macro under test.
#define NLOHMANN_JSON_VERSION_MAJOR 0
#define NLOHMANN_JSON_VERSION_MINOR 0
#define NLOHMANN_JSON_VERSION_PATCH 0
#define JSON_SKIP_LIBRARY_VERSION_CHECK 1
#include <nlohmann/json.hpp>
int main()
{
// reaching this point at all already proves that the mismatched,
// pre-defined version macros above did not stop compilation -- which is
// exactly what JSON_SKIP_LIBRARY_VERSION_CHECK is for. The library must
// also still be fully usable.
const nlohmann::json j = {{"a", 1}, {"b", {1, 2, 3}}};
if (j.dump() != "{\"a\":1,\"b\":[1,2,3]}")
{
return 1;
}
// include/nlohmann/detail/abi_macros.hpp unconditionally (re)defines the
// version macros to the library's real, current version right after the
// (here, skipped) mismatch check, regardless of the deliberately wrong
// stand-in values defined above.
if (NLOHMANN_JSON_VERSION_MAJOR == 0 && NLOHMANN_JSON_VERSION_MINOR == 0 && NLOHMANN_JSON_VERSION_PATCH == 0)
{
return 1;
}
return 0;
}
+40 -32
View File
@@ -11,8 +11,10 @@
#include <nlohmann/json.hpp>
#include <cstdint>
#include <string>
#include <utility>
#include <vector>
/* forward declarations */
class alt_string;
@@ -22,6 +24,10 @@ void int_to_string(alt_string& target, std::size_t value); // NOLINT(misc-use-in
/*
* This is virtually a string class.
* It covers std::string under the hood.
*
* It deliberately does not provide c_str(), back(), find(str, pos), replace(),
* or substr(): the library must not rely on them. Do not add members here
* without checking that the library actually needs them.
*/
class alt_string
{
@@ -106,11 +112,6 @@ class alt_string
return str_impl < op.str_impl;
}
const char* c_str() const
{
return str_impl.c_str();
}
char& operator[](std::size_t index)
{
return str_impl[index];
@@ -121,16 +122,6 @@ class alt_string
return str_impl[index];
}
char& back()
{
return str_impl.back();
}
const char& back() const
{
return str_impl.back();
}
void clear()
{
str_impl.clear();
@@ -146,28 +137,11 @@ class alt_string
return str_impl.empty();
}
std::size_t find(const alt_string& str, std::size_t pos = 0) const
{
return str_impl.find(str.str_impl, pos);
}
std::size_t find_first_of(char c, std::size_t pos = 0) const
{
return str_impl.find_first_of(c, pos);
}
alt_string substr(std::size_t pos = 0, std::size_t count = npos) const
{
const std::string s = str_impl.substr(pos, count);
return {s.data(), s.size()};
}
alt_string& replace(std::size_t pos, std::size_t count, const alt_string& str)
{
str_impl.replace(pos, count, str.str_impl);
return *this;
}
void reserve( std::size_t new_cap = 0 )
{
str_impl.reserve(new_cap);
@@ -202,6 +176,31 @@ bool operator<(const char* op1, const alt_string& op2) noexcept
TEST_CASE("alternative string type")
{
SECTION("binary formats")
{
alt_json doc;
doc["pi"] = 3.141;
doc["happy"] = true;
doc["list"] = {1, 2, 3};
CHECK(alt_json::from_cbor(alt_json::to_cbor(doc)) == doc);
CHECK(alt_json::from_msgpack(alt_json::to_msgpack(doc)) == doc);
// BSON is not covered: it additionally needs string_t::find(value_type),
// which alt_string does not provide
CHECK(alt_json::from_ubjson(alt_json::to_ubjson(doc)) == doc);
// a UBJSON high-precision number is parsed into a std::string that the
// reader has to hand to the SAX interface as an alt_string
const std::vector<uint8_t> high_precision =
{
'H', 'i', 0x16, '3', '.', '1', '4', '1', '5', '9', '2', '6', '5', '3',
'5', '8', '9', '7', '9', '3', '2', '3', '8', '4', '6'
};
const auto number = alt_json::from_ubjson(high_precision);
CHECK(number.is_number_float());
CHECK(number.get<double>() == doctest::Approx(3.14159265358979323846));
}
SECTION("dump")
{
{
@@ -332,6 +331,15 @@ TEST_CASE("alternative string type")
CHECK(j.at(alt_json::json_pointer("/foo/0")) == j["foo"][0]);
CHECK(j.at(alt_json::json_pointer("/foo/1")) == j["foo"][1]);
// RFC 6901 escaping works without string_t::find(str, pos), replace(),
// and substr()
auto j2 = alt_json::parse(R"({"a/b": 1, "m~n": 2, "~/~~//": 3})");
CHECK(j2.at(alt_json::json_pointer("/a~1b")) == 1);
CHECK(j2.at(alt_json::json_pointer("/m~0n")) == 2);
CHECK(j2.at(alt_json::json_pointer("/~0~1~0~0~1~1")) == 3);
CHECK(alt_json::json_pointer("/~0~1~0~0~1~1").to_string() == alt_string("/~0~1~0~0~1~1"));
CHECK(j2.flatten().unflatten() == j2);
}
SECTION("patch")
+198
View File
@@ -0,0 +1,198 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#include "doctest_compatibility.h"
#include <nlohmann/json.hpp>
using nlohmann::json;
#include <cstdint>
#include <string>
#include <vector>
namespace
{
// a spread of values exercising every writer path: scalars of each width, the
// float paths, strings, binary, and containers big enough to reallocate
std::vector<json> test_values()
{
json big_array = json::array();
for (int i = 0; i < 5000; ++i)
{
big_array.push_back(i);
}
json big_object = json::object();
for (int i = 0; i < 1000; ++i)
{
big_object[std::to_string(i)] = i;
}
return
{
json(nullptr), json(true), json(false),
json(0), json(-1), json(255), json(-129), json(65535), json(-32769),
json(4294967295U), json(-2147483649LL), json(18446744073709551615ULL),
json(0.0), json(-0.5), json(3.1415926535897932),
json(""), json("hello"), json(std::string(1000, 'x')),
json::binary({0x00, 0x01, 0x02}, 42),
json::array(), json::object(),
json::array({1, 2, 3}), json({{"a", 1}, {"b", nullptr}}),
json({{"nested", {{"deep", json::array({1, "two", 3.0, nullptr})}}}}),
big_array, big_object
};
}
// values to_bson() accepts: the document must be an object
std::vector<json> bson_values()
{
json big_object = json::object();
for (int i = 0; i < 1000; ++i)
{
big_object[std::to_string(i)] = i;
}
return
{
json::object(),
json({{"a", 1}, {"b", nullptr}, {"c", true}, {"d", 2.5}, {"e", "text"}}),
json({{"arr", json::array({1, 2, 3})}, {"obj", {{"k", "v"}}}}),
big_object
};
}
} // namespace
// The vector-returning to_*(j) overloads write through the non-virtual
// output_vector_sink, while to_*(j, adapter) goes through output_adapter_sink.
// The two are separate code paths that must stay byte-for-byte identical; these
// checks fail if either overload is ever changed without the other.
TEST_CASE("binary writer output sinks")
{
SECTION("vector sink and adapter sink agree")
{
// note: no SUBCASE inside these loops - doctest keys subcases by
// name/file/line, so a subcase in a loop body would only ever run for
// the first iteration
for (const auto& j : test_values())
{
CAPTURE(j.dump(-1, ' ', false, json::error_handler_t::replace));
std::vector<std::uint8_t> cbor;
json::to_cbor(j, cbor);
CHECK(json::to_cbor(j) == cbor);
std::vector<std::uint8_t> msgpack;
json::to_msgpack(j, msgpack);
CHECK(json::to_msgpack(j) == msgpack);
for (const bool use_size :
{
false, true
})
{
for (const bool use_type :
{
false, true
})
{
if (use_type && !use_size)
{
continue; // not a supported combination
}
CAPTURE(use_size);
CAPTURE(use_type);
std::vector<std::uint8_t> ubjson;
json::to_ubjson(j, ubjson, use_size, use_type);
CHECK(json::to_ubjson(j, use_size, use_type) == ubjson);
}
}
for (const auto version :
{
json::bjdata_version_t::draft2, json::bjdata_version_t::draft3
})
{
std::vector<std::uint8_t> bjdata;
json::to_bjdata(j, bjdata, false, false, version);
CHECK(json::to_bjdata(j, false, false, version) == bjdata);
}
}
for (const auto& j : bson_values())
{
CAPTURE(j.dump());
std::vector<std::uint8_t> bson;
json::to_bson(j, bson);
CHECK(json::to_bson(j) == bson);
}
}
SECTION("the char adapter produces the same bytes")
{
for (const auto& j : test_values())
{
CAPTURE(j.dump(-1, ' ', false, json::error_handler_t::replace));
const std::vector<std::uint8_t> expected = json::to_cbor(j);
std::vector<char> as_char;
json::to_cbor(j, as_char);
REQUIRE(as_char.size() == expected.size());
std::vector<std::uint8_t> as_bytes;
as_bytes.reserve(as_char.size());
for (const char c : as_char)
{
as_bytes.push_back(static_cast<std::uint8_t>(c));
}
CHECK(as_bytes == expected);
}
}
}
// binary_reserve_hint() is documented as a *lower* bound on the serialized size,
// so that reserving it up front can never leave the returned vector holding
// capacity beyond what the value actually needs.
TEST_CASE("binary_reserve_hint never over-reserves")
{
for (const auto& j : test_values())
{
CAPTURE(j.dump(-1, ' ', false, json::error_handler_t::replace));
const std::size_t hint = nlohmann::detail::binary_reserve_hint(j);
CHECK(hint <= json::to_cbor(j).size());
CHECK(hint <= json::to_msgpack(j).size());
CHECK(hint <= json::to_ubjson(j).size());
CHECK(hint <= json::to_ubjson(j, true, true).size());
CHECK(hint <= json::to_bjdata(j).size());
}
for (const auto& j : bson_values())
{
CAPTURE(j.dump());
CHECK(nlohmann::detail::binary_reserve_hint(j) <= json::to_bson(j).size());
}
SECTION("scalars get no hint")
{
CHECK(nlohmann::detail::binary_reserve_hint(json(nullptr)) == 0);
CHECK(nlohmann::detail::binary_reserve_hint(json(42)) == 0);
CHECK(nlohmann::detail::binary_reserve_hint(json("a string")) == 0);
CHECK(nlohmann::detail::binary_reserve_hint(json::binary({0x01})) == 0);
}
SECTION("containers are hinted from their element count")
{
CHECK(nlohmann::detail::binary_reserve_hint(json::array()) == 1);
CHECK(nlohmann::detail::binary_reserve_hint(json::array({1, 2, 3})) == 4);
CHECK(nlohmann::detail::binary_reserve_hint(json::object()) == 1);
CHECK(nlohmann::detail::binary_reserve_hint(json({{"a", 1}, {"b", 2}})) == 5);
}
}
+124 -2
View File
@@ -2586,7 +2586,12 @@ TEST_CASE("BJData")
CHECK(json::to_bjdata(json::from_bjdata(v_d), true, true) == v_d);
CHECK(json::to_bjdata(json::from_bjdata(v_D), true, true) == v_D);
CHECK(json::to_bjdata(json::from_bjdata(v_C), true, true) == v_C);
CHECK(json::to_bjdata(json::from_bjdata(v_B), true, true) == v_B);
// v_B uses the Draft-3-only 'B' marker, so it round-trips only when
// Draft 3 is explicitly selected (see GitHub issue #5404); the
// default Draft 2 falls back to a plain object instead, covered by
// the "ndarray with _ArrayType_ "byte" is gated by the BJData draft
// version" section below
CHECK(json::to_bjdata(json::from_bjdata(v_B), true, true, json::bjdata_version_t::draft3) == v_B);
}
SECTION("ndarray with data not matching _ArrayType_ is written as an object")
@@ -2629,8 +2634,10 @@ TEST_CASE("BJData")
// the C++ API stores an int literal as number_integer, so _ArrayType_
// names the wire type rather than the storage. Both storages have to
// produce the same typed array for every type.
// "byte" is checked separately below since it additionally requires
// BJData Draft 3 to be selected explicitly (see GitHub issue #5404).
for (const char* type :
{"uint8", "int8", "uint16", "int16", "uint32", "int32", "uint64", "int64", "char", "byte"
{"uint8", "int8", "uint16", "int16", "uint32", "int32", "uint64", "int64", "char"
})
{
CAPTURE(type);
@@ -2641,6 +2648,14 @@ TEST_CASE("BJData")
CHECK(from_text == json::to_bjdata(json({{"_ArrayType_", type}, {"_ArraySize_", {2, 3}}, {"_ArrayData_", {1, 2, 3, 4, 5, 6}}})));
}
{
const std::string text = R"({"_ArrayType_":"byte","_ArraySize_":[2,3],"_ArrayData_":[1,2,3,4,5,6]})";
const auto from_text = json::to_bjdata(json::parse(text), true, true, json::bjdata_version_t::draft3);
CHECK(from_text.at(0) == '[');
CHECK(from_text == json::to_bjdata(json({{"_ArrayType_", "byte"}, {"_ArraySize_", {2, 3}}, {"_ArrayData_", {1, 2, 3, 4, 5, 6}}}),
true, true, json::bjdata_version_t::draft3));
}
// negative values under a signed type behave the same way
const auto from_neg = json::to_bjdata(json::parse(R"({"_ArrayType_":"int32","_ArraySize_":[2],"_ArrayData_":[-5,7]})"));
CHECK(from_neg.at(0) == '[');
@@ -2731,6 +2746,83 @@ TEST_CASE("BJData")
CHECK(json::from_bjdata(json::to_bjdata(j_size), true, true) == j_size);
}
SECTION("ndarray whose _ArrayType_ is not a string stays as object")
{
// the type name is looked up as a string below the annotation
// check; a non-string _ArrayType_ cannot name a known dtype,
// so calling get<string_t>() on it would throw type_error.302
// instead of falling back like an unrecognized type name
// already does (see GitHub issue #5398)
json const j_number = json({{"_ArrayType_", 1}, {"_ArraySize_", {2}}, {"_ArrayData_", {1, 2}}});
const auto out_number = json::to_bjdata(j_number);
CHECK(out_number.at(0) == '{');
CHECK(json::from_bjdata(out_number) == j_number);
json const j_null = json({{"_ArrayType_", nullptr}, {"_ArraySize_", {2}}, {"_ArrayData_", {1, 2}}});
const auto out_null = json::to_bjdata(j_null);
CHECK(out_null.at(0) == '{');
CHECK(json::from_bjdata(out_null) == j_null);
json const j_bool = json({{"_ArrayType_", true}, {"_ArraySize_", {2}}, {"_ArrayData_", {1, 2}}});
const auto out_bool = json::to_bjdata(j_bool);
CHECK(out_bool.at(0) == '{');
CHECK(json::from_bjdata(out_bool) == j_bool);
json const j_array = json({{"_ArrayType_", {"uint8"}}, {"_ArraySize_", {2}}, {"_ArrayData_", {1, 2}}});
const auto out_array = json::to_bjdata(j_array);
CHECK(out_array.at(0) == '{');
CHECK(json::from_bjdata(out_array) == j_array);
json const j_object = json({{"_ArrayType_", {{"a", 1}}}, {"_ArraySize_", {2}}, {"_ArrayData_", {1, 2}}});
const auto out_object = json::to_bjdata(j_object);
CHECK(out_object.at(0) == '{');
CHECK(json::from_bjdata(out_object) == j_object);
}
SECTION("re-serializing a value containing a plain-array-of-bytes is value-stable but not byte-stable")
{
// OSS-Fuzz found this input (an array whose first element is a
// binary_t byte, followed by an object whose _ArrayType_ is
// not a string) while exercising the fix for #5398 above: once
// the fix stops to_bjdata() from throwing type_error.302 for
// the third element, serialization proceeds far enough to
// reach a pre-existing, unrelated round-trip quirk in how a
// single-byte binary_t value is re-encoded.
std::vector<std::uint8_t> const input
{
0x5b, 0x5b, 0x24, 0x42, 0x23, 0x5b, 0x69, 0x01, 0x5d, 0x5b, 0x5b, 0x5d, 0x7b, 0x55, 0x0b,
0x5f, 0x41, 0x72, 0x72, 0x61, 0x79, 0x44, 0x61, 0x74, 0x61, 0x5f, 0x54, 0x55, 0x0b, 0x5f,
0x41, 0x72, 0x72, 0x61, 0x79, 0x53, 0x69, 0x7a, 0x65, 0x5f, 0x5a, 0x55, 0x0b, 0x5f, 0x41,
0x72, 0x72, 0x61, 0x79, 0x54, 0x79, 0x70, 0x65, 0x5f, 0x54, 0x7d, 0x5d
};
json const j1 = json::from_bjdata(input);
// to_bjdata() must not throw (this is what #5398 fixes)
std::vector<std::uint8_t> vec2;
CHECK_NOTHROW(vec2 = json::to_bjdata(j1, false, false));
// parsing back a plain (non-optimized) array of bytes cannot
// recover that it used to be a binary_t: from_bjdata() has no
// way to distinguish "array of uint8 numbers" from "array of
// bytes" unless the compact "$U#" array header is used, so
// the binary_t collapses into a plain JSON array
json const j2 = json::from_bjdata(vec2);
CHECK(j1 != j2);
CHECK(j2 == json({{91}, json::array(), {{"_ArrayData_", true}, {"_ArraySize_", nullptr}, {"_ArrayType_", true}}}));
// re-serializing j2 no longer goes through the dedicated
// binary_t writer (which always uses the 'U' marker for raw
// bytes); the now-plain number 91 goes through the generic
// smallest-type writer instead, which - like the rest of the
// UBJSON/BJData writer, and unchanged by this fix - prefers
// the 'i' (int8) marker over 'U' (uint8) for values that fit
// both. Both markers are valid BJData and both decode back to
// 91, so this is not byte-for-byte identical to vec2, but it
// is value-stable: parsing it again reproduces j2 exactly.
std::vector<std::uint8_t> const vec3 = json::to_bjdata(j2, false, false);
CHECK(json::from_bjdata(vec3) == j2);
}
SECTION("ndarray whose dimensions overflow stays as object")
{
// the product of the dimensions wraps around std::size_t to 0
@@ -2823,6 +2915,36 @@ TEST_CASE("BJData")
CHECK(out_single_ok.at(0) == '[');
CHECK(json::from_bjdata(out_single_ok) == json({1.5f}));
}
SECTION("ndarray with _ArrayType_ \"byte\" is gated by the BJData draft version")
{
// the 'B' (byte) marker used by _ArrayType_ "byte" is only defined
// by BJData Draft 3; Draft 2 (the default) has no such marker, so
// emitting it unconditionally produced a stream that a Draft 2
// reader could not parse as intended (see GitHub issue #5404).
// Two dimensions are used so that a successfully written ndarray
// round-trips back into the annotated object (a single dimension
// is, by the BJData ndarray convention, read back as a plain
// binary value rather than the annotated object, same as every
// other single-dimension ndarray of a non-"byte" type is read
// back as a plain array instead of the annotated object).
json const j_byte = json({{"_ArrayType_", "byte"}, {"_ArraySize_", {2, 3}}, {"_ArrayData_", {1, 2, 3, 4, 5, 6}}});
// default (Draft 2): falls back to a plain object and round-trips
const auto out_draft2 = json::to_bjdata(j_byte);
CHECK(out_draft2.at(0) == '{');
CHECK(json::from_bjdata(out_draft2) == j_byte);
// explicit Draft 2: same as the default
const auto out_draft2_explicit = json::to_bjdata(j_byte, true, true, json::bjdata_version_t::draft2);
CHECK(out_draft2_explicit.at(0) == '{');
CHECK(json::from_bjdata(out_draft2_explicit) == j_byte);
// Draft 3 explicitly selected: still uses the compact 'B' ndarray encoding
const auto out_draft3 = json::to_bjdata(j_byte, true, true, json::bjdata_version_t::draft3);
CHECK(out_draft3 == std::vector<uint8_t>({'[', '$', 'B', '#', '[', '$', 'i', '#', 'i', 2, 2, 3, 1, 2, 3, 4, 5, 6}));
CHECK(json::from_bjdata(out_draft3) == j_byte);
}
}
}
+9
View File
@@ -791,6 +791,15 @@ TEST_CASE("BSON")
}
}
TEST_CASE("regression test - BSON binary subtype rejects a value that doesn't fit a single byte")
{
json const doc255 = {{"b", json::binary({1, 2}, 255)}};
CHECK(json::from_bson(json::to_bson(doc255))["b"].get_binary().subtype() == 255);
CHECK_THROWS_AS(json::to_bson(json{{"b", json::binary({1, 2}, 256)}}), json::out_of_range);
CHECK_THROWS_WITH_AS(json::to_bson(json{{"b", json::binary({1, 2}, 300)}}), "[json.exception.out_of_range.415] subtype 300 is too large for the BSON binary subtype (max 255)", json::out_of_range);
}
TEST_CASE("BSON input/output_adapters")
{
const json json_representation =
@@ -42,6 +42,39 @@ TEST_CASE("byte_container_with_subtype")
CHECK(container.subtype() == static_cast<subtype_type>(-1));
}
SECTION("move semantics")
{
// the rvalue-reference constructor (without a subtype) must actually move
// the passed-in container rather than copy it; comparing the buffer address
// before and after is a stronger check than just observing the source is
// empty afterward, since a copy-then-clear could also leave it empty
{
std::vector<std::uint8_t> bytes = {{0xCA, 0xFE, 0xBA, 0xBE}};
const auto* const data_ptr = bytes.data();
nlohmann::byte_container_with_subtype<std::vector<std::uint8_t>> container(std::move(bytes));
CHECK(container.size() == 4);
CHECK(container.data() == data_ptr);
CHECK(!container.has_subtype());
CHECK(bytes.empty()); // NOLINT(bugprone-use-after-move,clang-analyzer-cplusplus.Move,hicpp-invalid-access-moved)
}
// same check for the rvalue-reference constructor that also takes a subtype
{
std::vector<std::uint8_t> bytes = {{0xCA, 0xFE, 0xBA, 0xBE}};
const auto* const data_ptr = bytes.data();
nlohmann::byte_container_with_subtype<std::vector<std::uint8_t>> container(std::move(bytes), 42);
CHECK(container.size() == 4);
CHECK(container.data() == data_ptr);
CHECK(container.has_subtype());
CHECK(container.subtype() == 42);
CHECK(bytes.empty()); // NOLINT(bugprone-use-after-move,clang-analyzer-cplusplus.Move,hicpp-invalid-access-moved)
}
}
SECTION("comparisons")
{
std::vector<std::uint8_t> const bytes = {{0xCA, 0xFE, 0xBA, 0xBE}};
+435 -81
View File
@@ -8,6 +8,14 @@
#include "doctest_compatibility.h"
// capture whether JSON_STRICT_NUL_HANDLING was enabled on the command line
// (e.g. -DJSON_STRICT_NUL_HANDLING=1) *before* including json.hpp, since the
// library #undefs JSON_STRICT_NUL_HANDLING itself once the header has been
// fully processed (see include/nlohmann/detail/macro_unscope.hpp)
#if defined(JSON_STRICT_NUL_HANDLING) && (JSON_STRICT_NUL_HANDLING == 1)
#define JSON_TEST_STRICT_NUL_HANDLING_ENABLED 1
#endif
#define JSON_TESTS_PRIVATE
#include <nlohmann/json.hpp>
using nlohmann::json;
@@ -17,6 +25,8 @@ using nlohmann::json;
#include <valarray>
#include <algorithm>
#include <cstdio>
#include <fstream>
#include <list>
#include <sstream>
#include <string>
@@ -543,6 +553,88 @@ TEST_CASE("parser class")
}
}
SECTION("NUL byte handling (issue #5530, JSON_STRICT_NUL_HANDLING)")
{
// by default, a NUL byte anywhere in the input (not inside a quoted
// string, which is covered above) is silently treated the same as
// real end of input; JSON_STRICT_NUL_HANDLING (off by default, see
// docs/mkdocs/docs/api/macros/json_strict_nul_handling.md) makes a
// NUL byte an error like any other unexpected byte instead.
//
// The two sections below are mutually exclusive: this whole test
// binary is compiled once, with JSON_STRICT_NUL_HANDLING either
// left at its default or forced to 1 (e.g. by the dedicated
// ci_test_strict_nul_handling CI target), so only the section
// matching the actual, compiled-in behavior can pass.
#if !defined(JSON_TEST_STRICT_NUL_HANDLING_ENABLED)
SECTION("default behavior (macro not enabled)")
{
// a NUL byte after a complete value silently truncates the input
std::string s = "123";
s.push_back('\0');
s += "4";
CHECK(json::parse(s) == json(123));
CHECK(json::accept(s));
// parsing from a string literal is unaffected either way
CHECK(json::parse("123") == json(123));
}
#endif
#if defined(JSON_TEST_STRICT_NUL_HANDLING_ENABLED)
SECTION("opt-in strict behavior (JSON_STRICT_NUL_HANDLING == 1)")
{
// a NUL byte after a complete value is now a parse error,
// instead of silently truncating the input
{
std::string s = "123";
s.push_back('\0');
json _; // NOLINT(readability-identifier-naming)
CHECK_THROWS_WITH_AS(_ = json::parse(s),
"[json.exception.parse_error.101] parse error at line 1, column 4: syntax error while parsing value - invalid literal; last read: '123<U+0000>'; expected end of input",
json::parse_error&);
CHECK_FALSE(json::accept(s));
}
// a NUL byte where a value is expected is now a parse error,
// instead of being treated the same as an empty input
{
const std::string s(1, '\0');
json _; // NOLINT(readability-identifier-naming)
CHECK_THROWS_WITH_AS(_ = json::parse(s),
"[json.exception.parse_error.101] parse error at line 1, column 1: syntax error while parsing value - invalid literal; last read: '<U+0000>'",
json::parse_error&);
CHECK_FALSE(json::accept(s));
}
// a NUL byte inside a // comment no longer stops the comment
// scan early; scanning continues correctly past it
{
std::string s = "1 // a";
s.push_back('\0');
s += "b\n";
CHECK(json::parse(s, nullptr, true, true) == json(1));
CHECK(json::accept(s, true, true));
}
// a NUL byte inside a /* */ comment no longer stops the
// comment scan early either
{
std::string s = "1 /* a";
s.push_back('\0');
s += "b */ ";
CHECK(json::parse(s, nullptr, true, true) == json(1));
CHECK(json::accept(s, true, true));
}
// regression guard: parsing from a string literal (which
// carries a compiler-appended trailing '\0') still works,
// even though a NUL byte is now rejected everywhere else
CHECK(json::parse("123") == json(123));
}
#endif
}
SECTION("number")
{
SECTION("integers")
@@ -1892,7 +1984,13 @@ TEST_CASE("parser class")
SECTION("from std::array")
{
std::array<uint8_t, 5> v { {'t', 'r', 'u', 'e'} };
// NOTE: this array is sized to exactly the length of "true" (unlike
// the trailing-NUL-tolerant default behavior elsewhere in this file,
// see the "NUL byte handling" section above); a size of 5 here would
// leave a value-initialized trailing 0x00 element that is only
// silently accepted as end-of-input by default and would fail under
// JSON_STRICT_NUL_HANDLING
std::array<uint8_t, 4> v { {'t', 'r', 'u', 'e'} };
json j;
json::parser(nlohmann::detail::input_adapter(std::begin(v), std::end(v))).parse(true, j);
CHECK(j == json(true));
@@ -2033,7 +2131,17 @@ TEST_CASE("parser class")
{
json _;
CHECK_THROWS_WITH_AS(_ = json::parse("/a", nullptr, true, true), "[json.exception.parse_error.101] parse error at line 1, column 2: syntax error while parsing value - invalid comment; expecting '/' or '*' after '/'; last read: '/a'", json::parse_error);
// "/*" is a string literal, so it carries a compiler-appended trailing
// '\0'; by default that NUL is read like any other byte and shows up
// in "last read", but JSON_STRICT_NUL_HANDLING trims exactly that one
// trailing byte from a char array (see
// docs/mkdocs/docs/api/macros/json_strict_nul_handling.md), so it no
// longer appears in the message in that state
#if defined(JSON_TEST_STRICT_NUL_HANDLING_ENABLED)
CHECK_THROWS_WITH_AS(_ = json::parse("/*", nullptr, true, true), "[json.exception.parse_error.101] parse error at line 1, column 3: syntax error while parsing value - invalid comment; missing closing '*/'; last read: '/*'", json::parse_error);
#else
CHECK_THROWS_WITH_AS(_ = json::parse("/*", nullptr, true, true), "[json.exception.parse_error.101] parse error at line 1, column 3: syntax error while parsing value - invalid comment; missing closing '*/'; last read: '/*<U+0000>'", json::parse_error);
#endif
}
#if JSON_DIAGNOSTIC_POSITIONS
@@ -2259,86 +2367,6 @@ TEST_CASE("parser class")
#endif
}
#if JSON_DIAGNOSTIC_POSITIONS
TEST_CASE("diagnostic positions: value lifetime")
{
SECTION("copy constructor copies positions, recursively")
{
const std::string s = R"({"a":1,"b":[1,2,3]})";
const json a = json::parse(s);
const json b = a; // NOLINT(performance-unnecessary-copy-initialization)
CHECK(b.start_pos() == a.start_pos());
CHECK(b.end_pos() == a.end_pos());
CHECK(b["b"].start_pos() == a["b"].start_pos());
CHECK(b["b"].end_pos() == a["b"].end_pos());
}
SECTION("move constructor resets the moved-from value to npos")
{
const std::string s = R"({"a":1,"b":[1,2,3]})";
json a = json::parse(s);
const auto a_start = a.start_pos();
const auto a_end = a.end_pos();
const json b(std::move(a));
CHECK(b.start_pos() == a_start);
CHECK(b.end_pos() == a_end);
CHECK(a.start_pos() == std::string::npos); // NOLINT(bugprone-use-after-move,clang-analyzer-cplusplus.Move)
CHECK(a.end_pos() == std::string::npos); // NOLINT(bugprone-use-after-move,clang-analyzer-cplusplus.Move)
}
SECTION("swap() exchanges positions along with the values")
{
// basic_json::swap() (and the friend swap() that forwards to it) used
// to swap only m_data.m_type/m_data.m_value, leaving
// start_position/end_position untouched -- unlike copy-assignment's
// operator=(basic_json), which swaps positions as part of its
// copy-and-swap implementation. After swap(a, b), each value ended up
// with the *other* value's content but its *own* original position.
// This is now fixed so that swap() is consistent with copy-assignment.
json a = json::parse(R"({"a":1})");
json b = json::parse(R"([1,2,3,4,5])");
const auto a_start = a.start_pos();
const auto a_end = a.end_pos();
const auto b_start = b.start_pos();
const auto b_end = b.end_pos();
// lengths (and thus end positions) differ, which is enough to tell
// after the swap whether positions actually moved with the values
CHECK(a_end != b_end);
using std::swap;
swap(a, b);
CHECK(a == json::parse(R"([1,2,3,4,5])"));
CHECK(b == json::parse(R"({"a":1})"));
CHECK(a.start_pos() == b_start);
CHECK(a.end_pos() == b_end);
CHECK(b.start_pos() == a_start);
CHECK(b.end_pos() == a_end);
// member swap() behaves the same as the free function
json c = json::parse(R"({"a":1})");
json d = json::parse(R"([1,2,3,4,5])");
const auto c_start = c.start_pos();
const auto c_end = c.end_pos();
const auto d_start = d.start_pos();
const auto d_end = d.end_pos();
c.swap(d);
CHECK(c.start_pos() == d_start);
CHECK(c.end_pos() == d_end);
CHECK(d.start_pos() == c_start);
CHECK(d.end_pos() == c_end);
}
}
#endif
// this test relies on parse errors being thrown, so it is skipped when
// exceptions are disabled (json::parse aborts instead of throwing there)
#if !defined(JSON_NOEXCEPTION)
@@ -2445,3 +2473,329 @@ TEST_CASE("last-read diagnostics are identical across input adapters")
}
}
#endif // !defined(JSON_NOEXCEPTION)
// this test characterizes the current (documented-by-example, not otherwise
// specified) behavior of JSON_DIAGNOSTIC_POSITIONS positions with respect to
// value lifetime (copy/move/swap/mutation), the various input adapters, and
// user-driven SAX usage. It is regression protection, not a behavior
// specification: if any of these checks fail after a change to json.hpp,
// that change deliberately altered observable behavior and the test (and
// this comment) should be updated accordingly, rather than "fixed" blindly.
#if JSON_DIAGNOSTIC_POSITIONS
TEST_CASE("diagnostic positions: value lifetime, input adapters, and SAX")
{
SECTION("value lifetime")
{
SECTION("copy constructor copies positions, recursively")
{
// basic_json(const basic_json&) (json.hpp, around line 1192) copies
// start_position/end_position for the value itself; nested values
// are copied via their own copy constructor (through the copied
// object/array container), so positions are preserved throughout
// the whole tree.
const std::string s = R"({"a":1,"b":[1,2,3]})";
const json a = json::parse(s);
const json b = a; // NOLINT(performance-unnecessary-copy-initialization)
CHECK(b.start_pos() == a.start_pos());
CHECK(b.end_pos() == a.end_pos());
CHECK(b["b"].start_pos() == a["b"].start_pos());
CHECK(b["b"].end_pos() == a["b"].end_pos());
CHECK(b["b"][0].start_pos() == a["b"][0].start_pos());
CHECK(b["b"][0].end_pos() == a["b"][0].end_pos());
// sanity: the positions are meaningful (not all npos)
CHECK(b.start_pos() == 0);
CHECK(b.end_pos() == s.size());
}
SECTION("move constructor resets the moved-from value to npos")
{
// basic_json(basic_json&&) (json.hpp, around line 1265) copies
// other's start_position/end_position into *this and then resets
// other's to npos (see the cppcheck-suppress[accessForwarded]
// annotation there, which flags this reset as worth a second
// look). Only the top-level moved-from value is affected; its
// (moved-away) children are gone along with it.
const std::string s = R"({"a":1,"b":[1,2,3]})";
json a = json::parse(s);
const auto a_start = a.start_pos();
const auto a_end = a.end_pos();
const auto nested_start = a["b"].start_pos();
const auto nested_end = a["b"].end_pos();
const json b(std::move(a));
// the destination retains the original positions, recursively
CHECK(b.start_pos() == a_start);
CHECK(b.end_pos() == a_end);
CHECK(b["b"].start_pos() == nested_start);
CHECK(b["b"].end_pos() == nested_end);
// the moved-from value is reset to a null and reports npos
CHECK(a.is_null()); // NOLINT(bugprone-use-after-move,clang-analyzer-cplusplus.Move)
CHECK(a.start_pos() == std::string::npos); // NOLINT(bugprone-use-after-move,clang-analyzer-cplusplus.Move)
CHECK(a.end_pos() == std::string::npos); // NOLINT(bugprone-use-after-move,clang-analyzer-cplusplus.Move)
}
SECTION("swap() exchanges positions along with values")
{
// basic_json::swap() (json.hpp, around line 3626, and the friend
// swap() that forwards to it) swaps start_position/end_position
// together with m_data.m_type and m_data.m_value, so after
// swap(a, b) each variable's position describes its own new
// content, consistent with copy-assignment's
// operator=(basic_json) (json.hpp, around line 1291), which also
// swaps positions as part of its copy-and-swap implementation.
json a = json::parse(R"({"a":1})");
json b = json::parse(R"([1,2,3,4,5])");
const auto a_start = a.start_pos();
const auto a_end = a.end_pos();
const auto b_start = b.start_pos();
const auto b_end = b.end_pos();
// both start at 0 (root values start right away), but their
// lengths (and thus end positions) differ, which is enough to
// tell after the swap whether positions actually moved with
// the values
CHECK(a_end != b_end);
using std::swap;
swap(a, b);
// values were exchanged as expected ...
CHECK(a == json::parse(R"([1,2,3,4,5])"));
CHECK(b == json::parse(R"({"a":1})"));
// ... and so were positions: each variable now carries the
// other's original position, describing its own new content
CHECK(a.start_pos() == b_start);
CHECK(a.end_pos() == b_end);
CHECK(b.start_pos() == a_start);
CHECK(b.end_pos() == a_end);
// member swap() behaves the same as the free function
json c = json::parse(R"({"a":1})");
json d = json::parse(R"([1,2,3,4,5])");
const auto c_start = c.start_pos();
const auto c_end = c.end_pos();
const auto d_start = d.start_pos();
const auto d_end = d.end_pos();
c.swap(d);
CHECK(c.start_pos() == d_start);
CHECK(c.end_pos() == d_end);
CHECK(d.start_pos() == c_start);
CHECK(d.end_pos() == c_end);
}
SECTION("mutating a parsed document leaves positions of unrelated values untouched")
{
// Positions are recorded once, during parsing, and are not
// recomputed on mutation. As a consequence, after a mutation the
// parent's own recorded span may no longer describe its current
// (serialized) content -- it still describes what was originally
// parsed. This is characterized here as current behavior, not
// asserted to be desirable or specified.
SECTION("operator[] adding a new object key")
{
const std::string s = R"({"a":1})";
json j = json::parse(s);
const auto root_start = j.start_pos();
const auto root_end = j.end_pos();
const auto a_start = j["a"].start_pos();
const auto a_end = j["a"].end_pos();
j["c"] = 42;
// the newly-added value was never parsed, so it has no position
CHECK(j["c"].start_pos() == std::string::npos);
CHECK(j["c"].end_pos() == std::string::npos);
// the existing sibling's position is unaffected
CHECK(j["a"].start_pos() == a_start);
CHECK(j["a"].end_pos() == a_end);
// the parent's own recorded span is left as-is (now stale:
// it still reflects the original, shorter `{"a":1}` string)
CHECK(j.start_pos() == root_start);
CHECK(j.end_pos() == root_end);
}
SECTION("push_back on a parsed array")
{
const std::string s = R"([1,2,3])";
json j = json::parse(s);
const auto root_start = j.start_pos();
const auto root_end = j.end_pos();
const auto first_start = j[0].start_pos();
j.push_back(4);
CHECK(j.back().start_pos() == std::string::npos);
CHECK(j.back().end_pos() == std::string::npos);
CHECK(j[0].start_pos() == first_start);
CHECK(j.start_pos() == root_start);
CHECK(j.end_pos() == root_end);
}
SECTION("erase on a parsed array shifts elements but keeps their own positions")
{
const std::string s = R"([1,2,3])";
json j = json::parse(s);
const auto second_start = j[1].start_pos();
const auto third_start = j[2].start_pos();
const auto root_start = j.start_pos();
const auto root_end = j.end_pos();
j.erase(0);
// remaining elements moved down an index, but each one still
// reports the position it had *before* the erase (i.e. its
// position in the original source string, not a
// recalculated one)
CHECK(j[0].start_pos() == second_start);
CHECK(j[1].start_pos() == third_start);
// the parent's own recorded span is again left as-is
CHECK(j.start_pos() == root_start);
CHECK(j.end_pos() == root_end);
}
}
}
SECTION("input adapters")
{
SECTION("wide string input: positions count transcoded UTF-8 bytes, not wide characters")
{
// 'é' (U+00E9) is a single code unit in a wchar_t/UTF-16 string, but
// transcodes to 2 bytes in UTF-8; the lexer only ever sees the
// transcoded UTF-8 byte stream, so reported positions are byte
// offsets into that UTF-8 stream, not indices into the original
// std::wstring.
// é (rather than a literal 'é' byte sequence in this source
// file) so the wide-string literal's meaning does not depend on
// the compiler's assumed source character set (MSVC, without
// /utf-8, would otherwise decode the raw UTF-8 bytes using the
// system code page instead of as UTF-8)
const std::wstring ws = L"{\"a\":\"\u00e9\u00e9\"}";
CHECK(ws.size() == 10); // 10 wide characters
const json j = json::parse(ws);
CHECK(j.start_pos() == 0);
// the transcoded UTF-8 form is 2 bytes longer than the wide string,
// because each of the two 'é' characters becomes 2 UTF-8 bytes
CHECK(j.end_pos() == 12);
CHECK(j.end_pos() != ws.size());
const json& a = j["a"];
CHECK(a.start_pos() == 5);
CHECK(a.end_pos() == 11);
}
SECTION("BOM-prefixed input: start_pos() reflects the skipped 3-byte BOM")
{
const std::string s = "\xEF\xBB\xBF{\"a\":1}";
const json j = json::parse(s);
// the lexer silently skips the BOM before parsing the value, so
// the root value's recorded span starts right after it
CHECK(j.start_pos() == 3);
CHECK(j.end_pos() == s.size());
}
SECTION("std::istringstream: positions are consistent, not npos")
{
const std::string s = R"({"a":1,"b":2})";
std::istringstream ss(s);
const json j = json::parse(ss);
CHECK(j.start_pos() == 0);
CHECK(j.end_pos() == s.size());
CHECK(j["a"].start_pos() == 5);
}
SECTION("std::ifstream: positions are consistent, not npos")
{
const std::string s = R"({"a":1,"b":2})";
{
std::ofstream file("unit-class_parser_diagnostic_positions.tmp");
file << s;
}
{
std::ifstream f("unit-class_parser_diagnostic_positions.tmp");
const json j = json::parse(f);
CHECK(j.start_pos() == 0);
CHECK(j.end_pos() == s.size());
CHECK(j["a"].start_pos() == 5);
}
static_cast<void>(std::remove("unit-class_parser_diagnostic_positions.tmp"));
}
SECTION("iterator-pair input: positions are consistent, not npos")
{
const std::string s = R"({"a":1,"b":2})";
const json j = json::parse(s.begin(), s.end());
CHECK(j.start_pos() == 0);
CHECK(j.end_pos() == s.size());
CHECK(j["a"].start_pos() == 5);
}
SECTION("binary formats have no text positions")
{
// binary formats (CBOR, MessagePack, UBJSON, BSON, BJData) are
// parsed via detail::binary_reader, which never sets
// start_position/end_position on the values it produces (they
// have no notion of a text offset), so every value's position
// stays at its default of npos.
const json src = json::parse(R"({"a":1,"b":[1,2]})");
const json from_cbor = json::from_cbor(json::to_cbor(src));
CHECK(from_cbor.start_pos() == std::string::npos);
CHECK(from_cbor.end_pos() == std::string::npos);
CHECK(from_cbor["a"].start_pos() == std::string::npos);
CHECK(from_cbor["b"][0].start_pos() == std::string::npos);
const json from_msgpack = json::from_msgpack(json::to_msgpack(src));
CHECK(from_msgpack.start_pos() == std::string::npos);
CHECK(from_msgpack.end_pos() == std::string::npos);
const json from_ubjson = json::from_ubjson(json::to_ubjson(src));
CHECK(from_ubjson.start_pos() == std::string::npos);
CHECK(from_ubjson.end_pos() == std::string::npos);
const json from_bson_val = json::from_bson(json::to_bson(src));
CHECK(from_bson_val.start_pos() == std::string::npos);
CHECK(from_bson_val.end_pos() == std::string::npos);
}
}
SECTION("user-driven SAX consumers with no lexer report npos")
{
// json::parse() internally wires up its json_sax_dom_parser with a
// pointer to its own lexer (see parser.hpp), which is how positions
// get set at all. A user who constructs a json_sax_dom_parser
// directly (e.g. to drive it via json::sax_parse()) and does not
// supply a lexer pointer gets a consumer with m_lexer_ref == nullptr;
// every "if (m_lexer_ref)" guard in json_sax.hpp is then skipped, so
// every value it produces keeps its default, unset position (npos).
// This was previously true but silently unasserted (operator==
// ignores positions), see #5420.
json result;
nlohmann::detail::json_sax_dom_parser<json, nlohmann::detail::string_input_adapter_type> sdp(result);
const std::string s = R"({"a":1,"b":[1,2,3]})";
CHECK(json::sax_parse(s, &sdp));
CHECK(result.start_pos() == std::string::npos);
CHECK(result.end_pos() == std::string::npos);
CHECK(result["a"].start_pos() == std::string::npos);
CHECK(result["a"].end_pos() == std::string::npos);
CHECK(result["b"][0].start_pos() == std::string::npos);
CHECK(result["b"][0].end_pos() == std::string::npos);
}
}
#endif
+34
View File
@@ -1792,6 +1792,40 @@ TEST_CASE("std::filesystem::path")
}
#endif
// the ADL to_json overload for std::u8string only exists under the same guard
// as std::filesystem::path support (it is otherwise only reached indirectly,
// via std::filesystem::path::u8string()) -- mirror both #if conditions from
// include/nlohmann/detail/conversions/to_json.hpp exactly
#if JSON_HAS_FILESYSTEM || JSON_HAS_EXPERIMENTAL_FILESYSTEM
#if defined(__cpp_lib_char8_t)
TEST_CASE("std::u8string")
{
SECTION("ascii")
{
const std::u8string s = u8"Path";
json const j = s;
CHECK(j.template get<std::string>() == "Path");
}
SECTION("utf-8")
{
// use \u universal-character-names (rather than raw \x byte escapes
// or literal non-ASCII source bytes) to compose the multi-byte UTF-8
// encoding -- MSVC treats \x escapes used that way inside a u8
// literal as a nonstandard extension (warning C5321), which some of
// our CI configs promote to an error; \u is portable and produces
// the exact same encoded bytes without depending on the source
// file's encoding
const std::u8string s = u8"P\u011B\u0161ina";
json const j = s;
CHECK(j.template get<std::string>() == "P\xc4\x9b\xc5\xa1ina");
}
}
#endif
#endif
TEST_CASE("std::optional")
{
SECTION("null")
+150
View File
@@ -0,0 +1,150 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#include "doctest_compatibility.h"
#include <nlohmann/json.hpp>
#include <deque>
#include <map>
#include <memory>
#include <string>
#include <type_traits>
#include <vector>
namespace
{
// std::deque has no capacity() member function, which the library only needs
// to detect a reallocation for JSON_DIAGNOSTICS
using deque_json = nlohmann::basic_json<std::map, std::deque>;
// a std::vector whose at() is hidden: the library performs its own bounds
// check and must not fall back to the container's checked accessor
template<class T, class Allocator = std::allocator<T>>
class vector_without_at : public std::vector<T, Allocator>
{
public:
vector_without_at() = default;
// the array of an initializer list is built from a range
template<class InputIt>
vector_without_at(InputIt first, InputIt last) : std::vector<T, Allocator>(first, last) {}
void at() = delete;
};
using no_at_json = nlohmann::basic_json<std::map, vector_without_at>;
} // namespace
TEST_CASE("array type without capacity()")
{
SECTION("the iterators take their exception specification from the container")
{
// basic_json's iterators move exactly as the container iterators do:
// their move operations are defaulted without a declared noexcept,
// because an array or object type whose iterator is not nothrow move
// constructible would otherwise have them deleted (std::deque's is not
// with libstdc++ before 11, and neither are MSVC's debug iterators)
CHECK(std::is_nothrow_move_constructible<nlohmann::json::iterator>::value ==
(std::is_nothrow_move_constructible<nlohmann::json::object_t::iterator>::value
&& std::is_nothrow_move_constructible<nlohmann::json::array_t::iterator>::value));
CHECK(std::is_nothrow_move_assignable<nlohmann::json::iterator>::value ==
(std::is_nothrow_move_assignable<nlohmann::json::object_t::iterator>::value
&& std::is_nothrow_move_assignable<nlohmann::json::array_t::iterator>::value));
CHECK(std::is_nothrow_move_constructible<nlohmann::json::const_iterator>::value ==
(std::is_nothrow_move_constructible<nlohmann::json::object_t::const_iterator>::value
&& std::is_nothrow_move_constructible<nlohmann::json::array_t::const_iterator>::value));
// and they are movable at all, which is what dropping the declared
// noexcept buys for a std::deque array
CHECK(std::is_move_constructible<deque_json::iterator>::value);
CHECK(std::is_move_assignable<deque_json::iterator>::value);
}
SECTION("adding elements")
{
deque_json j = deque_json::array();
j.push_back(1);
j.push_back("two");
j.emplace_back(3);
j += 4;
CHECK(j.size() == 4);
CHECK(j == deque_json({1, "two", 3, 4}));
CHECK(j.back() == 4);
CHECK(j.front() == 1);
}
SECTION("accessing and modifying elements")
{
auto j = deque_json::parse(R"([1,2,3])");
CHECK(j[1] == 2);
CHECK(j.at(2) == 3);
// growing through operator[] fills up with null values
j[5] = 6;
CHECK(j.size() == 6);
CHECK(j[4].is_null());
CHECK(j[5] == 6);
j.erase(0);
CHECK(j == deque_json({2, 3, nullptr, nullptr, 6}));
auto it = j.erase(j.begin());
CHECK(*it == 3);
j.insert(j.begin(), 1);
CHECK(j.front() == 1);
}
SECTION("serialization and deserialization")
{
const auto j = deque_json::parse(R"({"a":[1,[2,3]],"b":[]})");
CHECK(j.dump() == R"({"a":[1,[2,3]],"b":[]})");
CHECK(deque_json::parse(j.dump()) == j);
CHECK(deque_json::from_cbor(deque_json::to_cbor(j)) == j);
// empty containers are flattened to null and cannot be restored
const auto nested = deque_json::parse(R"({"a":[1,[2,3]]})");
CHECK(nested.flatten().unflatten() == nested);
}
SECTION("references stay valid while the array grows")
{
deque_json j = deque_json::array();
j.push_back(1);
auto& first = j[0];
for (int i = 0; i < 100; ++i)
{
j.push_back(i);
}
CHECK(&first == &j[0]);
CHECK(first == 1);
}
}
TEST_CASE("array type without at()")
{
// built in memory rather than parsed, so that the exception message does
// not gain a byte range with JSON_DIAGNOSTIC_POSITIONS
no_at_json j = {1, 2, 3};
const auto& jc = j;
CHECK(j.at(0) == 1);
CHECK(j.at(2) == 3);
CHECK(jc.at(2) == 3);
CHECK_THROWS_WITH_AS(j.at(3), "[json.exception.out_of_range.401] array index 3 is out of range", no_at_json::out_of_range);
CHECK_THROWS_WITH_AS(jc.at(3), "[json.exception.out_of_range.401] array index 3 is out of range", no_at_json::out_of_range);
CHECK(j.at(no_at_json::json_pointer("/1")) == 2);
CHECK_THROWS_AS(j.at(no_at_json::json_pointer("/3")), no_at_json::out_of_range);
}
+79
View File
@@ -0,0 +1,79 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#include "doctest_compatibility.h"
#include <nlohmann/json.hpp>
#include <cstdint>
#include <functional>
#include <map>
#include <memory>
#include <string>
#include <vector>
#ifdef JSON_HAS_CPP_17
#include <cstddef>
#endif
namespace
{
// a BinaryType whose value type is signed: the elements must still be
// processed as the numbers 0..255
using char_binary_json = nlohmann::basic_json <
std::map, std::vector, std::string, bool, std::int64_t, std::uint64_t,
double, std::allocator, nlohmann::adl_serializer, std::vector<char>, void >;
#ifdef JSON_HAS_CPP_17
// a BinaryType whose value type is not an integer type at all
using byte_binary_json = nlohmann::basic_json <
std::map, std::vector, std::string, bool, std::int64_t, std::uint64_t,
double, std::allocator, nlohmann::adl_serializer, std::vector<std::byte>, void >;
#endif
} // namespace
TEST_CASE("binary type whose value type is not std::uint8_t")
{
SECTION("a signed value type does not dump negative numbers")
{
const std::vector<char> chars{'\0', '\x01', '\xFF'};
CHECK(char_binary_json::binary(chars).dump() == R"({"bytes":[0,1,255],"subtype":null})");
CHECK(char_binary_json::binary(chars, 42).dump() == R"({"bytes":[0,1,255],"subtype":42})");
CHECK(char_binary_json::binary({}).dump() == R"({"bytes":[],"subtype":null})");
}
SECTION("the default binary type is unchanged")
{
CHECK(nlohmann::json::binary({0, 1, 255}, 42).dump() == R"({"bytes":[0,1,255],"subtype":42})");
}
#ifdef JSON_HAS_CPP_17
SECTION("dumping a value type that is not an integer")
{
const std::vector<std::byte> bytes{std::byte{0}, std::byte{1}, std::byte{0xFF}};
CHECK(byte_binary_json::binary(bytes).dump() == R"({"bytes":[0,1,255],"subtype":null})");
CHECK(byte_binary_json::binary(bytes, 42).dump() == R"({"bytes":[0,1,255],"subtype":42})");
CHECK(byte_binary_json::binary({}).dump() == R"({"bytes":[],"subtype":null})");
}
SECTION("hashing and the binary formats")
{
const std::vector<std::byte> bytes{std::byte{0}, std::byte{1}, std::byte{0xFF}};
const auto j = byte_binary_json::binary(bytes);
CHECK(std::hash<byte_binary_json> {}(j) == std::hash<byte_binary_json> {}(j));
CHECK(byte_binary_json::from_cbor(byte_binary_json::to_cbor(j)) == j);
CHECK(byte_binary_json::from_msgpack(byte_binary_json::to_msgpack(j)) == j);
// UBJSON has no binary type, so binary values are written as an array
CHECK(byte_binary_json::from_ubjson(byte_binary_json::to_ubjson(j)) == byte_binary_json({0, 1, 255}));
}
#endif
}
+323
View File
@@ -0,0 +1,323 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#include "doctest_compatibility.h"
#include <nlohmann/json.hpp>
#include <cstdint>
#include <map>
#include <string>
#include <type_traits>
#include <utility>
#include <vector>
namespace
{
// An ObjectType that does *not* define a key_compare member type, which is
// what every hash map looks like to the library.
//
// A hash map is deliberately not used here: object_t is probed for
// key_compare inside the definition of basic_json, that is, while basic_json
// is still an incomplete type, and whether a hash map can be instantiated
// with an incomplete mapped type depends on the standard library (libstdc++ 9
// needs the size of the mapped type for its node type and rejects it). So the
// object type wraps a std::map instead of inheriting from it: an earlier
// version derived from std::map and shadowed the inherited key_compare type
// with a same-named member function, relying on ordinary member hiding to
// make key_compare unreachable as a type. MSVC 2017 (AppVeyor, /std:c++17)
// does not honor that hiding for a typename-qualified lookup performed from
// outside the class and still resolves key_compare to the base's comparator
// type, so the library's probe incorrectly found one. Composition sidesteps
// the question entirely: with no base class, there is no key_compare to find
// under any lookup rule.
template<class Key, class T, class Compare, class Allocator>
class no_key_compare_map
{
using map_t = std::map<Key, T, Compare, Allocator>;
map_t data;
public:
using key_type = typename map_t::key_type;
using mapped_type = typename map_t::mapped_type;
using value_type = typename map_t::value_type;
using size_type = typename map_t::size_type;
using allocator_type = typename map_t::allocator_type;
using iterator = typename map_t::iterator;
using const_iterator = typename map_t::const_iterator;
// -Weffc++ asks for the member to be initialized in the member
// initialization list, which a defaulted constructor does not do; the
// exception specification a defaulted one would have carried has to be
// written out as well, or -Wnoexcept objects where the standard library
// takes noexcept(construct(...))
no_key_compare_map() noexcept(std::is_nothrow_default_constructible<map_t>::value) : data() {}
// converting between two basic_json types builds the object from a range
template<class InputIt>
no_key_compare_map(InputIt first, InputIt last) : data(first, last) {}
iterator begin() noexcept
{
return data.begin();
}
iterator end() noexcept
{
return data.end();
}
const_iterator begin() const noexcept
{
return data.begin();
}
const_iterator end() const noexcept
{
return data.end();
}
const_iterator cbegin() const noexcept
{
return data.cbegin();
}
const_iterator cend() const noexcept
{
return data.cend();
}
bool empty() const noexcept
{
return data.empty();
}
size_type size() const noexcept
{
return data.size();
}
size_type max_size() const noexcept
{
return data.max_size();
}
void clear() noexcept
{
data.clear();
}
iterator find(const key_type& key)
{
return data.find(key);
}
const_iterator find(const key_type& key) const
{
return data.find(key);
}
size_type count(const key_type& key) const
{
return data.count(key);
}
std::pair<iterator, bool> emplace(const key_type& key, const mapped_type& value)
{
return data.emplace(key, value);
}
std::pair<iterator, bool> insert(const value_type& value)
{
return data.insert(value);
}
template<class InputIt>
void insert(InputIt first, InputIt last)
{
data.insert(first, last);
}
mapped_type& operator[](const key_type& key)
{
return data[key];
}
mapped_type& at(const key_type& key)
{
return data.at(key);
}
const mapped_type& at(const key_type& key) const
{
return data.at(key);
}
iterator erase(iterator pos)
{
return data.erase(pos);
}
iterator erase(iterator first, iterator last)
{
return data.erase(first, last);
}
size_type erase(const key_type& key)
{
return data.erase(key);
}
void swap(no_key_compare_map& other) noexcept(noexcept(data.swap(other.data)))
{
data.swap(other.data);
}
friend bool operator==(const no_key_compare_map& lhs, const no_key_compare_map& rhs)
{
return lhs.data == rhs.data;
}
friend bool operator<(const no_key_compare_map& lhs, const no_key_compare_map& rhs)
{
return lhs.data < rhs.data;
}
};
using no_key_compare_json = nlohmann::basic_json<no_key_compare_map>;
// An ObjectType whose erase(iterator) returns void rather than the following
// iterator, as for instance Abseil's hash maps do
template<class Key, class T, class Compare, class Allocator>
struct void_erase_map : std::map<Key, T, Compare, Allocator>
{
using base_t = std::map<Key, T, Compare, Allocator>;
using iterator = typename base_t::iterator;
using base_t::erase;
void erase(iterator pos)
{
base_t::erase(pos);
}
};
using void_erase_json = nlohmann::basic_json<void_erase_map>;
} // namespace
TEST_CASE("object type whose erase() returns void")
{
SECTION("erasing every element through the returned iterator")
{
void_erase_json j;
for (int i = 0; i < 8; ++i)
{
j["k" + std::to_string(i)] = i;
}
std::size_t erased = 0;
for (auto it = j.begin(); it != j.end(); ++erased)
{
it = j.erase(it);
}
CHECK(erased == 8);
CHECK(j.empty());
}
SECTION("erasing in the middle returns the following element")
{
void_erase_json j;
for (int i = 0; i < 4; ++i)
{
j["k" + std::to_string(i)] = i;
}
auto it = j.begin();
++it;
const auto after = j.erase(it);
CHECK(j.size() == 3);
CHECK(after.key() == "k2");
CHECK(after.value() == 2);
CHECK(!j.contains("k1"));
}
SECTION("the other erase overloads are unaffected")
{
void_erase_json j;
j["a"] = 1;
j["b"] = 2;
j["c"] = 3;
CHECK(j.erase("a") == 1);
CHECK(j.erase("nope") == 0);
j.erase(j.begin(), j.end());
CHECK(j.empty());
}
}
TEST_CASE("object type without key_compare")
{
SECTION("object_comparator_t falls back to default_object_comparator_t")
{
CHECK(std::is_same < no_key_compare_json::object_comparator_t,
no_key_compare_json::default_object_comparator_t >::value);
}
SECTION("object types defining key_compare are unaffected")
{
CHECK(std::is_same<nlohmann::json::object_comparator_t,
nlohmann::json::object_t::key_compare>::value);
CHECK(std::is_same<nlohmann::ordered_json::object_comparator_t,
nlohmann::ordered_json::object_t::key_compare>::value);
}
SECTION("creating and accessing values")
{
no_key_compare_json j;
j["one"] = 1;
j["two"] = "zwei";
j["three"]["nested"] = true;
CHECK(j.size() == 3);
CHECK(j.at("one") == 1);
CHECK(j["two"] == "zwei");
CHECK(j["three"]["nested"] == true);
CHECK(j.contains("one"));
CHECK(!j.contains("four"));
CHECK(j.find("one") != j.end());
CHECK(j.count("one") == 1);
CHECK(j.erase("one") == 1);
CHECK(j.size() == 2);
}
SECTION("serialization and deserialization")
{
const auto j = no_key_compare_json::parse(R"({"a":[1,2,3],"b":{"c":null}})");
CHECK(j["a"].size() == 3);
CHECK(j["a"][2] == 3);
CHECK(j["b"]["c"].is_null());
CHECK(no_key_compare_json::parse(j.dump()) == j);
}
SECTION("binary formats")
{
const auto j = no_key_compare_json::parse(R"({"a":[1,2,3],"b":"x"})");
CHECK(no_key_compare_json::from_cbor(no_key_compare_json::to_cbor(j)) == j);
CHECK(no_key_compare_json::from_msgpack(no_key_compare_json::to_msgpack(j)) == j);
}
SECTION("flatten and unflatten")
{
// "o" has a key that looks like an array index, so unflatten() must
// not turn it into an array
const auto j = no_key_compare_json::parse(
R"({"c":[1,2,3],"d":{"e":"s"},"n":[[0,1],[2]],"o":{"2":"x"}})");
CHECK(j.flatten().unflatten() == j);
}
SECTION("conversion to and from nlohmann::json")
{
const auto j = no_key_compare_json::parse(R"({"a":1,"b":[true,null]})");
const nlohmann::json converted(j);
CHECK(converted.is_object());
CHECK(converted["a"] == 1);
CHECK(converted["b"][0] == true);
CHECK(converted["b"][1].is_null());
CHECK(no_key_compare_json(converted) == j);
}
}
+54 -2
View File
@@ -8,6 +8,14 @@
#include "doctest_compatibility.h"
// capture whether JSON_STRICT_NUL_HANDLING was enabled on the command line
// (e.g. -DJSON_STRICT_NUL_HANDLING=1) *before* including json.hpp, since the
// library #undefs JSON_STRICT_NUL_HANDLING itself once the header has been
// fully processed (see include/nlohmann/detail/macro_unscope.hpp)
#if defined(JSON_STRICT_NUL_HANDLING) && (JSON_STRICT_NUL_HANDLING == 1)
#define JSON_TEST_STRICT_NUL_HANDLING_ENABLED 1
#endif
#include <nlohmann/json.hpp>
using nlohmann::json;
#ifdef JSON_TEST_NO_GLOBAL_UDLS
@@ -323,6 +331,23 @@ TEST_CASE("deserialization")
CHECK(j == json({"foo", 1, 2, 3, false, {{"one", 1}}}));
}
SECTION("operator>> with a NUL byte after the value (issue #5530)")
{
// operator>> parses non-strictly (it does not require the whole
// stream to be consumed), so a NUL byte following a complete
// value is simply left unread on the stream and never reaches
// the "expected end of input" check that JSON_STRICT_NUL_HANDLING
// affects; this holds regardless of the macro (verified below for
// the opt-in state as well)
std::string data = "123";
data.push_back('\0');
std::istringstream ss(data);
json j;
ss >> j;
CHECK(j == json(123));
CHECK(ss.good());
}
SECTION("user-defined string literal")
{
CHECK("[\"foo\",1,2,3,false,{\"one\":1}]"_json == json({"foo", 1, 2, 3, false, {{"one", 1}}}));
@@ -405,6 +430,27 @@ TEST_CASE("deserialization")
CHECK_THROWS_WITH_AS(ss >> j, "[json.exception.parse_error.101] parse error at line 1, column 29: syntax error while parsing array - unexpected end of input; expected ']'", json::parse_error&);
}
#if defined(JSON_TEST_STRICT_NUL_HANDLING_ENABLED)
SECTION("operator>> with a NUL byte where a value is expected (JSON_STRICT_NUL_HANDLING == 1, issue #5530)")
{
// a trailing NUL byte *after* a complete value is unaffected by the
// macro (see the successful-deserialization "operator>> with a NUL
// byte after the value" section above): operator>> parses
// non-strictly and never reaches the "expected end of input" check
// that the macro changes. A NUL byte where a *value* is expected,
// however, goes through the same token dispatch as any other input
// and is affected: with the macro enabled it now raises
// parse_error.101 (like any other unrecognized byte) instead of
// being silently treated the same as an empty stream.
std::string const data(1, '\0');
std::istringstream ss(data);
json j;
CHECK_THROWS_WITH_AS(ss >> j,
"[json.exception.parse_error.101] parse error at line 1, column 1: syntax error while parsing value - invalid literal; last read: '<U+0000>'",
json::parse_error&);
}
#endif
SECTION("user-defined string literal")
{
CHECK_THROWS_WITH_AS("[\"foo\",1,2,3,false,{\"one\":1}"_json, "[json.exception.parse_error.101] parse error at line 1, column 29: syntax error while parsing array - unexpected end of input; expected ']'", json::parse_error&);
@@ -453,7 +499,11 @@ TEST_CASE("deserialization")
SECTION("from std::array")
{
std::array<uint8_t, 5> const v { {'t', 'r', 'u', 'e'} };
// sized to exactly the length of "true": a size of 5 would leave
// a value-initialized trailing 0x00 element that is only
// silently accepted as end-of-input by default and would fail
// under JSON_STRICT_NUL_HANDLING
std::array<uint8_t, 4> const v { {'t', 'r', 'u', 'e'} };
CHECK(json::parse(v) == json(true));
CHECK(json::accept(v));
@@ -549,7 +599,9 @@ TEST_CASE("deserialization")
SECTION("from std::array")
{
std::array<uint8_t, 5> v { {'t', 'r', 'u', 'e'} };
// sized to exactly the length of "true", see the analogous
// "from std::array" section above for why
std::array<uint8_t, 4> v { {'t', 'r', 'u', 'e'} };
CHECK(json::parse(std::begin(v), std::end(v)) == json(true));
CHECK(json::accept(std::begin(v), std::end(v)));
+31
View File
@@ -273,5 +273,36 @@ TEST_CASE("Regression tests for extended diagnostics")
CHECK(j1["numbers"]["two"] == 2);
CHECK(j1["string"] == "t");
}
SECTION("Regression test - swap(array_t&)/swap(object_t&) must update JSON_DIAGNOSTICS parent pointers")
{
// swap(array_t&)
{
json j = json::array();
json::array_t arr = {json::array({1})};
j.swap(arr);
// parent pointers of the moved-in elements must point into j, not
// into the now-defunct free-standing array_t
CHECK_THROWS_WITH_AS(j[0][0].get<std::string>(), "[json.exception.type_error.302] (/0/0) type must be string, but is number", json::type_error);
// must not trigger assert_invariant() in a debug/assert-enabled build
json const k = j;
CHECK(k == j);
}
// swap(object_t&)
{
json o = json::object();
json::object_t obj = {{"a", json::array({1})}};
o.swap(obj);
CHECK_THROWS_WITH_AS(o["a"][0].get<std::string>(), "[json.exception.type_error.302] (/a/0) type must be string, but is number", json::type_error);
// must not trigger assert_invariant() in a debug/assert-enabled build
json const p = o;
CHECK(p == o);
}
}
}
+96
View File
@@ -672,6 +672,102 @@ TEST_CASE("JSON patch")
}
}
SECTION("patch_inplace")
{
SECTION("happy path: patch_inplace mirrors patch() on success")
{
// mirrors "A.5. Replacing a Value" above, but applies the patch with
// patch_inplace() to a mutable copy instead of using patch()'s
// returned copy
json doc = R"(
{
"baz": "qux",
"foo": "bar"
}
)"_json;
json const patch = R"(
[
{ "op": "replace", "path": "/baz", "value": "boo" }
]
)"_json;
json const expected = R"(
{
"baz": "boo",
"foo": "bar"
}
)"_json;
doc.patch_inplace(patch);
CHECK(doc == expected);
}
// this test relies on the "test" operation actually throwing so the
// partial-application state can be observed right after the throw
// point; under JSON_NOEXCEPTION, JSON_THROW() calls std::abort()
// instead (there is no C++ exception to throw), and doctest's
// CHECK_THROWS_AS() is compiled out to a no-op that never even
// invokes the given expression (see doctest's "--no-throw" test
// filter, which ci_test_noexceptions passes) -- so patch()/
// patch_inplace() would never be called at all and the follow-up
// state assertions below would fail against the untouched original
#if !defined(JSON_NOEXCEPTION)
SECTION("distinguishing contract vs patch(): partial application on failure")
{
// Unlike patch(), which is all-or-nothing because it applies the
// patch to an internal copy that is simply discarded when an
// exception is thrown (leaving the original untouched no matter
// what), patch_inplace() mutates the document it is called on
// directly and immediately, operation by operation. So if a JSON
// Patch fails partway through, whatever operations already
// succeeded remain applied -- the document is left in a partially
// patched state. This is empirically verified current behavior,
// not just documented intent, and is pinned here as such.
json const original = R"(
{
"baz": "qux",
"foo": "bar"
}
)"_json;
// the first operation ("replace") succeeds; the second ("test")
// fails because the value at "/baz" no longer (and never did)
// equal "not boo"
json const patch = R"(
[
{ "op": "replace", "path": "/baz", "value": "boo" },
{ "op": "test", "path": "/baz", "value": "not boo" }
]
)"_json;
// patch() never modifies the object it is called on -- it always
// operates on (and returns) a separate copy, so the original is
// left completely untouched, regardless of success or failure.
// copy_for_patch is intentionally a real copy, not a reference
// to `original`: the whole point of this check is to catch a
// hypothetical future regression where patch() *does* mutate its
// receiver. Using a reference here would make the assertion
// below compare `original` to itself -- trivially true even if
// such a bug existed -- which is exactly what a static analyzer
// can't see when it suggests "this copy is never modified, use
// a reference instead".
json copy_for_patch = original; // NOLINT(performance-unnecessary-copy-initialization)
CHECK_THROWS_AS(copy_for_patch.patch(patch), json::other_error&);
CHECK(copy_for_patch == original);
// patch_inplace(), in contrast, already applied the successful
// "replace" operation to the document before the "test" operation
// threw -- that change is not rolled back
json doc = original;
CHECK_THROWS_AS(doc.patch_inplace(patch), json::other_error&);
CHECK(doc != original);
CHECK(doc.at("baz") == "boo");
CHECK(doc.at("foo") == "bar");
}
#endif // !defined(JSON_NOEXCEPTION)
}
SECTION("errors")
{
SECTION("unknown operation")
+10
View File
@@ -507,6 +507,16 @@ TEST_CASE("JSON pointers")
// explicit roundtrip check
CHECK(j.flatten().unflatten() == j);
// an object is only unflattened to an array if one of its keys is the
// reference token 0; this must not depend on which key is seen first
CHECK(json({{"/2", "x"}}).unflatten() == json({{"2", "x"}}));
CHECK(json({{"/10", "y"}, {"/2", "z"}}).unflatten() == json({{"10", "y"}, {"2", "z"}}));
CHECK(json({{"/0", 1}, {"/1", 2}}).unflatten() == json({1, 2}));
CHECK(json({{"/1", 2}, {"/0", 1}}).unflatten() == json({1, 2}));
CHECK(json({{"/0", 1}, {"/2", 3}}).unflatten() == json({1, nullptr, 3}));
CHECK(json({{"/a/1", 2}, {"/a/0", 1}}).unflatten() == json({{"a", {1, 2}}}));
CHECK(json({{"/a/1", 2}, {"/a/x", 1}}).unflatten() == json({{"a", {{"1", 2}, {"x", 1}}}}));
// roundtrip for primitive values
json j_null;
CHECK(j_null.flatten().unflatten() == j_null);
+14
View File
@@ -641,6 +641,20 @@ TEST_CASE("modifiers")
CHECK_THROWS_WITH_AS(j_array.insert(j_array.end(), j_other_array.begin(), j_other_array2.end()), "[json.exception.invalid_iterator.210] iterators do not fit",
json::invalid_iterator&);
}
SECTION("iterators not pointing into an array")
{
json j_object2 = {{"k", 1}, {"l", 2}};
json j_primitive = 5;
json j_null;
CHECK_THROWS_WITH_AS(j_array.insert(j_array.begin(), j_object2.begin(), j_object2.end()), "[json.exception.invalid_iterator.202] iterators first and last must point to arrays",
json::invalid_iterator&);
CHECK_THROWS_WITH_AS(j_array.insert(j_array.begin(), j_primitive.begin(), j_primitive.end()), "[json.exception.invalid_iterator.202] iterators first and last must point to arrays",
json::invalid_iterator&);
CHECK_THROWS_WITH_AS(j_array.insert(j_array.begin(), j_null.begin(), j_null.end()), "[json.exception.invalid_iterator.202] iterators first and last must point to arrays",
json::invalid_iterator&);
}
}
SECTION("range for object")
+15
View File
@@ -1682,6 +1682,21 @@ TEST_CASE("issue #5405 - array reserve for definite-length MessagePack arrays")
}
}
TEST_CASE("regression test - MessagePack ext type rejects a subtype that doesn't fit a single byte")
{
// subtype 0-255 must still round-trip correctly (regression guard, pre-existing behavior)
CHECK(json::from_msgpack(json::to_msgpack(json::binary({1, 2}, 0))).get_binary().subtype() == 0);
CHECK(json::from_msgpack(json::to_msgpack(json::binary({1, 2}, 200))).get_binary().subtype() == 200);
CHECK(json::from_msgpack(json::to_msgpack(json::binary({1, 2}, 255))).get_binary().subtype() == 255);
// a subtype > 255 must throw instead of silently truncating
CHECK_THROWS_AS(json::to_msgpack(json::binary({1, 2}, 256)), json::out_of_range);
CHECK_THROWS_WITH_AS(json::to_msgpack(json::binary({1, 2}, 70000)), "[json.exception.out_of_range.415] subtype 70000 is too large for the MessagePack ext type (max 255)", json::out_of_range);
// a binary value with no subtype at all must be unaffected
CHECK(json::from_msgpack(json::to_msgpack(json::binary({1, 2}))).get_binary().has_subtype() == false);
}
// use this testcase outside [hide] to run it with Valgrind
TEST_CASE("MessagePack nesting does not consume the call stack")
{
@@ -70,7 +70,7 @@ TEST_CASE("check_for_mem_leak_on_adl_to_json-2")
}
}
TEST_CASE("check_for_mem_leak_on_adl_to_json-2")
TEST_CASE("check_for_mem_leak_on_adl_to_json-3")
{
try
{
@@ -0,0 +1,91 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
// This translation unit is a dedicated, small compile-and-run check for two
// configuration macros that (per #5423) were never exercised anywhere in the
// test matrix:
// - JSON_NO_IO, which removes the library's <istream>/<ostream> support
// (operator<<, operator>>, and the stream-based overloads of dump()/parse())
// - the JSON_THROW_USER / JSON_TRY_USER / JSON_CATCH_USER trio, which lets a
// user replace the library's internal exception handling
//
// Both macros are about excluding/replacing a facility the library would
// otherwise pull in on its own, and defining one has no bearing on the other,
// so -- to keep the test matrix small -- they are exercised together in a
// single dedicated file instead of two.
//
// JSON_NO_IO requires this file itself to never rely on <iostream>/<sstream>;
// only string-based parsing/dumping is used below.
#define JSON_NO_IO 1
// The user-supplied exception macros below are a *conforming* replacement:
// they simply forward to the real throw/try/catch keywords (via a counter so
// the test can assert each macro was actually invoked, not just defined), so
// every exception-related behavior the library relies on internally --
// including rethrowing std::out_of_range as json::out_of_range in at() --
// keeps working exactly as it would with the library's own default macros.
static int json_throw_user_call_count = 0; // NOLINT(cppcoreguidelines-avoid-non-const-global-variables)
#define JSON_THROW_USER(exception) do { ++json_throw_user_call_count; throw (exception); } while (false) // NOLINT(cppcoreguidelines-macro-usage)
#define JSON_TRY_USER try // NOLINT(cppcoreguidelines-macro-usage)
#define JSON_CATCH_USER(exception) catch (exception) // NOLINT(cppcoreguidelines-macro-usage)
#include "doctest_compatibility.h"
#include <nlohmann/json.hpp>
using json = nlohmann::json;
TEST_CASE("JSON_NO_IO")
{
// everything that does not touch <istream>/<ostream> must keep working:
// parsing from and dumping to std::string
const json j = json::parse(R"({"a":[1,2,3],"b":true})");
CHECK(j.dump() == R"({"a":[1,2,3],"b":true})");
CHECK(j.at("a").size() == 3);
CHECK(j.at("b").get<bool>() == true);
}
// this test relies on CHECK_THROWS_AS() actually invoking the guarded
// expression so json_throw_user_call_count gets bumped and can be observed
// afterwards; doctest's "--no-throw" test filter (which ci_test_noexceptions
// passes, together with a global -DJSON_NOEXCEPTION added to CMAKE_CXX_FLAGS
// for every translation unit in that build, this file included) compiles
// CHECK_THROWS_AS() out to a no-op that never even invokes the given
// expression -- so json::parse()/at() below would never be called at all and
// the call-count assertions would fail even though our JSON_THROW_USER
// override (which always really throws, regardless of JSON_NOEXCEPTION) would
// have worked fine on its own
#if !defined(JSON_NOEXCEPTION)
TEST_CASE("JSON_THROW_USER, JSON_TRY_USER, JSON_CATCH_USER")
{
json_throw_user_call_count = 0;
// json::parse() is [[nodiscard]] (JSON_HEDLEY_WARN_UNUSED_RESULT); under
// GCC in C++11 mode that expands to __attribute__((warn_unused_result)),
// which -- unlike a [[nodiscard]] attribute proper -- GCC does not
// consider satisfied by doctest's CHECK_THROWS_AS() wrapping the
// expression in a (void) cast, so the discarded return value would still
// be flagged under -Werror=unused-result; assign it to discard it instead,
// matching the established `json _ = json::parse(...)` pattern used
// elsewhere in the test suite (see unit-class_parser.cpp)
json _; // NOLINT(readability-identifier-naming)
// a parse error goes through JSON_THROW directly, i.e., through our
// JSON_THROW_USER override
CHECK_THROWS_AS(_ = json::parse("this is not JSON"), json::parse_error&);
CHECK(json_throw_user_call_count > 0);
// at() on an out-of-range array index internally catches std::out_of_range
// (JSON_TRY_USER/JSON_CATCH_USER) and rethrows it as json::out_of_range
// (JSON_THROW_USER again), so this exercises all three macros together
const int count_before = json_throw_user_call_count;
const json arr = json::array({1, 2, 3});
CHECK_THROWS_AS(arr.at(10), json::out_of_range&);
CHECK(json_throw_user_call_count > count_before);
}
#endif
+81
View File
@@ -81,3 +81,84 @@ TEST_CASE("regression test for issue #3732 - iteration_proxy_value<iter_impl<ord
};
static_cast<void>(fn);
}
TEST_CASE("regression test - diff() must account for ordered_json member order")
{
SECTION("pure reorder, no value changes")
{
ordered_json a = {{"a", 1}, {"b", 2}};
ordered_json b = {{"b", 2}, {"a", 1}};
CHECK(a != b); // order-sensitive equality
CHECK(a.patch(ordered_json::diff(a, b)) == b);
}
SECTION("new key must land at the front")
{
ordered_json c = {{"b", 2}};
ordered_json e = {{"a", 1}, {"b", 2}};
CHECK(c.patch(ordered_json::diff(c, e)) == e);
}
SECTION("reorder plus a value change on one of the reordered keys")
{
ordered_json a = {{"a", 1}, {"b", 2}};
ordered_json b = {{"b", 20}, {"a", 1}};
CHECK(a != b);
CHECK(a.patch(ordered_json::diff(a, b)) == b);
}
SECTION("reorder plus a deleted key")
{
ordered_json a = {{"a", 1}, {"b", 2}, {"c", 3}};
ordered_json b = {{"b", 2}, {"a", 1}};
CHECK(a != b);
CHECK(a.patch(ordered_json::diff(a, b)) == b);
}
SECTION("reorder plus a nested value that itself needs a recursive diff")
{
ordered_json a = {{"a", {{"x", 1}, {"y", 2}}}, {"b", 2}};
ordered_json b = {{"b", 2}, {"a", {{"x", 1}, {"y", 99}}}};
CHECK(a != b);
CHECK(a.patch(ordered_json::diff(a, b)) == b);
}
SECTION("three or more keys shuffled into a different order")
{
ordered_json a = {{"a", 1}, {"b", 2}, {"c", 3}, {"d", 4}};
ordered_json b = {{"d", 4}, {"b", 2}, {"a", 1}, {"c", 3}};
CHECK(a != b);
CHECK(a.patch(ordered_json::diff(a, b)) == b);
}
SECTION("matching order still produces a minimal patch (fast path unaffected)")
{
ordered_json a = {{"a", 1}, {"b", 2}, {"c", 3}};
ordered_json b = {{"a", 1}, {"b", 20}, {"c", 3}};
auto p = ordered_json::diff(a, b);
// only the changed value should be touched, not a wholesale remove+add
CHECK(p.size() == 1);
CHECK(p[0]["op"] == "replace");
CHECK(p[0]["path"] == "/b");
CHECK(a.patch(p) == b);
}
SECTION("plain json (std::map-backed) is unaffected by same-key-different-insertion-order")
{
json a;
a["b"] = 2;
a["a"] = 1;
json b;
b["a"] = 1;
b["b"] = 2;
// std::map iteration is always sorted by key, so a == b regardless of
// insertion order, and diff() must still produce the same minimal
// (empty) result as before this fix
CHECK(a == b);
auto p = json::diff(a, b);
CHECK(p.empty());
CHECK(a.patch(p) == b);
}
}
+489
View File
@@ -0,0 +1,489 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-FileCopyrightText: 2018 Vitaliy Manushkin <agri@akamo.info>
// SPDX-License-Identifier: MIT
// This file closes a test-coverage gap described in GitHub issue #5421:
// nlohmann::ordered_json (and other non-default basic_json specializations,
// such as the alt_string-based one from unit-alt-string.cpp) were never
// exercised through the binary formats (CBOR/MessagePack/UBJSON/BSON/BJData)
// or through flatten()/unflatten()/diff()/patch()/merge_patch().
#include "doctest_compatibility.h"
#include <nlohmann/json.hpp>
#include <cstdint>
#include <string>
#include <utility>
#include <vector>
using nlohmann::json;
using nlohmann::ordered_json;
/////////////////////////////////////////////////////////////////////////////
// alt_json: a second, independent copy of the custom-string_t basic_json
// specialization defined in unit-alt-string.cpp.
//
// It is duplicated here (rather than shared via a header) because every
// unit-*.cpp file in this test suite is compiled into its own standalone
// executable (see tests/CMakeLists.txt), so there is no ODR concern in
// having the same class name defined in multiple translation units.
//
// Two members had to be added relative to the original alt_string
// (a constructor from std::string, and a find(char, pos) overload) because
// the original type was never used with the binary writers/readers before
// this file: BSON's array/document writer converts std::to_string() results
// and checks for embedded NUL characters via find(char), and the UBJSON/BSON
// high-precision-number path constructs the SAX string_t argument from a
// std::string. Neither path is exercised anywhere else in the test suite for
// this type, which is presumably why the gap was never noticed.
/////////////////////////////////////////////////////////////////////////////
class alt_string;
bool operator<(const char* op1, const alt_string& op2) noexcept; // NOLINT(misc-use-internal-linkage)
void int_to_string(alt_string& target, std::size_t value); // NOLINT(misc-use-internal-linkage)
class alt_string
{
public:
using value_type = std::string::value_type;
static constexpr auto npos = (std::numeric_limits<std::size_t>::max)();
alt_string(const char* str): str_impl(str) {}
alt_string(const char* str, std::size_t count): str_impl(str, count) {}
alt_string(std::string str): str_impl(std::move(str)) {}
alt_string(size_t count, char chr): str_impl(count, chr) {}
alt_string() = default;
alt_string& append(char ch)
{
str_impl.push_back(ch);
return *this;
}
alt_string& append(const alt_string& str)
{
str_impl.append(str.str_impl);
return *this;
}
alt_string& append(const char* s, std::size_t length)
{
str_impl.append(s, length);
return *this;
}
void push_back(char c)
{
str_impl.push_back(c);
}
template <typename op_type>
bool operator==(const op_type& op) const
{
return str_impl == op;
}
bool operator==(const alt_string& op) const
{
return str_impl == op.str_impl;
}
template <typename op_type>
bool operator!=(const op_type& op) const
{
return str_impl != op;
}
bool operator!=(const alt_string& op) const
{
return str_impl != op.str_impl;
}
std::size_t size() const noexcept
{
return str_impl.size();
}
void resize(std::size_t n)
{
str_impl.resize(n);
}
void resize(std::size_t n, char c)
{
str_impl.resize(n, c);
}
template <typename op_type>
bool operator<(const op_type& op) const noexcept
{
return str_impl < op;
}
bool operator<(const alt_string& op) const noexcept
{
return str_impl < op.str_impl;
}
const char* c_str() const
{
return str_impl.c_str();
}
char& operator[](std::size_t index)
{
return str_impl[index];
}
const char& operator[](std::size_t index) const
{
return str_impl[index];
}
char& back()
{
return str_impl.back();
}
const char& back() const
{
return str_impl.back();
}
void clear()
{
str_impl.clear();
}
const value_type* data() const
{
return str_impl.data();
}
bool empty() const
{
return str_impl.empty();
}
std::size_t find(const alt_string& str, std::size_t pos = 0) const
{
return str_impl.find(str.str_impl, pos);
}
// needed by binary_writer's BSON support, which probes string keys for
// embedded NUL characters via find(char)
std::size_t find(char c, std::size_t pos = 0) const
{
return str_impl.find(c, pos);
}
std::size_t find_first_of(char c, std::size_t pos = 0) const
{
return str_impl.find_first_of(c, pos);
}
alt_string substr(std::size_t pos = 0, std::size_t count = npos) const
{
const std::string s = str_impl.substr(pos, count);
return {s.data(), s.size()};
}
alt_string& replace(std::size_t pos, std::size_t count, const alt_string& str)
{
str_impl.replace(pos, count, str.str_impl);
return *this;
}
void reserve(std::size_t new_cap = 0)
{
str_impl.reserve(new_cap);
}
private:
std::string str_impl {}; // NOLINT(readability-redundant-member-init)
friend bool operator<(const char* /*op1*/, const alt_string& /*op2*/) noexcept;
};
void int_to_string(alt_string& target, std::size_t value)
{
target = std::to_string(value).c_str();
}
using alt_json = nlohmann::basic_json <
std::map,
std::vector,
alt_string,
bool,
std::int64_t,
std::uint64_t,
double,
std::allocator,
nlohmann::adl_serializer >;
bool operator<(const char* op1, const alt_string& op2) noexcept
{
return op1 < op2.str_impl;
}
namespace
{
// collects the object keys of j, in iteration order
std::vector<std::string> collect_keys(const ordered_json& j)
{
std::vector<std::string> result;
for (auto it = j.cbegin(); it != j.cend(); ++it)
{
result.push_back(it.key());
}
return result;
}
// a nested object/array value with keys inserted in non-alphabetical order,
// used to check both round-trip equality and (for ordered_json) that
// insertion order survives a trip through a binary format
ordered_json make_rich_ordered_json()
{
ordered_json j;
j["zebra"] = 1;
j["apple"] = ordered_json::array({1, 2, 3});
j["mango"]["z_nested"] = true;
j["mango"]["a_nested"] = nullptr;
j["banana"] = "some text";
j["cherry"] = 3.14;
return j;
}
alt_json make_rich_alt_json()
{
alt_json j;
j["zebra"] = 1;
j["apple"] = alt_json::array({1, 2, 3});
j["mango"]["z_nested"] = true;
j["mango"]["a_nested"] = nullptr;
j["banana"] = "some text";
j["cherry"] = 3.14;
return j;
}
} // namespace
TEST_CASE("ordered_json across binary formats")
{
const ordered_json original = make_rich_ordered_json();
const std::vector<std::string> original_keys = collect_keys(original);
const std::vector<std::string> original_mango_keys = collect_keys(original["mango"]);
SECTION("CBOR")
{
const auto bytes = ordered_json::to_cbor(original);
const auto restored = ordered_json::from_cbor(bytes);
CHECK(restored == original);
CHECK(collect_keys(restored) == original_keys);
CHECK(collect_keys(restored["mango"]) == original_mango_keys);
}
SECTION("MessagePack")
{
const auto bytes = ordered_json::to_msgpack(original);
const auto restored = ordered_json::from_msgpack(bytes);
CHECK(restored == original);
CHECK(collect_keys(restored) == original_keys);
CHECK(collect_keys(restored["mango"]) == original_mango_keys);
}
SECTION("UBJSON")
{
const auto bytes = ordered_json::to_ubjson(original);
const auto restored = ordered_json::from_ubjson(bytes);
CHECK(restored == original);
CHECK(collect_keys(restored) == original_keys);
CHECK(collect_keys(restored["mango"]) == original_mango_keys);
}
SECTION("BSON")
{
const auto bytes = ordered_json::to_bson(original);
const auto restored = ordered_json::from_bson(bytes);
CHECK(restored == original);
CHECK(collect_keys(restored) == original_keys);
CHECK(collect_keys(restored["mango"]) == original_mango_keys);
}
SECTION("BJData")
{
const auto bytes = ordered_json::to_bjdata(original);
const auto restored = ordered_json::from_bjdata(bytes);
CHECK(restored == original);
CHECK(collect_keys(restored) == original_keys);
CHECK(collect_keys(restored["mango"]) == original_mango_keys);
}
}
TEST_CASE("alt_json (custom string_t) across binary formats")
{
const alt_json original = make_rich_alt_json();
SECTION("CBOR")
{
const auto bytes = alt_json::to_cbor(original);
const auto restored = alt_json::from_cbor(bytes);
CHECK(restored == original);
}
SECTION("MessagePack")
{
const auto bytes = alt_json::to_msgpack(original);
const auto restored = alt_json::from_msgpack(bytes);
CHECK(restored == original);
}
SECTION("UBJSON")
{
const auto bytes = alt_json::to_ubjson(original);
const auto restored = alt_json::from_ubjson(bytes);
CHECK(restored == original);
}
SECTION("BSON")
{
const auto bytes = alt_json::to_bson(original);
const auto restored = alt_json::from_bson(bytes);
CHECK(restored == original);
}
SECTION("BJData")
{
const auto bytes = alt_json::to_bjdata(original);
const auto restored = alt_json::from_bjdata(bytes);
CHECK(restored == original);
}
}
TEST_CASE("ordered_json operator== is sensitive to key order")
{
// Unlike nlohmann::json (whose object_t is a std::map, so equality never
// depends on insertion order), ordered_json's object_t (ordered_map) is a
// std::vector<std::pair<Key, T>> under the hood, and does not define its
// own operator==: it inherits std::vector's element-wise comparison. As a
// result, two ordered_json objects holding the very same key/value pairs
// in different insertion order compare *unequal*. This is the property
// that makes the round-trip `CHECK(restored == original)` checks above a
// meaningful order-preservation check by themselves (the explicit
// collect_keys() comparisons make that check explicit/readable, and
// guard against this operator== behavior ever changing).
ordered_json a;
a["x"] = 1;
a["y"] = 2;
ordered_json b;
b["y"] = 2;
b["x"] = 1;
CHECK(a.size() == b.size());
CHECK(a["x"] == b["x"]);
CHECK(a["y"] == b["y"]);
CHECK_FALSE(a == b);
}
TEST_CASE("duplicate keys in a binary-encoded object")
{
// CBOR encoding of a map with two entries under the same key "a": {"a": 1, "a": 2}
const std::vector<std::uint8_t> cbor_bytes
{
0xA2, 0x61, 'a', 0x01, 0x61, 'a', 0x02
};
// Both json (std::map, via operator[]) and ordered_json (ordered_map, via
// operator[]) build binary-decoded objects by looking up/creating the
// entry for each incoming key and then assigning the value into it. This
// means a repeated key does *not* produce two entries in either case;
// instead, the *first* occurrence's position is kept (relevant only for
// ordered_json) while the *last* occurrence's value wins (for both) --
// this matches operator[]'s "assign the referenced slot" semantics, and
// is worth noting because it differs from the initializer-list
// construction path (`ordered_json{{"a",1},{"a",2}}`), which builds
// through insert()/emplace() and therefore keeps the *first* value, not
// the last (see the "There are no dup keys..." case in
// unit-ordered_json.cpp).
const auto j = json::from_cbor(cbor_bytes);
const auto oj = ordered_json::from_cbor(cbor_bytes);
CHECK(j.size() == 1);
CHECK(oj.size() == 1);
CHECK(j["a"] == 2);
CHECK(oj["a"] == 2);
CHECK(j == json(oj));
}
TEST_CASE("ordered_json through flatten/unflatten")
{
const ordered_json original = make_rich_ordered_json();
const std::vector<std::string> original_keys = collect_keys(original);
const std::vector<std::string> original_mango_keys = collect_keys(original["mango"]);
const ordered_json flat = original.flatten();
const ordered_json unflattened = flat.unflatten();
CHECK(unflattened == original);
// flatten() walks the value depth-first in iteration order and
// unflatten() re-inserts each flattened key via operator[] in the flat
// object's iteration order, so for ordered_json the original key order
// (both top-level and nested) is preserved end-to-end.
CHECK(collect_keys(unflattened) == original_keys);
CHECK(collect_keys(unflattened["mango"]) == original_mango_keys);
}
TEST_CASE("ordered_json through diff/patch/patch_inplace")
{
ordered_json original;
original["one"] = 1;
original["two"] = 2;
original["three"] = 3;
ordered_json target = original;
target["one"] = 100; // replace
target.erase("two"); // remove
target["four"] = 4; // add
const ordered_json patch = ordered_json::diff(original, target);
SECTION("patch")
{
const ordered_json patched = original.patch(patch);
CHECK(patched == target);
}
SECTION("patch_inplace")
{
ordered_json copy = original;
copy.patch_inplace(patch);
CHECK(copy == target);
}
}
TEST_CASE("ordered_json through merge_patch")
{
ordered_json original;
original["a"] = 1;
original["b"] = 2;
const ordered_json patch = {{"b", nullptr}, {"c", 3}};
original.merge_patch(patch);
ordered_json expected;
expected["a"] = 1;
expected["c"] = 3;
CHECK(original == expected);
CHECK(collect_keys(original) == collect_keys(expected));
}
+107 -1
View File
@@ -606,7 +606,11 @@ TEST_CASE("regression tests 2")
SECTION("issue #2546 - parsing containers of std::byte")
{
const char DATA[] = R"("Hello, world!")"; // NOLINT(misc-const-correctness,cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
const auto s = std::as_bytes(std::span(DATA));
// exclude the trailing '\0' that string-literal initialization adds to
// DATA: std::span(DATA) would span the full array extent (including
// that NUL), which is only silently accepted as end-of-input by default
// and would fail under JSON_STRICT_NUL_HANDLING
const auto s = std::as_bytes(std::span(DATA, sizeof(DATA) - 1));
const json j = json::parse(s);
CHECK(j.dump() == "\"Hello, world!\"");
}
@@ -763,4 +767,106 @@ TEST_CASE("regression tests 2")
}
TEST_CASE("regression test - parser callback must not lose a duplicate key's prior value")
{
// a callback that rejects only the scalar value 2
const json::parser_callback_t drop_value_2 = [](int /*depth*/, json::parse_event_t ev, json & v) noexcept
{
return !(ev == json::parse_event_t::value && v == 2);
};
SECTION("duplicate key, second (scalar) value rejected - prior value is restored")
{
const json j = json::parse(R"({"a":1,"a":2})", drop_value_2);
CHECK(j.dump() == "{\"a\":1}");
}
SECTION("duplicate key, second value is an object rejected at object_end - prior value is restored")
{
const json j = json::parse(R"({"a":1,"a":{"x":2}})",
[](int depth, json::parse_event_t ev, json& /*parsed*/) noexcept
{
return !(ev == json::parse_event_t::object_end && depth == 1);
});
CHECK(j.dump() == "{\"a\":1}");
}
SECTION("duplicate key, second value is an array rejected at array_end - prior value is restored")
{
const json j = json::parse(R"({"a":1,"a":[9,9]})",
[](int depth, json::parse_event_t ev, json& /*parsed*/) noexcept
{
return !(ev == json::parse_event_t::array_end && depth == 1);
});
CHECK(j.dump() == "{\"a\":1}");
}
SECTION("duplicate key, second value accepted (scalar) - last value wins")
{
const json j = json::parse(R"({"a":1,"a":2})", [](int, json::parse_event_t, json&) noexcept
{
return true;
});
CHECK(j.dump() == "{\"a\":2}");
}
SECTION("duplicate key, second value accepted (object) - last value wins")
{
const json j = json::parse(R"({"a":1,"a":{"x":2}})", [](int, json::parse_event_t, json&) noexcept
{
return true;
});
CHECK(j.dump() == "{\"a\":{\"x\":2}}");
}
SECTION("brand new (non-duplicate) key, value rejected - member is fully absent")
{
const json j = json::parse(R"({"a":1,"b":2})", drop_value_2);
CHECK(j.dump() == "{\"a\":1}");
}
SECTION("duplicate key nested two levels deep")
{
const json j = json::parse(R"({"outer":{"a":1,"a":2}})", drop_value_2);
CHECK(j.dump() == "{\"outer\":{\"a\":1}}");
}
SECTION("three occurrences of the same key - middle rejected, last accepted")
{
const json j = json::parse(R"({"k":1,"k":2,"k":3})", drop_value_2);
CHECK(j.dump() == "{\"k\":3}");
}
}
TEST_CASE("regression test - excessive binary container size honors allow_exceptions=false")
{
// CBOR array with declared length 2^63
const std::vector<std::uint8_t> cbor = {0x9b, 0x80, 0, 0, 0, 0, 0, 0, 0};
// CBOR map with declared length 2^63
const std::vector<std::uint8_t> cbor_m = {0xbb, 0x80, 0, 0, 0, 0, 0, 0, 0};
// UBJSON array with declared length 2^63-1
const std::vector<std::uint8_t> ubj = {'[', '#', 'L', 0x7f, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff};
// BJData array with declared length 2^63-1 (little endian)
const std::vector<std::uint8_t> bjd = {'[', '#', 'L', 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x7f};
// allow_exceptions=false must report failure instead of throwing/aborting
CHECK(json::from_cbor(cbor, true, false).is_discarded());
CHECK(json::from_cbor(cbor_m, true, false).is_discarded());
CHECK(json::from_ubjson(ubj, true, false).is_discarded());
CHECK(json::from_bjdata(bjd, true, false).is_discarded());
// allow_exceptions=true (the default) must still throw exactly as before.
// The exact message text is not checked here: on platforms where
// std::size_t is 32-bit, the CBOR reader's own length-narrowing check
// (get_cbor_container_size(), unrelated to this fix) intercepts a
// declared length of 2^63 before it ever reaches the check this test
// targets, with different (but equally valid, and already correct)
// wording -- see unit-cbor.cpp for coverage of that message.
json _;
CHECK_THROWS_AS(_ = json::from_cbor(cbor), json::out_of_range);
// regression guard: a genuinely truncated CBOR input must remain discarded
CHECK(json::from_cbor(std::vector<std::uint8_t> {0x9b, 0, 0, 0, 0, 0, 0, 0, 0x02}, true, false).is_discarded());
}
DOCTEST_CLANG_SUPPRESS_WARNING_POP
+10
View File
@@ -18,6 +18,14 @@
// for some reason including this after the json header leads to linker errors with VS 2017...
#include <locale>
// skip tests if JSON_DisableEnumSerialization=ON (#4384): std::byte is a
// scoped enum, so get<std::byte>() (needed below to get<std::vector<std::byte>>()
// from a plain JSON array, not just from an already-binary value) relies on
// enum serialization being enabled
#if defined(JSON_DISABLE_ENUM_SERIALIZATION) && (JSON_DISABLE_ENUM_SERIALIZATION == 1)
#define SKIP_TESTS_FOR_ENUM_SERIALIZATION
#endif
#define JSON_TESTS_PRIVATE
#include <nlohmann/json.hpp>
using json = nlohmann::json;
@@ -466,6 +474,7 @@ TEST_CASE("regression tests 3")
CHECK((decoded == json_4804::array()));
}
#ifndef SKIP_TESTS_FOR_ENUM_SERIALIZATION
SECTION("discussion #4209 - custom BinaryType direct assignment and round-tripping")
{
// Test that assigning a custom BinaryType directly creates a binary value, not an array
@@ -499,6 +508,7 @@ TEST_CASE("regression tests 3")
CHECK(extracted[1] == std::byte{2});
CHECK(extracted[2] == std::byte{3});
}
#endif
SECTION("issue #5046 - implicit conversion of return json to std::optional no longer implicit")
{
+13
View File
@@ -522,6 +522,19 @@ TEST_CASE("indentation is written straight into the write buffer")
CHECK(json::parse(out) == j);
}
SECTION("binary values are indented the same way")
{
// a binary value is serialized as an object with "bytes" and
// "subtype" keys; the byte array itself is always written compactly
// (see dump_byte()), so only the surrounding object's indentation
// goes through put_indent()
const json j = json::binary({1, 2, 3}, 128);
CHECK(j.dump(2000) == "{\n" + std::string(2000, ' ') + "\"bytes\": [1, 2, 3],\n"
+ std::string(2000, ' ') + "\"subtype\": 128\n}");
CHECK(j.dump(2000, '\t') == "{\n" + std::string(2000, '\t') + "\"bytes\": [1, 2, 3],\n"
+ std::string(2000, '\t') + "\"subtype\": 128\n}");
}
SECTION("indentation is unchanged for ordinary widths")
{
const json j = {{"a", {1, 2}}, {"b", nullptr}};
+30
View File
@@ -17,6 +17,7 @@
#include <nlohmann/json.hpp>
using json = nlohmann::json;
using ordered_json = nlohmann::ordered_json;
// JSON_HAS_CPP_20 (do not remove; see note at top of file)
#if JSON_HAS_STD_FORMAT
@@ -52,6 +53,23 @@ TEST_CASE("std::formatter<nlohmann::json>")
CHECK(std::format("{:2}", j) == j.dump(2));
CHECK(std::format("{:#2}", j) == j.dump(2));
CHECK(std::format("{:8}", j) == j.dump(8));
// multi-digit widths must accumulate every digit, not just the first
CHECK(std::format("{:12}", j) == j.dump(12));
CHECK(std::format("{:#12}", j) == j.dump(12));
CHECK(std::format("{:10}", j) == j.dump(10));
}
SECTION("bare alignment with no fill character defaults to a space indent character")
{
const json j = {{"foo", 1}, {"bar", {1, 2, 3}}};
// without a preceding fill character, the alignment character itself must not
// be mistaken for the indent character -- the default space is kept
CHECK(std::format("{:<}", j) == j.dump());
CHECK(std::format("{:>}", j) == j.dump());
CHECK(std::format("{:^}", j) == j.dump());
CHECK(std::format("{:<3}", j) == j.dump(3, ' '));
CHECK(std::format("{:>3}", j) == j.dump(3, ' '));
CHECK(std::format("{:^3}", j) == j.dump(3, ' '));
}
SECTION("fill-and-align sets the indent character, like dump(indent, indent_char)")
@@ -93,4 +111,16 @@ TEST_CASE("std::formatter<nlohmann::json>")
}
}
TEST_CASE("std::formatter<nlohmann::ordered_json>")
{
// spot-check a non-default basic_json instantiation, since the formatter
// is written against the generic NLOHMANN_BASIC_JSON_TPL_DECLARATION
// template and must actually instantiate (and behave correctly) for
// template arguments other than nlohmann::json
const ordered_json j = {{"foo", 1}, {"bar", {1, 2, 3}}};
CHECK(std::format("{}", j) == j.dump());
CHECK(std::format("{:#}", j) == j.dump(4));
CHECK(std::format("{:2}", j) == j.dump(2));
}
#endif
+2 -2
View File
@@ -306,8 +306,8 @@ TEST_CASE("Unicode (3/5)" * doctest::skip())
{
for (int byte4 = 0x00; byte4 <= 0xFF; ++byte4)
{
// skip fourth second byte
if (0x80 <= byte3 && byte3 <= 0xBF)
// skip correct fourth byte
if (0x80 <= byte4 && byte4 <= 0xBF)
{
continue;
}
+1 -1
View File
@@ -307,7 +307,7 @@ TEST_CASE("Unicode (4/5)" * doctest::skip())
for (int byte4 = 0x00; byte4 <= 0xFF; ++byte4)
{
// skip correct fourth byte
if (0x80 <= byte3 && byte3 <= 0xBF)
if (0x80 <= byte4 && byte4 <= 0xBF)
{
continue;
}
+1 -1
View File
@@ -307,7 +307,7 @@ TEST_CASE("Unicode (5/5)" * doctest::skip())
for (int byte4 = 0x00; byte4 <= 0xFF; ++byte4)
{
// skip correct fourth byte
if (0x80 <= byte3 && byte3 <= 0xBF)
if (0x80 <= byte4 && byte4 <= 0xBF)
{
continue;
}