Commit Graph
5470 Commits
Author SHA1 Message Date
Niels Lohmann bae6216491 Merge develop into remove-dead-checks/binary-formats
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-02 18:27:09 +01:00
Niels Lohmann 6d5581c4e8 CBOR/MessagePack readers: leave one-element arrays to the general path (#6008)
enter_cbor_array and enter_msgpack_array called the out-of-line scalar
run for every non-empty array. When the array's element is an object or
an array, as on every level of [[...]] or the single ring of a GeoJSON
polygon, the call reads nothing; it made 1,000 arrays nested in each
other cost 4 % more instructions (GCC 13) and up to 8 % more time (Apple
clang) than without the run. #5975 had fixed that with an inline
pre-check, which was later dropped after measuring canada only.

The run is now entered only for arrays of two or more elements: the same
comparison as before, so no other row changes. Arrays of one scalar,
which the run read faster, now take the general path; they are rare in
documents (one in citm_catalog, none in twitter or canada). The new
benchmark FromBinaryShape/singles measures them, next to
FromBinaryShape/nested.

Instructions per repetition, new/old, GCC 13 callgrind: nested arrays
0.973 (CBOR) and 0.971 (MessagePack), arrays of one scalar 1.037 and
1.021, every other row within 0.15 %.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-02 18:25:44 +01:00
Niels LohmannandClaude 590b622d48 parse(): propagate std::bad_alloc instead of crashing on a half-constructed member (#5991)
* Create the DOM parser's member value before placing it (#5985)

json_sax_dom_parser::set_member() constructed the new value in place over
the null member. basic_json(value_t) sets the type before the json_value
allocates, so a std::bad_alloc left a member typed object/array with a
null pointer, and the destructor crashed instead of propagating the
exception. Construct the value first and move it over the member (the
move constructor is noexcept).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix CI for the parse() bad_alloc sweep test (#5985)

- sweep_allocator: provide max_size(), construct() and destroy(), which
  GCC 4.8's std::map calls on the allocator directly.
- Declare the non-throwing test lambdas noexcept (-Werror=noexcept on
  recent GCC).
- Give the sax_parse lambda an explicit return type; Clang 3.4 crashes on
  the deduced one in C++11 mode.
- Skip the parser callback sweep with iterator debugging (MSVC debug
  builds, clang-cl), where containers allocate debug proxies inside
  noexcept move constructors and terminate instead of throwing, as in the
  other allocator tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N5NasAgbDGG28MqXjQvLqp
Signed-off-by: Claude <noreply@anthropic.com>

* Construct a scalar member in place; only a throwing value goes through a temporary

Creating the value first and moving it over the member is needed only when
creating it can throw (a string, a binary value or a container). A scalar
or null is again constructed in place, without the extra move and destroy
per member (and, with JSON_DIAGNOSTICS, without the extra set_parents()).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Spell the nothrow trait without && and fix the construct_member comments

The dispatch in set_member tests std::is_nothrow_constructible, not
whether the value is a scalar. Say so in the comments, and drop the &&
from the trait argument (equivalent for a forwarding reference) so that
astyle no longer inserts spaces around the template brackets.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Say "for example" in the construct_member comments

The overloads are selected by std::is_nothrow_constructible, not by the
kind of value: a custom type that cannot throw takes the first one.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Signed-off-by: Claude <noreply@anthropic.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-11-02 18:25:30 +01:00
Niels Lohmann 100f676b3e Restore the two commits of #6005 that the stack merge dropped (fixes the build of develop) (#6012)
* basic_json, iterators, serializer: remove checks that invariants make dead

Branch coverage of the test suite and of 24k mutated inputs showed
these conditions as never taken; each is implied by the code around it:

- copy_object_level(): `src_it != src_object.cend()`; dst was built from
  exactly src_object.size() unique keys and src_it advances at most once
  per member
- the general comparison operators (equal_general, less_general,
  three_way_general): the five same-type scalar cases, which the inline
  fast path of every operator handles before calling them (now an
  assert); and the NaN half of compares_unordered() in the mixed-type
  branch, reached only after all six number pairs returned, where only
  a discarded operand is unordered
- serializer, ensure_ascii: the reset of `bytes` under error_handler_t::keep
  and the surrogate-pair escape in the UTF8_ACCEPT case; every
  well-formed multi-byte character is escaped by write_u_escapes() before
  decode() sees it, so decode() only ever accepts DEL here, a single byte
  of which nothing was buffered (now asserts)
- iter_impl(pointer): the if/else chain assigned value-initialized
  iterators to the value-initialized m_it

No change in behaviour: a build that aborts on each removed side passes
the test suite, the fuzz corpora and 23.2M dumps of all 1-, 2- and
boundary 4-byte strings in both ensure_ascii modes.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* iter_impl: keep the type-dependent initialization in the constructor

The check is not needed for correctness (m_it is value-initialized), but
without it GCC 13 generates slower iteration loops: a recursive walk over
a parsed document takes 0.5-2.7 % longer (callgrind: +1.9 % instructions
on twitter.json, +0.95 % on canada.json). Clang generates identical code
either way. See the measurements in #6005.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-02 18:23:44 +01:00
dependabot[bot] bd067fd08c Bump mkdocs-git-revision-date-localized-plugin in /docs/mkdocs (#6011)
Bumps [mkdocs-git-revision-date-localized-plugin](https://github.com/timvink/mkdocs-git-revision-date-localized-plugin) from 1.6.0 to 1.6.2.
- [Release notes](https://github.com/timvink/mkdocs-git-revision-date-localized-plugin/releases)
- [Commits](https://github.com/timvink/mkdocs-git-revision-date-localized-plugin/compare/v1.6.0...v1.6.2)

---
updated-dependencies:
- dependency-name: mkdocs-git-revision-date-localized-plugin
  dependency-version: 1.6.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-02 17:57:02 +01:00
Niels Lohmann 3354436ec4 json_view: remove checks that invariants make dead (#6006)
Branch coverage of the test suite and of 24k mutated inputs showed
these conditions as never taken; each is implied by the code around it:

- view_serializer::write_codepoint<EnsureAscii>(): the only caller
  passes the byte a run of plain output stopped at (a quote, a
  backslash or a control character), so the function is now
  write_escaped(), which writes that byte's escape sequence
- load_image(): `owned_image.empty() || image != owned_image.data()`;
  both callers hand in a fresh document_data, and for load(vector&&)
  the image is owned_image itself
- set_moved(): reserve_moved(); all three callers reserved first or
  have the moved flag set (the documented precondition)
- view_side::string() and json_side::string() in equal(): only used
  after both sides were found to be strings, so they read the string
  directly (and are noexcept)
- number<false>(): the '1'..'9' test after a sign-less dispatch, which
  only happens for a digit
- value_range(): the pre-loop `is_new || off >= size` test, which the
  loop's first pass repeats on the same node

No change in behaviour: a build that aborts on each removed side passes
the test suite and the fuzz corpora.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-02 17:54:40 +01:00
Niels Lohmann 596028c460 Drop the null_result parameter JSON_IMPLEMENT_OPERATOR no longer uses (#6005)
Without the same-type scalar cases, the macro body has no use for it, and
JSON_IMPLEMENT_SCALAR_CASES is now used only by JSON_IMPLEMENT_SCALAR_FAST.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-02 17:54:39 +01:00
Niels Lohmann 5c4f585ac2 Lexer, parser, stream adapter, SAX: remove checks that invariants make dead (#6004)
Branch coverage of the test suite and of 24k mutated inputs showed
these conditions as never taken; each is implied by the code around it:

- lexer: `next_unget` in get_codepoint_bulk() and scan_literal_bulk().
  unget() is only called at the end of scan_number(), in scan_comment()
  and in skip_bom(); every token scan starts with a plain get() in
  skip_whitespace() that consumes a pending unget, and nothing between
  that get() and the bulk helpers ungets (now asserts, as in
  get_after_string_run())
- lexer: `number_type == value_integer` in convert_integer(): both
  callers exclude value_float and the type has three values
- parser: `!context.empty()` in exception_message(): all callers pass a
  literal
- input_stream_adapter: `refill` in get_character_slow(): the function
  is only called after get_character() found the get area exhausted,
  and the variable recomputed that comparison
- json_sax_dom_callback_parser: `ref_stack.back()` in key() (the null
  case returned a few lines earlier, and the callback in between cannot
  reach ref_stack), `is_structured()` in end_object() (non-null entries
  are only pushed for objects and arrays), `!key_stack.empty()` in
  current_key() (a kept object pushed its key before any value)

No change in behaviour: a build that aborts on each removed side passes
the test suite, the fuzz corpora and 40k random documents through every
input adapter in four macro configurations.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-02 17:54:39 +01:00
Niels LohmannandClaude Opus 5.5 1d69ef9ab4 Raise line coverage to 99.9 %: fix the measurement, remove unreachable code, add tests (#5997)
* CI: measure line coverage without lcov's unexecuted-block adjustment

Drop geninfo_unexecuted_blocks=1 from the lcov capture of ci_test_coverage:
it reset every line of an inlined or templated function to 0 as soon as one
copy had an unexecuted block, which understated the line coverage by about
three points.

Add -fvar-tracking-assignments to both coverage builds. At -O0, GCC inlines
always_inline functions before it places the coverage counters, so the call
sites of such helpers (put_literal() and others) got no line record and
showed as 0 in an -O3 translation unit that never reaches them. With the
flag, the call sites carry counts in every translation unit. The generated
code is identical.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Remove unreachable code found by the coverage analysis

Remove code that no input can reach, and mark the lines that cannot be
covered with LCOV_EXCL_LINE:

- BJData writer: the dry_run parameter of the ND-array element writers was
  always true; replace them by range checks.
- BON8 writer: call throw_integer_out_of_range() directly instead of
  write_bon8_value() for values that always throw.
- binary_writer: integer_bits() is only called for integers of up to 64 bits.
- serializer, view serializer, lexer: replace guards that the callers make
  unreachable by assertions or by the simpler code (surrogate pairs under
  ensure_ascii, next_unget, '-' at the end of the buffer).
- json_view: drop unreachable branches in scalar(), set(), check_utf8_scan(),
  append_input_impl() and unescape().
- Mark exception-cleanup braces, 2 GiB limits, big-endian and 32-bit code, and
  fall-through case labels with LCOV_EXCL_LINE.

No behavior changes for any input that reaches these functions.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Tests: cover the remaining lines of the library headers

Add tests for the lines of include/nlohmann that were not covered: the
binary readers and writers, the lexer and locale paths, the json_view
modules, the custom number, key and array types, json_pointer, json_patch,
diff and the iterators.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix test failures in the coverage tests

- unit-custom-object-key-type: remove the unused make_strict_key(strict_key)
  overload, the unused range constructor and default constructor, and guard
  data() with JSON_DIAGNOSTICS (unused-function/-member-function errors with
  GCC and clang -Weverything).
- unit-json_view_compare: move the two new sections to the new
  unit-json_view_compare_values.cpp, as the object file got too large for the
  MinGW assembler again, and use a std::map with std::greater instead of
  std::unordered_map, which does not accept the incomplete basic_json
  (clang 3.9, icpc).
- unit-json_view_dump: rename the variables that hid outer ones (MSVC C4456).
- unit-stream-bulk-scan: restore the real size of the oversized get area
  before the adapter is destroyed (JSON_PRECISE_STREAM_POSITION reads the
  character at the read position), and skip the test where the streambuf
  cannot hold a get area larger than an int (MSVC).
- unit-regression1: with JSON_NO_THREAD_LOCAL, copies are built member by
  member and keep correct timestamps, so check the corresponding outcome.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N5NasAgbDGG28MqXjQvLqp
Signed-off-by: Claude <noreply@anthropic.com>

* Return std::size_t from the hash helper in the ordered_json test

MSVC (Win32) rejected the implicit conversion of the unsigned long long
product to the 32-bit std::size_t (C4244 as an error). Also note when
JSON_NO_THREAD_LOCAL is set in the regression test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N5NasAgbDGG28MqXjQvLqp
Signed-off-by: Claude <noreply@anthropic.com>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Signed-off-by: Claude <noreply@anthropic.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-11-02 17:54:38 +01:00
Niels Lohmann ff0bdfdef3 Keep the byte range in exceptions about the root value (#5990)
* Keep the byte range in exceptions about the root value (#5987)

With JSON_DIAGNOSTICS and JSON_DIAGNOSTIC_POSITIONS both set, exception::diagnostics()
and the json_view diagnostics() returned early for a value without a path (the
root), dropping "(bytes a-b)". Build the "(path) " prefix only when there are
tokens and always append the byte range.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Guard the null value of a value-initialized iterator; drop the unneeded found flag

With the root's byte range, diagnostics() reaches get_byte_positions() also
with JSON_DIAGNOSTICS when there is no value, as for a value-initialized
iterator (invalid_iterator.212). get_byte_positions() now returns no range
for a null value, the same change as #6003, so either can be merged first.

In the view, a value that value_path() does not find leaves no tokens, so the
found flag added nothing to the tokens.empty() test.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-02 17:50:40 +01:00
dependabot[bot] 7328310136 Bump mkdocs-material from 9.7.7 to 9.8.0 in /docs/mkdocs (#6010)
Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.7.7 to 9.8.0.
- [Release notes](https://github.com/squidfunk/mkdocs-material/releases)
- [Changelog](https://github.com/squidfunk/mkdocs-material/blob/master/CHANGELOG)
- [Commits](https://github.com/squidfunk/mkdocs-material/compare/9.7.7...9.8.0)

---
updated-dependencies:
- dependency-name: mkdocs-material
  dependency-version: 9.8.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-02 17:48:55 +01:00
Niels LohmannandClaude Opus 5.5 bfe69c73c2 json_view: get<T>() of a boolean throws for the exact number types, as basic_json does (#5992)
* json_view: get<T>() of a boolean throws for number_integer_t, number_unsigned_t and number_float_t like basic_json (#5988)

basic_json::get<T>() of a boolean throws type_error.302 for exactly
number_integer_t, number_unsigned_t and number_float_t (dedicated from_json()
overloads) and gives 0/1 for every other arithmetic type. The view returned
0/1 for all of them, contradicting its documentation. The view now throws
the same exception (same message, with path/bytes decoration) for these three
types. basic_json is unchanged; its behavior is documented and pinned by a test.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* tests: skip throwing json_view bool get<T>() combinations without exceptions (#5988)

check_boolean_get_parity() called get<T>() for number_integer_t,
number_unsigned_t and number_float_t in its JSON_NOEXCEPTION branch, but a
boolean throws type_error.302 for exactly these types, so JSON_THROW aborted
test-json_view_cpp11/cpp17 in ci_test_noexceptions. Skip these combinations
there; the throwing checks stay in the exceptions-enabled build.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N5NasAgbDGG28MqXjQvLqp
Signed-off-by: Claude <noreply@anthropic.com>

* Expect the byte range of the root in the json_view boolean get<T>() test

With JSON_DIAGNOSTIC_POSITIONS, the type_error.302 message starts with
"(bytes 0-4)"; build the expected text with json_view_test::message(),
as the other tests of the root value do.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N5NasAgbDGG28MqXjQvLqp
Signed-off-by: Claude <noreply@anthropic.com>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Signed-off-by: Claude <noreply@anthropic.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-11-02 17:48:43 +01:00
Niels LohmannandClaude 494a74c242 insert(): strong guarantee for single elements, basic guarantee for the rest (#5996)
* Make basic_json::insert() provide its documented strong exception guarantee (#5986)

A failed allocation while the new elements were copied left the value
changed, and with JSON_DIAGNOSTICS stale parent pointers behind. The
extra work now happens almost only on the failure path:

- Array insert(pos, cnt, val) and insert(pos, first, last) append the
  copies at the end and rotate them into place. If a copy throws, the
  appended elements are erased again (noexcept) and the parents are set.
  Moving a basic_json cannot throw, so reallocation is safe. This is no
  slower than vector::insert (faster for numbers), needs no temporary
  array, and keeps the elements that libstdc++ lost on reallocation.
- Array insert(pos, val) copies a non-scalar value first and inserts it
  by move, except at the end, where vector::insert is strong. Scalars
  keep the direct path.
- The object range insert(first, last) into an empty object uses the
  container's own range insertion and clears the object on an exception.
  Into a non-empty object it inserts member by member and logs one bit
  per source member (a new member is detected by the size change; no
  allocation for the first 256 members); on an exception the members
  added are erased by walking the source range. No assumption that
  key_type is string_t.
- The parents are set when a failed insertion is undone.

The regression test fails every allocation of every insert() overload in
turn, for json and ordered_json, and checks the value and its parents.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix clang-tidy and -Wnrvo findings in insert rollback helpers

Declare the rollback helpers non-movable, replace the nested conditional
and C array in object_insert_rollback, silence the intentional copy in the
allocator test, and avoid the missed NRVO in the test factory lambda.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N5NasAgbDGG28MqXjQvLqp
Signed-off-by: Claude <noreply@anthropic.com>

* insert(): strong guarantee for single elements, basic for the rest

Rework of the previous two commits, following the discussion in #5996:
the standard containers give the strong guarantee only for a single
element (std::vector: at the end, or if the exception does not come from
copying T; std::map: single-element insert), and copying a basic_json
allocates. Rebuilding a rollback for the count, range and object-range
overloads on top of array_t/object_t made the code verbose and slower.

- insert(pos, const basic_json&): at end() forward as before (the
  standard covers it); elsewhere copy first and insert by move, as
  libstdc++ does internally (libc++ shifts the tail before copying).
- insert_iterator() and object insert(first, last) set the parents in a
  scope guard, so a failed insertion leaves a valid value with
  JSON_DIAGNOSTICS too (the next assert_invariant() aborted before).
- Docs: strong guarantee for (1), basic guarantee for (2)-(5); the
  argument checks throw before anything is changed.
- Test: allocation-failure sweep checks the strong guarantee for (1) and
  validity, type and parents for the other overloads.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* insert(): document the exception safety of overloads 2 to 5 once

The four basic-guarantee statements were identical up to array/object.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* insert(): simplify the wording of the basic guarantee

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Signed-off-by: Claude <noreply@anthropic.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-11-02 17:48:29 +01:00
Niels LohmannandClaude Opus 5.5 bdc737db6d json_view: dump() without StringType::capacity() (#5999)
* json_view: dump() without StringType::capacity() (#5994)

output_buffer::grow() compared the output size with m_out.capacity(),
which the StringType requirements do not include, so dump() did not
compile with the tests' alt_string. Keep what was last reserved in the
buffer itself and use it as a lower bound of the capacity instead.

Add json_view tests with alt_string (read-only and editable documents,
dump with several growth steps, save/load), and document reserve(n) and
resize(n) as the StringType requirements of basic_json_view::dump.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Use a raw string literal in the alt-string json_view test

clang-tidy (modernize-raw-string-literal) rejects the escaped literal.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N5NasAgbDGG28MqXjQvLqp
Signed-off-by: Claude <noreply@anthropic.com>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Signed-off-by: Claude <noreply@anthropic.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-11-02 17:48:16 +01:00
Niels LohmannandClaude 66e2eeeec0 Do not construct the base class when destroying a container (#6000)
* Do not construct the base class when destroying a container (#5995)

json_value::destroy_container() and destroy_walk() used default-
constructed basic_json values as stand-ins, which construct a
CustomBaseClass that belongs to no value. A base class whose default
constructor allocates made the noexcept destructor allocate again, and
terminate if that allocation failed, which #5762 set out to prevent.

Use data (type and value, no base class) as the stand-in instead, and
pass the children's m_data to the walk helpers.

Add a test with a counting base class: destroying a value constructs no
base object and destroys exactly one per value, also for values nested
deeper than the recursion depth limit.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Release the test value with = nullptr instead of reset()

clang-tidy (readability-ambiguous-smartptr-reset-call) rejects j.reset()
because the pointee inherits a reset() from counting_base.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N5NasAgbDGG28MqXjQvLqp
Signed-off-by: Claude <noreply@anthropic.com>

* Use std::swap for the container stand-in in destroy_container

The unqualified call after `using std::swap` would let a user-provided
swap(json_value&, json_value&) intercept the destruction path. Call
std::swap directly.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Signed-off-by: Claude <noreply@anthropic.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-11-02 17:48:03 +01:00
Niels LohmannandClaude Opus 5.5 03d408aafa JSON_DIAGNOSTIC_POSITIONS: comparing a value-initialized iterator throws instead of crashing (#6003)
* JSON_DIAGNOSTIC_POSITIONS: comparing a value-initialized iterator throws instead of crashing

With JSON_DIAGNOSTIC_POSITIONS, comparing a value-initialized iterator
(which has no container) with another iterator built the
invalid_iterator.212 message from the byte positions of the iterator's
value, which is a null pointer: UBSan reports "member call on null
pointer" in exception::get_byte_positions(), then the process crashes.
Every other configuration throws invalid_iterator.212.

get_byte_positions() now returns no positions for a null value, as the
JSON_DIAGNOSTICS path already does.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Declare the value-initialized iterator in the test const

clang-tidy (misc-const-correctness) flags it, as it is only compared.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N5NasAgbDGG28MqXjQvLqp
Signed-off-by: Claude <noreply@anthropic.com>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Signed-off-by: Claude <noreply@anthropic.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-11-02 17:47:48 +01:00
Niels LohmannandClaude Opus 5.5 76dbe8f9c9 Fix the red CI on develop: Visual Studio 2017 build of the BSON writer, Valgrind timeouts (#6009)
* CI: give the Valgrind tests a timeout of one hour

Under Valgrind, test-json_view_builder_cpp11 and test-ordered_json_cpp11
took 1435 s and 1390 s on develop (ddc34f5), close to ctest's default
timeout of 1500 s. On the next run (b2674a4) the whole job was about 10 %
slower and both timed out, failing ci_test_valgrind on develop and on
every PR based on it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N5NasAgbDGG28MqXjQvLqp
Signed-off-by: Claude <noreply@anthropic.com>

* BSON writer: drop the SFINAE guard that Visual Studio 2017 cannot deduce

write_bson_document_single_pass() was a member template enabled by
is_detected<sink_written_t, SinkType>. MSVC 19.16 (Visual Studio 2017)
fails to deduce it through the class-scope alias template (C2672, C2783)
when it compiles to_bson() and to_bon8(), so every Visual Studio 2017
job on AppVeyor fails since #5979.

The guard is not needed: as a non-template member of the class template,
the function is only instantiated when called, and its only caller is
the write_bson_document() overload for sinks that have written(), which
already calls oa.written() and oa.rewind() itself.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N5NasAgbDGG28MqXjQvLqp
Signed-off-by: Claude <noreply@anthropic.com>

---------

Signed-off-by: Claude <noreply@anthropic.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-11-02 17:47:21 +01:00
Niels Lohmann 7503394a30 Remove is_bjd_excluded_optimized_type(), which has no caller left; fix two comments
is_valid_optimized_type() rejects the markers BJData excludes, so the reader
no longer needs its own list; the test now checks is_valid_optimized_type()
of a BJData reader against the writer's list. The BSON size comment now says
that open_bson_document() rejects a size below 5.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-02 09:40:51 +01:00
Niels Lohmann caabb97337 Binary reader and writer: remove checks that invariants make dead
Branch coverage of the test suite and of 24k mutated inputs showed
these conditions as never taken; each is implied by a check made
earlier on the same path:

- check_bson_document_size: `document_size < 0` (the size was rejected
  below 5 before it was stored)
- get_ubjson_ndarray_size: `second != 'N'` ('N' is rejected as a type
  marker before `second` can be set)
- bjd_type_name() == nullptr for an ndarray dtype: every marker
  is_valid_optimized_type() accepts for BJData has a name (now an assert)
- the early is_bjd_excluded_optimized_type() check in
  get_ubjson_size_type(): is_valid_optimized_type() rejects the same
  eight markers with the same error
- `input_format == bjdata` next to `is_ndarray`, bit 8 of the type, or a
  'B'/'u'/'m'/'M' marker: these only arise for BJData
- `first != npos` next to bit 8: the bit is only set after a count
- MessagePack writer: the `<= max` halves of the negative int8/16/32
  ladder (the ladder is in the `else` of `n >= 0`)

No change in behaviour: a build that aborts on each removed side passes
the test suite and the fuzz corpora, and a 1.6M-input differential run
of the reader against the original headers found no difference.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-02 06:58:13 +01:00
Niels Lohmann b2674a4a2f BSON reader: check array element names in place in strict mode (#5984) 2026-11-01 21:03:35 +01:00
Niels Lohmann e2423a1857 CI: pin ubuntu-latest jobs to ubuntu-26.04 (#5983)
GitHub is moving the ubuntu-latest label from Ubuntu 24.04 to 26.04
between October 19 and November 19, 2026 (actions/runner-images#14748).
Pin every ubuntu-latest job to ubuntu-26.04 so the switch does not
change the toolchain under our jobs, and so the behaviour does not
depend on when GitHub reaches our runs.

Jobs pinned to ubuntu-24.04 and ubuntu-24.04-arm are unchanged.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-01 20:43:07 +01:00
Niels Lohmann 67b103e506 Fix clang-tidy errors in json_view tests on develop (#5989)
readability-math-missing-parentheses in unit-json_view_builder.cpp (#5974)
and performance-inefficient-string-concatenation in
unit-json_view_positions.cpp.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-01 20:42:58 +01:00
Niels Lohmann ddc34f546a json_view: exceptions name the path and the bytes of the value (#5974)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-01 17:25:26 +01:00
Niels Lohmann f022e3c59c Binary readers and writers: faster UTF-8 checks of strings (#5982)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-01 17:23:27 +01:00
Niels Lohmann 9742edc294 BSON writer: write to_bson(j) in one pass, with sizes written afterwards (#5979)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-01 17:23:12 +01:00
Niels Lohmann 517444b504 accept() checks strings without copying them (#5978)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-01 17:22:58 +01:00
Niels Lohmann fb515ddc34 UBJSON/BJData readers: read optimized arrays of numbers in one loop (#5981)
* UBJSON/BJData readers: read optimized arrays of numbers in one loop

An array with a type marker for a fixed-size number ($U, $i, $I, $l, $L,
$d, $D, and BJData's $u, $m, $M), including the data of a BJData ndarray,
gives its elements no markers. When the input is contiguous, read the
elements that are in the input in one loop - load, byte swap for the
format's byte order, emit - instead of one trip through
parse_ubjson_internal and get_ubjson_value per element. Elements the
input ends before, other types, and non-contiguous inputs are read as
before.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* UBJSON/BJData tests: assign the parsed value instead of casting it to void

GCC reports -Werror=unused-result for static_cast<void>(json::from_ubjson(...))
and json::from_bjdata(...): a cast to void does not silence warn_unused_result
on GCC. Assign to a json, as the other tests do.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-01 17:22:33 +01:00
Niels Lohmann edf8d17601 MessagePack reader: read runs of scalar array elements in one loop (#5980)
* MessagePack reader: read runs of scalar array elements in one loop

As for CBOR: when an array opens and the input is contiguous, read the
integers, floats, booleans, and nils that follow in a tight loop instead
of going through parse_msgpack_internal and parse_msgpack_value for every
element. The loop stops at any other kind of value and when fewer than 9
bytes are left; those values are read by the general path as before.
Non-contiguous inputs are not affected.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* MessagePack reader: enter the scalar run only when a scalar follows

As for CBOR: enter_msgpack_array tests the first byte and the remaining length
inline and calls the out-of-line run only for a scalar, so arrays of maps or
strings no longer pay for a call that reads nothing; the run's loop condition
moves to the end (do-while).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* MessagePack reader: mark the initial accepted in the scalar run as a false-positive dead store

ci_infer reports DEAD_STORE for 'bool accepted = true;' in the MessagePack
scalar run, as in the CBOR one, because every case assigns accepted before it
is read.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* MessagePack tests: assign the parsed value instead of casting it to void

GCC (ci_test_gcc) reports -Werror=unused-result for
static_cast<void>(json::from_msgpack(...)): a cast to void does not silence
warn_unused_result on GCC. Assign to a json, as the other tests in the file do.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* MessagePack reader: drop the scalar pre-check before the run

enter_msgpack_array now calls the run directly, and parse_msgpack_scalar_run
tests the remaining count and length at the top of a while loop instead of
relying on msgpack_scalar_follows(); its own switch already leaves on any
value that is not a scalar.

Instructions per repetition, new/old: SAX canada 0.946 (clang) and 0.973
(GCC 13), DOM read canada 0.991 / 0.992; no row more than 0.3 % higher.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* MessagePack reader: say why the initial accepted in the scalar run stays

As for CBOR: Infer's DEAD_STORE report is right, not a false positive. The
initializer is kept for clang-tidy's cppcoreguidelines-init-variables.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* MessagePack reader: give the scalar run the bulk tag instead of a wrapper

As for CBOR: parse_msgpack_scalar_run takes the bulk-scan tag itself and is
the function kept out of line, so parse_msgpack_array_start goes.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* MessagePack reader: start the scalar run's accepted at false

As for CBOR: Infer ignores dead stores of 0, so the suppression goes.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* MessagePack reader: let the scalar run find its array itself

As for CBOR: parse_msgpack_scalar_run reads container_stack.back() in its
bulk version, so the caller no longer evaluates it for input that is not
contiguous.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-01 17:22:32 +01:00
Niels Lohmann aecbef3949 Binary readers: swap the bytes of floats with a single byteswap (#5976)
* Binary readers: swap the bytes of floats with a single byteswap

byte_swap reversed float and double values byte by byte. GCC does not
merge that loop into one bswap, and C++23's std::byteswap accepts only
integers. Overloads for float and double now copy the bits into a
uint32_t/uint64_t, call detail::byteswap, and copy them back. This
applies to every binary reader that uses get_number.

In the CBOR reader, the single- and double-precision float cases call a
non-inlined helper. Without it, GCC stopped inlining the stream
adapter's get_character() into parse_cbor_internal, which made reading
from a std::istream about 0.5 % slower.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* CBOR reader: inline get_cbor_float() and always inline get() instead

The never-inlined get_cbor_float() costs a call for every float read
from input without bulk access: reading canada.json from CBOR through a
FILE* became 5.6 % slower on Apple clang in the benchmark suite. The
helper existed only because GCC 13 stopped inlining the stream adapter's
get_character() into parse_cbor_internal. Always inlining
binary_reader::get() fixes that as well.

callgrind, all binary formats on buffer, std::istringstream and FILE*
input (canada and citm_catalog): GCC 13 0.986-1.003, GCC 16 0.994-1.006,
Clang 23 identical. Apple clang suite: FromBinaryFile/cbor/canada 0.939,
geomean of the 53 binary reader rows 0.998.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-01 17:22:32 +01:00
Niels Lohmann 6c3ab966d3 CBOR reader: read runs of scalar array elements in one loop (#5975)
* CBOR reader: read runs of scalar array elements in one loop

When a definite-length CBOR array opens and the input is contiguous, read
the integers, floats, booleans, and nulls that follow in a tight loop
instead of going through parse_cbor_internal's container logic and
parse_cbor_value's dispatch for every element. The loop stops at any other
kind of value and when fewer than 9 bytes are left; those values are read
by the general path exactly as before. Non-contiguous inputs are not
affected (tag dispatch on the bulk-scan capability).

Also pass a member empty string as the number source to number_float
instead of constructing a std::string for every float.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* CBOR reader: address review of the scalar-run reader

- parse_cbor_scalar_run: set the size explicitly in every case that
  reads a value, instead of relying on a default that every other case
  overwrites.
- Remove cbor_scalar_follows(): parse_cbor_scalar_run already returns on
  the same conditions, and without the pre-check SAX canada needs another
  8.5 % fewer instructions with GCC 13 (0.915 against the previous
  version), with no row more than 0.5 % worse (Mac and GCC).
- Restore the comment on why parse_cbor_internal copies the container
  frame.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* CBOR reader: enter the scalar run only when a scalar follows

An array of containers or strings called the out-of-line run start once per
array only to leave it at once; on nested data (arrays of objects) that call
made reading slower than before the run reader. enter_cbor_array now tests the
first byte and the remaining length inline and calls the run only for a
scalar, so the run's loop condition moves to the end (do-while).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* CBOR reader: mark the initial accepted in the scalar run as a false-positive dead store

ci_infer reports DEAD_STORE for 'bool accepted = true;' in
parse_cbor_scalar_run because every case assigns accepted before it is read.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* CBOR reader: drop the scalar pre-check before the run

enter_cbor_array calls the run directly again, and parse_cbor_scalar_run
tests the remaining count and length at the top of a while loop; its own
switch already leaves on any value that is not a scalar. The MessagePack
reader is built the same way.

Instructions per repetition, new/old: clang SAX canada 0.983, read canada
0.993; GCC 13 read canada 0.997, the other rows at most 1.004.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* CBOR reader: say why the initial accepted in the scalar run stays

Infer's DEAD_STORE report is right, not a false positive: every case that
reaches the read assigns accepted. The initializer is kept for clang-tidy's
cppcoreguidelines-init-variables.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* CBOR reader: give the scalar run the bulk tag instead of a wrapper

parse_cbor_scalar_run takes the bulk-scan tag itself and is the function that
is kept out of line, so the one-line parse_cbor_array_start wrapper goes, as
the UBJSON typed run already does.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* CBOR reader: start the scalar run's accepted at false

Infer ignores dead stores of 0 (--liveness-ignored-constant defaults to "0"),
and false is 0 to it, so the suppression and its comment go.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* CBOR reader: let the scalar run find its array itself

parse_cbor_scalar_run reads container_stack.back() in its bulk version, so
the caller no longer evaluates it for input that is not contiguous.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-01 17:22:31 +01:00
Niels Lohmann 5337d841d4 Document and test the lifetime of string_view members decoded from a json_view (#5973)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-01 17:21:31 +01:00
Niels Lohmann 162eda2574 json_view: skip the digits of a fraction 16 bytes at a time (SSE2) (#5971)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-01 17:21:19 +01:00
Niels Lohmann 1a018577cc json_view: accept() checks the text without writing an index (#5970)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-01 17:21:06 +01:00
Niels Lohmann 76001c6b4c BSON reader: find C-string terminators 8 bytes at a time, skip array element names (#5968)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-01 17:20:54 +01:00
Niels Lohmann ec6986c918 Add natvis visualizers for json_document and json_view (#5967)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-01 17:20:41 +01:00
Niels Lohmann 3fd3e0fdaa json_view: dump and read canada.json-style floats faster (batched float decisions) (#5908)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-11-01 17:20:29 +01:00
Niels Lohmann 9c349b1d7c Write integers of 17 to 20 digits eight digits at a time in dump() (#5964)
#5634 made dump() write integers below 10^16 eight digits at a time; larger
ones still took the two-digits-per-division loop and count_digits(). Random
64-bit integers almost all have 17 to 20 digits (99.9 % of signed_ints.json
and unsigned_ints.json), so Dump/signed_ints and Dump/unsigned_ints gained
only 1.4x over 3.12.0, against 2.3x for small integers.

A 64-bit magnitude is now written as its first one to four digits and two
groups of eight. Wider magnitudes (__int128 and class types, #5805) keep the
loop: number types of at most 64 bits are selected by type, so no comparison
is reported as always true; for others the value is compared. The sign
handling of #1447/#5815 (remove_sign() into number_magnitude_t) is unchanged.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-31 13:54:45 +01:00
Niels Lohmann 367963f6bb Test escaped object keys decoded through NLOHMANN_DEFINE_TYPE from a json_view (#5972)
A json_view or json_document decodes a macro-defined struct directly and looks
the members up by their decoded keys. Only plain keys were covered so far.

The new section decodes structs defined with NLOHMANN_DEFINE_TYPE_INTRUSIVE,
..._WITH_DEFAULT and ..._WITH_NAMES from documents whose keys contain escape
sequences, and compares the result with the conversion of the materialized
value. It also pins the difference for a duplicate key written once plain and
once escaped (the view takes the first member, the materialized value keeps
the last one), and that an escaped backslash is part of the decoded key.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-31 13:54:20 +01:00
Niels Lohmann 4fe670b5f4 json_view: decode chains of unicode escapes in a loop (#5969)
After the first \uXXXX escape of a string, slow_string() now decodes the
escapes that follow it directly, while the next six bytes are \u and four hex
digits that are not a surrogate, instead of going round its main loop for each
of them (that is how Python's json.dumps writes non-ASCII text by default).
A surrogate, a bad digit, or fewer than six bytes leave the position at the
backslash: the main loop handles them as before and reports the same error at
the same position.

twitterescaped.json (simdjson-data, 549 KB, 33,036 escapes), json_document::parse(),
A/B in one binary, 6 code layouts, time after / before:

- Apple M1 Max, Apple clang 21: 0.897
- x86-64 Haswell, GCC 16.2: 0.907
- x86-64 Haswell, Clang 23: 0.910

The other files of the benchmark set (twitter, citm_catalog, canada, gsoc-2018,
apache_builds, instruments, tree-pretty, github_events, random, update-center,
semanticscholar-corpus, twitter_timeline) stay within the noise of each machine.

New test: chains of every length from 1 to 40 of escapes that give one, two,
and three bytes of UTF-8, followed by the end of the string, text, escapes,
surrogates, bad digits, and cut in every place, in keys and in values, against
json::parse().

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-31 13:54:02 +01:00
Niels Lohmann 755bb16d7a Fix the red CI on develop: remove the duplicate is_editable in basic_json_view (#5966)
#5919 replaced the bool template parameter Editable of basic_json_view by
view_mode Mode and defined is_editable from Mode. #5890 was written before
that and added a second is_editable = Editable to the private section, which
no longer compiles ("duplicate member", "undeclared identifier Editable"). It
broke every job that builds the json_view tests (macOS, MSVC, clang-cl,
clang on Windows, the C++20 module job, CodeQL).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-30 16:48:31 +01:00
Niels Lohmann d20c4dcd89 json_view benchmarks: compare fairly and state what each row means (#5914)
- bench_view: add "(editable)" rows for json_editable_document and a
  parse+dump workload, so the cost of lazy float conversion shows next to the
  parse row (the traverse workload already includes the parse). Check that
  all engines' dumps agree where outputs are comparable.
- compare.py: add --aligned, which also builds and runs with aligned code and
  reports both tables, so code-layout artifacts become visible. Off by
  default; the default output is unchanged. Not run end to end yet.
- README: say which engines are like-for-like peers (yyjson, simdjson DOM) and
  which are mutable reference points (Boost.JSON, json::parse, the editable
  json_view); explain the lazy float conversion and the eager prototype that
  was measured and dropped.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-30 15:20:12 +01:00
Niels Lohmann ea03d601af json_view: comparison without temporaries and a faster image float check (#5890)
* Compare json_view values without a vector per container and a basic_json per number

operator== of views (view/view, view/json, and editable documents) built two
vectors of members for every object it entered, sorted them with stable_sort
for every object, and converted each number to a basic_json value. It now:

- keeps the members of all open objects in one vector (reserved with
  geometric growth, so that nested objects do not reallocate), and orders
  them by insertion sort for up to 16 members, or not at all when the keys
  ascend already;
- compares numbers of one type directly, and builds a basic_json only for
  numbers of different types;
- compares short texts (keys, numbers, most strings) with two overlapping
  loads (bytes_equal(), in node.hpp);
- first walks both documents in lockstep, in the order of their text, which
  finds most pairs of equal values without the work of the comparison. The
  walk only enters an object if the comparison would visit its members in the
  same order (the object type keeps the order, the keys ascend, or all values
  are scalars), so a difference is never found later than before, and it
  records the containers it found different, so that the comparison does not
  walk them again;
- rejects values of different types and arrays of different sizes before any
  of this, and, for an object type that keeps the order, objects whose first
  (up to four) members have different keys, which ends a comparison that
  differs near the start without preparing the members.

Results are unchanged: numbers compare by value across types, duplicate keys
resolve as parse() resolves them, and objects compare in document order for
ordered_json and by key otherwise.

Instructions of one comparison of a document with an equal copy, macOS arm64
(develop -> now), json_document / ordered_json_document / editable:

  twitter    16.1M -> 7.0M   17.7M -> 1.5M    16.6M -> 7.3M
  citm       45.0M -> 3.4M   47.0M -> 3.2M    45.7M -> 4.3M
  canada    111.4M -> 13.7M  111.4M -> 13.7M  113.3M -> 15.0M
  jeopardy   26.2M -> 3.7M   27.3M -> 3.7M    26.6M -> 12.1M

and of a view with a basic_json value:

  twitter    11.4M -> 5.9M   11.2M -> 7.3M    11.6M -> 5.9M
  citm       33.9M -> 9.7M   32.6M -> 9.1M    34.6M -> 9.8M
  canada     92.9M -> 32.1M  92.9M -> 32.1M   93.9M -> 32.8M

and Linux instruction counts (GCC 13 / Clang 18) of the same comparisons:

  document with an equal copy   json_document     ordered_json_document   editable
    twitter                     -49 % / -52 %     -89 % / -89 %           -48 % / -50 %
    citm                        -90 % / -89 %     -91 % / -90 %           -88 % / -87 %
    jeopardy (4,000 questions)  -85 % / -85 %     -86 % / -86 %           -46 % / -49 %
  view with a basic_json value
    twitter                     -41 % / -45 %     -28 % / -35 %           -45 % / -44 %
    citm                        -63 % / -65 %     -63 % / -66 %           -65 % / -64 %
    jeopardy                    -39 % / -43 %     -14 % / -24 %           -44 % / -43 %

A sweep that places one difference at about 2,100 positions of eight
documents (a token at an even spacing, the first and last tokens, the first,
middle and last member of the root removed, one appended, the document
wrapped) in five kinds of comparison (view/view of read-only and of editable
documents, view/json, json_document, editable/view) finds the same results as
before and no placement slower on the Mac (Clang); on Linux no placement is
slower than 3 % (Clang and GCC).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Check the float tokens of a json_document image without a sort and a scan of every character

load(image_check::full) checked every float token of an image by sorting
the token ranges, scanning the token character by character, and comparing
the digit layout with the node's. Now:

- the ranges are sorted only if they are not sorted already (save() writes
  them in the order of the nodes, so they are);
- a token of the shape [-]digits.digits whose digit counts are the ones the
  node records (nearly all of them) is accepted by checking its digits eight
  at a time, without the general scan;
- the general scan skips runs of digits eight at a time.

Damaged tokens are still rejected as before: the check accepts exactly the
tokens that the general scan accepts.

Instructions of load(image_check::full) (develop -> now):

  macOS arm64        canada 44.6M -> 30.9M (-31 %), jeopardy 11.3M -> 10.7M,
                     citm 10.4M -> 10.0M, twitter 5.06M -> 4.94M
  Linux GCC 13       canada -50 %, jeopardy -43 %, twitter -31 %, citm -28 %
  Linux Clang 18     canada -42 %, jeopardy -48 %, twitter -35 %, citm -32 %

image_check::bounds is unchanged (within 0.4 %).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Rename names that Flawfinder takes for the file function open()

The comparison cursors' open() is now enter(), and a test's open and
close strings are head and tail.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix the compiler and lint failures of the json_view comparison

- all_digits(): the lambda declares its constant itself; MSVC does not
  let a lambda without captures read a constexpr local of the function
- equal(): the frame's "in document order" index is a namespace-scope
  constant (document_order); the local constexpr npos in the default
  member initializers of the local struct crashed GCC 9 and MinGW GCC
  and was not found by MSVC
- keys_differ_early() keeps the first keys in std::arrays (cpplint:
  variable-length arrays; clang-tidy: C arrays), and the basic_json
  side's first_keys() is not noexcept (key() may throw)
- the numbers of one type are compared in an if chain instead of a
  nested conditional operator
- the dump test avoids C arrays and temporary string concatenations

View comparison instructions are unchanged (Apple clang, twitter, citm,
canada: -0.1 % to +0.1 %).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Keep the byte comparison helpers in compare.hpp

Address review comments: bytes_equal() and bytes_less() move from node.hpp
to compare.hpp, their only user; key_less() no longer splits the comment of
resolve_indices() from its function; and the loop counters of bytes_equal()
and all_digits() are scoped to their loops.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Dispatch on the compile-time conditions of the json_view comparison

MSVC (/W4 /WX) reported C4127 (conditional expression is constant) for
the template parameter EA and the object type's ordering in an if
condition; both are now tag-dispatched.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Remove a nested duplicate guard from the json_view image test

clang-tidy reported the inner #if !defined(JSON_NOEXCEPTION) as
redundant (the test case is in the same #if) and a find() != npos, which
gets the NOLINT the other tests use.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Mark Infer's null dereference in materialize() as a false positive

Infer (ci_infer, also red on develop since #5882) reports a null
dereference in json_sax_dom_parser::null() called from materialize(). The
member pointer the parser stores a value of an object in is set by the
key() before it, so it is not null there.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Move the json_view comparison tests into their own file

With the tests this branch adds, the object file of unit-json_view_dump.cpp
got too large for the MinGW x64 assembler ("file too big"). The test case
"json_view comparison" is now unit-json_view_compare.cpp, unchanged and
also built with C++17.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Build the DOM parser with Visual Studio 2015 again

Since #5882, the three VS 2015 jobs of AppVeyor fail on develop:
- the detection of keys that can be indexed instantiated std::hash for
  every string type, which VS 2015 rejects with a static_assert for a type
  it does not hash (alt_string): std::hash is now looked at only for
  objects in a vector
- the frame stack copied its frames with std::copy to a raw pointer, which
  VS 2015 reports as unsafe (C4996, an error with /WX): a loop now

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* json_view: no per-object allocations in ordered_json comparisons

For an object type that keeps the order of its members (ordered_json),
resolve_indices() allocated a vector per compared object, and
resolve_members() two. They now use one working vector owned by equal(),
whose capacity is reused, and resolve_indices() needs none when the keys
are unique. resolve_members() sorts small objects by insertion, as the
other paths do.

Instructions per comparison (callgrind, GCC 13 and clang 18, x86-64):
twitter, ordered_json view == ordered_json: 0.939 / 0.941; 20000 small
objects with a duplicate key: 0.741-0.768; all other rows (json,
citm_catalog, canada, view == view) within 0.988-1.003.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-30 15:19:42 +01:00
dependabot[bot] e2cb15c479 Bump the codeql-action group across 1 directory with 4 updates (#5961)
Bumps the codeql-action group with 4 updates in the / directory: [github/codeql-action/init](https://github.com/github/codeql-action), [github/codeql-action/autobuild](https://github.com/github/codeql-action), [github/codeql-action/analyze](https://github.com/github/codeql-action) and [github/codeql-action/upload-sarif](https://github.com/github/codeql-action).


Updates `github/codeql-action/init` from 4.39.0 to 4.39.1
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/6f2ac81f5cd81fe351a197ba0b00d91d03cd2aee...9b3254085494b55a0f7c4c2b3d2d52cc712ce370)

Updates `github/codeql-action/autobuild` from 4.39.0 to 4.39.1
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/6f2ac81f5cd81fe351a197ba0b00d91d03cd2aee...9b3254085494b55a0f7c4c2b3d2d52cc712ce370)

Updates `github/codeql-action/analyze` from 4.39.0 to 4.39.1
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/6f2ac81f5cd81fe351a197ba0b00d91d03cd2aee...9b3254085494b55a0f7c4c2b3d2d52cc712ce370)

Updates `github/codeql-action/upload-sarif` from 4.39.0 to 4.39.1
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/6f2ac81f5cd81fe351a197ba0b00d91d03cd2aee...9b3254085494b55a0f7c4c2b3d2d52cc712ce370)

---
updated-dependencies:
- dependency-name: github/codeql-action/analyze
  dependency-version: 4.39.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: codeql-action
- dependency-name: github/codeql-action/autobuild
  dependency-version: 4.39.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: codeql-action
- dependency-name: github/codeql-action/init
  dependency-version: 4.39.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: codeql-action
- dependency-name: github/codeql-action/upload-sarif
  dependency-version: 4.39.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: codeql-action
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-30 15:19:08 +01:00
dependabot[bot] c451fac1bc Bump step-security/harden-runner from 2.22.1 to 2.22.2 (#5962)
Bumps [step-security/harden-runner](https://github.com/step-security/harden-runner) from 2.22.1 to 2.22.2.
- [Release notes](https://github.com/step-security/harden-runner/releases)
- [Commits](https://github.com/step-security/harden-runner/compare/ccd8616d44fd3846e67624a50d5aad6d37bf2d25...dc2769abcc0fcfbadfc1df54081c1608579ef6b2)

---
updated-dependencies:
- dependency-name: step-security/harden-runner
  dependency-version: 2.22.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-30 15:18:56 +01:00
dependabot[bot] 28f71914b5 Bump egor-tensin/setup-mingw (#5963)
Bumps [egor-tensin/setup-mingw](https://github.com/egor-tensin/setup-mingw) from 41b837e47d7f85214629d255b9c4bc3fcbe9fd63 to d54f5ef0ce585bd3fbb1f8a71b5b254f59dbf931.
- [Release notes](https://github.com/egor-tensin/setup-mingw/releases)
- [Changelog](https://github.com/egor-tensin/setup-mingw/blob/master/CHANGELOG.md)
- [Commits](https://github.com/egor-tensin/setup-mingw/compare/41b837e47d7f85214629d255b9c4bc3fcbe9fd63...d54f5ef0ce585bd3fbb1f8a71b5b254f59dbf931)

---
updated-dependencies:
- dependency-name: egor-tensin/setup-mingw
  dependency-version: d54f5ef0ce585bd3fbb1f8a71b5b254f59dbf931
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-30 15:18:44 +01:00
Niels Lohmann 0b91f9ddd8 json_view: check_utf8 skips well-formed runs with the parser's string scan (#5926)
* json_view: check_utf8 skips well-formed runs with the parser's string scan

Strings of 32 bytes or more that are written into an editable document
are validated with scan_string_run(), which skips well-formed UTF-8 16
bytes at a time; the byte-wise check runs only from where the scan found
ill-formed UTF-8, to report the same type_error.316. Shorter strings
keep the byte-wise check (inlined with a constant start, so that this
path is the code it was), where the vector scan costs more than it
saves.

set()/push_back() of strings of 32 bytes or more take 0.29-0.76 of the
time (x86-64 GCC, averaged over code layouts) and 0.25-0.80 (AArch64
clang); shorter strings take 1.5-4 % fewer instructions; the json_view
rows of bench_edit execute at most 0.04 % more instructions.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* tests: avoid sign conversion in json_view_edit UTF-8 check

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* json_view: use while in check_utf8_bytewise

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-30 15:17:55 +01:00
Niels Lohmann 0115bb47b3 Name the source and license of adapted code in the file comments (#5965)
The files with code adapted from fast_float, Zmij and simdjson carried the
authors' SPDX copyright lines, but their file comments did not say what
was taken or under which license: number_parse.hpp, to_chars.hpp,
simd_scan.hpp and view/simd.hpp now do, as zmij.hpp and the yyjson-based
view files already did. README and the license page now also name
fast_float's eight-digit parsing and the lexer's use of simdjson's UTF-8
check. Comments only.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-30 15:17:24 +01:00
Niels Lohmann ea8a05484b Fix implicit-conversion sanitizer failure in unit-class_parser.cpp (#5960)
Constructing a std::string from uint8_t iterators converts values >= 0x80
implicitly to char, which ci_test_clang_sanitizer rejects. Convert each
byte with an explicit static_cast.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-30 09:56:06 +01:00
Niels Lohmann 2aecd86b37 Document the contract of object keys passed to json_sax::key and key callbacks (#5956)
* Document the contract of object keys passed to json_sax::key and key callbacks

Describe what the name passed to json_sax::key and to a parser callback
for parse_event_t::key holds when parsing JSON text: escapes are decoded
to UTF-8, different spellings of a name yield identical strings, the name
is validated before the call (invalid escapes or UTF-8 give a parse error
and no key event), no normalization or case conversion is applied, the
event precedes the ':' check, and duplicate names are reported once per
occurrence. Also note that sax_parse passes binary-format keys unchanged.

See discussion #5955.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Add a test for the object key contract and correct the BON8 note

Pin the guarantees documented for json_sax::key and the key event of
parser callbacks: decoded names, validation before delivery, delivery
before the colon, one event per repeated name, and the binary formats.
BON8 rejects ill-formed UTF-8 in a name even with sax_parse, so the
note no longer lists it with the formats that pass bytes through.

See discussion #5955.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix the GCC build of the object key contract test

GCC 11 in C++11 mode rejects a default argument on a lambda parameter
(-Wc++14-extensions) and ignores a static_cast<void> on the
warn_unused_result of parse(). Pass the input format explicitly and bind
the parse result to a variable.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Add version history for the object key contract

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-29 07:37:33 +01:00
Niels Lohmann 5eccbbd2bc Fix clang-tidy misc-redundant-expression in unit-json_view_image.cpp (#5958)
Split the combined has_save/has_load static_asserts so clang-tidy no
longer sees both sides of && as equivalent (develop ci_clang_tidy failure).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-29 07:36:11 +01:00