amorfo77/json
1
0 Fork 0
mirror of https://github.com/nlohmann/json.git synced 2026-10-31 09:55:42 +00:00
Code Issues Packages Projects Releases Wiki Activity
5,447 Commits 28 Branches 57 Tags
cf9bc9f71b261b0e242ab50337316827b1a49378
Commit Graph
391 Commits
Author SHA1 Message Date
Niels Lohmann cf9bc9f71b Merge branch 'ci/tiers' into ci/windows-macos-sets 
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-30 15:36:35 +01:00
Niels Lohmann 9e1d35eda9 Merge branch 'ci/path-filters' into ci/tiers 
Keep the tiered job layout and apply the harden-runner 2.22.2 bump
from #5962.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-30 15:36:22 +01:00
Niels Lohmann 0115bb47b3 Name the source and license of adapted code in the file comments (#5965) 
The files with code adapted from fast_float, Zmij and simdjson carried the
authors' SPDX copyright lines, but their file comments did not say what
was taken or under which license: number_parse.hpp, to_chars.hpp,
simd_scan.hpp and view/simd.hpp now do, as zmij.hpp and the yyjson-based
view files already did. README and the license page now also name
fast_float's eight-digit parsing and the lexer's use of simdjson's UTF-8
check. Comments only.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-30 15:17:24 +01:00
Niels Lohmann 2aecd86b37 Document the contract of object keys passed to json_sax::key and key callbacks (#5956) 
* Document the contract of object keys passed to json_sax::key and key callbacks

Describe what the name passed to json_sax::key and to a parser callback
for parse_event_t::key holds when parsing JSON text: escapes are decoded
to UTF-8, different spellings of a name yield identical strings, the name
is validated before the call (invalid escapes or UTF-8 give a parse error
and no key event), no normalization or case conversion is applied, the
event precedes the ':' check, and duplicate names are reported once per
occurrence. Also note that sax_parse passes binary-format keys unchanged.

See discussion #5955.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Add a test for the object key contract and correct the BON8 note

Pin the guarantees documented for json_sax::key and the key event of
parser callbacks: decoded names, validation before delivery, delivery
before the colon, one event per repeated name, and the binary formats.
BON8 rejects ill-formed UTF-8 in a name even with sax_parse, so the
note no longer lists it with the formats that pass bytes through.

See discussion #5955.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix the GCC build of the object key contract test

GCC 11 in C++11 mode rejects a default argument on a lambda parameter
(-Wc++14-extensions) and ignores a static_cast<void> on the
warn_unused_result of parse(). Pass the input format explicitly and bind
the parse result to a variable.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Add version history for the object key contract

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-29 07:37:33 +01:00
Niels Lohmann a4c4e92dbd Merge wip/5949 into ci/windows-macos-sets 
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-28 20:32:12 +01:00
Niels Lohmann ae57b09bc6 Merge wip/5948 into ci/tiers 
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-28 20:32:11 +01:00
Niels Lohmann 54f6b3ef06 json_view: source positions in materialize() with JSON_DIAGNOSTIC_POSITIONS (#5945) 
* json_view: source positions in materialize() with JSON_DIAGNOSTIC_POSITIONS

With JSON_DIAGNOSTIC_POSITIONS, materialize() now gives every value the
start_pos()/end_pos() that basic_json::parse() would give it, so exceptions
about materialized values name the same bytes (#5864).

- json_sax_dom_parser takes the lexer type as a defaulted template
  parameter; materialize() passes position_source, which reports the
  positions it derives from the index. diagnostic_positions treats npos as
  "no position" and keeps ranges all-or-nothing.
- Strings with escapes keep the offset of their opening quote plus one in
  `next` (only with the macro); materialize() scans from there to the closing
  quote. The end of an array or object is found by scanning from the end of
  its last element over whitespace, comments and a trailing comma.
- In editable documents, values written by edits, arrays and objects whose
  elements changed, and containers whose last element has no position get
  none. Images keep the flag is_new, and load() checks that `next` of a
  string is within the text.

Nothing changes on the parse path without the macro. The view's own
exceptions still carry no byte range, so the tests that compare them with
those of materialized values skip it under the macro, as they skip the path
under JSON_DIAGNOSTICS.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Indent comment in unit-json_view_from_json.cpp

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Address review: clearer position code and docs

- position_source gets clear(); set_scalar() returns early for new values,
  switches on node_kind, and uses no local copies; the one position_source
  per materialize() call is documented
- no npos + 1 for containers without a position; a comment on why the end
  of a container is reported as its closing bracket
- less duplication under #if in materialize() and the builder
- diagnostic_positions sets npos explicitly
- docs: positions of basic_json values after changes, positions in images
  depend on how the document was parsed, when next is set, and the cost of
  positions while parsing

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Share the comment scanners of the view builder with materialize()

skip_line_comment() and find_block_comment_end() in scan.hpp are used by
the builder's comment() and by position_source::container_end(), which
had its own copy.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Address review: if/else for container positions, clearer next field

- the opening bracket of a container sets positions with an if/else instead
  of a ternary, and the frame takes its start from positions
- the closing bracket of a container is found in nested if/else; the parent's
  last_end is taken from positions, so it is npos whenever no range is set
- architecture.md: the next field reads "or 0 if it was not; all other
  values: 0" instead of "else 0; otherwise 0"

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix GCC unused-result and clang-tidy raw-string findings in unit-json_view_positions.cpp

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-28 20:24:49 +01:00
Niels Lohmann 091c10d4a5 Fix self-aliasing in json_pointer::operator/= and patch_inplace (#5936) 
* Fix self-aliasing in json_pointer::operator/= and patch_inplace

json_pointer::operator/=(const json_pointer&) passed iterators into its
own token vector to insert() when a pointer was appended to itself
(p /= p), which is undefined behavior; libstdc++ produced "/a/b//" for
"/a/b". The pointer now appends a copy in that case.

basic_json::patch_inplace iterated the patch while changing *this, so
j.patch_inplace(j) read freed memory. It now applies a copy of the
patch when the patch is the value itself. A patch stored inside the
value remains undefined behavior and is documented as such; copying the
patch unconditionally would double the time of a large patch.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix JSON_DIAGNOSTICS test crash and clang-tidy hit in patch_inplace self-aliasing

The spare-capacity test reallocated non-empty elements through get_ref(), which leaves stale parent pointers under JSON_DIAGNOSTICS; reserve while the array is empty instead. Suppress performance-unnecessary-copy-initialization for the copy that is needed because *this is modified.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Detect a patch stored inside the patched value with JSON_DIAGNOSTICS

patch_inplace applied a copy only when the patch was the value itself. With JSON_DIAGNOSTICS the parent pointers make it cheap to also detect a patch nested inside the value, so that case is handled the same way. Without diagnostics it stays documented undefined behavior.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Silence misc-const-correctness for patch_aliases_this

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix clang-tidy findings in the operator/= self-append test

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-28 20:23:55 +01:00
Niels Lohmann 4c3e92b96e Split the macOS and Windows jobs into a PR set, a full set, and a nightly set 
The 19 macOS and 32 Windows jobs ran on every push to every pull request.
They now use the same sets as the Ubuntu jobs:

- PR set (16): four Visual Studio 2022 and two Visual Studio 2026
  configurations, ARM64 Debug, Clang 20.1.8 with MinGW, MinGW x86, the
  C++20 module test, Xcode 16.0, 16.4, 26.3 and 26.6, C++14 on the
  latest Xcode, and the Swift Package Manager check.
- full set (34): every other Visual Studio configuration, Clang
  version, clang-cl, Xcode version and C++ standard.
- nightly set (1): 64-bit MinGW, which takes about 68 minutes. A failed
  nightly run opens or updates an issue.

No configuration is removed: tools/ci_jobs/ci_jobs.py --compare lists
157 configurations before and after, and all jobs keep their runner,
steps and environment. ci_jobs.py now checks all three workflows and
expands matrices that consist only of include entries correctly.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-27 19:45:39 +01:00
Niels Lohmann 087df7b0c2 Split the Ubuntu jobs into a PR set, a full set, and a nightly set 
All ~100 Ubuntu jobs ran on every push to every pull request. They now
belong to three sets, chosen by each job's if: condition:

- PR set (16 configurations): every push to a pull request and every
  push. Together with the checks that always run, these jobs would
  have caught at least 92 % of the failing commits of Sep/Oct 2026.
- full set (73 configurations): after the PR set passed, on pushes and
  on pull requests with the "🚀 ready to merge" label.
- nightly set (10 configurations): Infer, Valgrind, coverage, icpc,
  icpx, nvhpc, CUDA, and ARM64; nightly on develop, on pushes to master
  and release/*, and by hand. A failed nightly run opens or updates an
  issue.

A matrix whose entries belong to different sets is split into one job
per set; the jobs share their steps through YAML anchors. No
configuration is removed: tools/ci_jobs/ci_jobs.py --compare lists 150
configurations before and after. The same tool checks in CI that every
job belongs to exactly one set, that the full set waits for the whole
PR set, and that the job table in quality_assurance.md is up to date.

The documentation describes the sets in a new section "How CI runs",
and the QA pipeline figure shows the new badges.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-27 19:40:21 +01:00
Niels Lohmann eb8f675bc9 docs: explain signed zero in the diff() round-trip exceptions (#5944) 
Spell out that numbers comparing equal produce no operation even if one is
zero and the other negative zero, in diff.md and features/json_patch.md.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-27 17:16:08 +01:00
Niels Lohmann 78a0687147 Strong exception guarantee for operator[](json_pointer) (#5940) 
* Keep the value unchanged when operator[] with a JSON pointer throws

The non-const operator[](json_pointer) is documented with strong exception
safety, but get_unchecked created intermediate values while walking the
pointer and kept them when a later reference token failed, e.g.
json j; j["/a/99999999999999999999999"_json_pointer] threw 404 and left
{"a":[]}; padding an existing array or appending with "-" behaved alike,
and so did std::length_error/std::bad_alloc from a huge index.

get_unchecked now only walks existing values up to the last reference
token. The last token's key insertion or array growth is strongly
exception safe by itself. Any other change goes through
get_unchecked_create, which records the one change it makes to an existing
value (null converted, key inserted, array grown) in a RAII guard and
reverts it if a later step throws; everything else it creates lies below
that change. The guard needs no try/catch, so JSON_NOEXCEPTION builds are
unaffected. Successful calls create the same values as before.

Docs: state the guarantee in the exceptions list and version history of
operator[], fix the complexity of the pointer overloads of operator[] and
at, and warn that padding arrays up to an untrusted index allocates memory
proportional to the index (operator[].md, untrusted_input.md).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Make the strongly exception-safe pointer operator[] faster than before

The previous commit made get_unchecked strongly exception safe but slower
with GCC 16 when it creates values (up to +6 % instructions for appending
with "-"). This restructures it so that no benchmark row is slower than
before the strong guarantee, with GCC 16, Clang 22 and Apple Clang.

- get_unchecked walks existing values in one loop that inserts nothing.
  The last reference token is resolved in that walk: "-" appends with
  emplace_back(), an index grows the array with operator[], and a missing
  key is inserted with emplace_hint() at the position the walk's
  lower_bound() found, so the key is searched once.
- With std::less<string_t> or std::less<> as comparator, the walk checks
  the key at the lower bound for equality instead of equivalence, which is
  the same for these comparators and cheaper.
- get_unchecked_create (not inlined) handles all other changes. Its guard
  swaps a null value with a fresh null value instead of copying it and
  reverts the change in a non-inlined function, so a successful call pays
  one compare for it. As every null value converted afterwards is a fresh
  one, it is converted in place instead of assigning a new array or object.
- Keys are added to new objects with emplace(), which also replaces the
  basic_json::operator[] call for ordered_json.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix rollback of pointer operator[] for unusual object and string types

Follow-up to the strong exception guarantee of the non-const
operator[](json_pointer):

- get_unchecked_create assumed that emplace() always inserts the key that
  the walk did not find. For object types whose lookup and emplace()
  disagree (e.g. a copied fifo_map), nothing was inserted, the rollback
  erased the existing member, and the walk continued with a value of the
  wrong type (assertion, type_error.305, or a crash). It now checks the
  result of emplace(), arms the rollback only if a key was inserted, and
  otherwise continues the walk with the member found, as basic_json's
  operator[] does: such pointers throw out_of_range.404 and leave the value
  unchanged, as before the strong guarantee.
- The rollback erased the inserted key by key, which converts string_t to
  object_t::key_type and may allocate inside a noexcept function
  (std::terminate on bad_alloc with custom key types). It now erases by the
  iterator emplace() returned, and shrinks a grown array with erase() at
  the end instead of resize(). The documentation states the remaining
  requirement for custom template parameters.
- Creating values no longer calls string_t::begin(), so the non-const
  operator[](json_pointer) compiles with string types that lack it (as the
  alternative string type of the tests).
- Correct the @throw lists of get_unchecked and get_unchecked_create.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Address review comments on operator[](json_pointer) rollback

- undo() shrinks the array with resize() instead of erase()
- comments say what "will be" done in get_unchecked_create
- tests: avoid a clang-tidy performance-unnecessary-copy-initialization
  hit (unit-alt-string) and the word "access" in section names, which
  Flawfinder flags as race/access

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Let the pointer rollback guard call undo() unconditionally

The check in the destructor and JSON_HEDLEY_NEVER_INLINE on undo() made no
measurable difference: instruction counts of j[ptr] are identical with
GCC 13 -O3 and within 0.4 % with clang, timings are within noise.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-27 17:15:55 +01:00
Niels Lohmann a47170346a docs: new design for the documentation site (#5934) 
* docs: new design for the documentation site

Apply the brand to the Material for MkDocs site: Inter and JetBrains Mono
served by the site (upright and real italic), brand colours, blue header band
with the syntax pattern, wordmark in the header, new home page (hero, cards,
animated feature tour), 404 page, polish for tabs, search, footer, focus
rings, tables and code blocks, and theme features (code copy, previous/next
footer, TOC follow, search suggestions).

Draw the number railroad diagram, the parser callback events and the iterator
illustrations as inline SVGs that follow the light and dark scheme. The
generator scripts are in docs/mkdocs/scripts/figures; the feature slideshow
(json-light.svg, json-dark.svg, replacing json.gif on the home page) is built
by docs/mkdocs/scripts/slideshow. Decision nodes in five flowcharts become
hexagons, which need far less room.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: add figures, table captions and a macro chooser

- Figures with numbered captions: SAX stepper, json_view layout and edit
  stepper, exception map, JSON pointer and patch walkers, QA pipeline,
  scaling and binary size charts, roadmap lanes, release timeline,
  integration decision tree and namespace builder
- Table captions (nj-table-caption) on 9 pages
- Compile times: modules and PCH charts, filesystem section, re-measured
  Figure 1, links between features/modules.md and compile_times.md
- Interactive macro chooser for features/arbitrary_types.md
- Link check: exclude URLs that answer the plugin with 400/401/419 or are
  dead (customers.md)

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: macro chooser generates adl_serializer for non-default-constructible types

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: cross-link concept pages and API pages in See also sections

Add See also links in both directions between the concept pages (features,
home, integration) and the API pages, write API links in code font, give every
entry a comment after a dash, and show the callback event figure on the parser
callbacks page.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: address review comments on the new design

- Home: add a "Latest release" button
- Sponsors: update the list and group it by tier
- Features: list all feature pages on the index page
- Releases: add highlights per release, link the 4.0 roadmap, add the first commit to the timeline
- Exceptions: prefix all example messages with the exception name
- API: no code font for format names in the See also links of the to_/from_ pages
- Merge Patch: show the document before, the patch, and the result
- Replace "--" with typographic dashes, mark external links with an icon, center Mermaid diagrams

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-27 07:41:42 +01:00
Niels Lohmann 21ec9bd801 json_view: fewer allocations when parsing small documents (#5915) 
- Size the buffer of decoded strings on its first growth from the input (a
  quarter of it plus 16 bytes, at most 4 KiB) instead of 272 bytes, so that a
  document with escapes does not grow it twice, with a zero-filled copy each
  time.
- Allocate the node index of documents with an estimate of up to 1024 nodes
  (was 256) together with the header, which saves one allocation. When the
  estimate is too low (dense numbers), that block stays unused: at most 16 KiB
  instead of 4 KiB. shrink_to_fit() cannot release it separately; the
  documentation says so.

Fresh parse of a 2.5 KB minified tweet: 993 to 958 ns (-3.5 %), traverse and
select -3 %, with no row consistently slower (Apple M1 Max, 30 rounds, three
runs). Dense numeric documents parse about 9 % faster and keep up to 16 KiB
more memory.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-27 07:22:49 +01:00
Niels Lohmann 94fe31eb9b docs: writing generic code for json_view (#5924) 
* docs: writing generic code for json_view

A "Writing generic code" section on the json_view page: why a function
taking json_view accepts no other view, the three signatures for
editable, read-only, or either kind of view (passed by value), why the
modes do not convert, is_json_view, and how a from_json() reads views
(constrained to json values and views; the macros and the library's
conversions do it already) with what differs then (duplicate keys,
JSON_DIAGNOSTICS paths).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: name the documents the generic view function accepts

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: views of read-only documents convert to editable views

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-26 20:13:32 +01:00
Niels Lohmann 9e67bfa115 json_view: the library's std conversions read views directly (#5923) 
* json_view: the library's std conversions read views directly

The library's from_json() overloads (containers, maps, std::array, C
arrays, pairs, tuples, optional, valarray, forward_list, enums, binary,
paths, and scalars) accept views: their traits use the view's
basic_json type, and the few accesses that differ (get_ptr) go through
small helpers. from_json() of a view now finds them; only the probe
still looks up by ADL only. get<T>() of a view thus converts these types
without materialize(), with the same values and exceptions; from an
object, a repeated key keeps the last value, as parse() does.

The view keeps its direct paths for vectors of the types it converts
directly (faster than from_json()) and for containers of views; its
other vector and map paths are replaced by from_json(). For basic_json,
std::valarray and std::forward_list are read in one pass (40 % and 3.5 %
fewer instructions); all other conversions are unchanged.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* json_view: enable_if_not_view_t instead of '!' in the from_json() templates

astyle spaces out the template argument list when it contains a negation;
the alias spells the condition with negation<> and leaves the lines alone.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* json_view: compact the template parameter lists astyle had spaced out

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-26 20:13:31 +01:00
Niels Lohmann e5fe4ebdd2 json_view: from_json() of the macros reads views directly (#5922) 
get<T>() and get_to() of a view call T's from_json() with the view itself
when that from_json() accepts views but not an unrelated probe type, and
fall back to materialize() otherwise. The from_json() functions of the
NLOHMANN_DEFINE_TYPE_*, NLOHMANN_DEFINE_DERIVED_TYPE_* and
NLOHMANN_JSON_SERIALIZE_ENUM(_STRICT) macros accept basic_json values and
views; a derived type whose base reads only json reads the base part
through materialize(). The library's own from_json() overloads are not
called with views: from_json() of a view (and of the probe) only looks up
functions by ADL.

New public trait nlohmann::is_json_view<T>, and the member type
basic_json_type of basic_json_view. With duplicate keys, the members of
such a T are those the view's lookups find (the first ones).

bench_decode.cpp measures decoding into structs (4-6x faster than
materialize().get<T>() on twitter, citm_catalog, and synthetic records).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-26 20:13:31 +01:00
Niels Lohmann 5a2819c940 json_view: one SIMD macro set with the library (#5921) 
The view's vector code follows JSON_NO_SIMD, like the lexer and the float
writer, and skips the CPU check of SSSE3 when the code is compiled for
SSSE3 anyway (__SSSE3__), like the lexer. JSON_VIEW_NO_SIMD and
JSON_VIEW_USE_SSSE3 and their ABI tags _vns and _vssse3 are removed; they
were never released. JSON_NO_SIMD keeps its tag _ns.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-26 20:13:31 +01:00
Niels Lohmann 4cdfa6d0f4 json_view: no save() and load() on big-endian targets (#5920) 
The image format is little-endian only. Instead of throwing
type_error.320 at runtime, save_image(), load_image() and the members
save() and load() do not exist on a big-endian target, so a call fails to
compile, like set() on a read-only document. type_error.320 remains for
saving a discarded document.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-26 20:13:30 +01:00
Niels Lohmann 731f1b5bfb json_view: view_mode enum instead of bool Editable, node_kind for node::kind (#5919) 
* json_view: view_mode enum instead of bool Editable, node_kind for node::kind

basic_json_document and basic_json_view take a view_mode (read_only, the
default, or editable) instead of a bool: basic_json_document<json, true>
did not say what true means. The shortcut aliases are unchanged.

The kind of a node is a node_kind (the value_t values plus link, 10 as
before), which removes the static_cast<std::uint8_t>(value_t::...) of
every comparison. The node and image layouts are unchanged.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* json_view: node_kind in the node writers, spell out read_only, comment fixes

The builder and the edit functions take and store a node_kind instead of
casting a value_t; the numbering assertions and the comments now speak of
node_kind, and the link kind notes that it follows value_t::discarded.
The read-only aliases name view_mode::read_only like the editable ones.
view_mode.md says the mode is a compile-time property of the type.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* json_view: views of read-only documents convert to editable views

A read-only document has no edits, so a view of it reads correctly as a
view of an editable document before its first edit. The implicit
conversion lets a function taking a json_editable_view accept the views
of both modes. There is no conversion the other way: a read-only view of
an editable document would read wrong values after the next edit.

The mode comparisons in the cross-mode operator==/!= use an
is_other_mode trait, which astyle keeps compact.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-26 20:13:30 +01:00
Niels Lohmann 24078c862d docs: fix BON8 documentation gaps found in the format review (#5932) 
* docs: fix BON8 documentation gaps found in the format review

Verified each claim with small C++ probes before writing it.

- index.md: BON8 serialization is also incomplete for invalid UTF-8.
- bon8.md: document duplicate keys (last wins), the error ids for
  malformed input incl. 0xF5..0xF7 + 0x80..0xBF, exact string
  termination wording, wide integers and discarded values in the
  unsupported list, and title two untitled admonitions.
- exceptions.md: to_bon8 in type_error.316, BON8 in parse_error.112
  (with example) and .113, wide integers in out_of_range.407.
- binary_values.md: bytes 0..39 take one byte in BON8.
- from_bon8.md: note on strict UTF-8 and duplicate keys.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Address review of the BON8 documentation

Restore the correct statement that marker bytes cannot begin a UTF-8 character (0x80..0xBF occur in valid UTF-8 as continuation bytes), and reword two sentences.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: drop the discarded-value remark from the BON8 page

Discarded values are not mentioned on the individual format pages.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: do not list invalid UTF-8 strings as a BON8 serialization gap

JSON text must be valid UTF-8, and to_bon8 rejects ill-formed strings by design (type_error.316), as documented on the BON8 page. Restore the original cell and realign the table.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-26 18:25:46 +01:00
Niels Lohmann 60107eda3f Read BJData ND-arrays of float16, restrict ND-array char to 0..127, correct BJData docs (#5931) 
* Read ND-arrays of half floats and enforce the char limit in ND-arrays

BJData N-dimensional arrays with element type 'h' (float16) failed to
read with a misleading parse_error.112, because the reader's dtype name
table lacked 'h'. It now maps to "half", the JData name. The writer still
does not pack "half"; such an object is written as a plain object and
reads back equal.

BJData limits 'C' (char) to 0..127 in Draft 2 and Draft 3, and the 1-D
reader enforces this, but ND-arrays with _ArrayType_ "char" wrote values
up to 255 as 'C' elements and read them back without a check. The writer
now only packs values in 0..127 (others fall back to a plain object that
round-trips), and the reader rejects 'C' elements above 127 with
parse_error.113 like the 1-D reader.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: correct and complete the BJData documentation

Verify the BJData chapter against the library and the Draft 2/3/4
specifications and fix what did not match:

- bjdata.md: correct the list of unused markers (N, not Z), the ND-array
  example (3x2, not 2x3), the conditions under which an object is packed
  as an ND-array (exactly three keys, row vectors, float-only elements for
  single/double, char 0..127, byte only in Draft 3), the binary table row
  and notes (which flag/draft combinations write [$B), the "$" type
  restriction (Draft 3, Z is excluded too), the H marker in the
  deserialization table, and the claim of a complete deserialization
  mapping. Document the supported drafts, that Draft 4 (SoA containers,
  extension type E) and column-major ND-arrays are rejected, ND-arrays of
  half floats, duplicate object keys, and link the Draft 3 specifications
  instead of master.
- Add an API page for basic_json::bjdata_version_t with an example, and
  link it from the nav, the API index and to_bjdata.
- Update binary_formats/index.md and binary_values.md, which still claimed
  that BJData has no binary type, and fix a wrong byte comment there.
- Document parse_error.115 and the out_of_range.408 cases for from_bjdata,
  the precise other_error.502 condition for to_bjdata, and reword the note
  on truncated unsigned integers in exceptions.md (only ND-array elements
  were truncated before 3.13.0).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Regenerate single_include and apply astyle

Result of make amalgamate (including make pretty) for the preceding changes.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Address review of the BJData documentation

The $ type restriction exists since Draft 2 (Draft 3 added B), $ requires a count, ND char above 127 is rejected on read, and the new API page is listed in alphabetical order in the nav.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: format tables, use the Oxford comma

Reformat the tables changed by this PR (BJData serialization and
deserialization mappings, completeness and binary-value tables) so all
rows are aligned, and add the Oxford comma to lists of three or more
items in the added text.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-26 18:25:46 +01:00
Niels Lohmann dc768a7d5d Fix BSON binary subtype 0x02 and check the declared document size earlier (#5930) 
* Write and read the inner length of BSON binary subtype 0x02

The BSON specification defines the payload of the "binary (old)" subtype
0x02 as an int32 followed by that many bytes. to_bson wrote the plain
payload instead, so decoders such as pymongo and libbson-based readers
rejected the bytes, and from_bson returned the inner length as part of the
payload (documented as lenient behavior in gh-5333).

The writer now emits the outer length (payload + 4), the subtype, the inner
int32 holding the payload size, and the payload. The reader requires an
outer length of at least 4 and an inner length equal to the outer length
minus 4 (the cases the bson-corpus lists as decode errors), reports parse
error 112 otherwise, and returns only the payload. Both directions change
together so that the round trip keeps working.

The single-reservation fast path for scalar entries declines subtype 0x02
because it has no room for the inner length.

This is a deliberate exception to the opt-in rule for behavior changes:
the previous output was invalid BSON.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Check the declared BSON document size before reading its content

The BSON reader compared a document's declared size with the number of bytes
read only when it reached the terminator. The declared size never bounded
the reads, so a SAX handler saw the keys and values of a document whose size
was too small before parse error 112 was reported.

The container frame now stores the position where the document ends
according to its declared size. open_bson_document rejects a size below 5
(size prefix plus terminator) and an embedded document that does not end
before the terminator of its enclosing document; the element loop rejects an
element whose type byte leaves no room for the name terminator and the
document terminator. The terminator check stays and still catches a declared
size that is too large. No input that was accepted before is rejected now;
documents with a wrong size are reported earlier, with fewer SAX events, a
different byte position and, in three cases, a different message. The error
id (parse_error.112) is unchanged and strict=false still does not skip the
check.

The tests that asserted the old messages and positions are updated, with the
outer sizes of two inputs corrected so that they still exercise the embedded
document. New tests cover the SAX events, a size below 5, and an embedded
document that fits exactly.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Document BSON subtype 0x02, document size checks, and lenient reads

Describe the inner length of binary subtype 0x02 that to_bson now writes and
from_bson checks and removes. Add the document size checks (parse_error.112,
not affected by strict=false) to bson.md and from_bson.md, and extend the
list of lenient reads with the UUID and MD5 payload lengths (not checked),
duplicate keys (last value wins), embedded NUL bytes in strings, and the
ignored trailing bytes with strict=false. State that the members of an object
are written in iteration order.

Correct the statement that element names are never validated on read: they
are checked and sanitized like string values by the error handler. Add the
BSON message to the examples of out_of_range.415 and type_error.321, the
subtype limit of 255 to the binary values page, CBOR to the formats that use
byte_container_with_subtype, and fix a typo in the incomplete mapping box.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Regenerate single_include and apply astyle

Result of make amalgamate (including make pretty) for the preceding changes.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Address review of the BSON documentation

Describe exactly when a too small document size is detected and note that subtype 2 data written by earlier versions is not readable.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: use the Oxford comma, drop discarded-value remarks

Add the serial comma in a list added by this PR and remove the BSON
example of type_error.321 (with the type_error.317 remark) so the docs do
not mention discarded values for BSON; restore the original singular
heading of that section. No tables were changed by this PR.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix -Weffc++ in the BSON recorder test and a clang-tidy use-auto hit

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix clang-tidy readability-redundant-member-init in BSON test

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-26 18:25:46 +01:00
Niels Lohmann 0c3570d44a Validate the UBJSON/BJData container type marker for count 0; fix UBJSON docs (#5929) 
* Validate the optimized-container type marker for count 0 in UBJSON and BJData

The '$' type marker of an optimized container was only checked when an
element was read, so containers with count 0 accepted any marker:
`[$x#i0`, `{$x#i0` and `[$E#i0` parsed as empty containers although a
non-empty container with the same marker is rejected and the UBJSON
specification requires a legal marker after '$'.

get_ubjson_size_type() now checks the marker right after '$' against the
allowed set (UBJSON: Z N T F i U I l L d D H C S [ {; BJData: B U i u I m
l M L h d D C, with the existing exclusion of T F Z N S H [ {) and reports
parse_error.112 "marker ... is not a permitted optimized array type".

Input that was accepted and is now rejected: an optimized container with
count 0 and an invalid '$' type marker. Valid input is unaffected. Two
existing tests that used an invalid marker incidentally ("[$#" and a
negative-count case) were adapted.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Correct and complete the UBJSON documentation

Fix statements that disagree with the behavior (checked with probes):
- `Z` is null; the unused marker is `N`.
- `$` requires `#`; use_type with use_size=false throws other_error.502 only
  for non-empty arrays, objects and binary values.
- NaN and infinity are written as IEEE 754 float64 and read back unchanged;
  the UBJSON specification encodes infinity as null.
- `H` is documented (semantics, parse_error.115, out_of_range.406); the
  "complete mapping" claim is qualified.
- use_type "same type" means same marker.
- Long Z/T/F arrays lose only the `$` annotation; the `#` count and the
  element markers are kept.
- from_ubjson lists the 112, 113, 115, 406 and 408 causes that occur for
  UBJSON, without the BJData-only ND-array wording.
- Duplicate object keys keep the last value; the `$` type marker is
  validated even for count 0.
- Smaller fixes: float32 row, "map" -> "object", https link, exception 110
  strict case, to_ubjson history of discarded values.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Regenerate single_include and apply astyle

Result of make amalgamate (including make pretty) for the preceding changes.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Address review of the UBJSON documentation

Clarify that empty containers carry no $ annotation of their own and note that a counted N object is not supported.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: format tables, use the Oxford comma, drop discarded-value remarks

Reformat the two UBJSON mapping tables that this PR changed (uniform column
widths, padded cells), use the serial comma in lists of three or more in the
added text, and restore the original type_error.321 bullet in to_ubjson.md.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix narrowing of the optimized-container count in BJData and UBJSON tests

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-26 18:25:45 +01:00
Niels Lohmann 2bbe80c043 Document MessagePack ext types and fix related doc errors (#5928) 
* Document MessagePack ext types and fix related doc errors

Add an "Extension types (ext, fixext)" section to the MessagePack page:
ext/fixext are read as binary, the signed ext type is exposed as an
unsigned subtype 0..255 (-1 reads as 255), Timestamp is not interpreted,
subtypes are always written as ext/fixext, and subtypes above 255 throw
out_of_range.415. Add the 0xC1 row to the deserialization table, note
that duplicate keys keep the last value, and correct the NaN/Infinity
explanation, the "31 bytes / 15 elements" wording, the ill-formed UTF-8
sentence, and the generic 0xC1-key error message.

Add fixext16 and the unsigned subtype mapping to binary_t.md and
binary_values.md, reword "maximum of the MessagePack ext type" in
to_msgpack.md and exceptions.md, and mark MessagePack deserialization
as incomplete (non-string keys) in the binary formats overview.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Address review of the MessagePack documentation

Name parse_error.113 for a 0xC1 map key, move the duplicate-key sentence out of the ext section, and keep the 0xC1 row in marker order.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-26 18:25:44 +01:00
Niels Lohmann d5da84249f Skip CBOR tags in front of map keys, validate indefinite text chunks, fix CBOR docs (#5927) 
* Skip CBOR tags in front of map keys and validate indefinite text chunks

A tag in front of a text string map key was rejected with parse_error.113
even with cbor_tag_handler_t::ignore or store, although tags in front of
values are handled. The tags are now skipped iteratively before the key is
read; the handler error still rejects them.

The chunks of an indefinite-length text string were concatenated before
the single UTF-8 check, so a code point split over two chunks, or an
invalid chunk, was accepted. RFC 8949, Section 3.2.3 requires every chunk
to be valid on its own. With error_handler_t::strict, replace and ignore
each chunk is now checked; keep is unchanged. The chunks are appended with
append(ptr, len), so StringType needs nothing new.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: correct and extend the CBOR documentation

Fix the negative-integer rows (they used MessagePack boundaries), the
number_integer range of the 0x1B row, and the half-precision entry in the
list of unused types; correct the NaN history (0xF9 since 3.8.0) and the
tagged-item wording (only the tag directly in front of a byte string is
stored, and only in the 0xD8..0xDB form). Document that to_cbor is not
RFC 8949 deterministic, that duplicate keys are accepted (last wins),
tags in front of keys, per-chunk UTF-8 validation, the exceptions
parse_error.113, out_of_range.406/408 of from_cbor, the pointer and
C array input traps, the partially written output on exceptions, and the
store/ignore results in the binary values example. Drop the claim that
the CBOR reader needs StringType::append(const StringType&) and
BinaryType::insert(pos, first, last).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Regenerate single_include and apply astyle

Result of make amalgamate (including make pretty) for the preceding changes.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Address review of the CBOR changes

Cite RFC 8949 Section 5.3.1 for duplicate keys, describe the out_of_range.408 condition precisely, rewrap a long line, and update the stale get_string comment (per-chunk UTF-8 check). single_include regenerated.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: drop discarded-value remarks

Remove the mention that a discarded value cannot be serialized to CBOR
(type_error.321) from the exception-safety and mapping sections. The
integer table and all added prose already follow the table-format and
Oxford-comma rules.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-26 18:25:44 +01:00
Niels Lohmann 81d66291c5 Create objects for pointer tokens that cannot be array indices (#5358) 
* create objects for tokens that cannot be array indices (#5357)

When operator[](json_pointer) traverses a level that does not exist yet,
the null value is turned into an array or an object depending on the
reference token. The check only tested whether all characters are digits,
so tokens that can never be a valid array index selected an array and
then failed:

- "01" (and any other token with a leading '0') threw parse_error.106
- the empty token threw out_of_range.404

Both tokens are valid object keys, and both work when the level already
exists as an object, so creating the level changed the outcome. Test the
token against the RFC 6901, Sect. 4 grammar for array indices instead, so
that such tokens create an object. This only affects pointers that threw
before.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Clarify that the token 0 is a valid array index in operator[] docs

The note described valid array indices as "a nonempty sequence of digits
without a leading 0", which read literally excludes the token "0" itself,
contradicting both the code and the /foo/0/0/0 example in the next
sentence. Spell out the RFC 6901 grammar instead.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-26 17:13:48 +01:00
Niels LohmannandClaude Sonnet 5.5 c712df0637 docs: make the diff() and patch() documentation accurate; add tests (#5939) 
* Throw out_of_range.411 for JSON Patch add into a null parent (#5917)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MKJf48vnGtTE25KscmjfaW
Signed-off-by: Niels Lohmann <niels.lohmann@gmail.com>

* docs: make the diff() and patch() documentation accurate; add tests

Document the exceptions to the diff round-trip invariant (NaN, discarded
values, number types), the path parameter, non-minimal patches, array and
object operation order, the ordered_json fallback, binary values in
serialized patches, the real complexity, duplicate member names in
operations, test on missing paths, and the exact out_of_range.403 message.
Add tests for these behaviors.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <niels.lohmann@gmail.com>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
 2026-10-26 17:12:05 +01:00
Niels Lohmann 7a10bf25d2 docs: state merge_patch complexity and RFC 7396 limitations; test non-object, discarded, binary cases (#5938) 
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-26 17:11:55 +01:00
Niels Lohmann c20b6eddf4 docs: correct JSON Pointer documentation and add tests (#5937) 
Fix inaccurate or missing statements about unflatten (third round-trip
failure, empty token, typo), value (type_error.306 before pointer lookup),
contains (pointer overload), parent_pointer and operator<< (bad_alloc),
out_of_range.410 example, template parameter requirements (resize(n)), and
document the URI fragment form and non-unique member names of RFC 6901.
Add tests for the unflatten and value() cases.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-26 17:11:46 +01:00
Niels LohmannandClaude Sonnet 5.5 d6431a9104 Throw out_of_range.411 for JSON Patch add into a null parent (#5917) (#5918) 
Claude-Session: https://claude.ai/code/session_01MKJf48vnGtTE25KscmjfaW

Signed-off-by: Niels Lohmann <niels.lohmann@gmail.com>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
 2026-10-26 17:11:25 +01:00
Niels Lohmann 1cf5492bcd docs: byte layout sections, glossary entries, and QA page update (#5910) 
* docs: byte layout sections, glossary, and QA page updates

- Move "Byte layout" of each binary format to its own section before
  "Serialization" and describe the idea of the format.
- Glossary: add LLDB and other terms used throughout the documentation.
- quality_assurance: correct Clang Static Analyzer and CMake entries; add
  CodeQL, Semgrep, Flawfinder, CIFuzz, OpenSSF Scorecard, REUSE, Meson/SwiftPM,
  and reproducibility checks.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: drop unused length colour from CBOR and MessagePack byte layout legends

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-25 12:08:14 +01:00
Niels Lohmann 330fbead59 Document clang-tidy exclusions, drop unneeded ones, and clean up doc examples (#5909) 
* Document clang-tidy exclusions and drop unneeded ones

Re-enable every excluded check in a run of clang-tidy 22 over all tests
and remove the seven that never fire (cert-err58-cpp,
clang-analyzer-webkit.NoUncountedMemberChecker,
cppcoreguidelines-virtual-class-destructor, google-readability-function-size,
hicpp-function-size, hicpp-no-assembler, readability-function-size).

Document the reason for every remaining exclusion at the top of the file.

Exclude in advance the checks that only exist in clang-tidy 23 and are
noise here (bugprone-signed-bitwise, *-explicit-constructor,
readability-trailing-comma, modernize-use-string-view), plus
modernize-use-integer-sign-comparison, which needs C++20 std::cmp_less
and flags deliberate casts.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix clang-tidy findings in documentation examples and replace redundant aliases

Apply clang-tidy fixes to the documentation examples:
- performance-avoid-endl: std::endl -> '\n'
- readability-qualified-auto: auto -> auto* / const auto*
- performance-for-range-copy: copy -> const reference
- google-readability-casting: C-style cast -> static_cast

Replace redundant aliases (readability-redundant-qualified-alias) such as
"using json = nlohmann::json;" with "using nlohmann::json;" in the
documentation, README, examples, tests, and benchmarks. Aliases at class
scope and aliases of other types (with_base_class_t etc.) are kept.

No changes to the library or its public API. All changed examples were
compiled and run; their output matches the stored .output files (except
max_size and meta, which depend on the platform and compiler).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Format doc examples with the pinned astyle

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-25 12:07:51 +01:00
Niels Lohmann fe231dc692 docs: write std::chrono durations as unit/value objects; reword a title with escaped quotes (#5911) 
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-25 12:07:32 +01:00
Niels Lohmann ffd4242dc2 Docs: fill documentation gaps found in user questions (#5907) 
* Docs: fill documentation gaps found in user questions

Add pages on class-hierarchy/smart-pointer/variant/chrono recipes, working
without exceptions, and large, streamed and partial input. Extend the FAQ
(23 entries, Known bugs first), parse error documentation (empty-input
checklist, line/column, diagnostic positions), element access, conversions,
creating values, ADL troubleshooting, memory, and build/distribution notes.
Add 14 examples with generated output. Correct the statement that parse errors
cannot be recovered from with -fno-exceptions.

The topics come from recurring questions on Stack Overflow, Reddit, Hacker
News, YouTube, and the project's GitHub discussions and issues.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: declare range-for lists in variables to keep astyle quiet

astyle reformats a braced init list placed directly in a range-for
header; declare the list first so check_amalgamation passes.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-25 12:07:13 +01:00
Niels Lohmann cc917d7123 docs: cross-reference json_view and json_document for 3.13.0 (#5892) 
* docs: cross-reference json_view and json_document for 3.13.0

Document the new zero-copy document/view API across the docs: add a
"Documents and views" section to the basic_json API hub linking to all
specializations, point parsing/conversion/modifying pages to the view-based
alternatives, add document/view sections to feature pages, update the
landing page to mention json_view.hpp/json_literals.hpp, and cross-link
debugging/exceptions.

Also document wide-string inputs (and ill-formed UTF-16/32 rejection),
add parse/accept examples for wide strings, fix related doc gaps, and
align features/parsing and FAQ with the added support.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: fix article before std::istream in FAQ

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: clarify document inputs, decoding, get<T>() and editing

Address review comments: say which inputs are borrowed and that a wide
string pointer is an extra input, which values are decoded during parsing,
that get<T>() reads only built-in types directly, all forms of set() and
that the document's insert() is array-only, and how to store wide strings.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* docs: integers are converted while parsing a document

Also say "They accept every input" in basic_json_document::parse.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-25 06:18:07 +01:00
Niels Lohmann 52b72b5684 Improve documentation: byte layouts, release pages, See also, dark-mode images (#5906) 
- Make the iterator range illustrations legible in dark mode (prefers-color-scheme
  rules inside the SVGs).
- Add a "Byte layout" section with a color-coded byte table to each binary format
  page (BJData, BON8, BSON, CBOR, MessagePack, UBJSON), and highlight the smallest
  size per file in the comparison table.
- Add "See also" sections to the overview pages that lacked one.
- Split the release notes into one page per release, grouped by minor version on
  the overview page, with an "API changes" section per release generated from the
  API pages' version histories (hooks/api_changes.py).
- Fact-check the release notes and version histories against the release tags and
  correct wrong or misleading entries (e.g., json_serializer 2.1.0, merge_patch
  3.1.0, items, count, operator[], to_bjdata, std::string_view key support).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-24 17:57:44 +02:00
Niels Lohmann dd7686a269 json_view: faster reading, indexing and C-stream parsing (#5888) 
* Read C streams into a json_document in blocks

json_document::parse(FILE*) (and read(), accept(), parse_copy()) went through the
library's file input adapter and appended the text character by character
with fgetc. Read it with fread in blocks instead, like the std::istream
overload already does.

Parsing citm_catalog.json (1.7 MB) from a FILE*:
  Apple M1: 80.6 M -> 8.4 M instructions, 4.8 ms -> 0.55 ms (parse of the same text from a std::string: 8.7 M)
  Linux x86-64 Ir (whole run), GCC 13 / Clang 18: 47.0 M -> 13.7 M / 45.8 M -> 14.2 M
The parsed documents and the error cases are identical (checked over the test corpus
and 30 mutants of each file). A null FILE* is handled as before (by the library's adapter).

Belongs to the PR "json_view: read-side speedups".

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Index arrays without subtrees directly

operator[](idx), at(idx) and back() skipped the elements of an array one
after the other, whatever they were. In an array none of whose elements is
an array or object (numbers, strings, booleans, null) the elements are
adjacent nodes, which the array node's own sizes tell: its subtree has
exactly one node more than it has elements. Such an element is now the
node idx after the array node; any other array is walked as before.

Reading r[i] for every i of an array of 3000 integers:
  Apple M1: 28.8 M instructions, 3.1 ms -> 0.1 M instructions, 4 us
  Linux x86-64 Ir (whole run), GCC 13 / Clang 18: 84.1 M -> 3.1 M / 75.0 M -> 3.0 M
Arrays that contain arrays are unchanged.

Belongs to the PR "json_view: read-side speedups".

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Skip the bounds checks of number and string tokens in NUL-terminated input

For a NUL-terminated input (std::string, C string, string literal), the
parser reads the terminator instead of comparing p with the end, via
cursor::cur(). Number and string scanning still tested `p != e` before each
look at the next byte (after the integer digits for '.', for 'e', after
the exponent sign and digits, and after a string for its closing quote),
although the terminator can never be one of those characters. They now use
cur(), which is the plain load for NUL-terminated input and the bounds check
for ptr+len input, as before.

Parsing, Apple M1 instructions per parse: twitter 3.126 M -> 3.089 M (-1.2 %),
citm_catalog 8.70 M -> 8.58 M (-1.3 %), jeopardy 306.1 M -> 302.5 M (-1.2 %), signed
integers 281 M -> 266 M (-5.4 %), canada and floats -0.1 % / -0.9 %.
Linux x86-64 Ir, GCC 13 / Clang 18: twitter -0.5 % / -0.0 %, signed integers +0.2 % / -0.7 %.

Belongs to the PR "json_view: read-side speedups".

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Make the index text of items() on arrays lazy

basic_json_view::items() on an array formatted every element's index with
std::to_string when the item was made, whether or not the loop asks for the
key. The item now keeps the position and makes the text on the first key()
call. Objects are not affected.

Iterating over the items of an array of 3000 numbers and reading only the values:
  Apple M1: 25.5 us -> 4.6 us (0.45 M -> ~0 net instructions)
  Linux x86-64 Ir (whole run), GCC 13 / Clang 18: 14.9 M -> 4.1 M / 15.3 M -> 4.6 M
Reading key() costs what it did (24.5 us).

Belongs to the PR "json_view: read-side speedups".

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Make the default of value() with a C string only if the member is missing

basic_json_view::value(key, "text") and value(pointer, "text") built a
std::string from the default first and passed it on, so a call allocated
for the default even when the member exists, and copied it again when it
does not. They now look the member up and make the result directly: from the
member if there is one, else from the C string. Behavior, including the
exceptions, is the same.

Two calls (member found, member missing) with a 40 character default, Apple M1:
  1.6 K -> 0.7 K instructions
  Linux x86-64 Ir (500 rounds), GCC 13 / Clang 18: 2.46 M -> 2.22 M / 2.40 M -> 2.15 M

Belongs to the PR "json_view: read-side speedups".

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Materialize scalars directly and keep the stack of small subtrees local

basic_json_view::materialize() (and get<BasicJsonType>()) always set up the
SAX handler that parse() uses and a std::vector for the open containers,
sized from the document's deepest nesting. For a scalar, which is most of what
is materialized one value at a time, neither is needed: the value is made
directly (the same constructors the handler ends up calling). For arrays and
objects the stack of open containers is now a local array for the first 16
levels, and the vector is only used for deeper documents.

Materializing five scalars of a small document, Apple M1: 3.6 K -> 0.9 K instructions
  Linux x86-64 Ir (500 rounds), GCC 13 / Clang 18: 2.96 M -> 2.37 M / 2.90 M -> 2.29 M

Belongs to the PR "json_view: read-side speedups".

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Copy the text between escapes while scanning it

A string that contains an escape is decoded by builder::slow_string(). After
each escape it scanned the next run of plain bytes (8 or 16 byte-by-byte
steps, then a vector loop), and then copied that run to the decode arena in a
second pass. The run is now copied 16 bytes at a time while it is scanned
(one vector load, a store of the same 16 bytes, and the mask of the bytes
that end the run), and the write position lives in a local variable, not in
the builder, since a store of a character may alias the builder's members and
made the compiler reload it after every escape. A run that ends at a quote
needs no scan, and non-ASCII text goes through the UTF-8 check as before;
the portable code is unchanged.

Parsing, Apple M1 instructions: jeopardy.json 304.0 M -> 293.8 M (-3.4 %), 3000 strings with
escapes 1.72 M -> 1.54 M (-10 %), twitter -0.6 %.
Linux x86-64 Ir, GCC 13 / Clang 18: escaped strings -7.6 % / -11.4 %, twitter 0.0 % / 0.0 %.
Documents and error messages are identical on 900 k random strings of escapes, UTF-8 and control
characters (also under ASan), and on mutants of the test corpus.

Belongs to the PR "json_view: read-side speedups".

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Hash the last bytes of a key of a large object without a loop

key_hash() mixed a key eight bytes at a time and then packed the remaining
0-7 bytes one by one, in a loop. The last word of a key of eight or more
bytes is now read as the last eight bytes of the key (overlapping the word
before), and a key of 2-7 bytes as two overlapping words of four or two
bytes. A product only moves bits upwards, and the slot is taken from the low
bits: the high bits of the last word are folded down before the
multiplication, since keys that differ only in their last characters ("user_17",
"user_18") would otherwise share slots. The hash is not stored, so its values
may change.

Looking up every key of an object of 1000 random keys, Apple M1 instructions per 1000 lookups:
  keys of 5 bytes 175 K -> 93 K, hex strings of 8 bytes 177 K -> 168 K, keys of 12 bytes 196 K -> 186 K
Linux x86-64 Ir (whole run), GCC 13 / Clang 18: 5-byte keys -4.7 % / -8.8 %, 12-byte keys -3.3 % / -7.4 %.
Building the tables while parsing such objects is cheaper as well (5-byte keys: 13.3 us -> 10.1 us).

Belongs to the PR "json_view: read-side speedups".

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Compare short keys of a large object without a call of memcmp

find_indexed() compared the key of a member the table points to with
memcmp, a library call for any length. Up to 16 bytes, the bytes are now
compared as two overlapping words (the first and the last 8, 4, or 2 bytes),
as find_member() already does for the objects without a table; longer keys
still use memcmp.

Looking up every key of an object of 1000 random keys, Apple M1 instructions per 1000 lookups
(with the previous commit): keys of 5 bytes 93 K -> 72 K, hex strings of 8 bytes 168 K -> 148 K,
keys of 12 bytes 186 K -> 170 K, keys of 24 bytes 198 K -> 193 K (unchanged, memcmp).
Linux x86-64 Ir (whole run), GCC 13 / Clang 18: 5-byte keys -7.2 % / -7.3 %, 12-byte keys -5.9 % / -6.6 %.

Belongs to the PR "json_view: read-side speedups".

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Read an unedited editable document like a read-only one

Views of a json_editable_document followed edits at every step: the
lookup, the element access, back(), dump() and materialize() were built
for nodes that can be moved or linked, though no edit changes the index until
the first one (every edit creates the edit state before it touches a node).
Until then these now take the read-only code: one check of the document's
edit state per call, not per member or per node. After the first edit
nothing changes. Views that were made earlier keep working, as the check is
made when they are used.

Unedited editable document, Apple M1: dump() of twitter.json 1.40 M -> 1.22 M instructions (-12.6 %);
lookups unchanged. Linux x86-64 Ir, GCC 13 / Clang 18: dump -1.7 % / -1.0 %, lookups -0.1 % / 0.0 %.
Walking with iterators is unchanged, and so is dump() after an edit.

Belongs to the PR "json_view: read-side speedups".

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Format the numbered test keys of json_view's index with a stream

The keys were made with snprintf() and a format from a list, which
Flawfinder reports as an error (CWE-134, a format string that is not a
literal). A stream now formats them; the 30000 keys of the test are the
same as before.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Put the key number of the json_view index test in parentheses

astyle formatted i * step after << as a pointer declaration (i* step),
which failed the amalgamation check.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix the compiler and lint findings of json_view's read side and tests

- find() and contains() dispatch on Editable for the member lookup; for
  documents that cannot be edited both branches were the same call
  (GCC -Wduplicated-branches)
- json_view.hpp includes <cstdio> for the FILE overload (cpplint)
- the arena check macro of the builder is on one line (cpplint read it
  as an if body without braces)
- tests: no nested conditional (also MSVC C4018), no useless cast,
  parentheses around mixed operators, raw string literals, += instead
  of temporary concatenations, and NOLINT for the fixed seeds and a
  find() != npos, as elsewhere in the tests

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Grow the decode arena through a function and store its position back on every return

slow_string() kept its write position in locals and grew the arena through
the macro NLOHMANN_VIEW_ENSURE. The macro is now the always-inline member
arena_ensure() (a lambda is not inlined at every call site by clang, which
then keeps the position in memory: +11 % instructions on escape-heavy
strings), and a guard stores the position back into m_arena_write on every
return instead of by hand on the successful one. Instruction counts are
unchanged (clang) or 1 % lower (GCC).

Also word the complexity of back() as that of at().

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-24 07:41:06 +02:00
dependabot[bot] 1e9222f955 Bump dompurify from 3.4.16 to 3.4.17 in /docs/mkdocs/scripts/mermaid (#5904) 
Bumps [dompurify](https://github.com/cure53/DOMPurify) from 3.4.16 to 3.4.17.
- [Release notes](https://github.com/cure53/DOMPurify/releases)
- [Commits](https://github.com/cure53/DOMPurify/compare/3.4.16...3.4.17)

---
updated-dependencies:
- dependency-name: dompurify
  dependency-version: 3.4.17
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-10-24 07:37:42 +02:00
Niels Lohmann ffcd392be0 Fix Semgrep findings in Python tooling (#5899) 
- generate_docset.py: use SHA-256 instead of SHA-1 for the cache file
  name digest of downloaded images.
- generate_docset.py: suppress dynamic-urllib-use-detected; the URL
  scheme is restricted to http(s) right before the call.
- generate_natvis.py: suppress direct-use-of-jinja2; the template
  renders a local XML file with autoescape enabled.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-23 07:11:17 +02:00
dependabot[bot] e28506dd86 Bump jsdom from 30.1.2 to 30.1.3 in /docs/mkdocs/scripts/mermaid (#5898) 
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-23 06:54:23 +02:00
Niels Lohmann 6fa14d8f95 Write binary formats through reserved sink space and number runs (#5885) 
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-23 06:52:58 +02:00
Niels Lohmann 533cbfee51 Let generate_natvis.py cover custom ABI tag combinations (#5880) 
Since #5877, the checked-in nlohmann_json.natvis only covers namespaces
with up to two ABI tags. The generator gets options to produce a file
for other configurations:

- --abi-tags TAGS adds the namespaces of one configuration (repeatable)
- --all generates every combination of ABI tags
- --list-tags prints the tags with the macros that enable them
- --output FILE writes somewhere else than the repository root

`make natvis NATVIS_ARGS="..."` passes the options through. The default
output is unchanged. The options are documented in the generator's
README, the debugging page, and FILES.md, and checked in CI.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-21 22:57:54 +02:00
Niels Lohmann e9970066c5 Scan strings with SSE2, SSSE3, or NEON (#5853) 
* Scan strings with SSE2, SSSE3, or NEON

The string fast path found the end of a run of plain characters 8 bytes at
a time (SWAR) and validated non-ASCII text one sequence at a time, calling
validate_one_utf8() out of line for each.

The vector code of json_view's scanner, in the new shared header
detail/input/simd_scan.hpp, now does both 16 bytes at a time: SSE2 finds
the end of a run on x86-64, and the UTF-8 check of Keiser and Lemire runs
with NEON on AArch64 and with SSSE3 on x86-64. SSSE3 is not part of x86-64,
so the check is compiled for it with a function attribute and used when the
CPU has it (cached in a constant-initialized atomic), else the scalar check;
code compiled with -mssse3 skips the CPU check. The first word of a run is
still checked with SWAR (the first four with NEON, where a vector mask
costs more), as most keys and short values end there, and the scalar
helpers are always inlined. On anything else, the scalar code is unchanged.

The new macro JSON_NO_SIMD selects the portable code everywhere, including
the SSE2/NEON digit formatting of the floating-point writer. The lexer tests
now also run with JSON_NO_SIMD and, on x86-64, with -mssse3, and a new test
places every byte that ends a run at each offset across several 16-byte
blocks, after ASCII and after CJK text.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Keep the string scan from slowing down short strings and dump() on Arm

Three benchmarks were 7-9% slower than develop on Apple M1 (Apple Clang,
-O3 -flto): Accept/jeopardy, Dump/citm_catalog, and Dump/jeopardy.

- dump(): the serializer passes a string without its closing quote, so
  plain_run_end() ran out of input and called the out-of-line NEON loop
  for every string, with fewer than 16 bytes left. The loop is now only
  called for 16 bytes or more.
- accept(): no string of jeopardy.json reaches the vector UTF-8 check,
  yet compiling it into the lexer's scan function made it 5% slower: the
  function runs once per token and now saved more registers and kept a
  stack slot. lexer_string_bulk_run() checks the first four words inline
  and continues out of line, and the serializer keeps string_bulk_run()
  inline, which is faster there.
- The byte that ends a run is read from the scanned buffer instead of
  through get(), which Apple Clang merged with the other get() calls that
  load the read position again right after it was stored (3%).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Define string_bulk_run_from() only where lexer_string_bulk_run() calls it

With NEON and simdutf in C++17, lexer_string_bulk_run() takes the
string_bulk_run() path, so the out-of-line helper was unused.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Do not dispatch the SSSE3 string scan with GCC's C++20 modules

GCC does not keep the target("ssse3") attribute on functions it reads
back from a module interface, so the module build (ci_module_cpp20,
gcc:latest) rejected the intrinsics in vector_utf8_run() with "needs isa
option -mssse3". With __cpp_modules, GCC now uses the portable UTF-8 check
like other compilers without the target attribute; the SSE2 scan stays,
and -mssse3 or more still compiles the SSSE3 check directly. Clang is not
affected.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Include <cpuid.h> only once when json.hpp and json_view.hpp are both used

The SSSE3 dispatch of the string scan and the one of json_view each
include <cpuid.h>. Before GCC 11 the header has no include guard, so a
translation unit that includes both headers fails with GCC 10:
"redefinition of 'unsigned int __get_cpuid_max(unsigned int, unsigned int*)'".
Guard both includes with NLOHMANN_JSON_CPUID_INCLUDED.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Add an ABI tag for JSON_NO_SIMD

JSON_NO_SIMD selects between two definitions of the internal scanning and
formatting functions. Like JSON_VIEW_NO_SIMD (_vns), it now appends _ns to
the inline namespace, so translation units built with and without it can
be linked into one program without ODR violations.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-21 20:00:50 +02:00
Niels LohmannandKartikey 3f8677d028 Limit Natvis namespace variants to two ABI tags (#5877) 
The generator wrote a namespace entry for every combination of ABI tags,
so nlohmann_json.natvis doubled with each new tag. With the 12 tags on
develop it had reached 57.9 MB, and one more tag would push it past
GitHub's 100 MB file limit. It now covers the default namespace, every
single tag, and every pair of tags (versioned and unversioned), which is
158 entries / 1 MB with 12 tags.

Taken over from #5868. Refs #5863.

Signed-off-by: Kartikey <kg704925@gmail.com>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Co-authored-by: Kartikey <kg704925@gmail.com>
 2026-10-21 18:26:07 +02:00
Niels Lohmann ef1cdb59ac Update performance docs for fast std::istream and FILE* parsing (#5875) 
performance.md still claimed streams have no fast path (outdated since #5828).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-21 12:25:10 +02:00
Niels Lohmann bd5227c2ec Fix silent wrong results for custom number types (#5805) 
* Fix silent wrong results for custom number types

Squashed onto develop cd6b428ef (CI fixes from #5860). Original commits:

- Fix silent wrong results for custom number types
- Fix CI: build the out_of_range.407 message from string_t and fix a link
- Fix CI: diagnostics message, clang-tidy, and MinGW section limit
- Format the out_of_range.407 number without dump()
- Move the std::deque array test out of unit-regression3.cpp
- Fix clang-tidy and clang 3.4/3.5 failures in the custom number type changes
- Fix CI: cast the 128-bit unsigned value explicitly in the integer fast path

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Name the range checks of integer_fits_max/min

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-21 08:36:31 +00:00
Niels Lohmann 08d7fbb0a1 Read FILE* input with std::fgetc under MemorySanitizer (#5874) 
With glibc and optimization, <stdio.h> turns getc_unlocked into an extern
inline function that reads the stdio buffer (fp->_IO_read_ptr) directly.
glibc, which is not instrumented, fills that buffer, so MemorySanitizer
treats the bytes as uninitialized and reports a false positive when the
parsers read from a FILE* (e.g., the fuzz harness FILE* leg, OSS-Fuzz
575915982). std::fgetc is an opaque call whose result MemorySanitizer treats
as initialized.

Select JSON_UNLOCKED_FILE_IO 0 when compiled with MemorySanitizer. This is a
regression from #5828 and not part of any release.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-21 10:22:50 +02:00
Niels Lohmann 4971200b54 Add images of json_documents: save() and load() (#5632) 
* Add images of json_documents: save() and load()

An image is a document stored so that loading it needs no
parsing: save() writes the node index, the text and the decoded
strings; the static load() reads an image written by save().

load() takes a pointer and size, a borrowed vector, or an owned
rvalue vector; the nodes are copied so they are aligned and can
be edited, while the text and decoded strings stay in the image.

image_check controls how much load() trusts the input: full
checks structure, bounds, strings and numbers, the parser's own
guarantees; bounds checks structure and bounds only; none skips
all checks, for images from a trusted source.

Layout is little-endian only ("NJVI" header, nodes, text, decoded
strings), following the idea of zero-copy formats such as
FlatBuffers and YaFF; the check follows FlatBuffers' Verifier.

New errors: parse_error.116 for a malformed image or a failed
check, type_error.320 for a discarded document or a big-endian
target.

A dedicated fuzzer and 6,000 seeded corruptions, checked under
ASan/UBSan, found and fixed two gaps: unchecked reserved header
fields, and unbounded null/boolean offsets that could make
dump() throw std::length_error.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix MSVC errors in image test and add image_check doc anchor

Move the raw string literal out of the CHECK macro (MSVC preprocessor),
cast 64-bit header fields to std::size_t (C4244 on 32-bit), and give the
image_check section in load.md a real anchor for the mkdocs strict build.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Check each distinct string and float range once in the full image check

The full check scanned the contents of every string node and every float token over its own range, so many nodes pointing to one large range made load() quadratic. The structural walk now only collects the ranges; after it, each kind is sorted and checked once per distinct range. Ranges of one kind that overlap without being identical are rejected, as save() never writes them (nodes that share a value share the whole range). The cost is linear in the size of the image plus sorting the ranges.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Adapt the image tests to named documents and last-wins lookups

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Test that the targets of links never reach an image

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Test lookups in loaded images with duplicate and colliding keys

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Release the list of large objects once load() has built the indexes

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Test that invalid UTF-8 of a loaded image is not copied into an editable document

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Document that editable documents refuse invalid UTF-8 copied from a loaded image

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Expect first-wins lookups in the image tests

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Avoid useless casts of header fields in the image tests

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix clang-tidy 22 findings in unit-json_view_image.cpp and image.hpp

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix MSVC C4244 and C2017 in the image test

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix the indentation of a loop in the image test

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Make json_view.hpp pass include-what-you-use

ci_single_binaries runs IWYU with --error on every header. For json_view.hpp
it suggested adding <array> (std::array is used), the headers that json.hpp
already provides (abi_config, abi_macros, input_adapters, json_pointer,
cpp_future, string_concat, value_t, json_fwd), and <version> for
std::nullptr_t, and removing <cstddef>.

- include <array>
- keep <cstddef> (nullptr_t, size_t; IWYU attributes them to <version> and <cstring>)
- tell IWYU not to suggest the headers that json.hpp provides: the amalgamated
  json_view.hpp only includes json.hpp, so including them here would duplicate
  their definitions
- drop edit_storage.hpp, which edit.hpp includes

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Silence a clang-tidy C++17 finding in the image test

bugprone-suspicious-stringview-data-usage flags data() of the source view
under C++17, although the length is passed and checked by the REQUIRE
before it.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Build the image link test without exceptions

load_result() exists only with exceptions; with JSON_NOEXCEPTION the test
loads the image with the check directly (a failed check aborts).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Regenerate the amalgamated headers

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Follow the renames from the review of the view builder

The review of #5620 renamed document_data's tape/tape_size/tape_cap to
nodes/node_count/node_capacity and made the decode arena a
std::vector<char>; adapt the code of this change to it.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Rename the emit lambda in the image writer for Qt compatibility

Qt defines emit as an empty macro, so a Qt header included before
json_view.hpp broke the lambda's declaration (see #5865). The slots
variables in the image tests are renamed as well.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Address the review of json_document images

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Regenerate the amalgamated headers

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-21 10:21:29 +02:00
Niels Lohmann 823eee3b59 Add editable json_documents: set, push_back, insert, and erase (#5630) 
* Add editable json_documents: set, push_back, insert, and erase

basic_json_document gets a second template parameter, Editable
(false by default), plus the aliases json_editable_document,
json_editable_view, ordered_json_editable_document and
ordered_json_editable_view.

Editable documents can change values and structure without
rewriting the source text: set()/push_back() on values, keys,
array indices and JSON pointers; insert() before an array
element; erase() of an object key, array index or JSON pointer.

New values and element sequences go into edit storage that the
document owns and never moves, so views keep referring to their
value across edits and a parsed node never moves. Read-only
documents walk the plain node array and are unaffected.

Strings are checked for UTF-8 on entry, so dump() of an editable
document never throws type_error.316. Binary values cannot be
stored (type_error.319).

A seeded differential test applies random edits to an editable
document and to the equivalent ordered_json and compares both
after every step.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix old clang and MSVC C4127 in editable json documents

Value-initialize the const std::less in find_parent (clang 3.4/3.6 do not
implement DR 253), and test the Editable template argument through a
function to avoid MSVC C4127.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Copy nested values into a document without recursion

Counting and copying the nodes of a view or a basic_json value into an
editable document recursed once per nesting level, so that a deeply
nested value overflowed the stack. The four functions now walk the value
with an explicit stack, as materialize() does.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Do not leave a half-assigned container when set_moved() throws

assign() rewrote the kind, length and extent of a slot before set_moved()
registered its new element sequence. set_moved() can throw
(std::bad_alloc from reserving the bookkeeping vectors) for a slot that
is not moved yet, which left a container without the moved flag that
showed its old children. The bookkeeping is now reserved first
(reserve_moved()), so that nothing after the first write to the slot can
throw. block_of() reserves before it marks anything for the same reason.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Check strings copied from another document for valid UTF-8

An editable document only holds valid UTF-8, but copy_scalar() copied the
strings and keys of a view of another document unchecked. A document of
a weaker check (or a borrowed text that changed after parsing) could
therefore bring ill-formed UTF-8 into it. The copy is checked now, with
the error that dump() reports for the string; copies within the same
document stay unchecked.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Allow the edit arena to grow up to 4 GiB - 1 bytes

append_text() doubled the capacity of the arena and rejected the result
if it exceeded 4 GiB, so that an arena of more than 2 GiB could not grow
even though the 32-bit offsets of nodes address 4 GiB - 1 bytes. The
capacity is now clamped to that limit (text_capacity()), and an append
is rejected only if the bytes themselves do not fit.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Set a null value below a pointer as basic_json does

set(json_pointer, value) turned a null parent into an object for every
last reference token, so "/a/0" and "/a/-" below {"a":null} made
{"a":{"0":1}}, where basic_json's operator[](json_pointer) makes an
array. A null parent now becomes an array for "-" and for digit tokens
(filled with nulls up to the index) and an object otherwise. An invalid
index is reported before the parent changes.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Replace a container by a scalar in constant time once it is linked

Replacing an array or object that spans several nodes by a scalar looks
up its parent from the root (find_parent), which is linear in the size
of the document: setting every element of a 20,000 element array took
more than half a second. The parent only has to switch to links once;
afterward the extent of the replaced value no longer matters. Nodes that
an entry of a moved sequence links to are now marked (node_flags::linked),
and assign() skips the lookup for them.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Update the json_view banner and type_error.319 for editable documents

The banner still described a read-only view, and the documentation of
type_error.319 listed set and push_back only, although insert rejects
binary values as well.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Assign the member that lookups find when set() meets duplicate keys

An editable document reads the last member of a repeated key (as the
read-only document does), but set(object, key, value) assigned the first
one: a view taken from object[key] before the call did not show the new
value. It now assigns the last member's value, keeps the key at the
position of its first occurrence (where materialize() puts it), and drops
the other members.

The seeded differential test now also edits documents whose objects
repeat keys and compares every lookup (operator[], at, find, value,
contains, count, and JSON pointers) with the parsed basic_json value.
A targeted test covers duplicates before and after edits that move the
object, in large objects with an index, in moved arrays, and in values
copied from other documents.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Assign the first member when set() meets duplicate keys again

Lookups find the first member of a repeated key again, so set(object, key,
value) assigns that member (keeping the key where it is) and drops the
others, as documented before, and a view taken from object[key] shows the
new value. This undoes the code and documentation changes of 5cbb8e6d6.
Lookups in edited objects (navigation of editable documents) stop at the
first match, like those of parsed objects.

The tests that commit added expect the first member from lookups now. In the
seeded differential test, the documents with repeated keys repeat them after
the real members; the reference holds the first members, which the edits
address and the lookups are compared with, while materialize() is compared
with what parse() makes of the document's dump.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix MSVC C4127 in the lookups of editable json documents

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix clang-tidy 22 findings in the json_view headers

Silence bugprone-casting-through-void for the SSE loads (a reinterpret_cast
would trip -Wcast-align=strict), use auto for a cast initialiser, add
parentheses to a mixed expression, and name bugprone-std-namespace-modification
in the NOLINTs of the tuple_size/tuple_element specialisations.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
(cherry picked from commit 0effbaa7a2)

* Fix clang-tidy 22 findings in unit-json_view_edit.cpp and edit.hpp

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Keep a raw string with a backslash out of a CHECK (MSVC C2017)

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Make json_view.hpp pass include-what-you-use

ci_single_binaries runs IWYU with --error on every header. For json_view.hpp
it suggested adding <array> (std::array is used), and the headers that json.hpp
already provides (abi_config, abi_macros, input_adapters, json_pointer,
cpp_future, string_concat, value_t, json_fwd), plus <version> for
std::nullptr_t, and removing <cstddef> and edit_storage.hpp.

- include <array>
- keep <cstddef> (nullptr_t, size_t; IWYU attributes them to <version> and <cstring>)
- tell IWYU not to suggest the headers that json.hpp provides: the amalgamated
  json_view.hpp only includes json.hpp, so including them here would duplicate
  their definitions
- drop edit_storage.hpp, which edit.hpp includes

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Keep the view's edit headers clean for clang-tidy in C++17 mode

clang-tidy 22 checks the headers with -std=gnu++17 as well, where it asks
for a transparent comparator and std::make_unique. The regions map uses its
default comparator, and the allocation of the edit state keeps new (the
library is C++11) with a NOLINT.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Regenerate the amalgamated headers

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Follow the renames from the review of the view builder

The review of #5620 renamed document_data's tape/tape_size/tape_cap to
nodes/node_count/node_capacity and made the decode arena a
std::vector<char>; adapt the code of this change to it.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Expect count() to count every member in the edit tests

Since #5622, count() counts every member with the key, so documents
with duplicate keys no longer give 1.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Count matching keys as unsigned in the json_view edit tests

Like in the access test, adding the int result of the conditional to a
std::size_t fails -Werror=sign-conversion with GCC.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Address the review of erase() in editable json documents

From gregmarr's review of #5630:

- invalid_iterator.202 is thrown by overloads 1 and 2 only; the page said
  "every overload" and then excluded 3 in parentheses.
- The note on links says what moves (the links after the removed element,
  as insert() moves them the other way) and that every iterator over the
  container is invalidated, not only those after the removed element.
- edit.hpp compares keys with document_data::key_equals().

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Regenerate the amalgamated headers

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Address the review of editable json documents

From gregmarr's review of #5630:

- set() with a JSON pointer below a null value checks for "-" before
  scanning the token for digits.
- node_flags::storage is defined as escaped | edited, and the comment on
  document_data::base says that the fourth slot is unused.
- source_extent() names its root check.
- The integer-index overloads of set(), insert(), and erase() use a named
  trait instead of a negated condition, which astyle formats like the
  library's other enable_ifs.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Regenerate the amalgamated headers

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Address the rest of the review of editable json documents

From gregmarr's review of #5630:

- alloc_nodes() gives a request of half of the next chunk or more a chunk
  of its own, so that the rest of the current chunk stays available for
  smaller requests instead of being abandoned.
- The comments on edit_state, append_text(), and reserve_moved() say that
  views stay valid across every edit (not just the first), that n is 0 for
  empty strings, and why both capacities are checked.
- The architecture page names flag bits 3, 4, and 5 (linked) separately
  and says that a discarded view has no node; erase.md rewords overload 3
  and says why it never throws invalid_iterator.202.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Call alloc_nodes() before reading chunks.back() in the json_view chunk test

The operands of == are unsequenced, and MSVC evaluated
d->edits->chunks.back() before alloc_nodes() added the new chunk, so the
check compared the result with the previous chunk and failed on
msvc-arm64 (Debug and Release).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
 2026-10-21 10:21:29 +02:00