GitHub is moving the ubuntu-latest label from Ubuntu 24.04 to 26.04
between October 19 and November 19, 2026 (actions/runner-images#14748).
Pin every ubuntu-latest job to ubuntu-26.04 so the switch does not
change the toolchain under our jobs, and so the behaviour does not
depend on when GitHub reaches our runs.
Jobs pinned to ubuntu-24.04 and ubuntu-24.04-arm are unchanged.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
readability-math-missing-parentheses in unit-json_view_builder.cpp (#5974)
and performance-inefficient-string-concatenation in
unit-json_view_positions.cpp.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* UBJSON/BJData readers: read optimized arrays of numbers in one loop
An array with a type marker for a fixed-size number ($U, $i, $I, $l, $L,
$d, $D, and BJData's $u, $m, $M), including the data of a BJData ndarray,
gives its elements no markers. When the input is contiguous, read the
elements that are in the input in one loop - load, byte swap for the
format's byte order, emit - instead of one trip through
parse_ubjson_internal and get_ubjson_value per element. Elements the
input ends before, other types, and non-contiguous inputs are read as
before.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* UBJSON/BJData tests: assign the parsed value instead of casting it to void
GCC reports -Werror=unused-result for static_cast<void>(json::from_ubjson(...))
and json::from_bjdata(...): a cast to void does not silence warn_unused_result
on GCC. Assign to a json, as the other tests do.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* MessagePack reader: read runs of scalar array elements in one loop
As for CBOR: when an array opens and the input is contiguous, read the
integers, floats, booleans, and nils that follow in a tight loop instead
of going through parse_msgpack_internal and parse_msgpack_value for every
element. The loop stops at any other kind of value and when fewer than 9
bytes are left; those values are read by the general path as before.
Non-contiguous inputs are not affected.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* MessagePack reader: enter the scalar run only when a scalar follows
As for CBOR: enter_msgpack_array tests the first byte and the remaining length
inline and calls the out-of-line run only for a scalar, so arrays of maps or
strings no longer pay for a call that reads nothing; the run's loop condition
moves to the end (do-while).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* MessagePack reader: mark the initial accepted in the scalar run as a false-positive dead store
ci_infer reports DEAD_STORE for 'bool accepted = true;' in the MessagePack
scalar run, as in the CBOR one, because every case assigns accepted before it
is read.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* MessagePack tests: assign the parsed value instead of casting it to void
GCC (ci_test_gcc) reports -Werror=unused-result for
static_cast<void>(json::from_msgpack(...)): a cast to void does not silence
warn_unused_result on GCC. Assign to a json, as the other tests in the file do.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* MessagePack reader: drop the scalar pre-check before the run
enter_msgpack_array now calls the run directly, and parse_msgpack_scalar_run
tests the remaining count and length at the top of a while loop instead of
relying on msgpack_scalar_follows(); its own switch already leaves on any
value that is not a scalar.
Instructions per repetition, new/old: SAX canada 0.946 (clang) and 0.973
(GCC 13), DOM read canada 0.991 / 0.992; no row more than 0.3 % higher.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* MessagePack reader: say why the initial accepted in the scalar run stays
As for CBOR: Infer's DEAD_STORE report is right, not a false positive. The
initializer is kept for clang-tidy's cppcoreguidelines-init-variables.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* MessagePack reader: give the scalar run the bulk tag instead of a wrapper
As for CBOR: parse_msgpack_scalar_run takes the bulk-scan tag itself and is
the function kept out of line, so parse_msgpack_array_start goes.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* MessagePack reader: start the scalar run's accepted at false
As for CBOR: Infer ignores dead stores of 0, so the suppression goes.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* MessagePack reader: let the scalar run find its array itself
As for CBOR: parse_msgpack_scalar_run reads container_stack.back() in its
bulk version, so the caller no longer evaluates it for input that is not
contiguous.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Binary readers: swap the bytes of floats with a single byteswap
byte_swap reversed float and double values byte by byte. GCC does not
merge that loop into one bswap, and C++23's std::byteswap accepts only
integers. Overloads for float and double now copy the bits into a
uint32_t/uint64_t, call detail::byteswap, and copy them back. This
applies to every binary reader that uses get_number.
In the CBOR reader, the single- and double-precision float cases call a
non-inlined helper. Without it, GCC stopped inlining the stream
adapter's get_character() into parse_cbor_internal, which made reading
from a std::istream about 0.5 % slower.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* CBOR reader: inline get_cbor_float() and always inline get() instead
The never-inlined get_cbor_float() costs a call for every float read
from input without bulk access: reading canada.json from CBOR through a
FILE* became 5.6 % slower on Apple clang in the benchmark suite. The
helper existed only because GCC 13 stopped inlining the stream adapter's
get_character() into parse_cbor_internal. Always inlining
binary_reader::get() fixes that as well.
callgrind, all binary formats on buffer, std::istringstream and FILE*
input (canada and citm_catalog): GCC 13 0.986-1.003, GCC 16 0.994-1.006,
Clang 23 identical. Apple clang suite: FromBinaryFile/cbor/canada 0.939,
geomean of the 53 binary reader rows 0.998.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* CBOR reader: read runs of scalar array elements in one loop
When a definite-length CBOR array opens and the input is contiguous, read
the integers, floats, booleans, and nulls that follow in a tight loop
instead of going through parse_cbor_internal's container logic and
parse_cbor_value's dispatch for every element. The loop stops at any other
kind of value and when fewer than 9 bytes are left; those values are read
by the general path exactly as before. Non-contiguous inputs are not
affected (tag dispatch on the bulk-scan capability).
Also pass a member empty string as the number source to number_float
instead of constructing a std::string for every float.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* CBOR reader: address review of the scalar-run reader
- parse_cbor_scalar_run: set the size explicitly in every case that
reads a value, instead of relying on a default that every other case
overwrites.
- Remove cbor_scalar_follows(): parse_cbor_scalar_run already returns on
the same conditions, and without the pre-check SAX canada needs another
8.5 % fewer instructions with GCC 13 (0.915 against the previous
version), with no row more than 0.5 % worse (Mac and GCC).
- Restore the comment on why parse_cbor_internal copies the container
frame.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* CBOR reader: enter the scalar run only when a scalar follows
An array of containers or strings called the out-of-line run start once per
array only to leave it at once; on nested data (arrays of objects) that call
made reading slower than before the run reader. enter_cbor_array now tests the
first byte and the remaining length inline and calls the run only for a
scalar, so the run's loop condition moves to the end (do-while).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* CBOR reader: mark the initial accepted in the scalar run as a false-positive dead store
ci_infer reports DEAD_STORE for 'bool accepted = true;' in
parse_cbor_scalar_run because every case assigns accepted before it is read.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* CBOR reader: drop the scalar pre-check before the run
enter_cbor_array calls the run directly again, and parse_cbor_scalar_run
tests the remaining count and length at the top of a while loop; its own
switch already leaves on any value that is not a scalar. The MessagePack
reader is built the same way.
Instructions per repetition, new/old: clang SAX canada 0.983, read canada
0.993; GCC 13 read canada 0.997, the other rows at most 1.004.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* CBOR reader: say why the initial accepted in the scalar run stays
Infer's DEAD_STORE report is right, not a false positive: every case that
reaches the read assigns accepted. The initializer is kept for clang-tidy's
cppcoreguidelines-init-variables.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* CBOR reader: give the scalar run the bulk tag instead of a wrapper
parse_cbor_scalar_run takes the bulk-scan tag itself and is the function that
is kept out of line, so the one-line parse_cbor_array_start wrapper goes, as
the UBJSON typed run already does.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* CBOR reader: start the scalar run's accepted at false
Infer ignores dead stores of 0 (--liveness-ignored-constant defaults to "0"),
and false is 0 to it, so the suppression and its comment go.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* CBOR reader: let the scalar run find its array itself
parse_cbor_scalar_run reads container_stack.back() in its bulk version, so
the caller no longer evaluates it for input that is not contiguous.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The baseline is now develop 9c349b1d7c with every open non-draft pull
request merged, including #5968, #5970, #5971, #5975, #5976, #5978,
#5979, #5980, #5981 and #5982. The suite was rerun on 11-01 with every
step repeated when the load rose above 5 while it ran; the comparison
with other libraries and the held-out files were rerun on 10-31.
- suite: 3.0x geometric mean (median 2.9x); Appendix B, the overview
and the distribution regenerated
- timeline endpoint, Appendix D and the cliffs figure updated
- the held-out figure compares json_document with the fastest other
library
- new paragraphs for the pull requests listed above
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
#5634 made dump() write integers below 10^16 eight digits at a time; larger
ones still took the two-digits-per-division loop and count_digits(). Random
64-bit integers almost all have 17 to 20 digits (99.9 % of signed_ints.json
and unsigned_ints.json), so Dump/signed_ints and Dump/unsigned_ints gained
only 1.4x over 3.12.0, against 2.3x for small integers.
A 64-bit magnitude is now written as its first one to four digits and two
groups of eight. Wider magnitudes (__int128 and class types, #5805) keep the
loop: number types of at most 64 bits are selected by type, so no comparison
is reported as always true; for others the value is compared. The sign
handling of #1447/#5815 (remove_sign() into number_magnitude_t) is unchanged.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
A json_view or json_document decodes a macro-defined struct directly and looks
the members up by their decoded keys. Only plain keys were covered so far.
The new section decodes structs defined with NLOHMANN_DEFINE_TYPE_INTRUSIVE,
..._WITH_DEFAULT and ..._WITH_NAMES from documents whose keys contain escape
sequences, and compares the result with the conversion of the materialized
value. It also pins the difference for a duplicate key written once plain and
once escaped (the view takes the first member, the materialized value keeps
the last one), and that an escaped backslash is part of the decoded key.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
After the first \uXXXX escape of a string, slow_string() now decodes the
escapes that follow it directly, while the next six bytes are \u and four hex
digits that are not a surrogate, instead of going round its main loop for each
of them (that is how Python's json.dumps writes non-ASCII text by default).
A surrogate, a bad digit, or fewer than six bytes leave the position at the
backslash: the main loop handles them as before and reports the same error at
the same position.
twitterescaped.json (simdjson-data, 549 KB, 33,036 escapes), json_document::parse(),
A/B in one binary, 6 code layouts, time after / before:
- Apple M1 Max, Apple clang 21: 0.897
- x86-64 Haswell, GCC 16.2: 0.907
- x86-64 Haswell, Clang 23: 0.910
The other files of the benchmark set (twitter, citm_catalog, canada, gsoc-2018,
apache_builds, instruments, tree-pretty, github_events, random, update-center,
semanticscholar-corpus, twitter_timeline) stay within the noise of each machine.
New test: chains of every length from 1 to 40 of escapes that give one, two,
and three bytes of UTF-8, followed by the end of the string, text, escapes,
surrogates, bad digits, and cut in every place, in keys and in values, against
json::parse().
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
- short strings: two overlapping 16-byte loads and stores
- eight digits at a time: the three multiplications, lane by lane
- validate on the 19 held-out files, per file, Mac and x86-64
The prose the figures replace is trimmed.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The suite (Appendix B, the short-version table, the overview and distribution
figures) and the timeline endpoint now measure develop d20c4dcd89 with #5964
and #5908 merged. The geometric mean stays 2.9x (2.95); Dump/signed_ints and
Dump/unsigned_ints gain 1.5x and 2.1x from #5964; GetVectorDouble/floats drops
from 10.3x to 7.0x because the old candidate's suite binary happened to compile
that loop faster (a standalone build takes 0.97 ms with both headers).
The timeline endpoint is the 10-27 value scaled by the ratio of the new to the
old candidate, both measured on 10-30, so that it stays comparable with the
other points of the timeline.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
#5919 replaced the bool template parameter Editable of basic_json_view by
view_mode Mode and defined is_editable from Mode. #5890 was written before
that and added a second is_editable = Editable to the private section, which
no longer compiles ("duplicate member", "undeclared identifier Editable"). It
broke every job that builds the json_view tests (macOS, MSVC, clang-cl,
clang on Windows, the C++20 module job, CodeQL).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The figures and Appendix E use the reruns with each library's fastest
modes. Skipping number conversion while reading is now allowed for
every library; copying number text is still not chosen when writing.
The text now says who set the rules, where libraries that check less
win select, how minified files change the picture, and that the
reused jeopardy.json claim holds with every library reused. A new
section covers 19 held-out files and JSONTestSuite, including where
json_document loses, and lists what was not measured. Every
comparison number was recomputed; Glaze is credited in the intro.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Address the points of a review from the other libraries' side:
- simdjson 5.0.8 (On-Demand structs, key selectors) and Glaze 9.1.0
- number-skipping modes: yyjson and RapidJSON raw numbers, Boost.JSON
number_precision::none
- reused parsers and allocators for yyjson, RapidJSON and Boost.JSON
- Boost.JSON parse_into, Glaze generic with double numbers, and the
same struct types for nlohmann, Glaze, simdjson and serde
- minified copies of twitter.json and citm_catalog.json
- a fourth probe (trailing garbage) and Glaze "(strict)" variants
- a new random variant order in every round, as a variant's time could
depend on the one before it by up to 15 % on the Mac
- --files for other test files and --conformance for JSONTestSuite,
with conformance-rs.sh for the Rust program
summarize.py also reports each library's best reused variant.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
- deeper technical detail and code in the eight themes; pull request
numbers move to the appendix, timeline steps are labelled by theme
- comparison with other libraries by use case, as heatmaps per
platform, with validation levels and the caveats next to the results
- appendix: test files, all 182 suite rows with a histogram, timeline
steps, algorithmic fixes, per-file tables, behaviour changes, setup,
pull requests by theme
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
bench_all and bench_rs now time use cases (validate, sum, select, edit,
minify, write) instead of library functions. Every library may solve a
use case in every way it offers; each variant's result is checked, and
three malformed inputs give its validation level (full, no UTF-8,
partial). summarize.py reports each library's fastest fully validating
variant and a faster one at a lower level, as text and as JSON.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The dump of canada.json with GCC on x86-64 lost to yyjson by 0.78 because
GCC compiled #5908's token-digit selections as branches; 9fad526888 fixed
that. The post's x86-64 dump and parse+dump rows of canada.json are now
measured with it (GCC 0.95, Clang 0.91 against yyjson), theme 5 tells the
story, and "How we measured" says which rows include the commit.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
- a dump workload in bench_all and bench_rs: each engine writes a
document parsed beforehand, and every output must read back with
bit-identical numbers; an optional argument selects one workload
- the post's comparison gains a write table, a throughput chart, a dump
row for all three configurations, and the x86-64 cases it loses
- the thanks section is split into adapted code and borrowed ideas
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
- author and date below the title, styled as .nj-byline; its link gets
no external-link icon
- the license of every library in the thanks section, checked against
each project's license files; credit sonic-rs for QUOTE_TAB, which
orjson adapted from it
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
- caption every table and figure
- describe what makes twitter.json, citm_catalog.json, canada.json and
jeopardy.json hard, and what each timeline panel exercises
- split the json_view layout figure into the tree and the index, with
the example text above; fix the arrows into the string boxes
- arrows and a "less is better" axis label in the cliffs figure
- name nlohmann/json in the comparison tables and chart
- licenses of the adapted code in the thanks section
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Extends the comparison in "How 3.13.0 got faster" from yyjson, simdjson and
Boost.JSON to eleven engines: sonic-cpp, RapidJSON and Glaze (generic DOM,
lazy_json, reading into structs) in C++, sonic-rs, simd-json and serde_json in
Rust. Measured on an Apple M1 Max (Apple clang 21, Rust 1.99) and an x86-64
Haswell server (GCC 13, Clang 18). Every engine must return the same values,
string bytes and bit-identical number sums before it is timed.
The post now answers whether json_document is the fastest parser: it is the
fastest document parser at parse+dump everywhere and at parsing on Apple
silicon, ties with simdjson's DOM on x86-64, and loses to simdjson On-Demand
and Glaze structs when only a few fields are read.
tests/benchmarks/json_view/others holds the programs (bench_all.cpp, the Rust
crate with its Cargo.lock), summarize.py and the pinned versions and checksums.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
A post for the documentation (Home > Blog) on how 3.13.0 became faster:
eight themes with the pull requests behind them, the ideas taken from other
libraries, what did not work, how the fast paths are tested, and the
compile-time cost. Every number comes from a run on 2026-10-27 (3.12.0 against
develop plus the open non-draft PRs, Apple M1 Max) or from the linked PR.
- docs/mkdocs/docs/home/blog/how_3_13_0_got_faster.md: the post.
- docs/mkdocs/scripts/figures/perf_blog.py: draws its 12 figures (overview,
eight timeline panels, the SWAR test, the cliffs, json_view throughput) as
inline SVGs in the nj-charts style; data/perf_timeline.csv holds the
timeline.
- tests/benchmarks/history: history.cpp and history.py time one fixed program
against the header of every commit that changed it, interleaved, and write
one CSV row per commit with its PR. The timeline of the post comes from it.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
- bench_view: add "(editable)" rows for json_editable_document and a
parse+dump workload, so the cost of lazy float conversion shows next to the
parse row (the traverse workload already includes the parse). Check that
all engines' dumps agree where outputs are comparable.
- compare.py: add --aligned, which also builds and runs with aligned code and
reports both tables, so code-layout artifacts become visible. Off by
default; the default output is unchanged. Not run end to end yet.
- README: say which engines are like-for-like peers (yyjson, simdjson DOM) and
which are mutable reference points (Boost.JSON, json::parse, the editable
json_view); explain the lazy float conversion and the eager prototype that
was measured and dropped.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Compare json_view values without a vector per container and a basic_json per number
operator== of views (view/view, view/json, and editable documents) built two
vectors of members for every object it entered, sorted them with stable_sort
for every object, and converted each number to a basic_json value. It now:
- keeps the members of all open objects in one vector (reserved with
geometric growth, so that nested objects do not reallocate), and orders
them by insertion sort for up to 16 members, or not at all when the keys
ascend already;
- compares numbers of one type directly, and builds a basic_json only for
numbers of different types;
- compares short texts (keys, numbers, most strings) with two overlapping
loads (bytes_equal(), in node.hpp);
- first walks both documents in lockstep, in the order of their text, which
finds most pairs of equal values without the work of the comparison. The
walk only enters an object if the comparison would visit its members in the
same order (the object type keeps the order, the keys ascend, or all values
are scalars), so a difference is never found later than before, and it
records the containers it found different, so that the comparison does not
walk them again;
- rejects values of different types and arrays of different sizes before any
of this, and, for an object type that keeps the order, objects whose first
(up to four) members have different keys, which ends a comparison that
differs near the start without preparing the members.
Results are unchanged: numbers compare by value across types, duplicate keys
resolve as parse() resolves them, and objects compare in document order for
ordered_json and by key otherwise.
Instructions of one comparison of a document with an equal copy, macOS arm64
(develop -> now), json_document / ordered_json_document / editable:
twitter 16.1M -> 7.0M 17.7M -> 1.5M 16.6M -> 7.3M
citm 45.0M -> 3.4M 47.0M -> 3.2M 45.7M -> 4.3M
canada 111.4M -> 13.7M 111.4M -> 13.7M 113.3M -> 15.0M
jeopardy 26.2M -> 3.7M 27.3M -> 3.7M 26.6M -> 12.1M
and of a view with a basic_json value:
twitter 11.4M -> 5.9M 11.2M -> 7.3M 11.6M -> 5.9M
citm 33.9M -> 9.7M 32.6M -> 9.1M 34.6M -> 9.8M
canada 92.9M -> 32.1M 92.9M -> 32.1M 93.9M -> 32.8M
and Linux instruction counts (GCC 13 / Clang 18) of the same comparisons:
document with an equal copy json_document ordered_json_document editable
twitter -49 % / -52 % -89 % / -89 % -48 % / -50 %
citm -90 % / -89 % -91 % / -90 % -88 % / -87 %
jeopardy (4,000 questions) -85 % / -85 % -86 % / -86 % -46 % / -49 %
view with a basic_json value
twitter -41 % / -45 % -28 % / -35 % -45 % / -44 %
citm -63 % / -65 % -63 % / -66 % -65 % / -64 %
jeopardy -39 % / -43 % -14 % / -24 % -44 % / -43 %
A sweep that places one difference at about 2,100 positions of eight
documents (a token at an even spacing, the first and last tokens, the first,
middle and last member of the root removed, one appended, the document
wrapped) in five kinds of comparison (view/view of read-only and of editable
documents, view/json, json_document, editable/view) finds the same results as
before and no placement slower on the Mac (Clang); on Linux no placement is
slower than 3 % (Clang and GCC).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Check the float tokens of a json_document image without a sort and a scan of every character
load(image_check::full) checked every float token of an image by sorting
the token ranges, scanning the token character by character, and comparing
the digit layout with the node's. Now:
- the ranges are sorted only if they are not sorted already (save() writes
them in the order of the nodes, so they are);
- a token of the shape [-]digits.digits whose digit counts are the ones the
node records (nearly all of them) is accepted by checking its digits eight
at a time, without the general scan;
- the general scan skips runs of digits eight at a time.
Damaged tokens are still rejected as before: the check accepts exactly the
tokens that the general scan accepts.
Instructions of load(image_check::full) (develop -> now):
macOS arm64 canada 44.6M -> 30.9M (-31 %), jeopardy 11.3M -> 10.7M,
citm 10.4M -> 10.0M, twitter 5.06M -> 4.94M
Linux GCC 13 canada -50 %, jeopardy -43 %, twitter -31 %, citm -28 %
Linux Clang 18 canada -42 %, jeopardy -48 %, twitter -35 %, citm -32 %
image_check::bounds is unchanged (within 0.4 %).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Rename names that Flawfinder takes for the file function open()
The comparison cursors' open() is now enter(), and a test's open and
close strings are head and tail.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix the compiler and lint failures of the json_view comparison
- all_digits(): the lambda declares its constant itself; MSVC does not
let a lambda without captures read a constexpr local of the function
- equal(): the frame's "in document order" index is a namespace-scope
constant (document_order); the local constexpr npos in the default
member initializers of the local struct crashed GCC 9 and MinGW GCC
and was not found by MSVC
- keys_differ_early() keeps the first keys in std::arrays (cpplint:
variable-length arrays; clang-tidy: C arrays), and the basic_json
side's first_keys() is not noexcept (key() may throw)
- the numbers of one type are compared in an if chain instead of a
nested conditional operator
- the dump test avoids C arrays and temporary string concatenations
View comparison instructions are unchanged (Apple clang, twitter, citm,
canada: -0.1 % to +0.1 %).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Keep the byte comparison helpers in compare.hpp
Address review comments: bytes_equal() and bytes_less() move from node.hpp
to compare.hpp, their only user; key_less() no longer splits the comment of
resolve_indices() from its function; and the loop counters of bytes_equal()
and all_digits() are scoped to their loops.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Dispatch on the compile-time conditions of the json_view comparison
MSVC (/W4 /WX) reported C4127 (conditional expression is constant) for
the template parameter EA and the object type's ordering in an if
condition; both are now tag-dispatched.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Remove a nested duplicate guard from the json_view image test
clang-tidy reported the inner #if !defined(JSON_NOEXCEPTION) as
redundant (the test case is in the same #if) and a find() != npos, which
gets the NOLINT the other tests use.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Mark Infer's null dereference in materialize() as a false positive
Infer (ci_infer, also red on develop since #5882) reports a null
dereference in json_sax_dom_parser::null() called from materialize(). The
member pointer the parser stores a value of an object in is set by the
key() before it, so it is not null there.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Move the json_view comparison tests into their own file
With the tests this branch adds, the object file of unit-json_view_dump.cpp
got too large for the MinGW x64 assembler ("file too big"). The test case
"json_view comparison" is now unit-json_view_compare.cpp, unchanged and
also built with C++17.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Build the DOM parser with Visual Studio 2015 again
Since #5882, the three VS 2015 jobs of AppVeyor fail on develop:
- the detection of keys that can be indexed instantiated std::hash for
every string type, which VS 2015 rejects with a static_assert for a type
it does not hash (alt_string): std::hash is now looked at only for
objects in a vector
- the frame stack copied its frames with std::copy to a raw pointer, which
VS 2015 reports as unsafe (C4996, an error with /WX): a loop now
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* json_view: no per-object allocations in ordered_json comparisons
For an object type that keeps the order of its members (ordered_json),
resolve_indices() allocated a vector per compared object, and
resolve_members() two. They now use one working vector owned by equal(),
whose capacity is reused, and resolve_indices() needs none when the keys
are unique. resolve_members() sorts small objects by insertion, as the
other paths do.
Instructions per comparison (callgrind, GCC 13 and clang 18, x86-64):
twitter, ordered_json view == ordered_json: 0.939 / 0.941; 20000 small
objects with a duplicate key: 0.741-0.768; all other rows (json,
citm_catalog, canada, view == view) within 0.988-1.003.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* json_view: check_utf8 skips well-formed runs with the parser's string scan
Strings of 32 bytes or more that are written into an editable document
are validated with scan_string_run(), which skips well-formed UTF-8 16
bytes at a time; the byte-wise check runs only from where the scan found
ill-formed UTF-8, to report the same type_error.316. Shorter strings
keep the byte-wise check (inlined with a constant start, so that this
path is the code it was), where the vector scan costs more than it
saves.
set()/push_back() of strings of 32 bytes or more take 0.29-0.76 of the
time (x86-64 GCC, averaged over code layouts) and 0.25-0.80 (AArch64
clang); shorter strings take 1.5-4 % fewer instructions; the json_view
rows of bench_edit execute at most 0.04 % more instructions.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* tests: avoid sign conversion in json_view_edit UTF-8 check
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* json_view: use while in check_utf8_bytewise
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The files with code adapted from fast_float, Zmij and simdjson carried the
authors' SPDX copyright lines, but their file comments did not say what
was taken or under which license: number_parse.hpp, to_chars.hpp,
simd_scan.hpp and view/simd.hpp now do, as zmij.hpp and the yyjson-based
view files already did. README and the license page now also name
fast_float's eight-digit parsing and the lexer's use of simdjson's UTF-8
check. Comments only.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Constructing a std::string from uint8_t iterators converts values >= 0x80
implicitly to char, which ci_test_clang_sanitizer rejects. Convert each
byte with an explicit static_cast.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Document the contract of object keys passed to json_sax::key and key callbacks
Describe what the name passed to json_sax::key and to a parser callback
for parse_event_t::key holds when parsing JSON text: escapes are decoded
to UTF-8, different spellings of a name yield identical strings, the name
is validated before the call (invalid escapes or UTF-8 give a parse error
and no key event), no normalization or case conversion is applied, the
event precedes the ':' check, and duplicate names are reported once per
occurrence. Also note that sax_parse passes binary-format keys unchanged.
See discussion #5955.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Add a test for the object key contract and correct the BON8 note
Pin the guarantees documented for json_sax::key and the key event of
parser callbacks: decoded names, validation before delivery, delivery
before the colon, one event per repeated name, and the binary formats.
BON8 rejects ill-formed UTF-8 in a name even with sax_parse, so the
note no longer lists it with the formats that pass bytes through.
See discussion #5955.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix the GCC build of the object key contract test
GCC 11 in C++11 mode rejects a default argument on a lambda parameter
(-Wc++14-extensions) and ignores a static_cast<void> on the
warn_unused_result of parse(). Pass the input format explicitly and bind
the parse result to a variable.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Add version history for the object key contract
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Split the combined has_save/has_load static_asserts so clang-tidy no
longer sees both sides of && as equivalent (develop ci_clang_tidy failure).
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
VS 2015 does not accept std::string::npos in a constant expression (C2131),
and the constant is only used at run time.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* json_view: source positions in materialize() with JSON_DIAGNOSTIC_POSITIONS
With JSON_DIAGNOSTIC_POSITIONS, materialize() now gives every value the
start_pos()/end_pos() that basic_json::parse() would give it, so exceptions
about materialized values name the same bytes (#5864).
- json_sax_dom_parser takes the lexer type as a defaulted template
parameter; materialize() passes position_source, which reports the
positions it derives from the index. diagnostic_positions treats npos as
"no position" and keeps ranges all-or-nothing.
- Strings with escapes keep the offset of their opening quote plus one in
`next` (only with the macro); materialize() scans from there to the closing
quote. The end of an array or object is found by scanning from the end of
its last element over whitespace, comments and a trailing comma.
- In editable documents, values written by edits, arrays and objects whose
elements changed, and containers whose last element has no position get
none. Images keep the flag is_new, and load() checks that `next` of a
string is within the text.
Nothing changes on the parse path without the macro. The view's own
exceptions still carry no byte range, so the tests that compare them with
those of materialized values skip it under the macro, as they skip the path
under JSON_DIAGNOSTICS.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Indent comment in unit-json_view_from_json.cpp
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Address review: clearer position code and docs
- position_source gets clear(); set_scalar() returns early for new values,
switches on node_kind, and uses no local copies; the one position_source
per materialize() call is documented
- no npos + 1 for containers without a position; a comment on why the end
of a container is reported as its closing bracket
- less duplication under #if in materialize() and the builder
- diagnostic_positions sets npos explicitly
- docs: positions of basic_json values after changes, positions in images
depend on how the document was parsed, when next is set, and the cost of
positions while parsing
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Share the comment scanners of the view builder with materialize()
skip_line_comment() and find_block_comment_end() in scan.hpp are used by
the builder's comment() and by position_source::container_end(), which
had its own copy.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Address review: if/else for container positions, clearer next field
- the opening bracket of a container sets positions with an if/else instead
of a ternary, and the frame takes its start from positions
- the closing bracket of a container is found in nested if/else; the parent's
last_end is taken from positions, so it is npos whenever no range is set
- architecture.md: the next field reads "or 0 if it was not; all other
values: 0" instead of "else 0; otherwise 0"
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix GCC unused-result and clang-tidy raw-string findings in unit-json_view_positions.cpp
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix self-aliasing in json_pointer::operator/= and patch_inplace
json_pointer::operator/=(const json_pointer&) passed iterators into its
own token vector to insert() when a pointer was appended to itself
(p /= p), which is undefined behavior; libstdc++ produced "/a/b//" for
"/a/b". The pointer now appends a copy in that case.
basic_json::patch_inplace iterated the patch while changing *this, so
j.patch_inplace(j) read freed memory. It now applies a copy of the
patch when the patch is the value itself. A patch stored inside the
value remains undefined behavior and is documented as such; copying the
patch unconditionally would double the time of a large patch.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix JSON_DIAGNOSTICS test crash and clang-tidy hit in patch_inplace self-aliasing
The spare-capacity test reallocated non-empty elements through get_ref(), which leaves stale parent pointers under JSON_DIAGNOSTICS; reserve while the array is empty instead. Suppress performance-unnecessary-copy-initialization for the copy that is needed because *this is modified.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Detect a patch stored inside the patched value with JSON_DIAGNOSTICS
patch_inplace applied a copy only when the patch was the value itself. With JSON_DIAGNOSTICS the parent pointers make it cheap to also detect a patch nested inside the value, so that case is handled the same way. Without diagnostics it stays documented undefined behavior.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Silence misc-const-correctness for patch_aliases_this
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix clang-tidy findings in the operator/= self-append test
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
MSVC 2015 treats two function templates that differ only in a decltype
return type as a redefinition (C2995), which breaks all three Visual
Studio 2015 jobs on AppVeyor. Overload on the view's mode in the
parameter type instead; template argument deduction ignores the
read-only-to-editable conversion, so each view still selects the
overload of its own mode.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Fix clang-tidy findings in unit-json_view_from_json.cpp
Silence misc-use-internal-linkage on the macro-generated ADL functions,
reconcile -Weffc++ with readability-redundant-member-init, use auto for
get<T>() initializers, and use value() on the optional access.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Satisfy bugprone-unchecked-optional-access in unit-json_view_from_json.cpp
clang-tidy 22 also flags value(); check has_value() in the same expression.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
---------
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The test relies on a copied fifo_map restarting its timestamps. Copying a
basic_json does that only on the recursive path; with JSON_NO_THREAD_LOCAL
the copy is built member by member and keeps correct timestamps, so the
premise was missing and the test failed (ci_test_no_thread_local, MinGW
clang). Copy the object_t explicitly to create the state on every path.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Niels Lohmann
dependabot[bot]