Compare commits

..
Author SHA1 Message Date
Niels Lohmann cf9bc9f71b Merge branch 'ci/tiers' into ci/windows-macos-sets
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-30 15:36:35 +01:00
Niels Lohmann 9e1d35eda9 Merge branch 'ci/path-filters' into ci/tiers
Keep the tiered job layout and apply the harden-runner 2.22.2 bump
from #5962.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-30 15:36:22 +01:00
Niels Lohmann 4b538d80f6 Merge branch 'develop' into ci/path-filters
Keep ci_test_documentation in repository_checks.yml and apply the
harden-runner 2.22.2 bump from #5962 there.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-30 15:35:39 +01:00
Niels Lohmann d20c4dcd89 json_view benchmarks: compare fairly and state what each row means (#5914)
- bench_view: add "(editable)" rows for json_editable_document and a
  parse+dump workload, so the cost of lazy float conversion shows next to the
  parse row (the traverse workload already includes the parse). Check that
  all engines' dumps agree where outputs are comparable.
- compare.py: add --aligned, which also builds and runs with aligned code and
  reports both tables, so code-layout artifacts become visible. Off by
  default; the default output is unchanged. Not run end to end yet.
- README: say which engines are like-for-like peers (yyjson, simdjson DOM) and
  which are mutable reference points (Boost.JSON, json::parse, the editable
  json_view); explain the lazy float conversion and the eager prototype that
  was measured and dropped.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-30 15:20:12 +01:00
Niels Lohmann ea03d601af json_view: comparison without temporaries and a faster image float check (#5890)
* Compare json_view values without a vector per container and a basic_json per number

operator== of views (view/view, view/json, and editable documents) built two
vectors of members for every object it entered, sorted them with stable_sort
for every object, and converted each number to a basic_json value. It now:

- keeps the members of all open objects in one vector (reserved with
  geometric growth, so that nested objects do not reallocate), and orders
  them by insertion sort for up to 16 members, or not at all when the keys
  ascend already;
- compares numbers of one type directly, and builds a basic_json only for
  numbers of different types;
- compares short texts (keys, numbers, most strings) with two overlapping
  loads (bytes_equal(), in node.hpp);
- first walks both documents in lockstep, in the order of their text, which
  finds most pairs of equal values without the work of the comparison. The
  walk only enters an object if the comparison would visit its members in the
  same order (the object type keeps the order, the keys ascend, or all values
  are scalars), so a difference is never found later than before, and it
  records the containers it found different, so that the comparison does not
  walk them again;
- rejects values of different types and arrays of different sizes before any
  of this, and, for an object type that keeps the order, objects whose first
  (up to four) members have different keys, which ends a comparison that
  differs near the start without preparing the members.

Results are unchanged: numbers compare by value across types, duplicate keys
resolve as parse() resolves them, and objects compare in document order for
ordered_json and by key otherwise.

Instructions of one comparison of a document with an equal copy, macOS arm64
(develop -> now), json_document / ordered_json_document / editable:

  twitter    16.1M -> 7.0M   17.7M -> 1.5M    16.6M -> 7.3M
  citm       45.0M -> 3.4M   47.0M -> 3.2M    45.7M -> 4.3M
  canada    111.4M -> 13.7M  111.4M -> 13.7M  113.3M -> 15.0M
  jeopardy   26.2M -> 3.7M   27.3M -> 3.7M    26.6M -> 12.1M

and of a view with a basic_json value:

  twitter    11.4M -> 5.9M   11.2M -> 7.3M    11.6M -> 5.9M
  citm       33.9M -> 9.7M   32.6M -> 9.1M    34.6M -> 9.8M
  canada     92.9M -> 32.1M  92.9M -> 32.1M   93.9M -> 32.8M

and Linux instruction counts (GCC 13 / Clang 18) of the same comparisons:

  document with an equal copy   json_document     ordered_json_document   editable
    twitter                     -49 % / -52 %     -89 % / -89 %           -48 % / -50 %
    citm                        -90 % / -89 %     -91 % / -90 %           -88 % / -87 %
    jeopardy (4,000 questions)  -85 % / -85 %     -86 % / -86 %           -46 % / -49 %
  view with a basic_json value
    twitter                     -41 % / -45 %     -28 % / -35 %           -45 % / -44 %
    citm                        -63 % / -65 %     -63 % / -66 %           -65 % / -64 %
    jeopardy                    -39 % / -43 %     -14 % / -24 %           -44 % / -43 %

A sweep that places one difference at about 2,100 positions of eight
documents (a token at an even spacing, the first and last tokens, the first,
middle and last member of the root removed, one appended, the document
wrapped) in five kinds of comparison (view/view of read-only and of editable
documents, view/json, json_document, editable/view) finds the same results as
before and no placement slower on the Mac (Clang); on Linux no placement is
slower than 3 % (Clang and GCC).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Check the float tokens of a json_document image without a sort and a scan of every character

load(image_check::full) checked every float token of an image by sorting
the token ranges, scanning the token character by character, and comparing
the digit layout with the node's. Now:

- the ranges are sorted only if they are not sorted already (save() writes
  them in the order of the nodes, so they are);
- a token of the shape [-]digits.digits whose digit counts are the ones the
  node records (nearly all of them) is accepted by checking its digits eight
  at a time, without the general scan;
- the general scan skips runs of digits eight at a time.

Damaged tokens are still rejected as before: the check accepts exactly the
tokens that the general scan accepts.

Instructions of load(image_check::full) (develop -> now):

  macOS arm64        canada 44.6M -> 30.9M (-31 %), jeopardy 11.3M -> 10.7M,
                     citm 10.4M -> 10.0M, twitter 5.06M -> 4.94M
  Linux GCC 13       canada -50 %, jeopardy -43 %, twitter -31 %, citm -28 %
  Linux Clang 18     canada -42 %, jeopardy -48 %, twitter -35 %, citm -32 %

image_check::bounds is unchanged (within 0.4 %).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Rename names that Flawfinder takes for the file function open()

The comparison cursors' open() is now enter(), and a test's open and
close strings are head and tail.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix the compiler and lint failures of the json_view comparison

- all_digits(): the lambda declares its constant itself; MSVC does not
  let a lambda without captures read a constexpr local of the function
- equal(): the frame's "in document order" index is a namespace-scope
  constant (document_order); the local constexpr npos in the default
  member initializers of the local struct crashed GCC 9 and MinGW GCC
  and was not found by MSVC
- keys_differ_early() keeps the first keys in std::arrays (cpplint:
  variable-length arrays; clang-tidy: C arrays), and the basic_json
  side's first_keys() is not noexcept (key() may throw)
- the numbers of one type are compared in an if chain instead of a
  nested conditional operator
- the dump test avoids C arrays and temporary string concatenations

View comparison instructions are unchanged (Apple clang, twitter, citm,
canada: -0.1 % to +0.1 %).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Keep the byte comparison helpers in compare.hpp

Address review comments: bytes_equal() and bytes_less() move from node.hpp
to compare.hpp, their only user; key_less() no longer splits the comment of
resolve_indices() from its function; and the loop counters of bytes_equal()
and all_digits() are scoped to their loops.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Dispatch on the compile-time conditions of the json_view comparison

MSVC (/W4 /WX) reported C4127 (conditional expression is constant) for
the template parameter EA and the object type's ordering in an if
condition; both are now tag-dispatched.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Remove a nested duplicate guard from the json_view image test

clang-tidy reported the inner #if !defined(JSON_NOEXCEPTION) as
redundant (the test case is in the same #if) and a find() != npos, which
gets the NOLINT the other tests use.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Mark Infer's null dereference in materialize() as a false positive

Infer (ci_infer, also red on develop since #5882) reports a null
dereference in json_sax_dom_parser::null() called from materialize(). The
member pointer the parser stores a value of an object in is set by the
key() before it, so it is not null there.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Move the json_view comparison tests into their own file

With the tests this branch adds, the object file of unit-json_view_dump.cpp
got too large for the MinGW x64 assembler ("file too big"). The test case
"json_view comparison" is now unit-json_view_compare.cpp, unchanged and
also built with C++17.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Build the DOM parser with Visual Studio 2015 again

Since #5882, the three VS 2015 jobs of AppVeyor fail on develop:
- the detection of keys that can be indexed instantiated std::hash for
  every string type, which VS 2015 rejects with a static_assert for a type
  it does not hash (alt_string): std::hash is now looked at only for
  objects in a vector
- the frame stack copied its frames with std::copy to a raw pointer, which
  VS 2015 reports as unsafe (C4996, an error with /WX): a loop now

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* json_view: no per-object allocations in ordered_json comparisons

For an object type that keeps the order of its members (ordered_json),
resolve_indices() allocated a vector per compared object, and
resolve_members() two. They now use one working vector owned by equal(),
whose capacity is reused, and resolve_indices() needs none when the keys
are unique. resolve_members() sorts small objects by insertion, as the
other paths do.

Instructions per comparison (callgrind, GCC 13 and clang 18, x86-64):
twitter, ordered_json view == ordered_json: 0.939 / 0.941; 20000 small
objects with a duplicate key: 0.741-0.768; all other rows (json,
citm_catalog, canada, view == view) within 0.988-1.003.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-30 15:19:42 +01:00
dependabot[bot] e2cb15c479 Bump the codeql-action group across 1 directory with 4 updates (#5961)
Bumps the codeql-action group with 4 updates in the / directory: [github/codeql-action/init](https://github.com/github/codeql-action), [github/codeql-action/autobuild](https://github.com/github/codeql-action), [github/codeql-action/analyze](https://github.com/github/codeql-action) and [github/codeql-action/upload-sarif](https://github.com/github/codeql-action).


Updates `github/codeql-action/init` from 4.39.0 to 4.39.1
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/6f2ac81f5cd81fe351a197ba0b00d91d03cd2aee...9b3254085494b55a0f7c4c2b3d2d52cc712ce370)

Updates `github/codeql-action/autobuild` from 4.39.0 to 4.39.1
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/6f2ac81f5cd81fe351a197ba0b00d91d03cd2aee...9b3254085494b55a0f7c4c2b3d2d52cc712ce370)

Updates `github/codeql-action/analyze` from 4.39.0 to 4.39.1
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/6f2ac81f5cd81fe351a197ba0b00d91d03cd2aee...9b3254085494b55a0f7c4c2b3d2d52cc712ce370)

Updates `github/codeql-action/upload-sarif` from 4.39.0 to 4.39.1
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/6f2ac81f5cd81fe351a197ba0b00d91d03cd2aee...9b3254085494b55a0f7c4c2b3d2d52cc712ce370)

---
updated-dependencies:
- dependency-name: github/codeql-action/analyze
  dependency-version: 4.39.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: codeql-action
- dependency-name: github/codeql-action/autobuild
  dependency-version: 4.39.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: codeql-action
- dependency-name: github/codeql-action/init
  dependency-version: 4.39.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: codeql-action
- dependency-name: github/codeql-action/upload-sarif
  dependency-version: 4.39.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: codeql-action
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-30 15:19:08 +01:00
dependabot[bot] c451fac1bc Bump step-security/harden-runner from 2.22.1 to 2.22.2 (#5962)
Bumps [step-security/harden-runner](https://github.com/step-security/harden-runner) from 2.22.1 to 2.22.2.
- [Release notes](https://github.com/step-security/harden-runner/releases)
- [Commits](https://github.com/step-security/harden-runner/compare/ccd8616d44fd3846e67624a50d5aad6d37bf2d25...dc2769abcc0fcfbadfc1df54081c1608579ef6b2)

---
updated-dependencies:
- dependency-name: step-security/harden-runner
  dependency-version: 2.22.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-30 15:18:56 +01:00
dependabot[bot] 28f71914b5 Bump egor-tensin/setup-mingw (#5963)
Bumps [egor-tensin/setup-mingw](https://github.com/egor-tensin/setup-mingw) from 41b837e47d7f85214629d255b9c4bc3fcbe9fd63 to d54f5ef0ce585bd3fbb1f8a71b5b254f59dbf931.
- [Release notes](https://github.com/egor-tensin/setup-mingw/releases)
- [Changelog](https://github.com/egor-tensin/setup-mingw/blob/master/CHANGELOG.md)
- [Commits](https://github.com/egor-tensin/setup-mingw/compare/41b837e47d7f85214629d255b9c4bc3fcbe9fd63...d54f5ef0ce585bd3fbb1f8a71b5b254f59dbf931)

---
updated-dependencies:
- dependency-name: egor-tensin/setup-mingw
  dependency-version: d54f5ef0ce585bd3fbb1f8a71b5b254f59dbf931
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-30 15:18:44 +01:00
Niels Lohmann 0b91f9ddd8 json_view: check_utf8 skips well-formed runs with the parser's string scan (#5926)
* json_view: check_utf8 skips well-formed runs with the parser's string scan

Strings of 32 bytes or more that are written into an editable document
are validated with scan_string_run(), which skips well-formed UTF-8 16
bytes at a time; the byte-wise check runs only from where the scan found
ill-formed UTF-8, to report the same type_error.316. Shorter strings
keep the byte-wise check (inlined with a constant start, so that this
path is the code it was), where the vector scan costs more than it
saves.

set()/push_back() of strings of 32 bytes or more take 0.29-0.76 of the
time (x86-64 GCC, averaged over code layouts) and 0.25-0.80 (AArch64
clang); shorter strings take 1.5-4 % fewer instructions; the json_view
rows of bench_edit execute at most 0.04 % more instructions.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* tests: avoid sign conversion in json_view_edit UTF-8 check

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* json_view: use while in check_utf8_bytewise

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-30 15:17:55 +01:00
Niels Lohmann 0115bb47b3 Name the source and license of adapted code in the file comments (#5965)
The files with code adapted from fast_float, Zmij and simdjson carried the
authors' SPDX copyright lines, but their file comments did not say what
was taken or under which license: number_parse.hpp, to_chars.hpp,
simd_scan.hpp and view/simd.hpp now do, as zmij.hpp and the yyjson-based
view files already did. README and the license page now also name
fast_float's eight-digit parsing and the lexer's use of simdjson's UTF-8
check. Comments only.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-30 15:17:24 +01:00
Niels Lohmann ea8a05484b Fix implicit-conversion sanitizer failure in unit-class_parser.cpp (#5960)
Constructing a std::string from uint8_t iterators converts values >= 0x80
implicitly to char, which ci_test_clang_sanitizer rejects. Convert each
byte with an explicit static_cast.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-30 09:56:06 +01:00
Niels Lohmann 2aecd86b37 Document the contract of object keys passed to json_sax::key and key callbacks (#5956)
* Document the contract of object keys passed to json_sax::key and key callbacks

Describe what the name passed to json_sax::key and to a parser callback
for parse_event_t::key holds when parsing JSON text: escapes are decoded
to UTF-8, different spellings of a name yield identical strings, the name
is validated before the call (invalid escapes or UTF-8 give a parse error
and no key event), no normalization or case conversion is applied, the
event precedes the ':' check, and duplicate names are reported once per
occurrence. Also note that sax_parse passes binary-format keys unchanged.

See discussion #5955.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Add a test for the object key contract and correct the BON8 note

Pin the guarantees documented for json_sax::key and the key event of
parser callbacks: decoded names, validation before delivery, delivery
before the colon, one event per repeated name, and the binary formats.
BON8 rejects ill-formed UTF-8 in a name even with sax_parse, so the
note no longer lists it with the formats that pass bytes through.

See discussion #5955.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix the GCC build of the object key contract test

GCC 11 in C++11 mode rejects a default argument on a lambda parameter
(-Wc++14-extensions) and ignores a static_cast<void> on the
warn_unused_result of parse(). Pass the input format explicitly and bind
the parse result to a variable.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Add version history for the object key contract

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-29 07:37:33 +01:00
Niels Lohmann 5eccbbd2bc Fix clang-tidy misc-redundant-expression in unit-json_view_image.cpp (#5958)
Split the combined has_save/has_load static_asserts so clang-tidy no
longer sees both sides of && as equivalent (develop ci_clang_tidy failure).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-29 07:36:11 +01:00
Niels Lohmann 91a8503f83 Merge ci/tiers into ci/windows-macos-sets
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-28 22:09:54 +01:00
Niels Lohmann 66ee5d1b84 Merge ci/path-filters into ci/tiers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-28 22:09:54 +01:00
Niels Lohmann 7a4e6b7a5d Merge develop into ci/path-filters
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-28 22:09:53 +01:00
Niels Lohmann 46bead6e13 Fix the VS2015 build of unit-json_view_positions.cpp (#5957)
VS 2015 does not accept std::string::npos in a constant expression (C2131),
and the constant is only used at run time.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-28 22:08:55 +01:00
Niels Lohmann a4c4e92dbd Merge wip/5949 into ci/windows-macos-sets
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-28 20:32:12 +01:00
Niels Lohmann ae57b09bc6 Merge wip/5948 into ci/tiers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-28 20:32:11 +01:00
Niels Lohmann fd37b04813 Merge origin/develop into ci/path-filters
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-28 20:32:10 +01:00
Niels Lohmann 54f6b3ef06 json_view: source positions in materialize() with JSON_DIAGNOSTIC_POSITIONS (#5945)
* json_view: source positions in materialize() with JSON_DIAGNOSTIC_POSITIONS

With JSON_DIAGNOSTIC_POSITIONS, materialize() now gives every value the
start_pos()/end_pos() that basic_json::parse() would give it, so exceptions
about materialized values name the same bytes (#5864).

- json_sax_dom_parser takes the lexer type as a defaulted template
  parameter; materialize() passes position_source, which reports the
  positions it derives from the index. diagnostic_positions treats npos as
  "no position" and keeps ranges all-or-nothing.
- Strings with escapes keep the offset of their opening quote plus one in
  `next` (only with the macro); materialize() scans from there to the closing
  quote. The end of an array or object is found by scanning from the end of
  its last element over whitespace, comments and a trailing comma.
- In editable documents, values written by edits, arrays and objects whose
  elements changed, and containers whose last element has no position get
  none. Images keep the flag is_new, and load() checks that `next` of a
  string is within the text.

Nothing changes on the parse path without the macro. The view's own
exceptions still carry no byte range, so the tests that compare them with
those of materialized values skip it under the macro, as they skip the path
under JSON_DIAGNOSTICS.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Indent comment in unit-json_view_from_json.cpp

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Address review: clearer position code and docs

- position_source gets clear(); set_scalar() returns early for new values,
  switches on node_kind, and uses no local copies; the one position_source
  per materialize() call is documented
- no npos + 1 for containers without a position; a comment on why the end
  of a container is reported as its closing bracket
- less duplication under #if in materialize() and the builder
- diagnostic_positions sets npos explicitly
- docs: positions of basic_json values after changes, positions in images
  depend on how the document was parsed, when next is set, and the cost of
  positions while parsing

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Share the comment scanners of the view builder with materialize()

skip_line_comment() and find_block_comment_end() in scan.hpp are used by
the builder's comment() and by position_source::container_end(), which
had its own copy.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Address review: if/else for container positions, clearer next field

- the opening bracket of a container sets positions with an if/else instead
  of a ternary, and the frame takes its start from positions
- the closing bracket of a container is found in nested if/else; the parent's
  last_end is taken from positions, so it is npos whenever no range is set
- architecture.md: the next field reads "or 0 if it was not; all other
  values: 0" instead of "else 0; otherwise 0"

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix GCC unused-result and clang-tidy raw-string findings in unit-json_view_positions.cpp

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-28 20:24:49 +01:00
Niels Lohmann 091c10d4a5 Fix self-aliasing in json_pointer::operator/= and patch_inplace (#5936)
* Fix self-aliasing in json_pointer::operator/= and patch_inplace

json_pointer::operator/=(const json_pointer&) passed iterators into its
own token vector to insert() when a pointer was appended to itself
(p /= p), which is undefined behavior; libstdc++ produced "/a/b//" for
"/a/b". The pointer now appends a copy in that case.

basic_json::patch_inplace iterated the patch while changing *this, so
j.patch_inplace(j) read freed memory. It now applies a copy of the
patch when the patch is the value itself. A patch stored inside the
value remains undefined behavior and is documented as such; copying the
patch unconditionally would double the time of a large patch.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix JSON_DIAGNOSTICS test crash and clang-tidy hit in patch_inplace self-aliasing

The spare-capacity test reallocated non-empty elements through get_ref(), which leaves stale parent pointers under JSON_DIAGNOSTICS; reserve while the array is empty instead. Suppress performance-unnecessary-copy-initialization for the copy that is needed because *this is modified.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Detect a patch stored inside the patched value with JSON_DIAGNOSTICS

patch_inplace applied a copy only when the patch was the value itself. With JSON_DIAGNOSTICS the parent pointers make it cheap to also detect a patch nested inside the value, so that case is handled the same way. Without diagnostics it stays documented undefined behavior.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Silence misc-const-correctness for patch_aliases_this

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix clang-tidy findings in the operator/= self-append test

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-28 20:23:55 +01:00
Niels Lohmann 2120927a7a Fix the VS2015 build of unit-json_view_edit_values.cpp (#5947)
MSVC 2015 treats two function templates that differ only in a decltype
return type as a redefinition (C2995), which breaks all three Visual
Studio 2015 jobs on AppVeyor. Overload on the view's mode in the
parameter type instead; template argument deduction ignores the
read-only-to-editable conversion, so each view still selects the
overload of its own mode.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-28 19:46:28 +01:00
Niels Lohmann 282079ba82 Fix clang-tidy findings in unit-json_view_from_json.cpp (#5954)
* Fix clang-tidy findings in unit-json_view_from_json.cpp

Silence misc-use-internal-linkage on the macro-generated ADL functions,
reconcile -Weffc++ with readability-redundant-member-init, use auto for
get<T>() initializers, and use value() on the optional access.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Satisfy bugprone-unchecked-optional-access in unit-json_view_from_json.cpp

clang-tidy 22 also flags value(); check has_value() in the same expression.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-28 19:44:53 +01:00
Niels Lohmann e89d5fdbdd Merge ci/tiers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-28 07:11:45 +01:00
Niels Lohmann d587214ab2 Merge ci/path-filters
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-28 07:11:44 +01:00
Niels Lohmann bd442d5d07 Merge develop
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-28 07:11:43 +01:00
Niels Lohmann da83aa92c8 Reconcile -Weffc++ and readability-redundant-member-init in unit-bson.cpp (#5953) 2026-10-28 07:11:04 +01:00
Niels Lohmann 84c5421644 Fix regression1 operator[](json_pointer) test under JSON_NO_THREAD_LOCAL (#5952)
The test relies on a copied fifo_map restarting its timestamps. Copying a
basic_json does that only on the recursive path; with JSON_NO_THREAD_LOCAL
the copy is built member by member and keeps correct timestamps, so the
premise was missing and the test failed (ci_test_no_thread_local, MinGW
clang). Copy the object_t explicitly to create the state on every path.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-27 20:31:53 +01:00
Niels Lohmann 4c3e92b96e Split the macOS and Windows jobs into a PR set, a full set, and a nightly set
The 19 macOS and 32 Windows jobs ran on every push to every pull request.
They now use the same sets as the Ubuntu jobs:

- PR set (16): four Visual Studio 2022 and two Visual Studio 2026
  configurations, ARM64 Debug, Clang 20.1.8 with MinGW, MinGW x86, the
  C++20 module test, Xcode 16.0, 16.4, 26.3 and 26.6, C++14 on the
  latest Xcode, and the Swift Package Manager check.
- full set (34): every other Visual Studio configuration, Clang
  version, clang-cl, Xcode version and C++ standard.
- nightly set (1): 64-bit MinGW, which takes about 68 minutes. A failed
  nightly run opens or updates an issue.

No configuration is removed: tools/ci_jobs/ci_jobs.py --compare lists
157 configurations before and after, and all jobs keep their runner,
steps and environment. ci_jobs.py now checks all three workflows and
expands matrices that consist only of include entries correctly.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-27 19:45:39 +01:00
Niels Lohmann 087df7b0c2 Split the Ubuntu jobs into a PR set, a full set, and a nightly set
All ~100 Ubuntu jobs ran on every push to every pull request. They now
belong to three sets, chosen by each job's if: condition:

- PR set (16 configurations): every push to a pull request and every
  push. Together with the checks that always run, these jobs would
  have caught at least 92 % of the failing commits of Sep/Oct 2026.
- full set (73 configurations): after the PR set passed, on pushes and
  on pull requests with the "🚀 ready to merge" label.
- nightly set (10 configurations): Infer, Valgrind, coverage, icpc,
  icpx, nvhpc, CUDA, and ARM64; nightly on develop, on pushes to master
  and release/*, and by hand. A failed nightly run opens or updates an
  issue.

A matrix whose entries belong to different sets is split into one job
per set; the jobs share their steps through YAML anchors. No
configuration is removed: tools/ci_jobs/ci_jobs.py --compare lists 150
configurations before and after. The same tool checks in CI that every
job belongs to exactly one set, that the full set waits for the whole
PR set, and that the job table in quality_assurance.md is up to date.

The documentation describes the sets in a new section "How CI runs",
and the QA pipeline figure shows the new badges.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-27 19:40:21 +01:00
Niels Lohmann eae3c8217b Split the checks of check_path_filters.py into smaller functions
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-27 19:39:20 +01:00
Niels Lohmann 960c234cc6 Do not report the excluded files that check_path_filters.py lists itself
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-27 18:58:36 +01:00
Niels Lohmann b6e24d19a9 Skip the compiler workflows for pull requests that change no build input
ubuntu.yml, macos.yml and windows.yml run about 155 jobs on every pull
request, including those that change only the documentation. They now
have a pull_request path filter: it starts with '**' and excludes the
documentation, Markdown files, GitHub metadata, other workflows and the
license files, so a new file runs them unless it is excluded on purpose.
Pushes to develop, master and release/* are not filtered.

The documentation jobs and the REUSE check move from ubuntu.yml to the
new repository_checks.yml, which has no path filter and also runs
tools/check_path_filters. That check keeps the three filters identical
and fails when a file the compiler jobs keep mentions an excluded file.
It found two: tools/check_build_options reads the Meson section of
package_managers.md, and the swiftpm job builds the Swift example from
the documentation; both are included in the filter.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-27 18:35:03 +01:00
Niels Lohmann 8617d9c94c Fix test lints that are red on develop (effc++, clang-tidy) (#5946)
- unit-bson: initialize the SAX recorder's member (-Weffc++)
- unit-json_patch: build the reference document independently instead of
  copying it (performance-unnecessary-copy-initialization)
- unit-json_view_builder: use node_kind::link instead of an out-of-range
  enum cast (clang-analyzer-optin.core.EnumCastOutOfRange)

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-27 17:25:18 +01:00
Niels Lohmann eb8f675bc9 docs: explain signed zero in the diff() round-trip exceptions (#5944)
Spell out that numbers comparing equal produce no operation even if one is
zero and the other negative zero, in diff.md and features/json_patch.md.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-27 17:16:08 +01:00
Niels Lohmann 78a0687147 Strong exception guarantee for operator[](json_pointer) (#5940)
* Keep the value unchanged when operator[] with a JSON pointer throws

The non-const operator[](json_pointer) is documented with strong exception
safety, but get_unchecked created intermediate values while walking the
pointer and kept them when a later reference token failed, e.g.
json j; j["/a/99999999999999999999999"_json_pointer] threw 404 and left
{"a":[]}; padding an existing array or appending with "-" behaved alike,
and so did std::length_error/std::bad_alloc from a huge index.

get_unchecked now only walks existing values up to the last reference
token. The last token's key insertion or array growth is strongly
exception safe by itself. Any other change goes through
get_unchecked_create, which records the one change it makes to an existing
value (null converted, key inserted, array grown) in a RAII guard and
reverts it if a later step throws; everything else it creates lies below
that change. The guard needs no try/catch, so JSON_NOEXCEPTION builds are
unaffected. Successful calls create the same values as before.

Docs: state the guarantee in the exceptions list and version history of
operator[], fix the complexity of the pointer overloads of operator[] and
at, and warn that padding arrays up to an untrusted index allocates memory
proportional to the index (operator[].md, untrusted_input.md).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Make the strongly exception-safe pointer operator[] faster than before

The previous commit made get_unchecked strongly exception safe but slower
with GCC 16 when it creates values (up to +6 % instructions for appending
with "-"). This restructures it so that no benchmark row is slower than
before the strong guarantee, with GCC 16, Clang 22 and Apple Clang.

- get_unchecked walks existing values in one loop that inserts nothing.
  The last reference token is resolved in that walk: "-" appends with
  emplace_back(), an index grows the array with operator[], and a missing
  key is inserted with emplace_hint() at the position the walk's
  lower_bound() found, so the key is searched once.
- With std::less<string_t> or std::less<> as comparator, the walk checks
  the key at the lower bound for equality instead of equivalence, which is
  the same for these comparators and cheaper.
- get_unchecked_create (not inlined) handles all other changes. Its guard
  swaps a null value with a fresh null value instead of copying it and
  reverts the change in a non-inlined function, so a successful call pays
  one compare for it. As every null value converted afterwards is a fresh
  one, it is converted in place instead of assigning a new array or object.
- Keys are added to new objects with emplace(), which also replaces the
  basic_json::operator[] call for ordered_json.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix rollback of pointer operator[] for unusual object and string types

Follow-up to the strong exception guarantee of the non-const
operator[](json_pointer):

- get_unchecked_create assumed that emplace() always inserts the key that
  the walk did not find. For object types whose lookup and emplace()
  disagree (e.g. a copied fifo_map), nothing was inserted, the rollback
  erased the existing member, and the walk continued with a value of the
  wrong type (assertion, type_error.305, or a crash). It now checks the
  result of emplace(), arms the rollback only if a key was inserted, and
  otherwise continues the walk with the member found, as basic_json's
  operator[] does: such pointers throw out_of_range.404 and leave the value
  unchanged, as before the strong guarantee.
- The rollback erased the inserted key by key, which converts string_t to
  object_t::key_type and may allocate inside a noexcept function
  (std::terminate on bad_alloc with custom key types). It now erases by the
  iterator emplace() returned, and shrinks a grown array with erase() at
  the end instead of resize(). The documentation states the remaining
  requirement for custom template parameters.
- Creating values no longer calls string_t::begin(), so the non-const
  operator[](json_pointer) compiles with string types that lack it (as the
  alternative string type of the tests).
- Correct the @throw lists of get_unchecked and get_unchecked_create.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Address review comments on operator[](json_pointer) rollback

- undo() shrinks the array with resize() instead of erase()
- comments say what "will be" done in get_unchecked_create
- tests: avoid a clang-tidy performance-unnecessary-copy-initialization
  hit (unit-alt-string) and the word "access" in section names, which
  Flawfinder flags as race/access

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Let the pointer rollback guard call undo() unconditionally

The check in the destructor and JSON_HEDLEY_NEVER_INLINE on undo() made no
measurable difference: instruction counts of j[ptr] are identical with
GCC 13 -O3 and within 0.4 % with clang, timings are within noise.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-27 17:15:55 +01:00
83 changed files with 8228 additions and 3235 deletions

No files matched your search

+9
View File
@@ -154,6 +154,15 @@ and each page's title (H1) and declaration by
[`docs/docset/generate_docset.py`](https://github.com/nlohmann/json/blob/develop/docs/docset/generate_docset.py);
`make build` reports API pages that cannot be classified.
A pull request that changes only the documentation, Markdown files, GitHub metadata, or license files skips the
compiler workflows (Ubuntu, macOS, Windows); the documentation and license checks still run. See
[`tools/check_path_filters`](https://github.com/nlohmann/json/tree/develop/tools/check_path_filters) for the list of
skipped files and what to do when a compiler job starts reading one of them.
Most compiler jobs run only once a maintainer adds the label "🚀 ready to merge" to a pull request; until then, a
smaller set of jobs runs on every push. See
[How CI runs](https://json.nlohmann.me/community/quality_assurance/#how-ci-runs) for the sets and when they run.
### Amalgamate the source code
The single-header files
+1 -1
View File
@@ -28,7 +28,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ccd8616d44fd3846e67624a50d5aad6d37bf2d25 # v2.22.1
uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
with:
egress-policy: audit
+2 -2
View File
@@ -22,7 +22,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@ccd8616d44fd3846e67624a50d5aad6d37bf2d25 # v2.22.1
uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
with:
egress-policy: audit
@@ -46,7 +46,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ccd8616d44fd3846e67624a50d5aad6d37bf2d25 # v2.22.1
uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
with:
egress-policy: audit
+1 -1
View File
@@ -20,7 +20,7 @@ jobs:
timeout-minutes: 30
steps:
- name: Harden Runner
uses: step-security/harden-runner@ccd8616d44fd3846e67624a50d5aad6d37bf2d25 # v2.22.1
uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
with:
egress-policy: audit
+1 -1
View File
@@ -13,7 +13,7 @@ jobs:
runs-on: ubuntu-22.04
steps:
- name: Harden Runner
uses: step-security/harden-runner@ccd8616d44fd3846e67624a50d5aad6d37bf2d25 # v2.22.1
uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
with:
egress-policy: audit
+4 -4
View File
@@ -27,7 +27,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ccd8616d44fd3846e67624a50d5aad6d37bf2d25 # v2.22.1
uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
with:
egress-policy: audit
@@ -38,14 +38,14 @@ jobs:
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@6f2ac81f5cd81fe351a197ba0b00d91d03cd2aee # v4.39.0
uses: github/codeql-action/init@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4.39.1
with:
languages: c-cpp
# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@6f2ac81f5cd81fe351a197ba0b00d91d03cd2aee # v4.39.0
uses: github/codeql-action/autobuild@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4.39.1
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@6f2ac81f5cd81fe351a197ba0b00d91d03cd2aee # v4.39.0
uses: github/codeql-action/analyze@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4.39.1
@@ -20,7 +20,7 @@ jobs:
pull-requests: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@ccd8616d44fd3846e67624a50d5aad6d37bf2d25 # v2.22.1
uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
with:
egress-policy: audit
+1 -1
View File
@@ -21,7 +21,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@ccd8616d44fd3846e67624a50d5aad6d37bf2d25 # v2.22.1
uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
with:
egress-policy: audit
+2 -2
View File
@@ -31,7 +31,7 @@ jobs:
security-events: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@ccd8616d44fd3846e67624a50d5aad6d37bf2d25 # v2.22.1
uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
with:
egress-policy: audit
@@ -48,6 +48,6 @@ jobs:
output: 'flawfinder_results.sarif'
- name: Upload analysis results to GitHub Security tab
uses: github/codeql-action/upload-sarif@6f2ac81f5cd81fe351a197ba0b00d91d03cd2aee # v4.39.0
uses: github/codeql-action/upload-sarif@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4.39.1
with:
sarif_file: ${{github.workspace}}/flawfinder_results.sarif
+1 -1
View File
@@ -54,7 +54,7 @@ jobs:
timeout-minutes: 60
steps:
- name: Harden Runner
uses: step-security/harden-runner@ccd8616d44fd3846e67624a50d5aad6d37bf2d25 # v2.22.1
uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
with:
egress-policy: audit
+1 -1
View File
@@ -23,7 +23,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ccd8616d44fd3846e67624a50d5aad6d37bf2d25 # v2.22.1
uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
with:
egress-policy: audit
+83 -6
View File
@@ -7,25 +7,60 @@ on:
- master
- release/*
pull_request:
# labeled: the "🚀 ready to merge" label starts the full set (see below)
types: [opened, synchronize, reopened, labeled]
# Skip pull requests that change only files no job here reads. The list is checked by
# tools/check_path_filters/check_path_filters.py (repository_checks.yml); change both together.
paths:
- '**'
- '!docs/**'
- '!**.md'
- '!.github/*'
- '!.github/ISSUE_TEMPLATE/**'
- '!.github/external_ci/**'
- '!.github/workflows/**'
- '!.reuse/**'
- '!LICENSES/**'
- '!CITATION.cff'
- '!REUSE.toml'
# read by tools/check_build_options in the ci_meson_install job (ubuntu.yml)
- 'docs/mkdocs/docs/integration/package_managers.md'
# built by the swiftpm job (macos.yml)
- 'docs/mkdocs/docs/integration/swift/**'
- '.github/workflows/macos.yml'
workflow_dispatch:
# A run triggered by any other label gets a group of its own, so it does not
# cancel the run of the last push; its jobs are skipped.
concurrency:
group: ${{ github.workflow }}-${{ github.ref || github.run_id }}
group: ${{ github.workflow }}-${{ github.event_name == 'schedule' && 'nightly' || github.ref || github.run_id }}${{ github.event.action == 'labeled' && github.event.label.name != '🚀 ready to merge' && format('-{0}', github.run_id) || '' }}
cancel-in-progress: true
permissions:
contents: read
# The jobs are split into a PR set and a full set like in ubuntu.yml (see "How
# CI runs" in docs/mkdocs/docs/community/quality_assurance.md): the PR set runs
# on every push; the full set after the PR set passed, on pushes and on pull
# requests with the "🚀 ready to merge" label. tools/ci_jobs checks that every
# job belongs to exactly one set.
jobs:
###########################################################################
# PR set
###########################################################################
macos-15:
if: &pr_set ${{ github.event_name != 'schedule' && (github.event.action != 'labeled' || github.event.label.name == '🚀 ready to merge') }}
runs-on: macos-15 # https://github.com/actions/runner-images/blob/main/images/macos/macos-15-Readme.md
strategy:
matrix:
xcode: ['16.0', '16.1', '16.2', '16.3', '16.4', '26.0.1', '26.1.1', '26.2', '26.3']
xcode: ['16.0', '16.4', '26.3']
env:
DEVELOPER_DIR: /Applications/Xcode_${{ matrix.xcode }}.app/Contents/Developer
steps:
steps: &xcode_steps
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
@@ -37,10 +72,11 @@ jobs:
run: cd build ; ctest -j 10 --output-on-failure
macos-26:
if: *pr_set
runs-on: macos-26 # https://github.com/actions/runner-images/blob/main/images/macos/macos-26-arm64-Readme.md
strategy:
matrix:
xcode: ['26.4.1', '26.5', '26.6']
xcode: ['26.6']
env:
DEVELOPER_DIR: /Applications/Xcode_${{ matrix.xcode }}.app/Contents/Developer
@@ -56,12 +92,13 @@ jobs:
run: cd build ; ctest -j 10 --output-on-failure
xcode_standards:
if: *pr_set
runs-on: macos-latest
strategy:
matrix:
standard: [11, 14, 17, 20, 23, 26]
standard: [14]
steps:
steps: &xcode_standards_steps
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
@@ -73,6 +110,7 @@ jobs:
run: cd build ; ctest -j 10 --output-on-failure
swiftpm:
if: *pr_set
runs-on: macos-15
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
@@ -106,3 +144,42 @@ jobs:
EOF
cd /tmp/json-swiftpm-consumer
swift build
###########################################################################
# full set: starts after the PR set passed
###########################################################################
macos-15_full:
needs: &pr_set_jobs [macos-15, macos-26, xcode_standards, swiftpm]
if: &full_set ${{ github.event_name == 'push' || github.event_name == 'workflow_dispatch' || contains(github.event.pull_request.labels.*.name, '🚀 ready to merge') }}
runs-on: macos-15 # https://github.com/actions/runner-images/blob/main/images/macos/macos-15-Readme.md
strategy:
matrix:
xcode: ['16.1', '16.2', '16.3', '26.0.1', '26.1.1', '26.2']
env:
DEVELOPER_DIR: /Applications/Xcode_${{ matrix.xcode }}.app/Contents/Developer
steps: *xcode_steps
macos-26_full:
needs: *pr_set_jobs
if: *full_set
runs-on: macos-26 # https://github.com/actions/runner-images/blob/main/images/macos/macos-26-arm64-Readme.md
strategy:
matrix:
xcode: ['26.4.1', '26.5']
env:
DEVELOPER_DIR: /Applications/Xcode_${{ matrix.xcode }}.app/Contents/Developer
steps: *xcode_steps
xcode_standards_full:
needs: *pr_set_jobs
if: *full_set
runs-on: macos-latest
strategy:
matrix:
standard: [11, 17, 20, 23, 26]
steps: *xcode_standards_steps
+1 -1
View File
@@ -37,7 +37,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@ccd8616d44fd3846e67624a50d5aad6d37bf2d25 # v2.22.1
uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
with:
egress-policy: audit
+83
View File
@@ -0,0 +1,83 @@
name: Repository checks
# Checks that concern files the compiler workflows (ubuntu.yml, macos.yml,
# windows.yml) skip with their path filter: the documentation, the license
# information, and the CI configuration itself. This workflow has no path filter.
on:
push:
branches:
- develop
- master
- release/*
pull_request:
workflow_dispatch:
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref || github.run_id }}
cancel-in-progress: true
jobs:
check_ci_configuration:
runs-on: ubuntu-26.04
steps:
- name: Harden Runner
uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
with:
egress-policy: audit
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Check the path filters of the compiler workflows
run: python3 tools/check_path_filters/check_path_filters.py
- name: Check the job sets of the compiler workflows and their documentation
run: |
python3 -m venv venv
venv/bin/pip install --quiet -r tools/ci_jobs/requirements.txt
venv/bin/python tools/ci_jobs/ci_jobs.py
ci_reuse_compliance:
runs-on: ubuntu-26.04
steps:
- name: Harden Runner
uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
with:
egress-policy: audit
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Get latest CMake and ninja
uses: lukka/get-cmake@a89e43e22a96abafcd7a62db652a65ecd33052c9 # v4.4.4
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ci_reuse_compliance
ci_test_documentation:
runs-on: ubuntu-26.04
strategy:
matrix:
target: [ci_test_examples, ci_test_build_documentation, ci_test_documentation_mermaid]
steps:
- name: Harden Runner
uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
with:
egress-policy: audit
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
# the git-revision-date-localized plugin needs the history; a shallow clone makes the strict build fail
fetch-depth: ${{ matrix.target == 'ci_test_build_documentation' && '0' || '1' }}
- uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
if: matrix.target == 'ci_test_documentation_mermaid'
with:
node-version: 24
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ${{ matrix.target }}
+2 -2
View File
@@ -40,7 +40,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ccd8616d44fd3846e67624a50d5aad6d37bf2d25 # v2.22.1
uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
with:
egress-policy: audit
@@ -80,6 +80,6 @@ jobs:
# Upload the results to GitHub's code scanning dashboard.
- name: "Upload to code-scanning"
uses: github/codeql-action/upload-sarif@6f2ac81f5cd81fe351a197ba0b00d91d03cd2aee # v4.39.0
uses: github/codeql-action/upload-sarif@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4.39.1
with:
sarif_file: results.sarif
+2 -2
View File
@@ -36,7 +36,7 @@ jobs:
runs-on: ubuntu-26.04
steps:
- name: Harden Runner
uses: step-security/harden-runner@ccd8616d44fd3846e67624a50d5aad6d37bf2d25 # v2.22.1
uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
with:
egress-policy: audit
@@ -65,7 +65,7 @@ jobs:
# Upload SARIF file generated in previous step
- name: Upload SARIF file
uses: github/codeql-action/upload-sarif@6f2ac81f5cd81fe351a197ba0b00d91d03cd2aee # v4.39.0
uses: github/codeql-action/upload-sarif@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4.39.1
with:
sarif_file: semgrep.sarif
if: always()
+1 -1
View File
@@ -16,7 +16,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ccd8616d44fd3846e67624a50d5aad6d37bf2d25 # v2.22.1
uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
with:
egress-policy: audit
+346 -186
View File
@@ -7,17 +7,62 @@ on:
- master
- release/*
pull_request:
# labeled: the "🚀 ready to merge" label starts the full set (see below)
types: [opened, synchronize, reopened, labeled]
# Skip pull requests that change only files no job here reads. The list is checked by
# tools/check_path_filters/check_path_filters.py (repository_checks.yml); change both together.
paths:
- '**'
- '!docs/**'
- '!**.md'
- '!.github/*'
- '!.github/ISSUE_TEMPLATE/**'
- '!.github/external_ci/**'
- '!.github/workflows/**'
- '!.reuse/**'
- '!LICENSES/**'
- '!CITATION.cff'
- '!REUSE.toml'
# read by tools/check_build_options in the ci_meson_install job (ubuntu.yml)
- 'docs/mkdocs/docs/integration/package_managers.md'
# built by the swiftpm job (macos.yml)
- 'docs/mkdocs/docs/integration/swift/**'
- '.github/workflows/ubuntu.yml'
schedule:
# the nightly set (see below), on develop
- cron: '0 2 * * *'
workflow_dispatch:
permissions:
contents: read
# A run triggered by any other label gets a group of its own, so it does not
# cancel the run of the last push; its jobs are skipped.
concurrency:
group: ${{ github.workflow }}-${{ github.ref || github.run_id }}
group: ${{ github.workflow }}-${{ github.event_name == 'schedule' && 'nightly' || github.ref || github.run_id }}${{ github.event.action == 'labeled' && github.event.label.name != '🚀 ready to merge' && format('-{0}', github.run_id) || '' }}
cancel-in-progress: true
# The jobs are split into three sets (see "How CI runs" in
# docs/mkdocs/docs/community/quality_assurance.md):
#
# - PR set: every pull request push and every push; chosen because these jobs
# caught 92 % of the failing commits in Sep/Oct 2026.
# - full set: after the PR set passed, on pushes and on pull requests with the
# "🚀 ready to merge" label; every remaining compiler, version and standard.
# - nightly set: slow tools and rare compilers, nightly on develop, on pushes
# to master and release/*, and when started by hand.
#
# A matrix whose entries belong to different sets is split into one job per
# set; the jobs share their steps through YAML anchors. tools/ci_jobs checks
# that every job belongs to exactly one set.
jobs:
###########################################################################
# PR set
###########################################################################
ci_test_gcc:
if: &pr_set ${{ github.event_name != 'schedule' && (github.event.action != 'labeled' || github.event.label.name == '🚀 ready to merge') }}
runs-on: ubuntu-latest
container: gcc:latest
steps:
@@ -32,6 +77,7 @@ jobs:
run: cmake --build build --target ci_test_gcc
ci_meson_install:
if: *pr_set
runs-on: ubuntu-26.04
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
@@ -76,43 +122,15 @@ jobs:
cmake -S tests/cmake_import/project -B build-import-split -DCMAKE_PREFIX_PATH=${{ github.workspace }}/install-split
cmake --build build-import-split
ci_infer:
runs-on: ubuntu-26.04
steps:
- name: Harden Runner
uses: step-security/harden-runner@ccd8616d44fd3846e67624a50d5aad6d37bf2d25 # v2.22.1
with:
egress-policy: audit
- name: Install Infer
run: |
wget -q -O - "https://github.com/facebook/infer/releases/download/v1.3.0/infer-linux-x86_64-v1.3.0.tar.xz" | sudo tar -C /opt -xJ
sudo ln -s /opt/infer-linux-x86_64-v1.3.0/bin/infer /usr/local/bin/infer
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Get latest CMake and ninja
uses: lukka/get-cmake@a89e43e22a96abafcd7a62db652a65ecd33052c9 # v4.4.4
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ci_infer
- name: Archive Infer report
if: always()
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: infer-report
path: ${{ github.workspace }}/build/build_infer/infer-out/report.txt
if-no-files-found: ignore
ci_static_analysis_ubuntu:
if: *pr_set
runs-on: ubuntu-26.04
strategy:
matrix:
target: [ci_test_single_header, ci_cppcheck, ci_cpplint, ci_reproducible_tests, ci_non_git_tests, ci_offline_testdata, ci_reuse_compliance, ci_test_valgrind]
steps:
target: [ci_cpplint]
steps: &static_analysis_ubuntu_steps
- name: Harden Runner
uses: step-security/harden-runner@ccd8616d44fd3846e67624a50d5aad6d37bf2d25 # v2.22.1
uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
with:
egress-policy: audit
@@ -130,12 +148,13 @@ jobs:
run: cmake --build build --target ${{ matrix.target }}
ci_static_analysis_clang:
if: *pr_set
runs-on: ubuntu-latest
container: silkeh/clang:22
strategy:
matrix:
target: [ci_test_clang, ci_clang_tidy, ci_test_clang_sanitizer, ci_clang_analyze, ci_single_binaries]
steps:
target: [ci_test_clang, ci_clang_tidy, ci_test_clang_sanitizer, ci_single_binaries]
steps: &static_analysis_clang_steps
- name: Install git, clang-tools, iwyu (ci_single_binaries), and unzip
run: apt-get update ; apt-get install -y git clang-tools iwyu unzip
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
@@ -149,12 +168,13 @@ jobs:
run: cmake --build build --target ${{ matrix.target }}
ci_cmake_options:
if: *pr_set
runs-on: ubuntu-latest
container: ubuntu:24.04
strategy:
matrix:
target: [ci_cmake_flags, ci_test_diagnostics, ci_test_diagnostic_positions, ci_test_noexceptions, ci_test_noimplicitconversions, ci_test_legacycomparison, ci_test_noglobaludls, ci_test_disableenumserialization, ci_test_disabletuplereferenceconversion, ci_test_skiplibraryversioncheck, ci_test_simdutf, ci_test_strict_nul_handling, ci_test_delete_deprecated_functions, ci_test_no_thread_local]
steps:
target: [ci_test_diagnostics, ci_test_diagnostic_positions, ci_test_noexceptions, ci_test_noglobaludls]
steps: &cmake_options_steps
- name: Install build-essential
run: apt-get update ; apt-get install -y build-essential unzip wget git
# the CMake 3.5.0 release binary that ci_cmake_flags downloads links
@@ -174,50 +194,17 @@ jobs:
- name: Build
run: cmake --build build --target ${{ matrix.target }}
ci_test_coverage:
runs-on: ubuntu-26.04
steps:
- name: Harden Runner
uses: step-security/harden-runner@ccd8616d44fd3846e67624a50d5aad6d37bf2d25 # v2.22.1
with:
egress-policy: audit
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Install dependencies and de_DE locale
run: |
sudo apt-get clean
sudo apt-get update
sudo apt-get install -y build-essential cmake lcov ninja-build make locales gcc-multilib g++-multilib
sudo locale-gen de_DE
sudo update-locale
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ci_test_coverage
- name: Archive coverage report
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: code-coverage-report
path: ${{ github.workspace }}/build/html
- name: Publish report to Coveralls
uses: coverallsapp/github-action@8d6379e14d29928660c4ba802d8e85393440b329 # v2.3.8
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
path-to-lcov: ${{ github.workspace }}/build/json.info.filtered.noexcept
fail-on-error: false
ci_test_compilers_gcc_old:
if: *pr_set
runs-on: ubuntu-latest
strategy:
matrix:
compiler: ['4.8', '4.9', '5', '6']
compiler: ['4.8']
# official gcc:4.8/4.9/5/6 images fail to check out code (too old for
# actions/checkout); install the old compilers on top of official ubuntu:20.04
# instead, mirroring what the (now retired) custom json-ci image did.
container: ubuntu:20.04
steps:
steps: &compilers_gcc_old_steps
- name: Install g++-${{ matrix.compiler }}
run: |
export DEBIAN_FRONTEND=noninteractive
@@ -250,7 +237,137 @@ jobs:
- name: Build
run: cmake --build build --target ci_test_compiler_g++-${{ matrix.compiler }}
ci_test_compilers_gcc:
ci_test_compilers_clang:
if: *pr_set
runs-on: ubuntu-latest
strategy:
matrix:
compiler: ['3.4']
container: silkeh/clang:${{ matrix.compiler }}
steps: &compilers_clang_steps
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Get latest CMake and ninja
uses: lukka/get-cmake@a89e43e22a96abafcd7a62db652a65ecd33052c9 # v4.4.4
- name: Disable filesystem support (std::filesystem is unusable) for clang 7 / 8 / 9 / 10
run: echo "JSON_FORCED_GLOBAL_COMPILE_OPTIONS=-DJSON_HAS_FILESYSTEM=0;-DJSON_HAS_EXPERIMENTAL_FILESYSTEM=0" >> "$GITHUB_ENV"
if: ${{ matrix.compiler == '7' || matrix.compiler == '8' || matrix.compiler == '9' || matrix.compiler == '10' }}
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ci_test_compiler_default
ci_test_standards_gcc:
if: *pr_set
runs-on: ubuntu-latest
container: gcc:latest
strategy:
matrix:
standard: [26]
steps: &standards_gcc_steps
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Get latest CMake and ninja
uses: lukka/get-cmake@a89e43e22a96abafcd7a62db652a65ecd33052c9 # v4.4.4
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ci_test_gcc_cxx${{ matrix.standard }}
ci_test_standards_clang:
if: *pr_set
runs-on: ubuntu-latest
container: silkeh/clang:latest
strategy:
matrix:
standard: [26]
stdlib: [libcxx]
steps: &standards_clang_steps
- name: Install git and unzip
run: apt-get update ; apt-get install -y git unzip
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Get latest CMake and ninja
uses: lukka/get-cmake@a89e43e22a96abafcd7a62db652a65ecd33052c9 # v4.4.4
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build with libc++
run: cmake --build build --target ci_test_clang_libcxx_cxx${{ matrix.standard }}
if: ${{ matrix.stdlib == 'libcxx' }}
- name: Build with libstdc++
run: cmake --build build --target ci_test_clang_cxx${{ matrix.standard }}
if: ${{ matrix.stdlib == 'libstdcxx' }}
ci_emscripten:
if: *pr_set
runs-on: ubuntu-26.04
steps:
- name: Harden Runner
uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
with:
egress-policy: audit
- name: Install emscripten
uses: mymindstorm/setup-emsdk@4528d102f7230f0e7b276855c01ea1159be0e984 # v16
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Get latest CMake and ninja
uses: lukka/get-cmake@a89e43e22a96abafcd7a62db652a65ecd33052c9 # v4.4.4
- name: Run CMake
run: cmake -S . -B build -DCMAKE_TOOLCHAIN_FILE=$EMSDK/upstream/emscripten/cmake/Modules/Platform/Emscripten.cmake -GNinja
- name: Build
run: cmake --build build
###########################################################################
# full set: starts after the PR set passed
###########################################################################
ci_static_analysis_ubuntu_full:
needs: &pr_set_jobs [ci_test_gcc, ci_meson_install, ci_static_analysis_ubuntu, ci_static_analysis_clang, ci_cmake_options, ci_test_compilers_gcc_old, ci_test_compilers_clang, ci_test_standards_gcc, ci_test_standards_clang, ci_emscripten]
if: &full_set ${{ github.event_name == 'push' || github.event_name == 'workflow_dispatch' || contains(github.event.pull_request.labels.*.name, '🚀 ready to merge') }}
runs-on: ubuntu-26.04
strategy:
matrix:
target: [ci_test_single_header, ci_cppcheck, ci_reproducible_tests, ci_non_git_tests, ci_offline_testdata]
steps: *static_analysis_ubuntu_steps
ci_static_analysis_clang_full:
needs: *pr_set_jobs
if: *full_set
runs-on: ubuntu-latest
container: silkeh/clang:22
strategy:
matrix:
target: [ci_clang_analyze]
steps: *static_analysis_clang_steps
ci_cmake_options_full:
needs: *pr_set_jobs
if: *full_set
runs-on: ubuntu-latest
container: ubuntu:24.04
strategy:
matrix:
target: [ci_cmake_flags, ci_test_noimplicitconversions, ci_test_legacycomparison, ci_test_disableenumserialization, ci_test_disabletuplereferenceconversion, ci_test_skiplibraryversioncheck, ci_test_simdutf, ci_test_strict_nul_handling, ci_test_delete_deprecated_functions, ci_test_no_thread_local]
steps: *cmake_options_steps
ci_test_compilers_gcc_old_full:
needs: *pr_set_jobs
if: *full_set
runs-on: ubuntu-latest
strategy:
matrix:
compiler: ['4.9', '5', '6']
container: ubuntu:20.04
steps: *compilers_gcc_old_steps
ci_test_compilers_gcc_full:
needs: *pr_set_jobs
if: *full_set
runs-on: ubuntu-latest
strategy:
matrix:
@@ -268,91 +385,44 @@ jobs:
- name: Build
run: cmake --build build --target ci_test_compiler_default
ci_test_compilers_clang:
ci_test_compilers_clang_full:
needs: *pr_set_jobs
if: *full_set
runs-on: ubuntu-latest
strategy:
matrix:
compiler: ['3.4', '3.5', '3.6', '3.7', '3.8', '3.9', '4', '5', '6', '7', '8', '9', '10', '11', '12', '13', '14', '15-bullseye', '16', '17', '18', '19', '20', '21', '22', 'latest']
compiler: ['3.5', '3.6', '3.7', '3.8', '3.9', '4', '5', '6', '7', '8', '9', '10', '11', '12', '13', '14', '15-bullseye', '16', '17', '18', '19', '20', '21', '22', 'latest']
container: silkeh/clang:${{ matrix.compiler }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Get latest CMake and ninja
uses: lukka/get-cmake@a89e43e22a96abafcd7a62db652a65ecd33052c9 # v4.4.4
- name: Disable filesystem support (std::filesystem is unusable) for clang 7 / 8 / 9 / 10
run: echo "JSON_FORCED_GLOBAL_COMPILE_OPTIONS=-DJSON_HAS_FILESYSTEM=0;-DJSON_HAS_EXPERIMENTAL_FILESYSTEM=0" >> "$GITHUB_ENV"
if: ${{ matrix.compiler == '7' || matrix.compiler == '8' || matrix.compiler == '9' || matrix.compiler == '10' }}
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ci_test_compiler_default
steps: *compilers_clang_steps
ci_test_standards_gcc:
ci_test_standards_gcc_full:
needs: *pr_set_jobs
if: *full_set
runs-on: ubuntu-latest
container: gcc:latest
strategy:
matrix:
standard: [11, 14, 17, 20, 23, 26]
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Get latest CMake and ninja
uses: lukka/get-cmake@a89e43e22a96abafcd7a62db652a65ecd33052c9 # v4.4.4
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ci_test_gcc_cxx${{ matrix.standard }}
standard: [11, 14, 17, 20, 23]
steps: *standards_gcc_steps
ci_test_standards_clang:
ci_test_standards_clang_full:
needs: *pr_set_jobs
if: *full_set
runs-on: ubuntu-latest
container: silkeh/clang:latest
strategy:
matrix:
standard: [11, 14, 17, 20, 23, 26]
stdlib: [libcxx, libstdcxx]
steps:
- name: Install git and unzip
run: apt-get update ; apt-get install -y git unzip
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Get latest CMake and ninja
uses: lukka/get-cmake@a89e43e22a96abafcd7a62db652a65ecd33052c9 # v4.4.4
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build with libc++
run: cmake --build build --target ci_test_clang_libcxx_cxx${{ matrix.standard }}
if: ${{ matrix.stdlib == 'libcxx' }}
- name: Build with libstdc++
run: cmake --build build --target ci_test_clang_cxx${{ matrix.standard }}
if: ${{ matrix.stdlib == 'libstdcxx' }}
ci_cuda_example:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
# 11.8.0: newest pre-C++20 CUDA release, exercises the C++17 fallback
# path (tests/cuda_example/CMakeLists.txt picks the standard per nvcc
# version); 12.1.1: permanent regression guard for #3907 (nvcc 12.0/12.1
# choke on enable_borrowed_range at C++20, fixed in 12.2); 12.6.3: recent
# CUDA/C++20 coverage.
cuda: ['11.8.0', '12.1.1', '12.6.3']
container: nvidia/cuda:${{ matrix.cuda }}-devel-ubuntu22.04
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Get latest CMake and ninja
uses: lukka/get-cmake@a89e43e22a96abafcd7a62db652a65ecd33052c9 # v4.4.4
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ci_cuda_example
exclude:
# in the PR set
- standard: 26
stdlib: libcxx
steps: *standards_clang_steps
ci_module_cpp20:
needs: *pr_set_jobs
if: *full_set
strategy:
matrix:
container: ['gcc:latest', 'silkeh/clang:latest']
@@ -382,7 +452,109 @@ jobs:
- name: Build
run: cmake --build build --target ci_module_cpp20
###########################################################################
# nightly set
###########################################################################
ci_infer:
if: &nightly_set ${{ github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || (github.event_name == 'push' && github.ref != 'refs/heads/develop') }}
runs-on: ubuntu-26.04
steps:
- name: Harden Runner
uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
with:
egress-policy: audit
- name: Install Infer
run: |
wget -q -O - "https://github.com/facebook/infer/releases/download/v1.3.0/infer-linux-x86_64-v1.3.0.tar.xz" | sudo tar -C /opt -xJ
sudo ln -s /opt/infer-linux-x86_64-v1.3.0/bin/infer /usr/local/bin/infer
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Get latest CMake and ninja
uses: lukka/get-cmake@a89e43e22a96abafcd7a62db652a65ecd33052c9 # v4.4.4
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ci_infer
- name: Archive Infer report
if: always()
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: infer-report
path: ${{ github.workspace }}/build/build_infer/infer-out/report.txt
if-no-files-found: ignore
ci_static_analysis_ubuntu_nightly:
if: *nightly_set
runs-on: ubuntu-26.04
strategy:
matrix:
target: [ci_test_valgrind]
steps: *static_analysis_ubuntu_steps
ci_test_coverage:
if: *nightly_set
runs-on: ubuntu-26.04
steps:
- name: Harden Runner
uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
with:
egress-policy: audit
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Install dependencies and de_DE locale
run: |
sudo apt-get clean
sudo apt-get update
sudo apt-get install -y build-essential cmake lcov ninja-build make locales gcc-multilib g++-multilib
sudo locale-gen de_DE
sudo update-locale
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ci_test_coverage
- name: Archive coverage report
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: code-coverage-report
path: ${{ github.workspace }}/build/html
- name: Publish report to Coveralls
uses: coverallsapp/github-action@8d6379e14d29928660c4ba802d8e85393440b329 # v2.3.8
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
path-to-lcov: ${{ github.workspace }}/build/json.info.filtered.noexcept
fail-on-error: false
ci_cuda_example:
if: *nightly_set
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
# 11.8.0: newest pre-C++20 CUDA release, exercises the C++17 fallback
# path (tests/cuda_example/CMakeLists.txt picks the standard per nvcc
# version); 12.1.1: permanent regression guard for #3907 (nvcc 12.0/12.1
# choke on enable_borrowed_range at C++20, fixed in 12.2); 12.6.3: recent
# CUDA/C++20 coverage.
cuda: ['11.8.0', '12.1.1', '12.6.3']
container: nvidia/cuda:${{ matrix.cuda }}-devel-ubuntu22.04
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Get latest CMake and ninja
uses: lukka/get-cmake@a89e43e22a96abafcd7a62db652a65ecd33052c9 # v4.4.4
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ci_cuda_example
ci_icpc:
if: *nightly_set
runs-on: ubuntu-latest
# Intel discontinued the classic icc/icpc compiler in oneAPI 2024.0; this is
# Intel's own last officially published image that still includes it.
@@ -402,6 +574,7 @@ jobs:
run: cmake --build build --target ci_icpc
ci_icpx:
if: *nightly_set
runs-on: ubuntu-latest
container: intel/oneapi-hpckit:latest
steps:
@@ -416,6 +589,7 @@ jobs:
run: cmake --build build --target ci_icpx
ci_nvhpc:
if: *nightly_set
runs-on: ubuntu-latest
container: nvcr.io/nvidia/nvhpc:25.5-devel-cuda12.9-ubuntu22.04
steps:
@@ -429,54 +603,10 @@ jobs:
- name: Build
run: cmake --build build --target ci_nvhpc
ci_emscripten:
runs-on: ubuntu-26.04
steps:
- name: Harden Runner
uses: step-security/harden-runner@ccd8616d44fd3846e67624a50d5aad6d37bf2d25 # v2.22.1
with:
egress-policy: audit
- name: Install emscripten
uses: mymindstorm/setup-emsdk@4528d102f7230f0e7b276855c01ea1159be0e984 # v16
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Get latest CMake and ninja
uses: lukka/get-cmake@a89e43e22a96abafcd7a62db652a65ecd33052c9 # v4.4.4
- name: Run CMake
run: cmake -S . -B build -DCMAKE_TOOLCHAIN_FILE=$EMSDK/upstream/emscripten/cmake/Modules/Platform/Emscripten.cmake -GNinja
- name: Build
run: cmake --build build
ci_test_documentation:
runs-on: ubuntu-26.04
strategy:
matrix:
target: [ci_test_examples, ci_test_build_documentation, ci_test_documentation_mermaid]
steps:
- name: Harden Runner
uses: step-security/harden-runner@ccd8616d44fd3846e67624a50d5aad6d37bf2d25 # v2.22.1
with:
egress-policy: audit
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
# the git-revision-date-localized plugin needs the history; a shallow clone makes the strict build fail
fetch-depth: ${{ matrix.target == 'ci_test_build_documentation' && '0' || '1' }}
- uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
if: matrix.target == 'ci_test_documentation_mermaid'
with:
node-version: 24
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ${{ matrix.target }}
# Native Linux ARM64 runner with the latest GCC. The ubuntu-24.04-arm label is
# only available for public repositories.
ci_test_arm64:
if: *nightly_set
runs-on: ubuntu-24.04-arm
container: gcc:latest
steps:
@@ -491,3 +621,33 @@ jobs:
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ci_test_compiler_default
# A failure of a scheduled run opens an issue, or comments on the open one.
nightly_report:
needs: [ci_infer, ci_static_analysis_ubuntu_nightly, ci_test_coverage, ci_cuda_example, ci_icpc, ci_icpx, ci_nvhpc, ci_test_arm64]
if: ${{ always() && github.event_name == 'schedule' && contains(needs.*.result, 'failure') }}
runs-on: ubuntu-26.04
permissions:
issues: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@dc2769abcc0fcfbadfc1df54081c1608579ef6b2 # v2.22.2
with:
egress-policy: audit
- name: Open or update the nightly failure issue
env:
GH_TOKEN: ${{ github.token }}
GH_REPO: ${{ github.repository }}
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
WORKFLOW: ${{ github.workflow }}
SHA: ${{ github.sha }}
run: |
title="Nightly CI failure (${WORKFLOW})"
body="The nightly run of ${WORKFLOW} on ${SHA:0:10} failed: ${RUN_URL}"
number=$(gh issue list --state open --search "in:title \"${title}\"" --json number,title --jq "map(select(.title == \"${title}\")) | .[0].number // empty")
if [ -n "$number" ]; then
gh issue comment "$number" --body "$body"
else
gh issue create --title "$title" --body "$body"
fi
+217 -35
View File
@@ -7,28 +7,67 @@ on:
- master
- release/*
pull_request:
# labeled: the "🚀 ready to merge" label starts the full set (see below)
types: [opened, synchronize, reopened, labeled]
# Skip pull requests that change only files no job here reads. The list is checked by
# tools/check_path_filters/check_path_filters.py (repository_checks.yml); change both together.
paths:
- '**'
- '!docs/**'
- '!**.md'
- '!.github/*'
- '!.github/ISSUE_TEMPLATE/**'
- '!.github/external_ci/**'
- '!.github/workflows/**'
- '!.reuse/**'
- '!LICENSES/**'
- '!CITATION.cff'
- '!REUSE.toml'
# read by tools/check_build_options in the ci_meson_install job (ubuntu.yml)
- 'docs/mkdocs/docs/integration/package_managers.md'
# built by the swiftpm job (macos.yml)
- 'docs/mkdocs/docs/integration/swift/**'
- '.github/workflows/windows.yml'
schedule:
# the nightly set (see below), on develop
- cron: '0 2 * * *'
workflow_dispatch:
permissions:
contents: read
# A run triggered by any other label gets a group of its own, so it does not
# cancel the run of the last push; its jobs are skipped.
concurrency:
group: ${{ github.workflow }}-${{ github.ref || github.run_id }}
group: ${{ github.workflow }}-${{ github.event_name == 'schedule' && 'nightly' || github.ref || github.run_id }}${{ github.event.action == 'labeled' && github.event.label.name != '🚀 ready to merge' && format('-{0}', github.run_id) || '' }}
cancel-in-progress: true
# The jobs are split into three sets like in ubuntu.yml (see "How CI runs" in
# docs/mkdocs/docs/community/quality_assurance.md): the PR set runs on every
# push; the full set after the PR set passed, on pushes and on pull requests
# with the "🚀 ready to merge" label; the nightly set nightly on develop, on
# pushes to master and release/*, and when started by hand. tools/ci_jobs
# checks that every job belongs to exactly one set.
jobs:
###########################################################################
# PR set
###########################################################################
mingw:
if: &pr_set ${{ github.event_name != 'schedule' && (github.event.action != 'labeled' || github.event.label.name == '🚀 ready to merge') }}
runs-on: windows-2022
strategy:
matrix:
architecture: [x64, x86]
architecture: [x86]
steps:
steps: &mingw_steps
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Set up MinGW
uses: egor-tensin/setup-mingw@41b837e47d7f85214629d255b9c4bc3fcbe9fd63 # v3.0
uses: egor-tensin/setup-mingw@d54f5ef0ce585bd3fbb1f8a71b5b254f59dbf931 # v3.0
with:
platform: ${{ matrix.architecture }}
version: 12.2.0 # https://github.com/egor-tensin/setup-mingw/issues/14
@@ -40,15 +79,26 @@ jobs:
run: cd build ; ctest -j 10 -C Debug --output-on-failure
msvc:
if: *pr_set
strategy:
matrix:
build_type: [Debug, Release]
architecture: [Win32, x64]
std_version: [default, latest]
include:
- build_type: Debug
architecture: x64
std_version: default
- build_type: Release
architecture: x64
std_version: default
- build_type: Debug
architecture: Win32
std_version: default
- build_type: Release
architecture: Win32
std_version: latest
runs-on: windows-2022
steps:
steps: &msvc_steps
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
@@ -77,15 +127,20 @@ jobs:
# Visual Studio 2026 (v145 toolset) on the windows-2025 image. The "Visual Studio
# 18 2026" generator requires CMake 4.2+, so a recent CMake is fetched explicitly.
msvc-vs2026:
if: *pr_set
strategy:
matrix:
build_type: [Debug, Release]
architecture: [Win32, x64]
std_version: [default, latest]
include:
- build_type: Debug
architecture: x64
std_version: latest
- build_type: Release
architecture: Win32
std_version: default
runs-on: windows-2025
steps:
steps: &msvc_vs2026_steps
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
@@ -117,13 +172,14 @@ jobs:
# Native ARM64 Windows runner with the MSVC ARM64 toolset. The windows-11-arm
# label is only available for public repositories.
msvc-arm64:
if: *pr_set
strategy:
matrix:
build_type: [Debug, Release]
build_type: [Debug]
runs-on: windows-11-arm
steps:
steps: &msvc_arm64_steps
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
@@ -141,19 +197,20 @@ jobs:
run: cd build ; ctest -j 10 -C ${{ matrix.build_type }} --output-on-failure
clang:
if: *pr_set
runs-on: windows-2022
strategy:
matrix:
version: [11.0.1, 12.0.1, 13.0.1, 14.0.6, 15.0.7, 16.0.6, 18.1.8, 19.1.7, 20.1.8]
version: [20.1.8]
steps:
steps: &clang_steps
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Install Clang
run: curl -fsSL -o LLVM${{ matrix.version }}.exe https://github.com/llvm/llvm-project/releases/download/llvmorg-${{ matrix.version }}/LLVM-${{ matrix.version }}-win64.exe ; 7z x LLVM${{ matrix.version }}.exe -y -o"C:/Program Files/LLVM"
- name: Set up MinGW
uses: egor-tensin/setup-mingw@41b837e47d7f85214629d255b9c4bc3fcbe9fd63 # v3.0
uses: egor-tensin/setup-mingw@d54f5ef0ce585bd3fbb1f8a71b5b254f59dbf931 # v3.0
with:
platform: x64
version: 12.2.0 # https://github.com/egor-tensin/setup-mingw/issues/14
@@ -189,24 +246,8 @@ jobs:
- name: Test
run: cd build ; ctest -j 10 -C Debug --output-on-failure
clang-cl:
runs-on: windows-2022
strategy:
matrix:
architecture: [Win32, x64]
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Run CMake
run: cmake -S . -B build -G "Visual Studio 17 2022" -A ${{ matrix.architecture }} -T ClangCL -DJSON_BuildTests=On
- name: Build
run: cmake --build build --config Debug --parallel 10
- name: Test
run: cd build ; ctest -j 10 -C Debug --output-on-failure
ci_module_cpp20:
if: *pr_set
runs-on: windows-2022
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
@@ -225,4 +266,145 @@ jobs:
run: cmake -S . -B build -G "Visual Studio 17 2022" -DJSON_CI=ON -DCMAKE_CXX_FLAGS="/permissive- /std:c++latest /utf-8 /EHsc /W4 /WX"
- name: Build
run: cmake --build build --config Debug --target ci_module_cpp20
###########################################################################
# full set: starts after the PR set passed
###########################################################################
msvc_full:
needs: &pr_set_jobs [mingw, msvc, msvc-vs2026, msvc-arm64, clang, ci_module_cpp20]
if: &full_set ${{ github.event_name == 'push' || github.event_name == 'workflow_dispatch' || contains(github.event.pull_request.labels.*.name, '🚀 ready to merge') }}
strategy:
matrix:
build_type: [Debug, Release]
architecture: [Win32, x64]
std_version: [default, latest]
exclude:
# in the PR set
- build_type: Debug
architecture: x64
std_version: default
- build_type: Release
architecture: x64
std_version: default
- build_type: Debug
architecture: Win32
std_version: default
- build_type: Release
architecture: Win32
std_version: latest
runs-on: windows-2022
steps: *msvc_steps
# Visual Studio 2026 (v145 toolset) on the windows-2025 image. The "Visual Studio
# 18 2026" generator requires CMake 4.2+, so a recent CMake is fetched explicitly.
msvc-vs2026_full:
needs: *pr_set_jobs
if: *full_set
strategy:
matrix:
build_type: [Debug, Release]
architecture: [Win32, x64]
std_version: [default, latest]
exclude:
# in the PR set
- build_type: Debug
architecture: x64
std_version: latest
- build_type: Release
architecture: Win32
std_version: default
runs-on: windows-2025
steps: *msvc_vs2026_steps
# Native ARM64 Windows runner with the MSVC ARM64 toolset. The windows-11-arm
# label is only available for public repositories.
msvc-arm64_full:
needs: *pr_set_jobs
if: *full_set
strategy:
matrix:
build_type: [Release]
runs-on: windows-11-arm
steps: *msvc_arm64_steps
clang_full:
needs: *pr_set_jobs
if: *full_set
runs-on: windows-2022
strategy:
matrix:
version: [11.0.1, 12.0.1, 13.0.1, 14.0.6, 15.0.7, 16.0.6, 18.1.8, 19.1.7]
steps: *clang_steps
clang-cl_full:
needs: *pr_set_jobs
if: *full_set
runs-on: windows-2022
strategy:
matrix:
architecture: [Win32, x64]
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Run CMake
run: cmake -S . -B build -G "Visual Studio 17 2022" -A ${{ matrix.architecture }} -T ClangCL -DJSON_BuildTests=On
- name: Build
run: cmake --build build --config Debug --parallel 10
- name: Test
run: cd build ; ctest -j 10 -C Debug --output-on-failure
###########################################################################
# nightly set
###########################################################################
mingw_nightly:
if: ${{ github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || (github.event_name == 'push' && github.ref != 'refs/heads/develop') }}
runs-on: windows-2022
strategy:
matrix:
architecture: [x64]
steps: *mingw_steps
# A failure of a scheduled run opens an issue, or comments on the open one.
nightly_report:
needs: [mingw_nightly]
if: ${{ always() && github.event_name == 'schedule' && contains(needs.*.result, 'failure') }}
runs-on: ubuntu-26.04
permissions:
issues: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@ccd8616d44fd3846e67624a50d5aad6d37bf2d25 # v2.22.1
with:
egress-policy: audit
- name: Open or update the nightly failure issue
env:
GH_TOKEN: ${{ github.token }}
GH_REPO: ${{ github.repository }}
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
WORKFLOW: ${{ github.workflow }}
SHA: ${{ github.sha }}
run: |
title="Nightly CI failure (${WORKFLOW})"
body="The nightly run of ${WORKFLOW} on ${SHA:0:10} failed: ${RUN_URL}"
number=$(gh issue list --state open --search "in:title \"${title}\"" --json number,title --jq "map(select(.title == \"${title}\")) | .[0].number // empty")
if [ -n "$number" ]; then
gh issue comment "$number" --body "$body"
else
gh issue create --title "$title" --body "$body"
fi
+2 -2
View File
@@ -1408,9 +1408,9 @@ THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR I
- The class contains a port of the shortest double-to-decimal conversion of [Żmij](https://github.com/vitaut/zmij) by Victor Zverovich, including the conversion of the digits to text by Xiang JunBo and the SIMD instruction sequence of Dougall Johnson, which is licensed under the [MIT License](https://opensource.org/licenses/MIT) (see above). Copyright &copy; 2025 [Victor Zverovich](https://github.com/vitaut)
- The class contains a copy of [Hedley](https://nemequ.github.io/hedley/) from Evan Nemerson which is licensed as [CC0-1.0](https://creativecommons.org/publicdomain/zero/1.0/).
- The class contains parts of [Google Abseil](https://github.com/abseil/abseil-cpp) which is licensed under the [Apache 2.0 License](https://opensource.org/licenses/Apache-2.0).
- The class contains an adapted version of the Eisel-Lemire algorithm, its table of powers of five, and its digit comparison for long numbers from [fast_float](https://github.com/fastfloat/fast_float) by Daniel Lemire and contributors, which is available under the [MIT License](https://opensource.org/licenses/MIT) (used here), the Apache 2.0 License, and the Boost Software License. Copyright &copy; 2021 The fast_float authors
- The class contains an adapted version of the Eisel-Lemire algorithm, its table of powers of five, its digit comparison for long numbers, and its parsing of eight digits at a time from [fast_float](https://github.com/fastfloat/fast_float) by Daniel Lemire and contributors, which is available under the [MIT License](https://opensource.org/licenses/MIT) (used here), the Apache 2.0 License, and the Boost Software License. Copyright &copy; 2021 The fast_float authors
- The view's parser (`<nlohmann/json_view.hpp>`) contains techniques and code adapted from [yyjson](https://github.com/ibireme/yyjson) by YaoYuan, which is licensed under the [MIT License](https://opensource.org/licenses/MIT) (see above): table-driven decoding of `\u` escapes and fixed-offset unrolled checks.
- The view's parser (`<nlohmann/json_view.hpp>`) validates non-ASCII strings with the vector UTF-8 check of [simdjson](https://github.com/simdjson/simdjson) by Daniel Lemire, Geoff Langdale, John Keiser, and contributors (its "lookup4" algorithm and tables, after J. Keiser and D. Lemire, "Validating UTF-8 In Less Than One Instruction Per Byte", 2021), which is available under the [MIT License](https://opensource.org/licenses/MIT) (used here) and the Apache 2.0 License. Copyright &copy; 2018-2025 The simdjson authors
- The class's lexer and the view's parser (`<nlohmann/json_view.hpp>`) validate non-ASCII strings with the vector UTF-8 check of [simdjson](https://github.com/simdjson/simdjson) by Daniel Lemire, Geoff Langdale, John Keiser, and contributors (its "lookup4" algorithm and tables, after J. Keiser and D. Lemire, "Validating UTF-8 In Less Than One Instruction Per Byte", 2021), which is available under the [MIT License](https://opensource.org/licenses/MIT) (used here) and the Apache 2.0 License. Copyright &copy; 2018-2025 The simdjson authors
<img align="right" src="https://git.fsfe.org/reuse/reuse-ci/raw/branch/master/reuse-horizontal.png" alt="REUSE Software">
+2 -1
View File
@@ -90,7 +90,8 @@ Strong exception safety: if an exception occurs, the original value stays intact
1. Constant.
2. Logarithmic in the size of the container.
3. Logarithmic in the size of the container.
4. Logarithmic in the size of the container.
4. Linear in the number of reference tokens of `ptr`; for each of them, logarithmic in the size of the object, or
constant for an array.
## Notes
+4 -3
View File
@@ -19,9 +19,10 @@ The equality holds up to [`operator==`](operator_eq.md), so it has the following
- A value that is not equal to itself cannot round-trip. This applies to `NaN` (also nested in an array or object) and
to [discarded](is_discarded.md) values: `diff` emits a `replace` operation for them. A `NaN` in the patch is
serialized as `#!json null`, so a patch that was dumped and parsed again would set `#!json null`.
- Numbers that compare equal produce no operation, even if their types or signs differ: `diff(1, 1.0)`,
`diff(1u, 1)` (unsigned and signed integer) and `diff(0.0, -0.0)` all return `#!json []`. The patched value is then
the `source` and keeps its number type.
- Numbers that compare equal produce no operation, even if their types differ or if one is zero and the other is
negative zero: `diff(1, 1.0)` (integer and floating-point), `diff(1u, 1)` (unsigned and signed integer) and
`diff(0.0, -0.0)` (positive and negative zero) all return `#!json []`. The patched value is then the `source` and
keeps its number type and sign.
## Parameters
+20 -2
View File
@@ -92,12 +92,20 @@ Strong exception safety: if an exception occurs, the original value stays intact
For the **const** version, an object key or array index in `ptr` that does not exist is not reported by an
exception, but is undefined behavior (see the notes below). Use [`at`](at.md) for checked access.
If the **non-const** version throws, the value is left unchanged: values it created for earlier reference tokens
of `ptr` are removed again. This also holds for exceptions such as `#!cpp std::bad_alloc` or
`#!cpp std::length_error` thrown while padding an array up to a very large index. With custom
[template parameters](../../features/types/template_parameters.md), this requires that `erase()` of the object and
array types and the move operations of the custom base class do not throw.
## Complexity
1. Constant if `idx` is in the range of the array. Otherwise, linear in `idx - size()`.
2. Logarithmic in the size of the container.
3. Logarithmic in the size of the container.
4. Logarithmic in the size of the container.
4. Linear in the number of reference tokens of `ptr`; for each of them, logarithmic in the size of the object, or
constant for an array. When the non-const version pads an array with `#!json null` values, it is additionally
linear in `idx - size()` for that array.
## Notes
@@ -145,6 +153,14 @@ Strong exception safety: if an exception occurs, the original value stays intact
specified by the JSON Pointer RFC; it is this library's own, intentional disambiguation rule. See also
[JSON Pointer](../../features/json_pointer.md).
!!! warning "Large array indices"
As the array is filled up with `#!json null` values to the given index, the memory used grows with the index
and not with the size of the pointer: on a 64-bit platform, the index `10000000` allocates about 160 MB.
The library does not limit the index, so check the array indices of a JSON pointer from untrusted input
before using it with the non-const version. See
[Parsing untrusted input](../../features/parsing/untrusted_input.md#json-pointers-and-flattened-keys).
!!! warning "Deprecation"
Overload (4) also accepts a [`json_pointer`](../json_pointer/index.md) whose template argument is a `basic_json`
@@ -296,4 +312,6 @@ Strong exception safety: if an exception occurs, the original value stays intact
4. Added in version 2.0.0. A missing array index in the const version is guarded by a runtime assertion since
version 3.13.0. Throws [`parse_error.109`](../../home/exceptions.md#jsonexceptionparse_error109) instead of
[`out_of_range.404`](../../home/exceptions.md#jsonexceptionout_of_range404) for a one-character array index that is not
a digit (e.g., `/x`) in version 3.13.0, as it already did for longer ones.
a digit (e.g., `/x`) in version 3.13.0, as it already did for longer ones. Fixed in version 3.13.0 to leave the value
unchanged if the non-const version throws (strong exception safety, as documented); before, values created for earlier
reference tokens of `ptr` were kept.
@@ -63,6 +63,15 @@ called:
Whether the JSON value which called the function during parsing should be kept (`#!cpp true`) or not (`#!cpp false`). In
the latter case, it is skipped completely, or replaced by `null` if it is the top-level value.
## Notes
For a `parse_event_t::key` event, `parsed` holds the decoded object name, with the same guarantees as the `val`
parameter of [`json_sax::key`](../json_sax/key.md#notes): all escape sequences are decoded and `\uXXXX` escapes are
stored as UTF-8, so different spellings of the same name yield identical strings; the name is validated completely
before the callback is called, so a name with an invalid escape sequence or ill-formed UTF-8 causes a parse error and
no `key` event; no Unicode normalization or case conversion is applied; and a name that occurs several times in the
same object yields one `key` event per occurrence.
## Examples
??? example "Example: skip an object key while parsing"
@@ -69,6 +69,19 @@ Duplicate member names inside an operation object are resolved by the parser bef
is kept, see [`object_t`](object_t.md#behavior)), so `#!json {"op": "add", "op": "remove", "path": "/a"}` is a
`remove` operation.
The JSON patch may be the JSON value itself (`#!cpp j.patch_inplace(j)`): the operations are then read from a copy of
the patch made before the first operation is applied, so the result is the same as that of
`#!cpp j = j.patch(j)`, and the exception safety is the same as above.
If [`JSON_DIAGNOSTICS`](../macros/json_diagnostics.md) is enabled, the same holds for a JSON patch stored inside the
JSON value (e.g., `#!cpp j.patch_inplace(j["patch"])`): it is detected via the parent pointers.
!!! danger "Undefined behavior"
Without [`JSON_DIAGNOSTICS`](../macros/json_diagnostics.md), the JSON patch must not be a value stored inside the
JSON value (e.g., `#!cpp j.patch_inplace(j["patch"])`), as the operations may change or destroy it while it is
read. Pass a copy (`#!cpp j.patch_inplace(json(j["patch"]))`) or use [`patch`](patch.md) instead.
## Examples
??? example "Example: apply a JSON patch in place"
@@ -122,3 +135,5 @@ is kept, see [`object_t`](object_t.md#behavior)), so `#!json {"op": "add", "op":
- Throws [`parse_error.109`](../../home/exceptions.md#jsonexceptionparse_error109) instead of
[`out_of_range.404`](../../home/exceptions.md#jsonexceptionout_of_range404) for a one-character array index that is
not a digit (e.g., `/x`) in version 3.13.0, as it already did for longer ones.
- Applying a JSON value as patch to itself (or, with `JSON_DIAGNOSTICS`, to a value stored inside it) no longer results
in undefined behavior in version 3.13.0.
@@ -32,9 +32,24 @@ uses, so the result matches it exactly — including, for an object, that a repe
last value. The replay is iterative, so it is not limited by the call stack the way a naive recursive conversion
would be; the JSON nesting depth is limited only by available memory, as for `BasicJsonType::parse()` itself.
Unlike parsing with [`JSON_DIAGNOSTIC_POSITIONS`](../macros/json_diagnostic_positions.md) enabled, the values
produced by `materialize()` do not carry source positions. The index has the start offset of every value, but not the
end of arrays, objects, and escaped strings, which the positions also need.
With [`JSON_DIAGNOSTIC_POSITIONS`](../macros/json_diagnostic_positions.md) enabled, the values get the source
positions [`BasicJsonType::parse()`](../basic_json/parse.md) would give them, also for the subtree of a view (the
positions are those in the document's whole source text), and exceptions about them name the same bytes. For values of
an [editable document](../json_editable_document.md), positions are all-or-nothing, and [`start_pos()`](../basic_json/start_pos.md)
and [`end_pos()`](../basic_json/end_pos.md) return `std::string::npos` for
- a value written by an edit (a copy of an array or object of the document is also new; copies of its scalars keep
their positions),
- an array or object whose elements were changed (its elements keep theirs), and
- an array or object whose last element has no positions (its end is found from that element).
This differs from a `basic_json` value, whose positions are not updated when it is changed: they stay as they were
parsed, even if they no longer match the value (see [`start_pos()`](../basic_json/start_pos.md)).
Documents [loaded](../basic_json_document/load.md) from an image have the positions of the saved document. Its strings
with escapes have positions only if the document was parsed with `JSON_DIAGNOSTIC_POSITIONS`: the position of their
opening quote is recorded while parsing, and `save()` and `load()` keep it whether or not they are compiled with the
macro.
Calling `materialize()` on the same view repeatedly builds a new, independent `BasicJsonType` value each time; it
never caches the result.
@@ -62,8 +77,7 @@ never caches the result.
- [`root`](../basic_json_document/root.md) - the view of a document's root value
- [`BasicJsonType::parse`](../basic_json/parse.md) - build a `BasicJsonType` value directly from a JSON text
- [`JSON_DIAGNOSTIC_POSITIONS`](../macros/json_diagnostic_positions.md) - source positions on parsed values (not
produced by `materialize()`)
- [`JSON_DIAGNOSTIC_POSITIONS`](../macros/json_diagnostic_positions.md) - source positions on parsed values
- [Zero-copy JSON views](../../features/json_view.md#getting-values-out-without-copying) - when to build a basic_json value from a view
## Version history
@@ -18,7 +18,8 @@ json_pointer& operator/=(std::size_t array_idx)
## Parameters
`ptr` (in)
: JSON pointer to append
: JSON pointer to append; may be the JSON pointer itself (`#!cpp p /= p`), which then appends a copy of its
reference tokens
`token` (in)
: reference token to append
@@ -69,6 +70,7 @@ json_pointer& operator/=(std::size_t array_idx)
## Version history
1. Added in version 3.6.0.
1. Added in version 3.6.0. Appending a JSON pointer to itself no longer results in undefined behavior in version
3.13.0.
2. Added in version 3.6.0. Changed type of `token` to `string_t` in version 3.11.0.
3. Added in version 3.6.0.
+21
View File
@@ -19,6 +19,23 @@ Whether parsing should proceed.
It is safe to move the passed object key value.
When parsing JSON text, `val` is the decoded object name, not its spelling in the input:
- All escape sequences (`\"`, `\\`, `\/`, `\b`, `\f`, `\n`, `\r`, `\t`, `\uXXXX`, and surrogate pairs) are
decoded, and `\uXXXX` escapes are stored as UTF-8. Different spellings of the same name, such as `#!json "/"`
and `#!json "\/"`, or `#!json "\u00e9"` and `#!json "\u00E9"`, yield identical strings. An escaped `\u0000` yields
an embedded NUL byte.
- The name is scanned and validated completely before `key` is called. If it contains an invalid escape sequence, an
unpaired surrogate, a control character, or ill-formed UTF-8, the parse fails with a
[`parse_error`](../../home/exceptions.md#parse-errors), and `key` is not called for it.
- No Unicode normalization or case conversion is applied: names that differ in any byte are different strings.
- `key` is called before the following `:` is read, so a parse error may still follow for the same member.
- A name that occurs several times in the same object is passed once per occurrence.
For the binary formats, object keys contain no escape sequences. For CBOR, MessagePack, UBJSON, and BSON, `val` holds
the bytes stored in the input, and [`sax_parse`](../basic_json/sax_parse.md) passes them unchanged, without checking them
for valid UTF-8. BON8 encodes characters itself, so an ill-formed name is a parse error and `key` is not called for it.
## Examples
??? example
@@ -45,3 +62,7 @@ It is safe to move the passed object key value.
## Version history
- Added in version 3.2.0.
- The behavior described in the notes above (decoded names, validation before `key` is called, no normalization, `key`
before the `:` check, one call per duplicate, bytes of binary formats passed unchanged) is the same in all versions
from 3.2.0 on, and for the `key` event of a [parser callback](../basic_json/parser_callback_t.md) from 3.0.0 on. BSON
is only supported from version 3.4.0, and BON8 only from version 3.13.0.
@@ -113,6 +113,7 @@ When the macro is not defined, the library will define it to its default value.
- [:simple-cmake: JSON_Diagnostic_Positions](../../integration/cmake.md#json_diagnostic_positions) - CMake option to control the macro
- [`JSON_DIAGNOSTICS`](json_diagnostics.md) - macro to control extended diagnostics
- [`materialize()`](../basic_json_view/materialize.md) - values of a JSON view, with the same positions
- [Parsing and Exceptions](../../features/parsing/parse_exceptions.md) - handling parse errors and locating them in the input
## Version history
@@ -13,6 +13,80 @@ describe each of them.
<p class="nj-caption" markdown>The checks grouped by the stage at which they run, from push or pull request through the merge into develop to a release. The badge on each card says whether the check runs on every push or PR, on a schedule, or continuously.</p>
## How CI runs
Not every job runs for every change. The compiler jobs on Ubuntu, macOS, and Windows are split into three sets, so a
pull request gets fast feedback and the long jobs run when they matter:
| Set | When it runs | What it contains |
|-------------|-------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| PR set | every push to a pull request, and every push to `develop`, `master`, and `release/*` | GCC and Clang with all warnings, Clang-Tidy, sanitizers, IWYU, cpplint, the oldest and newest GCC and Clang, common CMake options, Meson, Emscripten, four Visual Studio 2022 and two Visual Studio 2026 configurations, MinGW, Clang on Windows, ARM64 Windows, five Xcode versions, Swift Package Manager; with the checks that always run, they would have caught at least 92 % of the failing commits of September and October 2026 |
| full set | after the PR set passed: on pull requests with the label :rocket: ready to merge, and on every push | every other compiler, version, C++ standard, and CMake option, the Clang Static Analyzer, Cppcheck, and the C++20 module tests |
| nightly set | every night on `develop`, on every push to `master` and `release/*`, and when started by hand | Infer, Valgrind, coverage, the Intel and NVIDIA compilers, CUDA, ARM64 Linux, and 64-bit MinGW; a failure opens an issue |
The documentation and license checks and the amalgamation check run on every push and pull request; AppVeyor (Visual
Studio 2015, 2017, and 2019) has its own schedule. A pull request that changes only the documentation, Markdown files, GitHub metadata, or license files
skips the compiler workflows (see [`tools/check_path_filters`](https://github.com/nlohmann/json/tree/develop/tools/check_path_filters)).
A maintainer adds the label :rocket: ready to merge when a pull request is ready; the full set then runs, and again on
every later push to that pull request. A pull request is merged only when the full set passed.
??? abstract "Jobs of the compiler workflows and their sets"
The table is generated by [`tools/ci_jobs`](https://github.com/nlohmann/json/tree/develop/tools/ci_jobs), which
also checks in CI that it is up to date and that every job belongs to exactly one set.
<!-- ci_jobs: begin -->
| Set | Workflow | Job | Configurations |
|-----|----------|-----|----------------|
| PR set | macos.yml | `macos-15` | 16.0; 16.4; 26.3 |
| PR set | macos.yml | `macos-26` | 26.6 |
| PR set | macos.yml | `xcode_standards` | 14 |
| PR set | macos.yml | `swiftpm` | - |
| PR set | ubuntu.yml | `ci_test_gcc` | - |
| PR set | ubuntu.yml | `ci_meson_install` | - |
| PR set | ubuntu.yml | `ci_static_analysis_ubuntu` | ci_cpplint |
| PR set | ubuntu.yml | `ci_static_analysis_clang` | ci_test_clang; ci_clang_tidy; ci_test_clang_sanitizer; ci_single_binaries |
| PR set | ubuntu.yml | `ci_cmake_options` | ci_test_diagnostics; ci_test_diagnostic_positions; ci_test_noexceptions; ci_test_noglobaludls |
| PR set | ubuntu.yml | `ci_test_compilers_gcc_old` | 4.8 |
| PR set | ubuntu.yml | `ci_test_compilers_clang` | 3.4 |
| PR set | ubuntu.yml | `ci_test_standards_gcc` | 26 |
| PR set | ubuntu.yml | `ci_test_standards_clang` | 26, libcxx |
| PR set | ubuntu.yml | `ci_emscripten` | - |
| PR set | windows.yml | `mingw` | x86 |
| PR set | windows.yml | `msvc` | Debug, x64, default; Release, x64, default; Debug, Win32, default; Release, Win32, latest |
| PR set | windows.yml | `msvc-vs2026` | Debug, x64, latest; Release, Win32, default |
| PR set | windows.yml | `msvc-arm64` | Debug |
| PR set | windows.yml | `clang` | 20.1.8 |
| PR set | windows.yml | `ci_module_cpp20` | - |
| full set | macos.yml | `macos-15_full` | 16.1; 16.2; 16.3; 26.0.1; 26.1.1; 26.2 |
| full set | macos.yml | `macos-26_full` | 26.4.1; 26.5 |
| full set | macos.yml | `xcode_standards_full` | 11; 17; 20; 23; 26 |
| full set | ubuntu.yml | `ci_static_analysis_ubuntu_full` | ci_test_single_header; ci_cppcheck; ci_reproducible_tests; ci_non_git_tests; ci_offline_testdata |
| full set | ubuntu.yml | `ci_static_analysis_clang_full` | ci_clang_analyze |
| full set | ubuntu.yml | `ci_cmake_options_full` | ci_cmake_flags; ci_test_noimplicitconversions; ci_test_legacycomparison; ci_test_disableenumserialization; ci_test_disabletuplereferenceconversion; ci_test_skiplibraryversioncheck; ci_test_simdutf; ci_test_strict_nul_handling; ci_test_delete_deprecated_functions; ci_test_no_thread_local |
| full set | ubuntu.yml | `ci_test_compilers_gcc_old_full` | 4.9; 5; 6 |
| full set | ubuntu.yml | `ci_test_compilers_gcc_full` | 7; 8; 9; 10; 11; 12; 13; 14; 15; 16; latest |
| full set | ubuntu.yml | `ci_test_compilers_clang_full` | 3.5; 3.6; 3.7; 3.8; 3.9; 4; 5; 6; 7; 8; 9; 10; 11; 12; 13; 14; 15-bullseye; 16; 17; 18; 19; 20; 21; 22; latest |
| full set | ubuntu.yml | `ci_test_standards_gcc_full` | 11; 14; 17; 20; 23 |
| full set | ubuntu.yml | `ci_test_standards_clang_full` | 11, libcxx; 11, libstdcxx; 14, libcxx; 14, libstdcxx; 17, libcxx; 17, libstdcxx; 20, libcxx; 20, libstdcxx; 23, libcxx; 23, libstdcxx; 26, libstdcxx |
| full set | ubuntu.yml | `ci_module_cpp20` | gcc:latest; silkeh/clang:latest |
| full set | windows.yml | `msvc_full` | Debug, Win32, latest; Debug, x64, latest; Release, Win32, default; Release, x64, latest |
| full set | windows.yml | `msvc-vs2026_full` | Debug, Win32, default; Debug, Win32, latest; Debug, x64, default; Release, Win32, latest; Release, x64, default; Release, x64, latest |
| full set | windows.yml | `msvc-arm64_full` | Release |
| full set | windows.yml | `clang_full` | 11.0.1; 12.0.1; 13.0.1; 14.0.6; 15.0.7; 16.0.6; 18.1.8; 19.1.7 |
| full set | windows.yml | `clang-cl_full` | Win32; x64 |
| nightly set | ubuntu.yml | `ci_infer` | - |
| nightly set | ubuntu.yml | `ci_static_analysis_ubuntu_nightly` | ci_test_valgrind |
| nightly set | ubuntu.yml | `ci_test_coverage` | - |
| nightly set | ubuntu.yml | `ci_cuda_example` | 11.8.0; 12.1.1; 12.6.3 |
| nightly set | ubuntu.yml | `ci_icpc` | - |
| nightly set | ubuntu.yml | `ci_icpx` | - |
| nightly set | ubuntu.yml | `ci_nvhpc` | - |
| nightly set | ubuntu.yml | `ci_test_arm64` | - |
| nightly set | windows.yml | `mingw_nightly` | x64 |
<!-- ci_jobs: end -->
## C++ language compliance and compiler compatibility
!!! success "Requirement: Compiler support"
+3 -2
View File
@@ -78,8 +78,9 @@ flowchart LR
- `NaN` and [discarded](../api/basic_json/is_discarded.md) values are not equal to themselves, so the invariant
cannot hold for them. `diff` emits a `replace` operation for such a value, and a `NaN` in a patch is serialized
as `#!json null`.
- Numbers that compare equal produce no operation even if their types differ: `diff(1, 1.0)`, `diff(1u, 1)`, and
`diff(0.0, -0.0)` return `#!json []`, so the patched value keeps the number type of *source*.
- Numbers that compare equal produce no operation even if their types differ or if one is zero and the other is
negative zero: `diff(1, 1.0)`, `diff(1u, 1)`, and `diff(0.0, -0.0)` return `#!json []`, so the patched value
keeps the number type and sign of *source*.
- A [binary value](binary_values.md) in a patch is serialized as an object
(`#!json {"bytes":[...],"subtype":null}`), so a patch that was dumped and parsed again no longer holds a binary
value.
+5 -3
View File
@@ -149,10 +149,12 @@ document: `#!cpp auto v = json_document::parse(text).root();` does not compile.
end must still fit.
- **A stream is always read to its end.** There is no partial/streaming read of an `#!cpp std::istream`: the stream
is read to its end into a buffer the document owns, and parsing starts after that.
- **No source positions on `materialize()`.** Even with
- **Fewer source positions after edits.** With
[`JSON_DIAGNOSTIC_POSITIONS`](../api/macros/json_diagnostic_positions.md) enabled,
[`materialize()`](../api/basic_json_view/materialize.md) does not set them. The index has the start offset of every
value, but not the end of arrays, objects, and escaped strings, which the positions also need.
[`materialize()`](../api/basic_json_view/materialize.md) sets the same positions as `#!cpp basic_json::parse()`.
In an [editable document](../api/json_editable_document.md), a value written by an edit has none, and neither has
an array or object whose elements changed or whose last element has none; a `basic_json` value keeps its positions
when it is changed. See [`materialize()`](../api/basic_json_view/materialize.md#notes).
- **Objects iterate in document order.** [`begin()`](../api/basic_json_view/begin.md)/
[`end()`](../api/basic_json_view/end.md) and [`items()`](../api/basic_json_view/items.md) visit an object's
members in the order they appear in the source text. `basic_json`'s default `object_t` is a `std::map`, which
@@ -64,6 +64,9 @@ bool parse_error(std::size_t position, const std::string& last_token, const json
The return value of each function determines whether parsing should proceed.
Object keys are passed to `key` decoded and validated: escape sequences are already resolved, so different spellings of
the same name yield the same string. See [`key`](../../api/json_sax/key.md#notes) for the exact guarantees.
## Stepping through the events
The events for a small input, step by step, and what two different handlers do with them. Press *Play* or step with *Next*:
@@ -130,6 +130,17 @@ parameters of [`parse()`](../../api/basic_json/parse.md) or [`accept()`](../../a
enable either for input whose conformance you cannot otherwise control, since interoperability with strictly
conforming JSON consumers is exactly what the default rejects.
## JSON Pointers and flattened keys
A [JSON Pointer](../json_pointer.md) can come from untrusted input as well, for instance as a path in a request or as a
key of a flattened object. The non-const [`operator[]`](../../api/basic_json/operator%5B%5D.md) with a JSON Pointer and
[`unflatten`](../../api/basic_json/unflatten.md) create missing values, and when an array index lies beyond the end of
an array, they fill the array with `#!json null` values up to that index. The memory used therefore grows with the
index, not with the size of the pointer: on a 64-bit platform, the index `10000000` allocates about 160 MB, the index
`100000000` about 1.6 GB. The library does not limit the index. For pointers from untrusted input, check the array
indices (or the pointer as a whole) before using them, or use [`at`](../../api/basic_json/at.md) or
[`contains`](../../api/basic_json/contains.md), which never add values.
## Checklist
- Wrap parsing in a `#!cpp try`/`#!cpp catch` block, or use `allow_exceptions=false`/`accept()` if your environment
@@ -145,6 +156,8 @@ conforming JSON consumers is exactly what the default rejects.
`JSON_STRICT_NUL_HANDLING`.
- Decide whether duplicate object keys should be an error for your application, and add a callback if so.
- Do not enable `ignore_comments` or `ignore_trailing_commas` for input that must be strictly conforming JSON.
- Check the array indices of untrusted JSON Pointers and flattened keys before passing them to the non-const
`operator[]` or `unflatten()`, which fill arrays with `#!json null` values up to the index.
For the broader design rationale and how it is tested (fuzzing, sanitizers, static analysis), see the
[assurance case](../../community/assurance_case.md) and [quality assurance](../../community/quality_assurance.md). To
+8 -2
View File
@@ -203,7 +203,7 @@ packet-beta
| 2-3 | `extra` | `uint16_t` | numbers: the number of integer digits (low byte) and fraction digits (high byte), 255 for more; objects: the number of their key hash index (1-based), or 0; otherwise 0 |
| 4-7 | `off` | `uint32_t` | where the value starts: the first byte after a string's opening quote (or its position in the buffer of decoded strings), the first byte of a number or literal, the bracket of an array or object |
| 8-11 | `len` | `uint32_t` | strings: the length after decoding; floats and literals: the length of the token; arrays and objects: the number of elements |
| 12-15 | `next` | `uint32_t` | arrays and objects: the number of nodes of the subtree, including the node itself |
| 12-15 | `next` | `uint32_t` | arrays and objects: the number of nodes of the subtree, including the node itself; strings with escapes: the position of the opening quote in the source plus one if the document was parsed with [`JSON_DIAGNOSTIC_POSITIONS`](../api/macros/json_diagnostic_positions.md), or 0 if it was not; all other values: 0 |
<p class="nj-table-caption" markdown>Fields of a node, by their byte offset within the 16-byte node. Integers use bytes 8-15 differently, see below.</p>
@@ -218,6 +218,12 @@ packet-beta
step from element to element this way and skip whole subtrees in constant time.
- **Offsets** are 32 bits wide, so a document is limited to 4294967279 bytes, 4 GiB minus 16 bytes (a margin below
2^32 for positions one scanner step past the end of the text; `out_of_range.416`).
- **Source positions** ([`JSON_DIAGNOSTIC_POSITIONS`](../api/macros/json_diagnostic_positions.md)) are derived by
[`materialize()`](../api/basic_json_view/materialize.md) and cost almost nothing while parsing: the only work is
storing the offset of the opening quote of a string with escapes in its `next`, which is otherwise unused. A
scalar's range follows from `off` and its length, a string with escapes is scanned from its opening quote (`next`) to the closing one,
and the end of an array or object is found by scanning from the end of its last element over whitespace, comments,
and a trailing comma. `materialize()` hands these positions to the SAX builder of `parse()` in place of its lexer.
- **Large objects** (128 members or more) get a hash index for their keys after parsing
([`detail/view/object_index.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/view/object_index.hpp)):
an open-addressing table whose slots hold the distance from the object's node to a key's node, so that a lookup does
@@ -277,7 +283,7 @@ Images ([`save`](../api/basic_json_document/save.md) and [`load`](../api/basic_j
the nodes as they are: a 64-byte header (the magic bytes `NJVI`, a format version, the sizes, and reserved bytes that
must be zero), the nodes, the text, and the decoded strings. An edited document is first written in document order, as
the parser would have written it (without links), and the numbers of the hash indexes are cleared, since `load`
rebuilds the indexes. So a change of the node layout is a change of the image format: it must raise `image_version`,
rebuilds the indexes. The flag *new* is kept, as these values have no position in the text. So a change of the node layout is a change of the image format: it must raise `image_version`,
and `load` then rejects images of other versions (`parse_error.116`) instead of misreading them.
## Input adapters
+2 -2
View File
@@ -22,8 +22,8 @@ The class contains a port of the shortest double-to-decimal conversion of [Żmij
The class contains a copy of [Hedley](https://nemequ.github.io/hedley/) from Evan Nemerson which is licensed as [CC0-1.0](https://creativecommons.org/publicdomain/zero/1.0/).
The class contains an adapted version of the Eisel-Lemire algorithm, its table of powers of five, and its digit comparison for long numbers from [fast_float](https://github.com/fastfloat/fast_float) by Daniel Lemire and contributors, which is available under the [MIT License](https://opensource.org/licenses/MIT) (used here), the Apache 2.0 License, and the Boost Software License. Copyright &copy; 2021 The fast_float authors
The class contains an adapted version of the Eisel-Lemire algorithm, its table of powers of five, its digit comparison for long numbers, and its parsing of eight digits at a time from [fast_float](https://github.com/fastfloat/fast_float) by Daniel Lemire and contributors, which is available under the [MIT License](https://opensource.org/licenses/MIT) (used here), the Apache 2.0 License, and the Boost Software License. Copyright &copy; 2021 The fast_float authors
The view's parser (`<nlohmann/json_view.hpp>`) contains techniques and code adapted from [yyjson](https://github.com/ibireme/yyjson) by YaoYuan, which is licensed under the [MIT License](https://opensource.org/licenses/MIT) (see above): table-driven decoding of `\u` escapes and fixed-offset unrolled checks.
The view's parser (`<nlohmann/json_view.hpp>`) validates non-ASCII strings with the vector UTF-8 check of [simdjson](https://github.com/simdjson/simdjson) by Daniel Lemire, Geoff Langdale, John Keiser, and contributors (its "lookup4" algorithm and tables, after J. Keiser and D. Lemire, "Validating UTF-8 In Less Than One Instruction Per Byte", 2021), which is available under the [MIT License](https://opensource.org/licenses/MIT) (used here) and the Apache 2.0 License. Copyright &copy; 2018-2025 The simdjson authors
The class's lexer and the view's parser (`<nlohmann/json_view.hpp>`) validate non-ASCII strings with the vector UTF-8 check of [simdjson](https://github.com/simdjson/simdjson) by Daniel Lemire, Geoff Langdale, John Keiser, and contributors (its "lookup4" algorithm and tables, after J. Keiser and D. Lemire, "Validating UTF-8 In Less Than One Instruction Per Byte", 2021), which is available under the [MIT License](https://opensource.org/licenses/MIT) (used here) and the Apache 2.0 License. Copyright &copy; 2018-2025 The simdjson authors
+174 -160
View File
@@ -1,26 +1,31 @@
<svg class="nj-fig nj-exceptions-qa nj-exceptions-qa--wide" viewBox="0 0 760 808" role="img" aria-labelledby="nj-qa-title-w nj-qa-desc-w" xmlns="http://www.w3.org/2000/svg">
<svg class="nj-fig nj-exceptions-qa nj-exceptions-qa--wide" viewBox="0 0 760 862" role="img" aria-labelledby="nj-qa-title-w nj-qa-desc-w" xmlns="http://www.w3.org/2000/svg">
<title id="nj-qa-title-w">Quality assurance pipeline</title>
<desc id="nj-qa-desc-w">Every push or pull request starts these checks in parallel: Build matrix (every push / PR): 50+ compilers: G&#8203;CC, Clang, M&#8203;SVC, AppleClang, Intel, NVIDIA, ... C++11, 14, 17, 20, 23, and 26 libc++ and libstdc++ warnings: Clang -Weverything, G&#8203;CC 300+ warnings; Unit tests (doctest) (every push / PR): every public A&#8203;PI function every exception id and message all Unicode code points single header and split headers custom number, string, array, and object types; Coverage (every push / PR): unit tests cover all lines of the code base; Static analysis (every push / PR, scheduled): Clang-Tidy, Cppcheck, Infer Clang Static Analyzer (52 checkers), I&#8203;WYU, Codacy CodeQL, Semgrep, Flawfinder (also scheduled); Dynamic analysis (every push / PR): runtime assertions Valgrind (Memcheck) address and undefined behavior sanitizers, integer overflow, nullability violations; Fuzzing (every PR, continuous): CIFuzz on every pull request OSS-Fuzz: billions of inputs every crash fixed with a reproducing unit test; Style (every push / PR): Artistic Style (astyle), enforced in the C&#8203;I cpplint with 61 rules; Docs and examples (every push / PR, scheduled): every documentation example is tested; changed output fails site built with warnings as errors; links checked regularly (scheduled); Integration, supply chain (every push / PR, scheduled): CMake 3.5, 3.31.6, and 4.0.0 Meson, Swift Package Manager dependency review on every PR Dependabot daily, OpenSSF Scorecard, R&#8203;EUSE. When they pass, the change is merged into develop; a release is made only when no vulnerability alert is open.</desc>
<desc id="nj-qa-desc-w">A push or pull request starts these checks; the badges say when each runs: Build matrix (every push / PR, before merge): 50+ compilers: G&#8203;CC, Clang, M&#8203;SVC, AppleClang, Intel, NVIDIA, ... C++11, 14, 17, 20, 23, and 26 libc++ and libstdc++ warnings: Clang -Weverything, G&#8203;CC 300+ warnings a subset on every push and PR, all before merge; Intel, NVIDIA, C&#8203;UDA, and ARM64 nightly; Unit tests (doctest) (every push / PR): every public A&#8203;PI function every exception id and message all Unicode code points single header and split headers custom number, string, array, and object types; Coverage (nightly): unit tests cover all lines of the code base; Static analysis (every push / PR, scheduled): Clang-Tidy, Cppcheck, Infer Clang Static Analyzer (52 checkers), I&#8203;WYU, Codacy CodeQL, Semgrep, Flawfinder (also scheduled); Infer nightly; Dynamic analysis (every push / PR, nightly): runtime assertions Valgrind (Memcheck), nightly address and undefined behavior sanitizers, integer overflow, nullability violations; Fuzzing (every PR, continuous): CIFuzz on every pull request OSS-Fuzz: billions of inputs every crash fixed with a reproducing unit test; Style (every push / PR): Artistic Style (astyle), enforced in the C&#8203;I cpplint with 61 rules; Docs and examples (every push / PR, scheduled): every documentation example is tested; changed output fails site built with warnings as errors; links checked regularly (scheduled); Integration, supply chain (every push / PR, scheduled): CMake 3.5, 3.31.6, and 4.0.0 Meson, Swift Package Manager dependency review on every PR Dependabot daily, OpenSSF Scorecard, R&#8203;EUSE. When they pass, the change is merged into develop; a release is made only when no vulnerability alert is open.</desc>
<defs><marker id="nj-qa-arrow-w" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto"><path class="fg-head" d="M0 1 L10 5 L0 9 z"/></marker></defs>
<rect class="nj-exceptions-groupbox" x="12" y="70" width="736" height="594" rx="12"/>
<rect class="nj-exceptions-groupbox" x="12" y="70" width="736" height="648" rx="12"/>
<rect class="nj-exceptions-stage" x="265" y="8" width="230" height="34" rx="17"/>
<text class="nj-exceptions-stagetext" x="380" y="30">push or pull request</text>
<line class="nj-exceptions-edge" x1="380" y1="42" x2="380" y2="68" marker-end="url(#nj-qa-arrow-w)"/>
<text class="nj-exceptions-group" x="26" y="94">checks, run in parallel on GitHub Actions and AppVeyor</text>
<g class="nj-exceptions-card">
<rect class="nj-exceptions-box" x="26" y="110" width="228" height="184" rx="8"/>
<rect class="nj-exceptions-box" x="26" y="110" width="228" height="238" rx="8"/>
<text class="nj-exceptions-ctitle" x="40" y="135">Build matrix</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--push" x="40" y="146" width="110.5" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--push" x="95.25" y="160">every push / PR</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--push" x="156.5" y="146" width="91.6" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--push" x="202.3" y="160">before merge</text>
<text class="nj-exceptions-line" x="40" y="188">50+ compilers: G&#8203;CC, Clang, M&#8203;SVC,</text>
<text class="nj-exceptions-line" x="40" y="206">AppleClang, Intel, NVIDIA, ...</text>
<text class="nj-exceptions-line" x="40" y="224">C++11, 14, 17, 20, 23, and 26</text>
<text class="nj-exceptions-line" x="40" y="242">libc++ and libstdc++</text>
<text class="nj-exceptions-line" x="40" y="260">warnings: Clang -Weverything,</text>
<text class="nj-exceptions-line" x="40" y="278">G&#8203;CC 300+ warnings</text>
<text class="nj-exceptions-line" x="40" y="296">a subset on every push and PR,</text>
<text class="nj-exceptions-line" x="40" y="314">all before merge; Intel, NVIDIA,</text>
<text class="nj-exceptions-line" x="40" y="332">C&#8203;UDA, and ARM64 nightly</text>
</g>
<g class="nj-exceptions-card">
<rect class="nj-exceptions-box" x="266" y="110" width="228" height="184" rx="8"/>
<rect class="nj-exceptions-box" x="266" y="110" width="228" height="238" rx="8"/>
<text class="nj-exceptions-ctitle" x="280" y="135">Unit tests (doctest)</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--push" x="280" y="146" width="110.5" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--push" x="335.25" y="160">every push / PR</text>
@@ -32,209 +37,218 @@
<text class="nj-exceptions-line" x="280" y="278">and object types</text>
</g>
<g class="nj-exceptions-card">
<rect class="nj-exceptions-box" x="506" y="110" width="228" height="184" rx="8"/>
<rect class="nj-exceptions-box" x="506" y="110" width="228" height="238" rx="8"/>
<text class="nj-exceptions-ctitle" x="520" y="135">Coverage</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--push" x="520" y="146" width="110.5" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--push" x="575.25" y="160">every push / PR</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--sched" x="520" y="146" width="60.1" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--sched" x="550.05" y="160">nightly</text>
<text class="nj-exceptions-line" x="520" y="188">unit tests cover all lines</text>
<text class="nj-exceptions-line" x="520" y="206">of the code base</text>
</g>
<g class="nj-exceptions-card">
<rect class="nj-exceptions-box" x="26" y="306" width="228" height="166" rx="8"/>
<text class="nj-exceptions-ctitle" x="40" y="331">Static analysis</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--push" x="40" y="342" width="110.5" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--push" x="95.25" y="356">every push / PR</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--sched" x="156.5" y="342" width="72.7" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--sched" x="192.85" y="356">scheduled</text>
<text class="nj-exceptions-line" x="40" y="384">Clang-Tidy, Cppcheck, Infer</text>
<text class="nj-exceptions-line" x="40" y="402">Clang Static Analyzer (52</text>
<text class="nj-exceptions-line" x="40" y="420">checkers), I&#8203;WYU, Codacy</text>
<text class="nj-exceptions-line" x="40" y="438">CodeQL, Semgrep, Flawfinder</text>
<text class="nj-exceptions-line" x="40" y="456">(also scheduled)</text>
<rect class="nj-exceptions-box" x="26" y="360" width="228" height="166" rx="8"/>
<text class="nj-exceptions-ctitle" x="40" y="385">Static analysis</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--push" x="40" y="396" width="110.5" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--push" x="95.25" y="410">every push / PR</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--sched" x="156.5" y="396" width="72.7" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--sched" x="192.85" y="410">scheduled</text>
<text class="nj-exceptions-line" x="40" y="438">Clang-Tidy, Cppcheck, Infer</text>
<text class="nj-exceptions-line" x="40" y="456">Clang Static Analyzer (52</text>
<text class="nj-exceptions-line" x="40" y="474">checkers), I&#8203;WYU, Codacy</text>
<text class="nj-exceptions-line" x="40" y="492">CodeQL, Semgrep, Flawfinder</text>
<text class="nj-exceptions-line" x="40" y="510">(also scheduled); Infer nightly</text>
</g>
<g class="nj-exceptions-card">
<rect class="nj-exceptions-box" x="266" y="306" width="228" height="166" rx="8"/>
<text class="nj-exceptions-ctitle" x="280" y="331">Dynamic analysis</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--push" x="280" y="342" width="110.5" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--push" x="335.25" y="356">every push / PR</text>
<text class="nj-exceptions-line" x="280" y="384">runtime assertions</text>
<text class="nj-exceptions-line" x="280" y="402">Valgrind (Memcheck)</text>
<text class="nj-exceptions-line" x="280" y="420">address and undefined behavior</text>
<text class="nj-exceptions-line" x="280" y="438">sanitizers, integer overflow,</text>
<text class="nj-exceptions-line" x="280" y="456">nullability violations</text>
<rect class="nj-exceptions-box" x="266" y="360" width="228" height="166" rx="8"/>
<text class="nj-exceptions-ctitle" x="280" y="385">Dynamic analysis</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--push" x="280" y="396" width="110.5" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--push" x="335.25" y="410">every push / PR</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--sched" x="396.5" y="396" width="60.1" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--sched" x="426.55" y="410">nightly</text>
<text class="nj-exceptions-line" x="280" y="438">runtime assertions</text>
<text class="nj-exceptions-line" x="280" y="456">Valgrind (Memcheck), nightly</text>
<text class="nj-exceptions-line" x="280" y="474">address and undefined behavior</text>
<text class="nj-exceptions-line" x="280" y="492">sanitizers, integer overflow,</text>
<text class="nj-exceptions-line" x="280" y="510">nullability violations</text>
</g>
<g class="nj-exceptions-card">
<rect class="nj-exceptions-box" x="506" y="306" width="228" height="166" rx="8"/>
<text class="nj-exceptions-ctitle" x="520" y="331">Fuzzing</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--push" x="520" y="342" width="66.4" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--push" x="553.2" y="356">every PR</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--cont" x="592.4" y="342" width="79" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--cont" x="631.9" y="356">continuous</text>
<text class="nj-exceptions-line" x="520" y="384">CIFuzz on every pull request</text>
<text class="nj-exceptions-line" x="520" y="402">OSS-Fuzz: billions of inputs</text>
<text class="nj-exceptions-line" x="520" y="420">every crash fixed with a</text>
<text class="nj-exceptions-line" x="520" y="438">reproducing unit test</text>
<rect class="nj-exceptions-box" x="506" y="360" width="228" height="166" rx="8"/>
<text class="nj-exceptions-ctitle" x="520" y="385">Fuzzing</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--push" x="520" y="396" width="66.4" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--push" x="553.2" y="410">every PR</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--cont" x="592.4" y="396" width="79" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--cont" x="631.9" y="410">continuous</text>
<text class="nj-exceptions-line" x="520" y="438">CIFuzz on every pull request</text>
<text class="nj-exceptions-line" x="520" y="456">OSS-Fuzz: billions of inputs</text>
<text class="nj-exceptions-line" x="520" y="474">every crash fixed with a</text>
<text class="nj-exceptions-line" x="520" y="492">reproducing unit test</text>
</g>
<g class="nj-exceptions-card">
<rect class="nj-exceptions-box" x="26" y="484" width="228" height="166" rx="8"/>
<text class="nj-exceptions-ctitle" x="40" y="509">Style</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--push" x="40" y="520" width="110.5" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--push" x="95.25" y="534">every push / PR</text>
<text class="nj-exceptions-line" x="40" y="562">Artistic Style (astyle),</text>
<text class="nj-exceptions-line" x="40" y="580">enforced in the C&#8203;I</text>
<text class="nj-exceptions-line" x="40" y="598">cpplint with 61 rules</text>
<rect class="nj-exceptions-box" x="26" y="538" width="228" height="166" rx="8"/>
<text class="nj-exceptions-ctitle" x="40" y="563">Style</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--push" x="40" y="574" width="110.5" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--push" x="95.25" y="588">every push / PR</text>
<text class="nj-exceptions-line" x="40" y="616">Artistic Style (astyle),</text>
<text class="nj-exceptions-line" x="40" y="634">enforced in the C&#8203;I</text>
<text class="nj-exceptions-line" x="40" y="652">cpplint with 61 rules</text>
</g>
<g class="nj-exceptions-card">
<rect class="nj-exceptions-box" x="266" y="484" width="228" height="166" rx="8"/>
<text class="nj-exceptions-ctitle" x="280" y="509">Docs and examples</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--push" x="280" y="520" width="110.5" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--push" x="335.25" y="534">every push / PR</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--sched" x="396.5" y="520" width="72.7" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--sched" x="432.85" y="534">scheduled</text>
<text class="nj-exceptions-line" x="280" y="562">every documentation example is</text>
<text class="nj-exceptions-line" x="280" y="580">tested; changed output fails</text>
<text class="nj-exceptions-line" x="280" y="598">site built with warnings</text>
<text class="nj-exceptions-line" x="280" y="616">as errors; links checked</text>
<text class="nj-exceptions-line" x="280" y="634">regularly (scheduled)</text>
<rect class="nj-exceptions-box" x="266" y="538" width="228" height="166" rx="8"/>
<text class="nj-exceptions-ctitle" x="280" y="563">Docs and examples</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--push" x="280" y="574" width="110.5" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--push" x="335.25" y="588">every push / PR</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--sched" x="396.5" y="574" width="72.7" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--sched" x="432.85" y="588">scheduled</text>
<text class="nj-exceptions-line" x="280" y="616">every documentation example is</text>
<text class="nj-exceptions-line" x="280" y="634">tested; changed output fails</text>
<text class="nj-exceptions-line" x="280" y="652">site built with warnings</text>
<text class="nj-exceptions-line" x="280" y="670">as errors; links checked</text>
<text class="nj-exceptions-line" x="280" y="688">regularly (scheduled)</text>
</g>
<g class="nj-exceptions-card">
<rect class="nj-exceptions-box" x="506" y="484" width="228" height="166" rx="8"/>
<text class="nj-exceptions-ctitle" x="520" y="509">Integration, supply chain</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--push" x="520" y="520" width="110.5" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--push" x="575.25" y="534">every push / PR</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--sched" x="636.5" y="520" width="72.7" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--sched" x="672.85" y="534">scheduled</text>
<text class="nj-exceptions-line" x="520" y="562">CMake 3.5, 3.31.6, and 4.0.0</text>
<text class="nj-exceptions-line" x="520" y="580">Meson, Swift Package Manager</text>
<text class="nj-exceptions-line" x="520" y="598">dependency review on every PR</text>
<text class="nj-exceptions-line" x="520" y="616">Dependabot daily, OpenSSF</text>
<text class="nj-exceptions-line" x="520" y="634">Scorecard, R&#8203;EUSE</text>
<rect class="nj-exceptions-box" x="506" y="538" width="228" height="166" rx="8"/>
<text class="nj-exceptions-ctitle" x="520" y="563">Integration, supply chain</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--push" x="520" y="574" width="110.5" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--push" x="575.25" y="588">every push / PR</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--sched" x="636.5" y="574" width="72.7" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--sched" x="672.85" y="588">scheduled</text>
<text class="nj-exceptions-line" x="520" y="616">CMake 3.5, 3.31.6, and 4.0.0</text>
<text class="nj-exceptions-line" x="520" y="634">Meson, Swift Package Manager</text>
<text class="nj-exceptions-line" x="520" y="652">dependency review on every PR</text>
<text class="nj-exceptions-line" x="520" y="670">Dependabot daily, OpenSSF</text>
<text class="nj-exceptions-line" x="520" y="688">Scorecard, R&#8203;EUSE</text>
</g>
<line class="nj-exceptions-edge" x1="380" y1="664" x2="380" y2="690" marker-end="url(#nj-qa-arrow-w)"/>
<rect class="nj-exceptions-stage nj-exceptions-stage--merge" x="265" y="692" width="230" height="34" rx="17"/>
<text class="nj-exceptions-stagetext" x="380" y="714">merge into develop</text>
<line class="nj-exceptions-edge" x1="380" y1="726" x2="380" y2="752" marker-end="url(#nj-qa-arrow-w)"/>
<rect class="nj-exceptions-stage nj-exceptions-stage--merge" x="230" y="754" width="300" height="46" rx="23"/>
<text class="nj-exceptions-stagetext" x="380" y="776">release</text>
<text class="nj-exceptions-stagesub" x="380" y="792">no open vulnerability alert</text>
<line class="nj-exceptions-edge" x1="380" y1="718" x2="380" y2="744" marker-end="url(#nj-qa-arrow-w)"/>
<rect class="nj-exceptions-stage nj-exceptions-stage--merge" x="265" y="746" width="230" height="34" rx="17"/>
<text class="nj-exceptions-stagetext" x="380" y="768">merge into develop</text>
<line class="nj-exceptions-edge" x1="380" y1="780" x2="380" y2="806" marker-end="url(#nj-qa-arrow-w)"/>
<rect class="nj-exceptions-stage nj-exceptions-stage--merge" x="230" y="808" width="300" height="46" rx="23"/>
<text class="nj-exceptions-stagetext" x="380" y="830">release</text>
<text class="nj-exceptions-stagesub" x="380" y="846">no open vulnerability alert</text>
</svg>
<svg class="nj-fig nj-exceptions-qa nj-exceptions-qa--narrow" viewBox="0 0 360 1786" role="img" aria-labelledby="nj-qa-title-n nj-qa-desc-n" xmlns="http://www.w3.org/2000/svg">
<svg class="nj-fig nj-exceptions-qa nj-exceptions-qa--narrow" viewBox="0 0 360 1840" role="img" aria-labelledby="nj-qa-title-n nj-qa-desc-n" xmlns="http://www.w3.org/2000/svg">
<title id="nj-qa-title-n">Quality assurance pipeline</title>
<desc id="nj-qa-desc-n">Every push or pull request starts these checks in parallel: Build matrix (every push / PR): 50+ compilers: G&#8203;CC, Clang, M&#8203;SVC, AppleClang, Intel, NVIDIA, ... C++11, 14, 17, 20, 23, and 26 libc++ and libstdc++ warnings: Clang -Weverything, G&#8203;CC 300+ warnings; Unit tests (doctest) (every push / PR): every public A&#8203;PI function every exception id and message all Unicode code points single header and split headers custom number, string, array, and object types; Coverage (every push / PR): unit tests cover all lines of the code base; Static analysis (every push / PR, scheduled): Clang-Tidy, Cppcheck, Infer Clang Static Analyzer (52 checkers), I&#8203;WYU, Codacy CodeQL, Semgrep, Flawfinder (also scheduled); Dynamic analysis (every push / PR): runtime assertions Valgrind (Memcheck) address and undefined behavior sanitizers, integer overflow, nullability violations; Fuzzing (every PR, continuous): CIFuzz on every pull request OSS-Fuzz: billions of inputs every crash fixed with a reproducing unit test; Style (every push / PR): Artistic Style (astyle), enforced in the C&#8203;I cpplint with 61 rules; Docs and examples (every push / PR, scheduled): every documentation example is tested; changed output fails site built with warnings as errors; links checked regularly (scheduled); Integration, supply chain (every push / PR, scheduled): CMake 3.5, 3.31.6, and 4.0.0 Meson, Swift Package Manager dependency review on every PR Dependabot daily, OpenSSF Scorecard, R&#8203;EUSE. When they pass, the change is merged into develop; a release is made only when no vulnerability alert is open.</desc>
<desc id="nj-qa-desc-n">A push or pull request starts these checks; the badges say when each runs: Build matrix (every push / PR, before merge): 50+ compilers: G&#8203;CC, Clang, M&#8203;SVC, AppleClang, Intel, NVIDIA, ... C++11, 14, 17, 20, 23, and 26 libc++ and libstdc++ warnings: Clang -Weverything, G&#8203;CC 300+ warnings a subset on every push and PR, all before merge; Intel, NVIDIA, C&#8203;UDA, and ARM64 nightly; Unit tests (doctest) (every push / PR): every public A&#8203;PI function every exception id and message all Unicode code points single header and split headers custom number, string, array, and object types; Coverage (nightly): unit tests cover all lines of the code base; Static analysis (every push / PR, scheduled): Clang-Tidy, Cppcheck, Infer Clang Static Analyzer (52 checkers), I&#8203;WYU, Codacy CodeQL, Semgrep, Flawfinder (also scheduled); Infer nightly; Dynamic analysis (every push / PR, nightly): runtime assertions Valgrind (Memcheck), nightly address and undefined behavior sanitizers, integer overflow, nullability violations; Fuzzing (every PR, continuous): CIFuzz on every pull request OSS-Fuzz: billions of inputs every crash fixed with a reproducing unit test; Style (every push / PR): Artistic Style (astyle), enforced in the C&#8203;I cpplint with 61 rules; Docs and examples (every push / PR, scheduled): every documentation example is tested; changed output fails site built with warnings as errors; links checked regularly (scheduled); Integration, supply chain (every push / PR, scheduled): CMake 3.5, 3.31.6, and 4.0.0 Meson, Swift Package Manager dependency review on every PR Dependabot daily, OpenSSF Scorecard, R&#8203;EUSE. When they pass, the change is merged into develop; a release is made only when no vulnerability alert is open.</desc>
<defs><marker id="nj-qa-arrow-n" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto"><path class="fg-head" d="M0 1 L10 5 L0 9 z"/></marker></defs>
<rect class="nj-exceptions-groupbox" x="12" y="70" width="336" height="1572" rx="12"/>
<rect class="nj-exceptions-groupbox" x="12" y="70" width="336" height="1626" rx="12"/>
<rect class="nj-exceptions-stage" x="65" y="8" width="230" height="34" rx="17"/>
<text class="nj-exceptions-stagetext" x="180" y="30">push or pull request</text>
<line class="nj-exceptions-edge" x1="180" y1="42" x2="180" y2="68" marker-end="url(#nj-qa-arrow-n)"/>
<text class="nj-exceptions-group" x="26" y="94">checks, run in parallel</text>
<g class="nj-exceptions-card">
<rect class="nj-exceptions-box" x="26" y="110" width="308" height="184" rx="8"/>
<rect class="nj-exceptions-box" x="26" y="110" width="308" height="238" rx="8"/>
<text class="nj-exceptions-ctitle" x="40" y="135">Build matrix</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--push" x="40" y="146" width="110.5" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--push" x="95.25" y="160">every push / PR</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--push" x="156.5" y="146" width="91.6" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--push" x="202.3" y="160">before merge</text>
<text class="nj-exceptions-line" x="40" y="188">50+ compilers: G&#8203;CC, Clang, M&#8203;SVC,</text>
<text class="nj-exceptions-line" x="40" y="206">AppleClang, Intel, NVIDIA, ...</text>
<text class="nj-exceptions-line" x="40" y="224">C++11, 14, 17, 20, 23, and 26</text>
<text class="nj-exceptions-line" x="40" y="242">libc++ and libstdc++</text>
<text class="nj-exceptions-line" x="40" y="260">warnings: Clang -Weverything,</text>
<text class="nj-exceptions-line" x="40" y="278">G&#8203;CC 300+ warnings</text>
<text class="nj-exceptions-line" x="40" y="296">a subset on every push and PR,</text>
<text class="nj-exceptions-line" x="40" y="314">all before merge; Intel, NVIDIA,</text>
<text class="nj-exceptions-line" x="40" y="332">C&#8203;UDA, and ARM64 nightly</text>
</g>
<g class="nj-exceptions-card">
<rect class="nj-exceptions-box" x="26" y="306" width="308" height="184" rx="8"/>
<text class="nj-exceptions-ctitle" x="40" y="331">Unit tests (doctest)</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--push" x="40" y="342" width="110.5" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--push" x="95.25" y="356">every push / PR</text>
<text class="nj-exceptions-line" x="40" y="384">every public A&#8203;PI function</text>
<text class="nj-exceptions-line" x="40" y="402">every exception id and message</text>
<text class="nj-exceptions-line" x="40" y="420">all Unicode code points</text>
<text class="nj-exceptions-line" x="40" y="438">single header and split headers</text>
<text class="nj-exceptions-line" x="40" y="456">custom number, string, array,</text>
<text class="nj-exceptions-line" x="40" y="474">and object types</text>
<rect class="nj-exceptions-box" x="26" y="360" width="308" height="184" rx="8"/>
<text class="nj-exceptions-ctitle" x="40" y="385">Unit tests (doctest)</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--push" x="40" y="396" width="110.5" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--push" x="95.25" y="410">every push / PR</text>
<text class="nj-exceptions-line" x="40" y="438">every public A&#8203;PI function</text>
<text class="nj-exceptions-line" x="40" y="456">every exception id and message</text>
<text class="nj-exceptions-line" x="40" y="474">all Unicode code points</text>
<text class="nj-exceptions-line" x="40" y="492">single header and split headers</text>
<text class="nj-exceptions-line" x="40" y="510">custom number, string, array,</text>
<text class="nj-exceptions-line" x="40" y="528">and object types</text>
</g>
<g class="nj-exceptions-card">
<rect class="nj-exceptions-box" x="26" y="502" width="308" height="112" rx="8"/>
<text class="nj-exceptions-ctitle" x="40" y="527">Coverage</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--push" x="40" y="538" width="110.5" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--push" x="95.25" y="552">every push / PR</text>
<text class="nj-exceptions-line" x="40" y="580">unit tests cover all lines</text>
<text class="nj-exceptions-line" x="40" y="598">of the code base</text>
<rect class="nj-exceptions-box" x="26" y="556" width="308" height="112" rx="8"/>
<text class="nj-exceptions-ctitle" x="40" y="581">Coverage</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--sched" x="40" y="592" width="60.1" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--sched" x="70.05" y="606">nightly</text>
<text class="nj-exceptions-line" x="40" y="634">unit tests cover all lines</text>
<text class="nj-exceptions-line" x="40" y="652">of the code base</text>
</g>
<g class="nj-exceptions-card">
<rect class="nj-exceptions-box" x="26" y="626" width="308" height="166" rx="8"/>
<text class="nj-exceptions-ctitle" x="40" y="651">Static analysis</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--push" x="40" y="662" width="110.5" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--push" x="95.25" y="676">every push / PR</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--sched" x="156.5" y="662" width="72.7" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--sched" x="192.85" y="676">scheduled</text>
<text class="nj-exceptions-line" x="40" y="704">Clang-Tidy, Cppcheck, Infer</text>
<text class="nj-exceptions-line" x="40" y="722">Clang Static Analyzer (52</text>
<text class="nj-exceptions-line" x="40" y="740">checkers), I&#8203;WYU, Codacy</text>
<text class="nj-exceptions-line" x="40" y="758">CodeQL, Semgrep, Flawfinder</text>
<text class="nj-exceptions-line" x="40" y="776">(also scheduled)</text>
<rect class="nj-exceptions-box" x="26" y="680" width="308" height="166" rx="8"/>
<text class="nj-exceptions-ctitle" x="40" y="705">Static analysis</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--push" x="40" y="716" width="110.5" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--push" x="95.25" y="730">every push / PR</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--sched" x="156.5" y="716" width="72.7" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--sched" x="192.85" y="730">scheduled</text>
<text class="nj-exceptions-line" x="40" y="758">Clang-Tidy, Cppcheck, Infer</text>
<text class="nj-exceptions-line" x="40" y="776">Clang Static Analyzer (52</text>
<text class="nj-exceptions-line" x="40" y="794">checkers), I&#8203;WYU, Codacy</text>
<text class="nj-exceptions-line" x="40" y="812">CodeQL, Semgrep, Flawfinder</text>
<text class="nj-exceptions-line" x="40" y="830">(also scheduled); Infer nightly</text>
</g>
<g class="nj-exceptions-card">
<rect class="nj-exceptions-box" x="26" y="804" width="308" height="166" rx="8"/>
<text class="nj-exceptions-ctitle" x="40" y="829">Dynamic analysis</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--push" x="40" y="840" width="110.5" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--push" x="95.25" y="854">every push / PR</text>
<text class="nj-exceptions-line" x="40" y="882">runtime assertions</text>
<text class="nj-exceptions-line" x="40" y="900">Valgrind (Memcheck)</text>
<text class="nj-exceptions-line" x="40" y="918">address and undefined behavior</text>
<text class="nj-exceptions-line" x="40" y="936">sanitizers, integer overflow,</text>
<text class="nj-exceptions-line" x="40" y="954">nullability violations</text>
<rect class="nj-exceptions-box" x="26" y="858" width="308" height="166" rx="8"/>
<text class="nj-exceptions-ctitle" x="40" y="883">Dynamic analysis</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--push" x="40" y="894" width="110.5" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--push" x="95.25" y="908">every push / PR</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--sched" x="156.5" y="894" width="60.1" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--sched" x="186.55" y="908">nightly</text>
<text class="nj-exceptions-line" x="40" y="936">runtime assertions</text>
<text class="nj-exceptions-line" x="40" y="954">Valgrind (Memcheck), nightly</text>
<text class="nj-exceptions-line" x="40" y="972">address and undefined behavior</text>
<text class="nj-exceptions-line" x="40" y="990">sanitizers, integer overflow,</text>
<text class="nj-exceptions-line" x="40" y="1008">nullability violations</text>
</g>
<g class="nj-exceptions-card">
<rect class="nj-exceptions-box" x="26" y="982" width="308" height="148" rx="8"/>
<text class="nj-exceptions-ctitle" x="40" y="1007">Fuzzing</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--push" x="40" y="1018" width="66.4" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--push" x="73.2" y="1032">every PR</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--cont" x="112.4" y="1018" width="79" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--cont" x="151.9" y="1032">continuous</text>
<text class="nj-exceptions-line" x="40" y="1060">CIFuzz on every pull request</text>
<text class="nj-exceptions-line" x="40" y="1078">OSS-Fuzz: billions of inputs</text>
<text class="nj-exceptions-line" x="40" y="1096">every crash fixed with a</text>
<text class="nj-exceptions-line" x="40" y="1114">reproducing unit test</text>
<rect class="nj-exceptions-box" x="26" y="1036" width="308" height="148" rx="8"/>
<text class="nj-exceptions-ctitle" x="40" y="1061">Fuzzing</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--push" x="40" y="1072" width="66.4" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--push" x="73.2" y="1086">every PR</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--cont" x="112.4" y="1072" width="79" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--cont" x="151.9" y="1086">continuous</text>
<text class="nj-exceptions-line" x="40" y="1114">CIFuzz on every pull request</text>
<text class="nj-exceptions-line" x="40" y="1132">OSS-Fuzz: billions of inputs</text>
<text class="nj-exceptions-line" x="40" y="1150">every crash fixed with a</text>
<text class="nj-exceptions-line" x="40" y="1168">reproducing unit test</text>
</g>
<g class="nj-exceptions-card">
<rect class="nj-exceptions-box" x="26" y="1142" width="308" height="130" rx="8"/>
<text class="nj-exceptions-ctitle" x="40" y="1167">Style</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--push" x="40" y="1178" width="110.5" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--push" x="95.25" y="1192">every push / PR</text>
<text class="nj-exceptions-line" x="40" y="1220">Artistic Style (astyle),</text>
<text class="nj-exceptions-line" x="40" y="1238">enforced in the C&#8203;I</text>
<text class="nj-exceptions-line" x="40" y="1256">cpplint with 61 rules</text>
<rect class="nj-exceptions-box" x="26" y="1196" width="308" height="130" rx="8"/>
<text class="nj-exceptions-ctitle" x="40" y="1221">Style</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--push" x="40" y="1232" width="110.5" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--push" x="95.25" y="1246">every push / PR</text>
<text class="nj-exceptions-line" x="40" y="1274">Artistic Style (astyle),</text>
<text class="nj-exceptions-line" x="40" y="1292">enforced in the C&#8203;I</text>
<text class="nj-exceptions-line" x="40" y="1310">cpplint with 61 rules</text>
</g>
<g class="nj-exceptions-card">
<rect class="nj-exceptions-box" x="26" y="1284" width="308" height="166" rx="8"/>
<text class="nj-exceptions-ctitle" x="40" y="1309">Docs and examples</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--push" x="40" y="1320" width="110.5" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--push" x="95.25" y="1334">every push / PR</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--sched" x="156.5" y="1320" width="72.7" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--sched" x="192.85" y="1334">scheduled</text>
<text class="nj-exceptions-line" x="40" y="1362">every documentation example is</text>
<text class="nj-exceptions-line" x="40" y="1380">tested; changed output fails</text>
<text class="nj-exceptions-line" x="40" y="1398">site built with warnings</text>
<text class="nj-exceptions-line" x="40" y="1416">as errors; links checked</text>
<text class="nj-exceptions-line" x="40" y="1434">regularly (scheduled)</text>
<rect class="nj-exceptions-box" x="26" y="1338" width="308" height="166" rx="8"/>
<text class="nj-exceptions-ctitle" x="40" y="1363">Docs and examples</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--push" x="40" y="1374" width="110.5" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--push" x="95.25" y="1388">every push / PR</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--sched" x="156.5" y="1374" width="72.7" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--sched" x="192.85" y="1388">scheduled</text>
<text class="nj-exceptions-line" x="40" y="1416">every documentation example is</text>
<text class="nj-exceptions-line" x="40" y="1434">tested; changed output fails</text>
<text class="nj-exceptions-line" x="40" y="1452">site built with warnings</text>
<text class="nj-exceptions-line" x="40" y="1470">as errors; links checked</text>
<text class="nj-exceptions-line" x="40" y="1488">regularly (scheduled)</text>
</g>
<g class="nj-exceptions-card">
<rect class="nj-exceptions-box" x="26" y="1462" width="308" height="166" rx="8"/>
<text class="nj-exceptions-ctitle" x="40" y="1487">Integration, supply chain</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--push" x="40" y="1498" width="110.5" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--push" x="95.25" y="1512">every push / PR</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--sched" x="156.5" y="1498" width="72.7" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--sched" x="192.85" y="1512">scheduled</text>
<text class="nj-exceptions-line" x="40" y="1540">CMake 3.5, 3.31.6, and 4.0.0</text>
<text class="nj-exceptions-line" x="40" y="1558">Meson, Swift Package Manager</text>
<text class="nj-exceptions-line" x="40" y="1576">dependency review on every PR</text>
<text class="nj-exceptions-line" x="40" y="1594">Dependabot daily, OpenSSF</text>
<text class="nj-exceptions-line" x="40" y="1612">Scorecard, R&#8203;EUSE</text>
<rect class="nj-exceptions-box" x="26" y="1516" width="308" height="166" rx="8"/>
<text class="nj-exceptions-ctitle" x="40" y="1541">Integration, supply chain</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--push" x="40" y="1552" width="110.5" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--push" x="95.25" y="1566">every push / PR</text>
<rect class="nj-exceptions-badge nj-exceptions-badge--sched" x="156.5" y="1552" width="72.7" height="20" rx="10"/>
<text class="nj-exceptions-badgetext nj-exceptions-badgetext--sched" x="192.85" y="1566">scheduled</text>
<text class="nj-exceptions-line" x="40" y="1594">CMake 3.5, 3.31.6, and 4.0.0</text>
<text class="nj-exceptions-line" x="40" y="1612">Meson, Swift Package Manager</text>
<text class="nj-exceptions-line" x="40" y="1630">dependency review on every PR</text>
<text class="nj-exceptions-line" x="40" y="1648">Dependabot daily, OpenSSF</text>
<text class="nj-exceptions-line" x="40" y="1666">Scorecard, R&#8203;EUSE</text>
</g>
<line class="nj-exceptions-edge" x1="180" y1="1642" x2="180" y2="1668" marker-end="url(#nj-qa-arrow-n)"/>
<rect class="nj-exceptions-stage nj-exceptions-stage--merge" x="65" y="1670" width="230" height="34" rx="17"/>
<text class="nj-exceptions-stagetext" x="180" y="1692">merge into develop</text>
<line class="nj-exceptions-edge" x1="180" y1="1704" x2="180" y2="1730" marker-end="url(#nj-qa-arrow-n)"/>
<rect class="nj-exceptions-stage nj-exceptions-stage--merge" x="30" y="1732" width="300" height="46" rx="23"/>
<text class="nj-exceptions-stagetext" x="180" y="1754">release</text>
<text class="nj-exceptions-stagesub" x="180" y="1770">no open vulnerability alert</text>
<line class="nj-exceptions-edge" x1="180" y1="1696" x2="180" y2="1722" marker-end="url(#nj-qa-arrow-n)"/>
<rect class="nj-exceptions-stage nj-exceptions-stage--merge" x="65" y="1724" width="230" height="34" rx="17"/>
<text class="nj-exceptions-stagetext" x="180" y="1746">merge into develop</text>
<line class="nj-exceptions-edge" x1="180" y1="1758" x2="180" y2="1784" marker-end="url(#nj-qa-arrow-n)"/>
<rect class="nj-exceptions-stage nj-exceptions-stage--merge" x="30" y="1786" width="300" height="46" rx="23"/>
<text class="nj-exceptions-stagetext" x="180" y="1808">release</text>
<text class="nj-exceptions-stagesub" x="180" y="1824">no open vulnerability alert</text>
</svg>

Before

Width:  |  Height:  |  Size: 22 KiB

After

Width:  |  Height:  |  Size: 23 KiB

+11 -6
View File
@@ -22,19 +22,24 @@ P = "nj-exceptions-"
TRIGGERS = {
"push": ("every push / PR", "push"),
"pr": ("every PR", "push"),
"merge": ("before merge", "push"),
"nightly": ("nightly", "sched"),
"sched": ("scheduled", "sched"),
"cont": ("continuous", "cont"),
}
# title, triggers, lines (at most 31 characters, so they fit a card of the three-column drawing)
LANES = [
("Build matrix", ["push"], [
("Build matrix", ["push", "merge"], [
"50+ compilers: GCC, Clang, MSVC,",
"AppleClang, Intel, NVIDIA, ...",
"C++11, 14, 17, 20, 23, and 26",
"libc++ and libstdc++",
"warnings: Clang -Weverything,",
"GCC 300+ warnings",
"a subset on every push and PR,",
"all before merge; Intel, NVIDIA,",
"CUDA, and ARM64 nightly",
]),
("Unit tests (doctest)", ["push"], [
"every public API function",
@@ -44,7 +49,7 @@ LANES = [
"custom number, string, array,",
"and object types",
]),
("Coverage", ["push"], [
("Coverage", ["nightly"], [
"unit tests cover all lines",
"of the code base",
]),
@@ -53,11 +58,11 @@ LANES = [
"Clang Static Analyzer (52",
"checkers), IWYU, Codacy",
"CodeQL, Semgrep, Flawfinder",
"(also scheduled)",
"(also scheduled); Infer nightly",
]),
("Dynamic analysis", ["push"], [
("Dynamic analysis", ["push", "nightly"], [
"runtime assertions",
"Valgrind (Memcheck)",
"Valgrind (Memcheck), nightly",
"address and undefined behavior",
"sanitizers, integer overflow,",
"nullability violations",
@@ -178,7 +183,7 @@ def drawing(W, cols, suffix):
defs = ('<defs><marker id="%s" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto">'
'<path class="fg-head" d="M0 1 L10 5 L0 9 z"/></marker></defs>' % mid)
tid, did = "nj-qa-title-" + suffix, "nj-qa-desc-" + suffix
desc = ("Every push or pull request starts these checks in parallel: " +
desc = ("A push or pull request starts these checks; the badges say when each runs: " +
"; ".join("%s (%s): %s" % (t, ", ".join(TRIGGERS[k][0] for k in trig), " ".join(lines)) for t, trig, lines in LANES) +
". When they pass, the change is merged into develop; a release is made only when no vulnerability alert is open.")
out = ['<svg class="nj-fig %sqa %sqa--%s" viewBox="0 0 %d %d" role="img" aria-labelledby="%s %s" xmlns="http://www.w3.org/2000/svg">'
@@ -19,6 +19,11 @@
#include <type_traits> // conditional
// The shortest-digit conversion of doubles (see zmij.hpp) and the writing of
// its digits, including the conversion of the digits to text by Xiang JunBo
// and the SIMD instruction sequence by Dougall Johnson, are ported from Zmij
// by Victor Zverovich (https://github.com/vitaut/zmij, MIT license).
//
// SSE2 (every x86-64 CPU) and NEON (every 64-bit Arm CPU) convert the 16
// digits of a double at once (unless JSON_NO_SIMD selects the portable code)
#if !defined(JSON_NO_SIMD) && (defined(__x86_64__) || (defined(_M_X64) && !defined(_M_ARM64EC)))
+22 -3
View File
@@ -230,7 +230,10 @@ struct diagnostic_positions
{
// Lexer has read the first character of the container, so
// subtract 1 from the position to get the correct start position.
v.start_position = lexer->get_chars_read_total() - 1;
// (npos: a value without a source position, see materialize() of
// json_view)
const std::size_t read = lexer->get_chars_read_total();
v.start_position = read == std::string::npos ? std::string::npos : read - 1;
}
}
@@ -245,6 +248,12 @@ struct diagnostic_positions
{
// Lexer's position is past the closing brace or bracket, so set that as the end position.
v.end_position = lexer->get_chars_read_total();
if (v.end_position == std::string::npos)
{
// a range has both ends or none (exception::diagnostics()
// prints only complete ones)
v.start_position = std::string::npos;
}
}
}
@@ -261,6 +270,12 @@ struct diagnostic_positions
// The start position will be set below based on the length of the string representation
// of the value.
v.end_position = lexer->get_chars_read_total();
if (v.end_position == std::string::npos)
{
// a value without a source position
v.start_position = std::string::npos;
return;
}
switch (v.type())
{
@@ -473,8 +488,12 @@ read up to it.
@tparam Collect whether to collect the values of containers of unknown size on
a stack (only for a caller that does not use the value after a
parse error)
@tparam LexerType the type of the lexer that reports the source positions
(JSON_DIAGNOSTIC_POSITIONS); only diagnostic_positions uses
it, so any type with its three functions works (as
materialize() of json_view uses it)
*/
template<typename BasicJsonType, typename InputAdapterType, bool Collect = false>
template<typename BasicJsonType, typename InputAdapterType, bool Collect = false, typename LexerType = lexer<BasicJsonType, InputAdapterType>>
class json_sax_dom_parser : public json_sax_dom_error_state
{
public:
@@ -483,7 +502,7 @@ class json_sax_dom_parser : public json_sax_dom_error_state
using number_float_t = typename BasicJsonType::number_float_t;
using string_t = typename BasicJsonType::string_t;
using binary_t = typename BasicJsonType::binary_t;
using lexer_t = lexer<BasicJsonType, InputAdapterType>;
using lexer_t = LexerType;
/*!
@param[in,out] r reference to a JSON value that is manipulated while
@@ -44,6 +44,13 @@
// use std::from_chars or std::strtold. They are free functions so the lexer
// stays focused on scanning (see lexer::convert_number()) and so that other
// parsers of JSON text can convert tokens exactly like it does.
//
// The Eisel-Lemire conversion (eisel_lemire()), the digit comparison for the
// cases it cannot decide, and the eight-digits-at-a-time helpers
// (is_eight_digits(), parse_eight_digits()) are adapted from fast_float
// (https://github.com/fastfloat/fast_float, Daniel Lemire and contributors,
// available under the MIT, Apache 2.0, and Boost Software licenses; used here
// under the MIT license). The table of powers of five is in pow5_table.hpp.
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
+2 -2
View File
@@ -56,8 +56,8 @@ using parser_callback_t =
template<typename SAX>
struct sax_ignores_number_text : std::false_type {};
template<typename BasicJsonType, typename InputAdapterType, bool Collect>
struct sax_ignores_number_text<json_sax_dom_parser<BasicJsonType, InputAdapterType, Collect>> : std::true_type {};
template<typename BasicJsonType, typename InputAdapterType, bool Collect, typename LexerType>
struct sax_ignores_number_text<json_sax_dom_parser<BasicJsonType, InputAdapterType, Collect, LexerType>> : std::true_type {};
template<typename BasicJsonType, typename InputAdapterType>
struct sax_ignores_number_text<json_sax_dom_callback_parser<BasicJsonType, InputAdapterType>> : std::true_type {};
@@ -30,6 +30,13 @@
// NLOHMANN_JSON_NO_TARGET_ATTRIBUTE (for compilers that ignore the target
// attribute and then reject the SSSE3 intrinsics) keeps the check portable
// without disabling the SSE2 code.
//
// The vector UTF-8 check (utf8_lookup4) is adapted from simdjson
// (https://github.com/simdjson/simdjson, Daniel Lemire, Geoff Langdale, John
// Keiser, and contributors, available under the Apache 2.0 and MIT licenses;
// used here under the MIT license): the "lookup4" algorithm and tables of
// J. Keiser and D. Lemire, "Validating UTF-8 In Less Than One Instruction Per
// Byte" (2021).
#if !defined(JSON_NO_SIMD) && defined(__aarch64__) && (defined(__GNUC__) || defined(__clang__)) && defined(__BYTE_ORDER__) && defined(__ORDER_LITTLE_ENDIAN__) && __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
#include <arm_neon.h>
#define JSON_SCAN_NEON 1
+492 -30
View File
@@ -8,7 +8,7 @@
#pragma once
#include <algorithm> // all_of, copy
#include <algorithm> // copy
#include <array> // array
#include <cctype> // isdigit
#include <cerrno> // errno, ERANGE
@@ -16,9 +16,11 @@
#include <cstdint> // uint64_t
#include <cstdlib> // strtoull
#include <cstring> // memchr, memcmp, memcpy
#include <functional> // less
#ifndef JSON_NO_IO
#include <iosfwd> // ostream
#endif // JSON_NO_IO
#include <iterator> // next
#include <limits> // max
#include <string> // string
#include <utility> // move
@@ -28,6 +30,7 @@
#include <nlohmann/detail/macro_scope.hpp>
#include <nlohmann/detail/meta/cpp_future.hpp>
#include <nlohmann/detail/meta/type_traits.hpp>
#include <nlohmann/detail/meta/void_t.hpp>
#include <nlohmann/detail/string_concat.hpp>
#include <nlohmann/detail/string_escape.hpp>
#include <nlohmann/detail/string_utils.hpp>
@@ -134,6 +137,26 @@ JSON_HEDLEY_ALWAYS_INLINE array_index_status parse_array_index(const StringType&
return array_index_status::ok;
}
/// whether two keys of type @a StringType are equivalent under @a Compare
/// exactly if they are equal: std::less<StringType> and std::less<>
template<typename Compare, typename StringType>
struct is_less_comparator : std::false_type {};
template<typename StringType>
struct is_less_comparator<std::less<StringType>, StringType> : std::true_type {};
template<typename StringType>
struct is_less_comparator<std::less<void>, StringType> : std::true_type {};
/// whether the objects of type @a ObjectType find a key of type @a StringType
/// exactly if it equals a stored key
template<typename ObjectType, typename StringType, typename = void>
struct finds_equal_keys : std::false_type {};
template<typename ObjectType, typename StringType>
struct finds_equal_keys<ObjectType, StringType, void_t<typename ObjectType::key_compare, typename ObjectType::key_type>>
: conjunction<is_less_comparator<typename ObjectType::key_compare, StringType>, std::is_same<typename ObjectType::key_type, StringType>> {};
} // namespace detail
/// @brief JSON Pointer defines a string syntax for identifying a specific value within a JSON document
@@ -243,6 +266,13 @@ class json_pointer
/// @sa https://json.nlohmann.me/api/json_pointer/operator_slasheq/
json_pointer& operator/=(const json_pointer& ptr)
{
// insert() must not be given iterators into the vector it inserts
// into: append a copy when a pointer is appended to itself
if (JSON_HEDLEY_UNLIKELY(&ptr == this))
{
return *this /= json_pointer(ptr);
}
reference_tokens.insert(reference_tokens.end(),
ptr.reference_tokens.begin(),
ptr.reference_tokens.end());
@@ -589,6 +619,225 @@ class json_pointer
return object_member_impl(object, key, detail::priority_tag<1> {});
}
/// @brief append a null value to @a array, like operator[] with the size of @a array
template<typename BasicJsonType>
static BasicJsonType& array_append(BasicJsonType& array)
{
auto& elements = *array.m_data.m_value.array;
const auto old_capacity = array.container_capacity();
elements.emplace_back();
return array.set_parent_after_insert(elements.back(), old_capacity);
}
/// @brief add @a key, which is missing, to @a object
template<typename BasicJsonType>
static auto object_add_impl(BasicJsonType& object, const string_t& key, detail::priority_tag<1> /*unused*/)
-> decltype(object.m_data.m_value.object->emplace(key, nullptr), std::declval<BasicJsonType&>())
{
const auto old_capacity = object.container_capacity();
const auto it = object.m_data.m_value.object->emplace(key, nullptr).first;
return object.set_parent_after_insert(it->second, old_capacity);
}
template<typename BasicJsonType>
static BasicJsonType& object_add_impl(BasicJsonType& object, const string_t& key, detail::priority_tag<0> /*unused*/)
{
return object.operator[](key);
}
template<typename BasicJsonType>
static BasicJsonType& object_add(BasicJsonType& object, const string_t& key)
{
return object_add_impl(object, key, detail::priority_tag<1> {});
}
/// @brief search @a key in @a members: the result is where the key is or,
/// for objects that can insert at a position (std::map), where it belongs
template<typename ObjectType>
static auto object_search_impl(ObjectType& members, const string_t& key, detail::priority_tag<1> /*unused*/)
-> decltype(members.key_comp()(key, members.lower_bound(key)->first), members.lower_bound(key))
{
return members.lower_bound(key);
}
template<typename ObjectType>
static auto object_search_impl(ObjectType& members, const string_t& key, detail::priority_tag<0> /*unused*/)
-> decltype(members.find(key))
{
return members.find(key);
}
template<typename ObjectType>
static auto object_search(ObjectType& members, const string_t& key)
-> decltype(object_search_impl(members, key, detail::priority_tag<1> {}))
{
return object_search_impl(members, key, detail::priority_tag<1> {});
}
/// @brief whether @a it, the result of @ref object_search for @a key, points to @a key
///
/// With std::less, the key at the lower bound is equivalent to @a key
/// exactly if it is equal to it, and comparing for equality is cheaper.
template<typename ObjectType, typename Iterator,
detail::enable_if_t<detail::finds_equal_keys<ObjectType, string_t>::value, int> = 0>
static auto object_search_found_impl(ObjectType& members, const Iterator& it, const string_t& key, detail::priority_tag<2> /*unused*/)
-> decltype(members.lower_bound(key), it->first == key)
{
return it != members.end() && it->first == key;
}
template<typename ObjectType, typename Iterator>
static auto object_search_found_impl(ObjectType& members, const Iterator& it, const string_t& key, detail::priority_tag<1> /*unused*/)
-> decltype(members.key_comp()(key, it->first), members.lower_bound(key), bool())
{
return it != members.end() && !members.key_comp()(key, it->first);
}
template<typename ObjectType, typename Iterator>
static bool object_search_found_impl(ObjectType& members, const Iterator& it, const string_t& /*key*/, detail::priority_tag<0> /*unused*/)
{
return it != members.end();
}
template<typename ObjectType, typename Iterator>
static bool object_search_found(ObjectType& members, const Iterator& it, const string_t& key)
{
return object_search_found_impl(members, it, key, detail::priority_tag<2> {});
}
/// @brief insert @a key, which is missing, into @a object at @a it, the
/// result of @ref object_search for @a key
template<typename BasicJsonType, typename Iterator>
static auto object_insert_impl(BasicJsonType& object, const Iterator& it, const string_t& key, detail::priority_tag<1> /*unused*/)
-> decltype(object.m_data.m_value.object->emplace_hint(it, key, nullptr), std::declval<BasicJsonType&>())
{
const auto old_capacity = object.container_capacity();
const auto pos = object.m_data.m_value.object->emplace_hint(it, key, nullptr);
return object.set_parent_after_insert(pos->second, old_capacity);
}
template<typename BasicJsonType, typename Iterator>
static BasicJsonType& object_insert_impl(BasicJsonType& object, const Iterator& /*it*/, const string_t& key, detail::priority_tag<0> /*unused*/)
{
return object_add(object, key);
}
template<typename BasicJsonType, typename Iterator>
static BasicJsonType& object_insert(BasicJsonType& object, const Iterator& it, const string_t& key)
{
return object_insert_impl(object, it, key, detail::priority_tag<1> {});
}
/*!
@brief undoes the change @ref get_unchecked_create made to an existing value
@ref get_unchecked walks existing values without changing them. At the
first reference token that does not select an existing value, @ref
get_unchecked_create makes exactly one change to an existing value: it
converts a null value, inserts a key into an object, or grows an array.
Everything it creates afterwards lies below that change. If a later step
throws, the destructor of this guard reverts that one change, which
removes everything created after it, so the value is left as it was
(strong exception safety). The guard needs no try/catch and therefore also
compiles with JSON_NOEXCEPTION.
Reverting the change must not throw, as it happens in a destructor. It
erases the inserted member by its iterator (no key is converted or
compared, nothing is allocated), erases the added elements at the end of
the array, or swaps the null value back. This holds for std::map,
std::vector, ordered_map, and every object_t, array_t, and
json_base_class_t whose erase() and move operations do not throw.
Between recording and reverting, nothing is inserted into or erased from
the object or array, so the iterator stays valid.
*/
template<typename BasicJsonType>
class get_unchecked_rollback
{
public:
using object_iterator = typename BasicJsonType::object_t::iterator;
/// record that @a member was inserted into the object @a value
get_unchecked_rollback(BasicJsonType* value, const object_iterator& member)
: m_value(value)
, m_member(member)
, m_change(change::key_inserted)
{}
/// record the change that the next reference token makes to the null value or array @a value
explicit get_unchecked_rollback(BasicJsonType* value)
: m_value(value)
{
if (value->is_null())
{
// keep the null value (with its base class and positions) and
// continue with a fresh null value in its place, which is what
// assigning an array or object to it would have left there
m_saved_null.swap(*value);
m_change = change::null_converted;
}
else
{
JSON_ASSERT(value->is_array());
m_old_size = value->m_data.m_value.array->size();
m_change = change::array_grown;
}
}
get_unchecked_rollback(const get_unchecked_rollback&) = delete;
get_unchecked_rollback(get_unchecked_rollback&&) = delete;
get_unchecked_rollback& operator=(const get_unchecked_rollback&) = delete;
get_unchecked_rollback& operator=(get_unchecked_rollback&&) = delete;
~get_unchecked_rollback()
{
undo();
}
/// keep the change
void commit() noexcept
{
m_change = change::none;
}
private:
void undo() noexcept
{
switch (m_change)
{
case change::null_converted:
// the values created since go away with m_saved_null
m_value->swap(m_saved_null);
break;
case change::key_inserted:
m_value->m_data.m_value.object->erase(m_member);
break;
case change::array_grown:
{
auto& elements = *m_value->m_data.m_value.array;
elements.resize(m_old_size);
break;
}
case change::none:
default:
break;
}
}
enum class change
{
none,
null_converted,
key_inserted,
array_grown
};
BasicJsonType* m_value;
object_iterator m_member {};
typename BasicJsonType::size_type m_old_size = 0;
change m_change = change::none;
BasicJsonType m_saved_null {};
};
JSON_PRIVATE_UNLESS_TESTED:
json_pointer top() const
{
@@ -972,47 +1221,245 @@ class json_pointer
Complexity: Linear in the length of the JSON pointer.
@throw parse_error.106 if an array index begins with '0'
@throw parse_error.109 if an array index was not a number
@throw out_of_range.404 if the JSON pointer can not be resolved
Exception safety: Strong guarantee. This function walks existing values
until the first reference token that does not select one. If that is the
last reference token, inserting the key into the object or growing the
array is strongly exception safe by itself; any other change is made by
@ref get_unchecked_create, which undoes it if a later step throws.
@throw parse_error.106 if an array index of an existing array begins with '0'
@throw parse_error.109 if an array index of an existing array was not a number
@throw out_of_range.404 if the JSON pointer can not be resolved, or if an
array index is too large for strtoull
@throw out_of_range.410 if an array index exceeds size_type
@throw std::length_error if padding an array exceeds its maximal size
@throw std::bad_alloc if an allocation fails
*/
template<typename BasicJsonType>
BasicJsonType& get_unchecked(BasicJsonType* ptr) const
{
for (const auto& reference_token : reference_tokens)
const auto* token = reference_tokens.data();
const auto* const end = token + reference_tokens.size();
if (token == end)
{
return *ptr;
}
if (ptr->is_null())
{
// the null value is converted to an array or an object
return get_unchecked_create(ptr, token, end);
}
// walk existing values; the walk itself inserts nothing, which keeps
// the loop fast (insertions are done below or in get_unchecked_create)
typename BasicJsonType::object_t::iterator it;
do
{
if (ptr->is_object())
{
auto& members = *ptr->m_data.m_value.object;
it = object_search(members, *token);
if (!object_search_found(members, it, *token))
{
// the key is inserted below or in get_unchecked_create
break;
}
ptr = &it->second;
}
else if (ptr->is_array())
{
if (is_dash(*token))
{
if (token + 1 == end)
{
// explicitly treat "-" as index beyond the end
return array_append(*ptr);
}
// an element will be appended in get_unchecked_create
break;
}
if (token + 1 == end)
{
// convert array index to number; unchecked access
return ptr->operator[](array_index<BasicJsonType>(*token));
}
const auto idx = array_index<BasicJsonType>(*token);
if (idx >= ptr->m_data.m_value.array->size())
{
// the array will be padded in get_unchecked_create
break;
}
ptr = &(*ptr->m_data.m_value.array)[idx];
}
else if (ptr->is_null())
{
// the null value will be converted in get_unchecked_create
break;
}
else
{
// primitive value with a reference token left
throw_unresolved(*token, ptr);
}
}
while (++token != end);
if (token == end)
{
return *ptr;
}
if (token + 1 == end && ptr->is_object())
{
// insert the missing key where the walk found it belongs
return object_insert(*ptr, it, *token);
}
return get_unchecked_create(ptr, token, end);
}
/*!
@brief create the values for the reference tokens from @a first on
Continues @ref get_unchecked at the first reference token that does not
select an existing value. That token changes @a ptr, an existing value (see
@ref get_unchecked_rollback); all later tokens only change values created
here.
If the emplace() of the object type finds a member that the lookup in @ref
get_unchecked did not find (the two disagree, e.g. for fifo_map), nothing
is inserted, and the walk continues with that member as in @ref
get_unchecked, which is what basic_json::operator[] does.
@param[in] ptr the value the reference token @a first is applied to
@param[in] first the first reference token to resolve (an element of
@ref reference_tokens)
@param[in] end the end of @ref reference_tokens
@throw parse_error.106 if an array index of an existing array begins with
'0' (only after emplace() found a member, see above)
@throw parse_error.109 if an array index of an existing array was not a
number (only after emplace() found a member)
@throw out_of_range.404 if an array index is too large for strtoull, or (only
after emplace() found a member) if the JSON pointer
can not be resolved
@throw out_of_range.410 if an array index exceeds size_type
@throw std::length_error if padding an array exceeds its maximal size
@throw std::bad_alloc if an allocation fails
*/
template<typename BasicJsonType>
JSON_HEDLEY_NEVER_INLINE static BasicJsonType& get_unchecked_create(BasicJsonType* ptr, const string_t* first, const string_t* end)
{
const auto* token = first;
while (ptr->is_object())
{
const auto old_capacity = ptr->container_capacity();
const auto inserted = ptr->m_data.m_value.object->emplace(*token, nullptr);
if (JSON_HEDLEY_LIKELY(inserted.second))
{
get_unchecked_rollback<BasicJsonType> rollback(ptr, inserted.first);
BasicJsonType& result = create_values(&ptr->set_parent_after_insert(inserted.first->second, old_capacity), token + 1, end);
rollback.commit();
return result;
}
// nothing was inserted: walk existing values from the member
// emplace() found on, like get_unchecked
ptr = &inserted.first->second;
while (++token != end)
{
if (ptr->is_object())
{
auto& members = *ptr->m_data.m_value.object;
const auto it = object_search(members, *token);
if (!object_search_found(members, it, *token))
{
break;
}
ptr = &it->second;
}
else if (ptr->is_array())
{
if (is_dash(*token))
{
break;
}
const auto idx = array_index<BasicJsonType>(*token);
if (idx >= ptr->m_data.m_value.array->size())
{
break;
}
ptr = &(*ptr->m_data.m_value.array)[idx];
}
else if (ptr->is_null())
{
break;
}
else
{
// primitive value with a reference token left
throw_unresolved(*token, ptr);
}
}
if (token == end)
{
return *ptr;
}
}
get_unchecked_rollback<BasicJsonType> rollback(ptr);
BasicJsonType& result = create_values(ptr, token, end);
rollback.commit();
return result;
}
/*!
@brief the loop of @ref get_unchecked_create
@a ptr is a null value or an array. Every value created here is a fresh
one, so every object a key is added to is empty.
@throw out_of_range.404 if an array index is too large for strtoull
@throw out_of_range.410 if an array index exceeds size_type
@throw std::length_error if padding an array exceeds its maximal size
@throw std::bad_alloc if an allocation fails
*/
template<typename BasicJsonType>
static BasicJsonType& create_values(BasicJsonType* ptr, const string_t* first, const string_t* end)
{
for (const auto* token = first; token != end; ++token)
{
const auto& reference_token = *token;
// convert null values to arrays or objects before continuing
if (ptr->is_null())
{
// check if the reference token is a valid array index, that is
// a nonempty sequence of digits without a leading '0'
// (cf. RFC 6901, Sect. 4); tokens that could never be a valid
// array index (such as "01" or "") are treated as object keys
const bool nums = !reference_token.empty()
&& (reference_token.size() == 1 || reference_token[0] != '0')
&& std::all_of(reference_token.begin(), reference_token.end(),
[](const unsigned char x)
// change value to an array for array indices or "-" or to an
// object otherwise; every null value here is a fresh one (see
// get_unchecked_rollback), so converting it in place leaves
// the same value as assigning an array or object to it
if (is_array_index_token(reference_token) || is_dash(reference_token))
{
return std::isdigit(x);
});
// change value to an array for array indices or "-" or to object otherwise
*ptr = (nums || reference_token == "-")
? detail::value_t::array
: detail::value_t::object;
ptr->template convert_null_to<typename BasicJsonType::array_t>();
}
else
{
ptr->template convert_null_to<typename BasicJsonType::object_t>();
}
}
if (ptr->is_object())
{
// use unchecked object access
ptr = &object_member(*ptr, reference_token);
// the object is new and empty
ptr = &object_add(*ptr, reference_token);
}
else if (ptr->is_array())
else
{
if (reference_token == "-")
// only the first value is not created here, and it is a null
// value or an array (see get_unchecked_create)
JSON_ASSERT(ptr->is_array());
if (is_dash(reference_token))
{
// explicitly treat "-" as index beyond the end
ptr = &ptr->operator[](ptr->m_data.m_value.array->size());
ptr = &array_append(*ptr);
}
else
{
@@ -1020,16 +1467,31 @@ class json_pointer
ptr = &ptr->operator[](array_index<BasicJsonType>(reference_token));
}
}
else
{
// primitive value with a reference token left
throw_unresolved(reference_token, ptr);
}
}
return *ptr;
}
/// whether @a s is a valid array index, that is a nonempty sequence of
/// digits without a leading '0' (cf. RFC 6901, Sect. 4); tokens that could
/// never be a valid array index (such as "01" or "") are object keys
static bool is_array_index_token(const string_t& s)
{
const std::size_t size = s.size();
if (size == 0 || (size > 1 && s[0] == '0'))
{
return false;
}
for (std::size_t i = 0; i < size; ++i)
{
if (std::isdigit(static_cast<unsigned char>(s[i])) == 0)
{
return false;
}
}
return true;
}
/*!
@throw parse_error.106 if an array index begins with '0'
@throw parse_error.109 if an array index was not a number
+13 -21
View File
@@ -173,31 +173,16 @@ class builder
}
if (p[1] == '/')
{
p += 2;
// (as in parse(), a null byte is the end of the input, so it is
// left for the caller to see)
while (p != m_end && *p != '\n' && *p != '\r' && !(NulIsEnd && *p == 0))
{
++p;
}
return p;
return skip_line_comment<NulIsEnd>(p + 2, m_end);
}
if (p[1] == '*')
{
p += 2;
for (;;)
p = find_block_comment_end<NulIsEnd>(p + 2, m_end);
if (p == m_end || *p != '*')
{
if (p == m_end || (NulIsEnd && *p == 0))
{
return fail_at(error_code::comment_unterminated, p);
}
if (*p == '*' && p + 1 != m_end && p[1] == '/')
{
p += 2;
return p;
}
++p;
return fail_at(error_code::comment_unterminated, p);
}
return p + 2;
}
++p;
return fail_at(error_code::comment_start, p);
@@ -1193,7 +1178,14 @@ indent_done:
return false;
}
p = r.p;
push_node(node_kind::string, node_flags::escaped, 0, r.start, r.len);
#if JSON_DIAGNOSTIC_POSITIONS
// the decoded string is in the arena: next keeps the source offset
// of the opening quote plus one (0: unknown) for materialize()
const std::uint64_t second = static_cast<std::uint64_t>(r.len) | (static_cast<std::uint64_t>(s - b) << 32);
#else
const std::uint64_t second = r.len;
#endif
push_node(node_kind::string, node_flags::escaped, 0, r.start, second);
return true;
}
};
File diff suppressed because it is too large. Load diff
+40 -3
View File
@@ -26,6 +26,7 @@
#include <nlohmann/detail/view/lookup.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
#include <nlohmann/detail/view/node.hpp>
#include <nlohmann/detail/view/scan.hpp>
#include <nlohmann/detail/view/view_mode.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN
@@ -46,10 +47,9 @@ struct first_true<false, Conditions...> : std::integral_constant < int, 1 + firs
/// (type_error.316 with the same message), so that an editable document
/// only holds valid UTF-8: the error is at the first byte that no
/// well-formed sequence can continue with (Unicode, Table 3-7).
inline void check_utf8(const char* s, std::size_t n)
NLOHMANN_VIEW_ALWAYS_INLINE void check_utf8_bytewise(const unsigned char* p, std::size_t n, std::size_t i)
{
const auto* const p = reinterpret_cast<const unsigned char*>(s); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
for (std::size_t i = 0; i < n;)
while (i < n)
{
const unsigned char c = p[i];
if (c < 0x80)
@@ -79,6 +79,43 @@ inline void check_utf8(const char* s, std::size_t n)
}
}
/// The long-string path of check_utf8(): skips well-formed runs with the
/// parser's string scan (which stops at a quote, a backslash, or a control
/// character, all legal here, and at ill-formed UTF-8); the byte-wise check
/// runs only from where the scan found ill-formed UTF-8, to report it.
NLOHMANN_VIEW_NOINLINE inline void check_utf8_scan(const unsigned char* p, std::size_t n)
{
const unsigned char* const e = p + n;
const unsigned char* q = p;
for (;;)
{
q = scan_string_run<true>(q, e);
if (q == e)
{
return;
}
if (*q >= 0x80)
{
check_utf8_bytewise(p, n, static_cast<std::size_t>(q - p));
return; // not reached: the scan stopped at ill-formed UTF-8
}
++q; // a quote, backslash, or control character
}
}
inline void check_utf8(const char* s, std::size_t n)
{
const auto* const p = reinterpret_cast<const unsigned char*>(s); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
if (NLOHMANN_VIEW_UNLIKELY(n >= 32))
{
check_utf8_scan(p, n);
return;
}
// short strings: the vector scan costs more than it saves; (the loop is
// inlined with a constant start, so this path is the plain byte-wise check)
check_utf8_bytewise(p, n, 0);
}
/*!
@brief the edits of an editable basic_json_document
+118 -30
View File
@@ -80,7 +80,8 @@ constexpr std::uint64_t image_limit = 0xFFFFFFF0u;
/// Copy the current structure of an edited document into nodes in document
/// order, as the parser would have written them. Text written by edits is
/// appended to text_tail (number tokens) and arena_tail (strings); floats that
/// are not finite become null, as dump() writes them.
/// are not finite become null, as dump() writes them. Nodes keep the flag
/// is_new: they have no source position (JSON_DIAGNOSTIC_POSITIONS).
inline void compact_nodes(const document_data& d, std::size_t arena_size, std::vector<node>& out, std::string& text_tail, std::string& arena_tail)
{
struct frame
@@ -103,16 +104,18 @@ inline void compact_nodes(const document_data& d, std::size_t arena_size, std::v
arena_tail.append(d.str(s), s.len);
r.flags = node_flags::escaped;
}
r.flags = static_cast<std::uint8_t>(r.flags | (s.flags & node_flags::is_new));
return r;
};
const auto copy_node = [&](const node * v)
{
node r = *v;
const auto is_new = static_cast<std::uint8_t>(v->flags & node_flags::is_new);
switch (static_cast<value_t>(v->kind))
{
case value_t::object:
case value_t::array:
r.flags = 0;
r.flags = is_new;
r.extra = 0;
r.off = (v->flags & (node_flags::moved | node_flags::is_new)) != 0 ? 0 : v->off;
r.len = 0; // counted below
@@ -129,7 +132,7 @@ inline void compact_nodes(const document_data& d, std::size_t arena_size, std::v
r.off = static_cast<std::uint32_t>(d.size + text_tail.size());
text_tail.append(d.str(*v), number_length(*v));
}
r.flags = 0;
r.flags = is_new;
break;
case value_t::number_float:
if ((v->flags & node_flags::storage) == node_flags::edited)
@@ -139,22 +142,23 @@ inline void compact_nodes(const document_data& d, std::size_t arena_size, std::v
{
r = node{}; // nan and infinity: null, as dump() writes them
r.kind = node_kind::null;
r.flags = is_new;
break;
}
r.off = static_cast<std::uint32_t>(d.size + text_tail.size());
text_tail.append(t, v->len);
r.extra = 0xFFFFu; // the digit layout is not recorded
}
r.flags = 0;
r.flags = is_new;
break;
case value_t::boolean:
r.flags = static_cast<std::uint8_t>(v->flags & node_flags::is_true);
r.flags = static_cast<std::uint8_t>(v->flags & (node_flags::is_true | node_flags::is_new));
break;
case value_t::null:
case value_t::binary:
case value_t::discarded:
default:
r.flags = 0;
r.flags = is_new;
break;
}
out.push_back(r);
@@ -254,6 +258,81 @@ struct number_token
bool is_float; ///< a fraction or an exponent
};
/// advance over ASCII digits, eight bytes at a time
inline const unsigned char* digit_run(const unsigned char* p, const unsigned char* e) noexcept
{
constexpr std::uint64_t ones = 0x0101010101010101ull;
while (e - p >= 8)
{
// (the digits become 0..9; a byte above 9, or with bit 7 set, is no digit. A carry
// out of a byte with bit 7 set only affects the bytes above, after the first one
// that count_trailing_zeros finds.)
const std::uint64_t x = read_eight_bytes(p) ^ (ones * 0x30u);
const std::uint64_t not_digit = ((x + (ones * 0x76u)) | x) & (ones * 0x80u);
if (not_digit != 0)
{
return p + (static_cast<std::size_t>(count_trailing_zeros(not_digit)) / 8);
}
p += 8;
}
while (p != e && is_digit(*p))
{
++p;
}
return p;
}
/// whether the n bytes at p are ASCII digits (n > 0)
inline bool all_digits(const unsigned char* p, std::size_t n) noexcept
{
const auto digits = [](const unsigned char* q)
{
constexpr std::uint64_t ones = 0x0101010101010101ull;
const std::uint64_t x = read_eight_bytes(q) ^ (ones * 0x30u);
return (((x + (ones * 0x76u)) | x) & (ones * 0x80u)) == 0;
};
if (n >= 8)
{
for (std::size_t i = 0; i + 8 < n; i += 8)
{
if (!digits(p + i))
{
return false;
}
}
return digits(p + n - 8); // (overlapping the bytes before: they are digits)
}
for (std::size_t i = 0; i < n; ++i)
{
if (!is_digit(p[i]))
{
return false;
}
}
return true;
}
/// Whether a float token has the shape that the parser records for most of
/// them: [-]digits.digits, with the numbers of digits of the node's layout
/// (extra). Then the token is a number of that layout, and the general scan
/// that follows is not needed.
inline bool plain_float_token(const unsigned char* s, std::size_t len, std::uint16_t extra) noexcept
{
const std::size_t int_digits = extra & 0xFFu;
const std::size_t frac_digits = extra >> 8u;
if (int_digits == 0 || int_digits >= 255 || frac_digits == 0 || frac_digits >= 255 || len == 0)
{
return false;
}
const std::size_t negative = *s == '-' ? 1 : 0;
if (len != negative + int_digits + 1 + frac_digits)
{
return false;
}
const unsigned char* const p = s + negative;
return !(int_digits > 1 && *p == '0') && p[int_digits] == '.' && all_digits(p, int_digits) && all_digits(p + int_digits + 1, frac_digits);
}
/// whether [s, s + len) is a JSON number (the grammar the parser accepts)
inline bool scan_number_token(const unsigned char* s, std::size_t len, number_token& t)
{
@@ -272,11 +351,7 @@ inline bool scan_number_token(const unsigned char* s, std::size_t len, number_to
}
else if (*p >= '1' && *p <= '9')
{
do
{
++p;
}
while (p != e && is_digit(*p));
p = digit_run(p + 1, e);
}
else
{
@@ -288,10 +363,7 @@ inline bool scan_number_token(const unsigned char* s, std::size_t len, number_to
if (p != e && *p == '.')
{
const unsigned char* const f0 = ++p;
while (p != e && is_digit(*p))
{
++p;
}
p = digit_run(p, e);
if (p == f0)
{
return false;
@@ -402,10 +474,15 @@ struct float_range
template<typename Check>
bool check_distinct_ranges(std::vector<byte_range>& ranges, Check check)
{
std::sort(ranges.begin(), ranges.end(), [](const byte_range & a, const byte_range & b)
// (save() writes the ranges in the order of the nodes, which is sorted already)
const auto before = [](const byte_range & a, const byte_range & b)
{
return a.off != b.off ? a.off < b.off : a.len < b.len;
});
};
if (!std::is_sorted(ranges.begin(), ranges.end(), before))
{
std::sort(ranges.begin(), ranges.end(), before);
}
std::size_t end = 0;
for (std::size_t i = 0; i < ranges.size(); ++i)
{
@@ -427,7 +504,8 @@ bool check_distinct_ranges(std::vector<byte_range>& ranges, Check check)
/// record layouts that match it), tokens must not overlap.
inline bool check_float_ranges(std::vector<float_range>& ranges, const unsigned char* text)
{
std::sort(ranges.begin(), ranges.end(), [](const float_range & a, const float_range & b)
// (save() writes the tokens in the order of the nodes, which is sorted already)
const auto before = [](const float_range & a, const float_range & b)
{
if (a.off != b.off)
{
@@ -438,7 +516,11 @@ inline bool check_float_ranges(std::vector<float_range>& ranges, const unsigned
return a.len < b.len;
}
return a.extra < b.extra;
});
};
if (!std::is_sorted(ranges.begin(), ranges.end(), before))
{
std::sort(ranges.begin(), ranges.end(), before);
}
std::size_t end = 0;
std::size_t i = 0;
while (i < ranges.size())
@@ -448,14 +530,18 @@ inline bool check_float_ranges(std::vector<float_range>& ranges, const unsigned
{
return false;
}
number_token t{};
const unsigned char* const s = text + r.off;
if (!scan_number_token(s, r.len, t) || !float_token_finite(s, r.len, t))
std::uint16_t layout = r.extra;
if (!plain_float_token(s, r.len, r.extra))
{
return false;
number_token t{};
if (!scan_number_token(s, r.len, t) || !float_token_finite(s, r.len, t))
{
return false;
}
// the digit layout the parser records (or "many", as compaction writes it)
layout = float_layout(t);
}
// the digit layout the parser records (or "many", as compaction writes it)
const std::uint16_t layout = float_layout(t);
for (; i < ranges.size() && ranges[i].off == r.off && ranges[i].len == r.len; ++i)
{
if (ranges[i].extra != layout && ranges[i].extra != 0xFFFFu)
@@ -515,13 +601,15 @@ inline bool check_image(const node* nodes, std::size_t count, const unsigned cha
std::vector<float_range> floats;
const auto check_string = [&](const node & n) -> bool
{
if ((n.flags & ~node_flags::escaped) != 0 || n.extra != 0)
if ((n.flags & ~(node_flags::escaped | node_flags::is_new)) != 0 || n.extra != 0)
{
return false;
}
const bool decoded = (n.flags & node_flags::escaped) != 0;
const std::size_t limit = decoded ? arena_size : text_size;
if (n.off > limit || n.len > limit - n.off)
// (next: the source offset of a decoded string's opening quote plus
// one, or 0; see node)
if (n.off > limit || n.len > limit - n.off || n.next > text_size)
{
return false;
}
@@ -593,13 +681,13 @@ inline bool check_image(const node* nodes, std::size_t count, const unsigned cha
{
case value_t::null:
// (the offset of a literal is read to size the output of dump())
if (n.flags != 0 || n.extra != 0 || n.off > text_size)
if ((n.flags & ~node_flags::is_new) != 0 || n.extra != 0 || n.off > text_size)
{
return false;
}
break;
case value_t::boolean:
if ((n.flags & ~node_flags::is_true) != 0 || n.extra != 0 || n.off > text_size)
if ((n.flags & ~(node_flags::is_true | node_flags::is_new)) != 0 || n.extra != 0 || n.off > text_size)
{
return false;
}
@@ -613,7 +701,7 @@ inline bool check_image(const node* nodes, std::size_t count, const unsigned cha
case value_t::number_integer:
case value_t::number_unsigned:
case value_t::number_float:
if (n.flags != 0 || !number_in_bounds(n))
if ((n.flags & ~node_flags::is_new) != 0 || !number_in_bounds(n))
{
return false;
}
@@ -635,7 +723,7 @@ inline bool check_image(const node* nodes, std::size_t count, const unsigned cha
case value_t::object:
{
const std::size_t limit = stack.empty() ? count : stack.back().end;
if (n.flags != 0 || n.extra != 0 || n.next == 0 || n.next > limit - i || n.off > text_size)
if ((n.flags & ~node_flags::is_new) != 0 || n.extra != 0 || n.next == 0 || n.next > limit - i || n.off > text_size)
{
return false;
}
+263 -5
View File
@@ -19,6 +19,7 @@
#include <nlohmann/detail/view/macro_scope.hpp>
#include <nlohmann/detail/view/node.hpp>
#include <nlohmann/detail/view/number.hpp>
#include <nlohmann/detail/view/scan.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
@@ -26,6 +27,189 @@ namespace detail
namespace view
{
#if JSON_DIAGNOSTIC_POSITIONS
/*!
@brief the source positions of the values materialize() replays
Stands in for the lexer of json_sax_dom_parser, which reads the positions of
the values it stores from these three functions (diagnostic_positions). npos
is a value without a source position. One position_source serves a whole
materialize() call: before each value is handed to the handler, the range of
that value is set (or cleared), overwriting that of the value before.
*/
struct position_source
{
/// the token, of which diagnostic_positions only reads the length
struct token
{
std::size_t length;
std::size_t size() const noexcept
{
return length;
}
};
std::size_t start = std::string::npos; ///< the first byte of the next value
std::size_t end = std::string::npos; ///< one past the next value (start_object/start_array: one past the bracket)
std::size_t get_chars_read_total() const noexcept
{
return end;
}
std::size_t get_token_start_position() const noexcept
{
return start;
}
token get_string() const noexcept
{
return token{end - start};
}
/// the range [first, last), or none if either is npos
void set(std::size_t first, std::size_t last) noexcept
{
if (first == std::string::npos || last == std::string::npos)
{
clear();
return;
}
start = first;
end = last;
}
/// no range: the next value has no source position
void clear() noexcept
{
start = std::string::npos;
end = std::string::npos;
}
/// one past the closing quote of the string whose opening quote is at
/// src[quote], or npos (the scan is bounded by size, as images can be
/// loaded without a check)
static std::size_t string_end(const char* src, std::size_t size, std::size_t quote) noexcept
{
for (std::size_t i = quote + 1; i < size; ++i)
{
if (src[i] == '\\')
{
++i;
}
else if (src[i] == '"')
{
return i + 1;
}
}
return std::string::npos;
}
/// one past the closing bracket of the array or object whose last element
/// ends at src[pos] (or whose opening bracket is at src[pos - 1]), or npos;
/// in between, there can only be whitespace, comments, and a trailing comma
/// (the source was validated when it was parsed)
static std::size_t container_end(const char* src, std::size_t size, std::size_t pos) noexcept
{
const auto* const b = reinterpret_cast<const unsigned char*>(src); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast): the text as bytes, as the builder reads it
const unsigned char* const e = b + size;
while (pos < size)
{
switch (src[pos])
{
case ' ':
case '\t':
case '\n':
case '\r':
case ',':
++pos;
break;
case ']':
case '}':
return pos + 1;
case '/':
if (pos + 1 < size && src[pos + 1] == '/')
{
pos = static_cast<std::size_t>(skip_line_comment(b + pos + 2, e) - b);
break;
}
if (pos + 1 < size && src[pos + 1] == '*')
{
// (unterminated: pos is size, which ends the loop)
pos = static_cast<std::size_t>(find_block_comment_end(b + pos + 2, e) - b) + 2;
break;
}
return std::string::npos;
default:
return std::string::npos;
}
}
return std::string::npos;
}
/// the range of the scalar n of document d
void set_scalar(const document_data& d, const node& n) noexcept
{
if ((n.flags & node_flags::is_new) != 0)
{
clear();
return;
}
std::size_t first = std::string::npos;
std::size_t last = std::string::npos;
switch (n.kind)
{
case node_kind::string:
if ((n.flags & node_flags::storage) == 0)
{
// off is the first byte after the opening quote
if (n.off != 0)
{
first = n.off - 1;
last = n.off + std::size_t{n.len} + 1;
}
}
else if ((n.flags & node_flags::storage) == node_flags::escaped && n.next != 0)
{
// next is the offset of the opening quote plus one
first = n.next - 1;
last = string_end(d.base[0], d.size, first);
}
break;
case node_kind::number_integer:
case node_kind::number_unsigned:
case node_kind::number_float:
if ((n.flags & node_flags::storage) == 0)
{
first = n.off;
last = n.off + number_length(n);
}
break;
case node_kind::boolean:
first = n.off;
last = n.off + ((n.flags & node_flags::is_true) != 0 ? 4u : 5u);
break;
case node_kind::null:
first = n.off;
last = n.off + 4u;
break;
case node_kind::object:
case node_kind::array:
case node_kind::link:
default:
break;
}
// (a crafted image can point past the text)
if (last == std::string::npos || last > d.size)
{
clear();
}
else
{
set(first, last);
}
}
};
#endif
/*!
@brief the basic_json value of the subtree at n
@@ -33,14 +217,24 @@ The subtree is replayed into the SAX handler that parse() uses to build its
values, so the result is the value parse() would produce: duplicate keys keep
the last value, and with JSON_DIAGNOSTICS the parent pointers are set. It is
iterative, so the nesting depth is limited by memory only, as for parse().
Without a lexer the handler records no source positions
(JSON_DIAGNOSTIC_POSITIONS).
With JSON_DIAGNOSTIC_POSITIONS, the values get their source positions as with
parse(): position_source stands in for the lexer. Values written by edits have
none (npos), and neither has an array or object whose elements changed, or
whose last element has none (the end of its closing bracket is found by
scanning from the end of the last element).
*/
template<typename BasicJsonType, bool Editable>
BasicJsonType materialize(const document_data& d, const node* n)
{
using string_t = typename BasicJsonType::string_t;
using sax_t = json_sax_dom_parser<BasicJsonType, iterator_input_adapter<const char*>, true>;
using input_t = iterator_input_adapter<const char*>;
#if JSON_DIAGNOSTIC_POSITIONS
using lexer_t = position_source;
#else
using lexer_t = lexer<BasicJsonType, input_t>; // (the default: unused, as no lexer is passed)
#endif
using sax_t = json_sax_dom_parser<BasicJsonType, input_t, true, lexer_t>;
using nav = navigation<Editable>;
// the open containers: their next element (or key of the next member),
@@ -52,9 +246,15 @@ BasicJsonType materialize(const document_data& d, const node* n)
const node* pos;
const node* end;
bool object;
#if JSON_DIAGNOSTIC_POSITIONS
std::size_t start; ///< the opening bracket (npos: unknown)
std::size_t last_end; ///< the end of the last element read (npos: unknown)
#endif
};
#if !JSON_DIAGNOSTIC_POSITIONS
// a scalar needs neither the handler nor the stack of open containers
// (with positions, it goes through the handler, which sets them)
switch (static_cast<value_t>(n->kind))
{
case value_t::string:
@@ -76,10 +276,16 @@ BasicJsonType materialize(const document_data& d, const node* n)
default:
break;
}
#endif
BasicJsonType result;
#if JSON_DIAGNOSTIC_POSITIONS
position_source positions;
sax_t sax(result, true, &positions);
#else
sax_t sax(result, true);
const string_t no_token{};
#endif
const string_t no_token {};
// (shallow subtrees on the stack: a call for a small value then allocates only for the value)
std::array<frame, 16> shallow; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): filled as containers open
std::vector<frame> deep{};
@@ -94,6 +300,16 @@ BasicJsonType materialize(const document_data& d, const node* n)
std::size_t depth = 0;
for (;;)
{
#if JSON_DIAGNOSTIC_POSITIONS
if (!is_container(*n))
{
positions.set_scalar(d, *n);
if (depth != 0)
{
open[depth - 1].last_end = positions.end;
}
}
#endif
// false positive: n comes from nav::value(), which never returns null for a valid index
// @infer-ignore NULLPTR_DEREFERENCE
switch (static_cast<value_t>(n->kind))
@@ -102,6 +318,17 @@ BasicJsonType materialize(const document_data& d, const node* n)
case value_t::array:
{
const bool object = n->kind == node_kind::object;
#if JSON_DIAGNOSTIC_POSITIONS
// (moved containers are also new)
if ((n->flags & node_flags::is_new) != 0 || n->off >= d.size)
{
positions.clear();
}
else
{
positions.set(std::size_t{n->off}, std::size_t{n->off} + 1);
}
#endif
if (object)
{
sax.start_object(n->len);
@@ -118,7 +345,13 @@ BasicJsonType materialize(const document_data& d, const node* n)
open = deep.data();
capacity = deep.size();
}
open[depth++] = frame{nav::first(d, n), nav::end(d, n), object};
// (with positions: until an element is read, the container's
// own bracket is the last thing read)
open[depth++] = frame{nav::first(d, n), nav::end(d, n), object
#if JSON_DIAGNOSTIC_POSITIONS
, positions.start, positions.end
#endif
};
break;
}
case value_t::string:
@@ -159,6 +392,31 @@ BasicJsonType materialize(const document_data& d, const node* n)
frame& f = open[depth - 1];
if (f.pos == f.end)
{
#if JSON_DIAGNOSTIC_POSITIONS
if (f.start == std::string::npos || f.last_end == std::string::npos)
{
positions.clear();
}
else
{
const std::size_t end = position_source::container_end(d.base[0], d.size, f.last_end);
if (end == std::string::npos)
{
positions.clear();
}
else
{
// the closing bracket, as the lexer reports it (end_object()
// and end_array() only read its end; the start of the
// container was set when it was opened)
positions.set(end - 1, end);
}
}
if (depth > 1)
{
open[depth - 2].last_end = positions.end;
}
#endif
if (f.object)
{
sax.end_object();
+1 -1
View File
@@ -69,7 +69,7 @@ struct node
std::uint16_t extra; ///< numbers: integer digits (low byte) and fraction digits (high byte), 255 = "many"; objects: number of the object's key hash index (1-based, 0 = none); otherwise 0
std::uint32_t off; ///< source offset (string content, number token, literal, bracket); arena offset if escaped/edited; number of the element sequence if moved
std::uint32_t len; ///< string: decoded bytes; float: token bytes; array/object: element count
std::uint32_t next; ///< array/object: number of nodes of the subtree (its extent in the enclosing sequence)
std::uint32_t next; ///< array/object: number of nodes of the subtree (its extent in the enclosing sequence); escaped string: source offset of the opening quote plus one (JSON_DIAGNOSTIC_POSITIONS; 0: unknown)
};
static_assert(sizeof(node) == 16, "node must stay 16 bytes");
+8 -163
View File
@@ -19,7 +19,6 @@
#include <nlohmann/detail/view/macro_scope.hpp>
#include <nlohmann/detail/view/node.hpp>
#include <nlohmann/detail/view/scan.hpp>
#include <nlohmann/detail/view/simd.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
@@ -104,56 +103,12 @@ NLOHMANN_VIEW_NOINLINE FloatType float_value(const char* first, const node& n)
return convert_float<FloatType>(first, last, dot, mantissa_end);
}
#if NLOHMANN_VIEW_VECTOR
/*!
@brief the value of the first 16 digits of a run of digits with a point after
int_digits of them (or none, if int_digits >= 16)
The 17 bytes at p are read at once: the bytes after the point move down by one,
then pairs, quadruples, and octets of digits are combined by multiplications
(after simdjson). The bytes [p, p + 32) must be readable. Bytes that are not
digits (only in a damaged image loaded with image_check::bounds) give a wrong
value, but no undefined behavior.
*/
NLOHMANN_VIEW_ALWAYS_INLINE std::uint64_t sixteen_digits(const unsigned char* p, unsigned int_digits) noexcept
{
#if NLOHMANN_VIEW_NEON
const uint8x16_t low = vld1q_u8(p);
const uint8x16_t index = vcombine_u8(vcreate_u8(0x0706050403020100u), vcreate_u8(0x0F0E0D0C0B0A0908u));
const uint8x16_t before_point = vcltq_u8(index, vdupq_n_u8(static_cast<std::uint8_t>(int_digits)));
const uint8x16_t digits = vsubq_u8(vbslq_u8(before_point, low, vextq_u8(low, vld1q_u8(p + 16), 1)), vdupq_n_u8('0'));
const uint16x8_t twos = vpaddlq_u8(vmulq_u8(digits, vreinterpretq_u8_u16(vdupq_n_u16(0x010A))));
const uint32x4_t fours = vpaddlq_u16(vmulq_u16(twos, vreinterpretq_u16_u32(vdupq_n_u32(0x00010064))));
const uint64x2_t eights = vpaddlq_u32(vmulq_u32(fours, vreinterpretq_u32_u64(vdupq_n_u64(0x0000000100002710u))));
return (vgetq_lane_u64(eights, 0) * 100000000u) + vgetq_lane_u64(eights, 1);
#else
// NOLINTBEGIN(portability-simd-intrinsics)
const __m128i low = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(p))); // NOLINT(bugprone-casting-through-void)
const __m128i high = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(p + 16))); // NOLINT(bugprone-casting-through-void)
const __m128i before_point = _mm_cmpgt_epi8(_mm_set1_epi8(static_cast<char>(int_digits)), _mm_setr_epi8(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15));
const __m128i moved = _mm_or_si128(_mm_srli_si128(low, 1), _mm_slli_si128(high, 15));
const __m128i digits = _mm_sub_epi8(_mm_or_si128(_mm_and_si128(before_point, low), _mm_andnot_si128(before_point, moved)), _mm_set1_epi8('0'));
// (16-bit lanes: SSE2 has no multiply-add of bytes)
const __m128i tens = _mm_setr_epi16(10, 1, 10, 1, 10, 1, 10, 1);
const __m128i twos = _mm_packs_epi32(_mm_madd_epi16(_mm_unpacklo_epi8(digits, _mm_setzero_si128()), tens),
_mm_madd_epi16(_mm_unpackhi_epi8(digits, _mm_setzero_si128()), tens));
const __m128i fours = _mm_madd_epi16(twos, _mm_setr_epi16(100, 1, 100, 1, 100, 1, 100, 1));
const __m128i eights = _mm_madd_epi16(_mm_packs_epi32(fours, fours), _mm_setr_epi16(10000, 1, 10000, 1, 10000, 1, 10000, 1));
const auto upper = static_cast<std::uint32_t>(_mm_cvtsi128_si32(eights));
const auto lower = static_cast<std::uint32_t>(_mm_cvtsi128_si32(_mm_srli_si128(eights, 4)));
// NOLINTEND(portability-simd-intrinsics)
return (std::uint64_t{upper} * 100000000u) + lower;
#endif
}
#endif
/*!
@brief the digits of a float token with at most 19 digits, from its layout
The digit layout recorded while parsing says where the integer digits, the
fraction digits, and the exponent are, so the digits are read eight at a
time without scanning (with SIMD, a token of 16 to 19 digits sixteen at a
time).
time without scanning.
@param[in] p first character of the token
@param[in] e end of the token
@@ -163,34 +118,15 @@ NLOHMANN_VIEW_ALWAYS_INLINE float_significand layout_decimal(const unsigned char
{
const bool negative = *p == '-';
p += negative ? 1 : 0;
std::uint64_t w = 0;
std::uint64_t w = parse_upto19(p, int_digits, limit);
p += int_digits;
std::int64_t q = 0;
#if NLOHMANN_VIEW_VECTOR
const unsigned count = int_digits + frac_digits;
if (count >= 16 && limit - p >= 32)
if (frac_digits != 0)
{
// (the point, if any, follows the integer digits; the digits after
// the first 16 are at most three)
w = sixteen_digits(p, int_digits);
for (unsigned i = 16; i < count; ++i)
{
w = (w * 10) + static_cast<std::uint64_t>(p[i + (i >= int_digits ? 1 : 0)] - '0');
}
p += count + (frac_digits != 0 ? 1 : 0);
w = (w * int_pow10(frac_digits)) + parse_upto19(p + 1, frac_digits, limit);
p += 1 + frac_digits;
q = -static_cast<std::int64_t>(frac_digits);
}
else /* NOLINT(readability/braces) */
#endif
{
w = parse_upto19(p, int_digits, limit);
p += int_digits;
if (frac_digits != 0)
{
w = (w * int_pow10(frac_digits)) + parse_upto19(p + 1, frac_digits, limit);
p += 1 + frac_digits;
q = -static_cast<std::int64_t>(frac_digits);
}
}
if (p != e)
{
// [eE][+-]digits; huge exponents saturate (the parser rejected
@@ -225,22 +161,12 @@ NLOHMANN_VIEW_ALWAYS_INLINE float_significand layout_decimal(const unsigned char
The result is correctly rounded by the lexer's conversion
(detail::decimal_to_float(): Clinger's fast path where both operands are
exact, else the Eisel-Lemire algorithm, which needs no fallback for up to 19
digits), so it is the value parse() produces. Tokens of 16 or more digits go
to Eisel-Lemire directly: the fast path takes few of them (none of 17 digits),
and its check is a branch that data like GeoJSON coordinates mispredicts.
digits), so it is the value parse() produces.
*/
template<typename FloatType>
NLOHMANN_VIEW_ALWAYS_INLINE FloatType layout_float(const unsigned char* p, const unsigned char* e, unsigned int_digits, unsigned frac_digits, const unsigned char* limit) noexcept
{
const float_significand d = layout_decimal(p, e, int_digits, frac_digits, limit);
if (int_digits + frac_digits >= 16)
{
using format = ieee_binary_format<std::numeric_limits<FloatType>::digits>;
native_float_t<FloatType> value{};
float_from_bits(eisel_lemire<format>(d.exponent, d.w) | (d.negative ? (std::uint64_t{1} << format::sign_bit()) : 0u), value);
return value;
}
return decimal_to_float<FloatType>(d);
return decimal_to_float<FloatType>(layout_decimal(p, e, int_digits, frac_digits, limit));
}
/// the value of a float set by an edit: its token (the shortest round-trip
@@ -259,80 +185,6 @@ NLOHMANN_VIEW_NOINLINE FloatType edited_float(const char* token, const node& n)
return float_value<FloatType>(token, n);
}
#if NLOHMANN_VIEW_VECTOR
/// eisel_lemire() for the token w * 10^q of coordinate_float()
template<typename Format>
NLOHMANN_VIEW_ALWAYS_INLINE std::uint64_t coordinate_bits(std::int64_t q, std::uint64_t w, Format /*format*/) noexcept
{
return eisel_lemire<Format>(q, w);
}
/*!
@brief (binary64) eisel_lemire() without the cases that tokens of 16 or 17
digits with 5 or more after the point cannot reach: a value of 0, a
subnormal, an infinity, or a tie (q < -4)
*/
NLOHMANN_VIEW_ALWAYS_INLINE std::uint64_t coordinate_bits(std::int64_t q, std::uint64_t w, ieee_binary_format<53> format) noexcept
{
if (NLOHMANN_VIEW_UNLIKELY(w == 0 || q > -5))
{
return eisel_lemire<ieee_binary_format<53>>(q, w);
}
static_cast<void>(format);
const int lz = count_leading_zeros(w);
const std::uint64_t normalized = w << static_cast<unsigned>(lz);
const auto index = static_cast<std::size_t>(2 * (q - pow5_128_smallest_power));
const uint128_parts product = full_multiplication(normalized, pow5_128()[index]);
std::uint64_t high = product.high;
if (NLOHMANN_VIEW_UNLIKELY((high & 0x1FFu) == 0x1FFu))
{
// (the lower bits may carry into the result)
const std::uint64_t second = full_multiplication(normalized, pow5_128()[index + 1]).high;
high += product.low + second < product.low ? 1u : 0u;
}
const auto upperbit = static_cast<unsigned>(high >> 63u);
std::uint64_t mantissa = high >> (upperbit + 9);
auto power2 = static_cast<std::uint64_t>(((217706 * q) >> 16) + 63 + static_cast<std::int64_t>(upperbit) - lz + 1023);
mantissa = (mantissa + (mantissa & 1u)) >> 1u;
const bool carry = mantissa >= (std::uint64_t{2} << 52u);
mantissa = carry ? (std::uint64_t{1} << 52u) : mantissa;
power2 += carry ? 1u : 0u;
return (mantissa & ~(std::uint64_t{1} << 52u)) | (power2 << 52u);
}
/*!
@brief (binary32 and binary64) the value of a float token of the source with
16 or 17 digits, a point, and no exponent, the shape of most coordinates
written with full precision; whether the token of n has this shape
These tokens skip the general layout of layout_decimal(): no digits beyond
the 17th, no exponent (and Eisel-Lemire directly, as in layout_float()).
*/
template<typename FloatType>
NLOHMANN_VIEW_ALWAYS_INLINE bool coordinate_float(const document_data& d, const node& n, FloatType& value) noexcept
{
const unsigned int_digits = n.extra & 0xFFu;
const unsigned frac_digits = n.extra >> 8u;
const unsigned count = int_digits + frac_digits;
const auto* const first = reinterpret_cast<const unsigned char*>(d.src + n.off); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
const unsigned negative = first[0] == '-' ? 1u : 0u;
if (count - 16u > 1u || frac_digits == 0 || n.len != negative + count + 1 || d.size - n.off < 33)
{
return false;
}
const unsigned char* const digits = first + negative;
const std::uint64_t w = sixteen_digits(digits, int_digits);
// (the 17th digit follows the point)
const std::uint64_t w17 = (w * 10) + static_cast<std::uint64_t>(digits[17] - '0');
using format = ieee_binary_format<std::numeric_limits<FloatType>::digits>;
native_float_t<FloatType> result{};
float_from_bits(coordinate_bits(-static_cast<std::int64_t>(frac_digits), count == 17 ? w17 : w, format{}) | (static_cast<std::uint64_t>(negative) << format::sign_bit()), result);
value = result;
return true;
}
#endif
/// the value of the float token of a node, as parse() converts it; floats and
/// doubles with at most 19 digits are converted from the digit layout
template<typename FloatType>
@@ -348,13 +200,6 @@ FloatType float_value(const document_data& d, const node& n)
template<typename FloatType>
FloatType float_value(const document_data& d, const node& n, std::true_type /*binary32 or binary64*/)
{
#if NLOHMANN_VIEW_VECTOR
FloatType value {};
if (coordinate_float(d, n, value))
{
return value;
}
#endif
const unsigned int_digits = n.extra & 0xFFu;
const unsigned frac_digits = n.extra >> 8u;
if (NLOHMANN_VIEW_LIKELY(int_digits + frac_digits <= 19)) // (255 marks "many")
+26
View File
@@ -172,6 +172,32 @@ NLOHMANN_VIEW_ALWAYS_INLINE const unsigned char* skip_digits(const unsigned char
return p;
}
/// advance over the text of a line comment (after its "//") to the line
/// break or the end (with NulIsEnd also a null byte, which is left for the
/// caller to see, as in parse())
template<bool NulIsEnd = false>
inline const unsigned char* skip_line_comment(const unsigned char* p, const unsigned char* e) noexcept
{
while (p != e && *p != '\n' && *p != '\r' && !(NulIsEnd && *p == 0))
{
++p;
}
return p;
}
/// the "*/" that ends a block comment whose text starts at p (after its "/*");
/// if it is unterminated, the end (with NulIsEnd also a null byte), which is
/// where the caller reports the error
template<bool NulIsEnd = false>
inline const unsigned char* find_block_comment_end(const unsigned char* p, const unsigned char* e) noexcept
{
while (p != e && !(NulIsEnd && *p == 0) && !(*p == '*' && p + 1 != e && p[1] == '/'))
{
++p;
}
return p;
}
/// powers of ten up to 10^19 as integers
inline std::uint64_t int_pow10(unsigned k) noexcept
{
+1 -387
View File
@@ -14,9 +14,7 @@
#include <cstddef> // size_t
#include <cstdint> // uint8_t, uint32_t
#include <cstring> // memcpy, memset
#include <functional> // less
#include <limits> // numeric_limits
#include <memory> // unique_ptr
#include <type_traits> // integral_constant
#include <vector> // vector
@@ -386,127 +384,6 @@ struct depth_counter<false>
}
};
/*!
@brief where the double x nearest to a token w * 10^q lies, for 10^15 <= w <
10^17 and -17 <= q <= -1 (Eisel-Lemire, then a fixed-point position)
dump() writes the shortest decimal in the rounding interval of x (the closest
one if there are several, see zmij::to_shortest()). Its digits are those of a
multiple of 10^g of the token's last digit, with g = dec_exp + 1 - q for x's
decimal exponent dec_exp: the multiple of 10^g nearest to x if it lies in the
interval (at most one does), else the multiple of 10^(g - 1) nearest to x
(see token_digits_1() and token_digits_2()). For g <= 0, the token itself is
in the interval.
The position is exact in a 64-bit fixed-point number with 56 bits after the
point (of the token's last digit): x / 10^q = 2c * 5^-q / 2^t for x = c * 2^e2
and t = q + 1 - e2, with at most 56 bits after the point here, and |x / 10^q -
w| is at most half the spacing of the doubles, below 12 units. So only the low
64 bits of 2c * 5^-q are needed. c and e2 come from Eisel-Lemire's product
directly (as eisel_lemire() rounds it: no tie is possible for q < -4, and its
second product is added where it can carry into the result).
@param[out] offset x / 10^q - w + 32, with 56 bits after the point
@param[out] radius half the spacing of the doubles at x, likewise (+ 1 if
the interval is closed)
@return g; 3 if the digits are not decided here (a power of two, whose
interval is not symmetric, or out of range)
*/
NLOHMANN_VIEW_ALWAYS_INLINE int locate_token(std::uint64_t w, int q, std::uint64_t& offset, std::uint64_t& radius) noexcept
{
const int lz = count_leading_zeros(w);
const std::uint64_t normalized = w << static_cast<unsigned>(lz);
const auto index = static_cast<std::size_t>(2 * (q - pow5_128_smallest_power));
const uint128_parts product = full_multiplication(normalized, pow5_128()[index]);
std::uint64_t high = product.high;
if (NLOHMANN_VIEW_UNLIKELY((high & 0x1FFu) == 0x1FFu))
{
// (as eisel_lemire(): the lower bits may carry into the result)
const std::uint64_t second = full_multiplication(normalized, pow5_128()[index + 1]).high;
high += product.low + second < product.low ? 1u : 0u;
}
const auto upperbit = static_cast<unsigned>(high >> 63u);
const std::uint64_t mantissa = high >> (upperbit + 9);
const std::uint64_t c = (mantissa + (mantissa & 1u)) >> 1u;
// e2 as eisel_lemire() computes the exponent (log2(10) ~ 217706 / 2^16)
const int e2 = static_cast<int>((217706 * static_cast<std::int64_t>(q)) >> 16) + 63 + static_cast<int>(upperbit) - lz - 52;
const int t = q + 1 - e2;
const int g = ((e2 * 315653) >> 20) + 1 - q; // (dec_exp as in zmij::to_shortest(), for a regular x)
static const std::array<std::uint64_t, 17> fives = {{1u, 5u, 25u, 125u, 625u, 3125u, 15625u, 78125u, 390625u, 1953125u, 9765625u, 48828125u, 244140625u, 1220703125u, 6103515625u, 30517578125u, 152587890625u}};
const std::uint64_t five = fives[static_cast<std::size_t>(-q)]; // 5^-q
const unsigned s = static_cast<unsigned>(56 - t) & 63u;
offset = ((2 * c * five) << s) - (w << 56u) + (std::uint64_t{32} << 56u);
radius = (five << s) + ((c & 1u) == 0 ? 1u : 0u);
// (c = 2^52: a power of two; c = 2^53: rounded up to one. With 8 to 16
// fraction digits, as the batch's tokens have, t is in [15, 40] and g at
// most 2. x lies halfway between two candidates only if its distance from
// the token is a whole unit (g == 2) or a half (g == 1): such tokens are
// left to the full conversion)
const bool decided = c - ((std::uint64_t{1} << 52u) + 1) < (std::uint64_t{1} << 52u) - 1
&& (offset & ((std::uint64_t{1} << 55u) - 1)) != 0;
return decided ? g : 3;
}
/// the distance of a position (see locate_token()) from the biased point 32
NLOHMANN_VIEW_ALWAYS_INLINE std::uint64_t biased_distance(std::uint64_t position) noexcept
{
constexpr std::uint64_t center = std::uint64_t{32} << 56u;
return position >= center ? position - center : center - position;
}
/*!
@brief (g == 1) the digits dump() writes for a token, from the position of its
double (see locate_token()): the multiple of 10 nearest to x if it is in the
interval, else w rounded to x (an exact tie between two candidates, like
754570942.33984373, whose double is 754570942.33984375, is excluded by
locate_token())
With |x - w| at most the radius, which is below half the spacing of the
candidates, the multiple of 10 nearest to x is w - r10 or the one above it.
*/
NLOHMANN_VIEW_ALWAYS_INLINE std::uint64_t token_digits_1(std::uint64_t w, std::uint64_t offset, std::uint64_t radius, std::uint64_t r10) noexcept
{
constexpr std::uint64_t one = std::uint64_t{1} << 56u;
const std::uint64_t from = offset + (r10 * one); // x - (w - r10) (+ 32)
const bool up = from > 37 * one; // (nearer to the multiple above)
const bool shorter = biased_distance(from - (up ? 10 * one : 0)) < radius;
const std::uint64_t a = w - r10 + (up ? 10u : 0u);
const std::uint64_t b = w + ((offset + (one / 2)) >> 56u) - 32; // w rounded to x
return shorter ? a : b;
}
/*!
@brief (g == 2) the digits dump() writes for a token, as token_digits_1(): the
multiple of 100 nearest to x if it is in the interval, else the multiple of 10
nearest to x
*/
NLOHMANN_VIEW_ALWAYS_INLINE std::uint64_t token_digits_2(std::uint64_t w, std::uint64_t offset, std::uint64_t radius, std::uint64_t r10, std::uint64_t r100) noexcept
{
constexpr std::uint64_t one = std::uint64_t{1} << 56u;
const std::uint64_t from = offset + (r100 * one);
const bool up = from > 82 * one;
const bool shorter = biased_distance(from - (up ? 100 * one : 0)) < radius;
const std::uint64_t a = w - r100 + (up ? 100u : 0u);
// x - (w - r10), in [-12, 21]: the multiple of 10 nearest to x is that one,
// or one or two above, or one below
const std::uint64_t y = offset + (r10 * one);
const std::uint64_t steps = (y > 37 * one ? 1u : 0u) + (y > 47 * one ? 1u : 0u);
const bool down = y < 27 * one;
const std::uint64_t b = w - r10 + (10 * steps) - (down ? 10u : 0u);
return shorter ? a : b;
}
/// the four ASCII digits of x < 10^4, the first in the lowest byte (as
/// little-endian memory holds them)
NLOHMANN_VIEW_ALWAYS_INLINE std::uint32_t four_digit_text(std::uint32_t x) noexcept
{
const std::uint32_t ab = (x * 5243u) >> 19u; // x / 100
std::uint32_t v = ab | ((x - (ab * 100u)) << 16u);
const std::uint32_t a = ((v * 103u) >> 10u) & 0x000F000Fu; // / 10 per 16-bit lane
v = a | ((v - (a * 10u)) << 8u);
return v + 0x30303030u;
}
/// how the view's dump() writes a value
struct dump_style
{
@@ -834,7 +711,6 @@ class view_serializer
std::array<frame, 32> buffer; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
std::vector<frame> heap;
small_stack<frame> stack(buffer.data(), buffer.size(), heap);
float_batch batch;
const char* const src = m_doc.src;
const char* const src_end = src + m_doc.size;
char* w = m_out.cursor();
@@ -974,7 +850,7 @@ class view_serializer
else if (enabled(fast_double::value))
{
room(64);
w = write_double_ahead(w, src, n, batch);
w = write_double_at(w, *n);
}
else
{
@@ -999,20 +875,6 @@ class view_serializer
case value_t::array:
{
const bool object = n->kind == node_kind::object;
// (read-only documents, compact output) a pair of floats,
// like the coordinates of GeoJSON, without a frame
if (enabled(!Editable && !Pretty && !SourceNumbers && fast_double::value) && n->next == 3 && n->len == 2
&& n[1].kind == node_kind::number_float && n[2].kind == node_kind::number_float)
{
room(2 * 64);
*w++ = '[';
w = write_double_ahead(w, src, n + 1, batch);
*w++ = ',';
w = write_double_ahead(w, src, n + 2, batch);
*w++ = ']';
n += 2; // (the element before the next node)
break;
}
room(8);
if (n->len == 0)
{
@@ -1122,254 +984,6 @@ class view_serializer
}
}
/// (float_batch) the mark of digits whose rounding carried beyond the
/// token's last four (see mark())
static constexpr std::uint64_t carried() noexcept
{
return std::uint64_t{1} << 63u;
}
/// floats of 16 or 17 digits ahead of the walk, converted together (see
/// convert_ahead()), with their digits as mark() keeps them
struct float_batch
{
static constexpr std::size_t capacity() noexcept
{
return 256;
}
/// (on the heap, allocated by the first conversion: a frame of more
/// than a page made every dump of a small document slower)
struct entries
{
std::array<const node*, 256> at; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
std::array<std::uint64_t, 256> digits; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
};
std::unique_ptr<entries> storage{};
const node** at = nullptr;
std::uint64_t* digits = nullptr;
std::size_t count = 0;
std::size_t next = 0;
const node* scanned = nullptr; ///< where the last scan of the node array ended
};
/// (doubles) the float at n as dump() writes it, at w (64 bytes of room):
/// from the batch if it holds n
NLOHMANN_VIEW_ALWAYS_INLINE char* write_double_ahead(char* w, const char* src, const node* n, float_batch& batch)
{
#if NLOHMANN_VIEW_VECTOR && NLOHMANN_VIEW_LITTLE_ENDIAN
// (digits 0: not written by the batch; carried(): rounding carried
// beyond the last four digits)
if (batch.next != batch.count && batch.at[batch.next] == n && batch.digits[batch.next] - 1 < carried() - 1)
{
return write_token_digits(w, src, *n, batch.digits[batch.next++]);
}
return write_double_beside(w, n, batch);
#else
static_cast<void>(src);
static_cast<void>(batch);
return write_double_at(w, *n);
#endif
}
#if NLOHMANN_VIEW_VECTOR && NLOHMANN_VIEW_LITTLE_ENDIAN
/// write_double_ahead() for a float that is not the batch's next one (out
/// of line: the walk is faster with less code in its loop)
NLOHMANN_VIEW_NOINLINE char* write_double_beside(char* w, const node* n, float_batch& batch)
{
// (an editable document's walk can skip nodes of the array)
const std::less<const node*> before{}; // NOLINT(modernize-use-transparent-functors) C++11 has no std::less<>
while (batch.next != batch.count && before(batch.at[batch.next], n))
{
++batch.next;
}
if ((batch.next != batch.count && batch.at[batch.next] == n) || convert_ahead(n, batch))
{
const std::uint64_t digits = batch.digits[batch.next++];
if ((digits & carried()) != 0)
{
return write_carried_digits(w, *n, digits & ~carried());
}
if (digits != 0)
{
return write_token_digits(w, m_doc.src, *n, digits);
}
}
return write_double_at(w, *n);
}
/*!
@brief convert the floats of 16 or 17 digits among the next nodes of the
array at once; whether n is the first of them
Such tokens (like the coordinates of GeoJSON, written with 17 digits) are
written as the token with its last four digits replaced
(write_token_digits()). Each step (the digits, locate_token(),
token_digits_1/2()) is a long chain of dependent operations; one loop per
step over many floats lets the processor overlap the floats, which it does
not across the steps of one float (on Apple M1, one loop for the digits and
locate_token() made the dump of canada.json 12 % slower). Each node of the array
is scanned once; a node that the walk does not reach in order (in an
editable document) is written without the batch.
*/
NLOHMANN_VIEW_NOINLINE bool convert_ahead(const node* n, float_batch& batch)
{
const std::less<const node*> before{}; // NOLINT(modernize-use-transparent-functors) C++11 has no std::less<>
const node* const last = m_doc.nodes + m_doc.node_count;
if (before(n, batch.scanned) || before(n, m_doc.nodes) || !before(n, last))
{
return false;
}
const char* const src = m_doc.src;
const char* const src_end = src + m_doc.size;
const node* const stop = last - n > 1024 ? n + 1024 : last;
batch.count = 0;
batch.next = 0;
// the digits of each token (and its last four from the text)
std::array<std::uint64_t, 256> digits; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
std::array<int, 256> exponent; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
std::array<std::uint32_t, 256> last_four; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
const node* p = n;
if (!batch.storage)
{
batch.storage.reset(new typename float_batch::entries); // NOLINT(cppcoreguidelines-owning-memory,modernize-make-unique): C++11
batch.at = batch.storage->at.data();
batch.digits = batch.storage->digits.data();
}
for (; p != stop && batch.count != float_batch::capacity(); ++p)
{
// a token of the source: 16 or 17 digits, at least 8 after the
// point, no exponent, 32 bytes to read
const unsigned frac_digits = p->extra >> 8u;
const unsigned int_digits = p->extra & 0xFFu;
const unsigned count = int_digits + frac_digits;
if (p->kind == node_kind::number_float && (p->flags & node_flags::storage) == 0
&& count - 16u <= 1u && frac_digits >= 8 && src_end - (src + p->off) >= 33)
{
const auto* const token = reinterpret_cast<const unsigned char*>(src + p->off); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
const unsigned sign = token[0] == '-' ? 1 : 0;
const unsigned char* const first = token + sign;
const std::uint64_t w = sixteen_digits(first, int_digits);
// (the 17th digit follows the point; a token with an exponent
// gets digits outside the range of locate_token(), and is not
// written by the batch. A token 0.xxx with 16 significant
// digits is: its double is written as "0." and its digits too)
const std::uint64_t w17 = (w * 10) + static_cast<std::uint64_t>(first[17] - '0');
const std::size_t i = batch.count++;
batch.at[i] = p;
if (p->len != sign + count + 1)
{
digits[i] = 0;
}
else
{
digits[i] = count == 17 ? w17 : w;
}
exponent[i] = -static_cast<int>(frac_digits);
// (byte 0: the last digit, byte 1: the last two, bits 16-29: the last four)
std::uint32_t four = 0;
std::memcpy(&four, token + p->len - 4, 4);
four -= 0x30303030u;
const std::uint32_t pairs = ((four & 0x0F0F0F0Fu) * 2561u) >> 8u; // (byte 0, 2: the pairs)
last_four[i] = (four >> 24u) | ((pairs >> 16u & 0xFFu) << 8u) | ((((pairs & 0x00FF00FFu) * 6553601u) >> 16u) << 16u);
}
}
batch.scanned = p;
// the position of each double, and the floats of either kind
std::array<std::uint64_t, 256> offset; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
std::array<std::uint64_t, 256> radius; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
std::array<std::uint8_t, 256> kind_1; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
std::array<std::uint8_t, 256> kind_2; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
std::size_t count_1 = 0;
std::size_t count_2 = 0;
for (std::size_t i = 0; i != batch.count; ++i)
{
const std::uint64_t w = digits[i];
// (the range excludes the values of bytes that are not digits, only
// in a damaged image loaded with image_check::bounds)
const int g = w - 1000000000000000u < 99000000000000000u ? locate_token(w, exponent[i], offset[i], radius[i]) : 3;
batch.digits[i] = g <= 0 ? (last_four[i] >> 16u) + 1 : 0;
kind_1[count_1] = static_cast<std::uint8_t>(i);
count_1 += g == 1 ? 1 : 0;
kind_2[count_2] = static_cast<std::uint8_t>(i);
count_2 += g == 2 ? 1 : 0;
}
// the digits to write, in one loop per kind (see mark())
for (std::size_t k = 0; k != count_1; ++k)
{
const std::size_t i = kind_1[k];
const std::uint64_t s = token_digits_1(digits[i], offset[i], radius[i], last_four[i] & 0xFFu);
batch.digits[i] = mark(digits[i], s, last_four[i] >> 16u);
}
for (std::size_t k = 0; k != count_2; ++k)
{
const std::size_t i = kind_2[k];
const std::uint64_t s = token_digits_2(digits[i], offset[i], radius[i], last_four[i] & 0xFFu, (last_four[i] >> 8u) & 0xFFu);
batch.digits[i] = mark(digits[i], s, last_four[i] >> 16u);
}
return batch.count != 0 && batch.at[0] == n;
}
/*!
@brief a float token of 16 or 17 digits as dump() writes it, at w (64 bytes
of room), from the digits of its double (see locate_token()) whose
rounding carried beyond the token's last four digits
(like 43.42027299999999 -> 43.420273): the digits go to the writer of
Zmij's results as its 15 or 16 leading digits and one more, which omits
the trailing zeros.
*/
NLOHMANN_VIEW_NOINLINE char* write_carried_digits(char* w, const node& n, std::uint64_t digits) const noexcept
{
const int exponent = -(n.extra >> 8u);
const std::uint64_t leading = digits / 10;
const auto last = static_cast<unsigned char>(digits - (leading * 10));
const bool seventeen = digits >= 10000000000000000u;
*w = '-';
w += m_doc.src[n.off] == '-' ? 1 : 0;
return ::nlohmann::detail::dtoa_impl::write_shortest(w, seventeen ? ::nlohmann::detail::zmij::shortest_decimal{leading, exponent, last, last != 0}
: ::nlohmann::detail::zmij::shortest_decimal{digits, exponent - 1, 0, false});
}
/// the digits s of a token with the digits w and the last four digits
/// r4, as the batch keeps them: the new last four digits + 1 (only those
/// are written over the token's), or s marked if it differs from w beyond
/// them (0 stays for a float the batch does not write)
static std::uint64_t mark(std::uint64_t w, std::uint64_t s, std::uint64_t r4) noexcept
{
const std::uint64_t last = s - (w - r4);
return last < 10000 ? last + 1 : s | carried();
}
/*!
@brief a float token of 16 or 17 digits as dump() writes it, at w (64 bytes
of room): the token with its last four digits replaced by digits - 1 (see
mark(); the others are the token's), and without trailing zeros after the
point, but one
*/
char* write_token_digits(char* w, const char* src, const node& n, std::uint64_t digits)
{
const char* const token = src + n.off;
std::memcpy(w, token, 32);
char* const end = w + n.len;
const std::uint32_t last = four_digit_text(static_cast<std::uint32_t>(digits - 1));
std::memcpy(end - 4, &last, 4);
// (the token's text before the new digits, from the source: reading
// the output back would wait for the stores)
std::uint64_t before = 0;
std::memcpy(&before, token + n.len - 12, 8);
const std::uint32_t last_zeros = last ^ 0x30303030u;
const std::uint64_t before_zeros = before ^ 0x3030303030303030u;
if (NLOHMANN_VIEW_UNLIKELY(last_zeros == 0 && before_zeros == 0))
{
return write_double_at(w, n); // (twelve or more trailing zeros)
}
const unsigned zeros = last_zeros != 0 ? (static_cast<unsigned>(count_leading_zeros(last_zeros)) / 8) - 4
: 4 + (static_cast<unsigned>(count_leading_zeros(before_zeros)) / 8);
const unsigned frac_digits = n.extra >> 8u;
return end - (zeros < frac_digits ? zeros : frac_digits - 1);
}
#endif
/// a float node as dump() writes it
void write_float_node(const node& n)
{
+7
View File
@@ -29,6 +29,13 @@
// NLOHMANN_VIEW_NO_TARGET_ATTRIBUTE (set by the GCC module interface, where GCC
// ignores the target attribute and then rejects the SSSE3 intrinsics) keeps the
// check portable without disabling the SSE2 code.
//
// The vector UTF-8 check (utf8_lookup4) is adapted from simdjson
// (https://github.com/simdjson/simdjson, Daniel Lemire, Geoff Langdale, John
// Keiser, and contributors, available under the Apache 2.0 and MIT licenses;
// used here under the MIT license): the "lookup4" algorithm and tables of
// J. Keiser and D. Lemire, "Validating UTF-8 In Less Than One Instruction Per
// Byte" (2021).
#if !defined(JSON_NO_SIMD) && defined(__aarch64__) && (defined(__GNUC__) || defined(__clang__)) && NLOHMANN_VIEW_LITTLE_ENDIAN
#include <arm_neon.h>
#define NLOHMANN_VIEW_NEON 1
+20 -1
View File
@@ -159,7 +159,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
friend class ::nlohmann::detail::binary_writer;
template<typename BasicJsonType, typename InputType, typename SAX>
friend class ::nlohmann::detail::binary_reader;
template<typename BasicJsonType, typename InputAdapterType, bool Collect>
template<typename BasicJsonType, typename InputAdapterType, bool Collect, typename LexerType>
friend class ::nlohmann::detail::json_sax_dom_parser;
template<typename BasicJsonType, typename InputAdapterType>
friend class ::nlohmann::detail::json_sax_dom_binary_parser;
@@ -6950,6 +6950,25 @@ public:
/// @sa https://json.nlohmann.me/api/basic_json/patch/
void patch_inplace(const basic_json& json_patch)
{
// the operations below change *this while iterating json_patch: apply
// a copy when a value is applied as patch to itself or, with
// JSON_DIAGNOSTICS, to a value stored inside it
// NOLINTNEXTLINE(misc-const-correctness): only the JSON_DIAGNOSTICS build modifies it
bool patch_aliases_this = (&json_patch == this);
#if JSON_DIAGNOSTICS
for (const basic_json* p = json_patch.m_parent; !patch_aliases_this && p != nullptr; p = p->m_parent)
{
patch_aliases_this = (p == this);
}
#endif
if (JSON_HEDLEY_UNLIKELY(patch_aliases_this))
{
// NOLINTNEXTLINE(performance-unnecessary-copy-initialization): *this is modified below
const basic_json json_patch_copy = json_patch;
patch_inplace(json_patch_copy);
return;
}
basic_json& result = *this;
// the valid JSON Patch operations
enum class patch_operations {add, remove, replace, move, copy, test, invalid};
+4
View File
@@ -763,8 +763,12 @@ class basic_json_view
template<typename, view_mode> friend class basic_json_document;
template<typename, view_mode> friend class basic_json_view;
template<typename, typename> friend class detail::view::editor;
template<typename, typename> friend class detail::view::view_side;
friend iterator;
/// whether the view is one of an editable document
static constexpr bool is_editable = Editable;
basic_json_view(const document_data* d, const node* n) noexcept
: m_doc(d), m_node(n)
{}
+556 -36
View File
@@ -10970,6 +10970,13 @@ NLOHMANN_JSON_NAMESPACE_END
// use std::from_chars or std::strtold. They are free functions so the lexer
// stays focused on scanning (see lexer::convert_number()) and so that other
// parsers of JSON text can convert tokens exactly like it does.
//
// The Eisel-Lemire conversion (eisel_lemire()), the digit comparison for the
// cases it cannot decide, and the eight-digits-at-a-time helpers
// (is_eight_digits(), parse_eight_digits()) are adapted from fast_float
// (https://github.com/fastfloat/fast_float, Daniel Lemire and contributors,
// available under the MIT, Apache 2.0, and Boost Software licenses; used here
// under the MIT license). The table of powers of five is in pow5_table.hpp.
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
@@ -12078,6 +12085,13 @@ NLOHMANN_JSON_NAMESPACE_END
// NLOHMANN_JSON_NO_TARGET_ATTRIBUTE (for compilers that ignore the target
// attribute and then reject the SSSE3 intrinsics) keeps the check portable
// without disabling the SSE2 code.
//
// The vector UTF-8 check (utf8_lookup4) is adapted from simdjson
// (https://github.com/simdjson/simdjson, Daniel Lemire, Geoff Langdale, John
// Keiser, and contributors, available under the Apache 2.0 and MIT licenses;
// used here under the MIT license): the "lookup4" algorithm and tables of
// J. Keiser and D. Lemire, "Validating UTF-8 In Less Than One Instruction Per
// Byte" (2021).
#if !defined(JSON_NO_SIMD) && defined(__aarch64__) && (defined(__GNUC__) || defined(__clang__)) && defined(__BYTE_ORDER__) && defined(__ORDER_LITTLE_ENDIAN__) && __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
#include <arm_neon.h>
#define JSON_SCAN_NEON 1
@@ -15629,7 +15643,10 @@ struct diagnostic_positions
{
// Lexer has read the first character of the container, so
// subtract 1 from the position to get the correct start position.
v.start_position = lexer->get_chars_read_total() - 1;
// (npos: a value without a source position, see materialize() of
// json_view)
const std::size_t read = lexer->get_chars_read_total();
v.start_position = read == std::string::npos ? std::string::npos : read - 1;
}
}
@@ -15644,6 +15661,12 @@ struct diagnostic_positions
{
// Lexer's position is past the closing brace or bracket, so set that as the end position.
v.end_position = lexer->get_chars_read_total();
if (v.end_position == std::string::npos)
{
// a range has both ends or none (exception::diagnostics()
// prints only complete ones)
v.start_position = std::string::npos;
}
}
}
@@ -15660,6 +15683,12 @@ struct diagnostic_positions
// The start position will be set below based on the length of the string representation
// of the value.
v.end_position = lexer->get_chars_read_total();
if (v.end_position == std::string::npos)
{
// a value without a source position
v.start_position = std::string::npos;
return;
}
switch (v.type())
{
@@ -15872,8 +15901,12 @@ read up to it.
@tparam Collect whether to collect the values of containers of unknown size on
a stack (only for a caller that does not use the value after a
parse error)
@tparam LexerType the type of the lexer that reports the source positions
(JSON_DIAGNOSTIC_POSITIONS); only diagnostic_positions uses
it, so any type with its three functions works (as
materialize() of json_view uses it)
*/
template<typename BasicJsonType, typename InputAdapterType, bool Collect = false>
template<typename BasicJsonType, typename InputAdapterType, bool Collect = false, typename LexerType = lexer<BasicJsonType, InputAdapterType>>
class json_sax_dom_parser : public json_sax_dom_error_state
{
public:
@@ -15882,7 +15915,7 @@ class json_sax_dom_parser : public json_sax_dom_error_state
using number_float_t = typename BasicJsonType::number_float_t;
using string_t = typename BasicJsonType::string_t;
using binary_t = typename BasicJsonType::binary_t;
using lexer_t = lexer<BasicJsonType, InputAdapterType>;
using lexer_t = LexerType;
/*!
@param[in,out] r reference to a JSON value that is manipulated while
@@ -22515,8 +22548,8 @@ using parser_callback_t =
template<typename SAX>
struct sax_ignores_number_text : std::false_type {};
template<typename BasicJsonType, typename InputAdapterType, bool Collect>
struct sax_ignores_number_text<json_sax_dom_parser<BasicJsonType, InputAdapterType, Collect>> : std::true_type {};
template<typename BasicJsonType, typename InputAdapterType, bool Collect, typename LexerType>
struct sax_ignores_number_text<json_sax_dom_parser<BasicJsonType, InputAdapterType, Collect, LexerType>> : std::true_type {};
template<typename BasicJsonType, typename InputAdapterType>
struct sax_ignores_number_text<json_sax_dom_callback_parser<BasicJsonType, InputAdapterType>> : std::true_type {};
@@ -24064,7 +24097,7 @@ NLOHMANN_JSON_NAMESPACE_END
#include <algorithm> // all_of, copy
#include <algorithm> // copy
#include <array> // array
#include <cctype> // isdigit
#include <cerrno> // errno, ERANGE
@@ -24072,9 +24105,11 @@ NLOHMANN_JSON_NAMESPACE_END
#include <cstdint> // uint64_t
#include <cstdlib> // strtoull
#include <cstring> // memchr, memcmp, memcpy
#include <functional> // less
#ifndef JSON_NO_IO
#include <iosfwd> // ostream
#endif // JSON_NO_IO
#include <iterator> // next
#include <limits> // max
#include <string> // string
#include <utility> // move
@@ -24088,6 +24123,8 @@ NLOHMANN_JSON_NAMESPACE_END
// #include <nlohmann/detail/meta/type_traits.hpp>
// #include <nlohmann/detail/meta/void_t.hpp>
// #include <nlohmann/detail/string_concat.hpp>
// #include <nlohmann/detail/string_escape.hpp>
@@ -24198,6 +24235,26 @@ JSON_HEDLEY_ALWAYS_INLINE array_index_status parse_array_index(const StringType&
return array_index_status::ok;
}
/// whether two keys of type @a StringType are equivalent under @a Compare
/// exactly if they are equal: std::less<StringType> and std::less<>
template<typename Compare, typename StringType>
struct is_less_comparator : std::false_type {};
template<typename StringType>
struct is_less_comparator<std::less<StringType>, StringType> : std::true_type {};
template<typename StringType>
struct is_less_comparator<std::less<void>, StringType> : std::true_type {};
/// whether the objects of type @a ObjectType find a key of type @a StringType
/// exactly if it equals a stored key
template<typename ObjectType, typename StringType, typename = void>
struct finds_equal_keys : std::false_type {};
template<typename ObjectType, typename StringType>
struct finds_equal_keys<ObjectType, StringType, void_t<typename ObjectType::key_compare, typename ObjectType::key_type>>
: conjunction<is_less_comparator<typename ObjectType::key_compare, StringType>, std::is_same<typename ObjectType::key_type, StringType>> {};
} // namespace detail
/// @brief JSON Pointer defines a string syntax for identifying a specific value within a JSON document
@@ -24307,6 +24364,13 @@ class json_pointer
/// @sa https://json.nlohmann.me/api/json_pointer/operator_slasheq/
json_pointer& operator/=(const json_pointer& ptr)
{
// insert() must not be given iterators into the vector it inserts
// into: append a copy when a pointer is appended to itself
if (JSON_HEDLEY_UNLIKELY(&ptr == this))
{
return *this /= json_pointer(ptr);
}
reference_tokens.insert(reference_tokens.end(),
ptr.reference_tokens.begin(),
ptr.reference_tokens.end());
@@ -24653,6 +24717,225 @@ class json_pointer
return object_member_impl(object, key, detail::priority_tag<1> {});
}
/// @brief append a null value to @a array, like operator[] with the size of @a array
template<typename BasicJsonType>
static BasicJsonType& array_append(BasicJsonType& array)
{
auto& elements = *array.m_data.m_value.array;
const auto old_capacity = array.container_capacity();
elements.emplace_back();
return array.set_parent_after_insert(elements.back(), old_capacity);
}
/// @brief add @a key, which is missing, to @a object
template<typename BasicJsonType>
static auto object_add_impl(BasicJsonType& object, const string_t& key, detail::priority_tag<1> /*unused*/)
-> decltype(object.m_data.m_value.object->emplace(key, nullptr), std::declval<BasicJsonType&>())
{
const auto old_capacity = object.container_capacity();
const auto it = object.m_data.m_value.object->emplace(key, nullptr).first;
return object.set_parent_after_insert(it->second, old_capacity);
}
template<typename BasicJsonType>
static BasicJsonType& object_add_impl(BasicJsonType& object, const string_t& key, detail::priority_tag<0> /*unused*/)
{
return object.operator[](key);
}
template<typename BasicJsonType>
static BasicJsonType& object_add(BasicJsonType& object, const string_t& key)
{
return object_add_impl(object, key, detail::priority_tag<1> {});
}
/// @brief search @a key in @a members: the result is where the key is or,
/// for objects that can insert at a position (std::map), where it belongs
template<typename ObjectType>
static auto object_search_impl(ObjectType& members, const string_t& key, detail::priority_tag<1> /*unused*/)
-> decltype(members.key_comp()(key, members.lower_bound(key)->first), members.lower_bound(key))
{
return members.lower_bound(key);
}
template<typename ObjectType>
static auto object_search_impl(ObjectType& members, const string_t& key, detail::priority_tag<0> /*unused*/)
-> decltype(members.find(key))
{
return members.find(key);
}
template<typename ObjectType>
static auto object_search(ObjectType& members, const string_t& key)
-> decltype(object_search_impl(members, key, detail::priority_tag<1> {}))
{
return object_search_impl(members, key, detail::priority_tag<1> {});
}
/// @brief whether @a it, the result of @ref object_search for @a key, points to @a key
///
/// With std::less, the key at the lower bound is equivalent to @a key
/// exactly if it is equal to it, and comparing for equality is cheaper.
template<typename ObjectType, typename Iterator,
detail::enable_if_t<detail::finds_equal_keys<ObjectType, string_t>::value, int> = 0>
static auto object_search_found_impl(ObjectType& members, const Iterator& it, const string_t& key, detail::priority_tag<2> /*unused*/)
-> decltype(members.lower_bound(key), it->first == key)
{
return it != members.end() && it->first == key;
}
template<typename ObjectType, typename Iterator>
static auto object_search_found_impl(ObjectType& members, const Iterator& it, const string_t& key, detail::priority_tag<1> /*unused*/)
-> decltype(members.key_comp()(key, it->first), members.lower_bound(key), bool())
{
return it != members.end() && !members.key_comp()(key, it->first);
}
template<typename ObjectType, typename Iterator>
static bool object_search_found_impl(ObjectType& members, const Iterator& it, const string_t& /*key*/, detail::priority_tag<0> /*unused*/)
{
return it != members.end();
}
template<typename ObjectType, typename Iterator>
static bool object_search_found(ObjectType& members, const Iterator& it, const string_t& key)
{
return object_search_found_impl(members, it, key, detail::priority_tag<2> {});
}
/// @brief insert @a key, which is missing, into @a object at @a it, the
/// result of @ref object_search for @a key
template<typename BasicJsonType, typename Iterator>
static auto object_insert_impl(BasicJsonType& object, const Iterator& it, const string_t& key, detail::priority_tag<1> /*unused*/)
-> decltype(object.m_data.m_value.object->emplace_hint(it, key, nullptr), std::declval<BasicJsonType&>())
{
const auto old_capacity = object.container_capacity();
const auto pos = object.m_data.m_value.object->emplace_hint(it, key, nullptr);
return object.set_parent_after_insert(pos->second, old_capacity);
}
template<typename BasicJsonType, typename Iterator>
static BasicJsonType& object_insert_impl(BasicJsonType& object, const Iterator& /*it*/, const string_t& key, detail::priority_tag<0> /*unused*/)
{
return object_add(object, key);
}
template<typename BasicJsonType, typename Iterator>
static BasicJsonType& object_insert(BasicJsonType& object, const Iterator& it, const string_t& key)
{
return object_insert_impl(object, it, key, detail::priority_tag<1> {});
}
/*!
@brief undoes the change @ref get_unchecked_create made to an existing value
@ref get_unchecked walks existing values without changing them. At the
first reference token that does not select an existing value, @ref
get_unchecked_create makes exactly one change to an existing value: it
converts a null value, inserts a key into an object, or grows an array.
Everything it creates afterwards lies below that change. If a later step
throws, the destructor of this guard reverts that one change, which
removes everything created after it, so the value is left as it was
(strong exception safety). The guard needs no try/catch and therefore also
compiles with JSON_NOEXCEPTION.
Reverting the change must not throw, as it happens in a destructor. It
erases the inserted member by its iterator (no key is converted or
compared, nothing is allocated), erases the added elements at the end of
the array, or swaps the null value back. This holds for std::map,
std::vector, ordered_map, and every object_t, array_t, and
json_base_class_t whose erase() and move operations do not throw.
Between recording and reverting, nothing is inserted into or erased from
the object or array, so the iterator stays valid.
*/
template<typename BasicJsonType>
class get_unchecked_rollback
{
public:
using object_iterator = typename BasicJsonType::object_t::iterator;
/// record that @a member was inserted into the object @a value
get_unchecked_rollback(BasicJsonType* value, const object_iterator& member)
: m_value(value)
, m_member(member)
, m_change(change::key_inserted)
{}
/// record the change that the next reference token makes to the null value or array @a value
explicit get_unchecked_rollback(BasicJsonType* value)
: m_value(value)
{
if (value->is_null())
{
// keep the null value (with its base class and positions) and
// continue with a fresh null value in its place, which is what
// assigning an array or object to it would have left there
m_saved_null.swap(*value);
m_change = change::null_converted;
}
else
{
JSON_ASSERT(value->is_array());
m_old_size = value->m_data.m_value.array->size();
m_change = change::array_grown;
}
}
get_unchecked_rollback(const get_unchecked_rollback&) = delete;
get_unchecked_rollback(get_unchecked_rollback&&) = delete;
get_unchecked_rollback& operator=(const get_unchecked_rollback&) = delete;
get_unchecked_rollback& operator=(get_unchecked_rollback&&) = delete;
~get_unchecked_rollback()
{
undo();
}
/// keep the change
void commit() noexcept
{
m_change = change::none;
}
private:
void undo() noexcept
{
switch (m_change)
{
case change::null_converted:
// the values created since go away with m_saved_null
m_value->swap(m_saved_null);
break;
case change::key_inserted:
m_value->m_data.m_value.object->erase(m_member);
break;
case change::array_grown:
{
auto& elements = *m_value->m_data.m_value.array;
elements.resize(m_old_size);
break;
}
case change::none:
default:
break;
}
}
enum class change
{
none,
null_converted,
key_inserted,
array_grown
};
BasicJsonType* m_value;
object_iterator m_member {};
typename BasicJsonType::size_type m_old_size = 0;
change m_change = change::none;
BasicJsonType m_saved_null {};
};
JSON_PRIVATE_UNLESS_TESTED:
json_pointer top() const
{
@@ -25036,47 +25319,245 @@ class json_pointer
Complexity: Linear in the length of the JSON pointer.
@throw parse_error.106 if an array index begins with '0'
@throw parse_error.109 if an array index was not a number
@throw out_of_range.404 if the JSON pointer can not be resolved
Exception safety: Strong guarantee. This function walks existing values
until the first reference token that does not select one. If that is the
last reference token, inserting the key into the object or growing the
array is strongly exception safe by itself; any other change is made by
@ref get_unchecked_create, which undoes it if a later step throws.
@throw parse_error.106 if an array index of an existing array begins with '0'
@throw parse_error.109 if an array index of an existing array was not a number
@throw out_of_range.404 if the JSON pointer can not be resolved, or if an
array index is too large for strtoull
@throw out_of_range.410 if an array index exceeds size_type
@throw std::length_error if padding an array exceeds its maximal size
@throw std::bad_alloc if an allocation fails
*/
template<typename BasicJsonType>
BasicJsonType& get_unchecked(BasicJsonType* ptr) const
{
for (const auto& reference_token : reference_tokens)
const auto* token = reference_tokens.data();
const auto* const end = token + reference_tokens.size();
if (token == end)
{
return *ptr;
}
if (ptr->is_null())
{
// the null value is converted to an array or an object
return get_unchecked_create(ptr, token, end);
}
// walk existing values; the walk itself inserts nothing, which keeps
// the loop fast (insertions are done below or in get_unchecked_create)
typename BasicJsonType::object_t::iterator it;
do
{
if (ptr->is_object())
{
auto& members = *ptr->m_data.m_value.object;
it = object_search(members, *token);
if (!object_search_found(members, it, *token))
{
// the key is inserted below or in get_unchecked_create
break;
}
ptr = &it->second;
}
else if (ptr->is_array())
{
if (is_dash(*token))
{
if (token + 1 == end)
{
// explicitly treat "-" as index beyond the end
return array_append(*ptr);
}
// an element will be appended in get_unchecked_create
break;
}
if (token + 1 == end)
{
// convert array index to number; unchecked access
return ptr->operator[](array_index<BasicJsonType>(*token));
}
const auto idx = array_index<BasicJsonType>(*token);
if (idx >= ptr->m_data.m_value.array->size())
{
// the array will be padded in get_unchecked_create
break;
}
ptr = &(*ptr->m_data.m_value.array)[idx];
}
else if (ptr->is_null())
{
// the null value will be converted in get_unchecked_create
break;
}
else
{
// primitive value with a reference token left
throw_unresolved(*token, ptr);
}
}
while (++token != end);
if (token == end)
{
return *ptr;
}
if (token + 1 == end && ptr->is_object())
{
// insert the missing key where the walk found it belongs
return object_insert(*ptr, it, *token);
}
return get_unchecked_create(ptr, token, end);
}
/*!
@brief create the values for the reference tokens from @a first on
Continues @ref get_unchecked at the first reference token that does not
select an existing value. That token changes @a ptr, an existing value (see
@ref get_unchecked_rollback); all later tokens only change values created
here.
If the emplace() of the object type finds a member that the lookup in @ref
get_unchecked did not find (the two disagree, e.g. for fifo_map), nothing
is inserted, and the walk continues with that member as in @ref
get_unchecked, which is what basic_json::operator[] does.
@param[in] ptr the value the reference token @a first is applied to
@param[in] first the first reference token to resolve (an element of
@ref reference_tokens)
@param[in] end the end of @ref reference_tokens
@throw parse_error.106 if an array index of an existing array begins with
'0' (only after emplace() found a member, see above)
@throw parse_error.109 if an array index of an existing array was not a
number (only after emplace() found a member)
@throw out_of_range.404 if an array index is too large for strtoull, or (only
after emplace() found a member) if the JSON pointer
can not be resolved
@throw out_of_range.410 if an array index exceeds size_type
@throw std::length_error if padding an array exceeds its maximal size
@throw std::bad_alloc if an allocation fails
*/
template<typename BasicJsonType>
JSON_HEDLEY_NEVER_INLINE static BasicJsonType& get_unchecked_create(BasicJsonType* ptr, const string_t* first, const string_t* end)
{
const auto* token = first;
while (ptr->is_object())
{
const auto old_capacity = ptr->container_capacity();
const auto inserted = ptr->m_data.m_value.object->emplace(*token, nullptr);
if (JSON_HEDLEY_LIKELY(inserted.second))
{
get_unchecked_rollback<BasicJsonType> rollback(ptr, inserted.first);
BasicJsonType& result = create_values(&ptr->set_parent_after_insert(inserted.first->second, old_capacity), token + 1, end);
rollback.commit();
return result;
}
// nothing was inserted: walk existing values from the member
// emplace() found on, like get_unchecked
ptr = &inserted.first->second;
while (++token != end)
{
if (ptr->is_object())
{
auto& members = *ptr->m_data.m_value.object;
const auto it = object_search(members, *token);
if (!object_search_found(members, it, *token))
{
break;
}
ptr = &it->second;
}
else if (ptr->is_array())
{
if (is_dash(*token))
{
break;
}
const auto idx = array_index<BasicJsonType>(*token);
if (idx >= ptr->m_data.m_value.array->size())
{
break;
}
ptr = &(*ptr->m_data.m_value.array)[idx];
}
else if (ptr->is_null())
{
break;
}
else
{
// primitive value with a reference token left
throw_unresolved(*token, ptr);
}
}
if (token == end)
{
return *ptr;
}
}
get_unchecked_rollback<BasicJsonType> rollback(ptr);
BasicJsonType& result = create_values(ptr, token, end);
rollback.commit();
return result;
}
/*!
@brief the loop of @ref get_unchecked_create
@a ptr is a null value or an array. Every value created here is a fresh
one, so every object a key is added to is empty.
@throw out_of_range.404 if an array index is too large for strtoull
@throw out_of_range.410 if an array index exceeds size_type
@throw std::length_error if padding an array exceeds its maximal size
@throw std::bad_alloc if an allocation fails
*/
template<typename BasicJsonType>
static BasicJsonType& create_values(BasicJsonType* ptr, const string_t* first, const string_t* end)
{
for (const auto* token = first; token != end; ++token)
{
const auto& reference_token = *token;
// convert null values to arrays or objects before continuing
if (ptr->is_null())
{
// check if the reference token is a valid array index, that is
// a nonempty sequence of digits without a leading '0'
// (cf. RFC 6901, Sect. 4); tokens that could never be a valid
// array index (such as "01" or "") are treated as object keys
const bool nums = !reference_token.empty()
&& (reference_token.size() == 1 || reference_token[0] != '0')
&& std::all_of(reference_token.begin(), reference_token.end(),
[](const unsigned char x)
// change value to an array for array indices or "-" or to an
// object otherwise; every null value here is a fresh one (see
// get_unchecked_rollback), so converting it in place leaves
// the same value as assigning an array or object to it
if (is_array_index_token(reference_token) || is_dash(reference_token))
{
return std::isdigit(x);
});
// change value to an array for array indices or "-" or to object otherwise
*ptr = (nums || reference_token == "-")
? detail::value_t::array
: detail::value_t::object;
ptr->template convert_null_to<typename BasicJsonType::array_t>();
}
else
{
ptr->template convert_null_to<typename BasicJsonType::object_t>();
}
}
if (ptr->is_object())
{
// use unchecked object access
ptr = &object_member(*ptr, reference_token);
// the object is new and empty
ptr = &object_add(*ptr, reference_token);
}
else if (ptr->is_array())
else
{
if (reference_token == "-")
// only the first value is not created here, and it is a null
// value or an array (see get_unchecked_create)
JSON_ASSERT(ptr->is_array());
if (is_dash(reference_token))
{
// explicitly treat "-" as index beyond the end
ptr = &ptr->operator[](ptr->m_data.m_value.array->size());
ptr = &array_append(*ptr);
}
else
{
@@ -25084,16 +25565,31 @@ class json_pointer
ptr = &ptr->operator[](array_index<BasicJsonType>(reference_token));
}
}
else
{
// primitive value with a reference token left
throw_unresolved(reference_token, ptr);
}
}
return *ptr;
}
/// whether @a s is a valid array index, that is a nonempty sequence of
/// digits without a leading '0' (cf. RFC 6901, Sect. 4); tokens that could
/// never be a valid array index (such as "01" or "") are object keys
static bool is_array_index_token(const string_t& s)
{
const std::size_t size = s.size();
if (size == 0 || (size > 1 && s[0] == '0'))
{
return false;
}
for (std::size_t i = 0; i < size; ++i)
{
if (std::isdigit(static_cast<unsigned char>(s[i])) == 0)
{
return false;
}
}
return true;
}
/*!
@throw parse_error.106 if an array index begins with '0'
@throw parse_error.109 if an array index was not a number
@@ -32013,6 +32509,11 @@ NLOHMANN_JSON_NAMESPACE_END
#include <type_traits> // conditional
// The shortest-digit conversion of doubles (see zmij.hpp) and the writing of
// its digits, including the conversion of the digits to text by Xiang JunBo
// and the SIMD instruction sequence by Dougall Johnson, are ported from Zmij
// by Victor Zverovich (https://github.com/vitaut/zmij, MIT license).
//
// SSE2 (every x86-64 CPU) and NEON (every 64-bit Arm CPU) convert the 16
// digits of a double at once (unless JSON_NO_SIMD selects the portable code)
#if !defined(JSON_NO_SIMD) && (defined(__x86_64__) || (defined(_M_X64) && !defined(_M_ARM64EC)))
@@ -36454,7 +36955,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
friend class ::nlohmann::detail::binary_writer;
template<typename BasicJsonType, typename InputType, typename SAX>
friend class ::nlohmann::detail::binary_reader;
template<typename BasicJsonType, typename InputAdapterType, bool Collect>
template<typename BasicJsonType, typename InputAdapterType, bool Collect, typename LexerType>
friend class ::nlohmann::detail::json_sax_dom_parser;
template<typename BasicJsonType, typename InputAdapterType>
friend class ::nlohmann::detail::json_sax_dom_binary_parser;
@@ -43245,6 +43746,25 @@ public:
/// @sa https://json.nlohmann.me/api/basic_json/patch/
void patch_inplace(const basic_json& json_patch)
{
// the operations below change *this while iterating json_patch: apply
// a copy when a value is applied as patch to itself or, with
// JSON_DIAGNOSTICS, to a value stored inside it
// NOLINTNEXTLINE(misc-const-correctness): only the JSON_DIAGNOSTICS build modifies it
bool patch_aliases_this = (&json_patch == this);
#if JSON_DIAGNOSTICS
for (const basic_json* p = json_patch.m_parent; !patch_aliases_this && p != nullptr; p = p->m_parent)
{
patch_aliases_this = (p == this);
}
#endif
if (JSON_HEDLEY_UNLIKELY(patch_aliases_this))
{
// NOLINTNEXTLINE(performance-unnecessary-copy-initialization): *this is modified below
const basic_json json_patch_copy = json_patch;
patch_inplace(json_patch_copy);
return;
}
basic_json& result = *this;
// the valid JSON Patch operations
enum class patch_operations {add, remove, replace, move, copy, test, invalid};
+1761 -1023
View File
File diff suppressed because it is too large. Load diff
+27 -1
View File
@@ -21,6 +21,10 @@ python3 tests/benchmarks/json_view/compare.py --data <json_test_data directory>
- The other libraries come from the system: pkg-config, or Homebrew (`brew install yyjson simdjson boost`). With
`--download`, pinned releases are downloaded instead and checked against their SHA-256. Without Boost headers (or
with `--no-boost`), the Boost.JSON columns are skipped, and the results say so.
- `--aligned` additionally builds and runs everything with all functions aligned to 64 bytes (`-mllvm
-align-all-functions=6` for Clang, `-falign-functions=64 -falign-loops=64` for GCC), and reports both result
tables as "default build" and "aligned build". Code layout alone can move a result by a few percent; a difference
that appears in only one of the two tables is a layout artifact. Off by default, which doubles the run time.
- `--corpus file...` adds files to the corpus benchmark, e.g. those of
[simdjson-data](https://github.com/simdjson/simdjson-data) or the
[yyjson benchmark](https://github.com/ibireme/yyjson_benchmark).
@@ -39,7 +43,7 @@ where `json_view` stands on another architecture; they are not meant for publica
## What is measured
`bench_view.cpp` runs four workloads on twitter, citm_catalog, canada, jeopardy, a single tweet (`status`), and a
`bench_view.cpp` runs these workloads on twitter, citm_catalog, canada, jeopardy, a single tweet (`status`), and a
JSON-RPC request (`rpc`):
| workload | what it does |
@@ -48,6 +52,7 @@ JSON-RPC request (`rpc`):
| traverse | parse, then visit every value, convert every number, touch every string and key |
| select | parse, then read a few fields per record (e.g. id, user name, and retweet count of each tweet) |
| dump | serialize a parsed document (compact) |
| parse+dump | build a fresh document, then serialize it (compact), in one timed call |
`bench_corpus.cpp` runs parse, traverse, and dump on any list of files, so that no library is tuned to a handful of
documents. Its dump also writes the numbers as they are in the input: `json_view` with `number_format::source`, and
@@ -72,11 +77,32 @@ call follows an untimed call of the same engine: otherwise the engine after `jso
cleaning up the tens of thousands of nodes `json::parse` just freed (with glibc, this made `json_view` look 1.7 times
slower on citm_catalog traverse).
`json_view` is an index into the source text, not a tree of owning nodes. Its like-for-like peers are yyjson and
simdjson DOM (immutable, indexed, no per-node allocation). Boost.JSON and `json::parse` build mutable, owning DOMs and
are reference points for what that costs, not equals; simdjson On-Demand is lazy and forward-only, so it is the other
way round: it has no random access to pay for. All engines validate UTF-8 and syntax by default.
Number conversion differs: `json_view` converts integers while parsing but converts floating-point numbers lazily,
when they are read or dumped; yyjson (default flags) and simdjson DOM convert every number while parsing. The cost of
float conversion therefore appears in the traverse and dump rows for `json_view`, and in the parse row for the others.
For float-heavy documents (canada), the fair reading is a sum of parse and use: the `traverse` workload already
includes the parse, and `parse+dump` measures a fresh document and its dump in one timed call; the `dump` row alone
leaves the conversion in `json_view` and out of the others. Eager conversion at parse time was prototyped as an opt-in
and measured: canada traverse (with its parse) was 4-7 % faster, but parse+dump 11-30 % slower and parse 36-90 % slower. It was
dropped, so lazy conversion is the only mode.
`json_view` can also be edited: `json_editable_document` has the same read API, and the `(editable)` rows in
`bench_view` measure it on the same workloads. They are the fair counterpart of Boost.JSON and `json::parse`, which are
mutable too; the plain `json_view` rows are the counterpart of yyjson and simdjson DOM, which are not. `bench_edit` adds
the read-modify-write workloads. Read the tables with that in mind: an immutable-only library is not compared with an
editable `json_view` unless it says so.
The engines do not all offer the same features, which the numbers should be read with:
| engine | document | random access | editable | notes |
|---|---|---|---|---|
| `json_view` | immutable index into the text | yes | no | a fresh document per parse; "reused" parses into the same document |
| `json_view` (editable) | index into the text plus edits | yes | yes, in place | `json_editable_document`; keeps member order and number spellings |
| yyjson | immutable (`yyjson_read`) | yes | via a mutable copy | |
| simdjson DOM | immutable, parser reused | yes | no | "fresh" uses a new parser per parse |
| simdjson On-Demand | none: forward-only, lazy | no | no | only traverse and select |
+92 -24
View File
@@ -8,8 +8,9 @@
// Same-feature-set benchmark: read-only JSON documents with random access.
//
// json_view nlohmann/json_view.hpp (fresh document per parse / reused)
// yyjson yyjson_read(): immutable document, random access
// json_view nlohmann/json_view.hpp (fresh document per parse / reused;
// "(editable)": json_editable_document, same read API)
// yyjson yyjson_read(): immutable document, random access
// simdjson DOM dom::parser (reused, as recommended; "fresh": a new parser
// per parse): immutable, random access
// references (different feature sets):
@@ -19,7 +20,11 @@
//
// Workloads: parse (build + free), traverse (visit everything, convert every
// number, touch every string and key), select (a few fields per document),
// dump (compact serialization of the parsed document).
// dump (compact serialization of the parsed document), and parse+dump (a fresh
// document, then its dump, in one timed call). json_view converts floating-point
// numbers lazily (when read or dumped), yyjson and simdjson DOM while parsing:
// for float-heavy documents, traverse (which includes the parse) and parse+dump
// are the fair reading of the parse and dump rows.
// All engines run interleaved in every round, each timed call after an untimed
// one of the same engine; the best round is reported.
#include <nlohmann/json_view.hpp>
@@ -43,6 +48,8 @@
using nlohmann::json;
using nlohmann::json_document;
using nlohmann::json_editable_document;
using nlohmann::json_editable_view;
using nlohmann::json_view;
static volatile double g_sink;
@@ -55,7 +62,8 @@ struct stats
// ---------------- traversal ----------------
static void walk(json_view v, stats& st)
template<class V>
static void walk_view(V v, stats& st)
{
++st.nodes;
switch (v.type())
@@ -64,32 +72,42 @@ static void walk(json_view v, stats& st)
for (auto it = v.begin(); it != v.end(); ++it)
{
st.str += it.key().size();
walk(*it, st);
walk_view<V>(*it, st);
}
break;
case json::value_t::array:
for (const json_view e : v)
for (const V e : v)
{
walk(e, st);
walk_view<V>(e, st);
}
break;
case json::value_t::string:
st.str += v.get_string().size();
break;
case json::value_t::number_integer:
st.num += static_cast<double>(v.get<std::int64_t>());
st.num += static_cast<double>(v.template get<std::int64_t>());
break;
case json::value_t::number_unsigned:
st.num += static_cast<double>(v.get<std::uint64_t>());
st.num += static_cast<double>(v.template get<std::uint64_t>());
break;
case json::value_t::number_float:
st.num += v.get<double>();
st.num += v.template get<double>();
break;
default:
break;
}
}
static void walk(json_view v, stats& st)
{
walk_view<json_view>(v, st);
}
static void walk(json_editable_view v, stats& st)
{
walk_view<json_editable_view>(v, st);
}
static void walk(const json& j, stats& st)
{
++st.nodes;
@@ -300,20 +318,21 @@ static void walk(const boost::json::value& v, stats& st)
// status (one tweet): id, user.screen_name, retweet_count
// rpc: method, params.minuend, id
static double pick(const std::string& name, json_view r)
template<class V>
static double pick_view(const std::string& name, V r)
{
double acc = 0;
if (name == "twitter" || name == "status")
{
auto one = [&](json_view s)
auto one = [&](V s)
{
acc += static_cast<double>(s["id"].get<std::uint64_t>());
acc += static_cast<double>(s["id"].template get<std::uint64_t>());
acc += static_cast<double>(s["user"]["screen_name"].get_string().size());
acc += static_cast<double>(s["retweet_count"].get<std::int64_t>());
acc += static_cast<double>(s["retweet_count"].template get<std::int64_t>());
};
if (name == "twitter")
{
for (const json_view s : r["statuses"])
for (const V s : r["statuses"])
{
one(s);
}
@@ -325,20 +344,20 @@ static double pick(const std::string& name, json_view r)
}
else if (name == "citm_catalog")
{
for (const json_view p : r["performances"])
for (const V p : r["performances"])
{
acc += static_cast<double>(p["id"].get<std::uint64_t>() + p["eventId"].get<std::uint64_t>() + p["seatCategories"].size());
acc += static_cast<double>(p["id"].template get<std::uint64_t>() + p["eventId"].template get<std::uint64_t>() + p["seatCategories"].size());
}
acc += static_cast<double>(r["events"].size());
}
else if (name == "canada")
{
const json_view g = r["features"][0]["geometry"];
const V g = r["features"][0]["geometry"];
acc += static_cast<double>(r["type"].get_string().size() + g["type"].get_string().size() + g["coordinates"].size());
}
else if (name == "jeopardy")
{
for (const json_view q : r)
for (const V q : r)
{
acc += q["round"].get_string() == "Final Jeopardy!" ? 1 : 0;
acc += static_cast<double>(q["category"].get_string().size());
@@ -347,11 +366,21 @@ static double pick(const std::string& name, json_view r)
else if (name == "rpc")
{
acc += static_cast<double>(r["method"].get_string().size());
acc += static_cast<double>(r["params"]["minuend"].get<std::int64_t>() + r["id"].get<std::int64_t>());
acc += static_cast<double>(r["params"]["minuend"].template get<std::int64_t>() + r["id"].template get<std::int64_t>());
}
return acc;
}
static double pick(const std::string& name, json_view r)
{
return pick_view<json_view>(name, r);
}
static double pick(const std::string& name, json_editable_view r)
{
return pick_view<json_editable_view>(name, r);
}
static double pick(const std::string& name, const json& r)
{
double acc = 0;
@@ -634,15 +663,35 @@ int main(int argc, char** argv)
walk(json::parse(dc.text), a);
auto d = json_document::parse(dc.text);
walk(d.root(), b);
auto ed = json_editable_document::parse(dc.text);
stats e;
walk(ed.root(), e);
yyjson_doc* y = yyjson_read(dc.text.data(), dc.text.size(), 0);
walk(yyjson_doc_get_root(y), c);
simdjson::dom::parser p;
walk(p.parse(dc.text).value(), dd);
const bool ok = a.nodes == b.nodes && a.nodes == c.nodes && a.nodes == dd.nodes && a.str == b.str && a.str == c.str && a.str == dd.str
const bool ok = a.nodes == b.nodes && a.nodes == e.nodes && a.str == e.str && std::fabs(a.num - e.num) <= 1e-9 * std::fabs(a.num) && a.nodes == c.nodes && a.nodes == dd.nodes && a.str == b.str && a.str == c.str && a.str == dd.str
&& std::fabs(a.num - b.num) <= 1e-9 * std::fabs(a.num) && std::fabs(a.num - c.num) <= 1e-9 * std::fabs(a.num);
const double pa = pick(dc.name, json::parse(dc.text)), pb = pick(dc.name, d.root()), pc = pick(dc.name, yyjson_doc_get_root(y)), pd = pick(dc.name, p.parse(dc.text).value());
std::printf("check %-13s traverse %s select %s\n", dc.name.c_str(), ok ? "OK" : "MISMATCH", (pa == pb && pa == pc && pa == pd) ? "OK" : "MISMATCH");
const double pa = pick(dc.name, json::parse(dc.text)), pb = pick(dc.name, d.root()), pe = pick(dc.name, ed.root()), pc = pick(dc.name, yyjson_doc_get_root(y)), pd = pick(dc.name, p.parse(dc.text).value());
std::printf("check %-13s traverse %s select %s\n", dc.name.c_str(), ok ? "OK" : "MISMATCH", (pa == pb && pa == pe && pa == pc && pa == pd) ? "OK" : "MISMATCH");
yyjson_doc_free(y);
// the dumps of the engines may format floats differently, so only: every dump is non-empty, and the
// dump of json_view (also editable) and of json::parse parse back to the same number of nodes
const std::string dj = json::parse(dc.text).dump();
const std::string dv = d.root().dump();
const std::string de = ed.root().dump();
std::size_t yn = 0;
yyjson_doc* y2 = yyjson_read(dc.text.data(), dc.text.size(), 0);
char* yo = yyjson_write(y2, 0, &yn);
const bool dumps_ok = !dj.empty() && !dv.empty() && !de.empty() && yo != nullptr && yn > 0 && !simdjson::to_string(p.parse(dc.text).value()).empty();
std::free(yo);
yyjson_doc_free(y2);
stats sj_nodes, sv_nodes, se_nodes;
walk(json::parse(dj), sj_nodes);
walk(json::parse(dv), sv_nodes);
walk(json::parse(de), se_nodes);
std::printf("check %-13s dump %s\n", dc.name.c_str(), (dumps_ok && sj_nodes.nodes == a.nodes && sv_nodes.nodes == a.nodes && se_nodes.nodes == a.nodes) ? "OK" : "MISMATCH");
}
std::FILE* csv = std::fopen("bench_view.csv", "w");
@@ -669,6 +718,7 @@ int main(int argc, char** argv)
workloads.push_back({"parse", {
{"json_view", [&] { auto d = json_document::parse(s); g_sink = static_cast<double>(d.node_count()); }},
{"json_view (reused)", [&] { reused.read(s); g_sink = static_cast<double>(reused.node_count()); }},
{"json_view (editable)", [&] { auto d = json_editable_document::parse(s); g_sink = static_cast<double>(d.root().size()); }},
{"yyjson", [&] { yyjson_doc* d = yyjson_read(s.data(), s.size(), 0); g_sink = static_cast<double>(yyjson_doc_get_val_count(d)); yyjson_doc_free(d); }},
{"simdjson DOM", [&] { auto e = sj.parse(ps).value_unsafe(); g_sink = e.is_object(); }},
{"simdjson DOM (fresh)", [&] { simdjson::dom::parser p; auto e = p.parse(ps).value_unsafe(); g_sink = e.is_object(); }},
@@ -680,6 +730,7 @@ int main(int argc, char** argv)
workloads.push_back({"traverse", {
{"json_view", [&] { auto d = json_document::parse(s); stats st; walk(d.root(), st); g_sink = st.num; }},
{"json_view (reused)", [&] { reused.read(s); stats st; walk(reused.root(), st); g_sink = st.num; }},
{"json_view (editable)", [&] { auto d = json_editable_document::parse(s); stats st; walk(d.root(), st); g_sink = st.num; }},
{"yyjson", [&] { yyjson_doc* d = yyjson_read(s.data(), s.size(), 0); stats st; walk(yyjson_doc_get_root(d), st); g_sink = st.num; yyjson_doc_free(d); }},
{"simdjson DOM", [&] { stats st; walk(sj.parse(ps).value_unsafe(), st); g_sink = st.num; }},
{"simdjson OD", [&] { auto d = od.iterate(ps).value_unsafe(); stats st; walk_od(d.get_value().value_unsafe(), st); g_sink = st.num; }},
@@ -691,15 +742,31 @@ int main(int argc, char** argv)
workloads.push_back({"select", {
{"json_view", [&] { auto d = json_document::parse(s); g_sink = pick(name, d.root()); }},
{"json_view (reused)", [&] { reused.read(s); g_sink = pick(name, reused.root()); }},
{"json_view (editable)", [&] { auto d = json_editable_document::parse(s); g_sink = pick(name, d.root()); }},
{"yyjson", [&] { yyjson_doc* d = yyjson_read(s.data(), s.size(), 0); g_sink = pick(name, yyjson_doc_get_root(d)); yyjson_doc_free(d); }},
{"simdjson DOM", [&] { g_sink = pick(name, sj.parse(ps).value_unsafe()); }},
{"simdjson OD", [&] { auto d = od.iterate(ps).value_unsafe(); g_sink = pick_od(name, d); }},
{"json::parse", [&] { json j = json::parse(s); g_sink = pick(name, j); }},
}});
// pipeline: a fresh document, then a dump of it, in one timed call
// (simdjson DOM reuses its parser, as in the parse row); the traverse
// workload above is already parse+traverse
workloads.push_back({"parse+dump", {
{"json_view", [&] { auto d = json_document::parse(s); std::string o = d.root().dump(); g_sink = static_cast<double>(o.size()); }},
{"json_view (editable)", [&] { auto d = json_editable_document::parse(s); std::string o = d.root().dump(); g_sink = static_cast<double>(o.size()); }},
{"yyjson", [&] { yyjson_doc* d = yyjson_read(s.data(), s.size(), 0); std::size_t n = 0; char* o = yyjson_write(d, 0, &n); g_sink = static_cast<double>(n); std::free(o); yyjson_doc_free(d); }},
{"simdjson DOM", [&] { std::string o = simdjson::to_string(sj.parse(ps).value_unsafe()); g_sink = static_cast<double>(o.size()); }},
#if JSON_VIEW_BENCH_BOOST
{"Boost.JSON", [&] { boost::json::monotonic_resource mr; auto v = boost::json::parse(s, &mr); std::string o = boost::json::serialize(v); g_sink = static_cast<double>(o.size()); }},
#endif
{"json::parse", [&] { json j = json::parse(s); std::string o = j.dump(); g_sink = static_cast<double>(o.size()); }},
}});
{
// serialization of an already parsed document
static json_document vd;
vd.read(s);
static json_editable_document ed = json_editable_document::parse(s);
ed = json_editable_document::parse(s);
static yyjson_doc* yd = nullptr;
if (yd)
{
@@ -713,6 +780,7 @@ int main(int argc, char** argv)
jd = json::parse(s);
workloads.push_back({"dump", {
{"json_view", [&] { std::string o = vd.root().dump(); g_sink = static_cast<double>(o.size()); }},
{"json_view (editable)", [&] { std::string o = ed.root().dump(); g_sink = static_cast<double>(o.size()); }},
{"yyjson", [&] { std::size_t n = 0; char* o = yyjson_write(yd, 0, &n); g_sink = static_cast<double>(n); std::free(o); }},
{"simdjson DOM", [&] { std::string o = simdjson::to_string(se); g_sink = static_cast<double>(o.size()); }},
{"json::parse", [&] { std::string o = jd.dump(); g_sink = static_cast<double>(o.size()); }},
@@ -740,7 +808,7 @@ int main(int argc, char** argv)
}
}
const double ref = best[0];
std::printf("%-13s %-9s", dc.name.c_str(), wl.first.c_str());
std::printf("%-13s %-14s", dc.name.c_str(), wl.first.c_str());
for (std::size_t k = 0; k < wl.second.size(); ++k)
{
const double us = best[k] / 1e3;
+47 -25
View File
@@ -239,6 +239,8 @@ def main():
ap.add_argument('--download', action='store_true', help='use pinned downloads instead of system libraries')
ap.add_argument('--no-boost', action='store_true', help='skip Boost.JSON')
ap.add_argument('--native', action='store_true', help='compile for this CPU (-march=native / -mcpu=native)')
ap.add_argument('--aligned', action='store_true',
help='also build and run with all functions (and loops) aligned, to make code layout artifacts visible')
ap.add_argument('--rounds', type=int, default=30, help='rounds of bench_view (default: 30)')
ap.add_argument('--corpus', nargs='*', default=[], help='more files for bench_corpus')
ap.add_argument('--build-dir', default=os.path.join(HERE, 'build'), help='where to build (default: build/ next to this script)')
@@ -282,32 +284,46 @@ def main():
+ ['-I' + d for d in lib.include])
objects.append(obj)
binaries = {}
for bench in ['bench_view', 'bench_corpus', 'bench_edit', 'bench_decode']:
exe = os.path.join(args.build_dir, bench)
run(cxx + flags + include + [os.path.join(HERE, bench + '.cpp')] + objects + link + ['-o', exe])
binaries[bench] = exe
cxx_version = output(cxx + ['--version'])
is_clang = 'clang' in cxx_version.lower()
# alignment flags: functions and (gcc) loops on 64-byte boundaries
align_flags = ['-mllvm', '-align-all-functions=6'] if is_clang else ['-falign-functions=64', '-falign-loops=64']
# the builds: the default one and, with --aligned, one with aligned code
variants = [('default build', args.build_dir, [])]
if args.aligned:
variants.append(('aligned build', os.path.join(args.build_dir, 'aligned'), align_flags))
# run: bench_view on its documents, bench_corpus on those and the given files
corpus = [os.path.join(args.data, f) for f in DEFAULT_CORPUS] + args.corpus
outputs = {}
outputs['bench_view'] = run([binaries['bench_view'], args.data, str(args.rounds)], cwd=args.build_dir,
capture_output=True, text=True).stdout
outputs['bench_corpus'] = run([binaries['bench_corpus']] + corpus, cwd=args.build_dir,
capture_output=True, text=True).stdout
outputs['bench_edit'] = run([binaries['bench_edit'], args.data, str(max(1, args.rounds // 2))], cwd=args.build_dir,
capture_output=True, text=True).stdout
outputs['bench_decode'] = run([binaries['bench_decode'], args.data, str(max(1, args.rounds // 2))], cwd=args.build_dir,
capture_output=True, text=True).stdout
for name, text in outputs.items():
print(text)
# label -> {benchmark: output}
all_outputs = {}
for label, build_dir, extra in variants:
os.makedirs(build_dir, exist_ok=True)
binaries = {}
for bench in ['bench_view', 'bench_corpus', 'bench_edit', 'bench_decode']:
exe = os.path.join(build_dir, bench)
run(cxx + flags + extra + include + [os.path.join(HERE, bench + '.cpp')] + objects + link + ['-o', exe])
binaries[bench] = exe
# run: bench_view on its documents, bench_corpus on those and the given files
outputs = {}
outputs['bench_view'] = run([binaries['bench_view'], args.data, str(args.rounds)], cwd=build_dir,
capture_output=True, text=True).stdout
outputs['bench_corpus'] = run([binaries['bench_corpus']] + corpus, cwd=build_dir,
capture_output=True, text=True).stdout
outputs['bench_edit'] = run([binaries['bench_edit'], args.data, str(max(1, args.rounds // 2))], cwd=build_dir,
capture_output=True, text=True).stdout
outputs['bench_decode'] = run([binaries['bench_decode'], args.data, str(max(1, args.rounds // 2))],
cwd=build_dir, capture_output=True, text=True).stdout
for name, text in outputs.items():
print(text)
all_outputs[label] = outputs
# results with their metadata
now = datetime.datetime.now()
host = re.sub(r'[^A-Za-z0-9-]+', '-', platform.node().split('.')[0]) or 'host'
stem = os.path.join(HERE, 'results', f'{now:%Y-%m-%d}-{host}')
os.makedirs(os.path.dirname(stem), exist_ok=True)
cxx_version = output(cxx + ['--version'])
compiler_version = cxx_version.splitlines()[0] if cxx_version else ' '.join(cxx)
meta = [
('date', f'{now:%Y-%m-%d %H:%M}'),
@@ -316,6 +332,7 @@ def main():
('OS', f'{platform.system()} {platform.release()} ({platform.machine()})'),
('compiler', compiler_version),
('flags', ' '.join(flags)),
('aligned results', ('included (flags: ' + ' '.join(align_flags) + ')') if args.aligned else 'not included'),
('yyjson', libs['yyjson'].version),
('simdjson', libs['simdjson'].version),
('Boost.JSON', libs['boost'].version if with_boost else 'skipped (not found)'),
@@ -328,15 +345,20 @@ def main():
f.write('| | |\n|---|---|\n')
for key, value in meta:
f.write(f'| {key} | {value} |\n')
for name, text in outputs.items():
f.write(f'\n## {name}\n\n```\n{text.rstrip()}\n```\n')
for label, outputs in all_outputs.items():
# (without --aligned there is only the default build, and no extra heading level)
prefix = f'{label}: ' if args.aligned else ''
for name, text in outputs.items():
f.write(f'\n## {prefix}{name}\n\n```\n{text.rstrip()}\n```\n')
with open(stem + '.csv', 'w', encoding='utf-8') as out:
out.write(''.join(f'# {key}: {value}\n' for key, value in meta))
for name in ['bench_view', 'bench_corpus', 'bench_edit', 'bench_decode']:
path = os.path.join(args.build_dir, name + '.csv')
if os.path.isfile(path):
with open(path, encoding='utf-8') as f:
out.write(f'# {name}\n' + f.read())
for label, build_dir, _ in variants:
for name in ['bench_view', 'bench_corpus', 'bench_edit', 'bench_decode']:
path = os.path.join(build_dir, name + '.csv')
if os.path.isfile(path):
with open(path, encoding='utf-8') as f:
# (the default build keeps its plain section names)
out.write(f'# {name}' + (f' ({label})' if args.aligned else '') + '\n' + f.read())
print(f'results: {stem}.md, {stem}.csv')
+17
View File
@@ -373,6 +373,23 @@ TEST_CASE("alternative string type")
CHECK_FALSE(j.contains(alt_json::json_pointer("/foo/bar")));
}
SECTION("non-const operator[](json_pointer)")
{
// creating values must compile without string_t::begin()
alt_json j = alt_json::parse(R"({"a":{"b":[1]},"c":null})");
const alt_json before = j;
j[alt_json::json_pointer("/a/b/3/x/-/y")] = 1; // pad, append, create
j[alt_json::json_pointer("/a/z")] = 2; // insert a key
j[alt_json::json_pointer("/c/0/q")] = 3; // convert null
j[alt_json::json_pointer("/a/b/-")] = 4; // append
CHECK(j.dump() == R"({"a":{"b":[1,null,null,{"x":[{"y":1}]},4],"z":2},"c":[{"q":3}]})");
// a failing access leaves the value unchanged
j = before;
CHECK_THROWS_AS(j[alt_json::json_pointer("/a/n/99999999999999999999999")], alt_json::out_of_range&);
CHECK(j == before);
}
SECTION("operator/(std::size_t)")
{
// json_pointer::operator/=(std::size_t) must compile without string_t
+3 -1
View File
@@ -1311,7 +1311,9 @@ TEST_CASE("BSON declared document size bounds the content")
// records the events that a SAX parser receives
struct recorder : nlohmann::json_sax<json>
{
std::string events;
// GCC's -Weffc++ wants the member initialized in the member initialization list
// NOLINTNEXTLINE(readability-redundant-member-init)
std::string events {};
bool null() override
{
+141
View File
@@ -3030,3 +3030,144 @@ TEST_CASE("a json_sax_dom_parser keeps the values read before a parse error")
CHECK(read_before_error<nlohmann::ordered_json>(c.first) == c.second);
}
}
TEST_CASE("object keys are delivered decoded and validated")
{
// json_sax::key and the key event of a parser callback see the decoded name, and
// only after the whole name was scanned and validated
const auto keys_of_format = [](const std::string & text, const json::input_format_t format)
{
SaxEventLogger l;
json::sax_parse(text, &l, format);
std::vector<std::string> keys;
for (const auto& e : l.events)
{
if (e.compare(0, 4, "key(") == 0)
{
keys.push_back(e.substr(4, e.size() - 5));
}
}
return keys;
};
const auto keys_of = [&keys_of_format](const std::string & text)
{
return keys_of_format(text, json::input_format_t::json);
};
const auto callback_keys_of = [](const std::string & text)
{
std::vector<std::string> keys;
const json::parser_callback_t cb = [&keys](int /*depth*/, json::parse_event_t event, json & parsed)
{
if (event == json::parse_event_t::key)
{
keys.push_back(parsed.get<std::string>());
}
return true;
};
const json parsed = json::parse(text, cb, false);
static_cast<void>(parsed);
return keys;
};
SECTION("different spellings of a name yield the same string")
{
const std::string e_acute = "\xc3\xa9";
const std::string text = std::string(R"({"/":1,"\/":2,"\u002f":3,"\u00e9":4,"\u00E9":5,")") + e_acute + R"(":6})";
const std::vector<std::string> expected = {"/", "/", "/", e_acute, e_acute, e_acute};
CHECK(keys_of(text) == expected);
CHECK(callback_keys_of(text) == expected);
}
SECTION("escape sequences are decoded")
{
const std::string simple = R"({"\"\\\/\b\f\n\r\t":1})";
const std::vector<std::string> simple_expected = {std::string("\"\\/\b\f\n\r\t")};
CHECK(keys_of(simple) == simple_expected);
CHECK(callback_keys_of(simple) == simple_expected);
const std::string nul = R"({"\u0000":1})";
const std::vector<std::string> nul_expected = {std::string(1, '\0')};
CHECK(keys_of(nul) == nul_expected);
CHECK(callback_keys_of(nul) == nul_expected);
const std::string pair = R"({"\ud83d\ude00":1})";
const std::vector<std::string> pair_expected = {"\xf0\x9f\x98\x80"};
CHECK(keys_of(pair) == pair_expected);
CHECK(callback_keys_of(pair) == pair_expected);
}
SECTION("no normalization and no case conversion")
{
const std::string text = std::string("{\"\xc3\xa9\":1,\"e\xcc\x81\":2,\"A\":3,\"a\":4}");
const std::vector<std::string> expected = {"\xc3\xa9", "e\xcc\x81", "A", "a"};
CHECK(keys_of(text) == expected);
CHECK(callback_keys_of(text) == expected);
}
SECTION("a name that is not valid is not delivered")
{
const std::vector<std::string> texts =
{
R"({"\u12G4":1})", // malformed escape
R"({"\udc00":1})", // lone low surrogate
R"({"\ud800":1})", // lone high surrogate
R"({"\ud800\u0041":1})", // high surrogate not followed by a low one
R"({"\x":1})", // unknown escape
std::string("{\"a\x01\":1}"), // control character
std::string("{\"\xff\":1}") // ill-formed UTF-8
};
for (const auto& text : texts)
{
CAPTURE(text)
CHECK(!json::accept(text));
CHECK(keys_of(text).empty());
CHECK(callback_keys_of(text).empty());
}
}
SECTION("a name is delivered before the following colon is read")
{
const std::string text = R"({"a" 1})";
const std::vector<std::string> expected = {"a"};
CHECK(!json::accept(text));
CHECK(keys_of(text) == expected);
CHECK(callback_keys_of(text) == expected);
}
SECTION("a repeated name is delivered once per occurrence")
{
const std::string text = R"({"a":1,"a":2})";
const std::vector<std::string> expected = {"a", "a"};
CHECK(keys_of(text) == expected);
CHECK(callback_keys_of(text) == expected);
}
SECTION("binary formats pass the stored bytes through")
{
const std::string stored = "\xff\\";
const std::vector<std::string> expected = {stored};
const auto keys_of_bytes = [&keys_of_format](const std::vector<std::uint8_t>& v, json::input_format_t format)
{
std::string text;
for (const auto byte : v)
{
text.push_back(static_cast<char>(byte));
}
return keys_of_format(text, format);
};
CHECK(keys_of_bytes({0xA1, 0x62, 0xFF, 0x5C, 0x01}, json::input_format_t::cbor) == expected);
CHECK(keys_of_bytes({0x81, 0xA2, 0xFF, 0x5C, 0x01}, json::input_format_t::msgpack) == expected);
CHECK(keys_of_bytes({'{', 'i', 2, 0xFF, 0x5C, 'i', 1, '}'}, json::input_format_t::ubjson) == expected);
CHECK(keys_of_bytes({0x0D, 0, 0, 0, 0x10, 0xFF, 0x5C, 0, 1, 0, 0, 0, 0}, json::input_format_t::bson) == expected);
}
SECTION("BON8 rejects ill-formed UTF-8 in a name")
{
const std::vector<std::string> valid = {"\\"};
CHECK(keys_of_format(std::string("\x87\x5C\x91"), json::input_format_t::bon8) == valid);
CHECK(keys_of_format(std::string("\x87\xC3\x28\x91"), json::input_format_t::bon8).empty());
}
}
+131 -3
View File
@@ -826,6 +826,129 @@ TEST_CASE("JSON patch")
CHECK(doc.at("foo") == "bar");
}
#endif // !defined(JSON_NOEXCEPTION)
SECTION("applying a value as patch to itself")
{
// a JSON patch is an array, so an array of operations can be
// applied to itself; the operations are read from the patch as
// it was before the first operation, as patch() does it
const json copy_op = {{"op", "copy"}, {"from", "/0"}, {"path", "/-"}};
json copy_ops = json::array();
for (int i = 0; i < 5; ++i)
{
copy_ops.push_back(copy_op);
}
json expected_copies = copy_ops;
for (int i = 0; i < 5; ++i)
{
expected_copies.push_back(copy_op);
}
SECTION("copy operations without spare capacity")
{
// a copy has exactly as much capacity as elements
json doc = copy_ops;
CHECK(doc.patch(doc) == expected_copies);
doc.patch_inplace(doc);
CHECK(doc == expected_copies);
}
#if JSON_DIAGNOSTICS
SECTION("patch stored inside the patched value")
{
// the parent pointers of JSON_DIAGNOSTICS make the aliasing detectable
const json op = {{"op", "replace"}, {"path", "/patch"}, {"value", 1}};
json doc = {{"a", 1}, {"patch", json::array({op})}};
const json expected = doc.patch(doc["patch"]);
doc.patch_inplace(doc["patch"]);
CHECK(doc == expected);
}
#endif
SECTION("copy operations with spare capacity")
{
// reserve while the array is empty: reallocating non-empty
// elements behind the library's back leaves stale parent
// pointers with JSON_DIAGNOSTICS
json doc = json::array();
doc.get_ref<json::array_t&>().reserve(100);
for (int i = 0; i < 5; ++i)
{
doc.push_back(copy_op);
}
CHECK(doc.patch(doc) == expected_copies);
doc.patch_inplace(doc);
CHECK(doc == expected_copies);
}
SECTION("remove operations")
{
json doc = R"([
{"op": "remove", "path": "/0"},
{"op": "remove", "path": "/0"},
{"op": "remove", "path": "/0"}
])"_json;
CHECK(doc.patch(doc) == json::array());
doc.patch_inplace(doc);
CHECK(doc == json::array());
}
SECTION("operations that read a changed element")
{
// the second operation is still a "test" operation, although
// the first one replaced it in the document
json doc = R"([
{"op": "replace", "path": "/1", "value": 1},
{"op": "test", "path": "/1", "value": 1},
{"op": "add", "path": "/-", "value": 2}
])"_json;
const json expected = R"([
{"op": "replace", "path": "/1", "value": 1},
1,
{"op": "add", "path": "/-", "value": 2},
2
])"_json;
CHECK(doc.patch(doc) == expected);
doc.patch_inplace(doc);
CHECK(doc == expected);
}
SECTION("replacing the root")
{
json doc = R"([
{"op": "add", "path": "", "value": [1, 2]},
{"op": "add", "path": "/-", "value": 3},
{"op": "replace", "path": "", "value": {"a": 4}}
])"_json;
const json expected = {{"a", 4}};
CHECK(doc.patch(doc) == expected);
doc.patch_inplace(doc);
CHECK(doc == expected);
}
#if !defined(JSON_NOEXCEPTION)
SECTION("failing operation")
{
// the operations before the failing one remain applied
json doc = R"([
{"op": "add", "path": "/-", "value": 1},
{"op": "test", "path": "/0/op", "value": "remove"}
])"_json;
CHECK_THROWS_AS(doc.patch(doc), json::other_error&);
CHECK_THROWS_AS(doc.patch_inplace(doc), json::other_error&);
CHECK(doc.size() == 3);
CHECK(doc.back() == 1);
}
#endif // !defined(JSON_NOEXCEPTION)
}
}
SECTION("errors")
@@ -2178,8 +2301,13 @@ TEST_CASE("JSON patch - behavior documented in the API reference")
SECTION("patch() has the strong guarantee")
{
const json doc = R"({"a": [1, 2], "o": {"k": "v"}})"_json;
const json before = doc;
// independent values, so that the comparison below checks the untouched document
const auto make_doc = []
{
return R"({"a": [1, 2], "o": {"k": "v"}})"_json;
};
const json doc = make_doc();
const json before = make_doc();
const std::vector<json> failing =
{
@@ -2202,7 +2330,7 @@ TEST_CASE("JSON patch - behavior documented in the API reference")
CHECK_THROWS_AS(doc.patch(patch), json::exception&);
CHECK(doc == before);
json copy = doc;
json copy = make_doc();
CHECK_THROWS_AS(copy.patch(patch), json::exception&);
CHECK(copy == before);
}
+112
View File
@@ -29,9 +29,64 @@ using nlohmann::json;
#include <map>
#include <random>
#include <sstream>
#include <stdexcept>
#include <string>
#include <vector>
namespace
{
// the non-const operator[] must throw for pointer and leave the value parsed
// from init unchanged (strong exception safety)
template<typename Json>
void check_strong_guarantee(const char* init, const std::string& pointer, const char* message)
{
CAPTURE(init)
CAPTURE(pointer)
Json j = Json::parse(init);
const Json before = Json::parse(init);
const typename Json::json_pointer ptr(pointer);
CHECK_THROWS_WITH_AS(j[ptr], message, typename Json::exception&);
CHECK(j == before);
}
// as above, for an index that fits into size_type, but not into an array
template<typename Json>
void check_strong_guarantee_length_error(const char* init, const std::string& pointer)
{
CAPTURE(init)
CAPTURE(pointer)
Json j = Json::parse(init);
const Json before = Json::parse(init);
const typename Json::json_pointer ptr(pointer);
CHECK_THROWS_AS(j[ptr], std::length_error&);
CHECK(j == before);
}
template<typename Json>
void check_strong_guarantee_cases()
{
const char* const unresolved = "[json.exception.out_of_range.404] unresolved reference token '99999999999999999999999'";
// the first change converts a null value
check_strong_guarantee<Json>("null", "/99999999999999999999999", unresolved);
check_strong_guarantee<Json>("null", "/a/99999999999999999999999", unresolved);
check_strong_guarantee<Json>(R"({"a":null})", "/a/b/-/99999999999999999999999", unresolved);
// the first change inserts a key into an object
check_strong_guarantee<Json>("{}", "/a/99999999999999999999999", unresolved);
check_strong_guarantee<Json>(R"({"a":{"b":1}})", "/a/c/d/99999999999999999999999", unresolved);
// the first change pads or appends to an array
check_strong_guarantee<Json>(R"({"a":[1]})", "/a/5/99999999999999999999999", unresolved);
check_strong_guarantee<Json>(R"({"a":[1]})", "/a/-/x/1/99999999999999999999999", unresolved);
// an index that does not fit into size_type
check_strong_guarantee<Json>("null", "/x/18446744073709551615",
"[json.exception.out_of_range.410] array index 18446744073709551615 exceeds size_type");
// an index that fits into size_type, but not into an array
const auto too_large = std::to_string((std::numeric_limits<typename Json::size_type>::max)() - 1);
check_strong_guarantee_length_error<Json>("null", "/" + too_large);
check_strong_guarantee_length_error<Json>(R"({"a":[[1]]})", "/a/0/3/" + too_large);
}
} // namespace
TEST_CASE("JSON pointers")
{
SECTION("errors")
@@ -557,6 +612,23 @@ TEST_CASE("JSON pointers")
CHECK_THROWS_WITH_AS(j.at("/01"_json_pointer),
"[json.exception.parse_error.106] parse error: array index '01' must not begin with '0'", json::parse_error&);
}
SECTION("a failing operator[] leaves the value unchanged")
{
check_strong_guarantee_cases<json>();
check_strong_guarantee_cases<nlohmann::ordered_json>();
}
SECTION("a successful operator[] still creates the same values")
{
json j = {{"a", {1}}};
j["/a/3/x/-/y"_json_pointer] = 1;
CHECK(j == json::parse(R"({"a":[1,null,null,{"x":[{"y":1}]}]})"));
j["/a/-"_json_pointer] = 2;
j["/a/0"_json_pointer] = 3;
j["/b"_json_pointer] = 4;
CHECK(j == json::parse(R"({"a":[3,null,null,{"x":[{"y":1}]},2],"b":4})"));
}
}
SECTION("flatten")
@@ -845,6 +917,46 @@ TEST_CASE("JSON pointers")
CHECK(ptr.to_string() == "/object/~1");
}
SECTION("appending a pointer to itself")
{
// operator/= must not pass iterators into its own token vector to
// insert(); each pointer is appended to itself and must yield the
// pointer whose string representation is the original one twice
const std::vector<std::string> pointers =
{
"",
"/",
"/a",
"/a/b",
"/foo/0/-",
"/a~1b/~0c/~01",
"/1/2/3/4/5/6/7/8/9/10/11/12/13/14/15/16/17"
};
for (const auto& s : pointers)
{
CAPTURE(s)
json::json_pointer ptr(s);
const json::json_pointer expected(s + s);
const json::json_pointer& ret = (ptr /= ptr);
CHECK(&ret == &ptr);
CHECK(ptr == expected);
CHECK(ptr.to_string() == s + s);
// appending to itself again doubles the pointer once more
ptr /= ptr;
std::string quadruple = s + s;
quadruple += quadruple;
CHECK(ptr.to_string() == quadruple);
// operator/ appends to a copy and leaves the operand unchanged
const json::json_pointer original(s);
CHECK((original / original) == expected);
CHECK(original.to_string() == s);
}
}
SECTION("equality comparison")
{
std::string ptr_string{"/foo/bar"};
+23 -9
View File
@@ -749,6 +749,20 @@ std::string exception_of(F f)
}
return "";
}
// the exception a call on a basic_json value throws, without the byte range of
// the value (JSON_DIAGNOSTIC_POSITIONS), which the exceptions of views lack
template<typename F>
std::string json_exception_of(F f)
{
std::string what = exception_of(f);
const std::size_t at = what.find("] (bytes ");
if (at != std::string::npos)
{
what.erase(at + 2, what.find(") ", at) - at);
}
return what;
}
#endif
// compares a view with the ordered_json value materialize() gives for it:
@@ -957,24 +971,24 @@ TEST_CASE("json_view element access and iteration")
#if !defined(JSON_NOEXCEPTION)
if (!j.is_object())
{
CHECK(exception_of([&] { static_cast<void>(v["a"]); }) == exception_of([&] { static_cast<void>(j["a"]); }));
CHECK(exception_of([&] { static_cast<void>(v["a"]); }) == json_exception_of([&] { static_cast<void>(j["a"]); }));
}
if (!j.is_array())
{
CHECK(exception_of([&] { static_cast<void>(v[0]); }) == exception_of([&] { static_cast<void>(j[0]); }));
CHECK(exception_of([&] { static_cast<void>(v[0]); }) == json_exception_of([&] { static_cast<void>(j[0]); }));
}
CHECK(exception_of([&] { static_cast<void>(v.at("a")); }) == exception_of([&] { static_cast<void>(j.at("a")); }));
CHECK(exception_of([&] { static_cast<void>(v.at("missing")); }) == exception_of([&] { static_cast<void>(j.at("missing")); }));
CHECK(exception_of([&] { static_cast<void>(v.at(0)); }) == exception_of([&] { static_cast<void>(j.at(0)); }));
CHECK(exception_of([&] { static_cast<void>(v.at(5)); }) == exception_of([&] { static_cast<void>(j.at(5)); }));
CHECK(exception_of([&] { static_cast<void>(v.at("a")); }) == json_exception_of([&] { static_cast<void>(j.at("a")); }));
CHECK(exception_of([&] { static_cast<void>(v.at("missing")); }) == json_exception_of([&] { static_cast<void>(j.at("missing")); }));
CHECK(exception_of([&] { static_cast<void>(v.at(0)); }) == json_exception_of([&] { static_cast<void>(j.at(0)); }));
CHECK(exception_of([&] { static_cast<void>(v.at(5)); }) == json_exception_of([&] { static_cast<void>(j.at(5)); }));
if (!(j.is_object() && j.empty())) // (key() of an end iterator)
{
CHECK(exception_of([&] { static_cast<void>(v.begin().key()); }) == exception_of([&] { static_cast<void>(j.begin().key()); }));
CHECK(exception_of([&] { static_cast<void>(v.begin().key()); }) == json_exception_of([&] { static_cast<void>(j.begin().key()); }));
}
if (!j.empty() || j.is_null())
{
CHECK(exception_of([&] { static_cast<void>(v.front()); }) == exception_of([&] { static_cast<void>(j.front()); }));
CHECK(exception_of([&] { static_cast<void>(v.back()); }) == exception_of([&] { static_cast<void>(j.back()); }));
CHECK(exception_of([&] { static_cast<void>(v.front()); }) == json_exception_of([&] { static_cast<void>(j.front()); }));
CHECK(exception_of([&] { static_cast<void>(v.back()); }) == json_exception_of([&] { static_cast<void>(j.back()); }));
}
#endif
CHECK(v.contains("a") == j.contains("a"));
+2 -2
View File
@@ -592,7 +592,7 @@ TEST_CASE("json_view node integer bits")
})
{
CAPTURE(v)
n.kind = static_cast<nlohmann::detail::view::node_kind>(0x5A);
n.kind = nlohmann::detail::view::node_kind::link;
n.flags = 0xA5;
n.extra = 0x1234;
n.off = 0x89ABCDEFu;
@@ -601,7 +601,7 @@ TEST_CASE("json_view node integer bits")
CHECK(n.len == static_cast<std::uint32_t>(v));
CHECK(n.next == static_cast<std::uint32_t>(v >> 32));
// the other fields are untouched
CHECK(static_cast<std::uint8_t>(n.kind) == 0x5A);
CHECK(n.kind == nlohmann::detail::view::node_kind::link);
CHECK(n.flags == 0xA5);
CHECK(n.extra == 0x1234);
CHECK(n.off == 0x89ABCDEFu);
+364
View File
@@ -0,0 +1,364 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#include "doctest_compatibility.h"
#include <nlohmann/json_view.hpp>
using nlohmann::json;
using nlohmann::ordered_json;
using nlohmann::json_document;
using nlohmann::ordered_json_editable_document;
using nlohmann::json_view;
using nlohmann::ordered_json_document;
#include "json_view_test_helpers.hpp"
using json_view_test::generator;
#include <array>
#include <cstddef>
#include <string>
#include <vector>
// These tests were split off unit-json_view_dump.cpp, whose object file got too
// large for the MinGW assembler ("file too big"). Like that file, this one is
// also built with C++17, as it mentions JSON_HAS_CPP_17.
TEST_CASE("json_view comparison")
{
SECTION("equality of the values parse() produces")
{
generator g;
std::vector<std::string> texts;
for (int i = 0; i < 600; ++i)
{
std::string text;
g.value(text, 0);
texts.push_back(text);
// the same value written differently: sorted keys, canonical numbers
texts.push_back(json::parse(text).dump(1));
}
for (std::size_t i = 0; i + 2 < texts.size(); ++i)
{
for (std::size_t k = i; k < i + 3; ++k)
{
CAPTURE(texts[i])
CAPTURE(texts[k])
const json_document a = json_document::parse(texts[i]);
const json_document b = json_document::parse(texts[k]);
const json ja = json::parse(texts[i]);
const json jb = json::parse(texts[k]);
CHECK((a.root() == b.root()) == (ja == jb));
CHECK((a.root() != b.root()) == (ja != jb));
CHECK((a.root() == jb) == (ja == jb));
CHECK((jb == a.root()) == (ja == jb));
CHECK((a.root() != jb) == (ja != jb));
CHECK((jb != a.root()) == (ja != jb));
// ordered_json compares members in order
const ordered_json_document oa = ordered_json_document::parse(texts[i]);
const ordered_json_document ob = ordered_json_document::parse(texts[k]);
const ordered_json oja = ordered_json::parse(texts[i]);
const ordered_json ojb = ordered_json::parse(texts[k]);
CHECK((oa.root() == ob.root()) == (oja == ojb));
CHECK((oa.root() == ojb) == (oja == ojb));
}
}
}
SECTION("objects whose first keys differ or repeat")
{
const std::array<const char*, 17> texts =
{
{
R"({"a":1,"b":2})", R"({"a":1,"c":2})", R"({"b":1,"a":2})", R"({"a":1,"a":2,"b":3})", R"({"a":2,"b":3})",
R"({"a":1,"b":1,"a":2})", R"({"a":2,"b":1})", R"({"a":1,"a":2})", R"({"b":1})", R"({"a":1,"b":2,"c":3,"d":4,"e":5})",
R"({"a":1,"b":2,"c":3,"d":4,"f":5})", R"({"a":1,"b":2,"c":3,"a":4,"e":5})", R"({"a":4,"b":2,"c":3,"e":5})",
R"({"\u0061":1,"b":2})", R"({})", R"([{"a":1,"b":2}])", R"([{"a":1,"c":2}])"
}
};
for (const char* const x : texts)
{
for (const char* const y : texts)
{
CAPTURE(x)
CAPTURE(y)
const ordered_json_document dx = ordered_json_document::parse(x);
const ordered_json_document dy = ordered_json_document::parse(y);
const ordered_json_editable_document ex = ordered_json_editable_document::parse(x);
const ordered_json jx = ordered_json::parse(x);
const ordered_json jy = ordered_json::parse(y);
CHECK((dx.root() == dy.root()) == (jx == jy));
CHECK((dx.root() == jy) == (jx == jy));
CHECK((ex.root() == dy.root()) == (jx == jy));
}
}
}
SECTION("objects with duplicate keys, small and large")
{
// n members with keys in descending order; with duplicates, some keys
// repeat later (the first after unique ones in key order); in reverse,
// the members are in ascending key order
const auto object = [](int n, bool duplicates, bool reverse, int value)
{
std::string s = "{";
for (int i = 0; i < n; ++i)
{
const int k = reverse ? i : n - 1 - i;
s += (i != 0 ? ",\"k" : "\"k") + std::to_string(100 + k) + "\":" + std::to_string(value + k);
}
if (duplicates)
{
for (int k = n / 2; k < n; k += 3)
{
s += ",\"k" + std::to_string(100 + k) + "\":" + std::to_string(value + k);
}
}
return s + "}";
};
std::vector<std::string> texts;
for (const int n :
{
3, 16, 17, 40
})
{
for (const int flags :
{
0, 1, 2, 3
})
{
texts.push_back(object(n, (flags & 1) != 0, (flags & 2) != 0, 0));
}
texts.push_back(object(n, true, false, 1)); // other values
}
// many objects in one comparison (working memory reused across objects)
std::string array = "[";
std::string array_other = "[";
for (std::size_t i = 0; i < texts.size(); ++i)
{
array += (i != 0 ? "," : "") + texts[i];
array_other += (i != 0 ? "," : "") + texts[texts.size() - 1 - i];
}
texts.push_back(array + "]");
texts.push_back(array_other + "]");
for (const std::string& x : texts)
{
for (const std::string& y : texts)
{
CAPTURE(x)
CAPTURE(y)
const ordered_json_document dx = ordered_json_document::parse(x);
const ordered_json_document dy = ordered_json_document::parse(y);
const ordered_json jx = ordered_json::parse(x);
const ordered_json jy = ordered_json::parse(y);
CHECK((dx.root() == dy.root()) == (jx == jy));
CHECK((dx.root() == jy) == (jx == jy));
const json_document ux = json_document::parse(x);
const json uy = json::parse(y);
CHECK((ux.root() == uy) == (json::parse(x) == uy));
}
}
}
SECTION("numbers, duplicate keys, member order")
{
const auto same = [](const char* x, const char* y)
{
const json_document dx = json_document::parse(x);
const json_document dy = json_document::parse(y);
return dx.root() == dy.root();
};
CHECK(same("1", "1.0"));
CHECK(same("[1, -1, 2.5]", "[1.0, -1.0, 25e-1]"));
CHECK(!same("1", "1.5"));
CHECK(same("18446744073709551615", "18446744073709551615"));
CHECK(same(R"({"a": 1, "a": 2})", R"({"a": 2})"));
CHECK(!same(R"({"a": 1, "a": 2})", R"({"a": 1})"));
CHECK(same(R"({"a": 1, "b": 2})", R"({"b": 2, "a": 1})"));
CHECK(!same(R"({"a": 1})", R"({"a": 1, "b": 2})"));
CHECK(!same("[1, 2]", "[2, 1]"));
CHECK(!same("\"a\"", "\"b\""));
CHECK(same("\"\\u00e9\"", "\"\xc3\xa9\""));
CHECK(!same("null", "false"));
CHECK(!same("[]", "{}"));
const ordered_json_document dup = ordered_json_document::parse(R"({"a": 1, "b": 2, "a": 3})");
const ordered_json_document last = ordered_json_document::parse(R"({"a": 3, "b": 2})");
CHECK(dup.root() == last.root());
// (== compares as parse() resolves duplicates, a lookup finds the first)
CHECK(dup.root()["a"].materialize() == 1);
CHECK(dup.root()["a"] != last.root()["a"]);
const ordered_json_document ab = ordered_json_document::parse(R"({"a": 1, "b": 2})");
const ordered_json_document ba = ordered_json_document::parse(R"({"b": 2, "a": 1})");
CHECK(ab.root() != ba.root());
// discarded values compare as basic_json's do
const json discarded(json::value_t::discarded);
CHECK((json_view() == json_view()) == (discarded == discarded)); // NOLINT(readability-container-size-empty): operator== is tested
CHECK((json_view() == discarded) == (discarded == discarded)); // NOLINT(readability-container-size-empty)
const json_document null_document = json_document::parse("null");
CHECK(!(json_view() == null_document.root())); // NOLINT(readability-container-size-empty)
CHECK(!(null_document.root() == discarded));
}
SECTION("values that are the same text, or the same value written differently, at any depth")
{
// nested arrays and objects around depths where the walk over equal text gives up
for (const int depth :
{
1, 5, 47, 48, 49, 50, 60, 300
})
{
CAPTURE(depth)
std::string head;
std::string tail;
for (int i = 0; i < depth; ++i)
{
head += i % 2 == 0 ? R"({"k":[1,"s",{"x":true},)" : R"([null,2.5,{"y":)";
tail.insert(0, i % 2 == 0 ? "]}" : "}]");
}
// the same text, one number changed, and one number written differently
const auto with_number = [&](const char* number)
{
std::string result = head;
result += number;
result += tail;
return result;
};
const std::string text = with_number("7");
const std::string changed = with_number("8");
const std::string spelled = with_number("7.0");
const json_document a = json_document::parse(text);
const json_document b = json_document::parse(text);
const json_document c = json_document::parse(changed);
const json_document d = json_document::parse(spelled);
CHECK(a.root() == b.root());
CHECK(a.root() != c.root());
CHECK(c.root() != a.root());
CHECK(a.root() == d.root());
CHECK(d.root() == a.root());
CHECK(a.root() == json::parse(text));
CHECK(a.root() != json::parse(changed));
CHECK(json::parse(spelled) == a.root());
const nlohmann::json_editable_document e = nlohmann::json_editable_document::parse(text);
CHECK(a.root() == e.root());
CHECK(e.root() == a.root());
CHECK(c.root() != e.root());
CHECK(e.root() == e.root());
const ordered_json_document oa = ordered_json_document::parse(text);
const ordered_json_document oc = ordered_json_document::parse(changed);
CHECK(oa.root() != oc.root());
CHECK(oa.root() == ordered_json::parse(text));
CHECK(oc.root() != ordered_json::parse(text));
}
// not the same text, but the same value
const auto same = [](const char* x, const char* y)
{
const json_document dx = json_document::parse(x);
const json_document dy = json_document::parse(y);
const bool equal = dx.root() == dy.root();
CHECK(equal == (json::parse(x) == json::parse(y)));
CHECK(equal == (dx.root() == json::parse(y)));
return equal;
};
CHECK(same(R"({"a":1,"a":2})", R"({"a":3,"a":2})"));
CHECK(same(R"({"a":1,"b":[1,{"c":2,"c":3}]})", R"({"b":[1,{"c":9,"c":3}],"a":1})"));
CHECK(!same(R"({"a":1,"a":2})", R"({"a":2,"a":3})"));
CHECK(same("[1.0,-0,0,1e2]", "[1.00,0,-0,100.0]"));
CHECK(same(R"(["\u00e9","\n"])", "[\"\xc3\xa9\",\"\\n\"]"));
CHECK(!same(R"({"a":[1,2,{"b":null}]})", R"({"a":[1,2,{"b":false}]})"));
CHECK(!same(R"({"a":[1,2,{"b":null}]})", R"({"a":[1,2,{"c":null}]})"));
CHECK(!same(R"({"a":[1,2,{"b":null}]})", R"({"a":[1,2,{"b":null,"c":1}]})"));
// edited documents: moved sequences, links, and values of new nodes
nlohmann::json_editable_document edited = nlohmann::json_editable_document::parse(R"({"a":[1,2,3],"b":{"c":"d"},"e":0.5})");
const json_document plain = json_document::parse(R"({"a":[1,2,3,4],"b":{"c":"x","y":null},"e":0.5,"z":[]})");
edited.push_back(edited.root()["a"], 4);
edited.set(edited.root()["b"], "c", "x");
edited.set(edited.root()["b"], "y", nullptr);
edited.set(edited.root(), "z", json::array());
CHECK(edited.root() == plain.root());
CHECK(plain.root() == edited.root());
edited.erase(edited.root()["a"], 0);
CHECK(edited.root() != plain.root());
CHECK(edited.root()["a"] == json::parse("[2,3,4]"));
const json_document tail = json_document::parse("[2,3,4]");
CHECK(edited.root()["a"] == tail.root());
}
SECTION("large documents, with a difference at any place")
{
const auto make = [](std::size_t count, std::size_t changed, bool swap_members)
{
std::string text = R"({"b":1,"list":[)";
for (std::size_t i = 0; i < count; ++i)
{
text += (i != 0 ? "," : "");
const std::string id = std::to_string(i == changed ? i + 1 : i);
if (swap_members && i % 3 == 0)
{
text += R"({"v":[1.5,"s",{"k":)";
text += id;
text += R"(}],"id":)";
text += id;
text += "}";
}
else
{
text += R"({"id":)";
text += id;
text += R"(,"v":[1.5,"s",{"k":)";
text += id;
text += "}]}";
}
}
return text + R"(],"a":{"y":2,"x":3}})";
};
const std::size_t count = 3000;
for (const std::size_t changed :
{
count, std::size_t{0}, std::size_t{1}, count / 2, count - 1
})
{
for (const bool swap :
{
false, true
})
{
CAPTURE(changed)
CAPTURE(swap)
const std::string text_a = make(count, count, false);
const std::string text_b = make(count, changed, swap);
const json_document a = json_document::parse(text_a);
const json_document b = json_document::parse(text_b);
const bool equal = changed == count;
CHECK((a.root() == b.root()) == equal);
CHECK((b.root() == a.root()) == equal);
CHECK((a.root() == json::parse(text_b)) == equal);
CHECK((json::parse(text_b) == a.root()) == equal);
CHECK((a.root()["list"] == b.root()["list"]) == equal);
const ordered_json_document oa = ordered_json_document::parse(text_a);
const ordered_json_document ob = ordered_json_document::parse(text_b);
CHECK((oa.root() == ob.root()) == (equal && !swap));
CHECK((oa.root() == ordered_json::parse(text_b)) == (equal && !swap));
}
}
}
SECTION("deep nesting")
{
const std::string deep = std::string(100000, '[') + std::string(100000, ']');
const json_document a = json_document::parse(deep);
const json_document b = json_document::parse(deep);
CHECK(a.root() == b.root());
CHECK(a.root() == json::parse(deep));
const std::string other = std::string(100000, '[') + "1" + std::string(100000, ']');
const json_document c = json_document::parse(other);
CHECK(a.root() != c.root());
}
}
+672 -85
View File
@@ -35,9 +35,9 @@ using json_view_test::has_duplicate_keys;
#include <string>
#include <vector>
// These tests were split off unit-json_view.cpp, whose object file got too large for the MinGW
// linker (see json_view_test_helpers.hpp). Like that file, this one is also built
// with C++17, as it mentions JSON_HAS_CPP_17.
// These tests were split off unit-json_view.cpp, whose object file got too large
// for the MinGW linker (see json_view_test_helpers.hpp). Like that file, this one
// is also built with C++17, as it mentions JSON_HAS_CPP_17.
TEST_CASE("json_view dump")
{
@@ -150,88 +150,6 @@ TEST_CASE("json_view dump")
CHECK(many_doc.root().dump() == json::parse(many_tokens).dump());
}
// tokens of 16 and 17 digits without an exponent (like GeoJSON
// coordinates), which dump() converts in batches: doubles exactly
// halfway between two candidates for the shortest digits, random
// digits, and digits near a shorter decimal (...99999997,
// ...00000003), whose rounding carries; in pairs and among other
// values, so that the batches are interrupted; the last token near the
// end of the text
{
std::mt19937_64 tokens(5627); // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed)
const auto draw = [&tokens](std::size_t n)
{
const std::uint64_t r = tokens() % n;
return static_cast<std::size_t>(r);
};
std::string coordinates = "[754570942.33984373,257769445.56835938,858106901.94921875,97944964.876953125,"
"[0.5,1e-7],[1.5,-0.0,2.5],[1E2,12.5,0.1,3.25,4.5,5.75,6.125,7.0,8.5],[],[1.5],[1,2.5],[2.5,\"x\"],[[1.5],2.5],";
for (int i = 0; i < 20000; ++i)
{
const std::size_t length = 16 + draw(2);
std::string digits(1, static_cast<char>('1' + draw(9)));
const std::size_t kept = draw(3) == 0 ? length : 2 + draw(length - 3);
for (std::size_t k = 1; k < kept; ++k)
{
digits += static_cast<char>('0' + draw(10));
}
digits += std::string(length - kept, draw(2) == 0 ? '9' : '0');
digits.back() = static_cast<char>('0' + draw(10));
const std::size_t point = 1 + draw(9);
const std::string token = (draw(2) == 0 ? "-" : "") + digits.substr(0, point) + "." + digits.substr(point);
if (i != 0)
{
coordinates += ',';
}
if (draw(8) == 0)
{
coordinates += R"({"x":)";
coordinates += token;
coordinates += R"(,"s":"a"})";
}
else
{
coordinates += '[';
coordinates += token;
coordinates += ',';
coordinates += token;
coordinates += ']';
}
}
coordinates += ",-65.613616999999977]";
const json_document parsed = json_document::parse(coordinates);
const ordered_json j = ordered_json::parse(coordinates);
CHECK(parsed.root().dump() == j.dump());
CHECK(parsed.root().dump(1) == j.dump(1));
CHECK(parsed.root().dump(-1, ' ', true) == j.dump(-1, ' ', true));
CHECK(parsed.root().back().get<double>() == j.back().get<double>());
CHECK(parsed.root().back().dump() == "-65.61361699999998");
std::size_t i = 0;
for (const json_view v : parsed.root())
{
const ordered_json& e = j[i++];
CHECK(v.dump() == e.dump());
if (v.is_array() && !v.empty() && v[0].is_number())
{
CHECK(v[0].get<double>() == e[0].get<double>());
}
}
// an editable document: edits move some arrays, so that the walk
// leaves the order of the parsed nodes
ordered_json_editable_document editable = ordered_json_editable_document::parse(coordinates);
ordered_json expected = j;
for (std::size_t k = 0; k < 20000; k += 97)
{
if (expected[k].is_array())
{
editable.push_back(editable.root()[k], ordered_json(1.5));
expected[k].push_back(1.5);
}
}
CHECK(editable.root().dump() == expected.dump());
}
// random doubles, written as parse() and dump() would
std::mt19937_64 rng(1170); // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed)
std::string many = "[";
@@ -253,4 +171,673 @@ TEST_CASE("json_view dump")
const nlohmann::basic_json_document<json_float> float_document = nlohmann::basic_json_document<json_float>::parse("[0.1, 1.5e10, 3.4028235e38]");
CHECK(float_document.root().dump() == json_float::parse("[0.1, 1.5e10, 3.4028235e38]").dump());
}
SECTION("members in document order, all of them")
{
const json_document d = json_document::parse(R"({"b": 1, "a": 2, "b": 3})");
CHECK(d.root().dump() == R"({"b":1,"a":2,"b":3})");
CHECK(d.root().dump(1) == "{\n \"b\": 1,\n \"a\": 2,\n \"b\": 3\n}");
// dump() writes all members, though the lookup finds the first
CHECK(d.root()["b"].dump() == "1");
}
SECTION("deep nesting")
{
const std::string deep = std::string(100000, '[') + std::string(100000, ']');
const json_document deep_document = json_document::parse(deep);
CHECK(deep_document.root().dump() == deep);
}
SECTION("ensure_ascii: a character that needs an escape at each position of a string")
{
// string lengths around the width of the vector steps, in a string value, a key, and a value near the end of the text
for (const char* const special :
{
"\xC3\xA9", "\xE4\xB8\xAD", "\xF0\x9F\x98\x80", "\x7F", "\\n", "\\u0001", "\\\"", "\\\\"
})
{
for (std::size_t length = 0; length <= 40; ++length)
{
for (std::size_t position = 0; position <= length; ++position)
{
std::string content(length, 'a');
content.insert(position, special);
const std::array<std::string, 3> texts =
{
{
"\"" + content + "\"",
"[\"" + content + "\"]",
"{\"" + content + "\":1}",
}
};
for (const std::string& text : texts)
{
CAPTURE(text)
const ordered_json_document d = ordered_json_document::parse(text);
const ordered_json j = ordered_json::parse(text);
CHECK(d.root().dump(-1, ' ', true) == j.dump(-1, ' ', true));
CHECK(d.root().dump(2, ' ', true) == j.dump(2, ' ', true));
CHECK(d.root().dump() == j.dump());
}
}
}
}
}
SECTION("ensure_ascii: every character of the Basic Multilingual Plane and some beyond")
{
// (the two- and three-byte sequences, including the ranges that need a check: E0, ED, EF BF BF)
std::string text = "[";
std::size_t in_chunk = 0;
const auto append = [&](std::uint32_t cp)
{
if (cp < 0x80)
{
if (cp < 0x20 || cp == '"' || cp == '\\')
{
return;
}
text += static_cast<char>(cp);
}
else if (cp < 0x800)
{
text += static_cast<char>(0xC0 | (cp >> 6));
text += static_cast<char>(0x80 | (cp & 0x3F));
}
else if (cp < 0x10000)
{
text += static_cast<char>(0xE0 | (cp >> 12));
text += static_cast<char>(0x80 | ((cp >> 6) & 0x3F));
text += static_cast<char>(0x80 | (cp & 0x3F));
}
else
{
text += static_cast<char>(0xF0 | (cp >> 18));
text += static_cast<char>(0x80 | ((cp >> 12) & 0x3F));
text += static_cast<char>(0x80 | ((cp >> 6) & 0x3F));
text += static_cast<char>(0x80 | (cp & 0x3F));
}
};
for (std::uint32_t cp = 0x20; cp < 0x10400; ++cp)
{
if (cp >= 0xD800 && cp < 0xE000)
{
continue;
}
if (in_chunk == 0)
{
text += text.size() > 1 ? ",\"" : "\"";
}
append(cp);
if (++in_chunk == 37)
{
text += "\"";
in_chunk = 0;
}
}
text += in_chunk != 0 ? "\"]" : "]";
const ordered_json_document d = ordered_json_document::parse(text);
const ordered_json j = ordered_json::parse(text);
CHECK(d.root().dump(-1, ' ', true) == j.dump(-1, ' ', true));
CHECK(d.root().dump(2, ' ', true) == j.dump(2, ' ', true));
CHECK(d.root().dump() == j.dump());
CHECK(d.root().dump(2) == j.dump(2));
}
SECTION("deep nesting, indented")
{
// indentations of up to and beyond the width of the writer's fixed-size copy
for (const int depth :
{
1, 5, 20, 70, 300
})
{
std::string text;
for (int i = 0; i < depth; ++i)
{
text += i % 2 == 0 ? "[1," : R"({"k":2,"\u00e9":)";
}
text += "[]";
for (int i = depth - 1; i >= 0; --i)
{
text += i % 2 == 0 ? "]" : "}";
}
const ordered_json_document d = ordered_json_document::parse(text);
const ordered_json j = ordered_json::parse(text);
for (const int indent :
{
0, 1, 2, 3, 8
})
{
CHECK(d.root().dump(indent) == j.dump(indent));
CHECK(d.root().dump(indent, '\t') == j.dump(indent, '\t'));
}
}
}
SECTION("the output buffer of a small value is small")
{
// an escaped key after the value: its node lies in the arena, so the
// source extent of the value cannot be read from the next node
const std::string big(100000, 'a');
const std::string text = R"({"small":1,"list":[1,2,3],"k\n":")" + big + R"("})";
const json_document d = json_document::parse(text);
const auto small = d.root()["small"].dump();
CHECK(small == "1");
CHECK(small.capacity() < 4096);
const auto list = d.root()["list"].dump();
CHECK(list == "[1,2,3]");
CHECK(list.capacity() < 4096);
CHECK(d.root()["list"].dump(2).capacity() < 4096);
// the whole document and the large value are unaffected
CHECK(d.root().dump() == ordered_json::parse(text).dump());
CHECK(d.root()["k\n"].dump() == "\"" + big + "\"");
}
SECTION("output that outgrows the estimate")
{
// ensure_ascii writes six bytes for each two-byte character
std::string chars;
for (int i = 0; i < 5000; ++i)
{
chars += "\xC3\xA9";
}
const json_document d = json_document::parse(R"({"a":")" + chars + R"(","k\n":1})");
const json expected = json::parse("\"" + chars + "\"");
CHECK(d.root()["a"].dump(-1, ' ', true) == expected.dump(-1, ' ', true));
CHECK(d.root()["a"].dump(-1, ' ', true).size() == 2 + (5000 * 6));
}
SECTION("ensure_ascii with strings of the source, decoded strings, and edited strings")
{
// plain ASCII, non-ASCII, escaped (decoded), and mixed strings, as values and as keys;
// the second text has a DEL, which ensure_ascii writes as an escape
for (const char* const del :
{"", "\x7F"
})
{
const std::string text = std::string("{\"plain\":\"abc\",\"caf\xC3\xA9\":\"\xC3\xA9\xE2\x82\xAC\xF0\x9F\x98\x80\",")
+ "\"esc\\n\\\"\":\"a\\tb\\u00e4\\ud83d\\ude00\",\"del\":\"a" + del + "b\",\"\":\"\"," // NOLINT(modernize-raw-string-literal)
+ "\"list\":[\"x\",\"\xC3\xBC\",\"\\\\\",\"long ASCII string with more than thirty-two bytes in it\","
+ "\"long string with more than thirty-two bytes and one non-ASCII byte \xC3\xB6\"],"
+ "\"n\":[1,2.5,-0,true,null]}";
const ordered_json j = ordered_json::parse(text);
const ordered_json_document d = ordered_json_document::parse(text);
CHECK(d.root().dump(-1, ' ', true) == j.dump(-1, ' ', true));
CHECK(d.root().dump(4, ' ', true) == j.dump(4, ' ', true));
CHECK(d.root().dump() == j.dump());
CHECK(d.root()["list"].dump(-1, ' ', true) == j["list"].dump(-1, ' ', true));
CHECK(d.root()["del"].dump(-1, ' ', true) == j["del"].dump(-1, ' ', true));
// strings that an edit wrote (the source strings that they replace were plain)
ordered_json_editable_document e = ordered_json_editable_document::parse(text);
ordered_json k = j;
e.set(e.root()["plain"], std::string("\xC3\xA4gypten"));
k["plain"] = "\xC3\xA4gypten";
e.set(e.root()["list"][0], std::string("ascii after edit"));
k["list"][0] = "ascii after edit";
e.set(e.root()["list"][1], std::string("\xC3\xA4"));
k["list"][1] = "\xC3\xA4";
e.set(e.root()["del"], std::string("tab\there"));
k["del"] = "tab\there";
CHECK(e.root().dump(-1, ' ', true) == k.dump(-1, ' ', true));
CHECK(e.root().dump(2, ' ', true) == k.dump(2, ' ', true));
CHECK(e.root().dump() == k.dump());
// strings that were edited from or to ASCII text
e.set(e.root()["list"][2], std::string("\xC3\xA4"));
k["list"][2] = "\xC3\xA4";
e.set(e.root()["list"][4], std::string("plain again"));
k["list"][4] = "plain again";
CHECK(e.root().dump(-1, ' ', true) == k.dump(-1, ' ', true));
}
}
SECTION("ensure_ascii with DEL or a non-ASCII character at every position")
{
// the check whether a string of the source has no byte >= 0x7F reads
// eight bytes at a time and then the rest one by one
for (std::size_t length = 1; length <= 20; ++length)
{
for (std::size_t pos = 0; pos < length; ++pos)
{
for (const char* const c :
{
"\x7F", "\x7E", "\xC3\xA4", "\xC3\xBF" // NOLINT(modernize-raw-string-literal)
})
{
const std::string s = std::string(pos, 'a') + c + std::string(length - pos - 1, 'b');
std::string text = "[\"";
text.append(s).append("\",{\"").append(s).append("\":1}]");
const json_document d = json_document::parse(text);
const json j = json::parse(text);
CHECK(d.root().dump(-1, ' ', true) == j.dump(-1, ' ', true));
CHECK(d.root().dump(1, ' ', true) == j.dump(1, ' ', true));
}
}
}
}
SECTION("streams and discarded views")
{
const json_document d = json_document::parse(R"({"a": [1, 2]})");
std::ostringstream compact;
compact << d.root();
CHECK(compact.str() == R"({"a":[1,2]})");
std::ostringstream pretty;
pretty << std::setw(2) << std::setfill('.') << d.root() << d.root()["a"];
CHECK(pretty.str() == "{\n..\"a\": [\n....1,\n....2\n..]\n}[1,2]");
CHECK(json_view().dump() == json(json::value_t::discarded).dump());
}
}
TEST_CASE("json_view large objects")
{
// objects with 128 members or more are looked up with a hash index
for (const std::size_t members :
{
127u, 128u, 129u, 10000u
})
{
CAPTURE(members)
std::string text = "{";
for (std::size_t i = 0; i < members; ++i)
{
text += (i != 0 ? ",\"" : "\"") + std::string(i % 23, 'k') + std::to_string(i) + (i % 7 == 0 ? "\\n" : "") + "\":" + std::to_string(i);
}
text += R"(,"":"empty key","k1":"a duplicate of an earlier key"})";
const json_document d = json_document::parse(text);
const json_view v = d.root();
const json j = json::parse(text);
for (std::size_t i = 0; i < members; ++i)
{
const std::string key = std::string(i % 23, 'k') + std::to_string(i) + (i % 7 == 0 ? "\n" : "");
CHECK(v[key].get<std::size_t>() == i);
CHECK(v.contains(key));
CHECK(v.find(key).key() == key);
CHECK(v.at(key).get<std::size_t>() == i);
CHECK(!v.contains(key + "x"));
}
CHECK(v[""].get_string() == "empty key");
CHECK(v["k1"].get<int>() == 1); // the first of duplicate keys, as for small objects
CHECK(!v.contains("missing"));
CHECK_THROWS_WITH_AS(v.at("missing"), "[json.exception.out_of_range.403] key 'missing' not found", json::out_of_range&);
CHECK(v == j);
CHECK(v.materialize() == j);
}
SECTION("duplicate keys: the first member wins, with and without a table")
{
// an object of `total` members: the keys "k0".."k<n-1>" in order, then
// three keys repeated twice more (one copy in the middle, one at the
// end), and two keys repeated once; the value of a member is its
// position, so that the first member of a key can be told apart
struct member
{
std::string key;
std::size_t position;
};
const auto make_members = [](std::size_t total)
{
std::vector<std::string> keys;
for (std::size_t i = 0; i + 8 < total; ++i)
{
keys.push_back("k" + std::to_string(i));
}
const std::size_t n = keys.size();
const std::array<std::size_t, 3> triple = {{3, 17, n - 1}};
const std::array<std::size_t, 2> twice = {{5, n / 2}};
std::vector<std::string> ordered = keys;
for (const std::size_t i : triple)
{
ordered.insert(ordered.begin() + static_cast<std::ptrdiff_t>(ordered.size() / 2), keys[i]);
}
for (const std::size_t i : twice)
{
ordered.insert(ordered.begin() + static_cast<std::ptrdiff_t>(ordered.size() / 3), keys[i]);
}
for (const std::size_t i : triple)
{
ordered.push_back(keys[i]);
}
std::vector<member> result;
result.reserve(ordered.size());
for (std::size_t i = 0; i < ordered.size(); ++i)
{
result.push_back({ordered[i], i});
}
return result;
};
// 100 and 127 members: no table; 128 members and more: a table
for (const std::size_t total :
{
100u, 127u, 128u, 200u, 5000u
})
{
CAPTURE(total)
const std::vector<member> members = make_members(total);
REQUIRE(members.size() >= total);
REQUIRE(members.size() >= 100);
std::string text = "{";
std::map<std::string, std::size_t> first;
std::map<std::string, std::size_t> last;
std::map<std::string, std::size_t> occurrences;
for (const member& m : members)
{
text += (text.size() > 1 ? ",\"" : "\"") + m.key + "\":" + std::to_string(m.position);
first.insert({m.key, m.position});
last[m.key] = m.position;
++occurrences[m.key];
}
text += '}';
REQUIRE(first.size() < members.size());
const json_document d = json_document::parse(text);
const json_view v = d.root();
const json j = json::parse(text);
CHECK(v.size() == members.size()); // every occurrence is visited
const json m = v.materialize();
for (const auto& entry : first)
{
CAPTURE(entry.first)
const std::size_t expected = entry.second;
CHECK(v[entry.first].get<std::size_t>() == expected);
CHECK(v.at(entry.first).get<std::size_t>() == expected);
CHECK(v.find(entry.first).value().get<std::size_t>() == expected);
CHECK(v.value(entry.first, std::size_t{0}) == expected);
CHECK(v.contains(entry.first));
CHECK(v.count(entry.first) == occurrences[entry.first]);
const std::string pointer = "/" + entry.first;
CHECK(v[json::json_pointer(pointer)].get<std::size_t>() == expected);
CHECK(v.at(json::json_pointer(pointer)).get<std::size_t>() == expected);
CHECK(v.value(json::json_pointer(pointer), std::size_t{0}) == expected);
CHECK(v.contains(json::json_pointer(pointer)));
// materialize() and parse() keep the last value instead
CHECK(j[entry.first].get<std::size_t>() == last.at(entry.first));
CHECK(m[entry.first].get<std::size_t>() == last.at(entry.first));
}
CHECK(!v.contains("k"));
CHECK(v["missing"].is_discarded());
CHECK(v.materialize() == j);
CHECK(v == j);
}
}
SECTION("colliding keys")
{
// the hash is not seeded, so keys that all land in one place must not
// make the table build quadratic: such an object gets no table and is
// searched linearly
constexpr std::size_t members = 300;
constexpr std::size_t table_size = 1024; // the table size for 300 members: the next power of two >= 600
std::vector<std::string> colliding;
std::vector<std::string> spread;
for (std::uint64_t counter = 0; colliding.size() < members || spread.size() < members; ++counter)
{
std::string key(8, 'a');
std::uint64_t x = counter;
for (std::size_t i = 0; i < 8; ++i, x /= 26)
{
key[i] = static_cast<char>('a' + (x % 26));
}
const bool lands_in_slot_zero = (nlohmann::detail::view::key_hash(key.data(), key.size()) & (table_size - 1)) == 0;
if (lands_in_slot_zero && colliding.size() < members)
{
colliding.push_back(key);
}
else if (!lands_in_slot_zero && spread.size() < members)
{
spread.push_back(key);
}
}
const auto make_text = [](const std::vector<std::string>& keys)
{
std::string text = "{";
for (std::size_t i = 0; i < keys.size(); ++i)
{
text += (i != 0 ? ",\"" : "\"") + keys[i] + "\":" + std::to_string(i % 10);
}
return text + "}";
};
const std::string colliding_text = make_text(colliding);
const std::string spread_text = make_text(spread);
json_document with_collisions = json_document::parse(colliding_text);
json_document without_collisions = json_document::parse(spread_text);
for (const auto* pair :
{
&colliding, &spread
})
{
const json_view v = (pair == &colliding ? with_collisions : without_collisions).root();
for (std::size_t i = 0; i < members; ++i)
{
CAPTURE(i)
CHECK(v[(*pair)[i]].get<std::size_t>() == i % 10);
CHECK(v.at((*pair)[i]).get<std::size_t>() == i % 10);
CHECK(v.find((*pair)[i]).key() == (*pair)[i]);
CHECK(!v.contains((*pair)[i] + "x"));
}
CHECK(!v.contains("missing"));
}
CHECK(with_collisions.root() == json::parse(colliding_text));
// only the object with the spread keys got a table (both texts have the
// same length, so the tables are the only difference)
with_collisions.shrink_to_fit();
without_collisions.shrink_to_fit();
CHECK(without_collisions.memory_usage() >= with_collisions.memory_usage() + (table_size * sizeof(std::uint32_t)));
}
SECTION("keys that differ in their last bytes are spread over the table")
{
// The hash must spread the usual families of keys (numbered names, hex
// strings, dates, short names): a key farther than index_max_displacement
// from its home slot would leave the object without a table. The keys
// are placed with linear probing, as the table does.
const auto max_displacement = [](const std::vector<std::string>& keys)
{
std::size_t slots = 16;
while (slots < 2 * keys.size())
{
slots *= 2;
}
std::vector<bool> used(slots, false);
std::size_t farthest = 0;
for (const std::string& key : keys)
{
std::size_t i = nlohmann::detail::conditional_static_cast<std::size_t>(nlohmann::detail::view::key_hash(key.data(), key.size())) & (slots - 1);
std::size_t distance = 0;
while (used[i])
{
i = (i + 1) & (slots - 1);
++distance;
}
used[i] = true;
farthest = (std::max)(farthest, distance);
}
return farthest;
};
// the keys of a printf-like pattern with one %d, %0Nd or %0Nx, formatted with a stream
// (a format string that is not a literal is a Flawfinder error)
const auto numbered = [](const char* format, int count, int step)
{
const std::string pattern(format);
const std::size_t percent = pattern.find('%');
std::size_t conversion = percent + 1;
std::size_t width = 0;
while (pattern[conversion] >= '0' && pattern[conversion] <= '9')
{
width = (width * 10) + static_cast<std::size_t>(pattern[conversion] - '0');
++conversion;
}
const bool hex = pattern[conversion] == 'x';
std::vector<std::string> keys;
for (int i = 0; i < count; ++i)
{
std::ostringstream key;
key << pattern.substr(0, percent) << std::setfill('0') << std::setw(static_cast<int>(width))
<< (hex ? std::hex : std::dec) << (i * step) << pattern.substr(conversion + 1);
keys.push_back(key.str());
}
return keys;
};
for (const char* format :
{"key%d", "%d", "item_%04d", "a%05d", "%03dxyz", "id%d", "user_%d", "%d_suffix", "0x%04x", "A%07d", "12345678%d", "1234567%d", "123456789%d",
"configuration_parameter_value_%04d_extra", "2026-01-%02dT00:00:00Z"
})
{
CAPTURE(format)
CHECK(max_displacement(numbered(format, std::string(format).find("%02d") != std::string::npos ? 60 : 1000, 1)) < nlohmann::detail::view::index_max_displacement); // NOLINT(abseil-string-find-str-contains)
CHECK(max_displacement(numbered(format, 600, 7)) < nlohmann::detail::view::index_max_displacement);
}
std::mt19937_64 rng(7); // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed)
for (const std::size_t length :
{
std::size_t{1}, std::size_t{2}, std::size_t{3}, std::size_t{5}, std::size_t{7}, std::size_t{8}, std::size_t{9}, std::size_t{12}, std::size_t{16}, std::size_t{17}, std::size_t{33}
})
{
CAPTURE(length)
std::set<std::string> distinct;
// (there are 26 and 676 keys of one and two letters)
std::size_t wanted = 700;
if (length == 1)
{
wanted = 26;
}
else if (length == 2)
{
wanted = 500;
}
while (distinct.size() < wanted)
{
std::string key(length, 'a');
for (char& c : key)
{
c = static_cast<char>('a' + (rng() % 26));
}
distinct.insert(key);
}
CHECK(max_displacement(std::vector<std::string>(distinct.begin(), distinct.end())) < nlohmann::detail::view::index_max_displacement);
}
}
SECTION("keys of every length that differ in one byte")
{
// In a large object, a key is found by comparing its bytes with those of the member the
// table points to: up to 16 bytes as two overlapping words, longer ones with memcmp.
// Keys of every length from 0 to 40, and keys that differ from them in one byte.
std::mt19937_64 rng(2026); // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed)
std::vector<std::string> keys;
std::set<std::string> distinct;
for (std::size_t length = 0; length <= 40; ++length)
{
for (int copy = 0; copy < 5; ++copy)
{
std::string key(length, 'a');
for (char& c : key)
{
c = static_cast<char>('a' + (rng() % 26));
}
if (distinct.insert(key).second)
{
keys.push_back(key);
}
}
}
std::string text = "{";
for (std::size_t i = 0; i < keys.size(); ++i)
{
text += (i != 0 ? ",\"" : "\"") + keys[i] + "\":" + std::to_string(i);
}
text += "}";
REQUIRE(keys.size() >= 128); // (a table is built for 128 members)
const json_document d = json_document::parse(text);
const json_view v = d.root();
for (std::size_t i = 0; i < keys.size(); ++i)
{
CAPTURE(keys[i])
CHECK(v[keys[i]].get<std::size_t>() == i);
CHECK(v.at(keys[i]).get<std::size_t>() == i);
CHECK(v.contains(keys[i]));
for (std::size_t at = 0; at < keys[i].size(); ++at)
{
std::string other = keys[i];
other[at] = other[at] == 'z' ? 'y' : 'z';
CAPTURE(other)
CHECK(v.contains(other) == (distinct.count(other) != 0));
}
CHECK(v.contains(keys[i] + "a") == (distinct.count(keys[i] + "a") != 0));
if (!keys[i].empty())
{
CHECK(v.contains(keys[i].substr(1)) == (distinct.count(keys[i].substr(1)) != 0));
}
}
}
SECTION("shrink_to_fit releases the tables' spare capacity")
{
const auto make_text = [](int objects, int members)
{
std::string text = "[";
for (int object = 0; object < objects; ++object)
{
text += object != 0 ? ",{" : "{";
for (int i = 0, n = members + object; i < n; ++i)
{
text += (i != 0 ? ",\"" : "\"") + std::to_string(i) + "\":" + std::to_string(i);
}
text += '}';
}
return text + "]";
};
const std::string small_text = make_text(5, 150);
const std::string big_text = make_text(40, 400);
// reading a big text, and then a small one, leaves the spare capacity
// of the big one: shrink_to_fit() brings the document to the size of
// one parsed from the small text alone
json_document d = json_document::parse(big_text);
const std::size_t big = d.memory_usage();
d.read(small_text);
CHECK(d.memory_usage() >= big);
d.shrink_to_fit();
json_document fresh = json_document::parse(small_text);
fresh.shrink_to_fit();
CHECK(d.memory_usage() == fresh.memory_usage());
CHECK(d.memory_usage() < big / 2);
CHECK(d.root() == json::parse(small_text));
for (int object = 0; object < 5; ++object)
{
const json_view v = d.root()[static_cast<std::size_t>(object)];
for (int i = 0, n = 150 + object; i < n; ++i)
{
CHECK(v[std::to_string(i)].get<int>() == i);
}
}
}
SECTION("nested, reused, and in arrays")
{
std::string inner = "{";
for (int i = 0; i < 300; ++i)
{
inner += (i != 0 ? ",\"m" : "\"m") + std::to_string(i) + "\":" + std::to_string(i);
}
inner += '}';
const std::string text = "[" + inner + ",{\"x\":" + inner + "}," + inner + "]";
json_document d = json_document::parse(text);
CHECK(d.root()[0]["m299"].get<int>() == 299);
CHECK(d.root()[1]["x"]["m150"].get<int>() == 150);
CHECK(d.root()[2]["m0"].get<int>() == 0);
const std::size_t with_index = d.memory_usage();
d.read(std::string("{\"small\": 1}"));
CHECK(d.root()["small"].get<int>() == 1);
d.read(text);
CHECK(d.root()[2]["m7"].get<int>() == 7);
CHECK(d.memory_usage() >= with_index / 2);
}
}
-298
View File
@@ -1,298 +0,0 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#include "doctest_compatibility.h"
#include <nlohmann/json_view.hpp>
using nlohmann::json;
using nlohmann::ordered_json;
using nlohmann::json_document;
using nlohmann::ordered_json_editable_document;
using nlohmann::json_view;
using nlohmann::ordered_json_document;
using nlohmann::ordered_json_view;
#include <algorithm>
#include <array>
#include <cmath>
#include <cstddef>
#include <cstdint>
#include <cstring>
#include <iomanip>
#include <map>
#include <random>
#include <set>
#include <sstream>
#include <string>
#include <vector>
// These tests were split off unit-json_view_dump.cpp, whose object file got too large for the MinGW
// linker (see json_view_test_helpers.hpp). Like that file, this one is also built
// with C++17, as it mentions JSON_HAS_CPP_17.
TEST_CASE("json_view dump: escaping, nesting, and output buffers")
{
SECTION("members in document order, all of them")
{
const json_document d = json_document::parse(R"({"b": 1, "a": 2, "b": 3})");
CHECK(d.root().dump() == R"({"b":1,"a":2,"b":3})");
CHECK(d.root().dump(1) == "{\n \"b\": 1,\n \"a\": 2,\n \"b\": 3\n}");
// dump() writes all members, though the lookup finds the first
CHECK(d.root()["b"].dump() == "1");
}
SECTION("deep nesting")
{
const std::string deep = std::string(100000, '[') + std::string(100000, ']');
const json_document deep_document = json_document::parse(deep);
CHECK(deep_document.root().dump() == deep);
}
SECTION("ensure_ascii: a character that needs an escape at each position of a string")
{
// string lengths around the width of the vector steps, in a string value, a key, and a value near the end of the text
for (const char* const special :
{
"\xC3\xA9", "\xE4\xB8\xAD", "\xF0\x9F\x98\x80", "\x7F", "\\n", "\\u0001", "\\\"", "\\\\"
})
{
for (std::size_t length = 0; length <= 40; ++length)
{
for (std::size_t position = 0; position <= length; ++position)
{
std::string content(length, 'a');
content.insert(position, special);
const std::array<std::string, 3> texts =
{
{
"\"" + content + "\"",
"[\"" + content + "\"]",
"{\"" + content + "\":1}",
}
};
for (const std::string& text : texts)
{
CAPTURE(text)
const ordered_json_document d = ordered_json_document::parse(text);
const ordered_json j = ordered_json::parse(text);
CHECK(d.root().dump(-1, ' ', true) == j.dump(-1, ' ', true));
CHECK(d.root().dump(2, ' ', true) == j.dump(2, ' ', true));
CHECK(d.root().dump() == j.dump());
}
}
}
}
}
SECTION("ensure_ascii: every character of the Basic Multilingual Plane and some beyond")
{
// (the two- and three-byte sequences, including the ranges that need a check: E0, ED, EF BF BF)
std::string text = "[";
std::size_t in_chunk = 0;
const auto append = [&](std::uint32_t cp)
{
if (cp < 0x80)
{
if (cp < 0x20 || cp == '"' || cp == '\\')
{
return;
}
text += static_cast<char>(cp);
}
else if (cp < 0x800)
{
text += static_cast<char>(0xC0 | (cp >> 6));
text += static_cast<char>(0x80 | (cp & 0x3F));
}
else if (cp < 0x10000)
{
text += static_cast<char>(0xE0 | (cp >> 12));
text += static_cast<char>(0x80 | ((cp >> 6) & 0x3F));
text += static_cast<char>(0x80 | (cp & 0x3F));
}
else
{
text += static_cast<char>(0xF0 | (cp >> 18));
text += static_cast<char>(0x80 | ((cp >> 12) & 0x3F));
text += static_cast<char>(0x80 | ((cp >> 6) & 0x3F));
text += static_cast<char>(0x80 | (cp & 0x3F));
}
};
for (std::uint32_t cp = 0x20; cp < 0x10400; ++cp)
{
if (cp >= 0xD800 && cp < 0xE000)
{
continue;
}
if (in_chunk == 0)
{
text += text.size() > 1 ? ",\"" : "\"";
}
append(cp);
if (++in_chunk == 37)
{
text += "\"";
in_chunk = 0;
}
}
text += in_chunk != 0 ? "\"]" : "]";
const ordered_json_document d = ordered_json_document::parse(text);
const ordered_json j = ordered_json::parse(text);
CHECK(d.root().dump(-1, ' ', true) == j.dump(-1, ' ', true));
CHECK(d.root().dump(2, ' ', true) == j.dump(2, ' ', true));
CHECK(d.root().dump() == j.dump());
CHECK(d.root().dump(2) == j.dump(2));
}
SECTION("deep nesting, indented")
{
// indentations of up to and beyond the width of the writer's fixed-size copy
for (const int depth :
{
1, 5, 20, 70, 300
})
{
std::string text;
for (int i = 0; i < depth; ++i)
{
text += i % 2 == 0 ? "[1," : R"({"k":2,"\u00e9":)";
}
text += "[]";
for (int i = depth - 1; i >= 0; --i)
{
text += i % 2 == 0 ? "]" : "}";
}
const ordered_json_document d = ordered_json_document::parse(text);
const ordered_json j = ordered_json::parse(text);
for (const int indent :
{
0, 1, 2, 3, 8
})
{
CHECK(d.root().dump(indent) == j.dump(indent));
CHECK(d.root().dump(indent, '\t') == j.dump(indent, '\t'));
}
}
}
SECTION("the output buffer of a small value is small")
{
// an escaped key after the value: its node lies in the arena, so the
// source extent of the value cannot be read from the next node
const std::string big(100000, 'a');
const std::string text = R"({"small":1,"list":[1,2,3],"k\n":")" + big + R"("})";
const json_document d = json_document::parse(text);
const auto small = d.root()["small"].dump();
CHECK(small == "1");
CHECK(small.capacity() < 4096);
const auto list = d.root()["list"].dump();
CHECK(list == "[1,2,3]");
CHECK(list.capacity() < 4096);
CHECK(d.root()["list"].dump(2).capacity() < 4096);
// the whole document and the large value are unaffected
CHECK(d.root().dump() == ordered_json::parse(text).dump());
CHECK(d.root()["k\n"].dump() == "\"" + big + "\"");
}
SECTION("output that outgrows the estimate")
{
// ensure_ascii writes six bytes for each two-byte character
std::string chars;
for (int i = 0; i < 5000; ++i)
{
chars += "\xC3\xA9";
}
const json_document d = json_document::parse(R"({"a":")" + chars + R"(","k\n":1})");
const json expected = json::parse("\"" + chars + "\"");
CHECK(d.root()["a"].dump(-1, ' ', true) == expected.dump(-1, ' ', true));
CHECK(d.root()["a"].dump(-1, ' ', true).size() == 2 + (5000 * 6));
}
SECTION("ensure_ascii with strings of the source, decoded strings, and edited strings")
{
// plain ASCII, non-ASCII, escaped (decoded), and mixed strings, as values and as keys;
// the second text has a DEL, which ensure_ascii writes as an escape
for (const char* const del :
{"", "\x7F"
})
{
const std::string text = std::string("{\"plain\":\"abc\",\"caf\xC3\xA9\":\"\xC3\xA9\xE2\x82\xAC\xF0\x9F\x98\x80\",")
+ "\"esc\\n\\\"\":\"a\\tb\\u00e4\\ud83d\\ude00\",\"del\":\"a" + del + "b\",\"\":\"\"," // NOLINT(modernize-raw-string-literal)
+ "\"list\":[\"x\",\"\xC3\xBC\",\"\\\\\",\"long ASCII string with more than thirty-two bytes in it\","
+ "\"long string with more than thirty-two bytes and one non-ASCII byte \xC3\xB6\"],"
+ "\"n\":[1,2.5,-0,true,null]}";
const ordered_json j = ordered_json::parse(text);
const ordered_json_document d = ordered_json_document::parse(text);
CHECK(d.root().dump(-1, ' ', true) == j.dump(-1, ' ', true));
CHECK(d.root().dump(4, ' ', true) == j.dump(4, ' ', true));
CHECK(d.root().dump() == j.dump());
CHECK(d.root()["list"].dump(-1, ' ', true) == j["list"].dump(-1, ' ', true));
CHECK(d.root()["del"].dump(-1, ' ', true) == j["del"].dump(-1, ' ', true));
// strings that an edit wrote (the source strings that they replace were plain)
ordered_json_editable_document e = ordered_json_editable_document::parse(text);
ordered_json k = j;
e.set(e.root()["plain"], std::string("\xC3\xA4gypten"));
k["plain"] = "\xC3\xA4gypten";
e.set(e.root()["list"][0], std::string("ascii after edit"));
k["list"][0] = "ascii after edit";
e.set(e.root()["list"][1], std::string("\xC3\xA4"));
k["list"][1] = "\xC3\xA4";
e.set(e.root()["del"], std::string("tab\there"));
k["del"] = "tab\there";
CHECK(e.root().dump(-1, ' ', true) == k.dump(-1, ' ', true));
CHECK(e.root().dump(2, ' ', true) == k.dump(2, ' ', true));
CHECK(e.root().dump() == k.dump());
// strings that were edited from or to ASCII text
e.set(e.root()["list"][2], std::string("\xC3\xA4"));
k["list"][2] = "\xC3\xA4";
e.set(e.root()["list"][4], std::string("plain again"));
k["list"][4] = "plain again";
CHECK(e.root().dump(-1, ' ', true) == k.dump(-1, ' ', true));
}
}
SECTION("ensure_ascii with DEL or a non-ASCII character at every position")
{
// the check whether a string of the source has no byte >= 0x7F reads
// eight bytes at a time and then the rest one by one
for (std::size_t length = 1; length <= 20; ++length)
{
for (std::size_t pos = 0; pos < length; ++pos)
{
for (const char* const c :
{
"\x7F", "\x7E", "\xC3\xA4", "\xC3\xBF" // NOLINT(modernize-raw-string-literal)
})
{
const std::string s = std::string(pos, 'a') + c + std::string(length - pos - 1, 'b');
std::string text = "[\"";
text.append(s).append("\",{\"").append(s).append("\":1}]");
const json_document d = json_document::parse(text);
const json j = json::parse(text);
CHECK(d.root().dump(-1, ' ', true) == j.dump(-1, ' ', true));
CHECK(d.root().dump(1, ' ', true) == j.dump(1, ' ', true));
}
}
}
}
SECTION("streams and discarded views")
{
const json_document d = json_document::parse(R"({"a": [1, 2]})");
std::ostringstream compact;
compact << d.root();
CHECK(compact.str() == R"({"a":[1,2]})");
std::ostringstream pretty;
pretty << std::setw(2) << std::setfill('.') << d.root() << d.root()["a"];
CHECK(pretty.str() == "{\n..\"a\": [\n....1,\n....2\n..]\n}[1,2]");
CHECK(json_view().dump() == json(json::value_t::discarded).dump());
}
}
+120
View File
@@ -941,3 +941,123 @@ TEST_CASE("json_view edits: duplicate keys")
CHECK(d.root()["o"]["d"].get<int>() == 0);
}
}
#if !defined(JSON_NOEXCEPTION)
TEST_CASE("json_view edits: check_utf8 agrees with the byte-wise check")
{
namespace dv = nlohmann::detail::view;
// "ok" or the exception message
const auto run = [](bool fast, const std::string & s) -> std::string
{
try
{
if (fast)
{
dv::check_utf8(s.data(), s.size());
}
else
{
dv::check_utf8_bytewise(reinterpret_cast<const unsigned char*>(s.data()), s.size(), 0);
}
}
catch (const json::type_error& e)
{
return {e.what()};
}
return {"ok"};
};
std::size_t errors = 0;
const auto same = [&](const std::string & s)
{
const std::string a = run(true, s);
const std::string b = run(false, s);
if (a != b)
{
INFO("fast: " << a << ", byte-wise: " << b << ", length " << s.size());
CHECK(a == b);
}
errors += b != "ok" ? 1u : 0u;
};
const std::vector<std::string> pieces =
{
"a", "Z", " ", "\"", "\\", "\n", "\x01", "\x7f", "\xc3\xa4", "\xe4\xb8\xad", "\xf0\x9f\x98\x80",
"\x80", "\xbf", "\xc0\x80", "\xc1\xbf", "\xc2", "\xe0\x80\x80", "\xe0\x9f\xbf", "\xed\xa0\x80", "\xed\xbf\xbf",
"\xf0\x80\x80\x80", "\xf0\x8f\xbf\xbf", "\xf4\x90\x80\x80", "\xf4\x8f\xbf\xbf", "\xf5", "\xf8", "\xff", "\xfe",
"\xe4\xb8", "\xf0\x9f\x98", "\xf0\x9f", "\xf0"
};
// every piece behind prefixes of every length, and followed by more bytes
for (std::size_t prefix = 0; prefix <= 64; ++prefix)
{
for (const auto& piece : pieces)
{
same(std::string(prefix, 'x') + piece);
same(std::string(prefix, 'x') + piece + "yz");
same(std::string(prefix, '"') + piece);
same(std::string(prefix, '\xc3') + piece);
same(std::string(prefix / 2, '\xc3').append(prefix / 2, '\xa4') + piece);
same(std::string(prefix, 'x') + "\xe4\xb8\xad" + piece);
}
}
// an invalid byte at every position of valid strings of every length
for (std::size_t len = 0; len <= 64; ++len)
{
const std::string ascii(len, 'a');
std::string mixed;
while (mixed.size() < len)
{
if (mixed.size() % 7 == 0)
{
mixed += "\xc3\xa4";
}
else if (mixed.size() % 11 == 0)
{
mixed += "\xf0\x9f\x98\x80";
}
else
{
mixed += "a";
}
}
same(ascii);
same(mixed);
for (std::size_t pos = 0; pos < len; ++pos)
{
for (const char bad :
{'\x80', '\xc0', '\xc2', '\xe0', '\xf4', '\xff', '"', '\\', '\x1f'
})
{
std::string s = ascii;
s[pos] = bad;
same(s);
s = mixed;
s[pos] = bad;
same(s);
}
}
}
// random bytes, biased to the interesting ones
std::mt19937 gen(5869); // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed): reproducible
static const std::array<unsigned char, 23> interesting = {{0x00, 0x1f, 0x22, 0x5c, 0x7f, 0x80, 0x8f, 0x90, 0x9f, 0xa0, 0xbf, 0xc0, 0xc2, 0xdf, 0xe0, 0xe1, 0xed, 0xef, 0xf0, 0xf1, 0xf4, 0xf5, 0xff}};
for (int i = 0; i < 100000; ++i)
{
std::string s(gen() % 80, 'a');
for (auto& c : s)
{
const unsigned r = gen() % 8;
if (r == 0)
{
c = static_cast<char>(gen() & 0xff);
}
else if (r == 1)
{
c = static_cast<char>(interesting[gen() % interesting.size()]);
}
}
same(s);
}
CHECK(errors > 1000);
}
#endif
+8 -4
View File
@@ -54,14 +54,18 @@ int accepts_any(nlohmann::basic_json_view<BasicJsonType, Mode> /*unused*/)
return Mode == nlohmann::view_mode::editable ? 3 : 4;
}
template<typename View>
auto overload(View v) -> decltype(accepts_editable(v))
// overloads that differ only in the mode; deduction ignores the conversion from
// read-only to editable views, so each view picks the overload of its own mode
// (two templates that differ only in a decltype return type are a redefinition
// for MSVC 2015)
template<typename BasicJsonType>
int overload(nlohmann::basic_json_view<BasicJsonType, nlohmann::view_mode::editable> v)
{
return accepts_editable(v);
}
template<typename View>
auto overload(View v) -> decltype(accepts_read_only(v))
template<typename BasicJsonType>
int overload(nlohmann::basic_json_view<BasicJsonType, nlohmann::view_mode::read_only> v)
{
return accepts_read_only(v);
}
+27 -14
View File
@@ -69,9 +69,12 @@ struct point
struct shape
{
// initialized for GCC -Weffc++
// NOLINTNEXTLINE(readability-redundant-member-init)
std::string name{};
// NOLINTNEXTLINE(readability-redundant-member-init)
std::vector<point> points{};
point center{};
// NOLINTNEXTLINE(readability-redundant-member-init)
std::map<std::string, point> named{};
NLOHMANN_DEFINE_TYPE_INTRUSIVE(shape, name, points, center, named)
};
@@ -82,13 +85,17 @@ struct with_defaults
std::string b = "default";
std::vector<int> c = {1, 2};
};
// NOLINTNEXTLINE(misc-use-internal-linkage): found by ADL, like the from_json() functions of users
NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE_WITH_DEFAULT(with_defaults, a, b, c)
struct with_names
{
int value = 0;
std::string text{}; // initialized for GCC -Weffc++
// initialized for GCC -Weffc++
// NOLINTNEXTLINE(readability-redundant-member-init)
std::string text{};
};
// NOLINTNEXTLINE(misc-use-internal-linkage): found by ADL, like the from_json() functions of users
NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE_WITH_NAMES(with_names, "Value", value, "Text", text)
struct empty
@@ -102,6 +109,7 @@ enum class color
green,
blue
};
// NOLINTNEXTLINE(misc-use-internal-linkage): found by ADL, like the from_json() functions of users
NLOHMANN_JSON_SERIALIZE_ENUM(color,
{
{color::red, "red"},
@@ -114,6 +122,7 @@ enum class strict_color
red,
green
};
// NOLINTNEXTLINE(misc-use-internal-linkage): found by ADL, like the from_json() functions of users
NLOHMANN_JSON_SERIALIZE_ENUM_STRICT(strict_color,
{
{strict_color::red, "red"},
@@ -154,6 +163,7 @@ struct derived_from_legacy : legacy_base
{
int extra = 0;
};
// NOLINTNEXTLINE(misc-use-internal-linkage): found by ADL, like the from_json() functions of users
NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE(derived_from_legacy, legacy_base, extra)
// from_json() for json only: converted through materialize()
@@ -301,18 +311,19 @@ TEST_CASE("json_view from_json")
CHECK(same<std::string>(v, j, "text"));
CHECK(same<std::nullptr_t>(v, j, "nothing"));
CHECK(same<json>(v, j, "object"));
const std::valarray<int> va = v["ints"].get<std::valarray<int>>();
const auto va = v["ints"].get<std::valarray<int>>();
CHECK(va.size() == 4);
CHECK(va[0] == 3);
int c_array[2][2] = {{0, 0}, {0, 0}}; // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays): the conversion to C arrays
v["nested"][0].get_to(c_array[0]);
CHECK(c_array[0][1] == 2);
#ifdef JSON_HAS_CPP_17
CHECK(v["points"][0].get<std::optional<point>>()->y == 2);
const auto first = v["points"][0].get<std::optional<point>>();
CHECK((first.has_value() && first->y == 2));
CHECK(!v["nothing"].get<std::optional<point>>().has_value());
#endif
// containers of views keep views of the elements
const std::vector<json_view> views = v["points"].get<std::vector<json_view>>();
const auto views = v["points"].get<std::vector<json_view>>();
CHECK(views.size() == 2);
CHECK(views[1]["x"].get<int>() == 3);
CHECK(v["object"].get<std::map<std::string, json_view>>().at("a").get<int>() == 2);
@@ -326,7 +337,8 @@ TEST_CASE("json_view from_json")
CHECK(d.root().get<std::unordered_map<std::string, int>>().at("a") == 3);
}
#if !defined(JSON_NOEXCEPTION) && !JSON_DIAGNOSTICS
// (the exceptions of views have neither the path nor the byte range of the value)
#if !defined(JSON_NOEXCEPTION) && !JSON_DIAGNOSTICS && !JSON_DIAGNOSTIC_POSITIONS
SECTION("std types: the same errors as from the materialized value")
{
const json_document d = json_document::parse(R"([[1, "x"], {"a": 1}, "text", [1], 5, null])");
@@ -352,7 +364,7 @@ TEST_CASE("json_view from_json")
{
const json_document d = json_document::parse(R"({"name": "s", "points": [{"x": 1, "y": 2}, {"x": 3, "y": 4}],
"center": {"x": 5, "y": 6}, "named": {"a": {"x": 7, "y": 8}}})");
const shape s = d.root().get<shape>();
const auto s = d.root().get<shape>();
CHECK(s.name == "s");
CHECK(s.points.size() == 2);
CHECK(s.points[1].y == 4);
@@ -370,13 +382,13 @@ TEST_CASE("json_view from_json")
SECTION("defaults and names")
{
const json_document d = json_document::parse(R"({"a": 5, "Value": 3, "Text": "t"})");
const with_defaults w = d.root().get<with_defaults>();
const auto w = d.root().get<with_defaults>();
CHECK(w.a == 5);
CHECK(w.b == "default");
CHECK(w.c == std::vector<int> {1, 2});
const json_document null_document = json_document::parse("null");
CHECK(null_document.root().get<with_defaults>().a == 1);
const with_names n = d.root().get<with_names>();
const auto n = d.root().get<with_names>();
CHECK(n.value == 3);
CHECK(n.text == "t");
}
@@ -392,7 +404,7 @@ TEST_CASE("json_view from_json")
#if !defined(JSON_NOEXCEPTION)
const json j = d.root().materialize();
CHECK(exception_of_call([&] { static_cast<void>(d.root()[2].get<strict_color>()); }) == "[json.exception.out_of_range.410] enum value out of range for strict_color: \"purple\"");
#if !JSON_DIAGNOSTICS
#if !JSON_DIAGNOSTICS && !JSON_DIAGNOSTIC_POSITIONS
CHECK(exception_of_call([&] { static_cast<void>(d.root()[2].get<strict_color>()); }) == exception_of_call([&] { static_cast<void>(j[2].get<strict_color>()); }));
#endif
#endif
@@ -401,13 +413,13 @@ TEST_CASE("json_view from_json")
SECTION("derived types")
{
const json_document d = json_document::parse(R"({"id": 7, "c": "green", "extra": 3})");
const derived x = d.root().get<derived>();
const auto x = d.root().get<derived>();
CHECK(x.id == 7);
CHECK(x.c == color::green);
const derived_with_defaults y = d.root().get<derived_with_defaults>();
const auto y = d.root().get<derived_with_defaults>();
CHECK(y.id == 7);
CHECK(y.label == "none");
const derived_from_legacy z = d.root().get<derived_from_legacy>();
const auto z = d.root().get<derived_from_legacy>();
CHECK(z.id == 7);
CHECK(z.extra == 3);
}
@@ -440,10 +452,11 @@ TEST_CASE("json_view from_json")
CHECK(o.root().get<shape>().center.y == 2);
}
#if !defined(JSON_NOEXCEPTION) && !JSON_DIAGNOSTICS
#if !defined(JSON_NOEXCEPTION) && !JSON_DIAGNOSTICS && !JSON_DIAGNOSTIC_POSITIONS
SECTION("errors are those of the same conversion of a json value")
{
// (with JSON_DIAGNOSTICS, those of json name the path of the value)
// (with JSON_DIAGNOSTICS, those of json name the path of the value,
// and with JSON_DIAGNOSTIC_POSITIONS its bytes)
const json_document d = json_document::parse(R"([{"x": 1}, {"x": "1", "y": 2}, [1, 2], {"id": "7", "c": "red"}])");
const json j = d.root().materialize();
for (std::size_t i = 0; i < 3; ++i)
+62 -2
View File
@@ -185,8 +185,10 @@ std::uint32_t rng()
TEST_CASE("json_view images: round trips")
{
static_assert(json_view_image_test::has_save<json_document>::value && json_view_image_test::has_load<json_document>::value, "save() and load() on a little-endian target");
static_assert(json_view_image_test::has_save<ordered_json_editable_document>::value && json_view_image_test::has_load<ordered_json_editable_document>::value, "save() and load() on a little-endian target");
static_assert(json_view_image_test::has_save<json_document>::value, "save() on a little-endian target");
static_assert(json_view_image_test::has_load<json_document>::value, "load() on a little-endian target");
static_assert(json_view_image_test::has_save<ordered_json_editable_document>::value, "save() on a little-endian target");
static_assert(json_view_image_test::has_load<ordered_json_editable_document>::value, "load() on a little-endian target");
SECTION("small documents")
{
@@ -699,6 +701,64 @@ TEST_CASE("json_view images: ownership")
#if !defined(JSON_NOEXCEPTION)
TEST_CASE("json_view images: errors")
{
SECTION("float tokens: a changed byte is accepted if the text is still a number with the digits the node records")
{
for (const char* const token :
{
"-12.375", "0.5", "123456.0", "-0.0", "9999999999999999999.5", "0.000000000000000000000001", "1.25e10", "5E-3"
})
{
CAPTURE(token)
const std::string text = std::string("[") + token + "]";
const json_document d = json_document::parse(text);
const std::vector<std::uint8_t> image = d.save();
const std::size_t at = text_at(image) + 1; // the first byte of the token
const std::uint16_t recorded = node_at(image, 1).extra;
CHECK(loaded_dump(image) == json::parse(text).dump());
for (std::size_t pos = 0; token[pos] != '\0'; ++pos)
{
for (unsigned int byte = 0; byte < 256; ++byte)
{
std::vector<std::uint8_t> changed = image;
changed[at + pos] = static_cast<std::uint8_t>(byte);
std::string t = token;
t[pos] = static_cast<char>(byte);
// the digits of the integer part and of the fraction of the changed token
std::size_t integer = 0;
std::size_t fraction = 0;
std::size_t k = (t[0] == '-') ? 1 : 0;
for (; k < t.size() && t[k] >= '0' && t[k] <= '9'; ++k)
{
++integer;
}
if (k < t.size() && t[k] == '.')
{
for (++k; k < t.size() && t[k] >= '0' && t[k] <= '9'; ++k)
{
++fraction;
}
}
// (as a JSON text, the token may have white space around it)
const bool float_token = json::accept(t) && t.find_first_of(std::string(" \t\r\n\0", 5)) == std::string::npos && (t.find('.') != std::string::npos || t.find_first_of("eE") != std::string::npos) // NOLINT(abseil-string-find-str-contains)
&& ((integer | (fraction << 8U)) == recorded);
CAPTURE(pos)
CAPTURE(byte)
std::string result;
try
{
result = loaded_dump(changed);
}
catch (const json::parse_error& e)
{
CHECK(std::string(e.what()) == check_failed);
result = "rejected";
}
CHECK((result != "rejected") == float_token);
}
}
}
}
SECTION("invalid UTF-8 of a decoded string: the compact dump writes it, the others throw")
{
const json_document d = json_document::parse(R"({"k\n":"\u00e9x"})");
-544
View File
@@ -1,544 +0,0 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#include "doctest_compatibility.h"
#include <nlohmann/json_view.hpp>
using nlohmann::json;
using nlohmann::ordered_json;
using nlohmann::json_document;
using nlohmann::ordered_json_editable_document;
using nlohmann::json_view;
using nlohmann::ordered_json_document;
using nlohmann::ordered_json_view;
#include "json_view_test_helpers.hpp"
using json_view_test::generator;
#include <algorithm>
#include <array>
#include <cmath>
#include <cstddef>
#include <cstdint>
#include <cstring>
#include <iomanip>
#include <map>
#include <random>
#include <set>
#include <sstream>
#include <string>
#include <vector>
// These tests were split off unit-json_view_dump.cpp, whose object file got too large for the MinGW
// linker (see json_view_test_helpers.hpp). Like that file, this one is also built
// with C++17, as it mentions JSON_HAS_CPP_17.
TEST_CASE("json_view comparison")
{
SECTION("equality of the values parse() produces")
{
generator g;
std::vector<std::string> texts;
for (int i = 0; i < 600; ++i)
{
std::string text;
g.value(text, 0);
texts.push_back(text);
// the same value written differently: sorted keys, canonical numbers
texts.push_back(json::parse(text).dump(1));
}
for (std::size_t i = 0; i + 2 < texts.size(); ++i)
{
for (std::size_t k = i; k < i + 3; ++k)
{
CAPTURE(texts[i])
CAPTURE(texts[k])
const json_document a = json_document::parse(texts[i]);
const json_document b = json_document::parse(texts[k]);
const json ja = json::parse(texts[i]);
const json jb = json::parse(texts[k]);
CHECK((a.root() == b.root()) == (ja == jb));
CHECK((a.root() != b.root()) == (ja != jb));
CHECK((a.root() == jb) == (ja == jb));
CHECK((jb == a.root()) == (ja == jb));
CHECK((a.root() != jb) == (ja != jb));
CHECK((jb != a.root()) == (ja != jb));
// ordered_json compares members in order
const ordered_json_document oa = ordered_json_document::parse(texts[i]);
const ordered_json_document ob = ordered_json_document::parse(texts[k]);
const ordered_json oja = ordered_json::parse(texts[i]);
const ordered_json ojb = ordered_json::parse(texts[k]);
CHECK((oa.root() == ob.root()) == (oja == ojb));
CHECK((oa.root() == ojb) == (oja == ojb));
}
}
}
SECTION("numbers, duplicate keys, member order")
{
const auto same = [](const char* x, const char* y)
{
const json_document dx = json_document::parse(x);
const json_document dy = json_document::parse(y);
return dx.root() == dy.root();
};
CHECK(same("1", "1.0"));
CHECK(same("[1, -1, 2.5]", "[1.0, -1.0, 25e-1]"));
CHECK(!same("1", "1.5"));
CHECK(same("18446744073709551615", "18446744073709551615"));
CHECK(same(R"({"a": 1, "a": 2})", R"({"a": 2})"));
CHECK(!same(R"({"a": 1, "a": 2})", R"({"a": 1})"));
CHECK(same(R"({"a": 1, "b": 2})", R"({"b": 2, "a": 1})"));
CHECK(!same(R"({"a": 1})", R"({"a": 1, "b": 2})"));
CHECK(!same("[1, 2]", "[2, 1]"));
CHECK(!same("\"a\"", "\"b\""));
CHECK(same("\"\\u00e9\"", "\"\xc3\xa9\""));
CHECK(!same("null", "false"));
CHECK(!same("[]", "{}"));
const ordered_json_document dup = ordered_json_document::parse(R"({"a": 1, "b": 2, "a": 3})");
const ordered_json_document last = ordered_json_document::parse(R"({"a": 3, "b": 2})");
CHECK(dup.root() == last.root());
// (== compares as parse() resolves duplicates, a lookup finds the first)
CHECK(dup.root()["a"].materialize() == 1);
CHECK(dup.root()["a"] != last.root()["a"]);
const ordered_json_document ab = ordered_json_document::parse(R"({"a": 1, "b": 2})");
const ordered_json_document ba = ordered_json_document::parse(R"({"b": 2, "a": 1})");
CHECK(ab.root() != ba.root());
// discarded values compare as basic_json's do
const json discarded(json::value_t::discarded);
CHECK((json_view() == json_view()) == (discarded == discarded)); // NOLINT(readability-container-size-empty): operator== is tested
CHECK((json_view() == discarded) == (discarded == discarded)); // NOLINT(readability-container-size-empty)
const json_document null_document = json_document::parse("null");
CHECK(!(json_view() == null_document.root())); // NOLINT(readability-container-size-empty)
CHECK(!(null_document.root() == discarded));
}
SECTION("deep nesting")
{
const std::string deep = std::string(100000, '[') + std::string(100000, ']');
const json_document a = json_document::parse(deep);
const json_document b = json_document::parse(deep);
CHECK(a.root() == b.root());
CHECK(a.root() == json::parse(deep));
const std::string other = std::string(100000, '[') + "1" + std::string(100000, ']');
const json_document c = json_document::parse(other);
CHECK(a.root() != c.root());
}
}
TEST_CASE("json_view large objects")
{
// objects with 128 members or more are looked up with a hash index
for (const std::size_t members :
{
127u, 128u, 129u, 10000u
})
{
CAPTURE(members)
std::string text = "{";
for (std::size_t i = 0; i < members; ++i)
{
text += (i != 0 ? ",\"" : "\"") + std::string(i % 23, 'k') + std::to_string(i) + (i % 7 == 0 ? "\\n" : "") + "\":" + std::to_string(i);
}
text += R"(,"":"empty key","k1":"a duplicate of an earlier key"})";
const json_document d = json_document::parse(text);
const json_view v = d.root();
const json j = json::parse(text);
for (std::size_t i = 0; i < members; ++i)
{
const std::string key = std::string(i % 23, 'k') + std::to_string(i) + (i % 7 == 0 ? "\n" : "");
CHECK(v[key].get<std::size_t>() == i);
CHECK(v.contains(key));
CHECK(v.find(key).key() == key);
CHECK(v.at(key).get<std::size_t>() == i);
CHECK(!v.contains(key + "x"));
}
CHECK(v[""].get_string() == "empty key");
CHECK(v["k1"].get<int>() == 1); // the first of duplicate keys, as for small objects
CHECK(!v.contains("missing"));
CHECK_THROWS_WITH_AS(v.at("missing"), "[json.exception.out_of_range.403] key 'missing' not found", json::out_of_range&);
CHECK(v == j);
CHECK(v.materialize() == j);
}
SECTION("duplicate keys: the first member wins, with and without a table")
{
// an object of `total` members: the keys "k0".."k<n-1>" in order, then
// three keys repeated twice more (one copy in the middle, one at the
// end), and two keys repeated once; the value of a member is its
// position, so that the first member of a key can be told apart
struct member
{
std::string key;
std::size_t position;
};
const auto make_members = [](std::size_t total)
{
std::vector<std::string> keys;
for (std::size_t i = 0; i + 8 < total; ++i)
{
keys.push_back("k" + std::to_string(i));
}
const std::size_t n = keys.size();
const std::array<std::size_t, 3> triple = {{3, 17, n - 1}};
const std::array<std::size_t, 2> twice = {{5, n / 2}};
std::vector<std::string> ordered = keys;
for (const std::size_t i : triple)
{
ordered.insert(ordered.begin() + static_cast<std::ptrdiff_t>(ordered.size() / 2), keys[i]);
}
for (const std::size_t i : twice)
{
ordered.insert(ordered.begin() + static_cast<std::ptrdiff_t>(ordered.size() / 3), keys[i]);
}
for (const std::size_t i : triple)
{
ordered.push_back(keys[i]);
}
std::vector<member> result;
result.reserve(ordered.size());
for (std::size_t i = 0; i < ordered.size(); ++i)
{
result.push_back({ordered[i], i});
}
return result;
};
// 100 and 127 members: no table; 128 members and more: a table
for (const std::size_t total :
{
100u, 127u, 128u, 200u, 5000u
})
{
CAPTURE(total)
const std::vector<member> members = make_members(total);
REQUIRE(members.size() >= total);
REQUIRE(members.size() >= 100);
std::string text = "{";
std::map<std::string, std::size_t> first;
std::map<std::string, std::size_t> last;
std::map<std::string, std::size_t> occurrences;
for (const member& m : members)
{
text += (text.size() > 1 ? ",\"" : "\"") + m.key + "\":" + std::to_string(m.position);
first.insert({m.key, m.position});
last[m.key] = m.position;
++occurrences[m.key];
}
text += '}';
REQUIRE(first.size() < members.size());
const json_document d = json_document::parse(text);
const json_view v = d.root();
const json j = json::parse(text);
CHECK(v.size() == members.size()); // every occurrence is visited
const json m = v.materialize();
for (const auto& entry : first)
{
CAPTURE(entry.first)
const std::size_t expected = entry.second;
CHECK(v[entry.first].get<std::size_t>() == expected);
CHECK(v.at(entry.first).get<std::size_t>() == expected);
CHECK(v.find(entry.first).value().get<std::size_t>() == expected);
CHECK(v.value(entry.first, std::size_t{0}) == expected);
CHECK(v.contains(entry.first));
CHECK(v.count(entry.first) == occurrences[entry.first]);
const std::string pointer = "/" + entry.first;
CHECK(v[json::json_pointer(pointer)].get<std::size_t>() == expected);
CHECK(v.at(json::json_pointer(pointer)).get<std::size_t>() == expected);
CHECK(v.value(json::json_pointer(pointer), std::size_t{0}) == expected);
CHECK(v.contains(json::json_pointer(pointer)));
// materialize() and parse() keep the last value instead
CHECK(j[entry.first].get<std::size_t>() == last.at(entry.first));
CHECK(m[entry.first].get<std::size_t>() == last.at(entry.first));
}
CHECK(!v.contains("k"));
CHECK(v["missing"].is_discarded());
CHECK(v.materialize() == j);
CHECK(v == j);
}
}
SECTION("colliding keys")
{
// the hash is not seeded, so keys that all land in one place must not
// make the table build quadratic: such an object gets no table and is
// searched linearly
constexpr std::size_t members = 300;
constexpr std::size_t table_size = 1024; // the table size for 300 members: the next power of two >= 600
std::vector<std::string> colliding;
std::vector<std::string> spread;
for (std::uint64_t counter = 0; colliding.size() < members || spread.size() < members; ++counter)
{
std::string key(8, 'a');
std::uint64_t x = counter;
for (std::size_t i = 0; i < 8; ++i, x /= 26)
{
key[i] = static_cast<char>('a' + (x % 26));
}
const bool lands_in_slot_zero = (nlohmann::detail::view::key_hash(key.data(), key.size()) & (table_size - 1)) == 0;
if (lands_in_slot_zero && colliding.size() < members)
{
colliding.push_back(key);
}
else if (!lands_in_slot_zero && spread.size() < members)
{
spread.push_back(key);
}
}
const auto make_text = [](const std::vector<std::string>& keys)
{
std::string text = "{";
for (std::size_t i = 0; i < keys.size(); ++i)
{
text += (i != 0 ? ",\"" : "\"") + keys[i] + "\":" + std::to_string(i % 10);
}
return text + "}";
};
const std::string colliding_text = make_text(colliding);
const std::string spread_text = make_text(spread);
json_document with_collisions = json_document::parse(colliding_text);
json_document without_collisions = json_document::parse(spread_text);
for (const auto* pair :
{
&colliding, &spread
})
{
const json_view v = (pair == &colliding ? with_collisions : without_collisions).root();
for (std::size_t i = 0; i < members; ++i)
{
CAPTURE(i)
CHECK(v[(*pair)[i]].get<std::size_t>() == i % 10);
CHECK(v.at((*pair)[i]).get<std::size_t>() == i % 10);
CHECK(v.find((*pair)[i]).key() == (*pair)[i]);
CHECK(!v.contains((*pair)[i] + "x"));
}
CHECK(!v.contains("missing"));
}
CHECK(with_collisions.root() == json::parse(colliding_text));
// only the object with the spread keys got a table (both texts have the
// same length, so the tables are the only difference)
with_collisions.shrink_to_fit();
without_collisions.shrink_to_fit();
CHECK(without_collisions.memory_usage() >= with_collisions.memory_usage() + (table_size * sizeof(std::uint32_t)));
}
SECTION("keys that differ in their last bytes are spread over the table")
{
// The hash must spread the usual families of keys (numbered names, hex
// strings, dates, short names): a key farther than index_max_displacement
// from its home slot would leave the object without a table. The keys
// are placed with linear probing, as the table does.
const auto max_displacement = [](const std::vector<std::string>& keys)
{
std::size_t slots = 16;
while (slots < 2 * keys.size())
{
slots *= 2;
}
std::vector<bool> used(slots, false);
std::size_t farthest = 0;
for (const std::string& key : keys)
{
std::size_t i = nlohmann::detail::conditional_static_cast<std::size_t>(nlohmann::detail::view::key_hash(key.data(), key.size())) & (slots - 1);
std::size_t distance = 0;
while (used[i])
{
i = (i + 1) & (slots - 1);
++distance;
}
used[i] = true;
farthest = (std::max)(farthest, distance);
}
return farthest;
};
// the keys of a printf-like pattern with one %d, %0Nd or %0Nx, formatted with a stream
// (a format string that is not a literal is a Flawfinder error)
const auto numbered = [](const char* format, int count, int step)
{
const std::string pattern(format);
const std::size_t percent = pattern.find('%');
std::size_t conversion = percent + 1;
std::size_t width = 0;
while (pattern[conversion] >= '0' && pattern[conversion] <= '9')
{
width = (width * 10) + static_cast<std::size_t>(pattern[conversion] - '0');
++conversion;
}
const bool hex = pattern[conversion] == 'x';
std::vector<std::string> keys;
for (int i = 0; i < count; ++i)
{
std::ostringstream key;
key << pattern.substr(0, percent) << std::setfill('0') << std::setw(static_cast<int>(width))
<< (hex ? std::hex : std::dec) << (i * step) << pattern.substr(conversion + 1);
keys.push_back(key.str());
}
return keys;
};
for (const char* format :
{"key%d", "%d", "item_%04d", "a%05d", "%03dxyz", "id%d", "user_%d", "%d_suffix", "0x%04x", "A%07d", "12345678%d", "1234567%d", "123456789%d",
"configuration_parameter_value_%04d_extra", "2026-01-%02dT00:00:00Z"
})
{
CAPTURE(format)
CHECK(max_displacement(numbered(format, std::string(format).find("%02d") != std::string::npos ? 60 : 1000, 1)) < nlohmann::detail::view::index_max_displacement); // NOLINT(abseil-string-find-str-contains)
CHECK(max_displacement(numbered(format, 600, 7)) < nlohmann::detail::view::index_max_displacement);
}
std::mt19937_64 rng(7); // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed)
for (const std::size_t length :
{
std::size_t{1}, std::size_t{2}, std::size_t{3}, std::size_t{5}, std::size_t{7}, std::size_t{8}, std::size_t{9}, std::size_t{12}, std::size_t{16}, std::size_t{17}, std::size_t{33}
})
{
CAPTURE(length)
std::set<std::string> distinct;
// (there are 26 and 676 keys of one and two letters)
std::size_t wanted = 700;
if (length == 1)
{
wanted = 26;
}
else if (length == 2)
{
wanted = 500;
}
while (distinct.size() < wanted)
{
std::string key(length, 'a');
for (char& c : key)
{
c = static_cast<char>('a' + (rng() % 26));
}
distinct.insert(key);
}
CHECK(max_displacement(std::vector<std::string>(distinct.begin(), distinct.end())) < nlohmann::detail::view::index_max_displacement);
}
}
SECTION("keys of every length that differ in one byte")
{
// In a large object, a key is found by comparing its bytes with those of the member the
// table points to: up to 16 bytes as two overlapping words, longer ones with memcmp.
// Keys of every length from 0 to 40, and keys that differ from them in one byte.
std::mt19937_64 rng(2026); // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed)
std::vector<std::string> keys;
std::set<std::string> distinct;
for (std::size_t length = 0; length <= 40; ++length)
{
for (int copy = 0; copy < 5; ++copy)
{
std::string key(length, 'a');
for (char& c : key)
{
c = static_cast<char>('a' + (rng() % 26));
}
if (distinct.insert(key).second)
{
keys.push_back(key);
}
}
}
std::string text = "{";
for (std::size_t i = 0; i < keys.size(); ++i)
{
text += (i != 0 ? ",\"" : "\"") + keys[i] + "\":" + std::to_string(i);
}
text += "}";
REQUIRE(keys.size() >= 128); // (a table is built for 128 members)
const json_document d = json_document::parse(text);
const json_view v = d.root();
for (std::size_t i = 0; i < keys.size(); ++i)
{
CAPTURE(keys[i])
CHECK(v[keys[i]].get<std::size_t>() == i);
CHECK(v.at(keys[i]).get<std::size_t>() == i);
CHECK(v.contains(keys[i]));
for (std::size_t at = 0; at < keys[i].size(); ++at)
{
std::string other = keys[i];
other[at] = other[at] == 'z' ? 'y' : 'z';
CAPTURE(other)
CHECK(v.contains(other) == (distinct.count(other) != 0));
}
CHECK(v.contains(keys[i] + "a") == (distinct.count(keys[i] + "a") != 0));
if (!keys[i].empty())
{
CHECK(v.contains(keys[i].substr(1)) == (distinct.count(keys[i].substr(1)) != 0));
}
}
}
SECTION("shrink_to_fit releases the tables' spare capacity")
{
const auto make_text = [](int objects, int members)
{
std::string text = "[";
for (int object = 0; object < objects; ++object)
{
text += object != 0 ? ",{" : "{";
for (int i = 0, n = members + object; i < n; ++i)
{
text += (i != 0 ? ",\"" : "\"") + std::to_string(i) + "\":" + std::to_string(i);
}
text += '}';
}
return text + "]";
};
const std::string small_text = make_text(5, 150);
const std::string big_text = make_text(40, 400);
// reading a big text, and then a small one, leaves the spare capacity
// of the big one: shrink_to_fit() brings the document to the size of
// one parsed from the small text alone
json_document d = json_document::parse(big_text);
const std::size_t big = d.memory_usage();
d.read(small_text);
CHECK(d.memory_usage() >= big);
d.shrink_to_fit();
json_document fresh = json_document::parse(small_text);
fresh.shrink_to_fit();
CHECK(d.memory_usage() == fresh.memory_usage());
CHECK(d.memory_usage() < big / 2);
CHECK(d.root() == json::parse(small_text));
for (int object = 0; object < 5; ++object)
{
const json_view v = d.root()[static_cast<std::size_t>(object)];
for (int i = 0, n = 150 + object; i < n; ++i)
{
CHECK(v[std::to_string(i)].get<int>() == i);
}
}
}
SECTION("nested, reused, and in arrays")
{
std::string inner = "{";
for (int i = 0; i < 300; ++i)
{
inner += (i != 0 ? ",\"m" : "\"m") + std::to_string(i) + "\":" + std::to_string(i);
}
inner += '}';
const std::string text = "[" + inner + ",{\"x\":" + inner + "}," + inner + "]";
json_document d = json_document::parse(text);
CHECK(d.root()[0]["m299"].get<int>() == 299);
CHECK(d.root()[1]["x"]["m150"].get<int>() == 150);
CHECK(d.root()[2]["m0"].get<int>() == 0);
const std::size_t with_index = d.memory_usage();
d.read(std::string("{\"small\": 1}"));
CHECK(d.root()["small"].get<int>() == 1);
d.read(text);
CHECK(d.root()[2]["m7"].get<int>() == 7);
CHECK(d.memory_usage() >= with_index / 2);
}
}
+347
View File
@@ -0,0 +1,347 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
// source positions of materialize() (JSON_DIAGNOSTIC_POSITIONS, #5864)
#include "doctest_compatibility.h"
#ifndef JSON_DIAGNOSTICS
#define JSON_DIAGNOSTICS 1
#endif
#define JSON_DIAGNOSTIC_POSITIONS 1
#include <nlohmann/json_view.hpp>
using nlohmann::json;
using nlohmann::ordered_json;
using nlohmann::json_document;
using nlohmann::json_editable_document;
#include <cstdint>
#include <cstring>
#include <fstream>
#include <sstream>
#include <string>
#include <vector>
#include "json_view_test_helpers.hpp"
#include <test_data.hpp>
namespace
{
// not constexpr: VS 2015 does not accept std::string::npos in a constant expression (C2131)
const std::size_t npos = std::string::npos;
// whether every value of a has the source positions of the value of b at the
// same place (a and b are equal; shallow test documents only)
template<typename BasicJsonType>
bool same_positions(const BasicJsonType& a, const BasicJsonType& b)
{
if (a.start_pos() != b.start_pos() || a.end_pos() != b.end_pos() || a.type() != b.type() || a.size() != b.size())
{
return false;
}
if (a.is_object())
{
for (auto it = a.begin(); it != a.end(); ++it)
{
if (!same_positions(it.value(), b.at(it.key())))
{
return false;
}
}
}
else if (a.is_array())
{
for (std::size_t i = 0; i < a.size(); ++i)
{
if (!same_positions(a[i], b[i]))
{
return false;
}
}
}
return true;
}
// the materialized root of text has the positions of parse()
template<typename BasicJsonType>
bool positions_as_parse(const std::string& text, bool comments = false, bool trailing_commas = false)
{
const auto j = BasicJsonType::parse(text, nullptr, true, comments, trailing_commas);
const auto d = nlohmann::basic_json_document<BasicJsonType>::parse(text, true, comments, trailing_commas);
const auto e = nlohmann::basic_json_document<BasicJsonType, nlohmann::view_mode::editable>::parse(text, true, comments, trailing_commas);
return same_positions(d.root().materialize(), j) && same_positions(e.root().materialize(), j);
}
std::string range(const json& j)
{
return j.start_pos() == npos && j.end_pos() == npos ? "none" : std::to_string(j.start_pos()) + "-" + std::to_string(j.end_pos());
}
} // namespace
TEST_CASE("json_view materialize() source positions")
{
SECTION("as parse()")
{
const std::vector<std::string> texts =
{
"null", "true", "false", "0", "-12", "3.25e-2", "18446744073709551615", "-9223372036854775809", R"("a")",
R"("\n")", R"("\u00e9")", R"("\ud83d\ude00")", "\"\xf0\x9f\x98\x80\"", R"("a\"b\\")", R"("x\\")",
" [ ] ", "{}", "[[[]]]", R"({"a":{"b":[1,{"c":[]}]}})",
R"({"\u0041": "v\n", "k": "\"", "\t":[ "\/" ]})",
R"([1, 2.5, true, false, null, "s", "\u0000", -0])",
"\n\t[\r\n 1 ,\n {\n \"a\" : [ ]\n }\n]\n",
"\xef\xbb\xbf[1, {\"a\": \"b\"}]", "\xef\xbb\xbf\"\\n\"",
};
for (const auto& text : texts)
{
CAPTURE(text)
CHECK(positions_as_parse<json>(text));
CHECK(positions_as_parse<ordered_json>(text));
}
}
SECTION("comments and trailing commas")
{
const std::vector<std::string> texts =
{
"[1 /* ] */]", "[1 // ]\n]", "{\"a\": 1 /* } */ , }", "[1, 2, ]", "[[1,],[2 , /**/ ] , ]",
"{\"a\": [] // }\n, \"b\": {} /* } ] */ }", "[ /* empty */ ]", "{ // }\n}", "/* a */ [1] // b",
R"(["\n" /* " */ , ])",
};
for (const auto& text : texts)
{
CAPTURE(text)
CHECK(positions_as_parse<json>(text, true, true));
}
}
SECTION("generated documents")
{
json_view_test::generator gen;
for (int i = 0; i < 300; ++i)
{
std::string text;
gen.value(text, 0);
CAPTURE(text)
CHECK(positions_as_parse<json>(text));
CHECK(positions_as_parse<ordered_json>(text));
}
}
SECTION("test data")
{
for (const char* name :
{
"/json_testsuite/sample.json", "/nativejson-benchmark/canada.json", "/nativejson-benchmark/citm_catalog.json",
"/nativejson-benchmark/twitter.json", "/json_tests/pass1.json"
})
{
CAPTURE(name)
const std::ifstream f(std::string(TEST_DATA_DIRECTORY) + name, std::ios::binary);
std::stringstream ss;
ss << f.rdbuf();
CHECK(positions_as_parse<json>(ss.str()));
}
}
SECTION("subtrees")
{
const std::string text = R"({"a": [1, "\n", {"b": null}], "s": "x\ty", "n": -1.5})";
const json j = json::parse(text);
const json_document d = json_document::parse(text);
CHECK(same_positions(d.root()["a"].materialize(), j["a"]));
CHECK(same_positions(d.root()["a"][2].materialize(), j["a"][2]));
CHECK(same_positions(d.root()["a"][1].materialize(), j["a"][1]));
CHECK(same_positions(d.root()["s"].materialize(), j["s"]));
CHECK(same_positions(d.root()["n"].materialize(), j["n"]));
CHECK(range(d.root()["a"][1].materialize()) == "10-14");
CHECK(range(d.root()["s"].materialize()) == "35-41");
}
#if !defined(JSON_NOEXCEPTION)
SECTION("exception messages")
{
const std::string text = "{\n \"address\": {\"street\": \"Fake\\tStreet\", \"housenumber\": \"1\"},\n \"list\": [1, [true]]\n}";
const json j = json::parse(text);
const json_document d = json_document::parse(text);
const json m = d.root().materialize();
const std::string expected = json_view_test::exception_of_call([&]
{
static_cast<void>(j.at("address").at("housenumber").get<int>());
});
CHECK(expected.find("(bytes ") != std::string::npos);
CHECK(json_view_test::exception_of_call([&]
{
static_cast<void>(m.at("address").at("housenumber").get<int>());
}) == expected);
CHECK(json_view_test::exception_of_call([&]
{
static_cast<void>(m.at("address").at("street").get<int>());
}) == json_view_test::exception_of_call([&]
{
static_cast<void>(j.at("address").at("street").get<int>());
}));
CHECK(json_view_test::exception_of_call([&]
{
static_cast<void>(m.at("list").at(1).get<int>());
}) == json_view_test::exception_of_call([&]
{
static_cast<void>(j.at("list").at(1).get<int>());
}));
}
#endif
SECTION("edited documents")
{
const std::string text = R"({"a": [1, 2], "b": {"c": "\n", "d": true}, "e": [3, [4]], "f": "x"})";
const json j = json::parse(text);
SECTION("new scalars have none")
{
json_editable_document d = json_editable_document::parse(text);
d.set(d.root()["f"], 5);
d.set(d.root()["b"]["d"], "s");
const json m = d.root().materialize();
CHECK(range(m["f"]) == "none");
CHECK(range(m["b"]["d"]) == "none");
// the object whose last member is new has none; the others keep theirs
CHECK(range(m) == "none");
CHECK(range(m["b"]) == "none");
CHECK(same_positions(m["a"], j["a"]));
CHECK(same_positions(m["e"], j["e"]));
CHECK(same_positions(m["b"]["c"], j["b"]["c"]));
}
SECTION("containers whose elements change have none")
{
json_editable_document d = json_editable_document::parse(text);
d.push_back(d.root()["a"], 7);
d.erase(d.root()["e"], 0);
const json m = d.root().materialize();
CHECK(range(m["a"]) == "none");
CHECK(range(m["a"][2]) == "none");
CHECK(range(m["e"]) == "none");
// the elements kept keep theirs
CHECK(range(m["a"][0]) == range(j["a"][0]));
CHECK(range(m["a"][1]) == range(j["a"][1]));
CHECK(same_positions(m["e"][0], j["e"][1]));
CHECK(same_positions(m["b"], j["b"]));
// the root's elements did not change
CHECK(range(m) == range(j));
}
SECTION("new containers have none")
{
json_editable_document d = json_editable_document::parse(text);
d.set(d.root()["b"], json::parse(R"({"x": [1]})"));
const json m = d.root().materialize();
CHECK(range(m["b"]) == "none");
CHECK(range(m["b"]["x"]) == "none");
CHECK(range(m["b"]["x"][0]) == "none");
CHECK(range(m) == range(j));
}
SECTION("copies of scalars of the document keep theirs")
{
// (a copy of an array or object is a new one; its scalars are the
// parsed ones)
json_editable_document d = json_editable_document::parse(text);
d.set(d.root()["f"], d.root()["e"]);
d.set(d.root()["a"], d.root()["b"]["c"]);
const json m = d.root().materialize();
CHECK(range(m["a"]) == range(j["b"]["c"]));
CHECK(range(m["f"]) == "none");
CHECK(range(m["f"][1]) == "none");
CHECK(range(m["f"][0]) == range(j["e"][0]));
CHECK(range(m["f"][1][0]) == range(j["e"][1][0]));
}
}
#if !(defined(__BYTE_ORDER__) && defined(__ORDER_BIG_ENDIAN__) && __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__)
SECTION("images")
{
const std::string text = R"({"a\n": ["\u00e9", 1.5, {"b": [true, null]}], "c": "d"})";
const json j = json::parse(text);
SECTION("round trip")
{
const json_document d = json_document::load(json_document::parse(text).save());
CHECK(same_positions(d.root().materialize(), j));
}
SECTION("edited")
{
json_editable_document e = json_editable_document::parse(text);
e.set(e.root()["c"], 1);
e.push_back(e.root()[std::string("a") + '\n'], false);
e.set(e.root()[std::string("a") + '\n'][2]["b"][0], "\"");
const json_document d = json_document::load(e.save());
const json m = d.root().materialize();
CHECK(range(m["c"]) == "none");
CHECK(range(m) == "none");
const json& a = m[std::string("a") + '\n'];
const json& ja = j[std::string("a") + '\n'];
CHECK(range(a) == "none");
CHECK(range(a[3]) == "none");
CHECK(range(a[0]) == range(ja[0]));
CHECK(range(a[1]) == range(ja[1]));
CHECK(range(a[2]["b"][0]) == "none");
CHECK(range(a[2]["b"][1]) == range(ja[2]["b"][1]));
// the last element of b is still the parsed one
CHECK(range(a[2]["b"]) == range(ja[2]["b"]));
CHECK(range(a[2]) == range(ja[2]));
}
SECTION("without positions")
{
// an image saved without JSON_DIAGNOSTIC_POSITIONS: next is 0 for
// the strings with escapes
std::vector<std::uint8_t> image = json_document::parse(text).save();
std::uint64_t count = 0;
std::memcpy(&count, image.data() + 8, sizeof(count));
for (std::size_t i = 0; i < count; ++i)
{
nlohmann::detail::view::node n{};
std::memcpy(&n, image.data() + 64 + (i * sizeof(n)), sizeof(n));
if (n.kind == nlohmann::detail::view::node_kind::string)
{
n.next = 0;
}
std::memcpy(image.data() + 64 + (i * sizeof(n)), &n, sizeof(n));
}
const json_document d = json_document::load(image);
const json m = d.root().materialize();
CHECK(range(m[std::string("a") + '\n'][0]) == "none");
CHECK(range(m["c"]) == range(j["c"]));
CHECK(same_positions(m[std::string("a") + '\n'][2], j[std::string("a") + '\n'][2]));
}
#if !defined(JSON_NOEXCEPTION)
SECTION("a decoded string's position must be within the text")
{
std::vector<std::uint8_t> image = json_document::parse(R"(["\n"])").save();
nlohmann::detail::view::node n{};
std::memcpy(&n, image.data() + 64 + sizeof(n), sizeof(n));
REQUIRE(n.next == 2);
n.next = 7;
std::memcpy(image.data() + 64 + sizeof(n), &n, sizeof(n));
const auto load_image = [&image]
{
// GCC ignores a cast to void for [[nodiscard]]-like results, so keep the document
const json_document d = json_document::load(image);
static_cast<void>(d);
};
CHECK_THROWS_AS(load_image(), json::parse_error&);
// without the check, the position is out of bounds and ignored
const json_document d = json_document::load(image, json_document::image_check::none);
CHECK(range(d.root()[0].materialize()) == "none");
}
#endif
}
#endif
}
+20
View File
@@ -1499,6 +1499,26 @@ TEST_CASE("regression tests 1")
CHECK(order.dump() == R"({"a":2,"m":{"y":1,"b":2},"new_key":3})");
}
SECTION("operator[](json_pointer) when emplace() finds a key the lookup does not")
{
// a copied fifo_map restarts its timestamps, so a new key "k" gets
// the timestamp of "x": the lookup does not find "k", but emplace()
// finds "x" and inserts nothing; the pointer then continues with the
// member of "x", as basic_json::operator[] does. The object_t is
// copied explicitly: copying the my_json would build the object
// member by member (with fresh timestamps) when built with
// JSON_NO_THREAD_LOCAL.
const my_json orig = my_json::parse(R"({"x":1})");
const std::array<const char*, 3> pointers = {{"/k/0", "/k/a/b", "/k/-"}};
for (const char* pointer : pointers)
{
CAPTURE(pointer)
my_json j(orig.get_ref<const my_json::object_t&>());
CHECK_THROWS_AS(j[my_json::json_pointer(pointer)], my_json::out_of_range&);
CHECK(j.dump() == R"({"x":1})");
}
}
SECTION("issue #977 - Assigning between different json types")
{
foo_json lj = ns::foo{3};
+34
View File
@@ -0,0 +1,34 @@
# check_path_filters
The compiler workflows [`ubuntu.yml`](../../.github/workflows/ubuntu.yml),
[`macos.yml`](../../.github/workflows/macos.yml), and [`windows.yml`](../../.github/workflows/windows.yml) skip pull
requests that change only files no compiler job reads: the documentation, Markdown files, GitHub metadata, other
workflows, and the license files. Pushes to `develop`, `master`, and `release/*` always run them. The documentation,
license, and path-filter checks run in [`repository_checks.yml`](../../.github/workflows/repository_checks.yml), which
has no path filter.
The filter starts with `'**'` and excludes these files explicitly, so a new file runs the compiler workflows unless it
is excluded on purpose. [`check_path_filters.py`](check_path_filters.py) holds the filter (`PATHS`) and checks that
- each of the three workflows has exactly this filter, with its own workflow file included,
- the CI targets that read excluded files (`UNFILTERED_TARGETS`) do not run in these workflows, and
- no file the filter keeps (sources, tests, build files, tools, workflows) mentions an excluded file or directory
outside a comment line, unless the mention is listed in `KNOWN_REFERENCES` with a reason.
When the check reports a mention, decide whether a compiler job reads that file:
- If it does, include the file in `PATHS` after the exclusions (as `package_managers.md` and the Swift example are) and
copy the new filter into the three workflows.
- If it does not, add the mention to `KNOWN_REFERENCES` with the reason.
Run the check with
```shell
python3 tools/check_path_filters/check_path_filters.py
```
To see which workflows a pull request would run, pass the changed files:
```shell
python3 tools/check_path_filters/check_path_filters.py --changed $(git diff --name-only origin/develop)
```
+273
View File
@@ -0,0 +1,273 @@
#!/usr/bin/env python3
"""Check the path filters of the compiler workflows.
ubuntu.yml, macos.yml and windows.yml skip pull requests that change only files
no compiler job reads (documentation, Markdown files, GitHub metadata, license
files). The filter starts with '**' and excludes these files explicitly, so a
new file runs the workflows unless it is excluded on purpose.
The script checks that
- each of the three workflows has exactly the filter defined in PATHS below,
with its own workflow file included, and
- no tracked file that the filter keeps (sources, tests, build files, tools,
workflows) mentions a file the filter excludes, unless the mention is listed
in KNOWN_REFERENCES with the reason why it does not matter.
A mention found this way must either be added to PATHS as an included path
(the file is read by a compiler job) or to KNOWN_REFERENCES (it is not).
With --changed FILE..., the script prints instead which workflows a pull
request changing these files would run.
"""
import argparse
import os
import re
import subprocess # nosec B404
import sys
REPO_ROOT = os.path.normpath(os.path.join(sys.path[0], '..', '..'))
SELF = 'tools/check_path_filters/check_path_filters.py'
WORKFLOWS = ['.github/workflows/ubuntu.yml', '.github/workflows/macos.yml', '.github/workflows/windows.yml']
# The pull_request paths filter of the workflows above; '{workflow}' is replaced
# with the workflow's own file. GitHub applies the patterns in order: a file
# matching a later pattern is included ('pattern') or excluded ('!pattern').
PATHS = [
'**',
'!docs/**',
'!**.md',
'!.github/*',
'!.github/ISSUE_TEMPLATE/**',
'!.github/external_ci/**',
'!.github/workflows/**',
'!.reuse/**',
'!LICENSES/**',
'!CITATION.cff',
'!REUSE.toml',
# read by tools/check_build_options in the ci_meson_install job (ubuntu.yml)
'docs/mkdocs/docs/integration/package_managers.md',
# built by the swiftpm job (macos.yml)
'docs/mkdocs/docs/integration/swift/**',
'{workflow}',
]
# Mentions of excluded files that no job of the three workflows depends on:
# (file that mentions it, the mention, reason)
KNOWN_REFERENCES = [
('cmake/ci.cmake', 'docs/', 'ci_test_examples; runs in repository_checks.yml, which has no path filter'),
('cmake/ci.cmake', 'docs/mkdocs/', 'ci_test_build_documentation and ci_test_documentation_mermaid; they run in '
'repository_checks.yml, which has no path filter'),
('cmake/ci.cmake', 'docs/mkdocs/docs/examples/', 'indentation check of ci_test_amalgamation; the same check runs in '
'check_amalgamation.yml, which has no path filter'),
('Makefile', 'docs/mkdocs/docs/examples/', 'make pretty; runs in check_amalgamation.yml, which has no path filter'),
('Makefile', 'ChangeLog.md', 'make ChangeLog.md writes the file; no CI job runs it'),
('Makefile', 'tests/fuzzing.md', 'help text'),
('Makefile', 'tools/generate_natvis/README.md', 'help text'),
('tests/src/make_test_data_available.hpp', 'README.md', 'a file of the external test data repository'),
('tools/check_build_options/check_build_options.py', 'docs/', 'builds the path of package_managers.md, which '
'PATHS includes'),
('tools/ci_jobs/ci_jobs.py', 'docs/mkdocs/docs/community/quality_assurance.md',
'checks the job table; runs in repository_checks.yml, which has no path filter'),
('tools/ci_jobs/ci_jobs.py', '.github/workflows/repository_checks.yml', 'read by --compare, a manual check'),
('tools/ci_jobs/ci_jobs.py', '.github/workflows/check_amalgamation.yml', 'read by --compare, a manual check'),
]
# CI targets that read excluded files; they must run in a workflow without a path filter
UNFILTERED_TARGETS = ['ci_test_examples', 'ci_test_build_documentation', 'ci_test_documentation_mermaid',
'ci_test_amalgamation', 'ci_reuse_compliance']
def git_files(root):
# fixed command, no user input; includes new files not yet added, so a local run
# sees what CI sees after the commit
out = subprocess.check_output(['git', 'ls-files', '-z', '--cached', '--others', '--exclude-standard'], cwd=root) # nosec B603, B607
return [f for f in out.decode('utf-8').split('\0') if f]
def glob_to_regex(pattern):
"""Translate a GitHub Actions path pattern to a regular expression."""
regex = ''
i = 0
while i < len(pattern):
if pattern.startswith('**', i):
regex += '.*'
i += 2
elif pattern[i] == '*':
regex += '[^/]*'
i += 1
elif pattern[i] == '?':
regex += '[^/]'
i += 1
else:
regex += re.escape(pattern[i])
i += 1
return re.compile(regex + r'\Z')
def compile_filter(paths, workflow):
return [(p.startswith('!'), glob_to_regex(p.lstrip('!').replace('{workflow}', workflow))) for p in paths]
def included(compiled, path):
result = False
for negated, regex in compiled:
if regex.match(path):
result = not negated
return result
def read_filter(root, workflow):
"""Return the pull_request paths list of a workflow file."""
with open(os.path.join(root, workflow), encoding='utf-8') as f:
lines = f.read().splitlines()
paths = None
in_pull_request = False
for line in lines:
if re.match(r'^ pull_request:', line):
in_pull_request = True
continue
if in_pull_request:
if re.match(r'^ \S', line) or re.match(r'^\S', line):
break
if re.match(r'^ paths:\s*$', line):
paths = []
continue
m = re.match(r"^ - '(.*)'\s*(#.*)?$", line)
if paths is not None and m:
paths.append(m.group(1))
elif paths is not None and line.strip() and not line.strip().startswith('#'):
break
return paths
TOKEN = re.compile(r'(["\'/]?)([\w.-]+(?:/[\w.-]+)*)')
COMMENT_LINE = re.compile(r'^[ \t]*(//|#|/?\*|--).*$', re.MULTILINE)
class Mentions:
"""Finds the words and paths in a text that name an excluded file or directory."""
def __init__(self, files, excluded, is_excluded):
self.files = set(files)
self.excluded = excluded
self.dirs = {'/'.join(f.split('/')[:n]) for f in files for n in range(1, f.count('/') + 1)}
# an excluded directory: a new file in it would be excluded
self.excluded_dirs = {d for d in self.dirs if is_excluded(d + '/new_file')}
# base names that only excluded files have, e.g. 'ChangeLog.md' but not 'CMakeLists.txt'
self.names = {os.path.basename(f) for f in excluded} - {os.path.basename(f) for f in files if f not in excluded}
def find(self, text):
"""Yield (offset, mention) for every word or path in text that names an excluded file or directory."""
for m in TOKEN.finditer(text):
parts = m.group(2).rstrip('/').split('/')
# a single word names a directory only in a path context, e.g. 'docs' in os.path.join('docs', ...)
path_context = len(parts) > 1 or m.group(1)
for i in range(len(parts)):
path = '/'.join(parts[i:])
if path in self.files:
if path in self.excluded:
yield m.start(2), path
break
if path in self.dirs and path_context:
if path in self.excluded_dirs:
yield m.start(2), path + '/'
break
else:
if parts[-1] in self.names and '.' in parts[-1]:
yield m.start(2), parts[-1]
def known(path, mention):
return any(path == file and mention == known_mention for file, known_mention, _ in KNOWN_REFERENCES)
def check_workflows(root):
"""Return the errors in the filters of the workflows and in the targets they run."""
errors = []
for workflow in WORKFLOWS:
expected = [p.replace('{workflow}', workflow) for p in PATHS]
actual = read_filter(root, workflow)
if actual != expected:
errors.append('{}: the pull_request paths filter differs from tools/check_path_filters/check_path_filters.py\n'
' expected: {}\n actual: {}'.format(workflow, expected, actual))
with open(os.path.join(root, workflow), encoding='utf-8') as f:
text = f.read()
errors += ['{}: runs {}, which reads excluded files; run it in repository_checks.yml'.format(workflow, target)
for target in UNFILTERED_TARGETS if re.search(r'\b' + target + r'\b', text)]
return errors
def scanned_text(root, path):
"""Return the text of a kept file without comment lines and without the filter itself, or None to skip it."""
# third-party code and Markdown files read nothing; this script lists the excluded files
if path.startswith('tests/thirdparty/') or path.endswith('.md') or path == SELF:
return None
try:
with open(os.path.join(root, path), encoding='utf-8') as f:
# a mention in a comment line reads nothing; keep the line numbers
text = COMMENT_LINE.sub('', f.read())
except (UnicodeDecodeError, OSError):
return None
if path in WORKFLOWS:
# the filter itself lists the excluded files
for p in PATHS:
text = text.replace(" - '{}'".format(p.replace('{workflow}', path)), '')
return text
def check_mentions(root, files, excluded, finder):
"""Return an error for every mention of an excluded file in a kept file that KNOWN_REFERENCES does not list."""
errors = []
for path in sorted(set(files) - excluded):
text = scanned_text(root, path)
reported = set()
for offset, mention in finder.find(text or ''):
if mention not in reported and not known(path, mention):
reported.add(mention)
line = text.count('\n', 0, offset) + 1
errors.append('{}:{}: mentions excluded {}; include it in PATHS if a compiler job reads it, '
'or add it to KNOWN_REFERENCES'.format(path, line, mention))
return errors
def check(root):
files = git_files(root)
union = [compile_filter(PATHS, w) for w in WORKFLOWS]
def is_excluded(path):
return not any(included(c, path) for c in union)
excluded = {f for f in files if is_excluded(f)}
errors = check_workflows(root) + check_mentions(root, files, excluded, Mentions(files, excluded, is_excluded))
for e in errors:
print(e)
if not errors:
print('path filters OK: {} of {} tracked files skip the compiler workflows'.format(len(excluded), len(files)))
return 1 if errors else 0
def explain(changed):
for workflow in WORKFLOWS:
compiled = compile_filter(PATHS, workflow)
hits = [f for f in changed if included(compiled, f)]
print('{}: {}'.format(workflow, 'runs ({})'.format(', '.join(hits[:3]) + (' ...' if len(hits) > 3 else ''))
if hits else 'skipped'))
return 0
def main():
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
parser.add_argument('--root', default=REPO_ROOT, help='repository root')
parser.add_argument('--changed', nargs='+', metavar='FILE', help='show which workflows these changed files run')
args = parser.parse_args()
return explain(args.changed) if args.changed else check(args.root)
if __name__ == '__main__':
sys.exit(main())
+30
View File
@@ -0,0 +1,30 @@
# ci_jobs
The jobs of the compiler workflows [`ubuntu.yml`](../../.github/workflows/ubuntu.yml),
[`macos.yml`](../../.github/workflows/macos.yml), and [`windows.yml`](../../.github/workflows/windows.yml) are split into
three sets, chosen by each job's
`if:` condition (see [How CI runs](https://json.nlohmann.me/community/quality_assurance/#how-ci-runs)):
- **PR set**: every push to a pull request, and every push to `develop`, `master`, and `release/*`;
- **full set**: after the PR set passed, on pull requests with the label "🚀 ready to merge" and on every push;
- **nightly set**: every night on `develop`, on every push to `master` and `release/*`, and when started by hand.
A matrix whose entries belong to different sets is split into one job per set: the full-set job gets the suffix
`_full`, the nightly-set job `_nightly`, and both share their steps with the PR-set job through YAML anchors.
[`ci_jobs.py`](ci_jobs.py) checks that
- every job has the `if:` condition of exactly one set (the conditions are in `SETS`),
- every full-set job waits for (`needs:`) exactly the PR set of its workflow, and `nightly_report` for exactly the
nightly set, and
- the job table in [`quality_assurance.md`](../../docs/mkdocs/docs/community/quality_assurance.md) is up to date.
```shell
python3 -m venv venv && venv/bin/pip install -r tools/ci_jobs/requirements.txt
venv/bin/python tools/ci_jobs/ci_jobs.py # check
venv/bin/python tools/ci_jobs/ci_jobs.py --markdown # print the job table for the documentation
venv/bin/python tools/ci_jobs/ci_jobs.py --compare origin/develop # list configurations added or removed since develop
```
Use `--compare` when changing the workflows: a configuration that disappears fails it, so no compiler, version, or
standard is dropped by accident when jobs are split, merged, or moved between sets.
+231
View File
@@ -0,0 +1,231 @@
#!/usr/bin/env python3
"""List and check the jobs of the compiler workflows and the set each job belongs to.
The jobs of the compiler workflows belong to one of three sets, chosen by the
job's `if:` condition:
- PR set: every pull request push and every push,
- full set: after the PR set passed, on pushes and on pull requests with the
"🚀 ready to merge" label,
- nightly set: nightly on develop, on pushes to master and release/*, and when
started by hand.
The script checks that every job belongs to exactly one set, that the full set
of a workflow waits for its whole PR set, that the nightly report waits for the
whole nightly set, and that the job table in
docs/mkdocs/docs/community/quality_assurance.md is up to date.
--markdown print the job table for the documentation
--compare REF list the configurations (job and matrix values) added or
removed since the git revision REF; fails if one was removed
"""
import argparse
import collections
import itertools
import os
import re
import subprocess # nosec B404
import sys
import yaml
REPO_ROOT = os.path.normpath(os.path.join(sys.path[0], '..', '..'))
DOCS = 'docs/mkdocs/docs/community/quality_assurance.md'
BEGIN, END = '<!-- ci_jobs: begin -->', '<!-- ci_jobs: end -->'
WORKFLOWS = ['.github/workflows/ubuntu.yml', '.github/workflows/macos.yml', '.github/workflows/windows.yml']
TIERED = WORKFLOWS
# --compare also counts the jobs of these workflows, so a job moved between workflows is not a removed one
COMPARED = WORKFLOWS + ['.github/workflows/repository_checks.yml', '.github/workflows/check_amalgamation.yml']
LABEL = '🚀 ready to merge'
SETS = collections.OrderedDict([
('PR set', "${{ github.event_name != 'schedule' && (github.event.action != 'labeled' || "
"github.event.label.name == '" + LABEL + "') }}"),
('full set', "${{ github.event_name == 'push' || github.event_name == 'workflow_dispatch' || "
"contains(github.event.pull_request.labels.*.name, '" + LABEL + "') }}"),
('nightly set', "${{ github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || "
"(github.event_name == 'push' && github.ref != 'refs/heads/develop') }}"),
])
UNTIERED = 'every push and PR'
REPORT_JOB = 'nightly_report'
# a job split by set keeps its name and gets one of these suffixes
SUFFIXES = {'full set': '_full', 'nightly set': '_nightly'}
def read(root, path, ref=None):
if ref is None:
with open(os.path.join(root, path), encoding='utf-8') as f:
return f.read()
# fixed command; ref is a git revision given on the command line
return subprocess.check_output(['git', 'show', '{}:{}'.format(ref, path)], cwd=root, # nosec B603, B607
stderr=subprocess.DEVNULL).decode('utf-8')
def add_include(combos, keys, inc):
"""Apply one include entry: extend the matching configurations, or add it as a new one."""
matching = [c for c in combos if all(c[k] == v for k, v in inc.items() if k in keys)]
for c in matching:
c.update({k: v for k, v in inc.items() if k not in keys})
if not matching:
combos.append(dict(inc))
def expand(matrix):
"""Return the configurations of a job's strategy.matrix like GitHub Actions does."""
if not matrix:
return [{}]
keys = [k for k in matrix if k not in ('include', 'exclude')]
if not keys:
# a matrix of only include entries runs each entry
return [dict(inc) for inc in matrix.get('include') or []]
combos = [dict(zip(keys, values)) for values in itertools.product(*(matrix[k] for k in keys))]
for ex in matrix.get('exclude') or []:
combos = [c for c in combos if any(c.get(k) != v for k, v in ex.items())]
for inc in matrix.get('include') or []:
add_include(combos, keys, inc)
return combos
def jobs_of(root, workflow, ref=None):
"""Yield (job id, set, list of configurations) for every job of a workflow."""
data = yaml.safe_load(read(root, workflow, ref))
for job_id, job in data['jobs'].items():
condition = job.get('if')
job_set = UNTIERED if condition is None else next((s for s, c in SETS.items() if c == condition), None)
matrix = (job.get('strategy') or {}).get('matrix')
yield job_id, job_set, job, expand(matrix)
def base_id(job_id, job_set):
suffix = SUFFIXES.get(job_set)
return job_id[:-len(suffix)] if suffix and job_id.endswith(suffix) else job_id
def label(config):
return ', '.join(str(v) for k, v in config.items() if k != 'os' or len(config) == 1)
def configurations(root, ref=None):
"""Return a Counter of (job name without set suffix, configuration) over the compared workflows."""
result = collections.Counter()
for workflow in COMPARED:
try:
jobs = list(jobs_of(root, workflow, ref))
except (OSError, subprocess.CalledProcessError):
continue # the workflow does not exist in this revision
for job_id, job_set, _, configs in jobs:
if job_id == REPORT_JOB:
continue
for c in configs:
result[(base_id(job_id, job_set), tuple(sorted((k, str(v)) for k, v in c.items())))] += 1
return result
def markdown(root):
rows = []
order = list(SETS) + [UNTIERED]
for workflow in WORKFLOWS:
for job_id, job_set, _, configs in jobs_of(root, workflow):
if job_id == REPORT_JOB:
continue
labels = [label(c) for c in configs if c]
rows.append((order.index(job_set) if job_set in order else len(order), os.path.basename(workflow), job_id,
job_set or 'no set', '; '.join(labels) or '-'))
rows.sort(key=lambda r: (r[0], r[1]))
lines = ['| Set | Workflow | Job | Configurations |', '|-----|----------|-----|----------------|']
lines += ['| {} | {} | `{}` | {} |'.format(s, w, j, c) for _, w, j, s, c in rows]
return lines
def check_sets(workflow, jobs):
"""Return the errors in the sets of a tiered workflow's jobs and the jobs of each set."""
errors = []
by_set = collections.defaultdict(set)
for job_id, job_set, _, _ in jobs:
if job_id == REPORT_JOB:
continue
if job_set in SETS:
by_set[job_set].add(job_id)
else:
errors.append('{}: job {} has an if: condition of no set; use one of the conditions in '
'tools/ci_jobs/ci_jobs.py'.format(workflow, job_id))
return errors, by_set
def check_needs(workflow, jobs, by_set):
"""Return the errors in the needs: of a tiered workflow's jobs."""
required = {'full set': by_set['PR set'], 'PR set': set(), 'nightly set': set()}
errors = []
for job_id, job_set, job, _ in jobs:
needs = job.get('needs') or []
needs = {needs} if isinstance(needs, str) else set(needs)
expected = by_set['nightly set'] if job_id == REPORT_JOB else required.get(job_set, needs)
if needs != expected:
errors.append('{}: job {} must need exactly: {}'.format(workflow, job_id, ', '.join(sorted(expected)) or 'no job'))
return errors
def check_docs(root):
"""Return an error if the job table in the documentation is not up to date."""
text = read(root, DOCS)
block = re.search(re.escape(BEGIN) + r'\n(.*?)\n[ \t]*' + re.escape(END), text, re.S)
if not block:
return ['{}: markers {} and {} not found'.format(DOCS, BEGIN, END)]
indent = re.match(r'[ \t]*', block.group(1)).group(0)
if block.group(1) != '\n'.join(indent + line for line in markdown(root)):
return ['{}: the job table is out of date; replace it with the output of '
'python3 tools/ci_jobs/ci_jobs.py --markdown'.format(DOCS)]
return []
def check(root):
errors = []
for workflow in TIERED:
jobs = list(jobs_of(root, workflow))
set_errors, by_set = check_sets(workflow, jobs)
errors += set_errors + check_needs(workflow, jobs, by_set)
errors += check_docs(root)
for e in errors:
print(e)
if not errors:
counts = collections.Counter()
for workflow in WORKFLOWS:
for job_id, job_set, _, configs in jobs_of(root, workflow):
if job_id != REPORT_JOB:
counts[job_set] += len(configs)
print('CI jobs OK: ' + ', '.join('{} {}'.format(n, s) for s, n in counts.items()))
return 1 if errors else 0
def compare(root, ref):
before, after = configurations(root, ref), configurations(root)
removed, added = before - after, after - before
for title, diff in (('removed', removed), ('added', added)):
for job, config in sorted(diff.elements()):
print('{}: {} ({})'.format(title, job, ', '.join('{}={}'.format(k, v) for k, v in config)))
print('{} configurations before, {} after; {} removed, {} added'.format(
sum(before.values()), sum(after.values()), sum(removed.values()), sum(added.values())))
return 1 if removed else 0
def main():
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
parser.add_argument('--root', default=REPO_ROOT, help='repository root')
group = parser.add_mutually_exclusive_group()
group.add_argument('--markdown', action='store_true', help='print the job table for the documentation')
group.add_argument('--compare', metavar='REF', help='list the configurations added or removed since REF')
args = parser.parse_args()
if args.markdown:
print('\n'.join(markdown(args.root)))
return 0
if args.compare:
return compare(args.root, args.compare)
return check(args.root)
if __name__ == '__main__':
sys.exit(main())
+1
View File
@@ -0,0 +1 @@
PyYAML==6.0.3