Compare commits

...
43 Commits
Author SHA1 Message Date
Niels Lohmann 3c3cade958 Regenerate the amalgamated headers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:24:19 +02:00
Niels Lohmann e1a2a16548 Compare keys with key_equals() in the object hash index
As the linear lookups do since the second review round of #5622.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:24:18 +02:00
Niels Lohmann d6712df6a1 Do not include type_traits.hpp directly in the object index
json.hpp, which object_index.hpp already includes, provides it; included
directly, the json_view amalgamation (which only treats nlohmann/json.hpp
as external) copied the json internals into json_view.hpp a second time,
so the fuzzers, which include both single headers, failed with
redefinitions (make_void, integer_sequence, priority_tag, ...).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:22:09 +02:00
Niels Lohmann 4fb6bcbc43 Regenerate the amalgamated headers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:22:09 +02:00
Niels Lohmann c7b77ed23d Encode JSON_VIEW_NO_SIMD and JSON_VIEW_USE_SSSE3 in the ABI namespace
Both macros select between definitions of the same inline functions, so
translation units that disagree on them violated the ODR. Like the other
ABI-affecting macros, they now add a tag to the inline namespace (_vns and
_vssse3), and the docs replace the "Define consistently" warning with the
usual ABI compatibility note. From gregmarr's review of #5628.

nlohmann_json.natvis covers twice as many tags now, so it lists four times
as many namespaces (14.7 MB instead of 3.4 MB).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:22:08 +02:00
Niels Lohmann 674bc53f8a Regenerate the amalgamated headers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:22:07 +02:00
Niels Lohmann 32576a314f Address the review of the json_view object index and SIMD scan
From gregmarr's review of #5628:

- Docs and comments say that the hash index is one of the object's keys.
- object_index.hpp: conditional_static_cast for the hash, "slot" for the
  table position, no flag for a degenerate table, and a static_assert
  that the smallest indexed object is larger than the probe window.
- scan.hpp unrolls with NLOHMANN_VIEW_REPEAT_FIRST_8/SECOND_8.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:22:06 +02:00
Niels Lohmann 7c74cb7e9e Rename slots in the object index for Qt compatibility
Qt defines slots as a macro, so a Qt header included before
json_view.hpp broke the declarations of the local variable (see #5865).
The test variable is renamed as well.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:22:05 +02:00
Niels Lohmann d5cb3e1f75 Follow the renames from the review of the view builder
The review of #5620 renamed document_data's tape/tape_size/tape_cap to
nodes/node_count/node_capacity and made the decode arena a
std::vector<char>; adapt the code of this change to it.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:22:04 +02:00
Niels Lohmann 08809fb592 Regenerate the amalgamated headers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:22:03 +02:00
Niels Lohmann 83f7786bd7 Move the dump, comparison, and large-object tests to unit-json_view_dump.cpp
The dump and comparison tests moved to their own file in the dump pull
request; the hash index tests of this branch join them there.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:22:01 +02:00
Niels Lohmann f41ef9b1b5 Make json_view.hpp pass include-what-you-use
ci_single_binaries runs IWYU with --error on every header. For json_view.hpp
it suggested adding <array> (std::array is used), the headers that json.hpp
already provides (abi_config, abi_macros, input_adapters, json_pointer,
cpp_future, string_concat, value_t, json_fwd), and <version> for
std::nullptr_t, and removing <cstddef>.

- include <array>
- keep <cstddef> (nullptr_t, size_t; IWYU attributes them to <version> and <cstring>)
- tell IWYU not to suggest the headers that json.hpp provides: the amalgamated
  json_view.hpp only includes json.hpp, so including them here would duplicate
  their definitions

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:22:00 +02:00
Niels Lohmann 359ef2cf37 Index strings with size_t in the colliding-keys test (MSVC C4244 on Win32)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:22:00 +02:00
Niels Lohmann f1cd9b802b Avoid GCC useless-cast and strict-overflow warnings in unit-json_view.cpp
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:21:59 +02:00
Niels Lohmann 245c73a29d Fix clang-tidy 22 findings in unit-json_view.cpp and unit-json_view_builder.cpp
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:21:58 +02:00
Niels Lohmann bf56cf22b2 Fix clang-tidy 22 findings in the json_view headers
Silence bugprone-casting-through-void for the SSE loads (a reinterpret_cast
would trip -Wcast-align=strict), use auto for a cast initialiser, add
parentheses to a mixed expression, and name bugprone-std-namespace-modification
in the NOLINTs of the tuple_size/tuple_element specialisations.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:21:57 +02:00
Niels Lohmann 1a554ed76a Keep the first of duplicate keys in the object hash index again
Lookups in objects with 128 members or more return the first member of a
repeated key again, like the linear search of smaller objects. This undoes
the code change of a69542046; its test now expects the first member from
lookups (with and without a table) and the last value from materialize()
and basic_json::parse().

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:21:56 +02:00
Niels Lohmann a683851c4a Keep the last of duplicate keys in the object hash index
Lookups in objects with 128 members or more now return the last member of
a repeated key, like the linear search of smaller objects and like
materialize() and basic_json::parse(). build_object_index let the first
occurrence win, so the same text gave different results depending on the
size of the object.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:21:56 +02:00
Niels Lohmann 52c6510509 Document the hash index number in node::extra
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:21:55 +02:00
Niels Lohmann 4ef1470976 Use the portable string scan on non-x86 targets that define __SSE2__
WebAssembly with -msse2 defines __SSE2__, but has no <cpuid.h> or x86 intrinsics headers.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:21:54 +02:00
Niels Lohmann d7024a2d1a Release the spare capacity of the hash index in shrink_to_fit
shrink_to_fit() now trims the tables of large objects like the node array and the decoded strings, and the list of large objects is released as soon as the tables are built.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:21:53 +02:00
Niels Lohmann f83b37303e Bound the probe length of the large-object hash index
The key hash is not seeded, so keys chosen to collide made building the table quadratic (20,000 colliding keys took 470 ms to parse). A key may now sit at most 64 slots from its home slot; if a key would sit further away, the table is dropped and the object is searched linearly. Lookups stop after the same distance.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:21:52 +02:00
Niels Lohmann ce38a7bc76 Remove an accidentally committed GCC module cache
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:21:51 +02:00
Niels Lohmann ab6c1f9fb6 Fix GCC module build and MSVC C4127 in the json_view SIMD code
GCC ignores the target attribute in modules, so the SSSE3 dispatch is
disabled for the module interface (the check stays portable, SSE2 is kept).
Make the 8-vs-16 byte unrolling condition in scan_string_run a
preprocessor/template split to avoid a constant condition (C4127).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:21:50 +02:00
Niels Lohmann 93dced7c3e Scan json_view strings with SIMD and index large objects
Speed up json_view's parser with SIMD scanning and a hash table
for large objects.

Long runs of string bytes are scanned 16 bytes at a time with NEON
(AArch64, GCC and Clang) and SSE2 (x86-64), both baseline
instruction sets. Keys keep 16 table checks before the vector
loop, because their lengths repeat from record to record; string
values get 8, because their lengths vary more. Non-ASCII text is
validated 16 bytes at a time with simdjson's "lookup4" check
(Keiser and Lemire, 2021), with NEON on AArch64 and, on x86-64,
with SSSE3. SSSE3 is not part of baseline x86-64, so the check is
compiled for SSSE3 with a function attribute and used only where
CPUID reports it, which all x86-64 CPUs since about 2011 do; the
answer is cached in a statically initialized atomic, so there is
no guard of a local static and no global constructor. The same
input is accepted either way. JSON_VIEW_NO_SIMD selects the
portable code.

On x86-64, string runs are now checked vector-first: one SSE2
compare from the first byte finds the end of most keys and short
values, instead of a branch per byte for the first 8-16 bytes.
AArch64 keeps the byte-wise steps, where a NEON mask costs more and
the branches predict well. Entering an object or array no longer
stalls: open() stores the parent's frame field by field instead of
building it on the stack and reading it back with wider loads,
which waited for the narrower stores to retire.

Objects with 128 members or more get an open-addressing hash table
built when the object closes, so operator[], at(), find(),
contains(), count(), value(), and JSON pointers take constant time
on average in such objects; of duplicate keys, the first is kept,
as for the linear search. The idea comes from Boost.JSON.

simdjson is credited in simd.hpp's SPDX block, the README, and
license.md.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:21:50 +02:00
Niels Lohmann 3d318ab74a Allow the string literal parameter of the view serializer's put() for clang-tidy
The overload takes the literal by array reference so that its length is a
template parameter, like scan_literal() and the C array overloads of
to_json() and from_json().

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:21:36 +02:00
Niels Lohmann fd09d50a3c Exclude all json_view test files from Infer's reports
The same std::unique_ptr false positive (USE_AFTER_DELETE) also shows for
two json_documents parsed in one lambda in unit-json_view_dump.cpp
(ci_infer). It affects any test that holds several documents, so the
exclusion now covers tests/src/unit-json_view*.cpp; ASan and Valgrind
cover these tests.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:21:35 +02:00
Niels Lohmann 8473496b7b Regenerate the amalgamated headers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:21:34 +02:00
Niels Lohmann 67377f58bd Address the review of json_view dump() and comparisons
From gregmarr's review of #5624:

- The output buffer takes string literals without a length.
- write_float() names the IEEE single/double condition once, as the
  library's serializer does.
- equal() checks the types first and compares with !=; the reference to
  the top frame is noted as dangling after pop_back().
- dump.md: one paragraph per case in the notes.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:21:33 +02:00
Niels Lohmann bdbfa33498 Keep the stream includes of json_view.hpp for IWYU
json_view.hpp uses std::istream and std::ostream itself, but IWYU sees
them through the exported json.hpp and asked to remove both includes,
which fails ci_single_binaries (single_json_view).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:21:33 +02:00
Niels Lohmann 860dc63a6f Follow the renames from the review of the view builder
The review of #5620 renamed document_data's tape/tape_size/tape_cap to
nodes/node_count/node_capacity and made the decode arena a
std::vector<char>; adapt the code of this change to it.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:21:32 +02:00
Niels Lohmann 6a53ab7fb0 Regenerate the amalgamated headers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:21:31 +02:00
Niels Lohmann e94606df9b Make json_view.hpp pass include-what-you-use
ci_single_binaries runs IWYU with --error on every header. For json_view.hpp
it suggested adding <array> (std::array is used), the headers that json.hpp
already provides (abi_config, abi_macros, input_adapters, json_pointer,
cpp_future, string_concat, value_t, json_fwd), and <version> for
std::nullptr_t, and removing <cstddef>.

- include <array>
- keep <cstddef> (nullptr_t, size_t; IWYU attributes them to <version> and <cstring>)
- tell IWYU not to suggest the headers that json.hpp provides: the amalgamated
  json_view.hpp only includes json.hpp, so including them here would duplicate
  their definitions

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:21:30 +02:00
Niels Lohmann 6403910b04 Split the dump and comparison tests off unit-json_view.cpp
The MinGW linker of the Windows clang jobs cannot link object files with more
than 32767 sections ("relocation truncated to fit: IMAGE_REL_AMD64_REL32
against `.rdata'"). unit-json_view.cpp reaches that limit as the stack
grows, so its "json_view dump" and "json_view comparison" test cases move
into unit-json_view_dump.cpp. The test generator and has_duplicate_keys()
that both files use move into json_view_test_helpers.hpp.

The new file mentions JSON_HAS_CPP_17, so it is built for C++17 like the file
it was split from.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:21:29 +02:00
Niels Lohmann 4d5e490929 Fix the clang-tidy 22 findings in json_view and its tests
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:21:28 +02:00
Niels Lohmann b6a793e300 Document and test first-wins lookups next to dump() and ==
dump() writes every member and == resolves duplicate keys as parse()
does, whereas lookups find the first member of a duplicate key.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:21:28 +02:00
Niels Lohmann 6eacc86ef6 Do not shrink the output of dump()
finish() copied the whole output when the buffer was more than twice as
large as the result (citm dump +11%). The tighter source_extent()
estimate already keeps the buffer of a small value small.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:21:27 +02:00
Niels Lohmann 61f9d0f978 Hold the documents of the dump and comparison tests in names
root() of a temporary document is deleted: its views would dangle.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:21:26 +02:00
Niels Lohmann b98881bd69 Size the dump buffer of a small value by its own extent
The source extent of a value was read from the next node, falling back to the rest of the document when that node held a decoded string. dump() of a small value could thus allocate a buffer as large as the document. Skip a few such nodes, cap the fallback estimate, and shrink a buffer that is much larger than its output.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:21:25 +02:00
Niels Lohmann af380c8438 Avoid raw string with escapes inside CHECK macro (MSVC C2017)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:21:24 +02:00
Niels Lohmann c00f0592bc Add dump() and comparisons to json_view
Add basic_json_view::dump() and the comparison operators, and read
floats from the parser's digit layout instead of rescanning the
token.

dump(indent, indent_char, ensure_ascii, number_format) writes a
value the way ordered_json::parse(text).dump() writes it for the
same arguments: members in document order, all of them should a
key occur more than once; strings escaped by the same rules, using
the library's scanning kernels; floats written with the library's
to_chars conversion, so the output equals basic_json's byte for
byte; integers copied from the source, where they are already
canonical, except -0, which parse() reads as 0. There is no
error_handler argument, because the view only holds valid UTF-8.
number_format::source copies numbers exactly as they appear in the
source (e.g. "1.50", "1E2", "-0"), which basic_json cannot provide.
operator<< takes the indentation from the stream width, as for
basic_json. The writer walks iteratively, so nesting depth is
limited by memory only.

operator== and operator!= compare two views, or a view and a
basic_json value in either order, by the rules basic_json's
operator== uses: numbers compare by value across their types,
objects compare by their members with duplicate keys resolved as
parse() resolves them, member order matters only where the object
type keeps one, and discarded views compare as discarded basic_json
values do, including under JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON.
Nothing is materialized except single scalars.

While parsing, the view now records where the integer digits, the
fraction digits, and the exponent of a float token are, so floats
and doubles with at most 19 digits are read from that layout with
the library's decimal_to_float() instead of rescanning the token.
Both round correctly, so the values are those of parse(). get<double>(),
materialize(), dump(), and the comparisons all use it.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:21:24 +02:00
Niels Lohmann 0bf9fcf52c Regenerate the amalgamated headers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:20:58 +02:00
Niels Lohmann 0660e6d482 Address the second review round of json_view element access
From gregmarr's review of #5622:

- begin()/end(): "a null value or a discarded view iterates as an empty
  range", singular like the primitive case before it.
- The class page's list of function groups has no trailing commas.
- document_data::key_equals() compares a key node with a string, for the
  key comparisons that were spelled out in each lookup.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-18 22:20:57 +02:00
74 changed files with 118612 additions and 164 deletions

No files matched your search

+1
View File
@@ -62,6 +62,7 @@ labels:
- "include/nlohmann/detail/view/.*"
- "single_include/nlohmann/json_view\\.hpp"
- "tests/src/unit-json_view.*"
- "tests/src/json_view_test_helpers\\.hpp"
- "tests/src/fuzzer-parse_json_view\\.cpp"
- "tools/amalgamate/config_json_view\\.json"
- "docs/mkdocs/docs/features/json_view\\.md"
+3
View File
@@ -60,3 +60,6 @@ MODULE.bazel.lock
/json.tar.xz.asc
/release_files/
/single_include/nlohmann/*.asc
# GCC module cache
/gcm.cache/
+2 -2
View File
@@ -1,7 +1,7 @@
{
"_comment": "Used by the ci_infer CMake target (#5715 item 4b). fail-on-issue makes CI fail on Infer findings; disable-issue-type is a type-level baseline for the ~174 pre-existing findings (all PULSE_UNNECESSARY_COPY*/PULSE_RESOURCE_LEAK/PULSE_CONST_REFABLE, mostly in test code) triaged in run https://github.com/nlohmann/json/actions/runs/35829411620 on commit 1054b2097, so CI fails only on a NEW issue type. Remove an entry here once its findings have been fixed or explicitly accepted.",
"_comment_pulse": "Pulse stops exploring paths after pulse-max-disjuncts (default 20). With the default, basic_json::replace_value() (destroy + assert_invariant) exceeds the limit, Pulse loses the stored type, and reports false NULLPTR_DEREFERENCE findings for get_ptr() results in tests/src/unit-pointer_access.cpp.",
"_comment_view": "Infer's model of std::unique_ptr mixes up the `built` values that the helper build() in tests/src/unit-json_view_builder.cpp returns by value: it reports USE_AFTER_DELETE for one result after the destructor of an unrelated temporary (e.g. ~built() on line 345 invalidating `shallow` on line 350). AddressSanitizer is clean on the same tests, so reports for that test file are excluded; findings in the library headers it includes are still reported.",
"_comment_view": "Infer's model of std::unique_ptr mixes up values that are returned by value and own one: basic_json_document (parse() results) and the `built` values of the build() helper in tests/src/unit-json_view_builder.cpp. It reports USE_AFTER_DELETE for one of them after the destructor of an unrelated object (e.g. ~built() on line 345 invalidating `shallow` on line 350 of that file, or two documents parsed in one lambda in tests/src/unit-json_view_dump.cpp). AddressSanitizer and Valgrind are clean on the same tests, so reports for the json_view test files are excluded; findings reported in the library headers are still reported.",
"pulse-max-disjuncts": 40,
"fail-on-issue": true,
"disable-issue-type": [
@@ -13,6 +13,6 @@
"PULSE_CONST_REFABLE"
],
"report-block-list-path-regex": [
"tests/src/unit-json_view_builder\\.cpp"
"tests/src/unit-json_view.*\\.cpp"
]
}
+4
View File
@@ -68,6 +68,7 @@ cc_library(
"include/nlohmann/detail/string_utils.hpp",
"include/nlohmann/detail/value_t.hpp",
"include/nlohmann/detail/view/builder.hpp",
"include/nlohmann/detail/view/compare.hpp",
"include/nlohmann/detail/view/document_data.hpp",
"include/nlohmann/detail/view/errors.hpp",
"include/nlohmann/detail/view/input.hpp",
@@ -78,8 +79,11 @@ cc_library(
"include/nlohmann/detail/view/materialize.hpp",
"include/nlohmann/detail/view/node.hpp",
"include/nlohmann/detail/view/number.hpp",
"include/nlohmann/detail/view/object_index.hpp",
"include/nlohmann/detail/view/pointer.hpp",
"include/nlohmann/detail/view/scan.hpp",
"include/nlohmann/detail/view/serializer.hpp",
"include/nlohmann/detail/view/simd.hpp",
"include/nlohmann/detail/view/string_ref.hpp",
"include/nlohmann/detail/view/value.hpp",
"include/nlohmann/json.hpp",
+1
View File
@@ -1410,6 +1410,7 @@ THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR I
- The class contains parts of [Google Abseil](https://github.com/abseil/abseil-cpp) which is licensed under the [Apache 2.0 License](https://opensource.org/licenses/Apache-2.0).
- The class contains an adapted version of the Eisel-Lemire algorithm, its table of powers of five, and its digit comparison for long numbers from [fast_float](https://github.com/fastfloat/fast_float) by Daniel Lemire and contributors, which is available under the [MIT License](https://opensource.org/licenses/MIT) (used here), the Apache 2.0 License, and the Boost Software License. Copyright &copy; 2021 The fast_float authors
- The view's parser (`<nlohmann/json_view.hpp>`) contains techniques and code adapted from [yyjson](https://github.com/ibireme/yyjson) by YaoYuan, which is licensed under the [MIT License](https://opensource.org/licenses/MIT) (see above): table-driven decoding of `\u` escapes and fixed-offset unrolled checks.
- The view's parser (`<nlohmann/json_view.hpp>`) validates non-ASCII strings with the vector UTF-8 check of [simdjson](https://github.com/simdjson/simdjson) by Daniel Lemire, Geoff Langdale, John Keiser, and contributors (its "lookup4" algorithm and tables, after J. Keiser and D. Lemire, "Validating UTF-8 In Less Than One Instruction Per Byte", 2021), which is available under the [MIT License](https://opensource.org/licenses/MIT) (used here) and the Apache 2.0 License. Copyright &copy; 2018-2025 The simdjson authors
<img align="right" src="https://git.fsfe.org/reuse/reuse-ci/raw/branch/master/reuse-horizontal.png" alt="REUSE Software">
+2
View File
@@ -91,6 +91,8 @@ Binary values are serialized as an object containing two keys:
- [to_string](to_string.md) returns a string representation of a JSON value
- [operator<<](../operator_ltlt.md) serialize to stream
- [`basic_json_view::dump`](../basic_json_view/dump.md) the corresponding function of `basic_json_view`, serializing
directly from a flat index without building a `basic_json` value
- [Serialization](../../features/serialization.md) - the serialization article
## Version history
@@ -170,6 +170,8 @@ Linear.
- [operator!=](operator_ne.md) compare for inequality
- [operator<=>](operator_spaceship.md) comparison: 3-way (C++20)
- [basic_json_view::operator==](../basic_json_view/operator_eq.md) - the same comparison on a zero-copy view, without
building a `basic_json` value for it
## Version history
@@ -95,6 +95,8 @@ Linear.
- [operator==](operator_eq.md) comparison: equal
- [operator<=>](operator_spaceship.md) comparison: 3-way (C++20)
- [basic_json_view::operator!=](../basic_json_view/operator_ne.md) - the same comparison on a zero-copy view, without
building a `basic_json` value for it
## Version history
@@ -51,6 +51,11 @@ own on the next, since ownership is decided freshly each time. An input that is
[`parse()`](parse.md#notes)) is written into the document's previous owned copy, reusing its memory; a moved-in
`#!cpp std::string` replaces it.
Reusing a document matters most for large inputs: the operating system provides the memory of a fresh node index one
page at a time, and every page costs a page fault the first time it is written. On x86-64 Linux (4 KiB pages), parsing
a 55 MB document into a reused document took about 40 % less time than parsing it into a fresh one. Programs that parse
many documents of similar size should therefore keep one document and call `read()`.
An integer argument that is not a `#!cpp bool` where the flags are expected, such as `#!cpp read(ptr, len)`, does not
compile; see [`parse`](parse.md#notes).
@@ -77,6 +77,8 @@ None of these exceptions carry a [`JSON_DIAGNOSTICS`](../macros/json_diagnostics
1. Linear in the number of members: as for [`ordered_json`](../ordered_json.md), members are compared one after
another, in document order, stopping at the first match. Each comparison first checks the
key's length -- already known from the index, without reading the key bytes -- before comparing its content.
Objects with 128 or more members get a hash index of their keys while parsing, so that a lookup in them takes
constant time on average.
2. Linear in `idx`: elements are skipped one at a time from the first one, since they are not a fixed size in the
index (unlike `BasicJsonType`'s array, which is random-access).
3. Linear in the number of reference tokens of `ptr` and, for each token, in the number of members of the object at
@@ -6,7 +6,7 @@ iterator begin() const noexcept;
Returns an iterator to the first element of an array, or the first member value of an object, in **document order**
-- the order the values appear in the source text, not sorted by key. A primitive value iterates as a range of one
element (itself); `#!json null` and a [discarded](is_discarded.md) view iterate as an empty range.
element (itself), and a `#!json null` value or a [discarded](is_discarded.md) view iterates as an empty range.
## Return value
@@ -35,6 +35,8 @@ No-throw guarantee: this function never throws exceptions.
1. Linear in the number of members: as for [`ordered_json`](../ordered_json.md), members are compared one after
another, in document order, stopping at the first match. Each comparison first checks the
key's length -- already known from the index, without reading the key bytes -- before comparing its content.
Objects with 128 or more members get a hash index of their keys while parsing, so that a lookup in them takes
constant time on average.
2. Linear in the number of reference tokens of `ptr` and, for each token, in the number of members of the object at
that level or the index into the array -- as for [`operator[]`](operator[].md#complexity) and
[`at`](at.md#complexity) with a JSON pointer.
@@ -28,7 +28,8 @@ No-throw guarantee: this function never throws exceptions.
Linear in the number of members: unlike the lookup functions, `count()` does not stop at the first match, because it
has to count every member with the key. Members are compared one after another, in document order. Each comparison
first checks the key's length -- already known from the index, without reading the key bytes -- before comparing its
content.
content. The hash index of the keys of objects with 128 or more members does not help here, since every member
is visited.
## Notes
@@ -0,0 +1,104 @@
# <small>nlohmann::basic_json_view::</small>dump
```cpp
string_t dump(const int indent = -1,
const char indent_char = ' ',
const bool ensure_ascii = false,
const number_format numbers = number_format::shortest) const;
```
Serializes this value (and its subtree) directly from the flat index, without ever building a `BasicJsonType` value
first. With the default `#!cpp numbers == number_format::shortest`, the result is the same string
[`BasicJsonType::dump`](../basic_json/dump.md) would produce for the value
[`BasicJsonType::parse()`](../basic_json/parse.md) builds from the same source text, called with the same `indent`,
`indent_char`, and `ensure_ascii` -- except that members of an object appear in document order rather than sorted by
key, and *every* occurrence of a repeated key is written rather than only the last one (see
[Notes on duplicate keys](operator[].md#notes)). For a `json_view` (whose `BasicJsonType` is not ordered), this means
`dump()` can print an object's members in a different order than [`materialize()`](materialize.md)`.dump()` of the
same subtree.
## Parameters
`indent` (in)
: If `indent` is nonnegative, array elements and object members are pretty-printed with that indent level. An
indent level of `0` only inserts newlines. `-1` (the default) selects the most compact representation.
`indent_char` (in)
: The character used for indentation if `indent` is greater than `0`. The default is ` ` (space).
`ensure_ascii` (in)
: If `ensure_ascii` is `#!cpp true`, all non-ASCII characters in the output are escaped with `\uXXXX` sequences, and
the result consists of ASCII characters only.
`numbers` (in)
: how to write numbers, see [`number_format`](number_format.md): `shortest` (the default) writes them the way
[`BasicJsonType::dump`](../basic_json/dump.md) would; `source` copies every number exactly as it appears in the
source text.
## Return value
string containing the serialization of this value, or `#!cpp "<discarded>"` if the view is
[discarded](is_discarded.md).
## Exception safety
Strong exception safety: if an exception is thrown, there are no changes to the view or the document it refers to.
## Exceptions
May throw `#!cpp std::bad_alloc` if allocating the output string fails. Unlike
[`BasicJsonType::dump`](../basic_json/dump.md), there is no `error_handler` parameter and no
[`type_error.316`](../../home/exceptions.md#jsonexceptiontype_error316): the view only ever holds text the parser
already validated as UTF-8, so there is nothing to replace or ignore.
## Complexity
Linear in the size of the output text.
## Notes
The walk over the subtree is iterative, so the nesting depth it can write is limited by available memory only, not by
the call stack -- as for [`materialize()`](materialize.md).
Strings are escaped by the same rules as [`BasicJsonType::dump`](../basic_json/dump.md).
With `#!cpp numbers == number_format::shortest`, floats are written with the library's shortest round-trip conversion,
exactly as [`BasicJsonType::dump`](../basic_json/dump.md) would (e.g. `#!cpp 1.5`, `#!cpp 100.0`, `#!cpp 1e+100`), and
integers are copied from the source text -- already canonical in JSON, so this matches their shortest form too --
except that `#!cpp -0` is written as `#!cpp 0`, the way [`BasicJsonType::parse()`](../basic_json/parse.md) reads it.
`#!cpp number_format::source` copies every number exactly as written in the source text instead, with no exception
for `#!cpp -0` -- `#!cpp 1.50`, `#!cpp 1E2`, `#!cpp -0.0`, `#!cpp -0`, or all digits of an integer literal with more
digits than any number type holds (such a literal is itself classified as a float, see
[What is different](../../features/json_view.md#what-is-different)) -- something `BasicJsonType` cannot do, since
parsing already reduces every number to its parsed value.
## Examples
??? example
The example below forwards a single record out of a larger batch, and re-serializes a configuration file, both
without ever building a `BasicJsonType` value for the surrounding array or for the parts of it that were not
needed. It also shows that [`materialize()`](materialize.md)`.dump()` of the configuration sorts its keys, where
`dump()` on the view keeps the order they appear in the source text.
```cpp
--8<-- "examples/basic_json_view__dump.cpp"
```
Output:
```json
--8<-- "examples/basic_json_view__dump.output"
```
## See also
- [`number_format`](number_format.md) - how `dump()` writes numbers
- [operator<<](operator_ltlt.md) - serialize this value to a stream
- [materialize](materialize.md) - build a `BasicJsonType` value, e.g. to use `BasicJsonType::dump`'s `error_handler`
- [`BasicJsonType::dump`](../basic_json/dump.md) - the corresponding function of `basic_json`
## Version history
- Added in version 3.13.0.
+2 -3
View File
@@ -5,9 +5,8 @@ iterator end() const noexcept;
```
Returns an iterator to one past the last element of an array, one past the last member value of an object, in
**document order** -- see [`begin()`](begin.md). A primitive value iterates as a range of one element (itself);
`#!json null` and a [discarded](is_discarded.md) view iterate as an empty range, so `#!cpp begin() == end()` for
them.
**document order** -- see [`begin()`](begin.md). A primitive value iterates as a range of one element (itself), and a
`#!json null` value or a [discarded](is_discarded.md) view iterates as an empty range (`#!cpp begin() == end()`).
## Return value
@@ -28,6 +28,8 @@ No-throw guarantee: this function never throws exceptions.
Linear in the number of members: as for [`ordered_json`](../ordered_json.md), members are compared one after another,
in document order, stopping at the first match. Each comparison first checks the key's length
-- already known from the index, without reading the key bytes -- before comparing its content.
Objects with 128 or more members get a hash index of their keys while parsing, so that a lookup in them takes
constant time on average.
## Notes
+21 -8
View File
@@ -22,17 +22,19 @@ Moving the document itself does not invalidate its views: the index is heap-allo
`basic_json_view` provides the read-only part of the `BasicJsonType` interface:
- the type-inspection functions,
- element access,
- lookup,
- iteration, and
- the type-inspection functions
- element access
- lookup
- iteration
- conversion: [`get<T>()`](get.md), [`get_string()`](get_string.md), [`number_token()`](number_token.md), and
[`materialize()`](materialize.md) to build the `BasicJsonType` value of a subtree on demand.
[`materialize()`](materialize.md) to build the `BasicJsonType` value of a subtree on demand
- comparison: [`operator==`](operator_eq.md) and [`operator!=`](operator_ne.md) compare two views, or a view and a
`BasicJsonType` value, without ever building a `BasicJsonType` value for a view; no ordering comparison
(`#!cpp operator<`) is provided
[`operator[]`](operator%5B%5D.md), [`at`](at.md), [`contains`](contains.md), and [`value`](value.md) also accept a
[`json_pointer`](../json_pointer/index.md). It does not (yet) provide `dump()` or comparison. Unlike `basic_json`,
there are no implicit conversion operators (no `#!cpp operator ValueType()`), and no `#!cpp get_ptr()` or
`#!cpp get_ref()`; see [`get`](get.md).
[`json_pointer`](../json_pointer/index.md). Unlike `basic_json`, there are no implicit conversion operators (no
`#!cpp operator ValueType()`), and no `#!cpp get_ptr()` or `#!cpp get_ref()`; see [`get`](get.md).
## Template parameters
@@ -54,6 +56,7 @@ there are no implicit conversion operators (no `#!cpp operator ValueType()`), an
- **string_view_t** - `#!cpp std::string_view` on C++17 and newer, a minimal internal substitute otherwise
- **iterator**, **const_iterator** - a forward iterator over the elements of an array or the member values of an
object, in document order; both names refer to the same type, since a view is always read-only
- [**number_format**](number_format.md) - how [`dump()`](dump.md) writes numbers
## Member functions
@@ -112,6 +115,16 @@ there are no implicit conversion operators (no `#!cpp operator ValueType()`), an
- [**number_token**](number_token.md) - get a number's token text without a copy
- [**materialize**](materialize.md) - build the `BasicJsonType` value of this subtree
### Comparison
- [**operator==**](operator_eq.md) - comparison: equal
- [**operator!=**](operator_ne.md) - comparison: not equal
### Serialization
- [**dump**](dump.md) - serialize to a JSON-formatted string
- [**operator<<**](operator_ltlt.md) - serialize to stream
### Source access
- [**source_offset**](source_offset.md) - byte offset of this value in the document's source text
@@ -0,0 +1,51 @@
# <small>nlohmann::basic_json_view::</small>number_format
```cpp
enum class number_format {
shortest,
source
};
```
This enumeration is used in [`dump`](dump.md) to choose how numbers are written. Two values are differentiated:
shortest
: integers are copied from the source text -- already canonical in JSON -- except that `#!cpp -0` becomes
`#!cpp 0`, the way [`BasicJsonType::parse()`](../basic_json/parse.md) reads it; floats are written with the
library's shortest round-trip conversion, exactly as [`BasicJsonType::dump()`](../basic_json/dump.md) would (e.g.
`#!cpp 1.5`, `#!cpp 100.0`, `#!cpp 1e+100`)
source
: every number is copied exactly as it appears in the source text -- `#!cpp 1.50`, `#!cpp 1E2`, `#!cpp -0`, all
digits of an integer literal with more digits than any number type holds -- something `BasicJsonType` cannot do,
since parsing already reduces every number to its parsed value
## Examples
??? example
The example below writes back a price list received from a supplier: with `number_format::shortest` (the
default), a trailing zero and scientific notation are normalized away and a long account number that overflows
every number type is rounded, the same way `#!cpp materialize().dump()` (or `basic_json::dump()`) would;
`number_format::source` keeps every number exactly as it was written in the source text instead.
```cpp
--8<-- "examples/basic_json_view__number_format.cpp"
```
Output:
```json
--8<-- "examples/basic_json_view__number_format.output"
```
## See also
- [dump](dump.md) - serialize to a JSON-formatted string
- [number_token](number_token.md) - get a single number's token text without dumping the whole value
- [`BasicJsonType::error_handler_t`](../basic_json/error_handler_t.md) - the analogous enumeration for
`BasicJsonType::dump`'s decoding-error behavior
## Version history
- Added in version 3.13.0.
@@ -83,6 +83,8 @@ None of these exceptions carry a [`JSON_DIAGNOSTICS`](../macros/json_diagnostics
another, in document order, stopping at the first match. Each comparison first checks the
key's length -- already known from the index, without reading the key bytes -- before comparing its content, so a
key of a different length than `key` is rejected without touching the source text.
Objects with 128 or more members get a hash index of their keys while parsing, so that a lookup in them takes
constant time on average.
2. Linear in `idx`: elements are skipped one at a time from the first one, since they are not a fixed size in the
index (unlike `BasicJsonType`'s array, which is random-access).
3. Linear in the number of reference tokens of `ptr` and, for each token, in the number of members of the object at
@@ -114,6 +116,7 @@ document.
member in order and so keep the *last* value for a repeated key, so `#!cpp v["a"]` and
`#!cpp v.materialize()["a"]` can differ. To get the value `parse()` would give, use
[`materialize()`](materialize.md) or iterate the members with [`items()`](items.md) and keep the last match.
The key hash index of a larger object (128 members or more) leads to the first member of a key as well.
[`begin()`](begin.md)/[`end()`](end.md) and [`items()`](items.md) iterate over *all* members, including
duplicates, in document order. See [Duplicate keys](../../features/json_view.md#duplicate-keys) and
[`size()`](size.md#notes).
@@ -0,0 +1,109 @@
# <small>nlohmann::basic_json_view::</small>operator==
```cpp
// (1)
bool operator==(const basic_json_view& lhs, const basic_json_view& rhs);
// (2)
bool operator==(const basic_json_view& lhs, const BasicJsonType& rhs);
bool operator==(const BasicJsonType& lhs, const basic_json_view& rhs);
```
1. Compares two views for equality: whether the values [`BasicJsonType::parse()`](../basic_json/parse.md) would
produce for `lhs` and `rhs` are equal, according to `BasicJsonType`'s [`operator==`](../basic_json/operator_eq.md).
2. Compares a view and a `BasicJsonType` value for equality, in either order: whether the value `parse()` would
produce for the view and the other operand are equal, according to `BasicJsonType`'s
[`operator==`](../basic_json/operator_eq.md).
Neither overload builds a `BasicJsonType` value for a view to do the comparison (see [Notes](#notes) below). Numbers
compare by value across their types (`#!cpp 1 == 1.0`), and an object compares by its members, with duplicate keys
resolved exactly as `parse()` resolves them -- the last value, at the position of the first occurrence of the key.
This is not what [`operator[]`](operator[].md) and the other lookups return for a duplicate key: they find the *first*
member (see [Duplicate keys](../../features/json_view.md#duplicate-keys)).
## Parameters
`lhs` (in)
: first value to consider
`rhs` (in)
: second value to consider
## Return value
whether the values `lhs` and `rhs` are equal
## Exception safety
Strong exception safety: if an exception is thrown, there are no changes to either operand, or to the document(s) a
view refers to.
## Exceptions
May throw `#!cpp std::bad_alloc`. Unlike the other comparison and most other `basic_json_view` functions,
`operator==` is not `#!cpp noexcept`: resolving an object's members needs a temporary array to sort them by key (see
[Complexity](#complexity) below), and that allocation can fail.
## Complexity
Linear in the size of the compared values: every number, string, array element, and object member is visited at most
once, and the walk is iterative, so the nesting depth it can compare is limited by available memory only, not by the
call stack (as for [`materialize()`](materialize.md)). Resolving an object's members takes an additional O(n log n)
in the number of members at that level, since they are sorted by key to detect and resolve duplicates before being
compared. Two arrays of different [`size()`](size.md) are rejected without visiting either one's elements.
## Notes
Only a single number, boolean, or `#!cpp null` value is ever materialized into a `BasicJsonType`, to reuse its
`operator==` -- for numbers, so that values written differently in the source text but equal in value (e.g. an
integer and a floating-point literal) still compare equal, following the same rules `BasicJsonType` does for special
values such as `#!cpp NaN`. Constructing one of these scalars never allocates. Strings are compared directly, without
allocating, either from the source text on both sides or, for overload 2, against `BasicJsonType`'s own string.
Arrays and objects are never materialized at all; only their elements or members are visited, one pair at a time.
!!! info "How objects are compared"
For a [`json_view`](../json_view.md) (`BasicJsonType::object_t` is `#!cpp std::map`), members are compared by
key, regardless of the order they appear in the source text. For an
[`ordered_json_view`](../ordered_json_view.md) (`object_t` is `ordered_map`), they are compared in the order
they occur, so the very same two objects with their members reordered can compare equal as `json_view`s but not
as `ordered_json_view`s. This is exactly how [`json`](../json.md) and [`ordered_json`](../ordered_json.md)
compare, see ["Comparing different `basic_json` specializations"](../basic_json/operator_eq.md#notes).
!!! info "Discarded views"
A [discarded](is_discarded.md) view compares the same way a discarded `BasicJsonType` value does, which is
governed by
[`JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON`](../macros/json_use_legacy_discarded_value_comparison.md): by
default, a discarded view is never equal to anything, not even another discarded view.
No ordering comparison (`#!cpp operator<`) is provided for `basic_json_view`; [`materialize()`](materialize.md) is
the way to get a `BasicJsonType` value that supports it.
## Examples
??? example
The example below checks whether a newly received configuration differs from the previous one, and whether a
received document matches what a test expects -- directly on views, without ever materializing a `BasicJsonType`
value for either side.
```cpp
--8<-- "examples/basic_json_view__operator_eq.cpp"
```
Output:
```json
--8<-- "examples/basic_json_view__operator_eq.output"
```
## See also
- [operator!=](operator_ne.md) - compare for inequality
- [materialize](materialize.md) - build a `BasicJsonType` value, e.g. to keep comparing after the document is gone
- [`BasicJsonType::operator==`](../basic_json/operator_eq.md) - the corresponding function of `basic_json`
## Version history
- Added in version 3.13.0.
@@ -0,0 +1,74 @@
# <small>nlohmann::basic_json_view::</small>operator<<
```cpp
std::ostream& operator<<(std::ostream& o, const basic_json_view& v);
```
Not available when [`JSON_NO_IO`](../macros/json_no_io.md) is defined.
Serializes the given view `v` to the output stream `o`, using [`dump`](dump.md) -- exactly as
`#!cpp operator<<(std::ostream&, const basic_json&)` does for a `basic_json` value.
- The indentation of the output can be controlled with the member variable `width` of the output stream `o`. For
instance, using the manipulator `std::setw(4)` on `o` sets the indentation level to `4`, and the serialization
result is the same as calling `#!cpp v.dump(4)`. A `width` of `0` or less (the default) selects the most compact
representation, as `#!cpp v.dump(-1)` does.
- The indentation character can be controlled with the member variable `fill` of the output stream `o`. For instance,
the manipulator `std::setfill('\t')` sets indentation to use a tab character rather than the default space
character.
- As for `basic_json`, `o`'s `width` is reset to `0` after this call, whether or not it was greater than `0` before.
Numbers are always written as `#!cpp v.dump()` writes them by default, i.e. as with
[`number_format::shortest`](number_format.md); there is no way to select `#!cpp number_format::source` through the
stream.
## Parameters
`o` (in, out)
: stream to write to
`v` (in)
: view to serialize
## Return value
the stream `o`
## Exceptions
May throw `#!cpp std::bad_alloc`, propagated from [`dump`](dump.md#exceptions). Unlike
`#!cpp operator<<(std::ostream&, const basic_json&)`, there is no UTF-8 decoding step that could throw
[`type_error.316`](../../home/exceptions.md#jsonexceptiontype_error316), and no `error_handler` to choose between --
see the [Exceptions](dump.md#exceptions) of `dump`.
## Complexity
Linear, as [`dump`](dump.md#complexity).
## Examples
??? example
The example below writes one record out of a larger batch straight to a log stream -- compact for a one-line
entry, and pretty-printed with `std::setw`/`std::setfill` for a readable dump -- without ever building a
`BasicJsonType` value for the record, or for the rest of the batch.
```cpp
--8<-- "examples/basic_json_view__operator_ltlt.cpp"
```
Output:
```json
--8<-- "examples/basic_json_view__operator_ltlt.output"
```
## See also
- [dump](dump.md) - serialize to a JSON-formatted string
- [`operator<<(std::ostream&)`](../operator_ltlt.md) - the corresponding operator for `basic_json`
- [`JSON_NO_IO`](../macros/json_no_io.md) - switch off functions relying on certain C++ I/O headers
## Version history
- Added in version 3.13.0.
@@ -0,0 +1,82 @@
# <small>nlohmann::basic_json_view::</small>operator!=
```cpp
// (1)
bool operator!=(const basic_json_view& lhs, const basic_json_view& rhs);
// (2)
bool operator!=(const basic_json_view& lhs, const BasicJsonType& rhs);
bool operator!=(const BasicJsonType& lhs, const basic_json_view& rhs);
```
1. Compares two views for inequality. Returns `#!cpp !(lhs == rhs)`, see [operator==](operator_eq.md).
2. Compares a view and a `BasicJsonType` value for inequality, in either order. Returns `#!cpp !(lhs == rhs)` (or,
for the reversed order, `#!cpp !(rhs == lhs)`), see [operator==](operator_eq.md).
Since `operator!=` is defined as the negation of [`operator==`](operator_eq.md), it follows the same rules for
special cases: for instance, since a [discarded](is_discarded.md) view is never equal to anything by default (see
[operator=='s Notes](operator_eq.md#notes)), it is never *unequal* to anything either -- `#!cpp discarded != discarded`
is also `#!cpp false`, exactly as for a discarded `BasicJsonType` value.
## Parameters
`lhs` (in)
: first value to consider
`rhs` (in)
: second value to consider
## Return value
whether the values `lhs` and `rhs` are not equal
## Exception safety
Strong exception safety: if an exception is thrown, there are no changes to either operand, or to the document(s) a
view refers to.
## Exceptions
May throw `#!cpp std::bad_alloc`, propagated from [`operator==`](operator_eq.md#exceptions). Unlike most other
`basic_json_view` functions, `operator!=` is not `#!cpp noexcept`.
## Complexity
Linear, as [`operator==`](operator_eq.md#complexity).
## Notes
See the [Notes](operator_eq.md#notes) of `operator==` -- in particular for how an object's members are compared
(order matters for [`ordered_json_view`](../ordered_json_view.md) but not for [`json_view`](../json_view.md)) and
for how discarded views compare.
No ordering comparison (`#!cpp operator<`) is provided for `basic_json_view`; [`materialize()`](materialize.md) is
the way to get a `BasicJsonType` value that supports it.
## Examples
??? example
The example below asserts, as a test would, that a received document differs from an unwanted value, and shows
that -- as for [`json`](../json.md)/[`ordered_json`](../ordered_json.md) -- reordering an object's members is
detected as a difference for an `ordered_json_view` but not for a `json_view`.
```cpp
--8<-- "examples/basic_json_view__operator_ne.cpp"
```
Output:
```json
--8<-- "examples/basic_json_view__operator_ne.output"
```
## See also
- [operator==](operator_eq.md) - compare for equality
- [materialize](materialize.md) - build a `BasicJsonType` value, e.g. to keep comparing after the document is gone
- [`BasicJsonType::operator!=`](../basic_json/operator_ne.md) - the corresponding function of `basic_json`
## Version history
- Added in version 3.13.0.
@@ -70,6 +70,8 @@ None of these exceptions carry a [`JSON_DIAGNOSTICS`](../macros/json_diagnostics
1. Linear in the number of members: as for [`operator[]`](operator[].md#complexity), members are compared one after
another, in document order, stopping at the first match. Plus the complexity of converting
the found member to `T` (see [`get`](get.md)).
Objects with 128 or more members get a hash index of their keys while parsing, so that a lookup in them takes
constant time on average.
2. Linear in the number of reference tokens of `ptr` and, for each token, in the number of members of the object at
that level or the index into the array -- as for the [`operator[]`](operator[].md#complexity) and
[`at`](at.md#complexity) overloads that take a JSON pointer. Plus the complexity of converting the resolved value
+2
View File
@@ -38,6 +38,8 @@ header. See also the [macro overview page](../../features/macros.md).
- [**JSON_SKIP_UNSUPPORTED_COMPILER_CHECK**](json_skip_unsupported_compiler_check.md) - do not warn about unsupported compilers
- [**JSON_USE_GLOBAL_UDLS**](json_use_global_udls.md) - place user-defined string literals (UDLs) into the global namespace
- [**JSON_USE_SIMDUTF**](json_use_simdutf.md) - use the simdutf library to accelerate UTF-8 validation
- [**JSON_VIEW_NO_SIMD**](json_view_no_simd.md) - use only portable code in the parser of `json_view.hpp`
- [**JSON_VIEW_USE_SSSE3**](json_view_use_ssse3.md) - validate non-ASCII strings with SSSE3 in the parser of `json_view.hpp`
## Library version
@@ -0,0 +1,52 @@
# JSON_VIEW_NO_SIMD
```cpp
#define JSON_VIEW_NO_SIMD
```
When defined, the parser of [`basic_json_document`](../basic_json_document/index.md) (`<nlohmann/json_view.hpp>`)
uses only portable C++ to scan strings. By default, it scans long runs of string bytes 16 at a time with NEON on
AArch64 (with GCC and Clang) and SSE2 on x86-64, which are part of the baseline instruction sets of these
architectures, and validates non-ASCII text with NEON (or SSSE3, see
[`JSON_VIEW_USE_SSSE3`](json_view_use_ssse3.md)).
The same input is accepted or rejected either way, with the same values, and errors are reported the same way; only
the speed differs. The macro exists for platforms whose compilers lack the intrinsics headers, and to test the portable
code.
!!! note "ABI compatibility"
The macro selects between two definitions of the same inline functions. Whether it is defined is encoded in the
[namespace](../../features/namespace.md) (tag `_vns`), resulting in distinct symbol names. Translation units
compiled with and without it can therefore be linked into the same program without One Definition Rule (ODR)
violations, but they cannot exchange instances of library types.
## Default definition
By default, `#!cpp JSON_VIEW_NO_SIMD` is not defined, and the vector code is used where available.
```cpp
#undef JSON_VIEW_NO_SIMD
```
## Examples
??? example
The code below uses the portable string scanning of the view.
```cpp
#define JSON_VIEW_NO_SIMD
#include <nlohmann/json_view.hpp>
...
```
## See also
- [JSON_VIEW_USE_SSSE3](json_view_use_ssse3.md) - validate non-ASCII strings with SSSE3 on x86-64
- [json_view](../../features/json_view.md) - the zero-copy view
## Version history
- Added in version 3.13.0.
@@ -0,0 +1,53 @@
# JSON_VIEW_USE_SSSE3
```cpp
#define JSON_VIEW_USE_SSSE3
```
When defined on x86-64, the parser of [`basic_json_document`](../basic_json_document/index.md)
(`<nlohmann/json_view.hpp>`) validates non-ASCII text in strings with SSSE3 without asking the CPU first.
By default, the parser checks once at run time whether the CPU has SSSE3 (all x86-64 CPUs since about 2011 have it)
and then validates non-ASCII text 16 bytes at a time, using the "lookup4" algorithm of
[simdjson](https://github.com/simdjson/simdjson); on CPUs without SSSE3, it validates one UTF-8 sequence at a time.
The vector check is compiled for SSSE3 with a function attribute (GCC 4.9 and later, Clang), so this needs no compiler
option. With MSVC, the check uses `__cpuid`. On AArch64, the vector check uses NEON and is always on.
Define the macro only together with a compiler option that enables SSSE3 (e.g. `-mssse3`, or `-march=` with a CPU that
has it), and only for programs that run on such CPUs. It saves the check of the CPU, which costs little. The same
input is accepted or rejected either way; only the speed of non-ASCII text differs.
!!! note "ABI compatibility"
The macro selects between two definitions of the same inline functions. Whether it is defined is encoded in the
[namespace](../../features/namespace.md) (tag `_vssse3`, on every architecture), resulting in distinct symbol
names. Translation units compiled with and without it can therefore be linked into the same program without One
Definition Rule (ODR) violations, but they cannot exchange instances of library types.
## Default definition
By default, `#!cpp JSON_VIEW_USE_SSSE3` is not defined.
```cpp
#undef JSON_VIEW_USE_SSSE3
```
## Examples
??? example
With CMake, for a program that only runs on CPUs with SSSE3:
```cmake
target_compile_definitions(your_target PRIVATE JSON_VIEW_USE_SSSE3)
target_compile_options(your_target PRIVATE -mssse3)
```
## See also
- [JSON_VIEW_NO_SIMD](json_view_no_simd.md) - use only portable code in the view's parser
- [JSON_USE_SIMDUTF](json_use_simdutf.md) - validate UTF-8 with simdutf in `basic_json`'s parser
## Version history
- Added in version 3.13.0.
@@ -39,5 +39,6 @@ the library.
## Version history
- Added in version 3.11.0. Changed inline namespace name in version 3.11.2. Added the ABI tag `_dp` in
version 3.12.0, and the ABI tags `_bics`, `_psp`, `_snul`, `_sbu8`, and `_ekmo` in version 3.13.0; see
version 3.12.0, and the ABI tags `_bics`, `_psp`, `_snul`, `_sbu8`, `_ekmo`, `_vns`, and `_vssse3` in version
3.13.0; see
[`nlohmann` Namespace](../../features/namespace.md#structure).
+2
View File
@@ -86,6 +86,8 @@ Linear.
## See also
- [dump](basic_json/dump.md) - serialize to a JSON-formatted string
- [`basic_json_view::operator<<`](basic_json_view/operator_ltlt.md) - the corresponding operator for
`basic_json_view`
- [Serialization](../features/serialization.md) - the serialization article
## Version history
@@ -0,0 +1,25 @@
#include <iostream>
#include <nlohmann/json_view.hpp>
using json_document = nlohmann::json_document;
using json_view = nlohmann::json_view;
int main()
{
// a large batch of sensor readings -- forward just the one that changed,
// without ever building a basic_json value for the batch or for the
// readings that are not needed
const json_document batch = json_document::parse(R"(
[{"id": 1, "temp": 21.5}, {"id": 2, "temp": 87.3}, {"id": 3, "temp": 21.7}]
)");
const json_view readings = batch.root();
std::cout << readings[1].dump() << '\n';
// a configuration file -- dump() on the view keeps the member order of
// the source text; a json value's object_t is std::map, so
// materialize().dump() of the very same view sorts the keys instead
const json_document config = json_document::parse(
R"({"name": "cache", "host": "db1", "port": 6379, "timeout": 30})");
std::cout << config.root().dump(2) << "\n\n";
std::cout << config.root().materialize().dump(2) << '\n';
}
@@ -0,0 +1,14 @@
{"id":2,"temp":87.3}
{
"name": "cache",
"host": "db1",
"port": 6379,
"timeout": 30
}
{
"host": "db1",
"name": "cache",
"port": 6379,
"timeout": 30
}
@@ -0,0 +1,28 @@
#include <iostream>
#include <nlohmann/json_view.hpp>
using json_document = nlohmann::json_document;
using json_view = nlohmann::json_view;
int main()
{
// a price list received from a supplier feed -- prices and account
// numbers must be forwarded exactly, e.g. into an invoice
const json_document doc = json_document::parse(R"(
[{"sku": "A1", "price": 19.90, "account_id": 12345678901234567890123456},
{"sku": "A2", "price": 1E2, "account_id": 98765432109876543210987654}]
)");
const json_view list = doc.root();
// number_format::shortest (the default) writes numbers the way
// basic_json::dump() would: "19.90" becomes "19.9", "1E2" becomes
// "100.0", and each account number -- far beyond any 64-bit integer --
// is rounded to the nearest double, exactly as materialize().dump()
// (or a plain nlohmann::json) would round it
std::cout << list.dump() << '\n';
// number_format::source copies every number exactly as it was written
// in the source text instead -- something basic_json cannot do at all,
// since parsing already reduces every number to its parsed value
std::cout << list.dump(-1, ' ', false, json_view::number_format::source) << '\n';
}
@@ -0,0 +1,2 @@
[{"sku":"A1","price":19.9,"account_id":1.2345678901234568e+25},{"sku":"A2","price":100.0,"account_id":9.876543210987655e+25}]
[{"sku":"A1","price":19.90,"account_id":12345678901234567890123456},{"sku":"A2","price":1E2,"account_id":98765432109876543210987654}]
@@ -0,0 +1,30 @@
#include <iostream>
#include <nlohmann/json_view.hpp>
using json_document = nlohmann::json_document;
using json = nlohmann::json;
int main()
{
// two snapshots of a polled configuration endpoint -- compare them
// directly as views, without ever building a nlohmann::json value for
// either one
const json_document previous = json_document::parse(
R"({"name": "cache", "port": 6379, "timeout": 30})");
const json_document current = json_document::parse(
R"({"port": 6379.0, "timeout": 30, "name": "cache"})");
// same members, reordered, and 6379 written as a float -- operator==
// treats them the same way BasicJsonType::operator== would
std::cout << std::boolalpha << (previous.root() == current.root()) << '\n';
// an actually changed value is detected the same way
const json_document changed = json_document::parse(
R"({"name": "cache", "port": 6380, "timeout": 30})");
std::cout << (previous.root() == changed.root()) << '\n';
// comparing a view directly against an expected json value -- handy in a
// test, without materializing the received document at all
const json expected = {{"name", "cache"}, {"port", 6379}, {"timeout", 30}};
std::cout << (previous.root() == expected) << '\n';
}
@@ -0,0 +1,3 @@
true
false
true
@@ -0,0 +1,26 @@
#include <iostream>
#include <iomanip>
#include <nlohmann/json_view.hpp>
using json_document = nlohmann::json_document;
using json_view = nlohmann::json_view;
int main()
{
// one order out of a large incoming batch -- write it straight to a log
// stream without ever building a basic_json value for it, or for the
// rest of the batch
const json_document doc = json_document::parse(R"(
[{"id": 1, "item": "cable"}, {"id": 2, "item": "adapter"}]
)");
const json_view orders = doc.root();
// compact, for a one-line log entry
std::cout << orders[1] << '\n';
// std::setw sets the indentation level, exactly as for basic_json
std::cout << std::setw(2) << orders[1] << "\n\n";
// std::setfill changes the indentation character
std::cout << std::setw(1) << std::setfill('\t') << orders[1] << '\n';
}
@@ -0,0 +1,10 @@
{"id":2,"item":"adapter"}
{
"id": 2,
"item": "adapter"
}
{
"id": 2,
"item": "adapter"
}
@@ -0,0 +1,28 @@
#include <iostream>
#include <nlohmann/json_view.hpp>
using json_document = nlohmann::json_document;
using ordered_json_document = nlohmann::ordered_json_document;
using json = nlohmann::json;
int main()
{
// assert, as a test would, that a received document differs from an
// unwanted shape -- without ever materializing it into a json value just
// to compare
const json_document received = json_document::parse(
R"({"status": "ok", "code": 200})");
const json unwanted = {{"status", "error"}, {"code", 500}};
std::cout << std::boolalpha << (received.root() != unwanted) << '\n';
// json (std::map) compares object members regardless of order ...
const json_document a = json_document::parse(R"({"a": 1, "b": 2})");
const json_document b = json_document::parse(R"({"b": 2, "a": 1})");
std::cout << (a.root() != b.root()) << '\n';
// ... but ordered_json (ordered_map) compares them in the order they
// appear, so the very same reordering is detected as a difference
const ordered_json_document oa = ordered_json_document::parse(R"({"a": 1, "b": 2})");
const ordered_json_document ob = ordered_json_document::parse(R"({"b": 2, "a": 1})");
std::cout << (oa.root() != ob.root()) << '\n';
}
@@ -0,0 +1,3 @@
true
false
true
+27 -5
View File
@@ -155,8 +155,12 @@ document: `#!cpp auto v = json_document::parse(text).root();` does not compile.
element access and lookup functions never carry the JSON Pointer path `JSON_DIAGNOSTICS` would otherwise add: the
view has no `basic_json` value to point at, so the exception is created without one, regardless of how
`BasicJsonType` was built.
- **`dump()` and comparison are not (yet) provided** by `basic_json_view`. For now,
[`materialize()`](../api/basic_json_view/materialize.md) is the way to get a value you can do those things with.
- **Ordering comparisons are not provided** by `basic_json_view` -- there is no `#!cpp operator<`.
[`operator==`](../api/basic_json_view/operator_eq.md) and [`operator!=`](../api/basic_json_view/operator_ne.md) are
provided, though: two views, or a view and a `BasicJsonType` value, compare equal exactly when
[`materialize()`](../api/basic_json_view/materialize.md) or [`parse()`](../api/basic_json/parse.md) would produce
equal values for them, without ever building a tree to do it. For ordering, too,
[`materialize()`](../api/basic_json_view/materialize.md) is the way to get a value you can compare.
## Duplicate keys
@@ -176,9 +180,10 @@ document: `#!cpp auto v = json_document::parse(text).root();` does not compile.
**Why the first?** A lookup can stop as soon as it finds a match. Returning the last member would force every lookup
to scan all members of the object, even when the key is found at the very first one: this made lookups in small
objects 1.6 to 3.4 times slower. Other zero-copy parsers that index the source text, such as yyjson and simdjson, also
return the first member. RFC 8259 only says that names within an object SHOULD be unique and that the behavior of a
receiver that sees duplicates is unpredictable, so neither choice is wrong.
objects 1.6 to 3.4 times slower. The hash index of the keys of objects with 128 members or more leads to the first member of a
key as well. Other zero-copy parsers that index the source text, such as yyjson and simdjson, also return the first
member. RFC 8259 only says that names within an object SHOULD be unique and that the behavior of a receiver that sees
duplicates is unpredictable, so neither choice is wrong.
**What stays the same as `parse()`?** [`materialize()`](../api/basic_json_view/materialize.md) and
[`get<std::map<...>>()`](../api/basic_json_view/get.md) replay every member in order, so they keep the *last* value
@@ -227,6 +232,23 @@ Two conversions never copy at all:
Both results are only valid as long as the view -- and, for a string with no escapes, the borrowed source text -- is.
## Writing a view back
[`dump()`](../api/basic_json_view/dump.md) serializes a view directly from the flat index, without ever building a
`basic_json` value. An object's members are written in document order, not sorted by key, and *every* occurrence of a
repeated key is written, not only the last one -- the same two ways [iteration](#what-is-different) already differs
from a [`materialize()`](../api/basic_json_view/materialize.md)d value, see [Duplicate keys](#duplicate-keys).
`#!cpp materialize().dump()` gives a different result in both respects for a `json_view`.
By default, numbers are written the way [`basic_json::dump()`](../api/basic_json/dump.md) would.
[`number_format::source`](../api/basic_json_view/number_format.md) instead copies every number exactly as it was
written in the source text -- a price like `#!cpp 19.90`, a long order or account ID with more digits than any number
type holds, or a high-precision coordinate -- something `basic_json` cannot do at all, since parsing already reduces
a number to its parsed `#!cpp double`/`#!cpp int64_t` value.
[`operator<<`](../api/basic_json_view/operator_ltlt.md) writes a view to a stream the way `basic_json`'s does, using
the stream's `width`/`fill` for indentation.
## Choosing between `json`, `ordered_json`, the SAX interface, and `json_view`
| | [`json`](../api/json.md) / [`ordered_json`](../api/ordered_json.md) | [SAX interface](parsing/sax_interface.md) | [`json_document`](../api/json_document.md) / [`json_view`](../api/json_view.md) |
+6 -2
View File
@@ -23,6 +23,8 @@ The complete default namespace name is derived as follows:
- [`JSON_STRICT_BINARY_UTF8`](../api/macros/json_strict_binary_utf8.md) defined non-zero appends `_sbu8`.
- [`JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS`](../api/macros/json_use_objects_for_enum_keyed_maps.md) defined non-zero
appends `_ekmo`.
- [`JSON_VIEW_NO_SIMD`](../api/macros/json_view_no_simd.md) defined appends `_vns`.
- [`JSON_VIEW_USE_SSSE3`](../api/macros/json_view_use_ssse3.md) defined appends `_vssse3`.
- The inline namespace ends with the suffix `_v` followed by the 3 components of the version number separated by
underscores. To omit the version component, see [Disabling the version component](#disabling-the-version-component)
below.
@@ -105,5 +107,7 @@ follows:
- Added ABI tags `_bics` ([`JSON_BRACE_INIT_COPY_SEMANTICS`](../api/macros/json_brace_init_copy_semantics.md)), `_psp`
([`JSON_PRECISE_STREAM_POSITION`](../api/macros/json_precise_stream_position.md)), `_snul`
([`JSON_STRICT_NUL_HANDLING`](../api/macros/json_strict_nul_handling.md)), `_sbu8`
([`JSON_STRICT_BINARY_UTF8`](../api/macros/json_strict_binary_utf8.md)), and `_ekmo`
([`JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS`](../api/macros/json_use_objects_for_enum_keyed_maps.md)) in version 3.13.0.
([`JSON_STRICT_BINARY_UTF8`](../api/macros/json_strict_binary_utf8.md)), `_ekmo`
([`JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS`](../api/macros/json_use_objects_for_enum_keyed_maps.md)), `_vns`
([`JSON_VIEW_NO_SIMD`](../api/macros/json_view_no_simd.md)), and `_vssse3`
([`JSON_VIEW_USE_SSSE3`](../api/macros/json_view_use_ssse3.md)) in version 3.13.0.
+6 -1
View File
@@ -196,7 +196,7 @@ packet-beta
|-------|---------|------------|-------------------------------------------------------------------------------------------------------------------------------|
| 0 | `kind` | `uint8_t` | the type, numbered as [`value_t`](../api/basic_json/value_t.md): 0 null, 1 object, 2 array, 3 string, 4 boolean, 5 signed integer, 6 unsigned integer, 7 float |
| 1 | `flags` | `uint8_t` | bits 0-1: where a string's bytes are (0: the source text, 1: the buffer of decoded strings, for strings with escapes); bit 2: the value of a boolean |
| 2-3 | `extra` | `uint16_t` | numbers: the number of integer digits (low byte) and fraction digits (high byte), 255 for more; otherwise 0 |
| 2-3 | `extra` | `uint16_t` | numbers: the number of integer digits (low byte) and fraction digits (high byte), 255 for more; objects: the number of their key hash index (1-based), or 0; otherwise 0 |
| 4-7 | `off` | `uint32_t` | where the value starts: the first byte after a string's opening quote (or its position in the buffer of decoded strings), the first byte of a number or literal, the bracket of an array or object |
| 8-11 | `len` | `uint32_t` | strings: the length after decoding; floats and literals: the length of the token; arrays and objects: the number of elements |
| 12-15 | `next` | `uint32_t` | arrays and objects: the number of nodes of the subtree, including the node itself |
@@ -212,6 +212,11 @@ packet-beta
step from element to element this way and skip whole subtrees in constant time.
- **Offsets** are 32 bits wide, so a document is limited to 4294967279 bytes, 4 GiB minus 16 bytes (a margin below
2^32 for positions one scanner step past the end of the text; `out_of_range.416`).
- **Large objects** (128 members or more) get a hash index for their keys after parsing
([`detail/view/object_index.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/view/object_index.hpp)):
an open-addressing table whose slots hold the distance from the object's node to a key's node, so that a lookup does
not compare every key. Of duplicate keys the table leads to the first, as a linear search does. The object's `extra`
holds the number of its table. Only 65,535 tables fit into `extra`; objects beyond them are searched linearly.
For example, `#!json {"a": [1, 2.5]}` becomes five nodes. Each node's elements follow it, and `next` leads from an
array or object past its subtree:
+2
View File
@@ -25,3 +25,5 @@ The class contains a copy of [Hedley](https://nemequ.github.io/hedley/) from Eva
The class contains an adapted version of the Eisel-Lemire algorithm, its table of powers of five, and its digit comparison for long numbers from [fast_float](https://github.com/fastfloat/fast_float) by Daniel Lemire and contributors, which is available under the [MIT License](https://opensource.org/licenses/MIT) (used here), the Apache 2.0 License, and the Boost Software License. Copyright &copy; 2021 The fast_float authors
The view's parser (`<nlohmann/json_view.hpp>`) contains techniques and code adapted from [yyjson](https://github.com/ibireme/yyjson) by YaoYuan, which is licensed under the [MIT License](https://opensource.org/licenses/MIT) (see above): table-driven decoding of `\u` escapes and fixed-offset unrolled checks.
The view's parser (`<nlohmann/json_view.hpp>`) validates non-ASCII strings with the vector UTF-8 check of [simdjson](https://github.com/simdjson/simdjson) by Daniel Lemire, Geoff Langdale, John Keiser, and contributors (its "lookup4" algorithm and tables, after J. Keiser and D. Lemire, "Validating UTF-8 In Less Than One Instruction Per Byte", 2021), which is available under the [MIT License](https://opensource.org/licenses/MIT) (used here) and the Apache 2.0 License. Copyright &copy; 2018-2025 The simdjson authors
+7
View File
@@ -259,6 +259,7 @@ nav:
- 'cend': api/basic_json_view/cend.md
- 'contains': api/basic_json_view/contains.md
- 'count': api/basic_json_view/count.md
- 'dump': api/basic_json_view/dump.md
- 'empty': api/basic_json_view/empty.md
- 'end': api/basic_json_view/end.md
- 'find': api/basic_json_view/find.md
@@ -281,8 +282,12 @@ nav:
- 'is_structured': api/basic_json_view/is_structured.md
- 'items': api/basic_json_view/items.md
- 'materialize': api/basic_json_view/materialize.md
- 'number_format': api/basic_json_view/number_format.md
- 'number_token': api/basic_json_view/number_token.md
- 'operator<<': api/basic_json_view/operator_ltlt.md
- 'operator[]': api/basic_json_view/operator[].md
- 'operator==': api/basic_json_view/operator_eq.md
- 'operator!=': api/basic_json_view/operator_ne.md
- 'size': api/basic_json_view/size.md
- 'source_offset': api/basic_json_view/source_offset.md
- 'type': api/basic_json_view/type.md
@@ -377,6 +382,8 @@ nav:
- 'JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON': api/macros/json_use_legacy_discarded_value_comparison.md
- 'JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS': api/macros/json_use_objects_for_enum_keyed_maps.md
- 'JSON_USE_SIMDUTF': api/macros/json_use_simdutf.md
- 'JSON_VIEW_NO_SIMD': api/macros/json_view_no_simd.md
- 'JSON_VIEW_USE_SSSE3': api/macros/json_view_use_ssse3.md
- 'NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE, NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE_WITH_DEFAULT, NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE_ONLY_SERIALIZE, NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE, NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_WITH_DEFAULT, NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_ONLY_SERIALIZE': api/macros/nlohmann_define_derived_type.md
- 'NLOHMANN_DEFINE_TYPE_INTRUSIVE, NLOHMANN_DEFINE_TYPE_INTRUSIVE_WITH_DEFAULT, NLOHMANN_DEFINE_TYPE_INTRUSIVE_ONLY_SERIALIZE': api/macros/nlohmann_define_type_intrusive.md
- 'NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE, NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE_WITH_DEFAULT, NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE_ONLY_SERIALIZE': api/macros/nlohmann_define_type_non_intrusive.md
+18 -4
View File
@@ -102,14 +102,26 @@
#define NLOHMANN_JSON_ABI_TAG_OBJECTS_FOR_ENUM_KEYED_MAPS
#endif
#ifdef JSON_VIEW_NO_SIMD
#define NLOHMANN_JSON_ABI_TAG_VIEW_NO_SIMD _vns
#else
#define NLOHMANN_JSON_ABI_TAG_VIEW_NO_SIMD
#endif
#ifdef JSON_VIEW_USE_SSSE3
#define NLOHMANN_JSON_ABI_TAG_VIEW_USE_SSSE3 _vssse3
#else
#define NLOHMANN_JSON_ABI_TAG_VIEW_USE_SSSE3
#endif
#ifndef NLOHMANN_JSON_NAMESPACE_NO_VERSION
#define NLOHMANN_JSON_NAMESPACE_NO_VERSION 0
#endif
// Construct the namespace ABI tags component
#define NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f, g, h) json_abi ## a ## b ## c ## d ## e ## f ## g ## h
#define NLOHMANN_JSON_ABI_TAGS_CONCAT(a, b, c, d, e, f, g, h) \
NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f, g, h)
#define NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f, g, h, i, j) json_abi ## a ## b ## c ## d ## e ## f ## g ## h ## i ## j
#define NLOHMANN_JSON_ABI_TAGS_CONCAT(a, b, c, d, e, f, g, h, i, j) \
NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f, g, h, i, j)
#define NLOHMANN_JSON_ABI_TAGS \
NLOHMANN_JSON_ABI_TAGS_CONCAT( \
@@ -120,7 +132,9 @@
NLOHMANN_JSON_ABI_TAG_PRECISE_STREAM_POSITION, \
NLOHMANN_JSON_ABI_TAG_STRICT_NUL_HANDLING, \
NLOHMANN_JSON_ABI_TAG_STRICT_BINARY_UTF8, \
NLOHMANN_JSON_ABI_TAG_OBJECTS_FOR_ENUM_KEYED_MAPS)
NLOHMANN_JSON_ABI_TAG_OBJECTS_FOR_ENUM_KEYED_MAPS, \
NLOHMANN_JSON_ABI_TAG_VIEW_NO_SIMD, \
NLOHMANN_JSON_ABI_TAG_VIEW_USE_SSSE3)
// Construct the namespace version component
#define NLOHMANN_JSON_NAMESPACE_VERSION_CONCAT_EX(major, minor, patch) \
+39 -16
View File
@@ -126,6 +126,13 @@ class builder
frame_stack m_shallow;
std::vector<frame> m_deep{};
/// remember an object to index after parsing (out of line, so that the
/// parse loop only has a call for it)
NLOHMANN_VIEW_NOINLINE void note_large_object(std::uint32_t idx)
{
m_doc.large_objects.push_back(idx);
}
NLOHMANN_VIEW_NOINLINE bool fail(error_code c, const unsigned char* at) noexcept
{
m_failure.code = c;
@@ -539,7 +546,7 @@ class builder
switch (cur()) \
{ \
case '"': \
if (NLOHMANN_VIEW_UNLIKELY(!string())) { return false; } \
if (NLOHMANN_VIEW_UNLIKELY(!string<true>())) { return false; } \
break; \
case '{': \
open(value_t::object); \
@@ -623,7 +630,7 @@ obj_key:
{
return fail(error_code::expected_key);
}
if (NLOHMANN_VIEW_UNLIKELY(!string()))
if (NLOHMANN_VIEW_UNLIKELY(!string<false>()))
{
return false;
}
@@ -665,17 +672,25 @@ obj_next:
if (enabled(TrailingCommas) && cur() == '}')
{
++p;
goto close_container;
goto close_object;
}
goto obj_key;
}
if (cur() == '}')
{
++p;
goto close_container;
goto close_object;
}
return fail(error_code::expected_object_end);
close_object:
// a large object gets a hash index for its keys (objects only, so that closing
// an array pays nothing for this)
if (NLOHMANN_VIEW_UNLIKELY(cur_count >= document_data::index_min_members))
{
cold.note_large_object(cur_idx);
}
close_container:
close();
if (NLOHMANN_VIEW_UNLIKELY(depth == 0))
@@ -728,7 +743,7 @@ root_done:
switch (cur())
{
case '"':
return string();
return string<true>();
case 't':
return literal_true();
case 'f':
@@ -867,14 +882,19 @@ indent_done:
const auto idx = static_cast<std::uint32_t>(n - base);
if (depth != 0)
{
const frame f = {cur_idx, cur_count, cur_is_object};
if (NLOHMANN_VIEW_LIKELY(depth <= 64))
{
cold.m_shallow[depth - 1] = f;
// field by field: a frame put together on the stack and
// copied would be read back wider than it was written,
// and that load waits until the stores are done
frame& f = cold.m_shallow[depth - 1];
f.idx = cur_idx;
f.count = cur_count;
f.is_object = cur_is_object;
}
else
{
cold.m_deep.push_back(f);
cold.m_deep.push_back(frame{cur_idx, cur_count, cur_is_object});
}
}
++depth;
@@ -891,19 +911,21 @@ indent_done:
n.next = static_cast<std::uint32_t>(out - base) - cur_idx;
if (--depth != 0)
{
frame f{};
if (NLOHMANN_VIEW_LIKELY(depth <= 64))
{
f = cold.m_shallow[depth - 1];
const frame& f = cold.m_shallow[depth - 1];
cur_idx = f.idx;
cur_count = f.count;
cur_is_object = f.is_object;
}
else
{
f = cold.m_deep.back();
const frame f = cold.m_deep.back();
cold.m_deep.pop_back();
cur_idx = f.idx;
cur_count = f.count;
cur_is_object = f.is_object;
}
cur_idx = f.idx;
cur_count = f.count;
cur_is_object = f.is_object;
}
}
@@ -1028,12 +1050,13 @@ indent_done:
return true;
}
/// a string (value or key) at p
/// a string at p: a value (Value) or a key
template<bool Value>
NLOHMANN_VIEW_ALWAYS_INLINE bool string()
{
++p; // opening quote
const unsigned char* const s = p;
p = scan_string_run(p, e);
p = scan_string_run<Value>(p, e);
if (NLOHMANN_VIEW_LIKELY(p != e && *p == '"'))
{
push_node(value_t::string, 0, 0, static_cast<std::size_t>(s - b), static_cast<std::uint64_t>(p - s));
+320
View File
@@ -0,0 +1,320 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#pragma once
#include <algorithm> // sort, stable_sort
#include <cstddef> // size_t
#include <string> // string
#include <utility> // move, pair
#include <vector> // vector
#include <nlohmann/json.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
{
namespace view
{
// Equality of views, and of views with basic_json values, with the semantics
// of basic_json's operator== applied to the values parse() would produce:
// numbers compare by value across their types, an object is compared by its
// members with duplicate keys resolved as parse() resolves them (the last
// value, at the position of the first occurrence), and in document order if
// the object type keeps an order (ordered_json), by key otherwise.
/// one side of a comparison: a view
template<typename BasicJsonType, typename View>
class view_side
{
public:
using string_view_t = typename View::string_view_t;
explicit view_side(const View& v) noexcept
: m_view(v)
{}
value_t type() const noexcept
{
return m_view.type();
}
std::size_t size() const noexcept
{
return m_view.size();
}
string_view_t string() const
{
return m_view.get_string();
}
/// a number, boolean, or null as a basic_json value (no allocation)
BasicJsonType scalar() const
{
switch (m_view.type())
{
case value_t::number_integer:
return BasicJsonType(m_view.template get<typename BasicJsonType::number_integer_t>());
case value_t::number_unsigned:
return BasicJsonType(m_view.template get<typename BasicJsonType::number_unsigned_t>());
case value_t::number_float:
return BasicJsonType(m_view.template get<typename BasicJsonType::number_float_t>());
case value_t::boolean:
return BasicJsonType(m_view.template get<bool>());
case value_t::null:
case value_t::object:
case value_t::array:
case value_t::string:
case value_t::binary:
case value_t::discarded:
default:
return BasicJsonType(nullptr);
}
}
void elements(std::vector<view_side>& out) const
{
out.reserve(m_view.size());
for (const View e : m_view)
{
out.emplace_back(e);
}
}
/// the members as parse() keeps them: one per key, the last value at the
/// position of the first occurrence; in that order, or sorted by key
void members(std::vector<std::pair<string_view_t, view_side>>& out, bool ordered) const
{
struct member
{
string_view_t key;
View value;
std::size_t position;
};
std::vector<member> all;
all.reserve(m_view.size());
std::size_t position = 0;
for (auto it = m_view.begin(); it != m_view.end(); ++it)
{
all.push_back(member{it.key(), it.value(), position++});
}
std::stable_sort(all.begin(), all.end(), [](const member & a, const member & b)
{
return a.key < b.key;
});
std::vector<member> unique;
unique.reserve(all.size());
for (std::size_t i = 0; i < all.size();)
{
std::size_t last = i;
while (last + 1 < all.size() && all[last + 1].key == all[i].key)
{
++last;
}
unique.push_back(member{all[i].key, all[last].value, all[i].position});
i = last + 1;
}
if (ordered)
{
std::sort(unique.begin(), unique.end(), [](const member & a, const member & b)
{
return a.position < b.position;
});
}
out.reserve(unique.size());
for (const member& m : unique)
{
out.emplace_back(m.key, view_side(m.value));
}
}
private:
View m_view;
};
/// the other side of a comparison: a basic_json value
template<typename BasicJsonType, typename StringView>
class json_side
{
public:
using string_view_t = StringView;
explicit json_side(const BasicJsonType& j) noexcept
: m_json(&j)
{}
value_t type() const noexcept
{
return m_json->type();
}
std::size_t size() const noexcept
{
return m_json->size();
}
string_view_t string() const
{
const auto& s = m_json->template get_ref<const typename BasicJsonType::string_t&>();
return string_view_t(s.data(), s.size());
}
BasicJsonType scalar() const
{
return *m_json;
}
void elements(std::vector<json_side>& out) const
{
out.reserve(m_json->size());
for (const auto& e : *m_json)
{
out.emplace_back(e);
}
}
void members(std::vector<std::pair<string_view_t, json_side>>& out, bool ordered) const
{
out.reserve(m_json->size());
for (auto it = m_json->cbegin(); it != m_json->cend(); ++it)
{
out.emplace_back(string_view_t(it.key().data(), it.key().size()), json_side(it.value()));
}
if (!ordered)
{
std::sort(out.begin(), out.end(), [](const std::pair<string_view_t, json_side>& a, const std::pair<string_view_t, json_side>& b)
{
return a.first < b.first;
});
}
}
private:
const BasicJsonType* m_json;
};
inline bool is_number(value_t t) noexcept
{
return t == value_t::number_integer || t == value_t::number_unsigned || t == value_t::number_float;
}
/// whether two sides are equal; iterative, so that the nesting depth is
/// limited by memory only
template<typename BasicJsonType, typename A, typename B>
bool equal(const A& a0, const B& b0)
{
using string_view_t = typename A::string_view_t;
const bool ordered = is_ordered_map<typename BasicJsonType::object_t>::value;
struct frame
{
std::vector<A> elements_a{};
std::vector<B> elements_b{};
std::vector<std::pair<string_view_t, A>> members_a{};
std::vector<std::pair<string_view_t, B>> members_b{};
bool object = false;
std::size_t next = 0;
};
std::vector<frame> stack;
A a = a0;
B b = b0;
for (;;)
{
const value_t ta = a.type();
const value_t tb = b.type();
if (ta == value_t::discarded || tb == value_t::discarded)
{
// basic_json decides (JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON)
if (ta != tb || !(BasicJsonType(value_t::discarded) == BasicJsonType(value_t::discarded)))
{
return false;
}
}
else
{
if (ta != tb && !(is_number(ta) && is_number(tb))) // numbers of different types can be equal
{
return false;
}
if (ta == value_t::string)
{
if (a.string() != b.string())
{
return false;
}
}
else if (ta == value_t::array || ta == value_t::object)
{
if (a.size() != b.size() && ta == value_t::array)
{
return false;
}
frame f;
f.object = ta == value_t::object;
if (f.object)
{
a.members(f.members_a, ordered);
b.members(f.members_b, ordered);
if (f.members_a.size() != f.members_b.size())
{
return false;
}
}
else
{
a.elements(f.elements_a);
b.elements(f.elements_b);
}
stack.push_back(std::move(f));
}
else if (a.scalar() != b.scalar()) // numbers (also of different types), null, boolean
{
return false;
}
}
// the next pair of values
for (;;)
{
if (stack.empty())
{
return true;
}
frame& f = stack.back();
const std::size_t count = f.object ? f.members_a.size() : f.elements_a.size();
if (f.next == count)
{
stack.pop_back(); // (f dangles now, and is bound again after continue)
continue;
}
if (f.object)
{
if (!(f.members_a[f.next].first == f.members_b[f.next].first))
{
return false;
}
a = f.members_a[f.next].second;
b = f.members_b[f.next].second;
}
else
{
a = f.elements_a[f.next];
b = f.elements_b[f.next];
}
++f.next;
break;
}
}
}
} // namespace view
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
+22 -1
View File
@@ -10,7 +10,8 @@
#include <array> // array
#include <cstddef> // size_t
#include <cstring> // memcpy
#include <cstdint> // uint32_t
#include <cstring> // memcmp, memcpy
#include <limits> // numeric_limits
#include <new> // bad_alloc, operator new, placement new
#include <string> // string
@@ -40,6 +41,17 @@ struct document_data
std::size_t max_depth = 0; ///< deepest nesting of arrays and objects (the root container is depth 1)
std::vector<char> arena; ///< decoded strings that contained escapes (no terminator: they are handed out with their length)
std::string owned; ///< owned copy of the input, if any (a std::string, so that its NUL serves as the parser's sentinel)
// hash indexes of the keys of large objects (see object_index.hpp)
static constexpr std::uint32_t index_min_members = 128;
struct object_index
{
std::size_t start; ///< first slot in index_slots
std::uint32_t mask; ///< slot count - 1 (a power of two minus one)
};
std::vector<object_index> indexes;
std::vector<std::uint32_t> index_slots;
std::vector<std::uint32_t> large_objects; ///< positions of the objects to index (noted while parsing)
std::array<const char*, 4> base = {{nullptr, nullptr, nullptr, nullptr}}; ///< string bases: source, arena (indexed by flags & node_flags::storage)
bool discarded = true;
@@ -82,6 +94,9 @@ struct document_data
document_data() noexcept
: arena() // NOLINT(readability-redundant-member-init)
, owned() // NOLINT(readability-redundant-member-init)
, indexes() // NOLINT(readability-redundant-member-init)
, index_slots() // NOLINT(readability-redundant-member-init)
, large_objects() // NOLINT(readability-redundant-member-init)
{}
document_data(const document_data&) = delete;
document_data(document_data&&) = delete;
@@ -141,6 +156,12 @@ struct document_data
return base[n.flags & node_flags::storage] + n.off;
}
/// whether the string of node k (a key) is the n bytes at key
bool key_equals(const node& k, const char* key, std::size_t n) const noexcept
{
return k.len == n && (n == 0 || std::memcmp(str(k), key, n) == 0);
}
/// the node after n's subtree (containers span `next` nodes, scalars one)
static NLOHMANN_VIEW_ALWAYS_INLINE const node* after(const node* n) noexcept
{
+7 -2
View File
@@ -18,6 +18,7 @@
#include <nlohmann/detail/view/document_data.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
#include <nlohmann/detail/view/node.hpp>
#include <nlohmann/detail/view/object_index.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
@@ -89,6 +90,10 @@ class short_key
/// their length, from the index alone
inline const node* find_member(const document_data& d, const node* object, const char* key, std::size_t n) noexcept
{
if (NLOHMANN_VIEW_UNLIKELY(object->extra != 0))
{
return find_indexed(d, object, key, n); // a large object
}
const node* const end = document_data::child_end(object);
const auto* const k = reinterpret_cast<const unsigned char*>(key); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
if (NLOHMANN_VIEW_LIKELY(n <= 16))
@@ -105,7 +110,7 @@ inline const node* find_member(const document_data& d, const node* object, const
}
for (const node* m = document_data::first_child(object); m != end; m = document_data::after(m + 1))
{
if (m->len == n && std::memcmp(d.str(*m), key, n) == 0)
if (d.key_equals(*m, key, n))
{
return m;
}
@@ -120,7 +125,7 @@ inline std::size_t count_members(const document_data& d, const node* object, con
const node* const end = document_data::child_end(object);
for (const node* m = document_data::first_child(object); m != end; m = document_data::after(m + 1))
{
if (m->len == n && (n == 0 || std::memcmp(d.str(*m), key, n) == 0))
if (d.key_equals(*m, key, n))
{
++result;
}
+7 -1
View File
@@ -70,5 +70,11 @@
#define NLOHMANN_VIEW_LITTLE_ENDIAN 0
#endif
/// eight checks at fixed offsets 0..7
#define NLOHMANN_VIEW_REPEAT_FIRST_8(X) X(0) X(1) X(2) X(3) X(4) X(5) X(6) X(7)
/// eight checks at fixed offsets 8..15
#define NLOHMANN_VIEW_REPEAT_SECOND_8(X) X(8) X(9) X(10) X(11) X(12) X(13) X(14) X(15)
/// sixteen checks at fixed offsets 0..15
#define NLOHMANN_VIEW_REPEAT16(X) X(0) X(1) X(2) X(3) X(4) X(5) X(6) X(7) X(8) X(9) X(10) X(11) X(12) X(13) X(14) X(15)
#define NLOHMANN_VIEW_REPEAT16(X) NLOHMANN_VIEW_REPEAT_FIRST_8(X) NLOHMANN_VIEW_REPEAT_SECOND_8(X)
@@ -19,3 +19,12 @@
#undef NLOHMANN_VIEW_THROW
#undef NLOHMANN_VIEW_LITTLE_ENDIAN
#undef NLOHMANN_VIEW_REPEAT16
#undef NLOHMANN_VIEW_REPEAT_SECOND_8
#undef NLOHMANN_VIEW_REPEAT_FIRST_8
#undef NLOHMANN_VIEW_NEON
#undef NLOHMANN_VIEW_SSE2
#undef NLOHMANN_VIEW_SSSE3
#undef NLOHMANN_VIEW_SSSE3_DISPATCH
#undef NLOHMANN_VIEW_SSSE3_TARGET
#undef NLOHMANN_VIEW_VECTOR
#undef NLOHMANN_VIEW_VECTOR_UTF8
+1 -1
View File
@@ -94,7 +94,7 @@ BasicJsonType materialize(const document_data& d, const node* n)
++n;
break;
case value_t::number_float:
sax.number_float(float_value<typename BasicJsonType::number_float_t>(d.str(*n), *n), no_token);
sax.number_float(float_value<typename BasicJsonType::number_float_t>(d, *n), no_token);
++n;
break;
case value_t::boolean:
+1 -1
View File
@@ -45,7 +45,7 @@ struct node
{
std::uint8_t kind; ///< value_t
std::uint8_t flags; ///< node_flags
std::uint16_t extra; ///< numbers: integer digits (low byte) and fraction digits (high byte), 255 = "many"; otherwise 0
std::uint16_t extra; ///< numbers: integer digits (low byte) and fraction digits (high byte), 255 = "many"; objects: number of the object's key hash index (1-based, 0 = none); otherwise 0
std::uint32_t off; ///< source offset (string content, number token, literal, bracket); arena offset if node_flags::escaped
std::uint32_t len; ///< string: decoded bytes; float: token bytes; array/object: element count
std::uint32_t next; ///< array/object: number of nodes of the subtree (its extent in the enclosing sequence)
+94
View File
@@ -9,11 +9,15 @@
#pragma once
#include <cstddef> // size_t
#include <cstdint> // int64_t, uint64_t
#include <string> // string
#include <type_traits> // integral_constant
#include <nlohmann/json.hpp>
#include <nlohmann/detail/view/document_data.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
#include <nlohmann/detail/view/node.hpp>
#include <nlohmann/detail/view/scan.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
@@ -62,6 +66,96 @@ NLOHMANN_VIEW_NOINLINE FloatType float_value(const char* first, const node& n)
return convert_float<FloatType>(first, last, dot, mantissa_end);
}
/*!
@brief the digits of a float token with at most 19 digits, from its layout
The digit layout recorded while parsing says where the integer digits, the
fraction digits, and the exponent are, so the digits are read eight at a
time without scanning.
@param[in] p first character of the token
@param[in] e end of the token
@param[in] limit end of the readable memory (the source text)
*/
NLOHMANN_VIEW_ALWAYS_INLINE float_significand layout_decimal(const unsigned char* p, const unsigned char* e, unsigned int_digits, unsigned frac_digits, const unsigned char* limit) noexcept
{
const bool negative = *p == '-';
p += negative ? 1 : 0;
std::uint64_t w = parse_upto19(p, int_digits, limit);
p += int_digits;
std::int64_t q = 0;
if (frac_digits != 0)
{
w = (w * int_pow10(frac_digits)) + parse_upto19(p + 1, frac_digits, limit);
p += 1 + frac_digits;
q = -static_cast<std::int64_t>(frac_digits);
}
if (p != e)
{
// [eE][+-]digits; huge exponents saturate (the parser rejected overflow)
++p;
const bool exp_negative = *p == '-';
p += (*p == '-' || *p == '+') ? 1 : 0;
std::int64_t exp_value = 0;
for (; p != e; ++p)
{
if (exp_value < 0x10000000)
{
exp_value = (exp_value * 10) + (*p - '0');
}
}
q += exp_negative ? -exp_value : exp_value;
}
float_significand d;
d.w = w;
d.exponent = q;
d.negative = negative;
return d;
}
/*!
@brief the value of a float token with at most 19 digits, from its layout
The result is correctly rounded by the lexer's conversion
(detail::decimal_to_float(): Clinger's fast path where both operands are
exact, else the Eisel-Lemire algorithm, which needs no fallback for up to 19
digits), so it is the value parse() produces.
*/
template<typename FloatType>
NLOHMANN_VIEW_ALWAYS_INLINE FloatType layout_float(const unsigned char* p, const unsigned char* e, unsigned int_digits, unsigned frac_digits, const unsigned char* limit) noexcept
{
return decimal_to_float<FloatType>(layout_decimal(p, e, int_digits, frac_digits, limit));
}
/// the value of the float token of a node, as parse() converts it; floats and
/// doubles with at most 19 digits are converted from the digit layout
template<typename FloatType>
FloatType float_value(const document_data& d, const node& n)
{
return float_value<FloatType>(d, n, std::integral_constant<bool, has_native_float_format<FloatType>::value> {});
}
template<typename FloatType>
FloatType float_value(const document_data& d, const node& n, std::true_type /*binary32 or binary64*/)
{
const unsigned int_digits = n.extra & 0xFFu;
const unsigned frac_digits = n.extra >> 8u;
if (NLOHMANN_VIEW_LIKELY(int_digits + frac_digits <= 19)) // (255 marks "many")
{
// (a float token not written by an edit is in the text)
const auto* const first = reinterpret_cast<const unsigned char*>(d.src + n.off); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
return layout_float<FloatType>(first, first + n.len, int_digits, frac_digits, reinterpret_cast<const unsigned char*>(d.src + d.size)); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
}
return float_value<FloatType>(d.str(n), n);
}
template<typename FloatType>
FloatType float_value(const document_data& d, const node& n, std::false_type /*other*/)
{
return float_value<FloatType>(d.str(n), n);
}
} // namespace view
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
@@ -0,0 +1,154 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#pragma once
#include <cstddef> // size_t
#include <cstdint> // uint32_t, uint64_t
#include <nlohmann/json.hpp>
#include <nlohmann/detail/view/document_data.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
#include <nlohmann/detail/view/node.hpp>
// Hash indexes of the keys of large objects, so that a lookup does not compare thousands
// of keys (as Boost.JSON switches from a linear search to a hash table for
// large objects). An object with document_data::index_min_members members or
// more gets an open-addressing table after parsing; its node stores the
// number of the table (1-based) in `extra`. A slot holds the offset of a key
// node from its object node (0: empty). Of duplicate keys, the first is kept,
// as for the linear search.
//
// The hash is not seeded, so keys chosen to collide could make the build
// quadratic. A key therefore sits at most index_max_displacement slots away
// from its home slot; if a key would sit further away, the table is dropped
// and the object is searched linearly (like a small one). For the same
// reason, a lookup visits at most index_max_displacement + 1 slots.
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
{
namespace view
{
/// the farthest a key may sit from its home slot (a table with at most half of
/// its slots in use gives random keys a distance of about 50 for millions of
/// members; and every member costs at most this many steps while building)
constexpr std::size_t index_max_displacement = 64;
// a table has at least 2 * len slots (len >= index_min_members), so the probe
// window of index_max_displacement + 1 slots stays far smaller than the table
// and a probe never wraps around onto its own start
static_assert(document_data::index_min_members >= 2 * index_max_displacement,
"a table must be much larger than the probe window of a lookup");
/// hash of a key: its bytes, eight at a time, in a fixed byte order
inline std::uint64_t key_hash(const char* s, std::size_t n) noexcept
{
std::uint64_t h = 0x9E3779B97F4A7C15u * (n + 1);
const auto* p = reinterpret_cast<const unsigned char*>(s); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
while (n >= 8)
{
h = (h ^ read_eight_bytes(p)) * 0xBF58476D1CE4E5B9u;
h ^= h >> 29u;
p += 8;
n -= 8;
}
std::uint64_t w = 0;
for (std::size_t i = 0; i < n; ++i)
{
w |= static_cast<std::uint64_t>(p[i]) << (8u * i);
}
h = (h ^ w) * 0x94D049BB133111EBu;
return h ^ (h >> 31u);
}
/// build the table of a large object
inline void build_object_index(document_data& d, node* obj)
{
if (d.indexes.size() >= 0xFFFFu)
{
return; // LCOV_EXCL_LINE (the number must fit `extra`; more large objects are searched linearly)
}
std::size_t cap = 16;
while (cap < 2 * static_cast<std::size_t>(obj->len))
{
cap *= 2;
}
const std::size_t start = d.index_slots.size();
d.index_slots.resize(start + cap, 0);
std::uint32_t* const table = d.index_slots.data() + start;
const std::size_t mask = cap - 1;
for (const node* k = document_data::first_child(obj), *end = document_data::child_end(obj); k != end; k = document_data::after(k + 1))
{
const char* const key = d.str(*k);
std::size_t slot = conditional_static_cast<std::size_t>(key_hash(key, k->len)) & mask;
bool duplicate = false;
std::size_t distance = 0;
while (table[slot] != 0)
{
const node* const other = obj + table[slot];
if (d.key_equals(*other, key, k->len))
{
duplicate = true; // keep the first
break;
}
if (distance == index_max_displacement)
{
// too many keys share a home region
d.index_slots.resize(start); // no table: the object is searched linearly
return;
}
++distance;
slot = (slot + 1) & mask;
}
if (!duplicate)
{
table[slot] = static_cast<std::uint32_t>(k - obj);
}
}
d.indexes.push_back(document_data::object_index{start, static_cast<std::uint32_t>(mask)});
obj->extra = static_cast<std::uint16_t>(d.indexes.size());
}
/// build the tables of the large objects the parser noted
inline void build_object_indexes(document_data& d)
{
for (const std::uint32_t i : d.large_objects)
{
build_object_index(d, d.nodes + i);
}
}
/// the key node of the first member with this key of an indexed object, or
/// nullptr
inline const node* find_indexed(const document_data& d, const node* obj, const char* key, std::size_t n) noexcept
{
const document_data::object_index& ix = d.indexes[obj->extra - 1u];
const std::uint32_t* const table = d.index_slots.data() + ix.start;
std::size_t slot = conditional_static_cast<std::size_t>(key_hash(key, n)) & ix.mask;
for (std::size_t distance = 0; distance <= index_max_displacement; ++distance)
{
const std::uint32_t s = table[slot];
if (s == 0)
{
return nullptr;
}
const node* const k = obj + s;
if (d.key_equals(*k, key, n))
{
return k;
}
slot = (slot + 1) & ix.mask;
}
return nullptr; // (no key sits further from its home slot)
}
} // namespace view
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
+44 -5
View File
@@ -16,6 +16,7 @@
#include <nlohmann/json.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
#include <nlohmann/detail/view/simd.hpp>
// Scanning primitives of the view's parser. The unrolled checks at fixed
// offsets follow yyjson (https://github.com/ibireme/yyjson, MIT license): the
@@ -62,20 +63,43 @@ NLOHMANN_VIEW_ALWAYS_INLINE std::uint16_t load16(const unsigned char* p) noexcep
/// Advance over plain string bytes and well-formed UTF-8. Stops at a quote,
/// a backslash, a control character, ill-formed UTF-8, or the end. The first
/// 16 bytes are checked one by one, so that the position advances by
/// constants in predicted branches (most strings are short); longer runs
/// continue eight bytes at a time.
/// bytes are checked one by one, so that the position advances by constants
/// in predicted branches: 16 for keys, whose lengths repeat from record to
/// record, and 8 for string values (Value) where a vector loop follows, as
/// their lengths vary more. Longer runs continue 16 bytes at a time with NEON
/// or SSE2, else eight bytes at a time. With SSE2, the run is checked 16 bytes
/// at a time from its first byte instead: on x86-64, one compare that finds
/// the end of most keys and short values is faster than a branch per byte (on
/// AArch64, where a NEON mask costs more and branches predict well, slower).
template<bool Value = false>
NLOHMANN_VIEW_ALWAYS_INLINE const unsigned char* scan_string_run(const unsigned char* p, const unsigned char* e) noexcept
{
const std::uint8_t* plain = string_plain();
for (;;)
{
#if NLOHMANN_VIEW_SSE2
p = vector_plain_run(p, e);
#else
if (e - p >= 16)
{
#define NLOHMANN_VIEW_STEP(i) if (NLOHMANN_VIEW_LIKELY(plain[p[i]] != 0)) {} else { p += (i); goto stop; }
NLOHMANN_VIEW_REPEAT16(NLOHMANN_VIEW_STEP)
NLOHMANN_VIEW_REPEAT_FIRST_8(NLOHMANN_VIEW_STEP)
#if NLOHMANN_VIEW_VECTOR
if (!Value) // (a plain `!Value || !NLOHMANN_VIEW_VECTOR` is a constant condition for MSVC, C4127)
#endif
{
NLOHMANN_VIEW_REPEAT_SECOND_8(NLOHMANN_VIEW_STEP)
p += 8;
}
#undef NLOHMANN_VIEW_STEP
p += 16;
p += 8;
#if NLOHMANN_VIEW_VECTOR
p = vector_plain_run(p, e);
if (p != e && plain[*p] == 0)
{
goto stop;
}
#else
while (e - p >= 8)
{
const std::uint64_t special = swar_string_special(read_eight_bytes(p));
@@ -86,8 +110,10 @@ NLOHMANN_VIEW_ALWAYS_INLINE const unsigned char* scan_string_run(const unsigned
}
p += 8;
}
#endif
continue;
}
#endif
while (p != e && plain[*p] != 0)
{
++p;
@@ -96,11 +122,23 @@ NLOHMANN_VIEW_ALWAYS_INLINE const unsigned char* scan_string_run(const unsigned
{
return p;
}
#if !NLOHMANN_VIEW_SSE2
stop:
#endif
if (*p < 0x80)
{
return p; // quote, backslash, or control character
}
#if NLOHMANN_VIEW_VECTOR_UTF8
#if NLOHMANN_VIEW_SSSE3_DISPATCH
if (NLOHMANN_VIEW_LIKELY(cpu_has_ssse3()))
#endif
{
// non-ASCII: the vector check, out of line
return scan_string_vector(p, e, plain);
}
#endif
#if !NLOHMANN_VIEW_VECTOR_UTF8 || NLOHMANN_VIEW_SSSE3_DISPATCH
// non-ASCII: a run of well-formed sequences (the library's check, so
// that exactly what json::parse accepts is accepted)
do
@@ -113,6 +151,7 @@ stop:
p += n;
}
while (p != e && *p >= 0x80);
#endif
}
}
+429
View File
@@ -0,0 +1,429 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#pragma once
#include <algorithm> // max
#include <array> // array
#include <cmath> // isfinite
#include <cstddef> // size_t
#include <cstdint> // uint8_t, uint32_t
#include <cstring> // memcpy, memset
#include <limits> // numeric_limits
#include <type_traits> // integral_constant
#include <vector> // vector
#include <nlohmann/json.hpp>
#include <nlohmann/detail/view/document_data.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
#include <nlohmann/detail/view/node.hpp>
#include <nlohmann/detail/view/number.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
{
namespace view
{
/// append-only output buffer: writes through a raw pointer into a string that
/// is resized ahead, and trimmed by finish()
template<typename StringType>
class output_buffer
{
public:
output_buffer(StringType& out, std::size_t estimate)
: m_out(sized(out, estimate))
, m_pos(&m_out[0])
, m_end(m_pos + m_out.size())
{}
void finish()
{
const auto size = static_cast<std::size_t>(m_pos - m_out.data());
m_out.resize(size);
}
NLOHMANN_VIEW_ALWAYS_INLINE void reserve(std::size_t n)
{
if (NLOHMANN_VIEW_UNLIKELY(static_cast<std::size_t>(m_end - m_pos) < n))
{
grow(n);
}
}
NLOHMANN_VIEW_ALWAYS_INLINE void put(char c)
{
reserve(1);
*m_pos++ = c;
}
NLOHMANN_VIEW_ALWAYS_INLINE void put(const char* s, std::size_t n)
{
reserve(n);
std::memcpy(m_pos, s, n);
m_pos += n;
}
/// a string literal, without its terminating NUL
template<std::size_t N>
NLOHMANN_VIEW_ALWAYS_INLINE void put(const char (&s)[N]) // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
{
put(s, N - 1);
}
void put_repeated(char c, std::size_t n)
{
reserve(n);
std::memset(m_pos, c, n);
m_pos += n;
}
private:
static StringType& sized(StringType& out, std::size_t estimate)
{
out.resize((std::max)(estimate, static_cast<std::size_t>(64)));
return out;
}
NLOHMANN_VIEW_NOINLINE void grow(std::size_t n)
{
const auto used = static_cast<std::size_t>(m_pos - m_out.data());
m_out.resize((std::max)(m_out.size() * 2, used + n + 256));
m_pos = &m_out[0] + used;
m_end = &m_out[0] + m_out.size();
}
StringType& m_out;
char* m_pos;
char* m_end;
};
/// how the view's dump() writes a value
struct dump_style
{
bool pretty = false; ///< indent >= 0
std::size_t indent = 0; ///< characters per level
char indent_char = ' ';
bool ensure_ascii = false;
bool source_numbers = false; ///< copy number tokens from the source
};
/*!
@brief write a view's subtree as basic_json::dump() writes the value
The output of a subtree equals ordered_json::parse(text).dump() of it for
the same arguments (members in document order): strings are escaped by the
same rules, with the library's scanning kernels; floats are written with
the library's conversion; integers are copied from the source, where they
are canonical (except "-0", which parse() reads as 0). The walk is
iterative, so the nesting depth is limited by memory only.
*/
template<typename BasicJsonType>
class view_serializer
{
using string_t = typename BasicJsonType::string_t;
using number_float_t = typename BasicJsonType::number_float_t;
public:
view_serializer(const document_data& d, string_t& out, std::size_t estimate, const dump_style& style)
: m_doc(d), m_out(out, estimate), m_style(style)
{}
void dump(const node* root)
{
struct frame
{
const node* pos; ///< next element, or key of the next member
const node* end;
bool object;
bool first; ///< nothing written yet
};
std::vector<frame> stack;
const node* n = root;
for (;;)
{
// write the value at n
if (is_container(*n))
{
const bool object = n->kind == static_cast<std::uint8_t>(value_t::object);
if (n->len == 0)
{
m_out.put(object ? "{}" : "[]", 2);
}
else
{
m_out.put(object ? '{' : '[');
stack.push_back(frame{document_data::first_child(n), document_data::child_end(n), object, true});
}
}
else
{
write_scalar(*n);
}
// go to the next value: close finished containers, then separate
for (;;)
{
if (stack.empty())
{
m_out.finish();
return;
}
frame& f = stack.back();
if (f.pos == f.end)
{
const bool object = f.object; // (f dangles after pop_back)
stack.pop_back();
newline(stack.size());
m_out.put(object ? '}' : ']');
continue;
}
if (!f.first)
{
m_out.put(',');
}
f.first = false;
newline(stack.size());
if (f.object)
{
write_string(*f.pos);
if (m_style.pretty)
{
m_out.put(": ");
}
else
{
m_out.put(':');
}
n = f.pos + 1;
}
else
{
n = f.pos;
}
f.pos = document_data::after(n);
break;
}
}
}
private:
void newline(std::size_t level)
{
if (m_style.pretty)
{
m_out.put('\n');
m_out.put_repeated(m_style.indent_char, level * m_style.indent);
}
}
void write_scalar(const node& n)
{
switch (static_cast<value_t>(n.kind))
{
case value_t::null:
m_out.put("null");
break;
case value_t::boolean:
if ((n.flags & node_flags::is_true) != 0)
{
m_out.put("true");
}
else
{
m_out.put("false");
}
break;
case value_t::string:
write_string(n);
break;
case value_t::number_integer:
case value_t::number_unsigned:
{
const char* const token = m_doc.str(n);
const std::uint32_t len = number_length(n);
if (!m_style.source_numbers && len == 2 && token[0] == '-' && token[1] == '0')
{
m_out.put('0'); // parse() reads -0 as the integer 0
}
else
{
m_out.put(token, len);
}
break;
}
case value_t::number_float:
if (m_style.source_numbers)
{
m_out.put(m_doc.str(n), n.len);
}
else
{
write_float(float_value<number_float_t>(m_doc, n));
}
break;
case value_t::object: // LCOV_EXCL_LINE (containers are written by dump())
case value_t::array: // LCOV_EXCL_LINE
case value_t::binary: // LCOV_EXCL_LINE (not in a document)
case value_t::discarded: // LCOV_EXCL_LINE
default: // LCOV_EXCL_LINE
break; // LCOV_EXCL_LINE
}
}
/// as serializer::dump_float()
void write_float(number_float_t x)
{
if (!std::isfinite(x))
{
m_out.put("null");
return;
}
static constexpr bool is_ieee_single_or_double
= std::numeric_limits<number_float_t>::is_iec559
&& ((std::numeric_limits<number_float_t>::digits == 24 && std::numeric_limits<number_float_t>::max_exponent == 128)
|| (std::numeric_limits<number_float_t>::digits == 53 && std::numeric_limits<number_float_t>::max_exponent == 1024));
write_float(x, std::integral_constant<bool, is_ieee_single_or_double>());
}
void write_float(number_float_t x, std::true_type /*is_ieee_single_or_double*/)
{
std::array<char, 64> buf{};
const char* const end = ::nlohmann::detail::to_chars(buf.data(), buf.data() + buf.size(), x);
m_out.put(buf.data(), static_cast<std::size_t>(end - buf.data()));
}
void write_float(number_float_t x, std::false_type /*is_ieee_single_or_double*/)
{
// other types (e.g. long double) are rare: the library writes them
const string_t s = BasicJsonType(x).dump();
m_out.put(s.data(), s.size());
}
void write_string(const node& n)
{
const char* const s = m_doc.str(n);
m_out.put('"');
if ((n.flags & node_flags::escaped) == 0 && !m_style.ensure_ascii)
{
// a string without escape sequences has nothing to escape
m_out.put(s, n.len);
}
else if (m_style.ensure_ascii)
{
write_escaped<true>(reinterpret_cast<const unsigned char*>(s), n.len); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
}
else
{
write_escaped<false>(reinterpret_cast<const unsigned char*>(s), n.len); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
}
m_out.put('"');
}
/// as serializer::dump_escaped() for valid UTF-8 (the view has no other)
template<bool EnsureAscii>
void write_escaped(const unsigned char* s, std::size_t n)
{
std::size_t i = 0;
while (i < n)
{
std::size_t run = 0;
if (!EnsureAscii)
{
run = string_bulk_run(s + i, n - i);
}
else if (is_ascii_copyable(s[i]))
{
run = find_ascii_copyable_run(s + i, n - i);
}
if (run != 0)
{
m_out.put(reinterpret_cast<const char*>(s + i), run); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
i += run;
continue;
}
std::uint32_t codepoint = s[i];
std::size_t len = 1;
if (codepoint >= 0xC0)
{
len = 2;
if (codepoint >= 0xE0)
{
len = codepoint >= 0xF0 ? 4 : 3;
}
codepoint &= 0xFFu >> (len + 1);
for (std::size_t k = 1; k < len; ++k)
{
codepoint = (codepoint << 6u) | (s[i + k] & 0x3Fu);
}
}
write_codepoint<EnsureAscii>(codepoint, s + i, len);
i += len;
}
}
template<bool EnsureAscii>
void write_codepoint(std::uint32_t codepoint, const unsigned char* bytes, std::size_t len)
{
switch (codepoint)
{
case 0x08:
m_out.put("\\b");
return;
case 0x09:
m_out.put("\\t");
return;
case 0x0A:
m_out.put("\\n");
return;
case 0x0C:
m_out.put("\\f");
return;
case 0x0D:
m_out.put("\\r");
return;
case 0x22:
m_out.put("\\\"");
return;
case 0x5C:
m_out.put("\\\\");
return;
default:
break;
}
if (codepoint <= 0x1F || (EnsureAscii && codepoint >= 0x7F))
{
if (codepoint <= 0xFFFF)
{
write_u_escape(codepoint);
}
else
{
write_u_escape(0xD7C0u + (codepoint >> 10u));
write_u_escape(0xDC00u + (codepoint & 0x3FFu));
}
return;
}
m_out.put(reinterpret_cast<const char*>(bytes), len); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast) LCOV_EXCL_LINE (printable characters are copied in runs)
}
void write_u_escape(std::uint32_t u)
{
static constexpr const char* hex = "0123456789abcdef";
const std::array<char, 6> e = {{'\\', 'u', hex[(u >> 12u) & 0xFu], hex[(u >> 8u) & 0xFu], hex[(u >> 4u) & 0xFu], hex[u & 0xFu]}};
m_out.put(e.data(), e.size());
}
const document_data& m_doc;
output_buffer<string_t> m_out;
const dump_style m_style;
};
} // namespace view
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
+339
View File
@@ -0,0 +1,339 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-FileCopyrightText: 2018-2025 The simdjson authors <https://github.com/simdjson/simdjson>
// SPDX-License-Identifier: MIT
#pragma once
#include <array> // array
#include <atomic> // atomic
#include <cstddef> // size_t
#include <cstdint> // uint8_t, uint64_t
#include <nlohmann/json.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
// Vector code for long runs of string bytes. NEON (AArch64) and SSE2 (x86-64)
// belong to the baseline instruction sets and are used by default. The vector
// UTF-8 check needs NEON or SSSE3. SSSE3 is not part of x86-64, and the code
// must not depend on the flags of a translation unit (two translation units
// with different flags would have different definitions of the same inline
// functions): the check is compiled for SSSE3 with a function attribute and
// used where the CPU has SSSE3 (all x86-64 CPUs since about 2011), else the
// portable check. JSON_VIEW_USE_SSSE3 skips the CPU check (for code compiled
// for SSSE3 anyway); JSON_VIEW_NO_SIMD selects the portable code.
// NLOHMANN_VIEW_NO_TARGET_ATTRIBUTE (set by the GCC module interface, where GCC
// ignores the target attribute and then rejects the SSSE3 intrinsics) keeps the
// check portable without disabling the SSE2 code.
#if !defined(JSON_VIEW_NO_SIMD) && defined(__aarch64__) && (defined(__GNUC__) || defined(__clang__)) && NLOHMANN_VIEW_LITTLE_ENDIAN
#include <arm_neon.h>
#define NLOHMANN_VIEW_NEON 1
#else
#define NLOHMANN_VIEW_NEON 0
#endif
// (x86 only: other targets can define __SSE2__ as well, e.g., WebAssembly with -msse2, but have no <cpuid.h>)
#if !defined(JSON_VIEW_NO_SIMD) && !NLOHMANN_VIEW_NEON && (defined(__x86_64__) || defined(__i386__) || defined(_M_X64) || defined(_M_IX86)) && (defined(__SSE2__) || defined(_M_X64) || (defined(_M_IX86_FP) && _M_IX86_FP >= 2))
#include <emmintrin.h>
#define NLOHMANN_VIEW_SSE2 1
#else
#define NLOHMANN_VIEW_SSE2 0
#endif
#if NLOHMANN_VIEW_SSE2 && defined(JSON_VIEW_USE_SSSE3)
#include <tmmintrin.h>
#define NLOHMANN_VIEW_SSSE3 1 // NOLINT(cppcoreguidelines-macro-to-enum,modernize-macro-to-enum)
#else
#define NLOHMANN_VIEW_SSSE3 0 // NOLINT(cppcoreguidelines-macro-to-enum,modernize-macro-to-enum)
#endif
#if NLOHMANN_VIEW_SSE2 && !NLOHMANN_VIEW_SSSE3 && !defined(NLOHMANN_VIEW_NO_TARGET_ATTRIBUTE) && ((defined(__clang__) && __clang_major__ >= 4) || (defined(__GNUC__) && !defined(__clang__) && (__GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 9))))
// (GCC before 4.9 has no SSSE3 intrinsics without -mssse3)
#include <cpuid.h>
#include <tmmintrin.h>
#define NLOHMANN_VIEW_SSSE3_DISPATCH 1 // NOLINT(cppcoreguidelines-macro-to-enum,modernize-macro-to-enum)
#define NLOHMANN_VIEW_SSSE3_TARGET __attribute__((target("ssse3")))
#elif NLOHMANN_VIEW_SSE2 && !NLOHMANN_VIEW_SSSE3 && defined(_MSC_VER)
// (MSVC compiles intrinsics of any instruction set)
#include <intrin.h>
#include <tmmintrin.h>
#define NLOHMANN_VIEW_SSSE3_DISPATCH 1 // NOLINT(cppcoreguidelines-macro-to-enum,modernize-macro-to-enum)
#define NLOHMANN_VIEW_SSSE3_TARGET
#else
#define NLOHMANN_VIEW_SSSE3_DISPATCH 0 // NOLINT(cppcoreguidelines-macro-to-enum,modernize-macro-to-enum)
#define NLOHMANN_VIEW_SSSE3_TARGET
#endif
#define NLOHMANN_VIEW_VECTOR (NLOHMANN_VIEW_NEON || NLOHMANN_VIEW_SSE2)
#define NLOHMANN_VIEW_VECTOR_UTF8 (NLOHMANN_VIEW_NEON || NLOHMANN_VIEW_SSSE3 || NLOHMANN_VIEW_SSSE3_DISPATCH)
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
{
namespace view
{
#if NLOHMANN_VIEW_VECTOR
/*!
@brief the first byte of a string run that is a quote, a backslash, a control
character, or not ASCII, 16 bytes per step
Stops at such a byte, or where fewer than 16 bytes are left (the caller tells
the two apart). A signed compare with 0x20 finds control characters and
non-ASCII bytes at once.
*/
NLOHMANN_VIEW_ALWAYS_INLINE const unsigned char* vector_plain_run(const unsigned char* p, const unsigned char* e) noexcept
{
while (e - p >= 16)
{
#if NLOHMANN_VIEW_NEON
const uint8x16_t in = vld1q_u8(p);
const uint8x16_t special = vorrq_u8(vorrq_u8(vceqq_u8(in, vdupq_n_u8('"')), vceqq_u8(in, vdupq_n_u8('\\'))),
vcltq_s8(vreinterpretq_s8_u8(in), vdupq_n_s8(0x20)));
// one nibble per byte (the usual NEON replacement of x86's movemask, see
// D. Kutenin, "Porting x86 vector bitmask optimizations to Arm NEON", 2022)
const std::uint64_t bits = vget_lane_u64(vreinterpret_u64_u8(vshrn_n_u16(vreinterpretq_u16_u8(special), 4)), 0);
if (bits != 0)
{
return p + (count_trailing_zeros(bits) >> 2u);
}
#else
const __m128i in = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(p))); // NOLINT(bugprone-casting-through-void)
const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('"')), _mm_cmpeq_epi8(in, _mm_set1_epi8('\\'))),
_mm_cmplt_epi8(in, _mm_set1_epi8(0x20)));
const auto bits = static_cast<std::uint64_t>(static_cast<unsigned>(_mm_movemask_epi8(special)));
if (bits != 0)
{
return p + count_trailing_zeros(bits);
}
#endif
p += 16;
}
return p;
}
#endif
#if NLOHMANN_VIEW_SSSE3_DISPATCH
/// whether the CPU has SSSE3 (CPUID leaf 1, ECX bit 9)
inline bool cpu_ssse3() noexcept
{
#if defined(_MSC_VER) && !defined(__clang__)
std::array<int, 4> regs {{}};
__cpuid(regs.data(), 1);
return (static_cast<unsigned>(regs[2]) & (1u << 9u)) != 0;
#else
unsigned eax = 0;
unsigned ebx = 0;
unsigned ecx = 0;
unsigned edx = 0;
return __get_cpuid(1, &eax, &ebx, &ecx, &edx) != 0 && (ecx & (1u << 9u)) != 0;
#endif
}
/// whether the CPU has SSSE3, asked once: the answer is kept in an atomic
/// that is initialized at compile time, so that neither a guard of a local
/// static nor a global constructor is needed (threads that ask at the same
/// time all store the same answer)
NLOHMANN_VIEW_ALWAYS_INLINE bool cpu_has_ssse3() noexcept
{
static std::atomic<int> known{0}; // 0: not asked yet, 1: no, 2: yes
int state = known.load(std::memory_order_relaxed);
if (NLOHMANN_VIEW_UNLIKELY(state == 0))
{
state = cpu_ssse3() ? 2 : 1;
known.store(state, std::memory_order_relaxed);
}
return state == 2;
}
#endif
#if NLOHMANN_VIEW_VECTOR_UTF8
/// Tables of the UTF-8 check of J. Keiser and D. Lemire, "Validating UTF-8 In
/// Less Than One Instruction Per Byte" (2021), as in simdjson ("lookup4"): each
/// maps a nibble (high and low nibble of the previous byte, high nibble of the
/// current byte) to the errors it allows; a byte pair is ill-formed if all
/// three have an error bit in common.
template<typename Dummy = void>
struct utf8_lookup4
{
static constexpr std::uint8_t too_short = 1u << 0u, too_long = 1u << 1u, overlong_3 = 1u << 2u, too_large = 1u << 3u;
static constexpr std::uint8_t surrogate = 1u << 4u, overlong_2 = 1u << 5u, too_large_1000 = 1u << 6u, overlong_4 = 1u << 6u;
static constexpr std::uint8_t two_conts = 1u << 7u, carry = too_short | too_long | two_conts;
static const std::array<std::uint8_t, 16> byte_1_high;
static const std::array<std::uint8_t, 16> byte_1_low;
static const std::array<std::uint8_t, 16> byte_2_high;
};
template<typename Dummy>
const std::array<std::uint8_t, 16> utf8_lookup4<Dummy>::byte_1_high =
{
{
too_long, too_long, too_long, too_long, too_long, too_long, too_long, too_long,
two_conts, two_conts, two_conts, two_conts,
too_short | overlong_2, too_short, too_short | overlong_3 | surrogate, too_short | too_large | too_large_1000 | overlong_4
}
};
template<typename Dummy>
const std::array<std::uint8_t, 16> utf8_lookup4<Dummy>::byte_1_low =
{
{
carry | overlong_3 | overlong_2 | overlong_4, carry | overlong_2, carry, carry,
carry | too_large, carry | too_large | too_large_1000, carry | too_large | too_large_1000, carry | too_large | too_large_1000,
carry | too_large | too_large_1000, carry | too_large | too_large_1000, carry | too_large | too_large_1000, carry | too_large | too_large_1000,
carry | too_large | too_large_1000, carry | too_large | too_large_1000 | surrogate, carry | too_large | too_large_1000, carry | too_large | too_large_1000
}
};
template<typename Dummy>
const std::array<std::uint8_t, 16> utf8_lookup4<Dummy>::byte_2_high =
{
{
too_short, too_short, too_short, too_short, too_short, too_short, too_short, too_short,
static_cast<std::uint8_t>(too_long | overlong_2 | two_conts | overlong_3 | too_large_1000 | overlong_4),
static_cast<std::uint8_t>(too_long | overlong_2 | two_conts | overlong_3 | too_large),
static_cast<std::uint8_t>(too_long | overlong_2 | two_conts | surrogate | too_large),
static_cast<std::uint8_t>(too_long | overlong_2 | two_conts | surrogate | too_large),
too_short, too_short, too_short, too_short
}
};
/// the end of scan_string_vector from block, where the vector loop stopped
/// (ill-formed UTF-8, or fewer than 16 bytes left): one byte or sequence at a
/// time, from the start of a sequence that crosses into the block
inline const unsigned char* scan_string_finish(const unsigned char* p, const unsigned char* block, const unsigned char* e, const std::uint8_t* plain) noexcept
{
for (int i = 1; i <= 3 && block - i >= p; ++i)
{
const unsigned char c = block[-i];
if (c < 0x80)
{
break;
}
if (c >= 0xC0)
{
const int len = 2 + static_cast<int>(c >= 0xE0) + static_cast<int>(c >= 0xF0);
if (len > i)
{
block -= i;
}
break;
}
}
for (p = block; p != e;)
{
if (*p < 0x80)
{
if (plain[*p] == 0)
{
return p;
}
++p;
continue;
}
const std::size_t n = validate_one_utf8(p, static_cast<std::size_t>(e - p));
if (n == 0)
{
return p;
}
p += n;
}
return p;
}
/*!
@brief the rest of a string from p (a character boundary), 16 bytes per step
The first quote, backslash, or control character is found with vector
compares, and the UTF-8 check covers the bytes up to it. Returns where the
string scan stops, like scan_string_run: before ill-formed UTF-8 and for the
last bytes of the input, the bytes are checked one sequence at a time. Out of
line, so that no constants of the check occupy registers in the parse loop.
On x86-64, it is compiled for SSSE3 (see cpu_has_ssse3()).
*/
NLOHMANN_VIEW_SSSE3_TARGET NLOHMANN_VIEW_NOINLINE inline const unsigned char* scan_string_vector(const unsigned char* p, const unsigned char* e, const std::uint8_t* plain) noexcept
{
using lookup = utf8_lookup4<>;
const unsigned char* block = p;
#if NLOHMANN_VIEW_NEON
const uint8x16_t t1h = vld1q_u8(lookup::byte_1_high.data());
const uint8x16_t t1l = vld1q_u8(lookup::byte_1_low.data());
const uint8x16_t t2h = vld1q_u8(lookup::byte_2_high.data());
uint8x16_t prev = vdupq_n_u8(0);
while (e - block >= 16)
{
const uint8x16_t in = vld1q_u8(block);
const uint8x16_t special = vorrq_u8(vorrq_u8(vceqq_u8(in, vdupq_n_u8('"')), vceqq_u8(in, vdupq_n_u8('\\'))), vcltq_u8(in, vdupq_n_u8(0x20)));
const uint8x16_t prev1 = vextq_u8(prev, in, 15);
const uint8x16_t sc = vandq_u8(vandq_u8(vqtbl1q_u8(t1h, vshrq_n_u8(prev1, 4)), vqtbl1q_u8(t1l, vandq_u8(prev1, vdupq_n_u8(0x0F)))), vqtbl1q_u8(t2h, vshrq_n_u8(in, 4)));
const uint8x16_t must23 = vorrq_u8(vqsubq_u8(vextq_u8(prev, in, 14), vdupq_n_u8(0xE0 - 0x80)), vqsubq_u8(vextq_u8(prev, in, 13), vdupq_n_u8(0xF0 - 0x80)));
const uint8x16_t err = veorq_u8(vandq_u8(must23, vdupq_n_u8(0x80)), sc);
const std::uint64_t special_bits = vget_lane_u64(vreinterpret_u64_u8(vshrn_n_u16(vreinterpretq_u16_u8(special), 4)), 0);
const std::uint64_t err_bits = vget_lane_u64(vreinterpret_u64_u8(vshrn_n_u16(vreinterpretq_u16_u8(vtstq_u8(err, err)), 4)), 0);
if (special_bits != 0)
{
// errors up to the special byte count (an incomplete sequence
// before a quote shows at the quote); the bytes after it do not
const unsigned k = static_cast<unsigned>(count_trailing_zeros(special_bits)) >> 2u;
const std::uint64_t upto = k == 15 ? ~std::uint64_t{0} :
(std::uint64_t{1} << (4u * (k + 1u))) - 1u;
if ((err_bits & upto) == 0)
{
return block + k;
}
break;
}
if (err_bits != 0)
{
break;
}
prev = in;
block += 16;
}
#else
// the same with SSSE3 (pshufb for the table lookups; nibbles from 16-bit
// shifts, as there are no byte shifts)
const __m128i t1h = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(lookup::byte_1_high.data()))); // NOLINT(bugprone-casting-through-void)
const __m128i t1l = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(lookup::byte_1_low.data()))); // NOLINT(bugprone-casting-through-void)
const __m128i t2h = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(lookup::byte_2_high.data()))); // NOLINT(bugprone-casting-through-void)
const __m128i nibble = _mm_set1_epi8(0x0F);
const __m128i zero = _mm_setzero_si128();
__m128i prev = zero;
while (e - block >= 16)
{
const __m128i in = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(block))); // NOLINT(bugprone-casting-through-void)
const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('"')), _mm_cmpeq_epi8(in, _mm_set1_epi8('\\'))),
_mm_cmpeq_epi8(_mm_subs_epu8(in, _mm_set1_epi8(0x1F)), zero)); // in < 0x20
const __m128i prev1 = _mm_alignr_epi8(in, prev, 15);
const __m128i sc = _mm_and_si128(_mm_and_si128(_mm_shuffle_epi8(t1h, _mm_and_si128(_mm_srli_epi16(prev1, 4), nibble)),
_mm_shuffle_epi8(t1l, _mm_and_si128(prev1, nibble))),
_mm_shuffle_epi8(t2h, _mm_and_si128(_mm_srli_epi16(in, 4), nibble)));
const __m128i must23 = _mm_or_si128(_mm_subs_epu8(_mm_alignr_epi8(in, prev, 14), _mm_set1_epi8(0xE0 - 0x80)),
_mm_subs_epu8(_mm_alignr_epi8(in, prev, 13), _mm_set1_epi8(0xF0 - 0x80)));
const __m128i err = _mm_xor_si128(_mm_and_si128(must23, _mm_set1_epi8(static_cast<char>(-128))), sc);
const auto special_bits = static_cast<unsigned>(_mm_movemask_epi8(special));
const auto err_bits = ~static_cast<unsigned>(_mm_movemask_epi8(_mm_cmpeq_epi8(err, zero))) & 0xFFFFu;
if (special_bits != 0)
{
const auto k = static_cast<unsigned>(count_trailing_zeros(static_cast<std::uint64_t>(special_bits)));
if ((err_bits & ((2u << k) - 1u)) == 0)
{
return block + k;
}
break;
}
if (err_bits != 0)
{
break;
}
prev = in;
block += 16;
}
#endif
return scan_string_finish(p, block, e, plain);
}
#endif
} // namespace view
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
+1 -1
View File
@@ -50,7 +50,7 @@ NLOHMANN_VIEW_ALWAYS_INLINE T arithmetic_value(const document_data& d, const nod
case value_t::number_integer:
return static_cast<T>(static_cast<typename BasicJsonType::number_integer_t>(static_cast<std::int64_t>(integer_bits(n))));
case value_t::number_float:
return static_cast<T>(float_value<typename BasicJsonType::number_float_t>(d.str(n), n));
return static_cast<T>(float_value<typename BasicJsonType::number_float_t>(d, n));
case value_t::boolean:
return static_cast<T>((n.flags & node_flags::is_true) != 0);
case value_t::null:
+155 -2
View File
@@ -24,13 +24,18 @@
#ifndef INCLUDE_NLOHMANN_JSON_VIEW_HPP_
#define INCLUDE_NLOHMANN_JSON_VIEW_HPP_
#include <algorithm> // min
#include <array> // array
#include <cstddef> // nullptr_t, size_t // IWYU pragma: keep
#include <cstdint> // uint32_t
#include <cstring> // memcpy, strlen
#include <istream> // istream
#include <istream> // istream // IWYU pragma: keep
#include <iterator> // distance, input_iterator_tag, iterator_traits
#include <map> // map
#include <memory> // unique_ptr
#ifndef JSON_NO_IO
#include <ostream> // ostream // IWYU pragma: keep
#endif
#include <string> // string
#include <tuple> // tuple_element, tuple_size // IWYU pragma: keep
#include <type_traits> // decay, enable_if, integral_constant, is_arithmetic, is_base_of, is_integral, is_same, remove_cv, remove_extent
@@ -60,6 +65,7 @@
#endif
#include <nlohmann/detail/view/builder.hpp>
#include <nlohmann/detail/view/compare.hpp>
#include <nlohmann/detail/view/document_data.hpp>
#include <nlohmann/detail/view/errors.hpp>
#include <nlohmann/detail/view/input.hpp>
@@ -68,7 +74,9 @@
#include <nlohmann/detail/view/macro_scope.hpp>
#include <nlohmann/detail/view/materialize.hpp>
#include <nlohmann/detail/view/node.hpp>
#include <nlohmann/detail/view/object_index.hpp>
#include <nlohmann/detail/view/pointer.hpp>
#include <nlohmann/detail/view/serializer.hpp>
#include <nlohmann/detail/view/string_ref.hpp>
#include <nlohmann/detail/view/value.hpp>
@@ -576,6 +584,96 @@ class basic_json_view
return {m_doc->str(*m_node), detail::view::number_length(*m_node)};
}
///////////////////
// serialization //
///////////////////
/// how dump() writes numbers
enum class number_format
{
/// as basic_json::dump(): integers canonically, floats with the
/// library's shortest round-trip digits ("1.5", "100.0", "1e+100")
shortest,
/// the number text of the source as it is ("1.50", "1E2", "-0", all
/// digits of a long integer)
source,
};
/// the text of this value; with number_format::shortest, the output of
/// ordered_json::parse(text).dump() with the same arguments (members in
/// document order, all of them should a key occur more than once)
string_t dump(const int indent = -1, const char indent_char = ' ', const bool ensure_ascii = false,
const number_format numbers = number_format::shortest) const
{
string_t out;
if (m_node == nullptr)
{
out = "<discarded>"; // as basic_json::dump() of a discarded value
return out;
}
detail::view::dump_style style;
style.pretty = indent >= 0;
style.indent = indent >= 0 ? static_cast<std::size_t>(indent) : 0;
style.indent_char = indent_char;
style.ensure_ascii = ensure_ascii;
style.source_numbers = numbers == number_format::source;
// the compact text is about as long as the source text of the value
const std::size_t estimate = source_extent() + (style.pretty ? source_extent() / 2 : 0) + 64;
detail::view::view_serializer<BasicJsonType>(*m_doc, out, estimate, style).dump(m_node);
return out;
}
#ifndef JSON_NO_IO
/// as operator<< of basic_json: a stream width > 0 is the indentation,
/// the fill character the indentation character
friend std::ostream& operator<<(std::ostream& o, const basic_json_view& v)
{
const bool pretty = o.width() > 0;
const auto indentation = pretty ? o.width() : 0;
o.width(0);
const string_t s = v.dump(pretty ? static_cast<int>(indentation) : -1, o.fill());
return o.write(s.data(), static_cast<std::streamsize>(s.size()));
}
#endif
////////////////
// comparison //
////////////////
/// whether the values parse() would produce for two views are equal, as
/// by BasicJsonType's operator== (numbers by value, objects by their
/// members with duplicate keys resolved as parse() resolves them)
friend bool operator==(const basic_json_view& a, const basic_json_view& b)
{
return detail::view::equal<BasicJsonType>(side(a), side(b));
}
friend bool operator!=(const basic_json_view& a, const basic_json_view& b)
{
return !(a == b);
}
/// whether the value parse() would produce for a view equals a value
friend bool operator==(const basic_json_view& a, const BasicJsonType& j)
{
return detail::view::equal<BasicJsonType>(side(a), json_side_t(j));
}
friend bool operator==(const BasicJsonType& j, const basic_json_view& a)
{
return a == j;
}
friend bool operator!=(const basic_json_view& a, const BasicJsonType& j)
{
return !(a == j);
}
friend bool operator!=(const BasicJsonType& j, const basic_json_view& a)
{
return !(a == j);
}
/////////////////
// materialize //
/////////////////
@@ -608,6 +706,43 @@ class basic_json_view
: m_doc(d), m_node(n)
{}
using json_side_t = detail::view::json_side<BasicJsonType, string_view_t>;
static detail::view::view_side<BasicJsonType, basic_json_view> side(const basic_json_view& v) noexcept
{
return detail::view::view_side<BasicJsonType, basic_json_view>(v);
}
/// the number of source bytes of this value (estimated for values with
/// decoded strings); the estimate sizes the output buffer of dump()
std::size_t source_extent() const noexcept
{
const node* const end = m_doc->nodes + m_doc->node_count;
if ((m_node->flags & detail::view::node_flags::storage) != 0)
{
return m_node->len;
}
// the value ends where the next node in the source begins; nodes
// with decoded strings (their offset is in the arena) are skipped,
// but only a few of them, to keep the walk short
const node* next = document_data::after(m_node);
for (int skipped = 0; next != end && skipped < 16; ++skipped, ++next)
{
if ((next->flags & detail::view::node_flags::storage) == 0)
{
return next->off >= m_node->off ? next->off - m_node->off : 0;
}
}
if (next == end)
{
return m_doc->size - m_node->off;
}
// the end is unknown: assume a few bytes per node, the output buffer
// grows should the value be larger
const auto nodes = static_cast<std::size_t>(document_data::after(m_node) - m_node);
return (std::min)(m_doc->size - m_node->off, static_cast<std::size_t>(1024) + (nodes * 16));
}
/// the value of the first member with this key, or a discarded view
/// (object required)
NLOHMANN_VIEW_ALWAYS_INLINE basic_json_view lookup(string_view_t key) const noexcept
@@ -876,7 +1011,9 @@ class basic_json_document
}
return sizeof(document_data) + (m_data->inline_capacity * sizeof(detail::view::node))
+ (m_data->nodes != m_data->inline_nodes ? m_data->node_capacity * sizeof(detail::view::node) : 0)
+ m_data->arena.capacity() + m_data->owned.capacity();
+ m_data->arena.capacity() + m_data->owned.capacity()
+ (m_data->indexes.capacity() * sizeof(document_data::object_index)) + (m_data->index_slots.capacity() * sizeof(std::uint32_t))
+ (m_data->large_objects.capacity() * sizeof(std::uint32_t));
}
/// release unused capacity of the index and the decoded strings; like
@@ -897,6 +1034,8 @@ class basic_json_document
std::vector<char> arena(shrink_arena ? d.arena : std::vector<char>());
const bool shrink_nodes = d.nodes != d.inline_nodes && d.node_count != d.node_capacity;
const bool into_header = d.node_count <= d.inline_capacity;
std::vector<document_data::object_index> indexes(d.indexes.capacity() > d.indexes.size() ? d.indexes : std::vector<document_data::object_index>());
std::vector<std::uint32_t> index_slots(d.index_slots.capacity() > d.index_slots.size() ? d.index_slots : std::vector<std::uint32_t>());
node* fresh = (shrink_nodes && !into_header) ? static_cast<node*>(::operator new (d.node_count * sizeof(node))) : d.inline_nodes;
if (shrink_nodes)
@@ -914,6 +1053,14 @@ class basic_json_document
d.arena.swap(arena);
d.base[1] = d.arena.data();
}
if (d.indexes.capacity() > d.indexes.size())
{
d.indexes.swap(indexes);
}
if (d.index_slots.capacity() > d.index_slots.size())
{
d.index_slots.swap(index_slots);
}
}
private:
@@ -973,6 +1120,9 @@ class basic_json_document
d.size = size;
d.node_count = 0;
d.arena.clear();
d.indexes.clear();
d.index_slots.clear();
d.large_objects.clear();
d.discarded = true;
detail::view::parse_failure failure;
bool ok = false;
@@ -988,9 +1138,12 @@ class basic_json_document
{
d.base[0] = d.src;
d.base[1] = d.arena.data();
detail::view::build_object_indexes(d);
std::vector<std::uint32_t>().swap(d.large_objects); // (only needed while parsing)
d.discarded = false;
return;
}
std::vector<std::uint32_t>().swap(d.large_objects);
if (allow_exceptions)
{
detail::view::throw_parse_failure<BasicJsonType>(failure, src, size, comments, trailing_commas);
+113664
View File
File diff suppressed because it is too large. Load diff
+18 -4
View File
@@ -166,14 +166,26 @@
#define NLOHMANN_JSON_ABI_TAG_OBJECTS_FOR_ENUM_KEYED_MAPS
#endif
#ifdef JSON_VIEW_NO_SIMD
#define NLOHMANN_JSON_ABI_TAG_VIEW_NO_SIMD _vns
#else
#define NLOHMANN_JSON_ABI_TAG_VIEW_NO_SIMD
#endif
#ifdef JSON_VIEW_USE_SSSE3
#define NLOHMANN_JSON_ABI_TAG_VIEW_USE_SSSE3 _vssse3
#else
#define NLOHMANN_JSON_ABI_TAG_VIEW_USE_SSSE3
#endif
#ifndef NLOHMANN_JSON_NAMESPACE_NO_VERSION
#define NLOHMANN_JSON_NAMESPACE_NO_VERSION 0
#endif
// Construct the namespace ABI tags component
#define NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f, g, h) json_abi ## a ## b ## c ## d ## e ## f ## g ## h
#define NLOHMANN_JSON_ABI_TAGS_CONCAT(a, b, c, d, e, f, g, h) \
NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f, g, h)
#define NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f, g, h, i, j) json_abi ## a ## b ## c ## d ## e ## f ## g ## h ## i ## j
#define NLOHMANN_JSON_ABI_TAGS_CONCAT(a, b, c, d, e, f, g, h, i, j) \
NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f, g, h, i, j)
#define NLOHMANN_JSON_ABI_TAGS \
NLOHMANN_JSON_ABI_TAGS_CONCAT( \
@@ -184,7 +196,9 @@
NLOHMANN_JSON_ABI_TAG_PRECISE_STREAM_POSITION, \
NLOHMANN_JSON_ABI_TAG_STRICT_NUL_HANDLING, \
NLOHMANN_JSON_ABI_TAG_STRICT_BINARY_UTF8, \
NLOHMANN_JSON_ABI_TAG_OBJECTS_FOR_ENUM_KEYED_MAPS)
NLOHMANN_JSON_ABI_TAG_OBJECTS_FOR_ENUM_KEYED_MAPS, \
NLOHMANN_JSON_ABI_TAG_VIEW_NO_SIMD, \
NLOHMANN_JSON_ABI_TAG_VIEW_USE_SSSE3)
// Construct the namespace version component
#define NLOHMANN_JSON_NAMESPACE_VERSION_CONCAT_EX(major, minor, patch) \
+18 -4
View File
@@ -119,14 +119,26 @@
#define NLOHMANN_JSON_ABI_TAG_OBJECTS_FOR_ENUM_KEYED_MAPS
#endif
#ifdef JSON_VIEW_NO_SIMD
#define NLOHMANN_JSON_ABI_TAG_VIEW_NO_SIMD _vns
#else
#define NLOHMANN_JSON_ABI_TAG_VIEW_NO_SIMD
#endif
#ifdef JSON_VIEW_USE_SSSE3
#define NLOHMANN_JSON_ABI_TAG_VIEW_USE_SSSE3 _vssse3
#else
#define NLOHMANN_JSON_ABI_TAG_VIEW_USE_SSSE3
#endif
#ifndef NLOHMANN_JSON_NAMESPACE_NO_VERSION
#define NLOHMANN_JSON_NAMESPACE_NO_VERSION 0
#endif
// Construct the namespace ABI tags component
#define NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f, g, h) json_abi ## a ## b ## c ## d ## e ## f ## g ## h
#define NLOHMANN_JSON_ABI_TAGS_CONCAT(a, b, c, d, e, f, g, h) \
NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f, g, h)
#define NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f, g, h, i, j) json_abi ## a ## b ## c ## d ## e ## f ## g ## h ## i ## j
#define NLOHMANN_JSON_ABI_TAGS_CONCAT(a, b, c, d, e, f, g, h, i, j) \
NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f, g, h, i, j)
#define NLOHMANN_JSON_ABI_TAGS \
NLOHMANN_JSON_ABI_TAGS_CONCAT( \
@@ -137,7 +149,9 @@
NLOHMANN_JSON_ABI_TAG_PRECISE_STREAM_POSITION, \
NLOHMANN_JSON_ABI_TAG_STRICT_NUL_HANDLING, \
NLOHMANN_JSON_ABI_TAG_STRICT_BINARY_UTF8, \
NLOHMANN_JSON_ABI_TAG_OBJECTS_FOR_ENUM_KEYED_MAPS)
NLOHMANN_JSON_ABI_TAG_OBJECTS_FOR_ENUM_KEYED_MAPS, \
NLOHMANN_JSON_ABI_TAG_VIEW_NO_SIMD, \
NLOHMANN_JSON_ABI_TAG_VIEW_USE_SSSE3)
// Construct the namespace version component
#define NLOHMANN_JSON_NAMESPACE_VERSION_CONCAT_EX(major, minor, patch) \
+1638 -30
View File
File diff suppressed because it is too large. Load diff
+6
View File
@@ -17,6 +17,12 @@ module;
// 3. Or upgrade to a newer GCC version with better modules support.
// See: https://github.com/nlohmann/json/issues/5103
// GCC ignores the target attribute in modules ("'target' attribute currently
// unsupported in modules") and then rejects the SSSE3 intrinsics of json_view.
#if defined(__GNUC__) && !defined(__clang__)
#define NLOHMANN_VIEW_NO_TARGET_ATTRIBUTE
#endif
#include <nlohmann/json.hpp>
#include <nlohmann/json_literals.hpp>
#include <nlohmann/json_view.hpp>
+19
View File
@@ -369,6 +369,25 @@ json_test_add_test_for(src/unit-diagnostic-positions.cpp
MAIN test_main CXX_STANDARDS ${test_cxx_standards} ${test_force}
)
# the json_view parser again with the portable string scanning instead of
# NEON/SSE2, and on x86-64 with the SSSE3 UTF-8 check (JSON_VIEW_USE_SSSE3)
json_test_set_test_options(test-json_view_builder_portable
COMPILE_DEFINITIONS JSON_VIEW_NO_SIMD
)
json_test_add_test_for(src/unit-json_view_builder.cpp
NAME test-json_view_builder_portable
MAIN test_main CXX_STANDARDS ${test_cxx_standards} ${test_force}
)
if(CMAKE_SYSTEM_PROCESSOR MATCHES "^(x86_64|AMD64|amd64)$" AND NOT MSVC)
json_test_set_test_options(test-json_view_builder_ssse3
COMPILE_DEFINITIONS JSON_VIEW_USE_SSSE3
COMPILE_OPTIONS -mssse3
)
json_test_add_test_for(src/unit-json_view_builder.cpp
NAME test-json_view_builder_ssse3
MAIN test_main CXX_STANDARDS ${test_cxx_standards} ${test_force}
)
endif()
endif()
# *DO NOT* use json_test_set_test_options() below this line
+3 -1
View File
@@ -21,7 +21,9 @@ foreach(test default noversion)
target_compile_definitions(abi_config_${test}_all_tags PRIVATE
JSON_DIAGNOSTICS=1
JSON_DIAGNOSTIC_POSITIONS=1
JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON=1)
JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON=1
JSON_VIEW_NO_SIMD
JSON_VIEW_USE_SSSE3)
target_link_libraries(abi_config_${test}_all_tags PRIVATE abi_compat_main)
add_test(
NAME test-abi_config_${test}_all_tags
+8
View File
@@ -52,6 +52,14 @@ TEST_CASE("default namespace")
expected += "_ekmo";
#endif
#ifdef JSON_VIEW_NO_SIMD
expected += "_vns";
#endif
#ifdef JSON_VIEW_USE_SSSE3
expected += "_vssse3";
#endif
expected += "_v" STRINGIZE(NLOHMANN_JSON_VERSION_MAJOR);
expected += "_" STRINGIZE(NLOHMANN_JSON_VERSION_MINOR);
expected += "_" STRINGIZE(NLOHMANN_JSON_VERSION_PATCH) "::basic_json";
+8
View File
@@ -53,6 +53,14 @@ TEST_CASE("default namespace without version component")
expected += "_ekmo";
#endif
#ifdef JSON_VIEW_NO_SIMD
expected += "_vns";
#endif
#ifdef JSON_VIEW_USE_SSSE3
expected += "_vssse3";
#endif
expected += "::basic_json";
// fallback for Clang
+1
View File
@@ -21,6 +21,7 @@ Micro-benchmarks for parsing, serialization and the binary formats, written with
| `ViewParseIndented` | as `ParseIndented`, with a reused `json_document` |
| `ViewAccept` | validate with `json_document::accept`; compare with `Accept` |
| `ViewMaterialize` | convert a parsed `json_document` into a `json` value |
| `ViewDump` | serialize a parsed `json_document`; compare with `Dump` |
The input files are those of [nativejson-benchmark](https://github.com/miloyip/nativejson-benchmark) (`canada`,
`citm_catalog`, `twitter`), a large `jeopardy` file, and number-heavy files (`floats`, `signed_ints`, ...).
+34
View File
@@ -144,3 +144,37 @@ static void ViewMaterialize(benchmark::State& state, const char* filename)
state.SetBytesProcessed(state.iterations() * str.size());
}
JSON_VIEW_BENCHMARK_FILES(ViewMaterialize);
//////////////////////////////////////////////////////////////////////////////
// serialize a parsed document (compare with Dump)
//////////////////////////////////////////////////////////////////////////////
static void ViewDump(benchmark::State& state, const char* filename, int indent)
{
const std::string str = read_file(filename);
const json_document d = json_document::parse(str);
while (state.KeepRunning())
{
std::string output = d.root().dump(indent);
benchmark::DoNotOptimize(output);
}
state.SetBytesProcessed(state.iterations() * d.root().dump(indent).size());
}
BENCHMARK_CAPTURE(ViewDump, jeopardy / -, TEST_DATA_DIRECTORY "/jeopardy/jeopardy.json", -1);
BENCHMARK_CAPTURE(ViewDump, jeopardy / 4, TEST_DATA_DIRECTORY "/jeopardy/jeopardy.json", 4);
BENCHMARK_CAPTURE(ViewDump, canada / -, TEST_DATA_DIRECTORY "/nativejson-benchmark/canada.json", -1);
BENCHMARK_CAPTURE(ViewDump, canada / 4, TEST_DATA_DIRECTORY "/nativejson-benchmark/canada.json", 4);
BENCHMARK_CAPTURE(ViewDump, citm_catalog / -, TEST_DATA_DIRECTORY "/nativejson-benchmark/citm_catalog.json", -1);
BENCHMARK_CAPTURE(ViewDump, citm_catalog / 4, TEST_DATA_DIRECTORY "/nativejson-benchmark/citm_catalog.json", 4);
BENCHMARK_CAPTURE(ViewDump, twitter / -, TEST_DATA_DIRECTORY "/nativejson-benchmark/twitter.json", -1);
BENCHMARK_CAPTURE(ViewDump, twitter / 4, TEST_DATA_DIRECTORY "/nativejson-benchmark/twitter.json", 4);
BENCHMARK_CAPTURE(ViewDump, floats / -, TEST_DATA_DIRECTORY "/regression/floats.json", -1);
BENCHMARK_CAPTURE(ViewDump, floats / 4, TEST_DATA_DIRECTORY "/regression/floats.json", 4);
BENCHMARK_CAPTURE(ViewDump, signed_ints / -, TEST_DATA_DIRECTORY "/regression/signed_ints.json", -1);
BENCHMARK_CAPTURE(ViewDump, signed_ints / 4, TEST_DATA_DIRECTORY "/regression/signed_ints.json", 4);
BENCHMARK_CAPTURE(ViewDump, unsigned_ints / -, TEST_DATA_DIRECTORY "/regression/unsigned_ints.json", -1);
BENCHMARK_CAPTURE(ViewDump, unsigned_ints / 4, TEST_DATA_DIRECTORY "/regression/unsigned_ints.json", 4);
BENCHMARK_CAPTURE(ViewDump, small_signed_ints / -, TEST_DATA_DIRECTORY "/regression/small_signed_ints.json", -1);
BENCHMARK_CAPTURE(ViewDump, small_signed_ints / 4, TEST_DATA_DIRECTORY "/regression/small_signed_ints.json", 4);
+89
View File
@@ -0,0 +1,89 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#pragma once
// helpers shared by the unit-json_view*.cpp files, which are split to keep every
// object file small enough for the MinGW linker (it fails to link objects with
// more than 32767 sections)
#include <nlohmann/json_view.hpp>
#include <algorithm> // any_of
#include <random> // mt19937
#include <string> // string
namespace json_view_test
{
// a small deterministic generator of documents
struct generator
{
std::mt19937 rng{5295}; // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed)
int r(int n)
{
return static_cast<int>(rng() % static_cast<unsigned>(n));
}
void str(std::string& o)
{
static const char* const pieces[] = {"a", "Z", " ", "\\n", "\\\"", "\\u00e9", "\\ud83d\\ude00", "\xc3\xa9", "\xe3\x81\x82", "long text beyond the first sixteen bytes"}; // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
o += '"';
for (int n = r(5); n > 0; --n)
{
o += pieces[r(10)];
}
o += '"';
}
void value(std::string& o, int depth)
{
static const char* const scalars[] = {"0", "-1", "123456789012", "18446744073709551615", "18446744073709551616", "-9223372036854775809", // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
"1.5", "-2.25e-3", "1E2", "0.1", "true", "false", "null"
};
const int k = depth > 5 ? 2 + r(4) : r(6);
if (k < 2)
{
const bool object = k == 0;
o += object ? '{' : '[';
for (int i = r(5); i > 0; --i)
{
if (object)
{
str(o);
o += r(2) == 0 ? ":" : " : ";
}
value(o, depth + 1);
o += i > 1 ? ", " : "";
}
o += object ? '}' : ']';
}
else if (k < 4)
{
str(o);
}
else
{
o += scalars[r(13)];
}
}
};
// whether an object of the view repeats a key
inline bool has_duplicate_keys(const nlohmann::ordered_json_view& v)
{
if (v.is_object() && v.size() != v.materialize().size())
{
return true;
}
return std::any_of(v.begin(), v.end(), [](const nlohmann::ordered_json_view e)
{
return e.is_structured() && has_duplicate_keys(e);
});
}
} // namespace json_view_test
+14 -67
View File
@@ -16,6 +16,10 @@ using nlohmann::json_view;
using nlohmann::ordered_json_document;
using nlohmann::ordered_json_view;
#include "json_view_test_helpers.hpp"
using json_view_test::generator;
using json_view_test::has_duplicate_keys;
#include <algorithm>
#include <array>
#include <cmath>
@@ -23,6 +27,7 @@ using nlohmann::ordered_json_view;
#include <cstdint>
#include <cstdio>
#include <cstring>
#include <iomanip>
#include <ios>
#include <istream>
#include <iterator>
@@ -122,60 +127,6 @@ std::string view_exception(const std::string& text, bool comments = false, bool
return "";
}
#endif
// a small deterministic generator of documents
struct generator
{
std::mt19937 rng{5295}; // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed)
int r(int n)
{
return static_cast<int>(rng() % static_cast<unsigned>(n));
}
void str(std::string& o)
{
static const char* const pieces[] = {"a", "Z", " ", "\\n", "\\\"", "\\u00e9", "\\ud83d\\ude00", "\xc3\xa9", "\xe3\x81\x82", "long text beyond the first sixteen bytes"}; // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
o += '"';
for (int n = r(5); n > 0; --n)
{
o += pieces[r(10)];
}
o += '"';
}
void value(std::string& o, int depth)
{
static const char* const scalars[] = {"0", "-1", "123456789012", "18446744073709551615", "18446744073709551616", "-9223372036854775809", // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
"1.5", "-2.25e-3", "1E2", "0.1", "true", "false", "null"
};
const int k = depth > 5 ? 2 + r(4) : r(6);
if (k < 2)
{
const bool object = k == 0;
o += object ? '{' : '[';
for (int i = r(5); i > 0; --i)
{
if (object)
{
str(o);
o += r(2) == 0 ? ":" : " : ";
}
value(o, depth + 1);
o += i > 1 ? ", " : "";
}
o += object ? '}' : ']';
}
else if (k < 4)
{
str(o);
}
else
{
o += scalars[r(13)];
}
}
};
} // namespace
TEST_CASE("json_view")
@@ -1140,18 +1091,6 @@ std::uint32_t bits(float x)
return r;
}
bool has_duplicate_keys(const ordered_json_view& v)
{
if (v.is_object() && v.size() != v.materialize().size())
{
return true;
}
return std::any_of(v.begin(), v.end(), [](const ordered_json_view e)
{
return e.is_structured() && has_duplicate_keys(e);
});
}
// compares the conversions of a view with those of ordered_json
void check_values(const ordered_json_view& v, const ordered_json& j, const std::string& text)
{
@@ -1295,7 +1234,15 @@ TEST_CASE("json_view values")
std::mt19937_64 rng(5295); // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed)
std::vector<std::string> tokens = {"0.1", "-0.0", "1e308", "1.7976931348623157e308", "2.2250738585072011e-308", "4.9e-324", "5e-324",
"0.1000000000000000055511151231257827021181583404541015625", "123456789012345678901234567890",
"9007199254740993", "1.00000000000000011102230246251565404236316680908203125", "7.2057594037927933e16"
"9007199254740993", "1.00000000000000011102230246251565404236316680908203125", "7.2057594037927933e16",
// around the limits of the conversion from the digit layout: 19 and 20
// digits, and those of Clinger's fast path (2^53, 10^22)
"1234567890.123456789", "1234567890.1234567891", "0.0000000000000000001", "123456789012345678.9",
"9007199254740992.0", "9007199254740993.0", "9007199254740994.0", "1.5e22", "1.5e23", "15e-22", "15e-23",
"1e-400", "0.0e0", "-0.0e-5", "12E+3", "12e-0",
// and of float: 2^24 + 1 and 2^24 + 3 (ties), the subnormal and normal limits
"16777217", "16777219", "1.4e-45", "7.006492321624085e-46", "7.006492321624086e-46",
"1.17549435e-38", "0.30000001192092896", "3.4028234e37"
};
for (int i = 0; i < 20000; ++i)
{
+105
View File
@@ -18,6 +18,7 @@
#endif
using nlohmann::json;
#include <array>
#include <cstdint>
#include <fstream>
#include <limits>
@@ -145,6 +146,32 @@ void check_same(const std::string& text)
}
}
// a string value and a key must be accepted or rejected as json::parse does,
// and give its value (cheaper than check_same: the options do not matter)
void check_string(const std::string& content)
{
for (const std::string& text :
{
"[\"" + content + "\"]", "{\"" + content + "\":1}"
})
{
const bool accepted = json::accept(text);
for (const bool sentinel :
{
true, false
})
{
const built b = build(text, false, false, sentinel);
if (b.ok != accepted || (accepted && value_of(b) != json::parse(text)))
{
CAPTURE(text)
CHECK(b.ok == accepted);
CHECK((b.ok && accepted ? value_of(b) == json::parse(text) : true));
}
}
}
}
// a small deterministic generator of documents
struct generator
{
@@ -418,6 +445,84 @@ TEST_CASE("json_view builder")
}
}
TEST_CASE("json_view builder: strings across vector blocks")
{
// Strings are scanned 8 or 16 bytes at a time (NEON, SSE2, or SWAR) and
// non-ASCII text with the vector UTF-8 check (NEON, SSSE3) or one sequence
// at a time. Sequences are placed so that they start at every offset
// around the block boundaries of keys (16, 32) and values (8, 24), with
// text of several lengths after them.
const std::array<std::size_t, 16> prefixes = {{0, 6, 7, 8, 13, 14, 15, 16, 21, 22, 23, 24, 29, 30, 31, 32}};
const std::array<std::size_t, 3> suffixes = {{0, 3, 17}};
const auto around = [&](const std::string & seq, std::size_t prefix, std::size_t suffix)
{
return std::string(prefix, 'a') + seq + std::string(suffix, 'b');
};
SECTION("every two-byte sequence")
{
for (unsigned lead = 0x80; lead <= 0xFF; ++lead)
{
for (unsigned second = 0; second <= 0xFF; ++second)
{
if (second == '"' || second == '\\')
{
continue;
}
const std::string seq = {static_cast<char>(lead), static_cast<char>(second)};
check_string(around(seq, prefixes[(lead + second) % 16], suffixes[second % 3]));
}
}
}
SECTION("three- and four-byte sequences")
{
const std::array<unsigned, 8> conts = {{0x7F, 0x80, 0x8F, 0x90, 0x9F, 0xA0, 0xBF, 0xC0}};
for (unsigned lead = 0xE0; lead <= 0xF7; ++lead)
{
for (const unsigned b2 : conts)
{
for (const unsigned b3 : conts)
{
for (const std::size_t prefix : prefixes)
{
std::string seq = {static_cast<char>(lead), static_cast<char>(b2), static_cast<char>(b3)};
if (lead >= 0xF0)
{
seq += static_cast<char>(prefix % 2 == 0 ? 0x80 : 0xBF);
}
check_string(around(seq, prefix, suffixes[prefix % 3]));
// cut short before the end of the string
check_string(around(seq.substr(0, seq.size() - 1), prefix, suffixes[prefix % 3]));
}
}
}
}
}
SECTION("long runs of text with one damaged byte")
{
const std::array<const char*, 7> chars = {{"a", "\xc3\xa9", "\xe3\x81\x82", "\xf0\x9f\x98\x80", "\xed\x9f\xbf", "\xef\xbf\xbf", "\xf4\x8f\xbf\xbf"}};
const std::array<char, 11> damage = {{'\x80', '\xbf', '\xc0', '\xc1', '\xe0', '\xed', '\xf5', '\xff', '\x1f', '"', '\\'}};
std::mt19937 rng(5295); // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed): reproducible
for (int i = 0; i < 4000; ++i)
{
std::string text;
const auto n = rng() % 60;
for (unsigned k = 0; k < n; ++k)
{
text += chars[rng() % chars.size()];
}
check_string(text);
if (!text.empty())
{
text[rng() % text.size()] = damage[rng() % damage.size()];
check_string(text);
}
}
}
}
TEST_CASE("json_view node integer bits")
{
using nlohmann::detail::view::integer_bits;
+540
View File
@@ -0,0 +1,540 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#include "doctest_compatibility.h"
#include <nlohmann/json_view.hpp>
using nlohmann::json;
using nlohmann::ordered_json;
using nlohmann::json_document;
using nlohmann::json_view;
using nlohmann::ordered_json_document;
using nlohmann::ordered_json_view;
#include "json_view_test_helpers.hpp"
using json_view_test::generator;
using json_view_test::has_duplicate_keys;
#include <array>
#include <cmath>
#include <cstddef>
#include <cstdint>
#include <cstring>
#include <iomanip>
#include <map>
#include <random>
#include <sstream>
#include <string>
#include <vector>
// These tests were split off unit-json_view.cpp, whose object file got too large
// for the MinGW linker (see json_view_test_helpers.hpp). Like that file, this one
// is also built with C++17, as it mentions JSON_HAS_CPP_17.
TEST_CASE("json_view dump")
{
SECTION("the output of ordered_json::dump()")
{
generator g;
for (int i = 0; i < 2000; ++i)
{
std::string text;
g.value(text, 0);
const ordered_json_document d = ordered_json_document::parse(text);
if (has_duplicate_keys(d.root()))
{
continue;
}
CAPTURE(text)
const ordered_json j = ordered_json::parse(text);
for (const int indent :
{
-1, 0, 2
})
{
for (const bool ensure_ascii :
{
false, true
})
{
CHECK(d.root().dump(indent, i % 2 == 0 ? ' ' : '\t', ensure_ascii) == j.dump(indent, i % 2 == 0 ? ' ' : '\t', ensure_ascii));
}
}
// also of each element
for (const ordered_json_view e : d.root())
{
CHECK(e.dump() == e.materialize().dump());
}
}
}
SECTION("strings")
{
const std::string text = R"(["plain", "\u0000\u0001\u001f\u007f\u0080é€￿😀", "\"\\\/\b\f\n\r\t", "aéあ😀b", "long text beyond the eight bytes of a word \n with an escape in the middle"])";
const ordered_json_document d = ordered_json_document::parse(text);
const ordered_json j = ordered_json::parse(text);
CHECK(d.root().dump() == j.dump());
CHECK(d.root().dump(-1, ' ', true) == j.dump(-1, ' ', true));
CHECK(d.root().dump(4, ' ', true) == j.dump(4, ' ', true));
const std::string key_text = R"({"é\n": {"\"": [], "": {}}})";
const ordered_json_document keys = ordered_json_document::parse(key_text);
const ordered_json key_json = ordered_json::parse(key_text);
CHECK(keys.root().dump(2, ' ', true) == key_json.dump(2, ' ', true));
}
SECTION("numbers")
{
const std::string text = "[1.50, 1E2, -0, -0.0, 123456789012345678901234567890, 18446744073709551615, -9223372036854775808, 0.1, 1e-7, 5e-324]";
const json_document d = json_document::parse(text);
CHECK(d.root().dump() == json::parse(text).dump());
CHECK(d.root().dump() == "[1.5,100.0,0,-0.0,1.2345678901234568e+29,18446744073709551615,-9223372036854775808,0.1,1e-07,5e-324]");
CHECK(d.root().dump(-1, ' ', false, json_view::number_format::source) == "[1.50,1E2,-0,-0.0,123456789012345678901234567890,18446744073709551615,-9223372036854775808,0.1,1e-7,5e-324]");
// random doubles, written as parse() and dump() would
std::mt19937_64 rng(1170); // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed)
std::string many = "[";
for (int i = 0; i < 5000; ++i)
{
const std::uint64_t bits = rng();
double x = 0;
std::memcpy(&x, &bits, sizeof(x));
if (std::isfinite(x))
{
many += (many.size() > 1 ? "," : "") + json(x).dump();
}
}
many += ']';
const json_document many_document = json_document::parse(many);
CHECK(many_document.root().dump() == json::parse(many).dump());
using json_float = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int64_t, std::uint64_t, float>;
const nlohmann::basic_json_document<json_float> float_document = nlohmann::basic_json_document<json_float>::parse("[0.1, 1.5e10, 3.4028235e38]");
CHECK(float_document.root().dump() == json_float::parse("[0.1, 1.5e10, 3.4028235e38]").dump());
}
SECTION("members in document order, all of them")
{
const json_document d = json_document::parse(R"({"b": 1, "a": 2, "b": 3})");
CHECK(d.root().dump() == R"({"b":1,"a":2,"b":3})");
CHECK(d.root().dump(1) == "{\n \"b\": 1,\n \"a\": 2,\n \"b\": 3\n}");
// dump() writes all members, though the lookup finds the first
CHECK(d.root()["b"].dump() == "1");
}
SECTION("deep nesting")
{
const std::string deep = std::string(100000, '[') + std::string(100000, ']');
const json_document deep_document = json_document::parse(deep);
CHECK(deep_document.root().dump() == deep);
}
SECTION("the output buffer of a small value is small")
{
// an escaped key after the value: its node lies in the arena, so the
// source extent of the value cannot be read from the next node
const std::string big(100000, 'a');
const std::string text = R"({"small":1,"list":[1,2,3],"k\n":")" + big + R"("})";
const json_document d = json_document::parse(text);
const auto small = d.root()["small"].dump();
CHECK(small == "1");
CHECK(small.capacity() < 4096);
const auto list = d.root()["list"].dump();
CHECK(list == "[1,2,3]");
CHECK(list.capacity() < 4096);
CHECK(d.root()["list"].dump(2).capacity() < 4096);
// the whole document and the large value are unaffected
CHECK(d.root().dump() == ordered_json::parse(text).dump());
CHECK(d.root()["k\n"].dump() == "\"" + big + "\"");
}
SECTION("output that outgrows the estimate")
{
// ensure_ascii writes six bytes for each two-byte character
std::string chars;
for (int i = 0; i < 5000; ++i)
{
chars += "\xC3\xA9";
}
const json_document d = json_document::parse(R"({"a":")" + chars + R"(","k\n":1})");
const json expected = json::parse("\"" + chars + "\"");
CHECK(d.root()["a"].dump(-1, ' ', true) == expected.dump(-1, ' ', true));
CHECK(d.root()["a"].dump(-1, ' ', true).size() == 2 + (5000 * 6));
}
SECTION("streams and discarded views")
{
const json_document d = json_document::parse(R"({"a": [1, 2]})");
std::ostringstream compact;
compact << d.root();
CHECK(compact.str() == R"({"a":[1,2]})");
std::ostringstream pretty;
pretty << std::setw(2) << std::setfill('.') << d.root() << d.root()["a"];
CHECK(pretty.str() == "{\n..\"a\": [\n....1,\n....2\n..]\n}[1,2]");
CHECK(json_view().dump() == json(json::value_t::discarded).dump());
}
}
TEST_CASE("json_view comparison")
{
SECTION("equality of the values parse() produces")
{
generator g;
std::vector<std::string> texts;
for (int i = 0; i < 600; ++i)
{
std::string text;
g.value(text, 0);
texts.push_back(text);
// the same value written differently: sorted keys, canonical numbers
texts.push_back(json::parse(text).dump(1));
}
for (std::size_t i = 0; i + 2 < texts.size(); ++i)
{
for (std::size_t k = i; k < i + 3; ++k)
{
CAPTURE(texts[i])
CAPTURE(texts[k])
const json_document a = json_document::parse(texts[i]);
const json_document b = json_document::parse(texts[k]);
const json ja = json::parse(texts[i]);
const json jb = json::parse(texts[k]);
CHECK((a.root() == b.root()) == (ja == jb));
CHECK((a.root() != b.root()) == (ja != jb));
CHECK((a.root() == jb) == (ja == jb));
CHECK((jb == a.root()) == (ja == jb));
CHECK((a.root() != jb) == (ja != jb));
CHECK((jb != a.root()) == (ja != jb));
// ordered_json compares members in order
const ordered_json_document oa = ordered_json_document::parse(texts[i]);
const ordered_json_document ob = ordered_json_document::parse(texts[k]);
const ordered_json oja = ordered_json::parse(texts[i]);
const ordered_json ojb = ordered_json::parse(texts[k]);
CHECK((oa.root() == ob.root()) == (oja == ojb));
CHECK((oa.root() == ojb) == (oja == ojb));
}
}
}
SECTION("numbers, duplicate keys, member order")
{
const auto same = [](const char* x, const char* y)
{
const json_document dx = json_document::parse(x);
const json_document dy = json_document::parse(y);
return dx.root() == dy.root();
};
CHECK(same("1", "1.0"));
CHECK(same("[1, -1, 2.5]", "[1.0, -1.0, 25e-1]"));
CHECK(!same("1", "1.5"));
CHECK(same("18446744073709551615", "18446744073709551615"));
CHECK(same(R"({"a": 1, "a": 2})", R"({"a": 2})"));
CHECK(!same(R"({"a": 1, "a": 2})", R"({"a": 1})"));
CHECK(same(R"({"a": 1, "b": 2})", R"({"b": 2, "a": 1})"));
CHECK(!same(R"({"a": 1})", R"({"a": 1, "b": 2})"));
CHECK(!same("[1, 2]", "[2, 1]"));
CHECK(!same("\"a\"", "\"b\""));
CHECK(same("\"\\u00e9\"", "\"\xc3\xa9\""));
CHECK(!same("null", "false"));
CHECK(!same("[]", "{}"));
const ordered_json_document dup = ordered_json_document::parse(R"({"a": 1, "b": 2, "a": 3})");
const ordered_json_document last = ordered_json_document::parse(R"({"a": 3, "b": 2})");
CHECK(dup.root() == last.root());
// (== compares as parse() resolves duplicates, a lookup finds the first)
CHECK(dup.root()["a"].materialize() == 1);
CHECK(dup.root()["a"] != last.root()["a"]);
const ordered_json_document ab = ordered_json_document::parse(R"({"a": 1, "b": 2})");
const ordered_json_document ba = ordered_json_document::parse(R"({"b": 2, "a": 1})");
CHECK(ab.root() != ba.root());
// discarded values compare as basic_json's do
const json discarded(json::value_t::discarded);
CHECK((json_view() == json_view()) == (discarded == discarded)); // NOLINT(readability-container-size-empty): operator== is tested
CHECK((json_view() == discarded) == (discarded == discarded)); // NOLINT(readability-container-size-empty)
const json_document null_document = json_document::parse("null");
CHECK(!(json_view() == null_document.root())); // NOLINT(readability-container-size-empty)
CHECK(!(null_document.root() == discarded));
}
SECTION("deep nesting")
{
const std::string deep = std::string(100000, '[') + std::string(100000, ']');
const json_document a = json_document::parse(deep);
const json_document b = json_document::parse(deep);
CHECK(a.root() == b.root());
CHECK(a.root() == json::parse(deep));
const std::string other = std::string(100000, '[') + "1" + std::string(100000, ']');
const json_document c = json_document::parse(other);
CHECK(a.root() != c.root());
}
}
TEST_CASE("json_view large objects")
{
// objects with 128 members or more are looked up with a hash index
for (const std::size_t members :
{
127u, 128u, 129u, 10000u
})
{
CAPTURE(members)
std::string text = "{";
for (std::size_t i = 0; i < members; ++i)
{
text += (i != 0 ? ",\"" : "\"") + std::string(i % 23, 'k') + std::to_string(i) + (i % 7 == 0 ? "\\n" : "") + "\":" + std::to_string(i);
}
text += R"(,"":"empty key","k1":"a duplicate of an earlier key"})";
const json_document d = json_document::parse(text);
const json_view v = d.root();
const json j = json::parse(text);
for (std::size_t i = 0; i < members; ++i)
{
const std::string key = std::string(i % 23, 'k') + std::to_string(i) + (i % 7 == 0 ? "\n" : "");
CHECK(v[key].get<std::size_t>() == i);
CHECK(v.contains(key));
CHECK(v.find(key).key() == key);
CHECK(v.at(key).get<std::size_t>() == i);
CHECK(!v.contains(key + "x"));
}
CHECK(v[""].get_string() == "empty key");
CHECK(v["k1"].get<int>() == 1); // the first of duplicate keys, as for small objects
CHECK(!v.contains("missing"));
CHECK_THROWS_WITH_AS(v.at("missing"), "[json.exception.out_of_range.403] key 'missing' not found", json::out_of_range&);
CHECK(v == j);
CHECK(v.materialize() == j);
}
SECTION("duplicate keys: the first member wins, with and without a table")
{
// an object of `total` members: the keys "k0".."k<n-1>" in order, then
// three keys repeated twice more (one copy in the middle, one at the
// end), and two keys repeated once; the value of a member is its
// position, so that the first member of a key can be told apart
struct member
{
std::string key;
std::size_t position;
};
const auto make_members = [](std::size_t total)
{
std::vector<std::string> keys;
for (std::size_t i = 0; i + 8 < total; ++i)
{
keys.push_back("k" + std::to_string(i));
}
const std::size_t n = keys.size();
const std::array<std::size_t, 3> triple = {{3, 17, n - 1}};
const std::array<std::size_t, 2> twice = {{5, n / 2}};
std::vector<std::string> ordered = keys;
for (const std::size_t i : triple)
{
ordered.insert(ordered.begin() + static_cast<std::ptrdiff_t>(ordered.size() / 2), keys[i]);
}
for (const std::size_t i : twice)
{
ordered.insert(ordered.begin() + static_cast<std::ptrdiff_t>(ordered.size() / 3), keys[i]);
}
for (const std::size_t i : triple)
{
ordered.push_back(keys[i]);
}
std::vector<member> result;
result.reserve(ordered.size());
for (std::size_t i = 0; i < ordered.size(); ++i)
{
result.push_back({ordered[i], i});
}
return result;
};
// 100 and 127 members: no table; 128 members and more: a table
for (const std::size_t total :
{
100u, 127u, 128u, 200u, 5000u
})
{
CAPTURE(total)
const std::vector<member> members = make_members(total);
REQUIRE(members.size() >= total);
REQUIRE(members.size() >= 100);
std::string text = "{";
std::map<std::string, std::size_t> first;
std::map<std::string, std::size_t> last;
std::map<std::string, std::size_t> occurrences;
for (const member& m : members)
{
text += (text.size() > 1 ? ",\"" : "\"") + m.key + "\":" + std::to_string(m.position);
first.insert({m.key, m.position});
last[m.key] = m.position;
++occurrences[m.key];
}
text += '}';
REQUIRE(first.size() < members.size());
const json_document d = json_document::parse(text);
const json_view v = d.root();
const json j = json::parse(text);
CHECK(v.size() == members.size()); // every occurrence is visited
const json m = v.materialize();
for (const auto& entry : first)
{
CAPTURE(entry.first)
const std::size_t expected = entry.second;
CHECK(v[entry.first].get<std::size_t>() == expected);
CHECK(v.at(entry.first).get<std::size_t>() == expected);
CHECK(v.find(entry.first).value().get<std::size_t>() == expected);
CHECK(v.value(entry.first, std::size_t{0}) == expected);
CHECK(v.contains(entry.first));
CHECK(v.count(entry.first) == occurrences[entry.first]);
const std::string pointer = "/" + entry.first;
CHECK(v[json::json_pointer(pointer)].get<std::size_t>() == expected);
CHECK(v.at(json::json_pointer(pointer)).get<std::size_t>() == expected);
CHECK(v.value(json::json_pointer(pointer), std::size_t{0}) == expected);
CHECK(v.contains(json::json_pointer(pointer)));
// materialize() and parse() keep the last value instead
CHECK(j[entry.first].get<std::size_t>() == last.at(entry.first));
CHECK(m[entry.first].get<std::size_t>() == last.at(entry.first));
}
CHECK(!v.contains("k"));
CHECK(v["missing"].is_discarded());
CHECK(v.materialize() == j);
CHECK(v == j);
}
}
SECTION("colliding keys")
{
// the hash is not seeded, so keys that all land in one place must not
// make the table build quadratic: such an object gets no table and is
// searched linearly
constexpr std::size_t members = 300;
constexpr std::size_t table_size = 1024; // the table size for 300 members: the next power of two >= 600
std::vector<std::string> colliding;
std::vector<std::string> spread;
for (std::uint64_t counter = 0; colliding.size() < members || spread.size() < members; ++counter)
{
std::string key(8, 'a');
std::uint64_t x = counter;
for (std::size_t i = 0; i < 8; ++i, x /= 26)
{
key[i] = static_cast<char>('a' + (x % 26));
}
const bool lands_in_slot_zero = (nlohmann::detail::view::key_hash(key.data(), key.size()) & (table_size - 1)) == 0;
if (lands_in_slot_zero && colliding.size() < members)
{
colliding.push_back(key);
}
else if (!lands_in_slot_zero && spread.size() < members)
{
spread.push_back(key);
}
}
const auto make_text = [](const std::vector<std::string>& keys)
{
std::string text = "{";
for (std::size_t i = 0; i < keys.size(); ++i)
{
text += (i != 0 ? ",\"" : "\"") + keys[i] + "\":" + std::to_string(i % 10);
}
return text + "}";
};
const std::string colliding_text = make_text(colliding);
const std::string spread_text = make_text(spread);
json_document with_collisions = json_document::parse(colliding_text);
json_document without_collisions = json_document::parse(spread_text);
for (const auto* pair :
{
&colliding, &spread
})
{
const json_view v = (pair == &colliding ? with_collisions : without_collisions).root();
for (std::size_t i = 0; i < members; ++i)
{
CAPTURE(i)
CHECK(v[(*pair)[i]].get<std::size_t>() == i % 10);
CHECK(v.at((*pair)[i]).get<std::size_t>() == i % 10);
CHECK(v.find((*pair)[i]).key() == (*pair)[i]);
CHECK(!v.contains((*pair)[i] + "x"));
}
CHECK(!v.contains("missing"));
}
CHECK(with_collisions.root() == json::parse(colliding_text));
// only the object with the spread keys got a table (both texts have the
// same length, so the tables are the only difference)
with_collisions.shrink_to_fit();
without_collisions.shrink_to_fit();
CHECK(without_collisions.memory_usage() >= with_collisions.memory_usage() + (table_size * sizeof(std::uint32_t)));
}
SECTION("shrink_to_fit releases the tables' spare capacity")
{
const auto make_text = [](int objects, int members)
{
std::string text = "[";
for (int object = 0; object < objects; ++object)
{
text += object != 0 ? ",{" : "{";
for (int i = 0, n = members + object; i < n; ++i)
{
text += (i != 0 ? ",\"" : "\"") + std::to_string(i) + "\":" + std::to_string(i);
}
text += '}';
}
return text + "]";
};
const std::string small_text = make_text(5, 150);
const std::string big_text = make_text(40, 400);
// reading a big text, and then a small one, leaves the spare capacity
// of the big one: shrink_to_fit() brings the document to the size of
// one parsed from the small text alone
json_document d = json_document::parse(big_text);
const std::size_t big = d.memory_usage();
d.read(small_text);
CHECK(d.memory_usage() >= big);
d.shrink_to_fit();
json_document fresh = json_document::parse(small_text);
fresh.shrink_to_fit();
CHECK(d.memory_usage() == fresh.memory_usage());
CHECK(d.memory_usage() < big / 2);
CHECK(d.root() == json::parse(small_text));
for (int object = 0; object < 5; ++object)
{
const json_view v = d.root()[static_cast<std::size_t>(object)];
for (int i = 0, n = 150 + object; i < n; ++i)
{
CHECK(v[std::to_string(i)].get<int>() == i);
}
}
}
SECTION("nested, reused, and in arrays")
{
std::string inner = "{";
for (int i = 0; i < 300; ++i)
{
inner += (i != 0 ? ",\"m" : "\"m") + std::to_string(i) + "\":" + std::to_string(i);
}
inner += '}';
const std::string text = "[" + inner + ",{\"x\":" + inner + "}," + inner + "]";
json_document d = json_document::parse(text);
CHECK(d.root()[0]["m299"].get<int>() == 299);
CHECK(d.root()[1]["x"]["m150"].get<int>() == 150);
CHECK(d.root()[2]["m0"].get<int>() == 0);
const std::size_t with_index = d.memory_usage();
d.read(std::string("{\"small\": 1}"));
CHECK(d.root()["small"].get<int>() == 1);
d.read(text);
CHECK(d.root()[2]["m7"].get<int>() == 7);
CHECK(d.memory_usage() >= with_index / 2);
}
}