Compare commits

..
Author SHA1 Message Date
Niels Lohmann 45ef51e214 Regenerate the amalgamated headers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 22:26:07 +02:00
Niels Lohmann 4e0e915b13 Bind the serializer's frame only after the check for its end
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 22:26:00 +02:00
Niels Lohmann b0e60cd382 Allow the string literal parameter of the view serializer's put() for clang-tidy
The overload takes the literal by array reference so that its length is a
template parameter, like scan_literal() and the C array overloads of
to_json() and from_json().

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:49:17 +02:00
Niels Lohmann ec04825b93 Exclude all json_view test files from Infer's reports
The same std::unique_ptr false positive (USE_AFTER_DELETE) also shows for
two json_documents parsed in one lambda in unit-json_view_dump.cpp
(ci_infer). It affects any test that holds several documents, so the
exclusion now covers tests/src/unit-json_view*.cpp; ASan and Valgrind
cover these tests.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:49:16 +02:00
Niels Lohmann d1d061c37f Regenerate the amalgamated headers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:49:15 +02:00
Niels Lohmann 75d58c0584 Address the review of json_view dump() and comparisons
From gregmarr's review of #5624:

- The output buffer takes string literals without a length.
- write_float() names the IEEE single/double condition once, as the
  library's serializer does.
- equal() checks the types first and compares with !=; the reference to
  the top frame is noted as dangling after pop_back().
- dump.md: one paragraph per case in the notes.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:49:15 +02:00
Niels Lohmann 391dcf64bd Keep the stream includes of json_view.hpp for IWYU
json_view.hpp uses std::istream and std::ostream itself, but IWYU sees
them through the exported json.hpp and asked to remove both includes,
which fails ci_single_binaries (single_json_view).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:49:14 +02:00
Niels Lohmann 225a2beedb Follow the renames from the review of the view builder
The review of #5620 renamed document_data's tape/tape_size/tape_cap to
nodes/node_count/node_capacity and made the decode arena a
std::vector<char>; adapt the code of this change to it.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:49:13 +02:00
Niels Lohmann 2c1f6aa58c Regenerate the amalgamated headers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:49:12 +02:00
Niels Lohmann 301e7892a3 Make json_view.hpp pass include-what-you-use
ci_single_binaries runs IWYU with --error on every header. For json_view.hpp
it suggested adding <array> (std::array is used), the headers that json.hpp
already provides (abi_config, abi_macros, input_adapters, json_pointer,
cpp_future, string_concat, value_t, json_fwd), and <version> for
std::nullptr_t, and removing <cstddef>.

- include <array>
- keep <cstddef> (nullptr_t, size_t; IWYU attributes them to <version> and <cstring>)
- tell IWYU not to suggest the headers that json.hpp provides: the amalgamated
  json_view.hpp only includes json.hpp, so including them here would duplicate
  their definitions

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:49:11 +02:00
Niels Lohmann 6002c4bc8d Split the dump and comparison tests off unit-json_view.cpp
The MinGW linker of the Windows clang jobs cannot link object files with more
than 32767 sections ("relocation truncated to fit: IMAGE_REL_AMD64_REL32
against `.rdata'"). unit-json_view.cpp reaches that limit as the stack
grows, so its "json_view dump" and "json_view comparison" test cases move
into unit-json_view_dump.cpp. The test generator and has_duplicate_keys()
that both files use move into json_view_test_helpers.hpp.

The new file mentions JSON_HAS_CPP_17, so it is built for C++17 like the file
it was split from.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:49:11 +02:00
Niels Lohmann 81041a33ed Fix the clang-tidy 22 findings in json_view and its tests
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:49:10 +02:00
Niels Lohmann a7ae5a6f04 Document and test first-wins lookups next to dump() and ==
dump() writes every member and == resolves duplicate keys as parse()
does, whereas lookups find the first member of a duplicate key.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:49:09 +02:00
Niels Lohmann 45dbd6ea6c Do not shrink the output of dump()
finish() copied the whole output when the buffer was more than twice as
large as the result (citm dump +11%). The tighter source_extent()
estimate already keeps the buffer of a small value small.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:49:08 +02:00
Niels Lohmann ab8e77df45 Hold the documents of the dump and comparison tests in names
root() of a temporary document is deleted: its views would dangle.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:49:08 +02:00
Niels Lohmann 3ab2c9d6f4 Size the dump buffer of a small value by its own extent
The source extent of a value was read from the next node, falling back to the rest of the document when that node held a decoded string. dump() of a small value could thus allocate a buffer as large as the document. Skip a few such nodes, cap the fallback estimate, and shrink a buffer that is much larger than its output.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:49:07 +02:00
Niels Lohmann de9b9a360d Avoid raw string with escapes inside CHECK macro (MSVC C2017)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:49:06 +02:00
Niels Lohmann 06effba9e6 Add dump() and comparisons to json_view
Add basic_json_view::dump() and the comparison operators, and read
floats from the parser's digit layout instead of rescanning the
token.

dump(indent, indent_char, ensure_ascii, number_format) writes a
value the way ordered_json::parse(text).dump() writes it for the
same arguments: members in document order, all of them should a
key occur more than once; strings escaped by the same rules, using
the library's scanning kernels; floats written with the library's
to_chars conversion, so the output equals basic_json's byte for
byte; integers copied from the source, where they are already
canonical, except -0, which parse() reads as 0. There is no
error_handler argument, because the view only holds valid UTF-8.
number_format::source copies numbers exactly as they appear in the
source (e.g. "1.50", "1E2", "-0"), which basic_json cannot provide.
operator<< takes the indentation from the stream width, as for
basic_json. The writer walks iteratively, so nesting depth is
limited by memory only.

operator== and operator!= compare two views, or a view and a
basic_json value in either order, by the rules basic_json's
operator== uses: numbers compare by value across their types,
objects compare by their members with duplicate keys resolved as
parse() resolves them, member order matters only where the object
type keeps one, and discarded views compare as discarded basic_json
values do, including under JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON.
Nothing is materialized except single scalars.

While parsing, the view now records where the integer digits, the
fraction digits, and the exponent of a float token are, so floats
and doubles with at most 19 digits are read from that layout with
the library's decimal_to_float() instead of rescanning the token.
Both round correctly, so the values are those of parse(). get<double>(),
materialize(), dump(), and the comparisons all use it.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:49:05 +02:00
Niels Lohmann bf5a42cf2c Regenerate the amalgamated headers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:49:04 +02:00
Niels Lohmann 9211df1074 Keep detail::negation out of json_view.hpp's enable_ifs
include-what-you-use asks json_view.hpp to include
detail/meta/type_traits.hpp for it, which the single header must not do
(the amalgamation would duplicate json's internals). The two conditions of
value() go through a trait in value.hpp, like the others.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:49:03 +02:00
Niels Lohmann 9a1f759feb Regenerate the amalgamated headers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:49:03 +02:00
Niels Lohmann a7cd43cc34 Write the negated enable_if conditions of json_view with negation<>
With a ! or && in the condition, astyle spaces the whole template
header (template < ... >); detail::negation and a named trait keep it
compact, as the library writes such conditions elsewhere. From gregmarr's
review of #5630.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:49:02 +02:00
Niels Lohmann 379032ae20 Regenerate the amalgamated headers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:49:01 +02:00
Niels Lohmann f8d44e8020 Address the second review round of json_view element access
From gregmarr's review of #5622:

- begin()/end(): "a null value or a discarded view iterates as an empty
  range", singular like the primitive case before it.
- The class page's list of function groups has no trailing commas.
- document_data::key_equals() compares a key node with a string, for the
  key comparisons that were spelled out in each lookup.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:49:00 +02:00
Niels Lohmann eb9d23d80a Count matching keys as unsigned in the json_view access test
The conditional added 1 or 0 as int to a std::size_t, which GCC rejects
with -Werror=sign-conversion (ci_test_standards_gcc).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:59 +02:00
Niels Lohmann a830a24bde Regenerate the amalgamated headers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:59 +02:00
Niels Lohmann 18ad1807c5 Address the review of json_view element access
From gregmarr's review of #5622:

- count() counts every member with the key, so count(key) > 1 detects
  duplicate keys; it was contains() as 0/1 before.
- get_to() passes types that get<T>() does not convert directly to
  from_json with the existing value, as basic_json::get_to() does, so that
  a from_json which sets only the fields present keeps the others.
- array_index() checks the empty token and "-" first; the results are
  unchanged.
- Docs: no "if an exception is thrown" where a view never changes
  anything; the borrowed source must also stay unmodified; items() on a
  temporary view is safe; back()/front() notes shortened; the basic_json
  see-also entries say how the view's results can differ.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:58 +02:00
Niels Lohmann 4cf6eb4447 Report one-character non-numeric array indices like basic_json in the view
Develop's #5800 made basic_json report a JSON pointer array index such as
"/a" as parse_error.109 instead of out_of_range.404, as longer ones were;
the view's array_index() follows, with "-" and the empty token still
unresolved.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:57 +02:00
Niels Lohmann 14098c7406 Regenerate the amalgamated headers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:56 +02:00
Niels Lohmann d1d1912928 Fix the clang-tidy 22 findings in json_view and its tests
The same changes as on the dump branch, where they were first made, so
that this branch passes clang-tidy on its own.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:55 +02:00
Niels Lohmann 5c03e57870 Make json_view.hpp pass include-what-you-use
ci_single_binaries runs IWYU with --error on every header. For json_view.hpp
it suggested adding <array> (std::array is used), the headers that json.hpp
already provides (abi_config, abi_macros, input_adapters, json_pointer,
cpp_future, string_concat, value_t, json_fwd), and <version> for
std::nullptr_t, and removing <cstddef>.

- include <array>
- keep <cstddef> (nullptr_t, size_t; IWYU attributes them to <version> and <cstring>)
- tell IWYU not to suggest the headers that json.hpp provides: the amalgamated
  json_view.hpp only includes json.hpp, so including them here would duplicate
  their definitions

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:55 +02:00
Niels Lohmann 50137b71d4 Avoid GCC useless-cast warnings in the integer index tests
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:54 +02:00
Niels Lohmann 8728a31d90 Make json_view.hpp pass include-what-you-use
Mark the includes that are needed by design (json.hpp, the macro
unscope, <tuple> for the tuple_size specialization), take size_t from
<cstring>, and drop the forward declaration of basic_json_document that
the friend declaration makes redundant.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:53 +02:00
Niels Lohmann db3125848e Do not force-inline the view's functions on MSVC
__forceinline makes MSVC report C4714 for function templates it cannot
inline (get<std::string>), an error under /WX. The forced inlining is
only a performance hint, so use plain inline there.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:52 +02:00
Niels Lohmann b91999b9fe Return the first member of duplicate keys from lookups again
Looking up the last member of a duplicate key cannot stop at a match, so
every lookup scanned the whole object (1.6 to 3.4 times slower for small
objects). operator[](key), at, find, value, contains, count, and JSON
pointer resolution return the first member again, as yyjson and simdjson
do; materialize() and get<map>() keep the last value, as parse().

The documentation says so in the feature page and on each lookup page,
and explains how to get the value parse() would give. The integer index
templates and the discarded chaining of operator[] stay.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:51 +02:00
Niels Lohmann ce003f131c Test discarded views with is_discarded() in the operator[] docs and examples
The explicit conversion to bool of basic_json_view is gone.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:51 +02:00
Niels Lohmann a1409b96af Use named documents in the json_view tests that called root() on a temporary
root() of an rvalue document is deleted, because the views would dangle.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:50 +02:00
Niels Lohmann 7ee40e9c96 Replace the removed operator bool of basic_json_view in the header and its tests
A view is tested with is_discarded(); the lookups in at(), value() and
contains(json_pointer) and the unit tests no longer rely on the explicit
conversion to bool.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:49 +02:00
Niels Lohmann 1c925a6ead Make view lookups last-wins and chained access safe
* Lookups (operator[], at, find, contains, count, value, JSON pointers)
  return the last member of a duplicate key, as materialize() and
  parse() keep it.
* operator[] on a discarded view returns a discarded view instead of
  throwing, so v["a"]["b"] is safe for a missing "a".
* operator[] and at() take any integer type (not only int and size_t),
  fixing ambiguous calls with unsigned, long, std::int64_t, ...
* Fix the operator[] documentation, which claimed a discarded view for
  a type mismatch where type_error.305 is thrown.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:48 +02:00
Niels Lohmann 40554fe72a Add element access, iteration, values, and JSON pointers to json_view
Give basic_json_view the read-only access functions of basic_json:
operator[] and at() with keys and indices, front()/back(), find(),
contains(), count(), begin()/end() and cbegin()/cend(), items()
with structured bindings from C++17 on, and type_name().

Exceptions have the ids and messages of the const functions of
basic_json. Where basic_json has undefined behavior the view
answers safely: operator[] with a missing key or an out-of-range
index returns a discarded view, and front()/back() of an empty
container throw invalid_iterator.214. Objects are iterated in
document order, and all members are visited; duplicate-key lookups
find the first member (as yyjson and simdjson do), while parse(),
materialize(), and the map conversions keep the last value, as
parse() does. Keys of up to 16 bytes are compared with two
overlapping loads.

Add value conversions: get<T>()/get_to() for arithmetic types,
bool, nullptr_t, strings (std::basic_string copied,
string_view_t without a copy), BasicJsonType, views, std::vector,
and maps with string keys; get_string() for the string without a
copy; number_token() for the number exactly as written in the
source; value() with keys and JSON pointers; and operator[]/at()/
contains() with JSON pointers. Everything else, including types
with from_json(), goes through materialize() of that subtree.
get<T>() of arithmetic types is inlined down to the conversion, so
reading an integer needs no call.

detail::json_pointer_access exposes a pointer's reference tokens
to code outside basic_json.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:47 +02:00
Niels Lohmann 75efea25b2 Skip the deleted-overload checks of json_view on MSVC 2015
MSVC 2015 does not treat selecting a deleted function in decltype as a
substitution failure, so is_detected reports the deleted
parse/parse_copy/accept/read(ptr, len) overloads as callable and the
static_asserts in "integer arguments do not compile" fail (AppVeyor
VS2015 jobs). Calling them still fails to compile there. Guard the
negative checks as unit-delete_deprecated_functions.cpp does.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:40 +02:00
Niels Lohmann 2f7ec1302d Exclude the view builder test from Infer's reports
Infer's std::unique_ptr model confuses the values that the test's build()
helper returns by value and reports USE_AFTER_DELETE for one of them after
the destructor of an unrelated temporary (ci_infer, 12 reports, all in
tests/src/unit-json_view_builder.cpp). AddressSanitizer is clean on the
same tests.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:39 +02:00
Niels Lohmann 6f67233d88 Do not copy an empty run into an unallocated arena
memcpy() must not be passed a null pointer even for zero bytes, which
arena_run() did for an empty run while the arena was still empty
(ci_test_clang_sanitizer: "null pointer passed as argument 1" in
builder.hpp). Only the slow path can see such a run.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:38 +02:00
Niels Lohmann 90b0752e13 Rename emit() in the view builder for Qt compatibility
Qt defines emit as an empty macro, so a Qt header included before
json_view.hpp broke the declaration of the builder's emit(), see
issue #5865. It is now called push_node().

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:37 +02:00
Niels Lohmann a0c84da14d Include materialize.hpp in the view builder test
The test calls detail::view::materialize(), which only the single header
pulled in; the multiple-header build (JSON_TEST_USING_MULTIPLE_HEADERS)
failed with "'materialize' is not a member of detail::view".

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:36 +02:00
Niels Lohmann 5bf6be7ce1 Address the review: null inline nodes, trait spelling, and max() in json_view
- inline_nodes is null when the document has no inline nodes, instead of
  pointing past the end of the allocation; shrink_to_fit() no longer
  passes it to memcpy() for an empty node array
- is_byte_char and is_wide_char use conjunction/disjunction, which
  astyle leaves unspaced
- append_stream() uses (std::max) for the count read

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:35 +02:00
Niels Lohmann 9c2a11f198 Say that json_view converts integers while parsing
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:35 +02:00
Niels Lohmann abe7d6c02a Apply the review's wording fixes to the json_view documentation
- Copying is disabled because of its cost, not because of two documents
  borrowing the same buffer; the unique_ptr follows from move-only.
- Inputs: streams, FILE*, and wide characters (strings, arrays, pointers)
  are all supported and owned; point to parse_copy() for owned copies.
- The error replay only runs for failing input, builds no values, and is
  skipped with allow_exceptions = false.
- read() writes copied inputs into the previous owned copy.
- Views can dangle like std::string_view, and why there is no
  shared/weak pointer design.
- The 4 GiB limit (16-byte nodes, 16-byte scanner steps), streams being
  read to the end first, which macros apply, why materialize() has no
  source positions, and wording nits (accept() vs basic_json::accept(),
  is_binary, is_discarded, see-also lists, size table, "back into").

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:34 +02:00
Niels Lohmann c67dce9b27 Address the review of the view builder and document
Changes promised in the replies to the review of #5620:

- read() reuses the owned copy: a copied input (an rvalue byte container,
  a stream, FILE*, wide characters, an iterator range) is appended to the
  cleared owned buffer instead of a fresh std::string being move-assigned
  over it. A moved-in std::string still replaces it.
- The error replay runs the library parser with a SAX handler that stores
  nothing and throws the reported exception, instead of
  BasicJsonType::parse(), which built a DOM up to the error.
- Pointers to and arrays of wide characters, and FILE*, take the input
  adapter path (converted to UTF-8 into an owned buffer) instead of
  failing a static_assert. Wide pointers are NUL-terminated.
- Streams are read in blocks through rdbuf() into the buffer, instead of
  character by character through the input adapter; eofbit is set at the
  end, as by the library's stream adapter.
- tape/tape_size/tape_cap are now nodes/node_count/node_capacity.
- The builder's members get descriptive, m_-prefixed names.
- The shallow frame stack is a std::array (in a wrapper with a
  user-provided constructor: -Weffc++ wants the member initialized, and
  value-initializing the 64 frames costs up to 30 % on small documents).
- The value dispatch macro has no NEXT parameter; the switch is left with
  break, and the macro is undefined after its last use.
- literal() and digits_exceed() deduce the length from the string literal;
  each literal has one helper; the two `if (!is_float)` blocks are merged.
- arena_ensure() sits right before the writes; the decode arena is a
  std::vector<char>.
- The builder records the deepest nesting, and materialize() sizes its
  stack from it once (growing it only if a subtree is deeper).
- Comment fixes (hex4, slow_string, the NUL rule, put_utf8, 0 - m, the
  arena resize, grow()) and formatting (macro backslashes, digit cases,
  one power of ten per line, value_t assertion next to is_container()).
- build_with's template parameters follow the builder's order.

Greg's idea of keeping the source offset in `off` and the arena offset in
`next` for escaped strings was measured and not adopted: string access got
7-18 % slower with a select, and 3.5-18 % slower with a branch-free field
pick (Apple M-series, clang -O3), parsing was unchanged.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:33 +02:00
Niels Lohmann fb7bed1e02 Regenerate the amalgamated headers
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:32 +02:00
Niels Lohmann d9dbe94719 Silence a clang-tidy use-after-move finding in the json_view test
const_text is const, so std::move does not move from it; the second call
is part of the const rvalue test.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:31 +02:00
Niels Lohmann 6cab54b75b Read the node array base after emit() in the view builder's open()
emit() moves the node array when it grows, and the subtraction read base
in the same expression, so the order was unspecified. MSVC Release builds
without forced inlining read the old base; the container index then
pointed outside the array and close() wrote out of bounds.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:31 +02:00
Niels Lohmann 3b9ec826e7 Make json_view.hpp pass include-what-you-use
ci_single_binaries runs IWYU with --error on every header. For json_view.hpp
it suggested adding <array> (std::array is used), the headers that json.hpp
already provides (abi_config, abi_macros, input_adapters, json_pointer,
cpp_future, string_concat, value_t, json_fwd), and <version> for
std::nullptr_t, and removing <cstddef>.

- include <array>
- keep <cstddef> (nullptr_t, size_t; IWYU attributes them to <version> and <cstring>)
- tell IWYU not to suggest the headers that json.hpp provides: the amalgamated
  json_view.hpp only includes json.hpp, so including them here would duplicate
  their definitions
- export json.hpp, keep macro_unscope.hpp, and drop the unused forward declaration of the document class

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:30 +02:00
Niels Lohmann 3f577fd5f3 Make max_input_size a function (GCC -Wunused-const-variable)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:29 +02:00
Niels Lohmann f6dac74212 Check the steps of the deep-nesting test separately
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:28 +02:00
Niels Lohmann a92af1ce3c Suppress Flawfinder's read() finding on the deleted overload
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:27 +02:00
Niels Lohmann fce4bf6bc4 Annotate intentional patterns in unit-json_view.cpp for clang-tidy
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:27 +02:00
Niels Lohmann 594703812e Fix clang-tidy findings in unit-json_view_builder.cpp
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:26 +02:00
Niels Lohmann a173ba8a8a Give document_data a user-provided constructor
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:25 +02:00
Niels Lohmann 75c5deb7bc Fuzz json_document with exact-size buffers and parse options
The fuzzer only parsed a std::string with default options. Also parse an
exact-size byte vector, which has no NUL after its last byte and takes the
bounds-checked path, and derive ignore_comments and ignore_trailing_commas
from the first input byte, comparing against json::parse and json::accept
with the same options.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:24 +02:00
Niels Lohmann bbefddb824 Align the banner comment of json_view.hpp
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:23 +02:00
Niels Lohmann 4b28d4fe90 Remove the unused legacy_discarded_value_comparison from abi_config
Nothing reads it: the view does not compare values.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:23 +02:00
Niels Lohmann 552be83b49 State the exact input size limit of json_document
The check rejects inputs of 0xFFFFFFF0 bytes or more, but the exception
message and the documentation said 4 GiB. Name the limit once
(max_input_size), and state 4 GiB minus 16 bytes in the message and the
documentation. Test the limit with a container that only claims the size.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:22 +02:00
Niels Lohmann fecbae0b0d Check the node array size for overflow
reserve(n) and the growth of the index computed n * sizeof(node) without a
check, which wraps around on 32-bit targets for inputs of about 1 GiB and
allocates a too small array. Throw std::bad_alloc for a count beyond the
address space and clamp the growth step to it.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:21 +02:00
Niels Lohmann e8f330682e Store node integers by halves on big-endian targets
The non-little-endian path of the node writer wrote the integer's native
word over len and next, so len got the high half there. Compose and split
the value explicitly (len is the low half, next the high half); the
little-endian path stays a plain memcpy.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:20 +02:00
Niels Lohmann 9cdf8daedd Delete json_document::root() on temporary documents
auto v = json_document::parse(text).root() compiled and left the view
dangling. Delete the overload for rvalue documents, take a named document
in the tests, and document the lifetime rule.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:19 +02:00
Niels Lohmann 8bd5518803 Remove the conversion to bool from json_view
explicit operator bool meant "refers to a value", which silently differs
from what a basic_json converts to. Use !v.is_discarded() instead.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:19 +02:00
Niels Lohmann 4dc226bb6e Copy const rvalue strings and document the accepted inputs
json_document::parse(std::move(const_string)) failed to compile with
"no matching read_kind": only a non-const rvalue std::string can be moved
from. Treat a const rvalue as a copied byte container.

parse() does not accept everything BasicJsonType::parse() does: a FILE*
and pointers to or arrays of wide characters are rejected at compile time.
List the supported inputs instead.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:18 +02:00
Niels Lohmann 4dc626b42b Reject integer lengths in json_document::parse
parse(ptr, len) compiled: len converted to allow_exceptions, and ptr was
read as a C string, past the end of a buffer without a terminating NUL.
Delete the overloads of parse, parse_copy, accept, and read that take an
integer other than bool where the flags are expected.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:17 +02:00
Niels Lohmann 1600ddd651 Exclude libstdc++'s string_view comparison from the sign-change check
basic_string_view::_S_compare stores the difference of two lengths in a
signed difference_type on purpose, which -fsanitize=integer reports for
every comparison with a shorter view (test-json_view_cpp17).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:16 +02:00
Niels Lohmann 28853b19a6 Fix document_data for old Clang (3.4-3.6)
Drop the empty braced NSDMIs of the std::string members: old Clang
rejects the defaulted constructor when it is used by a member
initializer before the end of the class definition.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:15 +02:00
Niels Lohmann 3efa4ce0f0 Add json_document and json_view: node index, parser, and document
Add json_document and json_view, a read-only, zero-copy index of a
JSON text, as the first public slice of the zero-copy view (#5295).

A parse produces a flat array of 16-byte nodes in document order,
one per value and one per object key. Strings stay in the source
text; escaped strings are decoded into an arena. Integers are
converted while their digits are in the cache; floats keep only
their digit layout and are converted on read. Containers store the
size of their subtree, so a reader can step over one in constant
time. A document makes a handful of allocations, however many
values it has.

The parser accepts exactly what json::parse accepts, with every
combination of ignore_comments and ignore_trailing_commas, with and
without a trailing NUL, and under JSON_STRICT_NUL_HANDLING. It is
portable C++11 and does not depend on byte order.

basic_json_document adds parse, parse_copy, accept, read (reuses a
document's memory), root, is_discarded, source, owns_source,
node_count, memory_usage, and shrink_to_fit. basic_json_view adds
type, the is_* queries, operator bool, size, empty, materialize,
and source_offset. A parse error throws the same exception
basic_json::parse would throw for the same input, message and
position included.

detail::abi_config keeps JSON_STRICT_NUL_HANDLING and
JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON readable after json.hpp
undefines them, in the ABI namespace so they always match the
basic_json in use.

A NUL byte that ends a // comment is the end of the input, as in
parse() since #5696.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-10-20 08:48:15 +02:00
155 changed files with 3612 additions and 474671 deletions

No files matched your search

+3 -10
View File
@@ -108,16 +108,9 @@ The tests are located in [`tests/src/unit-*.cpp`](https://github.com/nlohmann/js
are structured along the features of the library or the nature of the tests. Usually, it should be clear from the
context which existing file needs to be extended, and only very few cases require creating new test files.
When fixing a bug, edit `unit-regression3.cpp` and add a section referencing the fixed issue. Tests that need C++17 or
C++20 go into `unit-regression3-cpp17.cpp` or `unit-regression3-cpp20.cpp` instead, so the large file is not rebuilt
for every C++ standard. `unit-regression2.cpp` holds the older tests; the two files exist because a single one grew
large enough for the MinGW linker to fail relocating it, so please keep adding to the smaller file rather than growing
the larger one.
To keep compile times down, several test files are compiled together as one translation unit. This sets a few rules
for test files, such as giving file-scope helpers file-specific names. See the README in the
[`tests`](https://github.com/nlohmann/json/tree/develop/tests) folder for how the tests are built and what to keep in
mind when adding them.
When fixing a bug, edit `unit-regression3.cpp` and add a section referencing the fixed issue.
`unit-regression2.cpp` holds the older tests; the two files exist because a single one grew large enough for the
MinGW linker to fail relocating it, so please keep adding to the smaller file rather than growing the larger one.
#### Exceptions
+3 -3
View File
@@ -63,12 +63,12 @@ labels:
- "single_include/nlohmann/json_view\\.hpp"
- "tests/src/unit-json_view.*"
- "tests/src/json_view_test_helpers\\.hpp"
- "tests/src/fuzzer-(parse_json_view|json_view_image)\\.cpp"
- "tests/src/fuzzer-parse_json_view\\.cpp"
- "tools/amalgamate/config_json_view\\.json"
- "docs/mkdocs/docs/features/json_view\\.md"
- "docs/mkdocs/docs/api/basic_json_(document|view)/.*"
- "docs/mkdocs/docs/api/(ordered_)?json_(editable_)?(document|view)\\.md"
- "docs/mkdocs/docs/examples/(basic_json_(document|view)__|(ordered_)?json_(editable_)?(document|view)).*"
- "docs/mkdocs/docs/api/(ordered_)?json_(document|view)\\.md"
- "docs/mkdocs/docs/examples/(basic_json_(document|view)__|(ordered_)?json_(document|view)).*"
- "tests/benchmarks/src/benchmarks_view\\.cpp"
- label: "aspect: json_view"
+15 -14
View File
@@ -77,12 +77,13 @@ jobs:
cd $MAIN_DIR
# Built from this PR's own tools/macro_builder/main.cpp, not a
# develop checkout: unlike amalgamate.py, this tool has no other
# source of truth to check against (its own README documents that
# it must reproduce macro_scope.hpp byte for byte), so there is
# nothing to gain from checking out a separate copy, and doing so
# would make this step fail on a PR that adds support for a new
# dispatch table until that PR itself merges to develop.
# develop checkout: unlike amalgamate.py and generate_natvis.py,
# this tool has no other source of truth to check against (its
# own README documents that it must reproduce macro_scope.hpp
# byte for byte), so there is nothing to gain from checking out a
# separate copy, and doing so would make this step fail on a PR
# that adds support for a new dispatch table until that PR itself
# merges to develop.
TMPDIR=$(mktemp -d ./macro_builder_check.XXXXXX)
c++ -std=c++11 tools/macro_builder/main.cpp -o "$TMPDIR/macro_builder"
"$TMPDIR/macro_builder" > "$TMPDIR/paste.hpp"
@@ -145,14 +146,14 @@ jobs:
- name: Regenerate nlohmann_json.natvis
run: |
cd $MAIN_DIR
# The generator and its template are what this check verifies the
# committed natvis file against (which namespaces it lists, and
# what each entry looks like), so run this PR's copy rather than
# the develop checkout's: with develop's, a PR that changes either
# could never pass. generate_natvis.py loads the template from its
# own directory. Like the macro_builder step above, this only runs
# PR content in a job without credentials.
python3 tools/generate_natvis/generate_natvis.py $MAIN_DIR
# The template is what this check verifies the committed natvis
# file against, so take it from this PR rather than from the
# develop checkout: generate_natvis.py loads the template from its
# own directory, and with develop's copy a PR that changes the
# template could never pass. Like the macro_builder step above,
# this only renders PR content in a job without credentials.
cp tools/generate_natvis/nlohmann_json.natvis.j2 $NATVIS_TOOL_DIR/
python3 $NATVIS_TOOL_DIR/generate_natvis.py $MAIN_DIR
- name: Build patch and check for differences
id: diff
+1 -1
View File
@@ -50,7 +50,7 @@ jobs:
persist-credentials: false
- name: "Run analysis"
uses: ossf/scorecard-action@fa7117db83c52d787ae5a89f350999701505d78e # v2.5.0
uses: ossf/scorecard-action@2d1146689b8cda280b9bc96326124645441f03bc # v2.4.4
with:
results_file: results.sarif
results_format: sarif
-3
View File
@@ -60,6 +60,3 @@ MODULE.bazel.lock
/json.tar.xz.asc
/release_files/
/single_include/nlohmann/*.asc
# GCC module cache
/gcm.cache/
-5
View File
@@ -70,10 +70,7 @@ cc_library(
"include/nlohmann/detail/view/builder.hpp",
"include/nlohmann/detail/view/compare.hpp",
"include/nlohmann/detail/view/document_data.hpp",
"include/nlohmann/detail/view/edit.hpp",
"include/nlohmann/detail/view/edit_storage.hpp",
"include/nlohmann/detail/view/errors.hpp",
"include/nlohmann/detail/view/image.hpp",
"include/nlohmann/detail/view/input.hpp",
"include/nlohmann/detail/view/iterator.hpp",
"include/nlohmann/detail/view/lookup.hpp",
@@ -82,11 +79,9 @@ cc_library(
"include/nlohmann/detail/view/materialize.hpp",
"include/nlohmann/detail/view/node.hpp",
"include/nlohmann/detail/view/number.hpp",
"include/nlohmann/detail/view/object_index.hpp",
"include/nlohmann/detail/view/pointer.hpp",
"include/nlohmann/detail/view/scan.hpp",
"include/nlohmann/detail/view/serializer.hpp",
"include/nlohmann/detail/view/simd.hpp",
"include/nlohmann/detail/view/string_ref.hpp",
"include/nlohmann/detail/view/value.hpp",
"include/nlohmann/json.hpp",
-1
View File
@@ -1410,7 +1410,6 @@ THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR I
- The class contains parts of [Google Abseil](https://github.com/abseil/abseil-cpp) which is licensed under the [Apache 2.0 License](https://opensource.org/licenses/Apache-2.0).
- The class contains an adapted version of the Eisel-Lemire algorithm, its table of powers of five, and its digit comparison for long numbers from [fast_float](https://github.com/fastfloat/fast_float) by Daniel Lemire and contributors, which is available under the [MIT License](https://opensource.org/licenses/MIT) (used here), the Apache 2.0 License, and the Boost Software License. Copyright &copy; 2021 The fast_float authors
- The view's parser (`<nlohmann/json_view.hpp>`) contains techniques and code adapted from [yyjson](https://github.com/ibireme/yyjson) by YaoYuan, which is licensed under the [MIT License](https://opensource.org/licenses/MIT) (see above): table-driven decoding of `\u` escapes and fixed-offset unrolled checks.
- The view's parser (`<nlohmann/json_view.hpp>`) validates non-ASCII strings with the vector UTF-8 check of [simdjson](https://github.com/simdjson/simdjson) by Daniel Lemire, Geoff Langdale, John Keiser, and contributors (its "lookup4" algorithm and tables, after J. Keiser and D. Lemire, "Validating UTF-8 In Less Than One Instruction Per Byte", 2021), which is available under the [MIT License](https://opensource.org/licenses/MIT) (used here) and the Apache 2.0 License. Copyright &copy; 2018-2025 The simdjson authors
<img align="right" src="https://git.fsfe.org/reuse/reuse-ci/raw/branch/master/reuse-horizontal.png" alt="REUSE Software">
+3 -3
View File
@@ -890,11 +890,11 @@ add_custom_target(ci_icpx
# The following tests are excluded as they trigger known nvc++ 25.5 defects (not
# library bugs); see https://github.com/nlohmann/json for tracking. Only the
# affected language-standard variants are excluded so coverage is otherwise kept:
# - test-comparison-cpp20_cpp20, test-comparison_legacy-cpp20_cpp20
# - test-comparison_cpp20, test-comparison_legacy_cpp20
# miscompiles cross-type/<=> comparison (e.g. `-17 <= null`)
# - test-constructor1_cpp11
# std::initializer_list lifetime bug -> SIGSEGV
# - test-deserialization-cpp20_cpp20
# - test-deserialization_cpp20
# mangles the UTF-8 u8"" string literal in the char8_t (C++20) section
add_custom_target(ci_nvhpc
COMMAND ${CMAKE_COMMAND}
@@ -906,7 +906,7 @@ add_custom_target(ci_nvhpc
COMMAND ${CMAKE_COMMAND} --build ${PROJECT_BINARY_DIR}/build_nvhpc
# the pipes are escaped so the surrounding shell passes them to ctest verbatim
# instead of treating them as shell pipe operators
COMMAND cd ${PROJECT_BINARY_DIR}/build_nvhpc && ${CMAKE_CTEST_COMMAND} --parallel ${N} --exclude-regex "test-comparison-cpp20_cpp20\\|test-comparison_legacy-cpp20_cpp20\\|test-constructor1_cpp11\\|test-deserialization-cpp20_cpp20" --output-on-failure
COMMAND cd ${PROJECT_BINARY_DIR}/build_nvhpc && ${CMAKE_CTEST_COMMAND} --parallel ${N} --exclude-regex "test-comparison_cpp20\\|test-comparison_legacy_cpp20\\|test-constructor1_cpp11\\|test-deserialization_cpp20" --output-on-failure
COMMENT "Compile and test with NVIDIA HPC SDK (nvc++)"
)
+1 -1
View File
@@ -1 +1 @@
cppcheck==1.5.3
cppcheck==1.5.2
+39 -366
View File
@@ -109,55 +109,6 @@ function(_json_test_apply_test_properties test_target properties_target)
endif()
endfunction()
# for internal use by _json_test_add_test() and _json_test_add_unity_batch():
# registers the CTest test <test_name>_cpp<cxx_standard> (plus its Valgrind
# variant), which runs the executable target <test_target> with the arguments
# in ARGN, and applies the test properties of the test- and standard-specific
# interface targets
function(_json_test_register_test test_name test_target cxx_standard)
set(ctest_name ${test_name}_cpp${cxx_standard})
if (JSON_FastTests)
add_test(NAME ${ctest_name}
COMMAND ${test_target} ${DOCTEST_TEST_FILTER} ${ARGN}
WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
)
else()
add_test(NAME ${ctest_name}
COMMAND ${test_target} ${DOCTEST_TEST_FILTER} ${ARGN} --no-skip
WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
)
endif()
set_tests_properties(${ctest_name} PROPERTIES LABELS "all" FIXTURES_REQUIRED TEST_DATA)
# apply standard-specific test properties
if(TARGET _json_test_interface__cpp_${cxx_standard})
_json_test_apply_test_properties(${ctest_name} _json_test_interface__cpp_${cxx_standard})
endif()
# apply test-specific test properties
if(TARGET _json_test_interface_${test_name})
_json_test_apply_test_properties(${ctest_name} _json_test_interface_${test_name})
endif()
# apply test- and standard-specific test properties
if(TARGET _json_test_interface_${test_name}_cpp_${cxx_standard})
_json_test_apply_test_properties(${ctest_name}
_json_test_interface_${test_name}_cpp_${cxx_standard}
)
endif()
if(JSON_Valgrind)
add_test(NAME ${ctest_name}_valgrind
COMMAND ${memcheck_command} $<TARGET_FILE:${test_target}> ${DOCTEST_TEST_FILTER} ${ARGN}
WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
)
set_tests_properties(${ctest_name}_valgrind PROPERTIES
LABELS "valgrind" FIXTURES_REQUIRED TEST_DATA
)
endif()
endfunction()
# for internal use by json_test_add_test_for()
function(_json_test_add_test test_name file main cxx_standard)
set(test_target ${test_name}_cpp${cxx_standard})
@@ -192,7 +143,45 @@ function(_json_test_add_test test_name file main cxx_standard)
)
endif()
_json_test_register_test(${test_name} ${test_target} ${cxx_standard})
if (JSON_FastTests)
add_test(NAME ${test_target}
COMMAND ${test_target} ${DOCTEST_TEST_FILTER}
WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
)
else()
add_test(NAME ${test_target}
COMMAND ${test_target} ${DOCTEST_TEST_FILTER} --no-skip
WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
)
endif()
set_tests_properties(${test_target} PROPERTIES LABELS "all" FIXTURES_REQUIRED TEST_DATA)
# apply standard-specific test properties
if(TARGET _json_test_interface__cpp_${cxx_standard})
_json_test_apply_test_properties(${test_target} _json_test_interface__cpp_${cxx_standard})
endif()
# apply test-specific test properties
if(TARGET _json_test_interface_${test_name})
_json_test_apply_test_properties(${test_target} _json_test_interface_${test_name})
endif()
# apply test- and standard-specific test properties
if(TARGET _json_test_interface_${test_name}_cpp_${cxx_standard})
_json_test_apply_test_properties(${test_target}
_json_test_interface_${test_name}_cpp_${cxx_standard}
)
endif()
if(JSON_Valgrind)
add_test(NAME ${test_target}_valgrind
COMMAND ${memcheck_command} $<TARGET_FILE:${test_target}> ${DOCTEST_TEST_FILTER}
WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
)
set_tests_properties(${test_target}_valgrind PROPERTIES
LABELS "valgrind" FIXTURES_REQUIRED TEST_DATA
)
endif()
endfunction()
#############################################################################
@@ -211,14 +200,6 @@ endfunction()
# Use NAME <name> to override the filename-derived test name.
# Use FORCE to create the test regardless of the file containing
# JSON_HAS_CPP_<version_number>.
#
# Tests that depend on the C++ standard (e.g., because they use the macros
# JSON_HAS_FILESYSTEM, JSON_HAS_RANGES, or JSON_HAS_THREE_WAY_COMPARISON)
# should not make the whole of a large unit-foo.cpp be rebuilt for every
# standard. Put them into a separate file unit-foo-cpp<NN>.cpp (see, e.g.,
# unit-items-cpp17.cpp) which wraps its content in #ifdef JSON_HAS_CPP_<NN>.
# Then, unit-foo.cpp itself contains no JSON_HAS_CPP_<NN> and is only built for
# C++11.
# Test targets are linked against <main>.
# CXX_STANDARDS defaults to "11".
#############################################################################
@@ -262,314 +243,6 @@ function(json_test_add_test_for file)
endforeach()
endfunction()
# for internal use by json_test_add_unity_tests(): sets <result> to whether the
# (absolute) <file> is built for <cxx_standard>; same rule as in
# json_test_add_test_for(): C++11 always, others only if the file contains
# JSON_HAS_CPP_<cxx_standard> or <force> is set
function(_json_test_unity_applies file cxx_standard force result)
set(${result} TRUE PARENT_SCOPE)
if(NOT ("${cxx_standard}" STREQUAL 11 OR force))
file(READ ${file} file_content)
string(FIND "${file_content}" JSON_HAS_CPP_${cxx_standard} has_cpp_found)
if(${has_cpp_found} EQUAL -1)
set(${result} FALSE PARENT_SCOPE)
endif()
endif()
endfunction()
# for internal use by json_test_add_unity_tests(): creates the executable
# test-unity-<batch_name>_cpp<cxx_standard> from the (absolute) source files
# in ARGN, which are #include-d by a generated source file, and registers one
# CTest test per source file that runs only the test cases of that file; if
# <private> is true, the generated file defines JSON_TESTS_PRIVATE first
function(_json_test_add_unity_batch batch_name cxx_standard main private)
set(batch_target test-unity-${batch_name}_cpp${cxx_standard})
set(batch_source ${PROJECT_BINARY_DIR}/tests/unity/${batch_target}.cpp)
set(batch_content "// generated by cmake/test.cmake; do not edit\n")
if(private)
string(APPEND batch_content "// at least one file of this batch needs access to private members of the library\n")
string(APPEND batch_content "#define JSON_TESTS_PRIVATE\n")
# the files define the macro again; mark this definition as used, as
# -Wunused-macros reports an unused definition when it is redefined
string(APPEND batch_content "#ifdef JSON_TESTS_PRIVATE\n#endif\n")
endif()
foreach(file ${ARGN})
string(APPEND batch_content "#include \"${file}\" // NOLINT(bugprone-suspicious-include)\n")
endforeach()
# only touch the generated file if it changed to keep incremental builds incremental
set(old_content "")
if(EXISTS ${batch_source})
file(READ ${batch_source} old_content)
endif()
if(NOT "${old_content}" STREQUAL "${batch_content}")
file(WRITE ${batch_source} "${batch_content}")
endif()
add_executable(${batch_target} ${batch_source})
target_link_libraries(${batch_target} PRIVATE ${main})
set_target_properties(${batch_target} PROPERTIES
CXX_STANDARD ${cxx_standard}
CXX_STANDARD_REQUIRED ON
)
if(TARGET _json_test_interface__cpp_${cxx_standard})
target_link_libraries(${batch_target} PRIVATE _json_test_interface__cpp_${cxx_standard})
endif()
# rebuild the batch when one of its files changes, and show the files in IDEs;
# files that are also built standalone (VARIANT_FILES) are left out, as
# HEADER_FILE_ONLY is a per-file property and would also affect those targets
set_source_files_properties(${batch_source} PROPERTIES OBJECT_DEPENDS "${ARGN}")
foreach(file ${ARGN})
if(NOT file IN_LIST _json_test_unity_variant_files)
set_source_files_properties(${file} PROPERTIES HEADER_FILE_ONLY ON)
target_sources(${batch_target} PRIVATE ${file})
endif()
endforeach()
foreach(file ${ARGN})
get_filename_component(file_basename ${file} NAME_WE)
string(REGEX REPLACE "unit-(.+)" "test-\\1" test_name ${file_basename})
# run only the test cases defined in this file (and in the shared
# make_test_data_available.hpp if the file uses it)
file(READ ${file} file_content)
set(source_filter "--source-file=*${file_basename}.cpp")
string(FIND "${file_content}" make_test_data_available.hpp uses_test_data)
if(NOT ${uses_test_data} EQUAL -1)
string(APPEND source_filter ",*make_test_data_available.hpp")
endif()
_json_test_register_test(${test_name} ${batch_target} ${cxx_standard} "${source_filter}")
endforeach()
endfunction()
#############################################################################
# json_test_add_unity_tests(
# FILES <files>...
# MAIN <main>
# [CXX_STANDARDS <version_number>...] [FORCE]
# [BATCH_SIZE <size>]
# [GROUPS <group>...]
# [VARIANT_FILES <files>...])
#
# Like calling json_test_add_test_for(<file> MAIN <main> ...) for each of the
# <files>, but compiles several files together to speed up the build: for each
# C++ standard, the files are split into batches of <size> files (default: 8)
# and each batch is built as a single executable
#
# test-unity-<pool><index>_cpp<version_number>
#
# whose generated source file #include-s the files of the batch (so they share
# the template instantiations of the library). The tests are still named
# test-foo_cpp<version_number>, one per file, but run the batch executable with
# a doctest filter that selects the test cases of that file only.
#
# Files are only batched with files that agree on the macros defined before
# the library is included: files that define at most JSON_TESTS_PRIVATE (or
# macros derived from global compile definitions) form the pools "plain" and
# "private". All other files are added with json_test_add_test_for() as usual,
# as are files with test-specific build settings (see
# json_test_set_test_options()) and the files listed in the explicit
# exclusion list below.
# Each <group> names a list variable json_test_unity_group_<group> of test file
# stems (file names without "unit-" and ".cpp"). The batchable files of a group
# are compiled together (regardless of BATCH_SIZE) as one executable
#
# test-unity-<group>_cpp<version_number>
#
# so that related tests, which instantiate the same templates, share one
# translation unit. A group may mix the pools "plain" and "private"; if any of
# its files needs JSON_TESTS_PRIVATE, the whole group is built with it. Files
# that cannot be batched stay standalone even if they are listed in a group.
# Files in no group are batched by BATCH_SIZE as described above.
# <files> in VARIANT_FILES are also built standalone with other settings, so
# they are not marked as header-only sources of the batch.
#############################################################################
function(json_test_add_unity_tests)
cmake_parse_arguments(args "FORCE" "MAIN;BATCH_SIZE" "FILES;CXX_STANDARDS;VARIANT_FILES;GROUPS" ${ARGN})
if("${args_MAIN}" STREQUAL "")
message(FATAL_ERROR "Required argument MAIN <main> missing.")
endif()
if("${args_BATCH_SIZE}" STREQUAL "")
set(args_BATCH_SIZE 8)
endif()
if("${args_CXX_STANDARDS}" STREQUAL "")
set(args_CXX_STANDARDS 11)
endif()
if(args_FORCE)
set(force FORCE)
else()
set(force "")
endif()
set(_json_test_unity_variant_files "")
foreach(file ${args_VARIANT_FILES})
get_filename_component(file ${file} ABSOLUTE)
list(APPEND _json_test_unity_variant_files ${file})
endforeach()
# files that must not be merged into a batch: unit-32bit.cpp is only built
# for 32bit targets, and unit-no-macro-leak.cpp checks that including the
# library defines no unprefixed macro, which any other file would disturb
set(standalone_files unit-32bit.cpp unit-no-macro-leak.cpp)
set(harmless_macros "^(DOCTEST_.*|SKIP_TESTS_FOR_.*|JSON_TEST_DEPRECATED_FUNCTIONS_DELETED|JSON_TEST_STRICT_NUL_HANDLING_ENABLED|JSON_TEST_STRINGIZE)$")
# classify the files
set(plain_files "")
set(private_files "")
set(standalone_abs_files "")
foreach(file ${args_FILES})
get_filename_component(file_name ${file} NAME)
get_filename_component(file_basename ${file} NAME_WE)
string(REGEX REPLACE "unit-(.+)" "test-\\1" test_name ${file_basename})
set(batchable TRUE)
if(file_name IN_LIST standalone_files)
set(batchable FALSE)
endif()
if(TARGET _json_test_interface_${test_name})
set(batchable FALSE)
endif()
foreach(cxx_standard ${args_CXX_STANDARDS})
if(TARGET _json_test_interface_${test_name}_cpp_${cxx_standard})
set(batchable FALSE)
endif()
endforeach()
# files that suppress GCC's -Wnoexcept around their include of the
# library: the suppression has no effect once another file of the
# batch included the library first
if(batchable)
file(READ ${file} file_content)
string(FIND "${file_content}" "SUPPRESS_WARNING(\"-Wnoexcept\")" suppresses_noexcept)
if(NOT ${suppresses_noexcept} EQUAL -1)
set(batchable FALSE)
endif()
endif()
set(pool plain)
if(batchable)
# collect the macros (un)defined before the library is included
file(READ ${file} file_content)
string(FIND "${file_content}" "#include <nlohmann/" include_position)
if(NOT ${include_position} EQUAL -1)
string(SUBSTRING "${file_content}" 0 ${include_position} file_content)
endif()
string(REGEX MATCHALL "(^|\n)[ \t]*#[ \t]*(define|undef)[ \t]+[A-Za-z_0-9]+" directives "${file_content}")
foreach(directive ${directives})
string(REGEX REPLACE "^.*[ \t]([A-Za-z_0-9]+)$" "\\1" macro "${directive}")
if(macro MATCHES "${harmless_macros}")
continue()
elseif("${macro}" STREQUAL JSON_TESTS_PRIVATE)
set(pool private)
else()
set(batchable FALSE)
endif()
endforeach()
endif()
get_filename_component(file_abs ${file} ABSOLUTE)
if(NOT batchable)
list(APPEND standalone_abs_files ${file_abs})
json_test_add_test_for(${file} MAIN ${args_MAIN} CXX_STANDARDS ${args_CXX_STANDARDS} ${force})
continue()
endif()
get_filename_component(file ${file} ABSOLUTE)
list(APPEND ${pool}_files ${file})
endforeach()
# resolve the explicit groups: group_<name>_files are the (absolute) files
# of the group, group_<name>_private tells whether one of them is private
set(grouped_files "")
foreach(group ${args_GROUPS})
if(NOT DEFINED json_test_unity_group_${group})
message(FATAL_ERROR "Unity test group '${group}' is not defined (json_test_unity_group_${group}).")
endif()
set(group_${group}_files "")
set(group_${group}_private FALSE)
foreach(stem ${json_test_unity_group_${group}})
# check against the source directory, because FILES may be filtered (JSON_TestShard)
get_filename_component(file ${CMAKE_CURRENT_SOURCE_DIR}/src/unit-${stem}.cpp ABSOLUTE)
if(NOT EXISTS ${file})
message(FATAL_ERROR "Unity test group '${group}' lists '${stem}', but ${file} does not exist.")
endif()
if(file IN_LIST grouped_files)
message(FATAL_ERROR "Unity test file unit-${stem}.cpp is listed in more than one group (second: '${group}').")
endif()
list(APPEND grouped_files ${file})
if(file IN_LIST standalone_abs_files)
message(STATUS "Unity test group '${group}': unit-${stem}.cpp cannot be batched and stays standalone")
elseif(file IN_LIST plain_files)
list(APPEND group_${group}_files ${file})
list(REMOVE_ITEM plain_files ${file})
elseif(file IN_LIST private_files)
list(APPEND group_${group}_files ${file})
list(REMOVE_ITEM private_files ${file})
set(group_${group}_private TRUE)
endif()
endforeach()
endforeach()
foreach(cxx_standard ${args_CXX_STANDARDS})
if(NOT compiler_supports_cpp_${cxx_standard})
continue()
endif()
# explicit groups: one batch per group
foreach(group ${args_GROUPS})
set(batch_files "")
foreach(file ${group_${group}_files})
_json_test_unity_applies(${file} ${cxx_standard} "${force}" applies)
if(applies)
list(APPEND batch_files ${file})
endif()
endforeach()
if(batch_files)
_json_test_add_unity_batch(${group} ${cxx_standard} ${args_MAIN} ${group_${group}_private} ${batch_files})
endif()
endforeach()
# remaining files: batches of BATCH_SIZE files per pool
foreach(pool plain private)
set(is_private FALSE)
if(pool STREQUAL private)
set(is_private TRUE)
endif()
set(batch_files "")
set(batch_count 0)
set(batch_index 0)
foreach(file ${${pool}_files})
_json_test_unity_applies(${file} ${cxx_standard} "${force}" applies)
if(NOT applies)
continue()
endif()
list(APPEND batch_files ${file})
math(EXPR batch_count "${batch_count} + 1")
if(batch_count EQUAL args_BATCH_SIZE)
_json_test_add_unity_batch(${pool}${batch_index} ${cxx_standard} ${args_MAIN} ${is_private} ${batch_files})
set(batch_files "")
set(batch_count 0)
math(EXPR batch_index "${batch_index} + 1")
endif()
endforeach()
if(batch_files)
_json_test_add_unity_batch(${pool}${batch_index} ${cxx_standard} ${args_MAIN} ${is_private} ${batch_files})
endif()
endforeach()
endforeach()
endfunction()
#############################################################################
# json_test_should_build_32bit_test(
# <build_32bit_var> <build_32bit_only_var> <input>)
+1 -3
View File
@@ -123,6 +123,4 @@ Linear in the size of the JSON value `j`.
that is not valid UTF-8 unchanged, as before; `strict` (the default if
[`JSON_STRICT_BINARY_UTF8`](../macros/json_strict_binary_utf8.md) is enabled) throws `type_error.316`.
- Throws `type_error.321` for a discarded value since version 3.13.0; previously, a discarded value nested in an
array or object was silently skipped, producing invalid BJData.
- Writes unsigned integers wider than 64 bits as high-precision numbers since version 3.13.0; previously, they were
silently truncated to 64 bits.
array or object was silently skipped, producing invalid BJData.
+2 -3
View File
@@ -37,9 +37,8 @@ With (2), the bytes written before the exception remain in the output adapter.
## Exceptions
- Throws [out_of_range.407](../../home/exceptions.md#jsonexceptionout_of_range407) if `j` contains an integer outside
the range of int64 (an unsigned integer above 9223372036854775807, or, with a number type wider than 64 bits, any
integer beyond int64), which BON8 cannot represent
- Throws [out_of_range.407](../../home/exceptions.md#jsonexceptionout_of_range407) if `j` contains an unsigned integer
above 9223372036854775807, which BON8 cannot represent
- Throws [type_error.316](../../home/exceptions.md#jsonexceptiontype_error316) if `j` contains a string that is not
valid UTF-8
- Throws [type_error.321](../../home/exceptions.md#jsonexceptiontype_error321) if `j` or a value nested in it is
@@ -47,10 +47,6 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
- Throws [`type_error.317`](../../home/exceptions.md#jsonexceptiontype_error317) if the top-level type of the JSON value
is not an object; example: `"to serialize to BSON, top-level type must be object, but is string"`
- Throws [`out_of_range.407`](../../home/exceptions.md#jsonexceptionout_of_range407) if `j` contains a signed integer
outside the range of int64 or an unsigned integer outside the range of uint64, which is only possible with a number
type wider than 64 bits; example:
`"integer number 9223372036854775808 cannot be represented by BSON as it does not fit int64"`
- Throws [`out_of_range.409`](../../home/exceptions.md#jsonexceptionout_of_range409) if a key in the JSON object contains
a null byte (code point U+0000); example: `"BSON key cannot contain code point U+0000 (at byte 2)"`
- Throws [`out_of_range.412`](../../home/exceptions.md#jsonexceptionout_of_range412) if the length of a document, array,
@@ -123,5 +119,3 @@ pass before anything is written.
that is not valid UTF-8 unchanged, as before; `strict` (the default if
[`JSON_STRICT_BINARY_UTF8`](../macros/json_strict_binary_utf8.md) is enabled) throws `type_error.316` before anything
is written.
- Throws `out_of_range.407` for integers that do not fit 64 bits since version 3.13.0; previously, integers of a
number type wider than 64 bits were silently truncated.
@@ -46,9 +46,6 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
## Exceptions
- Throws [`out_of_range.407`](../../home/exceptions.md#jsonexceptionout_of_range407) if `j` contains an integer
outside [-2^64, 2^64-1], which is only possible with a number type wider than 64 bits; example:
`"integer number 18446744073709551616 cannot be represented by CBOR as it does not fit [-2^64, 2^64-1]"`
- Throws [type_error.316](../../home/exceptions.md#jsonexceptiontype_error316) if a string or object key in `j` is
not valid UTF-8 and `error_handler` is `strict` (the default only if
[`JSON_STRICT_BINARY_UTF8`](../macros/json_strict_binary_utf8.md) is enabled)
@@ -93,5 +90,3 @@ Linear in the size of the JSON value `j`.
[`JSON_STRICT_BINARY_UTF8`](../macros/json_strict_binary_utf8.md) is enabled) throws `type_error.316`.
- Throws `type_error.321` for a discarded value since version 3.13.0; previously, a discarded value nested in an
array or object was silently skipped, producing invalid CBOR.
- Throws `out_of_range.407` for integers that do not fit 64 bits since version 3.13.0; previously, integers of a
number type wider than 64 bits were silently truncated.
@@ -46,9 +46,6 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
## Exceptions
- Throws [`out_of_range.407`](../../home/exceptions.md#jsonexceptionout_of_range407) if `j` contains an integer
outside [-2^63, 2^64-1], which is only possible with a number type wider than 64 bits; example:
`"integer number 18446744073709551616 cannot be represented by MessagePack as it does not fit [-2^63, 2^64-1]"`
- Throws [`out_of_range.412`](../../home/exceptions.md#jsonexceptionout_of_range412) if the length of a string, binary
value, array, or object exceeds 4294967295, the maximum MessagePack can store; example:
`"MessagePack length 4294967296 exceeds maximum of 4294967295"`
@@ -115,5 +112,3 @@ Linear in the size of the JSON value `j`.
`number_unsigned_t`.
- Throws `type_error.321` for a discarded value since version 3.13.0; previously, a discarded value nested in an
array or object was silently skipped, producing invalid MessagePack.
- Throws `out_of_range.407` for integers that do not fit 64 bits since version 3.13.0; previously, integers of a
number type wider than 64 bits were silently truncated.
@@ -1,129 +0,0 @@
# <small>nlohmann::basic_json_document::</small>erase
```cpp
// (1)
std::size_t erase(view_type object, string_view_t key);
// (2)
template<typename I>
void erase(view_type array, I idx);
// (3)
std::size_t erase(const json_pointer& ptr);
```
Only an **editable** document (`#!cpp Editable == true`, e.g. [`json_editable_document`](../json_editable_document.md))
has `erase`; calling it on a read-only `basic_json_document` fails to compile (`#!cpp static_assert`).
1. Removes every member of `object` whose key is `key` (see [Notes](#notes) on duplicate keys) and returns how many
were removed; `#!cpp 0` if `object` has no member with this key.
2. Removes the element at index `idx` of `array`, which must already exist (`#!cpp idx < array.size()`).
3. Removes the value the JSON pointer `ptr` refers to, relative to [`root()`](root.md). `ptr` must not be empty
([`root()`](root.md) itself cannot be erased), and the *parent* of the target must already exist. The target is
removed as in 1. if the parent is an object (returns the number of removed members, `#!cpp 0` or more) and as in 2.
if it is an array (always returns `#!cpp 1`).
## Template parameters
`I`
: an integral type other than `#!cpp bool`, deduced (overloads taking a `#!cpp bool` or a non-integral type for
`idx` do not participate in overload resolution).
## Parameters
`object` (in)
: the object to remove a member of
`array` (in)
: the array to remove an element of
`key` (in)
: the key of the member(s) to remove
`idx` (in)
: the index of the element to remove; a negative value throws (see [Exceptions](#exceptions))
`ptr` (in)
: a JSON pointer to the value to remove, relative to `root()`
## Return value
1. the number of removed members (`#!cpp 0` if `object` had none with this `key`)
2. (nothing)
3. the number of removed values (`#!cpp 0` or more for an object member, always `#!cpp 1` for an array element)
## Exceptions
1. Throws [`type_error.307`](../../home/exceptions.md#jsonexceptiontype_error307) if `object` is not an object -- the
same message [`BasicJsonType::erase`](../basic_json/erase.md) throws for the same type.
2. Throws `type_error.307` if `array` is not an array. Throws
[`out_of_range.401`](../../home/exceptions.md#jsonexceptionout_of_range401) if `idx` is negative, or if
`#!cpp idx >= array.size()`.
3. Throws [`out_of_range.405`](../../home/exceptions.md#jsonexceptionout_of_range405) ("JSON pointer has no parent")
if `ptr` is empty. Throws what [`at`](../basic_json_view/at.md) throws (overload 3) for resolving `ptr`'s parent.
For the last reference token itself: if the parent is an array, throws what 2. throws for an index that is out of
range, or, for a token that is not a valid array index,
[`parse_error.106`](../../home/exceptions.md#jsonexceptionparse_error106) (a leading `#!cpp '0'`),
[`parse_error.109`](../../home/exceptions.md#jsonexceptionparse_error109) (not a number),
[`out_of_range.410`](../../home/exceptions.md#jsonexceptionout_of_range410) (too large for `size_type`), or
[`out_of_range.404`](../../home/exceptions.md#jsonexceptionout_of_range404) (an empty token); otherwise (an
object, or a primitive value the pointer's parent resolves to) throws what 1. throws.
Overloads 1. and 2. also throw [`invalid_iterator.202`](../../home/exceptions.md#jsonexceptioninvalid_iterator202)
("view does not belong to this document") if `object`/`array` is a [discarded](../basic_json_view/is_discarded.md)
view or a view of a *different* document. Overload 3. never throws `invalid_iterator.202`: it takes no view, and it
always starts from this document's own [`root()`](root.md).
## Complexity
1. Linear in the number of members of `object`.
2. Linear in the number of elements of `array` at or after `idx` (they move one slot over).
3. Linear in the number of reference tokens of `ptr` and, for each token, in the number of members of the object at
that level or the index into the array (as [`at`](../basic_json_view/at.md)), plus the complexity of 1. or 2. for
the last token.
## Notes
!!! info "Duplicate keys"
Overload 1. removes *every* member with `key`, not just the first -- unlike [`set`](set.md), which assigns the
first occurrence and drops the rest. This is why it returns a count rather than a single view: there may be
more than one member removed, or none.
Like [`set`](set.md) and [`push_back`](push_back.md), `erase` never moves an element's *value*: a view still
referring to a removed member or element keeps showing what it last held (see [Edits](index.md#edits)) -- it just no
longer appears when `array`/`object` is read, dumped, or iterated. What does change is the container's sequence of
*links* to its elements: removing an element of `array` (2.) shifts the links to the elements after it by one, as
[`insert`](insert.md) does in the other direction. *Every* iterator already taken over `array`/`object` is invalidated
by an erase, not only those at or after the removed element, since the container may now use a different sequence.
## Examples
??? example
The example below drops a deprecated field and a decommissioned entry from a configuration document -- using all
three overloads -- and shows what stays intact that would not with a plain `json`/`ordered_json` value: the order
of the fields around the ones removed, and the exact spelling of a number that was never touched.
```cpp
--8<-- "examples/basic_json_document__erase.cpp"
```
Output:
```json
--8<-- "examples/basic_json_document__erase.output"
```
## See also
- [insert](insert.md) - insert an element into an array
- [set](set.md) - replace a value, or set an object member, an array element, or the value a JSON pointer refers to
- [push_back](push_back.md) - append to an array
- [root](root.md) - the view of the root value, the starting point of overload 3
- [`BasicJsonType::erase`](../basic_json/erase.md) - the corresponding function of `basic_json`
- [Edits](index.md#edits) - what an edit guarantees, for every overload
## Version history
- Added in version 3.13.0.
@@ -3,7 +3,7 @@
<small>Defined in header `<nlohmann/json_view.hpp>`</small>
```cpp
template<typename BasicJsonType, bool Editable = false>
template<typename BasicJsonType>
class basic_json_document;
```
@@ -20,11 +20,6 @@ is move-only: a copy would duplicate the whole index, the decoded-string buffer,
enough in every use we know of. The index, the decoded strings, and the owned text live in one heap block that the
document holds through a `#!cpp std::unique_ptr`; this is also why views survive a move of the document.
With `#!cpp Editable == true`, the document also offers [`set`](set.md), [`push_back`](push_back.md),
[`insert`](insert.md), and [`erase`](erase.md) to change values in place, see [Edits](#edits) below. The source text
itself is never written; a read-only document (`#!cpp Editable == false`, the default) does not carry any of the
bookkeeping edits need, and calling any of them on one fails to compile (`#!cpp static_assert`).
## Template parameters
`BasicJsonType`
@@ -32,38 +27,23 @@ bookkeeping edits need, and calling any of them on one fails to compile (`#!cpp
[`ordered_json`](../ordered_json.md). Only 64-bit `number_integer_t`/`number_unsigned_t` types are supported; this
is checked with a `static_assert`.
`Editable`
: whether the document supports [`set`](set.md), [`push_back`](push_back.md), [`insert`](insert.md), and
[`erase`](erase.md) (optional, `#!cpp false` by default). See [Edits](#edits) below.
## Specializations
- [**json_document**](../json_document.md) - read-only documents of the default specialization [`json`](../json.md)
- [**ordered_json_document**](../ordered_json_document.md) - read-only documents of
[`ordered_json`](../ordered_json.md)
- [**json_editable_document**](../json_editable_document.md) - editable documents of [`json`](../json.md)
- [**ordered_json_editable_document**](../ordered_json_editable_document.md) - editable documents of
[`ordered_json`](../ordered_json.md)
- [**json_document**](../json_document.md) - documents of the default specialization [`json`](../json.md)
- [**ordered_json_document**](../ordered_json_document.md) - documents of [`ordered_json`](../ordered_json.md)
## Member types
- **view_type** - the type of view returned by [`root()`](root.md) (`#!cpp basic_json_view<BasicJsonType, Editable>`)
- **view_type** - the type of view returned by [`root()`](root.md) (`#!cpp basic_json_view<BasicJsonType>`)
- **value_t** - the JSON type enumeration, see [`basic_json::value_t`](../basic_json/value_t.md)
- **image_check** - how [`load()`](load.md) validates an image (`full`, `bounds`, `none`), see [`image_check`](load.md#image_check)
## Member functions
- [(constructor)](basic_json_document.md)
### Parsing
- [**parse**](parse.md) (_static_) - deserialize from a compatible input, borrowing or owning it as appropriate
- [**parse_copy**](parse_copy.md) (_static_) - deserialize a copy of a compatible input
- [**accept**](accept.md) (_static_) - check whether the input is valid JSON
- [**read**](read.md) - (re-)parse into this document, reusing its memory
### Access
- [**root**](root.md) - the view of the root value
- [**is_discarded**](is_discarded.md) - return whether the last parse failed
- [**source**](source.md) - the parsed text
@@ -72,49 +52,6 @@ bookkeeping edits need, and calling any of them on one fails to compile (`#!cpp
- [**memory_usage**](memory_usage.md) - the number of bytes held by the document
- [**shrink_to_fit**](shrink_to_fit.md) - release unused index capacity
### Images
- [**save**](save.md) - the document as an image that `load()` reads without parsing
- [**load**](load.md) (_static_) - read an image written by `save()`
### Edits
- [**set**](set.md) - replace a value, or set an object member, an array element, or the value a JSON pointer refers
to (`#!cpp Editable` documents only)
- [**push_back**](push_back.md) - append to an array (`#!cpp Editable` documents only)
- [**insert**](insert.md) - insert an element into an array before a given position (`#!cpp Editable` documents only)
- [**erase**](erase.md) - remove an object member, an array element, or the value a JSON pointer refers to
(`#!cpp Editable` documents only)
## Edits
An editable document (`#!cpp Editable == true`) can be changed after parsing, with [`set`](set.md),
[`push_back`](push_back.md), [`insert`](insert.md), and [`erase`](erase.md);
[`json_editable_document`](../json_editable_document.md) and
[`ordered_json_editable_document`](../ordered_json_editable_document.md) are the corresponding specializations. A few
points apply to every edit:
- **The source text is never written**, and the parsed index never moves: every value keeps the node it was parsed
into, so [views](../basic_json_view/index.md) taken before an edit stay valid, including
[`root()`](root.md). New values (and the element sequences of an edited array/object) go to storage owned by the
document, allocated on demand.
- **A view keeps referring to the same value.** After [`set`](set.md) replaces the value a view refers to, that view
sees the new value; a view of a value that a later edit replaces or drops keeps showing what it last held. An edit
of an array or object, however, **invalidates the iterators taken over it** (its members may now live in a
different sequence), and a string obtained with [`get_string()`](../basic_json_view/get_string.md) stays valid even
as further edits happen (earlier buffers of edited text are kept alive, not overwritten).
- **Values are accepted three ways:** a [`basic_json_view`](../basic_json_view/index.md) of *any* document
(read-only or editable; it is copied, nothing is shared with the source document), a `BasicJsonType` value, or
anything `BasicJsonType` can be constructed from (numbers, strings, `#!cpp bool`, `#!cpp nullptr`, containers, ...).
- [`dump()`](../basic_json_view/dump.md) writes an edited document with members in document order, new members at
the end, and, with [`number_format::source`](../basic_json_view/number_format.md), keeps the spelling of every
number that was not itself edited -- see [Editing a document](../../features/json_view.md#editing-a-document) for
why this matters.
- [`read()`](read.md) discards all edits, [`shrink_to_fit()`](shrink_to_fit.md) does not move the node index once
there are edits, and [`memory_usage()`](memory_usage.md) includes the memory edits use.
[`source_offset()`](../basic_json_view/source_offset.md) of a value introduced by an edit is
`#!cpp static_cast<std::size_t>(-1)`, the same value it reports for a decoded string.
## Version history
- Added in version 3.13.0.
@@ -1,114 +0,0 @@
# <small>nlohmann::basic_json_document::</small>insert
```cpp
template<typename I, typename V>
view_type insert(view_type array, I idx, V&& value);
```
Only an **editable** document (`#!cpp Editable == true`, e.g. [`json_editable_document`](../json_editable_document.md))
has `insert`; calling it on a read-only `basic_json_document` fails to compile (`#!cpp static_assert`).
Inserts `value` into `array` as a new element before position `idx`, which must not be past the end
(`#!cpp idx <= array.size()`; `#!cpp idx == array.size()` appends, like [`push_back`](push_back.md)). Unlike
[`push_back`](push_back.md), a [null](../basic_json_view/is_null.md) `array` does *not* first become an empty array:
`array` must already be an array.
`value` is accepted three ways: a [`basic_json_view`](../basic_json_view/index.md) of *any* document -- read-only or
editable, and it does not have to be `array`'s own document -- which is copied so that nothing is shared with the
source document afterward; a `BasicJsonType` value; or anything `BasicJsonType` can be constructed from (numbers,
strings, `#!cpp bool`, `#!cpp nullptr`, containers, ...).
## Template parameters
`I`
: an integral type other than `#!cpp bool`, deduced (overloads taking a `#!cpp bool` or a non-integral type for
`idx` do not participate in overload resolution).
`V`
: the type of `value`, deduced; see above for what is accepted.
## Parameters
`array` (in)
: the array to insert into
`idx` (in)
: the position to insert `value` before; a negative value throws (see [Exceptions](#exceptions))
`value` (in)
: the value to insert
## Return value
a view of the new element, now holding `value`
## Exception safety
Basic exception safety: `value` is fully encoded -- including the checks below -- into storage owned by the document
before anything already reachable from [`root()`](root.md) is touched, so a failure while encoding `value` (an
invalid argument, or `#!cpp std::bad_alloc`) leaves the document completely unchanged, other than memory allocated
for the encoding that is not reclaimed. A failure of a later allocation -- while `array` switches from its parsed
layout to a growable block, or while that block grows, see [Notes](#notes) -- can still leave `array` already
switched to that layout even though `value` itself was not inserted.
## Exceptions
Throws [`type_error.309`](../../home/exceptions.md#jsonexceptiontype_error309) if `array` is not an array -- the same
message [`BasicJsonType::insert`](../basic_json/insert.md) throws for the same type; a null `array` throws this too
(see above). Throws [`out_of_range.401`](../../home/exceptions.md#jsonexceptionout_of_range401) if `idx` is negative,
or if `#!cpp idx > array.size()`. Throws
[`invalid_iterator.202`](../../home/exceptions.md#jsonexceptioninvalid_iterator202) ("view does not belong to this
document") if `array` is a [discarded](../basic_json_view/is_discarded.md) view or a view of a *different* document.
Throws [`type_error.302`](../../home/exceptions.md#jsonexceptiontype_error302) if `value` is a
[discarded](../basic_json_view/is_discarded.md) view or a [discarded](../basic_json/is_discarded.md) `BasicJsonType`
value, and [`type_error.319`](../../home/exceptions.md#jsonexceptiontype_error319) if `value` is (or contains) a
binary value -- `BasicJsonType` can hold one, but a `json_document` cannot. Throws
[`type_error.316`](../../home/exceptions.md#jsonexceptiontype_error316) if `value` is (or contains) a string that is
not valid UTF-8, with the same message [`BasicJsonType::dump()`](../basic_json/dump.md) gives for that string.
## Complexity
Linear in the number of elements of `array` at or after `idx` (they move one slot over), plus time linear in the
size of `value` to encode it into the document's storage (constant for a scalar, linear in the number of nested
values for an array or object): like [`push_back`](push_back.md), the elements of `array` move to a growable block
of links the first time it is inserted into (or [`set`](set.md)/[`push_back`](push_back.md) on), and that block
grows in amortized constant time; inserting before the end within that block still shifts every later element.
## Notes
Like [`set`](set.md) on a member or an element, `insert` never moves an existing *element's value* -- only where
`array`'s *links* to its elements live -- so a view of an existing element of `array` stays valid across an
`insert`, and keeps referring to the same element even though its index shifts. Any iterator already taken over
`array` is invalidated, since it was walking the old layout. See [Edits](index.md#edits) for what stays valid across
an edit in general.
## Examples
??? example
The example below inserts a step into the middle of a deployment plan, without touching the steps that come
after it, and shows that a view taken before the insert keeps referring to the same element even though its
index shifts -- something a plain `json`/`ordered_json` array, or its `std::vector`-based storage, has no
equivalent for.
```cpp
--8<-- "examples/basic_json_document__insert.cpp"
```
Output:
```json
--8<-- "examples/basic_json_document__insert.output"
```
## See also
- [push_back](push_back.md) - append to an array
- [erase](erase.md) - remove an object member, an array element, or the value a JSON pointer refers to
- [set](set.md) - replace a value, or set an object member, an array element, or the value a JSON pointer refers to
- [`BasicJsonType::insert`](../basic_json/insert.md) - the corresponding function of `basic_json`
- [Edits](index.md#edits) - what an edit guarantees, for every overload
## Version history
- Added in version 3.13.0.
@@ -1,179 +0,0 @@
# <small>nlohmann::basic_json_document::</small>load
```cpp
// (1)
static basic_json_document load(const std::uint8_t* image, std::size_t size,
const image_check check = image_check::full);
// (2)
static basic_json_document load(const std::vector<std::uint8_t>& image,
const image_check check = image_check::full);
// (3)
static basic_json_document load(std::vector<std::uint8_t>&& image,
const image_check check = image_check::full);
```
1. Reads an image [`save()`](save.md) wrote, from a pointer and a byte count. The image is **borrowed**: `image`
must stay alive and unchanged for as long as the returned document, and any view taken from it, is used.
2. Reads an image from a `#!cpp std::vector`. Also **borrowed** -- equivalent to overload 1 called with
`#!cpp image.data()` and `#!cpp image.size()`.
3. Reads an image, keeping the vector instead of copying it: `image` is moved into the document (no copy), which
then owns it for as long as it needs the text and the decoded strings. [`owns_source()`](owns_source.md) is
`#!cpp true` afterward.
In every overload, the node index is copied into storage the document itself owns -- so that it is properly aligned,
and, for an [editable](index.md#edits) document, can be edited -- while the text and the decoded strings stay in
`image`. The hash indexes [large objects](../../features/json_view.md) use for lookup are rebuilt, exactly as after
parsing.
## Parameters
`image` (in)
: the image [`save()`](save.md) wrote (overloads 1 and 2), or one to take ownership of (overload 3)
`size` (in)
: the number of bytes at `image` (overload 1)
`check` (in)
: how thoroughly to validate `image` before trusting it; see [`image_check`](#image_check) below (optional,
`#!cpp image_check::full` by default)
## Return value
The document read from the image.
## Exception safety
Overloads 1 and 2 give the strong guarantee: `image` is only read, never written, so a thrown exception leaves the
caller's buffer untouched.
Overload 3 moves `image` into the document *before* validating it, so that a good image is kept without a copy. If
loading then fails, the partially built document -- and the vector now inside it -- is discarded along with the
exception, and `image` itself is left **empty**, not restored to what was passed in. Move a copy in instead, or
validate with overload 2 first, if the original vector must survive a failed load.
## Exceptions
On a big-endian target, throws [`type_error.320`](../../home/exceptions.md#jsonexceptiontype_error320) -- the same
exception [`save()`](save.md#exceptions) throws there, since the image format is little-endian only.
Otherwise throws [`parse_error.116`](../../home/exceptions.md#jsonexceptionparse_error116) if `image` is not one
`save()` could have written, or fails the requested `check`:
| message | when |
|------------------------|------------------------------------------------------------------------------------------------------|
| `too short` | `image` is `#!cpp nullptr`, or `size` is smaller than the 64-byte header |
| `unknown format` | the header's magic bytes or version do not match, or a reserved header field is not zero |
| `sizes out of range` | the node count, text size, or decoded-string size the header describes does not fit `size`, or the `#!cpp '\0'` after the text or after the decoded strings is missing |
| `the check failed` | `check` is not `#!cpp image_check::none`, and the image fails it -- see [`image_check`](#image_check) |
!!! failure "Example messages"
```
[json.exception.parse_error.116] parse error: invalid json_document image: too short
```
```
[json.exception.parse_error.116] parse error: invalid json_document image: unknown format
```
```
[json.exception.parse_error.116] parse error: invalid json_document image: sizes out of range
```
```
[json.exception.parse_error.116] parse error: invalid json_document image: the check failed
```
## Complexity
Linear in the number of nodes, which are always copied into the document. `#!cpp image_check::none` does not read
the text or the decoded strings. `#!cpp image_check::bounds` reads one byte of the text for each float token (its sign,
to check the recorded digits against the token's length), and nothing else of the text or the decoded strings.
With `#!cpp image_check::full`, linear in the size of the image (the nodes, the text, and the decoded strings), plus
sorting the ranges of the strings and of the float tokens (at most one per node): the contents of each distinct range
are checked once, however many nodes refer to it.
## Notes
### `image_check` { #image_check }
```cpp
using image_check = detail::view::image_check;
enum class image_check
{
full,
bounds,
none
};
```
How thoroughly `load()` validates `image` before trusting it.
`full` (the default)
: Checks everything the parser itself guarantees: structure and bounds; that every string is valid UTF-8 (and, for a
string still in the source text, that it contains no quote, backslash, or control character); and that every number
token is well-formed and matches the value stored for it. Strings and float tokens may share a range only if the
ranges are identical (as nodes that share a value do), and never overlap otherwise. Reading and serializing a
checked image is safe and always produces valid JSON, exactly as for a parsed document. This is the right choice for
an image from anything you do not fully control -- a file, a cache shared with other processes, a peer on the
network.
`bounds`
: Checks structure and bounds only: that every offset and count in the node index stays inside the image. So
`load()`, [`root()`](root.md), element access, and [`materialize()`](../basic_json_view/materialize.md) can never
read outside the image, even on a damaged one. It does *not* guarantee that the image describes valid JSON: a
string that a `full` check would have rejected can make [`dump()`](../basic_json_view/dump.md) write invalid UTF-8
or invalid JSON, or throw [`type_error.316`](../../home/exceptions.md#jsonexceptiontype_error316), and a number can
read back with a value that does not match how it is spelled. An [editable document](../json_editable_document.md)
does not take such a string over: [`set`](set.md), [`push_back`](push_back.md) and [`insert`](insert.md) throw
`type_error.316` when they copy it from a view of the loaded document, and leave the editable document unchanged.
`bounds` skips scanning the text and the decoded strings, so it fits a cache your own process just wrote and reads
straight back, where damage would mean a bug or a hardware fault rather than adversarial input.
`none`
: Checks nothing. Reading a damaged image is undefined behavior, so use it only for an image you trust as much as
your own memory.
**Lifetime.** Overloads 1 and 2 borrow `image`: it must stay alive and byte-for-byte unchanged for as long as the
returned document, and any [view](../basic_json_view/index.md) taken from it, is used -- exactly like a document
[`parse()`](parse.md) borrowed its input for. Overload 3 avoids this by keeping the vector itself; see
[`owns_source`](owns_source.md).
!!! warning "Experimental"
The image format is versioned but not yet stable, and may change in an incompatible way before it is declared
stable; `load()` already rejects an image written by a different format version with `parse_error.116`
("unknown format"). Use images to cache a document within one build of the library, or to hand one to another
process running the *same* build on the *same* (little-endian) machine -- not as a long-term storage format.
## Examples
??? example "Caching a document, ownership, and a rejected image"
The example below saves a parsed document as an image, checks that `load()` reproduces the original
[`dump()`](../basic_json_view/dump.md) without parsing, and shows the difference between
`load(std::move(image))` (owned) and `load(image)` (borrowed). It then damages one byte of the image and shows
`image_check::full` rejecting it with `parse_error.116`, while `image_check::bounds` -- meant for a cache the
process already trusts -- still reads it without going out of bounds.
```cpp
--8<-- "examples/basic_json_document__load.cpp"
```
Output:
```json
--8<-- "examples/basic_json_document__load.output"
```
## See also
- [save](save.md) - write the document as an image
- [owns_source](owns_source.md) - return whether the document holds its own copy of the text
- [parse](parse.md) - deserialize from JSON text instead of an image
- [Images](../../features/json_view.md#images) - why and when to use images
## Version history
- Added in version 3.13.0.
@@ -149,7 +149,6 @@ buffer of a given length, pass a pair of pointers: `#!cpp parse(ptr, ptr + len)`
- [accept](accept.md) - check whether the input is valid JSON
- [read](read.md) - (re-)parse into this document, reusing its memory
- [owns_source](owns_source.md) - return whether the document holds its own copy of the text
- [load](load.md) - read a document from an image instead of parsing JSON text
- [`BasicJsonType::parse`](../basic_json/parse.md) - the corresponding function of `basic_json`
## Version history
@@ -1,102 +0,0 @@
# <small>nlohmann::basic_json_document::</small>push_back
```cpp
template<typename V>
view_type push_back(view_type array, V&& value);
```
Appends `value` as a new last element of `array`. A [null](../basic_json_view/is_null.md) `array` first becomes an
empty array, the same way [`set`](set.md) turns a null `object` into an empty object.
`value` is accepted three ways: a [`basic_json_view`](../basic_json_view/index.md) of *any* document -- read-only or
editable, and it does not have to be `array`'s own document -- which is copied so that nothing is shared with the
source document afterward; a `BasicJsonType` value; or anything `BasicJsonType` can be constructed from (numbers,
strings, `#!cpp bool`, `#!cpp nullptr`, containers, ...).
Only an **editable** document (`#!cpp Editable == true`, e.g. [`json_editable_document`](../json_editable_document.md))
has `push_back`; calling it on a read-only `basic_json_document` fails to compile (`#!cpp static_assert`).
## Template parameters
`V`
: the type of `value`, deduced; see above for what is accepted.
## Parameters
`array` (in)
: the array (or null value) to append to
`value` (in)
: the value to append
## Return value
a view of the new last element of `array`, now holding `value`
## Exception safety
Basic exception safety: `value` is fully encoded -- including the checks below -- into storage owned by the document
before anything already reachable from [`root()`](root.md) is touched, so a failure while encoding `value` (an
invalid argument, or `#!cpp std::bad_alloc`) leaves the document completely unchanged, other than memory allocated
for the encoding that is not reclaimed. A failure of a later allocation -- while `array` switches from its parsed
layout to a growable block, or while that block grows, see [Notes](#notes) -- can still leave a partial effect, such
as a null `array` argument already turned into an empty array even though `value` itself was not appended.
## Exceptions
Throws [`type_error.308`](../../home/exceptions.md#jsonexceptiontype_error308) if `array` is neither an array nor
null -- the same message [`BasicJsonType::push_back`](../basic_json/push_back.md) throws for the same type. Throws
[`invalid_iterator.202`](../../home/exceptions.md#jsonexceptioninvalid_iterator202) ("view does not belong to this
document") if `array` is a [discarded](../basic_json_view/is_discarded.md) view or a view of a *different* document.
Throws [`type_error.302`](../../home/exceptions.md#jsonexceptiontype_error302) if `value` is a
[discarded](../basic_json_view/is_discarded.md) view or a [discarded](../basic_json/is_discarded.md) `BasicJsonType`
value, and [`type_error.319`](../../home/exceptions.md#jsonexceptiontype_error319) if `value` is (or contains) a
binary value -- `BasicJsonType` can hold one, but a `json_document` cannot. Throws
[`type_error.316`](../../home/exceptions.md#jsonexceptiontype_error316) if `value` is (or contains) a string that is
not valid UTF-8, with the same message [`BasicJsonType::dump()`](../basic_json/dump.md) gives for that string.
## Complexity
Amortized constant, plus time linear in the size of `value` to encode it into the document's storage (constant for
a scalar, linear in the number of nested values for an array or object): the elements of `array` move to a growable
block of links the first time it is appended to (or [`set`](set.md) on), and that block itself grows -- doubling its
capacity, so the cost of growing it amortizes to constant per element -- only once it runs out of room. See
[Notes](#notes).
## Notes
Like [`set`](set.md) on a member or an element, `push_back` never moves an existing element itself -- only where
`array`'s *links* to its elements live -- so a view of an existing element of `array` stays valid across a
`push_back`, but any iterator already taken over `array` is invalidated, since it was walking the old layout. See
[Edits](index.md#edits) for what stays valid across an edit in general.
## Examples
??? example
The example below appends records to an array one at a time, as they might arrive from a stream of events,
without ever building a `BasicJsonType` value for the array or for the records already in it, and shows that a
view taken from an earlier `push_back` still refers to the same element once later ones have run.
```cpp
--8<-- "examples/basic_json_document__push_back.cpp"
```
Output:
```json
--8<-- "examples/basic_json_document__push_back.output"
```
## See also
- [set](set.md) - replace a value, or set an object member, an array element, or the value a JSON pointer refers to
- [insert](insert.md) - insert an element into an array before a given position
- [erase](erase.md) - remove an object member, an array element, or the value a JSON pointer refers to
- [root](root.md) - the view of the root value
- [`BasicJsonType::push_back`](../basic_json/push_back.md) - the corresponding function of `basic_json`
- [Edits](index.md#edits) - what an edit guarantees, for every overload
## Version history
- Added in version 3.13.0.
@@ -51,11 +51,6 @@ own on the next, since ownership is decided freshly each time. An input that is
[`parse()`](parse.md#notes)) is written into the document's previous owned copy, reusing its memory; a moved-in
`#!cpp std::string` replaces it.
Reusing a document matters most for large inputs: the operating system provides the memory of a fresh node index one
page at a time, and every page costs a page fault the first time it is written. On x86-64 Linux (4 KiB pages), parsing
a 55 MB document into a reused document took about 40 % less time than parsing it into a fresh one. Programs that parse
many documents of similar size should therefore keep one document and call `read()`.
An integer argument that is not a `#!cpp bool` where the flags are expected, such as `#!cpp read(ptr, len)`, does not
compile; see [`parse`](parse.md#notes).
@@ -80,7 +75,6 @@ compile; see [`parse`](parse.md#notes).
- [parse](parse.md) - deserialize from a compatible input
- [root](root.md) - the view of the root value
- [load](load.md) - read a document from an image instead of parsing JSON text
## Version history
@@ -1,104 +0,0 @@
# <small>nlohmann::basic_json_document::</small>save
```cpp
std::vector<std::uint8_t> save() const;
```
Writes the document as an *image*: a byte buffer that [`load`](load.md) reads back without parsing. The image holds
the node index, the source text (plus, for an edited document, the number tokens edits wrote), and the decoded
strings (plus the strings edits wrote) -- everything [`root()`](root.md) needs, with nothing left to parse.
An edited document is written in its *current* state, with its values in document order, the way the library's own
parser would have produced them for that JSON text: a member [`set`](set.md) added goes at the end, an
[`erase`](erase.md)d member leaves no trace, and a float that is not finite (NaN or positive/negative infinity)
becomes null, the same substitution [`dump()`](../basic_json_view/dump.md) makes. The same document always saves to
the same bytes -- also across `BasicJsonType` and `#!cpp Editable`, since the image reflects document order and
values only, not which specialization produced them.
## Return value
The image, as a `#!cpp std::vector<std::uint8_t>`. Pass it, or a pointer to its data together with its size, to
[`load`](load.md) to read the document back.
## Exception safety
Strong guarantee: `save()` does not modify `#!cpp *this` (it is `#!cpp const`), so if it throws, the document is left
exactly as it was, and the partially built image is discarded with the exception.
## Exceptions
Throws [`type_error.320`](../../home/exceptions.md#jsonexceptiontype_error320) if the document is
[discarded](is_discarded.md) -- a default-constructed document, or one a failed [`parse()`](parse.md)/
[`read()`](read.md) with `allow_exceptions == false` left discarded.
On a big-endian target, throws `type_error.320` with a different message instead: the image format is little-endian
only (see [Notes](#notes)).
Throws [`out_of_range.416`](../../home/exceptions.md#jsonexceptionout_of_range416) if the node count, the text, or
the decoded strings of the image would individually reach 4 GiB -- the same 32-bit offsets
[`parse()`](parse.md#exceptions) and, for edits, [`set`](set.md)/[`push_back`](push_back.md) are already limited to.
!!! failure "Example messages"
```
[json.exception.type_error.320] cannot save a discarded json_document
```
```
[json.exception.type_error.320] json_document images need a little-endian target
```
```
[json.exception.out_of_range.416] images of 4 GiB or more are not supported by json_document
```
## Complexity
Linear in the size of the document: the number of nodes, plus the length of the text and the decoded strings that end
up in the image.
## Notes
**Format.** The image begins with a 64-byte header (the magic bytes `#!cpp "NJVI"`, a version number, the node count,
and the sizes of the text and the decoded strings, all little-endian), followed by the nodes
([16 bytes each](../../home/architecture.md#node-index-of-json-views)), the text and a `#!cpp '\0'`, and the decoded
strings and a `#!cpp '\0'`. [`load`](load.md) checks the header, and the sizes it describes, before reading anything
else -- see [`load`'s Exceptions](load.md#exceptions).
!!! warning "Experimental"
The image format is versioned but not yet stable: it may change in an incompatible way before it is declared
stable. Use images to cache a document within one build of the library, or to hand one to another process running
the *same* build on the *same* (little-endian) machine -- not as a long-term storage format. Keep the original
JSON text if you need to read a saved document back with a future library version.
**Little-endian only.** The image is written as raw little-endian bytes, with no byte-swapping. `save()` (and
[`load`](load.md)) throw `type_error.320` on a big-endian target rather than silently produce bytes a big-endian
reader could not interpret correctly.
## Examples
??? example "Caching a parsed document as an image"
The example below saves a parsed configuration as an image -- the way a service might cache one to answer later
requests without parsing the text again -- and confirms that loading it back gives exactly the same result as
parsing did, and that saving is deterministic.
```cpp
--8<-- "examples/basic_json_document__save.cpp"
```
Output:
```json
--8<-- "examples/basic_json_document__save.output"
```
## See also
- [load](load.md) - read an image written by `save()`
- [owns_source](owns_source.md) - return whether the document holds its own copy of the text
- [`basic_json_view::dump`](../basic_json_view/dump.md) - serialize the document to JSON text instead of an image
- [Images](../../features/json_view.md#images) - why and when to use images
## Version history
- Added in version 3.13.0.
@@ -1,203 +0,0 @@
# <small>nlohmann::basic_json_document::</small>set
```cpp
// (1)
template<typename V>
view_type set(view_type target, V&& value);
// (2)
template<typename V>
view_type set(view_type object, string_view_t key, V&& value);
// (3)
template<typename I, typename V>
view_type set(view_type array, I idx, V&& value);
// (4)
template<typename V>
view_type set(const json_pointer& ptr, V&& value);
```
Only an **editable** document (`#!cpp Editable == true`, e.g. [`json_editable_document`](../json_editable_document.md))
has `set`; calling it on a read-only `basic_json_document` fails to compile (`#!cpp static_assert`).
1. Replaces the value `target` refers to with `value`.
2. Sets the member `key` of the object `object` to `value`: assigns it if `object` already has a member with this
key -- the first one, should the key occur more than once, and the later duplicates are then dropped (see the
[Notes](#notes) below) -- or appends a new member at the end otherwise. A [null](../basic_json_view/is_null.md)
`object` first becomes an empty object.
3. Assigns `value` to the element at index `idx` of the array `array`, which must already exist (`#!cpp idx <
array.size()`).
4. Sets the value the JSON pointer `ptr` refers to, relative to [`root()`](root.md), to `value`. The *parent* of the
target must already exist: an object member is set as in 2. (added if it does not exist yet), an array element is
assigned as in 3., and a last reference token of `#!cpp "-"`, or equal to the size of the array, appends `value`
instead, exactly as [`push_back`](push_back.md) would. A [null](../basic_json_view/is_null.md) parent becomes what
[`basic_json::operator[]`](../basic_json/operator%5B%5D.md) with a JSON pointer makes of it: an array if the last
reference token is `#!cpp "-"` or consists of digits only (for an index beyond 0, the array is first filled with
null values up to that index), an object otherwise. An empty `ptr` sets [`root()`](root.md) itself, as in 1.
In every overload, `value` is accepted three ways: a [`basic_json_view`](../basic_json_view/index.md) of *any*
document -- read-only or editable, and it does not have to be `target`'s/`object`'s/`array`'s own document -- which
is copied so that nothing is shared with the source document afterward; a `BasicJsonType` value; or anything
`BasicJsonType` can be constructed from (numbers, strings, `#!cpp bool`, `#!cpp nullptr`, containers, ...).
## Template parameters
`V`
: the type of `value`, deduced; see above for what is accepted.
`I`
: an integral type other than `#!cpp bool`, deduced (overloads taking a `#!cpp bool` or a non-integral type for
`idx` do not participate in overload resolution).
## Parameters
`target` (in)
: the value to replace
`object` (in)
: the object (or null value) whose member to set
`array` (in)
: the array whose element to assign
`key` (in)
: the key of the member to set
`idx` (in)
: the index of the element to assign; a negative value throws (see [Exceptions](#exceptions))
`ptr` (in)
: a JSON pointer to the value to set, relative to `root()`
`value` (in)
: the new value
## Return value
1. a view of `target`, now holding `value`
2. a view of the member `key` of `object`, now holding `value`
3. a view of the element `idx` of `array`, now holding `value`
4. a view of the value `ptr` refers to, now holding `value`
## Exception safety
Basic exception safety: `value` is fully encoded -- including the checks below -- into storage owned by the document
before anything already reachable from [`root()`](root.md) is touched, so a failure while encoding `value` (an
invalid argument, or `#!cpp std::bad_alloc`) leaves the document completely unchanged, other than memory allocated
for the encoding that is not reclaimed. A failure of a later allocation -- while an edited array or object switches
from its parsed layout to a growable block, see [Notes](#notes) -- can still leave a partial effect, such as a
[null](../basic_json_view/is_null.md) `object`/`array` argument already turned into an empty object/array even
though `value` itself was not linked in. Likewise, a failure of `value` in 4. leaves a null parent that is set with an
index beyond 0 already filled with the null values before the index.
## Exceptions
1. Throws [`type_error.302`](../../home/exceptions.md#jsonexceptiontype_error302) if `value` is a
[discarded](../basic_json_view/is_discarded.md) view, or a [discarded](../basic_json/is_discarded.md)
`BasicJsonType` value (e.g. `#!cpp BasicJsonType(value_t::discarded)`) -- an object or array `value`, of either
kind, is fine and is encoded as a whole subtree.
2. Throws [`type_error.305`](../../home/exceptions.md#jsonexceptiontype_error305) if `object` is neither an object
nor null -- the same message [`operator[]`](../basic_json_view/operator%5B%5D.md) throws for a string argument on
such a value. Throws [`type_error.316`](../../home/exceptions.md#jsonexceptiontype_error316) if `key` is not
valid UTF-8, with the same message [`BasicJsonType::dump()`](../basic_json/dump.md) gives for that string.
Also throws what 1. throws for `value`.
3. Throws `type_error.305` if `array` is not an array -- the same message `operator[]` throws for a numeric argument
on such a value. Throws [`out_of_range.401`](../../home/exceptions.md#jsonexceptionout_of_range401) if `idx` is
negative, or if `#!cpp idx >= array.size()`. Also throws what 1. throws for `value`.
4. Throws what [`at`](../basic_json_view/at.md) throws (overload 3) for resolving `ptr`'s parent, except that a
missing object member or an array index equal to the array's size at the very last reference token is not an
error there (it becomes a new member or an appended element) instead of
[`out_of_range.403`](../../home/exceptions.md#jsonexceptionout_of_range403)/[`out_of_range.402`](../../home/exceptions.md#jsonexceptionout_of_range402).
For the last reference token itself: if the parent is an object (or a primitive value, where it throws
`type_error.305`), throws what 2. throws; if the parent is an array, throws what 3. throws for an index that is
out of range, or, for a token that is not a valid array index,
[`parse_error.106`](../../home/exceptions.md#jsonexceptionparse_error106) (a leading `#!cpp '0'`),
[`parse_error.109`](../../home/exceptions.md#jsonexceptionparse_error109) (not a number),
[`out_of_range.410`](../../home/exceptions.md#jsonexceptionout_of_range410) (too large for `size_type`), or
[`out_of_range.404`](../../home/exceptions.md#jsonexceptionout_of_range404) (an empty token); the same errors are
thrown for a null parent and a token of digits (the parent is not changed then), and
[`out_of_range.401`](../../home/exceptions.md#jsonexceptionout_of_range401) if the index is 4294967295 or more.
Also throws what 1. throws for `value`.
Every overload also throws [`type_error.319`](../../home/exceptions.md#jsonexceptiontype_error319) if `value` is (or
contains) a binary value -- `BasicJsonType` can hold one, but a `json_document` cannot -- and
[`invalid_iterator.202`](../../home/exceptions.md#jsonexceptioninvalid_iterator202) ("view does not belong to this
document") if `target`/`object`/`array` is a [discarded](../basic_json_view/is_discarded.md) view or a view of a
*different* document (overloads 1-3 only; overload 4 always starts from this document's own [`root()`](root.md)).
## Complexity
1. Linear in the size of `value` (encoding it into the document's storage): constant for a scalar, linear in the
number of nested values for an array or object. If `target` is itself an array or object that spans more than one
node in its parent's original, unedited layout, and `value` is a scalar, replacing it additionally costs time
linear in the size of the document, the *first* time (the parent of `target` is looked up from
[`root()`](root.md), and then switches to links) -- see [Notes](#notes). Once the parent has links, `target` is
replaced in constant time: setting every element of a large array one after the other is linear overall. To avoid
the lookup altogether, use 3. (or 2. for an object), which know the parent.
2. Linear in the number of members of `object`, to find an existing member with `key`, plus the complexity of 1. for
`value`.
3. Constant, plus the complexity of 1. for `value`.
4. Linear in the number of reference tokens of `ptr` and, for each token, in the number of members of the object at
that level or the index into the array (as [`at`](../basic_json_view/at.md)), plus the complexity of 2. or 3. for
the last token; for a null parent and an index, linear in the index.
## Notes
!!! info "Duplicate keys"
If `object` already has more than one member with `key` (2.), the *first* one is assigned `value` and every
later member with the same key is removed -- so that a lookup, an iteration, and
[`materialize()`](../basic_json_view/materialize.md) of `object` afterward all agree on a single value for
`key`, the same way [`operator[]`](../basic_json_view/operator%5B%5D.md) already picks the first occurrence of a
duplicate key for reading. See the [Notes on duplicate keys](../basic_json_view/operator%5B%5D.md#notes) of
`operator[]`.
Setting a member (2.) or an element (3., through 4.) of an array or object whose elements have not been edited
before switches it from its parsed layout to a growable block holding links to its elements; a later
[`push_back`](push_back.md) or `set` on the same container reuses that block, growing it (amortized constant time)
only once it runs out of room. This never moves an element itself -- only where the container's *links* to its
elements live -- so a view of an element stays valid, but any iterator already taken over the container is
invalidated, since it was walking the old layout. See [Edits](index.md#edits) for what stays valid across an edit in
general.
The same switch happens, for the same reason, when overload 1. replaces a multi-node array/object value with a
scalar: the *parent's* element sequence is what has to switch to links, not `target` itself, because the parent
originally stepped over `target`'s whole subtree by its node count, which no longer applies once `target` is a
one-node scalar.
## Examples
??? example "Example: (1)/(2)/(3)/(4) replace a value, set a member, assign an element, set via a JSON pointer"
The example below edits a small configuration document -- replacing a value, adding an object member, assigning
an array element, and reaching a field through a JSON pointer -- and shows what
[`dump()`](../basic_json_view/dump.md) preserves that is lost once the same edits are made on a `BasicJsonType`
value instead: the order object members were written in, and the exact spelling of a number that was never
touched.
```cpp
--8<-- "examples/basic_json_document__set.cpp"
```
Output:
```json
--8<-- "examples/basic_json_document__set.output"
```
## See also
- [push_back](push_back.md) - append to an array
- [insert](insert.md) - insert an element into an array
- [erase](erase.md) - remove an object member, an array element, or the value a JSON pointer refers to
- [root](root.md) - the view of the root value, the starting point of overload 4
- [`basic_json_view::dump`](../basic_json_view/dump.md) - serialize the document, keeping an untouched number's
spelling with `#!cpp number_format::source`
- [Edits](index.md#edits) - what an edit guarantees, for every overload
- [Editing a document](../../features/json_view.md#editing-a-document) - why editable documents keep the source
text's order and number spelling
## Version history
- Added in version 3.13.0.
@@ -77,8 +77,6 @@ None of these exceptions carry a [`JSON_DIAGNOSTICS`](../macros/json_diagnostics
1. Linear in the number of members: as for [`ordered_json`](../ordered_json.md), members are compared one after
another, in document order, stopping at the first match. Each comparison first checks the
key's length -- already known from the index, without reading the key bytes -- before comparing its content.
Objects with 128 or more members get a hash index of their keys while parsing, so that a lookup in them takes
constant time on average.
2. Linear in `idx`: elements are skipped one at a time from the first one, since they are not a fixed size in the
index (unlike `BasicJsonType`'s array, which is random-access).
3. Linear in the number of reference tokens of `ptr` and, for each token, in the number of members of the object at
@@ -35,8 +35,6 @@ No-throw guarantee: this function never throws exceptions.
1. Linear in the number of members: as for [`ordered_json`](../ordered_json.md), members are compared one after
another, in document order, stopping at the first match. Each comparison first checks the
key's length -- already known from the index, without reading the key bytes -- before comparing its content.
Objects with 128 or more members get a hash index of their keys while parsing, so that a lookup in them takes
constant time on average.
2. Linear in the number of reference tokens of `ptr` and, for each token, in the number of members of the object at
that level or the index into the array -- as for [`operator[]`](operator[].md#complexity) and
[`at`](at.md#complexity) with a JSON pointer.
@@ -28,8 +28,7 @@ No-throw guarantee: this function never throws exceptions.
Linear in the number of members: unlike the lookup functions, `count()` does not stop at the first match, because it
has to count every member with the key. Members are compared one after another, in document order. Each comparison
first checks the key's length -- already known from the index, without reading the key bytes -- before comparing its
content. The hash index of the keys of objects with 128 or more members does not help here, since every member
is visited.
content.
## Notes
@@ -28,8 +28,6 @@ No-throw guarantee: this function never throws exceptions.
Linear in the number of members: as for [`ordered_json`](../ordered_json.md), members are compared one after another,
in document order, stopping at the first match. Each comparison first checks the key's length
-- already known from the index, without reading the key bytes -- before comparing its content.
Objects with 128 or more members get a hash index of their keys while parsing, so that a lookup in them takes
constant time on average.
## Notes
+1 -15
View File
@@ -3,7 +3,7 @@
<small>Defined in header `<nlohmann/json_view.hpp>`</small>
```cpp
template<typename BasicJsonType, bool Editable = false>
template<typename BasicJsonType>
class basic_json_view;
```
@@ -36,30 +36,16 @@ Moving the document itself does not invalidate its views: the index is heap-allo
[`json_pointer`](../json_pointer/index.md). Unlike `basic_json`, there are no implicit conversion operators (no
`#!cpp operator ValueType()`), and no `#!cpp get_ptr()` or `#!cpp get_ref()`; see [`get`](get.md).
`basic_json_view` itself is always read-only -- it never has a `set` or `push_back` of its own. A view of an
**editable** document (`#!cpp Editable == true`) sees every edit made through
[`basic_json_document::set`](../basic_json_document/set.md) and
[`basic_json_document::push_back`](../basic_json_document/push_back.md): once a value is changed, every view that
still refers to it -- including ones taken before the change -- reads the new value. See
[Edits](../basic_json_document/index.md#edits).
## Template parameters
`BasicJsonType`
: a specialization of [`basic_json`](../basic_json/index.md), matching the
[`basic_json_document`](../basic_json_document/index.md) the view was taken from.
`Editable`
: whether the view is of an editable document, matching the [`basic_json_document`](../basic_json_document/index.md)
it was taken from (optional, `#!cpp false` by default). See [Edits](../basic_json_document/index.md#edits).
## Specializations
- [**json_view**](../json_view.md) - views of a [`json_document`](../json_document.md)
- [**ordered_json_view**](../ordered_json_view.md) - views of an [`ordered_json_document`](../ordered_json_document.md)
- [**json_editable_view**](../json_editable_view.md) - views of a [`json_editable_document`](../json_editable_document.md)
- [**ordered_json_editable_view**](../ordered_json_editable_view.md) - views of an
[`ordered_json_editable_document`](../ordered_json_editable_document.md)
## Member types
@@ -83,8 +83,6 @@ None of these exceptions carry a [`JSON_DIAGNOSTICS`](../macros/json_diagnostics
another, in document order, stopping at the first match. Each comparison first checks the
key's length -- already known from the index, without reading the key bytes -- before comparing its content, so a
key of a different length than `key` is rejected without touching the source text.
Objects with 128 or more members get a hash index of their keys while parsing, so that a lookup in them takes
constant time on average.
2. Linear in `idx`: elements are skipped one at a time from the first one, since they are not a fixed size in the
index (unlike `BasicJsonType`'s array, which is random-access).
3. Linear in the number of reference tokens of `ptr` and, for each token, in the number of members of the object at
@@ -116,7 +114,6 @@ document.
member in order and so keep the *last* value for a repeated key, so `#!cpp v["a"]` and
`#!cpp v.materialize()["a"]` can differ. To get the value `parse()` would give, use
[`materialize()`](materialize.md) or iterate the members with [`items()`](items.md) and keep the last match.
The key hash index of a larger object (128 members or more) leads to the first member of a key as well.
[`begin()`](begin.md)/[`end()`](end.md) and [`items()`](items.md) iterate over *all* members, including
duplicates, in document order. See [Duplicate keys](../../features/json_view.md#duplicate-keys) and
[`size()`](size.md#notes).
@@ -70,8 +70,6 @@ None of these exceptions carry a [`JSON_DIAGNOSTICS`](../macros/json_diagnostics
1. Linear in the number of members: as for [`operator[]`](operator[].md#complexity), members are compared one after
another, in document order, stopping at the first match. Plus the complexity of converting
the found member to `T` (see [`get`](get.md)).
Objects with 128 or more members get a hash index of their keys while parsing, so that a lookup in them takes
constant time on average.
2. Linear in the number of reference tokens of `ptr` and, for each token, in the number of members of the object at
that level or the index into the array -- as for the [`operator[]`](operator[].md#complexity) and
[`at`](at.md#complexity) overloads that take a JSON pointer. Plus the complexity of converting the resolved value
@@ -1,43 +0,0 @@
# <small>nlohmann::</small>json_editable_document
<small>Defined in header `<nlohmann/json_view.hpp>`</small>
```cpp
using json_editable_document = basic_json_document<json, true>;
```
This type is an **editable** [`basic_json_document`](basic_json_document/index.md) of the default
[`json`](json.md) specialization: in addition to everything [`json_document`](json_document.md) offers,
[`set`](basic_json_document/set.md) and [`push_back`](basic_json_document/push_back.md) change values after
parsing, without ever rewriting the source text -- see [Edits](basic_json_document/index.md#edits) and
[Editing a document](../features/json_view.md#editing-a-document).
## Examples
??? example
The example below patches two fields of a small configuration document -- changing one and adding another --
and dumps it back out with the member order and the spelling of the untouched number preserved, something a
plain [`json`](json.md) value cannot do.
```cpp
--8<-- "examples/json_editable_document.cpp"
```
Output:
```json
--8<-- "examples/json_editable_document.output"
```
## See also
- [json_editable_view](json_editable_view.md) - a view of a value of a `json_editable_document`
- [json_document](json_document.md) - the read-only document this type adds edits to
- [ordered_json_editable_document](ordered_json_editable_document.md) - the corresponding editable document for
`ordered_json`
- [Edits](basic_json_document/index.md#edits) - what an edit guarantees
## Version history
Since version 3.13.0.
@@ -1,41 +0,0 @@
# <small>nlohmann::</small>json_editable_view
<small>Defined in header `<nlohmann/json_view.hpp>`</small>
```cpp
using json_editable_view = basic_json_view<json, true>;
```
This type is a [`basic_json_view`](basic_json_view/index.md) of a value of a
[`json_editable_document`](json_editable_document.md). It offers the same read-only interface as
[`json_view`](json_view.md); what is different is what it can be a view *of* -- a value that
[`set`](basic_json_document/set.md) and [`push_back`](basic_json_document/push_back.md) can change, with every view
still referring to it seeing the change, see [Edits](basic_json_document/index.md#edits).
## Examples
??? example
The example below is the same as [`json_editable_document`'s](json_editable_document.md): every view read back
out of the document -- `#!cpp doc.root()` and the views nested under it -- sees the edits made through `set`.
```cpp
--8<-- "examples/json_editable_document.cpp"
```
Output:
```json
--8<-- "examples/json_editable_document.output"
```
## See also
- [json_editable_document](json_editable_document.md) - the document type this view refers into
- [json_view](json_view.md) - the corresponding read-only view
- [ordered_json_editable_view](ordered_json_editable_view.md) - the corresponding view for
`ordered_json_editable_document`
## Version history
Since version 3.13.0.
-2
View File
@@ -42,8 +42,6 @@ header. See also the [macro overview page](../../features/macros.md).
- [**JSON_SKIP_UNSUPPORTED_COMPILER_CHECK**](json_skip_unsupported_compiler_check.md) - do not warn about unsupported compilers
- [**JSON_USE_GLOBAL_UDLS**](json_use_global_udls.md) - place user-defined string literals (UDLs) into the global namespace
- [**JSON_USE_SIMDUTF**](json_use_simdutf.md) - use the simdutf library to accelerate UTF-8 validation
- [**JSON_VIEW_NO_SIMD**](json_view_no_simd.md) - use only portable code in the parser of `json_view.hpp`
- [**JSON_VIEW_USE_SSSE3**](json_view_use_ssse3.md) - validate non-ASCII strings with SSSE3 in the parser of `json_view.hpp`
## Library version
@@ -10,9 +10,6 @@ earlier, instead of locking the file once and reading it with the unlocked stdio
It does not change any result: values, exceptions, and the position of the file after parsing are the same either way.
Only the speed and the locking of the file differ (see the notes on locking below).
Builds with MemorySanitizer use `std::fgetc` automatically, because the inline `getc_unlocked` of glibc reads the stdio
buffer directly, which glibc (not instrumented) filled, and MemorySanitizer would report it as uninitialized.
## Default definition
The default value is `0`: the file is locked once and read with the unlocked stdio functions where they are available.
@@ -1,52 +0,0 @@
# JSON_VIEW_NO_SIMD
```cpp
#define JSON_VIEW_NO_SIMD
```
When defined, the parser of [`basic_json_document`](../basic_json_document/index.md) (`<nlohmann/json_view.hpp>`)
uses only portable C++ to scan strings. By default, it scans long runs of string bytes 16 at a time with NEON on
AArch64 (with GCC and Clang) and SSE2 on x86-64, which are part of the baseline instruction sets of these
architectures, and validates non-ASCII text with NEON (or SSSE3, see
[`JSON_VIEW_USE_SSSE3`](json_view_use_ssse3.md)).
The same input is accepted or rejected either way, with the same values, and errors are reported the same way; only
the speed differs. The macro exists for platforms whose compilers lack the intrinsics headers, and to test the portable
code.
!!! note "ABI compatibility"
The macro selects between two definitions of the same inline functions. Whether it is defined is encoded in the
[namespace](../../features/namespace.md) (tag `_vns`), resulting in distinct symbol names. Translation units
compiled with and without it can therefore be linked into the same program without One Definition Rule (ODR)
violations, but they cannot exchange instances of library types.
## Default definition
By default, `#!cpp JSON_VIEW_NO_SIMD` is not defined, and the vector code is used where available.
```cpp
#undef JSON_VIEW_NO_SIMD
```
## Examples
??? example
The code below uses the portable string scanning of the view.
```cpp
#define JSON_VIEW_NO_SIMD
#include <nlohmann/json_view.hpp>
...
```
## See also
- [JSON_VIEW_USE_SSSE3](json_view_use_ssse3.md) - validate non-ASCII strings with SSSE3 on x86-64
- [json_view](../../features/json_view.md) - the zero-copy view
## Version history
- Added in version 3.13.0.
@@ -1,53 +0,0 @@
# JSON_VIEW_USE_SSSE3
```cpp
#define JSON_VIEW_USE_SSSE3
```
When defined on x86-64, the parser of [`basic_json_document`](../basic_json_document/index.md)
(`<nlohmann/json_view.hpp>`) validates non-ASCII text in strings with SSSE3 without asking the CPU first.
By default, the parser checks once at run time whether the CPU has SSSE3 (all x86-64 CPUs since about 2011 have it)
and then validates non-ASCII text 16 bytes at a time, using the "lookup4" algorithm of
[simdjson](https://github.com/simdjson/simdjson); on CPUs without SSSE3, it validates one UTF-8 sequence at a time.
The vector check is compiled for SSSE3 with a function attribute (GCC 4.9 and later, Clang), so this needs no compiler
option. With MSVC, the check uses `__cpuid`. On AArch64, the vector check uses NEON and is always on.
Define the macro only together with a compiler option that enables SSSE3 (e.g. `-mssse3`, or `-march=` with a CPU that
has it), and only for programs that run on such CPUs. It saves the check of the CPU, which costs little. The same
input is accepted or rejected either way; only the speed of non-ASCII text differs.
!!! note "ABI compatibility"
The macro selects between two definitions of the same inline functions. Whether it is defined is encoded in the
[namespace](../../features/namespace.md) (tag `_vssse3`, on every architecture), resulting in distinct symbol
names. Translation units compiled with and without it can therefore be linked into the same program without One
Definition Rule (ODR) violations, but they cannot exchange instances of library types.
## Default definition
By default, `#!cpp JSON_VIEW_USE_SSSE3` is not defined.
```cpp
#undef JSON_VIEW_USE_SSSE3
```
## Examples
??? example
With CMake, for a program that only runs on CPUs with SSSE3:
```cmake
target_compile_definitions(your_target PRIVATE JSON_VIEW_USE_SSSE3)
target_compile_options(your_target PRIVATE -mssse3)
```
## See also
- [JSON_VIEW_NO_SIMD](json_view_no_simd.md) - use only portable code in the view's parser
- [JSON_USE_SIMDUTF](json_use_simdutf.md) - validate UTF-8 with simdutf in `basic_json`'s parser
## Version history
- Added in version 3.13.0.
@@ -39,6 +39,6 @@ the library.
## Version history
- Added in version 3.11.0. Changed inline namespace name in version 3.11.2. Added the ABI tag `_dp` in
version 3.12.0, and the ABI tags `_bics`, `_psp`, `_snul`, `_sbu8`, `_ekmo`, `_dsbs`, `_dufio`, `_vns`, and
`_vssse3` in version 3.13.0; see
version 3.12.0, and the ABI tags `_bics`, `_psp`, `_snul`, `_sbu8`, `_ekmo`, `_dsbs`, and `_dufio` in version
3.13.0; see
[`nlohmann` Namespace](../../features/namespace.md#structure).
@@ -1,47 +0,0 @@
# <small>nlohmann::</small>ordered_json_editable_document
<small>Defined in header `<nlohmann/json_view.hpp>`</small>
```cpp
using ordered_json_editable_document = basic_json_document<ordered_json, true>;
```
This type is an **editable** [`basic_json_document`](basic_json_document/index.md) of the
[`ordered_json`](ordered_json.md) specialization: [`set`](basic_json_document/set.md) and
[`push_back`](basic_json_document/push_back.md) change values after parsing, as for
[`json_editable_document`](json_editable_document.md), and
[`materialize()`](basic_json_view/materialize.md) preserves the document order of object members -- including
members [`set`](basic_json_document/set.md) added -- instead of sorting them like
[`json_editable_document`](json_editable_document.md) does.
## Examples
??? example
The example below edits a document with `set`, then shows that `materialize()` keeps the member order of the
source text (with the new member at the end) for `ordered_json_editable_document`, where it would sort the
members alphabetically for [`json_editable_document`](json_editable_document.md).
```cpp
--8<-- "examples/ordered_json_editable_document.cpp"
```
Output:
```json
--8<-- "examples/ordered_json_editable_document.output"
```
## See also
- [ordered_json_editable_view](ordered_json_editable_view.md) - a view of a value of an
`ordered_json_editable_document`
- [ordered_json_document](ordered_json_document.md) - the read-only document this type adds edits to
- [json_editable_document](json_editable_document.md) - the corresponding editable document for the default `json`
specialization
- [Object Order](../features/object_order.md)
- [Edits](basic_json_document/index.md#edits) - what an edit guarantees
## Version history
Since version 3.13.0.
@@ -1,40 +0,0 @@
# <small>nlohmann::</small>ordered_json_editable_view
<small>Defined in header `<nlohmann/json_view.hpp>`</small>
```cpp
using ordered_json_editable_view = basic_json_view<ordered_json, true>;
```
This type is a [`basic_json_view`](basic_json_view/index.md) of a value of an
[`ordered_json_editable_document`](ordered_json_editable_document.md), the corresponding view for
[`ordered_json_view`](ordered_json_view.md) the way [`json_editable_view`](json_editable_view.md) is for
[`json_view`](json_view.md).
## Examples
??? example
The example below is the same as [`ordered_json_editable_document`'s](ordered_json_editable_document.md): the
views `set` returns see the document's member order preserved on `materialize()`, unlike for a
[`json_editable_document`](json_editable_document.md).
```cpp
--8<-- "examples/ordered_json_editable_document.cpp"
```
Output:
```json
--8<-- "examples/ordered_json_editable_document.output"
```
## See also
- [ordered_json_editable_document](ordered_json_editable_document.md) - the document type this view refers into
- [ordered_json_view](ordered_json_view.md) - the corresponding read-only view
- [json_editable_view](json_editable_view.md) - the corresponding view for `json_editable_document`
## Version history
Since version 3.13.0.
@@ -1,38 +0,0 @@
#include <iostream>
#include <nlohmann/json_view.hpp>
using json = nlohmann::json;
using json_editable_document = nlohmann::json_editable_document;
using json_editable_view = nlohmann::json_editable_view;
int main()
{
// a deprecated field is dropped from a configuration file, and a
// decommissioned replica is removed from the list -- "price" keeps its
// trailing zero, and the fields around the removed ones keep their order
const std::string text = R"({
"name": "cache",
"legacy_host": "db0",
"host": "db1",
"price": 19.90,
"replicas": ["db2", "db3", "db4"]
})";
json_editable_document doc = json_editable_document::parse(text);
doc.erase(doc.root(), "legacy_host"); // (1) an object member
doc.erase(doc.root()["replicas"], 1); // (2) an array element ("db3")
const std::size_t removed = doc.erase(json::json_pointer("/replicas/0")); // (3) via a JSON pointer
std::cout << removed << '\n';
std::cout << doc.root().dump(2, ' ', false, json_editable_view::number_format::source) << "\n\n";
// the same edits on a plain json value: object_t is a std::map, so
// parsing already sorted the keys, and dump() rewrites every number to
// its shortest form, even "price", which was never touched
json plain = json::parse(text);
plain.erase("legacy_host");
plain["replicas"].erase(1);
plain["replicas"].erase(0);
std::cout << plain.dump(2) << '\n';
}
@@ -1,18 +0,0 @@
1
{
"name": "cache",
"host": "db1",
"price": 19.90,
"replicas": [
"db4"
]
}
{
"host": "db1",
"name": "cache",
"price": 19.9,
"replicas": [
"db4"
]
}
@@ -1,38 +0,0 @@
#include <iostream>
#include <nlohmann/json_view.hpp>
using json = nlohmann::json;
using json_editable_document = nlohmann::json_editable_document;
using json_editable_view = nlohmann::json_editable_view;
int main()
{
// a deployment plan -- "budget" is written with a trailing zero that has
// no effect on its value
const std::string text = R"({
"release": "2026.09",
"steps": ["build", "test", "deploy"],
"budget": 19.90
})";
json_editable_document doc = json_editable_document::parse(text);
const std::size_t deploy_index = 2;
const auto deploy = doc.root()["steps"][deploy_index]; // held across the insert
doc.insert(doc.root()["steps"], deploy_index, "smoke-test"); // insert before "deploy"
// the held view still refers to "deploy", even though its index moved
// from 2 to 3, and nothing else in the document was touched
std::cout << deploy.dump() << '\n';
std::cout << doc.root().dump(2, ' ', false, json_editable_view::number_format::source) << "\n\n";
// the same edit on a plain json value: an index held from before the
// insert now refers to whatever moved into that slot, and dump()
// rewrites "budget" to its shortest form even though it was never
// touched
json plain = json::parse(text);
plain["steps"].insert(plain["steps"].begin() + static_cast<std::ptrdiff_t>(deploy_index), "smoke-test");
std::cout << plain["steps"][deploy_index].dump() << '\n';
std::cout << plain.dump(2) << '\n';
}
@@ -1,23 +0,0 @@
"deploy"
{
"release": "2026.09",
"steps": [
"build",
"test",
"smoke-test",
"deploy"
],
"budget": 19.90
}
"smoke-test"
{
"budget": 19.9,
"release": "2026.09",
"steps": [
"build",
"test",
"smoke-test",
"deploy"
]
}
@@ -1,52 +0,0 @@
#include <cstdint>
#include <iostream>
#include <string>
#include <vector>
#include <nlohmann/json_view.hpp>
using json = nlohmann::json;
using json_document = nlohmann::json_document;
using image_check = json_document::image_check;
int main()
{
std::cout << std::boolalpha;
// the image of a parsed document -- as if read back from a cache file or
// received from another process running the same build of the library
const std::string text = R"({"name": "cache", "note": "caf\u00e9", "replicas": ["db2", "db3"]})";
const json_document parsed = json_document::parse(text);
const std::vector<std::uint8_t> image = parsed.save();
// (1)/(2) load() needs no parsing, yet dumps exactly what parsing did
const json_document borrowed = json_document::load(image);
std::cout << (borrowed.root().dump() == parsed.root().dump()) << '\n';
std::cout << borrowed.owns_source() << '\n'; // borrowed: still points into `image`
// (3) load(std::move(image)) keeps the vector instead of copying it
std::vector<std::uint8_t> to_move = image;
const json_document owned = json_document::load(std::move(to_move));
std::cout << owned.owns_source() << '\n';
// a damaged image -- the last byte of the decoded string "note" holds
// (an escape sequence, so it was unescaped into the document's own
// buffer), flipped, as storage or transport corruption might do
std::vector<std::uint8_t> damaged = image;
damaged[damaged.size() - 2] = 0xFF;
// image_check::full inspects strings and numbers, so it catches the damage
try
{
static_cast<void>(json_document::load(damaged, image_check::full));
}
catch (const json::parse_error& e)
{
std::cout << e.id << '\n';
}
// image_check::bounds only checks structure and bounds, so a cache the
// process already trusts loads without the extra scan -- reading a value
// the damage did not touch is still safe
const json_document trusted = json_document::load(damaged, image_check::bounds);
std::cout << trusted.root()["name"].get<std::string>() << '\n';
}
@@ -1,5 +0,0 @@
true
false
true
116
cache
@@ -1,24 +0,0 @@
#include <iostream>
#include <nlohmann/json_view.hpp>
using json = nlohmann::json;
using json_editable_document = nlohmann::json_editable_document;
int main()
{
// "events" starts out null -- the first push_back() turns it into an
// array, exactly like set() turns a null object member into an object
json_editable_document doc = json_editable_document::parse(R"({"source": "sensor-1", "events": null})");
const auto first = doc.push_back(doc.root()["events"], json{{"type", "start"}, {"t", 0}});
for (int t = 1; t <= 3; ++t)
{
doc.push_back(doc.root()["events"], json{{"type", "tick"}, {"t", t}});
}
// push_back() never moves an existing element: a view taken from an
// earlier call still refers to the same element after later ones
std::cout << first.dump() << '\n';
std::cout << doc.root()["events"].size() << '\n';
std::cout << doc.root().dump(2) << '\n';
}
@@ -1,23 +0,0 @@
{"t":0,"type":"start"}
4
{
"source": "sensor-1",
"events": [
{
"t": 0,
"type": "start"
},
{
"t": 1,
"type": "tick"
},
{
"t": 2,
"type": "tick"
},
{
"t": 3,
"type": "tick"
}
]
}
@@ -1,30 +0,0 @@
#include <cstdint>
#include <iostream>
#include <string>
#include <vector>
#include <nlohmann/json_view.hpp>
using json_document = nlohmann::json_document;
int main()
{
std::cout << std::boolalpha;
// a configuration a service parses once and then caches as an image, so
// that later requests can load() it instead of parsing the text again
const std::string text = R"({"name": "cache", "host": "db1", "port": 6379, "replicas": ["db2", "db3"]})";
const json_document config = json_document::parse(text);
// save() turns the parsed document into a byte buffer: a 64-byte header,
// the node index, the source text, and the decoded strings
const std::vector<std::uint8_t> image = config.save();
std::cout << image.size() << '\n';
// the same document always saves to the same bytes
std::cout << (image == json_document::parse(text).save()) << '\n';
// loading the image back needs no parsing, yet dumps exactly what
// parsing the text produced
const json_document reloaded = json_document::load(image);
std::cout << (reloaded.root().dump() == config.root().dump()) << '\n';
}
@@ -1,3 +0,0 @@
316
true
true
@@ -1,41 +0,0 @@
#include <iostream>
#include <nlohmann/json_view.hpp>
using json = nlohmann::json;
using json_editable_document = nlohmann::json_editable_document;
using json_editable_view = nlohmann::json_editable_view;
int main()
{
// a configuration file, as it might be read from disk -- "price" is
// written with a trailing zero that has no effect on its value
const std::string text = R"({
"name": "cache",
"host": "db1",
"port": 6379,
"price": 19.90,
"replicas": ["db2", "db3"],
"timeout": 30
})";
json_editable_document doc = json_editable_document::parse(text);
doc.set(doc.root()["port"], 6380); // (1) replace a value
doc.set(doc.root(), "region", "us-east"); // (2) add a member
doc.set(doc.root()["replicas"], 0, "db4"); // (3) assign an element
doc.set(json::json_pointer("/timeout"), 45); // (4) via a JSON pointer
// members stay in document order (the new one at the end), and a number
// that was not itself edited keeps its exact spelling
std::cout << doc.root().dump(2, ' ', false, json_editable_view::number_format::source) << "\n\n";
// the same edits on a plain json value: object_t is a std::map, so
// parsing already sorted the keys, and dump() rewrites every number to
// its shortest form, even "price", which was never touched
json plain = json::parse(text);
plain["port"] = 6380;
plain["region"] = "us-east";
plain["replicas"][0] = "db4";
plain[json::json_pointer("/timeout")] = 45;
std::cout << plain.dump(2) << '\n';
}
@@ -1,25 +0,0 @@
{
"name": "cache",
"host": "db1",
"port": 6380,
"price": 19.90,
"replicas": [
"db4",
"db3"
],
"timeout": 45,
"region": "us-east"
}
{
"host": "db1",
"name": "cache",
"port": 6380,
"price": 19.9,
"region": "us-east",
"replicas": [
"db4",
"db3"
],
"timeout": 45
}
@@ -1,30 +0,0 @@
#include <iostream>
#include <nlohmann/json_view.hpp>
using json = nlohmann::json;
using json_editable_document = nlohmann::json_editable_document;
using json_editable_view = nlohmann::json_editable_view;
int main()
{
// a configuration file, as it might be read from disk
const std::string text = R"({"name": "cache", "host": "db1", "port": 6379, "price": 19.90})";
std::cout << text << "\n\n";
// patch two fields -- "price" is never touched
json_editable_document doc = json_editable_document::parse(text);
doc.set(doc.root(), "host", "db2");
doc.set(doc.root(), "retries", 3);
// member order (the new member at the end) and the untouched number's
// exact spelling survive
std::cout << doc.root().dump(-1, ' ', false, json_editable_view::number_format::source) << '\n';
// the same patch on a plain json value: keys are sorted (object_t is a
// std::map), and "price" is rewritten even though the patch never
// touched it
json plain = json::parse(text);
plain["host"] = "db2";
plain["retries"] = 3;
std::cout << plain.dump() << '\n';
}
@@ -1,4 +0,0 @@
{"name": "cache", "host": "db1", "port": 6379, "price": 19.90}
{"name":"cache","host":"db2","port":6379,"price":19.90,"retries":3}
{"host":"db2","name":"cache","port":6379,"price":19.9,"retries":3}
@@ -1,15 +0,0 @@
#include <iostream>
#include <nlohmann/json_view.hpp>
using ordered_json_editable_document = nlohmann::ordered_json_editable_document;
int main()
{
// ordered_json_editable_document is basic_json_document<nlohmann::ordered_json, true>
ordered_json_editable_document doc = ordered_json_editable_document::parse(R"({"z": 1, "a": 2, "m": 3})");
doc.set(doc.root(), "b", 4); // set() always appends a new member at the end
// materialize() preserves the document order (with "b" at the end),
// instead of sorting the keys the way json_editable_document does
std::cout << doc.root().materialize().dump() << '\n';
}
@@ -1 +0,0 @@
{"z":1,"a":2,"m":3,"b":4}
-2
View File
@@ -14,8 +14,6 @@ C++ types, and finally serialize it again.
[parsing untrusted input](parsing/untrusted_input.md).
- [Zero-copy JSON views](json_view.md) — read a JSON text through a flat index instead of building a `json` tree;
strings and numbers stay in the input and are only decoded when needed.
- [Editable documents](json_view.md#editing-a-document) — modify a parsed document without building a `json` tree.
- [Images](json_view.md#images) — save a parsed document and load it again without parsing it.
- [Comments](comments.md) and [trailing commas](trailing_commas.md) — opt-in relaxations of the JSON grammar.
## Accessing and modifying values
+9 -125
View File
@@ -180,10 +180,9 @@ document: `#!cpp auto v = json_document::parse(text).root();` does not compile.
**Why the first?** A lookup can stop as soon as it finds a match. Returning the last member would force every lookup
to scan all members of the object, even when the key is found at the very first one: this made lookups in small
objects 1.6 to 3.4 times slower. The hash index of the keys of objects with 128 members or more leads to the first member of a
key as well. Other zero-copy parsers that index the source text, such as yyjson and simdjson, also return the first
member. RFC 8259 only says that names within an object SHOULD be unique and that the behavior of a receiver that sees
duplicates is unpredictable, so neither choice is wrong.
objects 1.6 to 3.4 times slower. Other zero-copy parsers that index the source text, such as yyjson and simdjson, also
return the first member. RFC 8259 only says that names within an object SHOULD be unique and that the behavior of a
receiver that sees duplicates is unpredictable, so neither choice is wrong.
**What stays the same as `parse()`?** [`materialize()`](../api/basic_json_view/materialize.md) and
[`get<std::map<...>>()`](../api/basic_json_view/get.md) replay every member in order, so they keep the *last* value
@@ -249,129 +248,14 @@ a number to its parsed `#!cpp double`/`#!cpp int64_t` value.
[`operator<<`](../api/basic_json_view/operator_ltlt.md) writes a view to a stream the way `basic_json`'s does, using
the stream's `width`/`fill` for indentation.
## Editing a document
Everything above is read-only: a `json_document`/`json_view` lets you look at a parsed text without copying it, but
not change it. [`basic_json_document<BasicJsonType, true>`](../api/basic_json_document/index.md) -- more conveniently
spelled [`json_editable_document`](../api/json_editable_document.md) or
[`ordered_json_editable_document`](../api/ordered_json_editable_document.md) -- also lets you
[`set`](../api/basic_json_document/set.md) a value, [`push_back`](../api/basic_json_document/push_back.md) onto or
[`insert`](../api/basic_json_document/insert.md) into an array, and [`erase`](../api/basic_json_document/erase.md)
an object member or an array element, still without ever building a `basic_json` tree for parts you do not touch.
`#!cpp Editable` defaults to `#!cpp false`, so `json_document`/`ordered_json_document` are unaffected -- they carry
none of the bookkeeping edits need, and calling `set`/`push_back`/`insert`/`erase` on one is a compile error, not a
runtime one.
### Why: editing without reformatting
The `#!cpp 19.90` price from [above](#writing-a-view-back) is exactly the kind of value that makes editing a `json`
or `ordered_json` value in place lossy. Say you parse a configuration file, patch one field, and write it back:
- **`json`** re-sorts every key on the way in (`object_t` is a `#!cpp std::map`) and rewrites every number to its
shortest round-trip form on the way out -- a one-field patch turns into a diff that reorders the whole file and
rewrites `#!cpp 19.90` to `#!cpp 19.9`.
- **`ordered_json`** keeps the key order, but still rewrites every number the same way: parsing has already reduced
it to a `#!cpp double`/`#!cpp int64_t`, and there is no way back to how it was spelled in the source text.
An editable document keeps both. [`dump()`](../api/basic_json_view/dump.md) of an edited document writes members in
document order -- a member [`set`](../api/basic_json_document/set.md) added goes at the end, exactly where it was
inserted, and an [`erase`](../api/basic_json_document/erase.md)d member simply leaves a gap: everything around it
keeps its place -- and [`number_format::source`](../api/basic_json_view/number_format.md) keeps the exact spelling
of every number an edit did not itself touch; a number an edit *did* touch is written the way
[`BasicJsonType::dump()`](../api/basic_json/dump.md) would write it, since there is no source spelling for a brand
new value.
??? example "Example: patch a configuration, keeping member order and number spellings"
```cpp
--8<-- "examples/json_editable_document.cpp"
```
Output:
```json
--8<-- "examples/json_editable_document.output"
```
### What stays valid, and what an edit costs
The source text itself is **never written**, and the parsed index never moves -- a value keeps the node it was
parsed into for as long as it is not itself replaced. So every [view](../api/basic_json_view/index.md) taken before
an edit, including a previously obtained [`root()`](../api/basic_json_document/root.md), stays valid and, if it
still refers to the edited value, sees the edit; a view of a value a later edit drops or replaces just keeps showing
what it last held. New values go to storage the document allocates and owns on demand. The one thing an edit does
invalidate is the **iterators** taken over an edited array or object: the first time one of its elements is set,
appended to, inserted into, or erased, its elements move from the parsed, fixed layout to a growable block of links
so that [`push_back`](../api/basic_json_document/push_back.md) can later grow it in amortized constant time --
existing elements are not touched, but an iterator that was walking the old layout no longer matches. A string
obtained with
[`get_string()`](../api/basic_json_view/get_string.md) is unaffected either way and stays valid across further
edits. See [`basic_json_document`'s Edits](../api/basic_json_document/index.md#edits) for the details, and
[`set`'s Exception safety](../api/basic_json_document/set.md#exception-safety) for what an edit guarantees if it
throws (the *basic* guarantee, not the strong one `dump()` and the read-only functions provide). How edits are kept in
the index is described in the [architecture overview](../home/architecture.md#node-index-of-json-views).
## Images
[`save()`](../api/basic_json_document/save.md) writes a document as an *image*: a byte buffer that
[`load()`](../api/basic_json_document/load.md) reads back into a document without parsing -- no lexing, no building
the node index, nothing but copying the nodes and pointing the text and the decoded strings at the image. Where
[`parse_copy()`](../api/basic_json_document/parse_copy.md) still has to scan the whole input,
[`load()`](../api/basic_json_document/load.md) turns that scan into a copy of the node index alone.
**Why.** A document that is parsed once and then read many times -- a configuration loaded at startup, a template
rendered on every request, a large reference dataset a worker process needs in memory -- pays for parsing once but
can amortize [`save()`](../api/basic_json_document/save.md)'s cost across every later load. That makes images useful
for a cache: save a document the first time it is parsed (to a file, a shared-memory segment, an in-process cache),
and [`load()`](../api/basic_json_document/load.md) it on every later use instead of parsing the source text again.
They are just as useful for handing a parsed document to another process (or a forked worker) running the same build
of the library, since [`load()`](../api/basic_json_document/load.md) turns the transfer into a copy of the node index
plus pointers into the received bytes, not a re-parse.
**Choosing a check.** [`load()`](../api/basic_json_document/load.md) takes an
[`image_check`](../api/basic_json_document/load.md#image_check) that trades validation against speed:
`image_check::full` (the default) checks everything the parser itself guarantees, so a checked image is exactly as
safe to read and serialize as a freshly parsed document -- the right choice whenever the image did not come straight
from this process's own [`save()`](../api/basic_json_document/save.md), such as a file or a network peer.
`image_check::bounds` only checks structure and bounds -- cheaper, since it skips scanning the text and the decoded
strings -- and fits a cache the process trusts, one it wrote and reads back itself. `image_check::none` skips
validation entirely, for an image trusted as much as the process's own memory. See
[`load()`'s Notes](../api/basic_json_document/load.md#notes) for exactly what each level does and does not guarantee.
??? example "Example: cache a parsed configuration as an image"
```cpp
--8<-- "examples/basic_json_document__save.cpp"
```
Output:
```json
--8<-- "examples/basic_json_document__save.output"
```
!!! warning "Experimental"
The image format is versioned but not yet stable, and may change in an incompatible way before it is declared
stable. It is little-endian only, and tied to the library build that wrote it -- use it to cache a document or to
hand one to another process running the *same* build, not as a long-term storage format; keep the original JSON
text if a saved document needs to be readable by a future library version.
The idea of a document you can read without parsing comes from zero-copy formats such as
[FlatBuffers](https://github.com/google/flatbuffers) and [YaFF](https://github.com/yandex/yaff); the check
[`load()`](../api/basic_json_document/load.md) runs follows the idea of FlatBuffers' Verifier. No code is taken from
either.
## Choosing between `json`, `ordered_json`, the SAX interface, and `json_view`
| | [`json`](../api/json.md) / [`ordered_json`](../api/ordered_json.md) | [SAX interface](parsing/sax_interface.md) | [`json_document`](../api/json_document.md) / [`json_view`](../api/json_view.md) | [`json_editable_document`](../api/json_editable_document.md) / [`json_editable_view`](../api/json_editable_view.md) |
|---|---|---|---|---|
| **Ownership** | owns every value | owns nothing; you decide what to keep, in your handler | borrows or owns the *text*; the index is always owned by the document | same as `json_document`; edits go to storage the document owns |
| **Mutability** | freely mutable | not applicable (a one-shot event stream) | read-only | [`set`](../api/basic_json_document/set.md)/[`push_back`](../api/basic_json_document/push_back.md)/[`insert`](../api/basic_json_document/insert.md)/[`erase`](../api/basic_json_document/erase.md) edit in place; the source text is never rewritten |
| **What you get** | a full tree you can read, write, and keep as long as you like | a sequence of callbacks; whatever your handler builds from them | a flat index plus, on demand, [`materialize()`](../api/basic_json_view/materialize.md)d `json`/`ordered_json` values for the parts you actually use | the same, plus [`dump()`](../api/basic_json_view/dump.md) of an edited document that keeps the member order and, with [`number_format::source`](../api/basic_json_view/number_format.md), the spelling of every untouched number |
| **Typical use** | general-purpose JSON handling: config, request/response bodies you build or modify, anything you hold onto | validating or projecting a text into your own data structure without ever holding the whole thing as JSON | large or high-volume input where you only need part of it, or need it repeatedly, and can keep the source text (or a copy) alive for as long as the document lives | a document you read, patch a few fields of, and write back -- a configuration file, for instance -- where the rest of it should come back exactly as it was |
| **Caching/reload** | not applicable -- re-parse, or roll your own serialization | not applicable | [`save()`](../api/basic_json_document/save.md)/[`load()`](../api/basic_json_document/load.md): cache the parsed index as an image and reload it without parsing | same, saving the document's current -- possibly edited -- state |
| | [`json`](../api/json.md) / [`ordered_json`](../api/ordered_json.md) | [SAX interface](parsing/sax_interface.md) | [`json_document`](../api/json_document.md) / [`json_view`](../api/json_view.md) |
|---|---|---|---|
| **Ownership** | owns every value | owns nothing; you decide what to keep, in your handler | borrows or owns the *text*; the index is always owned by the document |
| **Mutability** | freely mutable | not applicable (a one-shot event stream) | read-only |
| **What you get** | a full tree you can read, write, and keep as long as you like | a sequence of callbacks; whatever your handler builds from them | a flat index plus, on demand, [`materialize()`](../api/basic_json_view/materialize.md)d `json`/`ordered_json` values for the parts you actually use |
| **Typical use** | general-purpose JSON handling: config, request/response bodies you build or modify, anything you hold onto | validating or projecting a text into your own data structure without ever holding the whole thing as JSON | large or high-volume input where you only need part of it, or need it repeatedly, and can keep the source text (or a copy) alive for as long as the document lives |
## Version history
+2 -6
View File
@@ -27,8 +27,6 @@ The complete default namespace name is derived as follows:
`_dsbs`.
- [`JSON_DISABLE_UNLOCKED_FILE_IO`](../api/macros/json_disable_unlocked_file_io.md) defined non-zero appends
`_dufio`.
- [`JSON_VIEW_NO_SIMD`](../api/macros/json_view_no_simd.md) defined appends `_vns`.
- [`JSON_VIEW_USE_SSSE3`](../api/macros/json_view_use_ssse3.md) defined appends `_vssse3`.
- The inline namespace ends with the suffix `_v` followed by the 3 components of the version number separated by
underscores. To omit the version component, see [Disabling the version component](#disabling-the-version-component)
below.
@@ -113,7 +111,5 @@ follows:
([`JSON_STRICT_NUL_HANDLING`](../api/macros/json_strict_nul_handling.md)), `_sbu8`
([`JSON_STRICT_BINARY_UTF8`](../api/macros/json_strict_binary_utf8.md)), `_ekmo`
([`JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS`](../api/macros/json_use_objects_for_enum_keyed_maps.md)), `_dsbs`
([`JSON_DISABLE_STREAM_BULK_SCAN`](../api/macros/json_disable_stream_bulk_scan.md)), `_dufio`
([`JSON_DISABLE_UNLOCKED_FILE_IO`](../api/macros/json_disable_unlocked_file_io.md)), `_vns`
([`JSON_VIEW_NO_SIMD`](../api/macros/json_view_no_simd.md)), and `_vssse3`
([`JSON_VIEW_USE_SSSE3`](../api/macros/json_view_use_ssse3.md)) in version 3.13.0.
([`JSON_DISABLE_STREAM_BULK_SCAN`](../api/macros/json_disable_stream_bulk_scan.md)), and `_dufio`
([`JSON_DISABLE_UNLOCKED_FILE_IO`](../api/macros/json_disable_unlocked_file_io.md)) in version 3.13.0.
+3 -33
View File
@@ -194,9 +194,9 @@ packet-beta
| Bytes | Field | Type | Contents |
|-------|---------|------------|-------------------------------------------------------------------------------------------------------------------------------|
| 0 | `kind` | `uint8_t` | the type, numbered as [`value_t`](../api/basic_json/value_t.md): 0 null, 1 object, 2 array, 3 string, 4 boolean, 5 signed integer, 6 unsigned integer, 7 float; 10 for a link (see below). There is no node for a discarded value: a [discarded](../api/basic_json_view/is_discarded.md) view has none |
| 1 | `flags` | `uint8_t` | bits 0-1: where a string's bytes (or a number's token) are (0: the source text, 1: the buffer of decoded strings, for strings with escapes, 2: the edit buffer); bit 2: the value of a boolean; bit 3: moved, bit 4: new, bit 5: linked (see below) |
| 2-3 | `extra` | `uint16_t` | numbers: the number of integer digits (low byte) and fraction digits (high byte), 255 for more; objects: the number of their key hash index (1-based), or 0; otherwise 0 |
| 0 | `kind` | `uint8_t` | the type, numbered as [`value_t`](../api/basic_json/value_t.md): 0 null, 1 object, 2 array, 3 string, 4 boolean, 5 signed integer, 6 unsigned integer, 7 float |
| 1 | `flags` | `uint8_t` | bits 0-1: where a string's bytes are (0: the source text, 1: the buffer of decoded strings, for strings with escapes); bit 2: the value of a boolean |
| 2-3 | `extra` | `uint16_t` | numbers: the number of integer digits (low byte) and fraction digits (high byte), 255 for more; otherwise 0 |
| 4-7 | `off` | `uint32_t` | where the value starts: the first byte after a string's opening quote (or its position in the buffer of decoded strings), the first byte of a number or literal, the bracket of an array or object |
| 8-11 | `len` | `uint32_t` | strings: the length after decoding; floats and literals: the length of the token; arrays and objects: the number of elements |
| 12-15 | `next` | `uint32_t` | arrays and objects: the number of nodes of the subtree, including the node itself |
@@ -212,11 +212,6 @@ packet-beta
step from element to element this way and skip whole subtrees in constant time.
- **Offsets** are 32 bits wide, so a document is limited to 4294967279 bytes, 4 GiB minus 16 bytes (a margin below
2^32 for positions one scanner step past the end of the text; `out_of_range.416`).
- **Large objects** (128 members or more) get a hash index for their keys after parsing
([`detail/view/object_index.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/view/object_index.hpp)):
an open-addressing table whose slots hold the distance from the object's node to a key's node, so that a lookup does
not compare every key. Of duplicate keys the table leads to the first, as a linear search does. The object's `extra`
holds the number of its table. Only 65,535 tables fit into `extra`; objects beyond them are searched linearly.
For example, `#!json {"a": [1, 2.5]}` becomes five nodes. Each node's elements follow it, and `next` leads from an
array or object past its subtree:
@@ -245,31 +240,6 @@ flowchart LR
All `flags` are 0. The integer's bytes 8-15 hold its value, 1; its `extra` says it has one digit. The float's `extra`
says it has one integer and one fraction digit, and its `len` is that of the token `2.5`.
Editable documents ([`json_editable_document`](../api/json_editable_document.md),
[`detail/view/edit.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/view/edit.hpp) and
[`detail/view/edit_storage.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/view/edit_storage.hpp))
never write the source text and never move or resize the parsed index, so views stay valid while the document is
edited:
- A new scalar is written over its node. Its text (a string, or the token of a number as `dump()` writes it) goes to
the edit buffer, which `flags` bits 0-1 then name.
- An array or object whose elements change gets the flag *moved* (bit 3): its elements then live in a separate
sequence (a header node, then the entries), whose number is in `off`. The entries are links (`kind` 10), whose bytes
8-15 hold the address of the value's node, so values never move.
- A node written by an edit gets the flag *new* (bit 4): it has no position in the source text.
- A value that an entry of such a sequence links to gets the flag *linked* (bit 5): its extent in the parsed layout
(`next`) no longer matters, so a scalar can replace an array or object there in place.
- Views of read-only documents compile without any of this: how views walk the index is a template parameter
(`navigation<Editable>`).
Images ([`save`](../api/basic_json_document/save.md) and [`load`](../api/basic_json_document/load.md),
[`detail/view/image.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/view/image.hpp)) store
the nodes as they are: a 64-byte header (the magic bytes `NJVI`, a format version, the sizes, and reserved bytes that
must be zero), the nodes, the text, and the decoded strings. An edited document is first written in document order, as
the parser would have written it (without links), and the numbers of the hash indexes are cleared, since `load`
rebuilds the indexes. So a change of the node layout is a change of the image format: it must raise `image_version`,
and `load` then rejects images of other versions (`parse_error.116`) instead of misreading them.
## Input adapters
Input is read via **input adapters** that abstract a source. Every input adapter provides this interface:
+6 -89
View File
@@ -395,23 +395,6 @@ A UBJSON high-precision number could not be parsed.
[json.exception.parse_error.115] parse error at byte 5: syntax error while parsing UBJSON high-precision number: invalid number text: 1A
```
### json.exception.parse_error.116
[`basic_json_document::load()`](../api/basic_json_document/load.md) rejected an
[image](../features/json_view.md#images): either the bytes are not one [`save()`](../api/basic_json_document/save.md)
could have written (too short, an unknown magic number or format version, or sizes that do not fit the buffer), or
they are, but fail the requested [`image_check`](../api/basic_json_document/load.md#image_check).
!!! failure "Example message"
```
[json.exception.parse_error.116] parse error: invalid json_document image: the check failed
```
!!! note
This exception was added in version 3.13.0, together with [images](../features/json_view.md#images).
## Iterator errors
This exception is thrown if iterators passed to a library function do not match
@@ -829,45 +812,6 @@ does not list an enumerator and it is therefore converted like the first listed
[json.exception.type_error.318] duplicate object key 'red'
```
### json.exception.type_error.319
[`basic_json_document::set`](../api/basic_json_document/set.md),
[`basic_json_document::push_back`](../api/basic_json_document/push_back.md), and
[`basic_json_document::insert`](../api/basic_json_document/insert.md) can store any `basic_json` value except
a binary one: a `json_document` has no representation for [binary values](../features/binary_values.md), which only
ever arise from parsing a binary format or from an explicit [`json::binary`](../api/basic_json/binary.md) value, not
from JSON text.
!!! failure "Example message"
```
[json.exception.type_error.319] cannot store a binary value in a json_document
```
!!! note
This exception was added in version 3.13.0, together with editable [`json_document`s](../features/json_view.md).
### json.exception.type_error.320
[`basic_json_document::save()`](../api/basic_json_document/save.md) cannot write an
[image](../features/json_view.md#images) of a [discarded](../api/basic_json_document/is_discarded.md) document.
[`save()`](../api/basic_json_document/save.md) and [`load()`](../api/basic_json_document/load.md) also throw this
exception on a big-endian target, since the image format is little-endian only.
!!! failure "Example messages"
```
[json.exception.type_error.320] cannot save a discarded json_document
```
```
[json.exception.type_error.320] json_document images need a little-endian target
```
!!! note
This exception was added in version 3.13.0, together with [images](../features/json_view.md#images).
### json.exception.type_error.321
A discarded value (one created by [`parse()`](../api/basic_json/parse.md) with a callback that returns `false` for the
@@ -972,19 +916,9 @@ double-precision number when `number_float_t` is `#!cpp float`.
### json.exception.out_of_range.407
An integer number cannot be represented by the binary format it is serialized to:
- [BON8](../features/binary_formats/bon8.md) only stores integers that fit into int64, so
[`to_bon8`](../api/basic_json/to_bon8.md) throws for unsigned integers greater than 9223372036854775807.
- [CBOR](../features/binary_formats/cbor.md), [MessagePack](../features/binary_formats/messagepack.md), and
[BSON](../features/binary_formats/bson.md) store integers in at most 64 bits. With the default number types, every
integer fits, but a [`number_integer_t`](../api/basic_json/number_integer_t.md) or
[`number_unsigned_t`](../api/basic_json/number_unsigned_t.md) wider than 64 bits (e.g., `__int128`) can hold values
outside the range of the format: [-2^64, 2^64-1] for CBOR, [-2^63, 2^64-1] for MessagePack, and the range of int64
(signed integers) or uint64 (unsigned integers) for BSON.
[UBJSON](../features/binary_formats/ubjson.md) and [BJData](../features/binary_formats/bjdata.md) never throw this
exception, because they serialize integers beyond 64 bits as high-precision numbers.
An unsigned integer number greater than 9223372036854775807 cannot be represented in the binary format. This exception
is only thrown by [`to_bon8`](../api/basic_json/to_bon8.md), as [BON8](../features/binary_formats/bon8.md) has no
unsigned 64-bit integer type.
!!! note "Former occurrences"
@@ -993,17 +927,11 @@ exception, because they serialize integers beyond 64 bits as high-precision numb
- Since version 3.9.0, integer numbers beyond int64 are serialized as high-precision UBJSON numbers.
- Since version 3.12.0, integer numbers beyond int64 are serialized as uint64 BSON numbers.
Before version 3.13.0, CBOR, MessagePack, and BSON silently truncated integers wider than 64 bits, and BJData
truncated unsigned integers wider than 64 bits.
!!! failure "Example messages"
!!! failure "Example message"
```
integer number 18446744073709551615 cannot be represented by BON8 as it does not fit int64
```
```
integer number 1267650600228229401496703205376 cannot be represented by CBOR as it does not fit [-2^64, 2^64-1]
```
### json.exception.out_of_range.408
@@ -1138,24 +1066,13 @@ MessagePack's ext type and BSON's binary subtype are each stored in a single byt
[`basic_json_document::parse()`](../api/basic_json_document/parse.md) and the other parsing functions of
[`basic_json_document`](../api/basic_json_document/index.md) index a value's position in the source text in 32 bits,
so they do not support an input of 4294967280 bytes (4 GiB minus 16 bytes) or more. The same 32-bit limit applies to an **editable** document's own
storage: [`set`](../api/basic_json_document/set.md) and [`push_back`](../api/basic_json_document/push_back.md) throw
this exception once the strings and number tokens written by edits reach 4 GiB in total, or once more than
4294967295 arrays/objects have had an element set or appended to them. The same limit applies to an
[image](../features/json_view.md#images): [`save()`](../api/basic_json_document/save.md) throws it if the node
count, the text, or the decoded strings it would write would individually reach 4 GiB.
so they do not support an input of 4294967280 bytes (4 GiB minus 16 bytes) or more.
!!! failure "Example messages"
!!! failure "Example message"
```
[json.exception.out_of_range.416] input of 4294967280 bytes or more is not supported by json_document
```
```
[json.exception.out_of_range.416] edits of 4 GiB or more are not supported by json_document
```
```
[json.exception.out_of_range.416] images of 4 GiB or more are not supported by json_document
```
!!! note
-2
View File
@@ -25,5 +25,3 @@ The class contains a copy of [Hedley](https://nemequ.github.io/hedley/) from Eva
The class contains an adapted version of the Eisel-Lemire algorithm, its table of powers of five, and its digit comparison for long numbers from [fast_float](https://github.com/fastfloat/fast_float) by Daniel Lemire and contributors, which is available under the [MIT License](https://opensource.org/licenses/MIT) (used here), the Apache 2.0 License, and the Boost Software License. Copyright &copy; 2021 The fast_float authors
The view's parser (`<nlohmann/json_view.hpp>`) contains techniques and code adapted from [yyjson](https://github.com/ibireme/yyjson) by YaoYuan, which is licensed under the [MIT License](https://opensource.org/licenses/MIT) (see above): table-driven decoding of `\u` escapes and fixed-offset unrolled checks.
The view's parser (`<nlohmann/json_view.hpp>`) validates non-ASCII strings with the vector UTF-8 check of [simdjson](https://github.com/simdjson/simdjson) by Daniel Lemire, Geoff Langdale, John Keiser, and contributors (its "lookup4" algorithm and tables, after J. Keiser and D. Lemire, "Validating UTF-8 In Less Than One Instruction Per Byte", 2021), which is available under the [MIT License](https://opensource.org/licenses/MIT) (used here) and the Apache 2.0 License. Copyright &copy; 2018-2025 The simdjson authors
-11
View File
@@ -235,17 +235,6 @@ Build the unit tests against the [simdutf](https://github.com/simdutf/simdutf) U
its version is set by the cache variable `JSON_SIMDUTF_VERSION`. This option is `OFF` by default. Depends on
`JSON_BuildTests`.
### `JSON_TestUnityBuild`
Build the unit tests in batches of several test files per executable to speed up compilation: the files of a batch are
compiled as one translation unit, so the template instantiations of the library are shared. Every test file still gets
its own CTest test (e.g., `test-foo_cpp11`), which runs only the test cases of that file from the shared executable.
This option is `ON` by default (and `OFF` with MinGW). Set it to `OFF` to get one executable per test file, e.g., when
debugging a single file. The number of files per executable is set by the cache variable `JSON_TestUnityBatchSize`
(default: `8`). Related test files (e.g., all binary formats) are grouped explicitly in `tests/CMakeLists.txt`; each
group is compiled as one executable per C++ standard regardless of the batch size, and only the remaining files are
batched by size. Depends on `JSON_BuildTests`.
### `JSON_Valgrind`
Execute the test suite with [Valgrind](https://valgrind.org). This option is `OFF` by default. Depends on `JSON_BuildTests`.
-12
View File
@@ -239,20 +239,14 @@ nav:
- 'Overview': api/basic_json_document/index.md
- '(Constructor)': api/basic_json_document/basic_json_document.md
- 'accept': api/basic_json_document/accept.md
- 'erase': api/basic_json_document/erase.md
- 'insert': api/basic_json_document/insert.md
- 'is_discarded': api/basic_json_document/is_discarded.md
- 'load': api/basic_json_document/load.md
- 'memory_usage': api/basic_json_document/memory_usage.md
- 'node_count': api/basic_json_document/node_count.md
- 'owns_source': api/basic_json_document/owns_source.md
- 'parse': api/basic_json_document/parse.md
- 'parse_copy': api/basic_json_document/parse_copy.md
- 'push_back': api/basic_json_document/push_back.md
- 'read': api/basic_json_document/read.md
- 'root': api/basic_json_document/root.md
- 'save': api/basic_json_document/save.md
- 'set': api/basic_json_document/set.md
- 'shrink_to_fit': api/basic_json_document/shrink_to_fit.md
- 'source': api/basic_json_document/source.md
- basic_json_view:
@@ -314,8 +308,6 @@ nav:
- 'to_json': api/adl_serializer/to_json.md
- 'json': api/json.md
- 'json_document': api/json_document.md
- 'json_editable_document': api/json_editable_document.md
- 'json_editable_view': api/json_editable_view.md
- json_pointer:
- 'Overview': api/json_pointer/index.md
- '(Constructor)': api/json_pointer/json_pointer.md
@@ -357,8 +349,6 @@ nav:
- 'operator""_json_pointer': api/operator_literal_json_pointer.md
- 'ordered_json': api/ordered_json.md
- 'ordered_json_document': api/ordered_json_document.md
- 'ordered_json_editable_document': api/ordered_json_editable_document.md
- 'ordered_json_editable_view': api/ordered_json_editable_view.md
- 'ordered_json_view': api/ordered_json_view.md
- 'ordered_map': api/ordered_map.md
- macros:
@@ -394,8 +384,6 @@ nav:
- 'JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON': api/macros/json_use_legacy_discarded_value_comparison.md
- 'JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS': api/macros/json_use_objects_for_enum_keyed_maps.md
- 'JSON_USE_SIMDUTF': api/macros/json_use_simdutf.md
- 'JSON_VIEW_NO_SIMD': api/macros/json_view_no_simd.md
- 'JSON_VIEW_USE_SSSE3': api/macros/json_view_use_ssse3.md
- 'NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE, NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE_WITH_DEFAULT, NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE_ONLY_SERIALIZE, NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE, NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_WITH_DEFAULT, NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_ONLY_SERIALIZE': api/macros/nlohmann_define_derived_type.md
- 'NLOHMANN_DEFINE_TYPE_INTRUSIVE, NLOHMANN_DEFINE_TYPE_INTRUSIVE_WITH_DEFAULT, NLOHMANN_DEFINE_TYPE_INTRUSIVE_ONLY_SERIALIZE': api/macros/nlohmann_define_type_intrusive.md
- 'NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE, NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE_WITH_DEFAULT, NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE_ONLY_SERIALIZE': api/macros/nlohmann_define_type_non_intrusive.md
+4 -18
View File
@@ -122,26 +122,14 @@
#define NLOHMANN_JSON_ABI_TAG_DISABLE_UNLOCKED_FILE_IO
#endif
#ifdef JSON_VIEW_NO_SIMD
#define NLOHMANN_JSON_ABI_TAG_VIEW_NO_SIMD _vns
#else
#define NLOHMANN_JSON_ABI_TAG_VIEW_NO_SIMD
#endif
#ifdef JSON_VIEW_USE_SSSE3
#define NLOHMANN_JSON_ABI_TAG_VIEW_USE_SSSE3 _vssse3
#else
#define NLOHMANN_JSON_ABI_TAG_VIEW_USE_SSSE3
#endif
#ifndef NLOHMANN_JSON_NAMESPACE_NO_VERSION
#define NLOHMANN_JSON_NAMESPACE_NO_VERSION 0
#endif
// Construct the namespace ABI tags component
#define NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f, g, h, i, j, k, l) json_abi ## a ## b ## c ## d ## e ## f ## g ## h ## i ## j ## k ## l
#define NLOHMANN_JSON_ABI_TAGS_CONCAT(a, b, c, d, e, f, g, h, i, j, k, l) \
NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f, g, h, i, j, k, l)
#define NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f, g, h, i, j) json_abi ## a ## b ## c ## d ## e ## f ## g ## h ## i ## j
#define NLOHMANN_JSON_ABI_TAGS_CONCAT(a, b, c, d, e, f, g, h, i, j) \
NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f, g, h, i, j)
#define NLOHMANN_JSON_ABI_TAGS \
NLOHMANN_JSON_ABI_TAGS_CONCAT( \
@@ -154,9 +142,7 @@
NLOHMANN_JSON_ABI_TAG_STRICT_BINARY_UTF8, \
NLOHMANN_JSON_ABI_TAG_OBJECTS_FOR_ENUM_KEYED_MAPS, \
NLOHMANN_JSON_ABI_TAG_DISABLE_STREAM_BULK_SCAN, \
NLOHMANN_JSON_ABI_TAG_DISABLE_UNLOCKED_FILE_IO, \
NLOHMANN_JSON_ABI_TAG_VIEW_NO_SIMD, \
NLOHMANN_JSON_ABI_TAG_VIEW_USE_SSSE3)
NLOHMANN_JSON_ABI_TAG_DISABLE_UNLOCKED_FILE_IO)
// Construct the namespace version component
#define NLOHMANN_JSON_NAMESPACE_VERSION_CONCAT_EX(major, minor, patch) \
@@ -1342,66 +1342,6 @@ inline char* write_shortest(char* first, const zmij::shortest_decimal d) noexcep
return end + (three ? 5 : 4);
}
/// the powers of ten up to 10^19 (all that fit in std::uint64_t)
inline const std::array<std::uint64_t, 20>& powers_of_ten() noexcept
{
static const std::array<std::uint64_t, 20> powers =
{
{
1u,
10u,
100u,
1000u,
10000u,
100000u,
1000000u,
10000000u,
100000000u,
1000000000u,
10000000000u,
100000000000u,
1000000000000u,
10000000000000u,
100000000000000u,
1000000000000000u,
10000000000000000u,
100000000000000000u,
1000000000000000000u,
10000000000000000000u
}
};
return powers;
}
/*!
@brief digits * 10^exp, as write_decimal() writes it, for the digits of a
double that need no conversion (count digits, at most 15, the first not 0;
trailing zeros allowed): extended to 16 digits and written by write_shortest()
@return a pointer past the text; up to 41 bytes at @a first are written
(some beyond the returned end)
*/
JSON_HEDLEY_NON_NULL(1)
JSON_HEDLEY_RETURNS_NON_NULL
inline char* write_short_decimal(char* first, std::uint64_t digits, int count, int exp) noexcept
{
JSON_ASSERT(digits >= powers_of_ten()[static_cast<std::size_t>(count - 1)] && count <= 15);
const int scale = 16 - count;
return write_shortest(first, zmij::shortest_decimal{digits * powers_of_ten()[static_cast<std::size_t>(scale)], exp - scale - 1, 0, false});
}
/// as write_short_decimal(), counting the digits (not 0, less than 10^15)
JSON_HEDLEY_NON_NULL(1)
JSON_HEDLEY_RETURNS_NON_NULL
inline char* write_short_decimal(char* first, std::uint64_t digits, int exp) noexcept
{
JSON_ASSERT(digits != 0 && digits < 1000000000000000u);
// floor(log10(2^bits)) + 1 digits, or one less
const int log2_bound = ((64 - count_leading_zeros(digits)) * 1233) >> 12;
const int count = log2_bound + (digits >= powers_of_ten()[static_cast<std::size_t>(log2_bound)] ? 1 : 0);
return write_short_decimal(first, digits, count, exp);
}
/*!
@brief whether FloatType is an IEEE 754 binary64 type (a double, or a long double
that has the same format, as with MSVC and on Apple's Arm CPUs)
@@ -38,16 +38,8 @@
// 2 = Microsoft CRT _lock_file/_fgetc_nolock, 0 = none (std::fgetc, which
// locks the file for every character). Only C libraries known to declare
// the functions are listed; the macros are tested after <cstdio>, which
// defines __GLIBC__. MemorySanitizer builds always use std::fgetc: glibc's
// inline getc_unlocked reads the stdio buffer directly, which glibc (not
// instrumented) filled, so MemorySanitizer would report a false positive.
#if defined(__has_feature)
#if __has_feature(memory_sanitizer)
#define JSON_UNLOCKED_FILE_IO_MSAN 1
#endif
#endif
#if JSON_DISABLE_UNLOCKED_FILE_IO || defined(JSON_UNLOCKED_FILE_IO_MSAN)
// defines __GLIBC__.
#if JSON_DISABLE_UNLOCKED_FILE_IO
#define JSON_UNLOCKED_FILE_IO 0
#elif defined(_MSC_VER)
#define JSON_UNLOCKED_FILE_IO 2
@@ -56,7 +48,6 @@
#else
#define JSON_UNLOCKED_FILE_IO 0
#endif
#undef JSON_UNLOCKED_FILE_IO_MSAN
#endif // JSON_NO_IO
NLOHMANN_JSON_NAMESPACE_BEGIN
+2 -2
View File
@@ -19,6 +19,8 @@
#undef NLOHMANN_CAN_CALL_STD_FUNC_IMPL
#undef JSON_INLINE_VARIABLE
#undef JSON_NO_UNIQUE_ADDRESS
#undef JSON_DISABLE_ENUM_SERIALIZATION
#undef JSON_DISABLE_TUPLE_REFERENCE_CONVERSION
#undef JSON_UNLOCKED_FILE_IO
#undef JSON_DTOA_SSE2
#undef JSON_DTOA_NEON
@@ -46,8 +48,6 @@
#undef JSON_STRICT_BINARY_UTF8
#undef JSON_DELETE_DEPRECATED_FUNCTIONS
#undef JSON_USE_OBJECTS_FOR_ENUM_KEYED_MAPS
#undef JSON_DISABLE_ENUM_SERIALIZATION
#undef JSON_DISABLE_TUPLE_REFERENCE_CONVERSION
#undef JSON_DISABLE_STREAM_BULK_SCAN
#undef JSON_DISABLE_UNLOCKED_FILE_IO
#endif
+88 -547
View File
@@ -23,7 +23,6 @@
#include <cstdlib> // _byteswap_ushort, _byteswap_ulong, _byteswap_uint64
#endif
#include <nlohmann/detail/bit_ops.hpp>
#include <nlohmann/detail/input/binary_reader.hpp>
#include <nlohmann/detail/input/string_scan.hpp>
#include <nlohmann/detail/macro_scope.hpp>
@@ -205,12 +204,24 @@ class binary_writer
}
case value_t::number_integer:
{
if (j.m_data.m_value.number_integer >= 0)
{
// CBOR does not differentiate between positive signed
// integers and unsigned integers
write_cbor_head(0x00, static_cast<std::uint64_t>(j.m_data.m_value.number_integer));
}
else
{
// a negative integer n is encoded as -1 - n
write_cbor_head(0x20, static_cast<std::uint64_t>(-1 - j.m_data.m_value.number_integer));
}
break;
}
case value_t::number_unsigned:
{
// one case for both, so that an array mixing negative
// (number_integer) and non-negative (number_unsigned) values
// does not mispredict the switch
write_cbor_integer(j, std::integral_constant<bool, integers_fit_64_bits()>());
write_cbor_head(0x00, conditional_static_cast<std::uint64_t>(j.m_data.m_value.number_unsigned));
break;
}
@@ -245,7 +256,13 @@ class binary_writer
case value_t::array:
{
write_cbor_array(j, depth);
// step 1: write control byte and the array size
write_cbor_head(0x80, j.m_data.m_value.array->size());
for (const auto& el : *j.m_data.m_value.array)
{
write_cbor(el, depth + 1);
}
break;
}
@@ -403,12 +420,54 @@ class binary_writer
}
case value_t::number_integer:
{
if (j.m_data.m_value.number_integer >= 0)
{
// MessagePack does not differentiate between positive
// signed integers and unsigned integers.
write_msgpack_unsigned(static_cast<std::uint64_t>(j.m_data.m_value.number_integer));
}
else
{
if (j.m_data.m_value.number_integer >= -32)
{
// negative fixnum
write_number(static_cast<std::int8_t>(j.m_data.m_value.number_integer));
}
else if (j.m_data.m_value.number_integer >= (std::numeric_limits<std::int8_t>::min)() &&
j.m_data.m_value.number_integer <= (std::numeric_limits<std::int8_t>::max)())
{
// int 8
oa.write_character(to_char_type(0xD0));
write_number(static_cast<std::int8_t>(j.m_data.m_value.number_integer));
}
else if (j.m_data.m_value.number_integer >= (std::numeric_limits<std::int16_t>::min)() &&
j.m_data.m_value.number_integer <= (std::numeric_limits<std::int16_t>::max)())
{
// int 16
oa.write_character(to_char_type(0xD1));
write_number(static_cast<std::int16_t>(j.m_data.m_value.number_integer));
}
else if (j.m_data.m_value.number_integer >= (std::numeric_limits<std::int32_t>::min)() &&
j.m_data.m_value.number_integer <= (std::numeric_limits<std::int32_t>::max)())
{
// int 32
oa.write_character(to_char_type(0xD2));
write_number(static_cast<std::int32_t>(j.m_data.m_value.number_integer));
}
else
{
// int 64
oa.write_character(to_char_type(0xD3));
write_number(static_cast<std::int64_t>(j.m_data.m_value.number_integer));
}
}
break;
}
case value_t::number_unsigned:
{
// one case for both, so that an array mixing negative
// (number_integer) and non-negative (number_unsigned) values
// does not mispredict the switch
write_msgpack_integer(j, std::integral_constant<bool, integers_fit_64_bits()>());
write_msgpack_unsigned(static_cast<std::uint64_t>(j.m_data.m_value.number_unsigned));
break;
}
@@ -426,7 +485,13 @@ class binary_writer
case value_t::array:
{
write_msgpack_array(j, depth);
// step 1: write control byte and the array size
write_msgpack_array_prefix(j.m_data.m_value.array->size(), j);
for (const auto& el : *j.m_data.m_value.array)
{
write_msgpack(el, depth + 1);
}
break;
}
@@ -759,142 +824,6 @@ class binary_writer
JSON_THROW(type_error::create(exception_id::discarded_value_used, concat("cannot serialize discarded value to ", format_name), &j));
}
/// how the range of integer type @a NumberType relates to that of @a TargetType
template<typename TargetType, typename NumberType>
struct integer_range
{
/// whether NumberType can hold values above the maximum of TargetType
static constexpr bool may_exceed_max = std::numeric_limits<NumberType>::digits > std::numeric_limits<TargetType>::digits;
/// whether NumberType can hold values below the minimum of TargetType
static constexpr bool may_fall_below_min = std::numeric_limits<NumberType>::is_signed
&& (!std::numeric_limits<TargetType>::is_signed || may_exceed_max);
};
/*!
@brief whether integer @a n does not exceed the maximum of @a TargetType
The binary formats store integers in at most 64 bits, but number_integer_t
and number_unsigned_t may be wider (e.g., __int128). The range of the
integer is taken from std::numeric_limits, so a number type whose range
fits into @a TargetType is never checked: the function is then constant
true, and the default int64_t/uint64_t types pay nothing.
*/
template<typename TargetType, typename NumberType>
static constexpr bool integer_fits_max(const NumberType n) noexcept
{
return integer_fits_max<TargetType>(n, std::integral_constant<bool, integer_range<TargetType, NumberType>::may_exceed_max>());
}
template<typename TargetType, typename NumberType>
static constexpr bool integer_fits_max(const NumberType /*unused*/, std::false_type /*may_exceed*/) noexcept
{
return true;
}
template<typename TargetType, typename NumberType>
static constexpr bool integer_fits_max(const NumberType n, std::true_type /*may_exceed*/) noexcept
{
// NumberType has more digits than TargetType, so it can hold its maximum
return n <= static_cast<NumberType>((std::numeric_limits<TargetType>::max)());
}
/*!
@brief whether integer @a n is not below the minimum of @a TargetType;
constant true if NumberType cannot hold such a value
*/
template<typename TargetType, typename NumberType>
static constexpr bool integer_fits_min(const NumberType n) noexcept
{
return integer_fits_min<TargetType>(n, std::integral_constant<bool, integer_range<TargetType, NumberType>::may_fall_below_min>());
}
template<typename TargetType, typename NumberType>
static constexpr bool integer_fits_min(const NumberType /*unused*/, std::false_type /*may_fall_below*/) noexcept
{
return true;
}
template<typename TargetType, typename NumberType>
static constexpr bool integer_fits_min(const NumberType n, std::true_type /*may_fall_below*/) noexcept
{
// NumberType is signed and either TargetType is unsigned (minimum 0)
// or NumberType has more digits, so it can hold the minimum
return n >= static_cast<NumberType>((std::numeric_limits<TargetType>::min)());
}
/*!
@brief whether integer @a n lies in the range of @a TargetType
*/
template<typename TargetType, typename NumberType>
static constexpr bool integer_fits(const NumberType n) noexcept
{
return integer_fits_min<TargetType>(n) && integer_fits_max<TargetType>(n);
}
/*!
@brief the decimal digits of the non-negative integer @a n
*/
template<typename NumberType>
static std::string integer_to_string(NumberType n, std::false_type /*is_signed*/)
{
std::string result;
do
{
result.push_back(static_cast<char>('0' + (n % 10)));
n = static_cast<NumberType>(n / 10);
}
while (n != 0);
std::reverse(result.begin(), result.end());
return result;
}
/*!
@brief the decimal digits of the integer @a n, with a leading '-' if it
is negative
*/
template<typename NumberType>
static std::string integer_to_string(NumberType n, std::true_type /*is_signed*/)
{
// the remainder of a negative n is not positive; each digit is
// negated on its own, because negating the minimum of NumberType
// as a whole would overflow
const bool negative = n < 0;
std::string result;
do
{
const auto remainder = n % 10;
result.push_back(static_cast<char>('0' + (remainder < 0 ? -remainder : remainder)));
n = static_cast<NumberType>(n / 10);
}
while (n != 0);
if (negative)
{
result.push_back('-');
}
std::reverse(result.begin(), result.end());
return result;
}
/*!
@brief throws because the integer @a j is too large for @a format_name
@throw out_of_range.407 always
*/
JSON_HEDLEY_NO_RETURN static void throw_integer_out_of_range(const BasicJsonType& j, const char* format_name, const char* range)
{
// neither std::to_string(), which has no overload for integer types
// wider than 64 bits, nor dump(), which would also instantiate the
// serializer for objects and fail for key types it cannot write
using integer_limits = std::numeric_limits<typename BasicJsonType::number_integer_t>;
using unsigned_limits = std::numeric_limits<typename BasicJsonType::number_unsigned_t>;
const std::string number = j.is_number_unsigned()
? integer_to_string(j.m_data.m_value.number_unsigned, std::integral_constant<bool, unsigned_limits::is_signed>())
: integer_to_string(j.m_data.m_value.number_integer, std::integral_constant<bool, integer_limits::is_signed>());
static_cast<void>(number); // unused when JSON_NOEXCEPTION is defined
static_cast<void>(format_name);
static_cast<void>(range);
JSON_THROW(out_of_range::create(exception_id::integer_too_large, concat("integer number ", number, " cannot be represented by ", format_name, " as it does not fit ", range), &j));
}
void write_msgpack_array_prefix(const std::size_t N, const BasicJsonType& j)
{
const auto n = to_msgpack_length(N, j);
@@ -1651,8 +1580,6 @@ class binary_writer
is neither an object nor an array
@throw out_of_range.415 if @a j is binary with a subtype that does not fit
into a byte, before anything is written
@throw out_of_range.407 if @a j is an integer that does not fit the 64 bits
of a BSON integer, before anything is written
@throw type_error.316 if @a j is a string that is not valid UTF-8, before
anything is written
@throw type_error.321 if @a j is discarded
@@ -1671,17 +1598,9 @@ class binary_writer
return 8ul;
case value_t::number_integer:
if (JSON_HEDLEY_UNLIKELY(!integer_fits<std::int64_t>(j.m_data.m_value.number_integer)))
{
throw_integer_out_of_range(j, "BSON", "int64");
}
return calc_bson_integer_size(static_cast<std::int64_t>(j.m_data.m_value.number_integer));
return calc_bson_integer_size(j.m_data.m_value.number_integer);
case value_t::number_unsigned:
if (JSON_HEDLEY_UNLIKELY(!integer_fits_max<std::uint64_t>(j.m_data.m_value.number_unsigned)))
{
throw_integer_out_of_range(j, "BSON", "uint64");
}
return calc_bson_unsigned_size(conditional_static_cast<std::uint64_t>(j.m_data.m_value.number_unsigned));
case value_t::string:
@@ -1720,9 +1639,8 @@ class binary_writer
case value_t::number_float:
return write_bson_double(name, j.m_data.m_value.number_float);
// calc_bson_value_size() checked that integers fit 64 bits
case value_t::number_integer:
return write_bson_integer(name, static_cast<std::int64_t>(j.m_data.m_value.number_integer));
return write_bson_integer(name, j.m_data.m_value.number_integer);
case value_t::number_unsigned:
return write_bson_unsigned(name, conditional_static_cast<std::uint64_t>(j.m_data.m_value.number_unsigned));
@@ -2033,376 +1951,6 @@ class binary_writer
}
}
/*!
@brief size class of an unsigned integer from its bit length
1, 2, 3, or 4 if @a x fits 1, 2, 4, or 8 bytes. Looked up from the number
of leading zero bits instead of found by branches: the sizes of a stream
of numbers are hard to predict, and a mispredicted branch costs more than
writing the number. (A table, as comparisons are vectorized by clang into
slower code.)
*/
static unsigned integer_size_class(const std::uint64_t x) noexcept
{
static constexpr std::array<std::uint8_t, 64> size_class_by_leading_zeros =
{
{
4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4,
3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3,
2, 2, 2, 2, 2, 2, 2, 2,
1, 1, 1, 1, 1, 1, 1, 1
}
};
// x | 1 keeps count_leading_zeros() away from 0
return size_class_by_leading_zeros[static_cast<std::size_t>(count_leading_zeros(x | 1u))];
}
/*!
@brief encode @a first_byte and the low-order bytes of @a value that
@a size_class calls for (none, or 1, 2, 4, or 8) in network order
to @a out
@return the number of characters of the encoding; up to
@ref integer_encoding_space characters at @a out are overwritten
The bytes are written as one byte and one 8-byte word whose bytes after
the value are scratch that the caller drops, so that writing them does not
depend on the size class. For class 0 the word is all scratch; it is
shifted by 60 bits rather than 64, which would be undefined.
*/
std::size_t encode_first_byte_and_tail(CharType* out, const std::uint8_t first_byte, const unsigned size_class, const std::uint64_t value) const noexcept
{
out[0] = to_char_type(first_byte);
const unsigned unused_bits = 64u - ((8u << size_class) >> 1u);
const std::uint64_t tail = is_little_endian ? byte_swap(value) >> unused_bits : value << unused_bits;
std::memcpy(out + 1, &tail, sizeof(tail));
return ((std::size_t{1} << size_class) >> 1u) + 1;
}
/// characters encode_first_byte_and_tail() may overwrite: the first byte
/// and one 8-byte word
static constexpr std::size_t integer_encoding_space = 9;
/*!
@brief the 64 bits of a number_integer or number_unsigned value, and a mask
that is all ones if it is negative, else zero
Selected through masks rather than branches, as the type and the sign of
the values in an array of mixed signs alternate unpredictably (non-negative
integers are number_unsigned after parsing). Up to 64 bits, the sign is bit
63 of the result (a narrower integer is sign-extended); computing it with
masks keeps GCC from branching on the type. A wider integer has to be
compared.
*/
static std::uint64_t integer_bits(const BasicJsonType& j, std::uint64_t& sign_mask) noexcept
{
const bool is_signed = j.m_data.m_type == value_t::number_integer;
const std::uint64_t raw = is_signed
? static_cast<std::uint64_t>(j.m_data.m_value.number_integer)
: static_cast<std::uint64_t>(j.m_data.m_value.number_unsigned);
sign_mask = sizeof(typename BasicJsonType::number_integer_t) <= sizeof(std::uint64_t)
? (std::uint64_t{0} - static_cast<std::uint64_t>(is_signed)) & (std::uint64_t{0} - (raw >> 63u))
: std::uint64_t{0} - static_cast<std::uint64_t>(is_signed && j.m_data.m_value.number_integer < 0);
return raw;
}
/*!
@brief encode the head of a CBOR integer to @a out without branching on its
size
@return the number of characters of the head
Same output as @ref write_cbor_head. The lengths of strings and containers
are mostly predictable, so their heads keep the branches; the sizes of a
run of integers are not, so an integer's head is assembled without them.
*/
std::size_t encode_cbor_integer_head(CharType* out, const std::uint8_t major_type, const std::uint64_t argument) const noexcept
{
// an argument of at most 23 is in size class 1 (one byte) and is moved
// to class 0, the first byte, by subtracting 1
const auto is_immediate = static_cast<unsigned>(argument <= 0x17u);
const unsigned size_class = integer_size_class(argument) - is_immediate;
// the first byte: the argument itself if it is immediate, else
// 0x17 + size_class, selected through a mask
const std::uint64_t immediate_mask = std::uint64_t{0} - is_immediate;
const auto first_byte = static_cast<std::uint8_t>(major_type + ((argument & immediate_mask) | ((0x17u + size_class) & ~immediate_mask)));
return encode_first_byte_and_tail(out, first_byte, size_class, argument);
}
/*!
@brief write an array value (head and elements) as CBOR
Integer elements, the most common scalar, are written without a call to
@ref write_cbor. The loop is kept out of line: inlined into write_cbor, its
registers would be saved and restored on every call of write_cbor, which
is one call per value nested in an object.
*/
JSON_HEDLEY_NEVER_INLINE
void write_cbor_array(const BasicJsonType& j, const std::size_t depth)
{
write_cbor_head(0x80, j.m_data.m_value.array->size());
write_cbor_array_elements(j, depth, std::integral_constant<bool, integers_fit_64_bits()>());
}
void write_cbor_array_elements(const BasicJsonType& j, const std::size_t depth, std::true_type /*fits_64_bits*/)
{
write_array_elements(j, [this](CharType * out, const BasicJsonType & el)
{
return encode_cbor_integer(out, el);
}, [this, depth](const BasicJsonType & el)
{
write_cbor(el, depth + 1);
});
}
void write_cbor_array_elements(const BasicJsonType& j, const std::size_t depth, std::false_type /*fits_64_bits*/)
{
for (const auto& el : *j.m_data.m_value.array)
{
write_cbor(el, depth + 1);
}
}
/*!
@brief write the elements of the array @a j, runs of integers with
@a encode_integer and all other values with @a write_value
Runs of integers are encoded straight into space reserved from the sink
for a whole run: the write pointer stays a local, where it would otherwise
be reloaded from the sink for every value, as a store of a character may
alias it.
*/
template<typename EncodeInteger, typename WriteValue>
void write_array_elements(const BasicJsonType& j, EncodeInteger encode_integer, WriteValue write_value)
{
constexpr std::size_t max_run = max_reserve_size / integer_encoding_space;
auto it = j.m_data.m_value.array->begin();
const auto end = j.m_data.m_value.array->end();
while (it != end)
{
if (it->is_number_integer())
{
CharType* const start = oa.reserve(max_run * integer_encoding_space);
CharType* out = start;
for (std::size_t n = 0; n < max_run && it != end && it->is_number_integer(); ++n, ++it)
{
out += encode_integer(out, *it);
}
oa.commit(static_cast<std::size_t>(out - start));
}
else
{
write_value(*it);
++it;
}
}
}
/*!
@brief write a number_integer or number_unsigned value as a CBOR integer
A negative integer n has major type 1 and the argument -1 - n, which is ~n.
Kept out of line, like the other per-value paths, so that write_cbor stays
small (runs of integers in an array are encoded inline instead).
*/
JSON_HEDLEY_NEVER_INLINE
void write_cbor_integer(const BasicJsonType& j, std::true_type /*fits_64_bits*/)
{
oa.commit(encode_cbor_integer(oa.reserve(integer_encoding_space), j));
}
/// write a number_integer or number_unsigned value of a number type wider
/// than 64 bits as a CBOR integer, with range checks on the whole value
void write_cbor_integer(const BasicJsonType& j, std::false_type /*fits_64_bits*/)
{
if (j.m_data.m_type == value_t::number_unsigned)
{
if (JSON_HEDLEY_UNLIKELY(!integer_fits_max<std::uint64_t>(j.m_data.m_value.number_unsigned)))
{
throw_integer_out_of_range(j, "CBOR", "uint64");
}
write_cbor_head(0x00, conditional_static_cast<std::uint64_t>(j.m_data.m_value.number_unsigned));
return;
}
if (j.m_data.m_value.number_integer >= 0)
{
if (JSON_HEDLEY_UNLIKELY(!integer_fits_max<std::uint64_t>(j.m_data.m_value.number_integer)))
{
throw_integer_out_of_range(j, "CBOR", "[-2^64, 2^64-1]");
}
// CBOR does not differentiate between positive signed
// integers and unsigned integers
write_cbor_head(0x00, static_cast<std::uint64_t>(j.m_data.m_value.number_integer));
}
else
{
// a negative integer n is encoded as -1 - n
if (JSON_HEDLEY_UNLIKELY(!integer_fits_max<std::uint64_t>(-1 - j.m_data.m_value.number_integer)))
{
throw_integer_out_of_range(j, "CBOR", "[-2^64, 2^64-1]");
}
write_cbor_head(0x20, static_cast<std::uint64_t>(-1 - j.m_data.m_value.number_integer));
}
}
/// encode a number_integer or number_unsigned value to @a out (see
/// @ref write_cbor_integer) and return the number of characters
std::size_t encode_cbor_integer(CharType* out, const BasicJsonType& j) const noexcept
{
std::uint64_t sign_mask = 0;
const std::uint64_t raw = integer_bits(j, sign_mask);
// ~raw is -1 - n in the 64 bits that are written
return encode_cbor_integer_head(out, static_cast<std::uint8_t>(sign_mask & 0x20u), raw ^ sign_mask);
}
/*!
@brief whether the CBOR and MessagePack integers of BasicJsonType are
written by @ref encode_cbor_integer and @ref encode_msgpack_integer
These pick the format from the low 64 bits; a wider number type is written
with range checks on the whole value instead.
*/
static constexpr bool integers_fit_64_bits() noexcept
{
return sizeof(typename BasicJsonType::number_integer_t) <= sizeof(std::uint64_t)
&& sizeof(typename BasicJsonType::number_unsigned_t) <= sizeof(std::uint64_t);
}
/*!
@brief write a number_integer or number_unsigned value as a MessagePack
integer without branching on its sign or size
Same output as @ref write_msgpack: a non-negative integer as positive fixint
(< 128) or uint 8/16/32/64, a negative one as negative fixint (>= -32) or
int 8/16/32/64. Kept out of line (see @ref write_cbor_integer).
*/
JSON_HEDLEY_NEVER_INLINE
void write_msgpack_integer(const BasicJsonType& j, std::true_type /*fits_64_bits*/)
{
oa.commit(encode_msgpack_integer(oa.reserve(integer_encoding_space), j));
}
/// encode a number_integer or number_unsigned value to @a out (see
/// @ref write_msgpack_integer) and return the number of characters
std::size_t encode_msgpack_integer(CharType* out, const BasicJsonType& j) const noexcept
{
std::uint64_t sign_mask = 0;
const std::uint64_t raw = integer_bits(j, sign_mask);
// A negative n fits int 8/16/32 exactly if 2 * (-1 - n) + 1 fits
// uint 8/16/32, so x gives the size class of either sign.
const std::uint64_t is_negative = sign_mask & 1u;
const std::uint64_t x = ((raw ^ sign_mask) << is_negative) | is_negative;
// A fixint is at most 127, or at least -32 (x at most 63). It is in
// size class 1 and is moved to class 0, the first byte, by subtracting 1.
const auto is_fixint = static_cast<unsigned>(x <= (127u ^ (sign_mask & 64u)));
const unsigned size_class = integer_size_class(x) - is_fixint;
// the first byte: the low byte of the integer for a fixint, else the
// marker 0xCC..0xCF (uint) or 0xD0..0xD3 (int), selected through a mask
const std::uint64_t fixint_mask = std::uint64_t{0} - is_fixint;
const std::uint64_t marker = 0xCBu + size_class + (sign_mask & 4u);
const auto first_byte = static_cast<std::uint8_t>((raw & fixint_mask) | (marker & ~fixint_mask));
return encode_first_byte_and_tail(out, first_byte, size_class, raw);
}
/// write a number_integer or number_unsigned value of a number type wider
/// than 64 bits as a MessagePack integer, with range checks on the whole value
void write_msgpack_integer(const BasicJsonType& j, std::false_type /*fits_64_bits*/)
{
if (j.m_data.m_type == value_t::number_unsigned)
{
if (JSON_HEDLEY_UNLIKELY(!integer_fits_max<std::uint64_t>(j.m_data.m_value.number_unsigned)))
{
throw_integer_out_of_range(j, "MessagePack", "uint64");
}
write_msgpack_unsigned(static_cast<std::uint64_t>(j.m_data.m_value.number_unsigned));
return;
}
if (j.m_data.m_value.number_integer >= 0)
{
if (JSON_HEDLEY_UNLIKELY(!integer_fits_max<std::uint64_t>(j.m_data.m_value.number_integer)))
{
throw_integer_out_of_range(j, "MessagePack", "[-2^63, 2^64-1]");
}
// MessagePack does not differentiate between positive
// signed integers and unsigned integers.
write_msgpack_unsigned(static_cast<std::uint64_t>(j.m_data.m_value.number_integer));
}
else
{
if (JSON_HEDLEY_UNLIKELY(!integer_fits_min<std::int64_t>(j.m_data.m_value.number_integer)))
{
throw_integer_out_of_range(j, "MessagePack", "[-2^63, 2^64-1]");
}
if (j.m_data.m_value.number_integer >= -32)
{
// negative fixnum
write_number(static_cast<std::int8_t>(j.m_data.m_value.number_integer));
}
else if (j.m_data.m_value.number_integer >= (std::numeric_limits<std::int8_t>::min)() &&
j.m_data.m_value.number_integer <= (std::numeric_limits<std::int8_t>::max)())
{
// int 8
oa.write_character(to_char_type(0xD0));
write_number(static_cast<std::int8_t>(j.m_data.m_value.number_integer));
}
else if (j.m_data.m_value.number_integer >= (std::numeric_limits<std::int16_t>::min)() &&
j.m_data.m_value.number_integer <= (std::numeric_limits<std::int16_t>::max)())
{
// int 16
oa.write_character(to_char_type(0xD1));
write_number(static_cast<std::int16_t>(j.m_data.m_value.number_integer));
}
else if (j.m_data.m_value.number_integer >= (std::numeric_limits<std::int32_t>::min)() &&
j.m_data.m_value.number_integer <= (std::numeric_limits<std::int32_t>::max)())
{
// int 32
oa.write_character(to_char_type(0xD2));
write_number(static_cast<std::int32_t>(j.m_data.m_value.number_integer));
}
else
{
// int 64
oa.write_character(to_char_type(0xD3));
write_number(static_cast<std::int64_t>(j.m_data.m_value.number_integer));
}
}
}
/*!
@brief write an array value (head and elements) as MessagePack
See @ref write_cbor_array.
*/
JSON_HEDLEY_NEVER_INLINE
void write_msgpack_array(const BasicJsonType& j, const std::size_t depth)
{
write_msgpack_array_prefix(j.m_data.m_value.array->size(), j);
write_msgpack_array_elements(j, depth, std::integral_constant<bool, integers_fit_64_bits()>());
}
void write_msgpack_array_elements(const BasicJsonType& j, const std::size_t depth, std::true_type /*fits_64_bits*/)
{
write_array_elements(j, [this](CharType * out, const BasicJsonType & el)
{
return encode_msgpack_integer(out, el);
}, [this, depth](const BasicJsonType & el)
{
write_msgpack(el, depth + 1);
});
}
void write_msgpack_array_elements(const BasicJsonType& j, const std::size_t depth, std::false_type /*fits_64_bits*/)
{
for (const auto& el : *j.m_data.m_value.array)
{
write_msgpack(el, depth + 1);
}
}
/*!
@brief write an object key as a CBOR text string
@@ -2599,7 +2147,7 @@ class binary_writer
{
return 'L';
}
if (use_bjdata && std::is_unsigned<NumberType>::value && integer_fits_max<std::uint64_t>(n))
if (use_bjdata && std::is_unsigned<NumberType>::value)
{
return 'M';
}
@@ -2724,16 +2272,14 @@ class binary_writer
@brief checks whether a JSON number fits into @a TargetType
@param[in] el a JSON number of either the signed or unsigned integer kind
@return whether @a el's value can be represented by @a TargetType without
wrapping, regardless of which of the two kinds it is stored as;
false for a value that does not even fit the 64-bit type it is
read as (possible for number types wider than 64 bits)
wrapping, regardless of which of the two kinds it is stored as
*/
template<typename TargetType>
static bool bjdata_ndarray_value_in_range(const BasicJsonType& el)
{
return el.is_number_unsigned()
? integer_fits_max<std::uint64_t>(el.m_data.m_value.number_unsigned) && value_in_range_of<TargetType>(el.template get<std::uint64_t>())
: integer_fits<std::int64_t>(el.m_data.m_value.number_integer) && value_in_range_of<TargetType>(el.template get<std::int64_t>());
? value_in_range_of<TargetType>(el.template get<std::uint64_t>())
: value_in_range_of<TargetType>(el.template get<std::int64_t>());
}
/*!
@@ -2965,11 +2511,10 @@ class binary_writer
for (const auto& el : dims)
{
// a dimension is read as an unsigned value below, so anything that
// is not a non-negative integer in the range of std::uint64_t is
// rejected: a non-integer entry would pun unrelated bytes as the
// dimension, and a negative or wider one would wrap into a
// nonsensical length
if (!el.is_number_integer() || !bjdata_ndarray_value_in_range<std::uint64_t>(el))
// is not a non-negative integer is rejected: a non-integer entry
// would pun unrelated bytes as the dimension, and a negative one
// would wrap into a nonsensical length
if (!el.is_number_integer() || (!el.is_number_unsigned() && el.template get<std::int64_t>() < 0))
{
return true;
}
@@ -3101,9 +2646,9 @@ class binary_writer
case value_t::number_unsigned:
{
if (JSON_HEDLEY_UNLIKELY(!integer_fits_max<std::int64_t>(j.m_data.m_value.number_unsigned)))
if (j.m_data.m_value.number_unsigned > static_cast<typename BasicJsonType::number_unsigned_t>((std::numeric_limits<std::int64_t>::max)()))
{
throw_integer_out_of_range(j, "BON8", "int64");
JSON_THROW(out_of_range::create(exception_id::integer_too_large, concat("integer number ", std::to_string(j.m_data.m_value.number_unsigned), " cannot be represented by BON8 as it does not fit int64"), &j));
}
write_bon8_integer(static_cast<std::int64_t>(j.m_data.m_value.number_unsigned));
string_open = false;
@@ -3112,10 +2657,6 @@ class binary_writer
case value_t::number_integer:
{
if (JSON_HEDLEY_UNLIKELY(!integer_fits<std::int64_t>(j.m_data.m_value.number_integer)))
{
throw_integer_out_of_range(j, "BON8", "int64");
}
write_bon8_integer(static_cast<std::int64_t>(j.m_data.m_value.number_integer));
string_open = false;
break;
@@ -8,10 +8,7 @@
#pragma once
#include <algorithm> // max
#include <array> // array
#include <cstddef> // size_t
#include <cstring> // memcpy
#include <memory> // shared_ptr, make_shared
#include <string> // basic_string
#include <type_traits> // conditional, integral_constant, is_same
@@ -51,170 +48,32 @@ template<typename CharType> struct output_adapter_protocol
template<typename CharType>
using output_adapter_t = std::shared_ptr<output_adapter_protocol<CharType>>;
/// the largest number of characters a writer may reserve() from a sink at once;
/// output_vector_sink also appends writes of at least this size with insert()
constexpr std::size_t max_reserve_size = 256;
/// @brief non-virtual output sink writing into a std::vector
///
/// This sink is not part of the virtual output_adapter_protocol hierarchy: it is
/// passed to binary_writer by value as a template parameter, so
/// write_character()/write_characters() are ordinary (inlinable) calls with no
/// vtable lookup and no shared_ptr. It is used for the common
/// `to_cbor`/`to_msgpack`/... into a std::vector.
///
/// Most writes are a few bytes - a type marker, or a number of 1, 2, 4, or 8
/// bytes - and appending those with push_back() or insert() cost more than
/// encoding them: insert() is an out-of-line call into a routine that can insert
/// anywhere, and resize() is one as well with libstdc++. The sink therefore keeps
/// its own write position: it grows the vector geometrically ahead of the
/// position and copies each write to it, so a write is a capacity check and a
/// short copy. The destructor shrinks the vector to the bytes actually written,
/// also when a write throws; shrinking a vector never allocates. The sink owns
/// that trim, so it can be moved but not copied, and the vector must not be used
/// while the sink is alive.
/// `to_cbor`/`to_msgpack`/... into a std::vector. output_vector_adapter below
/// wraps this same sink to provide the virtual interface.
template<typename CharType, typename AllocatorType = std::allocator<CharType>>
class output_vector_sink
{
public:
explicit output_vector_sink(std::vector<CharType, AllocatorType>& vec) noexcept
: v(&vec), position(vec.size())
{}
output_vector_sink(const output_vector_sink&) = delete;
output_vector_sink& operator=(const output_vector_sink&) = delete;
output_vector_sink& operator=(output_vector_sink&&) = delete;
output_vector_sink(output_vector_sink&& other) noexcept
: v(other.v), position(other.position)
{
other.v = nullptr;
}
~output_vector_sink()
{
if (v != nullptr)
{
v->resize(position);
}
}
// always inlined: GCC otherwise keeps these out of line in a large
// translation unit, and a call per byte or number costs more than the write
JSON_HEDLEY_ALWAYS_INLINE
void write_character(CharType c)
{
if (JSON_HEDLEY_UNLIKELY(position == v->size()))
{
grow(1);
}
(*v)[position++] = c;
}
// no JSON_HEDLEY_NON_NULL here: binary_writer legitimately passes a null
// pointer with length 0 for empty strings/binary values; memcpy must not
// see it.
JSON_HEDLEY_ALWAYS_INLINE
void write_characters(const CharType* s, std::size_t length)
{
if (JSON_HEDLEY_UNLIKELY(v->size() - position < length))
{
if (length >= max_reserve_size)
{
append(s, length);
return;
}
grow(length);
}
if (length != 0)
{
std::memcpy(v->data() + position, s, length * sizeof(CharType));
position += length;
}
}
/// @brief space for writing up to @a length characters directly
///
/// Returns a pointer to at least @a length (at most @ref max_reserve_size)
/// writable characters at the write position; @ref commit then keeps the
/// first ones of them. A writer can store whole words there and keep only
/// the bytes it needs, and keep its write pointer in a register across many
/// values instead of updating the sink after each.
JSON_HEDLEY_ALWAYS_INLINE
CharType* reserve(const std::size_t length)
{
JSON_ASSERT(length <= max_reserve_size);
if (JSON_HEDLEY_UNLIKELY(v->size() - position < length))
{
grow(length);
}
return v->data() + position;
}
/// keep the first @a length characters written to the space from reserve()
JSON_HEDLEY_ALWAYS_INLINE
void commit(const std::size_t length) noexcept
{
position += length;
}
private:
/// make room for at least @a length more characters after the position;
/// kept out of line so that the write functions stay small enough for
/// GCC to inline them into the writer
JSON_HEDLEY_NEVER_INLINE
void grow(const std::size_t length)
{
// use the capacity the vector already has, else at least double it;
// never less than 64 characters, so that a small value does not
// reallocate for every few bytes
const std::size_t needed = position + length;
v->resize((std::max)(std::size_t{64}, needed <= v->capacity()
? v->capacity()
: (std::max)(needed, v->capacity() * 2)));
}
/// append @a length (at least max_reserve_size) characters that do not fit
/// the space ahead of the position: insert() copies them once into the
/// grown vector, where grow() would first zero the new space and then copy
/// over it - noticeable for a large string or binary value
JSON_HEDLEY_NEVER_INLINE
void append(const CharType* s, const std::size_t length)
{
v->resize(position);
if (v->capacity() - position < length)
{
// grow geometrically, with room for the writes that follow, so
// that the next grow() need not reallocate right away
v->reserve((std::max)(v->capacity() * 2, position + length + max_reserve_size));
}
v->insert(v->end(), s, s + length);
position = v->size();
}
std::vector<CharType, AllocatorType>* v;
/// number of characters of *v written so far
std::size_t position;
};
/// output adapter for byte vectors
template<typename CharType, typename AllocatorType = std::allocator<CharType>>
class output_vector_adapter : public output_adapter_protocol<CharType>
{
public:
explicit output_vector_adapter(std::vector<CharType, AllocatorType>& vec) noexcept
: v(vec)
{}
void write_character(CharType c) override
void write_character(CharType c)
{
v.push_back(c);
}
// no JSON_HEDLEY_NON_NULL here: binary_writer legitimately passes a null
// pointer with length 0 for empty strings/binary values; appending an empty
// range is a no-op
void write_characters(const CharType* s, std::size_t length) override
// pointer with length 0 for empty strings/binary values. Appending an empty
// range is a no-op; the type-erased path tolerates this via the (unattributed)
// virtual base, and the concrete sink must do the same.
void write_characters(const CharType* s, std::size_t length)
{
v.insert(v.end(), s, s + length);
}
@@ -223,6 +82,33 @@ class output_vector_adapter : public output_adapter_protocol<CharType>
std::vector<CharType, AllocatorType>& v;
};
/// output adapter for byte vectors
///
/// The appending itself lives in output_vector_sink; this class only adds the
/// virtual output_adapter_protocol interface on top of it, so both the
/// type-erased and the templated path share one implementation.
template<typename CharType, typename AllocatorType = std::allocator<CharType>>
class output_vector_adapter : public output_adapter_protocol<CharType>
{
public:
explicit output_vector_adapter(std::vector<CharType, AllocatorType>& vec) noexcept
: sink(vec)
{}
void write_character(CharType c) override
{
sink.write_character(c);
}
void write_characters(const CharType* s, std::size_t length) override
{
sink.write_characters(s, length);
}
private:
output_vector_sink<CharType, AllocatorType> sink;
};
#ifndef JSON_NO_IO
/// output adapter for output streams
template<typename CharType>
@@ -302,24 +188,8 @@ class output_adapter_sink
oa->write_characters(s, length);
}
/// space for writing up to @a length characters (see
/// output_vector_sink::reserve)
CharType* reserve(const std::size_t length) noexcept
{
JSON_ASSERT(length <= max_reserve_size);
static_cast<void>(length);
return reserved.data();
}
/// write the first @a length characters written to the space from reserve()
void commit(const std::size_t length)
{
oa->write_characters(reserved.data(), length);
}
private:
output_adapter_t<CharType> oa;
std::array<CharType, max_reserve_size> reserved{};
};
/// @brief whether std::basic_string<CharType> has a non-deprecated std::char_traits
@@ -1525,7 +1525,7 @@ class serializer
// without leading zeros
if (abs_value < 10000000000000000u)
{
const auto value = static_cast<std::uint64_t>(abs_value); // fits: abs_value < 10^16
const std::uint64_t value = abs_value;
const std::uint64_t upper = value / 100000000u;
const std::uint64_t first = dtoa_impl::eight_digit_bytes(upper != 0 ? upper : value);
const auto leading = static_cast<unsigned>(count_leading_zeros(first) / 8); // (first is not 0)
+4 -6
View File
@@ -14,6 +14,7 @@
#include <cstdint> // uint8_t
#include <limits> // numeric_limits
#include <string> // string
#include <type_traits> // is_signed
#include <nlohmann/detail/macro_scope.hpp>
#if JSON_HAS_THREE_WAY_COMPARISON
@@ -207,16 +208,13 @@ FloatType compare_integer_with_float(const IntegerType i, const FloatType f) noe
// values of IntegerType lie in [-bound, bound) when signed and in
// [0, bound) when unsigned; digits excludes the sign bit, so bound is a
// power of two that the float represents exactly; the signedness comes
// from numeric_limits as well, because std::is_signed is false for class
// types such as 128-bit or multiprecision integers
using limits = std::numeric_limits<IntegerType>;
const FloatType bound = std::ldexp(static_cast<FloatType>(1), limits::digits);
// power of two that the float represents exactly
const FloatType bound = std::ldexp(static_cast<FloatType>(1), std::numeric_limits<IntegerType>::digits);
if (f >= bound)
{
return ordered(-1);
}
if (limits::is_signed ? (f < -bound) : (f < static_cast<FloatType>(0)))
if (std::is_signed<IntegerType>::value ? (f < -bound) : (f < static_cast<FloatType>(0)))
{
return ordered(1);
}
+16 -39
View File
@@ -126,13 +126,6 @@ class builder
frame_stack m_shallow;
std::vector<frame> m_deep{};
/// remember an object to index after parsing (out of line, so that the
/// parse loop only has a call for it)
NLOHMANN_VIEW_NOINLINE void note_large_object(std::uint32_t idx)
{
m_doc.large_objects.push_back(idx);
}
NLOHMANN_VIEW_NOINLINE bool fail(error_code c, const unsigned char* at) noexcept
{
m_failure.code = c;
@@ -546,7 +539,7 @@ class builder
switch (cur()) \
{ \
case '"': \
if (NLOHMANN_VIEW_UNLIKELY(!string<true>())) { return false; } \
if (NLOHMANN_VIEW_UNLIKELY(!string())) { return false; } \
break; \
case '{': \
open(value_t::object); \
@@ -630,7 +623,7 @@ obj_key:
{
return fail(error_code::expected_key);
}
if (NLOHMANN_VIEW_UNLIKELY(!string<false>()))
if (NLOHMANN_VIEW_UNLIKELY(!string()))
{
return false;
}
@@ -672,25 +665,17 @@ obj_next:
if (enabled(TrailingCommas) && cur() == '}')
{
++p;
goto close_object;
goto close_container;
}
goto obj_key;
}
if (cur() == '}')
{
++p;
goto close_object;
goto close_container;
}
return fail(error_code::expected_object_end);
close_object:
// a large object gets a hash index for its keys (objects only, so that closing
// an array pays nothing for this)
if (NLOHMANN_VIEW_UNLIKELY(cur_count >= document_data::index_min_members))
{
cold.note_large_object(cur_idx);
}
close_container:
close();
if (NLOHMANN_VIEW_UNLIKELY(depth == 0))
@@ -743,7 +728,7 @@ root_done:
switch (cur())
{
case '"':
return string<true>();
return string();
case 't':
return literal_true();
case 'f':
@@ -882,19 +867,14 @@ indent_done:
const auto idx = static_cast<std::uint32_t>(n - base);
if (depth != 0)
{
const frame f = {cur_idx, cur_count, cur_is_object};
if (NLOHMANN_VIEW_LIKELY(depth <= 64))
{
// field by field: a frame put together on the stack and
// copied would be read back wider than it was written,
// and that load waits until the stores are done
frame& f = cold.m_shallow[depth - 1];
f.idx = cur_idx;
f.count = cur_count;
f.is_object = cur_is_object;
cold.m_shallow[depth - 1] = f;
}
else
{
cold.m_deep.push_back(frame{cur_idx, cur_count, cur_is_object});
cold.m_deep.push_back(f);
}
}
++depth;
@@ -911,21 +891,19 @@ indent_done:
n.next = static_cast<std::uint32_t>(out - base) - cur_idx;
if (--depth != 0)
{
frame f{};
if (NLOHMANN_VIEW_LIKELY(depth <= 64))
{
const frame& f = cold.m_shallow[depth - 1];
cur_idx = f.idx;
cur_count = f.count;
cur_is_object = f.is_object;
f = cold.m_shallow[depth - 1];
}
else
{
const frame f = cold.m_deep.back();
f = cold.m_deep.back();
cold.m_deep.pop_back();
cur_idx = f.idx;
cur_count = f.count;
cur_is_object = f.is_object;
}
cur_idx = f.idx;
cur_count = f.count;
cur_is_object = f.is_object;
}
}
@@ -1050,13 +1028,12 @@ indent_done:
return true;
}
/// a string at p: a value (Value) or a key
template<bool Value>
/// a string (value or key) at p
NLOHMANN_VIEW_ALWAYS_INLINE bool string()
{
++p; // opening quote
const unsigned char* const s = p;
p = scan_string_run<Value>(p, e);
p = scan_string_run(p, e);
if (NLOHMANN_VIEW_LIKELY(p != e && *p == '"'))
{
push_node(value_t::string, 0, 0, static_cast<std::size_t>(s - b), static_cast<std::uint64_t>(p - s));
+1 -129
View File
@@ -10,11 +10,8 @@
#include <array> // array
#include <cstddef> // size_t
#include <cstdint> // uint8_t, uint32_t
#include <cstring> // memcmp, memcpy
#include <limits> // numeric_limits
#include <map> // map
#include <memory> // unique_ptr
#include <new> // bad_alloc, operator new, placement new
#include <string> // string
#include <vector> // vector
@@ -42,46 +39,10 @@ struct document_data
std::size_t inline_capacity = 0;
std::size_t max_depth = 0; ///< deepest nesting of arrays and objects (the root container is depth 1)
std::vector<char> arena; ///< decoded strings that contained escapes (no terminator: they are handed out with their length)
std::size_t arena_size = 0; ///< bytes of decoded strings at base[1] (the arena, or those of a loaded image)
std::string owned; ///< owned copy of the input, if any (a std::string, so that its NUL serves as the parser's sentinel)
std::vector<std::uint8_t> owned_image; ///< a loaded image the document owns (the text and the decoded strings point into it)
// hash indexes of the keys of large objects (see object_index.hpp)
static constexpr std::uint32_t index_min_members = 128;
struct object_index
{
std::size_t start; ///< first slot in index_slots
std::uint32_t mask; ///< slot count - 1 (a power of two minus one)
};
std::vector<object_index> indexes;
std::vector<std::uint32_t> index_slots;
std::vector<std::uint32_t> large_objects; ///< positions of the objects to index (noted while parsing)
std::array<const char*, 4> base = {{nullptr, nullptr, nullptr, nullptr}}; ///< string bases: source, arena, edit arena, unused (indexed by flags & node_flags::storage)
std::array<const char*, 4> base = {{nullptr, nullptr, nullptr, nullptr}}; ///< string bases: source, arena (indexed by flags & node_flags::storage)
bool discarded = true;
/// The storage of edits (editable documents only; see edit_storage.hpp).
/// No edit moves or resizes the parsed index, and the nodes of values
/// written by edits never move either, so a view stays valid whenever it
/// was taken, before the first edit or between later ones: an array/object
/// whose elements change gets node_flags::moved, and its elements then live
/// in a separate sequence (a header node, then the entries), whose entries
/// link to the values.
struct edit_state
{
std::vector<node*> moved{}; ///< element sequences of moved arrays/objects (header node first) // NOLINT(readability-redundant-member-init)
std::vector<std::size_t> moved_cap{}; ///< capacity in nodes of a growable block; 0: a fixed sequence (a new value) // NOLINT(readability-redundant-member-init)
std::vector<std::unique_ptr<node[]>> chunks{}; ///< storage of new values and blocks; never moved // NOLINT(readability-redundant-member-init,cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
std::map<const node*, node*> regions{}; ///< new arrays/objects: root -> container that uses it as its element sequence (nullptr: linked from a block) // NOLINT(readability-redundant-member-init)
node* chunk_cur = nullptr;
node* chunk_end = nullptr;
std::size_t chunk_next = 64;
std::vector<std::unique_ptr<char[]>> texts{}; ///< edit arena, the current buffer last; earlier ones stay alive for string views // NOLINT(readability-redundant-member-init,cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
std::size_t text_used = 0;
std::size_t text_cap = 0;
std::size_t bytes = 0; ///< memory held by edits
};
std::unique_ptr<edit_state> edits; ///< created by the first edit
/// One allocation for the header and room for `count` nodes, so that a
/// small document needs one allocation instead of two. Large documents get
/// a separate node array instead (so it can be trimmed). When a document
@@ -121,11 +82,6 @@ struct document_data
document_data() noexcept
: arena() // NOLINT(readability-redundant-member-init)
, owned() // NOLINT(readability-redundant-member-init)
, owned_image() // NOLINT(readability-redundant-member-init)
, indexes() // NOLINT(readability-redundant-member-init)
, index_slots() // NOLINT(readability-redundant-member-init)
, large_objects() // NOLINT(readability-redundant-member-init)
, edits() // NOLINT(readability-redundant-member-init)
{}
document_data(const document_data&) = delete;
document_data(document_data&&) = delete;
@@ -208,90 +164,6 @@ struct document_data
{
return n + n->next;
}
/// (editable documents) first element or key, also of a moved container
NLOHMANN_VIEW_ALWAYS_INLINE const node* first_child_edited(const node* n) const noexcept
{
return NLOHMANN_VIEW_LIKELY((n->flags & node_flags::moved) == 0) ? n + 1 : edits->moved[n->off] + 1;
}
/// (editable documents) end of the elements, also of a moved container
NLOHMANN_VIEW_ALWAYS_INLINE const node* child_end_edited(const node* n) const noexcept
{
if (NLOHMANN_VIEW_LIKELY((n->flags & node_flags::moved) == 0))
{
return n + n->next;
}
const node* const h = edits->moved[n->off];
return h + h->next;
}
/// (editable documents) the value at an element position: entries of
/// moved sequences are links. The link case is out of line, so that this
/// compiles to a predicted branch rather than a select that delays the
/// following loads.
static NLOHMANN_VIEW_ALWAYS_INLINE const node* deref(const node* n) noexcept
{
return NLOHMANN_VIEW_LIKELY(n->kind != kind_link) ? n : follow_link(n);
}
static NLOHMANN_VIEW_NOINLINE const node* follow_link(const node* n) noexcept
{
return link_target(*n);
}
};
/// How the index is walked: views of read-only documents follow the node
/// array alone and compile without any of the edit handling; views of
/// editable documents also follow moved element sequences and links.
template<bool Editable>
struct navigation
{
/// whether the walk follows edits (the template argument as a runtime
/// condition: `Editable && ...` is a constant condition for MSVC, C4127)
static NLOHMANN_VIEW_ALWAYS_INLINE bool editable() noexcept
{
return false;
}
static NLOHMANN_VIEW_ALWAYS_INLINE const node* first(const document_data& /*d*/, const node* n) noexcept
{
return n + 1;
}
static NLOHMANN_VIEW_ALWAYS_INLINE const node* end(const document_data& /*d*/, const node* n) noexcept
{
return n + n->next;
}
static NLOHMANN_VIEW_ALWAYS_INLINE const node* value(const node* n) noexcept
{
return n;
}
};
template<>
struct navigation<true>
{
static NLOHMANN_VIEW_ALWAYS_INLINE bool editable() noexcept
{
return true;
}
static NLOHMANN_VIEW_ALWAYS_INLINE const node* first(const document_data& d, const node* n) noexcept
{
return d.first_child_edited(n);
}
static NLOHMANN_VIEW_ALWAYS_INLINE const node* end(const document_data& d, const node* n) noexcept
{
return d.child_end_edited(n);
}
static NLOHMANN_VIEW_ALWAYS_INLINE const node* value(const node* n) noexcept
{
return document_data::deref(n);
}
};
} // namespace view
-883
View File
@@ -1,883 +0,0 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#pragma once
#include <array> // array
#include <cmath> // isinf, isnan
#include <cstddef> // size_t
#include <cstdint> // int64_t, uint8_t, uint32_t, uint64_t
#include <cstring> // memmove
#include <limits> // numeric_limits
#include <string> // string, to_string
#include <type_traits> // decay, enable_if, integral_constant, is_arithmetic, is_convertible, is_floating_point, is_same, is_signed
#include <utility> // forward
#include <vector> // vector
#include <nlohmann/json.hpp>
#include <nlohmann/detail/view/document_data.hpp>
#include <nlohmann/detail/view/edit_storage.hpp>
#include <nlohmann/detail/view/errors.hpp>
#include <nlohmann/detail/view/lookup.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
#include <nlohmann/detail/view/node.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN
template<typename BasicJsonType, bool Editable>
class basic_json_view;
namespace detail
{
namespace view
{
/// the index of the first true condition (the number of conditions if none is)
template<bool... Conditions>
struct first_true : std::integral_constant<int, 0> {};
template<bool... Conditions>
struct first_true<false, Conditions...> : std::integral_constant < int, 1 + first_true<Conditions...>::value > {};
/// Checks a string the way basic_json's serializer does when it writes it
/// (type_error.316 with the same message), so that an editable document
/// only holds valid UTF-8: the error is at the first byte that no
/// well-formed sequence can continue with (Unicode, Table 3-7).
inline void check_utf8(const char* s, std::size_t n)
{
const auto* const p = reinterpret_cast<const unsigned char*>(s); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
const auto hex = [](unsigned char c)
{
constexpr const char* digits = "0123456789ABCDEF";
return std::string{digits[c >> 4u], digits[c & 0xFu]};
};
for (std::size_t i = 0; i < n;)
{
const unsigned char c = p[i];
if (c < 0x80)
{
++i;
continue;
}
std::size_t len = 0;
unsigned char lo = 0x80;
unsigned char hi = 0xBF;
if (c >= 0xC2 && c <= 0xDF)
{
len = 2;
}
else if (c >= 0xE0 && c <= 0xEF)
{
len = 3;
lo = c == 0xE0 ? 0xA0 : 0x80;
hi = c == 0xED ? 0x9F : 0xBF;
}
else if (c >= 0xF0 && c <= 0xF4)
{
len = 4;
lo = c == 0xF0 ? 0x90 : 0x80;
hi = c == 0xF4 ? 0x8F : 0xBF;
}
else
{
throw_type_error(316, concat("invalid UTF-8 byte at index ", std::to_string(i), ": 0x", hex(c)));
}
for (std::size_t k = 1; k < len; ++k)
{
if (i + k == n)
{
throw_type_error(316, concat("incomplete UTF-8 string; last byte: 0x", hex(p[n - 1])));
}
const unsigned char b = p[i + k];
if (b < (k == 1 ? lo : 0x80) || b > (k == 1 ? hi : 0xBF))
{
throw_type_error(316, concat("invalid UTF-8 byte at index ", std::to_string(i + k), ": 0x", hex(b)));
}
}
i += len;
}
}
/*!
@brief the edits of an editable basic_json_document
Values are accepted as views (of any document), BasicJsonType values, and
everything BasicJsonType can be constructed from. The source text is never
written: new values go to storage owned by the document (see
edit_storage.hpp).
*/
template<typename BasicJsonType, typename View>
class editor
{
using number_integer_t = typename BasicJsonType::number_integer_t;
using number_unsigned_t = typename BasicJsonType::number_unsigned_t;
using number_float_t = typename BasicJsonType::number_float_t;
using string_t = typename BasicJsonType::string_t;
using string_view_t = typename View::string_view_t;
using nav = navigation<true>;
public:
explicit editor(document_data& d) noexcept
: m_doc(d)
{}
/// replace a value; returns its view
template<typename V>
View set(const View& target, V&& value)
{
node* const slot = own(target);
const encoded e = encode(std::forward<V>(value));
assign(slot, e, nullptr, false);
return View(&m_doc, slot);
}
/// set a member (appended if missing; a null value becomes an object);
/// returns a view of the member value
template<typename V>
View set(const View& object, string_view_t key, V&& value)
{
node* const o = own(object);
if (o->kind != static_cast<std::uint8_t>(value_t::object) && o->kind != static_cast<std::uint8_t>(value_t::null))
{
throw_type_error(305, "cannot use operator[] with a string argument with ", object.type_name());
}
check_utf8(key.data(), key.size());
const encoded e = encode(std::forward<V>(value));
if (o->kind == static_cast<std::uint8_t>(value_t::null))
{
become_empty(o, value_t::object);
}
// an existing member: assign it (and drop later duplicates, so that
// lookups, iteration, and materialize() agree)
node* slot = nullptr;
bool duplicates = false;
for (const node* k = nav::first(m_doc, o), *end = nav::end(m_doc, o); k != end; k = document_data::after(k + 1))
{
if (key_equals(*k, key))
{
if (slot != nullptr)
{
duplicates = true;
break;
}
slot = const_cast<node*>(nav::value(k + 1)); // NOLINT(cppcoreguidelines-pro-type-const-cast): the nodes belong to this document
}
}
if (slot != nullptr)
{
if (duplicates)
{
erase_members(o, key, true);
}
assign(slot, e, o, true);
return View(&m_doc, slot);
}
const node k = string_node(key.data(), key.size());
slot = new_slot(e);
node* const h = block_of(m_doc, o, 2);
h[h->next] = k;
make_link(h[h->next + 1], slot);
h->next += 2;
++h->len;
++o->len;
return View(&m_doc, slot);
}
/// assign an existing array element; returns a view of it
template<typename V>
View set(const View& array, std::size_t idx, V&& value)
{
node* const a = own(array);
if (a->kind != static_cast<std::uint8_t>(value_t::array))
{
throw_type_error(305, "cannot use operator[] with a numeric argument with ", array.type_name());
}
check_index(idx, a->len);
const encoded e = encode(std::forward<V>(value));
node* const slot = const_cast<node*>(nav::value(element_at<true>(m_doc, a, idx))); // NOLINT(cppcoreguidelines-pro-type-const-cast)
assign(slot, e, a, true);
return View(&m_doc, slot);
}
/// append to an array (a null value becomes an array); returns a view of
/// the new element
template<typename V>
View push_back(const View& array, V&& value)
{
node* const a = own(array);
if (a->kind != static_cast<std::uint8_t>(value_t::array) && a->kind != static_cast<std::uint8_t>(value_t::null))
{
throw_type_error(308, "cannot use push_back() with ", array.type_name());
}
const encoded e = encode(std::forward<V>(value));
if (a->kind == static_cast<std::uint8_t>(value_t::null))
{
become_empty(a, value_t::array);
}
node* const slot = new_slot(e);
node* const h = block_of(m_doc, a, 1);
make_link(h[h->next], slot);
++h->next;
++h->len;
++a->len;
return View(&m_doc, slot);
}
/// insert into an array before position idx (idx <= size()); returns a
/// view of the new element
template<typename V>
View insert(const View& array, std::size_t idx, V&& value)
{
node* const a = own(array);
if (a->kind != static_cast<std::uint8_t>(value_t::array))
{
throw_type_error(309, "cannot use insert() with ", array.type_name());
}
check_index(idx, a->len + 1);
const encoded e = encode(std::forward<V>(value));
node* const slot = new_slot(e);
node* const h = block_of(m_doc, a, 1);
std::memmove(h + 2 + idx, h + 1 + idx, (h->next - 1 - idx) * sizeof(node));
make_link(h[1 + idx], slot);
++h->next;
++h->len;
++a->len;
return View(&m_doc, slot);
}
/// remove all members with this key; returns their number
std::size_t erase(const View& object, string_view_t key)
{
node* const o = own(object);
if (o->kind != static_cast<std::uint8_t>(value_t::object))
{
throw_type_error(307, "cannot use erase() with ", object.type_name());
}
for (const node* k = nav::first(m_doc, o), *end = nav::end(m_doc, o); k != end; k = document_data::after(k + 1))
{
if (key_equals(*k, key))
{
return erase_members(o, key, false);
}
}
return 0;
}
/// remove an array element
void erase(const View& array, std::size_t idx)
{
node* const a = own(array);
if (a->kind != static_cast<std::uint8_t>(value_t::array))
{
throw_type_error(307, "cannot use erase() with ", array.type_name());
}
check_index(idx, a->len);
node* const h = block_of(m_doc, a, 0);
std::memmove(h + 1 + idx, h + 2 + idx, (h->next - 2 - idx) * sizeof(node));
--h->next;
--h->len;
--a->len;
}
private:
/// an encoded value: a scalar node, or the root of a new array/object
struct encoded
{
node scalar{};
node* region = nullptr;
};
/// the node of a view of this document
node* own(const View& v)
{
if (NLOHMANN_VIEW_UNLIKELY(v.m_doc != &m_doc || v.m_node == nullptr))
{
throw_invalid_iterator(202, "view does not belong to this document");
}
edit_state_of(m_doc);
return const_cast<node*>(v.m_node); // NOLINT(cppcoreguidelines-pro-type-const-cast): the nodes belong to this document
}
static void check_index(std::size_t idx, std::size_t limit)
{
if (idx >= limit)
{
throw_out_of_range(401, concat("array index ", std::to_string(idx), " is out of range"));
}
}
bool key_equals(const node& k, string_view_t key) const noexcept
{
return m_doc.key_equals(k, key.data(), key.size());
}
/// remove the members with this key (all, or all but the first) from an object
std::size_t erase_members(node* o, string_view_t key, bool keep_first)
{
node* const h = block_of(m_doc, o, 0);
node* w = h + 1;
std::size_t erased = 0;
bool kept = false;
for (const node* r = h + 1, *end = h + h->next; r != end; r += 2)
{
const bool match = key_equals(*r, key);
if (match && (kept || !keep_first))
{
++erased;
continue;
}
kept = kept || match;
if (w != r)
{
w[0] = r[0];
w[1] = r[1];
}
w += 2;
}
h->next = static_cast<std::uint32_t>(w - h);
h->len -= static_cast<std::uint32_t>(erased);
o->len -= static_cast<std::uint32_t>(erased);
return erased;
}
/// turn a null into an empty array/object in place
static void become_empty(node* n, value_t k) noexcept
{
const std::uint8_t linked = n->flags & node_flags::linked;
*n = node{};
n->kind = static_cast<std::uint8_t>(k);
n->flags = static_cast<std::uint8_t>(node_flags::is_new | linked);
n->next = 1;
}
/// Replace the value at slot; `parent` is the container whose elements
/// include slot (if known).
void assign(node* slot, const encoded& e, node* parent, bool parent_known)
{
// an entry of a moved sequence links to the slot: it can take any extent
const std::uint8_t linked = slot->flags & node_flags::linked;
if (e.region == nullptr)
{
if (is_container(*slot) && slot->next > 1 && slot != m_doc.nodes && linked == 0)
{
// The slot spans its old elements in the enclosing sequence, but
// a scalar is one node: the enclosing container first switches to
// links (then the extent of the slot no longer matters). Looking
// for the container is linear in the size of the document, so
// links (which are marked in the slot) avoid it.
node* const p = parent_known ? parent : find_parent(m_doc, slot);
if (p != nullptr && ((p->flags & node_flags::moved) == 0 || moved_capacity(m_doc, p) == 0))
{
block_of(m_doc, p, 0);
}
}
*slot = e.scalar;
slot->flags = static_cast<std::uint8_t>(slot->flags | linked);
return;
}
// an array/object: the slot keeps its extent (so that the enclosing
// sequence still steps over it), and the elements come from the new
// sequence
const node* const r = e.region;
const std::uint32_t extent = is_container(*slot) ? slot->next : 1;
const bool was_moved = (slot->flags & node_flags::moved) != 0;
// Everything that can throw happens before the slot changes: a slot
// that is a container without the moved flag would show its old
// elements. reserve_moved() makes the set_moved() below, which sets
// the flag, safe; the entry of `regions` exists already (encode()
// added it), so that the assignment at the end does not allocate.
if (!was_moved)
{
reserve_moved(m_doc);
}
slot->kind = r->kind;
slot->extra = 0;
slot->len = r->len;
slot->next = extent;
// (set_moved() adds the moved flag to a slot that does not have it yet)
slot->flags = static_cast<std::uint8_t>((was_moved ? node_flags::moved | node_flags::is_new : 0) | linked);
set_moved(m_doc, slot, e.region, 0);
edit_state_of(m_doc).regions[e.region] = slot;
}
/// a node for a new element (links point to it; it never moves)
node* new_slot(const encoded& e)
{
if (e.region != nullptr)
{
return e.region;
}
node* const s = alloc_nodes(m_doc, 1);
*s = e.scalar;
return s;
}
//////////////
// encoding //
//////////////
template<int N>
using encode_tag = std::integral_constant<int, N>;
template<typename T>
struct is_view : std::false_type {};
template<typename J, bool E>
struct is_view<basic_json_view<J, E>> : std::true_type {};
template<typename V>
encoded encode(V&& v)
{
using D = typename std::decay<V>::type;
return encode_impl(std::forward<V>(v), encode_tag<first_true<is_view<D>::value,
std::is_same<D, BasicJsonType>::value,
std::is_same<D, std::nullptr_t>::value,
std::is_same<D, bool>::value,
std::is_arithmetic<D>::value,
std::is_convertible<const D&, string_view_t>::value>::value> {});
}
/// a view of any document (copied; nothing is shared with it)
template<typename J, bool E>
encoded encode_impl(const basic_json_view<J, E>& v, encode_tag<0> /*view*/)
{
if (NLOHMANN_VIEW_UNLIKELY(v.m_node == nullptr))
{
throw_type_error(302, "type must be a value, but is ", "discarded");
}
encoded r;
if (!is_container(*v.m_node))
{
r.scalar = copy_scalar(*v.m_doc, *v.m_node);
return r;
}
r.region = alloc_nodes(m_doc, count_nodes<E>(*v.m_doc, v.m_node));
fill_nodes<E>(*v.m_doc, v.m_node, r.region);
edit_state_of(m_doc).regions.emplace(r.region, nullptr);
return r;
}
encoded encode_impl(const BasicJsonType& j, encode_tag<1> /*json*/)
{
encoded r;
if (!j.is_structured())
{
r.scalar = json_scalar(j);
return r;
}
r.region = alloc_nodes(m_doc, count_nodes(j));
fill_nodes(j, r.region);
edit_state_of(m_doc).regions.emplace(r.region, nullptr);
return r;
}
encoded encode_impl(std::nullptr_t /*unused*/, encode_tag<2> /*null*/)
{
encoded r;
r.scalar = plain_node(value_t::null);
return r;
}
encoded encode_impl(bool b, encode_tag<3> /*boolean*/)
{
encoded r;
r.scalar = plain_node(value_t::boolean);
r.scalar.flags = static_cast<std::uint8_t>(r.scalar.flags | (b ? node_flags::is_true : 0));
return r;
}
template<typename T>
encoded encode_impl(T x, encode_tag<4> /*number*/)
{
encoded r;
r.scalar = number_node(x, std::integral_constant<int, first_true<std::is_floating_point<T>::value, std::is_signed<T>::value>::value> {});
return r;
}
template<typename T>
encoded encode_impl(const T& s, encode_tag<5> /*string*/)
{
const string_view_t sv(s);
check_utf8(sv.data(), sv.size());
encoded r;
r.scalar = string_node(sv.data(), sv.size());
return r;
}
template<typename T>
encoded encode_impl(T&& x, encode_tag<6> /*other*/)
{
return encode_impl(BasicJsonType(std::forward<T>(x)), encode_tag<1> {});
}
static node plain_node(value_t k) noexcept
{
node n{};
n.kind = static_cast<std::uint8_t>(k);
n.flags = node_flags::is_new;
return n;
}
template<typename T>
node number_node(T x, std::integral_constant<int, 0> /*floating-point*/)
{
return float_node(static_cast<number_float_t>(x));
}
template<typename T>
node number_node(T x, std::integral_constant<int, 1> /*signed*/)
{
return integer_node(static_cast<std::uint64_t>(static_cast<std::int64_t>(x)), value_t::number_integer);
}
template<typename T>
node number_node(T x, std::integral_constant<int, 2> /*unsigned*/)
{
return integer_node(static_cast<std::uint64_t>(x), value_t::number_unsigned);
}
/// an integer with its canonical token in the edit arena
node integer_node(std::uint64_t bits, value_t k)
{
const bool negative = k == value_t::number_integer && static_cast<std::int64_t>(bits) < 0;
std::uint64_t magnitude = negative ? 0 - bits : bits;
std::array<char, 24> buf{};
char* p = buf.data() + buf.size(); // NOLINT(misc-const-correctness): digits are written through p
do
{
*--p = static_cast<char>('0' + (magnitude % 10));
magnitude /= 10;
}
while (magnitude != 0);
if (negative)
{
*--p = '-';
}
const auto len = static_cast<std::size_t>(buf.data() + buf.size() - p);
node n = plain_node(k);
n.flags = static_cast<std::uint8_t>(n.flags | node_flags::edited);
n.off = append_text(m_doc, p, len);
// number_length() adds one for the sign of number_integer nodes
n.extra = static_cast<std::uint16_t>(k == value_t::number_integer ? len - 1 : len);
set_integer_bits(n, bits);
return n;
}
/// a float with its shortest round-trip token (as basic_json::dump()
/// writes it), or nan, inf, -inf, in the edit arena
node float_node(number_float_t x)
{
string_t text;
if (std::isnan(x))
{
text = "nan";
}
else if (std::isinf(x))
{
text = x > 0 ? "inf" : "-inf";
}
else
{
text = BasicJsonType(x).dump();
}
node n = plain_node(value_t::number_float);
n.flags = static_cast<std::uint8_t>(n.flags | node_flags::edited);
n.extra = 0xFFFFu; // (the digit layout is not recorded)
n.off = append_text(m_doc, text.data(), text.size());
n.len = static_cast<std::uint32_t>(text.size());
return n;
}
/// a string (or key) in the edit arena
node string_node(const char* s, std::size_t len)
{
if (NLOHMANN_VIEW_UNLIKELY(len >= 0xFFFFFFFFu))
{
throw_out_of_range(416, "strings of 4 GiB or more are not supported by json_document"); // LCOV_EXCL_LINE
}
node n = plain_node(value_t::string);
n.flags = static_cast<std::uint8_t>(n.flags | node_flags::edited);
n.off = append_text(m_doc, s, len);
n.len = static_cast<std::uint32_t>(len);
return n;
}
/// a scalar of a view (of any document) as a node of this document
node copy_scalar(const document_data& from, const node& n)
{
if (&from == &m_doc)
{
return n; // the same storage
}
switch (static_cast<value_t>(n.kind))
{
case value_t::string:
// (an editable document only holds valid UTF-8, whatever the check of the other document was)
check_utf8(from.str(n), n.len);
return string_node(from.str(n), n.len);
case value_t::number_integer:
case value_t::number_unsigned:
return integer_node(integer_bits(n), static_cast<value_t>(n.kind));
case value_t::number_float:
{
node r = plain_node(value_t::number_float);
r.flags = static_cast<std::uint8_t>(r.flags | node_flags::edited);
r.off = append_text(m_doc, from.str(n), n.len);
r.len = n.len;
r.extra = n.extra;
return r;
}
case value_t::boolean:
{
node r = plain_node(value_t::boolean);
r.flags = static_cast<std::uint8_t>(r.flags | (n.flags & node_flags::is_true));
return r;
}
case value_t::null:
case value_t::object:
case value_t::array:
case value_t::binary:
case value_t::discarded:
default:
return plain_node(value_t::null);
}
}
node json_scalar(const BasicJsonType& j)
{
switch (j.type())
{
case value_t::null:
return plain_node(value_t::null);
case value_t::boolean:
{
node r = plain_node(value_t::boolean);
r.flags = static_cast<std::uint8_t>(r.flags | (j.template get<bool>() ? node_flags::is_true : 0));
return r;
}
case value_t::number_integer:
return integer_node(static_cast<std::uint64_t>(static_cast<std::int64_t>(j.template get<number_integer_t>())), value_t::number_integer);
case value_t::number_unsigned:
return integer_node(static_cast<std::uint64_t>(j.template get<number_unsigned_t>()), value_t::number_unsigned);
case value_t::number_float:
return float_node(j.template get<number_float_t>());
case value_t::string:
{
const auto& s = j.template get_ref<const string_t&>();
check_utf8(s.data(), s.size());
return string_node(s.data(), s.size());
}
case value_t::binary:
throw_type_error(319, "cannot store a binary value in a json_document", "");
case value_t::discarded:
case value_t::object:
case value_t::array:
default:
throw_type_error(302, "type must be a value, but is ", "discarded");
}
}
// The subtrees are walked with an explicit stack (as materialize() does):
// the nesting depth is limited by memory only, not by the call stack.
/// number of nodes of a subtree (containers, keys, scalars)
template<bool E>
static std::size_t count_nodes(const document_data& d, const node* n)
{
using walk = navigation<E>;
struct frame
{
const node* pos; ///< next element, or key of the next member
const node* end;
bool object;
};
std::vector<frame> open;
std::size_t r = 0;
for (;;)
{
++r;
if (is_container(*n))
{
open.push_back(frame{walk::first(d, n), walk::end(d, n), n->kind == static_cast<std::uint8_t>(value_t::object)});
}
// the next value: close finished containers, then step over the key
for (;;)
{
if (open.empty())
{
return r;
}
frame& f = open.back();
if (f.pos == f.end)
{
open.pop_back();
continue;
}
const node* v = f.pos;
if (f.object)
{
++r; // the key
++v;
}
f.pos = document_data::after(v);
n = walk::value(v);
break;
}
}
}
/// copy a subtree (of any document) as a contiguous sequence of
/// count_nodes() nodes
template<bool E>
void fill_nodes(const document_data& d, const node* n, node* out)
{
using walk = navigation<E>;
struct frame
{
const node* pos; ///< next element, or key of the next member
const node* end;
bool object;
node* self; ///< the container in the copy
};
std::vector<frame> open;
for (;;)
{
if (is_container(*n))
{
node* const self = out++;
*self = plain_node(static_cast<value_t>(n->kind));
self->len = n->len;
open.push_back(frame{walk::first(d, n), walk::end(d, n), n->kind == static_cast<std::uint8_t>(value_t::object), self});
}
else
{
*out++ = copy_scalar(d, *n);
}
// the next value: close finished containers, then copy the key
for (;;)
{
if (open.empty())
{
return;
}
frame& f = open.back();
if (f.pos == f.end)
{
f.self->next = static_cast<std::uint32_t>(out - f.self);
open.pop_back();
continue;
}
const node* v = f.pos;
if (f.object)
{
*out++ = copy_scalar(d, *v);
++v;
}
f.pos = document_data::after(v);
n = walk::value(v);
break;
}
}
}
static std::size_t count_nodes(const BasicJsonType& j)
{
using iterator = typename BasicJsonType::const_iterator;
struct frame
{
iterator pos;
iterator end;
bool object;
};
std::vector<frame> open;
const BasicJsonType* n = &j;
std::size_t r = 0;
for (;;)
{
++r;
if (n->is_structured())
{
open.push_back(frame{n->cbegin(), n->cend(), n->is_object()});
}
for (;;)
{
if (open.empty())
{
return r;
}
frame& f = open.back();
if (f.pos == f.end)
{
open.pop_back();
continue;
}
r += f.object ? 1 : 0; // the key
n = &*f.pos;
++f.pos;
break;
}
}
}
void fill_nodes(const BasicJsonType& j, node* out)
{
using iterator = typename BasicJsonType::const_iterator;
struct frame
{
iterator pos;
iterator end;
bool object;
node* self; ///< the container in the copy
};
std::vector<frame> open;
const BasicJsonType* n = &j;
for (;;)
{
if (n->is_structured())
{
node* const self = out++;
*self = plain_node(n->type());
self->len = static_cast<std::uint32_t>(n->size());
open.push_back(frame{n->cbegin(), n->cend(), n->is_object(), self});
}
else
{
*out++ = json_scalar(*n);
}
for (;;)
{
if (open.empty())
{
return;
}
frame& f = open.back();
if (f.pos == f.end)
{
f.self->next = static_cast<std::uint32_t>(out - f.self);
open.pop_back();
continue;
}
if (f.object)
{
const auto& key = f.pos.key();
check_utf8(key.data(), key.size());
*out++ = string_node(key.data(), key.size());
}
n = &*f.pos;
++f.pos;
break;
}
}
}
document_data& m_doc;
};
} // namespace view
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
@@ -1,281 +0,0 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#pragma once
#include <algorithm> // max, min
#include <cstddef> // size_t
#include <cstdint> // uint8_t, uint32_t
#include <cstring> // memcpy
#include <functional> // less
#include <memory> // unique_ptr
#include <utility> // move
#include <nlohmann/json.hpp>
#include <nlohmann/detail/view/document_data.hpp>
#include <nlohmann/detail/view/errors.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
#include <nlohmann/detail/view/node.hpp>
// The storage of edits. Edits never move or resize the parsed index: every
// value keeps its node, so views stay valid. New values and element sequences
// live in chunks that never move; strings and number tokens written by edits
// live in the edit arena. An array/object whose elements change gets
// node_flags::moved: its elements then live in a separate sequence (a header
// node, then the entries), whose entries link to the values (kind_link).
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
{
namespace view
{
inline document_data::edit_state& edit_state_of(document_data& d)
{
if (!d.edits)
{
d.edits.reset(new document_data::edit_state()); // NOLINT(cppcoreguidelines-owning-memory,modernize-make-unique): owned by the unique_ptr; std::make_unique is C++14
}
return *d.edits;
}
/// k consecutive nodes that never move (new values and blocks)
inline node* alloc_nodes(document_data& d, std::size_t k)
{
document_data::edit_state& e = edit_state_of(d);
if (NLOHMANN_VIEW_UNLIKELY(static_cast<std::size_t>(e.chunk_end - e.chunk_cur) < k))
{
if (k >= e.chunk_next / 2)
{
// a large request (half of the next chunk or more) gets a chunk of
// its own, so that the rest of the current chunk stays available
// for smaller ones
std::unique_ptr<node[]> own(new node[k]()); // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
e.chunks.push_back(std::move(own));
e.bytes += k * sizeof(node);
return e.chunks.back().get();
}
// a smaller request leaves the rest of the current chunk unused
// (fewer than k nodes, and k is less than half of the new chunk)
const std::size_t count = e.chunk_next;
std::unique_ptr<node[]> fresh(new node[count]()); // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
e.chunks.push_back(std::move(fresh));
e.chunk_cur = e.chunks.back().get();
e.chunk_end = e.chunk_cur + count;
e.chunk_next = (std::min)(e.chunk_next * 2, std::size_t{65536});
e.bytes += count * sizeof(node);
}
node* const r = e.chunk_cur;
e.chunk_cur += k;
return r;
}
/// The capacity of the edit arena after it grows by n bytes (`used` of `cap`
/// are taken): doubled, or what is needed plus some room, but never more than
/// the 4 GiB - 1 bytes that the 32-bit offsets of nodes can address. An error
/// if n more bytes do not fit even then.
inline std::size_t text_capacity(std::size_t cap, std::size_t used, std::size_t n)
{
constexpr std::size_t limit = 0xFFFFFFFFu;
if (NLOHMANN_VIEW_UNLIKELY(used > limit || n > limit - used))
{
throw_out_of_range(416, "edits of 4 GiB or more are not supported by json_document");
}
const std::size_t needed = used + n;
const std::size_t wanted = needed + (std::min)(limit - needed, std::size_t{256});
const std::size_t doubled = cap > limit / 2 ? limit : cap * 2;
return (std::max)(doubled, wanted);
}
/// copy n bytes into the edit arena and return their offset; a new buffer
/// leaves the old one alive, so that string views into it remain valid; n is
/// 0 for an empty string (written by an edit, or the key of a new member)
inline std::uint32_t append_text(document_data& d, const char* s, std::size_t n)
{
document_data::edit_state& e = edit_state_of(d);
if (NLOHMANN_VIEW_UNLIKELY(e.text_cap - e.text_used < n))
{
const std::size_t cap = text_capacity(e.text_cap, e.text_used, n);
std::unique_ptr<char[]> fresh(new char[cap]); // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
if (NLOHMANN_VIEW_LIKELY(e.text_used != 0)) // (0 only for the first buffer, and then texts is empty)
{
std::memcpy(fresh.get(), e.texts.back().get(), e.text_used);
}
e.texts.push_back(std::move(fresh));
e.text_cap = cap;
e.bytes += cap;
d.base[2] = e.texts.back().get();
}
const auto off = static_cast<std::uint32_t>(e.text_used);
if (n != 0)
{
std::memcpy(e.texts.back().get() + e.text_used, s, n);
}
e.text_used += n;
return off;
}
/// the capacity in nodes of the block of a moved container (0: a fixed
/// sequence, the elements of a new value)
inline std::size_t moved_capacity(const document_data& d, const node* n) noexcept
{
return d.edits->moved_cap[n->off];
}
/// Make room for one more moved container. This is the part of set_moved()
/// that can throw: a caller that changes a node before it calls set_moved()
/// calls this first, so that a failure leaves the node as it was.
inline void reserve_moved(document_data& d)
{
document_data::edit_state& e = edit_state_of(d);
if (e.moved.size() >= 0xFFFFFFFFu)
{
throw_out_of_range(416, "more than 4294967295 edited arrays and objects are not supported by json_document"); // LCOV_EXCL_LINE
}
if (e.moved.size() == e.moved.capacity() || e.moved_cap.size() == e.moved_cap.capacity())
{
// both grow before either changes, so that the push_backs cannot throw
// (both conditions are needed: reserve() may allocate more than asked,
// so the two capacities can differ)
e.moved.reserve((e.moved.size() * 2) + 16);
e.moved_cap.reserve((e.moved.size() * 2) + 16);
}
}
/// let container n take its elements from `seq` (header node first); cannot
/// throw if n is moved already or reserve_moved() was called
inline void set_moved(document_data& d, node* n, node* seq, std::size_t cap)
{
document_data::edit_state& e = edit_state_of(d);
if ((n->flags & node_flags::moved) != 0)
{
e.moved[n->off] = seq;
e.moved_cap[n->off] = cap;
return;
}
reserve_moved(d);
e.moved.push_back(seq);
e.moved_cap.push_back(cap);
n->off = static_cast<std::uint32_t>(e.moved.size() - 1);
n->flags = static_cast<std::uint8_t>(n->flags | node_flags::moved | node_flags::is_new);
}
/// Make the elements of container n a growable block with room for `extra`
/// more nodes, and return its header. The entries link to the existing
/// values, which stay where they are. A block that grows is copied (its old
/// space is not reused).
inline node* block_of(document_data& d, node* n, std::size_t extra)
{
if ((n->flags & node_flags::moved) != 0 && moved_capacity(d, n) != 0)
{
node* const h = d.edits->moved[n->off];
if (h->next + extra <= moved_capacity(d, n))
{
return h;
}
const std::size_t cap = (std::max)(2 * moved_capacity(d, n), h->next + extra);
node* const nh = alloc_nodes(d, cap);
std::memcpy(nh, h, h->next * sizeof(node));
set_moved(d, n, nh, cap);
return nh;
}
reserve_moved(d); // (so that set_moved() below cannot throw: the links are marked before)
const bool object = n->kind == static_cast<std::uint8_t>(value_t::object);
const std::size_t used = 1 + (static_cast<std::size_t>(n->len) * (object ? 2 : 1));
const std::size_t cap = used + extra;
node* const h = alloc_nodes(d, cap);
*h = node{};
h->kind = n->kind;
h->len = n->len;
h->next = static_cast<std::uint32_t>(used);
node* o = h + 1;
for (const node* c = d.first_child_edited(n), *e = d.child_end_edited(n); c != e;)
{
if (object)
{
*o++ = *c++; // the key
}
make_link(*o, const_cast<node*>(document_data::deref(c))); // NOLINT(cppcoreguidelines-pro-type-const-cast): the nodes belong to the document
++o;
c = document_data::after(c);
}
set_moved(d, n, h, cap);
return h;
}
/// The container whose elements include `target`; nullptr for the root, for
/// a value that is no longer part of the document, and for a value that is
/// only reached through a link. Values never move between allocations, so
/// the path to `target` stays inside the allocation that holds it (the parsed
/// index, or one new value), where the extent of each container (`next`)
/// still covers its original subtree.
inline node* find_parent(const document_data& d, const node* target)
{
const std::less<const node*> lt{};
const node* lo = d.nodes;
const node* hi = d.nodes + d.node_count;
const node* c = d.nodes;
if (lt(target, lo) || !lt(target, hi))
{
if (!d.edits)
{
return nullptr; // LCOV_EXCL_LINE (nodes outside the index exist only after edits)
}
auto it = d.edits->regions.upper_bound(target);
if (it == d.edits->regions.begin())
{
return nullptr; // LCOV_EXCL_LINE (an array/object with elements is in the index or a new value)
}
--it;
lo = it->first;
hi = lo + lo->next;
if (!lt(target, hi))
{
return nullptr; // LCOV_EXCL_LINE (a single-node value, reached through a link)
}
// the root of a new value is the element sequence of its owner, or a linked value
c = it->second != nullptr ? it->second : lo;
}
if (target == lo)
{
return nullptr;
}
for (;;)
{
if (!is_container(*c))
{
return nullptr; // LCOV_EXCL_LINE (the value is inside c)
}
const bool object = c->kind == static_cast<std::uint8_t>(value_t::object);
const node* down = nullptr;
for (const node* p = d.first_child_edited(c), *e = d.child_end_edited(c); p != e;)
{
const node* const at = object ? p + 1 : p;
const node* const v = document_data::deref(at);
if (v == target)
{
return const_cast<node*>(c); // NOLINT(cppcoreguidelines-pro-type-const-cast): the nodes belong to the document
}
if (is_container(*v) && !lt(v, lo) && lt(v, target) && lt(target, v + v->next))
{
down = v;
break;
}
p = document_data::after(at);
}
if (down == nullptr)
{
return nullptr;
}
c = down;
}
}
} // namespace view
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
-5
View File
@@ -30,11 +30,6 @@ namespace view
NLOHMANN_VIEW_THROW(type_error::create(id, concat(prefix, type), nullptr));
}
[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_type_error(int id, const std::string& msg)
{
NLOHMANN_VIEW_THROW(type_error::create(id, msg, nullptr));
}
[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_out_of_range(int id, const std::string& msg)
{
NLOHMANN_VIEW_THROW(out_of_range::create(id, msg, nullptr));
-744
View File
@@ -1,744 +0,0 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#pragma once
#include <algorithm> // min, sort
#include <array> // array
#include <cstddef> // offsetof, size_t
#include <cstdint> // int64_t, uint8_t, uint16_t, uint32_t, uint64_t
#include <cstring> // memcmp, memcpy, memset
#include <limits> // numeric_limits
#include <string> // string
#include <vector> // vector
#include <nlohmann/json.hpp>
#include <nlohmann/detail/view/document_data.hpp>
#include <nlohmann/detail/view/errors.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
#include <nlohmann/detail/view/node.hpp>
#include <nlohmann/detail/view/number.hpp>
#include <nlohmann/detail/view/object_index.hpp>
#include <nlohmann/detail/view/scan.hpp>
// Images: a document stored so that loading it needs no parsing.
//
// Layout (little-endian): a 64-byte header, the nodes, the text (the source,
// followed by the number tokens written by edits), a NUL, the decoded strings
// (followed by the strings written by edits), a NUL. The idea is that of
// zero-copy formats such as FlatBuffers (https://github.com/google/flatbuffers)
// and YaFF (https://github.com/yandex/yaff); no code is taken from them.
// check_image follows the idea of FlatBuffers' Verifier (bounds and
// structure) and also checks what the parser guarantees about strings and
// numbers, so that reading and serializing a checked image is safe and yields
// valid JSON.
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
{
namespace view
{
/// how load() checks an image
enum class image_check
{
/// everything the parser guarantees: structure and bounds, strings (valid
/// UTF-8; source strings without quotes, backslashes, and control
/// characters), and numbers (well-formed, matching the stored values)
full,
/// structure and bounds only: reading and serializing are safe, but a
/// crafted image can yield invalid UTF-8, strings that serialize to
/// invalid JSON, or numbers that differ from their text
bounds,
/// none: for images from a trusted source only (a damaged image is
/// undefined behavior)
none,
};
struct image_header
{
std::array<char, 4> magic; ///< "NJVI"
std::uint32_t version; ///< 1
std::uint64_t node_count;
std::uint64_t text_size;
std::uint64_t arena_size;
std::array<std::uint64_t, 4> reserved; ///< zero (for later versions)
};
static_assert(sizeof(image_header) == 64, "the image header must be 64 bytes");
constexpr std::uint32_t image_version = 1;
/// the largest node count and text or string size of an image (as for parsed
/// documents, offsets and counts must fit 32 bits)
constexpr std::uint64_t image_limit = 0xFFFFFFF0u;
/// Copy the current structure of an edited document into nodes in document
/// order, as the parser would have written them. Text written by edits is
/// appended to text_tail (number tokens) and arena_tail (strings); floats that
/// are not finite become null, as dump() writes them.
inline void compact_nodes(const document_data& d, std::size_t arena_size, std::vector<node>& out, std::string& text_tail, std::string& arena_tail)
{
struct frame
{
const node* cur;
const node* end;
std::size_t index; ///< the container's node in out
std::uint32_t count;
bool object;
};
std::vector<frame> stack;
const auto string_node = [&](const node & s)
{
node r = s;
r.extra = 0;
r.flags = static_cast<std::uint8_t>(s.flags & node_flags::storage);
if (r.flags == node_flags::edited)
{
r.off = static_cast<std::uint32_t>(arena_size + arena_tail.size());
arena_tail.append(d.str(s), s.len);
r.flags = node_flags::escaped;
}
return r;
};
const auto copy_node = [&](const node * v)
{
node r = *v;
switch (static_cast<value_t>(v->kind))
{
case value_t::object:
case value_t::array:
r.flags = 0;
r.extra = 0;
r.off = (v->flags & (node_flags::moved | node_flags::is_new)) != 0 ? 0 : v->off;
r.len = 0; // counted below
r.next = 0; // set when the container is complete
stack.push_back(frame{d.first_child_edited(v), d.child_end_edited(v), out.size(), 0, v->kind == static_cast<std::uint8_t>(value_t::object)});
break;
case value_t::string:
r = string_node(*v);
break;
case value_t::number_integer:
case value_t::number_unsigned:
if ((v->flags & node_flags::storage) == node_flags::edited)
{
r.off = static_cast<std::uint32_t>(d.size + text_tail.size());
text_tail.append(d.str(*v), number_length(*v));
}
r.flags = 0;
break;
case value_t::number_float:
if ((v->flags & node_flags::storage) == node_flags::edited)
{
const char* const t = d.str(*v);
if (t[0] == 'n' || t[0] == 'i' || (v->len > 1 && t[1] == 'i'))
{
r = node{}; // nan and infinity: null, as dump() writes them
r.kind = static_cast<std::uint8_t>(value_t::null);
break;
}
r.off = static_cast<std::uint32_t>(d.size + text_tail.size());
text_tail.append(t, v->len);
r.extra = 0xFFFFu; // the digit layout is not recorded
}
r.flags = 0;
break;
case value_t::boolean:
r.flags = static_cast<std::uint8_t>(v->flags & node_flags::is_true);
break;
case value_t::null:
case value_t::binary:
case value_t::discarded:
default:
r.flags = 0;
break;
}
out.push_back(r);
};
copy_node(d.nodes);
while (!stack.empty())
{
const frame top = stack.back();
if (top.cur == top.end)
{
node& c = out[top.index];
c.len = top.count;
c.next = static_cast<std::uint32_t>(out.size() - top.index);
stack.pop_back();
continue;
}
if (top.object)
{
out.push_back(string_node(*top.cur));
}
const node* const value = top.cur + (top.object ? 1 : 0);
stack.back().cur = document_data::after(value);
++stack.back().count;
copy_node(document_data::deref(value)); // may grow the stack
}
}
/// the document as an image
inline std::vector<std::uint8_t> save_image(const document_data& d)
{
#if !NLOHMANN_VIEW_LITTLE_ENDIAN
throw_type_error(320, "json_document images need a little-endian target"); // LCOV_EXCL_LINE
#endif
const std::size_t arena_size = d.arena_size;
std::vector<node> compacted;
std::string text_tail;
std::string arena_tail;
if (d.edits)
{
compact_nodes(d, arena_size, compacted, text_tail, arena_tail);
}
const node* const nodes = d.edits ? compacted.data() : d.nodes;
const std::size_t count = d.edits ? compacted.size() : d.node_count;
const std::size_t text_size = d.size + text_tail.size();
const std::size_t total_arena = arena_size + arena_tail.size();
if (NLOHMANN_VIEW_UNLIKELY(text_size >= image_limit || total_arena >= image_limit || count >= image_limit))
{
// LCOV_EXCL_START (4 GiB)
throw_out_of_range(416, "images of 4 GiB or more are not supported by json_document");
// LCOV_EXCL_STOP
}
image_header h{};
h.magic = {{'N', 'J', 'V', 'I'}};
h.version = image_version;
h.node_count = count;
h.text_size = text_size;
h.arena_size = total_arena;
std::vector<std::uint8_t> image(sizeof(h) + (count * sizeof(node)) + text_size + 1 + total_arena + 1);
std::uint8_t* o = image.data();
std::memcpy(o, &h, sizeof(h));
o += sizeof(h);
std::memcpy(o, nodes, count * sizeof(node));
// the hash indexes are rebuilt by load(), whose check rejects any other
// extra of an object, and the same document saves to the same bytes
for (std::size_t i = 0; i < count; ++i)
{
if (nodes[i].kind == static_cast<std::uint8_t>(value_t::object) && nodes[i].extra != 0)
{
std::memset(o + (i * sizeof(node)) + offsetof(node, extra), 0, sizeof(nodes[i].extra));
}
}
o += count * sizeof(node);
const auto append = [&o](const char* s, std::size_t n)
{
if (n != 0)
{
std::memcpy(o, s, n);
o += n;
}
};
append(d.src, d.size);
append(text_tail.data(), text_tail.size());
*o++ = 0;
append(d.base[1], arena_size);
append(arena_tail.data(), arena_tail.size());
*o = 0;
return image;
}
/// the parts of a number token that the checks need
struct number_token
{
const unsigned char* int_start;
std::size_t int_digits;
std::size_t frac_digits;
std::int64_t exponent;
bool negative;
bool is_float; ///< a fraction or an exponent
};
/// whether [s, s + len) is a JSON number (the grammar the parser accepts)
inline bool scan_number_token(const unsigned char* s, std::size_t len, number_token& t)
{
const unsigned char* const e = s + len;
const unsigned char* p = s;
t.negative = p != e && *p == '-';
p += t.negative ? 1 : 0;
if (p == e)
{
return false;
}
t.int_start = p;
if (*p == '0')
{
++p;
}
else if (*p >= '1' && *p <= '9')
{
do
{
++p;
}
while (p != e && is_digit(*p));
}
else
{
return false;
}
t.int_digits = static_cast<std::size_t>(p - t.int_start);
t.frac_digits = 0;
t.is_float = false;
if (p != e && *p == '.')
{
const unsigned char* const f0 = ++p;
while (p != e && is_digit(*p))
{
++p;
}
if (p == f0)
{
return false;
}
t.frac_digits = static_cast<std::size_t>(p - f0);
t.is_float = true;
}
t.exponent = 0;
if (p != e && (*p | 0x20u) == 'e')
{
++p;
const bool exp_negative = p != e && *p == '-';
p += (p != e && (*p == '+' || *p == '-')) ? 1 : 0;
if (p == e || !is_digit(*p))
{
return false;
}
do
{
t.exponent = t.exponent < 100000 ? (t.exponent * 10) + (*p - '0') : t.exponent;
++p;
}
while (p != e && is_digit(*p));
t.exponent = exp_negative ? -t.exponent : t.exponent;
t.is_float = true;
}
return p == e;
}
/// the digit layout the parser records for a float token (compaction
/// writes "many" instead; the caller accepts both)
inline std::uint16_t float_layout(const number_token& t)
{
return static_cast<std::uint16_t>((std::min)(t.int_digits, std::size_t{255}) | ((std::min)(t.frac_digits, std::size_t{255}) << 8u));
}
/// whether a float token (well-formed, per scan_number_token) is finite as
/// double; parse() rejects floats that overflow, and as there, only a number
/// whose magnitude could reach 1e308 needs the conversion
inline bool float_token_finite(const unsigned char* s, std::size_t len, const number_token& t)
{
if (static_cast<std::int64_t>(t.int_digits) + t.exponent > 300)
{
node n{};
n.kind = static_cast<std::uint8_t>(value_t::number_float);
n.len = static_cast<std::uint32_t>(len);
const auto v = float_value<double>(reinterpret_cast<const char*>(s), n); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
return v <= (std::numeric_limits<double>::max)() && v >= -(std::numeric_limits<double>::max)();
}
return true;
}
/// whether an integer node matches its token the way the parser records it
/// (after the bounds check; the token has at most 256 characters)
inline bool check_integer(const node& n, const unsigned char* text)
{
const std::size_t len = number_length(n);
const unsigned char* const s = text + n.off;
number_token t{};
if (!scan_number_token(s, len, t))
{
return false;
}
// the token's value is the stored one; number_integer nodes of edits can
// be non-negative (as basic_json keeps the type of a value)
const bool integer = n.kind == static_cast<std::uint8_t>(value_t::number_integer);
if (t.is_float || t.int_digits > 20 || (t.negative && !integer))
{
return false;
}
// (at most 19 digits cannot overflow; 20 digits are compared with 2^64 - 1)
if (t.int_digits == 20 && std::memcmp(t.int_start, "18446744073709551615", 20) > 0)
{
return false;
}
std::uint64_t m = 0;
for (const unsigned char* d = t.int_start; d != t.int_start + t.int_digits; ++d)
{
m = (m * 10) + static_cast<std::uint64_t>(*d - '0');
}
if (integer && m > (t.negative ? std::uint64_t{1} << 63u : (std::uint64_t{1} << 63u) - 1))
{
return false;
}
return integer_bits(n) == (t.negative ? 0 - m : m);
}
/// a byte range of the text or of the decoded strings
struct byte_range
{
std::uint32_t off;
std::uint32_t len;
};
/// a float token and the digit layout its node records
struct float_range
{
std::uint32_t off;
std::uint32_t len;
std::uint16_t extra;
};
/// Run check once for each distinct range of ranges (which are within bounds
/// and not empty). Ranges that are not identical must not overlap: save()
/// writes every string and token to a place of its own, and nodes that share
/// a value (copies within a document) share the whole range. This bounds the
/// work by the size of the text, however many nodes point to the same bytes.
template<typename Check>
bool check_distinct_ranges(std::vector<byte_range>& ranges, Check check)
{
std::sort(ranges.begin(), ranges.end(), [](const byte_range & a, const byte_range & b)
{
return a.off != b.off ? a.off < b.off : a.len < b.len;
});
std::size_t end = 0;
for (std::size_t i = 0; i < ranges.size(); ++i)
{
const byte_range r = ranges[i];
if (i != 0 && r.off == ranges[i - 1].off && r.len == ranges[i - 1].len)
{
continue;
}
if (r.off < end || !check(r))
{
return false;
}
end = static_cast<std::size_t>(r.off) + r.len;
}
return true;
}
/// Check the float nodes: each token once (nodes of the same token must
/// record layouts that match it), tokens must not overlap.
inline bool check_float_ranges(std::vector<float_range>& ranges, const unsigned char* text)
{
std::sort(ranges.begin(), ranges.end(), [](const float_range & a, const float_range & b)
{
if (a.off != b.off)
{
return a.off < b.off;
}
if (a.len != b.len)
{
return a.len < b.len;
}
return a.extra < b.extra;
});
std::size_t end = 0;
std::size_t i = 0;
while (i < ranges.size())
{
const float_range r = ranges[i];
if (r.off < end)
{
return false;
}
number_token t{};
const unsigned char* const s = text + r.off;
if (!scan_number_token(s, r.len, t) || !float_token_finite(s, r.len, t))
{
return false;
}
// the digit layout the parser records (or "many", as compaction writes it)
const std::uint16_t layout = float_layout(t);
for (; i < ranges.size() && ranges[i].off == r.off && ranges[i].len == r.len; ++i)
{
if (ranges[i].extra != layout && ranges[i].extra != 0xFFFFu)
{
return false;
}
}
end = static_cast<std::size_t>(r.off) + r.len;
}
return true;
}
/// the content checks of check_image: source strings as the parser leaves
/// them (no quotes, backslashes, or control characters), decoded strings
/// (valid UTF-8), and float tokens
inline bool check_contents(const unsigned char* text, const unsigned char* arena,
std::vector<byte_range>& source_strings, std::vector<byte_range>& decoded_strings,
std::vector<float_range>& floats)
{
return check_distinct_ranges(source_strings, [text](const byte_range & r)
{
const unsigned char* const b = text + r.off;
return scan_string_run(b, b + r.len) == b + r.len;
})
&& check_distinct_ranges(decoded_strings, [arena](const byte_range & r)
{
return valid_utf8_prefix(arena + r.off, r.len) == r.len;
})
&& check_float_ranges(floats, text);
}
/// Check the nodes of a loaded image against its text and decoded strings:
/// kinds, flags, and `extra`; extents and element counts of arrays and
/// objects; keys; bounds; string contents (source strings as the parser
/// leaves them: no quotes, backslashes, or control characters; all strings
/// valid UTF-8); and number tokens.
///
/// The structure and the bounds are checked node by node. The contents of
/// strings and of float tokens are checked afterwards, once for each distinct
/// range (see check_distinct_ranges), so that the full check is linear in the
/// size of the image plus the sorting of the ranges, and not in the number of
/// nodes times the size of the text.
inline bool check_image(const node* nodes, std::size_t count, const unsigned char* text, std::size_t text_size,
const unsigned char* arena, std::size_t arena_size, bool full)
{
struct frame
{
std::size_t end;
std::uint32_t len;
std::uint32_t seen;
bool object;
bool expect_key;
};
std::vector<frame> stack;
std::vector<byte_range> source_strings;
std::vector<byte_range> decoded_strings;
std::vector<float_range> floats;
const auto check_string = [&](const node & n) -> bool
{
if ((n.flags & ~node_flags::escaped) != 0 || n.extra != 0)
{
return false;
}
const bool decoded = (n.flags & node_flags::escaped) != 0;
const std::size_t limit = decoded ? arena_size : text_size;
if (n.off > limit || n.len > limit - n.off)
{
return false;
}
if (full && n.len != 0)
{
(decoded ? decoded_strings : source_strings).push_back(byte_range{n.off, n.len});
}
return true;
};
// bounds of a number token; the recorded digit layout must lie within it
const auto number_in_bounds = [&](const node & n) -> bool
{
const std::size_t len = number_length(n);
if (len == 0 || n.off > text_size || len > text_size - n.off)
{
return false;
}
if (n.kind != static_cast<std::uint8_t>(value_t::number_float))
{
return (n.extra >> 8u) == 0;
}
// float_value() reads the sign, the integer digits, and the point and
// fraction digits the layout records (a layout of more than 19 digits
// means the general conversion, which stays within the token)
const std::size_t int_digits = n.extra & 0xFFu;
const std::size_t frac_digits = n.extra >> 8u;
const std::size_t need = (text[n.off] == '-' ? 1u : 0u) + int_digits + (frac_digits != 0 ? frac_digits + 1 : 0);
return int_digits + frac_digits > 19 || need <= len;
};
std::size_t i = 0;
for (;;)
{
// close finished arrays and objects
while (!stack.empty() && i == stack.back().end)
{
const frame f = stack.back();
if (f.seen != f.len || (f.object && !f.expect_key))
{
return false;
}
stack.pop_back();
if (!stack.empty())
{
++stack.back().seen;
stack.back().expect_key = true;
}
}
if (i == count)
{
return stack.empty() && (!full || check_contents(text, arena, source_strings, decoded_strings, floats));
}
if (i != 0 && stack.empty())
{
return false; // nodes after the root
}
const node& n = nodes[i];
if (!stack.empty() && stack.back().object && stack.back().expect_key)
{
if (n.kind != static_cast<std::uint8_t>(value_t::string) || !check_string(n))
{
return false;
}
stack.back().expect_key = false;
++i;
continue;
}
bool complete = true;
switch (static_cast<value_t>(n.kind))
{
case value_t::null:
// (the offset of a literal is read to size the output of dump())
if (n.flags != 0 || n.extra != 0 || n.off > text_size)
{
return false;
}
break;
case value_t::boolean:
if ((n.flags & ~node_flags::is_true) != 0 || n.extra != 0 || n.off > text_size)
{
return false;
}
break;
case value_t::string:
if (!check_string(n))
{
return false;
}
break;
case value_t::number_integer:
case value_t::number_unsigned:
case value_t::number_float:
if (n.flags != 0 || !number_in_bounds(n))
{
return false;
}
if (full)
{
// (the contents of float tokens are checked later; the token of an
// integer has at most 256 characters)
if (n.kind == static_cast<std::uint8_t>(value_t::number_float))
{
floats.push_back(float_range{n.off, n.len, n.extra});
}
else if (!check_integer(n, text))
{
return false;
}
}
break;
case value_t::array:
case value_t::object:
{
const std::size_t limit = stack.empty() ? count : stack.back().end;
if (n.flags != 0 || n.extra != 0 || n.next == 0 || n.next > limit - i || n.off > text_size)
{
return false;
}
stack.push_back(frame{i + n.next, n.len, 0, n.kind == static_cast<std::uint8_t>(value_t::object), true});
complete = false;
break;
}
case value_t::binary:
case value_t::discarded:
default:
return false;
}
++i;
if (complete && !stack.empty())
{
++stack.back().seen;
stack.back().expect_key = true;
}
}
}
[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_invalid_image(const char* what)
{
throw_parse_error(116, concat("invalid json_document image: ", what));
}
/// Read an image into d. The text and the decoded strings stay in the image;
/// the nodes are copied (so that they are aligned, and edits can change them).
inline void load_image(document_data& d, const std::uint8_t* image, std::size_t size, image_check check)
{
#if !NLOHMANN_VIEW_LITTLE_ENDIAN
throw_type_error(320, "json_document images need a little-endian target"); // LCOV_EXCL_LINE
#endif
if (image == nullptr || size < sizeof(image_header))
{
throw_invalid_image("too short");
}
image_header h{};
std::memcpy(&h, image, sizeof(h));
// (the reserved fields are for later versions)
if (std::memcmp(h.magic.data(), "NJVI", 4) != 0 || h.version != image_version
|| (h.reserved[0] | h.reserved[1] | h.reserved[2] | h.reserved[3]) != 0)
{
throw_invalid_image("unknown format");
}
const std::size_t room = size - sizeof(h);
if (h.node_count == 0 || h.node_count > room / sizeof(node) || h.node_count >= image_limit || h.text_size >= image_limit || h.arena_size >= image_limit)
{
throw_invalid_image("sizes out of range");
}
const auto count = static_cast<std::size_t>(h.node_count);
const auto text_size = static_cast<std::size_t>(h.text_size);
const auto arena_size = static_cast<std::size_t>(h.arena_size);
const std::size_t text_at = sizeof(h) + (count * sizeof(node));
// the text, a NUL, the decoded strings, a NUL, and nothing after them
if (size - text_at < 2 || text_size > size - text_at - 2 || arena_size != size - text_at - text_size - 2
|| image[text_at + text_size] != 0 || image[size - 1] != 0)
{
throw_invalid_image("sizes out of range");
}
d.discarded = true;
d.edits.reset();
d.base[2] = nullptr;
d.owned.clear();
if (d.owned_image.empty() || image != d.owned_image.data())
{
d.owned_image.clear();
}
d.arena.clear();
d.indexes.clear();
d.index_slots.clear();
d.large_objects.clear();
d.node_count = 0;
d.reserve(count);
std::memcpy(d.nodes, image + sizeof(h), count * sizeof(node));
d.node_count = count;
const std::uint8_t* const text = image + text_at;
const std::uint8_t* const arena = text + text_size + 1;
d.src = reinterpret_cast<const char*>(text); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
d.size = text_size;
d.base[0] = d.src;
d.base[1] = reinterpret_cast<const char*>(arena); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
d.arena_size = arena_size;
if (check != image_check::none && !check_image(d.nodes, count, text, text_size, arena, arena_size, check == image_check::full))
{
throw_invalid_image("the check failed");
}
// the hash indexes of large objects, as after parsing
for (std::size_t i = 0; i < count; ++i)
{
const node& n = d.nodes[i];
if (n.kind == static_cast<std::uint8_t>(value_t::object) && n.len >= document_data::index_min_members)
{
d.large_objects.push_back(static_cast<std::uint32_t>(i));
}
}
build_object_indexes(d);
std::vector<std::uint32_t>().swap(d.large_objects); // (only needed while building)
d.discarded = false;
}
} // namespace view
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
+1 -1
View File
@@ -73,7 +73,7 @@ class view_iterator
NLOHMANN_VIEW_ALWAYS_INLINE View operator*() const noexcept
{
return View(m_doc, View::navigation::value(m_pos + m_value_offset));
return View(m_doc, m_pos + m_value_offset);
}
pointer operator->() const noexcept
+15 -32
View File
@@ -18,7 +18,6 @@
#include <nlohmann/detail/view/document_data.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
#include <nlohmann/detail/view/node.hpp>
#include <nlohmann/detail/view/object_index.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
@@ -88,20 +87,14 @@ class short_key
/// nullptr (the search stops at the first match; materialize() and parse()
/// keep the last value of a duplicate key instead); most keys are rejected by
/// their length, from the index alone
template<bool Editable>
const node* find_member(const document_data& d, const node* object, const char* key, std::size_t n) noexcept
inline const node* find_member(const document_data& d, const node* object, const char* key, std::size_t n) noexcept
{
using nav = navigation<Editable>;
if (NLOHMANN_VIEW_UNLIKELY(object->extra != 0) && (!nav::editable() || (object->flags & node_flags::moved) == 0))
{
return find_indexed(d, object, key, n); // a large object (whose members have not been edited)
}
const node* const end = nav::end(d, object);
const node* const end = document_data::child_end(object);
const auto* const k = reinterpret_cast<const unsigned char*>(key); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
if (NLOHMANN_VIEW_LIKELY(n <= 16))
{
const short_key probe(k, n);
for (const node* m = nav::first(d, object); m != end; m = document_data::after(m + 1))
for (const node* m = document_data::first_child(object); m != end; m = document_data::after(m + 1))
{
if (m->len == n && probe.matches(reinterpret_cast<const unsigned char*>(d.str(*m)))) // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
{
@@ -110,7 +103,7 @@ const node* find_member(const document_data& d, const node* object, const char*
}
return nullptr;
}
for (const node* m = nav::first(d, object); m != end; m = document_data::after(m + 1))
for (const node* m = document_data::first_child(object); m != end; m = document_data::after(m + 1))
{
if (d.key_equals(*m, key, n))
{
@@ -121,13 +114,11 @@ const node* find_member(const document_data& d, const node* object, const char*
}
/// the number of members of an object with this key (duplicates included)
template<bool Editable>
std::size_t count_members(const document_data& d, const node* object, const char* key, std::size_t n) noexcept
inline std::size_t count_members(const document_data& d, const node* object, const char* key, std::size_t n) noexcept
{
using nav = navigation<Editable>;
std::size_t result = 0;
const node* const end = nav::end(d, object);
for (const node* m = nav::first(d, object); m != end; m = document_data::after(m + 1))
const node* const end = document_data::child_end(object);
for (const node* m = document_data::first_child(object); m != end; m = document_data::after(m + 1))
{
if (d.key_equals(*m, key, n))
{
@@ -155,17 +146,10 @@ SizeType to_index(IntegerType idx) noexcept
return (idx < zero || static_cast<IntegerType>(result) != idx) ? (std::numeric_limits<SizeType>::max)() : result;
}
/// the entry of the element of an array at an index below its size (a link
/// in the moved sequences of editable documents)
template<bool Editable>
const node* element_at(const document_data& d, const node* array, std::size_t idx) noexcept
/// the element of an array at an index below its size
inline const node* element_at(const node* array, std::size_t idx) noexcept
{
using nav = navigation<Editable>;
const node* e = nav::first(d, array);
if (nav::editable() && (array->flags & node_flags::moved) != 0 && d.edits->moved_cap[array->off] != 0)
{
return e + idx; // a growable block: one link per element
}
const node* e = document_data::first_child(array);
for (std::size_t i = 0; i < idx; ++i)
{
e = document_data::after(e);
@@ -173,14 +157,13 @@ const node* element_at(const document_data& d, const node* array, std::size_t id
return e;
}
/// the entry of the last element of a non-empty array, or the key of the
/// last member of a non-empty object
template<bool Editable>
const node* last_child(const document_data& d, const node* container) noexcept
/// the last element of a non-empty array, or the key of the last member of a
/// non-empty object
inline const node* last_child(const node* container) noexcept
{
const std::size_t value_offset = container->kind == static_cast<std::uint8_t>(value_t::object) ? 1 : 0;
const node* const end = navigation<Editable>::end(d, container);
const node* last = navigation<Editable>::first(d, container);
const node* const end = document_data::child_end(container);
const node* last = document_data::first_child(container);
for (const node* c = document_data::after(last + value_offset); c != end; c = document_data::after(c + value_offset))
{
last = c;
+1 -7
View File
@@ -70,11 +70,5 @@
#define NLOHMANN_VIEW_LITTLE_ENDIAN 0
#endif
/// eight checks at fixed offsets 0..7
#define NLOHMANN_VIEW_REPEAT_FIRST_8(X) X(0) X(1) X(2) X(3) X(4) X(5) X(6) X(7)
/// eight checks at fixed offsets 8..15
#define NLOHMANN_VIEW_REPEAT_SECOND_8(X) X(8) X(9) X(10) X(11) X(12) X(13) X(14) X(15)
/// sixteen checks at fixed offsets 0..15
#define NLOHMANN_VIEW_REPEAT16(X) NLOHMANN_VIEW_REPEAT_FIRST_8(X) NLOHMANN_VIEW_REPEAT_SECOND_8(X)
#define NLOHMANN_VIEW_REPEAT16(X) X(0) X(1) X(2) X(3) X(4) X(5) X(6) X(7) X(8) X(9) X(10) X(11) X(12) X(13) X(14) X(15)
@@ -19,12 +19,3 @@
#undef NLOHMANN_VIEW_THROW
#undef NLOHMANN_VIEW_LITTLE_ENDIAN
#undef NLOHMANN_VIEW_REPEAT16
#undef NLOHMANN_VIEW_REPEAT_SECOND_8
#undef NLOHMANN_VIEW_REPEAT_FIRST_8
#undef NLOHMANN_VIEW_NEON
#undef NLOHMANN_VIEW_SSE2
#undef NLOHMANN_VIEW_SSSE3
#undef NLOHMANN_VIEW_SSSE3_DISPATCH
#undef NLOHMANN_VIEW_SSSE3_TARGET
#undef NLOHMANN_VIEW_VECTOR
#undef NLOHMANN_VIEW_VECTOR_UTF8
+34 -39
View File
@@ -36,32 +36,26 @@ iterative, so the nesting depth is limited by memory only, as for parse().
Without a lexer the handler records no source positions
(JSON_DIAGNOSTIC_POSITIONS).
*/
template<typename BasicJsonType, bool Editable>
template<typename BasicJsonType>
BasicJsonType materialize(const document_data& d, const node* n)
{
using string_t = typename BasicJsonType::string_t;
using sax_t = json_sax_dom_parser<BasicJsonType, iterator_input_adapter<const char*>>;
using nav = navigation<Editable>;
// the open containers: their next element (or key of the next member),
// their end, and whether they are objects; sized once from the
// document's deepest nesting (for a subtree an upper bound)
struct frame
{
const node* pos;
const node* end;
bool object;
};
BasicJsonType result;
sax_t sax(result, true);
const string_t no_token{};
std::vector<frame> open((std::max)(d.max_depth, std::size_t{1}));
// the ends of the open containers, and whether they are objects; sized
// once from the document's deepest nesting (for a subtree an upper bound)
struct open_container
{
const node* end;
bool is_object;
};
std::vector<open_container> open((std::max)(d.max_depth, std::size_t{1}));
std::size_t depth = 0;
for (;;)
{
// false positive: n comes from nav::value(), which never returns null for a valid index
// @infer-ignore NULLPTR_DEREFERENCE
switch (static_cast<value_t>(n->kind))
{
case value_t::object:
@@ -78,68 +72,69 @@ BasicJsonType materialize(const document_data& d, const node* n)
}
if (NLOHMANN_VIEW_UNLIKELY(depth == open.size()))
{
open.resize(depth * 2); // (deeper than the recorded nesting, e.g. after edits)
open.resize(depth * 2); // (deeper than the document's recorded nesting)
}
open[depth++] = frame{nav::first(d, n), nav::end(d, n), object};
open[depth++] = open_container{document_data::child_end(n), object};
n = document_data::first_child(n);
break;
}
case value_t::string:
{
string_t s(d.str(*n), n->len);
sax.string(s);
++n;
break;
}
case value_t::number_integer:
sax.number_integer(static_cast<typename BasicJsonType::number_integer_t>(static_cast<std::int64_t>(integer_bits(*n))));
++n;
break;
case value_t::number_unsigned:
sax.number_unsigned(static_cast<typename BasicJsonType::number_unsigned_t>(integer_bits(*n)));
++n;
break;
case value_t::number_float:
sax.number_float(float_value<typename BasicJsonType::number_float_t>(d, *n), no_token);
++n;
break;
case value_t::boolean:
sax.boolean((n->flags & node_flags::is_true) != 0);
++n;
break;
case value_t::null:
case value_t::binary:
case value_t::discarded:
default:
sax.null();
++n;
break;
}
// the next value: close finished containers, then read the key
for (;;)
{
if (depth == 0)
{
return result;
}
frame& f = open[depth - 1];
if (f.pos == f.end)
if (n != open[depth - 1].end)
{
if (f.object)
{
sax.end_object();
}
else
{
sax.end_array();
}
--depth;
continue;
break;
}
const node* entry = f.pos;
if (f.object)
if (open[depth - 1].is_object)
{
string_t key(d.str(*entry), entry->len);
sax.key(key);
++entry;
sax.end_object();
}
n = nav::value(entry);
f.pos = document_data::after(entry);
break;
else
{
sax.end_array();
}
--depth;
}
if (open[depth - 1].is_object)
{
// the key of the next member
string_t key(d.str(*n), n->len);
sax.key(key);
++n;
}
}
}
+4 -29
View File
@@ -33,28 +33,20 @@ constexpr std::size_t max_input_size() noexcept
struct node_flags
{
static constexpr std::uint8_t escaped = 1; ///< string payload lives in the decode arena, not the source
static constexpr std::uint8_t edited = 2; ///< string or number token lives in the edit arena (editable documents)
static constexpr std::uint8_t storage = escaped | edited; ///< mask: where a string or number token lives (index into document_data::base)
static constexpr std::uint8_t storage = 3; ///< mask: where a string or number token lives (index into document_data::base)
static constexpr std::uint8_t is_true = 4; ///< boolean value
static constexpr std::uint8_t moved = 8; ///< array/object: the elements live in a separate sequence (editable documents)
static constexpr std::uint8_t is_new = 16; ///< written by an edit: no source position
static constexpr std::uint8_t linked = 32; ///< an entry of a moved sequence links to this value (editable documents): its extent in the parsed layout no longer matters
};
/// kind of an entry of an edited sequence that stands for a value stored
/// elsewhere (the address of the value's node is kept in the len/next bytes)
constexpr std::uint8_t kind_link = 10;
/// One entry of the flat index, in document order. An object's members are
/// stored as key node followed by the value's subtree. Integers keep their
/// converted 64-bit value in the len/next bytes (the node after a scalar is
/// always the next one, and the token length follows from `extra`).
struct node
{
std::uint8_t kind; ///< value_t, or kind_link
std::uint8_t kind; ///< value_t
std::uint8_t flags; ///< node_flags
std::uint16_t extra; ///< numbers: integer digits (low byte) and fraction digits (high byte), 255 = "many"; objects: number of the object's key hash index (1-based, 0 = none); otherwise 0
std::uint32_t off; ///< source offset (string content, number token, literal, bracket); arena offset if escaped/edited; number of the element sequence if moved
std::uint16_t extra; ///< numbers: integer digits (low byte) and fraction digits (high byte), 255 = "many"; otherwise 0
std::uint32_t off; ///< source offset (string content, number token, literal, bracket); arena offset if node_flags::escaped
std::uint32_t len; ///< string: decoded bytes; float: token bytes; array/object: element count
std::uint32_t next; ///< array/object: number of nodes of the subtree (its extent in the enclosing sequence)
};
@@ -73,23 +65,6 @@ NLOHMANN_VIEW_ALWAYS_INLINE bool is_container(const node& n) noexcept
return static_cast<unsigned>(n.kind) - 1u <= 1u;
}
/// the value a link node stands for
NLOHMANN_VIEW_ALWAYS_INLINE const node* link_target(const node& n) noexcept
{
const node* t = nullptr;
std::memcpy(static_cast<void*>(&t), reinterpret_cast<const unsigned char*>(&n) + 8, sizeof(const node*)); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
return t;
}
/// let the entry n stand for the value at target (and mark the value)
inline void make_link(node& n, node* target) noexcept
{
target->flags = static_cast<std::uint8_t>(target->flags | node_flags::linked);
n = node{};
n.kind = kind_link;
std::memcpy(reinterpret_cast<unsigned char*>(&n) + 8, static_cast<const void*>(&target), sizeof(const node*)); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
}
/// the converted value of an integer node: len is its low half, next its high
/// half (on little-endian targets the two words are the value in memory)
NLOHMANN_VIEW_ALWAYS_INLINE std::uint64_t integer_bits(const node& n) noexcept
+30 -91
View File
@@ -10,7 +10,6 @@
#include <cstddef> // size_t
#include <cstdint> // int64_t, uint64_t
#include <limits> // numeric_limits
#include <string> // string
#include <type_traits> // integral_constant
@@ -26,79 +25,43 @@ namespace detail
namespace view
{
/*!
@brief locate the decimal point and the end of the mantissa of a float token
Also checks that the token is a JSON number. Tokens of the parser and of edits
always are; an image loaded with image_check::bounds can hold any bytes, which
must not reach the conversion (it expects a well-formed token).
*/
inline bool float_token_layout(const char* first, const char* last, std::size_t& dot, std::size_t& mantissa_end) noexcept
{
const auto digit = [last](const char* q)
{
return q != last && is_digit(static_cast<unsigned char>(*q));
};
const char* p = first;
p += (p != last && *p == '-') ? 1 : 0;
if (!digit(p) || (*p == '0' && digit(p + 1)))
{
return false;
}
do
{
++p;
}
while (digit(p));
dot = std::string::npos;
if (p != last && *p == '.')
{
dot = static_cast<std::size_t>(p - first);
if (!digit(++p))
{
return false;
}
do
{
++p;
}
while (digit(p));
}
mantissa_end = static_cast<std::size_t>(p - first);
if (p != last && (*p == 'e' || *p == 'E'))
{
++p;
p += (p != last && (*p == '+' || *p == '-')) ? 1 : 0;
if (!digit(p))
{
return false;
}
do
{
++p;
}
while (digit(p));
}
return p == last;
}
/*!
@brief the value of the float token of a node, as parse() converts it
Uses the lexer's conversion (detail::convert_float), so that the values are
bit-identical to parse(): float and double are converted without allocation
and independent of the locale. A token that is not a JSON number (only in a
damaged image loaded with image_check::bounds) yields 0.
and independent of the locale. The digit layout recorded while parsing locates
the decimal point and the exponent without scanning the token.
*/
template<typename FloatType>
NLOHMANN_VIEW_NOINLINE FloatType float_value(const char* first, const node& n)
{
const char* const last = first + n.len;
std::size_t dot = 0;
std::size_t mantissa_end = 0;
if (NLOHMANN_VIEW_UNLIKELY(!float_token_layout(first, last, dot, mantissa_end)))
const std::size_t neg = first[0] == '-' ? 1 : 0;
const std::size_t int_digits = n.extra & 0xFFu;
const std::size_t frac_digits = n.extra >> 8u;
std::size_t dot = std::string::npos;
std::size_t mantissa_end = n.len;
if (int_digits != 255 && frac_digits != 255)
{
return FloatType{};
dot = frac_digits != 0 ? neg + int_digits : std::string::npos;
mantissa_end = neg + int_digits + (frac_digits != 0 ? 1 + frac_digits : 0);
}
else
{
// more digits than the layout records: locate them
for (std::size_t i = 0; i < n.len; ++i)
{
if (first[i] == '.')
{
dot = i;
}
else if (first[i] == 'e' || first[i] == 'E')
{
mantissa_end = i;
break;
}
}
}
return convert_float<FloatType>(first, last, dot, mantissa_end);
}
@@ -129,20 +92,16 @@ NLOHMANN_VIEW_ALWAYS_INLINE float_significand layout_decimal(const unsigned char
}
if (p != e)
{
// [eE][+-]digits; huge exponents saturate (the parser rejected
// overflow). The token is not read beyond e, and the digits are taken
// as unsigned, so that a token that is not well-formed (a damaged
// image loaded with image_check::bounds) yields a wrong value, but no
// overflow.
// [eE][+-]digits; huge exponents saturate (the parser rejected overflow)
++p;
const bool exp_negative = p != e && *p == '-';
p += (p != e && (*p == '-' || *p == '+')) ? 1 : 0;
const bool exp_negative = *p == '-';
p += (*p == '-' || *p == '+') ? 1 : 0;
std::int64_t exp_value = 0;
for (; p != e; ++p)
{
if (exp_value < 0x10000000)
{
exp_value = (exp_value * 10) + static_cast<unsigned char>(*p - '0');
exp_value = (exp_value * 10) + (*p - '0');
}
}
q += exp_negative ? -exp_value : exp_value;
@@ -169,31 +128,11 @@ NLOHMANN_VIEW_ALWAYS_INLINE FloatType layout_float(const unsigned char* p, const
return decimal_to_float<FloatType>(layout_decimal(p, e, int_digits, frac_digits, limit));
}
/// the value of a float set by an edit: its token (the shortest round-trip
/// text, or "nan", "inf", "-inf") in the edit arena
template<typename FloatType>
NLOHMANN_VIEW_NOINLINE FloatType edited_float(const char* token, const node& n)
{
if (token[0] == 'n')
{
return std::numeric_limits<FloatType>::quiet_NaN();
}
if (token[0] == 'i' || (token[0] == '-' && token[1] == 'i'))
{
return token[0] == 'i' ? std::numeric_limits<FloatType>::infinity() : -std::numeric_limits<FloatType>::infinity();
}
return float_value<FloatType>(token, n);
}
/// the value of the float token of a node, as parse() converts it; floats and
/// doubles with at most 19 digits are converted from the digit layout
template<typename FloatType>
FloatType float_value(const document_data& d, const node& n)
{
if (NLOHMANN_VIEW_UNLIKELY((n.flags & node_flags::storage) == node_flags::edited))
{
return edited_float<FloatType>(d.str(n), n);
}
return float_value<FloatType>(d, n, std::integral_constant<bool, has_native_float_format<FloatType>::value> {});
}
@@ -1,154 +0,0 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#pragma once
#include <cstddef> // size_t
#include <cstdint> // uint32_t, uint64_t
#include <nlohmann/json.hpp>
#include <nlohmann/detail/view/document_data.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
#include <nlohmann/detail/view/node.hpp>
// Hash indexes of the keys of large objects, so that a lookup does not compare thousands
// of keys (as Boost.JSON switches from a linear search to a hash table for
// large objects). An object with document_data::index_min_members members or
// more gets an open-addressing table after parsing; its node stores the
// number of the table (1-based) in `extra`. A slot holds the offset of a key
// node from its object node (0: empty). Of duplicate keys, the first is kept,
// as for the linear search.
//
// The hash is not seeded, so keys chosen to collide could make the build
// quadratic. A key therefore sits at most index_max_displacement slots away
// from its home slot; if a key would sit further away, the table is dropped
// and the object is searched linearly (like a small one). For the same
// reason, a lookup visits at most index_max_displacement + 1 slots.
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
{
namespace view
{
/// the farthest a key may sit from its home slot (a table with at most half of
/// its slots in use gives random keys a distance of about 50 for millions of
/// members; and every member costs at most this many steps while building)
constexpr std::size_t index_max_displacement = 64;
// a table has at least 2 * len slots (len >= index_min_members), so the probe
// window of index_max_displacement + 1 slots stays far smaller than the table
// and a probe never wraps around onto its own start
static_assert(document_data::index_min_members >= 2 * index_max_displacement,
"a table must be much larger than the probe window of a lookup");
/// hash of a key: its bytes, eight at a time, in a fixed byte order
inline std::uint64_t key_hash(const char* s, std::size_t n) noexcept
{
std::uint64_t h = 0x9E3779B97F4A7C15u * (n + 1);
const auto* p = reinterpret_cast<const unsigned char*>(s); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
while (n >= 8)
{
h = (h ^ read_eight_bytes(p)) * 0xBF58476D1CE4E5B9u;
h ^= h >> 29u;
p += 8;
n -= 8;
}
std::uint64_t w = 0;
for (std::size_t i = 0; i < n; ++i)
{
w |= static_cast<std::uint64_t>(p[i]) << (8u * i);
}
h = (h ^ w) * 0x94D049BB133111EBu;
return h ^ (h >> 31u);
}
/// build the table of a large object
inline void build_object_index(document_data& d, node* obj)
{
if (d.indexes.size() >= 0xFFFFu)
{
return; // LCOV_EXCL_LINE (the number must fit `extra`; more large objects are searched linearly)
}
std::size_t cap = 16;
while (cap < 2 * static_cast<std::size_t>(obj->len))
{
cap *= 2;
}
const std::size_t start = d.index_slots.size();
d.index_slots.resize(start + cap, 0);
std::uint32_t* const table = d.index_slots.data() + start;
const std::size_t mask = cap - 1;
for (const node* k = document_data::first_child(obj), *end = document_data::child_end(obj); k != end; k = document_data::after(k + 1))
{
const char* const key = d.str(*k);
std::size_t slot = conditional_static_cast<std::size_t>(key_hash(key, k->len)) & mask;
bool duplicate = false;
std::size_t distance = 0;
while (table[slot] != 0)
{
const node* const other = obj + table[slot];
if (d.key_equals(*other, key, k->len))
{
duplicate = true; // keep the first
break;
}
if (distance == index_max_displacement)
{
// too many keys share a home region
d.index_slots.resize(start); // no table: the object is searched linearly
return;
}
++distance;
slot = (slot + 1) & mask;
}
if (!duplicate)
{
table[slot] = static_cast<std::uint32_t>(k - obj);
}
}
d.indexes.push_back(document_data::object_index{start, static_cast<std::uint32_t>(mask)});
obj->extra = static_cast<std::uint16_t>(d.indexes.size());
}
/// build the tables of the large objects the parser noted
inline void build_object_indexes(document_data& d)
{
for (const std::uint32_t i : d.large_objects)
{
build_object_index(d, d.nodes + i);
}
}
/// the key node of the first member with this key of an indexed object, or
/// nullptr
inline const node* find_indexed(const document_data& d, const node* obj, const char* key, std::size_t n) noexcept
{
const document_data::object_index& ix = d.indexes[obj->extra - 1u];
const std::uint32_t* const table = d.index_slots.data() + ix.start;
std::size_t slot = conditional_static_cast<std::size_t>(key_hash(key, n)) & ix.mask;
for (std::size_t distance = 0; distance <= index_max_displacement; ++distance)
{
const std::uint32_t s = table[slot];
if (s == 0)
{
return nullptr;
}
const node* const k = obj + s;
if (d.key_equals(*k, key, n))
{
return k;
}
slot = (slot + 1) & ix.mask;
}
return nullptr; // (no key sits further from its home slot)
}
} // namespace view
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
+31 -45
View File
@@ -16,7 +16,6 @@
#include <nlohmann/json.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
#include <nlohmann/detail/view/simd.hpp>
// Scanning primitives of the view's parser. The unrolled checks at fixed
// offsets follow yyjson (https://github.com/ibireme/yyjson, MIT license): the
@@ -63,43 +62,20 @@ NLOHMANN_VIEW_ALWAYS_INLINE std::uint16_t load16(const unsigned char* p) noexcep
/// Advance over plain string bytes and well-formed UTF-8. Stops at a quote,
/// a backslash, a control character, ill-formed UTF-8, or the end. The first
/// bytes are checked one by one, so that the position advances by constants
/// in predicted branches: 16 for keys, whose lengths repeat from record to
/// record, and 8 for string values (Value) where a vector loop follows, as
/// their lengths vary more. Longer runs continue 16 bytes at a time with NEON
/// or SSE2, else eight bytes at a time. With SSE2, the run is checked 16 bytes
/// at a time from its first byte instead: on x86-64, one compare that finds
/// the end of most keys and short values is faster than a branch per byte (on
/// AArch64, where a NEON mask costs more and branches predict well, slower).
template<bool Value = false>
/// 16 bytes are checked one by one, so that the position advances by
/// constants in predicted branches (most strings are short); longer runs
/// continue eight bytes at a time.
NLOHMANN_VIEW_ALWAYS_INLINE const unsigned char* scan_string_run(const unsigned char* p, const unsigned char* e) noexcept
{
const std::uint8_t* plain = string_plain();
for (;;)
{
#if NLOHMANN_VIEW_SSE2
p = vector_plain_run(p, e);
#else
if (e - p >= 16)
{
#define NLOHMANN_VIEW_STEP(i) if (NLOHMANN_VIEW_LIKELY(plain[p[i]] != 0)) {} else { p += (i); goto stop; }
NLOHMANN_VIEW_REPEAT_FIRST_8(NLOHMANN_VIEW_STEP)
#if NLOHMANN_VIEW_VECTOR
if (!Value) // (a plain `!Value || !NLOHMANN_VIEW_VECTOR` is a constant condition for MSVC, C4127)
#endif
{
NLOHMANN_VIEW_REPEAT_SECOND_8(NLOHMANN_VIEW_STEP)
p += 8;
}
NLOHMANN_VIEW_REPEAT16(NLOHMANN_VIEW_STEP)
#undef NLOHMANN_VIEW_STEP
p += 8;
#if NLOHMANN_VIEW_VECTOR
p = vector_plain_run(p, e);
if (p != e && plain[*p] == 0)
{
goto stop;
}
#else
p += 16;
while (e - p >= 8)
{
const std::uint64_t special = swar_string_special(read_eight_bytes(p));
@@ -110,10 +86,8 @@ NLOHMANN_VIEW_ALWAYS_INLINE const unsigned char* scan_string_run(const unsigned
}
p += 8;
}
#endif
continue;
}
#endif
while (p != e && plain[*p] != 0)
{
++p;
@@ -122,23 +96,11 @@ NLOHMANN_VIEW_ALWAYS_INLINE const unsigned char* scan_string_run(const unsigned
{
return p;
}
#if !NLOHMANN_VIEW_SSE2
stop:
#endif
if (*p < 0x80)
{
return p; // quote, backslash, or control character
}
#if NLOHMANN_VIEW_VECTOR_UTF8
#if NLOHMANN_VIEW_SSSE3_DISPATCH
if (NLOHMANN_VIEW_LIKELY(cpu_has_ssse3()))
#endif
{
// non-ASCII: the vector check, out of line
return scan_string_vector(p, e, plain);
}
#endif
#if !NLOHMANN_VIEW_VECTOR_UTF8 || NLOHMANN_VIEW_SSSE3_DISPATCH
// non-ASCII: a run of well-formed sequences (the library's check, so
// that exactly what json::parse accepts is accepted)
do
@@ -151,7 +113,6 @@ stop:
p += n;
}
while (p != e && *p >= 0x80);
#endif
}
}
@@ -175,7 +136,32 @@ NLOHMANN_VIEW_ALWAYS_INLINE const unsigned char* skip_digits(const unsigned char
/// powers of ten up to 10^19 as integers
inline std::uint64_t int_pow10(unsigned k) noexcept
{
return ::nlohmann::detail::dtoa_impl::powers_of_ten()[k];
static const std::array<std::uint64_t, 20> table =
{
{
1u,
10u,
100u,
1000u,
10000u,
100000u,
1000000u,
10000000u,
100000000u,
1000000000u,
10000000000u,
100000000000u,
1000000000000u,
10000000000000u,
100000000000000u,
1000000000000000u,
10000000000000000u,
100000000000000000u,
1000000000000000000u,
10000000000000000000u
}
};
return table[k];
}
/// value of 0 < k < 8 digits at p in one step if [p, p + 8) lies below
+19 -563
View File
@@ -31,11 +31,7 @@ namespace view
{
/// append-only output buffer: writes through a raw pointer into a string that
/// is resized ahead, and trimmed by finish(). The estimate is reserved, and the
/// string grows in steps of 64 KiB within it: resize() fills the new bytes
/// with zeros (before C++23, a string cannot grow without), and a small step
/// is filled while the writer is about to use it, in the cache, instead of
/// filling the whole estimate in memory first.
/// is resized ahead, and trimmed by finish()
template<typename StringType>
class output_buffer
{
@@ -87,49 +83,17 @@ class output_buffer
m_pos += n;
}
/// the write position and the end of the writable space, for a writer
/// that keeps the position in a local variable (set_cursor() hands it back)
char* cursor() const noexcept
{
return m_pos;
}
char* limit() const noexcept
{
return m_end;
}
void set_cursor(char* p) noexcept
{
m_pos = p;
}
private:
/// the size of a growth step (a function: std::min() takes a reference,
/// which a static constexpr member does not have before C++17)
static constexpr std::size_t step() noexcept
{
return 0x10000;
}
static StringType& sized(StringType& out, std::size_t estimate)
{
out.reserve(estimate);
out.resize((std::min)((std::max)(estimate, static_cast<std::size_t>(64)), step()));
out.resize((std::max)(estimate, static_cast<std::size_t>(64)));
return out;
}
NLOHMANN_VIEW_NOINLINE void grow(std::size_t n)
{
const auto used = static_cast<std::size_t>(m_pos - m_out.data());
// (a step does not go beyond the reserved estimate, so that a good
// estimate is never copied to a larger allocation)
const std::size_t size = (std::max)((std::min)(m_out.size() + step(), m_out.capacity()), used + n + 256);
if (size > m_out.capacity())
{
m_out.reserve((std::max)(m_out.capacity() * 2, size));
}
m_out.resize(size);
m_out.resize((std::max)(m_out.size() * 2, used + n + 256));
m_pos = &m_out[0] + used;
m_end = &m_out[0] + m_out.size();
}
@@ -139,117 +103,6 @@ class output_buffer
char* m_end;
};
/// The length of the run at s that dump() writes unchanged without
/// ensure_ascii: all bytes but quotes, backslashes, and control characters.
/// Unlike detail::string_bulk_run(), non-ASCII bytes are not validated: the
/// strings of a document are valid UTF-8 (a damaged image loaded with
/// image_check::bounds can have others, which are then written unchanged).
inline std::size_t plain_output_run(const unsigned char* s, std::size_t n) noexcept
{
constexpr std::uint64_t ones = 0x0101010101010101ull;
constexpr std::uint64_t high = 0x8080808080808080ull;
std::size_t i = 0;
for (; i + 8 <= n; i += 8)
{
const std::uint64_t v = read_eight_bytes(s + i);
const std::uint64_t q = v ^ 0x2222222222222222ull; // '"'
const std::uint64_t b = v ^ 0x5C5C5C5C5C5C5C5Cull; // '\\'
const std::uint64_t stop = (((q - ones) & ~q) | ((b - ones) & ~b) | ((v - 0x2020202020202020ull) & ~v)) & high;
if (stop != 0)
{
// the lowest flagged byte is the first stop: borrows only flag bytes above a true one
return i + (static_cast<std::size_t>(count_trailing_zeros(stop)) / 8);
}
}
for (; i < n; ++i)
{
if (s[i] == '"' || s[i] == '\\' || s[i] < 0x20)
{
return i;
}
}
return n;
}
/// A stack that starts in a buffer of the caller (a local array) and moves to
/// the heap (a vector of the caller) only when that is full, so that dumps of
/// shallow documents need no allocation. The top is a pointer, as in
/// std::vector. grow() is static and out of line: it gets the vector and the
/// used range and returns the new storage, so the address of the stack never
/// reaches a call. Otherwise the compiler would have to keep the three
/// pointers in memory and load and store them on every push_back() and
/// pop_back(), not just when growing.
template<typename T>
class small_stack
{
public:
small_stack(T* buffer, std::size_t capacity, std::vector<T>& heap) noexcept
: m_begin(buffer), m_top(buffer), m_end(buffer + capacity), m_heap(heap)
{}
small_stack(const small_stack&) = delete;
small_stack(small_stack&&) = delete;
small_stack& operator=(const small_stack&) = delete;
small_stack& operator=(small_stack&&) = delete;
~small_stack() = default;
NLOHMANN_VIEW_ALWAYS_INLINE void push_back(const T& x)
{
if (NLOHMANN_VIEW_UNLIKELY(m_top == m_end))
{
const std::size_t used = size();
const std::size_t capacity = 2 * static_cast<std::size_t>(m_end - m_begin);
m_begin = grow(m_heap, m_begin, used, capacity);
m_top = m_begin + used;
m_end = m_begin + capacity;
}
*m_top++ = x;
}
NLOHMANN_VIEW_ALWAYS_INLINE T& back() noexcept
{
return m_top[-1];
}
NLOHMANN_VIEW_ALWAYS_INLINE void pop_back() noexcept
{
--m_top;
}
NLOHMANN_VIEW_ALWAYS_INLINE bool empty() const noexcept
{
return m_top == m_begin;
}
NLOHMANN_VIEW_ALWAYS_INLINE std::size_t size() const noexcept
{
return static_cast<std::size_t>(m_top - m_begin);
}
private:
/// the used entries moved to heap storage of the given capacity
NLOHMANN_VIEW_NOINLINE static T* grow(std::vector<T>& heap, const T* begin, std::size_t used, std::size_t capacity)
{
std::vector<T> bigger(capacity);
std::copy(begin, begin + used, bigger.begin());
heap.swap(bigger);
return heap.data();
}
T* m_begin;
T* m_top;
T* m_end;
std::vector<T>& m_heap;
};
/// a string literal at w, without its terminating NUL (after room())
/// @return a pointer past it
template<std::size_t N>
NLOHMANN_VIEW_ALWAYS_INLINE char* put_literal(char* w, const char (&s)[N]) noexcept // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
{
std::memcpy(w, s, N - 1);
return w + N - 1;
}
/// how the view's dump() writes a value
struct dump_style
{
@@ -270,14 +123,11 @@ the library's conversion; integers are copied from the source, where they
are canonical (except "-0", which parse() reads as 0). The walk is
iterative, so the nesting depth is limited by memory only.
*/
template<typename BasicJsonType, bool Editable>
template<typename BasicJsonType>
class view_serializer
{
using nav = navigation<Editable>;
using string_t = typename BasicJsonType::string_t;
using number_float_t = typename BasicJsonType::number_float_t;
/// doubles that are IEEE 754 binary64 are written from their bits (Zmij)
using fast_double = std::integral_constant < bool, std::is_same<number_float_t, double>::value && ::nlohmann::detail::dtoa_impl::is_binary64<double>::value >;
public:
view_serializer(const document_data& d, string_t& out, std::size_t estimate, const dump_style& style)
@@ -286,18 +136,6 @@ class view_serializer
void dump(const node* root)
{
if (!m_style.pretty && !m_style.ensure_ascii)
{
if (m_style.source_numbers)
{
dump_compact<true>(root);
}
else
{
dump_compact<false>(root);
}
return;
}
struct frame
{
const node* pos; ///< next element, or key of the next member
@@ -305,9 +143,7 @@ class view_serializer
bool object;
bool first; ///< nothing written yet
};
std::array<frame, 32> buffer; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
std::vector<frame> heap;
small_stack<frame> stack(buffer.data(), buffer.size(), heap);
std::vector<frame> stack;
const node* n = root;
for (;;)
{
@@ -322,7 +158,7 @@ class view_serializer
else
{
m_out.put(object ? '{' : '[');
stack.push_back(frame{nav::first(m_doc, n), nav::end(m_doc, n), object, true});
stack.push_back(frame{document_data::first_child(n), document_data::child_end(n), object, true});
}
}
else
@@ -364,307 +200,19 @@ class view_serializer
{
m_out.put(':');
}
n = nav::value(f.pos + 1);
f.pos = document_data::after(f.pos + 1);
n = f.pos + 1;
}
else
{
n = nav::value(f.pos);
f.pos = document_data::after(f.pos);
n = f.pos;
}
f.pos = document_data::after(n);
break;
}
}
}
private:
/// a string that is not a plain string of the source (decoded, or written
/// by an edit), without ensure_ascii: runs without characters to escape
/// are copied
NLOHMANN_VIEW_NOINLINE void write_decoded(const node& n)
{
const auto* const s = reinterpret_cast<const unsigned char*>(m_doc.str(n)); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
m_out.put('"');
for (std::size_t i = 0; i < n.len;)
{
const std::size_t run = plain_output_run(s + i, n.len - i);
if (run != 0)
{
m_out.put(reinterpret_cast<const char*>(s + i), run); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
i += run;
continue;
}
write_codepoint<false>(s[i], s + i, 1); // a quote, a backslash, or a control character
++i;
}
m_out.put('"');
}
/// the copies of dump_compact() that are not fixed-size moves (long
/// strings, or near the end of the source); out of line, so that the
/// compiler does not merge the fixed-size moves into this call
NLOHMANN_VIEW_NOINLINE static void copy_long(char* to, const char* from, std::size_t n) noexcept
{
std::memcpy(to, from, n);
}
/// a compile-time option as a runtime condition: testing the template
/// argument directly makes a condition like `Editable && c` constant when
/// the option is off, which MSVC reports as C4127
static NLOHMANN_VIEW_ALWAYS_INLINE bool enabled(bool option) noexcept
{
return option;
}
/*!
@brief the compact output without ensure_ascii (the default dump())
The same walk as dump(), with the write position in a local variable
(stores through char pointers would otherwise force a reload of the
buffer's members after each one), and with strings and number tokens of
the source copied by fixed-size moves of 32 bytes where the source has
that many bytes left, instead of a library call per token. The buffer
keeps 64 bytes of slack for the overshoot.
*/
template<bool SourceNumbers>
void dump_compact(const node* root)
{
struct frame
{
const node* pos; ///< (editable documents) next element, or key of the next member
const node* end;
bool object;
};
std::array<frame, 32> buffer; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
std::vector<frame> heap;
small_stack<frame> stack(buffer.data(), buffer.size(), heap);
const char* const src = m_doc.src;
const char* const src_end = src + m_doc.size;
char* w = m_out.cursor();
char* lim = m_out.limit();
// room for n bytes and the slack
const auto room = [&](std::size_t n)
{
if (NLOHMANN_VIEW_UNLIKELY(static_cast<std::size_t>(lim - w) < n + 64))
{
m_out.set_cursor(w);
m_out.reserve(n + 64);
w = m_out.cursor();
lim = m_out.limit();
}
};
// copy n bytes of the source (after room(n))
const auto copy = [&](const char* from, std::size_t n)
{
if (n <= 32 && static_cast<std::size_t>(src_end - from) >= 32)
{
std::memcpy(w, from, 32);
}
else if (n <= 256 && static_cast<std::size_t>(src_end - from) >= n + 32)
{
for (std::size_t i = 0; i < n; i += 32)
{
std::memcpy(w + i, from + i, 32);
}
}
else
{
copy_long(w, from, n);
}
w += n;
};
// a string that is not a plain string of the source (out of line, so
// that the cursor stays in a register here)
const auto escaped = [&](const node & n)
{
m_out.set_cursor(w);
write_decoded(n);
w = m_out.cursor();
lim = m_out.limit();
};
// Read-only documents: the elements of a container follow it in the
// node array, so the walk goes through the array in order, and a
// frame only needs the end of its container. Editable documents: the
// elements of a moved container live elsewhere, so a frame keeps the
// position of the next element (see navigation).
// The innermost open container is kept in registers (cur; end ==
// nullptr: none), the stack holds the ones around it.
frame cur{nullptr, nullptr, false};
const node* n = root;
for (;;)
{
// write the value at n (read-only documents: and advance n)
bool opened = false;
// false positive: n comes from nav::value(), which never returns null for a valid index
// @infer-ignore NULLPTR_DEREFERENCE
switch (static_cast<value_t>(n->kind))
{
case value_t::string:
if ((n->flags & node_flags::storage) == 0)
{
room(n->len + 2);
*w++ = '"';
copy(src + n->off, n->len);
*w++ = '"';
}
else
{
escaped(*n);
}
break;
case value_t::number_integer:
case value_t::number_unsigned:
{
const std::uint32_t len = number_length(*n);
room(len);
if (enabled(Editable) && (n->flags & node_flags::storage) != 0)
{
copy_long(w, m_doc.str(*n), len); // a canonical token written by an edit
w += len;
break;
}
const char* const token = src + n->off;
if (!enabled(SourceNumbers) && NLOHMANN_VIEW_UNLIKELY(len == 2 && token[0] == '-' && token[1] == '0'))
{
*w++ = '0'; // parse() reads -0 as the integer 0
}
else
{
copy(token, len);
}
break;
}
case value_t::number_float:
if (enabled(SourceNumbers) && (n->flags & node_flags::storage) != node_flags::edited)
{
room(n->len);
copy(src + n->off, n->len);
}
else if (enabled(fast_double::value))
{
room(64);
w = write_double_at(w, *n);
}
else
{
m_out.set_cursor(w);
write_float_node(*n);
w = m_out.cursor();
lim = m_out.limit();
}
break;
case value_t::boolean:
room(8);
if ((n->flags & node_flags::is_true) != 0)
{
w = put_literal(w, "true");
}
else
{
w = put_literal(w, "false");
}
break;
case value_t::object:
case value_t::array:
{
const bool object = n->kind == static_cast<std::uint8_t>(value_t::object);
room(8);
if (n->len == 0)
{
w = object ? put_literal(w, "{}") : put_literal(w, "[]");
}
else
{
*w++ = object ? '{' : '[';
stack.push_back(cur);
if (Editable)
{
cur = frame{nav::first(m_doc, n), nav::end(m_doc, n), object};
}
else
{
cur = frame{nullptr, n + n->next, object};
}
opened = true;
}
break;
}
case value_t::null:
room(8);
w = put_literal(w, "null");
break;
case value_t::binary: // LCOV_EXCL_LINE (not in a document)
case value_t::discarded: // LCOV_EXCL_LINE
default: // LCOV_EXCL_LINE
break; // LCOV_EXCL_LINE
}
if (!Editable)
{
++n; // the next node: the first element of an opened container, or the node after a scalar
}
// go to the next value: close finished containers, then separate
// (a container just opened has an element)
if (!opened)
{
for (;;)
{
if (cur.end == nullptr)
{
m_out.set_cursor(w);
m_out.finish();
return;
}
if ((Editable ? cur.pos : n) != cur.end)
{
break;
}
room(1);
*w++ = cur.object ? '}' : ']';
cur = stack.back();
stack.pop_back();
}
room(1);
*w++ = ',';
}
const node* const at = Editable ? cur.pos : n;
if (cur.object)
{
const node& key = *at;
if ((key.flags & node_flags::storage) == 0)
{
room(key.len + 3);
*w++ = '"';
copy(src + key.off, key.len);
w[0] = '"';
w[1] = ':';
w += 2;
}
else
{
escaped(key);
room(1);
*w++ = ':';
}
if (Editable)
{
n = nav::value(at + 1);
cur.pos = document_data::after(at + 1);
}
else
{
++n;
}
}
else if (Editable)
{
n = nav::value(at);
cur.pos = document_data::after(at);
}
}
}
void newline(std::size_t level)
{
if (m_style.pretty)
@@ -710,13 +258,13 @@ class view_serializer
break;
}
case value_t::number_float:
if (m_style.source_numbers && (n.flags & node_flags::storage) != node_flags::edited)
if (m_style.source_numbers)
{
m_out.put(m_doc.str(n), n.len); // (a float set by an edit is written as with shortest)
m_out.put(m_doc.str(n), n.len);
}
else
{
write_float_node(n);
write_float(float_value<number_float_t>(m_doc, n));
}
break;
case value_t::object: // LCOV_EXCL_LINE (containers are written by dump())
@@ -728,88 +276,6 @@ class view_serializer
}
}
/// a float node as dump() writes it
void write_float_node(const node& n)
{
write_float_node(n, fast_double {});
}
void write_float_node(const node& n, std::false_type /*other*/)
{
write_float(float_value<number_float_t>(m_doc, n));
}
void write_float_node(const node& n, std::true_type /*double*/)
{
m_out.reserve(64);
m_out.set_cursor(write_double_at(m_out.cursor(), n));
}
/*!
@brief (doubles) the float at n as dump() writes it, at w (64 bytes of room)
A token of at most 15 significant digits is written from its digits,
without a conversion: two decimals of at most 15 digits are farther
apart than the rounding interval of a (normal) double (the argument
behind DBL_DIG), so the token's digits are the shortest ones of its
double, which the library's conversion writes (Zmij). Other tokens are
converted from the digits already read.
*/
char* write_double_at(char* w, const node& n)
{
const unsigned int_digits = n.extra & 0xFFu;
const unsigned frac_digits = n.extra >> 8u;
if ((n.flags & node_flags::storage) != node_flags::edited && int_digits + frac_digits <= 19)
{
const auto* const first = reinterpret_cast<const unsigned char*>(m_doc.src + n.off); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
const float_significand d = layout_decimal(first, first + n.len, int_digits, frac_digits, reinterpret_cast<const unsigned char*>(m_doc.src + m_doc.size)); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
// (the exponent keeps the value far from subnormals and overflow)
if (d.w != 0 && d.w < 1000000000000000u && d.exponent >= -290 && d.exponent <= 290)
{
*w = '-';
w += d.negative ? 1 : 0;
// (without leading zeros, all digits of the token count; the
// check also keeps an image that was only checked for bounds,
// whose token may not be made of digits, from the counted
// overload)
const auto& powers = ::nlohmann::detail::dtoa_impl::powers_of_ten();
const unsigned count = int_digits + frac_digits;
return count - 1u < 15u && d.w >= powers[count - 1u] && d.w < powers[count]
? ::nlohmann::detail::dtoa_impl::write_short_decimal(w, d.w, static_cast<int>(count), static_cast<int>(d.exponent))
: ::nlohmann::detail::dtoa_impl::write_short_decimal(w, d.w, static_cast<int>(d.exponent));
}
return write_double_value_at(w, decimal_to_float<double>(d)); // (without reading the token again)
}
return write_double_value_at(w, static_cast<double>(float_value<number_float_t>(m_doc, n)));
}
/// n bytes of text at w
static char* write_text_at(char* w, const char* text, std::size_t n) noexcept
{
std::memcpy(w, text, n);
return w + n;
}
/// a double as dump() writes it, at w (64 bytes of room)
static char* write_double_value_at(char* w, double x)
{
// (from the bits: without the checks of to_chars())
std::uint64_t bits = 0;
std::memcpy(&bits, &x, sizeof(bits));
if (NLOHMANN_VIEW_UNLIKELY((bits & 0x7FF0000000000000u) == 0x7FF0000000000000u))
{
return write_text_at(w, "null", 4);
}
*w = '-';
w += bits >> 63u;
bits &= ~(std::uint64_t{1} << 63u);
if (bits == 0)
{
return write_text_at(w, "0.0", 3);
}
return ::nlohmann::detail::dtoa_impl::write_shortest(w, ::nlohmann::detail::zmij::to_shortest(bits));
}
/// as serializer::dump_float()
void write_float(number_float_t x)
{
@@ -843,9 +309,9 @@ class view_serializer
{
const char* const s = m_doc.str(n);
m_out.put('"');
if ((n.flags & node_flags::storage) == 0 && !m_style.ensure_ascii)
if ((n.flags & node_flags::escaped) == 0 && !m_style.ensure_ascii)
{
// a string of the source without escape sequences has nothing to escape
// a string without escape sequences has nothing to escape
m_out.put(s, n.len);
}
else if (m_style.ensure_ascii)
@@ -859,9 +325,7 @@ class view_serializer
m_out.put('"');
}
/// as serializer::dump_escaped(); strings of a document are valid UTF-8,
/// except in a damaged image loaded with image_check::bounds, for which
/// this throws what basic_json::dump() throws for the string
/// as serializer::dump_escaped() for valid UTF-8 (the view has no other)
template<bool EnsureAscii>
void write_escaped(const unsigned char* s, std::size_t n)
{
@@ -885,13 +349,12 @@ class view_serializer
}
std::uint32_t codepoint = s[i];
std::size_t len = 1;
if (codepoint >= 0x80)
if (codepoint >= 0xC0)
{
len = validate_one_utf8(s + i, n - i);
if (NLOHMANN_VIEW_UNLIKELY(len == 0))
len = 2;
if (codepoint >= 0xE0)
{
invalid_utf8(s, n);
return;
len = codepoint >= 0xF0 ? 4 : 3;
}
codepoint &= 0xFFu >> (len + 1);
for (std::size_t k = 1; k < len; ++k)
@@ -904,13 +367,6 @@ class view_serializer
}
}
/// throw what basic_json::dump() throws for a string that is not valid UTF-8
NLOHMANN_VIEW_NOINLINE static void invalid_utf8(const unsigned char* s, std::size_t n)
{
const string_t dumped = BasicJsonType(string_t(reinterpret_cast<const char*>(s), n)).dump(); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
static_cast<void>(dumped);
}
template<bool EnsureAscii>
void write_codepoint(std::uint32_t codepoint, const unsigned char* bytes, std::size_t len)
{
-339
View File
@@ -1,339 +0,0 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-FileCopyrightText: 2018-2025 The simdjson authors <https://github.com/simdjson/simdjson>
// SPDX-License-Identifier: MIT
#pragma once
#include <array> // array
#include <atomic> // atomic
#include <cstddef> // size_t
#include <cstdint> // uint8_t, uint64_t
#include <nlohmann/json.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
// Vector code for long runs of string bytes. NEON (AArch64) and SSE2 (x86-64)
// belong to the baseline instruction sets and are used by default. The vector
// UTF-8 check needs NEON or SSSE3. SSSE3 is not part of x86-64, and the code
// must not depend on the flags of a translation unit (two translation units
// with different flags would have different definitions of the same inline
// functions): the check is compiled for SSSE3 with a function attribute and
// used where the CPU has SSSE3 (all x86-64 CPUs since about 2011), else the
// portable check. JSON_VIEW_USE_SSSE3 skips the CPU check (for code compiled
// for SSSE3 anyway); JSON_VIEW_NO_SIMD selects the portable code.
// NLOHMANN_VIEW_NO_TARGET_ATTRIBUTE (set by the GCC module interface, where GCC
// ignores the target attribute and then rejects the SSSE3 intrinsics) keeps the
// check portable without disabling the SSE2 code.
#if !defined(JSON_VIEW_NO_SIMD) && defined(__aarch64__) && (defined(__GNUC__) || defined(__clang__)) && NLOHMANN_VIEW_LITTLE_ENDIAN
#include <arm_neon.h>
#define NLOHMANN_VIEW_NEON 1
#else
#define NLOHMANN_VIEW_NEON 0
#endif
// (x86 only: other targets can define __SSE2__ as well, e.g., WebAssembly with -msse2, but have no <cpuid.h>)
#if !defined(JSON_VIEW_NO_SIMD) && !NLOHMANN_VIEW_NEON && (defined(__x86_64__) || defined(__i386__) || defined(_M_X64) || defined(_M_IX86)) && (defined(__SSE2__) || defined(_M_X64) || (defined(_M_IX86_FP) && _M_IX86_FP >= 2))
#include <emmintrin.h>
#define NLOHMANN_VIEW_SSE2 1
#else
#define NLOHMANN_VIEW_SSE2 0
#endif
#if NLOHMANN_VIEW_SSE2 && defined(JSON_VIEW_USE_SSSE3)
#include <tmmintrin.h>
#define NLOHMANN_VIEW_SSSE3 1 // NOLINT(cppcoreguidelines-macro-to-enum,modernize-macro-to-enum)
#else
#define NLOHMANN_VIEW_SSSE3 0 // NOLINT(cppcoreguidelines-macro-to-enum,modernize-macro-to-enum)
#endif
#if NLOHMANN_VIEW_SSE2 && !NLOHMANN_VIEW_SSSE3 && !defined(NLOHMANN_VIEW_NO_TARGET_ATTRIBUTE) && ((defined(__clang__) && __clang_major__ >= 4) || (defined(__GNUC__) && !defined(__clang__) && (__GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 9))))
// (GCC before 4.9 has no SSSE3 intrinsics without -mssse3)
#include <cpuid.h>
#include <tmmintrin.h>
#define NLOHMANN_VIEW_SSSE3_DISPATCH 1 // NOLINT(cppcoreguidelines-macro-to-enum,modernize-macro-to-enum)
#define NLOHMANN_VIEW_SSSE3_TARGET __attribute__((target("ssse3")))
#elif NLOHMANN_VIEW_SSE2 && !NLOHMANN_VIEW_SSSE3 && defined(_MSC_VER)
// (MSVC compiles intrinsics of any instruction set)
#include <intrin.h>
#include <tmmintrin.h>
#define NLOHMANN_VIEW_SSSE3_DISPATCH 1 // NOLINT(cppcoreguidelines-macro-to-enum,modernize-macro-to-enum)
#define NLOHMANN_VIEW_SSSE3_TARGET
#else
#define NLOHMANN_VIEW_SSSE3_DISPATCH 0 // NOLINT(cppcoreguidelines-macro-to-enum,modernize-macro-to-enum)
#define NLOHMANN_VIEW_SSSE3_TARGET
#endif
#define NLOHMANN_VIEW_VECTOR (NLOHMANN_VIEW_NEON || NLOHMANN_VIEW_SSE2)
#define NLOHMANN_VIEW_VECTOR_UTF8 (NLOHMANN_VIEW_NEON || NLOHMANN_VIEW_SSSE3 || NLOHMANN_VIEW_SSSE3_DISPATCH)
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
{
namespace view
{
#if NLOHMANN_VIEW_VECTOR
/*!
@brief the first byte of a string run that is a quote, a backslash, a control
character, or not ASCII, 16 bytes per step
Stops at such a byte, or where fewer than 16 bytes are left (the caller tells
the two apart). A signed compare with 0x20 finds control characters and
non-ASCII bytes at once.
*/
NLOHMANN_VIEW_ALWAYS_INLINE const unsigned char* vector_plain_run(const unsigned char* p, const unsigned char* e) noexcept
{
while (e - p >= 16)
{
#if NLOHMANN_VIEW_NEON
const uint8x16_t in = vld1q_u8(p);
const uint8x16_t special = vorrq_u8(vorrq_u8(vceqq_u8(in, vdupq_n_u8('"')), vceqq_u8(in, vdupq_n_u8('\\'))),
vcltq_s8(vreinterpretq_s8_u8(in), vdupq_n_s8(0x20)));
// one nibble per byte (the usual NEON replacement of x86's movemask, see
// D. Kutenin, "Porting x86 vector bitmask optimizations to Arm NEON", 2022)
const std::uint64_t bits = vget_lane_u64(vreinterpret_u64_u8(vshrn_n_u16(vreinterpretq_u16_u8(special), 4)), 0);
if (bits != 0)
{
return p + (count_trailing_zeros(bits) >> 2u);
}
#else
const __m128i in = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(p))); // NOLINT(bugprone-casting-through-void)
const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('"')), _mm_cmpeq_epi8(in, _mm_set1_epi8('\\'))),
_mm_cmplt_epi8(in, _mm_set1_epi8(0x20)));
const auto bits = static_cast<std::uint64_t>(static_cast<unsigned>(_mm_movemask_epi8(special)));
if (bits != 0)
{
return p + count_trailing_zeros(bits);
}
#endif
p += 16;
}
return p;
}
#endif
#if NLOHMANN_VIEW_SSSE3_DISPATCH
/// whether the CPU has SSSE3 (CPUID leaf 1, ECX bit 9)
inline bool cpu_ssse3() noexcept
{
#if defined(_MSC_VER) && !defined(__clang__)
std::array<int, 4> regs {{}};
__cpuid(regs.data(), 1);
return (static_cast<unsigned>(regs[2]) & (1u << 9u)) != 0;
#else
unsigned eax = 0;
unsigned ebx = 0;
unsigned ecx = 0;
unsigned edx = 0;
return __get_cpuid(1, &eax, &ebx, &ecx, &edx) != 0 && (ecx & (1u << 9u)) != 0;
#endif
}
/// whether the CPU has SSSE3, asked once: the answer is kept in an atomic
/// that is initialized at compile time, so that neither a guard of a local
/// static nor a global constructor is needed (threads that ask at the same
/// time all store the same answer)
NLOHMANN_VIEW_ALWAYS_INLINE bool cpu_has_ssse3() noexcept
{
static std::atomic<int> known{0}; // 0: not asked yet, 1: no, 2: yes
int state = known.load(std::memory_order_relaxed);
if (NLOHMANN_VIEW_UNLIKELY(state == 0))
{
state = cpu_ssse3() ? 2 : 1;
known.store(state, std::memory_order_relaxed);
}
return state == 2;
}
#endif
#if NLOHMANN_VIEW_VECTOR_UTF8
/// Tables of the UTF-8 check of J. Keiser and D. Lemire, "Validating UTF-8 In
/// Less Than One Instruction Per Byte" (2021), as in simdjson ("lookup4"): each
/// maps a nibble (high and low nibble of the previous byte, high nibble of the
/// current byte) to the errors it allows; a byte pair is ill-formed if all
/// three have an error bit in common.
template<typename Dummy = void>
struct utf8_lookup4
{
static constexpr std::uint8_t too_short = 1u << 0u, too_long = 1u << 1u, overlong_3 = 1u << 2u, too_large = 1u << 3u;
static constexpr std::uint8_t surrogate = 1u << 4u, overlong_2 = 1u << 5u, too_large_1000 = 1u << 6u, overlong_4 = 1u << 6u;
static constexpr std::uint8_t two_conts = 1u << 7u, carry = too_short | too_long | two_conts;
static const std::array<std::uint8_t, 16> byte_1_high;
static const std::array<std::uint8_t, 16> byte_1_low;
static const std::array<std::uint8_t, 16> byte_2_high;
};
template<typename Dummy>
const std::array<std::uint8_t, 16> utf8_lookup4<Dummy>::byte_1_high =
{
{
too_long, too_long, too_long, too_long, too_long, too_long, too_long, too_long,
two_conts, two_conts, two_conts, two_conts,
too_short | overlong_2, too_short, too_short | overlong_3 | surrogate, too_short | too_large | too_large_1000 | overlong_4
}
};
template<typename Dummy>
const std::array<std::uint8_t, 16> utf8_lookup4<Dummy>::byte_1_low =
{
{
carry | overlong_3 | overlong_2 | overlong_4, carry | overlong_2, carry, carry,
carry | too_large, carry | too_large | too_large_1000, carry | too_large | too_large_1000, carry | too_large | too_large_1000,
carry | too_large | too_large_1000, carry | too_large | too_large_1000, carry | too_large | too_large_1000, carry | too_large | too_large_1000,
carry | too_large | too_large_1000, carry | too_large | too_large_1000 | surrogate, carry | too_large | too_large_1000, carry | too_large | too_large_1000
}
};
template<typename Dummy>
const std::array<std::uint8_t, 16> utf8_lookup4<Dummy>::byte_2_high =
{
{
too_short, too_short, too_short, too_short, too_short, too_short, too_short, too_short,
static_cast<std::uint8_t>(too_long | overlong_2 | two_conts | overlong_3 | too_large_1000 | overlong_4),
static_cast<std::uint8_t>(too_long | overlong_2 | two_conts | overlong_3 | too_large),
static_cast<std::uint8_t>(too_long | overlong_2 | two_conts | surrogate | too_large),
static_cast<std::uint8_t>(too_long | overlong_2 | two_conts | surrogate | too_large),
too_short, too_short, too_short, too_short
}
};
/// the end of scan_string_vector from block, where the vector loop stopped
/// (ill-formed UTF-8, or fewer than 16 bytes left): one byte or sequence at a
/// time, from the start of a sequence that crosses into the block
inline const unsigned char* scan_string_finish(const unsigned char* p, const unsigned char* block, const unsigned char* e, const std::uint8_t* plain) noexcept
{
for (int i = 1; i <= 3 && block - i >= p; ++i)
{
const unsigned char c = block[-i];
if (c < 0x80)
{
break;
}
if (c >= 0xC0)
{
const int len = 2 + static_cast<int>(c >= 0xE0) + static_cast<int>(c >= 0xF0);
if (len > i)
{
block -= i;
}
break;
}
}
for (p = block; p != e;)
{
if (*p < 0x80)
{
if (plain[*p] == 0)
{
return p;
}
++p;
continue;
}
const std::size_t n = validate_one_utf8(p, static_cast<std::size_t>(e - p));
if (n == 0)
{
return p;
}
p += n;
}
return p;
}
/*!
@brief the rest of a string from p (a character boundary), 16 bytes per step
The first quote, backslash, or control character is found with vector
compares, and the UTF-8 check covers the bytes up to it. Returns where the
string scan stops, like scan_string_run: before ill-formed UTF-8 and for the
last bytes of the input, the bytes are checked one sequence at a time. Out of
line, so that no constants of the check occupy registers in the parse loop.
On x86-64, it is compiled for SSSE3 (see cpu_has_ssse3()).
*/
NLOHMANN_VIEW_SSSE3_TARGET NLOHMANN_VIEW_NOINLINE inline const unsigned char* scan_string_vector(const unsigned char* p, const unsigned char* e, const std::uint8_t* plain) noexcept
{
using lookup = utf8_lookup4<>;
const unsigned char* block = p;
#if NLOHMANN_VIEW_NEON
const uint8x16_t t1h = vld1q_u8(lookup::byte_1_high.data());
const uint8x16_t t1l = vld1q_u8(lookup::byte_1_low.data());
const uint8x16_t t2h = vld1q_u8(lookup::byte_2_high.data());
uint8x16_t prev = vdupq_n_u8(0);
while (e - block >= 16)
{
const uint8x16_t in = vld1q_u8(block);
const uint8x16_t special = vorrq_u8(vorrq_u8(vceqq_u8(in, vdupq_n_u8('"')), vceqq_u8(in, vdupq_n_u8('\\'))), vcltq_u8(in, vdupq_n_u8(0x20)));
const uint8x16_t prev1 = vextq_u8(prev, in, 15);
const uint8x16_t sc = vandq_u8(vandq_u8(vqtbl1q_u8(t1h, vshrq_n_u8(prev1, 4)), vqtbl1q_u8(t1l, vandq_u8(prev1, vdupq_n_u8(0x0F)))), vqtbl1q_u8(t2h, vshrq_n_u8(in, 4)));
const uint8x16_t must23 = vorrq_u8(vqsubq_u8(vextq_u8(prev, in, 14), vdupq_n_u8(0xE0 - 0x80)), vqsubq_u8(vextq_u8(prev, in, 13), vdupq_n_u8(0xF0 - 0x80)));
const uint8x16_t err = veorq_u8(vandq_u8(must23, vdupq_n_u8(0x80)), sc);
const std::uint64_t special_bits = vget_lane_u64(vreinterpret_u64_u8(vshrn_n_u16(vreinterpretq_u16_u8(special), 4)), 0);
const std::uint64_t err_bits = vget_lane_u64(vreinterpret_u64_u8(vshrn_n_u16(vreinterpretq_u16_u8(vtstq_u8(err, err)), 4)), 0);
if (special_bits != 0)
{
// errors up to the special byte count (an incomplete sequence
// before a quote shows at the quote); the bytes after it do not
const unsigned k = static_cast<unsigned>(count_trailing_zeros(special_bits)) >> 2u;
const std::uint64_t upto = k == 15 ? ~std::uint64_t{0} :
(std::uint64_t{1} << (4u * (k + 1u))) - 1u;
if ((err_bits & upto) == 0)
{
return block + k;
}
break;
}
if (err_bits != 0)
{
break;
}
prev = in;
block += 16;
}
#else
// the same with SSSE3 (pshufb for the table lookups; nibbles from 16-bit
// shifts, as there are no byte shifts)
const __m128i t1h = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(lookup::byte_1_high.data()))); // NOLINT(bugprone-casting-through-void)
const __m128i t1l = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(lookup::byte_1_low.data()))); // NOLINT(bugprone-casting-through-void)
const __m128i t2h = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(lookup::byte_2_high.data()))); // NOLINT(bugprone-casting-through-void)
const __m128i nibble = _mm_set1_epi8(0x0F);
const __m128i zero = _mm_setzero_si128();
__m128i prev = zero;
while (e - block >= 16)
{
const __m128i in = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(block))); // NOLINT(bugprone-casting-through-void)
const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('"')), _mm_cmpeq_epi8(in, _mm_set1_epi8('\\'))),
_mm_cmpeq_epi8(_mm_subs_epu8(in, _mm_set1_epi8(0x1F)), zero)); // in < 0x20
const __m128i prev1 = _mm_alignr_epi8(in, prev, 15);
const __m128i sc = _mm_and_si128(_mm_and_si128(_mm_shuffle_epi8(t1h, _mm_and_si128(_mm_srli_epi16(prev1, 4), nibble)),
_mm_shuffle_epi8(t1l, _mm_and_si128(prev1, nibble))),
_mm_shuffle_epi8(t2h, _mm_and_si128(_mm_srli_epi16(in, 4), nibble)));
const __m128i must23 = _mm_or_si128(_mm_subs_epu8(_mm_alignr_epi8(in, prev, 14), _mm_set1_epi8(0xE0 - 0x80)),
_mm_subs_epu8(_mm_alignr_epi8(in, prev, 13), _mm_set1_epi8(0xF0 - 0x80)));
const __m128i err = _mm_xor_si128(_mm_and_si128(must23, _mm_set1_epi8(static_cast<char>(-128))), sc);
const auto special_bits = static_cast<unsigned>(_mm_movemask_epi8(special));
const auto err_bits = ~static_cast<unsigned>(_mm_movemask_epi8(_mm_cmpeq_epi8(err, zero))) & 0xFFFFu;
if (special_bits != 0)
{
const auto k = static_cast<unsigned>(count_trailing_zeros(static_cast<std::uint64_t>(special_bits)));
if ((err_bits & ((2u << k) - 1u)) == 0)
{
return block + k;
}
break;
}
if (err_bits != 0)
{
break;
}
prev = in;
block += 16;
}
#endif
return scan_string_finish(p, block, e, plain);
}
#endif
} // namespace view
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
-4
View File
@@ -114,10 +114,6 @@ struct is_non_bool_arithmetic : conjunction<std::is_arithmetic<T>, negation<std:
template<typename T>
struct is_non_c_string_default : negation<std::is_same<typename std::decay<T>::type, const char*>> {};
/// the integral types accepted as an array index by the edits of a document
template<typename T>
struct is_non_bool_integral : conjunction<std::is_integral<T>, negation<std::is_same<T, bool>>> {};
/// whether View::get<T>() converts to T without materialize(); get_to()
/// assigns these, and passes the others to from_json
template<typename T, typename BasicJsonType, typename View>
+28 -335
View File
@@ -7,16 +7,14 @@
// SPDX-License-Identifier: MIT
/****************************************************************************\
* Zero-copy view of a parsed JSON text. *
* Zero-copy, read-only view of a parsed JSON text. *
* *
* json_document::parse() builds a flat index of the values of a JSON text *
* (16 bytes per value) instead of a tree of basic_json values. Strings and *
* numbers stay in the source text; only strings with escapes are decoded, *
* into one buffer. json_view is a handle to one value of the document, with *
* the read-only part of the basic_json interface; materialize() turns a *
* subtree into the basic_json value that parse() would produce. An editable *
* document (json_editable_document) also has set(), push_back(), insert(), *
* and erase(): edits never write to the source text, and views stay valid. *
* subtree into the basic_json value that parse() would produce. *
* *
* The source text must outlive a document that borrows it (lvalue byte *
* containers, C strings); rvalue strings, streams, and other inputs are *
@@ -26,10 +24,9 @@
#ifndef INCLUDE_NLOHMANN_JSON_VIEW_HPP_
#define INCLUDE_NLOHMANN_JSON_VIEW_HPP_
#include <algorithm> // all_of, min
#include <algorithm> // min
#include <array> // array
#include <cstddef> // nullptr_t, size_t // IWYU pragma: keep
#include <cstdint> // uint8_t, uint32_t
#include <cstring> // memcpy, strlen
#include <istream> // istream // IWYU pragma: keep
#include <iterator> // distance, input_iterator_tag, iterator_traits
@@ -69,16 +66,13 @@
#include <nlohmann/detail/view/builder.hpp>
#include <nlohmann/detail/view/compare.hpp>
#include <nlohmann/detail/view/document_data.hpp>
#include <nlohmann/detail/view/edit.hpp>
#include <nlohmann/detail/view/errors.hpp>
#include <nlohmann/detail/view/image.hpp>
#include <nlohmann/detail/view/input.hpp>
#include <nlohmann/detail/view/iterator.hpp>
#include <nlohmann/detail/view/lookup.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
#include <nlohmann/detail/view/materialize.hpp>
#include <nlohmann/detail/view/node.hpp>
#include <nlohmann/detail/view/object_index.hpp>
#include <nlohmann/detail/view/pointer.hpp>
#include <nlohmann/detail/view/serializer.hpp>
#include <nlohmann/detail/view/string_ref.hpp>
@@ -92,13 +86,11 @@ NLOHMANN_JSON_NAMESPACE_BEGIN
Trivially copyable (two pointers). Valid as long as the document is alive and
has not been re-parsed, and as long as a borrowed source text is alive.
*/
template<typename BasicJsonType, bool Editable = false>
template<typename BasicJsonType>
class basic_json_view
{
using node = detail::view::node;
using document_data = detail::view::document_data;
/// how the index is walked (with edits only for editable documents)
using navigation = detail::view::navigation<Editable>;
public:
using value_t = detail::value_t;
@@ -295,7 +287,7 @@ class basic_json_view
}
detail::view::throw_type_error(305, "cannot use operator[] with a numeric argument with ", type_name());
}
return idx < m_node->len ? basic_json_view(m_doc, navigation::value(detail::view::element_at<Editable>(*m_doc, m_node, idx))) : basic_json_view();
return idx < m_node->len ? basic_json_view(m_doc, detail::view::element_at(m_node, idx)) : basic_json_view();
}
/// any other integer type (int, unsigned, long, std::int64_t, ...; a
@@ -358,7 +350,7 @@ class basic_json_view
{
detail::view::throw_out_of_range(401, detail::concat("array index ", std::to_string(idx), " is out of range"));
}
return basic_json_view(m_doc, navigation::value(detail::view::element_at<Editable>(*m_doc, m_node, idx)));
return basic_json_view(m_doc, detail::view::element_at(m_node, idx));
}
/// any other integer type, see operator[]; negative values are out of
@@ -433,7 +425,7 @@ class basic_json_view
{
if (is_structured() && m_node->len != 0)
{
return basic_json_view(m_doc, navigation::value(detail::view::last_child<Editable>(*m_doc, m_node) + (is_object() ? 1 : 0)));
return basic_json_view(m_doc, detail::view::last_child(m_node) + (is_object() ? 1 : 0));
}
return front();
}
@@ -450,7 +442,7 @@ class basic_json_view
{
return end();
}
const node* const k = detail::view::find_member<Editable>(*m_doc, m_node, key.data(), key.size());
const node* const k = detail::view::find_member(*m_doc, m_node, key.data(), key.size());
return k != nullptr ? iterator(m_doc, k, true) : end();
}
@@ -467,7 +459,7 @@ class basic_json_view
/// whether this is an object with a member with this key
bool contains(string_view_t key) const
{
return is_object() && detail::view::find_member<Editable>(*m_doc, m_node, key.data(), key.size()) != nullptr;
return is_object() && detail::view::find_member(*m_doc, m_node, key.data(), key.size()) != nullptr;
}
bool contains(const char* key) const
@@ -491,7 +483,7 @@ class basic_json_view
/// more than once); 0 if this is not an object
size_type count(string_view_t key) const
{
return is_object() ? detail::view::count_members<Editable>(*m_doc, m_node, key.data(), key.size()) : 0;
return is_object() ? detail::view::count_members(*m_doc, m_node, key.data(), key.size()) : 0;
}
size_type count(const char* key) const
@@ -514,7 +506,7 @@ class basic_json_view
{
if (NLOHMANN_VIEW_LIKELY(is_structured()))
{
return iterator(m_doc, navigation::first(*m_doc, m_node), is_object());
return iterator(m_doc, document_data::first_child(m_node), is_object());
}
return iterator(m_doc, m_node, false);
}
@@ -523,7 +515,7 @@ class basic_json_view
{
if (NLOHMANN_VIEW_LIKELY(is_structured()))
{
return iterator(m_doc, navigation::end(*m_doc, m_node), is_object());
return iterator(m_doc, document_data::child_end(m_node), is_object());
}
return iterator(m_doc, (is_null() || is_discarded()) ? m_node : m_node + 1, false);
}
@@ -623,11 +615,9 @@ class basic_json_view
style.indent_char = indent_char;
style.ensure_ascii = ensure_ascii;
style.source_numbers = numbers == number_format::source;
// the compact text is about as long as the source text of the value;
// the compact writer keeps 64 bytes of slack, so that it does not grow
// the buffer just before the end
const std::size_t estimate = source_extent() + (style.pretty ? source_extent() / 2 : 0) + 160;
detail::view::view_serializer<BasicJsonType, Editable>(*m_doc, out, estimate, style).dump(m_node);
// the compact text is about as long as the source text of the value
const std::size_t estimate = source_extent() + (style.pretty ? source_extent() / 2 : 0) + 64;
detail::view::view_serializer<BasicJsonType>(*m_doc, out, estimate, style).dump(m_node);
return out;
}
@@ -661,19 +651,6 @@ class basic_json_view
return !(a == b);
}
/// a view of an editable document compares with one of a read-only document
template < bool E, typename std::enable_if < E != Editable, int >::type = 0 >
friend bool operator==(const basic_json_view& a, const basic_json_view<BasicJsonType, E>& b)
{
return detail::view::equal<BasicJsonType>(side(a), detail::view::view_side<BasicJsonType, basic_json_view<BasicJsonType, E>>(b));
}
template < bool E, typename std::enable_if < E != Editable, int >::type = 0 >
friend bool operator!=(const basic_json_view& a, const basic_json_view<BasicJsonType, E>& b)
{
return !(a == b);
}
/// whether the value parse() would produce for a view equals a value
friend bool operator==(const basic_json_view& a, const BasicJsonType& j)
{
@@ -707,7 +684,7 @@ class basic_json_view
{
return BasicJsonType(value_t::discarded);
}
return detail::view::materialize<BasicJsonType, Editable>(*m_doc, m_node);
return detail::view::materialize<BasicJsonType>(*m_doc, m_node);
}
/// byte offset of this value in the source text (for strings: of the
@@ -715,14 +692,12 @@ class basic_json_view
/// a discarded view and for strings with escapes, which are decoded
std::size_t source_offset() const noexcept
{
return m_node != nullptr && (m_node->flags & (detail::view::node_flags::storage | detail::view::node_flags::moved | detail::view::node_flags::is_new)) == 0
return m_node != nullptr && (m_node->flags & detail::view::node_flags::storage) == 0
? m_node->off : static_cast<std::size_t>(-1);
}
private:
template<typename, bool> friend class basic_json_document;
template<typename, bool> friend class basic_json_view;
template<typename, typename> friend class detail::view::editor;
template<typename> friend class basic_json_document;
friend iterator;
basic_json_view(const document_data* d, const node* n) noexcept
@@ -736,25 +711,10 @@ class basic_json_view
return detail::view::view_side<BasicJsonType, basic_json_view>(v);
}
/// the template argument as a runtime condition: testing it directly in
/// `Editable && ...` makes the condition constant, which MSVC reports as
/// C4127
static bool editable() noexcept
{
return Editable;
}
/// the number of source bytes of this value (estimated for values with
/// decoded strings); the estimate sizes the output buffer of dump()
std::size_t source_extent() const noexcept
{
if (editable() && m_doc->edits != nullptr)
{
// positions of moved and new values are not source offsets; the root
// (the first node) spans the source text and every edit
const bool is_root = m_node == m_doc->nodes;
return is_root ? m_doc->size + m_doc->edits->text_used : 64;
}
const node* const end = m_doc->nodes + m_doc->node_count;
if ((m_node->flags & detail::view::node_flags::storage) != 0)
{
@@ -785,8 +745,8 @@ class basic_json_view
/// (object required)
NLOHMANN_VIEW_ALWAYS_INLINE basic_json_view lookup(string_view_t key) const noexcept
{
const node* const k = detail::view::find_member<Editable>(*m_doc, m_node, key.data(), key.size());
return k != nullptr ? basic_json_view(m_doc, navigation::value(k + 1)) : basic_json_view();
const node* const k = detail::view::find_member(*m_doc, m_node, key.data(), key.size());
return k != nullptr ? basic_json_view(m_doc, k + 1) : basic_json_view();
}
// --- get() dispatch ---
@@ -892,7 +852,7 @@ Borrowed parses keep a pointer to the caller's text, which must outlive the
document. Owned parses (parse_copy, rvalue std::string, streams, and inputs
that are not contiguous byte ranges) keep their own copy.
*/
template<typename BasicJsonType, bool Editable = false>
template<typename BasicJsonType>
class basic_json_document
{
using document_data = detail::view::document_data;
@@ -901,7 +861,7 @@ class basic_json_document
"json_view supports 64-bit integer types only");
public:
using view_type = basic_json_view<BasicJsonType, Editable>;
using view_type = basic_json_view<BasicJsonType>;
using value_t = detail::value_t;
/// an empty (discarded) document
@@ -1031,7 +991,7 @@ class basic_json_document
/// whether the document holds its own copy of the text
bool owns_source() const noexcept
{
return m_data && ((!m_data->owned.empty() && m_data->src == m_data->owned.data()) || !m_data->owned_image.empty());
return m_data && !m_data->owned.empty() && m_data->src == m_data->owned.data();
}
/// number of index nodes (values plus object keys)
@@ -1040,7 +1000,7 @@ class basic_json_document
return m_data ? m_data->node_count : 0;
}
/// bytes held by the document (index, decoded strings, and owned text or image)
/// bytes held by the document (index, decoded strings, owned text)
std::size_t memory_usage() const noexcept
{
if (!m_data)
@@ -1049,10 +1009,7 @@ class basic_json_document
}
return sizeof(document_data) + (m_data->inline_capacity * sizeof(detail::view::node))
+ (m_data->nodes != m_data->inline_nodes ? m_data->node_capacity * sizeof(detail::view::node) : 0)
+ m_data->arena.capacity() + m_data->owned.capacity() + m_data->owned_image.capacity()
+ (m_data->indexes.capacity() * sizeof(document_data::object_index)) + (m_data->index_slots.capacity() * sizeof(std::uint32_t))
+ (m_data->large_objects.capacity() * sizeof(std::uint32_t))
+ (m_data->edits != nullptr ? m_data->edits->bytes : 0);
+ m_data->arena.capacity() + m_data->owned.capacity();
}
/// release unused capacity of the index and the decoded strings; like
@@ -1069,15 +1026,10 @@ class basic_json_document
// allocate everything first, so that an exception leaves the document
// unchanged
// (the decoded strings of a loaded image stay in the image)
const bool arena_in_use = d.base[1] == d.arena.data();
const bool shrink_arena = d.arena.capacity() > d.arena.size();
std::vector<char> arena(shrink_arena && arena_in_use ? d.arena : std::vector<char>());
// (edits link to the nodes of the index, which then stays in place)
const bool shrink_nodes = d.nodes != d.inline_nodes && d.node_count != d.node_capacity && d.edits == nullptr;
std::vector<char> arena(shrink_arena ? d.arena : std::vector<char>());
const bool shrink_nodes = d.nodes != d.inline_nodes && d.node_count != d.node_capacity;
const bool into_header = d.node_count <= d.inline_capacity;
std::vector<document_data::object_index> indexes(d.indexes.capacity() > d.indexes.size() ? d.indexes : std::vector<document_data::object_index>());
std::vector<std::uint32_t> index_slots(d.index_slots.capacity() > d.index_slots.size() ? d.index_slots : std::vector<std::uint32_t>());
node* fresh = (shrink_nodes && !into_header) ? static_cast<node*>(::operator new (d.node_count * sizeof(node))) : d.inline_nodes;
if (shrink_nodes)
@@ -1093,254 +1045,13 @@ class basic_json_document
if (shrink_arena)
{
d.arena.swap(arena);
if (arena_in_use)
{
d.base[1] = d.arena.data();
}
d.base[1] = d.arena.data();
}
if (d.indexes.capacity() > d.indexes.size())
{
d.indexes.swap(indexes);
}
if (d.index_slots.capacity() > d.index_slots.size())
{
d.index_slots.swap(index_slots);
}
}
////////////
// images //
////////////
/// how load() checks an image (full, bounds, or none)
using image_check = detail::view::image_check;
/// The document as an image that load() reads without parsing: the node
/// index, the text, and the decoded strings. An edited document is
/// written in its current state (floats that are not finite become null,
/// as in dump()).
std::vector<std::uint8_t> save() const
{
if (NLOHMANN_VIEW_UNLIKELY(!m_data || m_data->discarded))
{
detail::view::throw_type_error(320, "cannot save a discarded json_document");
}
return detail::view::save_image(*m_data);
}
/// Read an image written by save(). The image is borrowed: it must stay
/// alive and unchanged while the document is used.
NLOHMANN_VIEW_NODISCARD
static basic_json_document load(const std::uint8_t* image, std::size_t size, const image_check check = image_check::full)
{
basic_json_document d;
d.ensure_data(nullptr, 0);
detail::view::load_image(*d.m_data, image, size, check);
return d;
}
/// read an image (borrowed)
NLOHMANN_VIEW_NODISCARD
static basic_json_document load(const std::vector<std::uint8_t>& image, const image_check check = image_check::full)
{
return load(image.data(), image.size(), check);
}
/// read an image and keep it (no copy)
NLOHMANN_VIEW_NODISCARD
static basic_json_document load(std::vector<std::uint8_t>&& image, const image_check check = image_check::full)
{
basic_json_document d;
d.ensure_data(nullptr, 0);
d.m_data->owned_image = std::move(image);
detail::view::load_image(*d.m_data, d.m_data->owned_image.data(), d.m_data->owned_image.size(), check);
return d;
}
///////////
// edits //
///////////
// The source text is never written; new values go to storage owned by
// the document. A view keeps referring to the same value: after an
// assignment it sees the new value, and edits elsewhere do not affect it.
// A view of an erased value keeps its last value. An edit of an
// array/object invalidates the iterators over it. Values are accepted as
// views (of any document), BasicJsonType values, and everything
// BasicJsonType can be constructed from.
using string_view_t = typename view_type::string_view_t;
using json_pointer = typename BasicJsonType::json_pointer;
/// replace a value (a view of this document); returns a view of it
template<typename V>
view_type set(view_type target, V&& value)
{
return editor().set(target, std::forward<V>(value));
}
/// set a member (added if missing; a null value becomes an object);
/// returns a view of the member value
template<typename V>
view_type set(view_type object, string_view_t key, V&& value)
{
return editor().set(object, key, std::forward<V>(value));
}
/// assign an existing array element; returns a view of it
template<typename I, typename V, typename std::enable_if<detail::view::is_non_bool_integral<I>::value, int>::type = 0>
view_type set(view_type array, I idx, V && value)
{
return editor().set(array, index(idx), std::forward<V>(value));
}
/// set the value at a JSON pointer: its parent must exist; an object
/// member is set (added if missing), an array element assigned, and "-"
/// or the size of the array appends. A null parent becomes what
/// basic_json's operator[](json_pointer) makes of it: an array for "-"
/// and for digits (padded with nulls up to the index), an object otherwise.
template<typename V>
view_type set(const json_pointer& ptr, V&& value)
{
if (ptr.empty())
{
return set(root(), std::forward<V>(value));
}
const view_type parent = root().at(ptr.parent_pointer());
const auto& token = ptr.back();
if (parent.is_null())
{
if (token == "-")
{
return push_back(parent, std::forward<V>(value));
}
const bool digits = std::all_of(token.begin(), token.end(), [](const char c)
{
return c >= '0' && c <= '9';
});
if (digits)
{
// (an invalid index is an error before the parent changes)
const std::size_t idx = pointer_index(token);
if (idx >= 0xFFFFFFFFu)
{
detail::view::throw_out_of_range(401, detail::concat("array index ", std::to_string(idx), " is out of range"));
}
for (std::size_t i = 0; i < idx; ++i)
{
push_back(parent, nullptr);
}
return push_back(parent, std::forward<V>(value));
}
}
if (parent.is_array())
{
const std::size_t idx = token == "-" ? parent.size() : pointer_index(token);
if (idx == parent.size())
{
return push_back(parent, std::forward<V>(value));
}
return set(parent, idx, std::forward<V>(value));
}
return set(parent, string_view_t(token.data(), token.size()), std::forward<V>(value));
}
/// append to an array (a null value becomes an array); returns a view of
/// the new element
template<typename V>
view_type push_back(view_type array, V&& value)
{
return editor().push_back(array, std::forward<V>(value));
}
/// insert into an array before position idx (idx <= size()); returns a
/// view of the new element
template<typename I, typename V, typename std::enable_if<detail::view::is_non_bool_integral<I>::value, int>::type = 0>
view_type insert(view_type array, I idx, V && value)
{
return editor().insert(array, index(idx), std::forward<V>(value));
}
/// remove all members with this key; returns their number
std::size_t erase(view_type object, string_view_t key)
{
return editor().erase(object, key);
}
/// remove an array element
template<typename I, typename std::enable_if<detail::view::is_non_bool_integral<I>::value, int>::type = 0>
void erase(view_type array, I idx)
{
editor().erase(array, index(idx));
}
/// remove the value at a JSON pointer; returns the number of removed
/// values
std::size_t erase(const json_pointer& ptr)
{
if (ptr.empty())
{
detail::view::throw_out_of_range(405, "JSON pointer has no parent");
}
const view_type parent = root().at(ptr.parent_pointer());
const auto& token = ptr.back();
if (parent.is_array())
{
erase(parent, pointer_index(token));
return 1;
}
return erase(parent, string_view_t(token.data(), token.size()));
}
private:
using input_kind = detail::view::input_kind;
detail::view::editor<BasicJsonType, view_type> editor()
{
static_assert(Editable, "only an editable document can be edited: use basic_json_document<BasicJsonType, true> (json_editable_document)");
if (NLOHMANN_VIEW_UNLIKELY(!m_data || m_data->discarded))
{
detail::view::throw_invalid_iterator(202, "view does not belong to this document");
}
return detail::view::editor<BasicJsonType, view_type>(*m_data);
}
/// an index (out_of_range.401 if negative)
template<typename I>
static std::size_t index(I idx)
{
return index(idx, std::is_signed<I> {});
}
template<typename I>
static std::size_t index(I idx, std::true_type /*signed*/)
{
if (idx < 0)
{
detail::view::throw_out_of_range(401, detail::concat("array index ", std::to_string(idx), " is out of range"));
}
return static_cast<std::size_t>(idx);
}
template<typename I>
static std::size_t index(I idx, std::false_type /*unsigned*/)
{
return static_cast<std::size_t>(idx);
}
/// the array index of a JSON pointer token (json_pointer's rules)
template<typename StringType>
static std::size_t pointer_index(const StringType& token)
{
std::size_t idx = 0;
const detail::view::index_status status = detail::view::array_index(token, idx);
if (status != detail::view::index_status::ok)
{
detail::view::throw_array_index_error(status, token);
}
return idx;
}
/// create the storage (sized for the input) on first use
void ensure_data(const char* src, std::size_t size)
{
@@ -1391,16 +1102,10 @@ class basic_json_document
{
d.owned.clear();
}
d.owned_image.clear();
d.src = src;
d.size = size;
d.node_count = 0;
d.edits.reset(); // (views of the previous text end here anyway)
d.base[2] = nullptr;
d.arena.clear();
d.indexes.clear();
d.index_slots.clear();
d.large_objects.clear();
d.discarded = true;
detail::view::parse_failure failure;
bool ok = false;
@@ -1416,13 +1121,9 @@ class basic_json_document
{
d.base[0] = d.src;
d.base[1] = d.arena.data();
d.arena_size = d.arena.size();
detail::view::build_object_indexes(d);
std::vector<std::uint32_t>().swap(d.large_objects); // (only needed while parsing)
d.discarded = false;
return;
}
std::vector<std::uint32_t>().swap(d.large_objects);
if (allow_exceptions)
{
detail::view::throw_parse_failure<BasicJsonType>(failure, src, size, comments, trailing_commas);
@@ -1567,14 +1268,6 @@ using json_view = basic_json_view<json>;
using ordered_json_document = basic_json_document<ordered_json>;
/// a value of an ordered_json_document
using ordered_json_view = basic_json_view<ordered_json>;
/// an editable parsed JSON text for json
using json_editable_document = basic_json_document<json, true>;
/// a value of a json_editable_document
using json_editable_view = basic_json_view<json, true>;
/// an editable parsed JSON text for ordered_json
using ordered_json_editable_document = basic_json_document<ordered_json, true>;
/// a value of an ordered_json_editable_document
using ordered_json_editable_view = basic_json_view<ordered_json, true>;
NLOHMANN_JSON_NAMESPACE_END
-454656
View File
File diff suppressed because it is too large. Load diff
+135 -810
View File
File diff suppressed because it is too large. Load diff
+4 -18
View File
@@ -139,26 +139,14 @@
#define NLOHMANN_JSON_ABI_TAG_DISABLE_UNLOCKED_FILE_IO
#endif
#ifdef JSON_VIEW_NO_SIMD
#define NLOHMANN_JSON_ABI_TAG_VIEW_NO_SIMD _vns
#else
#define NLOHMANN_JSON_ABI_TAG_VIEW_NO_SIMD
#endif
#ifdef JSON_VIEW_USE_SSSE3
#define NLOHMANN_JSON_ABI_TAG_VIEW_USE_SSSE3 _vssse3
#else
#define NLOHMANN_JSON_ABI_TAG_VIEW_USE_SSSE3
#endif
#ifndef NLOHMANN_JSON_NAMESPACE_NO_VERSION
#define NLOHMANN_JSON_NAMESPACE_NO_VERSION 0
#endif
// Construct the namespace ABI tags component
#define NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f, g, h, i, j, k, l) json_abi ## a ## b ## c ## d ## e ## f ## g ## h ## i ## j ## k ## l
#define NLOHMANN_JSON_ABI_TAGS_CONCAT(a, b, c, d, e, f, g, h, i, j, k, l) \
NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f, g, h, i, j, k, l)
#define NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f, g, h, i, j) json_abi ## a ## b ## c ## d ## e ## f ## g ## h ## i ## j
#define NLOHMANN_JSON_ABI_TAGS_CONCAT(a, b, c, d, e, f, g, h, i, j) \
NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c, d, e, f, g, h, i, j)
#define NLOHMANN_JSON_ABI_TAGS \
NLOHMANN_JSON_ABI_TAGS_CONCAT( \
@@ -171,9 +159,7 @@
NLOHMANN_JSON_ABI_TAG_STRICT_BINARY_UTF8, \
NLOHMANN_JSON_ABI_TAG_OBJECTS_FOR_ENUM_KEYED_MAPS, \
NLOHMANN_JSON_ABI_TAG_DISABLE_STREAM_BULK_SCAN, \
NLOHMANN_JSON_ABI_TAG_DISABLE_UNLOCKED_FILE_IO, \
NLOHMANN_JSON_ABI_TAG_VIEW_NO_SIMD, \
NLOHMANN_JSON_ABI_TAG_VIEW_USE_SSSE3)
NLOHMANN_JSON_ABI_TAG_DISABLE_UNLOCKED_FILE_IO)
// Construct the namespace version component
#define NLOHMANN_JSON_NAMESPACE_VERSION_CONCAT_EX(major, minor, patch) \
+479 -4054
View File
File diff suppressed because it is too large. Load diff
-6
View File
@@ -17,12 +17,6 @@ module;
// 3. Or upgrade to a newer GCC version with better modules support.
// See: https://github.com/nlohmann/json/issues/5103
// GCC ignores the target attribute in modules ("'target' attribute currently
// unsupported in modules") and then rejects the SSSE3 intrinsics of json_view.
#if defined(__GNUC__) && !defined(__clang__)
#define NLOHMANN_VIEW_NO_TARGET_ATTRIBUTE
#endif
#include <nlohmann/json.hpp>
#include <nlohmann/json_literals.hpp>
#include <nlohmann/json_view.hpp>
+3 -55
View File
@@ -2,14 +2,12 @@ cmake_minimum_required(VERSION 3.13...4.0)
option(JSON_Valgrind "Execute test suite with Valgrind." OFF)
option(JSON_FastTests "Skip expensive/slow tests." OFF)
option(JSON_TestUnityBuild "Build the unit tests in batches of several files per executable to speed up compilation." ON)
option(JSON_TestSimdutf "Build the unit tests against the simdutf UTF-8 validation backend." OFF)
set(JSON_SIMDUTF_VERSION 9.1.0 CACHE STRING "The simdutf version used by JSON_TestSimdutf.")
set(JSON_32bitTest AUTO CACHE STRING "Enable the 32bit unit test (ON/OFF/AUTO/ONLY).")
set(JSON_TestStandards "" CACHE STRING "The list of standards to test explicitly.")
set(JSON_TestUnityBatchSize 8 CACHE STRING "The number of unit test files compiled into one executable with JSON_TestUnityBuild.")
set(JSON_TestShard "" CACHE STRING "Build only a part of the unit tests, given as <index>/<count> (e.g. 0/2), to split them across CI jobs with a time limit.")
# using an env var, since this will also affect targets executing cmake (such as "ci_test_compiler_default")
@@ -22,11 +20,6 @@ endif()
include(test)
# MinGW's GNU ld hits section and relocation limits with the large merged translation units
if(MINGW)
set(JSON_TestUnityBuild OFF)
endif()
#############################################################################
# override standard support
#############################################################################
@@ -333,28 +326,9 @@ if(NOT "${JSON_TestShard}" STREQUAL "")
message(STATUS "Test shard ${JSON_TestShard}: ${shard_file_count} of ${test_file_count} unit test files")
endif()
# Explicit groups of test files (file names without "unit-" and ".cpp") for
# JSON_TestUnityBuild: all files of a group are compiled into one translation
# unit per C++ standard, so that related tests share their template
# instantiations (all binary formats instantiate the same readers and
# writers). Only add a group where this was measured to pay off: grouping
# files that share little just creates one slow translation unit. Files in no
# group are batched alphabetically by JSON_TestUnityBatchSize.
set(json_test_unity_groups binary)
set(json_test_unity_group_binary binary_formats binary_utf8_error_handler binary_writer_sinks bjdata bon8 bon8-cpp17 bson byte_container_with_subtype cbor msgpack msgpack-cpp17 ubjson)
if(JSON_TestUnityBuild)
message(STATUS "Building the unit tests in batches of ${JSON_TestUnityBatchSize} files (JSON_TestUnityBuild)")
json_test_add_unity_tests(FILES ${files} MAIN test_main CXX_STANDARDS ${test_cxx_standards} ${test_force}
BATCH_SIZE ${JSON_TestUnityBatchSize}
GROUPS ${json_test_unity_groups}
VARIANT_FILES src/unit-comparison.cpp src/unit-comparison-cpp20.cpp src/unit-class_parser.cpp src/unit-diagnostic-positions.cpp
)
else()
foreach(file ${files})
json_test_add_test_for(${file} MAIN test_main CXX_STANDARDS ${test_cxx_standards} ${test_force})
endforeach()
endif()
foreach(file ${files})
json_test_add_test_for(${file} MAIN test_main CXX_STANDARDS ${test_cxx_standards} ${test_force})
endforeach()
# tests/src/unit-no-macro-leak.cpp #include-s the generated leak-check file,
# so its test targets must be built after generate_hedley_undef_checks.
@@ -380,13 +354,6 @@ json_test_add_test_for(src/unit-comparison.cpp
NAME test-comparison_legacy
MAIN test_main CXX_STANDARDS ${test_cxx_standards} ${test_force}
)
json_test_set_test_options(test-comparison_legacy-cpp20
COMPILE_DEFINITIONS JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON=1
)
json_test_add_test_for(src/unit-comparison-cpp20.cpp
NAME test-comparison_legacy-cpp20
MAIN test_main CXX_STANDARDS ${test_cxx_standards} ${test_force}
)
# test the parser again with JSON_DIAGNOSTIC_POSITIONS enabled
json_test_set_test_options(test-class_parser_diagnostic_positions
@@ -433,25 +400,6 @@ json_test_add_test_for(src/unit-file_input.cpp
MAIN test_main CXX_STANDARDS ${test_cxx_standards} ${test_force}
)
# the json_view parser again with the portable string scanning instead of
# NEON/SSE2, and on x86-64 with the SSSE3 UTF-8 check (JSON_VIEW_USE_SSSE3)
json_test_set_test_options(test-json_view_builder_portable
COMPILE_DEFINITIONS JSON_VIEW_NO_SIMD
)
json_test_add_test_for(src/unit-json_view_builder.cpp
NAME test-json_view_builder_portable
MAIN test_main CXX_STANDARDS ${test_cxx_standards} ${test_force}
)
if(CMAKE_SYSTEM_PROCESSOR MATCHES "^(x86_64|AMD64|amd64)$" AND NOT MSVC)
json_test_set_test_options(test-json_view_builder_ssse3
COMPILE_DEFINITIONS JSON_VIEW_USE_SSSE3
COMPILE_OPTIONS -mssse3
)
json_test_add_test_for(src/unit-json_view_builder.cpp
NAME test-json_view_builder_ssse3
MAIN test_main CXX_STANDARDS ${test_cxx_standards} ${test_force}
)
endif()
endif()
# *DO NOT* use json_test_set_test_options() below this line
+1 -4
View File
@@ -10,7 +10,7 @@ CXXFLAGS += -std=c++11
CPPFLAGS += -I ../single_include
FUZZER_ENGINE = src/fuzzer-driver_afl.cpp
FUZZERS = parse_afl_fuzzer parse_bson_fuzzer parse_cbor_fuzzer parse_msgpack_fuzzer parse_ubjson_fuzzer parse_bjdata_fuzzer parse_bon8_fuzzer parse_json_view_fuzzer json_view_image_fuzzer
FUZZERS = parse_afl_fuzzer parse_bson_fuzzer parse_cbor_fuzzer parse_msgpack_fuzzer parse_ubjson_fuzzer parse_bjdata_fuzzer parse_bon8_fuzzer parse_json_view_fuzzer
fuzzers: $(FUZZERS)
parse_afl_fuzzer:
@@ -19,9 +19,6 @@ parse_afl_fuzzer:
parse_json_view_fuzzer:
$(CXX) $(CXXFLAGS) $(CPPFLAGS) $(FUZZER_ENGINE) src/fuzzer-parse_json_view.cpp -o $@
json_view_image_fuzzer:
$(CXX) $(CXXFLAGS) $(CPPFLAGS) $(FUZZER_ENGINE) src/fuzzer-json_view_image.cpp -o $@
parse_bson_fuzzer:
$(CXX) $(CXXFLAGS) $(CPPFLAGS) $(FUZZER_ENGINE) src/fuzzer-parse_bson.cpp -o $@
-92
View File
@@ -1,92 +0,0 @@
# Unit tests
The unit tests are in [`src/unit-*.cpp`](src) and use [doctest](https://github.com/doctest/doctest). Each file becomes
one CTest test per C++ standard it is built for, named after the file: `src/unit-foo.cpp` becomes `test-foo_cpp11`.
## Build and run
```sh
cmake -S . -B build -DJSON_BuildTests=ON
cmake --build build -j 10
ctest --test-dir build -j 10
```
The [CMake options](../docs/mkdocs/docs/integration/cmake.md) starting with `JSON_Test` (and `JSON_FastTests`,
`JSON_Valgrind`) control the test build. The most relevant ones:
- `JSON_TestStandards`: build every test file for the given standards, e.g., `-DJSON_TestStandards=17`. By default,
a file is built for C++11 and for each standard whose `JSON_HAS_CPP_<N>` macro it mentions (see below).
- `JSON_TestUnityBuild`: compile several test files together (see below). Set it to `OFF` to get one executable per
test file, e.g., to debug a single test in a debugger.
- `JSON_TestShard`: build only a part of the test files, e.g., `-DJSON_TestShard=0/2`.
To run the tests of one file only, run its CTest test, e.g., `ctest --test-dir build -R test-foo_cpp11`.
## How test files are built
Two mechanisms keep the compile time down. Both are automatic, but they set a few rules for test files.
### C++ standards
A test file is built for a C++ standard beyond C++11 if its text contains `JSON_HAS_CPP_<N>`, for instance in
`#ifdef JSON_HAS_CPP_17`, or in a comment `// JSON_HAS_CPP_17` next to code that needs a macro only defined for that
standard, such as `JSON_HAS_FILESYSTEM`. Then the *whole file* is compiled again for that standard.
So that this does not happen for the large files, tests that need C++14, C++17, or C++20 go into a separate file
`src/unit-<name>-cpp<N>.cpp` (e.g., [`src/unit-regression3-cpp17.cpp`](src/unit-regression3-cpp17.cpp)), whose body
is wrapped in `#ifdef JSON_HAS_CPP_<N>`. The main file `src/unit-<name>.cpp` must not mention `JSON_HAS_CPP_<N>` at
all, not even in a comment, so it is built for C++11 only. The CI jobs `ci_test_*_cxx<N>` still build every test file
for every standard.
### Unity build
With `JSON_TestUnityBuild` (`ON` by default, `OFF` with MinGW), CMake compiles several test files as one translation
unit, so they share the template instantiations of the library. CMake generates `build/tests/unity/test-unity-*.cpp`,
which `#include` the test files of a batch. Every test file still gets its own CTest test, which runs the batch
executable with `--source-file=*unit-foo.cpp`, so only the test cases of that file run.
CMake decides from the macros a file defines *before* including `<nlohmann/json.hpp>`:
| Macros defined before the include | Built as |
|-------------------------------------------------------------------------------------------|-----------------------------------|
| none (or only `SKIP_TESTS_FOR_*` and similar macros derived from global definitions) | batch with the other plain files |
| only `JSON_TESTS_PRIVATE` | batch with the other such files |
| any library configuration macro (`JSON_DIAGNOSTICS`, `JSON_NO_IO`, `JSON_ASSERT`, ...) | its own executable |
Files with test-specific build options (`json_test_set_test_options(test-foo ...)` in
[`CMakeLists.txt`](CMakeLists.txt)) also get their own executable. The batchable files are split alphabetically into
batches of `JSON_TestUnityBatchSize` files, except for explicit groups of related files: `json_test_unity_group_binary`
compiles all binary-format tests together, because they share the binary readers and writers. Only add a group if it
measurably pays off; files that share little just make one slow translation unit.
## Rules for test files
Because the files of a batch share one translation unit, a test file must not affect the files that follow it:
1. **Give file-scope helpers file-specific names.** An anonymous namespace does not help, as the batch is one
translation unit. Use a name such as `my_allocator_2982`, or wrap the helpers in a namespace named after the file
(e.g., `namespace unit_comparison_detail`). Clashes show up as compile errors.
2. **Do not `#undef` library macros**, such as `JSON_HAS_CPP_17`, at the end of a file: this silently disables the tests
that depend on them in the later files of the batch. `#undef` macros you define yourself after including the library.
3. **Put version-dependent tests into `unit-<name>-cpp<N>.cpp`** (see [C++ standards](#c-standards)).
4. **Define test cases in the `.cpp` file itself.** The batch executable selects the tests of a file by its name, so a
`TEST_CASE` in a shared header is not run. The only exception is the test case in
[`src/make_test_data_available.hpp`](src/make_test_data_available.hpp), which CMake handles explicitly.
To check a new file for clashes with all others at once, build with `-DJSON_TestUnityBatchSize=1000`, which puts all
batchable files of a kind into one translation unit.
## Where to add tests
Tests are structured along the features of the library. Usually, an existing file is the right place:
- For a bug fix, add a section referencing the issue to [`src/unit-regression3.cpp`](src/unit-regression3.cpp), or to
[`src/unit-regression3-cpp17.cpp`](src/unit-regression3-cpp17.cpp) /
[`src/unit-regression3-cpp20.cpp`](src/unit-regression3-cpp20.cpp) if the test needs C++17 / C++20.
[`src/unit-regression2.cpp`](src/unit-regression2.cpp) holds older tests and should not grow further.
- When testing exceptions, use `CHECK_THROWS_WITH_AS`, which also checks the `what()` message.
A new file `src/unit-<name>.cpp` is picked up automatically when CMake runs again; no change to `CMakeLists.txt` is
needed.
See also [`fuzzing.md`](fuzzing.md) for fuzz testing.
+1 -3
View File
@@ -23,9 +23,7 @@ foreach(test default noversion)
JSON_DIAGNOSTIC_POSITIONS=1
JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON=1
JSON_DISABLE_STREAM_BULK_SCAN=1
JSON_DISABLE_UNLOCKED_FILE_IO=1
JSON_VIEW_NO_SIMD
JSON_VIEW_USE_SSSE3)
JSON_DISABLE_UNLOCKED_FILE_IO=1)
target_link_libraries(abi_config_${test}_all_tags PRIVATE abi_compat_main)
add_test(
NAME test-abi_config_${test}_all_tags
Loaded 100 of 155 files, more files were not shown because too many files have changed in this diff. Show more