Compare commits

...
Author SHA1 Message Date
Niels Lohmann a04aa3d095 Write doubles with the shortest digits (Zmij)
dump() writes doubles with the conversion of Zmij by Victor Zverovich
(https://github.com/vitaut/zmij, MIT), ported to C++11 in
detail/conversions/zmij.hpp: the shortest decimal in the rounding
interval, the closest one if there are several. Grisu2 does not always
find the shortest digits; about 0.14% of random doubles are now written
differently (0.08% with fewer digits, 0.06% with the closest last
digit); short decimals such as 0.1 or 2555.56 are not affected. float
keeps Grisu2.

The layout of doubles is unchanged, but written differently: the digits
are converted eight at a time (the BCD conversion of Xiang JunBo, as in
Zmij) and stored with one byte swap per eight digits; leading and
trailing zeros are counted from those bytes; and the layouts of
format_buffer() are written with fixed-size moves instead of per-digit
loops and moves of the buffer (to_chars() uses a local buffer if the
caller's is shorter than the 41 bytes this may write).

The powers of ten come from the table for number parsing, adjusted
where it holds them rounded up, and from the compressed tables of Zmij
beyond 10^308. json::dump() gets faster on floats: canada -53%,
numbers -46%, mesh -37%, marine_ik -30%.

Tests: the powers of ten recomputed with a small big-integer; for random
doubles, all powers of two and of ten and their neighbors, and boundary
values: the output reads back as the same value, no decimal with one
digit fewer does, the layout equals that of format_buffer() for the same
digits, and (C++17) the digits equal those of std::to_chars.
The size ratios of canada.json in unit-binary_formats.cpp and one
expectation in unit-to_chars.cpp change with the shorter output.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-29 14:03:25 +02:00
Niels Lohmann cbdc502fbf Write compact dumps of json_view without a library call per token
The default dump() (no indentation, no ensure_ascii) gets its own
writer: the same walk and output, with the write position in a local
variable (stores through char pointers would otherwise force a reload
of the buffer's members after each one), strings and number tokens of
the source copied by fixed-size moves of 32 bytes where the source has
that many bytes left (the buffer keeps 64 bytes of slack), and decoded
strings copied in runs up to the next quote, backslash, or control
character. Documents that are not edited are walked through the node
array in order, so that a frame only needs the end of its container,
and integer tokens are read from the source directly. The innermost
open container is kept in local variables, and the stack holds only
the ones around it; the stack starts in a local array of 32 and moves
to the heap only for deeper nesting (its address does not escape, so
its pointers stay in registers). Dumps of shallow documents thus
allocate only the output, whose first size includes the slack, so it
does not grow just before the end.

The long copies are out of line: otherwise, the compiler merges the
fixed-size moves into the same library call.

These techniques come from the prototype; the writer lost them when the
view was split into pull requests, which made dump() 2 to 3 times
slower.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-29 14:03:24 +02:00
Niels Lohmann 300ab011b0 Document how images store the node index of json_view
The node index section on the architecture page says how save() writes
the nodes and that a change of their layout must raise the image
version, and save's format note links to it.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-29 14:03:23 +02:00
Niels Lohmann 8d9088ab05 Address the cpplint findings of json_document images
The exponent of the overflow check is an std::int64_t instead of a long
(runtime/int).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-29 14:03:22 +02:00
Niels Lohmann 5a936a2ef3 Document the images of json_documents
Add API pages for basic_json_document::save() and load(), including the
image_check enumeration and its three levels. Add examples that show
caching a parsed document as an image, loading it without parsing, the
difference between a borrowed and an owned image, and a full check
rejecting a damaged image that a bounds check still reads safely.

Add an "Images" section to the json_view feature page, register the new
pages in mkdocs.yml and docSet.sql, group basic_json_document's member
list by parsing/access/images/edits, and document parse_error.116 and
type_error.320 on the exceptions page, extending out_of_range.416 for
images.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-29 14:03:21 +02:00
Niels Lohmann 5876a22712 Add images of json_documents: save() and load()
An image is a document stored so that loading it needs no parsing: a
64-byte header, the nodes, the text, and the decoded strings
(little-endian; version 1).

- save() writes an edited document in its current state, in document
  order (floats that are not finite become null, as in dump()); the
  same document always gives the same bytes
- load(pointer, size) and load(const vector&) borrow the image;
  load(vector&&) keeps it without a copy. The nodes are copied (aligned,
  and editable); the hash indexes of large objects are rebuilt.
- image_check::full checks everything the parser guarantees (structure,
  bounds, UTF-8, strings of the source, number tokens and their values);
  bounds checks structure and bounds, so that reading and serializing
  stay safe; none trusts the image.

A malformed image or a failed check throws the new parse_error.116;
saving a discarded document (or images on a big-endian target) throws
the new type_error.320; images of 4 GiB or more out_of_range.416.

As images checked for bounds only can hold any bytes, the general float
conversion now checks the token's grammar (and locates the point and
the exponent itself), the exponent loop of the layout conversion takes
digits as unsigned, and the serializer validates each non-ASCII sequence
it decodes, throwing what basic_json::dump() throws for invalid UTF-8.
Parsed and edited documents are not affected.

The idea of images comes from zero-copy formats such as FlatBuffers and
YaFF, the check from FlatBuffers' Verifier; no code is taken from them.

Tests: round trips with every check (small documents, test files, large
objects, edited documents with every kind of edit), ownership, all
errors, one corruption per rejection branch of the check, and 12,000
seeded random corruptions, which must be rejected or read safely. The
fuzzer json_view_image_fuzzer uses each input as an image and as a JSON
text.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-29 14:03:20 +02:00
Niels Lohmann cc36e26254 Document insert() and erase() of editable json_documents
Add API reference pages for basic_json_document::insert and
basic_json_document::erase, matching the style of set.md and
push_back.md: signatures, parameters, return values, exception safety,
exceptions with their exact ids and messages, complexity, notes on
duplicate keys and view/iterator validity, and an example.

Add example programs basic_json_document__insert.cpp and
basic_json_document__erase.cpp with their expected output, each
comparing an edit on an editable document with the same edit on a
plain json value to show what is preserved: member order, the
spelling of untouched numbers, and, for insert, that a view taken
before the insert keeps referring to the same element after its index
shifts.

Register both new pages in mkdocs.yml, docSet.sql, and the member list
of basic_json_document/index.md, add cross-references to them from
set.md and push_back.md, and mention insert/erase in the "Editing a
document" section of features/json_view.md.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-29 14:03:19 +02:00
Niels Lohmann 3c5c000408 Address the clang-tidy findings of the structural edit tests
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-29 14:03:18 +02:00
Niels Lohmann 1e212bc50c Benchmark editing documents with json_view, yyjson, Boost.JSON, and json
bench_edit.cpp joins the comparison: parse, apply the same logical edits
with each library's own API (a handful at fixed places, or one in every
record), and serialize; all outputs must describe the same value.
compare.py builds and runs it with the other two programs.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-29 14:03:17 +02:00
Niels Lohmann 5d25f863c0 Add insert() and erase() to editable documents
- insert(array, index, value): insert before an element (index <= size)
- erase(object, key): remove all members with the key; returns their
  number
- erase(array, index): remove an element
- erase(json_pointer): remove the member or element a pointer names

The errors are those of basic_json (type_error.307/309,
out_of_range.401/403/405). A view of an erased value keeps its last value,
and views of other values keep referring to them when elements move.

Tests: the differential test now also inserts and erases members and
elements, directly and through JSON pointers; plus the errors, views
across inserts and erasures, duplicate keys, and large objects.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-29 14:03:16 +02:00
Niels Lohmann ada8f17230 Document how editable json_documents use the node index
The node index section on the architecture page says how edits are kept
(the edit buffer, moved elements, links, and new nodes), and the feature
page links to it.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-29 14:03:15 +02:00
Niels Lohmann 266c2249fe Cover the edit storage in tests
Test edits of an empty document and assignments through a view of a
value that is no longer part of the document; copy the entries of a
block through deref() (one path for links and values); mark the
4 GiB limit and the returns of find_parent() that no document reaches.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-29 14:03:15 +02:00
Niels Lohmann 2e5419eebc Label the pages and examples of the editable aliases
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-29 14:03:14 +02:00
Niels Lohmann 7e577fb2d4 Document editable json_documents
- API pages for set and push_back of basic_json_document and for the
  editable aliases; the class overviews name the Editable parameter
- type_error.319 on the exceptions page
- the feature page explains editing, and the examples show when it
  helps: a configuration file changed without reformatting it

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-29 14:03:13 +02:00
Niels Lohmann baf4ccfda8 Address the clang-tidy findings of the editable documents
Pick the overloads of encode() with a first_true trait instead of
nested conditionals, name the pointer type in the copies of links,
mark the owning pointers of the edit storage, and compare doubles
by their bits in the tests.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-29 14:03:11 +02:00
Niels Lohmann 76897a9b9b Add editable documents with set() and push_back()
basic_json_document<BasicJsonType, true> (json_editable_document,
ordered_json_editable_document) can be edited:

- set(view, value): replace a value
- set(object, key, value): assign a member, or add it (a null becomes an
  object); with duplicate keys, the first is assigned and the others go
- set(array, index, value): assign an element
- set(json_pointer, value): the member, element, or ("-", or the size of
  the array) the end of an array a pointer names
- push_back(array, value): append (a null becomes an array)

Values are views (of any document, copied), BasicJsonType values, and
everything BasicJsonType can be constructed from. The source text is never
written, and the parsed index never moves: new values and element
sequences go to storage owned by the document (edit_storage.hpp), so views
stay valid, and a view keeps referring to its value (after an assignment,
it sees the new one). Read-only documents are unchanged; editing one does
not compile.

Errors are those of basic_json where the operation corresponds
(type_error.305/308, out_of_range.401/403/405, parse_error.106/109); a view
of another document is invalid_iterator.202. Strings are checked for UTF-8
when they enter the document, with the type_error.316 that
basic_json::dump() throws for the same string, so that a document only
holds valid UTF-8. Binary values cannot be stored (the new
type_error.319), and edits of 4 GiB or more end with out_of_range.416.
Views of editable and read-only documents compare with each other.

Tests (unit-json_view_edit.cpp): random assignments, member and element
changes, copies within and between documents, and pushes, applied to an
ordered_json_editable_document and to the ordered_json value; after every
edit both must serialize (also indented and with ensure_ascii),
materialize, compare, and read back the same. Further: the errors, strings
that stay valid while the edit arena grows, numbers (NaN, infinities,
extremes; number_format::source), nulls that become containers, the root
replaced, duplicate keys, values of other documents, large objects, and
documents reused with read().

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-29 14:03:11 +02:00
Niels Lohmann 2b1a720fc0 Walk the index of json_view through a navigation policy
Preparation for editable documents, without a change in behavior: views and
documents get a template parameter Editable (false by default), and every
walk over the index (iterators, lookups, dump(), materialize()) goes
through detail::view::navigation<Editable>. For read-only documents it is
the plain node array, as before, so they compile without any of the edit
handling. For editable documents it also follows the representation of
edits, which this commit defines:

- node flags `edited` (a string or number token in the edit arena),
  `moved` (the elements of an array/object live in a separate sequence),
  and `is_new` (no source position), and link nodes (kind_link) that
  stand for a value stored elsewhere
- document_data::edit_state: the moved sequences, the storage of new
  values, and the edit arena

materialize() now keeps a frame per open container instead of returning
to the end of a closed one, as the serializer does, so that it can
follow moved sequences. Floats whose token lives in the edit arena (also
"nan", "inf", "-inf") are converted out of line. dump() copies only
strings of the source without escaping, and shrink_to_fit() leaves the
node array in place once there are edits, as they link into it.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-29 14:03:10 +02:00
70 changed files with 10507 additions and 661 deletions
+3 -3
View File
@@ -51,13 +51,13 @@ labels:
- "include/nlohmann/detail/view/.*"
- "single_include/nlohmann/json_view\\.hpp"
- "tests/src/unit-json_view.*"
- "tests/src/fuzzer-parse_json_view\\.cpp"
- "tests/src/fuzzer-(parse_json_view|json_view_image)\\.cpp"
- "tests/benchmarks/json_view/.*"
- "tools/amalgamate/config_json_view\\.json"
- "docs/mkdocs/docs/features/json_view\\.md"
- "docs/mkdocs/docs/api/basic_json_(document|view)/.*"
- "docs/mkdocs/docs/api/(ordered_)?json_(document|view)\\.md"
- "docs/mkdocs/docs/examples/(basic_json_(document|view)__|(ordered_)?json_(document|view)).*"
- "docs/mkdocs/docs/api/(ordered_)?json_(editable_)?(document|view)\\.md"
- "docs/mkdocs/docs/examples/(basic_json_(document|view)__|(ordered_)?json_(editable_)?(document|view)).*"
- "tests/benchmarks/src/benchmarks_view\\.cpp"
- label: "aspect: json_view"
+4
View File
@@ -26,6 +26,7 @@ cc_library(
"include/nlohmann/detail/conversions/from_json.hpp",
"include/nlohmann/detail/conversions/to_chars.hpp",
"include/nlohmann/detail/conversions/to_json.hpp",
"include/nlohmann/detail/conversions/zmij.hpp",
"include/nlohmann/detail/exceptions.hpp",
"include/nlohmann/detail/hash.hpp",
"include/nlohmann/detail/input/binary_reader.hpp",
@@ -69,7 +70,10 @@ cc_library(
"include/nlohmann/detail/view/builder.hpp",
"include/nlohmann/detail/view/compare.hpp",
"include/nlohmann/detail/view/document_data.hpp",
"include/nlohmann/detail/view/edit.hpp",
"include/nlohmann/detail/view/edit_storage.hpp",
"include/nlohmann/detail/view/errors.hpp",
"include/nlohmann/detail/view/image.hpp",
"include/nlohmann/detail/view/input.hpp",
"include/nlohmann/detail/view/iterator.hpp",
"include/nlohmann/detail/view/lookup.hpp",
+1
View File
@@ -1401,6 +1401,7 @@ THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR I
- The class contains the UTF-8 Decoder from Bjoern Hoehrmann which is licensed under the [MIT License](https://opensource.org/licenses/MIT) (see above). Copyright &copy; 2008-2009 [Björn Hoehrmann](https://bjoern.hoehrmann.de/) <bjoern@hoehrmann.de>
- The class contains a slightly modified version of the Grisu2 algorithm from Florian Loitsch which is licensed under the [MIT License](https://opensource.org/licenses/MIT) (see above). Copyright &copy; 2009 [Florian Loitsch](https://florian.loitsch.com/)
- The class contains a port of the shortest double-to-decimal conversion of [Żmij](https://github.com/vitaut/zmij) by Victor Zverovich, which is licensed under the [MIT License](https://opensource.org/licenses/MIT) (see above). Copyright &copy; 2025 [Victor Zverovich](https://github.com/vitaut)
- The class contains a copy of [Hedley](https://nemequ.github.io/hedley/) from Evan Nemerson which is licensed as [CC0-1.0](https://creativecommons.org/publicdomain/zero/1.0/).
- The class contains parts of [Google Abseil](https://github.com/abseil/abseil-cpp) which is licensed under the [Apache 2.0 License](https://opensource.org/licenses/Apache-2.0).
- The class contains an adapted version of the Eisel-Lemire algorithm and its table of powers of five from [fast_float](https://github.com/fastfloat/fast_float) by Daniel Lemire and contributors, which is available under the [MIT License](https://opensource.org/licenses/MIT) (used here), the Apache 2.0 License, and the Boost Software License. Copyright &copy; 2021 The fast_float authors
+10
View File
@@ -131,14 +131,20 @@ INSERT INTO searchIndex(name, type, path) VALUES ('basic_json::~basic_json', 'Me
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json_document', 'Class', 'api/basic_json_document/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json_document::basic_json_document', 'Constructor', 'api/basic_json_document/basic_json_document/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json_document::accept', 'Function', 'api/basic_json_document/accept/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json_document::erase', 'Method', 'api/basic_json_document/erase/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json_document::insert', 'Method', 'api/basic_json_document/insert/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json_document::is_discarded', 'Method', 'api/basic_json_document/is_discarded/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json_document::load', 'Function', 'api/basic_json_document/load/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json_document::memory_usage', 'Method', 'api/basic_json_document/memory_usage/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json_document::node_count', 'Method', 'api/basic_json_document/node_count/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json_document::owns_source', 'Method', 'api/basic_json_document/owns_source/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json_document::parse', 'Function', 'api/basic_json_document/parse/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json_document::parse_copy', 'Function', 'api/basic_json_document/parse_copy/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json_document::push_back', 'Method', 'api/basic_json_document/push_back/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json_document::read', 'Method', 'api/basic_json_document/read/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json_document::root', 'Method', 'api/basic_json_document/root/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json_document::save', 'Method', 'api/basic_json_document/save/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json_document::set', 'Method', 'api/basic_json_document/set/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json_document::shrink_to_fit', 'Method', 'api/basic_json_document/shrink_to_fit/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json_document::source', 'Method', 'api/basic_json_document/source/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json_view', 'Class', 'api/basic_json_view/index.html');
@@ -187,6 +193,8 @@ INSERT INTO searchIndex(name, type, path) VALUES ('basic_json_view::type_name',
INSERT INTO searchIndex(name, type, path) VALUES ('basic_json_view::value', 'Method', 'api/basic_json_view/value/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('json', 'Class', 'api/json/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('json_document', 'Class', 'api/json_document/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('json_editable_document', 'Class', 'api/json_editable_document/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('json_editable_view', 'Class', 'api/json_editable_view/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('json_view', 'Class', 'api/json_view/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('json_pointer', 'Class', 'api/json_pointer/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('json_pointer::back', 'Method', 'api/json_pointer/back/index.html');
@@ -222,6 +230,8 @@ INSERT INTO searchIndex(name, type, path) VALUES ('operator<<', 'Operator', 'api
INSERT INTO searchIndex(name, type, path) VALUES ('operator>>', 'Operator', 'api/operator_gtgt/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('ordered_json', 'Class', 'api/ordered_json/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('ordered_json_document', 'Class', 'api/ordered_json_document/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('ordered_json_editable_document', 'Class', 'api/ordered_json_editable_document/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('ordered_json_editable_view', 'Class', 'api/ordered_json_editable_view/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('ordered_json_view', 'Class', 'api/ordered_json_view/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('ordered_map', 'Class', 'api/ordered_map/index.html');
INSERT INTO searchIndex(name, type, path) VALUES ('std::hash<basic_json>', 'Class', 'api/basic_json/std_hash/index.html');
+5
View File
@@ -60,6 +60,9 @@ Linear.
## Notes
Floating-point numbers are written with the fewest digits that read back as the same value (for `#!cpp double`; see
[number handling](../../features/types/number_handling.md#number-serialization)).
Binary values are serialized as an object containing two keys:
- "bytes": an array of bytes as integers
@@ -96,3 +99,5 @@ Binary values are serialized as an object containing two keys:
- Indentation character `indent_char`, option `ensure_ascii` and exceptions added in version 3.0.0.
- Error handlers added in version 3.4.0.
- Serialization of binary values added in version 3.8.0.
- Doubles are written with the shortest digits (Żmij instead of Grisu2) since version 3.13.0; about 0.1% of doubles are
written differently, most of them with fewer digits.
@@ -0,0 +1,128 @@
# <small>nlohmann::basic_json_document::</small>erase
```cpp
// (1)
std::size_t erase(view_type object, string_view_t key);
// (2)
template<typename I>
void erase(view_type array, I idx);
// (3)
std::size_t erase(const json_pointer& ptr);
```
Only an **editable** document (`#!cpp Editable == true`, e.g. [`json_editable_document`](../json_editable_document.md))
has `erase`; calling it on a read-only `basic_json_document` fails to compile (`#!cpp static_assert`).
1. Removes every member of `object` whose key is `key` (see [Notes](#notes) on duplicate keys) and returns how many
were removed; `#!cpp 0` if `object` has no member with this key.
2. Removes the element at index `idx` of `array`, which must already exist (`#!cpp idx < array.size()`).
3. Removes the value the JSON pointer `ptr` refers to, relative to [`root()`](root.md), and returns how many values
were removed: the *parent* of the target must already exist, and the target itself is removed as in 1. (an object
member; `#!cpp 0` or more) or 2. (an array element; always `#!cpp 1`). `ptr` must not be empty -- [`root()`](root.md)
itself cannot be erased.
## Template parameters
`I`
: an integral type other than `#!cpp bool`, deduced (overloads taking a `#!cpp bool` or a non-integral type for
`idx` do not participate in overload resolution).
## Parameters
`object` (in)
: the object to remove a member of
`array` (in)
: the array to remove an element of
`key` (in)
: the key of the member(s) to remove
`idx` (in)
: the index of the element to remove; a negative value throws (see [Exceptions](#exceptions))
`ptr` (in)
: a JSON pointer to the value to remove, relative to `root()`
## Return value
1. the number of removed members (`#!cpp 0` if `object` had none with this `key`)
2. (nothing)
3. the number of removed values (`#!cpp 0` or more for an object member, always `#!cpp 1` for an array element)
## Exceptions
1. Throws [`type_error.307`](../../home/exceptions.md#jsonexceptiontype_error307) if `object` is not an object -- the
same message [`BasicJsonType::erase`](../basic_json/erase.md) throws for the same type.
2. Throws `type_error.307` if `array` is not an array. Throws
[`out_of_range.401`](../../home/exceptions.md#jsonexceptionout_of_range401) if `idx` is negative, or if
`#!cpp idx >= array.size()`.
3. Throws [`out_of_range.405`](../../home/exceptions.md#jsonexceptionout_of_range405) ("JSON pointer has no parent")
if `ptr` is empty. Throws what [`at`](../basic_json_view/at.md) throws (overload 3) for resolving `ptr`'s parent.
For the last reference token itself: if the parent is an array, throws what 2. throws for an index that is out of
range, or, for a token that is not a valid array index,
[`parse_error.106`](../../home/exceptions.md#jsonexceptionparse_error106) (a leading `#!cpp '0'`),
[`parse_error.109`](../../home/exceptions.md#jsonexceptionparse_error109) (not a number),
[`out_of_range.410`](../../home/exceptions.md#jsonexceptionout_of_range410) (too large for `size_type`), or
[`out_of_range.404`](../../home/exceptions.md#jsonexceptionout_of_range404) (an empty token); otherwise (an
object, or a primitive value the pointer's parent resolves to) throws what 1. throws.
Every overload also throws [`invalid_iterator.202`](../../home/exceptions.md#jsonexceptioninvalid_iterator202) ("view
does not belong to this document") if `object`/`array` is a [discarded](../basic_json_view/is_discarded.md) view or a
view of a *different* document (overloads 1-2 only; overload 3 always starts from this document's own
[`root()`](root.md)).
## Complexity
1. Linear in the number of members of `object`.
2. Linear in the number of elements of `array` at or after `idx` (they move one slot over).
3. Linear in the number of reference tokens of `ptr` and, for each token, in the number of members of the object at
that level or the index into the array (as [`at`](../basic_json_view/at.md)), plus the complexity of 1. or 2. for
the last token.
## Notes
!!! info "Duplicate keys"
Overload 1. removes *every* member with `key`, not just the first -- unlike [`set`](set.md), which assigns the
first occurrence and drops the rest. This is why it returns a count rather than a single view: there may be
more than one member removed, or none.
Like [`set`](set.md) and [`push_back`](push_back.md), `erase` never moves an element's *value*: a view still
referring to a removed member or element keeps showing what it last held (see [Edits](index.md#edits)) -- it just no
longer appears when `array`/`object` is read, dumped, or iterated. Removing an element of `array` (2.) does shift the
*links* to the elements after it, the same way `insert`, `set`, or `push_back` on the same array would; any iterator
already taken over `array`/`object` is invalidated by an erase, since it was walking the old layout.
## Examples
??? example
The example below drops a deprecated field and a decommissioned entry from a configuration document -- using all
three overloads -- and shows what stays intact that would not with a plain `json`/`ordered_json` value: the order
of the fields around the ones removed, and the exact spelling of a number that was never touched.
```cpp
--8<-- "examples/basic_json_document__erase.cpp"
```
Output:
```json
--8<-- "examples/basic_json_document__erase.output"
```
## See also
- [insert](insert.md) - insert an element into an array
- [set](set.md) - replace a value, or set an object member, an array element, or the value a JSON pointer refers to
- [push_back](push_back.md) - append to an array
- [root](root.md) - the view of the root value, the starting point of overload 3
- [`BasicJsonType::erase`](../basic_json/erase.md) - the corresponding function of `basic_json`
- [Edits](index.md#edits) - what an edit guarantees, for every overload
## Version history
- Added in version 3.13.0.
@@ -3,7 +3,7 @@
<small>Defined in header `<nlohmann/json_view.hpp>`</small>
```cpp
template<typename BasicJsonType>
template<typename BasicJsonType, bool Editable = false>
class basic_json_document;
```
@@ -19,6 +19,11 @@ it (a copy, or an rvalue `#!cpp std::string` that was moved in); see [`owns_sour
is move-only: copying a document would either duplicate a potentially large index and text, or leave two documents
claiming to borrow the same buffer, so it is disabled.
With `#!cpp Editable == true`, the document also offers [`set`](set.md), [`push_back`](push_back.md),
[`insert`](insert.md), and [`erase`](erase.md) to change values in place, see [Edits](#edits) below. The source text
itself is never written; a read-only document (`#!cpp Editable == false`, the default) does not carry any of the
bookkeeping edits need, and calling any of them on one fails to compile (`#!cpp static_assert`).
## Template parameters
`BasicJsonType`
@@ -26,23 +31,37 @@ claiming to borrow the same buffer, so it is disabled.
[`ordered_json`](../ordered_json.md). Only 64-bit `number_integer_t`/`number_unsigned_t` types are supported; this
is checked with a `static_assert`.
`Editable`
: whether the document supports [`set`](set.md), [`push_back`](push_back.md), [`insert`](insert.md), and
[`erase`](erase.md) (optional, `#!cpp false` by default). See [Edits](#edits) below.
## Specializations
- [**json_document**](../json_document.md) - documents of the default specialization [`json`](../json.md)
- [**ordered_json_document**](../ordered_json_document.md) - documents of [`ordered_json`](../ordered_json.md)
- [**json_document**](../json_document.md) - read-only documents of the default specialization [`json`](../json.md)
- [**ordered_json_document**](../ordered_json_document.md) - read-only documents of
[`ordered_json`](../ordered_json.md)
- [**json_editable_document**](../json_editable_document.md) - editable documents of [`json`](../json.md)
- [**ordered_json_editable_document**](../ordered_json_editable_document.md) - editable documents of
[`ordered_json`](../ordered_json.md)
## Member types
- **view_type** - the type of view returned by [`root()`](root.md) (`#!cpp basic_json_view<BasicJsonType>`)
- **view_type** - the type of view returned by [`root()`](root.md) (`#!cpp basic_json_view<BasicJsonType, Editable>`)
- **value_t** - the JSON type enumeration, see [`basic_json::value_t`](../basic_json/value_t.md)
## Member functions
- [(constructor)](basic_json_document.md)
### Parsing
- [**parse**](parse.md) (_static_) - deserialize from a compatible input, borrowing or owning it as appropriate
- [**parse_copy**](parse_copy.md) (_static_) - deserialize a copy of a compatible input
- [**accept**](accept.md) (_static_) - check whether the input is valid JSON
- [**read**](read.md) - (re-)parse into this document, reusing its memory
### Access
- [**root**](root.md) - the view of the root value
- [**is_discarded**](is_discarded.md) - return whether the last parse failed
- [**source**](source.md) - the parsed text
@@ -51,6 +70,49 @@ claiming to borrow the same buffer, so it is disabled.
- [**memory_usage**](memory_usage.md) - the number of bytes held by the document
- [**shrink_to_fit**](shrink_to_fit.md) - release unused index capacity
### Images
- [**save**](save.md) - the document as an image that `load()` reads without parsing
- [**load**](load.md) (_static_) - read an image written by `save()`
### Edits
- [**set**](set.md) - replace a value, or set an object member, an array element, or the value a JSON pointer refers
to (`#!cpp Editable` documents only)
- [**push_back**](push_back.md) - append to an array (`#!cpp Editable` documents only)
- [**insert**](insert.md) - insert an element into an array before a given position (`#!cpp Editable` documents only)
- [**erase**](erase.md) - remove an object member, an array element, or the value a JSON pointer refers to
(`#!cpp Editable` documents only)
## Edits
An editable document (`#!cpp Editable == true`) can be changed after parsing, with [`set`](set.md),
[`push_back`](push_back.md), [`insert`](insert.md), and [`erase`](erase.md);
[`json_editable_document`](../json_editable_document.md) and
[`ordered_json_editable_document`](../ordered_json_editable_document.md) are the corresponding specializations. A few
points apply to every edit:
- **The source text is never written**, and the parsed index never moves: every value keeps the node it was parsed
into, so [views](../basic_json_view/index.md) taken before an edit stay valid, including
[`root()`](root.md). New values (and the element sequences of an edited array/object) go to storage owned by the
document, allocated on demand.
- **A view keeps referring to the same value.** After [`set`](set.md) replaces the value a view refers to, that view
sees the new value; a view of a value that a later edit replaces or drops keeps showing what it last held. An edit
of an array or object, however, **invalidates the iterators taken over it** (its members may now live in a
different sequence), and a string obtained with [`get_string()`](../basic_json_view/get_string.md) stays valid even
as further edits happen (earlier buffers of edited text are kept alive, not overwritten).
- **Values are accepted three ways:** a [`basic_json_view`](../basic_json_view/index.md) of *any* document
(read-only or editable; it is copied, nothing is shared with the source document), a `BasicJsonType` value, or
anything `BasicJsonType` can be constructed from (numbers, strings, `#!cpp bool`, `#!cpp nullptr`, containers, ...).
- [`dump()`](../basic_json_view/dump.md) writes an edited document with members in document order, new members at
the end, and, with [`number_format::source`](../basic_json_view/number_format.md), keeps the spelling of every
number that was not itself edited -- see [Editing a document](../../features/json_view.md#editing-a-document) for
why this matters.
- [`read()`](read.md) discards all edits, [`shrink_to_fit()`](shrink_to_fit.md) does not move the node index once
there are edits, and [`memory_usage()`](memory_usage.md) includes the memory edits use.
[`source_offset()`](../basic_json_view/source_offset.md) of a value introduced by an edit is
`#!cpp static_cast<std::size_t>(-1)`, the same value it reports for a decoded string.
## Version history
- Added in version 3.13.0.
@@ -0,0 +1,114 @@
# <small>nlohmann::basic_json_document::</small>insert
```cpp
template<typename I, typename V>
view_type insert(view_type array, I idx, V&& value);
```
Only an **editable** document (`#!cpp Editable == true`, e.g. [`json_editable_document`](../json_editable_document.md))
has `insert`; calling it on a read-only `basic_json_document` fails to compile (`#!cpp static_assert`).
Inserts `value` into `array` as a new element before position `idx`, which must not be past the end
(`#!cpp idx <= array.size()`; `#!cpp idx == array.size()` appends, like [`push_back`](push_back.md)). Unlike
[`push_back`](push_back.md), a [null](../basic_json_view/is_null.md) `array` does *not* first become an empty array:
`array` must already be an array.
`value` is accepted three ways: a [`basic_json_view`](../basic_json_view/index.md) of *any* document -- read-only or
editable, and it does not have to be `array`'s own document -- which is copied so that nothing is shared with the
source document afterward; a `BasicJsonType` value; or anything `BasicJsonType` can be constructed from (numbers,
strings, `#!cpp bool`, `#!cpp nullptr`, containers, ...).
## Template parameters
`I`
: an integral type other than `#!cpp bool`, deduced (overloads taking a `#!cpp bool` or a non-integral type for
`idx` do not participate in overload resolution).
`V`
: the type of `value`, deduced; see above for what is accepted.
## Parameters
`array` (in)
: the array to insert into
`idx` (in)
: the position to insert `value` before; a negative value throws (see [Exceptions](#exceptions))
`value` (in)
: the value to insert
## Return value
a view of the new element, now holding `value`
## Exception safety
Basic exception safety: `value` is fully encoded -- including the checks below -- into storage owned by the document
before anything already reachable from [`root()`](root.md) is touched, so a failure while encoding `value` (an
invalid argument, or `#!cpp std::bad_alloc`) leaves the document completely unchanged, other than memory allocated
for the encoding that is not reclaimed. A failure of a later allocation -- while `array` switches from its parsed
layout to a growable block, or while that block grows, see [Notes](#notes) -- can still leave `array` already
switched to that layout even though `value` itself was not inserted.
## Exceptions
Throws [`type_error.309`](../../home/exceptions.md#jsonexceptiontype_error309) if `array` is not an array -- the same
message [`BasicJsonType::insert`](../basic_json/insert.md) throws for the same type; a null `array` throws this too
(see above). Throws [`out_of_range.401`](../../home/exceptions.md#jsonexceptionout_of_range401) if `idx` is negative,
or if `#!cpp idx > array.size()`. Throws
[`invalid_iterator.202`](../../home/exceptions.md#jsonexceptioninvalid_iterator202) ("view does not belong to this
document") if `array` is a [discarded](../basic_json_view/is_discarded.md) view or a view of a *different* document.
Throws [`type_error.302`](../../home/exceptions.md#jsonexceptiontype_error302) if `value` is a
[discarded](../basic_json_view/is_discarded.md) view or a [discarded](../basic_json/is_discarded.md) `BasicJsonType`
value, and [`type_error.319`](../../home/exceptions.md#jsonexceptiontype_error319) if `value` is (or contains) a
binary value -- `BasicJsonType` can hold one, but a `json_document` cannot. Throws
[`type_error.316`](../../home/exceptions.md#jsonexceptiontype_error316) if `value` is (or contains) a string that is
not valid UTF-8, with the same message [`BasicJsonType::dump()`](../basic_json/dump.md) gives for that string.
## Complexity
Linear in the number of elements of `array` at or after `idx` (they move one slot over), plus time linear in the
size of `value` to encode it into the document's storage (constant for a scalar, linear in the number of nested
values for an array or object): like [`push_back`](push_back.md), the elements of `array` move to a growable block
of links the first time it is inserted into (or [`set`](set.md)/[`push_back`](push_back.md) on), and that block
grows in amortized constant time; inserting before the end within that block still shifts every later element.
## Notes
Like [`set`](set.md) on a member or an element, `insert` never moves an existing *element's value* -- only where
`array`'s *links* to its elements live -- so a view of an existing element of `array` stays valid across an
`insert`, and keeps referring to the same element even though its index shifts. Any iterator already taken over
`array` is invalidated, since it was walking the old layout. See [Edits](index.md#edits) for what stays valid across
an edit in general.
## Examples
??? example
The example below inserts a step into the middle of a deployment plan, without touching the steps that come
after it, and shows that a view taken before the insert keeps referring to the same element even though its
index shifts -- something a plain `json`/`ordered_json` array, or its `std::vector`-based storage, has no
equivalent for.
```cpp
--8<-- "examples/basic_json_document__insert.cpp"
```
Output:
```json
--8<-- "examples/basic_json_document__insert.output"
```
## See also
- [push_back](push_back.md) - append to an array
- [erase](erase.md) - remove an object member, an array element, or the value a JSON pointer refers to
- [set](set.md) - replace a value, or set an object member, an array element, or the value a JSON pointer refers to
- [`BasicJsonType::insert`](../basic_json/insert.md) - the corresponding function of `basic_json`
- [Edits](index.md#edits) - what an edit guarantees, for every overload
## Version history
- Added in version 3.13.0.
@@ -0,0 +1,169 @@
# <small>nlohmann::basic_json_document::</small>load
```cpp
// (1)
static basic_json_document load(const std::uint8_t* image, std::size_t size,
const image_check check = image_check::full);
// (2)
static basic_json_document load(const std::vector<std::uint8_t>& image,
const image_check check = image_check::full);
// (3)
static basic_json_document load(std::vector<std::uint8_t>&& image,
const image_check check = image_check::full);
```
1. Reads an image [`save()`](save.md) wrote, from a pointer and a byte count. The image is **borrowed**: `image`
must stay alive and unchanged for as long as the returned document, and any view taken from it, is used.
2. Reads an image from a `#!cpp std::vector`. Also **borrowed** -- equivalent to overload 1 called with
`#!cpp image.data()` and `#!cpp image.size()`.
3. Reads an image, keeping the vector instead of copying it: `image` is moved into the document (no copy), which
then owns it for as long as it needs the text and the decoded strings. [`owns_source()`](owns_source.md) is
`#!cpp true` afterward.
In every overload, the node index is copied into storage the document itself owns -- so that it is properly aligned,
and, for an [editable](index.md#edits) document, can be edited -- while the text and the decoded strings stay in
`image`. The hash indexes [large objects](../../features/json_view.md) use for lookup are rebuilt, exactly as after
parsing.
## Parameters
`image` (in)
: the image [`save()`](save.md) wrote (overloads 1 and 2), or one to take ownership of (overload 3)
`size` (in)
: the number of bytes at `image` (overload 1)
`check` (in)
: how thoroughly to validate `image` before trusting it; see [`image_check`](#image_check) below (optional,
`#!cpp image_check::full` by default)
## Return value
The document read from the image.
## Exception safety
Overloads 1 and 2 give the strong guarantee: `image` is only read, never written, so a thrown exception leaves the
caller's buffer untouched.
Overload 3 moves `image` into the document *before* validating it, so that a good image is kept without a copy. If
loading then fails, the partially built document -- and the vector now inside it -- is discarded along with the
exception, and `image` itself is left **empty**, not restored to what was passed in. Move a copy in instead, or
validate with overload 2 first, if the original vector must survive a failed load.
## Exceptions
On a big-endian target, throws [`type_error.320`](../../home/exceptions.md#jsonexceptiontype_error320) -- the same
exception [`save()`](save.md#exceptions) throws there, since the image format is little-endian only.
Otherwise throws [`parse_error.116`](../../home/exceptions.md#jsonexceptionparse_error116) if `image` is not one
`save()` could have written, or fails the requested `check`:
| message | when |
|------------------------|------------------------------------------------------------------------------------------------------|
| `too short` | `image` is `#!cpp nullptr`, or `size` is smaller than the 64-byte header |
| `unknown format` | the header's magic bytes or version do not match, or a reserved header field is not zero |
| `sizes out of range` | the node count, text size, or decoded-string size the header describes does not fit `size`, or the `#!cpp '\0'` after the text or after the decoded strings is missing |
| `the check failed` | `check` is not `#!cpp image_check::none`, and the image fails it -- see [`image_check`](#image_check) |
!!! failure "Example messages"
```
[json.exception.parse_error.116] parse error: invalid json_document image: too short
```
```
[json.exception.parse_error.116] parse error: invalid json_document image: unknown format
```
```
[json.exception.parse_error.116] parse error: invalid json_document image: sizes out of range
```
```
[json.exception.parse_error.116] parse error: invalid json_document image: the check failed
```
## Complexity
Linear in the number of nodes, which are always copied into the document. With `#!cpp check == image_check::full`,
additionally linear in the combined length of the text and the decoded strings; `#!cpp image_check::bounds` and
`#!cpp image_check::none` do not read them.
## `image_check`
```cpp
using image_check = detail::view::image_check;
enum class image_check
{
full,
bounds,
none
};
```
How thoroughly `load()` validates `image` before trusting it.
| value | checks | guarantees |
|----------|--------------------------------------------------------------------------------------------------------------|------------|
| `full` | everything the parser itself guarantees: structure and bounds; that every string is valid UTF-8 (and, for a string still in the source text, that it contains no quote, backslash, or control character); and that every number token is well-formed and matches the value stored for it | reading and serializing a checked image is safe and always produces valid JSON, exactly as for a parsed document |
| `bounds` | structure and bounds only -- that every offset and count in the node index stays inside the image | reading and serializing stay memory-safe, but a crafted image can hold strings that are not valid UTF-8 or that serialize to invalid JSON ([`dump()`](../basic_json_view/dump.md) writes them unchanged or throws [`type_error.316`](../../home/exceptions.md#jsonexceptiontype_error316)), and numbers whose values differ from their text |
| `none` | nothing | images from a trusted source only -- reading a damaged image is undefined behavior |
`full` is the default and the right choice for an image from anything you do not fully control -- a file, a cache
shared with other processes, a peer on the network. `bounds` skips scanning the text and the decoded strings, so it
fits a cache your own process just wrote and reads straight back, where damage would mean a bug or a hardware fault
rather than adversarial input; it still cannot crash or read out of bounds. `none` skips validation entirely and
should only be used for an image you trust as much as your own memory.
## Notes
**Lifetime.** Overloads 1 and 2 borrow `image`: it must stay alive and byte-for-byte unchanged for as long as the
returned document, and any [view](../basic_json_view/index.md) taken from it, is used -- exactly like a document
[`parse()`](parse.md) borrowed its input for. Overload 3 avoids this by keeping the vector itself; see
[`owns_source`](owns_source.md).
!!! warning "Experimental"
The image format is versioned but not yet stable, and may change in an incompatible way before it is declared
stable; `load()` already rejects an image written by a different format version with `parse_error.116`
("unknown format"). Use images to cache a document within one build of the library, or to hand one to another
process running the *same* build on the *same* (little-endian) machine -- not as a long-term storage format.
**What `image_check::bounds` does not guarantee.** A bounds-checked image can never make `load()`,
[`root()`](root.md), element access, or [`materialize()`](../basic_json_view/materialize.md) read outside the image,
so those stay safe on a damaged one. It does *not* guarantee that the image describes valid JSON: a string
that a `full` check would have rejected can make [`dump()`](../basic_json_view/dump.md) write invalid UTF-8 or invalid
JSON, or throw `type_error.316`, and a number can read back with a value that does not match how it is spelled. Reserve `bounds` for images you already trust to be well-formed, and use it
only to skip the extra scan.
## Examples
??? example "Caching a document, ownership, and a rejected image"
The example below saves a parsed document as an image, checks that `load()` reproduces the original
[`dump()`](../basic_json_view/dump.md) without parsing, and shows the difference between
`load(std::move(image))` (owned) and `load(image)` (borrowed). It then damages one byte of the image and shows
`image_check::full` rejecting it with `parse_error.116`, while `image_check::bounds` -- meant for a cache the
process already trusts -- still reads it without going out of bounds.
```cpp
--8<-- "examples/basic_json_document__load.cpp"
```
Output:
```json
--8<-- "examples/basic_json_document__load.output"
```
## See also
- [save](save.md) - write the document as an image
- [owns_source](owns_source.md) - return whether the document holds its own copy of the text
- [parse](parse.md) - deserialize from JSON text instead of an image
- [Images](../../features/json_view.md#images) - why and when to use images
## Version history
- Added in version 3.13.0.
@@ -132,6 +132,7 @@ integer type becomes a floating-point value.
- [accept](accept.md) - check whether the input is valid JSON
- [read](read.md) - (re-)parse into this document, reusing its memory
- [owns_source](owns_source.md) - return whether the document holds its own copy of the text
- [load](load.md) - read a document from an image instead of parsing JSON text
- [`BasicJsonType::parse`](../basic_json/parse.md) - the corresponding function of `basic_json`
## Version history
@@ -0,0 +1,102 @@
# <small>nlohmann::basic_json_document::</small>push_back
```cpp
template<typename V>
view_type push_back(view_type array, V&& value);
```
Appends `value` as a new last element of `array`. A [null](../basic_json_view/is_null.md) `array` first becomes an
empty array, the same way [`set`](set.md) turns a null `object` into an empty object.
`value` is accepted three ways: a [`basic_json_view`](../basic_json_view/index.md) of *any* document -- read-only or
editable, and it does not have to be `array`'s own document -- which is copied so that nothing is shared with the
source document afterward; a `BasicJsonType` value; or anything `BasicJsonType` can be constructed from (numbers,
strings, `#!cpp bool`, `#!cpp nullptr`, containers, ...).
Only an **editable** document (`#!cpp Editable == true`, e.g. [`json_editable_document`](../json_editable_document.md))
has `push_back`; calling it on a read-only `basic_json_document` fails to compile (`#!cpp static_assert`).
## Template parameters
`V`
: the type of `value`, deduced; see above for what is accepted.
## Parameters
`array` (in)
: the array (or null value) to append to
`value` (in)
: the value to append
## Return value
a view of the new last element of `array`, now holding `value`
## Exception safety
Basic exception safety: `value` is fully encoded -- including the checks below -- into storage owned by the document
before anything already reachable from [`root()`](root.md) is touched, so a failure while encoding `value` (an
invalid argument, or `#!cpp std::bad_alloc`) leaves the document completely unchanged, other than memory allocated
for the encoding that is not reclaimed. A failure of a later allocation -- while `array` switches from its parsed
layout to a growable block, or while that block grows, see [Notes](#notes) -- can still leave a partial effect, such
as a null `array` argument already turned into an empty array even though `value` itself was not appended.
## Exceptions
Throws [`type_error.308`](../../home/exceptions.md#jsonexceptiontype_error308) if `array` is neither an array nor
null -- the same message [`BasicJsonType::push_back`](../basic_json/push_back.md) throws for the same type. Throws
[`invalid_iterator.202`](../../home/exceptions.md#jsonexceptioninvalid_iterator202) ("view does not belong to this
document") if `array` is a [discarded](../basic_json_view/is_discarded.md) view or a view of a *different* document.
Throws [`type_error.302`](../../home/exceptions.md#jsonexceptiontype_error302) if `value` is a
[discarded](../basic_json_view/is_discarded.md) view or a [discarded](../basic_json/is_discarded.md) `BasicJsonType`
value, and [`type_error.319`](../../home/exceptions.md#jsonexceptiontype_error319) if `value` is (or contains) a
binary value -- `BasicJsonType` can hold one, but a `json_document` cannot. Throws
[`type_error.316`](../../home/exceptions.md#jsonexceptiontype_error316) if `value` is (or contains) a string that is
not valid UTF-8, with the same message [`BasicJsonType::dump()`](../basic_json/dump.md) gives for that string.
## Complexity
Amortized constant, plus time linear in the size of `value` to encode it into the document's storage (constant for
a scalar, linear in the number of nested values for an array or object): the elements of `array` move to a growable
block of links the first time it is appended to (or [`set`](set.md) on), and that block itself grows -- doubling its
capacity, so the cost of growing it amortizes to constant per element -- only once it runs out of room. See
[Notes](#notes).
## Notes
Like [`set`](set.md) on a member or an element, `push_back` never moves an existing element itself -- only where
`array`'s *links* to its elements live -- so a view of an existing element of `array` stays valid across a
`push_back`, but any iterator already taken over `array` is invalidated, since it was walking the old layout. See
[Edits](index.md#edits) for what stays valid across an edit in general.
## Examples
??? example
The example below appends records to an array one at a time, as they might arrive from a stream of events,
without ever building a `BasicJsonType` value for the array or for the records already in it, and shows that a
view taken from an earlier `push_back` still refers to the same element once later ones have run.
```cpp
--8<-- "examples/basic_json_document__push_back.cpp"
```
Output:
```json
--8<-- "examples/basic_json_document__push_back.output"
```
## See also
- [set](set.md) - replace a value, or set an object member, an array element, or the value a JSON pointer refers to
- [insert](insert.md) - insert an element into an array before a given position
- [erase](erase.md) - remove an object member, an array element, or the value a JSON pointer refers to
- [root](root.md) - the view of the root value
- [`BasicJsonType::push_back`](../basic_json/push_back.md) - the corresponding function of `basic_json`
- [Edits](index.md#edits) - what an edit guarantees, for every overload
## Version history
- Added in version 3.13.0.
@@ -70,6 +70,7 @@ own on the next, since ownership is decided freshly each time.
- [parse](parse.md) - deserialize from a compatible input
- [root](root.md) - the view of the root value
- [load](load.md) - read a document from an image instead of parsing JSON text
## Version history
@@ -0,0 +1,104 @@
# <small>nlohmann::basic_json_document::</small>save
```cpp
std::vector<std::uint8_t> save() const;
```
Writes the document as an *image*: a byte buffer that [`load`](load.md) reads back without parsing. The image holds
the node index, the source text (plus, for an edited document, the number tokens edits wrote), and the decoded
strings (plus the strings edits wrote) -- everything [`root()`](root.md) needs, with nothing left to parse.
An edited document is written in its *current* state, with its values in document order, the way the library's own
parser would have produced them for that JSON text: a member [`set`](set.md) added goes at the end, an
[`erase`](erase.md)d member leaves no trace, and a float that is not finite (NaN or positive/negative infinity)
becomes null, the same substitution [`dump()`](../basic_json_view/dump.md) makes. The same document always saves to
the same bytes -- also across `BasicJsonType` and `#!cpp Editable`, since the image reflects document order and
values only, not which specialization produced them.
## Return value
The image, as a `#!cpp std::vector<std::uint8_t>`. Pass it, or a pointer to its data together with its size, to
[`load`](load.md) to read the document back.
## Exception safety
Strong guarantee: `save()` does not modify `#!cpp *this` (it is `#!cpp const`), so if it throws, the document is left
exactly as it was, and the partially built image is discarded with the exception.
## Exceptions
Throws [`type_error.320`](../../home/exceptions.md#jsonexceptiontype_error320) if the document is
[discarded](is_discarded.md) -- a default-constructed document, or one a failed [`parse()`](parse.md)/
[`read()`](read.md) with `allow_exceptions == false` left discarded.
On a big-endian target, throws `type_error.320` with a different message instead: the image format is little-endian
only (see [Notes](#notes)).
Throws [`out_of_range.416`](../../home/exceptions.md#jsonexceptionout_of_range416) if the node count, the text, or
the decoded strings of the image would individually reach 4 GiB -- the same 32-bit offsets
[`parse()`](parse.md#exceptions) and, for edits, [`set`](set.md)/[`push_back`](push_back.md) are already limited to.
!!! failure "Example messages"
```
[json.exception.type_error.320] cannot save a discarded json_document
```
```
[json.exception.type_error.320] json_document images need a little-endian target
```
```
[json.exception.out_of_range.416] images of 4 GiB or more are not supported by json_document
```
## Complexity
Linear in the size of the document: the number of nodes, plus the length of the text and the decoded strings that end
up in the image.
## Notes
**Format.** The image begins with a 64-byte header (the magic bytes `#!cpp "NJVI"`, a version number, the node count,
and the sizes of the text and the decoded strings, all little-endian), followed by the nodes
([16 bytes each](../../home/architecture.md#node-index-of-json-views)), the text and a `#!cpp '\0'`, and the decoded
strings and a `#!cpp '\0'`. [`load`](load.md) checks the header, and the sizes it describes, before reading anything
else -- see [`load`'s Exceptions](load.md#exceptions).
!!! warning "Experimental"
The image format is versioned but not yet stable: it may change in an incompatible way before it is declared
stable. Use images to cache a document within one build of the library, or to hand one to another process running
the *same* build on the *same* (little-endian) machine -- not as a long-term storage format. Keep the original
JSON text if you need to read a saved document back with a future library version.
**Little-endian only.** The image is written as raw little-endian bytes, with no byte-swapping. `save()` (and
[`load`](load.md)) throw `type_error.320` on a big-endian target rather than silently produce bytes a big-endian
reader could not interpret correctly.
## Examples
??? example "Caching a parsed document as an image"
The example below saves a parsed configuration as an image -- the way a service might cache one to answer later
requests without parsing the text again -- and confirms that loading it back gives exactly the same result as
parsing did, and that saving is deterministic.
```cpp
--8<-- "examples/basic_json_document__save.cpp"
```
Output:
```json
--8<-- "examples/basic_json_document__save.output"
```
## See also
- [load](load.md) - read an image written by `save()`
- [owns_source](owns_source.md) - return whether the document holds its own copy of the text
- [`basic_json_view::dump`](../basic_json_view/dump.md) - serialize the document to JSON text instead of an image
- [Images](../../features/json_view.md#images) - why and when to use images
## Version history
- Added in version 3.13.0.
@@ -0,0 +1,194 @@
# <small>nlohmann::basic_json_document::</small>set
```cpp
// (1)
template<typename V>
view_type set(view_type target, V&& value);
// (2)
template<typename V>
view_type set(view_type object, string_view_t key, V&& value);
// (3)
template<typename I, typename V>
view_type set(view_type array, I idx, V&& value);
// (4)
template<typename V>
view_type set(const json_pointer& ptr, V&& value);
```
Only an **editable** document (`#!cpp Editable == true`, e.g. [`json_editable_document`](../json_editable_document.md))
has `set`; calling it on a read-only `basic_json_document` fails to compile (`#!cpp static_assert`).
1. Replaces the value `target` refers to with `value`.
2. Sets the member `key` of the object `object` to `value`: assigns it if `object` already has a member with this
key -- the first one, should the key occur more than once, and the later duplicates are then dropped (see the
[Notes](#notes) below) -- or appends a new member at the end otherwise. A [null](../basic_json_view/is_null.md)
`object` first becomes an empty object.
3. Assigns `value` to the element at index `idx` of the array `array`, which must already exist (`#!cpp idx <
array.size()`).
4. Sets the value the JSON pointer `ptr` refers to, relative to [`root()`](root.md), to `value`. The *parent* of the
target must already exist: an object member is set as in 2. (added if it does not exist yet), an array element is
assigned as in 3., and a last reference token of `#!cpp "-"`, or equal to the size of the array, appends `value`
instead, exactly as [`push_back`](push_back.md) would. An empty `ptr` sets [`root()`](root.md) itself, as in 1.
In every overload, `value` is accepted three ways: a [`basic_json_view`](../basic_json_view/index.md) of *any*
document -- read-only or editable, and it does not have to be `target`'s/`object`'s/`array`'s own document -- which
is copied so that nothing is shared with the source document afterward; a `BasicJsonType` value; or anything
`BasicJsonType` can be constructed from (numbers, strings, `#!cpp bool`, `#!cpp nullptr`, containers, ...).
## Template parameters
`V`
: the type of `value`, deduced; see above for what is accepted.
`I`
: an integral type other than `#!cpp bool`, deduced (overloads taking a `#!cpp bool` or a non-integral type for
`idx` do not participate in overload resolution).
## Parameters
`target` (in)
: the value to replace
`object` (in)
: the object (or null value) whose member to set
`array` (in)
: the array whose element to assign
`key` (in)
: the key of the member to set
`idx` (in)
: the index of the element to assign; a negative value throws (see [Exceptions](#exceptions))
`ptr` (in)
: a JSON pointer to the value to set, relative to `root()`
`value` (in)
: the new value
## Return value
1. a view of `target`, now holding `value`
2. a view of the member `key` of `object`, now holding `value`
3. a view of the element `idx` of `array`, now holding `value`
4. a view of the value `ptr` refers to, now holding `value`
## Exception safety
Basic exception safety: `value` is fully encoded -- including the checks below -- into storage owned by the document
before anything already reachable from [`root()`](root.md) is touched, so a failure while encoding `value` (an
invalid argument, or `#!cpp std::bad_alloc`) leaves the document completely unchanged, other than memory allocated
for the encoding that is not reclaimed. A failure of a later allocation -- while an edited array or object switches
from its parsed layout to a growable block, see [Notes](#notes) -- can still leave a partial effect, such as a
[null](../basic_json_view/is_null.md) `object`/`array` argument already turned into an empty object/array even
though `value` itself was not linked in.
## Exceptions
1. Throws [`type_error.302`](../../home/exceptions.md#jsonexceptiontype_error302) if `value` is a
[discarded](../basic_json_view/is_discarded.md) view, or a [discarded](../basic_json/is_discarded.md)
`BasicJsonType` value (e.g. `#!cpp BasicJsonType(value_t::discarded)`) -- an object or array `value`, of either
kind, is fine and is encoded as a whole subtree.
2. Throws [`type_error.305`](../../home/exceptions.md#jsonexceptiontype_error305) if `object` is neither an object
nor null -- the same message [`operator[]`](../basic_json_view/operator%5B%5D.md) throws for a string argument on
such a value. Throws [`type_error.316`](../../home/exceptions.md#jsonexceptiontype_error316) if `key` is not
valid UTF-8, with the same message [`BasicJsonType::dump()`](../basic_json/dump.md) gives for that string.
Also throws what 1. throws for `value`.
3. Throws `type_error.305` if `array` is not an array -- the same message `operator[]` throws for a numeric argument
on such a value. Throws [`out_of_range.401`](../../home/exceptions.md#jsonexceptionout_of_range401) if `idx` is
negative, or if `#!cpp idx >= array.size()`. Also throws what 1. throws for `value`.
4. Throws what [`at`](../basic_json_view/at.md) throws (overload 3) for resolving `ptr`'s parent, except that a
missing object member or an array index equal to the array's size at the very last reference token is not an
error there (it becomes a new member or an appended element) instead of
[`out_of_range.403`](../../home/exceptions.md#jsonexceptionout_of_range403)/[`out_of_range.402`](../../home/exceptions.md#jsonexceptionout_of_range402).
For the last reference token itself: if the parent is an object (or a primitive value, where it throws
`type_error.305`), throws what 2. throws; if the parent is an array, throws what 3. throws for an index that is
out of range, or, for a token that is not a valid array index,
[`parse_error.106`](../../home/exceptions.md#jsonexceptionparse_error106) (a leading `#!cpp '0'`),
[`parse_error.109`](../../home/exceptions.md#jsonexceptionparse_error109) (not a number),
[`out_of_range.410`](../../home/exceptions.md#jsonexceptionout_of_range410) (too large for `size_type`), or
[`out_of_range.404`](../../home/exceptions.md#jsonexceptionout_of_range404) (an empty token). Also throws what 1.
throws for `value`.
Every overload also throws [`type_error.319`](../../home/exceptions.md#jsonexceptiontype_error319) if `value` is (or
contains) a binary value -- `BasicJsonType` can hold one, but a `json_document` cannot -- and
[`invalid_iterator.202`](../../home/exceptions.md#jsonexceptioninvalid_iterator202) ("view does not belong to this
document") if `target`/`object`/`array` is a [discarded](../basic_json_view/is_discarded.md) view or a view of a
*different* document (overloads 1-3 only; overload 4 always starts from this document's own [`root()`](root.md)).
## Complexity
1. Linear in the size of `value` (encoding it into the document's storage): constant for a scalar, linear in the
number of nested values for an array or object. If `target` is itself an array or object that spans more than one
node in its parent's original, unedited layout, and `value` is a scalar, replacing it additionally costs time
linear in the number of elements of that parent, the *first* time -- see [Notes](#notes).
2. Linear in the number of members of `object`, to find an existing member with `key`, plus the complexity of 1. for
`value`.
3. Constant, plus the complexity of 1. for `value`.
4. Linear in the number of reference tokens of `ptr` and, for each token, in the number of members of the object at
that level or the index into the array (as [`at`](../basic_json_view/at.md)), plus the complexity of 2. or 3. for
the last token.
## Notes
!!! info "Duplicate keys"
If `object` already has more than one member with `key` (2.), the *first* one is assigned `value` and every
later member with the same key is removed -- so that a lookup, an iteration, and
[`materialize()`](../basic_json_view/materialize.md) of `object` afterward all agree on a single value for
`key`, the same way [`operator[]`](../basic_json_view/operator%5B%5D.md) already picks the first occurrence of a
duplicate key for reading. See the [Notes on duplicate keys](../basic_json_view/operator%5B%5D.md#notes) of
`operator[]`.
Setting a member (2.) or an element (3., through 4.) of an array or object whose elements have not been edited
before switches it from its parsed layout to a growable block holding links to its elements; a later
[`push_back`](push_back.md) or `set` on the same container reuses that block, growing it (amortized constant time)
only once it runs out of room. This never moves an element itself -- only where the container's *links* to its
elements live -- so a view of an element stays valid, but any iterator already taken over the container is
invalidated, since it was walking the old layout. See [Edits](index.md#edits) for what stays valid across an edit in
general.
The same switch happens, for the same reason, when overload 1. replaces a multi-node array/object value with a
scalar: the *parent's* element sequence is what has to switch to links, not `target` itself, because the parent
originally stepped over `target`'s whole subtree by its node count, which no longer applies once `target` is a
one-node scalar.
## Examples
??? example "Example: (1)/(2)/(3)/(4) replace a value, set a member, assign an element, set via a JSON pointer"
The example below edits a small configuration document -- replacing a value, adding an object member, assigning
an array element, and reaching a field through a JSON pointer -- and shows what
[`dump()`](../basic_json_view/dump.md) preserves that is lost once the same edits are made on a `BasicJsonType`
value instead: the order object members were written in, and the exact spelling of a number that was never
touched.
```cpp
--8<-- "examples/basic_json_document__set.cpp"
```
Output:
```json
--8<-- "examples/basic_json_document__set.output"
```
## See also
- [push_back](push_back.md) - append to an array
- [insert](insert.md) - insert an element into an array
- [erase](erase.md) - remove an object member, an array element, or the value a JSON pointer refers to
- [root](root.md) - the view of the root value, the starting point of overload 4
- [`basic_json_view::dump`](../basic_json_view/dump.md) - serialize the document, keeping an untouched number's
spelling with `#!cpp number_format::source`
- [Edits](index.md#edits) - what an edit guarantees, for every overload
- [Editing a document](../../features/json_view.md#editing-a-document) - why editable documents keep the source
text's order and number spelling
## Version history
- Added in version 3.13.0.
+15 -1
View File
@@ -3,7 +3,7 @@
<small>Defined in header `<nlohmann/json_view.hpp>`</small>
```cpp
template<typename BasicJsonType>
template<typename BasicJsonType, bool Editable = false>
class basic_json_view;
```
@@ -27,16 +27,30 @@ subtree on demand. [`operator[]`](operator%5B%5D.md), [`at`](at.md), [`contains`
[`operator!=`](operator_ne.md) compare two views, or a view and a `BasicJsonType` value, without ever building a
`BasicJsonType` value for a view; no ordering comparison (`#!cpp operator<`) is provided.
`basic_json_view` itself is always read-only -- it never has a `set` or `push_back` of its own. A view of an
**editable** document (`#!cpp Editable == true`) sees every edit made through
[`basic_json_document::set`](../basic_json_document/set.md) and
[`basic_json_document::push_back`](../basic_json_document/push_back.md): once a value is changed, every view that
still refers to it -- including ones taken before the change -- reads the new value. See
[Edits](../basic_json_document/index.md#edits).
## Template parameters
`BasicJsonType`
: a specialization of [`basic_json`](../basic_json/index.md), matching the
[`basic_json_document`](../basic_json_document/index.md) the view was taken from.
`Editable`
: whether the view is of an editable document, matching the [`basic_json_document`](../basic_json_document/index.md)
it was taken from (optional, `#!cpp false` by default). See [Edits](../basic_json_document/index.md#edits).
## Specializations
- [**json_view**](../json_view.md) - views of a [`json_document`](../json_document.md)
- [**ordered_json_view**](../ordered_json_view.md) - views of an [`ordered_json_document`](../ordered_json_document.md)
- [**json_editable_view**](../json_editable_view.md) - views of a [`json_editable_document`](../json_editable_document.md)
- [**ordered_json_editable_view**](../ordered_json_editable_view.md) - views of an
[`ordered_json_editable_document`](../ordered_json_editable_document.md)
## Member types
@@ -0,0 +1,43 @@
# <small>nlohmann::</small>json_editable_document
<small>Defined in header `<nlohmann/json_view.hpp>`</small>
```cpp
using json_editable_document = basic_json_document<json, true>;
```
This type is an **editable** [`basic_json_document`](basic_json_document/index.md) of the default
[`json`](json.md) specialization: in addition to everything [`json_document`](json_document.md) offers,
[`set`](basic_json_document/set.md) and [`push_back`](basic_json_document/push_back.md) change values after
parsing, without ever rewriting the source text -- see [Edits](basic_json_document/index.md#edits) and
[Editing a document](../features/json_view.md#editing-a-document).
## Examples
??? example
The example below patches two fields of a small configuration document -- changing one and adding another --
and dumps it back out with the member order and the spelling of the untouched number preserved, something a
plain [`json`](json.md) value cannot do.
```cpp
--8<-- "examples/json_editable_document.cpp"
```
Output:
```json
--8<-- "examples/json_editable_document.output"
```
## See also
- [json_editable_view](json_editable_view.md) - a view of a value of a `json_editable_document`
- [json_document](json_document.md) - the read-only document this type adds edits to
- [ordered_json_editable_document](ordered_json_editable_document.md) - the corresponding editable document for
`ordered_json`
- [Edits](basic_json_document/index.md#edits) - what an edit guarantees
## Version history
Since version 3.13.0.
@@ -0,0 +1,41 @@
# <small>nlohmann::</small>json_editable_view
<small>Defined in header `<nlohmann/json_view.hpp>`</small>
```cpp
using json_editable_view = basic_json_view<json, true>;
```
This type is a [`basic_json_view`](basic_json_view/index.md) of a value of a
[`json_editable_document`](json_editable_document.md). It offers the same read-only interface as
[`json_view`](json_view.md); what is different is what it can be a view *of* -- a value that
[`set`](basic_json_document/set.md) and [`push_back`](basic_json_document/push_back.md) can change, with every view
still referring to it seeing the change, see [Edits](basic_json_document/index.md#edits).
## Examples
??? example
The example below is the same as [`json_editable_document`'s](json_editable_document.md): every view read back
out of the document -- `#!cpp doc.root()` and the views nested under it -- sees the edits made through `set`.
```cpp
--8<-- "examples/json_editable_document.cpp"
```
Output:
```json
--8<-- "examples/json_editable_document.output"
```
## See also
- [json_editable_document](json_editable_document.md) - the document type this view refers into
- [json_view](json_view.md) - the corresponding read-only view
- [ordered_json_editable_view](ordered_json_editable_view.md) - the corresponding view for
`ordered_json_editable_document`
## Version history
Since version 3.13.0.
@@ -0,0 +1,47 @@
# <small>nlohmann::</small>ordered_json_editable_document
<small>Defined in header `<nlohmann/json_view.hpp>`</small>
```cpp
using ordered_json_editable_document = basic_json_document<ordered_json, true>;
```
This type is an **editable** [`basic_json_document`](basic_json_document/index.md) of the
[`ordered_json`](ordered_json.md) specialization: [`set`](basic_json_document/set.md) and
[`push_back`](basic_json_document/push_back.md) change values after parsing, as for
[`json_editable_document`](json_editable_document.md), and
[`materialize()`](basic_json_view/materialize.md) preserves the document order of object members -- including
members [`set`](basic_json_document/set.md) added -- instead of sorting them like
[`json_editable_document`](json_editable_document.md) does.
## Examples
??? example
The example below edits a document with `set`, then shows that `materialize()` keeps the member order of the
source text (with the new member at the end) for `ordered_json_editable_document`, where it would sort the
members alphabetically for [`json_editable_document`](json_editable_document.md).
```cpp
--8<-- "examples/ordered_json_editable_document.cpp"
```
Output:
```json
--8<-- "examples/ordered_json_editable_document.output"
```
## See also
- [ordered_json_editable_view](ordered_json_editable_view.md) - a view of a value of an
`ordered_json_editable_document`
- [ordered_json_document](ordered_json_document.md) - the read-only document this type adds edits to
- [json_editable_document](json_editable_document.md) - the corresponding editable document for the default `json`
specialization
- [Object Order](../features/object_order.md)
- [Edits](basic_json_document/index.md#edits) - what an edit guarantees
## Version history
Since version 3.13.0.
@@ -0,0 +1,40 @@
# <small>nlohmann::</small>ordered_json_editable_view
<small>Defined in header `<nlohmann/json_view.hpp>`</small>
```cpp
using ordered_json_editable_view = basic_json_view<ordered_json, true>;
```
This type is a [`basic_json_view`](basic_json_view/index.md) of a value of an
[`ordered_json_editable_document`](ordered_json_editable_document.md), the corresponding view for
[`ordered_json_view`](ordered_json_view.md) the way [`json_editable_view`](json_editable_view.md) is for
[`json_view`](json_view.md).
## Examples
??? example
The example below is the same as [`ordered_json_editable_document`'s](ordered_json_editable_document.md): the
views `set` returns see the document's member order preserved on `materialize()`, unlike for a
[`json_editable_document`](json_editable_document.md).
```cpp
--8<-- "examples/ordered_json_editable_document.cpp"
```
Output:
```json
--8<-- "examples/ordered_json_editable_document.output"
```
## See also
- [ordered_json_editable_document](ordered_json_editable_document.md) - the document type this view refers into
- [ordered_json_view](ordered_json_view.md) - the corresponding read-only view
- [json_editable_view](json_editable_view.md) - the corresponding view for `json_editable_document`
## Version history
Since version 3.13.0.
@@ -0,0 +1,38 @@
#include <iostream>
#include <nlohmann/json_view.hpp>
using json = nlohmann::json;
using json_editable_document = nlohmann::json_editable_document;
using json_editable_view = nlohmann::json_editable_view;
int main()
{
// a deprecated field is dropped from a configuration file, and a
// decommissioned replica is removed from the list -- "price" keeps its
// trailing zero, and the fields around the removed ones keep their order
const std::string text = R"({
"name": "cache",
"legacy_host": "db0",
"host": "db1",
"price": 19.90,
"replicas": ["db2", "db3", "db4"]
})";
json_editable_document doc = json_editable_document::parse(text);
doc.erase(doc.root(), "legacy_host"); // (1) an object member
doc.erase(doc.root()["replicas"], 1); // (2) an array element ("db3")
const std::size_t removed = doc.erase(json::json_pointer("/replicas/0")); // (3) via a JSON pointer
std::cout << removed << '\n';
std::cout << doc.root().dump(2, ' ', false, json_editable_view::number_format::source) << "\n\n";
// the same edits on a plain json value: object_t is a std::map, so
// parsing already sorted the keys, and dump() rewrites every number to
// its shortest form, even "price", which was never touched
json plain = json::parse(text);
plain.erase("legacy_host");
plain["replicas"].erase(1);
plain["replicas"].erase(0);
std::cout << plain.dump(2) << '\n';
}
@@ -0,0 +1,18 @@
1
{
"name": "cache",
"host": "db1",
"price": 19.90,
"replicas": [
"db4"
]
}
{
"host": "db1",
"name": "cache",
"price": 19.9,
"replicas": [
"db4"
]
}
@@ -0,0 +1,38 @@
#include <iostream>
#include <nlohmann/json_view.hpp>
using json = nlohmann::json;
using json_editable_document = nlohmann::json_editable_document;
using json_editable_view = nlohmann::json_editable_view;
int main()
{
// a deployment plan -- "budget" is written with a trailing zero that has
// no effect on its value
const std::string text = R"({
"release": "2026.09",
"steps": ["build", "test", "deploy"],
"budget": 19.90
})";
json_editable_document doc = json_editable_document::parse(text);
const std::size_t deploy_index = 2;
const auto deploy = doc.root()["steps"][deploy_index]; // held across the insert
doc.insert(doc.root()["steps"], deploy_index, "smoke-test"); // insert before "deploy"
// the held view still refers to "deploy", even though its index moved
// from 2 to 3, and nothing else in the document was touched
std::cout << deploy.dump() << '\n';
std::cout << doc.root().dump(2, ' ', false, json_editable_view::number_format::source) << "\n\n";
// the same edit on a plain json value: an index held from before the
// insert now refers to whatever moved into that slot, and dump()
// rewrites "budget" to its shortest form even though it was never
// touched
json plain = json::parse(text);
plain["steps"].insert(plain["steps"].begin() + static_cast<std::ptrdiff_t>(deploy_index), "smoke-test");
std::cout << plain["steps"][deploy_index].dump() << '\n';
std::cout << plain.dump(2) << '\n';
}
@@ -0,0 +1,23 @@
"deploy"
{
"release": "2026.09",
"steps": [
"build",
"test",
"smoke-test",
"deploy"
],
"budget": 19.90
}
"smoke-test"
{
"budget": 19.9,
"release": "2026.09",
"steps": [
"build",
"test",
"smoke-test",
"deploy"
]
}
@@ -0,0 +1,52 @@
#include <cstdint>
#include <iostream>
#include <string>
#include <vector>
#include <nlohmann/json_view.hpp>
using json = nlohmann::json;
using json_document = nlohmann::json_document;
using image_check = json_document::image_check;
int main()
{
std::cout << std::boolalpha;
// the image of a parsed document -- as if read back from a cache file or
// received from another process running the same build of the library
const std::string text = R"({"name": "cache", "note": "caf\u00e9", "replicas": ["db2", "db3"]})";
const json_document parsed = json_document::parse(text);
const std::vector<std::uint8_t> image = parsed.save();
// (1)/(2) load() needs no parsing, yet dumps exactly what parsing did
const json_document borrowed = json_document::load(image);
std::cout << (borrowed.root().dump() == parsed.root().dump()) << '\n';
std::cout << borrowed.owns_source() << '\n'; // borrowed: still points into `image`
// (3) load(std::move(image)) keeps the vector instead of copying it
std::vector<std::uint8_t> to_move = image;
const json_document owned = json_document::load(std::move(to_move));
std::cout << owned.owns_source() << '\n';
// a damaged image -- the last byte of the decoded string "note" holds
// (an escape sequence, so it was unescaped into the document's own
// buffer), flipped, as storage or transport corruption might do
std::vector<std::uint8_t> damaged = image;
damaged[damaged.size() - 2] = 0xFF;
// image_check::full inspects strings and numbers, so it catches the damage
try
{
static_cast<void>(json_document::load(damaged, image_check::full));
}
catch (const json::parse_error& e)
{
std::cout << e.id << '\n';
}
// image_check::bounds only checks structure and bounds, so a cache the
// process already trusts loads without the extra scan -- reading a value
// the damage did not touch is still safe
const json_document trusted = json_document::load(damaged, image_check::bounds);
std::cout << trusted.root()["name"].get<std::string>() << '\n';
}
@@ -0,0 +1,5 @@
true
false
true
116
cache
@@ -0,0 +1,24 @@
#include <iostream>
#include <nlohmann/json_view.hpp>
using json = nlohmann::json;
using json_editable_document = nlohmann::json_editable_document;
int main()
{
// "events" starts out null -- the first push_back() turns it into an
// array, exactly like set() turns a null object member into an object
json_editable_document doc = json_editable_document::parse(R"({"source": "sensor-1", "events": null})");
const auto first = doc.push_back(doc.root()["events"], json{{"type", "start"}, {"t", 0}});
for (int t = 1; t <= 3; ++t)
{
doc.push_back(doc.root()["events"], json{{"type", "tick"}, {"t", t}});
}
// push_back() never moves an existing element: a view taken from an
// earlier call still refers to the same element after later ones
std::cout << first.dump() << '\n';
std::cout << doc.root()["events"].size() << '\n';
std::cout << doc.root().dump(2) << '\n';
}
@@ -0,0 +1,23 @@
{"t":0,"type":"start"}
4
{
"source": "sensor-1",
"events": [
{
"t": 0,
"type": "start"
},
{
"t": 1,
"type": "tick"
},
{
"t": 2,
"type": "tick"
},
{
"t": 3,
"type": "tick"
}
]
}
@@ -0,0 +1,30 @@
#include <cstdint>
#include <iostream>
#include <string>
#include <vector>
#include <nlohmann/json_view.hpp>
using json_document = nlohmann::json_document;
int main()
{
std::cout << std::boolalpha;
// a configuration a service parses once and then caches as an image, so
// that later requests can load() it instead of parsing the text again
const std::string text = R"({"name": "cache", "host": "db1", "port": 6379, "replicas": ["db2", "db3"]})";
const json_document config = json_document::parse(text);
// save() turns the parsed document into a byte buffer: a 64-byte header,
// the node index, the source text, and the decoded strings
const std::vector<std::uint8_t> image = config.save();
std::cout << image.size() << '\n';
// the same document always saves to the same bytes
std::cout << (image == json_document::parse(text).save()) << '\n';
// loading the image back needs no parsing, yet dumps exactly what
// parsing the text produced
const json_document reloaded = json_document::load(image);
std::cout << (reloaded.root().dump() == config.root().dump()) << '\n';
}
@@ -0,0 +1,3 @@
316
true
true
@@ -0,0 +1,41 @@
#include <iostream>
#include <nlohmann/json_view.hpp>
using json = nlohmann::json;
using json_editable_document = nlohmann::json_editable_document;
using json_editable_view = nlohmann::json_editable_view;
int main()
{
// a configuration file, as it might be read from disk -- "price" is
// written with a trailing zero that has no effect on its value
const std::string text = R"({
"name": "cache",
"host": "db1",
"port": 6379,
"price": 19.90,
"replicas": ["db2", "db3"],
"timeout": 30
})";
json_editable_document doc = json_editable_document::parse(text);
doc.set(doc.root()["port"], 6380); // (1) replace a value
doc.set(doc.root(), "region", "us-east"); // (2) add a member
doc.set(doc.root()["replicas"], 0, "db4"); // (3) assign an element
doc.set(json::json_pointer("/timeout"), 45); // (4) via a JSON pointer
// members stay in document order (the new one at the end), and a number
// that was not itself edited keeps its exact spelling
std::cout << doc.root().dump(2, ' ', false, json_editable_view::number_format::source) << "\n\n";
// the same edits on a plain json value: object_t is a std::map, so
// parsing already sorted the keys, and dump() rewrites every number to
// its shortest form, even "price", which was never touched
json plain = json::parse(text);
plain["port"] = 6380;
plain["region"] = "us-east";
plain["replicas"][0] = "db4";
plain[json::json_pointer("/timeout")] = 45;
std::cout << plain.dump(2) << '\n';
}
@@ -0,0 +1,25 @@
{
"name": "cache",
"host": "db1",
"port": 6380,
"price": 19.90,
"replicas": [
"db4",
"db3"
],
"timeout": 45,
"region": "us-east"
}
{
"host": "db1",
"name": "cache",
"port": 6380,
"price": 19.9,
"region": "us-east",
"replicas": [
"db4",
"db3"
],
"timeout": 45
}
@@ -0,0 +1,30 @@
#include <iostream>
#include <nlohmann/json_view.hpp>
using json = nlohmann::json;
using json_editable_document = nlohmann::json_editable_document;
using json_editable_view = nlohmann::json_editable_view;
int main()
{
// a configuration file, as it might be read from disk
const std::string text = R"({"name": "cache", "host": "db1", "port": 6379, "price": 19.90})";
std::cout << text << "\n\n";
// patch two fields -- "price" is never touched
json_editable_document doc = json_editable_document::parse(text);
doc.set(doc.root(), "host", "db2");
doc.set(doc.root(), "retries", 3);
// member order (the new member at the end) and the untouched number's
// exact spelling survive
std::cout << doc.root().dump(-1, ' ', false, json_editable_view::number_format::source) << '\n';
// the same patch on a plain json value: keys are sorted (object_t is a
// std::map), and "price" is rewritten even though the patch never
// touched it
json plain = json::parse(text);
plain["host"] = "db2";
plain["retries"] = 3;
std::cout << plain.dump() << '\n';
}
@@ -0,0 +1,4 @@
{"name": "cache", "host": "db1", "port": 6379, "price": 19.90}
{"name":"cache","host":"db2","port":6379,"price":19.90,"retries":3}
{"host":"db2","name":"cache","port":6379,"price":19.9,"retries":3}
@@ -0,0 +1,15 @@
#include <iostream>
#include <nlohmann/json_view.hpp>
using ordered_json_editable_document = nlohmann::ordered_json_editable_document;
int main()
{
// ordered_json_editable_document is basic_json_document<nlohmann::ordered_json, true>
ordered_json_editable_document doc = ordered_json_editable_document::parse(R"({"z": 1, "a": 2, "m": 3})");
doc.set(doc.root(), "b", 4); // set() always appends a new member at the end
// materialize() preserves the document order (with "b" at the end),
// instead of sorting the keys the way json_editable_document does
std::cout << doc.root().materialize().dump() << '\n';
}
@@ -0,0 +1 @@
{"z":1,"a":2,"m":3,"b":4}
+2
View File
@@ -13,6 +13,8 @@ C++ types, and finally serialize it again.
[SAX interface](parsing/sax_interface.md), and [error handling](parsing/parse_exceptions.md).
- [Zero-copy JSON views](json_view.md) — read a JSON text through a flat index instead of building a `json` tree;
strings and numbers stay in the input and are only decoded when needed.
[Editable documents](json_view.md#editing-a-document) can also be modified, and [images](json_view.md#images) load a
parsed document again without parsing it.
- [Comments](comments.md) and [trailing commas](trailing_commas.md) — opt-in relaxations of the JSON grammar.
## Accessing and modifying values
+121 -6
View File
@@ -187,14 +187,129 @@ a number to its parsed `#!cpp double`/`#!cpp int64_t` value.
[`operator<<`](../api/basic_json_view/operator_ltlt.md) writes a view to a stream the way `basic_json`'s does, using
the stream's `width`/`fill` for indentation.
## Editing a document
Everything above is read-only: a `json_document`/`json_view` lets you look at a parsed text without copying it, but
not change it. [`basic_json_document<BasicJsonType, true>`](../api/basic_json_document/index.md) -- more conveniently
spelled [`json_editable_document`](../api/json_editable_document.md) or
[`ordered_json_editable_document`](../api/ordered_json_editable_document.md) -- also lets you
[`set`](../api/basic_json_document/set.md) a value, [`push_back`](../api/basic_json_document/push_back.md) onto or
[`insert`](../api/basic_json_document/insert.md) into an array, and [`erase`](../api/basic_json_document/erase.md)
an object member or an array element, still without ever building a `basic_json` tree for parts you do not touch.
`#!cpp Editable` defaults to `#!cpp false`, so `json_document`/`ordered_json_document` are unaffected -- they carry
none of the bookkeeping edits need, and calling `set`/`push_back`/`insert`/`erase` on one is a compile error, not a
runtime one.
### Why: editing without reformatting
The `#!cpp 19.90` price from [above](#writing-a-view-back) is exactly the kind of value that makes editing a `json`
or `ordered_json` value in place lossy. Say you parse a configuration file, patch one field, and write it back:
- **`json`** re-sorts every key on the way in (`object_t` is a `#!cpp std::map`) and rewrites every number to its
shortest round-trip form on the way out -- a one-field patch turns into a diff that reorders the whole file and
rewrites `#!cpp 19.90` to `#!cpp 19.9`.
- **`ordered_json`** keeps the key order, but still rewrites every number the same way: parsing has already reduced
it to a `#!cpp double`/`#!cpp int64_t`, and there is no way back to how it was spelled in the source text.
An editable document keeps both. [`dump()`](../api/basic_json_view/dump.md) of an edited document writes members in
document order -- a member [`set`](../api/basic_json_document/set.md) added goes at the end, exactly where it was
inserted, and an [`erase`](../api/basic_json_document/erase.md)d member simply leaves a gap: everything around it
keeps its place -- and [`number_format::source`](../api/basic_json_view/number_format.md) keeps the exact spelling
of every number an edit did not itself touch; a number an edit *did* touch is written the way
[`BasicJsonType::dump()`](../api/basic_json/dump.md) would write it, since there is no source spelling for a brand
new value.
??? example "Example: patch a configuration, keeping member order and number spellings"
```cpp
--8<-- "examples/json_editable_document.cpp"
```
Output:
```json
--8<-- "examples/json_editable_document.output"
```
### What stays valid, and what an edit costs
The source text itself is **never written**, and the parsed index never moves -- a value keeps the node it was
parsed into for as long as it is not itself replaced. So every [view](../api/basic_json_view/index.md) taken before
an edit, including a previously obtained [`root()`](../api/basic_json_document/root.md), stays valid and, if it
still refers to the edited value, sees the edit; a view of a value a later edit drops or replaces just keeps showing
what it last held. New values go to storage the document allocates and owns on demand. The one thing an edit does
invalidate is the **iterators** taken over an edited array or object: the first time one of its elements is set,
appended to, inserted into, or erased, its elements move from the parsed, fixed layout to a growable block of links
so that [`push_back`](../api/basic_json_document/push_back.md) can later grow it in amortized constant time --
existing elements are not touched, but an iterator that was walking the old layout no longer matches. A string
obtained with
[`get_string()`](../api/basic_json_view/get_string.md) is unaffected either way and stays valid across further
edits. See [`basic_json_document`'s Edits](../api/basic_json_document/index.md#edits) for the details, and
[`set`'s Exception safety](../api/basic_json_document/set.md#exception-safety) for what an edit guarantees if it
throws (the *basic* guarantee, not the strong one `dump()` and the read-only functions provide). How edits are kept in
the index is described in the [architecture overview](../home/architecture.md#node-index-of-json-views).
## Images
[`save()`](../api/basic_json_document/save.md) writes a document as an *image*: a byte buffer that
[`load()`](../api/basic_json_document/load.md) reads back into a document without parsing -- no lexing, no building
the node index, nothing but copying the nodes and pointing the text and the decoded strings at the image. Where
[`parse_copy()`](../api/basic_json_document/parse_copy.md) still has to scan the whole input,
[`load()`](../api/basic_json_document/load.md) turns that scan into a copy of the node index alone.
**Why.** A document that is parsed once and then read many times -- a configuration loaded at startup, a template
rendered on every request, a large reference dataset a worker process needs in memory -- pays for parsing once but
can amortize [`save()`](../api/basic_json_document/save.md)'s cost across every later load. That makes images useful
for a cache: save a document the first time it is parsed (to a file, a shared-memory segment, an in-process cache),
and [`load()`](../api/basic_json_document/load.md) it on every later use instead of parsing the source text again.
They are just as useful for handing a parsed document to another process (or a forked worker) running the same build
of the library, since [`load()`](../api/basic_json_document/load.md) turns the transfer into a copy of the node index
plus pointers into the received bytes, not a re-parse.
**Choosing a check.** [`load()`](../api/basic_json_document/load.md) takes an
[`image_check`](../api/basic_json_document/load.md#image_check) that trades validation against speed:
`image_check::full` (the default) checks everything the parser itself guarantees, so a checked image is exactly as
safe to read and serialize as a freshly parsed document -- the right choice whenever the image did not come straight
from this process's own [`save()`](../api/basic_json_document/save.md), such as a file or a network peer.
`image_check::bounds` only checks structure and bounds -- cheaper, since it skips scanning the text and the decoded
strings -- and fits a cache the process trusts, one it wrote and reads back itself. `image_check::none` skips
validation entirely, for an image trusted as much as the process's own memory. See
[`load()`'s Notes](../api/basic_json_document/load.md#notes) for exactly what each level does and does not guarantee.
??? example "Example: cache a parsed configuration as an image"
```cpp
--8<-- "examples/basic_json_document__save.cpp"
```
Output:
```json
--8<-- "examples/basic_json_document__save.output"
```
!!! warning "Experimental"
The image format is versioned but not yet stable, and may change in an incompatible way before it is declared
stable. It is little-endian only, and tied to the library build that wrote it -- use it to cache a document or to
hand one to another process running the *same* build, not as a long-term storage format; keep the original JSON
text if a saved document needs to be readable by a future library version.
The idea of a document you can read without parsing comes from zero-copy formats such as
[FlatBuffers](https://github.com/google/flatbuffers) and [YaFF](https://github.com/yandex/yaff); the check
[`load()`](../api/basic_json_document/load.md) runs follows the idea of FlatBuffers' Verifier. No code is taken from
either.
## Choosing between `json`, `ordered_json`, the SAX interface, and `json_view`
| | [`json`](../api/json.md) / [`ordered_json`](../api/ordered_json.md) | [SAX interface](parsing/sax_interface.md) | [`json_document`](../api/json_document.md) / [`json_view`](../api/json_view.md) |
|---|---|---|---|
| **Ownership** | owns every value | owns nothing; you decide what to keep, in your handler | borrows or owns the *text*; the index is always owned by the document |
| **Mutability** | freely mutable | not applicable (a one-shot event stream) | read-only |
| **What you get** | a full tree you can read, write, and keep as long as you like | a sequence of callbacks; whatever your handler builds from them | a flat index plus, on demand, [`materialize()`](../api/basic_json_view/materialize.md)d `json`/`ordered_json` values for the parts you actually use |
| **Typical use** | general-purpose JSON handling: config, request/response bodies you build or modify, anything you hold onto | validating or projecting a text into your own data structure without ever holding the whole thing as JSON | large or high-volume input where you only need part of it, or need it repeatedly, and can keep the source text (or a copy) alive for as long as the document lives |
| | [`json`](../api/json.md) / [`ordered_json`](../api/ordered_json.md) | [SAX interface](parsing/sax_interface.md) | [`json_document`](../api/json_document.md) / [`json_view`](../api/json_view.md) | [`json_editable_document`](../api/json_editable_document.md) / [`json_editable_view`](../api/json_editable_view.md) |
|---|---|---|---|---|
| **Ownership** | owns every value | owns nothing; you decide what to keep, in your handler | borrows or owns the *text*; the index is always owned by the document | same as `json_document`; edits go to storage the document owns |
| **Mutability** | freely mutable | not applicable (a one-shot event stream) | read-only | [`set`](../api/basic_json_document/set.md)/[`push_back`](../api/basic_json_document/push_back.md)/[`insert`](../api/basic_json_document/insert.md)/[`erase`](../api/basic_json_document/erase.md) edit in place; the source text is never rewritten |
| **What you get** | a full tree you can read, write, and keep as long as you like | a sequence of callbacks; whatever your handler builds from them | a flat index plus, on demand, [`materialize()`](../api/basic_json_view/materialize.md)d `json`/`ordered_json` values for the parts you actually use | the same, plus [`dump()`](../api/basic_json_view/dump.md) of an edited document that keeps the member order and, with [`number_format::source`](../api/basic_json_view/number_format.md), the spelling of every untouched number |
| **Typical use** | general-purpose JSON handling: config, request/response bodies you build or modify, anything you hold onto | validating or projecting a text into your own data structure without ever holding the whole thing as JSON | large or high-volume input where you only need part of it, or need it repeatedly, and can keep the source text (or a copy) alive for as long as the document lives | a document you read, patch a few fields of, and write back -- a configuration file, for instance -- where the rest of it should come back exactly as it was |
| **Caching/reload** | not applicable -- re-parse, or roll your own serialization | not applicable | [`save()`](../api/basic_json_document/save.md)/[`load()`](../api/basic_json_document/load.md): cache the parsed index as an image and reload it without parsing | same, saving the document's current -- possibly edited -- state |
## Version history
@@ -118,9 +118,10 @@ That is, `-0` is stored as a signed integer, but the serialization does not repr
### Number serialization
- Integer numbers are serialized as is; that is, no scientific notation is used.
- Floating-point numbers are serialized as specified by the `#!c %g` printf modifier with
[`std::numeric_limits<double>::max_digits10`](https://en.cppreference.com/w/cpp/types/numeric_limits/max_digits10)
significant digits. The rationale is to use the shortest representation while still allowing round-tripping.
- Floating-point numbers are serialized with the fewest digits that read back as the same value (the closest such
digits if there are several), in the layout of the `#!c %g` printf modifier: `#!c 1.5`, `#!c 100.0`, `#!c 1e+100`.
Doubles are converted with the algorithm of [Żmij](https://github.com/vitaut/zmij), floats with Grisu2, which
can write more digits than necessary.
!!! hint "Notes regarding precision of floating-point numbers"
@@ -540,9 +540,10 @@ therefore silently changes parse results rather than raising an error. See
specifiers, for which the library likewise provides only `#!cpp double` and `#!cpp long double` overloads
(`#!cpp float` is promoted to `#!cpp double`).
If `#!cpp std::numeric_limits<NumberFloatType>` describes an IEEE 754 binary32 or binary64 number, `dump` uses the
Grisu2 algorithm, which produces the shortest representation that round-trips. Otherwise the `snprintf` fallback with
`max_digits10` digits is used.
If `#!cpp std::numeric_limits<NumberFloatType>` describes an IEEE 754 binary64 number, `dump` uses the algorithm of
Żmij, which produces the shortest representation that round-trips. For IEEE 754 binary32 numbers, it uses Grisu2,
which produces a short representation that round-trips. Otherwise the `snprintf` fallback with `max_digits10` digits is
used.
### Required for the binary formats
@@ -554,7 +555,7 @@ binary32 or binary64 field and have no encoding for `#!cpp long double`.
| Type | Support |
|--------------------------|-----------------------------------------------------------------------------------------------------------------------|
| `#!cpp double` (default) | full; short round-trip output through Grisu2 |
| `#!cpp double` (default) | full; shortest round-trip output through Żmij |
| `#!cpp float` | full; short round-trip output through Grisu2 |
| `#!cpp long double` | `dump` and `parse` only; the binary format writers do not compile, as they only handle IEEE 754 binary32 and binary64 |
| any other type | not usable |
+25 -2
View File
@@ -194,8 +194,8 @@ packet-beta
| Bytes | Field | Type | Contents |
|-------|---------|------------|-------------------------------------------------------------------------------------------------------------------------------|
| 0 | `kind` | `uint8_t` | the type, numbered as [`value_t`](../api/basic_json/value_t.md): 0 null, 1 object, 2 array, 3 string, 4 boolean, 5 signed integer, 6 unsigned integer, 7 float |
| 1 | `flags` | `uint8_t` | bits 0-1: where a string's bytes are (0: the source text, 1: the buffer of decoded strings, for strings with escapes); bit 2: the value of a boolean |
| 0 | `kind` | `uint8_t` | the type, numbered as [`value_t`](../api/basic_json/value_t.md): 0 null, 1 object, 2 array, 3 string, 4 boolean, 5 signed integer, 6 unsigned integer, 7 float; 10 for a link (see below) |
| 1 | `flags` | `uint8_t` | bits 0-1: where a string's bytes (or a number's token) are (0: the source text, 1: the buffer of decoded strings, for strings with escapes, 2: the edit buffer); bit 2: the value of a boolean; bits 3 and 4: moved and new (see below) |
| 2-3 | `extra` | `uint16_t` | numbers: the number of integer digits (low byte) and fraction digits (high byte), 255 for more; objects: the number of their hash index (1-based), or 0; otherwise 0 |
| 4-7 | `off` | `uint32_t` | where the value starts: the first byte after a string's opening quote (or its position in the buffer of decoded strings), the first byte of a number or literal, the bracket of an array or object |
| 8-11 | `len` | `uint32_t` | strings: the length after decoding; floats and literals: the length of the token; arrays and objects: the number of elements |
@@ -244,6 +244,29 @@ flowchart LR
All `flags` are 0. The integer's bytes 8-15 hold its value, 1; its `extra` says it has one digit. The float's `extra`
says it has one integer and one fraction digit, and its `len` is that of the token `2.5`.
Editable documents ([`json_editable_document`](../api/json_editable_document.md),
[`detail/view/edit.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/view/edit.hpp) and
[`detail/view/edit_storage.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/view/edit_storage.hpp))
never write the source text and never move or resize the parsed index, so views stay valid while the document is
edited:
- A new scalar is written over its node. Its text (a string, or the token of a number as `dump()` writes it) goes to
the edit buffer, which `flags` bits 0-1 then name.
- An array or object whose elements change gets the flag *moved* (bit 3): its elements then live in a separate
sequence (a header node, then the entries), whose number is in `off`. The entries are links (`kind` 10), whose bytes
8-15 hold the address of the value's node, so values never move.
- A node written by an edit gets the flag *new* (bit 4): it has no position in the source text.
- Views of read-only documents compile without any of this: how views walk the index is a template parameter
(`navigation<Editable>`).
Images ([`save`](../api/basic_json_document/save.md) and [`load`](../api/basic_json_document/load.md),
[`detail/view/image.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/view/image.hpp)) store
the nodes as they are: a 64-byte header (the magic bytes `NJVI`, a format version, the sizes, and reserved bytes that
must be zero), the nodes, the text, and the decoded strings. An edited document is first written in document order, as
the parser would have written it (without links), and the numbers of the hash indexes are cleared, since `load`
rebuilds the indexes. So a change of the node layout is a change of the image format: it must raise `image_version`,
and `load` then rejects images of other versions (`parse_error.116`) instead of misreading them.
## Input adapters
Input is read via **input adapters** that abstract a source. Every input adapter provides this interface:
+68 -2
View File
@@ -388,6 +388,23 @@ A UBJSON high-precision number could not be parsed.
[json.exception.parse_error.115] parse error at byte 5: syntax error while parsing UBJSON high-precision number: invalid number text: 1A
```
### json.exception.parse_error.116
[`basic_json_document::load()`](../api/basic_json_document/load.md) rejected an
[image](../features/json_view.md#images): either the bytes are not one [`save()`](../api/basic_json_document/save.md)
could have written (too short, an unknown magic number or format version, or sizes that do not fit the buffer), or
they are, but fail the requested [`image_check`](../api/basic_json_document/load.md#image_check).
!!! failure "Example message"
```
[json.exception.parse_error.116] parse error: invalid json_document image: the check failed
```
!!! note
This exception was added in version 3.13.0, together with [images](../features/json_view.md#images).
## Iterator errors
This exception is thrown if iterators passed to a library function do not match
@@ -782,6 +799,44 @@ The dynamic type of the object cannot be represented in the requested serializat
Encapsulate the JSON value in an object. That is, instead of serializing `#!json true`, serialize `#!json {"value": true}`
### json.exception.type_error.319
[`basic_json_document::set`](../api/basic_json_document/set.md) and
[`basic_json_document::push_back`](../api/basic_json_document/push_back.md) can store any `basic_json` value except
a binary one: a `json_document` has no representation for [binary values](../features/binary_values.md), which only
ever arise from parsing a binary format or from an explicit [`json::binary`](../api/basic_json/binary.md) value, not
from JSON text.
!!! failure "Example message"
```
[json.exception.type_error.319] cannot store a binary value in a json_document
```
!!! note
This exception was added in version 3.13.0, together with editable [`json_document`s](../features/json_view.md).
### json.exception.type_error.320
[`basic_json_document::save()`](../api/basic_json_document/save.md) cannot write an
[image](../features/json_view.md#images) of a [discarded](../api/basic_json_document/is_discarded.md) document.
[`save()`](../api/basic_json_document/save.md) and [`load()`](../api/basic_json_document/load.md) also throw this
exception on a big-endian target, since the image format is little-endian only.
!!! failure "Example messages"
```
[json.exception.type_error.320] cannot save a discarded json_document
```
```
[json.exception.type_error.320] json_document images need a little-endian target
```
!!! note
This exception was added in version 3.13.0, together with [images](../features/json_view.md#images).
## Out of range
This exception is thrown in case a library function is called on an input parameter that exceeds the expected range, for instance, in the case of array indices or nonexisting object keys.
@@ -1006,13 +1061,24 @@ MessagePack's ext type and BSON's binary subtype are each stored in a single byt
[`basic_json_document::parse()`](../api/basic_json_document/parse.md) and the other parsing functions of
[`basic_json_document`](../api/basic_json_document/index.md) index a value's position in the source text in 32 bits,
so they do not support an input of 4 GiB or more.
so they do not support an input of 4 GiB or more. The same 32-bit limit applies to an **editable** document's own
storage: [`set`](../api/basic_json_document/set.md) and [`push_back`](../api/basic_json_document/push_back.md) throw
this exception once the strings and number tokens written by edits reach 4 GiB in total, or once more than
4294967295 arrays/objects have had an element set or appended to them. The same limit applies to an
[image](../features/json_view.md#images): [`save()`](../api/basic_json_document/save.md) throws it if the node
count, the text, or the decoded strings it would write would individually reach 4 GiB.
!!! failure "Example message"
!!! failure "Example messages"
```
[json.exception.out_of_range.416] input of 4 GiB or more is not supported by json_document
```
```
[json.exception.out_of_range.416] edits of 4 GiB or more are not supported by json_document
```
```
[json.exception.out_of_range.416] images of 4 GiB or more are not supported by json_document
```
!!! note
+2
View File
@@ -18,6 +18,8 @@ The class contains the UTF-8 Decoder from Bjoern Hoehrmann which is licensed und
The class contains a slightly modified version of the Grisu2 algorithm from Florian Loitsch which is licensed under the [MIT License](https://opensource.org/licenses/MIT) (see above). Copyright &copy; 2009 [Florian Loitsch](https://florian.loitsch.com/)
The class contains a port of the shortest double-to-decimal conversion of [Żmij](https://github.com/vitaut/zmij) by Victor Zverovich, which is licensed under the [MIT License](https://opensource.org/licenses/MIT) (see above). Copyright &copy; 2025 [Victor Zverovich](https://github.com/vitaut)
The class contains a copy of [Hedley](https://nemequ.github.io/hedley/) from Evan Nemerson which is licensed as [CC0-1.0](https://creativecommons.org/publicdomain/zero/1.0/).
The class contains an adapted version of the Eisel-Lemire algorithm and its table of powers of five from [fast_float](https://github.com/fastfloat/fast_float) by Daniel Lemire and contributors, which is available under the [MIT License](https://opensource.org/licenses/MIT) (used here), the Apache 2.0 License, and the Boost Software License. Copyright &copy; 2021 The fast_float authors
+10
View File
@@ -233,14 +233,20 @@ nav:
- 'Overview': api/basic_json_document/index.md
- '(Constructor)': api/basic_json_document/basic_json_document.md
- 'accept': api/basic_json_document/accept.md
- 'erase': api/basic_json_document/erase.md
- 'insert': api/basic_json_document/insert.md
- 'is_discarded': api/basic_json_document/is_discarded.md
- 'load': api/basic_json_document/load.md
- 'memory_usage': api/basic_json_document/memory_usage.md
- 'node_count': api/basic_json_document/node_count.md
- 'owns_source': api/basic_json_document/owns_source.md
- 'parse': api/basic_json_document/parse.md
- 'parse_copy': api/basic_json_document/parse_copy.md
- 'push_back': api/basic_json_document/push_back.md
- 'read': api/basic_json_document/read.md
- 'root': api/basic_json_document/root.md
- 'save': api/basic_json_document/save.md
- 'set': api/basic_json_document/set.md
- 'shrink_to_fit': api/basic_json_document/shrink_to_fit.md
- 'source': api/basic_json_document/source.md
- basic_json_view:
@@ -301,6 +307,8 @@ nav:
- 'to_json': api/adl_serializer/to_json.md
- 'json': api/json.md
- 'json_document': api/json_document.md
- 'json_editable_document': api/json_editable_document.md
- 'json_editable_view': api/json_editable_view.md
- json_pointer:
- 'Overview': api/json_pointer/index.md
- '(Constructor)': api/json_pointer/json_pointer.md
@@ -341,6 +349,8 @@ nav:
- 'operator""_json_pointer': api/operator_literal_json_pointer.md
- 'ordered_json': api/ordered_json.md
- 'ordered_json_document': api/ordered_json_document.md
- 'ordered_json_editable_document': api/ordered_json_editable_document.md
- 'ordered_json_editable_view': api/ordered_json_editable_view.md
- 'ordered_json_view': api/ordered_json_view.md
- 'ordered_map': api/ordered_map.md
- macros:
+259 -23
View File
@@ -11,11 +11,17 @@
#include <array> // array
#include <cmath> // signbit, isfinite
#include <cstddef> // size_t
#include <cstdint> // intN_t, uintN_t
#include <cstring> // memcpy, memmove
#include <limits> // numeric_limits
#include <type_traits> // conditional
#ifdef _MSC_VER
#include <cstdlib> // _byteswap_uint64
#endif
#include <nlohmann/detail/conversions/zmij.hpp>
#include <nlohmann/detail/macro_scope.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN
@@ -918,6 +924,87 @@ void grisu2(char* buf, int& len, int& decimal_exponent, FloatType value)
grisu2(buf, len, decimal_exponent, w.minus, w.w, w.plus);
}
/*!
@brief the shortest digits of a positive finite float (other than double): Grisu2
*/
template<typename FloatType>
JSON_HEDLEY_NON_NULL(1)
void shortest_digits(char* buf, int& len, int& decimal_exponent, FloatType value)
{
grisu2(buf, len, decimal_exponent, value);
}
/*!
@brief the shortest digits of a positive finite double: the conversion of
Zmij (see zmij.hpp), which always finds the shortest digits that read back as
the same value (Grisu2 does not for about one double in a thousand), and the
closest of them if there are several
v = buf * 10^decimal_exponent, as for grisu2()
*/
JSON_HEDLEY_NON_NULL(1)
inline void shortest_digits(char* buf, int& len, int& decimal_exponent, double value)
{
static_assert(std::numeric_limits<double>::is_iec559 && std::numeric_limits<double>::digits == 53,
"internal error: the conversion of Zmij needs IEEE 754 binary64 doubles");
JSON_ASSERT(std::isfinite(value));
JSON_ASSERT(value > 0);
std::uint64_t bits = 0;
std::memcpy(&bits, &value, sizeof(bits));
zmij::decimal d = zmij::to_decimal(bits);
// without trailing zeros (up to 16): 8, 4, 2, 1 at a time
while (d.significand % 100000000 == 0)
{
d.significand /= 100000000;
d.exponent += 8;
}
if (d.significand % 10000 == 0)
{
d.significand /= 10000;
d.exponent += 4;
}
if (d.significand % 100 == 0)
{
d.significand /= 100;
d.exponent += 2;
}
if (d.significand % 10 == 0)
{
d.significand /= 10;
d.exponent += 1;
}
// at most 17 digits, written from the back two at a time
static constexpr const char* pairs =
"00010203040506070809101112131415161718192021222324252627282930313233343536373839"
"40414243444546474849505152535455565758596061626364656667686970717273747576777879"
"8081828384858687888990919293949596979899";
std::array<char, 20> digits{};
std::size_t n = digits.size();
while (d.significand >= 100)
{
const auto i = static_cast<std::size_t>(d.significand % 100) * 2;
d.significand /= 100;
n -= 2;
digits[n] = pairs[i];
digits[n + 1] = pairs[i + 1];
}
if (d.significand >= 10)
{
const auto i = static_cast<std::size_t>(d.significand) * 2;
n -= 2;
digits[n] = pairs[i];
digits[n + 1] = pairs[i + 1];
}
else
{
digits[--n] = static_cast<char>('0' + d.significand);
}
len = static_cast<int>(digits.size() - n);
std::memcpy(buf, digits.data() + n, static_cast<std::size_t>(len));
decimal_exponent = d.exponent;
}
/*!
@brief appends a decimal representation of e to buf
@return a pointer to the element following the exponent.
@@ -1047,6 +1134,177 @@ inline char* format_buffer(char* buf, int len, int decimal_exponent,
return append_exponent(buf, n - 1);
}
/// eight decimal digits (a value below 10^8) as bytes 0..9, the first digit
/// in the most significant byte: three steps that divide all lanes at once
/// by a multiplication (the conversion of Xiang JunBo, as in Zmij)
inline std::uint64_t eight_digit_bytes(std::uint64_t abcdefgh) noexcept
{
const std::uint64_t abcd_efgh = abcdefgh + (((std::uint64_t{1} << 32u) - 10000u) * ((abcdefgh * (((std::uint64_t{1} << 40u) / 10000u) + 1u)) >> 40u));
const std::uint64_t ab_cd_ef_gh = abcd_efgh + (((std::uint64_t{1} << 16u) - 100u) * (((abcd_efgh * (((std::uint64_t{1} << 19u) / 100u) + 1u)) >> 19u) & 0x7F0000007Fu));
return ab_cd_ef_gh + (((std::uint64_t{1} << 8u) - 10u) * (((ab_cd_ef_gh * (((std::uint64_t{1} << 10u) / 10u) + 1u)) >> 10u) & 0x000F000F000F000Fu));
}
/// store the bytes of v, the most significant one first (one byte swap and
/// one store where the byte order is known: compilers do not reliably merge
/// the byte stores once this is inlined)
inline void store_msb_first(char* p, std::uint64_t v) noexcept
{
#if defined(__BYTE_ORDER__) && defined(__ORDER_LITTLE_ENDIAN__) && __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
v = __builtin_bswap64(v);
std::memcpy(p, &v, sizeof(v));
#elif defined(__BYTE_ORDER__) && defined(__ORDER_BIG_ENDIAN__) && __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__
std::memcpy(p, &v, sizeof(v));
#elif defined(_MSC_VER) // (little-endian on all its targets)
v = _byteswap_uint64(v);
std::memcpy(p, &v, sizeof(v));
#else
for (unsigned i = 0; i < 8; ++i)
{
p[i] = static_cast<char>(v >> (56u - (8u * i)));
}
#endif
}
/*!
@brief digits * 10^exp for a double, in the layout of format_buffer()
The layout is that of format_buffer() with min_exp -4 and max_exp 15 (the
digits10 of double). The digits are converted eight at a time and placed
with fixed-size moves instead of per-digit loops and moves of the buffer.
@param[in] digits the digits (not 0, at most 17 digits; trailing zeros allowed)
@param[in] exp the decimal exponent of the last digit
@return a pointer past the text; up to 41 bytes at @a first are written
(some beyond the returned end)
*/
JSON_HEDLEY_NON_NULL(1)
JSON_HEDLEY_RETURNS_NON_NULL
inline char* write_decimal(char* first, std::uint64_t digits, int exp) noexcept
{
JSON_ASSERT(digits != 0 && digits < 100000000000000000u);
const std::uint64_t upper = digits / 100000000u;
const std::uint64_t b0 = upper / 100000000u; // (one digit: it is its own byte)
const std::uint64_t b1 = eight_digit_bytes(upper % 100000000u);
const std::uint64_t b2 = eight_digit_bytes(digits % 100000000u);
// leading and trailing zero digits: zero bytes, counted without division
int leading = 16;
int zeros = 16;
if (b0 != 0)
{
leading = count_leading_zeros(b0) / 8;
}
else if (b1 != 0)
{
leading = 8 + (count_leading_zeros(b1) / 8);
}
else
{
leading += count_leading_zeros(b2) / 8;
}
if (b2 != 0)
{
zeros = count_trailing_zeros(b2) / 8;
}
else if (b1 != 0)
{
zeros = 8 + (count_trailing_zeros(b1) / 8);
}
// (else: 16, b0 is the one digit that is not 0)
// the digits as text at text + leading, then '0's, so that fixed-size
// moves need not check how many digits there are
std::array<char, 64> text; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
store_msb_first(text.data(), b0 + 0x3030303030303030u);
store_msb_first(text.data() + 8, b1 + 0x3030303030303030u);
store_msb_first(text.data() + 16, b2 + 0x3030303030303030u);
std::memset(text.data() + 24, '0', 40);
const int k = 24 - leading - zeros; // significant digits
const int n = k + exp + zeros; // position of the decimal point after the first digit
const char* const s0 = text.data() + leading;
if (-4 < n && n <= 15)
{
// "0.[000]digits" (n <= 0) is the digits after 1 - n leading '0's
// with the point after the first; "digits[000].0" (n >= k) and
// "dig.its" put the point after n characters
const int pad = n <= 0 ? 1 - n : 0;
const char* const s = s0 - pad;
const int len = k + pad;
const int point = n + pad;
std::memcpy(first, s, 16);
std::memcpy(first + point + 1, s + point, 24);
first[point] = '.';
return first + (point >= len ? point + 2 : len + 1);
}
// d.igitse+XX, with at least two exponent digits (as append_exponent())
std::memcpy(first, s0, 16);
std::memcpy(first + 2, s0 + 1, 16);
first[1] = '.';
char* const end = first + (k == 1 ? 1 : k + 1);
const int e = n - 1;
const auto ea = static_cast<unsigned>(e < 0 ? -e : e);
const bool three = ea >= 100;
end[0] = 'e';
end[1] = e < 0 ? '-' : '+';
end[2] = static_cast<char>('0' + (three ? ea / 100 : (ea / 10) % 10));
end[3] = static_cast<char>('0' + (three ? (ea / 10) % 10 : ea % 10));
end[4] = static_cast<char>('0' + (ea % 10));
return end + (three ? 5 : 4);
}
/// a positive finite float (other than double): Grisu2 and format_buffer()
template<typename FloatType>
JSON_HEDLEY_NON_NULL(1, 2)
JSON_HEDLEY_RETURNS_NON_NULL
char* write_positive(char* first, const char* last, FloatType value)
{
JSON_ASSERT(last - first >= std::numeric_limits<FloatType>::max_digits10);
// Compute v = buffer * 10^decimal_exponent.
// The decimal digits are stored in the buffer, which needs to be interpreted
// as an unsigned decimal integer.
// len is the length of the buffer, i.e., the number of decimal digits.
int len = 0;
int decimal_exponent = 0;
shortest_digits(first, len, decimal_exponent, value);
JSON_ASSERT(len <= std::numeric_limits<FloatType>::max_digits10);
// Format the buffer like printf("%.*g", prec, value)
constexpr int kMinExp = -4;
// Use digits10 here to increase compatibility with version 2.
constexpr int kMaxExp = std::numeric_limits<FloatType>::digits10;
JSON_ASSERT(last - first >= kMaxExp + 2);
JSON_ASSERT(last - first >= 2 + (-kMinExp - 1) + std::numeric_limits<FloatType>::max_digits10);
JSON_ASSERT(last - first >= std::numeric_limits<FloatType>::max_digits10 + 6);
return format_buffer(first, len, decimal_exponent, kMinExp, kMaxExp);
}
/// a positive finite double: the shortest digits (Zmij), laid out by
/// write_decimal() (through a local buffer if [first, last) is shorter than
/// the 41 bytes it may write)
JSON_HEDLEY_NON_NULL(1, 2)
JSON_HEDLEY_RETURNS_NON_NULL
inline char* write_positive(char* first, const char* last, double value)
{
static_assert(std::numeric_limits<double>::is_iec559 && std::numeric_limits<double>::digits == 53,
"internal error: the conversion of Zmij needs IEEE 754 binary64 doubles");
std::uint64_t bits = 0;
std::memcpy(&bits, &value, sizeof(bits));
const zmij::decimal d = zmij::to_decimal(bits);
if (JSON_HEDLEY_LIKELY(last - first >= 41))
{
return write_decimal(first, d.significand, d.exponent);
}
std::array<char, 64> buf; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
const auto len = static_cast<std::size_t>(write_decimal(buf.data(), d.significand, d.exponent) - buf.data());
JSON_ASSERT(static_cast<std::size_t>(last - first) >= len);
std::memcpy(first, buf.data(), len);
return first + len;
}
} // namespace dtoa_impl
/*!
@@ -1064,7 +1322,6 @@ JSON_HEDLEY_NON_NULL(1, 2)
JSON_HEDLEY_RETURNS_NON_NULL
char* to_chars(char* first, const char* last, FloatType value)
{
static_cast<void>(last); // maybe unused - fix warning
JSON_ASSERT(std::isfinite(value));
// Use signbit(value) instead of (value < 0) since signbit works for -0.
@@ -1090,28 +1347,7 @@ char* to_chars(char* first, const char* last, FloatType value)
JSON_HEDLEY_DIAGNOSTIC_POP
#endif
JSON_ASSERT(last - first >= std::numeric_limits<FloatType>::max_digits10);
// Compute v = buffer * 10^decimal_exponent.
// The decimal digits are stored in the buffer, which needs to be interpreted
// as an unsigned decimal integer.
// len is the length of the buffer, i.e., the number of decimal digits.
int len = 0;
int decimal_exponent = 0;
dtoa_impl::grisu2(first, len, decimal_exponent, value);
JSON_ASSERT(len <= std::numeric_limits<FloatType>::max_digits10);
// Format the buffer like printf("%.*g", prec, value)
constexpr int kMinExp = -4;
// Use digits10 here to increase compatibility with version 2.
constexpr int kMaxExp = std::numeric_limits<FloatType>::digits10;
JSON_ASSERT(last - first >= kMaxExp + 2);
JSON_ASSERT(last - first >= 2 + (-kMinExp - 1) + std::numeric_limits<FloatType>::max_digits10);
JSON_ASSERT(last - first >= std::numeric_limits<FloatType>::max_digits10 + 6);
return dtoa_impl::format_buffer(first, len, decimal_exponent, kMinExp, kMaxExp);
return dtoa_impl::write_positive(first, last, value);
}
} // namespace detail
@@ -0,0 +1,218 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2025 Victor Zverovich <https://github.com/vitaut/zmij>
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#pragma once
#include <array> // array
#include <cstddef> // size_t
#include <cstdint> // uint32_t, uint64_t
#include <nlohmann/detail/abi_macros.hpp>
#include <nlohmann/detail/bit_ops.hpp>
#include <nlohmann/detail/input/pow5_table.hpp>
#include <nlohmann/detail/macro_scope.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
{
/*!
@brief the shortest decimal representation of a double
A C++11 port of the conversion of Zmij by Victor Zverovich
(https://github.com/vitaut/zmij, MIT license): the shortest decimal in the
rounding interval of a double, the closest one if there are several. Zmij
credits Xiang JunBo (producing the shorter candidate without a division) and
Dougall Johnson (the compressed powers of ten). The powers of ten are taken
from the table for number parsing (pow5_table.hpp) where it holds them, and
computed from the compressed tables of Zmij beyond it.
*/
namespace zmij
{
/// significand * 10^exponent
struct decimal
{
std::uint64_t significand;
int exponent;
};
/// the compressed powers of ten of Zmij
inline const std::array<std::uint64_t, 28>& pow10_minor() noexcept
{
static const std::array<std::uint64_t, 28> table =
{
{
0x8000000000000000u, 0xa000000000000000u, 0xc800000000000000u, 0xfa00000000000000u, 0x9c40000000000000u,
0xc350000000000000u, 0xf424000000000000u, 0x9896800000000000u, 0xbebc200000000000u, 0xee6b280000000000u,
0x9502f90000000000u, 0xba43b74000000000u, 0xe8d4a51000000000u, 0x9184e72a00000000u, 0xb5e620f480000000u,
0xe35fa931a0000000u, 0x8e1bc9bf04000000u, 0xb1a2bc2ec5000000u, 0xde0b6b3a76400000u, 0x8ac7230489e80000u,
0xad78ebc5ac620000u, 0xd8d726b7177a8000u, 0x878678326eac9000u, 0xa968163f0a57b400u, 0xd3c21bcecceda100u,
0x84595161401484a0u, 0xa56fa5b99019a5c8u, 0xcecb8f27f4200f3au
}
};
return table;
}
/// (high, low) pairs
inline const std::array<std::uint64_t, 50>& pow10_major() noexcept
{
static const std::array<std::uint64_t, 50> table =
{
{
0xaddcb9e83c6b1793u, 0xdf4abe242a1bbf3eu, 0xaf8e5410288e1b6fu, 0x07ecf0ae5ee44ddau, 0xb1442798f49ffb4au, 0x99cd11cfdf41779du,
0xb2fe3f0b8599ef07u, 0x861fa7e6dcb4aa15u, 0xb4bca50b065abe63u, 0x0fed077a756b53aau, 0xb67f6455292cbf08u, 0x1a3bc84c17b1d543u,
0xb84687c269ef3bfbu, 0x3d5d514f40eea742u, 0xba121a4650e4ddebu, 0x92f34d62616ce413u, 0xbbe226efb628afeau, 0x890489f70a55368cu,
0xbdb6b8e905cb600fu, 0x5400e987bbc1c921u, 0xbf8fdb78849a5f96u, 0xde98520472bdd034u, 0xc16d9a0095928a27u, 0x75b7053c0f178294u,
0xc350000000000000u, 0x0000000000000000u, 0xc5371912364ce305u, 0x6c28000000000000u, 0xc722f0ef9d80aad6u, 0x424d3ad2b7b97ef6u,
0xc913936dd571c84cu, 0x03bc3a19cd1e38eau, 0xcb090c8001ab551cu, 0x5cadf5bfd3072cc6u, 0xcd036837130890a1u, 0x36dba887c37a8c10u,
0xcf02b2c21207ef2eu, 0x94f967e45e03f4bcu, 0xd106f86e69d785c7u, 0xe13336d701beba52u, 0xd31045a8341ca07cu, 0x1ede48111209a051u,
0xd51ea6fa85785631u, 0x552a74227f3ea566u, 0xd732290fbacaf133u, 0xa97c177947ad4096u, 0xd94ad8b1c7380874u, 0x18375281ae7822bdu,
0xdb68c2ca82ed2a05u, 0xa67398db9f6820e1u
}
};
return table;
}
/// one bit per power: whether the computed value is one unit too large
inline const std::array<std::uint32_t, 21>& pow10_fixups() noexcept
{
static const std::array<std::uint32_t, 21> table =
{
{
0x8d8fc810u, 0x06100293u, 0x19000000u, 0x00100000u, 0x00000908u, 0x00000000u, 0x04e00300u, 0x3807e0b2u, 0x3d83d793u, 0x0006f5ccu,
0x00000000u, 0xffff0000u, 0x8076337du, 0x4ff45ba0u, 0x09405033u, 0x034376d9u, 0x09000000u, 0x4e100501u, 0x076d14dcu, 0xf964f45eu,
0x0000003du
}
};
return table;
}
/// the 128-bit significand of 10^k, rounded down, for k in [-307, 341]
/// (compute_pow10 of Zmij)
inline uint128_parts compute_pow10(int k) noexcept
{
const auto i = static_cast<unsigned>(k + 307);
const std::uint64_t m = pow10_minor()[(i + 24) % 28];
const std::size_t j = 2 * static_cast<std::size_t>((i + 24) / 28);
const std::uint64_t h_hi = pow10_major()[j];
const std::uint64_t h_lo = pow10_major()[j + 1];
const std::uint64_t h1 = full_multiplication(h_lo, m).high;
const std::uint64_t c0 = h_lo * m;
const std::uint64_t c1 = h1 + (h_hi * m);
const std::uint64_t c2 = (c1 < h1 ? 1u : 0u) + full_multiplication(h_hi, m).high;
uint128_parts r{};
if ((c2 >> 63u) != 0)
{
r.high = c2;
r.low = c1;
}
else
{
r.high = (c2 << 1u) | (c1 >> 63u);
r.low = (c1 << 1u) | (c0 >> 63u);
}
r.low -= (pow10_fixups()[i >> 5u] >> (i & 31u)) & 1u;
return r;
}
/// The 128-bit significand of 10^k, rounded down, for k in [-342, 341].
/// Up to 10^308, the table for number parsing holds the same significands
/// (those of 5^k), except for k in [-27, -1], where it holds them one unit
/// larger (as the Eisel-Lemire algorithm needs them).
inline uint128_parts pow10(int k) noexcept
{
if (k > pow5_128_largest_power)
{
return compute_pow10(k); // (only for the smallest doubles)
}
const auto i = 2 * static_cast<std::size_t>(k - pow5_128_smallest_power);
uint128_parts r{pow5_128()[i + 1], pow5_128()[i]};
const std::uint64_t adjust = static_cast<unsigned>(k + 27) < 27u ? 1u : 0u;
r.high -= r.low < adjust ? 1u : 0u;
r.low -= adjust;
return r;
}
/// (x_hi * 2^64 + x_lo) * y >> 64, as 128 bits
inline uint128_parts umul192_hi128(std::uint64_t x_hi, std::uint64_t x_lo, std::uint64_t y) noexcept
{
const uint128_parts p = full_multiplication(x_hi, y);
uint128_parts r{};
r.low = p.low + full_multiplication(x_lo, y).high;
r.high = p.high + (r.low < p.low ? 1u : 0u);
return r;
}
/// (x * y + c) >> 64
inline std::uint64_t umul128_add_hi64(std::uint64_t x, std::uint64_t y, std::uint64_t c) noexcept
{
const uint128_parts p = full_multiplication(x, y);
return p.high + (p.low + c < p.low ? 1u : 0u);
}
/// The shortest decimal in the rounding interval of a positive finite double
/// given by its bits, the closest one if there are several (to_decimal of
/// Zmij). The significand can end in zeros.
inline decimal to_decimal(std::uint64_t bits) noexcept
{
constexpr int extra_shift = 9;
const auto raw_exp = static_cast<int>((bits >> 52u) & 0x7FFu);
std::uint64_t bin_sig = bits & ((std::uint64_t{1} << 52u) - 1);
// a power of two has a narrower interval below (except the smallest normal)
const bool regular = bin_sig != 0 || raw_exp <= 1;
const int bin_exp = (raw_exp == 0 ? 1 : raw_exp) - 1075;
if (raw_exp != 0)
{
bin_sig |= std::uint64_t{1} << 52u;
}
// floor(log10(2^bin_exp)), or floor(log10(3/4 * 2^bin_exp)) for the irregular case
const int dec_exp = ((bin_exp * 315653) - (regular ? 0 : 131072)) >> 20;
// scaled by 10^(-dec_exp - 1): the integral part is the shorter candidate
const int shift = bin_exp + ((-(dec_exp + 1) * 217707) >> 16) + 1 + extra_shift;
const uint128_parts p10 = pow10(-dec_exp - 1);
const uint128_parts p = umul192_hi128(p10.high, p10.low, bin_sig << static_cast<unsigned>(shift));
std::uint64_t integral = p.high >> static_cast<unsigned>(extra_shift);
const std::uint64_t fractional = (p.high << static_cast<unsigned>(64 - extra_shift)) | (p.low >> static_cast<unsigned>(extra_shift));
std::uint64_t digit = 0;
bool round_up = false;
bool round_down = false;
if (JSON_HEDLEY_LIKELY(regular))
{
const std::uint64_t half_ulp = (p10.high >> static_cast<unsigned>(extra_shift + 1 - shift)) + (1 - (bin_sig & 1u));
round_up = fractional + half_ulp < fractional;
round_down = half_ulp > fractional;
// the last digit of the longer candidate, rounded to nearest
digit = umul128_add_hi64(fractional, 10, (std::uint64_t{1} << 63u) + 6);
if (fractional == (std::uint64_t{1} << 62u))
{
digit = 2; // 2.5 rounds to 2
}
}
else
{
const std::uint64_t half_ulp = p10.high >> static_cast<unsigned>(extra_shift + 1 - shift);
round_up = half_ulp > ~std::uint64_t{0} - fractional;
round_down = (half_ulp >> 1u) > fractional;
digit = umul128_add_hi64(fractional, 10, (std::uint64_t{1} << 63u) - 1);
const std::uint64_t lowest = umul128_add_hi64(fractional - (half_ulp >> 1u), 10, ~std::uint64_t{0});
digit = digit < lowest ? lowest : digit;
}
integral += round_up ? 1u : 0u;
if (!round_up && !round_down)
{
// the shorter candidate is outside the rounding interval: one digit more
return decimal{(integral * 10) + digit, dec_exp};
}
return decimal{integral, dec_exp + 1};
}
} // namespace zmij
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
+100 -2
View File
@@ -10,8 +10,11 @@
#include <array> // array
#include <cstddef> // size_t
#include <cstdint> // uint32_t
#include <cstdint> // uint8_t, uint32_t
#include <cstring> // memcpy
#include <functional> // less
#include <map> // map
#include <memory> // unique_ptr
#include <new> // operator new, placement new
#include <string> // string
#include <vector> // vector
@@ -38,7 +41,9 @@ struct document_data
node* inline_tape = nullptr; ///< node array allocated together with this header
std::size_t inline_cap = 0;
std::string arena{}; ///< decoded strings that contained escapes // NOLINT(readability-redundant-member-init)
std::size_t arena_size = 0; ///< bytes of decoded strings at base[1] (the arena, or those of a loaded image)
std::string owned{}; ///< owned copy of the input, if any // NOLINT(readability-redundant-member-init)
std::vector<std::uint8_t> owned_image{}; ///< a loaded image the document owns (the text and the decoded strings point into it) // NOLINT(readability-redundant-member-init)
// hash indexes of large objects (see object_index.hpp)
static constexpr std::uint32_t index_min_members = 128;
@@ -50,9 +55,30 @@ struct document_data
std::vector<object_index> indexes{}; // NOLINT(readability-redundant-member-init)
std::vector<std::uint32_t> index_slots{}; // NOLINT(readability-redundant-member-init)
std::vector<std::uint32_t> large_objects{}; ///< positions of the objects to index (noted while parsing) // NOLINT(readability-redundant-member-init)
std::array<const char*, 4> base = {{nullptr, nullptr, nullptr, nullptr}}; ///< string bases: source, arena (indexed by flags & node_flags::storage)
std::array<const char*, 4> base = {{nullptr, nullptr, nullptr, nullptr}}; ///< string bases: source, arena, edit arena (indexed by flags & node_flags::storage)
bool discarded = true;
/// The storage of edits (editable documents only; see edit_storage.hpp).
/// Edits never move or resize the parsed index, so views stay valid: an
/// array/object whose elements change gets node_flags::moved, and its
/// elements then live in a separate sequence (a header node, then the
/// entries), whose entries link to the values.
struct edit_state
{
std::vector<node*> moved{}; ///< element sequences of moved arrays/objects (header node first) // NOLINT(readability-redundant-member-init)
std::vector<std::size_t> moved_cap{}; ///< capacity in nodes of a growable block; 0: a fixed sequence (a new value) // NOLINT(readability-redundant-member-init)
std::vector<std::unique_ptr<node[]>> chunks{}; ///< storage of new values and blocks; never moved // NOLINT(readability-redundant-member-init,cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
std::map<const node*, node*, std::less<const node*>> regions{}; ///< new arrays/objects: root -> container that uses it as its element sequence (nullptr: linked from a block) // NOLINT(readability-redundant-member-init)
node* chunk_cur = nullptr;
node* chunk_end = nullptr;
std::size_t chunk_next = 64;
std::vector<std::unique_ptr<char[]>> texts{}; ///< edit arena, the current buffer last; earlier ones stay alive for string views // NOLINT(readability-redundant-member-init,cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
std::size_t text_used = 0;
std::size_t text_cap = 0;
std::size_t bytes = 0; ///< memory held by edits
};
std::unique_ptr<edit_state> edits{}; ///< created by the first edit // NOLINT(readability-redundant-member-init)
/// one allocation for the header and room for `nodes` nodes; large
/// documents get a separate node array instead (so it can be trimmed)
static document_data* create(std::size_t nodes)
@@ -136,6 +162,78 @@ struct document_data
{
return n + n->next;
}
/// (editable documents) first element or key, also of a moved container
NLOHMANN_VIEW_ALWAYS_INLINE const node* first_child_edited(const node* n) const noexcept
{
return NLOHMANN_VIEW_LIKELY((n->flags & node_flags::moved) == 0) ? n + 1 : edits->moved[n->off] + 1;
}
/// (editable documents) end of the elements, also of a moved container
NLOHMANN_VIEW_ALWAYS_INLINE const node* child_end_edited(const node* n) const noexcept
{
if (NLOHMANN_VIEW_LIKELY((n->flags & node_flags::moved) == 0))
{
return n + n->next;
}
const node* const h = edits->moved[n->off];
return h + h->next;
}
/// (editable documents) the value at an element position: entries of
/// moved sequences are links. The link case is out of line, so that this
/// compiles to a predicted branch rather than a select that delays the
/// following loads.
static NLOHMANN_VIEW_ALWAYS_INLINE const node* deref(const node* n) noexcept
{
return NLOHMANN_VIEW_LIKELY(n->kind != kind_link) ? n : follow_link(n);
}
static NLOHMANN_VIEW_NOINLINE const node* follow_link(const node* n) noexcept
{
return link_target(*n);
}
};
/// How the index is walked: views of read-only documents follow the node
/// array alone and compile without any of the edit handling; views of
/// editable documents also follow moved element sequences and links.
template<bool Editable>
struct navigation
{
static NLOHMANN_VIEW_ALWAYS_INLINE const node* first(const document_data& /*d*/, const node* n) noexcept
{
return n + 1;
}
static NLOHMANN_VIEW_ALWAYS_INLINE const node* end(const document_data& /*d*/, const node* n) noexcept
{
return n + n->next;
}
static NLOHMANN_VIEW_ALWAYS_INLINE const node* value(const node* n) noexcept
{
return n;
}
};
template<>
struct navigation<true>
{
static NLOHMANN_VIEW_ALWAYS_INLINE const node* first(const document_data& d, const node* n) noexcept
{
return d.first_child_edited(n);
}
static NLOHMANN_VIEW_ALWAYS_INLINE const node* end(const document_data& d, const node* n) noexcept
{
return d.child_end_edited(n);
}
static NLOHMANN_VIEW_ALWAYS_INLINE const node* value(const node* n) noexcept
{
return document_data::deref(n);
}
};
} // namespace view
+767
View File
@@ -0,0 +1,767 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#pragma once
#include <array> // array
#include <cmath> // isinf, isnan
#include <cstddef> // size_t
#include <cstdint> // int64_t, uint8_t, uint32_t, uint64_t
#include <cstring> // memcmp, memmove
#include <limits> // numeric_limits
#include <string> // string, to_string
#include <type_traits> // decay, enable_if, integral_constant, is_arithmetic, is_convertible, is_floating_point, is_same, is_signed
#include <utility> // forward
#include <nlohmann/json.hpp>
#include <nlohmann/detail/view/document_data.hpp>
#include <nlohmann/detail/view/edit_storage.hpp>
#include <nlohmann/detail/view/errors.hpp>
#include <nlohmann/detail/view/lookup.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
#include <nlohmann/detail/view/node.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN
template<typename BasicJsonType, bool Editable>
class basic_json_view;
namespace detail
{
namespace view
{
/// the index of the first true condition (the number of conditions if none is)
template<bool... Conditions>
struct first_true : std::integral_constant<int, 0> {};
template<bool... Conditions>
struct first_true<false, Conditions...> : std::integral_constant < int, 1 + first_true<Conditions...>::value > {};
/// Checks a string the way basic_json's serializer does when it writes it
/// (type_error.316 with the same message), so that an editable document
/// only holds valid UTF-8: the error is at the first byte that no
/// well-formed sequence can continue with (Unicode, Table 3-7).
inline void check_utf8(const char* s, std::size_t n)
{
const auto* const p = reinterpret_cast<const unsigned char*>(s); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
const auto hex = [](unsigned char c)
{
constexpr const char* digits = "0123456789ABCDEF";
return std::string{digits[c >> 4u], digits[c & 0xFu]};
};
for (std::size_t i = 0; i < n;)
{
const unsigned char c = p[i];
if (c < 0x80)
{
++i;
continue;
}
std::size_t len = 0;
unsigned char lo = 0x80;
unsigned char hi = 0xBF;
if (c >= 0xC2 && c <= 0xDF)
{
len = 2;
}
else if (c >= 0xE0 && c <= 0xEF)
{
len = 3;
lo = c == 0xE0 ? 0xA0 : 0x80;
hi = c == 0xED ? 0x9F : 0xBF;
}
else if (c >= 0xF0 && c <= 0xF4)
{
len = 4;
lo = c == 0xF0 ? 0x90 : 0x80;
hi = c == 0xF4 ? 0x8F : 0xBF;
}
else
{
throw_type_error(316, concat("invalid UTF-8 byte at index ", std::to_string(i), ": 0x", hex(c)));
}
for (std::size_t k = 1; k < len; ++k)
{
if (i + k == n)
{
throw_type_error(316, concat("incomplete UTF-8 string; last byte: 0x", hex(p[n - 1])));
}
const unsigned char b = p[i + k];
if (b < (k == 1 ? lo : 0x80) || b > (k == 1 ? hi : 0xBF))
{
throw_type_error(316, concat("invalid UTF-8 byte at index ", std::to_string(i + k), ": 0x", hex(b)));
}
}
i += len;
}
}
/*!
@brief the edits of an editable basic_json_document
Values are accepted as views (of any document), BasicJsonType values, and
everything BasicJsonType can be constructed from. The source text is never
written: new values go to storage owned by the document (see
edit_storage.hpp).
*/
template<typename BasicJsonType, typename View>
class editor
{
using number_integer_t = typename BasicJsonType::number_integer_t;
using number_unsigned_t = typename BasicJsonType::number_unsigned_t;
using number_float_t = typename BasicJsonType::number_float_t;
using string_t = typename BasicJsonType::string_t;
using string_view_t = typename View::string_view_t;
using nav = navigation<true>;
public:
explicit editor(document_data& d) noexcept
: m_doc(d)
{}
/// replace a value; returns its view
template<typename V>
View set(const View& target, V&& value)
{
node* const slot = own(target);
const encoded e = encode(std::forward<V>(value));
assign(slot, e, nullptr, false);
return View(&m_doc, slot);
}
/// set a member (appended if missing; a null value becomes an object);
/// returns a view of the member value
template<typename V>
View set(const View& object, string_view_t key, V&& value)
{
node* const o = own(object);
if (o->kind != static_cast<std::uint8_t>(value_t::object) && o->kind != static_cast<std::uint8_t>(value_t::null))
{
throw_type_error(305, "cannot use operator[] with a string argument with ", object.type_name());
}
check_utf8(key.data(), key.size());
const encoded e = encode(std::forward<V>(value));
if (o->kind == static_cast<std::uint8_t>(value_t::null))
{
become_empty(o, value_t::object);
}
// an existing member: assign it (and drop later duplicates, so that
// lookups, iteration, and materialize() agree)
node* slot = nullptr;
bool duplicates = false;
for (const node* k = nav::first(m_doc, o), *end = nav::end(m_doc, o); k != end; k = document_data::after(k + 1))
{
if (key_equals(*k, key))
{
if (slot != nullptr)
{
duplicates = true;
break;
}
slot = const_cast<node*>(nav::value(k + 1)); // NOLINT(cppcoreguidelines-pro-type-const-cast): the nodes belong to this document
}
}
if (slot != nullptr)
{
if (duplicates)
{
erase_members(o, key, true);
}
assign(slot, e, o, true);
return View(&m_doc, slot);
}
const node k = string_node(key.data(), key.size());
slot = new_slot(e);
node* const h = block_of(m_doc, o, 2);
h[h->next] = k;
make_link(h[h->next + 1], slot);
h->next += 2;
++h->len;
++o->len;
return View(&m_doc, slot);
}
/// assign an existing array element; returns a view of it
template<typename V>
View set(const View& array, std::size_t idx, V&& value)
{
node* const a = own(array);
if (a->kind != static_cast<std::uint8_t>(value_t::array))
{
throw_type_error(305, "cannot use operator[] with a numeric argument with ", array.type_name());
}
check_index(idx, a->len);
const encoded e = encode(std::forward<V>(value));
node* const slot = const_cast<node*>(nav::value(element_at<true>(m_doc, a, idx))); // NOLINT(cppcoreguidelines-pro-type-const-cast)
assign(slot, e, a, true);
return View(&m_doc, slot);
}
/// append to an array (a null value becomes an array); returns a view of
/// the new element
template<typename V>
View push_back(const View& array, V&& value)
{
node* const a = own(array);
if (a->kind != static_cast<std::uint8_t>(value_t::array) && a->kind != static_cast<std::uint8_t>(value_t::null))
{
throw_type_error(308, "cannot use push_back() with ", array.type_name());
}
const encoded e = encode(std::forward<V>(value));
if (a->kind == static_cast<std::uint8_t>(value_t::null))
{
become_empty(a, value_t::array);
}
node* const slot = new_slot(e);
node* const h = block_of(m_doc, a, 1);
make_link(h[h->next], slot);
++h->next;
++h->len;
++a->len;
return View(&m_doc, slot);
}
/// insert into an array before position idx (idx <= size()); returns a
/// view of the new element
template<typename V>
View insert(const View& array, std::size_t idx, V&& value)
{
node* const a = own(array);
if (a->kind != static_cast<std::uint8_t>(value_t::array))
{
throw_type_error(309, "cannot use insert() with ", array.type_name());
}
check_index(idx, a->len + 1);
const encoded e = encode(std::forward<V>(value));
node* const slot = new_slot(e);
node* const h = block_of(m_doc, a, 1);
std::memmove(h + 2 + idx, h + 1 + idx, (h->next - 1 - idx) * sizeof(node));
make_link(h[1 + idx], slot);
++h->next;
++h->len;
++a->len;
return View(&m_doc, slot);
}
/// remove all members with this key; returns their number
std::size_t erase(const View& object, string_view_t key)
{
node* const o = own(object);
if (o->kind != static_cast<std::uint8_t>(value_t::object))
{
throw_type_error(307, "cannot use erase() with ", object.type_name());
}
for (const node* k = nav::first(m_doc, o), *end = nav::end(m_doc, o); k != end; k = document_data::after(k + 1))
{
if (key_equals(*k, key))
{
return erase_members(o, key, false);
}
}
return 0;
}
/// remove an array element
void erase(const View& array, std::size_t idx)
{
node* const a = own(array);
if (a->kind != static_cast<std::uint8_t>(value_t::array))
{
throw_type_error(307, "cannot use erase() with ", array.type_name());
}
check_index(idx, a->len);
node* const h = block_of(m_doc, a, 0);
std::memmove(h + 1 + idx, h + 2 + idx, (h->next - 2 - idx) * sizeof(node));
--h->next;
--h->len;
--a->len;
}
private:
/// an encoded value: a scalar node, or the root of a new array/object
struct encoded
{
node scalar{};
node* region = nullptr;
};
/// the node of a view of this document
node* own(const View& v)
{
if (NLOHMANN_VIEW_UNLIKELY(v.m_doc != &m_doc || v.m_node == nullptr))
{
throw_invalid_iterator(202, "view does not belong to this document");
}
edit_state_of(m_doc);
return const_cast<node*>(v.m_node); // NOLINT(cppcoreguidelines-pro-type-const-cast): the nodes belong to this document
}
static void check_index(std::size_t idx, std::size_t limit)
{
if (idx >= limit)
{
throw_out_of_range(401, concat("array index ", std::to_string(idx), " is out of range"));
}
}
bool key_equals(const node& k, string_view_t key) const noexcept
{
return k.len == key.size() && (key.size() == 0 || std::memcmp(m_doc.str(k), key.data(), key.size()) == 0);
}
/// remove the members with this key (all, or all but the first) from an object
std::size_t erase_members(node* o, string_view_t key, bool keep_first)
{
node* const h = block_of(m_doc, o, 0);
node* w = h + 1;
std::size_t erased = 0;
bool kept = false;
for (node* r = h + 1, *end = h + h->next; r != end; r += 2)
{
const bool match = key_equals(*r, key);
if (match && (kept || !keep_first))
{
++erased;
continue;
}
kept = kept || match;
if (w != r)
{
w[0] = r[0];
w[1] = r[1];
}
w += 2;
}
h->next = static_cast<std::uint32_t>(w - h);
h->len -= static_cast<std::uint32_t>(erased);
o->len -= static_cast<std::uint32_t>(erased);
return erased;
}
/// turn a null into an empty array/object in place
static void become_empty(node* n, value_t k) noexcept
{
*n = node{};
n->kind = static_cast<std::uint8_t>(k);
n->flags = node_flags::is_new;
n->next = 1;
}
/// Replace the value at slot; `parent` is the container whose elements
/// include slot (if known).
void assign(node* slot, const encoded& e, node* parent, bool parent_known)
{
if (e.region == nullptr)
{
if (is_container(*slot) && slot->next > 1 && slot != m_doc.tape)
{
// The slot spans its old elements in the enclosing sequence, but
// a scalar is one node: the enclosing container first switches to
// links (then the extent of the slot no longer matters).
node* const p = parent_known ? parent : find_parent(m_doc, slot);
if (p != nullptr && ((p->flags & node_flags::moved) == 0 || moved_capacity(m_doc, p) == 0))
{
block_of(m_doc, p, 0);
}
}
*slot = e.scalar;
return;
}
// an array/object: the slot keeps its extent (so that the enclosing
// sequence still steps over it), and the elements come from the new
// sequence
const node* const r = e.region;
const std::uint32_t extent = is_container(*slot) ? slot->next : 1;
const bool was_moved = (slot->flags & node_flags::moved) != 0;
slot->kind = r->kind;
slot->extra = 0;
slot->len = r->len;
slot->next = extent;
slot->flags = was_moved ? static_cast<std::uint8_t>(node_flags::moved | node_flags::is_new) : std::uint8_t{0};
set_moved(m_doc, slot, e.region, 0);
edit_state_of(m_doc).regions[e.region] = slot;
}
/// a node for a new element (links point to it; it never moves)
node* new_slot(const encoded& e)
{
if (e.region != nullptr)
{
return e.region;
}
node* const s = alloc_nodes(m_doc, 1);
*s = e.scalar;
return s;
}
//////////////
// encoding //
//////////////
template<int N>
using encode_tag = std::integral_constant<int, N>;
template<typename T>
struct is_view : std::false_type {};
template<typename J, bool E>
struct is_view<basic_json_view<J, E>> : std::true_type {};
template<typename V>
encoded encode(V&& v)
{
using D = typename std::decay<V>::type;
return encode_impl(std::forward<V>(v), encode_tag<first_true<is_view<D>::value,
std::is_same<D, BasicJsonType>::value,
std::is_same<D, std::nullptr_t>::value,
std::is_same<D, bool>::value,
std::is_arithmetic<D>::value,
std::is_convertible<const D&, string_view_t>::value>::value> {});
}
/// a view of any document (copied; nothing is shared with it)
template<typename J, bool E>
encoded encode_impl(const basic_json_view<J, E>& v, encode_tag<0> /*view*/)
{
if (NLOHMANN_VIEW_UNLIKELY(v.m_node == nullptr))
{
throw_type_error(302, "type must be a value, but is ", "discarded");
}
encoded r;
if (!is_container(*v.m_node))
{
r.scalar = copy_scalar(*v.m_doc, *v.m_node);
return r;
}
r.region = alloc_nodes(m_doc, count_nodes<E>(*v.m_doc, v.m_node));
fill_nodes<E>(*v.m_doc, v.m_node, r.region);
edit_state_of(m_doc).regions.emplace(r.region, nullptr);
return r;
}
encoded encode_impl(const BasicJsonType& j, encode_tag<1> /*json*/)
{
encoded r;
if (!j.is_structured())
{
r.scalar = json_scalar(j);
return r;
}
r.region = alloc_nodes(m_doc, count_nodes(j));
fill_nodes(j, r.region);
edit_state_of(m_doc).regions.emplace(r.region, nullptr);
return r;
}
encoded encode_impl(std::nullptr_t /*unused*/, encode_tag<2> /*null*/)
{
encoded r;
r.scalar = plain_node(value_t::null);
return r;
}
encoded encode_impl(bool b, encode_tag<3> /*boolean*/)
{
encoded r;
r.scalar = plain_node(value_t::boolean);
r.scalar.flags = static_cast<std::uint8_t>(r.scalar.flags | (b ? node_flags::is_true : 0));
return r;
}
template<typename T>
encoded encode_impl(T x, encode_tag<4> /*number*/)
{
encoded r;
r.scalar = number_node(x, std::integral_constant<int, first_true<std::is_floating_point<T>::value, std::is_signed<T>::value>::value> {});
return r;
}
template<typename T>
encoded encode_impl(const T& s, encode_tag<5> /*string*/)
{
const string_view_t sv(s);
check_utf8(sv.data(), sv.size());
encoded r;
r.scalar = string_node(sv.data(), sv.size());
return r;
}
template<typename T>
encoded encode_impl(T&& x, encode_tag<6> /*other*/)
{
return encode_impl(BasicJsonType(std::forward<T>(x)), encode_tag<1> {});
}
static node plain_node(value_t k) noexcept
{
node n{};
n.kind = static_cast<std::uint8_t>(k);
n.flags = node_flags::is_new;
return n;
}
template<typename T>
node number_node(T x, std::integral_constant<int, 0> /*floating-point*/)
{
return float_node(static_cast<number_float_t>(x));
}
template<typename T>
node number_node(T x, std::integral_constant<int, 1> /*signed*/)
{
return integer_node(static_cast<std::uint64_t>(static_cast<std::int64_t>(x)), value_t::number_integer);
}
template<typename T>
node number_node(T x, std::integral_constant<int, 2> /*unsigned*/)
{
return integer_node(static_cast<std::uint64_t>(x), value_t::number_unsigned);
}
/// an integer with its canonical token in the edit arena
node integer_node(std::uint64_t bits, value_t k)
{
const bool negative = k == value_t::number_integer && static_cast<std::int64_t>(bits) < 0;
std::uint64_t magnitude = negative ? 0 - bits : bits;
std::array<char, 24> buf{};
char* p = buf.data() + buf.size();
do
{
*--p = static_cast<char>('0' + (magnitude % 10));
magnitude /= 10;
}
while (magnitude != 0);
if (negative)
{
*--p = '-';
}
const auto len = static_cast<std::size_t>(buf.data() + buf.size() - p);
node n = plain_node(k);
n.flags = static_cast<std::uint8_t>(n.flags | node_flags::edited);
n.off = append_text(m_doc, p, len);
// number_length() adds one for the sign of number_integer nodes
n.extra = static_cast<std::uint16_t>(k == value_t::number_integer ? len - 1 : len);
set_integer_bits(n, bits);
return n;
}
/// a float with its shortest round-trip token (as basic_json::dump()
/// writes it), or nan, inf, -inf, in the edit arena
node float_node(number_float_t x)
{
string_t text;
if (std::isnan(x))
{
text = "nan";
}
else if (std::isinf(x))
{
text = x > 0 ? "inf" : "-inf";
}
else
{
text = BasicJsonType(x).dump();
}
node n = plain_node(value_t::number_float);
n.flags = static_cast<std::uint8_t>(n.flags | node_flags::edited);
n.extra = 0xFFFFu; // (the digit layout is not recorded)
n.off = append_text(m_doc, text.data(), text.size());
n.len = static_cast<std::uint32_t>(text.size());
return n;
}
/// a string (or key) in the edit arena
node string_node(const char* s, std::size_t len)
{
if (NLOHMANN_VIEW_UNLIKELY(len >= 0xFFFFFFFFu))
{
throw_out_of_range(416, "strings of 4 GiB or more are not supported by json_document"); // LCOV_EXCL_LINE
}
node n = plain_node(value_t::string);
n.flags = static_cast<std::uint8_t>(n.flags | node_flags::edited);
n.off = append_text(m_doc, s, len);
n.len = static_cast<std::uint32_t>(len);
return n;
}
/// a scalar of a view (of any document) as a node of this document
node copy_scalar(const document_data& from, const node& n)
{
if (&from == &m_doc)
{
return n; // the same storage
}
switch (static_cast<value_t>(n.kind))
{
case value_t::string:
return string_node(from.str(n), n.len);
case value_t::number_integer:
case value_t::number_unsigned:
return integer_node(integer_bits(n), static_cast<value_t>(n.kind));
case value_t::number_float:
{
node r = plain_node(value_t::number_float);
r.flags = static_cast<std::uint8_t>(r.flags | node_flags::edited);
r.off = append_text(m_doc, from.str(n), n.len);
r.len = n.len;
r.extra = n.extra;
return r;
}
case value_t::boolean:
{
node r = plain_node(value_t::boolean);
r.flags = static_cast<std::uint8_t>(r.flags | (n.flags & node_flags::is_true));
return r;
}
case value_t::null:
case value_t::object:
case value_t::array:
case value_t::binary:
case value_t::discarded:
default:
return plain_node(value_t::null);
}
}
node json_scalar(const BasicJsonType& j)
{
switch (j.type())
{
case value_t::null:
return plain_node(value_t::null);
case value_t::boolean:
{
node r = plain_node(value_t::boolean);
r.flags = static_cast<std::uint8_t>(r.flags | (j.template get<bool>() ? node_flags::is_true : 0));
return r;
}
case value_t::number_integer:
return integer_node(static_cast<std::uint64_t>(static_cast<std::int64_t>(j.template get<number_integer_t>())), value_t::number_integer);
case value_t::number_unsigned:
return integer_node(static_cast<std::uint64_t>(j.template get<number_unsigned_t>()), value_t::number_unsigned);
case value_t::number_float:
return float_node(j.template get<number_float_t>());
case value_t::string:
{
const auto& s = j.template get_ref<const string_t&>();
check_utf8(s.data(), s.size());
return string_node(s.data(), s.size());
}
case value_t::binary:
throw_type_error(319, "cannot store a binary value in a json_document", "");
case value_t::discarded:
case value_t::object:
case value_t::array:
default:
throw_type_error(302, "type must be a value, but is ", "discarded");
}
}
/// number of nodes of a subtree (containers, keys, scalars)
template<bool E>
static std::size_t count_nodes(const document_data& d, const node* n)
{
if (!is_container(*n))
{
return 1;
}
const bool object = n->kind == static_cast<std::uint8_t>(value_t::object);
std::size_t r = 1;
for (const node* c = navigation<E>::first(d, n), *end = navigation<E>::end(d, n); c != end;)
{
const node* const v = object ? c + 1 : c;
r += (object ? 1 : 0) + count_nodes<E>(d, navigation<E>::value(v));
c = document_data::after(v);
}
return r;
}
/// copy a subtree (of any document) as a contiguous sequence; returns its end
template<bool E>
node* fill_nodes(const document_data& d, const node* n, node* out)
{
if (!is_container(*n))
{
*out = copy_scalar(d, *n);
return out + 1;
}
node* const self = out++;
*self = plain_node(static_cast<value_t>(n->kind));
self->len = n->len;
const bool object = n->kind == static_cast<std::uint8_t>(value_t::object);
for (const node* c = navigation<E>::first(d, n), *end = navigation<E>::end(d, n); c != end;)
{
if (object)
{
*out++ = copy_scalar(d, *c);
++c;
}
out = fill_nodes<E>(d, navigation<E>::value(c), out);
c = document_data::after(c);
}
self->next = static_cast<std::uint32_t>(out - self);
return out;
}
static std::size_t count_nodes(const BasicJsonType& j)
{
std::size_t r = 1;
if (j.is_object())
{
for (const auto& member : j.items())
{
r += 1 + count_nodes(member.value());
}
}
else if (j.is_array())
{
for (const auto& e : j)
{
r += count_nodes(e);
}
}
return r;
}
node* fill_nodes(const BasicJsonType& j, node* out)
{
if (!j.is_structured())
{
*out = json_scalar(j);
return out + 1;
}
node* const self = out++;
*self = plain_node(j.type());
self->len = static_cast<std::uint32_t>(j.size());
if (j.is_object())
{
for (const auto& member : j.items())
{
check_utf8(member.key().data(), member.key().size());
*out++ = string_node(member.key().data(), member.key().size());
out = fill_nodes(member.value(), out);
}
}
else
{
for (const auto& e : j)
{
out = fill_nodes(e, out);
}
}
self->next = static_cast<std::uint32_t>(out - self);
return out;
}
document_data& m_doc;
};
} // namespace view
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
@@ -0,0 +1,242 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#pragma once
#include <algorithm> // max, min
#include <cstddef> // size_t
#include <cstdint> // uint8_t, uint32_t
#include <cstring> // memcpy
#include <functional> // less
#include <memory> // unique_ptr
#include <utility> // move
#include <nlohmann/json.hpp>
#include <nlohmann/detail/view/document_data.hpp>
#include <nlohmann/detail/view/errors.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
#include <nlohmann/detail/view/node.hpp>
// The storage of edits. Edits never move or resize the parsed index: every
// value keeps its node, so views stay valid. New values and element sequences
// live in chunks that never move; strings and number tokens written by edits
// live in the edit arena. An array/object whose elements change gets
// node_flags::moved: its elements then live in a separate sequence (a header
// node, then the entries), whose entries link to the values (kind_link).
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
{
namespace view
{
inline document_data::edit_state& edit_state_of(document_data& d)
{
if (!d.edits)
{
d.edits.reset(new document_data::edit_state()); // NOLINT(cppcoreguidelines-owning-memory): owned by the unique_ptr
}
return *d.edits;
}
/// k consecutive nodes that never move (new values and blocks)
inline node* alloc_nodes(document_data& d, std::size_t k)
{
document_data::edit_state& e = edit_state_of(d);
if (NLOHMANN_VIEW_UNLIKELY(static_cast<std::size_t>(e.chunk_end - e.chunk_cur) < k))
{
const std::size_t count = (std::max)(k, e.chunk_next);
std::unique_ptr<node[]> fresh(new node[count]()); // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
e.chunks.push_back(std::move(fresh));
e.chunk_cur = e.chunks.back().get();
e.chunk_end = e.chunk_cur + count;
e.chunk_next = (std::min)(e.chunk_next * 2, std::size_t{65536});
e.bytes += count * sizeof(node);
}
node* const r = e.chunk_cur;
e.chunk_cur += k;
return r;
}
/// copy n bytes into the edit arena and return their offset; a new buffer
/// leaves the old one alive, so that string views into it remain valid
inline std::uint32_t append_text(document_data& d, const char* s, std::size_t n)
{
document_data::edit_state& e = edit_state_of(d);
if (NLOHMANN_VIEW_UNLIKELY(e.text_cap - e.text_used < n))
{
const std::size_t cap = (std::max)(e.text_cap * 2, e.text_used + n + 256);
if (cap > 0xFFFFFFFFu)
{
throw_out_of_range(416, "edits of 4 GiB or more are not supported by json_document"); // LCOV_EXCL_LINE (4 GiB)
}
std::unique_ptr<char[]> fresh(new char[cap]); // NOLINT(cppcoreguidelines-avoid-c-arrays,hicpp-avoid-c-arrays,modernize-avoid-c-arrays)
if (e.text_used != 0)
{
std::memcpy(fresh.get(), e.texts.back().get(), e.text_used);
}
e.texts.push_back(std::move(fresh));
e.text_cap = cap;
e.bytes += cap;
d.base[2] = e.texts.back().get();
}
const auto off = static_cast<std::uint32_t>(e.text_used);
if (n != 0)
{
std::memcpy(e.texts.back().get() + e.text_used, s, n);
}
e.text_used += n;
return off;
}
/// the capacity in nodes of the block of a moved container (0: a fixed
/// sequence, the elements of a new value)
inline std::size_t moved_capacity(const document_data& d, const node* n) noexcept
{
return d.edits->moved_cap[n->off];
}
/// let container n take its elements from `seq` (header node first)
inline void set_moved(document_data& d, node* n, node* seq, std::size_t cap)
{
document_data::edit_state& e = edit_state_of(d);
if ((n->flags & node_flags::moved) != 0)
{
e.moved[n->off] = seq;
e.moved_cap[n->off] = cap;
return;
}
if (e.moved.size() >= 0xFFFFFFFFu)
{
throw_out_of_range(416, "more than 4294967295 edited arrays and objects are not supported by json_document"); // LCOV_EXCL_LINE
}
if (e.moved.size() == e.moved.capacity() || e.moved_cap.size() == e.moved_cap.capacity())
{
// both grow before either changes, so that the push_backs cannot throw
e.moved.reserve((2 * e.moved.size()) + 16);
e.moved_cap.reserve((2 * e.moved.size()) + 16);
}
e.moved.push_back(seq);
e.moved_cap.push_back(cap);
n->off = static_cast<std::uint32_t>(e.moved.size() - 1);
n->flags = static_cast<std::uint8_t>(n->flags | node_flags::moved | node_flags::is_new);
}
/// Make the elements of container n a growable block with room for `extra`
/// more nodes, and return its header. The entries link to the existing
/// values, which stay where they are. A block that grows is copied (its old
/// space is not reused).
inline node* block_of(document_data& d, node* n, std::size_t extra)
{
if ((n->flags & node_flags::moved) != 0 && moved_capacity(d, n) != 0)
{
node* const h = d.edits->moved[n->off];
if (h->next + extra <= moved_capacity(d, n))
{
return h;
}
const std::size_t cap = (std::max)(2 * moved_capacity(d, n), h->next + extra);
node* const nh = alloc_nodes(d, cap);
std::memcpy(nh, h, h->next * sizeof(node));
set_moved(d, n, nh, cap);
return nh;
}
const bool object = n->kind == static_cast<std::uint8_t>(value_t::object);
const std::size_t used = 1 + (static_cast<std::size_t>(n->len) * (object ? 2 : 1));
const std::size_t cap = used + extra;
node* const h = alloc_nodes(d, cap);
*h = node{};
h->kind = n->kind;
h->len = n->len;
h->next = static_cast<std::uint32_t>(used);
node* o = h + 1;
for (const node* c = d.first_child_edited(n), *e = d.child_end_edited(n); c != e;)
{
if (object)
{
*o++ = *c++; // the key
}
make_link(*o, document_data::deref(c));
++o;
c = document_data::after(c);
}
set_moved(d, n, h, cap);
return h;
}
/// The container whose elements include `target`; nullptr for the root, for
/// a value that is no longer part of the document, and for a value that is
/// only reached through a link. Values never move between allocations, so
/// the path to `target` stays inside the allocation that holds it (the parsed
/// index, or one new value), where the extent of each container (`next`)
/// still covers its original subtree.
inline node* find_parent(const document_data& d, const node* target)
{
const std::less<const node*> lt;
const node* lo = d.tape;
const node* hi = d.tape + d.tape_size;
const node* c = d.tape;
if (lt(target, lo) || !lt(target, hi))
{
if (!d.edits)
{
return nullptr; // LCOV_EXCL_LINE (nodes outside the index exist only after edits)
}
auto it = d.edits->regions.upper_bound(target);
if (it == d.edits->regions.begin())
{
return nullptr; // LCOV_EXCL_LINE (an array/object with elements is in the index or a new value)
}
--it;
lo = it->first;
hi = lo + lo->next;
if (!lt(target, hi))
{
return nullptr; // LCOV_EXCL_LINE (a single-node value, reached through a link)
}
// the root of a new value is the element sequence of its owner, or a linked value
c = it->second != nullptr ? it->second : lo;
}
if (target == lo)
{
return nullptr;
}
for (;;)
{
if (!is_container(*c))
{
return nullptr; // LCOV_EXCL_LINE (the value is inside c)
}
const bool object = c->kind == static_cast<std::uint8_t>(value_t::object);
const node* down = nullptr;
for (const node* p = d.first_child_edited(c), *e = d.child_end_edited(c); p != e;)
{
const node* const at = object ? p + 1 : p;
const node* const v = document_data::deref(at);
if (v == target)
{
return const_cast<node*>(c); // NOLINT(cppcoreguidelines-pro-type-const-cast): the nodes belong to the document
}
if (is_container(*v) && !lt(v, lo) && lt(v, target) && lt(target, v + v->next))
{
down = v;
break;
}
p = document_data::after(at);
}
if (down == nullptr)
{
return nullptr;
}
c = down;
}
}
} // namespace view
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
+5
View File
@@ -30,6 +30,11 @@ namespace view
NLOHMANN_VIEW_THROW(type_error::create(id, concat(prefix, type), nullptr));
}
[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_type_error(int id, const std::string& msg)
{
NLOHMANN_VIEW_THROW(type_error::create(id, msg, nullptr));
}
[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_out_of_range(int id, const std::string& msg)
{
NLOHMANN_VIEW_THROW(out_of_range::create(id, msg, nullptr));
+602
View File
@@ -0,0 +1,602 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#pragma once
#include <array> // array
#include <cstddef> // size_t
#include <cstdint> // int64_t, uint8_t, uint16_t, uint32_t, uint64_t
#include <cstring> // memcmp, memcpy
#include <limits> // numeric_limits
#include <string> // string
#include <vector> // vector
#include <nlohmann/json.hpp>
#include <nlohmann/detail/view/document_data.hpp>
#include <nlohmann/detail/view/errors.hpp>
#include <nlohmann/detail/view/macro_scope.hpp>
#include <nlohmann/detail/view/node.hpp>
#include <nlohmann/detail/view/number.hpp>
#include <nlohmann/detail/view/object_index.hpp>
#include <nlohmann/detail/view/scan.hpp>
// Images: a document stored so that loading it needs no parsing.
//
// Layout (little-endian): a 64-byte header, the nodes, the text (the source,
// followed by the number tokens written by edits), a NUL, the decoded strings
// (followed by the strings written by edits), a NUL. The idea is that of
// zero-copy formats such as FlatBuffers (https://github.com/google/flatbuffers)
// and YaFF (https://github.com/yandex/yaff); no code is taken from them.
// check_image follows the idea of FlatBuffers' Verifier (bounds and
// structure) and also checks what the parser guarantees about strings and
// numbers, so that reading and serializing a checked image is safe and yields
// valid JSON.
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
{
namespace view
{
/// how load() checks an image
enum class image_check
{
/// everything the parser guarantees: structure and bounds, strings (valid
/// UTF-8; source strings without quotes, backslashes, and control
/// characters), and numbers (well-formed, matching the stored values)
full,
/// structure and bounds only: reading and serializing are safe, but a
/// crafted image can yield invalid UTF-8, strings that serialize to
/// invalid JSON, or numbers that differ from their text
bounds,
/// none: for images from a trusted source only (a damaged image is
/// undefined behavior)
none,
};
struct image_header
{
std::array<char, 4> magic; ///< "NJVI"
std::uint32_t version; ///< 1
std::uint64_t node_count;
std::uint64_t text_size;
std::uint64_t arena_size;
std::array<std::uint64_t, 4> reserved; ///< zero (for later versions)
};
static_assert(sizeof(image_header) == 64, "the image header must be 64 bytes");
constexpr std::uint32_t image_version = 1;
/// the largest node count and text or string size of an image (as for parsed
/// documents, offsets and counts must fit 32 bits)
constexpr std::uint64_t image_limit = 0xFFFFFFF0u;
/// Copy the current structure of an edited document into nodes in document
/// order, as the parser would have written them. Text written by edits is
/// appended to text_tail (number tokens) and arena_tail (strings); floats that
/// are not finite become null, as dump() writes them.
inline void compact_nodes(const document_data& d, std::size_t arena_size, std::vector<node>& out, std::string& text_tail, std::string& arena_tail)
{
struct frame
{
const node* cur;
const node* end;
std::size_t index; ///< the container's node in out
std::uint32_t count;
bool object;
};
std::vector<frame> stack;
const auto string_node = [&](const node & s)
{
node r = s;
r.extra = 0;
r.flags = static_cast<std::uint8_t>(s.flags & node_flags::storage);
if (r.flags == node_flags::edited)
{
r.off = static_cast<std::uint32_t>(arena_size + arena_tail.size());
arena_tail.append(d.str(s), s.len);
r.flags = node_flags::escaped;
}
return r;
};
const auto emit = [&](const node * v)
{
node r = *v;
switch (static_cast<value_t>(v->kind))
{
case value_t::object:
case value_t::array:
r.flags = 0;
r.extra = 0;
r.off = (v->flags & (node_flags::moved | node_flags::is_new)) != 0 ? 0 : v->off;
r.len = 0; // counted below
r.next = 0; // set when the container is complete
stack.push_back(frame{d.first_child_edited(v), d.child_end_edited(v), out.size(), 0, v->kind == static_cast<std::uint8_t>(value_t::object)});
break;
case value_t::string:
r = string_node(*v);
break;
case value_t::number_integer:
case value_t::number_unsigned:
if ((v->flags & node_flags::storage) == node_flags::edited)
{
r.off = static_cast<std::uint32_t>(d.size + text_tail.size());
text_tail.append(d.str(*v), number_length(*v));
}
r.flags = 0;
break;
case value_t::number_float:
if ((v->flags & node_flags::storage) == node_flags::edited)
{
const char* const t = d.str(*v);
if (t[0] == 'n' || t[0] == 'i' || (v->len > 1 && t[1] == 'i'))
{
r = node{}; // nan and infinity: null, as dump() writes them
r.kind = static_cast<std::uint8_t>(value_t::null);
break;
}
r.off = static_cast<std::uint32_t>(d.size + text_tail.size());
text_tail.append(t, v->len);
r.extra = 0xFFFFu; // the digit layout is not recorded
}
r.flags = 0;
break;
case value_t::boolean:
r.flags = static_cast<std::uint8_t>(v->flags & node_flags::is_true);
break;
case value_t::null:
case value_t::binary:
case value_t::discarded:
default:
r.flags = 0;
break;
}
out.push_back(r);
};
emit(d.tape);
while (!stack.empty())
{
frame& top = stack.back();
if (top.cur == top.end)
{
node& c = out[top.index];
c.len = top.count;
c.next = static_cast<std::uint32_t>(out.size() - top.index);
stack.pop_back();
continue;
}
++top.count;
const node* v = nullptr;
if (top.object)
{
out.push_back(string_node(*top.cur));
v = document_data::deref(top.cur + 1);
top.cur = document_data::after(top.cur + 1);
}
else
{
v = document_data::deref(top.cur);
top.cur = document_data::after(top.cur);
}
emit(v); // may grow the stack (top is not used afterwards)
}
}
/// the document as an image
inline std::vector<std::uint8_t> save_image(const document_data& d)
{
#if !NLOHMANN_VIEW_LITTLE_ENDIAN
throw_type_error(320, "json_document images need a little-endian target"); // LCOV_EXCL_LINE
#endif
const std::size_t arena_size = d.arena_size;
const node* nodes = d.tape;
std::size_t count = d.tape_size;
std::vector<node> compacted;
std::string text_tail;
std::string arena_tail;
if (d.edits)
{
compact_nodes(d, arena_size, compacted, text_tail, arena_tail);
nodes = compacted.data();
count = compacted.size();
}
const std::size_t text_size = d.size + text_tail.size();
const std::size_t total_arena = arena_size + arena_tail.size();
if (NLOHMANN_VIEW_UNLIKELY(text_size >= image_limit || total_arena >= image_limit || count >= image_limit))
{
// LCOV_EXCL_START (4 GiB)
throw_out_of_range(416, "images of 4 GiB or more are not supported by json_document");
// LCOV_EXCL_STOP
}
image_header h{};
h.magic = {{'N', 'J', 'V', 'I'}};
h.version = image_version;
h.node_count = count;
h.text_size = text_size;
h.arena_size = total_arena;
std::vector<std::uint8_t> image(sizeof(h) + (count * sizeof(node)) + text_size + 1 + total_arena + 1);
std::uint8_t* o = image.data();
std::memcpy(o, &h, sizeof(h));
o += sizeof(h);
std::memcpy(o, nodes, count * sizeof(node));
// the hash indexes are rebuilt by load()
for (std::size_t i = 0; i < count; ++i)
{
if (nodes[i].kind == static_cast<std::uint8_t>(value_t::object) && nodes[i].extra != 0)
{
node n = nodes[i];
n.extra = 0;
std::memcpy(o + (i * sizeof(node)), &n, sizeof(node));
}
}
o += count * sizeof(node);
const auto append = [&o](const char* s, std::size_t n)
{
if (n != 0)
{
std::memcpy(o, s, n);
o += n;
}
};
append(d.src, d.size);
append(text_tail.data(), text_tail.size());
*o++ = 0;
append(d.base[1], arena_size);
append(arena_tail.data(), arena_tail.size());
*o = 0;
return image;
}
/// whether a number node matches its token the way the parser records it
/// (after the bounds check)
inline bool check_number(const node& n, const unsigned char* text)
{
const std::size_t len = number_length(n);
const unsigned char* const s = text + n.off;
const unsigned char* const e = s + len;
const unsigned char* p = s;
const bool negative = *p == '-';
p += negative ? 1 : 0;
const unsigned char* const int_start = p;
if (p == e)
{
return false;
}
if (*p == '0')
{
++p;
}
else if (*p >= '1' && *p <= '9')
{
while (p != e && is_digit(*p))
{
++p;
}
}
else
{
return false;
}
const auto int_digits = static_cast<std::size_t>(p - int_start);
std::size_t frac_digits = 0;
bool is_float = false;
if (p != e && *p == '.')
{
const unsigned char* const f0 = ++p;
while (p != e && is_digit(*p))
{
++p;
}
if (p == f0)
{
return false;
}
frac_digits = static_cast<std::size_t>(p - f0);
is_float = true;
}
std::int64_t exponent = 0;
if (p != e && (*p | 0x20u) == 'e')
{
++p;
const bool exp_negative = p != e && *p == '-';
p += (p != e && (*p == '+' || *p == '-')) ? 1 : 0;
if (p == e || !is_digit(*p))
{
return false;
}
while (p != e && is_digit(*p))
{
exponent = exponent < 100000 ? (exponent * 10) + (*p - '0') : exponent;
++p;
}
exponent = exp_negative ? -exponent : exponent;
is_float = true;
}
if (p != e)
{
return false;
}
if (n.kind == static_cast<std::uint8_t>(value_t::number_float))
{
// the digit layout the parser records (or "many", as compaction
// writes it), and a finite value
const auto layout = static_cast<std::uint16_t>((int_digits < 255 ? int_digits : 255) | ((frac_digits < 255 ? frac_digits : 255) << 8u));
if (n.extra != layout && n.extra != 0xFFFFu)
{
return false;
}
// parse() rejects floats that overflow; as there, only a number whose
// magnitude could reach 1e308 needs the conversion
if (static_cast<std::int64_t>(int_digits) + exponent > 300)
{
const auto v = float_value<double>(reinterpret_cast<const char*>(s), n); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
return v <= (std::numeric_limits<double>::max)() && v >= -(std::numeric_limits<double>::max)();
}
return true;
}
// integers: the token's value is the stored one; number_integer nodes of
// edits can be non-negative (as basic_json keeps the type of a value)
const bool integer = n.kind == static_cast<std::uint8_t>(value_t::number_integer);
if (is_float || int_digits > 20 || (negative && !integer))
{
return false;
}
// (at most 19 digits cannot overflow; 20 digits are compared with 2^64 - 1)
if (int_digits == 20 && std::memcmp(int_start, "18446744073709551615", 20) > 0)
{
return false;
}
std::uint64_t m = 0;
for (const unsigned char* d = int_start; d != int_start + int_digits; ++d)
{
m = (m * 10) + static_cast<std::uint64_t>(*d - '0');
}
if (integer && m > (negative ? std::uint64_t{1} << 63u : (std::uint64_t{1} << 63u) - 1))
{
return false;
}
return integer_bits(n) == (negative ? 0 - m : m);
}
/// Check the nodes of a loaded image against its text and decoded strings:
/// kinds, flags, and `extra`; extents and element counts of arrays and
/// objects; keys; bounds; string contents (source strings as the parser
/// leaves them: no quotes, backslashes, or control characters; all strings
/// valid UTF-8); and number tokens.
inline bool check_image(const node* nodes, std::size_t count, const unsigned char* text, std::size_t text_size,
const unsigned char* arena, std::size_t arena_size, bool full)
{
struct frame
{
std::size_t end;
std::uint32_t len;
std::uint32_t seen;
bool object;
bool expect_key;
};
std::vector<frame> stack;
const auto check_string = [&](const node & n) -> bool
{
if ((n.flags & ~node_flags::escaped) != 0 || n.extra != 0)
{
return false;
}
const bool decoded = (n.flags & node_flags::escaped) != 0;
const unsigned char* const base = decoded ? arena : text;
const std::size_t limit = decoded ? arena_size : text_size;
if (n.off > limit || n.len > limit - n.off)
{
return false;
}
if (!full)
{
return true;
}
const unsigned char* const b = base + n.off;
return decoded ? valid_utf8_prefix(b, n.len) == n.len : scan_string_run(b, b + n.len) == b + n.len;
};
// bounds of a number token; the recorded digit layout must lie within it
const auto number_in_bounds = [&](const node & n) -> bool
{
const std::size_t len = number_length(n);
if (len == 0 || n.off > text_size || len > text_size - n.off)
{
return false;
}
if (n.kind != static_cast<std::uint8_t>(value_t::number_float))
{
return (n.extra >> 8u) == 0;
}
// float_value() reads the sign, the integer digits, and the point and
// fraction digits the layout records (a layout of more than 19 digits
// means the general conversion, which stays within the token)
const std::size_t int_digits = n.extra & 0xFFu;
const std::size_t frac_digits = n.extra >> 8u;
const std::size_t need = (text[n.off] == '-' ? 1u : 0u) + int_digits + (frac_digits != 0 ? frac_digits + 1 : 0);
return int_digits + frac_digits > 19 || need <= len;
};
std::size_t i = 0;
for (;;)
{
// close finished arrays and objects
while (!stack.empty() && i == stack.back().end)
{
const frame f = stack.back();
if (f.seen != f.len || (f.object && !f.expect_key))
{
return false;
}
stack.pop_back();
if (!stack.empty())
{
++stack.back().seen;
stack.back().expect_key = true;
}
}
if (i == count)
{
return stack.empty();
}
if (i != 0 && stack.empty())
{
return false; // nodes after the root
}
const node& n = nodes[i];
if (!stack.empty() && stack.back().object && stack.back().expect_key)
{
if (n.kind != static_cast<std::uint8_t>(value_t::string) || !check_string(n))
{
return false;
}
stack.back().expect_key = false;
++i;
continue;
}
bool complete = true;
switch (static_cast<value_t>(n.kind))
{
case value_t::null:
// (the offset of a literal is read to size the output of dump())
if (n.flags != 0 || n.extra != 0 || n.off > text_size)
{
return false;
}
break;
case value_t::boolean:
if ((n.flags & ~node_flags::is_true) != 0 || n.extra != 0 || n.off > text_size)
{
return false;
}
break;
case value_t::string:
if (!check_string(n))
{
return false;
}
break;
case value_t::number_integer:
case value_t::number_unsigned:
case value_t::number_float:
if (n.flags != 0 || !number_in_bounds(n) || (full && !check_number(n, text)))
{
return false;
}
break;
case value_t::array:
case value_t::object:
{
const std::size_t limit = stack.empty() ? count : stack.back().end;
if (n.flags != 0 || n.extra != 0 || n.next == 0 || n.next > limit - i || n.off > text_size)
{
return false;
}
stack.push_back(frame{i + n.next, n.len, 0, n.kind == static_cast<std::uint8_t>(value_t::object), true});
complete = false;
break;
}
case value_t::binary:
case value_t::discarded:
default:
return false;
}
++i;
if (complete && !stack.empty())
{
++stack.back().seen;
stack.back().expect_key = true;
}
}
}
[[noreturn]] NLOHMANN_VIEW_NOINLINE inline void throw_invalid_image(const char* what)
{
throw_parse_error(116, concat("invalid json_document image: ", what));
}
/// Read an image into d. The text and the decoded strings stay in the image;
/// the nodes are copied (so that they are aligned, and edits can change them).
inline void load_image(document_data& d, const std::uint8_t* image, std::size_t size, image_check check)
{
#if !NLOHMANN_VIEW_LITTLE_ENDIAN
throw_type_error(320, "json_document images need a little-endian target"); // LCOV_EXCL_LINE
#endif
if (image == nullptr || size < sizeof(image_header))
{
throw_invalid_image("too short");
}
image_header h{};
std::memcpy(&h, image, sizeof(h));
// (the reserved fields are for later versions)
if (std::memcmp(h.magic.data(), "NJVI", 4) != 0 || h.version != image_version
|| (h.reserved[0] | h.reserved[1] | h.reserved[2] | h.reserved[3]) != 0)
{
throw_invalid_image("unknown format");
}
const std::size_t room = size - sizeof(h);
if (h.node_count == 0 || h.node_count > room / sizeof(node) || h.node_count >= image_limit || h.text_size >= image_limit || h.arena_size >= image_limit)
{
throw_invalid_image("sizes out of range");
}
const auto count = static_cast<std::size_t>(h.node_count);
const auto text_size = static_cast<std::size_t>(h.text_size);
const auto arena_size = static_cast<std::size_t>(h.arena_size);
const std::size_t text_at = sizeof(h) + (count * sizeof(node));
// the text, a NUL, the decoded strings, a NUL, and nothing after them
if (size - text_at < 2 || text_size > size - text_at - 2 || arena_size != size - text_at - text_size - 2
|| image[text_at + text_size] != 0 || image[size - 1] != 0)
{
throw_invalid_image("sizes out of range");
}
d.discarded = true;
d.edits.reset();
d.base[2] = nullptr;
d.owned.clear();
if (d.owned_image.empty() || image != d.owned_image.data())
{
d.owned_image.clear();
}
d.arena.clear();
d.indexes.clear();
d.index_slots.clear();
d.large_objects.clear();
d.tape_size = 0;
d.reserve(count);
std::memcpy(d.tape, image + sizeof(h), count * sizeof(node));
d.tape_size = count;
const std::uint8_t* const text = image + text_at;
const std::uint8_t* const arena = text + text_size + 1;
d.src = reinterpret_cast<const char*>(text); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
d.size = text_size;
d.base[0] = d.src;
d.base[1] = reinterpret_cast<const char*>(arena); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
d.arena_size = arena_size;
if (check != image_check::none && !check_image(d.tape, count, text, text_size, arena, arena_size, check == image_check::full))
{
throw_invalid_image("the check failed");
}
// the hash indexes of large objects, as after parsing
for (std::size_t i = 0; i < count; ++i)
{
node& n = d.tape[i];
if (n.kind == static_cast<std::uint8_t>(value_t::object))
{
n.extra = 0;
if (n.len >= document_data::index_min_members)
{
d.large_objects.push_back(static_cast<std::uint32_t>(i));
}
}
}
build_object_indexes(d);
d.discarded = false;
}
} // namespace view
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
+1 -1
View File
@@ -73,7 +73,7 @@ class view_iterator
NLOHMANN_VIEW_ALWAYS_INLINE View operator*() const noexcept
{
return View(m_doc, m_pos + m_value_offset);
return View(m_doc, View::navigation::value(m_pos + m_value_offset));
}
pointer operator->() const noexcept
+23 -14
View File
@@ -84,18 +84,20 @@ class short_key
/// the key node of the first member of an object with the given key, or
/// nullptr; most keys are rejected by their length, from the index alone
inline const node* find_member(const document_data& d, const node* object, const char* key, std::size_t n) noexcept
template<bool Editable>
const node* find_member(const document_data& d, const node* object, const char* key, std::size_t n) noexcept
{
if (NLOHMANN_VIEW_UNLIKELY(object->extra != 0))
using nav = navigation<Editable>;
if (NLOHMANN_VIEW_UNLIKELY(object->extra != 0) && (!Editable || (object->flags & node_flags::moved) == 0))
{
return find_indexed(d, object, key, n); // a large object
return find_indexed(d, object, key, n); // a large object (whose members have not been edited)
}
const node* const end = document_data::child_end(object);
const node* const end = nav::end(d, object);
const auto* const k = reinterpret_cast<const unsigned char*>(key); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
if (NLOHMANN_VIEW_LIKELY(n <= 16))
{
const short_key probe(k, n);
for (const node* m = document_data::first_child(object); m != end; m = document_data::after(m + 1))
for (const node* m = nav::first(d, object); m != end; m = document_data::after(m + 1))
{
if (m->len == n && probe.matches(reinterpret_cast<const unsigned char*>(d.str(*m)))) // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
{
@@ -104,7 +106,7 @@ inline const node* find_member(const document_data& d, const node* object, const
}
return nullptr;
}
for (const node* m = document_data::first_child(object); m != end; m = document_data::after(m + 1))
for (const node* m = nav::first(d, object); m != end; m = document_data::after(m + 1))
{
if (m->len == n && std::memcmp(d.str(*m), key, n) == 0)
{
@@ -114,10 +116,16 @@ inline const node* find_member(const document_data& d, const node* object, const
return nullptr;
}
/// the element of an array at an index below its size
inline const node* element_at(const node* array, std::size_t idx) noexcept
/// the entry of the element of an array at an index below its size (a link
/// in the moved sequences of editable documents)
template<bool Editable>
const node* element_at(const document_data& d, const node* array, std::size_t idx) noexcept
{
const node* e = document_data::first_child(array);
const node* e = navigation<Editable>::first(d, array);
if (Editable && (array->flags & node_flags::moved) != 0 && d.edits->moved_cap[array->off] != 0)
{
return e + idx; // a growable block: one link per element
}
for (std::size_t i = 0; i < idx; ++i)
{
e = document_data::after(e);
@@ -125,13 +133,14 @@ inline const node* element_at(const node* array, std::size_t idx) noexcept
return e;
}
/// the last element of a non-empty array, or the key of the last member of a
/// non-empty object
inline const node* last_child(const node* container) noexcept
/// the entry of the last element of a non-empty array, or the key of the
/// last member of a non-empty object
template<bool Editable>
const node* last_child(const document_data& d, const node* container) noexcept
{
const std::size_t value_offset = container->kind == static_cast<std::uint8_t>(value_t::object) ? 1 : 0;
const node* const end = document_data::child_end(container);
const node* last = document_data::first_child(container);
const node* const end = navigation<Editable>::end(d, container);
const node* last = navigation<Editable>::first(d, container);
for (const node* c = document_data::after(last + value_offset); c != end; c = document_data::after(c + value_offset))
{
last = c;
+33 -27
View File
@@ -34,17 +34,24 @@ iterative, so the nesting depth is limited by memory only, as for parse().
Without a lexer the handler records no source positions
(JSON_DIAGNOSTIC_POSITIONS).
*/
template<typename BasicJsonType>
template<typename BasicJsonType, bool Editable>
BasicJsonType materialize(const document_data& d, const node* n)
{
using string_t = typename BasicJsonType::string_t;
using sax_t = json_sax_dom_parser<BasicJsonType, iterator_input_adapter<const char*>>;
using nav = navigation<Editable>;
struct frame
{
const node* pos; ///< next element, or key of the next member
const node* end;
bool object;
};
BasicJsonType result;
sax_t sax(result, true);
const string_t no_token{};
// the ends of the open containers, and whether they are objects
std::vector<std::pair<const node*, bool>> open;
std::vector<frame> open;
for (;;)
{
switch (static_cast<value_t>(n->kind))
@@ -61,67 +68,66 @@ BasicJsonType materialize(const document_data& d, const node* n)
{
sax.start_array(n->len);
}
open.emplace_back(document_data::child_end(n), object);
n = document_data::first_child(n);
open.push_back(frame{nav::first(d, n), nav::end(d, n), object});
break;
}
case value_t::string:
{
string_t s(d.str(*n), n->len);
sax.string(s);
++n;
break;
}
case value_t::number_integer:
sax.number_integer(static_cast<typename BasicJsonType::number_integer_t>(static_cast<std::int64_t>(integer_bits(*n))));
++n;
break;
case value_t::number_unsigned:
sax.number_unsigned(static_cast<typename BasicJsonType::number_unsigned_t>(integer_bits(*n)));
++n;
break;
case value_t::number_float:
sax.number_float(float_value<typename BasicJsonType::number_float_t>(d, *n), no_token);
++n;
break;
case value_t::boolean:
sax.boolean((n->flags & node_flags::is_true) != 0);
++n;
break;
case value_t::null:
case value_t::binary:
case value_t::discarded:
default:
sax.null();
++n;
break;
}
// the next value: close finished containers, then read the key
for (;;)
{
if (open.empty())
{
return result;
}
if (n != open.back().first)
frame& f = open.back();
if (f.pos == f.end)
{
break;
if (f.object)
{
sax.end_object();
}
else
{
sax.end_array();
}
open.pop_back();
continue;
}
if (open.back().second)
const node* entry = f.pos;
if (f.object)
{
sax.end_object();
string_t key(d.str(*entry), entry->len);
sax.key(key);
++entry;
}
else
{
sax.end_array();
}
open.pop_back();
}
if (open.back().second)
{
// the key of the next member
string_t key(d.str(*n), n->len);
sax.key(key);
++n;
n = nav::value(entry);
f.pos = document_data::after(entry);
break;
}
}
}
+25 -3
View File
@@ -33,20 +33,27 @@ static_assert(static_cast<std::uint8_t>(value_t::null) == 0 && static_cast<std::
struct node_flags
{
static constexpr std::uint8_t escaped = 1; ///< string payload lives in the decode arena, not the source
static constexpr std::uint8_t edited = 2; ///< string or number token lives in the edit arena (editable documents)
static constexpr std::uint8_t storage = 3; ///< mask: where a string or number token lives (index into document_data::base)
static constexpr std::uint8_t is_true = 4; ///< boolean value
static constexpr std::uint8_t moved = 8; ///< array/object: the elements live in a separate sequence (editable documents)
static constexpr std::uint8_t is_new = 16; ///< written by an edit: no source position
};
/// kind of an entry of an edited sequence that stands for a value stored
/// elsewhere (the address of the value's node is kept in the len/next bytes)
constexpr std::uint8_t kind_link = 10;
/// One entry of the flat index, in document order. An object's members are
/// stored as key node followed by the value's subtree. Integers keep their
/// converted 64-bit value in the len/next bytes (the node after a scalar is
/// always the next one, and the token length follows from `extra`).
struct node
{
std::uint8_t kind; ///< value_t
std::uint8_t kind; ///< value_t, or kind_link
std::uint8_t flags; ///< node_flags
std::uint16_t extra; ///< numbers: integer digits (low byte) and fraction digits (high byte), 255 = "many"; otherwise 0
std::uint32_t off; ///< source offset (string content, number token, literal, bracket); arena offset if node_flags::escaped
std::uint16_t extra; ///< numbers: integer digits (low byte) and fraction digits (high byte), 255 = "many"; objects: number of the hash index; otherwise 0
std::uint32_t off; ///< source offset (string content, number token, literal, bracket); arena offset if escaped/edited; number of the element sequence if moved
std::uint32_t len; ///< string: decoded bytes; float: token bytes; array/object: element count
std::uint32_t next; ///< array/object: number of nodes of the subtree (its extent in the enclosing sequence)
};
@@ -57,6 +64,21 @@ NLOHMANN_VIEW_ALWAYS_INLINE bool is_container(const node& n) noexcept
return static_cast<unsigned>(n.kind) - 1u <= 1u;
}
/// the value a link node stands for
NLOHMANN_VIEW_ALWAYS_INLINE const node* link_target(const node& n) noexcept
{
const node* t = nullptr;
std::memcpy(static_cast<void*>(&t), reinterpret_cast<const unsigned char*>(&n) + 8, sizeof(const node*)); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
return t;
}
inline void make_link(node& n, const node* target) noexcept
{
n = node{};
n.kind = kind_link;
std::memcpy(reinterpret_cast<unsigned char*>(&n) + 8, static_cast<const void*>(&target), sizeof(const node*)); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
}
/// the converted value of an integer node (stored in len/next)
NLOHMANN_VIEW_ALWAYS_INLINE std::uint64_t integer_bits(const node& n) noexcept
{
+89 -31
View File
@@ -13,6 +13,7 @@
#include <cstddef> // size_t
#include <cstdint> // int64_t, uint64_t
#include <cstring> // memcpy
#include <limits> // numeric_limits
#include <string> // string
#include <type_traits> // integral_constant, is_same
@@ -28,43 +29,76 @@ namespace detail
namespace view
{
/*!
@brief locate the decimal point and the end of the mantissa of a float token
Also checks that the token is a JSON number. Tokens of the parser and of edits
always are; an image loaded with image_check::bounds can hold any bytes, which
must not reach the conversion (it expects a well-formed token).
*/
inline bool float_token_layout(const char* first, const char* last, std::size_t& dot, std::size_t& mantissa_end) noexcept
{
const auto digit = [last](const char* q)
{
return q != last && is_digit(static_cast<unsigned char>(*q));
};
const char* p = first;
p += (p != last && *p == '-') ? 1 : 0;
if (!digit(p) || (*p == '0' && digit(p + 1)))
{
return false;
}
while (digit(p))
{
++p;
}
dot = std::string::npos;
if (p != last && *p == '.')
{
dot = static_cast<std::size_t>(p - first);
if (!digit(++p))
{
return false;
}
while (digit(p))
{
++p;
}
}
mantissa_end = static_cast<std::size_t>(p - first);
if (p != last && (*p == 'e' || *p == 'E'))
{
++p;
p += (p != last && (*p == '+' || *p == '-')) ? 1 : 0;
if (!digit(p))
{
return false;
}
while (digit(p))
{
++p;
}
}
return p == last;
}
/*!
@brief the value of the float token of a node, as parse() converts it
Uses the lexer's conversion (detail::convert_float_fast, then the locale-aware
strtod fallback), so that the values are bit-identical to parse(). The digit
layout recorded while parsing locates the decimal point and the exponent
without scanning the token.
strtod fallback), so that the values are bit-identical to parse(). A token
that is not a JSON number (only in a damaged image loaded with
image_check::bounds) yields 0.
*/
template<typename FloatType>
NLOHMANN_VIEW_NOINLINE FloatType float_value(const char* first, const node& n)
{
const char* const last = first + n.len;
const std::size_t neg = first[0] == '-' ? 1 : 0;
const std::size_t int_digits = n.extra & 0xFFu;
const std::size_t frac_digits = n.extra >> 8u;
std::size_t dot = std::string::npos;
std::size_t mantissa_end = n.len;
if (int_digits != 255 && frac_digits != 255)
std::size_t dot = 0;
std::size_t mantissa_end = 0;
if (NLOHMANN_VIEW_UNLIKELY(!float_token_layout(first, last, dot, mantissa_end)))
{
dot = frac_digits != 0 ? neg + int_digits : std::string::npos;
mantissa_end = neg + int_digits + (frac_digits != 0 ? 1 + frac_digits : 0);
}
else
{
// more digits than the layout records: locate them
for (std::size_t i = 0; i < n.len; ++i)
{
if (first[i] == '.')
{
dot = i;
}
else if (first[i] == 'e' || first[i] == 'E')
{
mantissa_end = i;
break;
}
}
return FloatType{};
}
FloatType v{};
if (!convert_float_fast(first, last, dot, mantissa_end, v))
@@ -103,16 +137,20 @@ NLOHMANN_VIEW_ALWAYS_INLINE double layout_double(const unsigned char* p, const u
}
if (p != e)
{
// [eE][+-]digits; huge exponents saturate (the parser rejected overflow)
// [eE][+-]digits; huge exponents saturate (the parser rejected
// overflow). The token is not read beyond e, and the digits are taken
// as unsigned, so that a token that is not well-formed (a damaged
// image loaded with image_check::bounds) yields a wrong value, but no
// overflow.
++p;
const bool exp_negative = *p == '-';
p += (*p == '-' || *p == '+') ? 1 : 0;
const bool exp_negative = p != e && *p == '-';
p += (p != e && (*p == '-' || *p == '+')) ? 1 : 0;
std::int64_t exp_value = 0;
for (; p != e; ++p)
{
if (exp_value < 0x10000000)
{
exp_value = (exp_value * 10) + (*p - '0');
exp_value = (exp_value * 10) + static_cast<unsigned char>(*p - '0');
}
}
q += exp_negative ? -exp_value : exp_value;
@@ -137,11 +175,31 @@ NLOHMANN_VIEW_ALWAYS_INLINE double layout_double(const unsigned char* p, const u
return negative ? -result : result;
}
/// the value of a float set by an edit: its token (the shortest round-trip
/// text, or "nan", "inf", "-inf") in the edit arena
template<typename FloatType>
NLOHMANN_VIEW_NOINLINE FloatType edited_float(const char* token, const node& n)
{
if (token[0] == 'n')
{
return std::numeric_limits<FloatType>::quiet_NaN();
}
if (token[0] == 'i' || (token[0] == '-' && token[1] == 'i'))
{
return token[0] == 'i' ? std::numeric_limits<FloatType>::infinity() : -std::numeric_limits<FloatType>::infinity();
}
return float_value<FloatType>(token, n);
}
/// the value of the float token of a node, as parse() converts it; doubles
/// with at most 19 digits are converted from the digit layout
template<typename FloatType>
FloatType float_value(const document_data& d, const node& n)
{
if (NLOHMANN_VIEW_UNLIKELY((n.flags & node_flags::storage) == node_flags::edited))
{
return edited_float<FloatType>(d.str(n), n);
}
return float_value<FloatType>(d, n, std::is_same<FloatType, double> {});
}
+435 -15
View File
@@ -75,6 +75,23 @@ class output_buffer
m_pos += n;
}
/// the write position and the end of the writable space, for a writer
/// that keeps the position in a local variable (set_cursor() hands it back)
char* cursor() const noexcept
{
return m_pos;
}
char* limit() const noexcept
{
return m_end;
}
void set_cursor(char* p) noexcept
{
m_pos = p;
}
private:
static StringType& sized(StringType& out, std::size_t estimate)
{
@@ -95,6 +112,105 @@ class output_buffer
char* m_end;
};
/// The length of the run at s that dump() writes unchanged without
/// ensure_ascii: all bytes but quotes, backslashes, and control characters.
/// Unlike detail::string_bulk_run(), non-ASCII bytes are not validated: the
/// strings of a document are valid UTF-8 (a damaged image loaded with
/// image_check::bounds can have others, which are then written unchanged).
inline std::size_t plain_output_run(const unsigned char* s, std::size_t n) noexcept
{
constexpr std::uint64_t ones = 0x0101010101010101ull;
constexpr std::uint64_t high = 0x8080808080808080ull;
std::size_t i = 0;
for (; i + 8 <= n; i += 8)
{
const std::uint64_t v = read_eight_bytes(s + i);
const std::uint64_t q = v ^ 0x2222222222222222ull; // '"'
const std::uint64_t b = v ^ 0x5C5C5C5C5C5C5C5Cull; // '\\'
const std::uint64_t stop = (((q - ones) & ~q) | ((b - ones) & ~b) | ((v - 0x2020202020202020ull) & ~v)) & high;
if (stop != 0)
{
// the lowest flagged byte is the first stop: borrows only flag bytes above a true one
return i + (static_cast<std::size_t>(count_trailing_zeros(stop)) / 8);
}
}
for (; i < n; ++i)
{
if (s[i] == '"' || s[i] == '\\' || s[i] < 0x20)
{
return i;
}
}
return n;
}
/// A stack that starts in a buffer of the caller (a local array) and moves to
/// the heap (a vector of the caller) only when that is full, so that dumps of
/// shallow documents need no allocation. The top is a pointer, as in
/// std::vector. The address of the stack never escapes (the growth gets the
/// vector and returns the new storage), so its pointers stay in registers.
template<typename T>
class small_stack
{
public:
small_stack(T* buffer, std::size_t capacity, std::vector<T>& heap) noexcept
: m_begin(buffer), m_top(buffer), m_end(buffer + capacity), m_heap(&heap)
{}
small_stack(const small_stack&) = delete;
small_stack(small_stack&&) = delete;
small_stack& operator=(const small_stack&) = delete;
small_stack& operator=(small_stack&&) = delete;
~small_stack() = default;
NLOHMANN_VIEW_ALWAYS_INLINE void push_back(const T& x)
{
if (NLOHMANN_VIEW_UNLIKELY(m_top == m_end))
{
const std::size_t used = size();
const std::size_t capacity = 2 * static_cast<std::size_t>(m_end - m_begin);
m_begin = grow(*m_heap, m_begin, used, capacity);
m_top = m_begin + used;
m_end = m_begin + capacity;
}
*m_top++ = x;
}
NLOHMANN_VIEW_ALWAYS_INLINE T& back() noexcept
{
return m_top[-1];
}
NLOHMANN_VIEW_ALWAYS_INLINE void pop_back() noexcept
{
--m_top;
}
NLOHMANN_VIEW_ALWAYS_INLINE bool empty() const noexcept
{
return m_top == m_begin;
}
NLOHMANN_VIEW_ALWAYS_INLINE std::size_t size() const noexcept
{
return static_cast<std::size_t>(m_top - m_begin);
}
private:
/// the used entries moved to heap storage of the given capacity
NLOHMANN_VIEW_NOINLINE static T* grow(std::vector<T>& heap, const T* begin, std::size_t used, std::size_t capacity)
{
std::vector<T> bigger(capacity);
std::copy(begin, begin + used, bigger.begin());
heap.swap(bigger);
return heap.data();
}
T* m_begin;
T* m_top;
T* m_end;
std::vector<T>* m_heap;
};
/// how the view's dump() writes a value
struct dump_style
{
@@ -115,9 +231,10 @@ the library's conversion; integers are copied from the source, where they
are canonical (except "-0", which parse() reads as 0). The walk is
iterative, so the nesting depth is limited by memory only.
*/
template<typename BasicJsonType>
template<typename BasicJsonType, bool Editable>
class view_serializer
{
using nav = navigation<Editable>;
using string_t = typename BasicJsonType::string_t;
using number_float_t = typename BasicJsonType::number_float_t;
@@ -128,6 +245,18 @@ class view_serializer
void dump(const node* root)
{
if (!m_style.pretty && !m_style.ensure_ascii)
{
if (m_style.source_numbers)
{
dump_compact<true>(root);
}
else
{
dump_compact<false>(root);
}
return;
}
struct frame
{
const node* pos; ///< next element, or key of the next member
@@ -135,7 +264,9 @@ class view_serializer
bool object;
bool first; ///< nothing written yet
};
std::vector<frame> stack;
std::array<frame, 32> buffer; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
std::vector<frame> heap;
small_stack<frame> stack(buffer.data(), buffer.size(), heap);
const node* n = root;
for (;;)
{
@@ -150,7 +281,7 @@ class view_serializer
else
{
m_out.put(object ? '{' : '[');
stack.push_back(frame{document_data::first_child(n), document_data::child_end(n), object, true});
stack.push_back(frame{nav::first(m_doc, n), nav::end(m_doc, n), object, true});
}
}
else
@@ -192,19 +323,298 @@ class view_serializer
{
m_out.put(':');
}
n = f.pos + 1;
n = nav::value(f.pos + 1);
f.pos = document_data::after(f.pos + 1);
}
else
{
n = f.pos;
n = nav::value(f.pos);
f.pos = document_data::after(f.pos);
}
f.pos = document_data::after(n);
break;
}
}
}
private:
/*!
@brief the compact output without ensure_ascii (the default dump())
The same walk as dump(), with the write position in a local variable
(stores through char pointers would otherwise force a reload of the
buffer's members after each one), and with strings and number tokens of
the source copied by fixed-size moves of 32 bytes where the source has
that many bytes left, instead of a library call per token. The buffer
keeps 64 bytes of slack for the overshoot.
*/
/// a string that is not a plain string of the source (decoded, or written
/// by an edit), without ensure_ascii: runs without characters to escape
/// are copied
NLOHMANN_VIEW_NOINLINE void write_decoded(const node& n)
{
const auto* const s = reinterpret_cast<const unsigned char*>(m_doc.str(n)); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
m_out.put('"');
for (std::size_t i = 0; i < n.len;)
{
const std::size_t run = plain_output_run(s + i, n.len - i);
if (run != 0)
{
m_out.put(reinterpret_cast<const char*>(s + i), run); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
i += run;
continue;
}
write_codepoint<false>(s[i], s + i, 1); // a quote, a backslash, or a control character
++i;
}
m_out.put('"');
}
/// the copies of dump_compact() that are not fixed-size moves (long
/// strings, or near the end of the source); out of line, so that the
/// compiler does not merge the fixed-size moves into this call
NLOHMANN_VIEW_NOINLINE static void copy_long(char* to, const char* from, std::size_t n) noexcept
{
std::memcpy(to, from, n);
}
template<bool SourceNumbers>
void dump_compact(const node* root)
{
struct frame
{
const node* pos; ///< (editable documents) next element, or key of the next member
const node* end;
bool object;
};
std::array<frame, 32> buffer; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
std::vector<frame> heap;
small_stack<frame> stack(buffer.data(), buffer.size(), heap);
const char* const src = m_doc.src;
const char* const src_end = src + m_doc.size;
char* w = m_out.cursor();
char* lim = m_out.limit();
// room for n bytes and the slack
const auto room = [&](std::size_t n)
{
if (NLOHMANN_VIEW_UNLIKELY(static_cast<std::size_t>(lim - w) < n + 64))
{
m_out.set_cursor(w);
m_out.reserve(n + 64);
w = m_out.cursor();
lim = m_out.limit();
}
};
// copy n bytes of the source (after room(n))
const auto copy = [&](const char* from, std::size_t n)
{
if (n <= 32 && src_end - from >= 32)
{
std::memcpy(w, from, 32);
}
else if (n <= 256 && src_end - from >= static_cast<std::ptrdiff_t>(n) + 32)
{
for (std::size_t i = 0; i < n; i += 32)
{
std::memcpy(w + i, from + i, 32);
}
}
else
{
copy_long(w, from, n);
}
w += n;
};
// a literal of n bytes (after room(n))
const auto literal = [&](const char* text, std::size_t n)
{
std::memcpy(w, text, n);
w += n;
};
// a string that is not a plain string of the source (out of line, so
// that the cursor stays in a register here)
const auto escaped = [&](const node & n)
{
m_out.set_cursor(w);
write_decoded(n);
w = m_out.cursor();
lim = m_out.limit();
};
// Read-only documents: the elements of a container follow it in the
// node array, so the walk goes through the array in order, and a
// frame only needs the end of its container. Editable documents: the
// elements of a moved container live elsewhere, so a frame keeps the
// position of the next element (see navigation).
// The innermost open container is kept in registers (cur; end ==
// nullptr: none), the stack holds the ones around it.
frame cur{nullptr, nullptr, false};
const node* n = root;
for (;;)
{
// write the value at n (read-only documents: and advance n)
bool opened = false;
switch (static_cast<value_t>(n->kind))
{
case value_t::string:
if ((n->flags & node_flags::storage) == 0)
{
room(n->len + 2);
*w++ = '"';
copy(src + n->off, n->len);
*w++ = '"';
}
else
{
escaped(*n);
}
break;
case value_t::number_integer:
case value_t::number_unsigned:
{
const std::uint32_t len = number_length(*n);
room(len);
if (Editable && (n->flags & node_flags::storage) != 0)
{
copy_long(w, m_doc.str(*n), len); // a canonical token written by an edit
w += len;
break;
}
const char* const token = src + n->off;
if (!SourceNumbers && NLOHMANN_VIEW_UNLIKELY(len == 2 && token[0] == '-' && token[1] == '0'))
{
*w++ = '0'; // parse() reads -0 as the integer 0
}
else
{
copy(token, len);
}
break;
}
case value_t::number_float:
if (SourceNumbers && (n->flags & node_flags::storage) != node_flags::edited)
{
room(n->len);
copy(src + n->off, n->len);
}
else
{
m_out.set_cursor(w);
write_float(float_value<number_float_t>(m_doc, *n));
w = m_out.cursor();
lim = m_out.limit();
}
break;
case value_t::boolean:
room(8);
if ((n->flags & node_flags::is_true) != 0)
{
literal("true", 4);
}
else
{
literal("false", 5);
}
break;
case value_t::object:
case value_t::array:
{
const bool object = n->kind == static_cast<std::uint8_t>(value_t::object);
room(8);
if (n->len == 0)
{
literal(object ? "{}" : "[]", 2);
}
else
{
*w++ = object ? '{' : '[';
stack.push_back(cur);
if (Editable)
{
cur = frame{nav::first(m_doc, n), nav::end(m_doc, n), object};
}
else
{
cur = frame{nullptr, n + n->next, object};
}
opened = true;
}
break;
}
case value_t::null:
room(8);
literal("null", 4);
break;
case value_t::binary: // LCOV_EXCL_LINE (not in a document)
case value_t::discarded: // LCOV_EXCL_LINE
default: // LCOV_EXCL_LINE
break; // LCOV_EXCL_LINE
}
if (!Editable)
{
++n; // the next node: the first element of an opened container, or the node after a scalar
}
// go to the next value: close finished containers, then separate
// (a container just opened has an element)
if (!opened)
{
for (;;)
{
if (cur.end == nullptr)
{
m_out.set_cursor(w);
m_out.finish();
return;
}
if ((Editable ? cur.pos : n) != cur.end)
{
break;
}
room(1);
*w++ = cur.object ? '}' : ']';
cur = stack.back();
stack.pop_back();
}
room(1);
*w++ = ',';
}
const node* const at = Editable ? cur.pos : n;
if (cur.object)
{
const node& key = *at;
if ((key.flags & node_flags::storage) == 0)
{
room(key.len + 3);
*w++ = '"';
copy(src + key.off, key.len);
w[0] = '"';
w[1] = ':';
w += 2;
}
else
{
escaped(key);
room(1);
*w++ = ':';
}
if (Editable)
{
n = nav::value(at + 1);
cur.pos = document_data::after(at + 1);
}
else
{
++n;
}
}
else if (Editable)
{
n = nav::value(at);
cur.pos = document_data::after(at);
}
}
}
void newline(std::size_t level)
{
if (m_style.pretty)
@@ -250,9 +660,9 @@ class view_serializer
break;
}
case value_t::number_float:
if (m_style.source_numbers)
if (m_style.source_numbers && (n.flags & node_flags::storage) != node_flags::edited)
{
m_out.put(m_doc.str(n), n.len);
m_out.put(m_doc.str(n), n.len); // (a float set by an edit is written as with shortest)
}
else
{
@@ -299,9 +709,9 @@ class view_serializer
{
const char* const s = m_doc.str(n);
m_out.put('"');
if ((n.flags & node_flags::escaped) == 0 && !m_style.ensure_ascii)
if ((n.flags & node_flags::storage) == 0 && !m_style.ensure_ascii)
{
// a string without escape sequences has nothing to escape
// a string of the source without escape sequences has nothing to escape
m_out.put(s, n.len);
}
else if (m_style.ensure_ascii)
@@ -315,7 +725,9 @@ class view_serializer
m_out.put('"');
}
/// as serializer::dump_escaped() for valid UTF-8 (the view has no other)
/// as serializer::dump_escaped(); strings of a document are valid UTF-8,
/// except in a damaged image loaded with image_check::bounds, for which
/// this throws what basic_json::dump() throws for the string
template<bool EnsureAscii>
void write_escaped(const unsigned char* s, std::size_t n)
{
@@ -339,12 +751,13 @@ class view_serializer
}
std::uint32_t codepoint = s[i];
std::size_t len = 1;
if (codepoint >= 0xC0)
if (codepoint >= 0x80)
{
len = 2;
if (codepoint >= 0xE0)
len = validate_one_utf8(s + i, n - i);
if (NLOHMANN_VIEW_UNLIKELY(len == 0))
{
len = codepoint >= 0xF0 ? 4 : 3;
invalid_utf8(s, n);
return;
}
codepoint &= 0xFFu >> (len + 1);
for (std::size_t k = 1; k < len; ++k)
@@ -357,6 +770,13 @@ class view_serializer
}
}
/// throw what basic_json::dump() throws for a string that is not valid UTF-8
NLOHMANN_VIEW_NOINLINE static void invalid_utf8(const unsigned char* s, std::size_t n)
{
const string_t dumped = BasicJsonType(string_t(reinterpret_cast<const char*>(s), n)).dump(); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
static_cast<void>(dumped);
}
template<bool EnsureAscii>
void write_codepoint(std::uint32_t codepoint, const unsigned char* bytes, std::size_t len)
{
+276 -27
View File
@@ -25,7 +25,7 @@
#define INCLUDE_NLOHMANN_JSON_VIEW_HPP_
#include <cstddef> // size_t
#include <cstdint> // uint32_t
#include <cstdint> // uint8_t, uint32_t
#include <cstring> // memcpy, strlen
#include <iterator> // distance, input_iterator_tag, iterator_traits
#include <map> // map
@@ -50,7 +50,10 @@
#include <nlohmann/detail/view/builder.hpp>
#include <nlohmann/detail/view/compare.hpp>
#include <nlohmann/detail/view/document_data.hpp>
#include <nlohmann/detail/view/edit.hpp>
#include <nlohmann/detail/view/edit_storage.hpp>
#include <nlohmann/detail/view/errors.hpp>
#include <nlohmann/detail/view/image.hpp>
#include <nlohmann/detail/view/input.hpp>
#include <nlohmann/detail/view/iterator.hpp>
#include <nlohmann/detail/view/lookup.hpp>
@@ -65,7 +68,7 @@
NLOHMANN_JSON_NAMESPACE_BEGIN
template<typename BasicJsonType>
template<typename BasicJsonType, bool Editable>
class basic_json_document;
/*!
@@ -74,11 +77,13 @@ class basic_json_document;
Trivially copyable (two pointers). Valid as long as the document is alive and
has not been re-parsed, and as long as a borrowed source text is alive.
*/
template<typename BasicJsonType>
template<typename BasicJsonType, bool Editable = false>
class basic_json_view
{
using node = detail::view::node;
using document_data = detail::view::document_data;
/// how the index is walked (with edits only for editable documents)
using navigation = detail::view::navigation<Editable>;
public:
using value_t = detail::value_t;
@@ -271,7 +276,7 @@ class basic_json_view
{
detail::view::throw_type_error(305, "cannot use operator[] with a numeric argument with ", type_name());
}
return idx < m_node->len ? basic_json_view(m_doc, detail::view::element_at(m_node, idx)) : basic_json_view();
return idx < m_node->len ? basic_json_view(m_doc, navigation::value(detail::view::element_at<Editable>(*m_doc, m_node, idx))) : basic_json_view();
}
/// (an int argument would be ambiguous between size_type and const char*)
@@ -327,7 +332,7 @@ class basic_json_view
{
detail::view::throw_out_of_range(401, detail::concat("array index ", std::to_string(idx), " is out of range"));
}
return basic_json_view(m_doc, detail::view::element_at(m_node, idx));
return basic_json_view(m_doc, navigation::value(detail::view::element_at<Editable>(*m_doc, m_node, idx)));
}
basic_json_view at(int idx) const
@@ -399,7 +404,7 @@ class basic_json_view
{
if (is_structured() && m_node->len != 0)
{
return basic_json_view(m_doc, detail::view::last_child(m_node) + (is_object() ? 1 : 0));
return basic_json_view(m_doc, navigation::value(detail::view::last_child<Editable>(*m_doc, m_node) + (is_object() ? 1 : 0)));
}
return front();
}
@@ -416,7 +421,7 @@ class basic_json_view
{
return end();
}
const node* const k = detail::view::find_member(*m_doc, m_node, key.data(), key.size());
const node* const k = detail::view::find_member<Editable>(*m_doc, m_node, key.data(), key.size());
return k != nullptr ? iterator(m_doc, k, true) : end();
}
@@ -433,7 +438,7 @@ class basic_json_view
/// whether this is an object with a member with this key
bool contains(string_view_t key) const
{
return is_object() && detail::view::find_member(*m_doc, m_node, key.data(), key.size()) != nullptr;
return is_object() && detail::view::find_member<Editable>(*m_doc, m_node, key.data(), key.size()) != nullptr;
}
bool contains(const char* key) const
@@ -480,7 +485,7 @@ class basic_json_view
{
if (NLOHMANN_VIEW_LIKELY(is_structured()))
{
return iterator(m_doc, document_data::first_child(m_node), is_object());
return iterator(m_doc, navigation::first(*m_doc, m_node), is_object());
}
return iterator(m_doc, m_node, false);
}
@@ -489,7 +494,7 @@ class basic_json_view
{
if (NLOHMANN_VIEW_LIKELY(is_structured()))
{
return iterator(m_doc, document_data::child_end(m_node), is_object());
return iterator(m_doc, navigation::end(*m_doc, m_node), is_object());
}
return iterator(m_doc, (is_null() || is_discarded()) ? m_node : m_node + 1, false);
}
@@ -587,9 +592,11 @@ class basic_json_view
style.indent_char = indent_char;
style.ensure_ascii = ensure_ascii;
style.source_numbers = numbers == number_format::source;
// the compact text is about as long as the source text of the value
const std::size_t estimate = source_extent() + (style.pretty ? source_extent() / 2 : 0) + 64;
detail::view::view_serializer<BasicJsonType>(*m_doc, out, estimate, style).dump(m_node);
// the compact text is about as long as the source text of the value;
// the compact writer keeps 64 bytes of slack, so that it does not grow
// the buffer just before the end
const std::size_t estimate = source_extent() + (style.pretty ? source_extent() / 2 : 0) + 160;
detail::view::view_serializer<BasicJsonType, Editable>(*m_doc, out, estimate, style).dump(m_node);
return out;
}
@@ -623,6 +630,19 @@ class basic_json_view
return !(a == b);
}
/// a view of an editable document compares with one of a read-only document
template < bool E, typename std::enable_if < E != Editable, int >::type = 0 >
friend bool operator==(const basic_json_view& a, const basic_json_view<BasicJsonType, E>& b)
{
return detail::view::equal<BasicJsonType>(side(a), detail::view::view_side<BasicJsonType, basic_json_view<BasicJsonType, E>>(b));
}
template < bool E, typename std::enable_if < E != Editable, int >::type = 0 >
friend bool operator!=(const basic_json_view& a, const basic_json_view<BasicJsonType, E>& b)
{
return !(a == b);
}
/// whether the value parse() would produce for a view equals a value
friend bool operator==(const basic_json_view& a, const BasicJsonType& j)
{
@@ -656,7 +676,7 @@ class basic_json_view
{
return BasicJsonType(value_t::discarded);
}
return detail::view::materialize<BasicJsonType>(*m_doc, m_node);
return detail::view::materialize<BasicJsonType, Editable>(*m_doc, m_node);
}
/// byte offset of this value in the source text (for strings: of the
@@ -664,12 +684,14 @@ class basic_json_view
/// a discarded view and for strings with escapes, which are decoded
std::size_t source_offset() const noexcept
{
return m_node != nullptr && (m_node->flags & detail::view::node_flags::storage) == 0
return m_node != nullptr && (m_node->flags & (detail::view::node_flags::storage | detail::view::node_flags::moved | detail::view::node_flags::is_new)) == 0
? m_node->off : static_cast<std::size_t>(-1);
}
private:
template<typename> friend class basic_json_document;
template<typename, bool> friend class basic_json_document;
template<typename, bool> friend class basic_json_view;
template<typename, typename> friend class detail::view::editor;
friend iterator;
basic_json_view(const document_data* d, const node* n) noexcept
@@ -687,6 +709,11 @@ class basic_json_view
/// decoded strings)
std::size_t source_extent() const noexcept
{
if (Editable && m_doc->edits != nullptr)
{
// positions of moved and new values are not source offsets
return m_node == m_doc->tape ? m_doc->size + m_doc->edits->text_used : 64;
}
const node* const next = document_data::after(m_node);
const bool in_source = (m_node->flags & detail::view::node_flags::storage) == 0;
if (!in_source)
@@ -704,8 +731,8 @@ class basic_json_view
/// (object required)
NLOHMANN_VIEW_ALWAYS_INLINE basic_json_view lookup(string_view_t key) const noexcept
{
const node* const k = detail::view::find_member(*m_doc, m_node, key.data(), key.size());
return k != nullptr ? basic_json_view(m_doc, k + 1) : basic_json_view();
const node* const k = detail::view::find_member<Editable>(*m_doc, m_node, key.data(), key.size());
return k != nullptr ? basic_json_view(m_doc, navigation::value(k + 1)) : basic_json_view();
}
// --- get() dispatch ---
@@ -796,7 +823,7 @@ Borrowed parses keep a pointer to the caller's text, which must outlive the
document. Owned parses (parse_copy, rvalue std::string, streams, and inputs
that are not contiguous byte ranges) keep their own copy.
*/
template<typename BasicJsonType>
template<typename BasicJsonType, bool Editable = false>
class basic_json_document
{
using document_data = detail::view::document_data;
@@ -805,7 +832,7 @@ class basic_json_document
"json_view supports 64-bit integer types only");
public:
using view_type = basic_json_view<BasicJsonType>;
using view_type = basic_json_view<BasicJsonType, Editable>;
using value_t = detail::value_t;
/// an empty (discarded) document
@@ -909,7 +936,7 @@ class basic_json_document
/// whether the document holds its own copy of the text
bool owns_source() const noexcept
{
return m_data && !m_data->owned.empty() && m_data->src == m_data->owned.data();
return m_data && ((!m_data->owned.empty() && m_data->src == m_data->owned.data()) || !m_data->owned_image.empty());
}
/// number of index nodes (values plus object keys)
@@ -918,7 +945,7 @@ class basic_json_document
return m_data ? m_data->tape_size : 0;
}
/// bytes held by the document (index, decoded strings, owned text)
/// bytes held by the document (index, decoded strings, owned text or image)
std::size_t memory_usage() const noexcept
{
if (!m_data)
@@ -927,9 +954,10 @@ class basic_json_document
}
return sizeof(document_data) + (m_data->inline_cap * sizeof(detail::view::node))
+ (m_data->tape != m_data->inline_tape ? m_data->tape_cap * sizeof(detail::view::node) : 0)
+ m_data->arena.capacity() + m_data->owned.capacity()
+ m_data->arena.capacity() + m_data->owned.capacity() + m_data->owned_image.capacity()
+ (m_data->indexes.capacity() * sizeof(document_data::object_index)) + (m_data->index_slots.capacity() * sizeof(std::uint32_t))
+ (m_data->large_objects.capacity() * sizeof(std::uint32_t));
+ (m_data->large_objects.capacity() * sizeof(std::uint32_t))
+ (m_data->edits != nullptr ? m_data->edits->bytes : 0);
}
/// release unused capacity of the index and the decoded strings; like
@@ -946,9 +974,12 @@ class basic_json_document
// allocate everything first, so that an exception leaves the document
// unchanged
// (the decoded strings of a loaded image stay in the image)
const bool arena_in_use = d.base[1] == d.arena.data();
const bool shrink_arena = d.arena.capacity() > d.arena.size();
std::string arena(shrink_arena ? d.arena : std::string());
const bool shrink_tape = d.tape != d.inline_tape && d.tape_size != d.tape_cap;
std::string arena(shrink_arena && arena_in_use ? d.arena : std::string());
// (edits link to the nodes of the index, which then stays in place)
const bool shrink_tape = d.tape != d.inline_tape && d.tape_size != d.tape_cap && d.edits == nullptr;
const bool into_header = d.tape_size <= d.inline_cap;
node* fresh = (shrink_tape && !into_header) ? static_cast<node*>(::operator new (d.tape_size * sizeof(node))) : d.inline_tape;
@@ -962,13 +993,219 @@ class basic_json_document
if (shrink_arena)
{
d.arena.swap(arena);
d.base[1] = d.arena.data();
if (arena_in_use)
{
d.base[1] = d.arena.data();
}
}
}
////////////
// images //
////////////
/// how load() checks an image (full, bounds, or none)
using image_check = detail::view::image_check;
/// The document as an image that load() reads without parsing: the node
/// index, the text, and the decoded strings. An edited document is
/// written in its current state (floats that are not finite become null,
/// as in dump()).
std::vector<std::uint8_t> save() const
{
if (NLOHMANN_VIEW_UNLIKELY(!m_data || m_data->discarded))
{
detail::view::throw_type_error(320, "cannot save a discarded json_document");
}
return detail::view::save_image(*m_data);
}
/// Read an image written by save(). The image is borrowed: it must stay
/// alive and unchanged while the document is used.
NLOHMANN_VIEW_NODISCARD
static basic_json_document load(const std::uint8_t* image, std::size_t size, const image_check check = image_check::full)
{
basic_json_document d;
d.ensure_data(nullptr, 0);
detail::view::load_image(*d.m_data, image, size, check);
return d;
}
/// read an image (borrowed)
NLOHMANN_VIEW_NODISCARD
static basic_json_document load(const std::vector<std::uint8_t>& image, const image_check check = image_check::full)
{
return load(image.data(), image.size(), check);
}
/// read an image and keep it (no copy)
NLOHMANN_VIEW_NODISCARD
static basic_json_document load(std::vector<std::uint8_t>&& image, const image_check check = image_check::full)
{
basic_json_document d;
d.ensure_data(nullptr, 0);
d.m_data->owned_image = std::move(image);
detail::view::load_image(*d.m_data, d.m_data->owned_image.data(), d.m_data->owned_image.size(), check);
return d;
}
///////////
// edits //
///////////
// The source text is never written; new values go to storage owned by
// the document. A view keeps referring to the same value: after an
// assignment it sees the new value, and edits elsewhere do not affect it.
// A view of an erased value keeps its last value. An edit of an
// array/object invalidates the iterators over it. Values are accepted as
// views (of any document), BasicJsonType values, and everything
// BasicJsonType can be constructed from.
using string_view_t = typename view_type::string_view_t;
using json_pointer = typename BasicJsonType::json_pointer;
/// replace a value (a view of this document); returns a view of it
template<typename V>
view_type set(view_type target, V&& value)
{
return editor().set(target, std::forward<V>(value));
}
/// set a member (added if missing; a null value becomes an object);
/// returns a view of the member value
template<typename V>
view_type set(view_type object, string_view_t key, V&& value)
{
return editor().set(object, key, std::forward<V>(value));
}
/// assign an existing array element; returns a view of it
template < typename I, typename V, typename std::enable_if < std::is_integral<I>::value && !std::is_same<I, bool>::value, int >::type = 0 >
view_type set(view_type array, I idx, V && value)
{
return editor().set(array, index(idx), std::forward<V>(value));
}
/// set the value at a JSON pointer: its parent must exist; an object
/// member is set (added if missing), an array element assigned, and "-"
/// or the size of the array appends
template<typename V>
view_type set(const json_pointer& ptr, V&& value)
{
if (ptr.empty())
{
return set(root(), std::forward<V>(value));
}
const view_type parent = root().at(ptr.parent_pointer());
const auto& token = ptr.back();
if (parent.is_array())
{
const std::size_t idx = token == "-" ? parent.size() : pointer_index(token);
if (idx == parent.size())
{
return push_back(parent, std::forward<V>(value));
}
return set(parent, idx, std::forward<V>(value));
}
return set(parent, string_view_t(token.data(), token.size()), std::forward<V>(value));
}
/// append to an array (a null value becomes an array); returns a view of
/// the new element
template<typename V>
view_type push_back(view_type array, V&& value)
{
return editor().push_back(array, std::forward<V>(value));
}
/// insert into an array before position idx (idx <= size()); returns a
/// view of the new element
template < typename I, typename V, typename std::enable_if < std::is_integral<I>::value && !std::is_same<I, bool>::value, int >::type = 0 >
view_type insert(view_type array, I idx, V && value)
{
return editor().insert(array, index(idx), std::forward<V>(value));
}
/// remove all members with this key; returns their number
std::size_t erase(view_type object, string_view_t key)
{
return editor().erase(object, key);
}
/// remove an array element
template < typename I, typename std::enable_if < std::is_integral<I>::value && !std::is_same<I, bool>::value, int >::type = 0 >
void erase(view_type array, I idx)
{
editor().erase(array, index(idx));
}
/// remove the value at a JSON pointer; returns the number of removed
/// values
std::size_t erase(const json_pointer& ptr)
{
if (ptr.empty())
{
detail::view::throw_out_of_range(405, "JSON pointer has no parent");
}
const view_type parent = root().at(ptr.parent_pointer());
const auto& token = ptr.back();
if (parent.is_array())
{
erase(parent, pointer_index(token));
return 1;
}
return erase(parent, string_view_t(token.data(), token.size()));
}
private:
using input_kind = detail::view::input_kind;
detail::view::editor<BasicJsonType, view_type> editor()
{
static_assert(Editable, "only an editable document can be edited: use basic_json_document<BasicJsonType, true> (json_editable_document)");
if (NLOHMANN_VIEW_UNLIKELY(!m_data || m_data->discarded))
{
detail::view::throw_invalid_iterator(202, "view does not belong to this document");
}
return detail::view::editor<BasicJsonType, view_type>(*m_data);
}
/// an index (out_of_range.401 if negative)
template<typename I>
static std::size_t index(I idx)
{
return index(idx, std::is_signed<I> {});
}
template<typename I>
static std::size_t index(I idx, std::true_type /*signed*/)
{
if (idx < 0)
{
detail::view::throw_out_of_range(401, detail::concat("array index ", std::to_string(idx), " is out of range"));
}
return static_cast<std::size_t>(idx);
}
template<typename I>
static std::size_t index(I idx, std::false_type /*unsigned*/)
{
return static_cast<std::size_t>(idx);
}
/// the array index of a JSON pointer token (json_pointer's rules)
template<typename StringType>
static std::size_t pointer_index(const StringType& token)
{
std::size_t idx = 0;
const detail::view::index_status status = detail::view::array_index(token, idx);
if (status != detail::view::index_status::ok)
{
detail::view::throw_array_index_error(status, token);
}
return idx;
}
/// create the storage (sized for the input) on first use
void ensure_data(const char* src, std::size_t size)
{
@@ -995,9 +1232,12 @@ class basic_json_document
{
d.owned.clear();
}
d.owned_image.clear();
d.src = src;
d.size = size;
d.tape_size = 0;
d.edits.reset(); // (views of the previous text end here anyway)
d.base[2] = nullptr;
d.arena.clear();
d.indexes.clear();
d.index_slots.clear();
@@ -1017,6 +1257,7 @@ class basic_json_document
{
d.base[0] = d.src;
d.base[1] = d.arena.data();
d.arena_size = d.arena.size();
detail::view::build_object_indexes(d);
d.discarded = false;
return;
@@ -1133,6 +1374,14 @@ using json_view = basic_json_view<json>;
using ordered_json_document = basic_json_document<ordered_json>;
/// a value of an ordered_json_document
using ordered_json_view = basic_json_view<ordered_json>;
/// an editable parsed JSON text for json
using json_editable_document = basic_json_document<json, true>;
/// a value of a json_editable_document
using json_editable_view = basic_json_view<json, true>;
/// an editable parsed JSON text for ordered_json
using ordered_json_editable_document = basic_json_document<ordered_json, true>;
/// a value of an ordered_json_editable_document
using ordered_json_editable_view = basic_json_view<ordered_json, true>;
NLOHMANN_JSON_NAMESPACE_END
+482 -23
View File
@@ -23918,11 +23918,240 @@ NLOHMANN_JSON_NAMESPACE_END
#include <array> // array
#include <cmath> // signbit, isfinite
#include <cstddef> // size_t
#include <cstdint> // intN_t, uintN_t
#include <cstring> // memcpy, memmove
#include <limits> // numeric_limits
#include <type_traits> // conditional
#ifdef _MSC_VER
#include <cstdlib> // _byteswap_uint64
#endif
// #include <nlohmann/detail/conversions/zmij.hpp>
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2025 Victor Zverovich <https://github.com/vitaut/zmij>
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#include <array> // array
#include <cstddef> // size_t
#include <cstdint> // uint32_t, uint64_t
// #include <nlohmann/detail/abi_macros.hpp>
// #include <nlohmann/detail/bit_ops.hpp>
// #include <nlohmann/detail/input/pow5_table.hpp>
// #include <nlohmann/detail/macro_scope.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN
namespace detail
{
/*!
@brief the shortest decimal representation of a double
A C++11 port of the conversion of Zmij by Victor Zverovich
(https://github.com/vitaut/zmij, MIT license): the shortest decimal in the
rounding interval of a double, the closest one if there are several. Zmij
credits Xiang JunBo (producing the shorter candidate without a division) and
Dougall Johnson (the compressed powers of ten). The powers of ten are taken
from the table for number parsing (pow5_table.hpp) where it holds them, and
computed from the compressed tables of Zmij beyond it.
*/
namespace zmij
{
/// significand * 10^exponent
struct decimal
{
std::uint64_t significand;
int exponent;
};
/// the compressed powers of ten of Zmij
inline const std::array<std::uint64_t, 28>& pow10_minor() noexcept
{
static const std::array<std::uint64_t, 28> table =
{
{
0x8000000000000000u, 0xa000000000000000u, 0xc800000000000000u, 0xfa00000000000000u, 0x9c40000000000000u,
0xc350000000000000u, 0xf424000000000000u, 0x9896800000000000u, 0xbebc200000000000u, 0xee6b280000000000u,
0x9502f90000000000u, 0xba43b74000000000u, 0xe8d4a51000000000u, 0x9184e72a00000000u, 0xb5e620f480000000u,
0xe35fa931a0000000u, 0x8e1bc9bf04000000u, 0xb1a2bc2ec5000000u, 0xde0b6b3a76400000u, 0x8ac7230489e80000u,
0xad78ebc5ac620000u, 0xd8d726b7177a8000u, 0x878678326eac9000u, 0xa968163f0a57b400u, 0xd3c21bcecceda100u,
0x84595161401484a0u, 0xa56fa5b99019a5c8u, 0xcecb8f27f4200f3au
}
};
return table;
}
/// (high, low) pairs
inline const std::array<std::uint64_t, 50>& pow10_major() noexcept
{
static const std::array<std::uint64_t, 50> table =
{
{
0xaddcb9e83c6b1793u, 0xdf4abe242a1bbf3eu, 0xaf8e5410288e1b6fu, 0x07ecf0ae5ee44ddau, 0xb1442798f49ffb4au, 0x99cd11cfdf41779du,
0xb2fe3f0b8599ef07u, 0x861fa7e6dcb4aa15u, 0xb4bca50b065abe63u, 0x0fed077a756b53aau, 0xb67f6455292cbf08u, 0x1a3bc84c17b1d543u,
0xb84687c269ef3bfbu, 0x3d5d514f40eea742u, 0xba121a4650e4ddebu, 0x92f34d62616ce413u, 0xbbe226efb628afeau, 0x890489f70a55368cu,
0xbdb6b8e905cb600fu, 0x5400e987bbc1c921u, 0xbf8fdb78849a5f96u, 0xde98520472bdd034u, 0xc16d9a0095928a27u, 0x75b7053c0f178294u,
0xc350000000000000u, 0x0000000000000000u, 0xc5371912364ce305u, 0x6c28000000000000u, 0xc722f0ef9d80aad6u, 0x424d3ad2b7b97ef6u,
0xc913936dd571c84cu, 0x03bc3a19cd1e38eau, 0xcb090c8001ab551cu, 0x5cadf5bfd3072cc6u, 0xcd036837130890a1u, 0x36dba887c37a8c10u,
0xcf02b2c21207ef2eu, 0x94f967e45e03f4bcu, 0xd106f86e69d785c7u, 0xe13336d701beba52u, 0xd31045a8341ca07cu, 0x1ede48111209a051u,
0xd51ea6fa85785631u, 0x552a74227f3ea566u, 0xd732290fbacaf133u, 0xa97c177947ad4096u, 0xd94ad8b1c7380874u, 0x18375281ae7822bdu,
0xdb68c2ca82ed2a05u, 0xa67398db9f6820e1u
}
};
return table;
}
/// one bit per power: whether the computed value is one unit too large
inline const std::array<std::uint32_t, 21>& pow10_fixups() noexcept
{
static const std::array<std::uint32_t, 21> table =
{
{
0x8d8fc810u, 0x06100293u, 0x19000000u, 0x00100000u, 0x00000908u, 0x00000000u, 0x04e00300u, 0x3807e0b2u, 0x3d83d793u, 0x0006f5ccu,
0x00000000u, 0xffff0000u, 0x8076337du, 0x4ff45ba0u, 0x09405033u, 0x034376d9u, 0x09000000u, 0x4e100501u, 0x076d14dcu, 0xf964f45eu,
0x0000003du
}
};
return table;
}
/// the 128-bit significand of 10^k, rounded down, for k in [-307, 341]
/// (compute_pow10 of Zmij)
inline uint128_parts compute_pow10(int k) noexcept
{
const auto i = static_cast<unsigned>(k + 307);
const std::uint64_t m = pow10_minor()[(i + 24) % 28];
const std::size_t j = 2 * static_cast<std::size_t>((i + 24) / 28);
const std::uint64_t h_hi = pow10_major()[j];
const std::uint64_t h_lo = pow10_major()[j + 1];
const std::uint64_t h1 = full_multiplication(h_lo, m).high;
const std::uint64_t c0 = h_lo * m;
const std::uint64_t c1 = h1 + (h_hi * m);
const std::uint64_t c2 = (c1 < h1 ? 1u : 0u) + full_multiplication(h_hi, m).high;
uint128_parts r{};
if ((c2 >> 63u) != 0)
{
r.high = c2;
r.low = c1;
}
else
{
r.high = (c2 << 1u) | (c1 >> 63u);
r.low = (c1 << 1u) | (c0 >> 63u);
}
r.low -= (pow10_fixups()[i >> 5u] >> (i & 31u)) & 1u;
return r;
}
/// The 128-bit significand of 10^k, rounded down, for k in [-342, 341].
/// Up to 10^308, the table for number parsing holds the same significands
/// (those of 5^k), except for k in [-27, -1], where it holds them one unit
/// larger (as the Eisel-Lemire algorithm needs them).
inline uint128_parts pow10(int k) noexcept
{
if (k > pow5_128_largest_power)
{
return compute_pow10(k); // (only for the smallest doubles)
}
const auto i = 2 * static_cast<std::size_t>(k - pow5_128_smallest_power);
uint128_parts r{pow5_128()[i + 1], pow5_128()[i]};
const std::uint64_t adjust = static_cast<unsigned>(k + 27) < 27u ? 1u : 0u;
r.high -= r.low < adjust ? 1u : 0u;
r.low -= adjust;
return r;
}
/// (x_hi * 2^64 + x_lo) * y >> 64, as 128 bits
inline uint128_parts umul192_hi128(std::uint64_t x_hi, std::uint64_t x_lo, std::uint64_t y) noexcept
{
const uint128_parts p = full_multiplication(x_hi, y);
uint128_parts r{};
r.low = p.low + full_multiplication(x_lo, y).high;
r.high = p.high + (r.low < p.low ? 1u : 0u);
return r;
}
/// (x * y + c) >> 64
inline std::uint64_t umul128_add_hi64(std::uint64_t x, std::uint64_t y, std::uint64_t c) noexcept
{
const uint128_parts p = full_multiplication(x, y);
return p.high + (p.low + c < p.low ? 1u : 0u);
}
/// The shortest decimal in the rounding interval of a positive finite double
/// given by its bits, the closest one if there are several (to_decimal of
/// Zmij). The significand can end in zeros.
inline decimal to_decimal(std::uint64_t bits) noexcept
{
constexpr int extra_shift = 9;
const auto raw_exp = static_cast<int>((bits >> 52u) & 0x7FFu);
std::uint64_t bin_sig = bits & ((std::uint64_t{1} << 52u) - 1);
// a power of two has a narrower interval below (except the smallest normal)
const bool regular = bin_sig != 0 || raw_exp <= 1;
const int bin_exp = (raw_exp == 0 ? 1 : raw_exp) - 1075;
if (raw_exp != 0)
{
bin_sig |= std::uint64_t{1} << 52u;
}
// floor(log10(2^bin_exp)), or floor(log10(3/4 * 2^bin_exp)) for the irregular case
const int dec_exp = ((bin_exp * 315653) - (regular ? 0 : 131072)) >> 20;
// scaled by 10^(-dec_exp - 1): the integral part is the shorter candidate
const int shift = bin_exp + ((-(dec_exp + 1) * 217707) >> 16) + 1 + extra_shift;
const uint128_parts p10 = pow10(-dec_exp - 1);
const uint128_parts p = umul192_hi128(p10.high, p10.low, bin_sig << static_cast<unsigned>(shift));
std::uint64_t integral = p.high >> static_cast<unsigned>(extra_shift);
const std::uint64_t fractional = (p.high << static_cast<unsigned>(64 - extra_shift)) | (p.low >> static_cast<unsigned>(extra_shift));
std::uint64_t digit = 0;
bool round_up = false;
bool round_down = false;
if (JSON_HEDLEY_LIKELY(regular))
{
const std::uint64_t half_ulp = (p10.high >> static_cast<unsigned>(extra_shift + 1 - shift)) + (1 - (bin_sig & 1u));
round_up = fractional + half_ulp < fractional;
round_down = half_ulp > fractional;
// the last digit of the longer candidate, rounded to nearest
digit = umul128_add_hi64(fractional, 10, (std::uint64_t{1} << 63u) + 6);
if (fractional == (std::uint64_t{1} << 62u))
{
digit = 2; // 2.5 rounds to 2
}
}
else
{
const std::uint64_t half_ulp = p10.high >> static_cast<unsigned>(extra_shift + 1 - shift);
round_up = half_ulp > ~std::uint64_t{0} - fractional;
round_down = (half_ulp >> 1u) > fractional;
digit = umul128_add_hi64(fractional, 10, (std::uint64_t{1} << 63u) - 1);
const std::uint64_t lowest = umul128_add_hi64(fractional - (half_ulp >> 1u), 10, ~std::uint64_t{0});
digit = digit < lowest ? lowest : digit;
}
integral += round_up ? 1u : 0u;
if (!round_up && !round_down)
{
// the shorter candidate is outside the rounding interval: one digit more
return decimal{(integral * 10) + digit, dec_exp};
}
return decimal{integral, dec_exp + 1};
}
} // namespace zmij
} // namespace detail
NLOHMANN_JSON_NAMESPACE_END
// #include <nlohmann/detail/macro_scope.hpp>
@@ -24826,6 +25055,87 @@ void grisu2(char* buf, int& len, int& decimal_exponent, FloatType value)
grisu2(buf, len, decimal_exponent, w.minus, w.w, w.plus);
}
/*!
@brief the shortest digits of a positive finite float (other than double): Grisu2
*/
template<typename FloatType>
JSON_HEDLEY_NON_NULL(1)
void shortest_digits(char* buf, int& len, int& decimal_exponent, FloatType value)
{
grisu2(buf, len, decimal_exponent, value);
}
/*!
@brief the shortest digits of a positive finite double: the conversion of
Zmij (see zmij.hpp), which always finds the shortest digits that read back as
the same value (Grisu2 does not for about one double in a thousand), and the
closest of them if there are several
v = buf * 10^decimal_exponent, as for grisu2()
*/
JSON_HEDLEY_NON_NULL(1)
inline void shortest_digits(char* buf, int& len, int& decimal_exponent, double value)
{
static_assert(std::numeric_limits<double>::is_iec559 && std::numeric_limits<double>::digits == 53,
"internal error: the conversion of Zmij needs IEEE 754 binary64 doubles");
JSON_ASSERT(std::isfinite(value));
JSON_ASSERT(value > 0);
std::uint64_t bits = 0;
std::memcpy(&bits, &value, sizeof(bits));
zmij::decimal d = zmij::to_decimal(bits);
// without trailing zeros (up to 16): 8, 4, 2, 1 at a time
while (d.significand % 100000000 == 0)
{
d.significand /= 100000000;
d.exponent += 8;
}
if (d.significand % 10000 == 0)
{
d.significand /= 10000;
d.exponent += 4;
}
if (d.significand % 100 == 0)
{
d.significand /= 100;
d.exponent += 2;
}
if (d.significand % 10 == 0)
{
d.significand /= 10;
d.exponent += 1;
}
// at most 17 digits, written from the back two at a time
static constexpr const char* pairs =
"00010203040506070809101112131415161718192021222324252627282930313233343536373839"
"40414243444546474849505152535455565758596061626364656667686970717273747576777879"
"8081828384858687888990919293949596979899";
std::array<char, 20> digits{};
std::size_t n = digits.size();
while (d.significand >= 100)
{
const auto i = static_cast<std::size_t>(d.significand % 100) * 2;
d.significand /= 100;
n -= 2;
digits[n] = pairs[i];
digits[n + 1] = pairs[i + 1];
}
if (d.significand >= 10)
{
const auto i = static_cast<std::size_t>(d.significand) * 2;
n -= 2;
digits[n] = pairs[i];
digits[n + 1] = pairs[i + 1];
}
else
{
digits[--n] = static_cast<char>('0' + d.significand);
}
len = static_cast<int>(digits.size() - n);
std::memcpy(buf, digits.data() + n, static_cast<std::size_t>(len));
decimal_exponent = d.exponent;
}
/*!
@brief appends a decimal representation of e to buf
@return a pointer to the element following the exponent.
@@ -24955,6 +25265,177 @@ inline char* format_buffer(char* buf, int len, int decimal_exponent,
return append_exponent(buf, n - 1);
}
/// eight decimal digits (a value below 10^8) as bytes 0..9, the first digit
/// in the most significant byte: three steps that divide all lanes at once
/// by a multiplication (the conversion of Xiang JunBo, as in Zmij)
inline std::uint64_t eight_digit_bytes(std::uint64_t abcdefgh) noexcept
{
const std::uint64_t abcd_efgh = abcdefgh + (((std::uint64_t{1} << 32u) - 10000u) * ((abcdefgh * (((std::uint64_t{1} << 40u) / 10000u) + 1u)) >> 40u));
const std::uint64_t ab_cd_ef_gh = abcd_efgh + (((std::uint64_t{1} << 16u) - 100u) * (((abcd_efgh * (((std::uint64_t{1} << 19u) / 100u) + 1u)) >> 19u) & 0x7F0000007Fu));
return ab_cd_ef_gh + (((std::uint64_t{1} << 8u) - 10u) * (((ab_cd_ef_gh * (((std::uint64_t{1} << 10u) / 10u) + 1u)) >> 10u) & 0x000F000F000F000Fu));
}
/// store the bytes of v, the most significant one first (one byte swap and
/// one store where the byte order is known: compilers do not reliably merge
/// the byte stores once this is inlined)
inline void store_msb_first(char* p, std::uint64_t v) noexcept
{
#if defined(__BYTE_ORDER__) && defined(__ORDER_LITTLE_ENDIAN__) && __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
v = __builtin_bswap64(v);
std::memcpy(p, &v, sizeof(v));
#elif defined(__BYTE_ORDER__) && defined(__ORDER_BIG_ENDIAN__) && __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__
std::memcpy(p, &v, sizeof(v));
#elif defined(_MSC_VER) // (little-endian on all its targets)
v = _byteswap_uint64(v);
std::memcpy(p, &v, sizeof(v));
#else
for (unsigned i = 0; i < 8; ++i)
{
p[i] = static_cast<char>(v >> (56u - (8u * i)));
}
#endif
}
/*!
@brief digits * 10^exp for a double, in the layout of format_buffer()
The layout is that of format_buffer() with min_exp -4 and max_exp 15 (the
digits10 of double). The digits are converted eight at a time and placed
with fixed-size moves instead of per-digit loops and moves of the buffer.
@param[in] digits the digits (not 0, at most 17 digits; trailing zeros allowed)
@param[in] exp the decimal exponent of the last digit
@return a pointer past the text; up to 41 bytes at @a first are written
(some beyond the returned end)
*/
JSON_HEDLEY_NON_NULL(1)
JSON_HEDLEY_RETURNS_NON_NULL
inline char* write_decimal(char* first, std::uint64_t digits, int exp) noexcept
{
JSON_ASSERT(digits != 0 && digits < 100000000000000000u);
const std::uint64_t upper = digits / 100000000u;
const std::uint64_t b0 = upper / 100000000u; // (one digit: it is its own byte)
const std::uint64_t b1 = eight_digit_bytes(upper % 100000000u);
const std::uint64_t b2 = eight_digit_bytes(digits % 100000000u);
// leading and trailing zero digits: zero bytes, counted without division
int leading = 16;
int zeros = 16;
if (b0 != 0)
{
leading = count_leading_zeros(b0) / 8;
}
else if (b1 != 0)
{
leading = 8 + (count_leading_zeros(b1) / 8);
}
else
{
leading += count_leading_zeros(b2) / 8;
}
if (b2 != 0)
{
zeros = count_trailing_zeros(b2) / 8;
}
else if (b1 != 0)
{
zeros = 8 + (count_trailing_zeros(b1) / 8);
}
// (else: 16, b0 is the one digit that is not 0)
// the digits as text at text + leading, then '0's, so that fixed-size
// moves need not check how many digits there are
std::array<char, 64> text; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
store_msb_first(text.data(), b0 + 0x3030303030303030u);
store_msb_first(text.data() + 8, b1 + 0x3030303030303030u);
store_msb_first(text.data() + 16, b2 + 0x3030303030303030u);
std::memset(text.data() + 24, '0', 40);
const int k = 24 - leading - zeros; // significant digits
const int n = k + exp + zeros; // position of the decimal point after the first digit
const char* const s0 = text.data() + leading;
if (-4 < n && n <= 15)
{
// "0.[000]digits" (n <= 0) is the digits after 1 - n leading '0's
// with the point after the first; "digits[000].0" (n >= k) and
// "dig.its" put the point after n characters
const int pad = n <= 0 ? 1 - n : 0;
const char* const s = s0 - pad;
const int len = k + pad;
const int point = n + pad;
std::memcpy(first, s, 16);
std::memcpy(first + point + 1, s + point, 24);
first[point] = '.';
return first + (point >= len ? point + 2 : len + 1);
}
// d.igitse+XX, with at least two exponent digits (as append_exponent())
std::memcpy(first, s0, 16);
std::memcpy(first + 2, s0 + 1, 16);
first[1] = '.';
char* const end = first + (k == 1 ? 1 : k + 1);
const int e = n - 1;
const auto ea = static_cast<unsigned>(e < 0 ? -e : e);
const bool three = ea >= 100;
end[0] = 'e';
end[1] = e < 0 ? '-' : '+';
end[2] = static_cast<char>('0' + (three ? ea / 100 : (ea / 10) % 10));
end[3] = static_cast<char>('0' + (three ? (ea / 10) % 10 : ea % 10));
end[4] = static_cast<char>('0' + (ea % 10));
return end + (three ? 5 : 4);
}
/// a positive finite float (other than double): Grisu2 and format_buffer()
template<typename FloatType>
JSON_HEDLEY_NON_NULL(1, 2)
JSON_HEDLEY_RETURNS_NON_NULL
char* write_positive(char* first, const char* last, FloatType value)
{
JSON_ASSERT(last - first >= std::numeric_limits<FloatType>::max_digits10);
// Compute v = buffer * 10^decimal_exponent.
// The decimal digits are stored in the buffer, which needs to be interpreted
// as an unsigned decimal integer.
// len is the length of the buffer, i.e., the number of decimal digits.
int len = 0;
int decimal_exponent = 0;
shortest_digits(first, len, decimal_exponent, value);
JSON_ASSERT(len <= std::numeric_limits<FloatType>::max_digits10);
// Format the buffer like printf("%.*g", prec, value)
constexpr int kMinExp = -4;
// Use digits10 here to increase compatibility with version 2.
constexpr int kMaxExp = std::numeric_limits<FloatType>::digits10;
JSON_ASSERT(last - first >= kMaxExp + 2);
JSON_ASSERT(last - first >= 2 + (-kMinExp - 1) + std::numeric_limits<FloatType>::max_digits10);
JSON_ASSERT(last - first >= std::numeric_limits<FloatType>::max_digits10 + 6);
return format_buffer(first, len, decimal_exponent, kMinExp, kMaxExp);
}
/// a positive finite double: the shortest digits (Zmij), laid out by
/// write_decimal() (through a local buffer if [first, last) is shorter than
/// the 41 bytes it may write)
JSON_HEDLEY_NON_NULL(1, 2)
JSON_HEDLEY_RETURNS_NON_NULL
inline char* write_positive(char* first, const char* last, double value)
{
static_assert(std::numeric_limits<double>::is_iec559 && std::numeric_limits<double>::digits == 53,
"internal error: the conversion of Zmij needs IEEE 754 binary64 doubles");
std::uint64_t bits = 0;
std::memcpy(&bits, &value, sizeof(bits));
const zmij::decimal d = zmij::to_decimal(bits);
if (JSON_HEDLEY_LIKELY(last - first >= 41))
{
return write_decimal(first, d.significand, d.exponent);
}
std::array<char, 64> buf; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
const auto len = static_cast<std::size_t>(write_decimal(buf.data(), d.significand, d.exponent) - buf.data());
JSON_ASSERT(static_cast<std::size_t>(last - first) >= len);
std::memcpy(first, buf.data(), len);
return first + len;
}
} // namespace dtoa_impl
/*!
@@ -24972,7 +25453,6 @@ JSON_HEDLEY_NON_NULL(1, 2)
JSON_HEDLEY_RETURNS_NON_NULL
char* to_chars(char* first, const char* last, FloatType value)
{
static_cast<void>(last); // maybe unused - fix warning
JSON_ASSERT(std::isfinite(value));
// Use signbit(value) instead of (value < 0) since signbit works for -0.
@@ -24998,28 +25478,7 @@ char* to_chars(char* first, const char* last, FloatType value)
JSON_HEDLEY_DIAGNOSTIC_POP
#endif
JSON_ASSERT(last - first >= std::numeric_limits<FloatType>::max_digits10);
// Compute v = buffer * 10^decimal_exponent.
// The decimal digits are stored in the buffer, which needs to be interpreted
// as an unsigned decimal integer.
// len is the length of the buffer, i.e., the number of decimal digits.
int len = 0;
int decimal_exponent = 0;
dtoa_impl::grisu2(first, len, decimal_exponent, value);
JSON_ASSERT(len <= std::numeric_limits<FloatType>::max_digits10);
// Format the buffer like printf("%.*g", prec, value)
constexpr int kMinExp = -4;
// Use digits10 here to increase compatibility with version 2.
constexpr int kMaxExp = std::numeric_limits<FloatType>::digits10;
JSON_ASSERT(last - first >= kMaxExp + 2);
JSON_ASSERT(last - first >= 2 + (-kMinExp - 1) + std::numeric_limits<FloatType>::max_digits10);
JSON_ASSERT(last - first >= std::numeric_limits<FloatType>::max_digits10 + 6);
return dtoa_impl::format_buffer(first, len, decimal_exponent, kMinExp, kMaxExp);
return dtoa_impl::write_positive(first, last, value);
}
} // namespace detail
File diff suppressed because it is too large Load Diff
+4 -1
View File
@@ -10,7 +10,7 @@ CXXFLAGS += -std=c++11
CPPFLAGS += -I ../single_include
FUZZER_ENGINE = src/fuzzer-driver_afl.cpp
FUZZERS = parse_afl_fuzzer parse_bson_fuzzer parse_cbor_fuzzer parse_msgpack_fuzzer parse_ubjson_fuzzer parse_bjdata_fuzzer parse_bon8_fuzzer parse_json_view_fuzzer
FUZZERS = parse_afl_fuzzer parse_bson_fuzzer parse_cbor_fuzzer parse_msgpack_fuzzer parse_ubjson_fuzzer parse_bjdata_fuzzer parse_bon8_fuzzer parse_json_view_fuzzer json_view_image_fuzzer
fuzzers: $(FUZZERS)
parse_afl_fuzzer:
@@ -19,6 +19,9 @@ parse_afl_fuzzer:
parse_json_view_fuzzer:
$(CXX) $(CXXFLAGS) $(CPPFLAGS) $(FUZZER_ENGINE) src/fuzzer-parse_json_view.cpp -o $@
json_view_image_fuzzer:
$(CXX) $(CXXFLAGS) $(CPPFLAGS) $(FUZZER_ENGINE) src/fuzzer-json_view_image.cpp -o $@
parse_bson_fuzzer:
$(CXX) $(CXXFLAGS) $(CPPFLAGS) $(FUZZER_ENGINE) src/fuzzer-parse_bson.cpp -o $@
+7 -1
View File
@@ -7,7 +7,7 @@ users ask when they pick a library. They are not built by CMake or run by CI.
## Reproducing the numbers
`compare.py` builds both programs against `include/` of this checkout, runs them, and writes the results together with
`compare.py` builds the programs against `include/` of this checkout, runs them, and writes the results together with
everything needed to reproduce them to `results/<date>-<host>.md` (and `.csv`): the date, the commit, the CPU, the
OS, the compiler, the flags, and the versions of all libraries.
@@ -52,6 +52,12 @@ JSON-RPC request (`rpc`):
`bench_corpus.cpp` runs parse, traverse, and dump on any list of files, so that no library is tuned to a handful of
documents.
`bench_edit.cpp` measures read-modify-write: parse, apply the same logical edits with each library's own API, and
serialize (compact). Workloads: `patch` (a handful of edits at fixed places) and `update` (edits in every record).
An editable `json_document` edits in place; yyjson copies its immutable document into a mutable one first
(`yyjson_doc_mut_copy`); Boost.JSON and `json::parse` build mutable DOMs; simdjson cannot edit a document. All
outputs are checked to describe the same value.
Before anything is timed, all engines must accept each document and agree on the traversal: the number of values, the
bytes of all strings and keys, and the sum of all numbers. All engines run interleaved in every round, and the best
round is reported, as time and as a factor of the `json_view` time (below 1 means faster than `json_view`).
+575
View File
@@ -0,0 +1,575 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
// Read-modify-write benchmark: parse a document, apply the same logical edits
// with each library's own API, and serialize it (compact).
//
// json_view json_editable_document: edits in place, unchanged values stay in the index
// yyjson yyjson_read + yyjson_doc_mut_copy (the way to edit a parsed document)
// Boost.JSON parse into a mutable DOM (monotonic resource, precise numbers), serialize
// json::parse nlohmann::json today
// simdjson has no mutable document and is not part of this comparison.
//
// Workloads:
// patch a handful of edits at fixed places (scalars, a new member, a new array element)
// update edits in every record (twitter: 100 statuses, citm: 243 performances,
// canada: 480 rings, jeopardy: 216,930 questions): set scalars, erase a
// member, add a member (canada: replace the first point of every ring)
//
// Build: see README.md (same flags as bench_view.cpp).
#include <nlohmann/json_view.hpp>
#if JSON_VIEW_BENCH_BOOST
#include <boost/json.hpp>
#include <boost/json/src.hpp>
#endif
#include <yyjson.h>
#include <algorithm>
#include <chrono>
#include <cstdio>
#include <cstdlib>
#include <fstream>
#include <functional>
#include <sstream>
using nlohmann::json;
using nlohmann::json_editable_document;
using nlohmann::json_editable_view;
#if JSON_VIEW_BENCH_BOOST
namespace bj = boost::json;
#endif
static volatile std::size_t g_sink;
// ---------------- json_view ----------------
static std::string edit_view(const std::string& name, const std::string& s, bool update)
{
json_editable_document d = json_editable_document::parse(s);
const json_editable_view r = d.root();
if (name == "twitter")
{
if (update)
{
std::int64_t i = 0;
for (const json_editable_view st : r["statuses"])
{
d.set(st, "retweet_count", i++);
d.set(st, "favorited", true);
d.set(st, "text", "redacted");
d.erase(st, "entities");
d.set(st, "edited", true);
}
}
else
{
d.set(r["search_metadata"], "count", 200);
d.set(r["statuses"][0], "text", "patched");
d.set(r["statuses"][0]["user"], "followers_count", 1);
d.set(r["statuses"][99], "favorited", true);
d.set(r, "patched", true);
}
}
else if (name == "citm_catalog")
{
if (update)
{
for (const json_editable_view p : r["performances"])
{
d.set(p, "name", "performance");
d.set(p, "start", p["start"].get<std::int64_t>() + 1);
d.erase(p, "seatMapImage");
d.set(p, "edited", true);
}
}
else
{
d.set(r["events"]["138586341"], "name", "patched");
d.set(r["performances"][0], "start", 0);
d.set(r["venueNames"], "PLEYEL_PLEYEL", "Salle");
d.set(r, "patched", true);
}
}
else if (name == "canada")
{
const json_editable_view coords = r["features"][0]["geometry"]["coordinates"];
if (update)
{
for (const json_editable_view ring : coords)
{
d.set(ring, 0, json::array({0.5, 0.5}));
}
}
else
{
d.set(r["features"][0]["properties"], "name", "patched");
d.set(r, "type", "FeatureCollection2");
d.set(coords[0], 0, json::array({0.0, 0.0}));
}
}
else if (name == "jeopardy")
{
if (update)
{
for (const json_editable_view q : r)
{
d.set(q, "value", "$1");
d.erase(q, "air_date");
}
}
else
{
d.set(r[0], "value", "$0");
d.set(r[100000], "answer", "patched");
d.set(r[216929], "round", "x");
d.push_back(r, json::object({{"category", "NEW"}, {"value", "$5"}}));
}
}
else if (name == "status")
{
d.set(r, "retweet_count", 1);
d.set(r["user"], "name", "x");
}
else if (name == "rpc")
{
d.set(r, "id", 4);
d.set(r["params"], "subtrahend", 24);
}
return r.dump();
}
// ---------------- nlohmann::json ----------------
static std::string edit_json(const std::string& name, const std::string& s, bool update)
{
json r = json::parse(s);
if (name == "twitter")
{
if (update)
{
std::int64_t i = 0;
for (auto& st : r["statuses"])
{
st["retweet_count"] = i++;
st["favorited"] = true;
st["text"] = "redacted";
st.erase("entities");
st["edited"] = true;
}
}
else
{
r["search_metadata"]["count"] = 200;
r["statuses"][0]["text"] = "patched";
r["statuses"][0]["user"]["followers_count"] = 1;
r["statuses"][99]["favorited"] = true;
r["patched"] = true;
}
}
else if (name == "citm_catalog")
{
if (update)
{
for (auto& p : r["performances"])
{
p["name"] = "performance";
p["start"] = p["start"].get<std::int64_t>() + 1;
p.erase("seatMapImage");
p["edited"] = true;
}
}
else
{
r["events"]["138586341"]["name"] = "patched";
r["performances"][0]["start"] = 0;
r["venueNames"]["PLEYEL_PLEYEL"] = "Salle";
r["patched"] = true;
}
}
else if (name == "canada")
{
json& coords = r["features"][0]["geometry"]["coordinates"];
if (update)
{
for (auto& ring : coords)
{
ring[0] = json::array({0.5, 0.5});
}
}
else
{
r["features"][0]["properties"]["name"] = "patched";
r["type"] = "FeatureCollection2";
coords[0][0] = json::array({0.0, 0.0});
}
}
else if (name == "jeopardy")
{
if (update)
{
for (auto& q : r)
{
q["value"] = "$1";
q.erase("air_date");
}
}
else
{
r[0]["value"] = "$0";
r[100000]["answer"] = "patched";
r[216929]["round"] = "x";
r.push_back(json::object({{"category", "NEW"}, {"value", "$5"}}));
}
}
else if (name == "status")
{
r["retweet_count"] = 1;
r["user"]["name"] = "x";
}
else if (name == "rpc")
{
r["id"] = 4;
r["params"]["subtrahend"] = 24;
}
return r.dump();
}
// ---------------- yyjson ----------------
static std::string edit_yyjson(const std::string& name, const std::string& s, bool update)
{
yyjson_doc* idoc = yyjson_read(s.data(), s.size(), 0);
yyjson_mut_doc* d = yyjson_doc_mut_copy(idoc, nullptr);
yyjson_doc_free(idoc);
yyjson_mut_val* r = yyjson_mut_doc_get_root(d);
auto get = [](yyjson_mut_val * o, const char* k)
{
return yyjson_mut_obj_get(o, k);
};
if (name == "twitter")
{
yyjson_mut_val* sts = get(r, "statuses");
if (update)
{
std::size_t idx, max;
yyjson_mut_val* st;
std::int64_t i = 0;
yyjson_mut_arr_foreach(sts, idx, max, st)
{
yyjson_mut_set_sint(get(st, "retweet_count"), i++);
yyjson_mut_set_bool(get(st, "favorited"), true);
yyjson_mut_set_str(get(st, "text"), "redacted");
yyjson_mut_obj_remove_key(st, "entities");
yyjson_mut_obj_add_bool(d, st, "edited", true);
}
}
else
{
yyjson_mut_set_sint(get(get(r, "search_metadata"), "count"), 200);
yyjson_mut_val* s0 = yyjson_mut_arr_get(sts, 0);
yyjson_mut_set_str(get(s0, "text"), "patched");
yyjson_mut_set_sint(get(get(s0, "user"), "followers_count"), 1);
yyjson_mut_set_bool(get(yyjson_mut_arr_get(sts, 99), "favorited"), true);
yyjson_mut_obj_add_bool(d, r, "patched", true);
}
}
else if (name == "citm_catalog")
{
if (update)
{
std::size_t idx, max;
yyjson_mut_val* p;
yyjson_mut_arr_foreach(get(r, "performances"), idx, max, p)
{
yyjson_mut_set_str(get(p, "name"), "performance");
yyjson_mut_val* start = get(p, "start");
yyjson_mut_set_sint(start, yyjson_mut_get_sint(start) + 1);
yyjson_mut_obj_remove_key(p, "seatMapImage");
yyjson_mut_obj_add_bool(d, p, "edited", true);
}
}
else
{
yyjson_mut_set_str(get(get(get(r, "events"), "138586341"), "name"), "patched");
yyjson_mut_set_sint(get(yyjson_mut_arr_get(get(r, "performances"), 0), "start"), 0);
yyjson_mut_set_str(get(get(r, "venueNames"), "PLEYEL_PLEYEL"), "Salle");
yyjson_mut_obj_add_bool(d, r, "patched", true);
}
}
else if (name == "canada")
{
yyjson_mut_val* f0 = yyjson_mut_arr_get(get(r, "features"), 0);
yyjson_mut_val* coords = get(get(f0, "geometry"), "coordinates");
if (update)
{
static const double half[2] = {0.5, 0.5};
std::size_t idx, max;
yyjson_mut_val* ring;
yyjson_mut_arr_foreach(coords, idx, max, ring)
{
yyjson_mut_arr_replace(ring, 0, yyjson_mut_arr_with_real(d, half, 2));
}
}
else
{
static const double zero[2] = {0.0, 0.0};
yyjson_mut_set_str(get(get(f0, "properties"), "name"), "patched");
yyjson_mut_set_str(get(r, "type"), "FeatureCollection2");
yyjson_mut_arr_replace(yyjson_mut_arr_get(coords, 0), 0, yyjson_mut_arr_with_real(d, zero, 2));
}
}
else if (name == "jeopardy")
{
if (update)
{
std::size_t idx, max;
yyjson_mut_val* q;
yyjson_mut_arr_foreach(r, idx, max, q)
{
yyjson_mut_set_str(get(q, "value"), "$1");
yyjson_mut_obj_remove_key(q, "air_date");
}
}
else
{
yyjson_mut_set_str(get(yyjson_mut_arr_get(r, 0), "value"), "$0");
yyjson_mut_set_str(get(yyjson_mut_arr_get(r, 100000), "answer"), "patched");
yyjson_mut_set_str(get(yyjson_mut_arr_get(r, 216929), "round"), "x");
yyjson_mut_val* o = yyjson_mut_obj(d);
yyjson_mut_obj_add_str(d, o, "category", "NEW");
yyjson_mut_obj_add_str(d, o, "value", "$5");
yyjson_mut_arr_append(r, o);
}
}
else if (name == "status")
{
yyjson_mut_set_sint(get(r, "retweet_count"), 1);
yyjson_mut_set_str(get(get(r, "user"), "name"), "x");
}
else if (name == "rpc")
{
yyjson_mut_set_sint(get(r, "id"), 4);
yyjson_mut_set_sint(get(get(r, "params"), "subtrahend"), 24);
}
std::size_t n = 0;
char* out = yyjson_mut_write(d, 0, &n);
std::string result(out, n);
std::free(out);
yyjson_mut_doc_free(d);
return result;
}
#if JSON_VIEW_BENCH_BOOST
// ---------------- Boost.JSON ----------------
static std::string edit_boost(const std::string& name, const std::string& s, bool update)
{
bj::monotonic_resource mr;
bj::parse_options opt;
opt.numbers = bj::number_precision::precise; // correctly rounded, like the others
bj::value v = bj::parse(s, &mr, opt);
bj::object* const obj = v.if_object(); // nullptr for jeopardy (an array)
if (name == "twitter")
{
bj::array& sts = (*obj)["statuses"].as_array();
if (update)
{
std::int64_t i = 0;
for (auto& e : sts)
{
bj::object& st = e.as_object();
st["retweet_count"] = i++;
st["favorited"] = true;
st["text"] = "redacted";
st.erase("entities");
st["edited"] = true;
}
}
else
{
(*obj)["search_metadata"].as_object()["count"] = 200;
bj::object& s0 = sts[0].as_object();
s0["text"] = "patched";
s0["user"].as_object()["followers_count"] = 1;
sts[99].as_object()["favorited"] = true;
(*obj)["patched"] = true;
}
}
else if (name == "citm_catalog")
{
if (update)
{
for (auto& e : (*obj)["performances"].as_array())
{
bj::object& p = e.as_object();
p["name"] = "performance";
p["start"] = p["start"].as_int64() + 1;
p.erase("seatMapImage");
p["edited"] = true;
}
}
else
{
(*obj)["events"].as_object()["138586341"].as_object()["name"] = "patched";
(*obj)["performances"].as_array()[0].as_object()["start"] = 0;
(*obj)["venueNames"].as_object()["PLEYEL_PLEYEL"] = "Salle";
(*obj)["patched"] = true;
}
}
else if (name == "canada")
{
bj::object& f0 = (*obj)["features"].as_array()[0].as_object();
bj::array& coords = f0["geometry"].as_object()["coordinates"].as_array();
if (update)
{
for (auto& ring : coords)
{
ring.as_array()[0] = bj::array({0.5, 0.5});
}
}
else
{
f0["properties"].as_object()["name"] = "patched";
(*obj)["type"] = "FeatureCollection2";
coords[0].as_array()[0] = bj::array({0.0, 0.0});
}
}
else if (name == "jeopardy")
{
bj::array& a = v.as_array();
if (update)
{
for (auto& e : a)
{
bj::object& q = e.as_object();
q["value"] = "$1";
q.erase("air_date");
}
}
else
{
a[0].as_object()["value"] = "$0";
a[100000].as_object()["answer"] = "patched";
a[216929].as_object()["round"] = "x";
a.push_back(bj::object({{"category", "NEW"}, {"value", "$5"}}));
}
}
else if (name == "status")
{
(*obj)["retweet_count"] = 1;
(*obj)["user"].as_object()["name"] = "x";
}
else if (name == "rpc")
{
(*obj)["id"] = 4;
(*obj)["params"].as_object()["subtrahend"] = 24;
}
return bj::serialize(v);
}
#endif
// ---------------- harness ----------------
static std::string slurp(const std::string& p)
{
std::ifstream f(p, std::ios::binary);
std::stringstream ss;
ss << f.rdbuf();
return ss.str();
}
int main(int argc, char** argv)
{
if (argc < 2)
{
std::fprintf(stderr, "usage: %s <json_test_data directory> [rounds] [document]\n", argv[0]);
return 1;
}
const std::string T = std::string(argv[1]) + "/";
const int rounds = argc > 2 ? std::atoi(argv[2]) : 20;
const std::string only = argc > 3 ? argv[3] : "";
struct doc
{
std::string name, text;
int batch;
};
std::vector<doc> docs;
for (const char* f :
{"nativejson-benchmark/twitter.json", "nativejson-benchmark/citm_catalog.json", "nativejson-benchmark/canada.json", "jeopardy/jeopardy.json"
})
{
std::string n = std::string(f).substr(std::string(f).find('/') + 1);
docs.push_back({n.substr(0, n.size() - 5), slurp(T + f), 1});
}
docs.push_back({"status", json::parse(docs[0].text)["statuses"][0].dump(), 200});
docs.push_back({"rpc", R"({"jsonrpc": "2.0", "method": "subtract", "params": {"minuend": 42, "subtrahend": 23}, "id": 3})", 5000});
using fn = std::string (*)(const std::string&, const std::string&, bool);
const std::vector<std::pair<std::string, fn>> engines =
{
{"json_view", edit_view}, {"yyjson", edit_yyjson},
#if JSON_VIEW_BENCH_BOOST
{"Boost.JSON", edit_boost},
#endif
{"json::parse", edit_json}
};
std::FILE* csv = std::fopen("bench_edit.csv", "w");
std::fprintf(csv, "doc,bytes,workload,engine,ns\n");
for (const auto& dc : docs)
{
if (!only.empty() && dc.name != only)
{
continue;
}
for (const bool update :
{
false, true
})
{
if (update && (dc.name == "status" || dc.name == "rpc"))
{
continue;
}
// all engines must produce the same value
const json expected = json::parse(edit_json(dc.name, dc.text, update));
bool ok = true;
for (const auto& e : engines)
{
ok = ok && json::parse(e.second(dc.name, dc.text, update)) == expected;
}
std::vector<double> best(engines.size(), 1e300);
const int r = dc.text.size() > 10000000 ? std::max(3, rounds / 4) : rounds;
for (int i = 0; i < r; ++i)
{
for (std::size_t k = 0; k < engines.size(); ++k)
{
const auto t0 = std::chrono::steady_clock::now();
for (int b = 0; b < dc.batch; ++b)
{
g_sink = engines[k].second(dc.name, dc.text, update).size();
}
const double ns = std::chrono::duration<double, std::nano>(std::chrono::steady_clock::now() - t0).count() / dc.batch;
best[k] = std::min(best[k], ns);
}
}
const char* wl = update ? "update" : "patch";
std::printf("%-13s %-7s %s", dc.name.c_str(), wl, ok ? "" : "[OUTPUT MISMATCH] ");
for (std::size_t k = 0; k < engines.size(); ++k)
{
const double us = best[k] / 1e3;
std::printf(" %s %.*fus (%.2fx)", engines[k].first.c_str(), us < 10 ? 3 : (us < 1000 ? 1 : 0), us, best[k] / best[0]);
std::fprintf(csv, "%s,%zu,%s,%s,%.1f\n", dc.name.c_str(), dc.text.size(), wl, engines[k].first.c_str(), best[k]);
}
std::printf("\n");
std::fflush(stdout);
}
}
std::fclose(csv);
}
+5 -3
View File
@@ -9,7 +9,7 @@
"""Compare json_view with yyjson, simdjson, Boost.JSON, and json::parse.
Builds bench_view.cpp and bench_corpus.cpp against the include/ directory of
Builds bench_view.cpp, bench_corpus.cpp, and bench_edit.cpp against the include/ directory of
this checkout, runs them, and writes the results with everything needed to
reproduce them (date, commit, CPU, OS, compiler, library versions, flags) to
results/<date>-<host>.md and .csv next to this script.
@@ -245,7 +245,7 @@ def main():
objects.append(obj)
binaries = {}
for bench in ['bench_view', 'bench_corpus']:
for bench in ['bench_view', 'bench_corpus', 'bench_edit']:
exe = os.path.join(args.build_dir, bench)
run([cxx] + flags + include + [os.path.join(HERE, bench + '.cpp')] + objects + link + ['-o', exe])
binaries[bench] = exe
@@ -257,6 +257,8 @@ def main():
capture_output=True, text=True).stdout
outputs['bench_corpus'] = run([binaries['bench_corpus']] + corpus, cwd=args.build_dir,
capture_output=True, text=True).stdout
outputs['bench_edit'] = run([binaries['bench_edit'], args.data, str(max(1, args.rounds // 2))], cwd=args.build_dir,
capture_output=True, text=True).stdout
for name, text in outputs.items():
print(text)
@@ -288,7 +290,7 @@ def main():
f.write(f'\n## {name}\n\n```\n{text.rstrip()}\n```\n')
with open(stem + '.csv', 'w', encoding='utf-8') as out:
out.write(''.join(f'# {key}: {value}\n' for key, value in meta))
for name in ['bench_view', 'bench_corpus']:
for name in ['bench_view', 'bench_corpus', 'bench_edit']:
path = os.path.join(args.build_dir, name + '.csv')
if os.path.isfile(path):
with open(path, encoding='utf-8') as f:
+6
View File
@@ -10,6 +10,12 @@ produces, and that a rejected input makes both parsers throw with an identical `
reuses the `corpus_json` corpus (or, for the `make fuzz_testing_json_view` target below, `tests/data/json_tests`) rather
than a format of its own.
`json_view_image_fuzzer` (`tests/src/fuzzer-json_view_image.cpp`) tests the images of `json_document` (`save()` and
`load()`). It uses each input twice: as an image, which `load()` must either reject with `parse_error.116` or read
safely (with `image_check::full`, the document must also serialize to the JSON it reads as), and as a JSON text, whose
image must load and serialize to the same text. A corpus of images can be made from JSON files with a small program
that calls `json_document::parse(text).save()`; plain JSON files work as well.
## Corpus creation
For most effective fuzzing, a [corpus](https://llvm.org/docs/LibFuzzer.html#corpus) should be provided. A corpus is a
+89
View File
@@ -0,0 +1,89 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
/*
This file implements a test of json_document images suitable for fuzz
testing. The input is used twice:
- as an image: json_document::load() with image_check::full must either throw
a parse_error or yield a document that serializes to the JSON text it reads
as; with image_check::bounds, reading and serializing must be safe (checked
by the sanitizers), and serializing may only throw type_error.316
- as a JSON text: if json_document::parse() accepts it, the image of the
document must load (with every check) and serialize to the same text
The provided function `LLVMFuzzerTestOneInput` can be used in different fuzzer
drivers.
*/
#include <cassert>
#include <cstdint>
#include <string>
#include <vector>
#include <nlohmann/json.hpp>
#include <nlohmann/json_view.hpp>
// the checks below are assertions; NDEBUG would compile them away
#ifdef NDEBUG
#error "the fuzzer drivers must be built without NDEBUG"
#endif
using json = nlohmann::json;
using json_document = nlohmann::json_document;
using image_check = json_document::image_check;
// see http://llvm.org/docs/LibFuzzer.html
extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
{
// the input as an image
for (const image_check check : {image_check::full, image_check::bounds})
{
json_document d;
try
{
d = json_document::load(data, size, check);
}
catch (const json::parse_error& e)
{
assert(e.id == 116);
continue;
}
std::string dumped;
try
{
dumped = d.root().dump();
}
catch (const json::type_error& e)
{
// invalid UTF-8 can only pass the bounds check
assert(check == image_check::bounds && e.id == 316);
continue;
}
const json j = d.root().materialize();
if (check == image_check::full)
{
assert(json::parse(dumped) == j);
// an image of the loaded document is the input
assert(d.save() == std::vector<std::uint8_t>(data, data + size));
}
}
// the input as a JSON text
const std::string text(reinterpret_cast<const char*>(data), size); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
const json_document parsed = json_document::parse(text, false);
if (!parsed.is_discarded())
{
const std::vector<std::uint8_t> image = parsed.save();
for (const image_check check : {image_check::full, image_check::bounds, image_check::none})
{
const json_document loaded = json_document::load(image, check);
assert(loaded.root().dump() == parsed.root().dump());
}
}
return 0;
}
+11 -11
View File
@@ -33,7 +33,7 @@ TEST_CASE("Binary Formats" * doctest::skip())
const auto ubjson_2_size = json::to_ubjson(j, true).size();
const auto ubjson_3_size = json::to_ubjson(j, true, true).size();
CHECK(json_size == 2090303);
CHECK(json_size == 2090234);
CHECK(bjdata_1_size == 1112030);
CHECK(bjdata_2_size == 1224148);
CHECK(bjdata_3_size == 1224148);
@@ -46,16 +46,16 @@ TEST_CASE("Binary Formats" * doctest::skip())
CHECK(ubjson_3_size == 1169069);
CHECK((100.0 * double(json_size) / double(json_size)) == Approx(100.0));
CHECK((100.0 * double(bjdata_1_size) / double(json_size)) == Approx(53.199));
CHECK((100.0 * double(bjdata_2_size) / double(json_size)) == Approx(58.563));
CHECK((100.0 * double(bjdata_3_size) / double(json_size)) == Approx(58.563));
CHECK((100.0 * double(bon8_size) / double(json_size)) == Approx(50.509));
CHECK((100.0 * double(bson_size) / double(json_size)) == Approx(85.849));
CHECK((100.0 * double(cbor_size) / double(json_size)) == Approx(50.497));
CHECK((100.0 * double(msgpack_size) / double(json_size)) == Approx(50.526));
CHECK((100.0 * double(ubjson_1_size) / double(json_size)) == Approx(53.199));
CHECK((100.0 * double(ubjson_2_size) / double(json_size)) == Approx(58.563));
CHECK((100.0 * double(ubjson_3_size) / double(json_size)) == Approx(55.928));
CHECK((100.0 * double(bjdata_1_size) / double(json_size)) == Approx(53.201));
CHECK((100.0 * double(bjdata_2_size) / double(json_size)) == Approx(58.565));
CHECK((100.0 * double(bjdata_3_size) / double(json_size)) == Approx(58.565));
CHECK((100.0 * double(bon8_size) / double(json_size)) == Approx(50.511));
CHECK((100.0 * double(bson_size) / double(json_size)) == Approx(85.853));
CHECK((100.0 * double(cbor_size) / double(json_size)) == Approx(50.499));
CHECK((100.0 * double(msgpack_size) / double(json_size)) == Approx(50.528));
CHECK((100.0 * double(ubjson_1_size) / double(json_size)) == Approx(53.201));
CHECK((100.0 * double(ubjson_2_size) / double(json_size)) == Approx(58.565));
CHECK((100.0 * double(ubjson_3_size) / double(json_size)) == Approx(55.930));
}
SECTION("twitter.json")
+3
View File
@@ -1089,6 +1089,9 @@ TEST_CASE("json_view dump")
CHECK(d.root().dump() == json::parse(text).dump());
CHECK(d.root().dump() == "[1.5,100.0,0,-0.0,1.2345678901234568e+29,18446744073709551615,-9223372036854775808,0.1,1e-07,5e-324]");
CHECK(d.root().dump(-1, ' ', false, json_view::number_format::source) == "[1.50,1E2,-0,-0.0,123456789012345678901234567890,18446744073709551615,-9223372036854775808,0.1,1e-7,5e-324]");
// also indented, and with ensure_ascii
CHECK(d.root().dump(0, ' ', false, json_view::number_format::source) == "[\n1.50,\n1E2,\n-0,\n-0.0,\n123456789012345678901234567890,\n18446744073709551615,\n-9223372036854775808,\n0.1,\n1e-7,\n5e-324\n]");
CHECK(d.root().dump(-1, ' ', true, json_view::number_format::source) == "[1.50,1E2,-0,-0.0,123456789012345678901234567890,18446744073709551615,-9223372036854775808,0.1,1e-7,5e-324]");
// random doubles, written as parse() and dump() would
std::mt19937_64 rng(1170); // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed)
+558
View File
@@ -0,0 +1,558 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#include "doctest_compatibility.h"
#include <nlohmann/json_view.hpp>
using nlohmann::json;
using nlohmann::ordered_json;
using nlohmann::json_document;
using nlohmann::json_editable_document;
using nlohmann::json_editable_view;
using nlohmann::ordered_json_document;
using nlohmann::ordered_json_editable_document;
using nlohmann::ordered_json_editable_view;
using ptr_t = ordered_json::json_pointer;
#include <array>
#include <cmath>
#include <cstring>
#include <cstdint>
#include <functional>
#include <iterator>
#include <limits>
#include <random>
#include <string>
#include <vector>
namespace
{
std::uint32_t rng()
{
static std::mt19937 generator(5295); // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed): reproducible
return generator();
}
int r(int n)
{
return static_cast<int>(rng() % static_cast<unsigned>(n));
}
int counter = 0;
std::uint64_t bits(double x)
{
std::uint64_t b = 0;
std::memcpy(&b, &x, sizeof(b));
return b;
}
std::string random_string()
{
static const std::array<const char*, 15> pieces = {{"a", "Z", " ", "~", "\n", "\"", "\\", "/", "\xc3\xa9", "\xe3\x81\x82", "\xf0\x9f\x98\x80", "\x7f", "\x1f", "0", "key"}};
std::string s;
for (int i = r(3) == 0 ? r(30) : r(6); i > 0; --i)
{
s += pieces[static_cast<std::size_t>(r(15))];
}
return s;
}
ordered_json random_scalar()
{
switch (r(9))
{
case 0:
return nullptr;
case 1:
return r(2) == 0;
case 2:
return static_cast<std::int64_t>(rng()) - 2147483648LL;
case 3:
return (static_cast<std::uint64_t>(rng()) * 4294967296ULL) + rng();
case 4:
return static_cast<double>(static_cast<std::int32_t>(rng())) / (1 + r(1000));
case 5:
return r(2) == 0 ? 1e300 * (r(2) == 0 ? 1 : -1) : 5e-324;
case 6:
return -0.0;
default:
return random_string();
}
}
ordered_json random_value(int depth)
{
const int k = depth > 3 ? 4 : r(7);
if (k == 0)
{
ordered_json o = ordered_json::object();
for (int i = r(5); i > 0; --i)
{
o[random_string() + "#" + std::to_string(counter++)] = random_value(depth + 1);
}
return o;
}
if (k == 1)
{
ordered_json a = ordered_json::array();
for (int i = r(5); i > 0; --i)
{
a.push_back(random_value(depth + 1));
}
return a;
}
return random_scalar();
}
void collect(const ordered_json& j, const ptr_t& p, std::vector<ptr_t>& out)
{
out.push_back(p);
if (j.is_object())
{
for (const auto& kv : j.items())
{
collect(kv.value(), p / kv.key(), out);
}
}
else if (j.is_array())
{
for (std::size_t i = 0; i < j.size(); ++i)
{
collect(j[i], p / i, out);
}
}
}
// the edited view and the ordered_json value read the same through the whole
// read API
void compare(const ordered_json_editable_view& v, const ordered_json& j)
{
REQUIRE(v.type() == j.type());
REQUIRE(v.size() == j.size());
if (j.is_object())
{
auto jt = j.begin();
for (auto it = v.begin(); it != v.end(); ++it, ++jt)
{
CHECK(std::string(it.key().data(), it.key().size()) == jt.key());
CHECK(v[jt.key()].dump() == jt.value().dump());
CHECK(v.contains(jt.key()));
compare(*it, jt.value());
}
}
else if (j.is_array())
{
for (std::size_t i = 0; i < j.size(); ++i)
{
CHECK(v[i].dump() == j[i].dump());
}
std::size_t i = 0;
for (const auto e : v)
{
compare(e, j[i++]);
}
if (!j.empty())
{
CHECK(v.back().dump() == j.back().dump());
}
}
else if (j.is_string())
{
CHECK(v.get<std::string>() == j.get<std::string>());
}
else if (j.is_number_float())
{
CHECK(bits(v.get<double>()) == bits(j.get<double>()));
}
else if (j.is_number_integer())
{
CHECK(v.get<std::int64_t>() == j.get<std::int64_t>());
CHECK(v.get<std::uint64_t>() == j.get<std::uint64_t>());
}
}
void check_all(const ordered_json_editable_document& d, const ordered_json& j, bool deep)
{
const std::string text = d.root().dump();
REQUIRE(text == j.dump());
CHECK(d.root().materialize() == j);
if (deep)
{
CHECK(d.root().dump(2) == j.dump(2));
CHECK(d.root().dump(-1, ' ', true) == j.dump(-1, ' ', true));
CHECK(d.root() == j);
compare(d.root(), j);
// a fresh document of the text reads the same
const ordered_json_document fresh = ordered_json_document::parse(text);
CHECK(fresh.root() == d.root());
}
}
} // namespace
TEST_CASE("json_view edits: differential")
{
// random edits are applied to an ordered_json_editable_document and to the
// ordered_json parse() produces; after every edit both must serialize,
// materialize, and read back the same
for (int n = 0; n < 150; ++n)
{
ordered_json j = random_value(0);
if (r(4) == 0)
{
j = ordered_json::object({{"a", random_value(1)}, {"b", random_value(1)}});
}
const std::string text = j.dump(r(2) == 0 ? -1 : 2);
CAPTURE(text);
ordered_json_editable_document d = ordered_json_editable_document::parse(text);
j = ordered_json::parse(text);
const ordered_json_document other = ordered_json_document::parse(random_value(0).dump());
const int edits = 30;
for (int e = 0; e < edits; ++e)
{
std::vector<ptr_t> paths;
collect(j, ptr_t(), paths);
const ptr_t p = paths[static_cast<std::size_t>(r(static_cast<int>(paths.size())))];
const ordered_json& target = j[p];
const ordered_json_editable_view tv = d.root().at(p);
const int op = r(12);
{
if (op == 0) // assign a scalar
{
const ordered_json v = random_scalar();
if (v.is_string() && r(2) == 0)
{
d.set(tv, v.get<std::string>());
}
else if (v.is_number_unsigned() && r(2) == 0)
{
d.set(tv, v.get<std::uint64_t>());
}
else
{
d.set(tv, v);
}
j[p] = v;
}
else if (op == 1) // assign a new array/object (or anything), sometimes via the pointer API
{
const ordered_json v = random_value(2);
if (r(2) == 0)
{
d.set(p, v);
}
else
{
d.set(tv, v);
}
j[p] = v;
}
else if (op == 2) // copy a value of the same document
{
const ptr_t& q = paths[static_cast<std::size_t>(r(static_cast<int>(paths.size())))];
const ordered_json v = j[q];
d.set(tv, d.root().at(q));
j[p] = v;
}
else if (op == 3) // copy a value of another document
{
d.set(tv, other.root());
j[p] = other.root().materialize();
}
else if ((op == 4 || op == 5) && target.is_object()) // set a member (new or existing)
{
std::string key = random_string() + "#" + std::to_string(counter++);
if (op == 5 && !target.empty())
{
key = std::next(target.begin(), r(static_cast<int>(target.size()))).key();
}
const ordered_json v = random_value(2);
d.set(tv, key, v);
j[p][key] = v;
}
else if (op == 6 && target.is_object() && !target.empty()) // erase a member
{
const std::string key = std::next(target.begin(), r(static_cast<int>(target.size()))).key();
if (r(2) == 0)
{
d.erase(tv, key);
}
else
{
d.erase(p / key);
}
j[p].erase(key);
}
else if (op == 7 && (target.is_array() || target.is_null())) // push_back
{
const ordered_json v = random_value(2);
d.push_back(tv, v);
j[p].push_back(v);
}
else if (op == 8 && target.is_array()) // insert
{
const auto i = static_cast<std::size_t>(r(static_cast<int>(target.size()) + 1));
const ordered_json v = random_value(2);
d.insert(tv, i, v);
j[p].insert(j[p].begin() + static_cast<std::ptrdiff_t>(i), v);
}
else if (op == 9 && target.is_array() && !target.empty()) // erase an element
{
const auto i = static_cast<std::size_t>(r(static_cast<int>(target.size())));
if (r(2) == 0)
{
d.erase(tv, i);
}
else
{
d.erase(p / i);
}
j[p].erase(i);
}
else if (op == 10 && target.is_array() && !target.empty()) // assign an element
{
const auto i = static_cast<std::size_t>(r(static_cast<int>(target.size())));
const ordered_json v = random_value(2);
d.set(tv, i, v);
j[p][i] = v;
}
else if (op == 11) // a held view sees the assignment
{
const ordered_json v = random_value(2);
const ordered_json_editable_view held = d.root().at(p);
d.set(p, v);
j[p] = v;
CHECK(held.dump() == v.dump());
}
else
{
continue;
}
}
CAPTURE(p.to_string());
CAPTURE(op);
check_all(d, j, e % 8 == 7 || e == edits - 1);
}
}
}
namespace
{
std::string exception_of_call(const std::function<void()>& f)
{
try
{
f();
}
catch (const json::exception& e)
{
return e.what();
}
return "";
}
} // namespace
TEST_CASE("json_view edits: errors")
{
SECTION("an empty document")
{
json_editable_document d;
CHECK_THROWS_WITH_AS(d.set(d.root(), 1), "[json.exception.invalid_iterator.202] view does not belong to this document", json::invalid_iterator&);
}
json_editable_document d = json_editable_document::parse(R"({"o": {"a": 1}, "a": [1, 2], "n": 1, "z": null})");
const json_editable_view root = d.root();
const json_document other = json_document::parse("[1]");
json_editable_document other_editable = json_editable_document::parse("[1]");
CHECK_THROWS_WITH_AS(d.set(other_editable.root(), 1), "[json.exception.invalid_iterator.202] view does not belong to this document", json::invalid_iterator&);
CHECK_THROWS_WITH_AS(d.set(json_editable_view(), 1), "[json.exception.invalid_iterator.202] view does not belong to this document", json::invalid_iterator&);
CHECK_THROWS_WITH_AS(d.set(root["a"], "k", 1), "[json.exception.type_error.305] cannot use operator[] with a string argument with array", json::type_error&);
CHECK_THROWS_WITH_AS(d.set(root["o"], 0, 1), "[json.exception.type_error.305] cannot use operator[] with a numeric argument with object", json::type_error&);
CHECK_THROWS_WITH_AS(d.set(root["a"], 2, 1), "[json.exception.out_of_range.401] array index 2 is out of range", json::out_of_range&);
CHECK_THROWS_WITH_AS(d.set(root["a"], -1, 1), "[json.exception.out_of_range.401] array index -1 is out of range", json::out_of_range&);
CHECK_THROWS_WITH_AS(d.push_back(root["o"], 1), "[json.exception.type_error.308] cannot use push_back() with object", json::type_error&);
CHECK_THROWS_WITH_AS(d.insert(root["n"], 0, 1), "[json.exception.type_error.309] cannot use insert() with number", json::type_error&);
CHECK_THROWS_WITH_AS(d.insert(root["a"], 3, 1), "[json.exception.out_of_range.401] array index 3 is out of range", json::out_of_range&);
CHECK_THROWS_WITH_AS(d.erase(root["n"], "k"), "[json.exception.type_error.307] cannot use erase() with number", json::type_error&);
CHECK_THROWS_WITH_AS(d.erase(root["o"], 0), "[json.exception.type_error.307] cannot use erase() with object", json::type_error&);
CHECK_THROWS_WITH_AS(d.erase(root["a"], 2), "[json.exception.out_of_range.401] array index 2 is out of range", json::out_of_range&);
CHECK_THROWS_WITH_AS(d.erase(json::json_pointer("")), "[json.exception.out_of_range.405] JSON pointer has no parent", json::out_of_range&);
CHECK_THROWS_WITH_AS(d.erase(json::json_pointer("/missing/x")), "[json.exception.out_of_range.403] key 'missing' not found", json::out_of_range&);
CHECK_THROWS_WITH_AS(d.set(json::json_pointer("/a/01"), 1), "[json.exception.parse_error.106] parse error: array index '01' must not begin with '0'", json::parse_error&);
CHECK_THROWS_WITH_AS(d.set(root, json_editable_view()), "[json.exception.type_error.302] type must be a value, but is discarded", json::type_error&);
CHECK_THROWS_WITH_AS(d.set(root, json::binary({1, 2})), "[json.exception.type_error.319] cannot store a binary value in a json_document", json::type_error&);
// invalid UTF-8 is rejected when it enters the document, with the error
// basic_json::dump() reports for the same string
for (const std::string bad :
{"\xC3\x28", "a\xE2\x28\xA1", "\xF0\x28\x8C\xBC", "\xE2\x82", "x\xF0\x9F\x98", "\xFF", "\xED\xA0\x80", "\xC0\xAF"
})
{
CAPTURE(bad);
const std::string expected = exception_of_call([&]
{
const std::string text = json(bad).dump();
static_cast<void>(text);
});
CHECK(!expected.empty());
CHECK(exception_of_call([&] { d.set(root["z"], bad); }) == expected);
CHECK(exception_of_call([&] { d.set(root["o"], bad, 1); }) == expected);
CHECK(exception_of_call([&] { d.set(root["z"], json{{"k", bad}}); }) == expected);
}
// nothing of the failed edits is visible
CHECK(root.dump() == R"({"o":{"a":1},"a":[1,2],"n":1,"z":null})");
static_cast<void>(other);
}
TEST_CASE("json_view edits: views and values")
{
SECTION("a value that is no longer part of the document")
{
json_editable_document d = json_editable_document::parse("[[[1,2]]]");
const json_editable_view inner = d.root()[0][0];
d.set(d.root()[0], json::array({7}));
d.set(inner, 5);
CHECK(d.root().dump() == "[[7]]");
CHECK(inner.get<int>() == 5);
}
SECTION("views keep referring to their value")
{
json_editable_document d = json_editable_document::parse(R"({"a": [10, 20, 30], "b": {"c": "text"}})");
const json_editable_view a = d.root()["a"];
const json_editable_view twenty = a[1];
const json_editable_view c = d.root()["b"]["c"];
d.insert(a, 0, 5);
d.push_back(a, 40);
CHECK(twenty.get<int>() == 20);
CHECK(a[2].get<int>() == 20);
d.erase(a, 2);
CHECK(twenty.get<int>() == 20); // an erased value keeps its last value
d.set(c, 7);
CHECK(c.get<int>() == 7); // a held view sees an assignment
d.set(d.root()["b"], json::array({1, 2}));
CHECK(d.root()["b"].dump() == "[1,2]");
CHECK(d.root().dump() == R"({"a":[5,10,30,40],"b":[1,2]})");
CHECK(d.root()["a"][0].source_offset() == static_cast<std::size_t>(-1)); // a new value
CHECK(d.root()["a"][1].source_offset() != static_cast<std::size_t>(-1));
}
SECTION("strings stay valid while more edits come")
{
json_editable_document d = json_editable_document::parse("[]");
const auto first = d.push_back(d.root(), std::string(100, 'x')).get_string();
for (int i = 0; i < 1000; ++i)
{
d.push_back(d.root(), std::string(static_cast<std::size_t>(i % 50), 'y'));
}
CHECK(std::string(first.data(), first.size()) == std::string(100, 'x'));
CHECK(d.root().size() == 1001);
}
SECTION("numbers")
{
json_editable_document d = json_editable_document::parse(R"([1.50, 1E2, 3])");
d.set(d.root(), 2, 0.1);
d.push_back(d.root(), std::numeric_limits<double>::quiet_NaN());
d.push_back(d.root(), -std::numeric_limits<double>::infinity());
d.push_back(d.root(), static_cast<std::uint64_t>(18446744073709551615u));
d.push_back(d.root(), static_cast<std::int64_t>(-9223372036854775807 - 1));
CHECK(d.root().dump() == "[1.5,100.0,0.1,null,null,18446744073709551615,-9223372036854775808]");
CHECK(d.root().dump(-1, ' ', false, json_editable_view::number_format::source) == "[1.50,1E2,0.1,null,null,18446744073709551615,-9223372036854775808]");
CHECK(std::isnan(d.root()[3].get<double>()));
CHECK(std::isinf(d.root()[4].get<double>()));
CHECK(d.root()[2].number_token() == "0.1");
CHECK(d.root()[5].get<std::uint64_t>() == 18446744073709551615u);
CHECK(d.root()[6].number_token() == "-9223372036854775808");
CHECK(d.root().materialize().dump() == json::parse(R"([1.5, 100.0, 0.1, null, null, 18446744073709551615, -9223372036854775808])").dump());
}
SECTION("nulls become containers, and the root can be replaced")
{
json_editable_document d = json_editable_document::parse("[null, null]");
d.set(d.root()[0], "k", 1);
d.push_back(d.root()[1], true);
CHECK(d.root().dump() == R"([{"k":1},[true]])");
d.set(d.root(), "scalar");
CHECK(d.root().dump() == R"("scalar")");
d.set(d.root(), json{{"x", {1, 2}}});
d.set(json::json_pointer("/x/-"), 3);
d.set(json::json_pointer("/x/3"), 4); // the size of the array appends too
d.set(json::json_pointer("/y"), false);
CHECK(d.root().dump() == R"({"x":[1,2,3,4],"y":false})");
CHECK(d.erase(json::json_pointer("/x/0")) == 1);
CHECK(d.erase(json::json_pointer("/y")) == 1);
CHECK(d.erase(json::json_pointer("/nothing")) == 0);
CHECK(d.root().dump() == R"({"x":[2,3,4]})");
}
SECTION("duplicate keys")
{
json_editable_document d = json_editable_document::parse(R"({"a": 1, "b": 2, "a": 3})");
d.set(d.root(), "a", 4); // the first member is assigned, the others dropped
CHECK(d.root().dump() == R"({"a":4,"b":2})");
d = json_editable_document::parse(R"({"a": 1, "b": 2, "a": 3})");
CHECK(d.erase(d.root(), "a") == 2);
CHECK(d.root().dump() == R"({"b":2})");
}
SECTION("values from other documents")
{
const json_document source = json_document::parse(R"({"list": [1, "two", {"three": 3.5}], "text": "a\nb"})");
json_editable_document edited = json_editable_document::parse("[0]");
edited.set(edited.root(), 0, json{{"inner", {1, 2}}});
json_editable_document d = json_editable_document::parse("{}");
d.set(d.root(), "copy", source.root()["list"]);
d.set(d.root(), "text", source.root()["text"]);
d.set(d.root(), "edited", edited.root()[0]);
d.set(d.root(), "self", d.root()["copy"]);
CHECK(d.root().dump() == R"({"copy":[1,"two",{"three":3.5}],"text":"a\nb","edited":{"inner":[1,2]},"self":[1,"two",{"three":3.5}]})");
CHECK(d.root()["copy"] == source.root()["list"]);
CHECK(source.root()["list"] == d.root()["self"]);
CHECK(d.root() != source.root());
}
SECTION("large objects")
{
std::string text = "{";
for (int i = 0; i < 200; ++i)
{
text += (i != 0 ? ",\"k" : "\"k") + std::to_string(i) + "\":" + std::to_string(i);
}
text += '}';
json_editable_document d = json_editable_document::parse(text);
d.set(d.root(), "k7", "seven"); // assigned in place: the index stays in use
CHECK(d.root()["k7"].get_string() == "seven");
d.set(d.root(), "new", 1); // appended: the members move, the lookup is linear
CHECK(d.root()["new"].get<int>() == 1);
CHECK(d.root()["k199"].get<int>() == 199);
d.erase(d.root(), "k0");
CHECK(!d.root().contains("k0"));
CHECK(d.root().size() == 200);
}
SECTION("reuse and memory")
{
json_editable_document d = json_editable_document::parse("[1, 2, 3]");
const std::size_t before = d.memory_usage();
for (int i = 0; i < 100; ++i)
{
d.push_back(d.root(), "some text");
}
CHECK(d.memory_usage() > before);
const json_editable_view first = d.root()[0];
d.shrink_to_fit(); // (with edits, the index stays in place)
CHECK(first.get<int>() == 1);
d.read(std::string("[true]"));
CHECK(d.root().dump() == "[true]");
d.push_back(d.root(), false);
CHECK(d.root().dump() == "[true,false]");
}
}
+791
View File
@@ -0,0 +1,791 @@
// __ _____ _____ _____
// __| | __| | | | JSON for Modern C++ (supporting code)
// | | |__ | | | | | | version 3.12.0
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
//
// SPDX-FileCopyrightText: 2013-2026 Niels Lohmann <https://nlohmann.me>
// SPDX-License-Identifier: MIT
#include "doctest_compatibility.h"
#include <nlohmann/json_view.hpp>
using nlohmann::json;
using nlohmann::ordered_json;
using nlohmann::json_document;
using nlohmann::json_editable_document;
using nlohmann::ordered_json_document;
using nlohmann::ordered_json_editable_document;
using image_check = json_document::image_check;
using nlohmann::detail::view::node;
#include <array>
#include <cstdint>
#include <cstring>
#include <fstream>
#include <functional>
#include <limits>
#include <random>
#include <sstream>
#include <string>
#include <utility>
#include <vector>
#include <test_data.hpp>
#if !(defined(__BYTE_ORDER__) && defined(__ORDER_BIG_ENDIAN__) && __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__)
namespace
{
std::string exception_of(const std::function<void()>& f)
{
try
{
f();
}
catch (const json::exception& e)
{
return e.what();
}
return "";
}
const char* const check_failed = "[json.exception.parse_error.116] parse error: invalid json_document image: the check failed";
std::string read_file(const std::string& name)
{
std::ifstream f(std::string(TEST_DATA_DIRECTORY) + name, std::ios::binary);
std::stringstream ss;
ss << f.rdbuf();
return ss.str();
}
// the offsets of the parts of an image
constexpr std::size_t header_size = 64;
std::uint64_t header_field(const std::vector<std::uint8_t>& image, std::size_t offset)
{
std::uint64_t v = 0;
std::memcpy(&v, image.data() + offset, sizeof(v));
return v;
}
void set_header_field(std::vector<std::uint8_t>& image, std::size_t offset, std::uint64_t v)
{
std::memcpy(image.data() + offset, &v, sizeof(v));
}
std::size_t node_count(const std::vector<std::uint8_t>& image)
{
return static_cast<std::size_t>(header_field(image, 8));
}
std::size_t text_at(const std::vector<std::uint8_t>& image)
{
return header_size + (node_count(image) * sizeof(node));
}
node node_at(const std::vector<std::uint8_t>& image, std::size_t i)
{
node n{};
std::memcpy(&n, image.data() + header_size + (i * sizeof(node)), sizeof(node));
return n;
}
void set_node(std::vector<std::uint8_t>& image, std::size_t i, const node& n)
{
std::memcpy(image.data() + header_size + (i * sizeof(node)), &n, sizeof(node));
}
/// the result of loading an image with a check: "" or the exception message
std::string load_result(const std::vector<std::uint8_t>& image, image_check check)
{
return exception_of([&]
{
const json_document d = json_document::load(image, check);
static_cast<void>(d);
});
}
/// a copy of the image with node i changed by f
template<typename F>
std::vector<std::uint8_t> corrupted(const std::vector<std::uint8_t>& image, std::size_t i, F f)
{
std::vector<std::uint8_t> b = image;
node n = node_at(b, i);
f(n);
set_node(b, i, n);
return b;
}
/// a document and the documents loaded from its image must be equal
template<typename Document>
void check_round_trip(const Document& d)
{
const std::vector<std::uint8_t> image = d.save();
for (const image_check check :
{
image_check::full, image_check::bounds, image_check::none
})
{
const json_document l = json_document::load(image, check);
CHECK(l.root().dump() == d.root().dump());
CHECK(l.root().dump(2) == d.root().dump(2));
CHECK(l.root().materialize() == json(d.root().materialize()));
// an image of a loaded document is the same image
CHECK(l.save() == image);
}
// an editable document can be loaded, too
const ordered_json_editable_document e = ordered_json_editable_document::load(image);
CHECK(e.root().dump() == d.root().dump());
}
std::uint32_t rng()
{
static std::mt19937 generator(5295); // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed): reproducible
return generator();
}
} // namespace
TEST_CASE("json_view images: round trips")
{
SECTION("small documents")
{
for (const char* text :
{
"null", "true", "false", "0", "-0", "42", "-42", "18446744073709551615", "-9223372036854775808",
"123456789012345678901234567890", "1.5", "-1.25e-300", "1E308", "0.1000000000000000000000000001",
"\"\"", "\"text\"", R"("esc\"aped\n\u00e9\ud83d\ude00")", "\"\xc3\xa9\xe3\x81\x82\"",
"[]", "{}", "[[]]", "[{}]", "{\"\":{}}",
R"({"a": [1, 2.5, "x\ty", true, null, {"b": []}], "c": {"d": -3, "eA": "f"}})",
R"({"k": 1, "k": 2, "l": [], "k": 3})",
" [1 , 2 ] "
})
{
CAPTURE(text);
check_round_trip(json_document::parse(text));
check_round_trip(ordered_json_document::parse(text));
}
}
SECTION("files")
{
for (const char* name :
{
"/json_testsuite/sample.json", "/nativejson-benchmark/canada.json", "/nativejson-benchmark/citm_catalog.json",
"/nativejson-benchmark/twitter.json", "/json_tests/pass1.json", "/json_tests/pass2.json", "/json_tests/pass3.json"
})
{
CAPTURE(name);
const std::string text = read_file(name);
const json_document d = json_document::parse(text);
check_round_trip(d);
// what a loaded document reads is what parse() produces
CHECK(json_document::load(d.save()).root().materialize() == json::parse(text));
}
}
SECTION("images are deterministic")
{
const std::string text = R"({"b": [1, 2, {"c": "\u00e9"}], "a": 1.5})";
const json_document d = json_document::parse(text);
CHECK(d.save() == json_document::parse(text).save());
CHECK(d.save() == json_editable_document::parse(text).save());
CHECK(d.save() == ordered_json_document::parse(text).save());
const json_document copy = json_document::parse_copy(text);
CHECK(copy.save() == d.save());
}
SECTION("large objects get their hash index again")
{
std::string text = "{";
for (int i = 0; i < 1000; ++i)
{
text += (i != 0 ? ",\"k" : "\"k") + std::to_string(i) + "\":" + std::to_string(i);
}
text += R"(,"k7":"a duplicate","inner":{)";
for (int i = 0; i < 200; ++i)
{
text += (i != 0 ? ",\"m" : "\"m") + std::to_string(i) + "\":" + std::to_string(-i);
}
text += "}}";
const json_document d = json_document::parse(text);
const std::vector<std::uint8_t> image = d.save();
for (const image_check check :
{
image_check::full, image_check::none
})
{
const json_document l = json_document::load(image, check);
for (int i = 0; i < 1000; ++i)
{
CHECK(l.root()["k" + std::to_string(i)] == d.root()["k" + std::to_string(i)]);
}
CHECK(l.root()["k7"].get<int>() == 7); // the first of duplicate keys
CHECK(l.root()["inner"]["m199"].get<int>() == -199);
CHECK(!l.root().contains("k1000"));
// the index is not part of the image
CHECK(l.save() == image);
}
// the nodes of objects in the image do not carry the number of an index
CHECK(node_at(image, 0).extra == 0);
}
}
TEST_CASE("json_view images: edited documents")
{
const std::string text = R"({"name": "x", "n": 1, "f": 2.5, "list": [1, 2, 3], "obj": {"a": "\u00e9", "b": [true]}, "s": "a\"b"})";
SECTION("every kind of edit")
{
ordered_json_editable_document d = ordered_json_editable_document::parse(text);
d.set(d.root()["name"], "a new \"name\""); // string in the edit arena
d.set(d.root()["n"], -17); // negative integer
d.set(d.root(), "p", 5); // non-negative number_integer
d.set(d.root(), "u", 18446744073709551615u); // unsigned
d.set(d.root()["f"], 0.1); // float token
d.set(d.root(), "nan", std::numeric_limits<double>::quiet_NaN());
d.set(d.root(), "inf", -std::numeric_limits<double>::infinity());
d.push_back(d.root()["list"], "pushed"); // moved array
d.insert(d.root()["list"], 0, ordered_json::object({{"new", {1, 2}}}));
d.erase(d.root()["list"], 2);
d.erase(d.root(), "s");
d.set(d.root()["obj"], "c", ordered_json::array({1, "two", 3.5, nullptr, false})); // new object member with a new array
d.set(d.root(), "copy", d.root()["obj"]); // a copy of a subtree
d.set(d.root(), "key \xc3\xa9", true); // a key in the edit arena
const std::vector<std::uint8_t> image = d.save();
const ordered_json expected = ordered_json::parse(d.root().dump());
for (const image_check check :
{
image_check::full, image_check::bounds, image_check::none
})
{
const ordered_json_document l = ordered_json_document::load(image, check);
CHECK(l.root().dump() == d.root().dump());
CHECK(l.root().materialize() == expected);
CHECK(l.root()["nan"].is_null());
CHECK(l.root()["inf"].is_null());
CHECK(l.root()["p"].is_number_integer());
CHECK(l.root()["p"].get<int>() == 5);
CHECK(l.root()["f"].get<double>() == 0.1);
CHECK(l.root()["u"].get<std::uint64_t>() == 18446744073709551615u);
}
// the node index is in document order again: an image of the loaded
// document is the same image
CHECK(ordered_json_document::load(image).save() == image);
// number tokens of edits follow the source; the text is the source's
// prefix
const ordered_json_document l = ordered_json_document::load(image);
REQUIRE(l.source().size() > text.size());
CHECK(std::string(l.source().data(), text.size()) == text);
}
SECTION("a loaded document can be edited and saved again")
{
const std::vector<std::uint8_t> first = json_editable_document::parse(text).save();
json_editable_document d = json_editable_document::load(first);
d.set(d.root()["obj"]["a"], "changed");
d.push_back(d.root()["list"], 4);
d.set(d.root(), "z", json::array({json::object()}));
const std::vector<std::uint8_t> second = d.save();
const json_document l = json_document::load(second);
CHECK(l.root().dump() == d.root().dump());
CHECK(l.root()["obj"]["a"] == "changed");
CHECK(l.root()["list"].size() == 4);
}
SECTION("the root replaced")
{
json_editable_document d = json_editable_document::parse(text);
d.set(d.root(), json::array({1, "x"}));
check_round_trip(d);
d.set(d.root(), 3.5);
check_round_trip(d);
d.set(d.root(), "text");
check_round_trip(d);
}
}
TEST_CASE("json_view images: ownership")
{
const std::string text = R"({"a": "esc\u00e9aped", "b": [1, 2]})";
const std::vector<std::uint8_t> image = json_document::parse(text).save();
SECTION("borrowed")
{
const json_document d = json_document::load(image);
CHECK(!d.owns_source());
CHECK(d.root()["a"] == "esc\xc3\xa9" "aped");
const json_document p = json_document::load(image.data(), image.size());
CHECK(!p.owns_source());
CHECK(p.root() == d.root());
// the text is the image's
CHECK(d.source().data() == reinterpret_cast<const char*>(image.data() + text_at(image)));
}
SECTION("owned")
{
std::vector<std::uint8_t> copy = image;
const std::uint8_t* const data = copy.data();
json_document d = json_document::load(std::move(copy));
CHECK(d.owns_source());
CHECK(d.source().data() == reinterpret_cast<const char*>(data + text_at(image)));
CHECK(d.memory_usage() >= image.size());
CHECK(d.root()["b"][1] == 2);
// read() replaces the image
d.read(std::string("[1]"));
CHECK(d.owns_source());
CHECK(d.root().dump() == "[1]");
const std::string borrowed = "[2]";
d.read(borrowed);
CHECK(!d.owns_source());
}
SECTION("shrink_to_fit keeps the decoded strings of the image")
{
json_document d = json_document::parse(R"(["\u00e9\u00e9\u00e9\u00e9\u00e9\u00e9\u00e9\u00e9\u00e9\u00e9\u00e9\u00e9\u00e9\u00e9\u00e9\u00e9"])");
d.shrink_to_fit();
const std::vector<std::uint8_t> img = d.save();
json_document l = json_document::load(img);
l.shrink_to_fit();
CHECK(l.root().dump() == d.root().dump());
CHECK(l.save() == img);
}
}
TEST_CASE("json_view images: errors")
{
SECTION("a literal as the root: dump() after loading")
{
for (const char* text :
{
"null", "true", "false"
})
{
std::vector<std::uint8_t> image = json_document::parse(text).save();
node n = node_at(image, 0);
n.off = static_cast<std::uint32_t>(image.size());
set_node(image, 0, n);
CHECK(load_result(image, image_check::full) == check_failed);
CHECK(load_result(image, image_check::bounds) == check_failed);
}
}
SECTION("saving a discarded document")
{
const json_document empty;
CHECK(exception_of([&] { static_cast<void>(empty.save()); }) == "[json.exception.type_error.320] cannot save a discarded json_document");
const json_document failed = json_document::parse("[1,", false);
CHECK(exception_of([&] { static_cast<void>(failed.save()); }) == "[json.exception.type_error.320] cannot save a discarded json_document");
}
const std::vector<std::uint8_t> image = json_document::parse(R"({"a": [1, "\u00e9"]})").save();
const std::string prefix = "[json.exception.parse_error.116] parse error: invalid json_document image: ";
SECTION("header and sizes")
{
CHECK(exception_of([]
{
const json_document d = json_document::load(nullptr, 0);
static_cast<void>(d);
}) == prefix + "too short");
CHECK(exception_of([&]
{
const json_document d = json_document::load(image.data(), 63);
static_cast<void>(d);
}) == prefix + "too short");
std::vector<std::uint8_t> bad = image;
bad[0] = 'X';
CHECK(load_result(bad, image_check::full) == prefix + "unknown format");
bad = image;
bad[4] = 2; // version
CHECK(load_result(bad, image_check::full) == prefix + "unknown format");
for (std::size_t reserved = 32; reserved < 64; reserved += 8)
{
bad = image;
bad[reserved + 3] = 1;
CHECK(load_result(bad, image_check::none) == prefix + "unknown format");
}
const auto sizes = [&](std::size_t offset, std::uint64_t v)
{
std::vector<std::uint8_t> b = image;
set_header_field(b, offset, v);
return load_result(b, image_check::none);
};
CHECK(sizes(8, 0) == prefix + "sizes out of range"); // no nodes
CHECK(sizes(8, 1000) == prefix + "sizes out of range"); // more nodes than bytes
CHECK(sizes(8, 0xFFFFFFF0u) == prefix + "sizes out of range");
CHECK(sizes(16, 0xFFFFFFF0u) == prefix + "sizes out of range"); // text size
CHECK(sizes(16, header_field(image, 16) + 1) == prefix + "sizes out of range");
CHECK(sizes(16, image.size()) == prefix + "sizes out of range");
CHECK(sizes(24, 0xFFFFFFF0u) == prefix + "sizes out of range"); // decoded string size
CHECK(sizes(24, header_field(image, 24) - 1) == prefix + "sizes out of range");
// the NULs after the text and the decoded strings
bad = image;
bad[text_at(image) + header_field(image, 16)] = 'x';
CHECK(load_result(bad, image_check::none) == prefix + "sizes out of range");
bad = image;
bad.back() = 'x';
CHECK(load_result(bad, image_check::none) == prefix + "sizes out of range");
// nothing after the image
bad = image;
bad.push_back(0);
CHECK(load_result(bad, image_check::none) == prefix + "sizes out of range");
// nodes, but not even room for the NULs
bad.assign(image.begin(), image.begin() + static_cast<std::ptrdiff_t>(text_at(image)));
CHECK(load_result(bad, image_check::none) == prefix + "sizes out of range");
}
}
TEST_CASE("json_view images: check")
{
// nodes: 0 { 1 "s" 2 "x\"y" (escaped) 3 "i" 4 -12 5 "u" 6 7 7 "f" 8 1.5e300 9 "b" 10 true 11 "n" 12 null
// 13 "a" 14 [ 15 "t" 16 {} ]
const std::string text = R"({"s":"x\"y","i":-12,"u":7,"f":1.5e300,"b":true,"n":null,"a":["t",{}]})";
const std::vector<std::uint8_t> image = json_document::parse(text).save();
REQUIRE(load_result(image, image_check::full).empty());
REQUIRE(node_count(image) == 17);
// bounds: rejected by both checks; content: only by the full one
const auto rejected = [&](const std::vector<std::uint8_t>& b, bool bounds)
{
CHECK(load_result(b, image_check::full) == check_failed);
CHECK(load_result(b, image_check::bounds) == (bounds ? check_failed : ""));
};
SECTION("kinds")
{
const std::array<std::uint8_t, 4> kinds = {{8, 9, 10, 200}}; // binary, discarded, link, unknown
for (const std::uint8_t kind : kinds)
{
rejected(corrupted(image, 12, [&](node & n)
{
n.kind = kind;
}), true);
}
// a key that is not a string
rejected(corrupted(image, 1, [](node & n)
{
n.kind = 0;
n.len = 0;
n.off = 0;
}), true);
}
SECTION("flags and extra")
{
rejected(corrupted(image, 12, [](node & n)
{
n.flags = 4;
}), true);
rejected(corrupted(image, 12, [](node & n)
{
n.extra = 1;
}), true);
rejected(corrupted(image, 10, [](node & n)
{
n.flags = 5;
}), true);
rejected(corrupted(image, 10, [](node & n)
{
n.extra = 1;
}), true);
rejected(corrupted(image, 1, [](node & n)
{
n.flags = 2; // a string in the edit arena
}), true);
rejected(corrupted(image, 1, [](node & n)
{
n.extra = 3;
}), true);
rejected(corrupted(image, 4, [](node & n)
{
n.flags = 2;
}), true);
rejected(corrupted(image, 4, [](node & n)
{
n.extra = static_cast<std::uint16_t>(n.extra | 0x100u); // an integer with fraction digits
}), true);
rejected(corrupted(image, 0, [](node & n)
{
n.flags = 8; // moved
}), true);
rejected(corrupted(image, 0, [](node & n)
{
n.extra = 1; // a hash index
}), true);
}
SECTION("bounds")
{
const std::size_t text_size = header_field(image, 16);
const std::size_t arena_size = header_field(image, 24);
rejected(corrupted(image, 1, [&](node & n)
{
n.off = static_cast<std::uint32_t>(text_size + 1);
}), true);
rejected(corrupted(image, 1, [&](node & n)
{
n.len = static_cast<std::uint32_t>(text_size);
}), true);
rejected(corrupted(image, 2, [&](node & n)
{
n.len = static_cast<std::uint32_t>(arena_size + 1);
}), true);
rejected(corrupted(image, 6, [&](node & n)
{
n.off = static_cast<std::uint32_t>(text_size);
}), true);
rejected(corrupted(image, 6, [&](node & n)
{
n.off = static_cast<std::uint32_t>(text_size + 5);
}), true);
rejected(corrupted(image, 6, [](node & n)
{
n.extra = 0; // no digits
}), true);
rejected(corrupted(image, 8, [&](node & n)
{
n.len = static_cast<std::uint32_t>(text_size);
}), true);
rejected(corrupted(image, 8, [](node & n)
{
n.len = 2; // shorter than the recorded digits
}), true);
rejected(corrupted(image, 14, [&](node & n)
{
n.off = static_cast<std::uint32_t>(text_size + 1);
}), true);
// literals: their offset sizes the output of dump()
rejected(corrupted(image, 10, [&](node & n)
{
n.off = static_cast<std::uint32_t>(text_size + 1);
}), true);
rejected(corrupted(image, 12, [&](node & n)
{
n.off = 0xFFFFFFFFu;
}), true);
}
SECTION("structure")
{
rejected(corrupted(image, 0, [](node & n)
{
n.next = 0;
}), true);
rejected(corrupted(image, 0, [](node & n)
{
n.next = 18; // beyond the image
}), true);
rejected(corrupted(image, 14, [](node & n)
{
n.next = 4; // beyond the enclosing object
}), true);
rejected(corrupted(image, 0, [](node & n)
{
n.len = 6; // member count
}), true);
rejected(corrupted(image, 14, [](node & n)
{
n.len = 3; // element count
}), true);
rejected(corrupted(image, 0, [](node & n)
{
n.next = 14; // the object ends after the key "a"
n.len = 7;
}), true);
rejected(corrupted(image, 0, [](node & n)
{
n.next = 13; // nodes after the root
n.len = 6;
}), true);
rejected(corrupted(image, 0, [](node & n)
{
n.kind = 2; // an array: the "keys" are values, and the counts do not match
}), true);
const std::vector<std::uint8_t> as_array = corrupted(image, 16, [](node & n)
{
n.kind = 2; // {} as []: fine
});
CHECK(load_result(as_array, image_check::full).empty());
CHECK(json_document::load(as_array).root().dump() == R"({"s":"x\"y","i":-12,"u":7,"f":1.5e+300,"b":true,"n":null,"a":["t",[]]})");
}
SECTION("strings")
{
// a quote in a source string (the full check only)
std::vector<std::uint8_t> b = image;
const std::size_t t = text_at(image);
const node t15 = node_at(image, 15);
b[t + t15.off] = '"';
rejected(b, false);
// a control character
b[t + t15.off] = '\n';
rejected(b, false);
// invalid UTF-8 in a decoded string
b = image;
const node s2 = node_at(image, 2);
b[t + header_field(image, 16) + 1 + s2.off] = 0xFF;
rejected(b, false);
}
SECTION("numbers")
{
const std::size_t t = text_at(image);
const node i4 = node_at(image, 4);
const node u6 = node_at(image, 6);
const node f8 = node_at(image, 8);
const auto at_token = [&](const node & n, std::size_t k, std::uint8_t c)
{
std::vector<std::uint8_t> b = image;
b[t + n.off + k] = c;
return b;
};
rejected(at_token(i4, 1, 'x'), false); // -x2
rejected(at_token(i4, 1, '0'), false); // -02
rejected(at_token(i4, 0, '1'), false); // 112 != -12
rejected(at_token(f8, 1, 'x'), false); // 1x5e300
rejected(at_token(f8, 2, 'e'), false); // 1.ee300
rejected(at_token(f8, 4, 'x'), false); // 1.5ex00
rejected(at_token(f8, 3, '0'), false); // 1.50300: another layout
rejected(at_token(f8, 4, '9'), false); // 1.5e900: overflow
rejected(at_token(f8, 0, 'x'), false);
rejected(at_token(u6, 0, '8'), false); // 8 != 7
rejected(corrupted(image, 4, [](node & n)
{
n.kind = 6; // "-12" as unsigned: a sign
n.extra = 3;
}), false);
// a non-negative number_integer (as edits write it): fine
const std::vector<std::uint8_t> positive = corrupted(image, 6, [](node & n)
{
n.kind = 5;
n.extra = 0;
});
CHECK(load_result(positive, image_check::full).empty());
CHECK(json_document::load(positive).root()["u"].is_number_integer());
rejected(corrupted(image, 8, [](node & n)
{
n.kind = 6; // a float token as integer
n.extra = 7;
}), false);
}
SECTION("float tokens of an image checked for bounds only")
{
// A float node whose layout records "many" digits is converted from
// its token alone; a token that is not a JSON number reads as 0.
const std::vector<std::uint8_t> img = json_document::parse("[1.5e300,2]").save();
const std::size_t t = text_at(img);
for (const char* token :
{
"x.5e300", "01.5e30", "1.xe300", "1.5ex00", "1.5e+x0", "1.5e30x", "-.5e300", "1.5E300"
})
{
CAPTURE(token);
std::vector<std::uint8_t> b = img;
node n = node_at(b, 1);
n.extra = 0xFFFFu;
set_node(b, 1, n);
std::memcpy(b.data() + t + n.off, token, n.len);
const json_document d = json_document::load(b, image_check::bounds);
const auto v = d.root()[0].get<double>();
CHECK(v == (std::string(token) == "1.5E300" ? 1.5e300 : 0.0));
CHECK(load_result(b, image_check::full) == (std::string(token) == "1.5E300" ? "" : check_failed));
}
}
SECTION("integer ranges")
{
// tokens of many digits, which the parser stores as floats
const std::string big = R"([123456789012345678901234, 99999999999999999999, 9223372036854775808])";
const std::vector<std::uint8_t> img = json_document::parse(big).save();
const auto as_integer = [&](std::size_t i, std::uint8_t kind, std::uint16_t extra)
{
std::vector<std::uint8_t> b = img;
node n = node_at(b, i);
n.kind = kind;
n.extra = extra;
set_node(b, i, n);
return load_result(b, image_check::full);
};
CHECK(as_integer(1, 6, 24) == check_failed); // more than 20 digits
CHECK(as_integer(2, 6, 20) == check_failed); // more than 2^64 - 1
CHECK(as_integer(3, 5, 18) == check_failed); // more than 2^63 - 1 as number_integer
}
}
TEST_CASE("json_view images: damaged images")
{
// A damaged image must be rejected, or read safely; with the full check,
// it also serializes to the JSON it reads as.
const std::vector<std::string> texts =
{
R"({"a": [1, -2, 3.25, "x\u00e9y", true, null], "b": {"c": "\"q\"", "d": 1e10}, "e": ""})",
R"([[[[]]], {"k": {"k": {"k": 12345678901234567890}}}, "\ud83d\ude00", -0.0, 0])",
};
for (const std::string& text : texts)
{
const std::vector<std::uint8_t> image = json_document::parse(text).save();
for (int round = 0; round < 3000; ++round)
{
std::vector<std::uint8_t> b = image;
const std::uint32_t flips = 1 + (rng() % 3);
for (std::uint32_t k = 0; k < flips; ++k)
{
// mostly the nodes, where the damage matters most
const std::size_t at = rng() % 4 != 0 ? header_size + (rng() % (b.size() - header_size)) : rng() % b.size();
b[at] = static_cast<std::uint8_t>(rng() % 3 == 0 ? rng() : b[at] ^ (1u << (rng() % 8)));
}
for (const image_check check :
{
image_check::full, image_check::bounds
})
{
json_document d;
try
{
d = json_document::load(b, check);
}
catch (const json::parse_error& e)
{
CHECK(e.id == 116);
continue;
}
std::string dumped;
std::string dumped_ascii;
try
{
dumped = d.root().dump();
dumped_ascii = d.root().dump(-1, ' ', true);
}
catch (const json::type_error& e)
{
// invalid UTF-8 (the bounds check only)
CHECK(check == image_check::bounds);
CHECK(e.id == 316);
continue;
}
const json j = d.root().materialize();
if (check == image_check::full)
{
CHECK(json::parse(dumped) == j);
CHECK(json::parse(dumped_ascii) == j);
}
}
}
}
}
#else
TEST_CASE("json_view images: big-endian targets")
{
const json_document d = json_document::parse("[1]");
CHECK_THROWS_WITH_AS(d.save(), "[json.exception.type_error.320] json_document images need a little-endian target", json::type_error&);
}
#endif
+255 -1
View File
@@ -15,6 +15,20 @@
#include <nlohmann/json.hpp>
using nlohmann::detail::dtoa_impl::reinterpret_bits;
#include <array>
#include <cmath>
#include <cstdint>
#include <cstdio>
#include <cstdlib>
#include <limits>
#include <random>
#include <string>
#include <utility>
#include <vector>
#if defined(JSON_HAS_CPP_17)
#include <charconv>
#endif
namespace
{
float make_float(uint32_t sign_bit, uint32_t biased_exponent, uint32_t significand)
@@ -450,7 +464,7 @@ TEST_CASE("formatting")
check_double( 1.2345e+18, "1.2345e+18" ); // 1.2345e+18 1.2345e+18 1.2345e18
check_double( 1.2345e+19, "1.2345e+19" ); // 1.2345e+19 1.2345e+19 1.2345e19
check_double( 1.2345e+20, "1.2345e+20" ); // 1.2345e+20 1.2345e+20 1.2345e20
check_double( 1.2345e+21, "1.2344999999999999e+21" ); // 1.2345e+21 1.2344999999999999e+21 1.2345e21
check_double( 1.2345e+21, "1.2345e+21" ); // 1.2345e+21 1.2344999999999999e+21 1.2345e21
check_double( 1.2345e+22, "1.2345e+22" ); // 1.2345e+22 1.2345e+22 1.2345e22
}
@@ -514,3 +528,243 @@ TEST_CASE("formatting")
check_integer(1000000000000000000LL, "1000000000000000000");
}
}
namespace
{
// a small unsigned big integer (32-bit limbs, least significant first), to
// recompute the powers of ten of the shortest double conversion
using big = std::vector<std::uint32_t>;
void big_mul_small(big& x, std::uint32_t m)
{
std::uint64_t carry = 0;
for (auto& limb : x)
{
const std::uint64_t v = (static_cast<std::uint64_t>(limb) * m) + carry;
limb = static_cast<std::uint32_t>(v);
carry = v >> 32u;
}
if (carry != 0)
{
x.push_back(static_cast<std::uint32_t>(carry));
}
}
void big_div_small(big& x, std::uint32_t d)
{
std::uint64_t rest = 0;
for (std::size_t i = x.size(); i-- > 0;)
{
const std::uint64_t v = (rest << 32u) | x[i];
x[i] = static_cast<std::uint32_t>(v / d);
rest = v % d;
}
while (!x.empty() && x.back() == 0)
{
x.pop_back();
}
}
std::size_t big_bit_length(const big& x)
{
std::size_t n = 32 * x.size();
for (std::uint32_t top = x.back(); (top & 0x80000000u) == 0; top <<= 1u)
{
--n;
}
return n;
}
bool big_bit(const big& x, std::size_t i)
{
return ((x[i / 32] >> (i % 32)) & 1u) != 0;
}
/// the 128 most significant bits of x (floor), shifted left if x has fewer bits
std::pair<std::uint64_t, std::uint64_t> big_top128(const big& x)
{
const std::size_t n = big_bit_length(x);
std::uint64_t high = 0;
std::uint64_t low = 0;
for (std::size_t k = 0; k < 128; ++k)
{
const bool bit = k < n && big_bit(x, n - 1 - k);
if (k < 64)
{
high = (high << 1u) | (bit ? 1u : 0u);
}
else
{
low = (low << 1u) | (bit ? 1u : 0u);
}
}
return {high, low};
}
/// the digits (without trailing zeros) and the decimal exponent of a
/// representation "[-]d[.ddd][e[+-]x]"
std::pair<std::string, int> digits_and_exponent(const std::string& s)
{
std::string digits;
int point = -1;
int exponent = 0;
for (std::size_t i = 0; i < s.size(); ++i)
{
const char c = s[i];
if (c >= '0' && c <= '9')
{
digits += c;
}
else if (c == '.')
{
point = static_cast<int>(digits.size());
}
else if (c == 'e' || c == 'E')
{
exponent = std::stoi(s.substr(i + 1));
break;
}
}
int e = exponent + (point < 0 ? static_cast<int>(digits.size()) : point) - static_cast<int>(digits.size());
const std::size_t first = digits.find_first_not_of('0');
digits = first == std::string::npos ? "0" : digits.substr(first);
while (digits.size() > 1 && digits.back() == '0')
{
digits.pop_back();
++e;
}
return {digits, e};
}
/// whether the decimal digits * 10^e reads back as v
bool reads_back(const std::string& digits, int e, double v)
{
const std::string text = digits + "e" + std::to_string(e);
return std::strtod(text.c_str(), nullptr) == v;
}
/// Check the representation of a positive finite double: it reads back as
/// the same value, and no representation with fewer digits does.
void check_shortest(double v)
{
std::array<char, 33> buf{};
char* end = nlohmann::detail::to_chars(buf.data(), buf.data() + 32, v);
const std::string text(buf.data(), end);
CAPTURE(text);
CHECK(std::strtod(text.c_str(), nullptr) == v);
// the layout is that of format_buffer() for the same digits
std::array<char, 64> reference{};
int len = 0;
int exponent = 0;
nlohmann::detail::dtoa_impl::shortest_digits(reference.data(), len, exponent, v);
const char* const reference_end = nlohmann::detail::dtoa_impl::format_buffer(reference.data(), len, exponent, -4, 15);
CHECK(text == std::string(reference.data(), static_cast<std::size_t>(reference_end - reference.data())));
const auto de = digits_and_exponent(text);
const std::string& digits = de.first;
if (digits.size() > 1)
{
// the decimals of one digit fewer next to the value
std::array<char, 64> shorter{};
const int n = std::snprintf(shorter.data(), shorter.size(), "%.*e", static_cast<int>(digits.size()) - 2, v); // NOLINT(cppcoreguidelines-pro-type-vararg,hicpp-vararg)
const auto near = digits_and_exponent(std::string(shorter.data(), static_cast<std::size_t>(n)));
// as an integer with digits.size() - 1 digits
std::string m = near.first;
int e = near.second;
while (m.size() < digits.size() - 1)
{
m += '0';
--e;
}
const std::uint64_t mid = std::stoull(m);
for (const std::uint64_t candidate :
{
mid - 1, mid, mid + 1
})
{
CAPTURE(candidate);
CHECK(!reads_back(std::to_string(candidate), e, v));
}
}
#if defined(JSON_HAS_CPP_17) && defined(__cpp_lib_to_chars)
// the closest of the shortest representations, as std::to_chars finds it
std::array<char, 64> std_text{};
const auto r = std::to_chars(std_text.data(), std_text.data() + std_text.size(), v, std::chars_format::scientific);
CHECK(digits_and_exponent(std::string(std_text.data(), r.ptr)) == de);
#endif
}
} // namespace
TEST_CASE("shortest digits of doubles")
{
SECTION("powers of ten")
{
// the 128-bit significands of 10^k, rounded down, recomputed
for (int k = -342; k <= 341; ++k)
{
CAPTURE(k);
big x{1};
if (k >= 0)
{
for (int i = 0; i < k; ++i)
{
big_mul_small(x, 10);
}
}
else
{
// floor(2^b / 10^-k) for a b that leaves more than 128 bits
const int b = 128 + 64 + (4 * -k);
x.assign(static_cast<std::size_t>(b / 32) + 1, 0);
x.back() = 1u << (b % 32);
for (int i = 0; i < -k; ++i)
{
big_div_small(x, 10);
}
}
const auto expected = big_top128(x);
const auto actual = nlohmann::detail::zmij::pow10(k);
CHECK(actual.high == expected.first);
CHECK(actual.low == expected.second);
}
}
SECTION("boundary values")
{
for (const double v :
{
std::numeric_limits<double>::min(), std::numeric_limits<double>::max(), std::numeric_limits<double>::denorm_min(),
std::nextafter(std::numeric_limits<double>::min(), 0.0), 1.0, 2.0, 0.1, 0.3, 1e21, 1e22, 1e23, 5e-324, 9007199254740993.0,
1.2345e+21, 2.2250738585072014e-308, 1.7976931348623157e308, 4.9406564584124654e-324, 123456789012345680.0
})
{
check_shortest(v);
}
// all powers of two (their rounding interval is narrower below)
for (int e = -1074; e <= 1023; ++e)
{
check_shortest(std::ldexp(1.0, e));
}
// powers of ten and their neighbors
for (int e = -323; e <= 308; ++e)
{
const double p = std::strtod(("1e" + std::to_string(e)).c_str(), nullptr);
check_shortest(p);
check_shortest(std::nextafter(p, 0.0));
check_shortest(std::nextafter(p, std::numeric_limits<double>::infinity()));
}
}
SECTION("random doubles")
{
std::mt19937_64 rng(5295); // NOLINT(cert-msc32-c,cert-msc51-cpp,bugprone-random-generator-seed): reproducible
for (int i = 0; i < 100000; ++i)
{
const std::uint64_t bits = rng() & 0x7FFFFFFFFFFFFFFFu;
const auto v = reinterpret_bits<double>(bits);
if (std::isfinite(v) && v != 0)
{
check_shortest(v);
}
}
}
}