Add images of json_documents: save() and load() (#5632)

* Add images of json_documents: save() and load()

An image is a document stored so that loading it needs no
parsing: save() writes the node index, the text and the decoded
strings; the static load() reads an image written by save().

load() takes a pointer and size, a borrowed vector, or an owned
rvalue vector; the nodes are copied so they are aligned and can
be edited, while the text and decoded strings stay in the image.

image_check controls how much load() trusts the input: full
checks structure, bounds, strings and numbers, the parser's own
guarantees; bounds checks structure and bounds only; none skips
all checks, for images from a trusted source.

Layout is little-endian only ("NJVI" header, nodes, text, decoded
strings), following the idea of zero-copy formats such as
FlatBuffers and YaFF; the check follows FlatBuffers' Verifier.

New errors: parse_error.116 for a malformed image or a failed
check, type_error.320 for a discarded document or a big-endian
target.

A dedicated fuzzer and 6,000 seeded corruptions, checked under
ASan/UBSan, found and fixed two gaps: unchecked reserved header
fields, and unbounded null/boolean offsets that could make
dump() throw std::length_error.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix MSVC errors in image test and add image_check doc anchor

Move the raw string literal out of the CHECK macro (MSVC preprocessor),
cast 64-bit header fields to std::size_t (C4244 on 32-bit), and give the
image_check section in load.md a real anchor for the mkdocs strict build.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Check each distinct string and float range once in the full image check

The full check scanned the contents of every string node and every float token over its own range, so many nodes pointing to one large range made load() quadratic. The structural walk now only collects the ranges; after it, each kind is sorted and checked once per distinct range. Ranges of one kind that overlap without being identical are rejected, as save() never writes them (nodes that share a value share the whole range). The cost is linear in the size of the image plus sorting the ranges.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Adapt the image tests to named documents and last-wins lookups

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Test that the targets of links never reach an image

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Test lookups in loaded images with duplicate and colliding keys

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Release the list of large objects once load() has built the indexes

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Test that invalid UTF-8 of a loaded image is not copied into an editable document

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Document that editable documents refuse invalid UTF-8 copied from a loaded image

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Expect first-wins lookups in the image tests

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Avoid useless casts of header fields in the image tests

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix clang-tidy 22 findings in unit-json_view_image.cpp and image.hpp

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix MSVC C4244 and C2017 in the image test

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Fix the indentation of a loop in the image test

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Make json_view.hpp pass include-what-you-use

ci_single_binaries runs IWYU with --error on every header. For json_view.hpp
it suggested adding <array> (std::array is used), the headers that json.hpp
already provides (abi_config, abi_macros, input_adapters, json_pointer,
cpp_future, string_concat, value_t, json_fwd), and <version> for
std::nullptr_t, and removing <cstddef>.

- include <array>
- keep <cstddef> (nullptr_t, size_t; IWYU attributes them to <version> and <cstring>)
- tell IWYU not to suggest the headers that json.hpp provides: the amalgamated
  json_view.hpp only includes json.hpp, so including them here would duplicate
  their definitions
- drop edit_storage.hpp, which edit.hpp includes

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Silence a clang-tidy C++17 finding in the image test

bugprone-suspicious-stringview-data-usage flags data() of the source view
under C++17, although the length is passed and checked by the REQUIRE
before it.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Build the image link test without exceptions

load_result() exists only with exceptions; with JSON_NOEXCEPTION the test
loads the image with the check directly (a failed check aborts).

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Regenerate the amalgamated headers

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Follow the renames from the review of the view builder

The review of #5620 renamed document_data's tape/tape_size/tape_cap to
nodes/node_count/node_capacity and made the decode arena a
std::vector<char>; adapt the code of this change to it.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Rename the emit lambda in the image writer for Qt compatibility

Qt defines emit as an empty macro, so a Qt header included before
json_view.hpp broke the lambda's declaration (see #5865). The slots
variables in the image tests are renamed as well.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Address the review of json_document images

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Regenerate the amalgamated headers

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

docs/mkdocs/docs/features/index.md

@@ -14,7 +14,8 @@ C++ types, and finally serialize it again.
   [parsing untrusted input](parsing/untrusted_input.md).
 - [Zero-copy JSON views](json_view.md) — read a JSON text through a flat index instead of building a `json` tree;
   strings and numbers stay in the input and are only decoded when needed.
-  [Editable documents](json_view.md#editing-a-document) can also be modified.
+- [Editable documents](json_view.md#editing-a-document) — modify a parsed document without building a `json` tree.
+- [Images](json_view.md#images) — save a parsed document and load it again without parsing it.
 - [Comments](comments.md) and [trailing commas](trailing_commas.md) — opt-in relaxations of the JSON grammar.
 
 ## Accessing and modifying values

docs/mkdocs/docs/features/json_view.md

@@ -312,6 +312,57 @@ edits. See [`basic_json_document`'s Edits](../api/basic_json_document/index.md#e
 throws (the *basic* guarantee, not the strong one `dump()` and the read-only functions provide). How edits are kept in
 the index is described in the [architecture overview](../home/architecture.md#node-index-of-json-views).
 
+## Images
+
+[`save()`](../api/basic_json_document/save.md) writes a document as an *image*: a byte buffer that
+[`load()`](../api/basic_json_document/load.md) reads back into a document without parsing -- no lexing, no building
+the node index, nothing but copying the nodes and pointing the text and the decoded strings at the image. Where
+[`parse_copy()`](../api/basic_json_document/parse_copy.md) still has to scan the whole input,
+[`load()`](../api/basic_json_document/load.md) turns that scan into a copy of the node index alone.
+
+**Why.** A document that is parsed once and then read many times -- a configuration loaded at startup, a template
+rendered on every request, a large reference dataset a worker process needs in memory -- pays for parsing once but
+can amortize [`save()`](../api/basic_json_document/save.md)'s cost across every later load. That makes images useful
+for a cache: save a document the first time it is parsed (to a file, a shared-memory segment, an in-process cache),
+and [`load()`](../api/basic_json_document/load.md) it on every later use instead of parsing the source text again.
+They are just as useful for handing a parsed document to another process (or a forked worker) running the same build
+of the library, since [`load()`](../api/basic_json_document/load.md) turns the transfer into a copy of the node index
+plus pointers into the received bytes, not a re-parse.
+
+**Choosing a check.** [`load()`](../api/basic_json_document/load.md) takes an
+[`image_check`](../api/basic_json_document/load.md#image_check) that trades validation against speed:
+`image_check::full` (the default) checks everything the parser itself guarantees, so a checked image is exactly as
+safe to read and serialize as a freshly parsed document -- the right choice whenever the image did not come straight
+from this process's own [`save()`](../api/basic_json_document/save.md), such as a file or a network peer.
+`image_check::bounds` only checks structure and bounds -- cheaper, since it skips scanning the text and the decoded
+strings -- and fits a cache the process trusts, one it wrote and reads back itself. `image_check::none` skips
+validation entirely, for an image trusted as much as the process's own memory. See
+[`load()`'s Notes](../api/basic_json_document/load.md#notes) for exactly what each level does and does not guarantee.
+
+??? example "Example: cache a parsed configuration as an image"
+
+    ```cpp
+    --8<-- "examples/basic_json_document__save.cpp"
+    ```
+
+    Output:
+
+    ```json
+    --8<-- "examples/basic_json_document__save.output"
+    ```
+
+!!! warning "Experimental"
+
+    The image format is versioned but not yet stable, and may change in an incompatible way before it is declared
+    stable. It is little-endian only, and tied to the library build that wrote it -- use it to cache a document or to
+    hand one to another process running the *same* build, not as a long-term storage format; keep the original JSON
+    text if a saved document needs to be readable by a future library version.
+
+The idea of a document you can read without parsing comes from zero-copy formats such as
+[FlatBuffers](https://github.com/google/flatbuffers) and [YaFF](https://github.com/yandex/yaff); the check
+[`load()`](../api/basic_json_document/load.md) runs follows the idea of FlatBuffers' Verifier. No code is taken from
+either.
+
 ## Choosing between `json`, `ordered_json`, the SAX interface, and `json_view`
 
 | | [`json`](../api/json.md) / [`ordered_json`](../api/ordered_json.md) | [SAX interface](parsing/sax_interface.md) | [`json_document`](../api/json_document.md) / [`json_view`](../api/json_view.md) | [`json_editable_document`](../api/json_editable_document.md) / [`json_editable_view`](../api/json_editable_view.md) |
@@ -320,6 +371,7 @@ the index is described in the [architecture overview](../home/architecture.md#no
 | **Mutability** | freely mutable | not applicable (a one-shot event stream) | read-only | [`set`](../api/basic_json_document/set.md)/[`push_back`](../api/basic_json_document/push_back.md)/[`insert`](../api/basic_json_document/insert.md)/[`erase`](../api/basic_json_document/erase.md) edit in place; the source text is never rewritten |
 | **What you get** | a full tree you can read, write, and keep as long as you like | a sequence of callbacks; whatever your handler builds from them | a flat index plus, on demand, [`materialize()`](../api/basic_json_view/materialize.md)d `json`/`ordered_json` values for the parts you actually use | the same, plus [`dump()`](../api/basic_json_view/dump.md) of an edited document that keeps the member order and, with [`number_format::source`](../api/basic_json_view/number_format.md), the spelling of every untouched number |
 | **Typical use** | general-purpose JSON handling: config, request/response bodies you build or modify, anything you hold onto | validating or projecting a text into your own data structure without ever holding the whole thing as JSON | large or high-volume input where you only need part of it, or need it repeatedly, and can keep the source text (or a copy) alive for as long as the document lives | a document you read, patch a few fields of, and write back -- a configuration file, for instance -- where the rest of it should come back exactly as it was |
+| **Caching/reload** | not applicable -- re-parse, or roll your own serialization | not applicable | [`save()`](../api/basic_json_document/save.md)/[`load()`](../api/basic_json_document/load.md): cache the parsed index as an image and reload it without parsing | same, saving the document's current -- possibly edited -- state |
 
 ## Version history