docs: cross-reference json_view and json_document for 3.13.0

Document the new zero-copy document/view API across the docs: add a
"Documents and views" section to the basic_json API hub linking to all
specializations, point parsing/conversion/modifying pages to the view-based
alternatives, add document/view sections to feature pages, update the
landing page to mention json_view.hpp/json_literals.hpp, and cross-link
debugging/exceptions.

Also document wide-string inputs (and ill-formed UTF-16/32 rejection),
add parse/accept examples for wide strings, fix related doc gaps, and
align features/parsing and FAQ with the added support.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

docs/mkdocs/docs/api/basic_json/accept.md

@@ -23,6 +23,9 @@ Checks whether the input is valid JSON.
     respectively as UTF-8, UTF-16, and UTF-32. If `SentinelType` differs from `IteratorType`, it must be comparable to
     the iterator type with `operator!=`.
     
+    Ill-formed UTF-16 and UTF-32 sequences are rejected: a lone surrogate, a high surrogate that is not followed by its
+    low surrogate, and a code point above `U+10FFFF` all make this function return `false`.
+
 Unlike the [`parse()`](parse.md) function, this function neither throws an exception in case of invalid JSON input
 (i.e., a parse error) nor creates diagnostic information.
 
@@ -35,9 +38,12 @@ Unlike the [`parse()`](parse.md) function, this function neither throws an excep
     - a `#!c FILE` pointer (throws if null)
     - a C-style array of characters
     - a pointer to a null-terminated string of single byte characters (throws if null)
-    - a `std::string`
+    - a `std::string` or a `std::string_view` (C++17)
     - a container `obj` for which `begin(obj)` and `end(obj)` produce a valid pair of iterators
       (as found via ADL or member functions, with semantics compatible to `std::begin` and `std::end`)
+    - wide characters (`wchar_t`, `char16_t`, `char32_t`), which are converted to UTF-8: a wide string object
+      (`std::wstring`, `std::u16string`, `std::u32string`) or an array (e.g. a wide string literal); a pointer to a
+      null-terminated wide string (e.g. `std::u16string::c_str()`) is not supported
 
 `IteratorType`
 :   a compatible iterator type, for instance.
@@ -125,6 +131,21 @@ outside of a string, invalid) byte; see the [FAQ entry](../../home/faq.md#nul-by
     --8<-- "examples/accept__iterator_pair.output"
     ```
 
+??? example "Example: (3) reading from UTF-16 strings"
+
+    The encoding is deduced from the character type of the input. String literals like `u"..."`, `U"..."`, and
+    `L"..."` work as well.
+
+    ```cpp
+    --8<-- "examples/accept__wide_string.cpp"
+    ```
+
+    Output:
+
+    ```json
+    --8<-- "examples/accept__wide_string.output"
+    ```
+
 ## See also
 
 - [parse](parse.md) - deserialize from a compatible input
@@ -151,6 +172,9 @@ outside of a string, invalid) byte; see the [FAQ entry](../../home/faq.md#nul-by
   ([`JSON_DISABLE_STREAM_BULK_SCAN`](../macros/json_disable_stream_bulk_scan.md)).
 - Parsing from a `FILE*` locks the file once instead of once per character in version 3.13.0
   ([`JSON_DISABLE_UNLOCKED_FILE_IO`](../macros/json_disable_unlocked_file_io.md)).
+- Ill-formed UTF-16 and UTF-32 input (a lone surrogate, a high surrogate that is not followed by its low surrogate, or
+  a code point above `U+10FFFF`) is rejected in version 3.13.0; before that, some such sequences were accepted or
+  ended the input silently.
 
 !!! warning "Deprecation"
 

docs/mkdocs/docs/api/basic_json/index.md

@@ -44,6 +44,22 @@ are collected in [Template Parameter Requirements](../../features/types/template
 - [**json**](../json.md) - default specialization
 - [**ordered_json**](../ordered_json.md) - a specialization that maintains the insertion order of object keys
 
+## Documents and views
+
+Besides `basic_json`, the library offers a second way to hold a parsed JSON text: a
+[`basic_json_document`](../basic_json_document/index.md), which owns or borrows the text and holds a flat index of the
+values in it, and a [`basic_json_view`](../basic_json_view/index.md), a read-only handle to one value of such a document.
+Both are defined in the header `<nlohmann/json_view.hpp>`; see
+[Zero-copy JSON views](../../features/json_view.md) for the concept and the trade-offs.
+
+- [**json_document**](../json_document.md) / [**ordered_json_document**](../ordered_json_document.md) - read-only
+  documents of the `basic_json` specialization they belong to
+- [**json_editable_document**](../json_editable_document.md) /
+  [**ordered_json_editable_document**](../ordered_json_editable_document.md) - documents that can be modified in place
+- [**json_view**](../json_view.md) / [**ordered_json_view**](../ordered_json_view.md) - views of a document
+- [**json_editable_view**](../json_editable_view.md) /
+  [**ordered_json_editable_view**](../ordered_json_editable_view.md) - views of an editable document
+
 ## Iterator invalidation
 
 All operations that add values to an **array** ([`push_back`](push_back.md) , [`operator+=`](operator+=.md),
@@ -343,6 +359,9 @@ Access to the JSON value
 ## See also
 
 - [RFC 8259: The JavaScript Object Notation (JSON) Data Interchange Format](https://tools.ietf.org/html/rfc8259)
+- [basic_json_document](../basic_json_document/index.md) / [basic_json_view](../basic_json_view/index.md) - parse a JSON
+  text into a flat index and read it through a view
+- [Zero-copy JSON views](../../features/json_view.md) - when to use a document and a view instead of a `basic_json`
 
 ## Version history
 

docs/mkdocs/docs/api/basic_json/parse.md

@@ -25,6 +25,10 @@ static basic_json parse(IteratorType first, SentinelType last,
     respectively as UTF-8, UTF-16, and UTF-32. If `SentinelType` differs from `IteratorType`, it must be comparable to
     the iterator type with `operator!=`.
 
+    Ill-formed UTF-16 and UTF-32 sequences are rejected: a lone surrogate, a high surrogate that is not followed by its
+    low surrogate, and a code point above `U+10FFFF` all yield a
+    [`parse_error.101`](../../home/exceptions.md#jsonexceptionparse_error101).
+
 ## Template parameters
 
 `InputType`
@@ -34,9 +38,12 @@ static basic_json parse(IteratorType first, SentinelType last,
     - a `FILE` pointer (throws if null)
     - a C-style array of characters
     - a pointer to a null-terminated string of single byte characters (throws if null)
-    - a `std::string`
+    - a `std::string` or a `std::string_view` (C++17)
     - a container `obj` for which `begin(obj)` and `end(obj)` produce a valid pair of iterators
       (as found via ADL or member functions, with semantics compatible to `std::begin` and `std::end`)
+    - wide characters (`wchar_t`, `char16_t`, `char32_t`), which are converted to UTF-8: a wide string object
+      (`std::wstring`, `std::u16string`, `std::u32string`) or an array (e.g. a wide string literal); a pointer to a
+      null-terminated wide string (e.g. `std::u16string::c_str()`) is not supported
 
 `IteratorType`
 :   a compatible iterator type, for instance.
@@ -112,6 +119,13 @@ By default, a `'\0'` (NUL) byte anywhere in the input is treated as end of input
 outside of a string, invalid) byte; see the [FAQ entry](../../home/faq.md#nul-bytes-in-the-input) for details and the
 [`JSON_STRICT_NUL_HANDLING`](../macros/json_strict_nul_handling.md) macro to opt into rejecting it instead.
 
+!!! tip "Parsing into a document instead of a value"
+
+    [`basic_json_document::parse`](../basic_json_document/parse.md) accepts the same inputs and the same options, but
+    keeps the result as a flat index of the parsed values instead of a tree of `basic_json` values. It is faster and uses
+    less memory when only parts of the document are read, and it also accepts a pointer to a null-terminated wide string,
+    which `parse()` does not. See [Zero-copy JSON views](../../features/json_view.md).
+
 !!! tip "Performance"
 
     Contiguous inputs (strings, string views, vectors, arrays, pointer ranges) are the fastest to parse. A
@@ -250,6 +264,22 @@ outside of a string, invalid) byte; see the [FAQ entry](../../home/faq.md#nul-by
     --8<-- "examples/trailing_commas.output"
     ```
 
+??? example "Example: parse from UTF-16, UTF-32, and wide strings"
+
+    The encoding is deduced from the character type of the input; `std::u16string` and `std::u32string` are interpreted
+    as UTF-16 and UTF-32, and `std::wstring` uses the platform's `wchar_t` width. String literals like `u"..."`,
+    `U"..."`, and `L"..."` work as well, as do iterators over wide characters.
+
+    ```cpp
+    --8<-- "examples/parse__wide_string.cpp"
+    ```
+
+    Output:
+
+    ```json
+    --8<-- "examples/parse__wide_string.output"
+    ```
+
 ## See also
 
 - [accept](accept.md) - check if the input is valid JSON
@@ -280,6 +310,9 @@ outside of a string, invalid) byte; see the [FAQ entry](../../home/faq.md#nul-by
   time).
 - Parsing from a `FILE*` locks the file once instead of once per character in version 3.13.0
   ([`JSON_DISABLE_UNLOCKED_FILE_IO`](../macros/json_disable_unlocked_file_io.md) restores `std::fgetc`).
+- Ill-formed UTF-16 and UTF-32 input (a lone surrogate, a high surrogate that is not followed by its low surrogate, or
+  a code point above `U+10FFFF`) is rejected with `parse_error.101` in version 3.13.0; before that, some such
+  sequences were accepted or ended the input silently.
 
 !!! warning "Deprecation"
 

docs/mkdocs/docs/api/basic_json/sax_parse.md

@@ -31,6 +31,10 @@ Read from input and generate SAX events
     respectively as UTF-8, UTF-16, and UTF-32. If `SentinelType` differs from `IteratorType`, it must be comparable to
     the iterator type with `operator!=`.
 
+    Ill-formed UTF-16 and UTF-32 sequences are rejected: a lone surrogate, a high surrogate that is not followed by its
+    low surrogate, and a code point above `U+10FFFF` all yield a
+    [`parse_error.101`](../../home/exceptions.md#jsonexceptionparse_error101).
+
 The SAX event lister must follow the interface of [`json_sax`](../json_sax/index.md).
 
 ## Template parameters
@@ -42,8 +46,12 @@ The SAX event lister must follow the interface of [`json_sax`](../json_sax/index
     - a `FILE` pointer
     - a C-style array of characters
     - a pointer to a null-terminated string of single byte characters
+    - a `std::string` or a `std::string_view` (C++17)
     - a container `obj` for which `begin(obj)` and `end(obj)` produce a valid pair of iterators
       (as found via ADL or member functions, with semantics compatible to `std::begin` and `std::end`)
+    - wide characters (`wchar_t`, `char16_t`, `char32_t`), which are converted to UTF-8: a wide string object
+      (`std::wstring`, `std::u16string`, `std::u32string`) or an array (e.g. a wide string literal); a pointer to a
+      null-terminated wide string (e.g. `std::u16string::c_str()`) is not supported
 
 `IteratorType`
 :   a compatible iterator type for overload (2); a pair of character iterators whose `value_type` is an integral type
@@ -160,6 +168,9 @@ use the same file with stdio functions, but it must not wait for another thread
   ([`JSON_DISABLE_STREAM_BULK_SCAN`](../macros/json_disable_stream_bulk_scan.md)).
 - Parsing from a `FILE*` locks the file once instead of once per character in version 3.13.0
   ([`JSON_DISABLE_UNLOCKED_FILE_IO`](../macros/json_disable_unlocked_file_io.md)).
+- Ill-formed UTF-16 and UTF-32 input (a lone surrogate, a high surrogate that is not followed by its low surrogate, or
+  a code point above `U+10FFFF`) is rejected with `parse_error.101` in version 3.13.0; before that, some such
+  sequences were accepted or ended the input silently.
 
 !!! warning "Deprecation"
 

docs/mkdocs/docs/api/basic_json_document/index.md

@@ -115,6 +115,12 @@ points apply to every edit:
   [`source_offset()`](../basic_json_view/source_offset.md) of a value introduced by an edit is
   `#!cpp static_cast<std::size_t>(-1)`, the same value it reports for a decoded string.
 
+## See also
+
+- [basic_json_view](../basic_json_view/index.md) - the read-only handle to a value of a document
+- [json_document](../json_document.md) - the document type of the default specialization
+- [Zero-copy JSON views](../../features/json_view.md) - the concept behind documents and views
+
 ## Version history
 
 - Added in version 3.13.0.

docs/mkdocs/docs/api/basic_json_document/parse.md

@@ -21,17 +21,18 @@ static basic_json_document parse(IteratorType first, IteratorType last,
 
 Both overloads accept the same JSON text as [`BasicJsonType::parse()`](../basic_json/parse.md), with the same
 `ignore_comments`/`ignore_trailing_commas` options, but build a [`basic_json_document`](index.md) (a flat index into
-the input) instead of a tree of `BasicJsonType` values. They take the same inputs as `BasicJsonType::parse()`. Byte
+the input) instead of a tree of `BasicJsonType` values. They take the same inputs as `BasicJsonType::parse()`, plus a
-inputs that stay alive can be borrowed; all others (streams, `#!cpp FILE*`, wide characters, non-contiguous
+pointer to a null-terminated wide string, which `BasicJsonType::parse()` does not accept. Byte inputs that stay alive
-iterators) are read into a copy that the document owns -- see the Notes. Use [`parse_copy()`](parse_copy.md) to get an
+can be borrowed; all others (streams, `#!cpp FILE*`, wide characters, non-contiguous iterators) are read into a copy
-owned copy of a borrowable input as well.
+that the document owns -- see the Notes. Use [`parse_copy()`](parse_copy.md) to get an owned copy of a borrowable input
+as well.
 
 ## Template parameters
 
 `InputType`
 :   A compatible input, for instance:
 
-    - a `#!cpp std::string`, `#!cpp std::string_view`, or a C-style array of single-byte characters
+    - a `#!cpp std::string`, a `#!cpp std::string_view` (C++17), or a C-style array of single-byte characters
     - a pointer to a null-terminated string of single-byte characters (`#!cpp char`, `#!cpp signed char`,
       `#!cpp unsigned char`, `#!cpp std::uint8_t`)
     - a container for which `#!cpp obj.data()` and `#!cpp obj.size()` give contiguous single-byte access, e.g.
@@ -47,6 +48,10 @@ owned copy of a borrowable input as well.
     `#!cpp std::string::iterator`; the iterators of single-byte characters are borrowed or read like the byte inputs
     above, those of wide characters are converted to UTF-8
 
+Ill-formed UTF-16 and UTF-32 sequences are rejected: a lone surrogate, a high surrogate that is not followed by its low
+surrogate, and a code point above `U+10FFFF` all yield a
+[`parse_error.101`](../../home/exceptions.md#jsonexceptionparse_error101).
+
 ## Parameters
 
 `input` (in)
@@ -155,3 +160,5 @@ buffer of a given length, pass a pair of pointers: `#!cpp parse(ptr, ptr + len)`
 ## Version history
 
 - Added in version 3.13.0.
+- Ill-formed UTF-16 and UTF-32 input (a lone surrogate, a high surrogate that is not followed by its low surrogate, or
+  a code point above `U+10FFFF`) is rejected with `parse_error.101`.

docs/mkdocs/docs/api/basic_json_view/index.md

@@ -143,6 +143,12 @@ still refers to it -- including ones taken before the change -- reads the new va
 
 - [**source_offset**](source_offset.md) - byte offset of this value in the document's source text
 
+## See also
+
+- [basic_json_document](../basic_json_document/index.md) - the document a view refers into
+- [materialize](materialize.md) - build the `BasicJsonType` value of a subtree
+- [Zero-copy JSON views](../../features/json_view.md) - the concept behind documents and views
+
 ## Version history
 
 - Added in version 3.13.0.

docs/mkdocs/docs/examples/accept__wide_string.cpp

@@ -0,0 +1,18 @@
+#include <iostream>
+#include <string>
+#include <nlohmann/json.hpp>
+
+using json = nlohmann::json;
+
+int main()
+{
+    // a valid JSON text, given as UTF-16
+    const std::u16string valid_text = u"{\"name\":\"\u00e9\"}";
+
+    // an invalid JSON text, given as UTF-16
+    const std::u16string invalid_text = u"{\"name\":\"\u00e9\",}";
+
+    std::cout << std::boolalpha
+              << json::accept(valid_text) << ' '
+              << json::accept(invalid_text) << '\n';
+}

docs/mkdocs/docs/examples/accept__wide_string.output

@@ -0,0 +1 @@
+true false

docs/mkdocs/docs/examples/parse__wide_string.cpp

@@ -0,0 +1,27 @@
+#include <iostream>
+#include <string>
+#include <nlohmann/json.hpp>
+
+using json = nlohmann::json;
+
+int main()
+{
+    // the same JSON text, given in different encodings
+    const std::string utf8 = "{\"name\":\"\u00e9\"}";
+    const std::u16string utf16 = u"{\"name\":\"\u00e9\"}";
+    const std::u32string utf32 = U"{\"name\":\"\u00e9\"}";
+    const std::wstring wide = L"{\"name\":\"\u00e9\"}";
+
+    // the encoding is deduced from the character type of the input, and all
+    // of them yield the same value
+    std::cout << json::parse(utf8) << '\n'
+              << json::parse(utf16) << '\n'
+              << json::parse(utf32) << '\n'
+              << json::parse(wide) << '\n';
+
+    // a wide string literal works as well: u"..." is UTF-16, U"..." is UTF-32
+    std::cout << json::parse(u"[1, 2, 3]") << '\n';
+
+    // iterators over wide characters work as well
+    std::cout << json::parse(utf16.begin(), utf16.end()) << '\n';
+}

docs/mkdocs/docs/examples/parse__wide_string.output

@@ -0,0 +1,6 @@
+{"name":"é"}
+{"name":"é"}
+{"name":"é"}
+{"name":"é"}
+[1,2,3]
+{"name":"é"}

docs/mkdocs/docs/features/conversions.md

@@ -38,6 +38,16 @@ which avoids repeating it:
     --8<-- "examples/get_to.output"
     ```
 
+!!! info "Getting values out of a document or a view"
+
+    A [view](../api/basic_json_view/index.md) offers the same `get<T>()` conversion, but reads it straight from the
+    document's index, without building a `basic_json` value first. Two conversions never copy at all:
+    [`get_string()`](../api/basic_json_view/get_string.md) returns a string as a view into the source text, and
+    [`number_token()`](../api/basic_json_view/number_token.md) returns a number exactly as it was written. Every other
+    type is obtained from a subtree that is built on demand with
+    [`materialize()`](../api/basic_json_view/materialize.md). See
+    [Zero-copy JSON views](json_view.md#getting-values-out-without-copying).
+
 The library already knows how to convert to and from the scalar types and the STL containers (such as
 `#!cpp std::vector`, `#!cpp std::map`, `#!cpp std::array`, `#!cpp std::optional`, and many more). Converting a JSON
 object back to a `#!cpp std::map` or a JSON array back to a `#!cpp std::vector` therefore works without any extra code:
@@ -234,3 +244,4 @@ The conversions above are built in for standard types. To make the same syntax w
 - [`get_ref`](../api/basic_json/get_ref.md) / [`get_ptr`](../api/basic_json/get_ptr.md) - access the stored value without copying
 - [Arbitrary types conversions](arbitrary_types.md) - support your own types
 - [`JSON_USE_IMPLICIT_CONVERSIONS`](../api/macros/json_use_implicit_conversions.md) - toggle implicit conversions
+- [Zero-copy JSON views](json_view.md) - get values out of a document or a view

docs/mkdocs/docs/features/creating_values.md

@@ -1,7 +1,9 @@
 # Creating JSON values
 
 There are several ways to create a JSON value in memory. This page gives an overview; to read a value from JSON text
-instead, see [parsing](parsing/index.md).
+instead, see [parsing](parsing/index.md). A JSON text can also be parsed into a
+[document](../api/basic_json_document/index.md) and read through a [view](../api/basic_json_view/index.md) instead of
+into a value; see [Zero-copy JSON views](json_view.md).
 
 ## From C++ values
 
@@ -101,3 +103,4 @@ string `#!json "42"`.
 - [`operator""_json`](../api/operator_literal_json.md) - the `_json` literal
 - [Converting values](conversions.md) - which C++ types can be used
 - [Parsing](parsing/index.md) - create a value from JSON text
+- [Zero-copy JSON views](json_view.md) - create a document and a view from JSON text instead of a value

docs/mkdocs/docs/features/macros.md

@@ -287,6 +287,23 @@ external dependency and is not defined by default.
 
 See [full documentation of `JSON_USE_SIMDUTF`](../api/macros/json_use_simdutf.md).
 
+## `JSON_VIEW_NO_SIMD`
+
+When defined, the parser of [`basic_json_document`](../api/basic_json_document/index.md)
+(`<nlohmann/json_view.hpp>`) uses only portable C++ to scan strings, instead of vector instructions (SSE2 on x86-64,
+NEON on AArch64). The same input is accepted or rejected either way; only the speed differs. It is not defined by
+default.
+
+See [full documentation of `JSON_VIEW_NO_SIMD`](../api/macros/json_view_no_simd.md).
+
+## `JSON_VIEW_USE_SSSE3`
+
+When defined on x86-64, the parser of [`basic_json_document`](../api/basic_json_document/index.md)
+(`<nlohmann/json_view.hpp>`) validates non-ASCII text in strings with SSSE3 without asking the CPU first. It is not
+defined by default.
+
+See [full documentation of `JSON_VIEW_USE_SSSE3`](../api/macros/json_view_use_ssse3.md).
+
 ## `NLOHMANN_DEFINE_TYPE_*(...)`, `NLOHMANN_DEFINE_DERIVED_TYPE_*(...)`
 
 The library defines 12 macros to simplify the serialization/deserialization of types. See the page on

docs/mkdocs/docs/features/modifying_values.md

@@ -85,6 +85,16 @@ json a = {1, 2, 3, 4};
 a.erase(1);             // [1,3,4]  (erase by index)
 ```
 
+## Editing a document
+
+The operations above work on a `basic_json` value. A JSON text that was parsed into a
+[document](json_view.md#editing-a-document) can also be changed in place, without building a `basic_json` tree for the
+parts that are not touched: [`set`](../api/basic_json_document/set.md) replaces a value or an object member,
+[`push_back`](../api/basic_json_document/push_back.md) appends to an array,
+[`insert`](../api/basic_json_document/insert.md) inserts into an array, and
+[`erase`](../api/basic_json_document/erase.md) removes a member or an element. These functions are available on the
+editable document types, such as [`json_editable_document`](../api/json_editable_document.md).
+
 ## See also
 
 - [`push_back`](../api/basic_json/push_back.md) / [`emplace_back`](../api/basic_json/emplace_back.md) - append to an array
@@ -93,3 +103,4 @@ a.erase(1);             // [1,3,4]  (erase by index)
 - [`merge_patch`](../api/basic_json/merge_patch.md) - apply an RFC 7396 merge patch
 - [`erase`](../api/basic_json/erase.md) / [`clear`](../api/basic_json/clear.md) - remove elements
 - [JSON Patch and Diff](json_patch.md) and [JSON Merge Patch](merge_patch.md) - structured modifications
+- [Editing a document](json_view.md#editing-a-document) - change a parsed document in place

docs/mkdocs/docs/features/parsing/index.md

@@ -18,6 +18,7 @@ flowchart LR
 The [`parse`](../../api/basic_json/parse.md) function reads a JSON value from an input. The input can be
 
 - a string (`#!cpp std::string`, C string, or string literal),
+- a wide string (`#!cpp std::u16string`, `#!cpp std::u32string`, or `#!cpp std::wstring`, or a wide string literal),
 - a `#!cpp std::istream` (e.g., an `#!cpp std::ifstream` reading from a file),
 - a `#!cpp FILE*` pointer,
 - a pair of iterators over a contiguous range (e.g., a `#!cpp std::vector<std::uint8_t>`), or
@@ -32,12 +33,18 @@ std::ifstream f("example.json");
 json data = json::parse(f);
 ```
 
-The input must be encoded in UTF-8; other encodings are not supported. A single input may contain only one JSON value.
+The input must be encoded in UTF-8; other encodings are not supported. The exception is wide-character input, which
+is read as UTF-16 or UTF-32 -- depending on the character type -- and converted to UTF-8; see the
+[`parse`](../../api/basic_json/parse.md) page for an example. A single input may contain only one JSON value.
 Inputs consisting of multiple values separated by newlines are handled by the [JSON Lines](json_lines.md) format.
 
 By default, the library rejects comments and trailing commas. Both can be enabled with parameters of the `parse`
 function — see [comments](../comments.md) and [trailing commas](../trailing_commas.md).
 
+[`basic_json_document::parse`](../../api/basic_json_document/parse.md) accepts the same inputs with the same options,
+but keeps the result in a document rather than in a `basic_json` value; see
+[Documents and views](#documents-and-views) below.
+
 ## Strictness and trailing data
 
 [`parse`](../../api/basic_json/parse.md) reads a single JSON value and requires the whole input to be consumed: any
@@ -98,6 +105,24 @@ The library offers two parsing models:
 You can influence a DOM parse without switching to the SAX interface by passing a
 [parser callback](parser_callbacks.md), which is called during parsing and can, for example, discard parts of the input.
 
+## Documents and views
+
+A third model sits between DOM and SAX: [`basic_json_document::parse`](../../api/basic_json_document/parse.md) parses
+the input into a document that keeps the text and a flat index of the values in it, and
+[`basic_json_view`](../../api/basic_json_view/index.md) gives read-only access to the values through that index.
+Strings and numbers stay in the source text and are only decoded when they are read, which is faster and uses less
+memory than building a tree of `basic_json` values when only parts of the document are needed. A document can also be
+modified in place, and it can be saved and loaded again without parsing it.
+
+```cpp
+#include <nlohmann/json_view.hpp>
+
+json_document doc = json_document::parse(R"({"happy": true, "pi": 3.141})");
+std::cout << doc.root()["happy"].get<bool>() << '\n';
+```
+
+See [Zero-copy JSON views](../json_view.md) for the concept and the trade-offs.
+
 ## Exceptions
 
 When the input is not valid JSON, the `parse` function throws an exception by default. If exceptions are undesired or
@@ -108,6 +133,7 @@ options.
 ## See also
 
 - [`parse`](../../api/basic_json/parse.md) - deserialize from a compatible input
+- [`basic_json_document::parse`](../../api/basic_json_document/parse.md) - deserialize into a document instead of a value
 - [`accept`](../../api/basic_json/accept.md) - check if the input is valid JSON
 - [`sax_parse`](../../api/basic_json/sax_parse.md) - generate SAX events
 - [JSON Lines](json_lines.md) - parse newline-delimited JSON
@@ -115,3 +141,4 @@ options.
 - [SAX interface](sax_interface.md) - implement a custom SAX handler
 - [parsing and exceptions](parse_exceptions.md) - control error handling
 - [parsing untrusted input](untrusted_input.md) - what to consider when parsing input from untrusted sources
+- [Zero-copy JSON views](../json_view.md) - parse into a document and read it through a view

docs/mkdocs/docs/features/parsing/untrusted_input.md

@@ -81,6 +81,9 @@ Invalid UTF-8 is rejected while parsing, not just while serializing:
 
 - In JSON text, an ill-formed UTF-8 byte in a string is a
   [`parse_error.101`](../../home/exceptions.md#jsonexceptionparse_error101) ("invalid string: ill-formed UTF-8 byte").
+- In UTF-16 or UTF-32 input, an ill-formed sequence is a
+  [`parse_error.101`](../../home/exceptions.md#jsonexceptionparse_error101) as well: a lone surrogate, a high surrogate
+  that is not followed by its low surrogate, and a code point above `U+10FFFF` are all rejected.
 - In a binary format, a string that is not valid UTF-8 is a
   [`parse_error.113`](../../home/exceptions.md#jsonexceptionparse_error113).
 

docs/mkdocs/docs/home/debugging.md

@@ -1,7 +1,7 @@
 # Debugging
 
-This page collects the library's built-in debugger integrations and other debugging-related features. They are
+This page collects the library's built-in debugger integrations and other debugging-related features. They are not
-not linked from a single place elsewhere in the docs, so are collected here.
+covered by any single page elsewhere in the docs, so they are collected here.
 
 ## Visual Studio (natvis)
 

docs/mkdocs/docs/home/exceptions.md

@@ -99,7 +99,8 @@ As this global context comes at the price of storing one additional pointer per
 
     Now the exception message contains a JSON Pointer `/address/housenumber` that indicates which value has the wrong type.
 
-See [documentation of `JSON_DIAGNOSTICS`](../api/macros/json_diagnostics.md) for more information.
+See [documentation of `JSON_DIAGNOSTICS`](../api/macros/json_diagnostics.md) for more information, and
+[Debugging](debugging.md) for the debugger integrations of the library.
 
 ## Parse errors
 

docs/mkdocs/docs/home/faq.md

@@ -81,8 +81,9 @@ rejected by default, and the library never adds them when serializing.
 
 The library supports **Unicode input** as follows:
 
-- Only **UTF-8** encoded input is supported, which is the default encoding for JSON, according to [RFC 8259](https://tools.ietf.org/html/rfc8259.html#section-8.1).
+- **UTF-8** encoded input is supported, which is the default encoding for JSON, according to [RFC 8259](https://tools.ietf.org/html/rfc8259.html#section-8.1).
-- `std::u16string` and `std::u32string` can be parsed, assuming UTF-16 and UTF-32 encoding, respectively. These encodings are not supported when reading from files or other input containers.
+- `std::u16string` and `std::u32string` can be parsed, assuming UTF-16 and UTF-32 encoding, respectively; `std::wstring` works as well and is interpreted as UTF-16 or UTF-32, depending on the width of the platform's `wchar_t`. These encodings cannot be read from a `FILE*` or an `std::istream`, which carry bytes only.
+- Ill-formed UTF-16 or UTF-32 input is rejected: a lone surrogate, a high surrogate that is not followed by its low surrogate, and a code point above `U+10FFFF` all yield a parse error. The same holds for iterators over 2- or 4-byte characters.
 - Other encodings such as Latin-1 or ISO 8859-1 are **not** supported and will yield parse or serialization errors.
 - The library will not replace [Unicode noncharacters](http://www.unicode.org/faq/private_use.html#nonchar1).
 - Invalid surrogates (e.g., incomplete pairs such as `\uDEAD`) will yield parse errors.
@@ -146,7 +147,7 @@ json::parse(std::string("\"") + '\0' + "\""); // throws parse_error.101: control
 
     Why are wide strings (e.g., `std::wstring`) dumped as arrays of numbers?
 
-As described [above](#parse-errors-reading-non-ascii-characters), the library assumes UTF-8 as encoding.  To store a wide string, you need to change the encoding.
+As described [above](#parse-errors-reading-non-ascii-characters), the library stores its strings as UTF-8. To store a wide string, you need to change the encoding.
 
 !!! example
 

docs/mkdocs/docs/index.md

@@ -6,7 +6,9 @@ JSON for Modern C++ is a header-only C++11 library that turns JSON into a first-
 magic of modern C++ so that creating, reading, and modifying JSON values feels as natural as it does in languages like
 Python. The whole library is available as a single header, `json.hpp`, with no dependencies, no subproject, and no
 complex build system to set up; a companion header, `json_fwd.hpp`, provides forward declarations to keep compile times
-down. See [header-only integration](integration/index.md) for details. It is heavily unit-tested with 100% code
+down, `<nlohmann/json_literals.hpp>` holds the user-defined string literals on their own, and
+`<nlohmann/json_view.hpp>` adds zero-copy views and editable documents. See
+[header-only integration](integration/index.md) for details. It is heavily unit-tested with 100% code
 coverage, checked with Valgrind and the Clang Sanitizers for memory leaks, and continuously fuzz-tested by Google
 OSS-Fuzz.
 
@@ -58,7 +60,8 @@ Conda, and more), and pkg-config.
 
     ---
 
-    Creating, parsing, accessing, and serializing JSON values, JSON Pointer/Patch, binary formats, and more.
+    Creating, parsing, accessing, and serializing JSON values, JSON Pointer/Patch, binary formats, zero-copy views,
+    and more.
 
     [:octicons-arrow-right-24: Features](features/index.md)
 
@@ -74,7 +77,8 @@ Conda, and more), and pkg-config.
 
     ---
 
-    The complete reference for `basic_json` and its member functions, types, and related classes.
+    The complete reference for `basic_json` and its member functions, types, and related classes, as well as for
+    `json_document` and `json_view`.
 
     [:octicons-arrow-right-24: API documentation](api/basic_json/index.md)
 

include/nlohmann/detail/view/serializer.hpp

@@ -23,7 +23,6 @@
 #include <nlohmann/detail/view/macro_scope.hpp>
 #include <nlohmann/detail/view/node.hpp>
 #include <nlohmann/detail/view/number.hpp>
-#include <nlohmann/detail/view/simd.hpp>
 
 NLOHMANN_JSON_NAMESPACE_BEGIN
 namespace detail
@@ -81,6 +80,13 @@ class output_buffer
         put(s, N - 1);
     }
 
+    void put_repeated(char c, std::size_t n)
+    {
+        reserve(n);
+        std::memset(m_pos, c, n);
+        m_pos += n;
+    }
+
     /// the write position and the end of the writable space, for a writer
     /// that keeps the position in a local variable (set_cursor() hands it back)
     char* cursor() const noexcept
@@ -137,25 +143,9 @@ class output_buffer
 /// image_check::bounds can have others, which are then written unchanged).
 inline std::size_t plain_output_run(const unsigned char* s, std::size_t n) noexcept
 {
-    std::size_t i = 0;
-#if NLOHMANN_VIEW_SSE2
-    // (16 bytes per step; with NEON, the mask of a step costs more than it saves on the
-    // short runs that are typical here)
-    for (; i + 16 <= n; i += 16)
-    {
-        const __m128i in = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(s + i))); // NOLINT(bugprone-casting-through-void)
-        // (a byte is below 0x20 if it is its own minimum with 0x1F)
-        const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('"')), _mm_cmpeq_epi8(in, _mm_set1_epi8('\\'))),
-                                             _mm_cmpeq_epi8(_mm_min_epu8(in, _mm_set1_epi8(0x1F)), in));
-        const auto mask = static_cast<std::uint64_t>(static_cast<unsigned>(_mm_movemask_epi8(special)));
-        if (mask != 0)
-        {
-            return i + static_cast<std::size_t>(count_trailing_zeros(mask));
-        }
-    }
-#endif
     constexpr std::uint64_t ones = 0x0101010101010101ull;
     constexpr std::uint64_t high = 0x8080808080808080ull;
+    std::size_t i = 0;
     for (; i + 8 <= n; i += 8)
     {
         const std::uint64_t v = read_eight_bytes(s + i);
@@ -206,70 +196,6 @@ inline bool is_ascii_without_delete(const char* s, std::size_t n) noexcept
     return true;
 }
 
-#if NLOHMANN_VIEW_VECTOR
-/// the bytes of the 16 at p that ensure_ascii cannot copy (a quote, a
-/// backslash, a control character, DEL, or not ASCII; a signed compare with
-/// 0x20 finds control characters and non-ASCII bytes at once), one bit per
-/// byte (x86) or one nibble per byte (NEON)
-NLOHMANN_VIEW_ALWAYS_INLINE std::uint64_t ascii_stop_mask(const unsigned char* p) noexcept
-{
-#if NLOHMANN_VIEW_NEON
-    const uint8x16_t in = vld1q_u8(p);
-    const uint8x16_t special = vorrq_u8(vorrq_u8(vceqq_u8(in, vdupq_n_u8('"')), vceqq_u8(in, vdupq_n_u8('\\'))),
-                                        vorrq_u8(vceqq_u8(in, vdupq_n_u8(0x7F)), vcltq_s8(vreinterpretq_s8_u8(in), vdupq_n_s8(0x20))));
-    return vget_lane_u64(vreinterpret_u64_u8(vshrn_n_u16(vreinterpretq_u16_u8(special), 4)), 0);
-#else
-    const __m128i in = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(p))); // NOLINT(bugprone-casting-through-void)
-    const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('"')), _mm_cmpeq_epi8(in, _mm_set1_epi8('\\'))),
-                                         _mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8(0x7F)), _mm_cmplt_epi8(in, _mm_set1_epi8(0x20))));
-    return static_cast<std::uint64_t>(static_cast<unsigned>(_mm_movemask_epi8(special)));
-#endif
-}
-
-/// the width in bits of a byte in the mask of ascii_stop_mask()
-constexpr unsigned ascii_stop_bits() noexcept
-{
-    return NLOHMANN_VIEW_NEON ? 4u : 1u;
-}
-#endif
-
-/// As find_ascii_copyable_run(), 16 bytes per step. end is the end of the
-/// buffer that holds the n bytes at p (a short run is read as 16 bytes if
-/// they are in the buffer).
-inline std::size_t ascii_copyable_run(const unsigned char* p, std::size_t n, const unsigned char* end) noexcept
-{
-#if NLOHMANN_VIEW_VECTOR
-    if (n >= 16)
-    {
-        std::size_t i = 0;
-        for (; i + 16 <= n; i += 16)
-        {
-            const std::uint64_t mask = ascii_stop_mask(p + i);
-            if (mask != 0)
-            {
-                return i + (static_cast<std::size_t>(count_trailing_zeros(mask)) / ascii_stop_bits());
-            }
-        }
-        if (i == n)
-        {
-            return n;
-        }
-        // the last 16 bytes: those before i are known to be copyable
-        const std::uint64_t mask = ascii_stop_mask(p + n - 16);
-        return mask != 0 ? n - 16 + (static_cast<std::size_t>(count_trailing_zeros(mask)) / ascii_stop_bits()) : n;
-    }
-    if (end - p >= 16)
-    {
-        // (the bytes after the run are masked out)
-        const std::uint64_t mask = ascii_stop_mask(p) & ((std::uint64_t{1} << (n * ascii_stop_bits())) - 1u);
-        return mask != 0 ? static_cast<std::size_t>(count_trailing_zeros(mask)) / ascii_stop_bits() : n;
-    }
-#else
-    static_cast<void>(end);
-#endif
-    return find_ascii_copyable_run(p, n);
-}
-
 /// A stack that starts in a buffer of the caller (a local array) and moves to
 /// the heap (a vector of the caller) only when that is full, so that dumps of
 /// shallow documents need no allocation. The top is a pointer, as in
@@ -349,39 +275,6 @@ NLOHMANN_VIEW_ALWAYS_INLINE char* put_literal(char* w, const char (&s)[N]) noexc
     return w + N - 1;
 }
 
-/// the depth of the innermost open container, counted only for indented output
-template<bool Counted>
-struct depth_counter
-{
-    std::size_t value = 0;
-    void enter() noexcept
-    {
-        ++value;
-    }
-    std::size_t leave() noexcept
-    {
-        return --value;
-    }
-    std::size_t get() const noexcept
-    {
-        return value;
-    }
-};
-
-template<>
-struct depth_counter<false>
-{
-    void enter() noexcept {}
-    std::size_t leave() noexcept
-    {
-        return 0;
-    }
-    std::size_t get() const noexcept
-    {
-        return 0;
-    }
-};
-
 /// how the view's dump() writes a value
 struct dump_style
 {
@@ -418,95 +311,139 @@ class view_serializer
 
     void dump(const node* root)
     {
-        if (m_style.ensure_ascii)
+        if (!m_style.pretty)
         {
-            dump_ascii(root);
+            if (m_style.ensure_ascii)
+            {
+                dump_compact_ascii(root);
+            }
+            else if (m_style.source_numbers)
+            {
+                dump_compact<true, false>(root);
+            }
+            else
+            {
+                dump_compact<false, false>(root);
+            }
+            return;
         }
-        else if (m_style.pretty)
+        struct frame
         {
-            dump_pretty(root);
+            const node* pos; ///< next element, or key of the next member
-        }
+            const node* end;
-        else if (m_style.source_numbers)
+            bool object;
+            bool first;  ///< nothing written yet
+        };
+        std::array<frame, 32> buffer; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
+        std::vector<frame> heap;
+        small_stack<frame> stack(buffer.data(), buffer.size(), heap);
+        const node* n = root;
+        for (;;)
         {
-            dump_text<true, false, false>(root);
+            // write the value at n
-        }
+            if (is_container(*n))
-        else
+            {
-        {
+                const bool object = n->kind == static_cast<std::uint8_t>(value_t::object);
-            dump_text<false, false, false>(root);
+                if (n->len == 0)
+                {
+                    m_out.put(object ? "{}" : "[]", 2);
+                }
+                else
+                {
+                    m_out.put(object ? '{' : '[');
+                    stack.push_back(frame{nav::first(m_doc, n), nav::end(m_doc, n), object, true});
+                }
+            }
+            else
+            {
+                write_scalar(*n);
+            }
+
+            // go to the next value: close finished containers, then separate
+            for (;;)
+            {
+                if (stack.empty())
+                {
+                    m_out.finish();
+                    return;
+                }
+                if (stack.back().pos == stack.back().end)
+                {
+                    const bool object = stack.back().object;
+                    stack.pop_back();
+                    newline(stack.size());
+                    m_out.put(object ? '}' : ']');
+                    continue;
+                }
+                frame& f = stack.back();
+                if (!f.first)
+                {
+                    m_out.put(',');
+                }
+                f.first = false;
+                newline(stack.size());
+                if (f.object)
+                {
+                    write_string(*f.pos);
+                    if (m_style.pretty)
+                    {
+                        m_out.put(": ");
+                    }
+                    else
+                    {
+                        m_out.put(':');
+                    }
+                    n = nav::value(f.pos + 1);
+                    f.pos = document_data::after(f.pos + 1);
+                }
+                else
+                {
+                    n = nav::value(f.pos);
+                    f.pos = document_data::after(f.pos);
+                }
+                break;
+            }
         }
     }
 
   private:
     /// a string that is not a plain string of the source (decoded, or written
     /// by an edit), without ensure_ascii: runs without characters to escape
-    /// are copied. If validate is set, the string must be valid UTF-8: the
+    /// are copied
-    /// runs are found by the lexer's scan, which validates the sequences, and
+    NLOHMANN_VIEW_NOINLINE void write_decoded(const node& n)
-    /// invalid UTF-8 throws what basic_json::dump() throws. (The compact output
-    /// writes the bytes of a damaged image unchanged, and scans without
-    /// validating them.)
-    NLOHMANN_VIEW_NOINLINE void write_decoded(const node& n, bool validate)
     {
         const auto* const s = reinterpret_cast<const unsigned char*>(m_doc.str(n)); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
         m_out.put('"');
         for (std::size_t i = 0; i < n.len;)
         {
-            const std::size_t run = validate ? string_bulk_run(s + i, n.len - i) : plain_output_run(s + i, n.len - i);
+            const std::size_t run = plain_output_run(s + i, n.len - i);
             if (run != 0)
             {
                 m_out.put(reinterpret_cast<const char*>(s + i), run); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
                 i += run;
                 continue;
             }
-            if (validate && s[i] >= 0x80)
-            {
-                invalid_utf8(s, n.len); // (the scan stopped at ill-formed UTF-8)
-            }
             write_codepoint<false>(s[i], s + i, 1); // a quote, a backslash, or a control character
             ++i;
         }
         m_out.put('"');
     }
 
-    /// dump_text() with indentation, out of line: the compact walk is faster
+    /// dump_compact() with ensure_ascii, out of line: the walk without it is
-    /// when it is not merged with the indented ones into dump()
+    /// faster when it is not merged with this one into dump()
-    NLOHMANN_VIEW_NOINLINE void dump_pretty(const node* root)
+    NLOHMANN_VIEW_NOINLINE void dump_compact_ascii(const node* root)
     {
         if (m_style.source_numbers)
         {
-            dump_text<true, true, false>(root);
+            dump_compact<true, true>(root);
         }
         else
         {
-            dump_text<false, true, false>(root);
+            dump_compact<false, true>(root);
         }
     }
 
-    /// dump_text() with ensure_ascii, out of line: the walk without it is
+    /// whether dump_compact() copies a string or key from the source: one of
-    /// faster when it is not merged with this one into dump()
-    NLOHMANN_VIEW_NOINLINE void dump_ascii(const node* root)
-    {
-        if (m_style.pretty)
-        {
-            if (m_style.source_numbers)
-            {
-                dump_text<true, true, true>(root);
-            }
-            else
-            {
-                dump_text<false, true, true>(root);
-            }
-        }
-        else if (m_style.source_numbers)
-        {
-            dump_text<true, false, true>(root);
-        }
-        else
-        {
-            dump_text<false, false, true>(root);
-        }
-    }
-
-    /// whether dump_text() copies a string or key from the source: one of
     /// the source, and with ensure_ascii one without bytes >= 0x7F (an overload
     /// each, so that the walk without ensure_ascii is compiled exactly as
     /// before: with GCC, a condition that only folds away later changed its
@@ -521,154 +458,13 @@ class view_serializer
         return (n.flags & node_flags::storage) == 0 && is_ascii_without_delete(src + n.off, n.len);
     }
 
-    /*!
+    /// a string with ensure_ascii (dump_compact()), out of line
-    @brief a string with ensure_ascii: the characters that are not printable ASCII as \\uXXXX
+    NLOHMANN_VIEW_NOINLINE void write_ascii_string(const node& n)
-    (a character beyond U+FFFF as a surrogate pair); strings of a document
-    are valid UTF-8, except in a damaged image loaded with image_check::bounds,
-    for which this throws what basic_json::dump() throws for the string
-    */
-    NLOHMANN_VIEW_NOINLINE void write_ascii(const node& n)
     {
-        const auto* const s = reinterpret_cast<const unsigned char*>(m_doc.str(n)); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
+        write_string(n);
-        const std::size_t size = n.len;
-        char* w = m_out.cursor();
-        char* lim = m_out.limit();
-        // room for k bytes
-        const auto room = [&](std::size_t k)
-        {
-            if (NLOHMANN_VIEW_UNLIKELY(static_cast<std::size_t>(lim - w) < k))
-            {
-                m_out.set_cursor(w);
-                m_out.reserve(k);
-                w = m_out.cursor();
-                lim = m_out.limit();
-            }
-        };
-        static constexpr const char* hex = "0123456789abcdef";
-        // \\uXXXX
-        const auto u_escape = [&](std::uint32_t u)
-        {
-            w[0] = '\\';
-            w[1] = 'u';
-            w[2] = hex[(u >> 12u) & 0xFu];
-            w[3] = hex[(u >> 8u) & 0xFu];
-            w[4] = hex[(u >> 4u) & 0xFu];
-            w[5] = hex[u & 0xFu];
-            w += 6;
-        };
-        room(1);
-        *w++ = '"';
-        std::size_t i = 0;
-        // (a string of the source starts with a run that has nothing to escape, 16 bytes per step)
-        const std::size_t prefix = (n.flags & node_flags::storage) == 0 ? ascii_copyable_run(s, size, reinterpret_cast<const unsigned char*>(m_doc.src + m_doc.size)) : 0; // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
-        if (prefix != 0)
-        {
-            room(prefix);
-            std::memcpy(w, s, prefix);
-            w += prefix;
-            i = prefix;
-        }
-        while (i < size)
-        {
-            // (a character that needs an escape sequence is not worth a call)
-            const std::size_t run = is_ascii_copyable(s[i]) ? find_ascii_copyable_run(s + i, size - i) : 0;
-            if (run != 0)
-            {
-                room(run);
-                std::memcpy(w, s + i, run);
-                w += run;
-                i += run;
-                continue;
-            }
-            room(12);
-            if (s[i] >= 0x80)
-            {
-                const unsigned char lead = s[i];
-                // (the two-byte sequences of Latin text and the three-byte sequences of most other
-                // text, with continuation bytes of any value: the ranges that are restricted (E0,
-                // ED, and the sequences of four bytes) go through validate_one_utf8())
-                if (lead >= 0xE1 && lead <= 0xEF && lead != 0xED && size - i >= 3 && (s[i + 1] & 0xC0u) == 0x80 && (s[i + 2] & 0xC0u) == 0x80)
-                {
-                    u_escape(((lead & 0x0Fu) << 12u) | ((s[i + 1] & 0x3Fu) << 6u) | (s[i + 2] & 0x3Fu));
-                    i += 3;
-                    continue;
-                }
-                if (lead >= 0xC2 && lead <= 0xDF && size - i >= 2 && (s[i + 1] & 0xC0u) == 0x80)
-                {
-                    u_escape(((lead & 0x1Fu) << 6u) | (s[i + 1] & 0x3Fu));
-                    i += 2;
-                    continue;
-                }
-                const std::size_t len = validate_one_utf8(s + i, size - i);
-                if (NLOHMANN_VIEW_UNLIKELY(len == 0))
-                {
-                    m_out.set_cursor(w);
-                    invalid_utf8(s, size);
-                    return; // LCOV_EXCL_LINE (invalid_utf8 throws)
-                }
-                std::uint32_t codepoint = s[i] & (0xFFu >> (len + 1));
-                for (std::size_t k = 1; k < len; ++k)
-                {
-                    codepoint = (codepoint << 6u) | (s[i + k] & 0x3Fu);
-                }
-                i += len;
-                if (codepoint <= 0xFFFF)
-                {
-                    u_escape(codepoint);
-                }
-                else
-                {
-                    u_escape(0xD7C0u + (codepoint >> 10u));
-                    u_escape(0xDC00u + (codepoint & 0x3FFu));
-                }
-                continue;
-            }
-            // a quote, a backslash, or a control character (or DEL)
-            const unsigned char c = s[i++];
-            char escape = 0;
-            switch (c)
-            {
-                case 0x08:
-                    escape = 'b';
-                    break;
-                case 0x09:
-                    escape = 't';
-                    break;
-                case 0x0A:
-                    escape = 'n';
-                    break;
-                case 0x0C:
-                    escape = 'f';
-                    break;
-                case 0x0D:
-                    escape = 'r';
-                    break;
-                case 0x22:
-                    escape = '"';
-                    break;
-                case 0x5C:
-                    escape = '\\';
-                    break;
-                default:
-                    break;
-            }
-            if (escape != 0)
-            {
-                w[0] = '\\';
-                w[1] = escape;
-                w += 2;
-            }
-            else
-            {
-                u_escape(c);
-            }
-        }
-        room(1);
-        *w++ = '"';
-        m_out.set_cursor(w);
     }
 
-    /// the copies of dump_text() that are not fixed-size moves (long
+    /// the copies of dump_compact() that are not fixed-size moves (long
     /// strings, or near the end of the source); out of line, so that the
     /// compiler does not merge the fixed-size moves into this call
     NLOHMANN_VIEW_NOINLINE static void copy_long(char* to, const char* from, std::size_t n) noexcept
@@ -685,7 +481,7 @@ class view_serializer
     }
 
     /*!
-    @brief the output, compact (the default dump()) or indented, with or without ensure_ascii
+    @brief the compact output (the default dump(), or with ensure_ascii)
 
     The same walk as dump(), with the write position in a local variable
     (stores through char pointers would otherwise force a reload of the
@@ -693,11 +489,11 @@ class view_serializer
     the source copied by fixed-size moves of 32 bytes where the source has
     that many bytes left, instead of a library call per token. The buffer
     keeps 64 bytes of slack for the overshoot. With ensure_ascii (Ascii), a
-    string of the source is copied only if it has nothing to escape (see
+    string of the source is copied only if it has no byte >= 0x7F (see
-    copied()); any other string is written by write_ascii().
+    is_ascii_without_delete()); any other string is written by write_string().
     */
-    template<bool SourceNumbers, bool Pretty, bool Ascii>
+    template<bool SourceNumbers, bool Ascii>
-    void dump_text(const node* root)
+    void dump_compact(const node* root)
     {
         struct frame
         {
@@ -712,16 +508,6 @@ class view_serializer
         const char* const src_end = src + m_doc.size;
         char* w = m_out.cursor();
         char* lim = m_out.limit();
-        // (indented output) a line break and 63 indentation characters, from
-        // which the line break of a level of up to 63 characters is a copy
-        std::array<char, 64> ruler; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): filled below if used
-        if (Pretty)
-        {
-            ruler.fill(m_style.indent_char);
-            ruler[0] = '\n';
-        }
-        const std::size_t indent = m_style.indent;
-        depth_counter<Pretty> level; // depth of the innermost open container (indented output)
         // room for n bytes and the slack
         const auto room = [&](std::size_t n)
         {
@@ -753,22 +539,6 @@ class view_serializer
             }
             w += n;
         };
-        // (indented output) the line break and the indentation of a level
-        const auto newline = [&](std::size_t depth)
-        {
-            const std::size_t count = 1 + (depth * indent);
-            room(count);
-            if (count <= ruler.size())
-            {
-                std::memcpy(w, ruler.data(), ruler.size());
-            }
-            else
-            {
-                *w = '\n';
-                std::memset(w + 1, m_style.indent_char, count - 1);
-            }
-            w += count;
-        };
         // a string that is not a plain string of the source (out of line, so
         // that the cursor stays in a register here)
         const auto escaped = [&](const node & n)
@@ -776,11 +546,11 @@ class view_serializer
             m_out.set_cursor(w);
             if (enabled(Ascii))
             {
-                write_ascii(n);
+                write_ascii_string(n);
             }
             else
             {
-                write_decoded(n, Pretty);
+                write_decoded(n);
             }
             w = m_out.cursor();
             lim = m_out.limit();
@@ -881,7 +651,6 @@ class view_serializer
                     {
                         *w++ = object ? '{' : '[';
                         stack.push_back(cur);
-                        level.enter();
                         if (Editable)
                         {
                             cur = frame{nav::first(m_doc, n), nav::end(m_doc, n), object};
@@ -924,10 +693,6 @@ class view_serializer
                     {
                         break;
                     }
-                    if (Pretty)
-                    {
-                        newline(level.leave());
-                    }
                     room(1);
                     *w++ = cur.object ? '}' : ']';
                     cur = stack.back();
@@ -936,17 +701,13 @@ class view_serializer
                 room(1);
                 *w++ = ',';
             }
-            if (Pretty)
-            {
-                newline(level.get());
-            }
             const node* const at = Editable ? cur.pos : n;
             if (cur.object)
             {
                 const node& key = *at;
                 if (copied(key, src, std::integral_constant<bool, Ascii>()))
                 {
-                    room(key.len + (Pretty ? 4 : 3));
+                    room(key.len + 3);
                     *w++ = '"';
                     copy(src + key.off, key.len);
                     w[0] = '"';
@@ -956,13 +717,9 @@ class view_serializer
                 else
                 {
                     escaped(key);
-                    room(Pretty ? 2 : 1);
+                    room(1);
                     *w++ = ':';
                 }
-                if (Pretty)
-                {
-                    *w++ = ' ';
-                }
                 if (Editable)
                 {
                     n = nav::value(at + 1);
@@ -981,6 +738,69 @@ class view_serializer
         }
     }
 
+    void newline(std::size_t level)
+    {
+        if (m_style.pretty)
+        {
+            m_out.put('\n');
+            m_out.put_repeated(m_style.indent_char, level * m_style.indent);
+        }
+    }
+
+    void write_scalar(const node& n)
+    {
+        switch (static_cast<value_t>(n.kind))
+        {
+            case value_t::null:
+                m_out.put("null");
+                break;
+            case value_t::boolean:
+                if ((n.flags & node_flags::is_true) != 0)
+                {
+                    m_out.put("true");
+                }
+                else
+                {
+                    m_out.put("false");
+                }
+                break;
+            case value_t::string:
+                write_string(n);
+                break;
+            case value_t::number_integer:
+            case value_t::number_unsigned:
+            {
+                const char* const token = m_doc.str(n);
+                const std::uint32_t len = number_length(n);
+                if (!m_style.source_numbers && len == 2 && token[0] == '-' && token[1] == '0')
+                {
+                    m_out.put('0'); // parse() reads -0 as the integer 0
+                }
+                else
+                {
+                    m_out.put(token, len);
+                }
+                break;
+            }
+            case value_t::number_float:
+                if (m_style.source_numbers && (n.flags & node_flags::storage) != node_flags::edited)
+                {
+                    m_out.put(m_doc.str(n), n.len); // (a float set by an edit is written as with shortest)
+                }
+                else
+                {
+                    write_float_node(n);
+                }
+                break;
+            case value_t::object:    // LCOV_EXCL_LINE (containers are written by dump())
+            case value_t::array:     // LCOV_EXCL_LINE
+            case value_t::binary:    // LCOV_EXCL_LINE (not in a document)
+            case value_t::discarded: // LCOV_EXCL_LINE
+            default:                 // LCOV_EXCL_LINE
+                break;               // LCOV_EXCL_LINE
+        }
+    }
+
     /// a float node as dump() writes it
     void write_float_node(const node& n)
     {
@@ -1091,6 +911,72 @@ class view_serializer
         m_out.put(s.data(), s.size());
     }
 
+    void write_string(const node& n)
+    {
+        const char* const s = m_doc.str(n);
+        m_out.put('"');
+        if ((n.flags & node_flags::storage) == 0 && (!m_style.ensure_ascii || is_ascii_without_delete(s, n.len)))
+        {
+            // a string of the source without escape sequences has nothing to
+            // escape (with ensure_ascii, if it is printable ASCII)
+            m_out.put(s, n.len);
+        }
+        else if (m_style.ensure_ascii)
+        {
+            write_escaped<true>(reinterpret_cast<const unsigned char*>(s), n.len); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
+        }
+        else
+        {
+            write_escaped<false>(reinterpret_cast<const unsigned char*>(s), n.len); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
+        }
+        m_out.put('"');
+    }
+
+    /// as serializer::dump_escaped(); strings of a document are valid UTF-8,
+    /// except in a damaged image loaded with image_check::bounds, for which
+    /// this throws what basic_json::dump() throws for the string
+    template<bool EnsureAscii>
+    void write_escaped(const unsigned char* s, std::size_t n)
+    {
+        std::size_t i = 0;
+        while (i < n)
+        {
+            std::size_t run = 0;
+            if (!EnsureAscii)
+            {
+                run = string_bulk_run(s + i, n - i);
+            }
+            else if (is_ascii_copyable(s[i]))
+            {
+                run = find_ascii_copyable_run(s + i, n - i);
+            }
+            if (run != 0)
+            {
+                m_out.put(reinterpret_cast<const char*>(s + i), run); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
+                i += run;
+                continue;
+            }
+            std::uint32_t codepoint = s[i];
+            std::size_t len = 1;
+            if (codepoint >= 0x80)
+            {
+                len = validate_one_utf8(s + i, n - i);
+                if (NLOHMANN_VIEW_UNLIKELY(len == 0))
+                {
+                    invalid_utf8(s, n);
+                    return;
+                }
+                codepoint &= 0xFFu >> (len + 1);
+                for (std::size_t k = 1; k < len; ++k)
+                {
+                    codepoint = (codepoint << 6u) | (s[i + k] & 0x3Fu);
+                }
+            }
+            write_codepoint<EnsureAscii>(codepoint, s + i, len);
+            i += len;
+        }
+    }
+
     /// throw what basic_json::dump() throws for a string that is not valid UTF-8
     NLOHMANN_VIEW_NOINLINE static void invalid_utf8(const unsigned char* s, std::size_t n)
     {

single_include/nlohmann/json_view.hpp

@@ -6089,8 +6089,6 @@ NLOHMANN_JSON_NAMESPACE_END
 
 // #include <nlohmann/detail/view/number.hpp>
 
-// #include <nlohmann/detail/view/simd.hpp>
-
 
 NLOHMANN_JSON_NAMESPACE_BEGIN
 namespace detail
@@ -6148,6 +6146,13 @@ class output_buffer
         put(s, N - 1);
     }
 
+    void put_repeated(char c, std::size_t n)
+    {
+        reserve(n);
+        std::memset(m_pos, c, n);
+        m_pos += n;
+    }
+
     /// the write position and the end of the writable space, for a writer
     /// that keeps the position in a local variable (set_cursor() hands it back)
     char* cursor() const noexcept
@@ -6204,25 +6209,9 @@ class output_buffer
 /// image_check::bounds can have others, which are then written unchanged).
 inline std::size_t plain_output_run(const unsigned char* s, std::size_t n) noexcept
 {
-    std::size_t i = 0;
-#if NLOHMANN_VIEW_SSE2
-    // (16 bytes per step; with NEON, the mask of a step costs more than it saves on the
-    // short runs that are typical here)
-    for (; i + 16 <= n; i += 16)
-    {
-        const __m128i in = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(s + i))); // NOLINT(bugprone-casting-through-void)
-        // (a byte is below 0x20 if it is its own minimum with 0x1F)
-        const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('"')), _mm_cmpeq_epi8(in, _mm_set1_epi8('\\'))),
-                                             _mm_cmpeq_epi8(_mm_min_epu8(in, _mm_set1_epi8(0x1F)), in));
-        const auto mask = static_cast<std::uint64_t>(static_cast<unsigned>(_mm_movemask_epi8(special)));
-        if (mask != 0)
-        {
-            return i + static_cast<std::size_t>(count_trailing_zeros(mask));
-        }
-    }
-#endif
     constexpr std::uint64_t ones = 0x0101010101010101ull;
     constexpr std::uint64_t high = 0x8080808080808080ull;
+    std::size_t i = 0;
     for (; i + 8 <= n; i += 8)
     {
         const std::uint64_t v = read_eight_bytes(s + i);
@@ -6273,70 +6262,6 @@ inline bool is_ascii_without_delete(const char* s, std::size_t n) noexcept
     return true;
 }
 
-#if NLOHMANN_VIEW_VECTOR
-/// the bytes of the 16 at p that ensure_ascii cannot copy (a quote, a
-/// backslash, a control character, DEL, or not ASCII; a signed compare with
-/// 0x20 finds control characters and non-ASCII bytes at once), one bit per
-/// byte (x86) or one nibble per byte (NEON)
-NLOHMANN_VIEW_ALWAYS_INLINE std::uint64_t ascii_stop_mask(const unsigned char* p) noexcept
-{
-#if NLOHMANN_VIEW_NEON
-    const uint8x16_t in = vld1q_u8(p);
-    const uint8x16_t special = vorrq_u8(vorrq_u8(vceqq_u8(in, vdupq_n_u8('"')), vceqq_u8(in, vdupq_n_u8('\\'))),
-                                        vorrq_u8(vceqq_u8(in, vdupq_n_u8(0x7F)), vcltq_s8(vreinterpretq_s8_u8(in), vdupq_n_s8(0x20))));
-    return vget_lane_u64(vreinterpret_u64_u8(vshrn_n_u16(vreinterpretq_u16_u8(special), 4)), 0);
-#else
-    const __m128i in = _mm_loadu_si128(static_cast<const __m128i*>(static_cast<const void*>(p))); // NOLINT(bugprone-casting-through-void)
-    const __m128i special = _mm_or_si128(_mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8('"')), _mm_cmpeq_epi8(in, _mm_set1_epi8('\\'))),
-                                         _mm_or_si128(_mm_cmpeq_epi8(in, _mm_set1_epi8(0x7F)), _mm_cmplt_epi8(in, _mm_set1_epi8(0x20))));
-    return static_cast<std::uint64_t>(static_cast<unsigned>(_mm_movemask_epi8(special)));
-#endif
-}
-
-/// the width in bits of a byte in the mask of ascii_stop_mask()
-constexpr unsigned ascii_stop_bits() noexcept
-{
-    return NLOHMANN_VIEW_NEON ? 4u : 1u;
-}
-#endif
-
-/// As find_ascii_copyable_run(), 16 bytes per step. end is the end of the
-/// buffer that holds the n bytes at p (a short run is read as 16 bytes if
-/// they are in the buffer).
-inline std::size_t ascii_copyable_run(const unsigned char* p, std::size_t n, const unsigned char* end) noexcept
-{
-#if NLOHMANN_VIEW_VECTOR
-    if (n >= 16)
-    {
-        std::size_t i = 0;
-        for (; i + 16 <= n; i += 16)
-        {
-            const std::uint64_t mask = ascii_stop_mask(p + i);
-            if (mask != 0)
-            {
-                return i + (static_cast<std::size_t>(count_trailing_zeros(mask)) / ascii_stop_bits());
-            }
-        }
-        if (i == n)
-        {
-            return n;
-        }
-        // the last 16 bytes: those before i are known to be copyable
-        const std::uint64_t mask = ascii_stop_mask(p + n - 16);
-        return mask != 0 ? n - 16 + (static_cast<std::size_t>(count_trailing_zeros(mask)) / ascii_stop_bits()) : n;
-    }
-    if (end - p >= 16)
-    {
-        // (the bytes after the run are masked out)
-        const std::uint64_t mask = ascii_stop_mask(p) & ((std::uint64_t{1} << (n * ascii_stop_bits())) - 1u);
-        return mask != 0 ? static_cast<std::size_t>(count_trailing_zeros(mask)) / ascii_stop_bits() : n;
-    }
-#else
-    static_cast<void>(end);
-#endif
-    return find_ascii_copyable_run(p, n);
-}
-
 /// A stack that starts in a buffer of the caller (a local array) and moves to
 /// the heap (a vector of the caller) only when that is full, so that dumps of
 /// shallow documents need no allocation. The top is a pointer, as in
@@ -6416,39 +6341,6 @@ NLOHMANN_VIEW_ALWAYS_INLINE char* put_literal(char* w, const char (&s)[N]) noexc
     return w + N - 1;
 }
 
-/// the depth of the innermost open container, counted only for indented output
-template<bool Counted>
-struct depth_counter
-{
-    std::size_t value = 0;
-    void enter() noexcept
-    {
-        ++value;
-    }
-    std::size_t leave() noexcept
-    {
-        return --value;
-    }
-    std::size_t get() const noexcept
-    {
-        return value;
-    }
-};
-
-template<>
-struct depth_counter<false>
-{
-    void enter() noexcept {}
-    std::size_t leave() noexcept
-    {
-        return 0;
-    }
-    std::size_t get() const noexcept
-    {
-        return 0;
-    }
-};
-
 /// how the view's dump() writes a value
 struct dump_style
 {
@@ -6485,95 +6377,139 @@ class view_serializer
 
     void dump(const node* root)
     {
-        if (m_style.ensure_ascii)
+        if (!m_style.pretty)
         {
-            dump_ascii(root);
+            if (m_style.ensure_ascii)
+            {
+                dump_compact_ascii(root);
+            }
+            else if (m_style.source_numbers)
+            {
+                dump_compact<true, false>(root);
+            }
+            else
+            {
+                dump_compact<false, false>(root);
+            }
+            return;
         }
-        else if (m_style.pretty)
+        struct frame
         {
-            dump_pretty(root);
+            const node* pos; ///< next element, or key of the next member
-        }
+            const node* end;
-        else if (m_style.source_numbers)
+            bool object;
+            bool first;  ///< nothing written yet
+        };
+        std::array<frame, 32> buffer; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
+        std::vector<frame> heap;
+        small_stack<frame> stack(buffer.data(), buffer.size(), heap);
+        const node* n = root;
+        for (;;)
         {
-            dump_text<true, false, false>(root);
+            // write the value at n
-        }
+            if (is_container(*n))
-        else
+            {
-        {
+                const bool object = n->kind == static_cast<std::uint8_t>(value_t::object);
-            dump_text<false, false, false>(root);
+                if (n->len == 0)
+                {
+                    m_out.put(object ? "{}" : "[]", 2);
+                }
+                else
+                {
+                    m_out.put(object ? '{' : '[');
+                    stack.push_back(frame{nav::first(m_doc, n), nav::end(m_doc, n), object, true});
+                }
+            }
+            else
+            {
+                write_scalar(*n);
+            }
+
+            // go to the next value: close finished containers, then separate
+            for (;;)
+            {
+                if (stack.empty())
+                {
+                    m_out.finish();
+                    return;
+                }
+                if (stack.back().pos == stack.back().end)
+                {
+                    const bool object = stack.back().object;
+                    stack.pop_back();
+                    newline(stack.size());
+                    m_out.put(object ? '}' : ']');
+                    continue;
+                }
+                frame& f = stack.back();
+                if (!f.first)
+                {
+                    m_out.put(',');
+                }
+                f.first = false;
+                newline(stack.size());
+                if (f.object)
+                {
+                    write_string(*f.pos);
+                    if (m_style.pretty)
+                    {
+                        m_out.put(": ");
+                    }
+                    else
+                    {
+                        m_out.put(':');
+                    }
+                    n = nav::value(f.pos + 1);
+                    f.pos = document_data::after(f.pos + 1);
+                }
+                else
+                {
+                    n = nav::value(f.pos);
+                    f.pos = document_data::after(f.pos);
+                }
+                break;
+            }
         }
     }
 
   private:
     /// a string that is not a plain string of the source (decoded, or written
     /// by an edit), without ensure_ascii: runs without characters to escape
-    /// are copied. If validate is set, the string must be valid UTF-8: the
+    /// are copied
-    /// runs are found by the lexer's scan, which validates the sequences, and
+    NLOHMANN_VIEW_NOINLINE void write_decoded(const node& n)
-    /// invalid UTF-8 throws what basic_json::dump() throws. (The compact output
-    /// writes the bytes of a damaged image unchanged, and scans without
-    /// validating them.)
-    NLOHMANN_VIEW_NOINLINE void write_decoded(const node& n, bool validate)
     {
         const auto* const s = reinterpret_cast<const unsigned char*>(m_doc.str(n)); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
         m_out.put('"');
         for (std::size_t i = 0; i < n.len;)
         {
-            const std::size_t run = validate ? string_bulk_run(s + i, n.len - i) : plain_output_run(s + i, n.len - i);
+            const std::size_t run = plain_output_run(s + i, n.len - i);
             if (run != 0)
             {
                 m_out.put(reinterpret_cast<const char*>(s + i), run); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
                 i += run;
                 continue;
             }
-            if (validate && s[i] >= 0x80)
-            {
-                invalid_utf8(s, n.len); // (the scan stopped at ill-formed UTF-8)
-            }
             write_codepoint<false>(s[i], s + i, 1); // a quote, a backslash, or a control character
             ++i;
         }
         m_out.put('"');
     }
 
-    /// dump_text() with indentation, out of line: the compact walk is faster
+    /// dump_compact() with ensure_ascii, out of line: the walk without it is
-    /// when it is not merged with the indented ones into dump()
+    /// faster when it is not merged with this one into dump()
-    NLOHMANN_VIEW_NOINLINE void dump_pretty(const node* root)
+    NLOHMANN_VIEW_NOINLINE void dump_compact_ascii(const node* root)
     {
         if (m_style.source_numbers)
         {
-            dump_text<true, true, false>(root);
+            dump_compact<true, true>(root);
         }
         else
         {
-            dump_text<false, true, false>(root);
+            dump_compact<false, true>(root);
         }
     }
 
-    /// dump_text() with ensure_ascii, out of line: the walk without it is
+    /// whether dump_compact() copies a string or key from the source: one of
-    /// faster when it is not merged with this one into dump()
-    NLOHMANN_VIEW_NOINLINE void dump_ascii(const node* root)
-    {
-        if (m_style.pretty)
-        {
-            if (m_style.source_numbers)
-            {
-                dump_text<true, true, true>(root);
-            }
-            else
-            {
-                dump_text<false, true, true>(root);
-            }
-        }
-        else if (m_style.source_numbers)
-        {
-            dump_text<true, false, true>(root);
-        }
-        else
-        {
-            dump_text<false, false, true>(root);
-        }
-    }
-
-    /// whether dump_text() copies a string or key from the source: one of
     /// the source, and with ensure_ascii one without bytes >= 0x7F (an overload
     /// each, so that the walk without ensure_ascii is compiled exactly as
     /// before: with GCC, a condition that only folds away later changed its
@@ -6588,154 +6524,13 @@ class view_serializer
         return (n.flags & node_flags::storage) == 0 && is_ascii_without_delete(src + n.off, n.len);
     }
 
-    /*!
+    /// a string with ensure_ascii (dump_compact()), out of line
-    @brief a string with ensure_ascii: the characters that are not printable ASCII as \\uXXXX
+    NLOHMANN_VIEW_NOINLINE void write_ascii_string(const node& n)
-    (a character beyond U+FFFF as a surrogate pair); strings of a document
-    are valid UTF-8, except in a damaged image loaded with image_check::bounds,
-    for which this throws what basic_json::dump() throws for the string
-    */
-    NLOHMANN_VIEW_NOINLINE void write_ascii(const node& n)
     {
-        const auto* const s = reinterpret_cast<const unsigned char*>(m_doc.str(n)); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
+        write_string(n);
-        const std::size_t size = n.len;
-        char* w = m_out.cursor();
-        char* lim = m_out.limit();
-        // room for k bytes
-        const auto room = [&](std::size_t k)
-        {
-            if (NLOHMANN_VIEW_UNLIKELY(static_cast<std::size_t>(lim - w) < k))
-            {
-                m_out.set_cursor(w);
-                m_out.reserve(k);
-                w = m_out.cursor();
-                lim = m_out.limit();
-            }
-        };
-        static constexpr const char* hex = "0123456789abcdef";
-        // \\uXXXX
-        const auto u_escape = [&](std::uint32_t u)
-        {
-            w[0] = '\\';
-            w[1] = 'u';
-            w[2] = hex[(u >> 12u) & 0xFu];
-            w[3] = hex[(u >> 8u) & 0xFu];
-            w[4] = hex[(u >> 4u) & 0xFu];
-            w[5] = hex[u & 0xFu];
-            w += 6;
-        };
-        room(1);
-        *w++ = '"';
-        std::size_t i = 0;
-        // (a string of the source starts with a run that has nothing to escape, 16 bytes per step)
-        const std::size_t prefix = (n.flags & node_flags::storage) == 0 ? ascii_copyable_run(s, size, reinterpret_cast<const unsigned char*>(m_doc.src + m_doc.size)) : 0; // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
-        if (prefix != 0)
-        {
-            room(prefix);
-            std::memcpy(w, s, prefix);
-            w += prefix;
-            i = prefix;
-        }
-        while (i < size)
-        {
-            // (a character that needs an escape sequence is not worth a call)
-            const std::size_t run = is_ascii_copyable(s[i]) ? find_ascii_copyable_run(s + i, size - i) : 0;
-            if (run != 0)
-            {
-                room(run);
-                std::memcpy(w, s + i, run);
-                w += run;
-                i += run;
-                continue;
-            }
-            room(12);
-            if (s[i] >= 0x80)
-            {
-                const unsigned char lead = s[i];
-                // (the two-byte sequences of Latin text and the three-byte sequences of most other
-                // text, with continuation bytes of any value: the ranges that are restricted (E0,
-                // ED, and the sequences of four bytes) go through validate_one_utf8())
-                if (lead >= 0xE1 && lead <= 0xEF && lead != 0xED && size - i >= 3 && (s[i + 1] & 0xC0u) == 0x80 && (s[i + 2] & 0xC0u) == 0x80)
-                {
-                    u_escape(((lead & 0x0Fu) << 12u) | ((s[i + 1] & 0x3Fu) << 6u) | (s[i + 2] & 0x3Fu));
-                    i += 3;
-                    continue;
-                }
-                if (lead >= 0xC2 && lead <= 0xDF && size - i >= 2 && (s[i + 1] & 0xC0u) == 0x80)
-                {
-                    u_escape(((lead & 0x1Fu) << 6u) | (s[i + 1] & 0x3Fu));
-                    i += 2;
-                    continue;
-                }
-                const std::size_t len = validate_one_utf8(s + i, size - i);
-                if (NLOHMANN_VIEW_UNLIKELY(len == 0))
-                {
-                    m_out.set_cursor(w);
-                    invalid_utf8(s, size);
-                    return; // LCOV_EXCL_LINE (invalid_utf8 throws)
-                }
-                std::uint32_t codepoint = s[i] & (0xFFu >> (len + 1));
-                for (std::size_t k = 1; k < len; ++k)
-                {
-                    codepoint = (codepoint << 6u) | (s[i + k] & 0x3Fu);
-                }
-                i += len;
-                if (codepoint <= 0xFFFF)
-                {
-                    u_escape(codepoint);
-                }
-                else
-                {
-                    u_escape(0xD7C0u + (codepoint >> 10u));
-                    u_escape(0xDC00u + (codepoint & 0x3FFu));
-                }
-                continue;
-            }
-            // a quote, a backslash, or a control character (or DEL)
-            const unsigned char c = s[i++];
-            char escape = 0;
-            switch (c)
-            {
-                case 0x08:
-                    escape = 'b';
-                    break;
-                case 0x09:
-                    escape = 't';
-                    break;
-                case 0x0A:
-                    escape = 'n';
-                    break;
-                case 0x0C:
-                    escape = 'f';
-                    break;
-                case 0x0D:
-                    escape = 'r';
-                    break;
-                case 0x22:
-                    escape = '"';
-                    break;
-                case 0x5C:
-                    escape = '\\';
-                    break;
-                default:
-                    break;
-            }
-            if (escape != 0)
-            {
-                w[0] = '\\';
-                w[1] = escape;
-                w += 2;
-            }
-            else
-            {
-                u_escape(c);
-            }
-        }
-        room(1);
-        *w++ = '"';
-        m_out.set_cursor(w);
     }
 
-    /// the copies of dump_text() that are not fixed-size moves (long
+    /// the copies of dump_compact() that are not fixed-size moves (long
     /// strings, or near the end of the source); out of line, so that the
     /// compiler does not merge the fixed-size moves into this call
     NLOHMANN_VIEW_NOINLINE static void copy_long(char* to, const char* from, std::size_t n) noexcept
@@ -6752,7 +6547,7 @@ class view_serializer
     }
 
     /*!
-    @brief the output, compact (the default dump()) or indented, with or without ensure_ascii
+    @brief the compact output (the default dump(), or with ensure_ascii)
 
     The same walk as dump(), with the write position in a local variable
     (stores through char pointers would otherwise force a reload of the
@@ -6760,11 +6555,11 @@ class view_serializer
     the source copied by fixed-size moves of 32 bytes where the source has
     that many bytes left, instead of a library call per token. The buffer
     keeps 64 bytes of slack for the overshoot. With ensure_ascii (Ascii), a
-    string of the source is copied only if it has nothing to escape (see
+    string of the source is copied only if it has no byte >= 0x7F (see
-    copied()); any other string is written by write_ascii().
+    is_ascii_without_delete()); any other string is written by write_string().
     */
-    template<bool SourceNumbers, bool Pretty, bool Ascii>
+    template<bool SourceNumbers, bool Ascii>
-    void dump_text(const node* root)
+    void dump_compact(const node* root)
     {
         struct frame
         {
@@ -6779,16 +6574,6 @@ class view_serializer
         const char* const src_end = src + m_doc.size;
         char* w = m_out.cursor();
         char* lim = m_out.limit();
-        // (indented output) a line break and 63 indentation characters, from
-        // which the line break of a level of up to 63 characters is a copy
-        std::array<char, 64> ruler; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): filled below if used
-        if (Pretty)
-        {
-            ruler.fill(m_style.indent_char);
-            ruler[0] = '\n';
-        }
-        const std::size_t indent = m_style.indent;
-        depth_counter<Pretty> level; // depth of the innermost open container (indented output)
         // room for n bytes and the slack
         const auto room = [&](std::size_t n)
         {
@@ -6820,22 +6605,6 @@ class view_serializer
             }
             w += n;
         };
-        // (indented output) the line break and the indentation of a level
-        const auto newline = [&](std::size_t depth)
-        {
-            const std::size_t count = 1 + (depth * indent);
-            room(count);
-            if (count <= ruler.size())
-            {
-                std::memcpy(w, ruler.data(), ruler.size());
-            }
-            else
-            {
-                *w = '\n';
-                std::memset(w + 1, m_style.indent_char, count - 1);
-            }
-            w += count;
-        };
         // a string that is not a plain string of the source (out of line, so
         // that the cursor stays in a register here)
         const auto escaped = [&](const node & n)
@@ -6843,11 +6612,11 @@ class view_serializer
             m_out.set_cursor(w);
             if (enabled(Ascii))
             {
-                write_ascii(n);
+                write_ascii_string(n);
             }
             else
             {
-                write_decoded(n, Pretty);
+                write_decoded(n);
             }
             w = m_out.cursor();
             lim = m_out.limit();
@@ -6948,7 +6717,6 @@ class view_serializer
                     {
                         *w++ = object ? '{' : '[';
                         stack.push_back(cur);
-                        level.enter();
                         if (Editable)
                         {
                             cur = frame{nav::first(m_doc, n), nav::end(m_doc, n), object};
@@ -6991,10 +6759,6 @@ class view_serializer
                     {
                         break;
                     }
-                    if (Pretty)
-                    {
-                        newline(level.leave());
-                    }
                     room(1);
                     *w++ = cur.object ? '}' : ']';
                     cur = stack.back();
@@ -7003,17 +6767,13 @@ class view_serializer
                 room(1);
                 *w++ = ',';
             }
-            if (Pretty)
-            {
-                newline(level.get());
-            }
             const node* const at = Editable ? cur.pos : n;
             if (cur.object)
             {
                 const node& key = *at;
                 if (copied(key, src, std::integral_constant<bool, Ascii>()))
                 {
-                    room(key.len + (Pretty ? 4 : 3));
+                    room(key.len + 3);
                     *w++ = '"';
                     copy(src + key.off, key.len);
                     w[0] = '"';
@@ -7023,13 +6783,9 @@ class view_serializer
                 else
                 {
                     escaped(key);
-                    room(Pretty ? 2 : 1);
+                    room(1);
                     *w++ = ':';
                 }
-                if (Pretty)
-                {
-                    *w++ = ' ';
-                }
                 if (Editable)
                 {
                     n = nav::value(at + 1);
@@ -7048,6 +6804,69 @@ class view_serializer
         }
     }
 
+    void newline(std::size_t level)
+    {
+        if (m_style.pretty)
+        {
+            m_out.put('\n');
+            m_out.put_repeated(m_style.indent_char, level * m_style.indent);
+        }
+    }
+
+    void write_scalar(const node& n)
+    {
+        switch (static_cast<value_t>(n.kind))
+        {
+            case value_t::null:
+                m_out.put("null");
+                break;
+            case value_t::boolean:
+                if ((n.flags & node_flags::is_true) != 0)
+                {
+                    m_out.put("true");
+                }
+                else
+                {
+                    m_out.put("false");
+                }
+                break;
+            case value_t::string:
+                write_string(n);
+                break;
+            case value_t::number_integer:
+            case value_t::number_unsigned:
+            {
+                const char* const token = m_doc.str(n);
+                const std::uint32_t len = number_length(n);
+                if (!m_style.source_numbers && len == 2 && token[0] == '-' && token[1] == '0')
+                {
+                    m_out.put('0'); // parse() reads -0 as the integer 0
+                }
+                else
+                {
+                    m_out.put(token, len);
+                }
+                break;
+            }
+            case value_t::number_float:
+                if (m_style.source_numbers && (n.flags & node_flags::storage) != node_flags::edited)
+                {
+                    m_out.put(m_doc.str(n), n.len); // (a float set by an edit is written as with shortest)
+                }
+                else
+                {
+                    write_float_node(n);
+                }
+                break;
+            case value_t::object:    // LCOV_EXCL_LINE (containers are written by dump())
+            case value_t::array:     // LCOV_EXCL_LINE
+            case value_t::binary:    // LCOV_EXCL_LINE (not in a document)
+            case value_t::discarded: // LCOV_EXCL_LINE
+            default:                 // LCOV_EXCL_LINE
+                break;               // LCOV_EXCL_LINE
+        }
+    }
+
     /// a float node as dump() writes it
     void write_float_node(const node& n)
     {
@@ -7158,6 +6977,72 @@ class view_serializer
         m_out.put(s.data(), s.size());
     }
 
+    void write_string(const node& n)
+    {
+        const char* const s = m_doc.str(n);
+        m_out.put('"');
+        if ((n.flags & node_flags::storage) == 0 && (!m_style.ensure_ascii || is_ascii_without_delete(s, n.len)))
+        {
+            // a string of the source without escape sequences has nothing to
+            // escape (with ensure_ascii, if it is printable ASCII)
+            m_out.put(s, n.len);
+        }
+        else if (m_style.ensure_ascii)
+        {
+            write_escaped<true>(reinterpret_cast<const unsigned char*>(s), n.len); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
+        }
+        else
+        {
+            write_escaped<false>(reinterpret_cast<const unsigned char*>(s), n.len); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
+        }
+        m_out.put('"');
+    }
+
+    /// as serializer::dump_escaped(); strings of a document are valid UTF-8,
+    /// except in a damaged image loaded with image_check::bounds, for which
+    /// this throws what basic_json::dump() throws for the string
+    template<bool EnsureAscii>
+    void write_escaped(const unsigned char* s, std::size_t n)
+    {
+        std::size_t i = 0;
+        while (i < n)
+        {
+            std::size_t run = 0;
+            if (!EnsureAscii)
+            {
+                run = string_bulk_run(s + i, n - i);
+            }
+            else if (is_ascii_copyable(s[i]))
+            {
+                run = find_ascii_copyable_run(s + i, n - i);
+            }
+            if (run != 0)
+            {
+                m_out.put(reinterpret_cast<const char*>(s + i), run); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
+                i += run;
+                continue;
+            }
+            std::uint32_t codepoint = s[i];
+            std::size_t len = 1;
+            if (codepoint >= 0x80)
+            {
+                len = validate_one_utf8(s + i, n - i);
+                if (NLOHMANN_VIEW_UNLIKELY(len == 0))
+                {
+                    invalid_utf8(s, n);
+                    return;
+                }
+                codepoint &= 0xFFu >> (len + 1);
+                for (std::size_t k = 1; k < len; ++k)
+                {
+                    codepoint = (codepoint << 6u) | (s[i + k] & 0x3Fu);
+                }
+            }
+            write_codepoint<EnsureAscii>(codepoint, s + i, len);
+            i += len;
+        }
+    }
+
     /// throw what basic_json::dump() throws for a string that is not valid UTF-8
     NLOHMANN_VIEW_NOINLINE static void invalid_utf8(const unsigned char* s, std::size_t n)
     {

tests/src/unit-json_view_dump.cpp

@@ -186,131 +186,6 @@ TEST_CASE("json_view dump")
         CHECK(deep_document.root().dump() == deep);
     }
 
-    SECTION("ensure_ascii: a character that needs an escape at each position of a string")
-    {
-        // string lengths around the width of the vector steps, in a string value, a key, and a value near the end of the text
-        for (const char* const special :
-                {
-                    "\xC3\xA9", "\xE4\xB8\xAD", "\xF0\x9F\x98\x80", "\x7F", "\\n", "\\u0001", "\\\"", "\\\\"
-                })
-        {
-            for (std::size_t length = 0; length <= 40; ++length)
-            {
-                for (std::size_t position = 0; position <= length; ++position)
-                {
-                    std::string content(length, 'a');
-                    content.insert(position, special);
-                    const std::string texts[] =
-                    {
-                        "\"" + content + "\"",
-                        "[\"" + content + "\"]",
-                        "{\"" + content + "\":1}",
-                    };
-                    for (const std::string& text : texts)
-                    {
-                        CAPTURE(text)
-                        const ordered_json_document d = ordered_json_document::parse(text);
-                        const ordered_json j = ordered_json::parse(text);
-                        CHECK(d.root().dump(-1, ' ', true) == j.dump(-1, ' ', true));
-                        CHECK(d.root().dump(2, ' ', true) == j.dump(2, ' ', true));
-                        CHECK(d.root().dump() == j.dump());
-                    }
-                }
-            }
-        }
-    }
-
-    SECTION("ensure_ascii: every character of the Basic Multilingual Plane and some beyond")
-    {
-        // (the two- and three-byte sequences, including the ranges that need a check: E0, ED, EF BF BF)
-        std::string text = "[";
-        std::size_t in_chunk = 0;
-        const auto append = [&](std::uint32_t cp)
-        {
-            if (cp < 0x80)
-            {
-                if (cp < 0x20 || cp == '"' || cp == '\\')
-                {
-                    return;
-                }
-                text += static_cast<char>(cp);
-            }
-            else if (cp < 0x800)
-            {
-                text += static_cast<char>(0xC0 | (cp >> 6));
-                text += static_cast<char>(0x80 | (cp & 0x3F));
-            }
-            else if (cp < 0x10000)
-            {
-                text += static_cast<char>(0xE0 | (cp >> 12));
-                text += static_cast<char>(0x80 | ((cp >> 6) & 0x3F));
-                text += static_cast<char>(0x80 | (cp & 0x3F));
-            }
-            else
-            {
-                text += static_cast<char>(0xF0 | (cp >> 18));
-                text += static_cast<char>(0x80 | ((cp >> 12) & 0x3F));
-                text += static_cast<char>(0x80 | ((cp >> 6) & 0x3F));
-                text += static_cast<char>(0x80 | (cp & 0x3F));
-            }
-        };
-        for (std::uint32_t cp = 0x20; cp < 0x10400; ++cp)
-        {
-            if (cp >= 0xD800 && cp < 0xE000)
-            {
-                continue;
-            }
-            if (in_chunk == 0)
-            {
-                text += text.size() > 1 ? ",\"" : "\"";
-            }
-            append(cp);
-            if (++in_chunk == 37)
-            {
-                text += "\"";
-                in_chunk = 0;
-            }
-        }
-        text += in_chunk != 0 ? "\"]" : "]";
-        const ordered_json_document d = ordered_json_document::parse(text);
-        const ordered_json j = ordered_json::parse(text);
-        CHECK(d.root().dump(-1, ' ', true) == j.dump(-1, ' ', true));
-        CHECK(d.root().dump(2, ' ', true) == j.dump(2, ' ', true));
-        CHECK(d.root().dump() == j.dump());
-        CHECK(d.root().dump(2) == j.dump(2));
-    }
-
-    SECTION("deep nesting, indented")
-    {
-        // indentations of up to and beyond the width of the writer's fixed-size copy
-        for (const int depth :
-                {
-                    1, 5, 20, 70, 300
-                })
-        {
-            std::string text;
-            for (int i = 0; i < depth; ++i)
-            {
-                text += i % 2 == 0 ? "[1," : "{\"k\":2,\"\\u00e9\":";
-            }
-            text += "[]";
-            for (int i = depth - 1; i >= 0; --i)
-            {
-                text += i % 2 == 0 ? "]" : "}";
-            }
-            const ordered_json_document d = ordered_json_document::parse(text);
-            const ordered_json j = ordered_json::parse(text);
-            for (const int indent :
-                    {
-                        0, 1, 2, 3, 8
-                    })
-            {
-                CHECK(d.root().dump(indent) == j.dump(indent));
-                CHECK(d.root().dump(indent, '\t') == j.dump(indent, '\t'));
-            }
-        }
-    }
-
     SECTION("the output buffer of a small value is small")
     {
         // an escaped key after the value: its node lies in the arena, so the

tests/src/unit-json_view_image.cpp

@@ -682,26 +682,6 @@ TEST_CASE("json_view images: ownership")
 #if !defined(JSON_NOEXCEPTION)
 TEST_CASE("json_view images: errors")
 {
-    SECTION("invalid UTF-8 of a decoded string: the compact dump writes it, the others throw")
-    {
-        const json_document d = json_document::parse(R"({"k\n":"\u00e9x"})");
-        std::vector<std::uint8_t> image = d.save();
-        const std::size_t arena = text_at(image) + header_size_field(image, 16) + 1;
-        image[arena + 3] = 0x28; // the key is "k\n"; the value starts at 3: 0xC3 0x28 is invalid
-        const json_document loaded = json_document::load(image, image_check::bounds);
-        CHECK(loaded.root().dump().size() == 13);
-        for (const int indent :
-                {
-                    0, 2
-                })
-        {
-            CHECK_THROWS_AS(loaded.root().dump(indent), json::type_error);
-            CHECK_THROWS_AS(loaded.root().dump(indent, ' ', true), json::type_error);
-        }
-        CHECK_THROWS_AS(loaded.root().dump(-1, ' ', true), json::type_error);
-        CHECK_THROWS_WITH_AS(loaded.root().dump(2), "[json.exception.type_error.316] invalid UTF-8 byte at index 1: 0x28", json::type_error&);
-    }
-
     SECTION("a literal as the root: dump() after loading")
     {
         for (const char* text :