docs: cross-reference json_view and json_document for 3.13.0

Document the new zero-copy document/view API across the docs: add a
"Documents and views" section to the basic_json API hub linking to all
specializations, point parsing/conversion/modifying pages to the view-based
alternatives, add document/view sections to feature pages, update the
landing page to mention json_view.hpp/json_literals.hpp, and cross-link
debugging/exceptions.

Also document wide-string inputs (and ill-formed UTF-16/32 rejection),
add parse/accept examples for wide strings, fix related doc gaps, and
align features/parsing and FAQ with the added support.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

docs/mkdocs/docs/api/basic_json/accept.md

@@ -23,6 +23,9 @@ Checks whether the input is valid JSON.
     respectively as UTF-8, UTF-16, and UTF-32. If `SentinelType` differs from `IteratorType`, it must be comparable to
     the iterator type with `operator!=`.
     
+    Ill-formed UTF-16 and UTF-32 sequences are rejected: a lone surrogate, a high surrogate that is not followed by its
+    low surrogate, and a code point above `U+10FFFF` all make this function return `false`.
+
 Unlike the [`parse()`](parse.md) function, this function neither throws an exception in case of invalid JSON input
 (i.e., a parse error) nor creates diagnostic information.
 
@@ -35,9 +38,12 @@ Unlike the [`parse()`](parse.md) function, this function neither throws an excep
     - a `#!c FILE` pointer (throws if null)
     - a C-style array of characters
     - a pointer to a null-terminated string of single byte characters (throws if null)
-    - a `std::string`
+    - a `std::string` or a `std::string_view` (C++17)
     - a container `obj` for which `begin(obj)` and `end(obj)` produce a valid pair of iterators
       (as found via ADL or member functions, with semantics compatible to `std::begin` and `std::end`)
+    - wide characters (`wchar_t`, `char16_t`, `char32_t`), which are converted to UTF-8: a wide string object
+      (`std::wstring`, `std::u16string`, `std::u32string`) or an array (e.g. a wide string literal); a pointer to a
+      null-terminated wide string (e.g. `std::u16string::c_str()`) is not supported
 
 `IteratorType`
 :   a compatible iterator type, for instance.
@@ -125,6 +131,21 @@ outside of a string, invalid) byte; see the [FAQ entry](../../home/faq.md#nul-by
     --8<-- "examples/accept__iterator_pair.output"
     ```
 
+??? example "Example: (3) reading from UTF-16 strings"
+
+    The encoding is deduced from the character type of the input. String literals like `u"..."`, `U"..."`, and
+    `L"..."` work as well.
+
+    ```cpp
+    --8<-- "examples/accept__wide_string.cpp"
+    ```
+
+    Output:
+
+    ```json
+    --8<-- "examples/accept__wide_string.output"
+    ```
+
 ## See also
 
 - [parse](parse.md) - deserialize from a compatible input
@@ -151,6 +172,9 @@ outside of a string, invalid) byte; see the [FAQ entry](../../home/faq.md#nul-by
   ([`JSON_DISABLE_STREAM_BULK_SCAN`](../macros/json_disable_stream_bulk_scan.md)).
 - Parsing from a `FILE*` locks the file once instead of once per character in version 3.13.0
   ([`JSON_DISABLE_UNLOCKED_FILE_IO`](../macros/json_disable_unlocked_file_io.md)).
+- Ill-formed UTF-16 and UTF-32 input (a lone surrogate, a high surrogate that is not followed by its low surrogate, or
+  a code point above `U+10FFFF`) is rejected in version 3.13.0; before that, some such sequences were accepted or
+  ended the input silently.
 
 !!! warning "Deprecation"
 

docs/mkdocs/docs/api/basic_json/index.md

@@ -44,6 +44,22 @@ are collected in [Template Parameter Requirements](../../features/types/template
 - [**json**](../json.md) - default specialization
 - [**ordered_json**](../ordered_json.md) - a specialization that maintains the insertion order of object keys
 
+## Documents and views
+
+Besides `basic_json`, the library offers a second way to hold a parsed JSON text: a
+[`basic_json_document`](../basic_json_document/index.md), which owns or borrows the text and holds a flat index of the
+values in it, and a [`basic_json_view`](../basic_json_view/index.md), a read-only handle to one value of such a document.
+Both are defined in the header `<nlohmann/json_view.hpp>`; see
+[Zero-copy JSON views](../../features/json_view.md) for the concept and the trade-offs.
+
+- [**json_document**](../json_document.md) / [**ordered_json_document**](../ordered_json_document.md) - read-only
+  documents of the `basic_json` specialization they belong to
+- [**json_editable_document**](../json_editable_document.md) /
+  [**ordered_json_editable_document**](../ordered_json_editable_document.md) - documents that can be modified in place
+- [**json_view**](../json_view.md) / [**ordered_json_view**](../ordered_json_view.md) - views of a document
+- [**json_editable_view**](../json_editable_view.md) /
+  [**ordered_json_editable_view**](../ordered_json_editable_view.md) - views of an editable document
+
 ## Iterator invalidation
 
 All operations that add values to an **array** ([`push_back`](push_back.md) , [`operator+=`](operator+=.md),
@@ -343,6 +359,9 @@ Access to the JSON value
 ## See also
 
 - [RFC 8259: The JavaScript Object Notation (JSON) Data Interchange Format](https://tools.ietf.org/html/rfc8259)
+- [basic_json_document](../basic_json_document/index.md) / [basic_json_view](../basic_json_view/index.md) - parse a JSON
+  text into a flat index and read it through a view
+- [Zero-copy JSON views](../../features/json_view.md) - when to use a document and a view instead of a `basic_json`
 
 ## Version history
 

docs/mkdocs/docs/api/basic_json/parse.md

@@ -25,6 +25,10 @@ static basic_json parse(IteratorType first, SentinelType last,
     respectively as UTF-8, UTF-16, and UTF-32. If `SentinelType` differs from `IteratorType`, it must be comparable to
     the iterator type with `operator!=`.
 
+    Ill-formed UTF-16 and UTF-32 sequences are rejected: a lone surrogate, a high surrogate that is not followed by its
+    low surrogate, and a code point above `U+10FFFF` all yield a
+    [`parse_error.101`](../../home/exceptions.md#jsonexceptionparse_error101).
+
 ## Template parameters
 
 `InputType`
@@ -34,9 +38,12 @@ static basic_json parse(IteratorType first, SentinelType last,
     - a `FILE` pointer (throws if null)
     - a C-style array of characters
     - a pointer to a null-terminated string of single byte characters (throws if null)
-    - a `std::string`
+    - a `std::string` or a `std::string_view` (C++17)
     - a container `obj` for which `begin(obj)` and `end(obj)` produce a valid pair of iterators
       (as found via ADL or member functions, with semantics compatible to `std::begin` and `std::end`)
+    - wide characters (`wchar_t`, `char16_t`, `char32_t`), which are converted to UTF-8: a wide string object
+      (`std::wstring`, `std::u16string`, `std::u32string`) or an array (e.g. a wide string literal); a pointer to a
+      null-terminated wide string (e.g. `std::u16string::c_str()`) is not supported
 
 `IteratorType`
 :   a compatible iterator type, for instance.
@@ -112,6 +119,13 @@ By default, a `'\0'` (NUL) byte anywhere in the input is treated as end of input
 outside of a string, invalid) byte; see the [FAQ entry](../../home/faq.md#nul-bytes-in-the-input) for details and the
 [`JSON_STRICT_NUL_HANDLING`](../macros/json_strict_nul_handling.md) macro to opt into rejecting it instead.
 
+!!! tip "Parsing into a document instead of a value"
+
+    [`basic_json_document::parse`](../basic_json_document/parse.md) accepts the same inputs and the same options, but
+    keeps the result as a flat index of the parsed values instead of a tree of `basic_json` values. It is faster and uses
+    less memory when only parts of the document are read, and it also accepts a pointer to a null-terminated wide string,
+    which `parse()` does not. See [Zero-copy JSON views](../../features/json_view.md).
+
 !!! tip "Performance"
 
     Contiguous inputs (strings, string views, vectors, arrays, pointer ranges) are the fastest to parse. A
@@ -250,6 +264,22 @@ outside of a string, invalid) byte; see the [FAQ entry](../../home/faq.md#nul-by
     --8<-- "examples/trailing_commas.output"
     ```
 
+??? example "Example: parse from UTF-16, UTF-32, and wide strings"
+
+    The encoding is deduced from the character type of the input; `std::u16string` and `std::u32string` are interpreted
+    as UTF-16 and UTF-32, and `std::wstring` uses the platform's `wchar_t` width. String literals like `u"..."`,
+    `U"..."`, and `L"..."` work as well, as do iterators over wide characters.
+
+    ```cpp
+    --8<-- "examples/parse__wide_string.cpp"
+    ```
+
+    Output:
+
+    ```json
+    --8<-- "examples/parse__wide_string.output"
+    ```
+
 ## See also
 
 - [accept](accept.md) - check if the input is valid JSON
@@ -280,6 +310,9 @@ outside of a string, invalid) byte; see the [FAQ entry](../../home/faq.md#nul-by
   time).
 - Parsing from a `FILE*` locks the file once instead of once per character in version 3.13.0
   ([`JSON_DISABLE_UNLOCKED_FILE_IO`](../macros/json_disable_unlocked_file_io.md) restores `std::fgetc`).
+- Ill-formed UTF-16 and UTF-32 input (a lone surrogate, a high surrogate that is not followed by its low surrogate, or
+  a code point above `U+10FFFF`) is rejected with `parse_error.101` in version 3.13.0; before that, some such
+  sequences were accepted or ended the input silently.
 
 !!! warning "Deprecation"
 

docs/mkdocs/docs/api/basic_json/sax_parse.md

@@ -31,6 +31,10 @@ Read from input and generate SAX events
     respectively as UTF-8, UTF-16, and UTF-32. If `SentinelType` differs from `IteratorType`, it must be comparable to
     the iterator type with `operator!=`.
 
+    Ill-formed UTF-16 and UTF-32 sequences are rejected: a lone surrogate, a high surrogate that is not followed by its
+    low surrogate, and a code point above `U+10FFFF` all yield a
+    [`parse_error.101`](../../home/exceptions.md#jsonexceptionparse_error101).
+
 The SAX event lister must follow the interface of [`json_sax`](../json_sax/index.md).
 
 ## Template parameters
@@ -42,8 +46,12 @@ The SAX event lister must follow the interface of [`json_sax`](../json_sax/index
     - a `FILE` pointer
     - a C-style array of characters
     - a pointer to a null-terminated string of single byte characters
+    - a `std::string` or a `std::string_view` (C++17)
     - a container `obj` for which `begin(obj)` and `end(obj)` produce a valid pair of iterators
       (as found via ADL or member functions, with semantics compatible to `std::begin` and `std::end`)
+    - wide characters (`wchar_t`, `char16_t`, `char32_t`), which are converted to UTF-8: a wide string object
+      (`std::wstring`, `std::u16string`, `std::u32string`) or an array (e.g. a wide string literal); a pointer to a
+      null-terminated wide string (e.g. `std::u16string::c_str()`) is not supported
 
 `IteratorType`
 :   a compatible iterator type for overload (2); a pair of character iterators whose `value_type` is an integral type
@@ -160,6 +168,9 @@ use the same file with stdio functions, but it must not wait for another thread
   ([`JSON_DISABLE_STREAM_BULK_SCAN`](../macros/json_disable_stream_bulk_scan.md)).
 - Parsing from a `FILE*` locks the file once instead of once per character in version 3.13.0
   ([`JSON_DISABLE_UNLOCKED_FILE_IO`](../macros/json_disable_unlocked_file_io.md)).
+- Ill-formed UTF-16 and UTF-32 input (a lone surrogate, a high surrogate that is not followed by its low surrogate, or
+  a code point above `U+10FFFF`) is rejected with `parse_error.101` in version 3.13.0; before that, some such
+  sequences were accepted or ended the input silently.
 
 !!! warning "Deprecation"
 

docs/mkdocs/docs/api/basic_json_document/index.md

@@ -115,6 +115,12 @@ points apply to every edit:
   [`source_offset()`](../basic_json_view/source_offset.md) of a value introduced by an edit is
   `#!cpp static_cast<std::size_t>(-1)`, the same value it reports for a decoded string.
 
+## See also
+
+- [basic_json_view](../basic_json_view/index.md) - the read-only handle to a value of a document
+- [json_document](../json_document.md) - the document type of the default specialization
+- [Zero-copy JSON views](../../features/json_view.md) - the concept behind documents and views
+
 ## Version history
 
 - Added in version 3.13.0.

docs/mkdocs/docs/api/basic_json_document/parse.md

@@ -21,17 +21,18 @@ static basic_json_document parse(IteratorType first, IteratorType last,
 
 Both overloads accept the same JSON text as [`BasicJsonType::parse()`](../basic_json/parse.md), with the same
 `ignore_comments`/`ignore_trailing_commas` options, but build a [`basic_json_document`](index.md) (a flat index into
-the input) instead of a tree of `BasicJsonType` values. They take the same inputs as `BasicJsonType::parse()`. Byte
+the input) instead of a tree of `BasicJsonType` values. They take the same inputs as `BasicJsonType::parse()`, plus a
-inputs that stay alive can be borrowed; all others (streams, `#!cpp FILE*`, wide characters, non-contiguous
+pointer to a null-terminated wide string, which `BasicJsonType::parse()` does not accept. Byte inputs that stay alive
-iterators) are read into a copy that the document owns -- see the Notes. Use [`parse_copy()`](parse_copy.md) to get an
+can be borrowed; all others (streams, `#!cpp FILE*`, wide characters, non-contiguous iterators) are read into a copy
-owned copy of a borrowable input as well.
+that the document owns -- see the Notes. Use [`parse_copy()`](parse_copy.md) to get an owned copy of a borrowable input
+as well.
 
 ## Template parameters
 
 `InputType`
 :   A compatible input, for instance:
 
-    - a `#!cpp std::string`, `#!cpp std::string_view`, or a C-style array of single-byte characters
+    - a `#!cpp std::string`, a `#!cpp std::string_view` (C++17), or a C-style array of single-byte characters
     - a pointer to a null-terminated string of single-byte characters (`#!cpp char`, `#!cpp signed char`,
       `#!cpp unsigned char`, `#!cpp std::uint8_t`)
     - a container for which `#!cpp obj.data()` and `#!cpp obj.size()` give contiguous single-byte access, e.g.
@@ -47,6 +48,10 @@ owned copy of a borrowable input as well.
     `#!cpp std::string::iterator`; the iterators of single-byte characters are borrowed or read like the byte inputs
     above, those of wide characters are converted to UTF-8
 
+Ill-formed UTF-16 and UTF-32 sequences are rejected: a lone surrogate, a high surrogate that is not followed by its low
+surrogate, and a code point above `U+10FFFF` all yield a
+[`parse_error.101`](../../home/exceptions.md#jsonexceptionparse_error101).
+
 ## Parameters
 
 `input` (in)
@@ -155,3 +160,5 @@ buffer of a given length, pass a pair of pointers: `#!cpp parse(ptr, ptr + len)`
 ## Version history
 
 - Added in version 3.13.0.
+- Ill-formed UTF-16 and UTF-32 input (a lone surrogate, a high surrogate that is not followed by its low surrogate, or
+  a code point above `U+10FFFF`) is rejected with `parse_error.101`.

docs/mkdocs/docs/api/basic_json_view/index.md

@@ -143,6 +143,12 @@ still refers to it -- including ones taken before the change -- reads the new va
 
 - [**source_offset**](source_offset.md) - byte offset of this value in the document's source text
 
+## See also
+
+- [basic_json_document](../basic_json_document/index.md) - the document a view refers into
+- [materialize](materialize.md) - build the `BasicJsonType` value of a subtree
+- [Zero-copy JSON views](../../features/json_view.md) - the concept behind documents and views
+
 ## Version history
 
 - Added in version 3.13.0.

docs/mkdocs/docs/examples/accept__wide_string.cpp

@@ -0,0 +1,18 @@
+#include <iostream>
+#include <string>
+#include <nlohmann/json.hpp>
+
+using json = nlohmann::json;
+
+int main()
+{
+    // a valid JSON text, given as UTF-16
+    const std::u16string valid_text = u"{\"name\":\"\u00e9\"}";
+
+    // an invalid JSON text, given as UTF-16
+    const std::u16string invalid_text = u"{\"name\":\"\u00e9\",}";
+
+    std::cout << std::boolalpha
+              << json::accept(valid_text) << ' '
+              << json::accept(invalid_text) << '\n';
+}

docs/mkdocs/docs/examples/accept__wide_string.output

@@ -0,0 +1 @@
+true false

docs/mkdocs/docs/examples/parse__wide_string.cpp

@@ -0,0 +1,27 @@
+#include <iostream>
+#include <string>
+#include <nlohmann/json.hpp>
+
+using json = nlohmann::json;
+
+int main()
+{
+    // the same JSON text, given in different encodings
+    const std::string utf8 = "{\"name\":\"\u00e9\"}";
+    const std::u16string utf16 = u"{\"name\":\"\u00e9\"}";
+    const std::u32string utf32 = U"{\"name\":\"\u00e9\"}";
+    const std::wstring wide = L"{\"name\":\"\u00e9\"}";
+
+    // the encoding is deduced from the character type of the input, and all
+    // of them yield the same value
+    std::cout << json::parse(utf8) << '\n'
+              << json::parse(utf16) << '\n'
+              << json::parse(utf32) << '\n'
+              << json::parse(wide) << '\n';
+
+    // a wide string literal works as well: u"..." is UTF-16, U"..." is UTF-32
+    std::cout << json::parse(u"[1, 2, 3]") << '\n';
+
+    // iterators over wide characters work as well
+    std::cout << json::parse(utf16.begin(), utf16.end()) << '\n';
+}

docs/mkdocs/docs/examples/parse__wide_string.output

@@ -0,0 +1,6 @@
+{"name":"é"}
+{"name":"é"}
+{"name":"é"}
+{"name":"é"}
+[1,2,3]
+{"name":"é"}

docs/mkdocs/docs/features/conversions.md

@@ -38,6 +38,16 @@ which avoids repeating it:
     --8<-- "examples/get_to.output"
     ```
 
+!!! info "Getting values out of a document or a view"
+
+    A [view](../api/basic_json_view/index.md) offers the same `get<T>()` conversion, but reads it straight from the
+    document's index, without building a `basic_json` value first. Two conversions never copy at all:
+    [`get_string()`](../api/basic_json_view/get_string.md) returns a string as a view into the source text, and
+    [`number_token()`](../api/basic_json_view/number_token.md) returns a number exactly as it was written. Every other
+    type is obtained from a subtree that is built on demand with
+    [`materialize()`](../api/basic_json_view/materialize.md). See
+    [Zero-copy JSON views](json_view.md#getting-values-out-without-copying).
+
 The library already knows how to convert to and from the scalar types and the STL containers (such as
 `#!cpp std::vector`, `#!cpp std::map`, `#!cpp std::array`, `#!cpp std::optional`, and many more). Converting a JSON
 object back to a `#!cpp std::map` or a JSON array back to a `#!cpp std::vector` therefore works without any extra code:
@@ -234,3 +244,4 @@ The conversions above are built in for standard types. To make the same syntax w
 - [`get_ref`](../api/basic_json/get_ref.md) / [`get_ptr`](../api/basic_json/get_ptr.md) - access the stored value without copying
 - [Arbitrary types conversions](arbitrary_types.md) - support your own types
 - [`JSON_USE_IMPLICIT_CONVERSIONS`](../api/macros/json_use_implicit_conversions.md) - toggle implicit conversions
+- [Zero-copy JSON views](json_view.md) - get values out of a document or a view

docs/mkdocs/docs/features/creating_values.md

@@ -1,7 +1,9 @@
 # Creating JSON values
 
 There are several ways to create a JSON value in memory. This page gives an overview; to read a value from JSON text
-instead, see [parsing](parsing/index.md).
+instead, see [parsing](parsing/index.md). A JSON text can also be parsed into a
+[document](../api/basic_json_document/index.md) and read through a [view](../api/basic_json_view/index.md) instead of
+into a value; see [Zero-copy JSON views](json_view.md).
 
 ## From C++ values
 
@@ -101,3 +103,4 @@ string `#!json "42"`.
 - [`operator""_json`](../api/operator_literal_json.md) - the `_json` literal
 - [Converting values](conversions.md) - which C++ types can be used
 - [Parsing](parsing/index.md) - create a value from JSON text
+- [Zero-copy JSON views](json_view.md) - create a document and a view from JSON text instead of a value

docs/mkdocs/docs/features/macros.md

@@ -287,6 +287,23 @@ external dependency and is not defined by default.
 
 See [full documentation of `JSON_USE_SIMDUTF`](../api/macros/json_use_simdutf.md).
 
+## `JSON_VIEW_NO_SIMD`
+
+When defined, the parser of [`basic_json_document`](../api/basic_json_document/index.md)
+(`<nlohmann/json_view.hpp>`) uses only portable C++ to scan strings, instead of vector instructions (SSE2 on x86-64,
+NEON on AArch64). The same input is accepted or rejected either way; only the speed differs. It is not defined by
+default.
+
+See [full documentation of `JSON_VIEW_NO_SIMD`](../api/macros/json_view_no_simd.md).
+
+## `JSON_VIEW_USE_SSSE3`
+
+When defined on x86-64, the parser of [`basic_json_document`](../api/basic_json_document/index.md)
+(`<nlohmann/json_view.hpp>`) validates non-ASCII text in strings with SSSE3 without asking the CPU first. It is not
+defined by default.
+
+See [full documentation of `JSON_VIEW_USE_SSSE3`](../api/macros/json_view_use_ssse3.md).
+
 ## `NLOHMANN_DEFINE_TYPE_*(...)`, `NLOHMANN_DEFINE_DERIVED_TYPE_*(...)`
 
 The library defines 12 macros to simplify the serialization/deserialization of types. See the page on

docs/mkdocs/docs/features/modifying_values.md

@@ -85,6 +85,16 @@ json a = {1, 2, 3, 4};
 a.erase(1);             // [1,3,4]  (erase by index)
 ```
 
+## Editing a document
+
+The operations above work on a `basic_json` value. A JSON text that was parsed into a
+[document](json_view.md#editing-a-document) can also be changed in place, without building a `basic_json` tree for the
+parts that are not touched: [`set`](../api/basic_json_document/set.md) replaces a value or an object member,
+[`push_back`](../api/basic_json_document/push_back.md) appends to an array,
+[`insert`](../api/basic_json_document/insert.md) inserts into an array, and
+[`erase`](../api/basic_json_document/erase.md) removes a member or an element. These functions are available on the
+editable document types, such as [`json_editable_document`](../api/json_editable_document.md).
+
 ## See also
 
 - [`push_back`](../api/basic_json/push_back.md) / [`emplace_back`](../api/basic_json/emplace_back.md) - append to an array
@@ -93,3 +103,4 @@ a.erase(1);             // [1,3,4]  (erase by index)
 - [`merge_patch`](../api/basic_json/merge_patch.md) - apply an RFC 7396 merge patch
 - [`erase`](../api/basic_json/erase.md) / [`clear`](../api/basic_json/clear.md) - remove elements
 - [JSON Patch and Diff](json_patch.md) and [JSON Merge Patch](merge_patch.md) - structured modifications
+- [Editing a document](json_view.md#editing-a-document) - change a parsed document in place

docs/mkdocs/docs/features/parsing/index.md

@@ -18,6 +18,7 @@ flowchart LR
 The [`parse`](../../api/basic_json/parse.md) function reads a JSON value from an input. The input can be
 
 - a string (`#!cpp std::string`, C string, or string literal),
+- a wide string (`#!cpp std::u16string`, `#!cpp std::u32string`, or `#!cpp std::wstring`, or a wide string literal),
 - a `#!cpp std::istream` (e.g., an `#!cpp std::ifstream` reading from a file),
 - a `#!cpp FILE*` pointer,
 - a pair of iterators over a contiguous range (e.g., a `#!cpp std::vector<std::uint8_t>`), or
@@ -32,12 +33,18 @@ std::ifstream f("example.json");
 json data = json::parse(f);
 ```
 
-The input must be encoded in UTF-8; other encodings are not supported. A single input may contain only one JSON value.
+The input must be encoded in UTF-8; other encodings are not supported. The exception is wide-character input, which
+is read as UTF-16 or UTF-32 -- depending on the character type -- and converted to UTF-8; see the
+[`parse`](../../api/basic_json/parse.md) page for an example. A single input may contain only one JSON value.
 Inputs consisting of multiple values separated by newlines are handled by the [JSON Lines](json_lines.md) format.
 
 By default, the library rejects comments and trailing commas. Both can be enabled with parameters of the `parse`
 function — see [comments](../comments.md) and [trailing commas](../trailing_commas.md).
 
+[`basic_json_document::parse`](../../api/basic_json_document/parse.md) accepts the same inputs with the same options,
+but keeps the result in a document rather than in a `basic_json` value; see
+[Documents and views](#documents-and-views) below.
+
 ## Strictness and trailing data
 
 [`parse`](../../api/basic_json/parse.md) reads a single JSON value and requires the whole input to be consumed: any
@@ -98,6 +105,24 @@ The library offers two parsing models:
 You can influence a DOM parse without switching to the SAX interface by passing a
 [parser callback](parser_callbacks.md), which is called during parsing and can, for example, discard parts of the input.
 
+## Documents and views
+
+A third model sits between DOM and SAX: [`basic_json_document::parse`](../../api/basic_json_document/parse.md) parses
+the input into a document that keeps the text and a flat index of the values in it, and
+[`basic_json_view`](../../api/basic_json_view/index.md) gives read-only access to the values through that index.
+Strings and numbers stay in the source text and are only decoded when they are read, which is faster and uses less
+memory than building a tree of `basic_json` values when only parts of the document are needed. A document can also be
+modified in place, and it can be saved and loaded again without parsing it.
+
+```cpp
+#include <nlohmann/json_view.hpp>
+
+json_document doc = json_document::parse(R"({"happy": true, "pi": 3.141})");
+std::cout << doc.root()["happy"].get<bool>() << '\n';
+```
+
+See [Zero-copy JSON views](../json_view.md) for the concept and the trade-offs.
+
 ## Exceptions
 
 When the input is not valid JSON, the `parse` function throws an exception by default. If exceptions are undesired or
@@ -108,6 +133,7 @@ options.
 ## See also
 
 - [`parse`](../../api/basic_json/parse.md) - deserialize from a compatible input
+- [`basic_json_document::parse`](../../api/basic_json_document/parse.md) - deserialize into a document instead of a value
 - [`accept`](../../api/basic_json/accept.md) - check if the input is valid JSON
 - [`sax_parse`](../../api/basic_json/sax_parse.md) - generate SAX events
 - [JSON Lines](json_lines.md) - parse newline-delimited JSON
@@ -115,3 +141,4 @@ options.
 - [SAX interface](sax_interface.md) - implement a custom SAX handler
 - [parsing and exceptions](parse_exceptions.md) - control error handling
 - [parsing untrusted input](untrusted_input.md) - what to consider when parsing input from untrusted sources
+- [Zero-copy JSON views](../json_view.md) - parse into a document and read it through a view

docs/mkdocs/docs/features/parsing/untrusted_input.md

@@ -81,6 +81,9 @@ Invalid UTF-8 is rejected while parsing, not just while serializing:
 
 - In JSON text, an ill-formed UTF-8 byte in a string is a
   [`parse_error.101`](../../home/exceptions.md#jsonexceptionparse_error101) ("invalid string: ill-formed UTF-8 byte").
+- In UTF-16 or UTF-32 input, an ill-formed sequence is a
+  [`parse_error.101`](../../home/exceptions.md#jsonexceptionparse_error101) as well: a lone surrogate, a high surrogate
+  that is not followed by its low surrogate, and a code point above `U+10FFFF` are all rejected.
 - In a binary format, a string that is not valid UTF-8 is a
   [`parse_error.113`](../../home/exceptions.md#jsonexceptionparse_error113).
 

docs/mkdocs/docs/home/architecture.md

@@ -339,11 +339,6 @@ The `serializer` (used by [`dump`](../api/basic_json/dump.md) and [`operator<<`]
 - `output_stream_adapter` writes to a `std::ostream`.
 - `output_string_adapter` appends to a string.
 
-The `binary_writer` does not call an adapter for every byte. The `to_*` functions that return a `std::vector` write
-into the vector through a sink (`output_vector_sink`) that is not virtual. The functions that take an output adapter
-write through a sink (`output_adapter_sink`) that collects the output in a buffer of one kibibyte and passes it to the
-adapter in blocks, so that the adapter is called (and a stream locked) once per block instead of once per value.
-
 ## Value conversion
 
 Values are converted from and to other types with the `JSONSerializer` template parameter. The default,

docs/mkdocs/docs/home/debugging.md

@@ -1,7 +1,7 @@
 # Debugging
 
-This page collects the library's built-in debugger integrations and other debugging-related features. They are
+This page collects the library's built-in debugger integrations and other debugging-related features. They are not
-not linked from a single place elsewhere in the docs, so are collected here.
+covered by any single page elsewhere in the docs, so they are collected here.
 
 ## Visual Studio (natvis)
 

docs/mkdocs/docs/home/exceptions.md

@@ -99,7 +99,8 @@ As this global context comes at the price of storing one additional pointer per
 
     Now the exception message contains a JSON Pointer `/address/housenumber` that indicates which value has the wrong type.
 
-See [documentation of `JSON_DIAGNOSTICS`](../api/macros/json_diagnostics.md) for more information.
+See [documentation of `JSON_DIAGNOSTICS`](../api/macros/json_diagnostics.md) for more information, and
+[Debugging](debugging.md) for the debugger integrations of the library.
 
 ## Parse errors
 

docs/mkdocs/docs/home/faq.md

@@ -81,8 +81,9 @@ rejected by default, and the library never adds them when serializing.
 
 The library supports **Unicode input** as follows:
 
-- Only **UTF-8** encoded input is supported, which is the default encoding for JSON, according to [RFC 8259](https://tools.ietf.org/html/rfc8259.html#section-8.1).
+- **UTF-8** encoded input is supported, which is the default encoding for JSON, according to [RFC 8259](https://tools.ietf.org/html/rfc8259.html#section-8.1).
-- `std::u16string` and `std::u32string` can be parsed, assuming UTF-16 and UTF-32 encoding, respectively. These encodings are not supported when reading from files or other input containers.
+- `std::u16string` and `std::u32string` can be parsed, assuming UTF-16 and UTF-32 encoding, respectively; `std::wstring` works as well and is interpreted as UTF-16 or UTF-32, depending on the width of the platform's `wchar_t`. These encodings cannot be read from a `FILE*` or an `std::istream`, which carry bytes only.
+- Ill-formed UTF-16 or UTF-32 input is rejected: a lone surrogate, a high surrogate that is not followed by its low surrogate, and a code point above `U+10FFFF` all yield a parse error. The same holds for iterators over 2- or 4-byte characters.
 - Other encodings such as Latin-1 or ISO 8859-1 are **not** supported and will yield parse or serialization errors.
 - The library will not replace [Unicode noncharacters](http://www.unicode.org/faq/private_use.html#nonchar1).
 - Invalid surrogates (e.g., incomplete pairs such as `\uDEAD`) will yield parse errors.
@@ -146,7 +147,7 @@ json::parse(std::string("\"") + '\0' + "\""); // throws parse_error.101: control
 
     Why are wide strings (e.g., `std::wstring`) dumped as arrays of numbers?
 
-As described [above](#parse-errors-reading-non-ascii-characters), the library assumes UTF-8 as encoding.  To store a wide string, you need to change the encoding.
+As described [above](#parse-errors-reading-non-ascii-characters), the library stores its strings as UTF-8. To store a wide string, you need to change the encoding.
 
 !!! example
 

docs/mkdocs/docs/index.md

@@ -6,7 +6,9 @@ JSON for Modern C++ is a header-only C++11 library that turns JSON into a first-
 magic of modern C++ so that creating, reading, and modifying JSON values feels as natural as it does in languages like
 Python. The whole library is available as a single header, `json.hpp`, with no dependencies, no subproject, and no
 complex build system to set up; a companion header, `json_fwd.hpp`, provides forward declarations to keep compile times
-down. See [header-only integration](integration/index.md) for details. It is heavily unit-tested with 100% code
+down, `<nlohmann/json_literals.hpp>` holds the user-defined string literals on their own, and
+`<nlohmann/json_view.hpp>` adds zero-copy views and editable documents. See
+[header-only integration](integration/index.md) for details. It is heavily unit-tested with 100% code
 coverage, checked with Valgrind and the Clang Sanitizers for memory leaks, and continuously fuzz-tested by Google
 OSS-Fuzz.
 
@@ -58,7 +60,8 @@ Conda, and more), and pkg-config.
 
     ---
 
-    Creating, parsing, accessing, and serializing JSON values, JSON Pointer/Patch, binary formats, and more.
+    Creating, parsing, accessing, and serializing JSON values, JSON Pointer/Patch, binary formats, zero-copy views,
+    and more.
 
     [:octicons-arrow-right-24: Features](features/index.md)
 
@@ -74,7 +77,8 @@ Conda, and more), and pkg-config.
 
     ---
 
-    The complete reference for `basic_json` and its member functions, types, and related classes.
+    The complete reference for `basic_json` and its member functions, types, and related classes, as well as for
+    `json_document` and `json_view`.
 
     [:octicons-arrow-right-24: API documentation](api/basic_json/index.md)
 

include/nlohmann/detail/output/output_adapters.hpp

@@ -55,66 +55,6 @@ using output_adapter_t = std::shared_ptr<output_adapter_protocol<CharType>>;
 /// output_vector_sink also appends writes of at least this size with insert()
 constexpr std::size_t max_reserve_size = 256;
 
-/// @brief copy @a length characters, at most 32, without a call of memcpy()
-///
-/// Short strings are the typical payload of a binary value, and a call of
-/// memcpy() costs more than copying them. A copy of 16 to 32 characters is two
-/// possibly overlapping 16-byte copies, one of 8 to 15 two 8-byte copies, and
-/// so on down to a single character; each is a fixed-size load and store.
-/// Nothing outside of [src, src + length) is read or written.
-template<typename CharType>
-inline void copy_short(CharType* dst, const CharType* src, const std::size_t length, std::true_type /*is_byte*/) noexcept
-{
-    JSON_ASSERT(length <= 32);
-    if (length >= 16)
-    {
-        std::array<char, 16> first; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
-        std::array<char, 16> last; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
-        std::memcpy(first.data(), src, 16);
-        std::memcpy(last.data(), src + (length - 16), 16);
-        std::memcpy(dst, first.data(), 16);
-        std::memcpy(dst + (length - 16), last.data(), 16);
-    }
-    else if (length >= 8)
-    {
-        std::array<char, 8> first; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
-        std::array<char, 8> last; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
-        std::memcpy(first.data(), src, 8);
-        std::memcpy(last.data(), src + (length - 8), 8);
-        std::memcpy(dst, first.data(), 8);
-        std::memcpy(dst + (length - 8), last.data(), 8);
-    }
-    else if (length >= 4)
-    {
-        std::array<char, 4> first; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
-        std::array<char, 4> last; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init): written before read
-        std::memcpy(first.data(), src, 4);
-        std::memcpy(last.data(), src + (length - 4), 4);
-        std::memcpy(dst, first.data(), 4);
-        std::memcpy(dst + (length - 4), last.data(), 4);
-    }
-    else if (length != 0)
-    {
-        // 1 to 3: the first, the middle, and the last character
-        const CharType a = src[0];
-        const CharType b = src[length >> 1];
-        const CharType c = src[length - 1];
-        dst[0] = a;
-        dst[length >> 1] = b;
-        dst[length - 1] = c;
-    }
-}
-
-/// @brief copy @a length characters, of a character type wider than a byte
-template<typename CharType>
-inline void copy_short(CharType* dst, const CharType* src, const std::size_t length, std::false_type /*is_byte*/) noexcept
-{
-    std::memcpy(dst, src, length * sizeof(CharType));
-}
-
-/// the number of characters an empty vector is grown to by output_vector_sink
-constexpr std::size_t min_vector_sink_capacity = 64;
-
 /// @brief non-virtual output sink writing into a std::vector
 ///
 /// This sink is not part of the virtual output_adapter_protocol hierarchy: it is
@@ -138,7 +78,7 @@ class output_vector_sink
 {
   public:
     explicit output_vector_sink(std::vector<CharType, AllocatorType>& vec) noexcept
-        : v(&vec), cursor(vec.data() + vec.size()), limit(cursor)
+        : v(&vec), position(vec.size())
     {}
 
     output_vector_sink(const output_vector_sink&) = delete;
@@ -146,7 +86,7 @@ class output_vector_sink
     output_vector_sink& operator=(output_vector_sink&&) = delete;
 
     output_vector_sink(output_vector_sink&& other) noexcept
-        : v(other.v), cursor(other.cursor), limit(other.limit)
+        : v(other.v), position(other.position)
     {
         other.v = nullptr;
     }
@@ -155,7 +95,7 @@ class output_vector_sink
     {
         if (v != nullptr)
         {
-            v->resize(position());
+            v->resize(position);
         }
     }
 
@@ -164,11 +104,11 @@ class output_vector_sink
     JSON_HEDLEY_ALWAYS_INLINE
     void write_character(CharType c)
     {
-        if (JSON_HEDLEY_UNLIKELY(cursor == limit))
+        if (JSON_HEDLEY_UNLIKELY(position == v->size()))
         {
             grow(1);
         }
-        *cursor++ = c;
+        (*v)[position++] = c;
     }
 
     // no JSON_HEDLEY_NON_NULL here: binary_writer legitimately passes a null
@@ -177,7 +117,7 @@ class output_vector_sink
     JSON_HEDLEY_ALWAYS_INLINE
     void write_characters(const CharType* s, std::size_t length)
     {
-        if (JSON_HEDLEY_UNLIKELY(space() < length))
+        if (JSON_HEDLEY_UNLIKELY(v->size() - position < length))
         {
             if (length >= max_reserve_size)
             {
@@ -186,15 +126,10 @@ class output_vector_sink
             }
             grow(length);
         }
-        if (length <= 32)
+        if (length != 0)
         {
-            copy_short(cursor, s, length, std::integral_constant<bool, sizeof(CharType) == 1>());
+            std::memcpy(v->data() + position, s, length * sizeof(CharType));
-            cursor += length;
+            position += length;
-        }
-        else
-        {
-            std::memcpy(cursor, s, length * sizeof(CharType));
-            cursor += length;
         }
     }
 
@@ -209,37 +144,21 @@ class output_vector_sink
     CharType* reserve(const std::size_t length)
     {
         JSON_ASSERT(length <= max_reserve_size);
-        if (JSON_HEDLEY_UNLIKELY(space() < length))
+        if (JSON_HEDLEY_UNLIKELY(v->size() - position < length))
         {
             grow(length);
         }
-        return cursor;
+        return v->data() + position;
     }
 
     /// keep the first @a length characters written to the space from reserve()
     JSON_HEDLEY_ALWAYS_INLINE
     void commit(const std::size_t length) noexcept
     {
-        cursor += length;
+        position += length;
     }
 
-    /// nothing to pass on: the characters are in the vector (see output_adapter_sink)
-    void flush() noexcept
-    {}
-
   private:
-    /// number of characters of *v written so far
-    std::size_t position() const noexcept
-    {
-        return static_cast<std::size_t>(cursor - v->data());
-    }
-
-    /// number of characters that fit the vector behind the write position
-    std::size_t space() const noexcept
-    {
-        return static_cast<std::size_t>(limit - cursor);
-    }
-
     /// make room for at least @a length more characters after the position;
     /// kept out of line so that the write functions stay small enough for
     /// GCC to inline them into the writer
@@ -247,15 +166,12 @@ class output_vector_sink
     void grow(const std::size_t length)
     {
         // use the capacity the vector already has, else at least double it;
-        // never less than min_vector_sink_capacity, so that a small value does not
+        // never less than 64 characters, so that a small value does not
         // reallocate for every few bytes
-        const std::size_t written = position();
+        const std::size_t needed = position + length;
-        const std::size_t needed = written + length;
+        v->resize((std::max)(std::size_t{64}, needed <= v->capacity()
-        v->resize((std::max)(min_vector_sink_capacity, needed <= v->capacity()
                              ? v->capacity()
                              : (std::max)(needed, v->capacity() * 2)));
-        cursor = v->data() + written;
-        limit = v->data() + v->size();
     }
 
     /// append @a length (at least max_reserve_size) characters that do not fit
@@ -265,24 +181,20 @@ class output_vector_sink
     JSON_HEDLEY_NEVER_INLINE
     void append(const CharType* s, const std::size_t length)
     {
-        const std::size_t written = position();
+        v->resize(position);
-        v->resize(written);
+        if (v->capacity() - position < length)
-        if (v->capacity() - written < length)
         {
             // grow geometrically, with room for the writes that follow, so
             // that the next grow() need not reallocate right away
-            v->reserve((std::max)(v->capacity() * 2, written + length + max_reserve_size));
+            v->reserve((std::max)(v->capacity() * 2, position + length + max_reserve_size));
         }
         v->insert(v->end(), s, s + length);
-        cursor = v->data() + v->size();
+        position = v->size();
-        limit = cursor;
     }
 
     std::vector<CharType, AllocatorType>* v;
-    /// the write position: in *v, at the number of characters written so far
+    /// number of characters of *v written so far
-    CharType* cursor;
+    std::size_t position;
-    /// the end of *v
-    CharType* limit;
 };
 
 /// output adapter for byte vectors
@@ -365,17 +277,9 @@ class output_string_adapter : public output_adapter_protocol<CharType>
 ///
 /// Wraps the polymorphic output_adapter_t so the same binary_writer template can
 /// also target arbitrary adapters (output streams, strings, user-provided
-/// adapters) via the `output_adapter`-based overloads.
+/// adapters) via the `output_adapter`-based overloads. Each write still goes
-///
+/// through one virtual call, exactly as before; only the concrete sinks above
-/// A write through the adapter is a virtual call, and for a stream a call with
+/// avoid it.
-/// a sentry on top, which costs more than encoding the few bytes of most values.
-/// The sink therefore collects what is written in a buffer and passes it on in
-/// one piece when it is full, a write is too large for it, or @ref flush is
-/// called. Everything written reaches the adapter in the same order and with
-/// the same bytes; the destructor passes on what is left, so that a write that
-/// throws leaves the adapter with exactly the bytes written before it. Only
-/// @ref flush can report an error of the adapter (the destructor must not
-/// throw), so it has to be called once the value is written.
 template<typename CharType>
 class output_adapter_sink
 {
@@ -386,121 +290,36 @@ class output_adapter_sink
         JSON_ASSERT(oa);
     }
 
-    output_adapter_sink(const output_adapter_sink&) = delete;
-    output_adapter_sink& operator=(const output_adapter_sink&) = delete;
-    output_adapter_sink& operator=(output_adapter_sink&&) = delete;
-
-    output_adapter_sink(output_adapter_sink&& other) noexcept
-        : oa(std::move(other.oa)), used(other.used)
-    {
-        std::memcpy(buffer.data(), other.buffer.data(), other.used * sizeof(CharType));
-        other.used = 0;
-    }
-
-    ~output_adapter_sink()
-    {
-        // a normal return has flushed already, so this is for a write that threw
-        if (used != 0)
-        {
-            JSON_TRY
-            {
-                flush();
-            }
-            JSON_CATCH(...)
-            {
-                // nothing can be done about an adapter that throws here
-            }
-        }
-    }
-
-    JSON_HEDLEY_ALWAYS_INLINE
     void write_character(CharType c)
     {
-        if (JSON_HEDLEY_UNLIKELY(used == buffer_size))
+        oa->write_character(c);
-        {
-            flush();
-        }
-        buffer[used++] = c;
     }
 
     // no JSON_HEDLEY_NON_NULL: forwards (null, 0) for empty payloads, exactly as
     // the type-erased path already did before this sink existed
-    JSON_HEDLEY_ALWAYS_INLINE
     void write_characters(const CharType* s, std::size_t length)
     {
-        if (JSON_HEDLEY_UNLIKELY(buffer_size - used < length))
+        oa->write_characters(s, length);
-        {
-            write_large(s, length);
-            return;
-        }
-        if (length <= 32)
-        {
-            copy_short(buffer.data() + used, s, length, std::integral_constant<bool, sizeof(CharType) == 1>());
-            used += length;
-        }
-        else
-        {
-            std::memcpy(buffer.data() + used, s, length * sizeof(CharType));
-            used += length;
-        }
     }
 
     /// space for writing up to @a length characters (see
     /// output_vector_sink::reserve)
-    JSON_HEDLEY_ALWAYS_INLINE
+    CharType* reserve(const std::size_t length) noexcept
-    CharType* reserve(const std::size_t length)
     {
         JSON_ASSERT(length <= max_reserve_size);
-        if (JSON_HEDLEY_UNLIKELY(buffer_size - used < length))
+        static_cast<void>(length);
-        {
+        return reserved.data();
-            flush();
-        }
-        return buffer.data() + used;
     }
 
-    /// keep the first @a length characters written to the space from reserve()
+    /// write the first @a length characters written to the space from reserve()
-    JSON_HEDLEY_ALWAYS_INLINE
+    void commit(const std::size_t length)
-    void commit(const std::size_t length) noexcept
     {
-        used += length;
+        oa->write_characters(reserved.data(), length);
-    }
-
-    /// pass everything written so far on to the adapter
-    void flush()
-    {
-        if (used != 0)
-        {
-            // reset first: the characters count as written even if the adapter throws
-            const std::size_t length = used;
-            used = 0;
-            oa->write_characters(buffer.data(), length);
-        }
     }
 
   private:
-    /// a write that does not fit the space left in the buffer
-    JSON_HEDLEY_NEVER_INLINE
-    void write_large(const CharType* s, const std::size_t length)
-    {
-        flush();
-        if (length >= buffer_size)
-        {
-            oa->write_characters(s, length);
-        }
-        else
-        {
-            std::memcpy(buffer.data(), s, length * sizeof(CharType));
-            used = length;
-        }
-    }
-
-    /// large enough for reserve() at any time
-    static constexpr std::size_t buffer_size = 4 * max_reserve_size;
-
     output_adapter_t<CharType> oa;
-    /// number of characters of @ref buffer not yet passed on
+    std::array<CharType, max_reserve_size> reserved{};
-    std::size_t used = 0;
-    std::array<CharType, buffer_size> buffer{};
 };
 
 /// @brief whether std::basic_string<CharType> has a non-deprecated std::char_traits

include/nlohmann/json.hpp

@@ -6321,7 +6321,7 @@ public:
             const error_handler_t error_handler = detail::binary_writer_default_error_handler())
     {
         std::vector<std::uint8_t> result;
-        result.reserve(detail::binary_initial_capacity(j));
+        result.reserve(detail::binary_reserve_hint(j));
         vector_writer(result, error_handler).write_cbor(j);
         return result;
     }
@@ -6331,9 +6331,7 @@ public:
     static void to_cbor(const basic_json& j, detail::output_adapter<std::uint8_t> o,
                         const error_handler_t error_handler = detail::binary_writer_default_error_handler())
     {
-        binary_writer<std::uint8_t> writer(o, error_handler);
+        binary_writer<std::uint8_t>(o, error_handler).write_cbor(j);
-        writer.write_cbor(j);
-        writer.flush();
     }
 
     /// @brief create a CBOR serialization of a given JSON value
@@ -6341,9 +6339,7 @@ public:
     static void to_cbor(const basic_json& j, detail::output_adapter<char> o,
                         const error_handler_t error_handler = detail::binary_writer_default_error_handler())
     {
-        binary_writer<char> writer(o, error_handler);
+        binary_writer<char>(o, error_handler).write_cbor(j);
-        writer.write_cbor(j);
-        writer.flush();
     }
 
     /// @brief create a MessagePack serialization of a given JSON value
@@ -6352,7 +6348,7 @@ public:
             const error_handler_t error_handler = error_handler_t::keep)
     {
         std::vector<std::uint8_t> result;
-        result.reserve(detail::binary_initial_capacity(j));
+        result.reserve(detail::binary_reserve_hint(j));
         vector_writer(result, error_handler).write_msgpack(j);
         return result;
     }
@@ -6362,9 +6358,7 @@ public:
     static void to_msgpack(const basic_json& j, detail::output_adapter<std::uint8_t> o,
                            const error_handler_t error_handler = error_handler_t::keep)
     {
-        binary_writer<std::uint8_t> writer(o, error_handler);
+        binary_writer<std::uint8_t>(o, error_handler).write_msgpack(j);
-        writer.write_msgpack(j);
-        writer.flush();
     }
 
     /// @brief create a MessagePack serialization of a given JSON value
@@ -6372,9 +6366,7 @@ public:
     static void to_msgpack(const basic_json& j, detail::output_adapter<char> o,
                            const error_handler_t error_handler = error_handler_t::keep)
     {
-        binary_writer<char> writer(o, error_handler);
+        binary_writer<char>(o, error_handler).write_msgpack(j);
-        writer.write_msgpack(j);
-        writer.flush();
     }
 
     /// @brief create a UBJSON serialization of a given JSON value
@@ -6385,7 +6377,7 @@ public:
             const error_handler_t error_handler = detail::binary_writer_default_error_handler())
     {
         std::vector<std::uint8_t> result;
-        result.reserve(detail::binary_initial_capacity(j));
+        result.reserve(detail::binary_reserve_hint(j));
         vector_writer(result, error_handler).write_ubjson(j, use_size, use_type);
         return result;
     }
@@ -6396,9 +6388,7 @@ public:
                           const bool use_size = false, const bool use_type = false,
                           const error_handler_t error_handler = detail::binary_writer_default_error_handler())
     {
-        binary_writer<std::uint8_t> writer(o, error_handler);
+        binary_writer<std::uint8_t>(o, error_handler).write_ubjson(j, use_size, use_type);
-        writer.write_ubjson(j, use_size, use_type);
-        writer.flush();
     }
 
     /// @brief create a UBJSON serialization of a given JSON value
@@ -6407,9 +6397,7 @@ public:
                           const bool use_size = false, const bool use_type = false,
                           const error_handler_t error_handler = detail::binary_writer_default_error_handler())
     {
-        binary_writer<char> writer(o, error_handler);
+        binary_writer<char>(o, error_handler).write_ubjson(j, use_size, use_type);
-        writer.write_ubjson(j, use_size, use_type);
-        writer.flush();
     }
 
     /// @brief create a BJData serialization of a given JSON value
@@ -6421,7 +6409,7 @@ public:
             const error_handler_t error_handler = detail::binary_writer_default_error_handler())
     {
         std::vector<std::uint8_t> result;
-        result.reserve(detail::binary_initial_capacity(j));
+        result.reserve(detail::binary_reserve_hint(j));
         vector_writer(result, error_handler).write_ubjson(j, use_size, use_type, true, true, version);
         return result;
     }
@@ -6433,9 +6421,7 @@ public:
                           const bjdata_version_t version = bjdata_version_t::draft2,
                           const error_handler_t error_handler = detail::binary_writer_default_error_handler())
     {
-        binary_writer<std::uint8_t> writer(o, error_handler);
+        binary_writer<std::uint8_t>(o, error_handler).write_ubjson(j, use_size, use_type, true, true, version);
-        writer.write_ubjson(j, use_size, use_type, true, true, version);
-        writer.flush();
     }
 
     /// @brief create a BJData serialization of a given JSON value
@@ -6445,9 +6431,7 @@ public:
                           const bjdata_version_t version = bjdata_version_t::draft2,
                           const error_handler_t error_handler = detail::binary_writer_default_error_handler())
     {
-        binary_writer<char> writer(o, error_handler);
+        binary_writer<char>(o, error_handler).write_ubjson(j, use_size, use_type, true, true, version);
-        writer.write_ubjson(j, use_size, use_type, true, true, version);
-        writer.flush();
     }
 
     /// @brief create a BSON serialization of a given JSON value
@@ -6456,7 +6440,7 @@ public:
             const error_handler_t error_handler = detail::binary_writer_default_error_handler())
     {
         std::vector<std::uint8_t> result;
-        result.reserve(detail::binary_initial_capacity(j));
+        result.reserve(detail::binary_reserve_hint(j));
         vector_writer(result, error_handler).write_bson(j);
         return result;
     }
@@ -6466,9 +6450,7 @@ public:
     static void to_bson(const basic_json& j, detail::output_adapter<std::uint8_t> o,
                         const error_handler_t error_handler = detail::binary_writer_default_error_handler())
     {
-        binary_writer<std::uint8_t> writer(o, error_handler);
+        binary_writer<std::uint8_t>(o, error_handler).write_bson(j);
-        writer.write_bson(j);
-        writer.flush();
     }
 
     /// @brief create a BSON serialization of a given JSON value
@@ -6476,9 +6458,7 @@ public:
     static void to_bson(const basic_json& j, detail::output_adapter<char> o,
                         const error_handler_t error_handler = detail::binary_writer_default_error_handler())
     {
-        binary_writer<char> writer(o, error_handler);
+        binary_writer<char>(o, error_handler).write_bson(j);
-        writer.write_bson(j);
-        writer.flush();
     }
 
     /// @brief create a BON8 serialization of a given JSON value
@@ -6486,7 +6466,7 @@ public:
     static std::vector<std::uint8_t> to_bon8(const basic_json& j)
     {
         std::vector<std::uint8_t> result;
-        result.reserve(detail::binary_initial_capacity(j));
+        result.reserve(detail::binary_reserve_hint(j));
         vector_writer(result).write_bon8(j);
         return result;
     }
@@ -6495,18 +6475,14 @@ public:
     /// @sa https://json.nlohmann.me/api/basic_json/to_bon8/
     static void to_bon8(const basic_json& j, detail::output_adapter<std::uint8_t> o)
     {
-        binary_writer<std::uint8_t> writer(o);
+        binary_writer<std::uint8_t>(o).write_bon8(j);
-        writer.write_bon8(j);
-        writer.flush();
     }
 
     /// @brief create a BON8 serialization of a given JSON value
     /// @sa https://json.nlohmann.me/api/basic_json/to_bon8/
     static void to_bon8(const basic_json& j, detail::output_adapter<char> o)
     {
-        binary_writer<char> writer(o);
+        binary_writer<char>(o).write_bon8(j);
-        writer.write_bon8(j);
-        writer.flush();
     }
 
     /// @brief create a JSON value from an input in CBOR format

tests/src/unit-binary_writer_sinks.cpp

@@ -492,275 +492,3 @@ TEST_CASE("binary_reserve_hint never over-reserves")
         CHECK(nlohmann::detail::binary_reserve_hint(json({{"a", 1}, {"b", 2}})) == 5);
     }
 }
-
-namespace
-{
-
-// numbers around every boundary of the integer encodings, and floats of each
-// kind (exact in float, needing double, special values)
-std::vector<json> boundary_numbers()
-{
-    std::vector<json> numbers;
-    for (int k = 0; k < 64; ++k)
-    {
-        const std::uint64_t p = std::uint64_t{1} << k;
-        for (const std::uint64_t d :
-                {
-                    p - 1, p, p + 1
-                })
-        {
-            numbers.emplace_back(d);
-            if (d <= static_cast<std::uint64_t>((std::numeric_limits<std::int64_t>::max)()))
-            {
-                numbers.emplace_back(static_cast<std::int64_t>(d));
-                numbers.emplace_back(-static_cast<std::int64_t>(d));
-                numbers.emplace_back(-static_cast<std::int64_t>(d) - 1);
-            }
-        }
-    }
-    numbers.emplace_back((std::numeric_limits<std::int64_t>::min)());
-    numbers.emplace_back((std::numeric_limits<std::uint64_t>::max)());
-    for (const double f :
-            {
-                0.0, -0.0, 1.0, -1.0, 0.5, 0.1, 3.25, 1e300, -1e-300, 3.4028234663852886e38, 1e39,
-                std::numeric_limits<double>::infinity(), -std::numeric_limits<double>::infinity(),
-                std::numeric_limits<double>::quiet_NaN()
-            })
-    {
-        numbers.emplace_back(f);
-    }
-    return numbers;
-}
-
-// the concatenation of the encodings of the elements of @a numbers[first, first + count)
-template<typename Single>
-std::vector<std::uint8_t> concat_singles(const std::vector<json>& numbers, const std::size_t first, const std::size_t count, Single single)
-{
-    std::vector<std::uint8_t> bytes;
-    for (std::size_t i = first; i < first + count; ++i)
-    {
-        const auto one = single(numbers[i]);
-        bytes.insert(bytes.end(), one.begin(), one.end());
-    }
-    return bytes;
-}
-
-} // namespace
-
-// Runs of numbers in arrays are encoded straight into space reserved from the
-// sink, by code that is separate from the encoding of a single value. Both have
-// to agree on every number: an array is its head and the encodings of its
-// elements, one after the other.
-TEST_CASE("runs of numbers in arrays are encoded as the numbers one by one")
-{
-    const std::vector<json> numbers = boundary_numbers();
-
-    // arrays of every length that a one-byte head covers, so that runs are as
-    // long as, and longer and shorter than, what fits one reservation
-    for (const std::size_t count :
-            {
-                1u, 2u, 3u, 5u, 14u, 15u, 23u
-            })
-    {
-        for (std::size_t first = 0; first + count <= numbers.size(); first += 7)
-        {
-            const json array(std::vector<json>(numbers.begin() + static_cast<std::ptrdiff_t>(first),
-                                               numbers.begin() + static_cast<std::ptrdiff_t>(first + count)));
-            CAPTURE(array.dump(-1, ' ', false, json::error_handler_t::replace))
-
-            std::vector<std::uint8_t> expected{static_cast<std::uint8_t>(0x80 + count)};
-            auto elements = concat_singles(numbers, first, count, [](const json & j)
-            {
-                return json::to_cbor(j);
-            });
-            expected.insert(expected.end(), elements.begin(), elements.end());
-            CHECK(json::to_cbor(array) == expected);
-
-            expected.assign(1, static_cast<std::uint8_t>(count <= 15 ? 0x90 + count : 0xDC));
-            if (count > 15)
-            {
-                expected.push_back(0);
-                expected.push_back(static_cast<std::uint8_t>(count));
-            }
-            elements = concat_singles(numbers, first, count, [](const json & j)
-            {
-                return json::to_msgpack(j);
-            });
-            expected.insert(expected.end(), elements.begin(), elements.end());
-            CHECK(json::to_msgpack(array) == expected);
-
-            for (const bool bjdata :
-                    {
-                        false, true
-                    })
-            {
-                expected.assign(1, static_cast<std::uint8_t>('['));
-                elements = concat_singles(numbers, first, count, [bjdata](const json & j)
-                {
-                    return bjdata ? json::to_bjdata(j) : json::to_ubjson(j);
-                });
-                expected.insert(expected.end(), elements.begin(), elements.end());
-                expected.push_back(static_cast<std::uint8_t>(']'));
-                CHECK((bjdata ? json::to_bjdata(array) : json::to_ubjson(array)) == expected);
-            }
-
-            // BON8 has no integers above the int64 range
-            bool representable = true;
-            for (std::size_t i = first; i < first + count; ++i)
-            {
-                representable = representable && bon8_representable(numbers[i]);
-            }
-            if (representable)
-            {
-                expected.assign(1, static_cast<std::uint8_t>(count <= 4 ? 0x80 + count : 0x85));
-                elements = concat_singles(numbers, first, count, [](const json & j)
-                {
-                    return json::to_bon8(j);
-                });
-                expected.insert(expected.end(), elements.begin(), elements.end());
-                if (count > 4)
-                {
-                    expected.push_back(0xFE);
-                }
-                CHECK(json::to_bon8(array) == expected);
-            }
-            else
-            {
-                CHECK_THROWS_AS(json::to_bon8(array), json::out_of_range&);
-            }
-        }
-    }
-}
-
-// An array that is written in more than one reservation (more numbers than fit
-// one), and arrays of an index with more digits, round trip for every format.
-TEST_CASE("large arrays and many digits of indices round trip")
-{
-    json array = json::array();
-    for (int i = 0; i < 12000; ++i)
-    {
-        switch (i % 4)
-        {
-            case 0:
-                array.push_back(i);
-                break;
-            case 1:
-                array.push_back(-i);
-                break;
-            case 2:
-                array.push_back(i * 0.25);
-                break;
-            default:
-                array.push_back(i % 8 == 3);
-                break;
-        }
-    }
-    json document = json::object();
-    document["numbers"] = array;
-    document["nested"] = json::array({array, "text", array});
-
-    CHECK(json::from_cbor(json::to_cbor(document)) == document);
-    CHECK(json::from_msgpack(json::to_msgpack(document)) == document);
-    CHECK(json::from_ubjson(json::to_ubjson(document)) == document);
-    CHECK(json::from_ubjson(json::to_ubjson(document, true, true)) == document);
-    CHECK(json::from_bjdata(json::to_bjdata(document)) == document);
-    CHECK(json::from_bson(json::to_bson(document)) == document);
-    CHECK(json::from_bon8(json::to_bon8(document)) == document);
-}
-
-// Strings and keys are written with their head in one reservation up to a
-// length; around that length, and with every kind of head, they must read back.
-TEST_CASE("strings of every head length round trip")
-{
-    json document = json::object();
-    json array = json::array();
-    for (const std::size_t length :
-            {
-                0u, 1u, 2u, 3u, 4u, 7u, 8u, 15u, 16u, 17u, 23u, 24u, 25u, 31u, 32u, 33u, 63u, 127u, 128u, 129u, 200u, 245u, 246u, 247u, 248u, 249u, 250u, 251u, 252u, 253u, 254u, 255u, 256u, 257u, 300u, 1000u, 65535u, 65536u, 70000u
-            })
-    {
-        const std::string text(length, 'x');
-        array.push_back(text);
-        document[std::string(length, 'k')] = text;
-    }
-    document["array"] = array;
-
-    CHECK(json::from_cbor(json::to_cbor(document)) == document);
-    CHECK(json::from_msgpack(json::to_msgpack(document)) == document);
-    CHECK(json::from_ubjson(json::to_ubjson(document)) == document);
-    CHECK(json::from_ubjson(json::to_ubjson(document, true, true)) == document);
-    CHECK(json::from_bjdata(json::to_bjdata(document)) == document);
-    CHECK(json::from_bson(json::to_bson(document)) == document);
-    CHECK(json::from_bon8(json::to_bon8(document)) == document);
-}
-
-// What a write that throws leaves behind in an output adapter is what was written
-// before it, whatever the adapter and however the adapter sink buffers.
-TEST_CASE("a write that throws leaves the same bytes in every kind of output")
-{
-    json array = json::array();
-    for (int i = 0; i < 700; ++i)
-    {
-        array.push_back(i * 1000);
-        array.push_back("text " + std::to_string(i));
-    }
-    array.push_back("bad \xFF byte");
-    json document = json::object();
-    document["first"] = array;
-    document["second"] = 1;
-
-    const auto strict = json::error_handler_t::strict;
-
-    SECTION("cbor")
-    {
-        std::vector<std::uint8_t> as_vector;
-        std::string as_string;
-        std::ostringstream as_stream;
-        CHECK_THROWS_AS(json::to_cbor(document, as_vector, strict), json::type_error&);
-        CHECK_THROWS_AS(json::to_cbor(document, as_string, strict), json::type_error&);
-        CHECK_THROWS_AS(json::to_cbor(document, as_stream, strict), json::type_error&);
-        CHECK(!as_vector.empty());
-        CHECK(std::string(reinterpret_cast<const char*>(as_vector.data()), as_vector.size()) == as_string); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
-        CHECK(as_stream.str() == as_string);
-    }
-
-    SECTION("msgpack")
-    {
-        std::vector<std::uint8_t> as_vector;
-        std::string as_string;
-        std::ostringstream as_stream;
-        CHECK_THROWS_AS(json::to_msgpack(document, as_vector, strict), json::type_error&);
-        CHECK_THROWS_AS(json::to_msgpack(document, as_string, strict), json::type_error&);
-        CHECK_THROWS_AS(json::to_msgpack(document, as_stream, strict), json::type_error&);
-        CHECK(!as_vector.empty());
-        CHECK(std::string(reinterpret_cast<const char*>(as_vector.data()), as_vector.size()) == as_string); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
-        CHECK(as_stream.str() == as_string);
-    }
-
-    SECTION("ubjson")
-    {
-        std::vector<std::uint8_t> as_vector;
-        std::string as_string;
-        std::ostringstream as_stream;
-        CHECK_THROWS_AS(json::to_ubjson(document, as_vector, false, false, strict), json::type_error&);
-        CHECK_THROWS_AS(json::to_ubjson(document, as_string, false, false, strict), json::type_error&);
-        CHECK_THROWS_AS(json::to_ubjson(document, as_stream, false, false, strict), json::type_error&);
-        CHECK(!as_vector.empty());
-        CHECK(std::string(reinterpret_cast<const char*>(as_vector.data()), as_vector.size()) == as_string); // NOLINT(cppcoreguidelines-pro-type-reinterpret-cast)
-        CHECK(as_stream.str() == as_string);
-    }
-
-    SECTION("bson")
-    {
-        // the size pass finds the string before anything is written
-        std::vector<std::uint8_t> as_vector;
-        std::string as_string;
-        std::ostringstream as_stream;
-        CHECK_THROWS_AS(json::to_bson(document, as_vector, strict), json::type_error&);
-        CHECK_THROWS_AS(json::to_bson(document, as_string, strict), json::type_error&);
-        CHECK_THROWS_AS(json::to_bson(document, as_stream, strict), json::type_error&);
-        CHECK(as_vector.empty());
-        CHECK(as_string.empty());
-        CHECK(as_stream.str().empty());
-    }
-}

tests/src/unit-custom-number-types.cpp

@@ -285,55 +285,6 @@ TEST_CASE("custom number types: integers beyond 64 bits")
         CHECK(wide_json::to_bjdata(wide_json::array({uint_big, uint_big}), true, true) == expected_bjdata_array);
     }
 
-    SECTION("integers in arrays and objects")
-    {
-        // runs of numbers and scalar object members are encoded by loops of
-        // their own for the number types up to 64 bits; a wider number type
-        // takes the generic path, which checks the range of every integer
-        const wide_json run = wide_json::array({make_int(1), make_int(-3), make_uint(7), wide_json(1.5), int_big, make_int(5)});
-        const wide_json run_negative = wide_json::array({make_int(1), make_int(-3), int_big_negative});
-        const wide_json run_unsigned = wide_json::array({make_uint(1), make_uint(7), uint_big});
-        const wide_json members = wide_json::object({{"a", make_int(1)}, {"b", wide_json(1.5)}, {"c", int_big}});
-
-        const std::vector<wide_json> failing = {run, run_negative, run_unsigned, members, wrap(run), wrap(members)};
-        for (const auto& value : failing)
-        {
-            CHECK_THROWS_AS(_ = wide_json::to_cbor(value), out_of_range&);
-            CHECK_THROWS_AS(_ = wide_json::to_msgpack(value), out_of_range&);
-            CHECK_THROWS_AS(_ = wide_json::to_bson(wrap(value)), out_of_range&);
-            CHECK_THROWS_AS(_ = wide_json::to_bon8(value), out_of_range&);
-        }
-
-        // a failing write leaves no more than the elements before the failing one
-        std::vector<std::uint8_t> out;
-        CHECK_THROWS_AS(wide_json::to_cbor(run, out), out_of_range&);
-        // (the head of the array, 1, -3, 7, and 1.5 as a single-precision float)
-        CHECK(out == std::vector<std::uint8_t>({0x86, 0x01, 0x22, 0x07, 0xFA, 0x3F, 0xC0, 0x00, 0x00}));
-        out.clear();
-        CHECK_THROWS_AS(wide_json::to_bon8(run, out), out_of_range&);
-        CHECK(!out.empty());
-
-        // the same values that fit are written exactly as with 64-bit number types
-        const wide_json fits = wide_json::array({make_int(1), make_int(-3), make_uint(7), wide_json(1.5), make_int(min_int64), make_int(max_int64), make_int(200), make_int(-70000)});
-        const nlohmann::json fits64 = nlohmann::json::array({1, -3, 7, 1.5, (std::numeric_limits<std::int64_t>::min)(), (std::numeric_limits<std::int64_t>::max)(), 200, -70000});
-        CHECK(wide_json::to_cbor(fits) == nlohmann::json::to_cbor(fits64));
-        CHECK(wide_json::to_msgpack(fits) == nlohmann::json::to_msgpack(fits64));
-        CHECK(wide_json::to_ubjson(fits) == nlohmann::json::to_ubjson(fits64));
-        CHECK(wide_json::to_ubjson(fits, true, true) == nlohmann::json::to_ubjson(fits64, true, true));
-        CHECK(wide_json::to_bjdata(fits) == nlohmann::json::to_bjdata(fits64));
-        CHECK(wide_json::to_bjdata(fits, true, true) == nlohmann::json::to_bjdata(fits64, true, true));
-        CHECK(wide_json::to_bon8(fits) == nlohmann::json::to_bon8(fits64));
-        CHECK(wide_json::to_bson(wrap(fits)) == nlohmann::json::to_bson(nlohmann::json::object({{"v", fits64}})));
-
-        const wide_json fit_members = wide_json::object({{"a", make_int(1)}, {"b", make_int(-300)}, {"c", make_uint(max_uint64)}, {"d", wide_json(2.5)}});
-        const nlohmann::json fit_members64 = nlohmann::json::object({{"a", 1}, {"b", -300}, {"c", (std::numeric_limits<std::uint64_t>::max)()}, {"d", 2.5}});
-        CHECK(wide_json::to_cbor(fit_members) == nlohmann::json::to_cbor(fit_members64));
-        CHECK(wide_json::to_msgpack(fit_members) == nlohmann::json::to_msgpack(fit_members64));
-        CHECK(wide_json::to_ubjson(fit_members) == nlohmann::json::to_ubjson(fit_members64));
-        CHECK(wide_json::to_bjdata(fit_members) == nlohmann::json::to_bjdata(fit_members64));
-        CHECK(wide_json::to_bson(fit_members) == nlohmann::json::to_bson(fit_members64));
-    }
-
     SECTION("BJData ND-array")
     {
         // an ND-array element beyond 64 bits does not fit any dtype, so the