From 0a4c1a2d4235030dc218ae2111663495a0c1f540 Mon Sep 17 00:00:00 2001 From: Niels Lohmann Date: Thu, 22 Oct 2026 13:58:45 +0200 Subject: [PATCH] docs: cross-reference json_view and json_document for 3.13.0 Document the new zero-copy document/view API across the docs: add a "Documents and views" section to the basic_json API hub linking to all specializations, point parsing/conversion/modifying pages to the view-based alternatives, add document/view sections to feature pages, update the landing page to mention json_view.hpp/json_literals.hpp, and cross-link debugging/exceptions. Also document wide-string inputs (and ill-formed UTF-16/32 rejection), add parse/accept examples for wide strings, fix related doc gaps, and align features/parsing and FAQ with the added support. Signed-off-by: Niels Lohmann --- docs/mkdocs/docs/api/basic_json/accept.md | 26 +++++++++++++- docs/mkdocs/docs/api/basic_json/index.md | 19 ++++++++++ docs/mkdocs/docs/api/basic_json/parse.md | 35 ++++++++++++++++++- docs/mkdocs/docs/api/basic_json/sax_parse.md | 11 ++++++ .../docs/api/basic_json_document/index.md | 6 ++++ .../docs/api/basic_json_document/parse.md | 17 ++++++--- docs/mkdocs/docs/api/basic_json_view/index.md | 6 ++++ .../docs/examples/accept__wide_string.cpp | 18 ++++++++++ .../docs/examples/accept__wide_string.output | 1 + .../docs/examples/parse__wide_string.cpp | 27 ++++++++++++++ .../docs/examples/parse__wide_string.output | 6 ++++ docs/mkdocs/docs/features/conversions.md | 11 ++++++ docs/mkdocs/docs/features/creating_values.md | 5 ++- docs/mkdocs/docs/features/macros.md | 17 +++++++++ docs/mkdocs/docs/features/modifying_values.md | 11 ++++++ docs/mkdocs/docs/features/parsing/index.md | 29 ++++++++++++++- .../docs/features/parsing/untrusted_input.md | 3 ++ docs/mkdocs/docs/home/debugging.md | 4 +-- docs/mkdocs/docs/home/exceptions.md | 3 +- docs/mkdocs/docs/home/faq.md | 7 ++-- docs/mkdocs/docs/index.md | 10 ++++-- 21 files changed, 254 insertions(+), 18 deletions(-) create mode 100644 docs/mkdocs/docs/examples/accept__wide_string.cpp create mode 100644 docs/mkdocs/docs/examples/accept__wide_string.output create mode 100644 docs/mkdocs/docs/examples/parse__wide_string.cpp create mode 100644 docs/mkdocs/docs/examples/parse__wide_string.output diff --git a/docs/mkdocs/docs/api/basic_json/accept.md b/docs/mkdocs/docs/api/basic_json/accept.md index 41c6969a1..345d0d507 100644 --- a/docs/mkdocs/docs/api/basic_json/accept.md +++ b/docs/mkdocs/docs/api/basic_json/accept.md @@ -23,6 +23,9 @@ Checks whether the input is valid JSON. respectively as UTF-8, UTF-16, and UTF-32. If `SentinelType` differs from `IteratorType`, it must be comparable to the iterator type with `operator!=`. + Ill-formed UTF-16 and UTF-32 sequences are rejected: a lone surrogate, a high surrogate that is not followed by its + low surrogate, and a code point above `U+10FFFF` all make this function return `false`. + Unlike the [`parse()`](parse.md) function, this function neither throws an exception in case of invalid JSON input (i.e., a parse error) nor creates diagnostic information. @@ -35,9 +38,12 @@ Unlike the [`parse()`](parse.md) function, this function neither throws an excep - a `#!c FILE` pointer (throws if null) - a C-style array of characters - a pointer to a null-terminated string of single byte characters (throws if null) - - a `std::string` + - a `std::string` or a `std::string_view` (C++17) - a container `obj` for which `begin(obj)` and `end(obj)` produce a valid pair of iterators (as found via ADL or member functions, with semantics compatible to `std::begin` and `std::end`) + - wide characters (`wchar_t`, `char16_t`, `char32_t`), which are converted to UTF-8: a wide string object + (`std::wstring`, `std::u16string`, `std::u32string`) or an array (e.g. a wide string literal); a pointer to a + null-terminated wide string (e.g. `std::u16string::c_str()`) is not supported `IteratorType` : a compatible iterator type, for instance. @@ -125,6 +131,21 @@ outside of a string, invalid) byte; see the [FAQ entry](../../home/faq.md#nul-by --8<-- "examples/accept__iterator_pair.output" ``` +??? example "Example: (3) reading from UTF-16 strings" + + The encoding is deduced from the character type of the input. String literals like `u"..."`, `U"..."`, and + `L"..."` work as well. + + ```cpp + --8<-- "examples/accept__wide_string.cpp" + ``` + + Output: + + ```json + --8<-- "examples/accept__wide_string.output" + ``` + ## See also - [parse](parse.md) - deserialize from a compatible input @@ -151,6 +172,9 @@ outside of a string, invalid) byte; see the [FAQ entry](../../home/faq.md#nul-by ([`JSON_DISABLE_STREAM_BULK_SCAN`](../macros/json_disable_stream_bulk_scan.md)). - Parsing from a `FILE*` locks the file once instead of once per character in version 3.13.0 ([`JSON_DISABLE_UNLOCKED_FILE_IO`](../macros/json_disable_unlocked_file_io.md)). +- Ill-formed UTF-16 and UTF-32 input (a lone surrogate, a high surrogate that is not followed by its low surrogate, or + a code point above `U+10FFFF`) is rejected in version 3.13.0; before that, some such sequences were accepted or + ended the input silently. !!! warning "Deprecation" diff --git a/docs/mkdocs/docs/api/basic_json/index.md b/docs/mkdocs/docs/api/basic_json/index.md index 8d945de9b..acb67401f 100644 --- a/docs/mkdocs/docs/api/basic_json/index.md +++ b/docs/mkdocs/docs/api/basic_json/index.md @@ -44,6 +44,22 @@ are collected in [Template Parameter Requirements](../../features/types/template - [**json**](../json.md) - default specialization - [**ordered_json**](../ordered_json.md) - a specialization that maintains the insertion order of object keys +## Documents and views + +Besides `basic_json`, the library offers a second way to hold a parsed JSON text: a +[`basic_json_document`](../basic_json_document/index.md), which owns or borrows the text and holds a flat index of the +values in it, and a [`basic_json_view`](../basic_json_view/index.md), a read-only handle to one value of such a document. +Both are defined in the header ``; see +[Zero-copy JSON views](../../features/json_view.md) for the concept and the trade-offs. + +- [**json_document**](../json_document.md) / [**ordered_json_document**](../ordered_json_document.md) - read-only + documents of the `basic_json` specialization they belong to +- [**json_editable_document**](../json_editable_document.md) / + [**ordered_json_editable_document**](../ordered_json_editable_document.md) - documents that can be modified in place +- [**json_view**](../json_view.md) / [**ordered_json_view**](../ordered_json_view.md) - views of a document +- [**json_editable_view**](../json_editable_view.md) / + [**ordered_json_editable_view**](../ordered_json_editable_view.md) - views of an editable document + ## Iterator invalidation All operations that add values to an **array** ([`push_back`](push_back.md) , [`operator+=`](operator+=.md), @@ -343,6 +359,9 @@ Access to the JSON value ## See also - [RFC 8259: The JavaScript Object Notation (JSON) Data Interchange Format](https://tools.ietf.org/html/rfc8259) +- [basic_json_document](../basic_json_document/index.md) / [basic_json_view](../basic_json_view/index.md) - parse a JSON + text into a flat index and read it through a view +- [Zero-copy JSON views](../../features/json_view.md) - when to use a document and a view instead of a `basic_json` ## Version history diff --git a/docs/mkdocs/docs/api/basic_json/parse.md b/docs/mkdocs/docs/api/basic_json/parse.md index f2e7eda69..a069f2086 100644 --- a/docs/mkdocs/docs/api/basic_json/parse.md +++ b/docs/mkdocs/docs/api/basic_json/parse.md @@ -25,6 +25,10 @@ static basic_json parse(IteratorType first, SentinelType last, respectively as UTF-8, UTF-16, and UTF-32. If `SentinelType` differs from `IteratorType`, it must be comparable to the iterator type with `operator!=`. + Ill-formed UTF-16 and UTF-32 sequences are rejected: a lone surrogate, a high surrogate that is not followed by its + low surrogate, and a code point above `U+10FFFF` all yield a + [`parse_error.101`](../../home/exceptions.md#jsonexceptionparse_error101). + ## Template parameters `InputType` @@ -34,9 +38,12 @@ static basic_json parse(IteratorType first, SentinelType last, - a `FILE` pointer (throws if null) - a C-style array of characters - a pointer to a null-terminated string of single byte characters (throws if null) - - a `std::string` + - a `std::string` or a `std::string_view` (C++17) - a container `obj` for which `begin(obj)` and `end(obj)` produce a valid pair of iterators (as found via ADL or member functions, with semantics compatible to `std::begin` and `std::end`) + - wide characters (`wchar_t`, `char16_t`, `char32_t`), which are converted to UTF-8: a wide string object + (`std::wstring`, `std::u16string`, `std::u32string`) or an array (e.g. a wide string literal); a pointer to a + null-terminated wide string (e.g. `std::u16string::c_str()`) is not supported `IteratorType` : a compatible iterator type, for instance. @@ -112,6 +119,13 @@ By default, a `'\0'` (NUL) byte anywhere in the input is treated as end of input outside of a string, invalid) byte; see the [FAQ entry](../../home/faq.md#nul-bytes-in-the-input) for details and the [`JSON_STRICT_NUL_HANDLING`](../macros/json_strict_nul_handling.md) macro to opt into rejecting it instead. +!!! tip "Parsing into a document instead of a value" + + [`basic_json_document::parse`](../basic_json_document/parse.md) accepts the same inputs and the same options, but + keeps the result as a flat index of the parsed values instead of a tree of `basic_json` values. It is faster and uses + less memory when only parts of the document are read, and it also accepts a pointer to a null-terminated wide string, + which `parse()` does not. See [Zero-copy JSON views](../../features/json_view.md). + !!! tip "Performance" Contiguous inputs (strings, string views, vectors, arrays, pointer ranges) are the fastest to parse. A @@ -250,6 +264,22 @@ outside of a string, invalid) byte; see the [FAQ entry](../../home/faq.md#nul-by --8<-- "examples/trailing_commas.output" ``` +??? example "Example: parse from UTF-16, UTF-32, and wide strings" + + The encoding is deduced from the character type of the input; `std::u16string` and `std::u32string` are interpreted + as UTF-16 and UTF-32, and `std::wstring` uses the platform's `wchar_t` width. String literals like `u"..."`, + `U"..."`, and `L"..."` work as well, as do iterators over wide characters. + + ```cpp + --8<-- "examples/parse__wide_string.cpp" + ``` + + Output: + + ```json + --8<-- "examples/parse__wide_string.output" + ``` + ## See also - [accept](accept.md) - check if the input is valid JSON @@ -280,6 +310,9 @@ outside of a string, invalid) byte; see the [FAQ entry](../../home/faq.md#nul-by time). - Parsing from a `FILE*` locks the file once instead of once per character in version 3.13.0 ([`JSON_DISABLE_UNLOCKED_FILE_IO`](../macros/json_disable_unlocked_file_io.md) restores `std::fgetc`). +- Ill-formed UTF-16 and UTF-32 input (a lone surrogate, a high surrogate that is not followed by its low surrogate, or + a code point above `U+10FFFF`) is rejected with `parse_error.101` in version 3.13.0; before that, some such + sequences were accepted or ended the input silently. !!! warning "Deprecation" diff --git a/docs/mkdocs/docs/api/basic_json/sax_parse.md b/docs/mkdocs/docs/api/basic_json/sax_parse.md index b13147b05..0b0f1dee2 100644 --- a/docs/mkdocs/docs/api/basic_json/sax_parse.md +++ b/docs/mkdocs/docs/api/basic_json/sax_parse.md @@ -31,6 +31,10 @@ Read from input and generate SAX events respectively as UTF-8, UTF-16, and UTF-32. If `SentinelType` differs from `IteratorType`, it must be comparable to the iterator type with `operator!=`. + Ill-formed UTF-16 and UTF-32 sequences are rejected: a lone surrogate, a high surrogate that is not followed by its + low surrogate, and a code point above `U+10FFFF` all yield a + [`parse_error.101`](../../home/exceptions.md#jsonexceptionparse_error101). + The SAX event lister must follow the interface of [`json_sax`](../json_sax/index.md). ## Template parameters @@ -42,8 +46,12 @@ The SAX event lister must follow the interface of [`json_sax`](../json_sax/index - a `FILE` pointer - a C-style array of characters - a pointer to a null-terminated string of single byte characters + - a `std::string` or a `std::string_view` (C++17) - a container `obj` for which `begin(obj)` and `end(obj)` produce a valid pair of iterators (as found via ADL or member functions, with semantics compatible to `std::begin` and `std::end`) + - wide characters (`wchar_t`, `char16_t`, `char32_t`), which are converted to UTF-8: a wide string object + (`std::wstring`, `std::u16string`, `std::u32string`) or an array (e.g. a wide string literal); a pointer to a + null-terminated wide string (e.g. `std::u16string::c_str()`) is not supported `IteratorType` : a compatible iterator type for overload (2); a pair of character iterators whose `value_type` is an integral type @@ -160,6 +168,9 @@ use the same file with stdio functions, but it must not wait for another thread ([`JSON_DISABLE_STREAM_BULK_SCAN`](../macros/json_disable_stream_bulk_scan.md)). - Parsing from a `FILE*` locks the file once instead of once per character in version 3.13.0 ([`JSON_DISABLE_UNLOCKED_FILE_IO`](../macros/json_disable_unlocked_file_io.md)). +- Ill-formed UTF-16 and UTF-32 input (a lone surrogate, a high surrogate that is not followed by its low surrogate, or + a code point above `U+10FFFF`) is rejected with `parse_error.101` in version 3.13.0; before that, some such + sequences were accepted or ended the input silently. !!! warning "Deprecation" diff --git a/docs/mkdocs/docs/api/basic_json_document/index.md b/docs/mkdocs/docs/api/basic_json_document/index.md index d1321959a..ff36d5a85 100644 --- a/docs/mkdocs/docs/api/basic_json_document/index.md +++ b/docs/mkdocs/docs/api/basic_json_document/index.md @@ -115,6 +115,12 @@ points apply to every edit: [`source_offset()`](../basic_json_view/source_offset.md) of a value introduced by an edit is `#!cpp static_cast(-1)`, the same value it reports for a decoded string. +## See also + +- [basic_json_view](../basic_json_view/index.md) - the read-only handle to a value of a document +- [json_document](../json_document.md) - the document type of the default specialization +- [Zero-copy JSON views](../../features/json_view.md) - the concept behind documents and views + ## Version history - Added in version 3.13.0. diff --git a/docs/mkdocs/docs/api/basic_json_document/parse.md b/docs/mkdocs/docs/api/basic_json_document/parse.md index 7bc00a9f1..fe59aa04e 100644 --- a/docs/mkdocs/docs/api/basic_json_document/parse.md +++ b/docs/mkdocs/docs/api/basic_json_document/parse.md @@ -21,17 +21,18 @@ static basic_json_document parse(IteratorType first, IteratorType last, Both overloads accept the same JSON text as [`BasicJsonType::parse()`](../basic_json/parse.md), with the same `ignore_comments`/`ignore_trailing_commas` options, but build a [`basic_json_document`](index.md) (a flat index into -the input) instead of a tree of `BasicJsonType` values. They take the same inputs as `BasicJsonType::parse()`. Byte -inputs that stay alive can be borrowed; all others (streams, `#!cpp FILE*`, wide characters, non-contiguous -iterators) are read into a copy that the document owns -- see the Notes. Use [`parse_copy()`](parse_copy.md) to get an -owned copy of a borrowable input as well. +the input) instead of a tree of `BasicJsonType` values. They take the same inputs as `BasicJsonType::parse()`, plus a +pointer to a null-terminated wide string, which `BasicJsonType::parse()` does not accept. Byte inputs that stay alive +can be borrowed; all others (streams, `#!cpp FILE*`, wide characters, non-contiguous iterators) are read into a copy +that the document owns -- see the Notes. Use [`parse_copy()`](parse_copy.md) to get an owned copy of a borrowable input +as well. ## Template parameters `InputType` : A compatible input, for instance: - - a `#!cpp std::string`, `#!cpp std::string_view`, or a C-style array of single-byte characters + - a `#!cpp std::string`, a `#!cpp std::string_view` (C++17), or a C-style array of single-byte characters - a pointer to a null-terminated string of single-byte characters (`#!cpp char`, `#!cpp signed char`, `#!cpp unsigned char`, `#!cpp std::uint8_t`) - a container for which `#!cpp obj.data()` and `#!cpp obj.size()` give contiguous single-byte access, e.g. @@ -47,6 +48,10 @@ owned copy of a borrowable input as well. `#!cpp std::string::iterator`; the iterators of single-byte characters are borrowed or read like the byte inputs above, those of wide characters are converted to UTF-8 +Ill-formed UTF-16 and UTF-32 sequences are rejected: a lone surrogate, a high surrogate that is not followed by its low +surrogate, and a code point above `U+10FFFF` all yield a +[`parse_error.101`](../../home/exceptions.md#jsonexceptionparse_error101). + ## Parameters `input` (in) @@ -155,3 +160,5 @@ buffer of a given length, pass a pair of pointers: `#!cpp parse(ptr, ptr + len)` ## Version history - Added in version 3.13.0. +- Ill-formed UTF-16 and UTF-32 input (a lone surrogate, a high surrogate that is not followed by its low surrogate, or + a code point above `U+10FFFF`) is rejected with `parse_error.101`. diff --git a/docs/mkdocs/docs/api/basic_json_view/index.md b/docs/mkdocs/docs/api/basic_json_view/index.md index 4d6d7dd05..69913d650 100644 --- a/docs/mkdocs/docs/api/basic_json_view/index.md +++ b/docs/mkdocs/docs/api/basic_json_view/index.md @@ -143,6 +143,12 @@ still refers to it -- including ones taken before the change -- reads the new va - [**source_offset**](source_offset.md) - byte offset of this value in the document's source text +## See also + +- [basic_json_document](../basic_json_document/index.md) - the document a view refers into +- [materialize](materialize.md) - build the `BasicJsonType` value of a subtree +- [Zero-copy JSON views](../../features/json_view.md) - the concept behind documents and views + ## Version history - Added in version 3.13.0. diff --git a/docs/mkdocs/docs/examples/accept__wide_string.cpp b/docs/mkdocs/docs/examples/accept__wide_string.cpp new file mode 100644 index 000000000..63dce67db --- /dev/null +++ b/docs/mkdocs/docs/examples/accept__wide_string.cpp @@ -0,0 +1,18 @@ +#include +#include +#include + +using json = nlohmann::json; + +int main() +{ + // a valid JSON text, given as UTF-16 + const std::u16string valid_text = u"{\"name\":\"\u00e9\"}"; + + // an invalid JSON text, given as UTF-16 + const std::u16string invalid_text = u"{\"name\":\"\u00e9\",}"; + + std::cout << std::boolalpha + << json::accept(valid_text) << ' ' + << json::accept(invalid_text) << '\n'; +} diff --git a/docs/mkdocs/docs/examples/accept__wide_string.output b/docs/mkdocs/docs/examples/accept__wide_string.output new file mode 100644 index 000000000..836a5934c --- /dev/null +++ b/docs/mkdocs/docs/examples/accept__wide_string.output @@ -0,0 +1 @@ +true false diff --git a/docs/mkdocs/docs/examples/parse__wide_string.cpp b/docs/mkdocs/docs/examples/parse__wide_string.cpp new file mode 100644 index 000000000..93a599947 --- /dev/null +++ b/docs/mkdocs/docs/examples/parse__wide_string.cpp @@ -0,0 +1,27 @@ +#include +#include +#include + +using json = nlohmann::json; + +int main() +{ + // the same JSON text, given in different encodings + const std::string utf8 = "{\"name\":\"\u00e9\"}"; + const std::u16string utf16 = u"{\"name\":\"\u00e9\"}"; + const std::u32string utf32 = U"{\"name\":\"\u00e9\"}"; + const std::wstring wide = L"{\"name\":\"\u00e9\"}"; + + // the encoding is deduced from the character type of the input, and all + // of them yield the same value + std::cout << json::parse(utf8) << '\n' + << json::parse(utf16) << '\n' + << json::parse(utf32) << '\n' + << json::parse(wide) << '\n'; + + // a wide string literal works as well: u"..." is UTF-16, U"..." is UTF-32 + std::cout << json::parse(u"[1, 2, 3]") << '\n'; + + // iterators over wide characters work as well + std::cout << json::parse(utf16.begin(), utf16.end()) << '\n'; +} diff --git a/docs/mkdocs/docs/examples/parse__wide_string.output b/docs/mkdocs/docs/examples/parse__wide_string.output new file mode 100644 index 000000000..cc75f0381 --- /dev/null +++ b/docs/mkdocs/docs/examples/parse__wide_string.output @@ -0,0 +1,6 @@ +{"name":"é"} +{"name":"é"} +{"name":"é"} +{"name":"é"} +[1,2,3] +{"name":"é"} diff --git a/docs/mkdocs/docs/features/conversions.md b/docs/mkdocs/docs/features/conversions.md index 5ba42b7cf..fda7eba77 100644 --- a/docs/mkdocs/docs/features/conversions.md +++ b/docs/mkdocs/docs/features/conversions.md @@ -38,6 +38,16 @@ which avoids repeating it: --8<-- "examples/get_to.output" ``` +!!! info "Getting values out of a document or a view" + + A [view](../api/basic_json_view/index.md) offers the same `get()` conversion, but reads it straight from the + document's index, without building a `basic_json` value first. Two conversions never copy at all: + [`get_string()`](../api/basic_json_view/get_string.md) returns a string as a view into the source text, and + [`number_token()`](../api/basic_json_view/number_token.md) returns a number exactly as it was written. Every other + type is obtained from a subtree that is built on demand with + [`materialize()`](../api/basic_json_view/materialize.md). See + [Zero-copy JSON views](json_view.md#getting-values-out-without-copying). + The library already knows how to convert to and from the scalar types and the STL containers (such as `#!cpp std::vector`, `#!cpp std::map`, `#!cpp std::array`, `#!cpp std::optional`, and many more). Converting a JSON object back to a `#!cpp std::map` or a JSON array back to a `#!cpp std::vector` therefore works without any extra code: @@ -234,3 +244,4 @@ The conversions above are built in for standard types. To make the same syntax w - [`get_ref`](../api/basic_json/get_ref.md) / [`get_ptr`](../api/basic_json/get_ptr.md) - access the stored value without copying - [Arbitrary types conversions](arbitrary_types.md) - support your own types - [`JSON_USE_IMPLICIT_CONVERSIONS`](../api/macros/json_use_implicit_conversions.md) - toggle implicit conversions +- [Zero-copy JSON views](json_view.md) - get values out of a document or a view diff --git a/docs/mkdocs/docs/features/creating_values.md b/docs/mkdocs/docs/features/creating_values.md index fc35b3401..42a927b78 100644 --- a/docs/mkdocs/docs/features/creating_values.md +++ b/docs/mkdocs/docs/features/creating_values.md @@ -1,7 +1,9 @@ # Creating JSON values There are several ways to create a JSON value in memory. This page gives an overview; to read a value from JSON text -instead, see [parsing](parsing/index.md). +instead, see [parsing](parsing/index.md). A JSON text can also be parsed into a +[document](../api/basic_json_document/index.md) and read through a [view](../api/basic_json_view/index.md) instead of +into a value; see [Zero-copy JSON views](json_view.md). ## From C++ values @@ -101,3 +103,4 @@ string `#!json "42"`. - [`operator""_json`](../api/operator_literal_json.md) - the `_json` literal - [Converting values](conversions.md) - which C++ types can be used - [Parsing](parsing/index.md) - create a value from JSON text +- [Zero-copy JSON views](json_view.md) - create a document and a view from JSON text instead of a value diff --git a/docs/mkdocs/docs/features/macros.md b/docs/mkdocs/docs/features/macros.md index 620cbe0da..773ea5ac3 100644 --- a/docs/mkdocs/docs/features/macros.md +++ b/docs/mkdocs/docs/features/macros.md @@ -287,6 +287,23 @@ external dependency and is not defined by default. See [full documentation of `JSON_USE_SIMDUTF`](../api/macros/json_use_simdutf.md). +## `JSON_VIEW_NO_SIMD` + +When defined, the parser of [`basic_json_document`](../api/basic_json_document/index.md) +(``) uses only portable C++ to scan strings, instead of vector instructions (SSE2 on x86-64, +NEON on AArch64). The same input is accepted or rejected either way; only the speed differs. It is not defined by +default. + +See [full documentation of `JSON_VIEW_NO_SIMD`](../api/macros/json_view_no_simd.md). + +## `JSON_VIEW_USE_SSSE3` + +When defined on x86-64, the parser of [`basic_json_document`](../api/basic_json_document/index.md) +(``) validates non-ASCII text in strings with SSSE3 without asking the CPU first. It is not +defined by default. + +See [full documentation of `JSON_VIEW_USE_SSSE3`](../api/macros/json_view_use_ssse3.md). + ## `NLOHMANN_DEFINE_TYPE_*(...)`, `NLOHMANN_DEFINE_DERIVED_TYPE_*(...)` The library defines 12 macros to simplify the serialization/deserialization of types. See the page on diff --git a/docs/mkdocs/docs/features/modifying_values.md b/docs/mkdocs/docs/features/modifying_values.md index 10814ac2e..7a55c6970 100644 --- a/docs/mkdocs/docs/features/modifying_values.md +++ b/docs/mkdocs/docs/features/modifying_values.md @@ -85,6 +85,16 @@ json a = {1, 2, 3, 4}; a.erase(1); // [1,3,4] (erase by index) ``` +## Editing a document + +The operations above work on a `basic_json` value. A JSON text that was parsed into a +[document](json_view.md#editing-a-document) can also be changed in place, without building a `basic_json` tree for the +parts that are not touched: [`set`](../api/basic_json_document/set.md) replaces a value or an object member, +[`push_back`](../api/basic_json_document/push_back.md) appends to an array, +[`insert`](../api/basic_json_document/insert.md) inserts into an array, and +[`erase`](../api/basic_json_document/erase.md) removes a member or an element. These functions are available on the +editable document types, such as [`json_editable_document`](../api/json_editable_document.md). + ## See also - [`push_back`](../api/basic_json/push_back.md) / [`emplace_back`](../api/basic_json/emplace_back.md) - append to an array @@ -93,3 +103,4 @@ a.erase(1); // [1,3,4] (erase by index) - [`merge_patch`](../api/basic_json/merge_patch.md) - apply an RFC 7396 merge patch - [`erase`](../api/basic_json/erase.md) / [`clear`](../api/basic_json/clear.md) - remove elements - [JSON Patch and Diff](json_patch.md) and [JSON Merge Patch](merge_patch.md) - structured modifications +- [Editing a document](json_view.md#editing-a-document) - change a parsed document in place diff --git a/docs/mkdocs/docs/features/parsing/index.md b/docs/mkdocs/docs/features/parsing/index.md index 07fb6500d..2944a71e2 100644 --- a/docs/mkdocs/docs/features/parsing/index.md +++ b/docs/mkdocs/docs/features/parsing/index.md @@ -18,6 +18,7 @@ flowchart LR The [`parse`](../../api/basic_json/parse.md) function reads a JSON value from an input. The input can be - a string (`#!cpp std::string`, C string, or string literal), +- a wide string (`#!cpp std::u16string`, `#!cpp std::u32string`, or `#!cpp std::wstring`, or a wide string literal), - a `#!cpp std::istream` (e.g., an `#!cpp std::ifstream` reading from a file), - a `#!cpp FILE*` pointer, - a pair of iterators over a contiguous range (e.g., a `#!cpp std::vector`), or @@ -32,12 +33,18 @@ std::ifstream f("example.json"); json data = json::parse(f); ``` -The input must be encoded in UTF-8; other encodings are not supported. A single input may contain only one JSON value. +The input must be encoded in UTF-8; other encodings are not supported. The exception is wide-character input, which +is read as UTF-16 or UTF-32 -- depending on the character type -- and converted to UTF-8; see the +[`parse`](../../api/basic_json/parse.md) page for an example. A single input may contain only one JSON value. Inputs consisting of multiple values separated by newlines are handled by the [JSON Lines](json_lines.md) format. By default, the library rejects comments and trailing commas. Both can be enabled with parameters of the `parse` function — see [comments](../comments.md) and [trailing commas](../trailing_commas.md). +[`basic_json_document::parse`](../../api/basic_json_document/parse.md) accepts the same inputs with the same options, +but keeps the result in a document rather than in a `basic_json` value; see +[Documents and views](#documents-and-views) below. + ## Strictness and trailing data [`parse`](../../api/basic_json/parse.md) reads a single JSON value and requires the whole input to be consumed: any @@ -98,6 +105,24 @@ The library offers two parsing models: You can influence a DOM parse without switching to the SAX interface by passing a [parser callback](parser_callbacks.md), which is called during parsing and can, for example, discard parts of the input. +## Documents and views + +A third model sits between DOM and SAX: [`basic_json_document::parse`](../../api/basic_json_document/parse.md) parses +the input into a document that keeps the text and a flat index of the values in it, and +[`basic_json_view`](../../api/basic_json_view/index.md) gives read-only access to the values through that index. +Strings and numbers stay in the source text and are only decoded when they are read, which is faster and uses less +memory than building a tree of `basic_json` values when only parts of the document are needed. A document can also be +modified in place, and it can be saved and loaded again without parsing it. + +```cpp +#include + +json_document doc = json_document::parse(R"({"happy": true, "pi": 3.141})"); +std::cout << doc.root()["happy"].get() << '\n'; +``` + +See [Zero-copy JSON views](../json_view.md) for the concept and the trade-offs. + ## Exceptions When the input is not valid JSON, the `parse` function throws an exception by default. If exceptions are undesired or @@ -108,6 +133,7 @@ options. ## See also - [`parse`](../../api/basic_json/parse.md) - deserialize from a compatible input +- [`basic_json_document::parse`](../../api/basic_json_document/parse.md) - deserialize into a document instead of a value - [`accept`](../../api/basic_json/accept.md) - check if the input is valid JSON - [`sax_parse`](../../api/basic_json/sax_parse.md) - generate SAX events - [JSON Lines](json_lines.md) - parse newline-delimited JSON @@ -115,3 +141,4 @@ options. - [SAX interface](sax_interface.md) - implement a custom SAX handler - [parsing and exceptions](parse_exceptions.md) - control error handling - [parsing untrusted input](untrusted_input.md) - what to consider when parsing input from untrusted sources +- [Zero-copy JSON views](../json_view.md) - parse into a document and read it through a view diff --git a/docs/mkdocs/docs/features/parsing/untrusted_input.md b/docs/mkdocs/docs/features/parsing/untrusted_input.md index 9c5fa4c9e..551f9770e 100644 --- a/docs/mkdocs/docs/features/parsing/untrusted_input.md +++ b/docs/mkdocs/docs/features/parsing/untrusted_input.md @@ -81,6 +81,9 @@ Invalid UTF-8 is rejected while parsing, not just while serializing: - In JSON text, an ill-formed UTF-8 byte in a string is a [`parse_error.101`](../../home/exceptions.md#jsonexceptionparse_error101) ("invalid string: ill-formed UTF-8 byte"). +- In UTF-16 or UTF-32 input, an ill-formed sequence is a + [`parse_error.101`](../../home/exceptions.md#jsonexceptionparse_error101) as well: a lone surrogate, a high surrogate + that is not followed by its low surrogate, and a code point above `U+10FFFF` are all rejected. - In a binary format, a string that is not valid UTF-8 is a [`parse_error.113`](../../home/exceptions.md#jsonexceptionparse_error113). diff --git a/docs/mkdocs/docs/home/debugging.md b/docs/mkdocs/docs/home/debugging.md index 36aa5f39f..362afaa6b 100644 --- a/docs/mkdocs/docs/home/debugging.md +++ b/docs/mkdocs/docs/home/debugging.md @@ -1,7 +1,7 @@ # Debugging -This page collects the library's built-in debugger integrations and other debugging-related features. They are -not linked from a single place elsewhere in the docs, so are collected here. +This page collects the library's built-in debugger integrations and other debugging-related features. They are not +covered by any single page elsewhere in the docs, so they are collected here. ## Visual Studio (natvis) diff --git a/docs/mkdocs/docs/home/exceptions.md b/docs/mkdocs/docs/home/exceptions.md index 7658fb3cb..00d4058dc 100644 --- a/docs/mkdocs/docs/home/exceptions.md +++ b/docs/mkdocs/docs/home/exceptions.md @@ -99,7 +99,8 @@ As this global context comes at the price of storing one additional pointer per Now the exception message contains a JSON Pointer `/address/housenumber` that indicates which value has the wrong type. -See [documentation of `JSON_DIAGNOSTICS`](../api/macros/json_diagnostics.md) for more information. +See [documentation of `JSON_DIAGNOSTICS`](../api/macros/json_diagnostics.md) for more information, and +[Debugging](debugging.md) for the debugger integrations of the library. ## Parse errors diff --git a/docs/mkdocs/docs/home/faq.md b/docs/mkdocs/docs/home/faq.md index 2381f4f7e..d6234c3bf 100644 --- a/docs/mkdocs/docs/home/faq.md +++ b/docs/mkdocs/docs/home/faq.md @@ -81,8 +81,9 @@ rejected by default, and the library never adds them when serializing. The library supports **Unicode input** as follows: -- Only **UTF-8** encoded input is supported, which is the default encoding for JSON, according to [RFC 8259](https://tools.ietf.org/html/rfc8259.html#section-8.1). -- `std::u16string` and `std::u32string` can be parsed, assuming UTF-16 and UTF-32 encoding, respectively. These encodings are not supported when reading from files or other input containers. +- **UTF-8** encoded input is supported, which is the default encoding for JSON, according to [RFC 8259](https://tools.ietf.org/html/rfc8259.html#section-8.1). +- `std::u16string` and `std::u32string` can be parsed, assuming UTF-16 and UTF-32 encoding, respectively; `std::wstring` works as well and is interpreted as UTF-16 or UTF-32, depending on the width of the platform's `wchar_t`. These encodings cannot be read from a `FILE*` or an `std::istream`, which carry bytes only. +- Ill-formed UTF-16 or UTF-32 input is rejected: a lone surrogate, a high surrogate that is not followed by its low surrogate, and a code point above `U+10FFFF` all yield a parse error. The same holds for iterators over 2- or 4-byte characters. - Other encodings such as Latin-1 or ISO 8859-1 are **not** supported and will yield parse or serialization errors. - The library will not replace [Unicode noncharacters](http://www.unicode.org/faq/private_use.html#nonchar1). - Invalid surrogates (e.g., incomplete pairs such as `\uDEAD`) will yield parse errors. @@ -146,7 +147,7 @@ json::parse(std::string("\"") + '\0' + "\""); // throws parse_error.101: control Why are wide strings (e.g., `std::wstring`) dumped as arrays of numbers? -As described [above](#parse-errors-reading-non-ascii-characters), the library assumes UTF-8 as encoding. To store a wide string, you need to change the encoding. +As described [above](#parse-errors-reading-non-ascii-characters), the library stores its strings as UTF-8. To store a wide string, you need to change the encoding. !!! example diff --git a/docs/mkdocs/docs/index.md b/docs/mkdocs/docs/index.md index 091eca5e8..cf9d05375 100644 --- a/docs/mkdocs/docs/index.md +++ b/docs/mkdocs/docs/index.md @@ -6,7 +6,9 @@ JSON for Modern C++ is a header-only C++11 library that turns JSON into a first- magic of modern C++ so that creating, reading, and modifying JSON values feels as natural as it does in languages like Python. The whole library is available as a single header, `json.hpp`, with no dependencies, no subproject, and no complex build system to set up; a companion header, `json_fwd.hpp`, provides forward declarations to keep compile times -down. See [header-only integration](integration/index.md) for details. It is heavily unit-tested with 100% code +down, `` holds the user-defined string literals on their own, and +`` adds zero-copy views and editable documents. See +[header-only integration](integration/index.md) for details. It is heavily unit-tested with 100% code coverage, checked with Valgrind and the Clang Sanitizers for memory leaks, and continuously fuzz-tested by Google OSS-Fuzz. @@ -58,7 +60,8 @@ Conda, and more), and pkg-config. --- - Creating, parsing, accessing, and serializing JSON values, JSON Pointer/Patch, binary formats, and more. + Creating, parsing, accessing, and serializing JSON values, JSON Pointer/Patch, binary formats, zero-copy views, + and more. [:octicons-arrow-right-24: Features](features/index.md) @@ -74,7 +77,8 @@ Conda, and more), and pkg-config. --- - The complete reference for `basic_json` and its member functions, types, and related classes. + The complete reference for `basic_json` and its member functions, types, and related classes, as well as for + `json_document` and `json_view`. [:octicons-arrow-right-24: API documentation](api/basic_json/index.md)