mirror of
https://github.com/nlohmann/json.git
synced 2026-10-26 15:58:55 +00:00
Docs: fill documentation gaps found in user questions
Add pages on class-hierarchy/smart-pointer/variant/chrono recipes, working without exceptions, and large, streamed and partial input. Extend the FAQ (23 entries, Known bugs first), parse error documentation (empty-input checklist, line/column, diagnostic positions), element access, conversions, creating values, ADL troubleshooting, memory, and build/distribution notes. Add 14 examples with generated output. Correct the statement that parse errors cannot be recovered from with -fno-exceptions. The topics come from recurring questions on Stack Overflow, Reddit, Hacker News, YouTube, and the project's GitHub discussions and issues. Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
45 files changed
+1666
-8
No files matched your search
@@ -409,6 +409,32 @@ namespace nlohmann {
|
||||
}
|
||||
```
|
||||
|
||||
## Why isn't my `to_json` or `from_json` found?
|
||||
|
||||
The library calls `to_json` and `from_json` without qualification, so the compiler finds them by
|
||||
[argument-dependent lookup](https://en.cppreference.com/w/cpp/language/adl) in the namespace of the type, or in the
|
||||
namespaces of its template arguments. The usual reasons for a function that is "not found" are:
|
||||
|
||||
- **The type is an alias of a standard type.** After `#!cpp using Hash = std::array<std::uint8_t, 6>;` the type
|
||||
*is* `std::array<std::uint8_t, 6>`, which lives in namespace `std`. A `to_json(json&, const Hash&)` in your namespace
|
||||
is never selected. Either give the type its own identity, for example `#!cpp struct Hash : std::array<std::uint8_t, 6> {};`
|
||||
(the function is then found in `Hash`'s namespace), or specialize
|
||||
[`adl_serializer<Hash>`](#how-do-i-convert-third-party-types).
|
||||
- **The function is in the wrong namespace.** It must be in the namespace of the type (a global type needs a global
|
||||
function), not in the namespace of the code that calls the conversion. Specializations of `adl_serializer` belong in
|
||||
namespace `nlohmann`.
|
||||
- **It is declared too late.** A conversion can only use what has been declared when it is instantiated. If
|
||||
`#!cpp json j = std::vector<third_party::string>{...};` appears before the `adl_serializer` specialization, the
|
||||
compiler reports an error such as "explicit specialization of `adl_serializer<...>` after instantiation". Put the
|
||||
specialization in a small header that every file includes **before** it uses the type with `json`.
|
||||
- **It is a template.** For `template <class T> struct box`, write `template <class T> void to_json(json&, const box<T>&)`,
|
||||
not an overload for one `T`.
|
||||
- **It worked by accident.** Some versions found `to_json` overloads in `nlohmann::detail` through argument-dependent
|
||||
lookup of the base class of `basic_json`. Do not rely on this; it is not part of the interface.
|
||||
|
||||
For smart pointers, class hierarchies, `std::variant`, and `std::chrono`, see
|
||||
[Recipes for common types](type_recipes.md).
|
||||
|
||||
## Why can't I convert to/from `std::any`?
|
||||
|
||||
`std::any` is intentionally excluded from `get<T>()`/generic conversion support, so `get<std::any>()` and
|
||||
@@ -505,6 +531,7 @@ struct bad_serializer
|
||||
|
||||
## See also
|
||||
|
||||
- [Recipes for common types](type_recipes.md) - smart pointers, class hierarchies, `std::variant`, and `std::chrono`
|
||||
- [Converting values](conversions.md) - the general overview of `get`/`get_to` and implicit conversions
|
||||
- [Specializing enum conversion](enum_conversion.md) - map enums to JSON strings instead of integers
|
||||
- [Supported macros](macros.md) - reference for `NLOHMANN_DEFINE_TYPE_*` and related macros
|
||||
|
||||
@@ -7,7 +7,9 @@ detect bugs in client code.
|
||||
## Switch off runtime assertions
|
||||
|
||||
Runtime assertions can be switched off by defining the preprocessor macro `NDEBUG` (see the
|
||||
[documentation of assert](https://en.cppreference.com/w/cpp/error/assert)) which is the default for release builds.
|
||||
[documentation of assert](https://en.cppreference.com/w/cpp/error/assert)) which is the default for release builds. Without `NDEBUG`, the assertions are compiled in and contain the name of the source
|
||||
file (`__FILE__`); see [notes for builds and distribution](../integration/index.md#notes-for-builds-and-distribution)
|
||||
if that path must not end up in your binaries.
|
||||
|
||||
## Change assertion behavior
|
||||
|
||||
|
||||
@@ -112,6 +112,29 @@ Implicit conversions are convenient but can be surprising (for example, in overl
|
||||
be disabled by defining [`JSON_USE_IMPLICIT_CONVERSIONS`](../api/macros/json_use_implicit_conversions.md) to `#!cpp 0`,
|
||||
which forces the explicit `get` form and can catch unintended conversions at compile time.
|
||||
|
||||
!!! warning "`auto` copies the JSON value; it does not convert it"
|
||||
|
||||
The type of `#!cpp j["name"]` is `json`, not the C++ type of the value inside. `#!cpp auto` therefore deduces
|
||||
`json`:
|
||||
|
||||
```cpp
|
||||
json j = {{"name", "Mary"}, {"age", 42}};
|
||||
|
||||
auto name = j["name"]; // json: a copy of the value; prints as "Mary" (with quotes)
|
||||
std::string s1 = j["name"]; // std::string: implicit conversion
|
||||
auto s2 = j["name"].get<std::string>(); // std::string: explicit conversion
|
||||
auto& age = j["age"]; // json&: a reference, no copy
|
||||
age = 43; // changes j
|
||||
```
|
||||
|
||||
This is also why `#!cpp std::cout << j["name"]` prints the quotes (see
|
||||
[getting a string without quotes](#getting-values-out)), and why a `json` value cannot be passed to `printf`: the
|
||||
argument is not an `int` or a `char*`, and a compiler either rejects passing it through `...` (Clang does) or
|
||||
compiles it and prints garbage. Convert first: `#!cpp std::printf("%d", j["age"].get<int>());`.
|
||||
|
||||
Prefer `get<T>()` over `auto` when you want the C++ value, and `auto&` or `const auto&` when you want to avoid a
|
||||
copy of a large value.
|
||||
|
||||
!!! warning "Conversions do not range-check numbers"
|
||||
|
||||
Just like C++ itself, the `get` family performs numeric conversions without range checks — retrieving a
|
||||
|
||||
@@ -59,6 +59,25 @@ with a string as the first element is treated as an object, everything else as a
|
||||
[FAQ](../home/faq.md#brace-initialization-yields-arrays) for details and the opt-in
|
||||
[`JSON_BRACE_INIT_COPY_SEMANTICS`](../api/macros/json_brace_init_copy_semantics.md) macro.
|
||||
|
||||
### What does this produce?
|
||||
|
||||
The following table shows how some common spellings are interpreted. When in doubt, use `#!cpp json::array()`,
|
||||
`#!cpp json::object()`, or the `#!cpp json(value)` form with parentheses.
|
||||
|
||||
| code | result | note |
|
||||
|-------------------------------------------------|------------------------------|----------------------------------------------|
|
||||
| `#!cpp json j = {};` / `#!cpp json j{};` | `#!json null` | use `json::object()` or `json::array()` |
|
||||
| `#!cpp json j = json::object();` | `#!json {}` | |
|
||||
| `#!cpp json j = json::array();` | `#!json []` | |
|
||||
| `#!cpp json j(true);` | `#!json true` | parentheses create the value itself |
|
||||
| `#!cpp json j = {true};` | `#!json [true]` | braces create a one-element array (`#!cpp json j{true};` depends on the compiler, see the [FAQ](../home/faq.md#brace-initialization-yields-arrays)) |
|
||||
| `#!cpp json j = {{"a", 1}};` | `#!json {"a":1}` | all elements are `[string, value]` pairs: object |
|
||||
| `#!cpp json j = {{"a", 1}, {"b"}};` | `#!json [["a",1],["b"]]` | one element is not a pair: array |
|
||||
| `#!cpp json j = {{"a", 1}, 2};` | `#!json [["a",1],2]` | a plain value among pairs: array |
|
||||
| `#!cpp json j = {{"a", 1, 2}};` | `#!json [["a",1,2]]` | a three-element list is not a pair: array |
|
||||
| `#!cpp json j = "{\"a\":1}";` | `#!json "{\"a\":1}"` | a string **value**; it is not parsed |
|
||||
| `#!cpp json j = json::parse("{\"a\":1}");` | `#!json {"a":1}` | parse text with `parse` or `_json` |
|
||||
|
||||
## Building incrementally
|
||||
|
||||
A value can also be built up piece by piece. Accessing a non-existing object key or array index with
|
||||
|
||||
@@ -21,3 +21,42 @@ flowchart TD
|
||||
A --> F{"just testing first?"}
|
||||
F -->|"yes"| G["contains() / find()"]
|
||||
```
|
||||
|
||||
## Does this key exist?
|
||||
|
||||
Use [`contains`](../../api/basic_json/contains.md) to test for a key; it does not modify the value and does not throw:
|
||||
|
||||
```cpp
|
||||
json j = {{"name", "Mary"}, {"address", {{"city", "Berlin"}}}};
|
||||
|
||||
j.contains("name"); // true
|
||||
j.contains("age"); // false
|
||||
j.contains("city"); // false: contains() only looks at the top level
|
||||
j["address"].contains("city"); // true
|
||||
j.contains(json::json_pointer("/address/city")); // true: a JSON pointer tests a nested path in one call
|
||||
```
|
||||
|
||||
If the value is needed afterwards, [`find`](../../api/basic_json/find.md) tests and accesses with a single lookup:
|
||||
|
||||
```cpp
|
||||
if (auto it = j.find("name"); it != j.end())
|
||||
{
|
||||
std::cout << *it << '\n';
|
||||
}
|
||||
```
|
||||
|
||||
`contains` also returns `true` for a key whose value is `null`; use `j.contains("k") && !j["k"].is_null()` if a
|
||||
`null` should count as missing. To read a value with a fallback instead, see [`value`](default_value.md).
|
||||
|
||||
!!! warning "`operator[]` inserts missing keys"
|
||||
|
||||
Like `#!cpp std::map::operator[]`, the non-const [`operator[]`](unchecked_access.md) adds a `null` value if the key
|
||||
is missing. Testing a key with it therefore *changes the object*:
|
||||
|
||||
```cpp
|
||||
json j = {{"foo", 1}};
|
||||
bool absent = j["bar"].is_null(); // true, but j is now {"bar": null, "foo": 1}
|
||||
```
|
||||
|
||||
Use `contains`/`find` to test, and [`at`](checked_access.md) to access a value that must exist (it throws an
|
||||
[`out_of_range`](../../home/exceptions.md#out-of-range) exception otherwise).
|
||||
@@ -0,0 +1,113 @@
|
||||
# Working Without Exceptions
|
||||
|
||||
By default, the library reports errors by throwing exceptions (see [Exceptions](../home/exceptions.md)). This page
|
||||
collects what to use when exceptions are unwanted or unavailable, for example in embedded systems, game engines, or
|
||||
code compiled with `-fno-exceptions`.
|
||||
|
||||
## Two different questions
|
||||
|
||||
1. **Can my code avoid *catching* exceptions?** Yes. For parsing and for accessing values there are functions that
|
||||
report failure through a return value. They work in a normal build and need no `#!cpp try`/`#!cpp catch`.
|
||||
2. **Can the library be *compiled* without exceptions?** Yes, with `-fno-exceptions` or by defining
|
||||
[`JSON_NOEXCEPTION`](../api/macros/json_noexception.md). Both have the same effect: every `#!cpp throw` in the
|
||||
library becomes a call to `std::abort()`. The functions listed below never reach a `#!cpp throw` for the failures
|
||||
they report, so they keep working in this configuration. Anything else that fails aborts the program.
|
||||
|
||||
## Parsing
|
||||
|
||||
| function | on invalid input |
|
||||
|---------------------------------------------------------------------------------------------|-----------------------------------------------------|
|
||||
| `#!cpp json::parse(input, nullptr, false)` | returns a *discarded* value; test with `is_discarded()` |
|
||||
| [`accept`](../api/basic_json/accept.md) | returns `#!cpp false` |
|
||||
| `#!cpp json::from_cbor(input, true, false)` (and the other `from_*` functions) | returns a *discarded* value |
|
||||
| [`sax_parse`](../api/basic_json/sax_parse.md) with your own handler | returns `#!cpp false` after calling `parse_error` |
|
||||
|
||||
??? example "Example: parse and validate without exceptions"
|
||||
|
||||
```cpp
|
||||
--8<-- "examples/no_exceptions_parse.cpp"
|
||||
```
|
||||
|
||||
Output:
|
||||
|
||||
```
|
||||
--8<-- "examples/no_exceptions_parse.output"
|
||||
```
|
||||
|
||||
None of the calls in this example throws, so it also compiles and runs with `-fno-exceptions`.
|
||||
|
||||
The discarded value carries no message. If you need the position or the reason, use the SAX interface; see
|
||||
[Parsing and exceptions](parsing/parse_exceptions.md).
|
||||
|
||||
## Accessing values
|
||||
|
||||
The access functions are not equally safe. Only some of them never throw:
|
||||
|
||||
| expression | if the key is missing or the type is wrong |
|
||||
|-------------------------------------|-------------------------------------------------------------------------------|
|
||||
| `#!cpp j.contains("k")` | returns `#!cpp false` for a missing key; never throws |
|
||||
| `#!cpp j.find("k")` | returns `end()` for a missing key; never throws |
|
||||
| `#!cpp j.is_string()`, ... | the `is_*` functions test the type; never throw |
|
||||
| `#!cpp j.get_ptr<const std::string*>()` | returns `#!cpp nullptr` for a different type; never throws |
|
||||
| `#!cpp j.at("k")` | throws `out_of_range.403` for a missing key |
|
||||
| `#!cpp j.get<int>()` | throws `type_error.302` for a value that is not a number |
|
||||
| `#!cpp j.value("k", 0)` | returns the default for a missing key, but **throws `type_error.302` if `k` exists with another type** |
|
||||
| `#!cpp j["k"]` (non-const) | inserts `null` for a missing key |
|
||||
| `#!cpp j["k"]` (const) | undefined behavior for a missing key |
|
||||
|
||||
So reading a value that may be absent or of the wrong type without exceptions takes two steps: look up, then test the
|
||||
type.
|
||||
|
||||
??? example "Example: look up a key and test its type"
|
||||
|
||||
```cpp
|
||||
--8<-- "examples/no_exceptions_access.cpp"
|
||||
```
|
||||
|
||||
Output:
|
||||
|
||||
```
|
||||
--8<-- "examples/no_exceptions_access.output"
|
||||
```
|
||||
|
||||
The last lines of the example show why `value` is not on the safe list: it returns the default only for a *missing*
|
||||
key.
|
||||
|
||||
For numbers, test with `is_number()` (or `is_number_integer()`, `is_number_unsigned()`, `is_number_float()`) before
|
||||
calling `get<int>()`, and see [number handling](types/number_handling.md) for range and precision. Whether a value
|
||||
fits into the target type is not checked by `get`.
|
||||
|
||||
## Converting to your own types
|
||||
|
||||
`from_json` functions (including the ones generated by the
|
||||
[`NLOHMANN_DEFINE_TYPE_*` macros](arbitrary_types.md#simplify-your-life-with-macros)) report problems by throwing.
|
||||
There is no variant that returns an error code. Without exceptions you have two options:
|
||||
|
||||
- Validate the JSON value yourself, as shown above, and fill your struct field by field.
|
||||
- Convert in a function that is compiled **with** exceptions enabled and catches them at the boundary, returning
|
||||
`std::optional` or an error code to the rest of your code:
|
||||
|
||||
??? example "Example: convert to a struct and report failure as std::optional"
|
||||
|
||||
```cpp
|
||||
--8<-- "examples/no_exceptions_convert.c++17.cpp"
|
||||
```
|
||||
|
||||
Output:
|
||||
|
||||
```
|
||||
--8<-- "examples/no_exceptions_convert.c++17.output"
|
||||
```
|
||||
|
||||
## Overriding what happens instead of an exception
|
||||
|
||||
With exceptions off, you can replace the `abort()` by your own code, for example to log the message first. See
|
||||
[Switch off exceptions](../home/exceptions.md#switch-off-exceptions) and
|
||||
[`JSON_THROW_USER`](../api/macros/json_throw_user.md). The handler must not return normally.
|
||||
|
||||
## See also
|
||||
|
||||
- [Exceptions](../home/exceptions.md) - all exceptions and their ids
|
||||
- [Parsing and exceptions](parsing/parse_exceptions.md) - the parsing alternatives in detail
|
||||
- [Parsing untrusted input](parsing/untrusted_input.md)
|
||||
- [`JSON_NOEXCEPTION`](../api/macros/json_noexception.md)
|
||||
@@ -33,7 +33,9 @@ json data = json::parse(f);
|
||||
```
|
||||
|
||||
The input must be encoded in UTF-8; other encodings are not supported. A single input may contain only one JSON value.
|
||||
Inputs consisting of multiple values separated by newlines are handled by the [JSON Lines](json_lines.md) format.
|
||||
Inputs consisting of multiple values separated by newlines are handled by the [JSON Lines](json_lines.md) format; input
|
||||
that arrives in chunks or contains several values is covered in
|
||||
[Large, streamed, and partial input](large_and_partial_input.md).
|
||||
|
||||
By default, the library rejects comments and trailing commas. Both can be enabled with parameters of the `parse`
|
||||
function — see [comments](../comments.md) and [trailing commas](../trailing_commas.md).
|
||||
@@ -85,7 +87,7 @@ again without changing any result.
|
||||
|
||||
## SAX vs. DOM parsing
|
||||
|
||||
The library offers two parsing models:
|
||||
The library offers two parsing models, and a third, read-only alternative:
|
||||
|
||||
- **DOM parsing** (the default): the complete input is read and stored as an in-memory `basic_json` value that can be
|
||||
traversed and modified freely. This is what [`parse`](../../api/basic_json/parse.md) does, and it is the right choice
|
||||
@@ -95,6 +97,13 @@ The library offers two parsing models:
|
||||
inputs or when you only need to extract parts of the input. See the [SAX interface](sax_interface.md) for details and
|
||||
[`sax_parse`](../../api/basic_json/sax_parse.md) for the API.
|
||||
|
||||
- **Document and views** (`#!cpp <nlohmann/json_view.hpp>`, added in version 3.13.0): [`json_document::parse`](../../api/basic_json_document/parse.md)
|
||||
reads the same JSON grammar but builds a flat, read-only index of the values instead of a `basic_json` tree. Strings and
|
||||
floating-point numbers are read from the text on demand, and only the parts you
|
||||
[`materialize()`](../../api/basic_json_view/materialize.md) become `basic_json` values. This avoids most of the work when you
|
||||
need only a few values from a large or frequently parsed text; see [Zero-copy JSON views](../json_view.md). It is read-only, so use
|
||||
`parse` when you need to modify the value.
|
||||
|
||||
You can influence a DOM parse without switching to the SAX interface by passing a
|
||||
[parser callback](parser_callbacks.md), which is called during parsing and can, for example, discard parts of the input.
|
||||
|
||||
@@ -111,7 +120,9 @@ options.
|
||||
- [`accept`](../../api/basic_json/accept.md) - check if the input is valid JSON
|
||||
- [`sax_parse`](../../api/basic_json/sax_parse.md) - generate SAX events
|
||||
- [JSON Lines](json_lines.md) - parse newline-delimited JSON
|
||||
- [Large, streamed, and partial input](large_and_partial_input.md) - chunked, concatenated, and very large input
|
||||
- [parser callbacks](parser_callbacks.md) - influence the parsing by a callback function
|
||||
- [SAX interface](sax_interface.md) - implement a custom SAX handler
|
||||
- [Zero-copy JSON views](../json_view.md) - parse into a read-only document and read it through views
|
||||
- [parsing and exceptions](parse_exceptions.md) - control error handling
|
||||
- [parsing untrusted input](untrusted_input.md) - what to consider when parsing input from untrusted sources
|
||||
@@ -0,0 +1,154 @@
|
||||
# Large, Streamed, and Partial Input
|
||||
|
||||
[`parse`](../../api/basic_json/parse.md) reads exactly one complete JSON value and builds it in memory. It cannot be
|
||||
paused and resumed, and it does not hand back a half-parsed value when the input ends early. This page shows what to do
|
||||
when the input does not fit that model: it arrives in pieces, it contains several values, or it is too large to hold
|
||||
as a `json` value.
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A["my input is..."] --> B{"several JSON values in one stream?"}
|
||||
B -->|"yes"| C["read them one at a time"]
|
||||
A --> D{"arriving in chunks (socket, pipe)?"}
|
||||
D -->|"yes"| E["frame the messages, or buffer until complete"]
|
||||
A --> F{"too large to keep in memory?"}
|
||||
F -->|"yes"| G["discard parts with a callback, or use SAX"]
|
||||
A --> H{"only a small part is needed?"}
|
||||
H -->|"yes"| I["stop early with SAX, or read through a view"]
|
||||
```
|
||||
|
||||
## Several values in one stream
|
||||
|
||||
For newline-delimited input, read line by line and parse each line, as described in [JSON Lines](json_lines.md). For
|
||||
values that are simply concatenated, [`operator>>`](../../api/operator_gtgt.md) reads one value and leaves the stream
|
||||
positioned after it. Check for the end of the stream before each read, because a read at the end throws:
|
||||
|
||||
??? example "Example: read concatenated values"
|
||||
|
||||
```cpp
|
||||
--8<-- "examples/partial_input_concatenated.cpp"
|
||||
```
|
||||
|
||||
Output:
|
||||
|
||||
```
|
||||
--8<-- "examples/partial_input_concatenated.output"
|
||||
```
|
||||
|
||||
The same works for binary formats. [`from_cbor`](../../api/basic_json/from_cbor.md) and its relatives accept
|
||||
`#!cpp strict = false` to stop after the first value and leave the rest of the stream untouched; `tellg()` then reports
|
||||
how many bytes the value took:
|
||||
|
||||
??? example "Example: read one CBOR value and find out how many bytes it used"
|
||||
|
||||
```cpp
|
||||
--8<-- "examples/partial_input_cbor.cpp"
|
||||
```
|
||||
|
||||
Output:
|
||||
|
||||
```
|
||||
--8<-- "examples/partial_input_cbor.output"
|
||||
```
|
||||
|
||||
## Input arriving in chunks
|
||||
|
||||
A socket, pipe, or HTTP body hands over data in pieces of arbitrary size, and a piece can end in the middle of a
|
||||
number, a string, or an object. The parser does not keep state between calls, so there are two ways to deal with this.
|
||||
|
||||
**Frame the messages.** The most robust approach is a protocol that says where a value ends: one value per line
|
||||
([JSON Lines](json_lines.md)), a length prefix, or a delimiter. Collect the bytes of one message and parse them when
|
||||
the frame is complete. Prefer this whenever you control the protocol.
|
||||
|
||||
**Buffer until the value is complete.** If the data has no framing, append every chunk to a buffer and try to parse
|
||||
it. A parse error at the very end of the buffer means that the input ended prematurely, while an error anywhere else is
|
||||
a real syntax error. The member `byte` of the [`parse_error`](../../home/exceptions.md#jsonexceptionparse_error101)
|
||||
exception is one past the buffer size in the first case:
|
||||
|
||||
??? example "Example: buffer chunks until the value is complete"
|
||||
|
||||
```cpp
|
||||
--8<-- "examples/partial_input_buffering.cpp"
|
||||
```
|
||||
|
||||
Output:
|
||||
|
||||
```
|
||||
--8<-- "examples/partial_input_buffering.output"
|
||||
```
|
||||
|
||||
The example feeds a document in five chunks. The first four are reported as incomplete, and a syntax error is reported as
|
||||
such, not as missing data.
|
||||
|
||||
!!! warning "Limits of buffering"
|
||||
|
||||
- The buffer must contain exactly one value. Data that belongs to the next value is a syntax error. Use framing if
|
||||
values follow each other without a separator.
|
||||
- A number at the end of the buffer is a complete value, even if more digits are still to come (`12` may be the
|
||||
start of `123`). Buffering cannot detect this; use framing for top-level numbers.
|
||||
- Every attempt parses the buffer from the beginning, so the total work grows quadratically with the size of the
|
||||
value. Limit the size of the buffer for input you do not trust (see [untrusted input](untrusted_input.md)).
|
||||
|
||||
## Input too large for memory
|
||||
|
||||
The input does not have to become one huge `json` value. The two tools below process it piece by piece.
|
||||
|
||||
### Handle and discard elements with a callback
|
||||
|
||||
A [parser callback](parser_callbacks.md) is called for every finished value. If it returns `#!cpp false`, the value is
|
||||
discarded and its memory is released at once. For a file that consists of one huge array (or one huge object) of
|
||||
records, handle each record at depth 1 and discard it:
|
||||
|
||||
??? example "Example: handle and discard each record of a large array"
|
||||
|
||||
```cpp
|
||||
--8<-- "examples/partial_input_discard_records.cpp"
|
||||
```
|
||||
|
||||
Output:
|
||||
|
||||
```
|
||||
--8<-- "examples/partial_input_discard_records.output"
|
||||
```
|
||||
|
||||
Only one record is in memory at any time. The same recipe, with more detail, is in the
|
||||
[parser callbacks](parser_callbacks.md#recipe-streaming-a-large-homogeneous-array) page. To skip entire members you do not
|
||||
need, discard them when their key is read; see the [example](parser_callbacks.md) for `parse_event_t::key`.
|
||||
|
||||
### Extract a few values and stop with SAX
|
||||
|
||||
With the [SAX interface](sax_interface.md), no value is built at all, and **returning `#!cpp false` from any handler
|
||||
function stops the parser**. [`sax_parse`](../../api/basic_json/sax_parse.md) then returns `#!cpp false`, and the rest
|
||||
of the input is not read. The example finds the first `id` in a document without parsing the remainder:
|
||||
|
||||
??? example "Example: stop parsing as soon as a value is found"
|
||||
|
||||
```cpp
|
||||
--8<-- "examples/partial_input_sax_stop.cpp"
|
||||
```
|
||||
|
||||
Output:
|
||||
|
||||
```
|
||||
--8<-- "examples/partial_input_sax_stop.output"
|
||||
```
|
||||
|
||||
Note that `sax_parse` returns `#!cpp false` both when a handler stopped the parse and when the input is invalid, so
|
||||
record the reason in the handler (here: `id`) if you need to tell the two apart.
|
||||
|
||||
### Read parts of a large document with a view
|
||||
|
||||
If the whole text is available and you only need to read some of it, a [`json_document`](../json_view.md) from
|
||||
`<nlohmann/json_view.hpp>` (added in version 3.13.0) avoids the `basic_json` tree altogether: it keeps the text, builds a flat index of the values,
|
||||
and gives read-only [views](../../api/basic_json_view/index.md) of the parts you look at. Only what you
|
||||
[`materialize()`](../../api/basic_json_view/materialize.md) becomes a `basic_json` value. See
|
||||
[Zero-copy JSON views](../json_view.md) for details and for the lifetime rules.
|
||||
|
||||
## See also
|
||||
|
||||
- [Parsing](index.md) - overview of the parsing functions
|
||||
- [JSON Lines](json_lines.md) - newline-delimited JSON
|
||||
- [Parser callbacks](parser_callbacks.md) - discard values while parsing
|
||||
- [SAX interface](sax_interface.md) - event-based parsing
|
||||
- [Zero-copy JSON views](../json_view.md) - read parts of a large document without building a tree
|
||||
- [Untrusted input](untrusted_input.md) - limits for input you do not control
|
||||
@@ -21,6 +21,10 @@ catch (json::parse_error& ex)
|
||||
In case exceptions are undesired or not supported by the environment, there are different ways to proceed:
|
||||
|
||||
|
||||
!!! tip
|
||||
|
||||
[Working without exceptions](../no_exceptions.md) also covers accessing values and converting to your own types.
|
||||
|
||||
## Switch off exceptions
|
||||
|
||||
The [`parse()`](../../api/basic_json/parse.md) function accepts a `#!cpp bool` parameter `allow_exceptions` which
|
||||
@@ -81,3 +85,61 @@ The return value indicates whether the parsing should continue, so the function
|
||||
```
|
||||
--8<-- "examples/sax_no_exception.output"
|
||||
```
|
||||
|
||||
## Where did the error occur?
|
||||
|
||||
The message of a [`parse_error`](../../home/exceptions.md#jsonexceptionparse_error101) for JSON text contains the line
|
||||
and column of the error:
|
||||
|
||||
```
|
||||
[json.exception.parse_error.101] parse error at line 3, column 8: syntax error while parsing value - unexpected ']'; expected '[', '{', or a literal
|
||||
```
|
||||
|
||||
The exception object itself only stores the **byte index** of the last read character in the member `byte` (the first
|
||||
byte of the input has index 1). The same index is passed to `parse_error` in the [SAX interface](sax_interface.md). If
|
||||
you need the line and column as numbers, for example to show them in an editor, compute them from the input:
|
||||
|
||||
??? example "Example: compute line and column of a parse error"
|
||||
|
||||
```cpp
|
||||
--8<-- "examples/parse_error_position.cpp"
|
||||
```
|
||||
|
||||
Output:
|
||||
|
||||
```
|
||||
--8<-- "examples/parse_error_position.output"
|
||||
```
|
||||
|
||||
Columns count bytes, so a line with multi-byte UTF-8 characters before the error may differ from the column your editor
|
||||
shows.
|
||||
|
||||
### Where is the value that caused a later error?
|
||||
|
||||
A `parse_error` is raised while the text is read. Many mistakes show up later: the text is valid JSON, but a value has
|
||||
the wrong type when you convert it (`type_error.302`), or a key is missing (`out_of_range.403`). The exception then says
|
||||
what is wrong, but not where in the input the value was. Define
|
||||
[`JSON_DIAGNOSTIC_POSITIONS`](../../api/macros/json_diagnostic_positions.md) before including the library to add the
|
||||
byte range of the offending value to such messages (since version 3.12.0), and to query the range of any value with
|
||||
[`start_pos()`](../../api/basic_json/start_pos.md) and [`end_pos()`](../../api/basic_json/end_pos.md):
|
||||
|
||||
??? example "Example: locate a value with diagnostic positions"
|
||||
|
||||
```cpp
|
||||
--8<-- "examples/parse_error_position_diagnostic.cpp"
|
||||
```
|
||||
|
||||
Output:
|
||||
|
||||
```
|
||||
--8<-- "examples/parse_error_position_diagnostic.output"
|
||||
```
|
||||
|
||||
Positions are only available for values created by [`parse`](../../api/basic_json/parse.md), they are not updated when a
|
||||
value is changed, and the macro makes every value 16 bytes larger on 64-bit platforms, so use it where the cost is
|
||||
acceptable, for example for configuration files. Note that `start_pos()` is a 0-based index into the input, whereas
|
||||
`byte` of a `parse_error` starts at 1.
|
||||
|
||||
For binary formats such as CBOR, there are no lines; `byte` is the position in the byte sequence. To find out how many
|
||||
bytes a successfully parsed value used, see
|
||||
[Several values in one stream](large_and_partial_input.md#several-values-in-one-stream).
|
||||
@@ -26,9 +26,11 @@ several alternatives; see [Parsing and exceptions](parse_exceptions.md) for the
|
||||
[example](parse_exceptions.md#user-defined-sax-interface) that overrides it to print instead of throw.
|
||||
|
||||
If exceptions are unavailable entirely (`-fno-exceptions`, or [`JSON_NOEXCEPTION`](../../api/macros/json_noexception.md)
|
||||
defined), every `#!cpp throw` in the library becomes a call to `std::abort()` -- there is no way to recover from a
|
||||
parse error of untrusted input in that configuration; see
|
||||
[Switch off exceptions](../../home/exceptions.md#switch-off-exceptions) for the details and for overriding this with
|
||||
defined), every `#!cpp throw` in the library becomes a call to `std::abort()`. The alternatives above do not throw, so
|
||||
`parse()` with `allow_exceptions` set to `#!cpp false` (and likewise `accept()` and the `from_*` functions with
|
||||
`allow_exceptions` set to `#!cpp false`) can still reject invalid input in that configuration; a plain `parse()` call
|
||||
aborts on invalid input. See [Working without exceptions](../no_exceptions.md) and
|
||||
[Switch off exceptions](../../home/exceptions.md#switch-off-exceptions) for the details and for overriding the abort with
|
||||
`JSON_THROW_USER`.
|
||||
|
||||
## Nesting depth
|
||||
|
||||
@@ -122,7 +122,9 @@ an adapter, since their template argument order does not match what `basic_json`
|
||||
[linear in its size](../api/basic_json/basic_json.md#complexity) for the copy constructor but
|
||||
[constant](../api/basic_json/basic_json.md#complexity) for the move constructor. The same applies to assigning a
|
||||
large `#!cpp std::string`, `#!cpp std::vector`, or other container into a value: pass it as `#!cpp std::move(x)`
|
||||
rather than `x` whenever `x` is no longer needed afterwards.
|
||||
rather than `x` whenever `x` is no longer needed afterwards. Since version 3.13.0, converting an rvalue container
|
||||
(`#!cpp json j = std::move(vec);`) moves the elements instead of copying them; the source is then left in a valid but
|
||||
unspecified state.
|
||||
- **Access without copying.** [`get<T>()`](../api/basic_json/get.md) returns a copy of the stored value converted to
|
||||
`T`. When a reference or pointer to the value already stored inside the `basic_json` is enough,
|
||||
[`get_ref()`](../api/basic_json/get_ref.md) and [`get_ptr()`](../api/basic_json/get_ptr.md) access it directly:
|
||||
@@ -151,6 +153,29 @@ an adapter, since their template argument order does not match what `basic_json`
|
||||
["reserving array capacity"](element_access/unchecked_access.md#performance-reserving-array-capacity) for the
|
||||
one-line recipe.
|
||||
|
||||
## Memory
|
||||
|
||||
- **A value is small, its content is on the heap.** A `basic_json` is 16 bytes on a 64-bit platform (a type tag and a
|
||||
union). Strings, arrays, objects, and binary values are allocated on the heap and owned by the value. Destroying the
|
||||
value frees everything it owns.
|
||||
- **Copies are deep.** After `#!cpp json b = a;`, changing `b` does not change `a`, and both own their memory. Copying a
|
||||
large value therefore doubles its memory; see [Avoiding copies](#avoiding-copies). `#!cpp auto x = j["key"];` is a
|
||||
copy, `#!cpp auto& x = j["key"];` is not.
|
||||
- **A moved-from value is `null`.** After `#!cpp json c = std::move(a);` the value `a` is `#!json null` and owns nothing.
|
||||
- **Reading does not require a tree.** If you only read the data, parse it into a [`json_document`](json_view.md)
|
||||
(`#!cpp <nlohmann/json_view.hpp>`, added in version 3.13.0) instead of a `basic_json`. It stores 16 bytes per value, does not copy floating-point
|
||||
numbers or strings without escape sequences out of the text, and creates no `basic_json` value except for the parts
|
||||
you [`materialize()`](../api/basic_json_view/materialize.md). A document that borrows the text needs the text to
|
||||
outlive it; see [Zero-copy JSON views](json_view.md).
|
||||
- **Containers keep their capacity.** `clear()` and `erase()` on an array remove the elements, but the underlying
|
||||
`#!cpp std::vector` keeps its allocated capacity, like `#!cpp std::vector::clear` does. To give the memory back,
|
||||
assign a new value (`#!cpp j = json::array();`) or destroy the value.
|
||||
- **Freed memory does not always go back to the operating system.** The library returns memory to the allocator
|
||||
(`#!cpp std::allocator` by default, or the [one you configured](types/template_parameters.md)). Whether the process
|
||||
size shrinks then depends on the allocator and the platform, as for any C++ program. A process that parses a large
|
||||
document and drops it is not leaking if its resident size stays high; run it under a tool such as
|
||||
[Valgrind](https://valgrind.org) or the sanitizers to tell a leak from retained memory.
|
||||
|
||||
## Serialization
|
||||
|
||||
[`dump()`](../api/basic_json/dump.md) with the default `#!cpp indent = -1` selects "the most compact representation"
|
||||
|
||||
@@ -0,0 +1,122 @@
|
||||
# Recipes for Common Types
|
||||
|
||||
The library converts the standard containers, strings, numbers, tuples, `std::optional`, and `std::filesystem::path`
|
||||
(the last two since version 3.12.0, C++17) out of the box, and your own types once you write conversion functions or use
|
||||
the macros (see [Arbitrary type conversions](arbitrary_types.md)). It deliberately does **not** convert smart pointers,
|
||||
`std::variant`, `std::chrono` types, or class hierarchies, because there is more than one reasonable JSON
|
||||
representation for each of them. This page shows a representation for each that you can copy and adapt.
|
||||
|
||||
All recipes use a specialization of [`adl_serializer`](../api/adl_serializer/index.md), which you may provide for types
|
||||
you do not own, or free `to_json`/`from_json` functions in the namespace of the type. Place them where they are
|
||||
visible before any code that converts the type; see
|
||||
[Why isn't my `to_json` found?](arbitrary_types.md#why-isnt-my-to_json-or-from_json-found).
|
||||
|
||||
## Smart pointers
|
||||
|
||||
A `std::shared_ptr<T>` or `std::unique_ptr<T>` is written as the value it points to, or as `#!json null` for an empty
|
||||
pointer, and read back by allocating a new `T`:
|
||||
|
||||
??? example "Example: serialize shared_ptr and unique_ptr"
|
||||
|
||||
```cpp
|
||||
--8<-- "examples/type_recipes_smart_pointers.c++14.cpp"
|
||||
```
|
||||
|
||||
Output:
|
||||
|
||||
```
|
||||
--8<-- "examples/type_recipes_smart_pointers.c++14.output"
|
||||
```
|
||||
|
||||
With this, containers of smart pointers, pointers to `const`, and members of these types in your own structs convert as
|
||||
usual. A pointer to a base class cannot be filled this way, because `T` would have
|
||||
to be created as a derived type; see the next section.
|
||||
|
||||
Reading creates a new object. Two `shared_ptr`s that pointed to the same object are written twice and read back as two
|
||||
separate objects.
|
||||
|
||||
## Class hierarchies
|
||||
|
||||
!!! tip "Derived classes whose type is known at compile time"
|
||||
|
||||
If you only need to serialize the data members of a derived class, and the code always knows the concrete type, you do
|
||||
not need a type tag. The [`NLOHMANN_DEFINE_DERIVED_TYPE_*` macros](arbitrary_types.md#simplify-your-life-with-macros)
|
||||
(since version 3.12.0) add the members of the derived class to those of its base class. This section is for the case
|
||||
where the concrete type is only known at run time, such as a `std::unique_ptr<shape>` that holds a circle or a
|
||||
rectangle.
|
||||
|
||||
JSON has no notion of types, so a value in a hierarchy needs a **type tag** that says which derived class to create.
|
||||
Write the tag in the object, and select the class from it when reading:
|
||||
|
||||
??? example "Example: serialize a class hierarchy with a type tag"
|
||||
|
||||
```cpp
|
||||
--8<-- "examples/type_recipes_class_hierarchy.c++14.cpp"
|
||||
```
|
||||
|
||||
Output:
|
||||
|
||||
```
|
||||
--8<-- "examples/type_recipes_class_hierarchy.c++14.output"
|
||||
```
|
||||
|
||||
A `#!cpp std::vector<std::unique_ptr<shape>>` converts to an array of tagged objects and back, creating a `circle`
|
||||
and a `rectangle`. The same works for `std::shared_ptr<shape>`. Remember the virtual destructor: without it, deleting a
|
||||
derived object through the base pointer is undefined behavior.
|
||||
|
||||
## `std::variant`
|
||||
|
||||
A variant has the same problem: the alternative must be recorded. If the alternatives are your own types, write a type
|
||||
tag next to their members, exactly as for a hierarchy:
|
||||
|
||||
??? example "Example: serialize a std::variant with a type tag"
|
||||
|
||||
```cpp
|
||||
--8<-- "examples/type_recipes_variant.c++17.cpp"
|
||||
```
|
||||
|
||||
Output:
|
||||
|
||||
```
|
||||
--8<-- "examples/type_recipes_variant.c++17.output"
|
||||
```
|
||||
|
||||
In the example, the functions are found
|
||||
because `geo` is the namespace of the template arguments of `#!cpp std::variant<circ, rect>`; for a variant of only
|
||||
standard types (for example `#!cpp std::variant<int, std::string>`) there is no such namespace, and you need an
|
||||
`adl_serializer` specialization or a wrapper type of your own.
|
||||
|
||||
If the alternatives are JSON-native (numbers, strings, ...), a variant such as `#!cpp std::variant<int, std::string>`
|
||||
can instead be written as the plain value, and read by testing the JSON type (`is_number_integer()`, `is_string()`)
|
||||
to choose the alternative.
|
||||
|
||||
## `std::chrono`
|
||||
|
||||
A duration is written as its count and read back with the same representation and period. The **unit is not part of
|
||||
the JSON value**, so document it, for example in the member name (`timeout_ms`):
|
||||
|
||||
??? example "Example: serialize durations and system_clock time points"
|
||||
|
||||
```cpp
|
||||
--8<-- "examples/type_recipes_chrono.cpp"
|
||||
```
|
||||
|
||||
Output:
|
||||
|
||||
```
|
||||
--8<-- "examples/type_recipes_chrono.output"
|
||||
```
|
||||
|
||||
A `system_clock::time_point` with millisecond precision is written as the number of milliseconds since the Unix epoch.
|
||||
The time point serializer is limited to `system_clock`,
|
||||
because the epoch of other clocks (for example `steady_clock`) has no meaning outside the running program.
|
||||
|
||||
If you prefer a text representation such as ISO 8601 (`2023-11-14T22:13:20Z`), write the string with
|
||||
`#!cpp std::format("{:%FT%TZ}", t)` and read it with `std::chrono::parse` (both C++20), or with a date library, in the
|
||||
same two functions.
|
||||
|
||||
## See also
|
||||
|
||||
- [Arbitrary type conversions](arbitrary_types.md) - the conversion mechanism and macros
|
||||
- [Why isn't my `to_json` found?](arbitrary_types.md#why-isnt-my-to_json-or-from_json-found)
|
||||
- [Converting values](conversions.md) - `get`, implicit conversions, and `std::optional`
|
||||
Reference in new issue
Block a user