From 40f3caad4df09c0d8f5c1c0c0696a64ad6beb65e Mon Sep 17 00:00:00 2001 From: Niels Lohmann Date: Fri, 10 Jul 2026 12:01:11 +0200 Subject: [PATCH] :memo: Fix documentation gaps found in a full GitHub Discussions review Reviewed all 1008 GitHub Discussions (2020-2026) for recurring questions that better or more visible documentation would have avoided. Adds/expands documentation for ~26 distinct gaps, including: - New "Debugging" page collecting natvis, GDB pretty printer, LLDB status, and JSON_DIAGNOSTICS pointers (previously scattered/undiscoverable) - Thread-safety and schema-validation FAQ entries - StringType's char-based requirement (no wstring/u16string/u32string) - Brace-initialization-yields-arrays warning directly on the constructor reference page (previously only in the FAQ, missed by users reading the constructor docs) - std::any exclusion from get(), with a manual-dispatch example - Non-string-keyed std::map serializing as an array of pairs - ordered_json compatibility with NLOHMANN_DEFINE_TYPE_* macros (already worked, was undocumented) - std::array truncation on size-mismatched conversion (no exception) - static_cast vs. get>() divergence - Recipe for omitting a std::optional field instead of emitting null - No built-in nesting-depth limit during parsing + a callback-based workaround recipe - Recipe for streaming a large homogeneous array via parser callbacks - operator>> stream-position semantics for concatenated JSON values - JSON Pointer array-vs-object creation rule for non-existing paths - CMake target name (nlohmann_json_modules) needed to link C++20 modules - ESP-IDF/PlatformIO: no official package, link to a community fork - get(key, default) as the Python dict.get() equivalent - reserve() recipe for pre-allocating array capacity - JSONC as an alias for the existing ignore_comments/ignore_trailing_commas combination (distinct from the unsupported JSON5) - items() dereferenced-element type: decltype() idiom + detail-namespace stability caveat - Various macro/type-conversion limitations (MSGPACK_DEFINE_ARRAY equivalent, char-array round-tripping, ADL serializer macro gap) Signed-off-by: Niels Lohmann --- docs/mkdocs/docs/api/basic_json/basic_json.md | 17 +++- docs/mkdocs/docs/api/basic_json/binary_t.md | 8 ++ docs/mkdocs/docs/api/basic_json/items.md | 11 +++ docs/mkdocs/docs/api/basic_json/operator[].md | 9 ++ docs/mkdocs/docs/api/basic_json/string_t.md | 5 ++ docs/mkdocs/docs/api/basic_json/value.md | 4 + .../macros/nlohmann_define_type_intrusive.md | 3 + .../nlohmann_define_type_non_intrusive.md | 3 + docs/mkdocs/docs/api/operator_gtgt.md | 12 +++ docs/mkdocs/docs/features/arbitrary_types.md | 86 +++++++++++++++++++ docs/mkdocs/docs/features/comments.md | 2 +- docs/mkdocs/docs/features/conversions.md | 43 ++++++++++ .../features/element_access/default_value.md | 3 +- .../element_access/unchecked_access.md | 14 +++ docs/mkdocs/docs/features/json_pointer.md | 5 ++ .../docs/features/parsing/json_lines.md | 3 + .../docs/features/parsing/parser_callbacks.md | 56 ++++++++++++ docs/mkdocs/docs/features/types/index.md | 12 +++ docs/mkdocs/docs/home/debugging.md | 31 +++++++ docs/mkdocs/docs/home/faq.md | 23 +++++ docs/mkdocs/docs/integration/cmake.md | 12 +++ .../docs/integration/package_managers.md | 9 ++ docs/mkdocs/mkdocs.yml | 1 + 23 files changed, 369 insertions(+), 3 deletions(-) create mode 100644 docs/mkdocs/docs/home/debugging.md diff --git a/docs/mkdocs/docs/api/basic_json/basic_json.md b/docs/mkdocs/docs/api/basic_json/basic_json.md index 07f3ae066..6aaab23c9 100644 --- a/docs/mkdocs/docs/api/basic_json/basic_json.md +++ b/docs/mkdocs/docs/api/basic_json/basic_json.md @@ -115,7 +115,22 @@ basic_json(basic_json&& other) noexcept; Function [`array()`](array.md) and [`object()`](object.md) force array and object creation from initializer lists, respectively. - + + !!! warning "Brace initialization yields arrays" + + Because this constructor takes an `initializer_list_t`, brace-initializing a `json`/`ordered_json` from + another `json` value wraps it in a single-element array rather than copying it: + + ```cpp + json j1 = "hello"; + json j2{j1}; // [!] j2 is ["hello"], NOT a copy of j1 + json j3(j1); // j3 is "hello" -- parentheses copy as expected + ``` + + See the FAQ entry on [brace initialization](../../home/faq.md#brace-initialization-yields-arrays) for the + full explanation, an opt-in macro to change this behavior, and how to explicitly create a single-element + array (`json::array({value})`) if that is what you want. + 6. Constructs a JSON array value by creating `cnt` copies of a passed value. In case `cnt` is `0`, an empty array is created. diff --git a/docs/mkdocs/docs/api/basic_json/binary_t.md b/docs/mkdocs/docs/api/basic_json/binary_t.md index 902ffe732..2c61ddb28 100644 --- a/docs/mkdocs/docs/api/basic_json/binary_t.md +++ b/docs/mkdocs/docs/api/basic_json/binary_t.md @@ -37,6 +37,14 @@ represent a byte array in modern C++. `BinaryType` : container type to store arrays + Although not formally expressed as a C++ concept, `BinaryType` must be default-constructible, + copy/move-constructible, and support `push_back()`, `.data()`, and `.size()`, because + [`byte_container_with_subtype`](../byte_container_with_subtype/index.md) derives directly from it. Its + `value_type` must additionally be exactly one byte wide (e.g., `std::uint8_t`/`char`/`std::byte`): the binary + serializers (CBOR, MessagePack, BSON, UBJSON) read and write the container's raw bytes via + `reinterpret_cast`, which is only correct for byte-sized elements -- a container like + `#!cpp std::vector` will not work as `BinaryType`. + ## Notes #### Default type diff --git a/docs/mkdocs/docs/api/basic_json/items.md b/docs/mkdocs/docs/api/basic_json/items.md index a45ebdf86..3f3d1f2ca 100644 --- a/docs/mkdocs/docs/api/basic_json/items.md +++ b/docs/mkdocs/docs/api/basic_json/items.md @@ -46,6 +46,17 @@ for (auto& [key, val] : j_object.items()) } ``` +If you need to name the type of the dereferenced element explicitly (e.g., to write a standalone function that +takes it as a parameter, or to use `items()` with `std::for_each`), use `decltype`: + +```cpp +using element_type = decltype(*j_object.items().begin()); +``` + +The per-element type (`iteration_proxy_value`) lives in the library's internal `detail` namespace and is +intentionally unspecified as a stable, named type -- `decltype` is the supported way to obtain it, but its exact +name/definition may change between versions. + ## Return value iteration proxy object wrapping the current value with an interface to use in range-based for loops diff --git a/docs/mkdocs/docs/api/basic_json/operator[].md b/docs/mkdocs/docs/api/basic_json/operator[].md index 4df77f14b..3195870e4 100644 --- a/docs/mkdocs/docs/api/basic_json/operator[].md +++ b/docs/mkdocs/docs/api/basic_json/operator[].md @@ -124,6 +124,15 @@ Strong exception safety: if an exception occurs, the original value stays intact filled with `#!json null`. - The special value `-` is treated as a synonym for the index past the end. + !!! note "Creating intermediate levels that don't exist yet" + + When the JSON pointer traverses intermediate levels that don't exist at all yet (not just a missing + leaf), each missing level is created as an array or an object depending on whether the corresponding + pointer token parses as a non-negative integer: a numeric token creates an array, a non-numeric token + creates an object. For example, on an initially `#!json null` value, `/foo/0/0/0` creates nested arrays, + while `/foo/one/one/one` creates nested objects. This is not specified by the JSON Pointer RFC; it is + this library's own, intentional disambiguation rule. See also [JSON Pointer](../../features/json_pointer.md). + ## Examples ??? example "Example: (1) access specified array element" diff --git a/docs/mkdocs/docs/api/basic_json/string_t.md b/docs/mkdocs/docs/api/basic_json/string_t.md index 78ef4ff45..97c586f28 100644 --- a/docs/mkdocs/docs/api/basic_json/string_t.md +++ b/docs/mkdocs/docs/api/basic_json/string_t.md @@ -18,6 +18,11 @@ JSON class into byte-sized characters during deserialization. : the container to store strings (e.g., `std::string`). Note this container is used for keys/names in objects, see [object_t](object_t.md). + `StringType` must have a `char`-compatible `value_type`: the library relies on UTF-8/`char`-based storage and + processing internally, so `std::wstring`, `std::u16string`, and `std::u32string` are **not** valid choices for + `StringType`. To work with wide-character data, convert it to/from UTF-8 at the boundary instead -- see the + FAQ's [wide string handling](../../home/faq.md#wide-string-handling) section for a conversion recipe. + ## Notes #### Default type diff --git a/docs/mkdocs/docs/api/basic_json/value.md b/docs/mkdocs/docs/api/basic_json/value.md index 4666f1b4a..e4f1bac5f 100644 --- a/docs/mkdocs/docs/api/basic_json/value.md +++ b/docs/mkdocs/docs/api/basic_json/value.md @@ -50,6 +50,10 @@ ValueType value(const json_pointer& ptr, - Unlike [`operator[]`](operator[].md), this function does not implicitly add an element to the position defined by `key`/`ptr` key. This function is furthermore also applicable to const objects. +!!! note + + This is equivalent to Python's `dict.get(key, default)`. + ## Template parameters `KeyType` diff --git a/docs/mkdocs/docs/api/macros/nlohmann_define_type_intrusive.md b/docs/mkdocs/docs/api/macros/nlohmann_define_type_intrusive.md index b1ccc3975..f0c32c557 100644 --- a/docs/mkdocs/docs/api/macros/nlohmann_define_type_intrusive.md +++ b/docs/mkdocs/docs/api/macros/nlohmann_define_type_intrusive.md @@ -62,6 +62,9 @@ See the examples below for the concrete generated code. - The current implementation is limited to at most 63 member variables. If you want to serialize/deserialize types with more than 63 member variables, you need to define the `to_json`/`from_json` functions manually. + - These macros always produce object-style (named-key) JSON, one key per member. There is no macro variant + that serializes a struct's members positionally into a JSON array; for that, write `to_json`/`from_json` by + hand, building/reading a `json::array()` of the members in order. ## Examples diff --git a/docs/mkdocs/docs/api/macros/nlohmann_define_type_non_intrusive.md b/docs/mkdocs/docs/api/macros/nlohmann_define_type_non_intrusive.md index 0b28189d1..cc8304750 100644 --- a/docs/mkdocs/docs/api/macros/nlohmann_define_type_non_intrusive.md +++ b/docs/mkdocs/docs/api/macros/nlohmann_define_type_non_intrusive.md @@ -63,6 +63,9 @@ See the examples below for the concrete generated code. - The current implementation is limited to at most 63 member variables. If you want to serialize/deserialize types with more than 63 member variables, you need to define the `to_json`/`from_json` functions manually. + - These macros always produce object-style (named-key) JSON, one key per member. There is no macro variant + that serializes a struct's members positionally into a JSON array; for that, write `to_json`/`from_json` by + hand, building/reading a `json::array()` of the members in order. ## Examples diff --git a/docs/mkdocs/docs/api/operator_gtgt.md b/docs/mkdocs/docs/api/operator_gtgt.md index 6306c87f5..e1d24d47f 100644 --- a/docs/mkdocs/docs/api/operator_gtgt.md +++ b/docs/mkdocs/docs/api/operator_gtgt.md @@ -33,6 +33,18 @@ A UTF-8 byte order mark is silently ignored. Invalid Unicode escapes and unpaired surrogates in the input are reported as [`parse_error.101`](../home/exceptions.md#jsonexceptionparse_error101) with a detailed message. +`operator>>` parses exactly one JSON value and leaves the stream positioned right after it, so it can be called +repeatedly to read a sequence of concatenated JSON values from the same stream: + +```cpp +json j1, j2; +input >> j1; // parses the first value, stream now positioned right after it +input >> j2; // parses the next value +``` + +Note this does **not** work for [JSON Lines](../features/parsing/json_lines.md) (newline-delimited JSON) input -- +see that page for why and for the recommended alternative. + !!! warning "Deprecation" This function replaces function `#!cpp std::istream& operator<<(basic_json& j, std::istream& i)` which has diff --git a/docs/mkdocs/docs/features/arbitrary_types.md b/docs/mkdocs/docs/features/arbitrary_types.md index 949037fd1..b472861b7 100644 --- a/docs/mkdocs/docs/features/arbitrary_types.md +++ b/docs/mkdocs/docs/features/arbitrary_types.md @@ -180,6 +180,49 @@ For _derived_ classes and structs, use the following macros } ``` +!!! warning "Overriding conversions for natively-supported types" + + The library already provides built-in `to_json`/`from_json` conversions for STL containers such as + `std::vector`, `std::array`, and `std::map`. Defining your own free-function `to_json`/`from_json` overload + for one of these container types directly (instead of for your own type) can conflict with the built-in + overload during overload resolution, producing compiler errors ("no matching overloaded function", + "call is ambiguous") that vary by compiler and library version. If you need different conversion behavior + for a container type the library already handles, wrap it in your own type (or use `adl_serializer` + specialization, as shown [above](#how-do-i-convert-third-party-types) for `boost::optional`) instead of + trying to re-specialize `to_json`/`from_json` for the container type itself. + +!!! warning "Raw C-style arrays" + + Members declared as raw C-style arrays (e.g., `char buf[1024]`) do not round-trip safely through + `NLOHMANN_DEFINE_TYPE_*` macros or the default (de)serializers: `to_json` serializes any `char` array as a + JSON *string* (matching the `std::string`-constructible overload), but the `from_json` overload for + fixed-size arrays expects a JSON *array* and iterates it element-wise, which fails with a `type_error` when + given a string. Use `std::string`, `std::array`, or a manually written `to_json`/`from_json` pair + for such members instead. + +!!! note "Macros and `nlohmann::ordered_json`" + + The `NLOHMANN_DEFINE_TYPE_*`/`NLOHMANN_DEFINE_DERIVED_TYPE_*` macros are generic over any `basic_json` + specialization, including `nlohmann::ordered_json`. Simply use `ordered_json` as the target type and members + are serialized in declaration order -- no separate macro or extra code is needed. + + ```cpp + namespace ns { + NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE(person, name, address, age) + } + + ns::person p{"Ned Flanders", "744 Evergreen Terrace", 60}; + nlohmann::ordered_json j = p; // keys appear in declaration order: name, address, age + ``` + +!!! note "No macro for non-default-constructible types" + + There is currently no `NLOHMANN_DEFINE_TYPE_*`-style macro for types that are not + [DefaultConstructible](https://en.cppreference.com/w/cpp/named_req/DefaultConstructible). This is not an + intentional omission of documentation -- no such macro exists yet; see + [How can I use `get()` for non-default constructible/non-copyable types?](#how-can-i-use-get-for-non-default-constructiblenon-copyable-types) + for the manual pattern to use instead. + ## How do I convert third-party types? This requires a bit more advanced technique. But first, let us see how this conversion mechanism works: @@ -270,6 +313,49 @@ namespace nlohmann { } ``` +## Why can't I convert to/from `std::any`? + +`std::any` is intentionally excluded from `get()`/generic conversion support, so `get()` and +containers like `std::map` fail to compile by design -- there is no way to know, from a +`json` value alone, which concrete type to store inside the `std::any`. To work with heterogeneous JSON values, +dispatch on the value's type manually and construct the `std::any` (or extract from it) yourself: + +```cpp +std::any value_to_any(const json& j) { + if (j.is_boolean()) { return j.get(); } + if (j.is_number_integer()) { return j.get(); } + if (j.is_number_float()) { return j.get(); } + if (j.is_string()) { return j.get(); } + // ... handle other types (arrays, objects) as needed for your use case + return {}; +} + +json any_to_json(const std::any& a) { + if (a.type() == typeid(bool)) { return std::any_cast(a); } + if (a.type() == typeid(int)) { return std::any_cast(a); } + if (a.type() == typeid(double)) { return std::any_cast(a); } + if (a.type() == typeid(std::string)) { return std::any_cast(a); } + return nullptr; +} +``` + +## Why does serializing a `std::map`/`std::unordered_map` with non-string keys produce an array? + +A `std::map`/`std::unordered_map` whose key type is not string-like (e.g., `std::map`) is +serialized as a JSON *array* of 2-element `[key, value]` arrays, not as a JSON object -- JSON object keys must be +strings, so the library cannot represent an integer-keyed map as an object. + +```cpp +std::map m{{1, "one"}, {2, "two"}}; +json j = m; +// j is [[1,"one"],[2,"two"]], not {"1":"one","2":"two"} +``` + +## Why does `std::wstring` convert or dump incorrectly? + +The library assumes UTF-8 encoding internally, so `std::wstring` is not supported out of the box -- see the FAQ +entry on [wide string handling](../home/faq.md#wide-string-handling) for why, and for a UTF-8 conversion recipe. + ## Can I write my own serializer? (Advanced use) Yes. You might want to take a look at [`unit-udt.cpp`](https://github.com/nlohmann/json/blob/develop/tests/src/unit-udt.cpp) in the test suite, to see a few examples. diff --git a/docs/mkdocs/docs/features/comments.md b/docs/mkdocs/docs/features/comments.md index dec04e38a..95ac72359 100644 --- a/docs/mkdocs/docs/features/comments.md +++ b/docs/mkdocs/docs/features/comments.md @@ -11,7 +11,7 @@ This library does not support comments *by default*. It does so for three reason 3. It is dangerous for interoperability if some libraries add comment support while others do not. Please check [The Harmful Consequences of the Robustness Principle](https://tools.ietf.org/html/draft-iab-protocol-maintenance-01) on this. -However, you can set parameter `ignore_comments` to `#!cpp true` in the [`parse`](../api/basic_json/parse.md) function to ignore `//` or `/* */` comments. Comments will then be treated as whitespace. +However, you can set parameter `ignore_comments` to `#!cpp true` in the [`parse`](../api/basic_json/parse.md) function to ignore `//` or `/* */` comments. Comments will then be treated as whitespace. Combined with `ignore_trailing_commas` (also a `parse` parameter), this covers what is commonly referred to as **JSONC** (JSON with Comments, as used e.g. by Visual Studio Code's `.jsonc` files) -- comments and trailing commas, nothing more. This is a different, smaller extension than [JSON5](https://json5.org), which additionally allows unquoted keys, single-quoted strings, and other syntax changes that this library does not support. For more information, see [JSON With Commas and Comments (JWCC)](https://nigeltao.github.io/blog/2021/json-with-commas-comments.html). diff --git a/docs/mkdocs/docs/features/conversions.md b/docs/mkdocs/docs/features/conversions.md index 7f63587fd..9681cc114 100644 --- a/docs/mkdocs/docs/features/conversions.md +++ b/docs/mkdocs/docs/features/conversions.md @@ -84,6 +84,49 @@ which forces the explicit `get` form and can catch unintended conversions at com j_null.get_to(opt); // ✅ std::nullopt ``` +!!! warning "`static_cast` and `get>()` are not guaranteed equivalent" + + `operator ValueType()` (used by `static_cast` and implicit conversions) intentionally excludes + `std::optional` from delegating to `get()`, to avoid a constructor ambiguity with + `std::optional`'s own converting constructor from `basic_json`. As a result, + `static_cast>(json_value)` goes through `std::optional`'s own converting + constructor rather than through `get>()`, which can behave differently -- for example, + with a custom `adl_serializer>` specialization. Prefer `get>()`/`get_to()` + over `static_cast` for optional types. + +!!! warning "Converting to a fixed-size `std::array` does not check length" + + Converting a JSON array to `#!cpp std::array` does not check that the JSON array's size matches `N`: + if the JSON array is longer, the extra elements are silently dropped; if it is shorter, the remaining + `std::array` elements are left default-constructed. No exception is thrown in either case. + + ```cpp + json j = {1, 2, 3, 4, 5}; + auto a = j.get>(); // {1, 2, 3} -- elements 4 and 5 silently dropped + ``` + +## Omitting a field when serializing `std::optional` + +By default, `to_json` for `std::optional` writes either the value or `#!json null` -- there is no built-in way +to make a field disappear from the serialized object entirely when the `std::optional` is `std::nullopt`. Because +a specialization of `adl_serializer>` only controls how the *value* is converted (it cannot +prevent the containing object's `to_json` from inserting the key in the first place), omission has to be +implemented in the *containing* type's `to_json`: + +```cpp +struct person { + std::string name; + std::optional age; +}; + +void to_json(json& j, const person& p) { + j = json{{"name", p.name}}; + if (p.age) { + j["age"] = *p.age; // key is only inserted when the optional has a value + } +} +``` + ## Putting values in The reverse direction works the same way: assigning or constructing a `json` from a C++ value converts it to JSON. diff --git a/docs/mkdocs/docs/features/element_access/default_value.md b/docs/mkdocs/docs/features/element_access/default_value.md index 4f714a287..7b613062b 100644 --- a/docs/mkdocs/docs/features/element_access/default_value.md +++ b/docs/mkdocs/docs/features/element_access/default_value.md @@ -4,7 +4,8 @@ In many situations, such as configuration files, missing values are not exceptional, but may be treated as if a default value was present. For this case, use [`value(key, default_value)`](../../api/basic_json/value.md) which takes the key -you want to access and a default value in case there is no value stored with that key. +you want to access and a default value in case there is no value stored with that key. This is equivalent to Python's +`dict.get(key, default)`. ## Example diff --git a/docs/mkdocs/docs/features/element_access/unchecked_access.md b/docs/mkdocs/docs/features/element_access/unchecked_access.md index f2de067d3..edaaa37a3 100644 --- a/docs/mkdocs/docs/features/element_access/unchecked_access.md +++ b/docs/mkdocs/docs/features/element_access/unchecked_access.md @@ -102,6 +102,20 @@ that the passed index is the new maximal index. Intermediate values are filled w `operator[]` can only be used with objects (with a string argument) or with arrays (with a numeric argument). For other types, a [`basic_json::type_error`](../../home/exceptions.md#jsonexceptiontype_error305) is thrown. +## Performance: reserving array capacity + +There is no public `reserve(count)` member on `basic_json` for pre-allocating array capacity. If you are building +a large array incrementally (e.g., via repeated `push_back()`) and know its final size ahead of time, you can +reserve capacity via `get_ref()` to access the underlying `array_t` directly: + +```cpp +json j = json::array(); +j.get_ref().reserve(1000); +for (int i = 0; i < 1000; ++i) { + j.push_back(i); +} +``` + ## Summary | scenario | non-const value | const value | diff --git a/docs/mkdocs/docs/features/json_pointer.md b/docs/mkdocs/docs/features/json_pointer.md index a16e8df24..c7237c266 100644 --- a/docs/mkdocs/docs/features/json_pointer.md +++ b/docs/mkdocs/docs/features/json_pointer.md @@ -77,6 +77,11 @@ auto val2 = j.at(json::json_pointer("/nested/three/1")); // false auto val3 = j.value(json::json_pointer("/nested/four"), 0); // 0 ``` +!!! note "Creating intermediate levels that don't exist" + + See the [`operator[]` notes](../api/basic_json/operator%5B%5D.md#return-value) for how array vs. object is + decided when a pointer creates intermediate levels that don't exist yet. + ## Flatten / unflatten The library implements a function [`flatten`](../api/basic_json/flatten.md) to convert any JSON document into a JSON diff --git a/docs/mkdocs/docs/features/parsing/json_lines.md b/docs/mkdocs/docs/features/parsing/json_lines.md index 659d31792..cda6f37c7 100644 --- a/docs/mkdocs/docs/features/parsing/json_lines.md +++ b/docs/mkdocs/docs/features/parsing/json_lines.md @@ -47,3 +47,6 @@ JSON Lines input with more than one value is treated as invalid JSON by the [`pa ``` with a JSON Lines input does not work, because the parser will try to parse one value after the last one. + + This is different from parsing a stream of *concatenated* (non-newline-delimited) JSON values, for which + `operator>>` does work -- see its [notes](../../api/operator_gtgt.md#notes) for details. diff --git a/docs/mkdocs/docs/features/parsing/parser_callbacks.md b/docs/mkdocs/docs/features/parsing/parser_callbacks.md index d398be325..e65ac0eb9 100644 --- a/docs/mkdocs/docs/features/parsing/parser_callbacks.md +++ b/docs/mkdocs/docs/features/parsing/parser_callbacks.md @@ -58,6 +58,14 @@ table describes the values of the parameters `depth`, `event`, and `parsed`. | `array_end` | 1 | `#!json [52.519444,13.406667]` | | `object_end` | 0 | `#!json {"location":[52.519444,13.406667],"name":"Berlin"}` | +!!! note "No built-in nesting depth limit" + + The library has no built-in limit on recursion/nesting depth while parsing. A parser callback can only + *discard* content it has already parsed (by returning `#!c false`); it cannot make parsing fail once a + nesting limit is exceeded partway through reading a deeply nested value. If you need to reject over-deep + untrusted input outright, track `depth` in a callback and `throw` from it once your limit is exceeded (a + thrown exception propagates out of `parse()` as usual). + ## Return value Discarding a value (i.e., returning `#!c false`) has different effects depending on the context in which the function @@ -112,3 +120,51 @@ This approach has two limitations: For strict validation with precise error positions, implementing a [SAX interface](sax_interface.md) instead gives access to the parser's position information directly. + +## Recipe: streaming a large homogeneous array + +A common use case is a huge top-level array of many similarly-shaped objects, too large to hold entirely in +memory as a `#!c json` value. A parser callback can hand off each completed element to a user function and then +discard it, so memory usage stays bounded by a single element (plus the not-yet-parsed tail of the input) rather +than the whole document. Since the top-level array's `array_start`/`array_end` are reported at `depth == 0` (its +parent is the document root), the object elements it contains are reported at `depth == 1`: + +??? example + + ```cpp + std::ifstream input("large_array.json"); + + auto callback = [](int depth, json::parse_event_t event, json& parsed) -> bool { + if (depth == 1 && event == json::parse_event_t::object_end) { + handle_element(parsed); // process the element, e.g. write it elsewhere + return false; // discard it -- frees its memory before the next one is parsed + } + return true; // keep everything else, including the (by then empty) top-level array + }; + + json::parse(input, callback); + ``` + +If the array's elements are scalars or nested arrays instead of objects, check for `parse_event_t::value` or +`parse_event_t::array_end` at `depth == 1` instead. The same approach works for a top-level *object* of many +homogeneous values by checking `object_end`/`value` events at `depth == 1` there too. + +## Recipe: max nesting depth via a callback + +Since there is no built-in nesting-depth limit (see the note above), a callback can enforce one manually by +tracking the maximum `depth` seen and throwing once it is exceeded: + +??? example + + ```cpp + constexpr int max_depth = 32; + + auto callback = [](int depth, json::parse_event_t /*event*/, json& /*parsed*/) -> bool { + if (depth > max_depth) { + throw std::runtime_error("maximum nesting depth exceeded"); + } + return true; + }; + + json::parse(input, callback); + ``` diff --git a/docs/mkdocs/docs/features/types/index.md b/docs/mkdocs/docs/features/types/index.md index f24da12a6..5990ac708 100644 --- a/docs/mkdocs/docs/features/types/index.md +++ b/docs/mkdocs/docs/features/types/index.md @@ -151,6 +151,18 @@ In this class, the object's limit of nesting is not explicitly constrained. Howe Objects are stored as pointers in a `basic_json` type. That is, for any access to object values, a pointer of type `object_t*` must be dereferenced. +### Converting maps with non-string keys + +A `std::map`/`std::unordered_map` whose key type is not string-like (e.g., `std::map`) is +converted to a JSON *array* of 2-element `[key, value]` arrays rather than a JSON object, because JSON object +keys must be strings: + +```cpp +std::map m{{1, "one"}, {2, "two"}}; +json j = m; +// j is [[1,"one"],[2,"two"]], not {"1":"one","2":"two"} +``` + ## Arrays diff --git a/docs/mkdocs/docs/home/debugging.md b/docs/mkdocs/docs/home/debugging.md new file mode 100644 index 000000000..803322150 --- /dev/null +++ b/docs/mkdocs/docs/home/debugging.md @@ -0,0 +1,31 @@ +# Debugging + +This page collects the library's built-in debugger integrations and other debugging-related features. They are +not linked from a single place elsewhere in the docs, so are collected here. + +## Visual Studio (natvis) + +The repository ships [`nlohmann_json.natvis`](https://github.com/nlohmann/json/blob/develop/nlohmann_json.natvis) +at its root, a [Natvis](https://learn.microsoft.com/en-us/visualstudio/debugger/create-custom-views-of-native-objects) +file that gives `json`/`ordered_json` values a friendly, key/value debugger view instead of showing raw internal +fields, when debugging with the MSVC debug engine (`cppvsdbg`) in Visual Studio or VS Code. + +Debug engines that wrap LLDB instead of the MSVC debug engine (for example, `codelldb` in VS Code) only have +partial/experimental Natvis support, and commonly fall back to showing raw internal fields even with the +`.natvis` file present. Switching to `cppvsdbg` where available, or checking your debug extension's own Natvis +support/version, are the next things to try if this happens. There is currently no bundled LLDB-native +pretty-printer script in this repository. + +## GDB + +The repository ships a [GDB Python pretty printer](https://github.com/nlohmann/json/tree/develop/tools/gdb_pretty_printer) +under `tools/gdb_pretty_printer`, with its own usage instructions in that directory's `README.md`. + +## Extended exception diagnostics + +Defining [`JSON_DIAGNOSTICS`](../api/macros/json_diagnostics.md) before including the library augments +`type_error`/`out_of_range`-style exceptions with a JSON Pointer to the offending value, which can help pinpoint +where in a large document a runtime error occurred. This only applies to exceptions thrown *after* a value +exists (e.g. during element access); parse errors, which happen before any value exists to point at, are not +covered by this mechanism -- see [Parsing and exceptions](../features/parsing/parse_exceptions.md) for how parse +errors report their own location instead. diff --git a/docs/mkdocs/docs/home/faq.md b/docs/mkdocs/docs/home/faq.md index c92889eca..a147a450c 100644 --- a/docs/mkdocs/docs/home/faq.md +++ b/docs/mkdocs/docs/home/faq.md @@ -129,6 +129,29 @@ As described [above](#parse-errors-reading-non-ascii-characters), the library as } ``` +## Usage + +### Thread safety + +!!! question + + Is `basic_json` thread-safe? + +No. `basic_json` provides no built-in synchronization, the same as `std::map` or `std::vector`. Concurrent reads of +the same value from multiple threads are safe, as are concurrent (non-overlapping) accesses to independent `json` +objects. However, any concurrent write to a `json` object -- or a concurrent read while another thread writes to the +same object -- is a data race and requires external synchronization (e.g., a `std::mutex`) by the caller. + +### Schema validation + +!!! question + + Does this library support JSON Schema validation? + +Not directly, but the companion project [json-schema-validator](https://github.com/pboettch/json-schema-validator) +builds JSON Schema (draft 4, 6, 7, and 2019-09) validation on top of this library and is a common recommendation +for this use case. + ## Exceptions ### Parsing without exceptions diff --git a/docs/mkdocs/docs/integration/cmake.md b/docs/mkdocs/docs/integration/cmake.md index 6bdaa4524..05d1d346f 100644 --- a/docs/mkdocs/docs/integration/cmake.md +++ b/docs/mkdocs/docs/integration/cmake.md @@ -181,3 +181,15 @@ Execute the test suite with [Valgrind](https://valgrind.org). This option is `OF Build the experimental [C++ module](../features/modules.md) `nlohmann.json` (requires CMake 3.28 or later and C++20). This option is `OFF` by default. + +A consuming project must link the dedicated `nlohmann_json_modules` CMake target (not just +`nlohmann_json::nlohmann_json`) for `import nlohmann.json;` to resolve: + +```cmake +set(NLOHMANN_JSON_BUILD_MODULES ON) +add_subdirectory(path/to/json) + +add_executable(myproject main.cpp) +target_link_libraries(myproject PRIVATE nlohmann_json_modules) +target_compile_definitions(myproject PRIVATE NLOHMANN_JSON_BUILD_MODULES) +``` diff --git a/docs/mkdocs/docs/integration/package_managers.md b/docs/mkdocs/docs/integration/package_managers.md index 79f872f85..909720454 100644 --- a/docs/mkdocs/docs/integration/package_managers.md +++ b/docs/mkdocs/docs/integration/package_managers.md @@ -930,6 +930,15 @@ If you are using [CocoaPods](https://cocoapods.org), you can use the library by to your podfile (see [an example](https://bitbucket.org/benman/nlohmann_json-cocoapod/src/master/)). Please file issues [here](https://bitbucket.org/benman/nlohmann_json-cocoapod/issues?status=new&status=open). +## ESP-IDF and PlatformIO + +There is no official package published to the [ESP-IDF Component Registry](https://components.espressif.com) or the +[PlatformIO Registry](https://registry.platformio.org). A community-maintained fork, +[Johboh/nlohmann-json](https://github.com/Johboh/nlohmann-json), publishes this library to both registries on each +new release and can be used as an unofficial component/package for ESP-IDF and PlatformIO projects. As the library +is header-only, it can otherwise be used directly by adding its `include/` directory to your component's/project's +include paths, like any other integration method described on this page. + ![](https://img.shields.io/cocoapods/v/nlohmann_json) !!! warning diff --git a/docs/mkdocs/mkdocs.yml b/docs/mkdocs/mkdocs.yml index 721b8e0b2..fe786f7d5 100644 --- a/docs/mkdocs/mkdocs.yml +++ b/docs/mkdocs/mkdocs.yml @@ -56,6 +56,7 @@ nav: - home/architecture.md - home/customers.md - home/sponsors.md + - home/debugging.md - Features: - features/index.md - features/arbitrary_types.md