Compare commits

..
Author SHA1 Message Date
Niels Lohmann 023ab67ecc Document options to reduce compile times
Add an integration page that collects the ways to reduce compile times
with measurements: json_fwd.hpp in headers, JSON_NO_AUTOMATIC_UDLS,
explicit instantiation with extern template, modules, and precompiled
headers, and notes that JSON_NO_IO and JSON_USE_GLOBAL_UDLS have no
measurable effect.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-30 20:08:22 +02:00
12 changed files with 526 additions and 1073 deletions
@@ -62,6 +62,7 @@ By default, `#!cpp JSON_NO_AUTOMATIC_UDLS` is not defined, and `<nlohmann/json.h
- [`operator""_json`](../operator_literal_json.md) - [`operator""_json`](../operator_literal_json.md)
- [`operator""_json_pointer`](../operator_literal_json_pointer.md) - [`operator""_json_pointer`](../operator_literal_json_pointer.md)
- [`JSON_USE_GLOBAL_UDLS`](json_use_global_udls.md) - place user-defined string literals (UDLs) into the global namespace - [`JSON_USE_GLOBAL_UDLS`](json_use_global_udls.md) - place user-defined string literals (UDLs) into the global namespace
- [Compile times](../../integration/compile_times.md) - options to reduce compile times
## Version history ## Version history
-6
View File
@@ -18,10 +18,6 @@ Deserializes an input stream to a JSON value.
the stream `i` the stream `i`
## Exception safety
Strong guarantee: if an exception is thrown, there are no changes in `j`.
## Exceptions ## Exceptions
- Throws [`parse_error.101`](../home/exceptions.md#jsonexceptionparse_error101) in case of an unexpected token, or if - Throws [`parse_error.101`](../home/exceptions.md#jsonexceptionparse_error101) in case of an unexpected token, or if
@@ -129,5 +125,3 @@ being read.
the stream; planned to become the default in version 4.0.0. the stream; planned to become the default in version 4.0.0.
- Fixed a null pointer dereference for an `std::istream` without a stream buffer (now throws `parse_error.101`), and a - Fixed a null pointer dereference for an `std::istream` without a stream buffer (now throws `parse_error.101`), and a
crash (`std::terminate`) when `i` has `eofbit` in its exception mask, in version 3.13.0. crash (`std::terminate`) when `i` has `eofbit` in its exception mask, in version 3.13.0.
- Changed to the strong exception safety guarantee in version 3.13.0: `j` is no longer left with a partially parsed
value if parsing throws.
@@ -389,7 +389,6 @@ using array_t = ArrayType<basic_json, AllocatorType<basic_json>>;
| Functionality | Additional requirement | | Functionality | Additional requirement |
|-----------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| |-----------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [`diff`](../../api/basic_json/diff.md), [`items`](../../api/basic_json/items.md), [`std::hash`](../../api/basic_json/std_hash.md) | conversion of a `#!cpp std::size_t` to `StringType`: either assignability from the result of `#!cpp std::to_string`, or an ADL overload `#!cpp void int_to_string(StringType&, std::size_t)` | | [`diff`](../../api/basic_json/diff.md), [`items`](../../api/basic_json/items.md), [`std::hash`](../../api/basic_json/std_hash.md) | conversion of a `#!cpp std::size_t` to `StringType`: either assignability from the result of `#!cpp std::to_string`, or an ADL overload `#!cpp void int_to_string(StringType&, std::size_t)` |
| [`operator/(std::size_t)`](../../api/json_pointer/operator_slash.md) | the same conversion of a `#!cpp std::size_t` to `StringType` as `diff`, `items`, and `std::hash` above |
| [`std::hash<basic_json>`](../../api/basic_json/std_hash.md) | additionally a specialization of `#!cpp std::hash<StringType>` | | [`std::hash<basic_json>`](../../api/basic_json/std_hash.md) | additionally a specialization of `#!cpp std::hash<StringType>` |
| [`to_bson`](../../api/basic_json/to_bson.md) | `find(value_type)` and `npos` | | [`to_bson`](../../api/basic_json/to_bson.md) | `find(value_type)` and `npos` |
| [`parse`](../../api/basic_json/parse.md) from a `string_t` | the input adapters must accept it; otherwise pass a character range | | [`parse`](../../api/basic_json/parse.md) from a `string_t` | the input adapters must accept it; otherwise pass a character range |
@@ -0,0 +1,149 @@
# Compile times
The library is header-only and makes heavy use of templates, so every translation unit that includes
`<nlohmann/json.hpp>` pays for parsing the header and instantiating what it uses. This page lists the options to reduce
that cost, ordered by how much they typically save.
!!! info "Measurements"
The numbers below are medians of nine runs compiling a single translation unit with `-std=c++17 -c` against the
single-header version, with Apple clang and GCC 16 on macOS (Apple silicon). They show the order of magnitude to
expect; measure your own code before and after a change.
## Include `json_fwd.hpp` in headers
Header files that only need to *name* the `json` type — for function declarations, members held by pointer or
reference, or friend declarations — can include `<nlohmann/json_fwd.hpp>` instead of `<nlohmann/json.hpp>`. It only
forward-declares `basic_json`, `json`, `ordered_json`, `json_pointer`, and `adl_serializer`. The translation units that
actually use the values then include `<nlohmann/json.hpp>`.
```cpp title="person.hpp"
#pragma once
#include <nlohmann/json_fwd.hpp>
struct person;
void to_json(nlohmann::json& j, const person& p);
void from_json(const nlohmann::json& j, person& p);
```
```cpp title="person.cpp"
#include "person.hpp"
#include <nlohmann/json.hpp>
void to_json(nlohmann::json& j, const person& p) { /* ... */ }
void from_json(const nlohmann::json& j, person& p) { /* ... */ }
```
| Compiler | `json.hpp` (`-O0`) | `json_fwd.hpp` (`-O0`) | Change |
|-------------|-------------------:|-----------------------:|-------:|
| Apple clang | 704 ms | 329 ms | −53% |
| GCC 16 | 779 ms | 242 ms | −69% |
This is the most effective option, because it avoids the full header in every translation unit that includes
*your* headers.
## Opt out of the automatic user-defined string literals
The user-defined string literals [`operator""_json`](../api/operator_literal_json.md) and
[`operator""_json_pointer`](../api/operator_literal_json_pointer.md) are ordinary inline functions whose bodies call the
parser. As `<nlohmann/json.hpp>` includes them by default, every translation unit instantiates the parser, even if it
never parses anything itself.
Define [`JSON_NO_AUTOMATIC_UDLS`](../api/macros/json_no_automatic_udls.md) for the whole project and include
`<nlohmann/json_literals.hpp>` only in the files that use the literals:
```cmake
target_compile_definitions(my_target PRIVATE JSON_NO_AUTOMATIC_UDLS)
```
```cpp
#include <nlohmann/json.hpp>
#include <nlohmann/json_literals.hpp> // only where "..."_json is used
```
The saving applies to translation units that do not parse JSON, for example ones that define types and their
conversions or only pass `json` values around:
| Compiler | Translation unit | Default (`-O0` / `-O2`) | `JSON_NO_AUTOMATIC_UDLS` (`-O0` / `-O2`) | Change |
|-------------|------------------|------------------------:|-----------------------------------------:|------------:|
| Apple clang | model | 776 ms / 846 ms | 629 ms / 692 ms | −19% / −18% |
| GCC 16 | model | 1022 ms / 1120 ms | 882 ms / 965 ms | −14% / −14% |
| Apple clang | parsing | 992 ms / 1815 ms | 1006 ms / 1823 ms | +1% / 0% |
| GCC 16 | parsing | 2018 ms / 3420 ms | 1990 ms / 3454 ms | −1% / +1% |
Translation units that include only the header save up to a third. Translation units that parse anyway instantiate
the parser regardless and see no difference.
## Instantiate `basic_json` once
Each translation unit instantiates the member functions of `nlohmann::json` it uses. An explicit instantiation
declaration tells the compiler that the non-template members are instantiated elsewhere, so it can skip them:
```cpp title="json_instance.hpp"
#pragma once
#include <nlohmann/json.hpp>
extern template class nlohmann::basic_json<>;
```
```cpp title="json_instance.cpp"
#include "json_instance.hpp"
template class nlohmann::basic_json<>;
```
Include `json_instance.hpp` instead of `<nlohmann/json.hpp>` and compile and link `json_instance.cpp` once.
| Compiler | Translation unit | Default (`-O0` / `-O2`) | `extern template` (`-O0` / `-O2`) | Change |
|-------------|---------------------|------------------------:|----------------------------------:|------------:|
| Apple clang | parsing | 992 ms / 1815 ms | 953 ms / 1625 ms | −4% / −10% |
| GCC 16 | parsing | 2018 ms / 3420 ms | 1522 ms / 2728 ms | −25% / −20% |
| Apple clang | `json_instance.cpp` | — | 2166 ms / 4660 ms | — |
| GCC 16 | `json_instance.cpp` | — | 5085 ms / 10616 ms | — |
Notes:
- The saving grows with the number of translation units that use `json`, while the instantiation translation unit is
compiled only once (and is rarely recompiled, as it does not depend on your code).
- Member function templates (such as `get<T>()`, `parse(InputType&&)`, or `value(key, default)`) are not covered by
the explicit instantiation and are still instantiated where they are used.
- The declaration covers exactly `nlohmann::json`. Add the same lines for `nlohmann::ordered_json`
(`nlohmann::basic_json<nlohmann::ordered_map>`) or your own `basic_json` specializations if you use them.
## Use C++20 modules
With a toolchain that supports named modules, `import nlohmann.json;` compiles the library once into a module and
avoids parsing the header in every translation unit. See [Modules](../features/modules.md) for requirements and known
issues. Module support is experimental and currently depends heavily on the compiler version.
## Use precompiled headers
Build systems can precompile `<nlohmann/json.hpp>` together with other stable headers, for example with CMake's
[`target_precompile_headers`](https://cmake.org/cmake/help/latest/command/target_precompile_headers.html):
```cmake
target_precompile_headers(my_target PRIVATE <nlohmann/json.hpp>)
```
This removes the cost of parsing the header, but not of instantiating templates in each translation unit, so it
combines well with the options above.
## Options without effect on compile times
Some configuration macros change what the library declares, but do not measurably change compile times:
| Macro | Apple clang, model (`-O0` / `-O2`) | GCC 16, model (`-O0` / `-O2`) |
|------------------------------------------------------------------------|-----------------------------------:|------------------------------:|
| default | 776 ms / 846 ms | 1022 ms / 1120 ms |
| [`JSON_NO_IO`](../api/macros/json_no_io.md) | 764 ms / 836 ms | 1022 ms / 1117 ms |
| [`JSON_USE_GLOBAL_UDLS`](../api/macros/json_use_global_udls.md)`=0` | 763 ms / 852 ms | 1019 ms / 1106 ms |
`JSON_USE_GLOBAL_UDLS` only controls *where* the literals are declared; to avoid their cost, use
`JSON_NO_AUTOMATIC_UDLS` instead.
## See also
- [`JSON_NO_AUTOMATIC_UDLS`](../api/macros/json_no_automatic_udls.md) - do not include the user-defined string
literals automatically
- [Modules](../features/modules.md) - C++20 module support
- [Header only](index.md) - including the library
+1 -1
View File
@@ -15,7 +15,7 @@ Clang).
You can further use file You can further use file
[`single_include/nlohmann/json_fwd.hpp`](https://github.com/nlohmann/json/blob/develop/single_include/nlohmann/json_fwd.hpp) [`single_include/nlohmann/json_fwd.hpp`](https://github.com/nlohmann/json/blob/develop/single_include/nlohmann/json_fwd.hpp)
for forward declarations, and file for forward declarations (see [Compile times](compile_times.md)), and file
[`single_include/nlohmann/json_literals.hpp`](https://github.com/nlohmann/json/blob/develop/single_include/nlohmann/json_literals.hpp) [`single_include/nlohmann/json_literals.hpp`](https://github.com/nlohmann/json/blob/develop/single_include/nlohmann/json_literals.hpp)
for the user-defined string literals if you define for the user-defined string literals if you define
[`JSON_NO_AUTOMATIC_UDLS`](../api/macros/json_no_automatic_udls.md). [`JSON_NO_AUTOMATIC_UDLS`](../api/macros/json_no_automatic_udls.md).
+1
View File
@@ -106,6 +106,7 @@ nav:
- integration/cmake.md - integration/cmake.md
- integration/package_managers.md - integration/package_managers.md
- integration/pkg-config.md - integration/pkg-config.md
- integration/compile_times.md
- API Documentation: - API Documentation:
- basic_json: - basic_json:
- 'Overview': api/basic_json/index.md - 'Overview': api/basic_json/index.md
+3 -4
View File
@@ -26,7 +26,6 @@
#include <nlohmann/detail/macro_scope.hpp> #include <nlohmann/detail/macro_scope.hpp>
#include <nlohmann/detail/string_concat.hpp> #include <nlohmann/detail/string_concat.hpp>
#include <nlohmann/detail/string_escape.hpp> #include <nlohmann/detail/string_escape.hpp>
#include <nlohmann/detail/string_utils.hpp>
#include <nlohmann/detail/value_t.hpp> #include <nlohmann/detail/value_t.hpp>
NLOHMANN_JSON_NAMESPACE_BEGIN NLOHMANN_JSON_NAMESPACE_BEGIN
@@ -117,7 +116,7 @@ class json_pointer
/// @sa https://json.nlohmann.me/api/json_pointer/operator_slasheq/ /// @sa https://json.nlohmann.me/api/json_pointer/operator_slasheq/
json_pointer& operator/=(std::size_t array_idx) json_pointer& operator/=(std::size_t array_idx)
{ {
return *this /= detail::to_string<string_t>(array_idx); return *this /= std::to_string(array_idx);
} }
/// @brief create a new JSON pointer by appending the right JSON pointer at the end of the left JSON pointer /// @brief create a new JSON pointer by appending the right JSON pointer at the end of the left JSON pointer
@@ -753,7 +752,7 @@ class json_pointer
// would throw out_of_range.404 -- contains() must not throw (see #5395) // would throw out_of_range.404 -- contains() must not throw (see #5395)
return false; return false;
} }
if (JSON_HEDLEY_UNLIKELY(reference_token.size() == 1 && !('0' <= reference_token[0] && reference_token[0] <= '9'))) if (JSON_HEDLEY_UNLIKELY(reference_token.size() == 1 && !("0" <= reference_token && reference_token <= "9")))
{ {
// invalid char // invalid char
return false; return false;
@@ -781,7 +780,7 @@ class json_pointer
// not throw (see #5395), so such a reference token is treated as "not found" // not throw (see #5395), so such a reference token is treated as "not found"
errno = 0; // strtoull() does not reset errno on success errno = 0; // strtoull() does not reset errno on success
char* p_end = nullptr; // NOLINT(misc-const-correctness) char* p_end = nullptr; // NOLINT(misc-const-correctness)
const unsigned long long magnitude = std::strtoull(reference_token.data(), &p_end, 10); // NOLINT(runtime/int) const unsigned long long magnitude = std::strtoull(reference_token.c_str(), &p_end, 10); // NOLINT(runtime/int)
if (JSON_HEDLEY_UNLIKELY(errno == ERANGE // the value exceeds ULLONG_MAX if (JSON_HEDLEY_UNLIKELY(errno == ERANGE // the value exceeds ULLONG_MAX
|| magnitude >= static_cast<unsigned long long>((std::numeric_limits<typename BasicJsonType::size_type>::max)()))) // NOLINT(runtime/int) || magnitude >= static_cast<unsigned long long>((std::numeric_limits<typename BasicJsonType::size_type>::max)()))) // NOLINT(runtime/int)
{ {
+104 -346
View File
@@ -5092,10 +5092,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
/// @sa https://json.nlohmann.me/api/basic_json/operator_gtgt/ /// @sa https://json.nlohmann.me/api/basic_json/operator_gtgt/
friend std::istream& operator>>(std::istream& i, basic_json& j) friend std::istream& operator>>(std::istream& i, basic_json& j)
{ {
// parse into a temporary so that j is left unchanged if parsing fails parser(detail::input_adapter(i)).parse(false, j);
basic_json result;
parser(detail::input_adapter(i)).parse(false, result);
j = std::move(result);
return i; return i;
} }
#endif // JSON_NO_IO #endif // JSON_NO_IO
@@ -6106,113 +6103,68 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
{ {
// the patch // the patch
basic_json result(value_t::array); basic_json result(value_t::array);
diff_recursively(result, source, target, path, 0);
// if the values are the same, return an empty patch
if (source == target)
{
return result; return result;
} }
private: if (source.type() != target.type())
/// @brief two arrays or two objects @ref diff_iteratively is diffing
struct diff_frame
{
diff_frame(const basic_json* source_, const basic_json* target_, const std::size_t path_length_) noexcept
: source(source_), target(target_), path_length(path_length_)
{}
// declared for GCC's -Weffc++, which asks for them in a class with
// pointer members and a non-trivial destructor; the exception
// specifications are left implicit, as GCC 4.8 rejects explicit ones
// that differ from them
diff_frame(const diff_frame&) = default;
diff_frame(diff_frame&&) = default;
diff_frame& operator=(const diff_frame&) = default;
diff_frame& operator=(diff_frame&&) = default;
~diff_frame() = default;
/// the values being diffed, both arrays or both objects
const basic_json* source;
const basic_json* target;
/// the length of their path in `current_path`
std::size_t path_length;
/// arrays: the next index to diff
std::size_t index = 0;
/// objects: the next member of source to look at
const_iterator member{}; // NOLINT(readability-redundant-member-init)
/// objects: the keys common to both, in source's order
std::vector<typename object_t::key_type> common_keys{}; // NOLINT(readability-redundant-member-init)
/// objects: the next entry of common_keys
std::size_t next_common = 0;
/// objects: the "add" operations for keys only target has
basic_json added_ops{}; // NOLINT(readability-redundant-member-init)
};
// The operations of a diff are built by the functions below rather than
// where they are needed: building one takes several temporaries, and
// unoptimized builds give each temporary a stack slot of its own in the
// function it appears in. In diff_recursively, which is on the call stack
// once per nesting level, that made every level cost kilobytes of stack.
/// @brief append a "replace" operation for @a path with @a value to @a result
static void diff_replace(basic_json& result, const string_t& path, const basic_json& value)
{ {
// different types: replace value
result.push_back( result.push_back(
{ {
{"op", "replace"}, {"path", path}, {"value", value} {"op", "replace"}, {"path", path}, {"value", target}
}); });
return result;
} }
/// @brief append a "remove" operation for @a path to @a result switch (source.type())
static void diff_remove(basic_json& result, const string_t& path) {
case value_t::array:
{
// first pass: traverse common elements
std::size_t i = 0;
while (i < source.size() && i < target.size())
{
// recursive call to compare array values at index i
auto temp_diff = diff(source[i], target[i], detail::concat<string_t>(path, '/', detail::to_string<string_t>(i)));
result.insert(result.end(), temp_diff.begin(), temp_diff.end());
++i;
}
// We now reached the end of at least one array
// in a second pass, traverse the remaining elements
// remove my remaining elements, highest index first; appending
// in that order avoids the quadratic reinsertion done before
for (std::size_t j = source.size(); j > i; --j)
{ {
result.push_back(object( result.push_back(object(
{ {
{"op", "remove"}, {"path", path} {"op", "remove"},
{"path", detail::concat<string_t>(path, '/', detail::to_string<string_t>(j - 1))}
})); }));
} }
i = source.size();
/// @brief append an "add" operation for @a path with @a value to @a result // add other remaining elements
static void diff_add(basic_json& result, const string_t& path, const basic_json& value) while (i < target.size())
{ {
result.push_back( result.push_back(
{ {
{"op", "add"}, {"path", path}, {"value", value} {"op", "add"},
{"path", detail::concat<string_t>(path, "/-")},
{"value", target[i]}
}); });
++i;
} }
/// @brief append the "remove" operations for the elements of array break;
/// @a source from @a index on, and the "add" operations for the
/// elements of array @a target from source's size on, to @a result
static void diff_array_tails(basic_json& result, const basic_json& source, const basic_json& target,
const string_t& path, const std::size_t index)
{
// remove my remaining elements, highest index first; appending
// in that order avoids the quadratic reinsertion done before
for (std::size_t j = source.size(); j > index; --j)
{
diff_remove(result, detail::concat<string_t>(path, '/', detail::to_string<string_t>(j - 1)));
} }
// add other remaining elements case value_t::object:
for (std::size_t i = source.size(); i < target.size(); ++i)
{
diff_add(result, detail::concat<string_t>(path, "/-"), target[i]);
}
}
/*!
@brief compare the keys of objects @a source and @a target
If object_t does not keep its members in insertion order, or if the keys
both objects have are in the same order in both, and the keys only
@a target has come after them, stores the keys common to both in
source's order in @a common_keys, stores the "add" operations for the keys
only @a target has in @a added_ops, and returns true: the caller then diffs
the objects member by member. Otherwise, appends operations that remove
every member of @a source and add every member of @a target to @a result,
and returns false.
*/
static bool diff_object_keys(basic_json& result, const basic_json& source, const basic_json& target,
const string_t& path, std::vector<typename object_t::key_type>& common_keys,
basic_json& added_ops)
{ {
// first pass: record, for every source key, whether it is // first pass: record, for every source key, whether it is
// common to both objects (in source's iteration order) or // common to both objects (in source's iteration order) or
@@ -6220,10 +6172,10 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
// a by-product of the target.find() call already needed to // a by-product of the target.find() call already needed to
// tell the two cases apart, so it adds no extra lookups. The // tell the two cases apart, so it adds no extra lookups. The
// "remove" ops themselves are emitted later, interleaved // "remove" ops themselves are emitted later, interleaved
// with the per-key diffs in the caller's fast path, to match // with the recursive per-key diffs in the fast path below,
// source's original iteration order (as the original, // to match source's original iteration order (as the
// pre-reordering-aware implementation did) instead of // original, pre-reordering-aware implementation did) instead
// grouping all removes before all per-key diffs. // of grouping all removes before all recursive diffs.
std::vector<typename object_t::key_type> common_keys_source_order; std::vector<typename object_t::key_type> common_keys_source_order;
for (auto it = source.cbegin(); it != source.cend(); ++it) for (auto it = source.cbegin(); it != source.cend(); ++it)
{ {
@@ -6239,17 +6191,18 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
// source.find() call already needed to detect added keys. At // source.find() call already needed to detect added keys. At
// the same time, determine whether every added key comes // the same time, determine whether every added key comes
// after every common key in target's order (a precondition // after every common key in target's order (a precondition
// for the fast path, which only ever appends new keys // for the fast path below, which only ever appends new keys
// at the very end). Both are only needed for an object_t that // at the very end). Both are only needed for an object_t that
// keeps its members in insertion order, such as the one // keeps its members in insertion order, such as the one
// backing `ordered_json`; for any other object_t, the fast // backing `ordered_json`; for any other object_t, the fast
// path is always taken and they are not computed. // path is always taken and they are not computed.
// The patch ops for keys that were added (i.e., in target but not // patch ops for keys that were added (i.e., in target but not
// in source) are built here so the fast path can reuse // in source); built here so the fast path below can reuse
// them without a second source.find() per target key. Only // them without a second source.find() per target key. Only
// used by the fast path -- the slow (reordering) path // used by the fast path -- the slow (reordering) path
// rebuilds "add" ops for every key itself. // rebuilds "add" ops for every key itself.
std::vector<typename object_t::key_type> common_keys_target_order; std::vector<typename object_t::key_type> common_keys_target_order;
basic_json added_ops(value_t::array);
bool new_keys_form_suffix = true; bool new_keys_form_suffix = true;
bool seen_new_key = false; bool seen_new_key = false;
for (auto it = target.cbegin(); it != target.cend(); ++it) for (auto it = target.cbegin(); it != target.cend(); ++it)
@@ -6257,7 +6210,12 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
if (source.find(it.key()) == source.end()) if (source.find(it.key()) == source.end())
{ {
seen_new_key = true; seen_new_key = true;
diff_add(added_ops, detail::concat<string_t>(path, '/', detail::escape(it.key())), it.value()); const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
added_ops.push_back(
{
{"op", "add"}, {"path", path_key},
{"value", it.value()}
});
} }
else else
{ {
@@ -6290,12 +6248,43 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
{ {
// fast path: order of common keys already matches (or the // fast path: order of common keys already matches (or the
// object_t's iteration order does not depend on // object_t's iteration order does not depend on
// insertion history), so a plain per-key diff is correct // insertion history), so a plain per-key recursive diff
// and minimal, as before // is correct and minimal, as before. common_keys_source_order
common_keys = std::move(common_keys_source_order); // is, by construction, the subsequence of source's keys
return true; // that are common to both objects, in source's iteration
// order -- so it can be walked in lockstep with `source`
// using a cheap key comparison instead of another lookup.
// Deleted keys (those source keys not in common_keys_source_order)
// are interleaved here too, in source's original order, to
// match the historical (pre-reordering-aware) output order.
auto common_it = common_keys_source_order.cbegin();
for (auto it = source.cbegin(); it != source.cend(); ++it)
{
if (common_it != common_keys_source_order.cend() && it.key() == *common_it)
{
const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
auto temp_diff = diff(it.value(), target[it.key()], path_key);
result.insert(result.end(), temp_diff.begin(), temp_diff.end());
++common_it;
}
else
{
// found a key that is not in target -> remove it
const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
result.push_back(object(
{
{"op", "remove"}, {"path", path_key}
}));
}
} }
// append the "add" ops for brand-new keys collected above
// during the pass over target -- no second source.find()
// per target key needed
result.insert(result.end(), added_ops.begin(), added_ops.end());
}
else
{
// slow path: the common keys are in a different relative // slow path: the common keys are in a different relative
// order in source and target (only possible for a // order in source and target (only possible for a
// reorderable object_t like ordered_map). Building a // reorderable object_t like ordered_map). Building a
@@ -6312,7 +6301,11 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
// it moves it to the end, fixing its position. // it moves it to the end, fixing its position.
for (auto it = source.cbegin(); it != source.cend(); ++it) for (auto it = source.cbegin(); it != source.cend(); ++it)
{ {
diff_remove(result, detail::concat<string_t>(path, '/', detail::escape(it.key()))); const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
result.push_back(object(
{
{"op", "remove"}, {"path", path_key}
}));
} }
// add every key that is either common (just removed // add every key that is either common (just removed
@@ -6321,96 +6314,15 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
// target exactly // target exactly
for (auto it = target.cbegin(); it != target.cend(); ++it) for (auto it = target.cbegin(); it != target.cend(); ++it)
{ {
diff_add(result, detail::concat<string_t>(path, '/', detail::escape(it.key())), it.value()); const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
} result.push_back(
return false;
}
/*!
@brief @ref diff, for values at nesting level @a depth, appending the
operations to @a result
Diffing two arrays or objects calls this function again, once per nesting
level, so values nested deeply enough used to exhaust the call stack and
terminate the process. The descent is bounded here: once @ref
detail::recursion_depth_limit levels have been entered, @ref
diff_iteratively diffs what is left without the call stack.
*/
static void diff_recursively(basic_json& result, const basic_json& source, const basic_json& target,
const string_t& path, const std::size_t depth)
{ {
// if the values are the same, there is nothing to do {"op", "add"}, {"path", path_key},
if (source == target) {"value", it.value()}
{ });
return;
}
if (JSON_HEDLEY_UNLIKELY(depth >= detail::recursion_depth_limit()))
{
diff_iteratively(result, source, target, path);
return;
}
if (source.type() != target.type())
{
// different types: replace value
diff_replace(result, path, target);
return;
}
switch (source.type())
{
case value_t::array:
{
// first pass: traverse common elements
std::size_t i = 0;
while (i < source.size() && i < target.size())
{
// recursive call to compare array values at index i
diff_recursively(result, source[i], target[i], detail::concat<string_t>(path, '/', detail::to_string<string_t>(i)), depth + 1);
++i;
}
// We now reached the end of at least one array
// in a second pass, traverse the remaining elements
diff_array_tails(result, source, target, path, i);
break;
}
case value_t::object:
{
std::vector<typename object_t::key_type> common_keys;
basic_json added_ops(value_t::array);
if (diff_object_keys(result, source, target, path, common_keys, added_ops))
{
// fast path: common_keys is, by construction, the
// subsequence of source's keys that are common to both
// objects, in source's iteration order -- so it can be
// walked in lockstep with `source` using a cheap key
// comparison instead of another lookup. Deleted keys
// (those source keys not in common_keys) are interleaved
// here too, in source's original order, to match the
// historical (pre-reordering-aware) output order.
auto common_it = common_keys.cbegin();
for (auto it = source.cbegin(); it != source.cend(); ++it)
{
if (common_it != common_keys.cend() && it.key() == *common_it)
{
diff_recursively(result, it.value(), target[it.key()], detail::concat<string_t>(path, '/', detail::escape(it.key())), depth + 1);
++common_it;
}
else
{
// found a key that is not in target -> remove it
diff_remove(result, detail::concat<string_t>(path, '/', detail::escape(it.key())));
} }
} }
// append the "add" ops for brand-new keys collected by
// diff_object_keys -- no second source.find() per target
// key needed
result.insert(result.end(), added_ops.begin(), added_ops.end());
}
break; break;
} }
@@ -6425,170 +6337,16 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
default: default:
{ {
// both primitive types: replace value // both primitive types: replace value
diff_replace(result, path, target); result.push_back(
{
{"op", "replace"}, {"path", path}, {"value", target}
});
break; break;
} }
} }
}
/*! return result;
@brief @ref diff without the call stack, appending the operations to
@a result
Produces the same operations as @ref diff_recursively. Only reached for
values nested more deeply than @ref detail::recursion_depth_limit.
*/
static void diff_iteratively(basic_json& result, const basic_json& source, const basic_json& target,
const string_t& path)
{
// The arrays and objects being diffed are kept on an explicit stack,
// and every pair of elements is still diffed completely before the
// next one, so the operations come out in the same order as in
// diff_recursively. The path of the values being diffed is kept in
// one buffer that grows and shrinks with the stack, rather than in a
// new string per level.
std::vector<diff_frame> stack;
string_t current_path = path;
// diff `s` against `t`, whose path is current_path: primitives,
// values of different types, and objects whose members were reordered
// are handled right away; arrays and other objects get a frame
const auto enter = [&result, &stack, &current_path](const basic_json & s, const basic_json & t)
{
// if the values are the same, there is nothing to do. Arrays and
// objects are not compared up front: comparing them visits
// everything below them, so doing that at every level would take
// quadratic time in the nesting depth - equal ones yield no
// operations anyway.
if ((!s.is_structured() || !t.is_structured()) && s == t)
{
return;
} }
if (s.type() != t.type())
{
// different types: replace value
diff_replace(result, current_path, t);
return;
}
switch (s.type())
{
case value_t::array:
{
stack.emplace_back(&s, &t, current_path.size());
return;
}
case value_t::object:
{
std::vector<typename object_t::key_type> common_keys;
basic_json added_ops(value_t::array);
if (diff_object_keys(result, s, t, current_path, common_keys, added_ops))
{
// fast path: the frame walks source in lockstep with
// common_keys, as diff_recursively does, and appends
// added_ops once all members are done
stack.emplace_back(&s, &t, current_path.size());
stack.back().member = s.cbegin();
stack.back().common_keys = std::move(common_keys);
stack.back().added_ops = std::move(added_ops);
}
return;
}
case value_t::null:
case value_t::string:
case value_t::boolean:
case value_t::number_integer:
case value_t::number_unsigned:
case value_t::number_float:
case value_t::binary:
case value_t::discarded:
default:
{
// both primitive types: replace value
diff_replace(result, current_path, t);
return;
}
}
};
enter(source, target);
while (!stack.empty())
{
// the frame is copied out member by member and changed through
// stack.back(): enter() may push a frame and the end of the loop
// pops it, either of which would invalidate a reference to it
const basic_json* const s = stack.back().source;
const basic_json* const t = stack.back().target;
const std::size_t path_length = stack.back().path_length;
const std::size_t depth = stack.size();
if (s->is_array())
{
const auto& source_array = *s->m_data.m_value.array;
const auto& target_array = *t->m_data.m_value.array;
// first pass: traverse common elements
const std::size_t i = stack.back().index;
if (i < source_array.size() && i < target_array.size())
{
++stack.back().index;
detail::concat_into(current_path, '/', detail::to_string<string_t>(i));
enter(source_array[i], target_array[i]);
if (stack.size() == depth)
{
current_path.resize(path_length);
}
continue;
}
// We now reached the end of at least one array
// in a second pass, traverse the remaining elements
diff_array_tails(result, *s, *t, current_path, i);
}
else
{
const const_iterator it = stack.back().member;
if (it != s->cend())
{
++stack.back().member;
const std::size_t next_common = stack.back().next_common;
if (next_common < stack.back().common_keys.size() && it.key() == stack.back().common_keys[next_common])
{
++stack.back().next_common;
const basic_json& target_value = (*t)[it.key()];
detail::concat_into(current_path, '/', detail::escape(it.key()));
enter(it.value(), target_value);
if (stack.size() == depth)
{
current_path.resize(path_length);
}
}
else
{
// found a key that is not in target -> remove it
diff_remove(result, detail::concat<string_t>(current_path, '/', detail::escape(it.key())));
}
continue;
}
// append the "add" ops for brand-new keys collected when the
// object was entered
result.insert(result.end(), stack.back().added_ops.begin(), stack.back().added_ops.end());
}
// this array or object is done: continue with the one it is in
stack.pop_back();
if (!stack.empty())
{
current_path.resize(stack.back().path_length);
}
}
}
public:
/// @} /// @}
//////////////////////////////// ////////////////////////////////
+107 -351
View File
@@ -19636,8 +19636,6 @@ NLOHMANN_JSON_NAMESPACE_END
// #include <nlohmann/detail/string_escape.hpp> // #include <nlohmann/detail/string_escape.hpp>
// #include <nlohmann/detail/string_utils.hpp>
// #include <nlohmann/detail/value_t.hpp> // #include <nlohmann/detail/value_t.hpp>
@@ -19729,7 +19727,7 @@ class json_pointer
/// @sa https://json.nlohmann.me/api/json_pointer/operator_slasheq/ /// @sa https://json.nlohmann.me/api/json_pointer/operator_slasheq/
json_pointer& operator/=(std::size_t array_idx) json_pointer& operator/=(std::size_t array_idx)
{ {
return *this /= detail::to_string<string_t>(array_idx); return *this /= std::to_string(array_idx);
} }
/// @brief create a new JSON pointer by appending the right JSON pointer at the end of the left JSON pointer /// @brief create a new JSON pointer by appending the right JSON pointer at the end of the left JSON pointer
@@ -20365,7 +20363,7 @@ class json_pointer
// would throw out_of_range.404 -- contains() must not throw (see #5395) // would throw out_of_range.404 -- contains() must not throw (see #5395)
return false; return false;
} }
if (JSON_HEDLEY_UNLIKELY(reference_token.size() == 1 && !('0' <= reference_token[0] && reference_token[0] <= '9'))) if (JSON_HEDLEY_UNLIKELY(reference_token.size() == 1 && !("0" <= reference_token && reference_token <= "9")))
{ {
// invalid char // invalid char
return false; return false;
@@ -20393,7 +20391,7 @@ class json_pointer
// not throw (see #5395), so such a reference token is treated as "not found" // not throw (see #5395), so such a reference token is treated as "not found"
errno = 0; // strtoull() does not reset errno on success errno = 0; // strtoull() does not reset errno on success
char* p_end = nullptr; // NOLINT(misc-const-correctness) char* p_end = nullptr; // NOLINT(misc-const-correctness)
const unsigned long long magnitude = std::strtoull(reference_token.data(), &p_end, 10); // NOLINT(runtime/int) const unsigned long long magnitude = std::strtoull(reference_token.c_str(), &p_end, 10); // NOLINT(runtime/int)
if (JSON_HEDLEY_UNLIKELY(errno == ERANGE // the value exceeds ULLONG_MAX if (JSON_HEDLEY_UNLIKELY(errno == ERANGE // the value exceeds ULLONG_MAX
|| magnitude >= static_cast<unsigned long long>((std::numeric_limits<typename BasicJsonType::size_type>::max)()))) // NOLINT(runtime/int) || magnitude >= static_cast<unsigned long long>((std::numeric_limits<typename BasicJsonType::size_type>::max)()))) // NOLINT(runtime/int)
{ {
@@ -32019,10 +32017,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
/// @sa https://json.nlohmann.me/api/basic_json/operator_gtgt/ /// @sa https://json.nlohmann.me/api/basic_json/operator_gtgt/
friend std::istream& operator>>(std::istream& i, basic_json& j) friend std::istream& operator>>(std::istream& i, basic_json& j)
{ {
// parse into a temporary so that j is left unchanged if parsing fails parser(detail::input_adapter(i)).parse(false, j);
basic_json result;
parser(detail::input_adapter(i)).parse(false, result);
j = std::move(result);
return i; return i;
} }
#endif // JSON_NO_IO #endif // JSON_NO_IO
@@ -33033,113 +33028,68 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
{ {
// the patch // the patch
basic_json result(value_t::array); basic_json result(value_t::array);
diff_recursively(result, source, target, path, 0);
// if the values are the same, return an empty patch
if (source == target)
{
return result; return result;
} }
private: if (source.type() != target.type())
/// @brief two arrays or two objects @ref diff_iteratively is diffing
struct diff_frame
{
diff_frame(const basic_json* source_, const basic_json* target_, const std::size_t path_length_) noexcept
: source(source_), target(target_), path_length(path_length_)
{}
// declared for GCC's -Weffc++, which asks for them in a class with
// pointer members and a non-trivial destructor; the exception
// specifications are left implicit, as GCC 4.8 rejects explicit ones
// that differ from them
diff_frame(const diff_frame&) = default;
diff_frame(diff_frame&&) = default;
diff_frame& operator=(const diff_frame&) = default;
diff_frame& operator=(diff_frame&&) = default;
~diff_frame() = default;
/// the values being diffed, both arrays or both objects
const basic_json* source;
const basic_json* target;
/// the length of their path in `current_path`
std::size_t path_length;
/// arrays: the next index to diff
std::size_t index = 0;
/// objects: the next member of source to look at
const_iterator member{}; // NOLINT(readability-redundant-member-init)
/// objects: the keys common to both, in source's order
std::vector<typename object_t::key_type> common_keys{}; // NOLINT(readability-redundant-member-init)
/// objects: the next entry of common_keys
std::size_t next_common = 0;
/// objects: the "add" operations for keys only target has
basic_json added_ops{}; // NOLINT(readability-redundant-member-init)
};
// The operations of a diff are built by the functions below rather than
// where they are needed: building one takes several temporaries, and
// unoptimized builds give each temporary a stack slot of its own in the
// function it appears in. In diff_recursively, which is on the call stack
// once per nesting level, that made every level cost kilobytes of stack.
/// @brief append a "replace" operation for @a path with @a value to @a result
static void diff_replace(basic_json& result, const string_t& path, const basic_json& value)
{ {
// different types: replace value
result.push_back( result.push_back(
{ {
{"op", "replace"}, {"path", path}, {"value", value} {"op", "replace"}, {"path", path}, {"value", target}
}); });
return result;
} }
/// @brief append a "remove" operation for @a path to @a result switch (source.type())
static void diff_remove(basic_json& result, const string_t& path) {
case value_t::array:
{
// first pass: traverse common elements
std::size_t i = 0;
while (i < source.size() && i < target.size())
{
// recursive call to compare array values at index i
auto temp_diff = diff(source[i], target[i], detail::concat<string_t>(path, '/', detail::to_string<string_t>(i)));
result.insert(result.end(), temp_diff.begin(), temp_diff.end());
++i;
}
// We now reached the end of at least one array
// in a second pass, traverse the remaining elements
// remove my remaining elements, highest index first; appending
// in that order avoids the quadratic reinsertion done before
for (std::size_t j = source.size(); j > i; --j)
{ {
result.push_back(object( result.push_back(object(
{ {
{"op", "remove"}, {"path", path} {"op", "remove"},
{"path", detail::concat<string_t>(path, '/', detail::to_string<string_t>(j - 1))}
})); }));
} }
i = source.size();
/// @brief append an "add" operation for @a path with @a value to @a result // add other remaining elements
static void diff_add(basic_json& result, const string_t& path, const basic_json& value) while (i < target.size())
{ {
result.push_back( result.push_back(
{ {
{"op", "add"}, {"path", path}, {"value", value} {"op", "add"},
{"path", detail::concat<string_t>(path, "/-")},
{"value", target[i]}
}); });
++i;
} }
/// @brief append the "remove" operations for the elements of array break;
/// @a source from @a index on, and the "add" operations for the
/// elements of array @a target from source's size on, to @a result
static void diff_array_tails(basic_json& result, const basic_json& source, const basic_json& target,
const string_t& path, const std::size_t index)
{
// remove my remaining elements, highest index first; appending
// in that order avoids the quadratic reinsertion done before
for (std::size_t j = source.size(); j > index; --j)
{
diff_remove(result, detail::concat<string_t>(path, '/', detail::to_string<string_t>(j - 1)));
} }
// add other remaining elements case value_t::object:
for (std::size_t i = source.size(); i < target.size(); ++i)
{
diff_add(result, detail::concat<string_t>(path, "/-"), target[i]);
}
}
/*!
@brief compare the keys of objects @a source and @a target
If object_t does not keep its members in insertion order, or if the keys
both objects have are in the same order in both, and the keys only
@a target has come after them, stores the keys common to both in
source's order in @a common_keys, stores the "add" operations for the keys
only @a target has in @a added_ops, and returns true: the caller then diffs
the objects member by member. Otherwise, appends operations that remove
every member of @a source and add every member of @a target to @a result,
and returns false.
*/
static bool diff_object_keys(basic_json& result, const basic_json& source, const basic_json& target,
const string_t& path, std::vector<typename object_t::key_type>& common_keys,
basic_json& added_ops)
{ {
// first pass: record, for every source key, whether it is // first pass: record, for every source key, whether it is
// common to both objects (in source's iteration order) or // common to both objects (in source's iteration order) or
@@ -33147,10 +33097,10 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
// a by-product of the target.find() call already needed to // a by-product of the target.find() call already needed to
// tell the two cases apart, so it adds no extra lookups. The // tell the two cases apart, so it adds no extra lookups. The
// "remove" ops themselves are emitted later, interleaved // "remove" ops themselves are emitted later, interleaved
// with the per-key diffs in the caller's fast path, to match // with the recursive per-key diffs in the fast path below,
// source's original iteration order (as the original, // to match source's original iteration order (as the
// pre-reordering-aware implementation did) instead of // original, pre-reordering-aware implementation did) instead
// grouping all removes before all per-key diffs. // of grouping all removes before all recursive diffs.
std::vector<typename object_t::key_type> common_keys_source_order; std::vector<typename object_t::key_type> common_keys_source_order;
for (auto it = source.cbegin(); it != source.cend(); ++it) for (auto it = source.cbegin(); it != source.cend(); ++it)
{ {
@@ -33166,17 +33116,18 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
// source.find() call already needed to detect added keys. At // source.find() call already needed to detect added keys. At
// the same time, determine whether every added key comes // the same time, determine whether every added key comes
// after every common key in target's order (a precondition // after every common key in target's order (a precondition
// for the fast path, which only ever appends new keys // for the fast path below, which only ever appends new keys
// at the very end). Both are only needed for an object_t that // at the very end). Both are only needed for an object_t that
// keeps its members in insertion order, such as the one // keeps its members in insertion order, such as the one
// backing `ordered_json`; for any other object_t, the fast // backing `ordered_json`; for any other object_t, the fast
// path is always taken and they are not computed. // path is always taken and they are not computed.
// The patch ops for keys that were added (i.e., in target but not // patch ops for keys that were added (i.e., in target but not
// in source) are built here so the fast path can reuse // in source); built here so the fast path below can reuse
// them without a second source.find() per target key. Only // them without a second source.find() per target key. Only
// used by the fast path -- the slow (reordering) path // used by the fast path -- the slow (reordering) path
// rebuilds "add" ops for every key itself. // rebuilds "add" ops for every key itself.
std::vector<typename object_t::key_type> common_keys_target_order; std::vector<typename object_t::key_type> common_keys_target_order;
basic_json added_ops(value_t::array);
bool new_keys_form_suffix = true; bool new_keys_form_suffix = true;
bool seen_new_key = false; bool seen_new_key = false;
for (auto it = target.cbegin(); it != target.cend(); ++it) for (auto it = target.cbegin(); it != target.cend(); ++it)
@@ -33184,7 +33135,12 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
if (source.find(it.key()) == source.end()) if (source.find(it.key()) == source.end())
{ {
seen_new_key = true; seen_new_key = true;
diff_add(added_ops, detail::concat<string_t>(path, '/', detail::escape(it.key())), it.value()); const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
added_ops.push_back(
{
{"op", "add"}, {"path", path_key},
{"value", it.value()}
});
} }
else else
{ {
@@ -33217,12 +33173,43 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
{ {
// fast path: order of common keys already matches (or the // fast path: order of common keys already matches (or the
// object_t's iteration order does not depend on // object_t's iteration order does not depend on
// insertion history), so a plain per-key diff is correct // insertion history), so a plain per-key recursive diff
// and minimal, as before // is correct and minimal, as before. common_keys_source_order
common_keys = std::move(common_keys_source_order); // is, by construction, the subsequence of source's keys
return true; // that are common to both objects, in source's iteration
// order -- so it can be walked in lockstep with `source`
// using a cheap key comparison instead of another lookup.
// Deleted keys (those source keys not in common_keys_source_order)
// are interleaved here too, in source's original order, to
// match the historical (pre-reordering-aware) output order.
auto common_it = common_keys_source_order.cbegin();
for (auto it = source.cbegin(); it != source.cend(); ++it)
{
if (common_it != common_keys_source_order.cend() && it.key() == *common_it)
{
const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
auto temp_diff = diff(it.value(), target[it.key()], path_key);
result.insert(result.end(), temp_diff.begin(), temp_diff.end());
++common_it;
}
else
{
// found a key that is not in target -> remove it
const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
result.push_back(object(
{
{"op", "remove"}, {"path", path_key}
}));
}
} }
// append the "add" ops for brand-new keys collected above
// during the pass over target -- no second source.find()
// per target key needed
result.insert(result.end(), added_ops.begin(), added_ops.end());
}
else
{
// slow path: the common keys are in a different relative // slow path: the common keys are in a different relative
// order in source and target (only possible for a // order in source and target (only possible for a
// reorderable object_t like ordered_map). Building a // reorderable object_t like ordered_map). Building a
@@ -33239,7 +33226,11 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
// it moves it to the end, fixing its position. // it moves it to the end, fixing its position.
for (auto it = source.cbegin(); it != source.cend(); ++it) for (auto it = source.cbegin(); it != source.cend(); ++it)
{ {
diff_remove(result, detail::concat<string_t>(path, '/', detail::escape(it.key()))); const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
result.push_back(object(
{
{"op", "remove"}, {"path", path_key}
}));
} }
// add every key that is either common (just removed // add every key that is either common (just removed
@@ -33248,96 +33239,15 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
// target exactly // target exactly
for (auto it = target.cbegin(); it != target.cend(); ++it) for (auto it = target.cbegin(); it != target.cend(); ++it)
{ {
diff_add(result, detail::concat<string_t>(path, '/', detail::escape(it.key())), it.value()); const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
} result.push_back(
return false;
}
/*!
@brief @ref diff, for values at nesting level @a depth, appending the
operations to @a result
Diffing two arrays or objects calls this function again, once per nesting
level, so values nested deeply enough used to exhaust the call stack and
terminate the process. The descent is bounded here: once @ref
detail::recursion_depth_limit levels have been entered, @ref
diff_iteratively diffs what is left without the call stack.
*/
static void diff_recursively(basic_json& result, const basic_json& source, const basic_json& target,
const string_t& path, const std::size_t depth)
{ {
// if the values are the same, there is nothing to do {"op", "add"}, {"path", path_key},
if (source == target) {"value", it.value()}
{ });
return;
}
if (JSON_HEDLEY_UNLIKELY(depth >= detail::recursion_depth_limit()))
{
diff_iteratively(result, source, target, path);
return;
}
if (source.type() != target.type())
{
// different types: replace value
diff_replace(result, path, target);
return;
}
switch (source.type())
{
case value_t::array:
{
// first pass: traverse common elements
std::size_t i = 0;
while (i < source.size() && i < target.size())
{
// recursive call to compare array values at index i
diff_recursively(result, source[i], target[i], detail::concat<string_t>(path, '/', detail::to_string<string_t>(i)), depth + 1);
++i;
}
// We now reached the end of at least one array
// in a second pass, traverse the remaining elements
diff_array_tails(result, source, target, path, i);
break;
}
case value_t::object:
{
std::vector<typename object_t::key_type> common_keys;
basic_json added_ops(value_t::array);
if (diff_object_keys(result, source, target, path, common_keys, added_ops))
{
// fast path: common_keys is, by construction, the
// subsequence of source's keys that are common to both
// objects, in source's iteration order -- so it can be
// walked in lockstep with `source` using a cheap key
// comparison instead of another lookup. Deleted keys
// (those source keys not in common_keys) are interleaved
// here too, in source's original order, to match the
// historical (pre-reordering-aware) output order.
auto common_it = common_keys.cbegin();
for (auto it = source.cbegin(); it != source.cend(); ++it)
{
if (common_it != common_keys.cend() && it.key() == *common_it)
{
diff_recursively(result, it.value(), target[it.key()], detail::concat<string_t>(path, '/', detail::escape(it.key())), depth + 1);
++common_it;
}
else
{
// found a key that is not in target -> remove it
diff_remove(result, detail::concat<string_t>(path, '/', detail::escape(it.key())));
} }
} }
// append the "add" ops for brand-new keys collected by
// diff_object_keys -- no second source.find() per target
// key needed
result.insert(result.end(), added_ops.begin(), added_ops.end());
}
break; break;
} }
@@ -33352,170 +33262,16 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
default: default:
{ {
// both primitive types: replace value // both primitive types: replace value
diff_replace(result, path, target); result.push_back(
{
{"op", "replace"}, {"path", path}, {"value", target}
});
break; break;
} }
} }
}
/*! return result;
@brief @ref diff without the call stack, appending the operations to
@a result
Produces the same operations as @ref diff_recursively. Only reached for
values nested more deeply than @ref detail::recursion_depth_limit.
*/
static void diff_iteratively(basic_json& result, const basic_json& source, const basic_json& target,
const string_t& path)
{
// The arrays and objects being diffed are kept on an explicit stack,
// and every pair of elements is still diffed completely before the
// next one, so the operations come out in the same order as in
// diff_recursively. The path of the values being diffed is kept in
// one buffer that grows and shrinks with the stack, rather than in a
// new string per level.
std::vector<diff_frame> stack;
string_t current_path = path;
// diff `s` against `t`, whose path is current_path: primitives,
// values of different types, and objects whose members were reordered
// are handled right away; arrays and other objects get a frame
const auto enter = [&result, &stack, &current_path](const basic_json & s, const basic_json & t)
{
// if the values are the same, there is nothing to do. Arrays and
// objects are not compared up front: comparing them visits
// everything below them, so doing that at every level would take
// quadratic time in the nesting depth - equal ones yield no
// operations anyway.
if ((!s.is_structured() || !t.is_structured()) && s == t)
{
return;
} }
if (s.type() != t.type())
{
// different types: replace value
diff_replace(result, current_path, t);
return;
}
switch (s.type())
{
case value_t::array:
{
stack.emplace_back(&s, &t, current_path.size());
return;
}
case value_t::object:
{
std::vector<typename object_t::key_type> common_keys;
basic_json added_ops(value_t::array);
if (diff_object_keys(result, s, t, current_path, common_keys, added_ops))
{
// fast path: the frame walks source in lockstep with
// common_keys, as diff_recursively does, and appends
// added_ops once all members are done
stack.emplace_back(&s, &t, current_path.size());
stack.back().member = s.cbegin();
stack.back().common_keys = std::move(common_keys);
stack.back().added_ops = std::move(added_ops);
}
return;
}
case value_t::null:
case value_t::string:
case value_t::boolean:
case value_t::number_integer:
case value_t::number_unsigned:
case value_t::number_float:
case value_t::binary:
case value_t::discarded:
default:
{
// both primitive types: replace value
diff_replace(result, current_path, t);
return;
}
}
};
enter(source, target);
while (!stack.empty())
{
// the frame is copied out member by member and changed through
// stack.back(): enter() may push a frame and the end of the loop
// pops it, either of which would invalidate a reference to it
const basic_json* const s = stack.back().source;
const basic_json* const t = stack.back().target;
const std::size_t path_length = stack.back().path_length;
const std::size_t depth = stack.size();
if (s->is_array())
{
const auto& source_array = *s->m_data.m_value.array;
const auto& target_array = *t->m_data.m_value.array;
// first pass: traverse common elements
const std::size_t i = stack.back().index;
if (i < source_array.size() && i < target_array.size())
{
++stack.back().index;
detail::concat_into(current_path, '/', detail::to_string<string_t>(i));
enter(source_array[i], target_array[i]);
if (stack.size() == depth)
{
current_path.resize(path_length);
}
continue;
}
// We now reached the end of at least one array
// in a second pass, traverse the remaining elements
diff_array_tails(result, *s, *t, current_path, i);
}
else
{
const const_iterator it = stack.back().member;
if (it != s->cend())
{
++stack.back().member;
const std::size_t next_common = stack.back().next_common;
if (next_common < stack.back().common_keys.size() && it.key() == stack.back().common_keys[next_common])
{
++stack.back().next_common;
const basic_json& target_value = (*t)[it.key()];
detail::concat_into(current_path, '/', detail::escape(it.key()));
enter(it.value(), target_value);
if (stack.size() == depth)
{
current_path.resize(path_length);
}
}
else
{
// found a key that is not in target -> remove it
diff_remove(result, detail::concat<string_t>(current_path, '/', detail::escape(it.key())));
}
continue;
}
// append the "add" ops for brand-new keys collected when the
// object was entered
result.insert(result.end(), stack.back().added_ops.begin(), stack.back().added_ops.end());
}
// this array or object is done: continue with the one it is in
stack.pop_back();
if (!stack.empty())
{
current_path.resize(stack.back().path_length);
}
}
}
public:
/// @} /// @}
//////////////////////////////// ////////////////////////////////
-35
View File
@@ -352,41 +352,6 @@ TEST_CASE("alternative string type")
CHECK(j2.flatten().unflatten() == j2); CHECK(j2.flatten().unflatten() == j2);
} }
SECTION("contains(json_pointer)")
{
// contains(json_pointer) must compile and work with a string_t that has
// no c_str() and no comparison with const char* (see #5666)
auto j = alt_json::parse(R"({"foo": ["bar", "baz"]})");
// present: object key and array indices
CHECK(j.contains(alt_json::json_pointer("/foo")));
CHECK(j.contains(alt_json::json_pointer("/foo/0")));
CHECK(j.contains(alt_json::json_pointer("/foo/1")));
// missing: absent object key and out-of-range array index
CHECK_FALSE(j.contains(alt_json::json_pointer("/bar")));
CHECK_FALSE(j.contains(alt_json::json_pointer("/foo/2")));
// "-" always fails the range check
CHECK_FALSE(j.contains(alt_json::json_pointer("/foo/-")));
// an array index must not have a leading zero
CHECK_FALSE(j.contains(alt_json::json_pointer("/foo/01")));
// a reference token that is not a number
CHECK_FALSE(j.contains(alt_json::json_pointer("/foo/bar")));
}
SECTION("operator/(std::size_t)")
{
// json_pointer::operator/=(std::size_t) must compile without string_t
// being constructible from std::string (see #5666)
auto j = alt_json::parse(R"({"foo": ["bar", "baz"]})");
CHECK(j.at(alt_json::json_pointer("/foo") / std::size_t(0)) == j["foo"][0]);
CHECK(j.at(alt_json::json_pointer("/foo") / std::size_t(1)) == j["foo"][1]);
}
SECTION("patch") SECTION("patch")
{ {
alt_json const patch1 = alt_json::parse(R"([{ "op": "add", "path": "/a/b", "value": [ "foo", "bar" ] }])"); alt_json const patch1 = alt_json::parse(R"([{ "op": "add", "path": "/a/b", "value": [ "foo", "bar" ] }])");
-16
View File
@@ -19,7 +19,6 @@ using nlohmann::json;
#include <map> #include <map>
#include <unordered_map> #include <unordered_map>
#include <sstream>
TEST_CASE("Better diagnostics") TEST_CASE("Better diagnostics")
{ {
@@ -493,21 +492,6 @@ TEST_CASE("Regression tests for extended diagnostics")
CHECK(copy == j); CHECK(copy == j);
} }
} }
SECTION("Regression test for issue #5652 - operator>> leaves a partial value in its target on a parse error")
{
json j = "old value";
std::istringstream is("[1, x");
CHECK_THROWS_WITH_AS(is >> j, "[json.exception.parse_error.101] parse error at line 1, column 5: syntax error while parsing value - invalid literal; last read: '1, x'", json::parse_error);
// j must be left unchanged, as json::parse() guarantees for its result
CHECK(j == "old value");
// copying j must not trigger assert_invariant(): a failed parse must
// not leave array/object elements without a parent pointer
json const copy = j; // NOLINT(performance-unnecessary-copy-initialization)
CHECK(copy == j);
}
} }
TEST_CASE("Better diagnostics past the descent bound of update() and merge_patch()") TEST_CASE("Better diagnostics past the descent bound of update() and merge_patch()")
-153
View File
@@ -15,65 +15,8 @@ using nlohmann::json;
#endif #endif
#include <fstream> #include <fstream>
#include <string>
#include <vector>
#include "make_test_data_available.hpp" #include "make_test_data_available.hpp"
namespace
{
// alternating objects and arrays nested `depth` levels deep, with members that
// depend on `variant` at some levels, so diffing two variants yields
// operations on many levels: replacing the innermost value, adding, removing,
// and (for ordered_json) reordering members, and changing array lengths
template<typename BasicJsonType>
BasicJsonType nested(const std::size_t depth, const int variant)
{
BasicJsonType value = variant;
for (std::size_t i = 0; i < depth; ++i)
{
if (i % 2 == 0)
{
BasicJsonType object = BasicJsonType::object();
if ((i + static_cast<std::size_t>(variant)) % 7 == 0)
{
object["x"] = i;
}
if (variant == 2 && i % 11 == 0)
{
object["z"] = "z";
}
object["a"] = std::move(value);
if (variant == 1 && i % 5 == 0)
{
object["y"] = 1;
}
value = std::move(object);
}
else
{
BasicJsonType array = BasicJsonType::array({std::move(value)});
if ((i + static_cast<std::size_t>(variant)) % 3 == 0)
{
array.push_back(i);
}
value = std::move(array);
}
}
return value;
}
// a path of `depth` reference tokens, as nested() nests its values
std::string nested_path(const std::size_t depth)
{
std::string path;
for (std::size_t i = depth; i > 0; --i)
{
path += (i - 1) % 2 == 0 ? "/a" : "/0";
}
return path;
}
} // namespace
TEST_CASE("JSON patch") TEST_CASE("JSON patch")
{ {
SECTION("examples from RFC 6902") SECTION("examples from RFC 6902")
@@ -1809,102 +1752,6 @@ TEST_CASE("JSON patch - diff emits array removals in descending index order")
} }
} }
TEST_CASE("JSON patch: diff of deeply nested values")
{
SECTION("the diff reproduces the target at every depth")
{
// depths on either side of the nesting depth up to which diff()
// recurses (detail::recursion_depth_limit(), 128); not every depth up
// to 300, as the test would then time out under Valgrind
std::vector<std::size_t> depths;
for (std::size_t depth = 0; depth <= 16; ++depth)
{
depths.push_back(depth);
}
for (std::size_t depth = 120; depth <= 136; ++depth)
{
depths.push_back(depth);
}
depths.push_back(300);
for (const auto depth : depths)
{
CAPTURE(depth);
for (int from = 0; from < 3; ++from)
{
for (int to = 0; to < 3; ++to)
{
CAPTURE(from);
CAPTURE(to);
const auto source = nested<json>(depth, from);
const auto target = nested<json>(depth, to);
const auto patch = json::diff(source, target);
CHECK(source.patch(patch) == target);
CHECK(patch.empty() == (from == to));
const auto ordered_source = nested<nlohmann::ordered_json>(depth, from);
const auto ordered_target = nested<nlohmann::ordered_json>(depth, to);
CHECK(ordered_source.patch(nlohmann::ordered_json::diff(ordered_source, ordered_target)) == ordered_target);
}
}
}
}
SECTION("a difference only in the innermost value is one replace operation")
{
for (std::size_t depth = 0; depth <= 300; ++depth)
{
CAPTURE(depth);
json source = 1;
json target = 2;
for (std::size_t i = 0; i < depth; ++i)
{
source = i % 2 == 0 ? json::object({{"a", std::move(source)}}) : json::array({std::move(source)});
target = i % 2 == 0 ? json::object({{"a", std::move(target)}}) : json::array({std::move(target)});
}
CHECK(json::diff(source, target, "/root") == json::array({{{"op", "replace"}, {"path", "/root" + nested_path(depth)}, {"value", 2}}}));
}
}
SECTION("values nested too deeply for the call stack (#5393)")
{
// diff() used to recurse once per nesting level, and compared the
// values with operator== on every level. The values are only
// parsed and diffed, never copied or compared, since those recurse
// too.
const std::size_t depth = 100000;
for (const bool objects :
{
false, true
})
{
CAPTURE(objects);
std::string source_text;
std::string target_text;
std::string equal_text;
std::string path;
for (std::size_t i = 0; i < depth; ++i)
{
source_text += objects ? "{\"a\":" : "[";
path += objects ? "/a" : "/0";
}
target_text = source_text + "2";
equal_text = source_text + "1";
source_text += "1";
const std::string closing(depth, objects ? '}' : ']');
const auto source = json::parse(source_text + closing);
const auto patch = json::diff(source, json::parse(target_text + closing));
REQUIRE(patch.size() == 1);
CHECK(patch[0]["op"] == "replace");
CHECK(patch[0]["path"] == path);
CHECK(patch[0]["value"] == 2);
CHECK(json::diff(source, json::parse(equal_text + closing)).empty());
}
}
}
TEST_CASE("JSON patch - diff() takes the fast path for non-reorderable object types (regression #5639)") TEST_CASE("JSON patch - diff() takes the fast path for non-reorderable object types (regression #5639)")
{ {
// #5465 added an order check to diff()'s object handling so a // #5465 added an order check to diff()'s object handling so a