Compare commits

..
Author SHA1 Message Date
Niels Lohmann 2dc43e8bf8 Merge branch 'develop' into claude/fix-issue-2649-738aba
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-27 20:58:11 +02:00
Niels Lohmann e0eb3e7490 Construct explicitly in get_to() and to_json(std::optional)
With JSON_USE_IMPLICIT_CONVERSIONS=0 the conversion from a basic_json
with a different string type is now explicit, but two library paths
still assigned such a value implicitly and failed to compile inside the
library:

- get_to() with a basic_json target (the #2175 overload) did
  `v = *this`, so json(42).get_to(alt_json&) broke although
  get<alt_json>() works.
- to_json(BasicJsonType&, const std::optional<T>&) is constrained on
  std::is_constructible (which accepts the explicit constructor) but did
  `j = *opt`, so converting a std::optional<alt_json> into a json broke.

Both now construct the value explicitly, as get_impl() already does.

Also replace static_cast<bool>(JSON_USE_IMPLICIT_CONVERSIONS) with a
comparison: clang-tidy's modernize-use-bool-literals rejected the cast
of the integer literal the macro expands to, failing ci_clang_tidy.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-27 18:31:22 +02:00
Niels Lohmann 3ae4d7c45e Make cross-string-type basic_json conversion explicit without implicit conversions
The converting constructor from another basic_json specialization was
always implicit, so a value with a different string_t (std::wstring, a
string with a custom allocator, ...) silently converted into a temporary,
e.g. when passed to a function taking const nlohmann::json&. Such
conversions do not produce correct values (#3425), and
JSON_USE_IMPLICIT_CONVERSIONS=0 did not catch them.

When JSON_USE_IMPLICIT_CONVERSIONS is 0, the constructor is now explicit
if the string types differ. Specializations sharing a string type (json
and ordered_json, different serializers or object maps) stay implicitly
convertible, so the NLOHMANN_DEFINE_TYPE_* macros keep working with
nested json members. get<BasicJsonType>() constructs explicitly and
works in both modes.

Fixes #2649.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-09-27 16:32:26 +02:00
7 changed files with 480 additions and 975 deletions
+11 -1
View File
@@ -272,6 +272,15 @@ basic_json(basic_json&& other) noexcept;
When used without parentheses around an empty initializer list, `basic_json()` is called instead of this When used without parentheses around an empty initializer list, `basic_json()` is called instead of this
function, yielding the JSON `#!json null` value. function, yielding the JSON `#!json null` value.
- Overload 4:
!!! info "Implicit conversion"
The conversion is implicit unless [`JSON_USE_IMPLICIT_CONVERSIONS`](../macros/json_use_implicit_conversions.md)
is defined to `0` and `BasicJsonType::string_t` differs from `string_t`. In that case, the constructor is
`explicit`, so a JSON value with a different string type is no longer silently converted, for example when it is
passed to a function taking `#!cpp const json&`. Write `#!cpp json(other)` or `#!cpp other.get<json>()` instead.
- Overload 7: - Overload 7:
!!! info "Preconditions" !!! info "Preconditions"
@@ -420,7 +429,8 @@ basic_json(basic_json&& other) noexcept;
1. Since version 1.0.0. 1. Since version 1.0.0.
2. Since version 1.0.0. 2. Since version 1.0.0.
3. Since version 2.1.0. 3. Since version 2.1.0.
4. Since version 3.2.0. 4. Since version 3.2.0. Explicit for different string types if `JSON_USE_IMPLICIT_CONVERSIONS` is `0` since
version 3.13.0.
5. Since version 1.0.0. 5. Since version 1.0.0.
6. Since version 1.0.0. 6. Since version 1.0.0.
7. Since version 1.0.0. 7. Since version 1.0.0.
@@ -5,7 +5,9 @@
``` ```
When defined to `0`, implicit conversions are switched off. By default, implicit conversions are switched on. The When defined to `0`, implicit conversions are switched off. By default, implicit conversions are switched on. The
value directly affects [`operator ValueType`](../basic_json/operator_ValueType.md). value directly affects [`operator ValueType`](../basic_json/operator_ValueType.md) and the
[converting constructor](../basic_json/basic_json.md) from a `basic_json` specialization with a different string
type (overload 4).
## Default definition ## Default definition
@@ -57,6 +59,25 @@ By default, implicit conversions are enabled.
auto s = j.get<std::string>(); auto s = j.get<std::string>();
``` ```
??? example "Conversion between `basic_json` specializations"
A `basic_json` specialization with a different string type is also no longer converted implicitly when
`JSON_USE_IMPLICIT_CONVERSIONS` is defined to `0`:
```cpp
using wjson = nlohmann::basic_json<std::map, std::vector, std::wstring>;
void load(const nlohmann::json& j);
wjson wj = /* ... */;
load(wj); // error: no implicit conversion
load(nlohmann::json(wj)); // OK: explicit conversion
load(wj.get<nlohmann::json>()); // OK: explicit conversion
```
Specializations that share the same string type, such as `json` and `ordered_json`, remain implicitly
convertible.
## See also ## See also
- [**operator ValueType**](../basic_json/operator_ValueType.md) - get a value (implicit) - [**operator ValueType**](../basic_json/operator_ValueType.md) - get a value (implicit)
@@ -66,3 +87,4 @@ By default, implicit conversions are enabled.
## Version history ## Version history
- Added in version 3.9.0. - Added in version 3.9.0.
- Also affects the conversion between `basic_json` specializations with different string types since version 3.13.0.
@@ -291,7 +291,9 @@ void to_json(BasicJsonType& j, const std::optional<T>& opt) noexcept
{ {
if (opt.has_value()) if (opt.has_value())
{ {
j = *opt; // explicit construction, as the conversion from a basic_json with a different
// string type is explicit if JSON_USE_IMPLICIT_CONVERSIONS is 0 (#2649)
j = BasicJsonType(*opt);
} }
else else
{ {
+202 -409
View File
@@ -1605,12 +1605,42 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
assert_invariant(); assert_invariant();
} }
private:
/// whether a basic_json specialization can be converted implicitly into this one;
/// with JSON_USE_IMPLICIT_CONVERSIONS set to 0, this is only the case if both share
/// the same string type (see https://github.com/nlohmann/json/issues/2649)
template<typename BasicJsonType>
using is_implicitly_convertible_basic_json = std::integral_constant < bool,
(JSON_USE_IMPLICIT_CONVERSIONS != 0)
|| std::is_same<typename BasicJsonType::string_t, string_t>::value >;
/// tag to select the constructor that performs the conversion from another basic_json specialization
struct convert_basic_json_tag {};
public:
/// @brief create a JSON value from an existing one /// @brief create a JSON value from an existing one
/// @sa https://json.nlohmann.me/api/basic_json/basic_json/ /// @sa https://json.nlohmann.me/api/basic_json/basic_json/
template < typename BasicJsonType, template < typename BasicJsonType,
detail::enable_if_t < detail::enable_if_t <
detail::is_basic_json<BasicJsonType>::value&& !std::is_same<basic_json, BasicJsonType>::value, int > = 0 > detail::is_basic_json<BasicJsonType>::value&& !std::is_same<basic_json, BasicJsonType>::value
&& is_implicitly_convertible_basic_json<BasicJsonType>::value, int > = 0 >
basic_json(const BasicJsonType& val) basic_json(const BasicJsonType& val)
: basic_json(val, convert_basic_json_tag{})
{}
/// @brief create a JSON value from an existing one
/// @sa https://json.nlohmann.me/api/basic_json/basic_json/
template < typename BasicJsonType,
detail::enable_if_t <
detail::is_basic_json<BasicJsonType>::value&& !std::is_same<basic_json, BasicJsonType>::value
&& !is_implicitly_convertible_basic_json<BasicJsonType>::value, int > = 0 >
explicit basic_json(const BasicJsonType& val)
: basic_json(val, convert_basic_json_tag{})
{}
private:
template<typename BasicJsonType>
basic_json(const BasicJsonType& val, convert_basic_json_tag /*unused*/)
#if JSON_DIAGNOSTIC_POSITIONS #if JSON_DIAGNOSTIC_POSITIONS
: start_position(val.start_pos()), : start_position(val.start_pos()),
end_position(val.end_pos()) end_position(val.end_pos())
@@ -1666,6 +1696,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
assert_invariant(); assert_invariant();
} }
public:
/// @brief create a container (array or object) from an initializer list /// @brief create a container (array or object) from an initializer list
/// @sa https://json.nlohmann.me/api/basic_json/basic_json/ /// @sa https://json.nlohmann.me/api/basic_json/basic_json/
basic_json(initializer_list_t init, basic_json(initializer_list_t init,
@@ -2432,7 +2463,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
int > = 0 > int > = 0 >
BasicJsonType get_impl(detail::priority_tag<2> /*unused*/) const BasicJsonType get_impl(detail::priority_tag<2> /*unused*/) const
{ {
return *this; return BasicJsonType(*this);
} }
/*! /*!
@@ -2571,7 +2602,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
int> = 0> int> = 0>
ValueType & get_to(ValueType& v) const ValueType & get_to(ValueType& v) const
{ {
v = *this; v = ValueType(*this);
return v; return v;
} }
@@ -6061,240 +6092,21 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
{ {
// the patch // the patch
basic_json result(value_t::array); basic_json result(value_t::array);
diff_recursively(result, source, target, path, 0);
return result;
}
private: // if the values are the same, return an empty patch
/// @brief two arrays or two objects @ref diff_iteratively is diffing
struct diff_frame
{
diff_frame(const basic_json* source_, const basic_json* target_, const std::size_t path_length_) noexcept
: source(source_), target(target_), path_length(path_length_)
{}
// declared for GCC's -Weffc++, which asks for them in a class with
// pointer members and a non-trivial destructor; the exception
// specifications are left implicit, as GCC 4.8 rejects explicit ones
// that differ from them
diff_frame(const diff_frame&) = default;
diff_frame(diff_frame&&) = default;
diff_frame& operator=(const diff_frame&) = default;
diff_frame& operator=(diff_frame&&) = default;
~diff_frame() = default;
/// the values being diffed, both arrays or both objects
const basic_json* source;
const basic_json* target;
/// the length of their path in `current_path`
std::size_t path_length;
/// arrays: the next index to diff
std::size_t index = 0;
/// objects: the next member of source to look at
const_iterator member{}; // NOLINT(readability-redundant-member-init)
/// objects: the keys common to both, in source's order
std::vector<typename object_t::key_type> common_keys{}; // NOLINT(readability-redundant-member-init)
/// objects: the next entry of common_keys
std::size_t next_common = 0;
/// objects: the "add" operations for keys only target has
basic_json added_ops{}; // NOLINT(readability-redundant-member-init)
};
// The operations of a diff are built by the functions below rather than
// where they are needed: building one takes several temporaries, and
// unoptimized builds give each temporary a stack slot of its own in the
// function it appears in. In diff_recursively, which is on the call stack
// once per nesting level, that made every level cost kilobytes of stack.
/// @brief append a "replace" operation for @a path with @a value to @a result
static void diff_replace(basic_json& result, const string_t& path, const basic_json& value)
{
result.push_back(
{
{"op", "replace"}, {"path", path}, {"value", value}
});
}
/// @brief append a "remove" operation for @a path to @a result
static void diff_remove(basic_json& result, const string_t& path)
{
result.push_back(object(
{
{"op", "remove"}, {"path", path}
}));
}
/// @brief append an "add" operation for @a path with @a value to @a result
static void diff_add(basic_json& result, const string_t& path, const basic_json& value)
{
result.push_back(
{
{"op", "add"}, {"path", path}, {"value", value}
});
}
/// @brief append the "remove" operations for the elements of array
/// @a source from @a index on, and the "add" operations for the
/// elements of array @a target from source's size on, to @a result
static void diff_array_tails(basic_json& result, const basic_json& source, const basic_json& target,
const string_t& path, const std::size_t index)
{
// remove my remaining elements, highest index first; appending
// in that order avoids the quadratic reinsertion done before
for (std::size_t j = source.size(); j > index; --j)
{
diff_remove(result, detail::concat<string_t>(path, '/', detail::to_string<string_t>(j - 1)));
}
// add other remaining elements
for (std::size_t i = source.size(); i < target.size(); ++i)
{
diff_add(result, detail::concat<string_t>(path, "/-"), target[i]);
}
}
/*!
@brief compare the keys of objects @a source and @a target
If the keys both objects have are in the same order in both, and the keys
only @a target has come after them, stores the keys common to both in
source's order in @a common_keys, stores the "add" operations for the keys
only @a target has in @a added_ops, and returns true: the caller then diffs
the objects member by member. Otherwise, appends operations that remove
every member of @a source and add every member of @a target to @a result,
and returns false.
*/
static bool diff_object_keys(basic_json& result, const basic_json& source, const basic_json& target,
const string_t& path, std::vector<typename object_t::key_type>& common_keys,
basic_json& added_ops)
{
// first pass: record, for every source key, whether it is
// common to both objects (in source's iteration order) or
// was deleted (i.e., in source but not in target) -- this is
// a by-product of the target.find() call already needed to
// tell the two cases apart, so it adds no extra lookups. The
// "remove" ops themselves are emitted later, interleaved
// with the per-key diffs in the caller's fast path, to match
// source's original iteration order (as the original,
// pre-reordering-aware implementation did) instead of
// grouping all removes before all per-key diffs.
std::vector<typename object_t::key_type> common_keys_source_order;
for (auto it = source.cbegin(); it != source.cend(); ++it)
{
if (target.find(it.key()) != target.end())
{
common_keys_source_order.push_back(it.key());
}
}
// second pass: find keys that were added (i.e., in target but
// not in source), and record the keys common to both, in
// target's iteration order -- again a by-product of the
// source.find() call already needed to detect added keys. At
// the same time, determine whether every added key comes
// after every common key in target's order (a precondition
// for the fast path, which only ever appends new keys
// at the very end): for an object_t whose iteration order is
// a pure function of the key set (e.g. the default std::map,
// which always iterates in sorted key order), the order
// check further below is always true and this whole
// mechanism is effectively a no-op; it only matters for a
// reorderable object_t such as the one backing `ordered_json`.
// The patch ops for keys that were added (i.e., in target but not
// in source) are built here so the fast path can reuse
// them without a second source.find() per target key. Only
// used by the fast path -- the slow (reordering) path
// rebuilds "add" ops for every key itself.
std::vector<typename object_t::key_type> common_keys_target_order;
bool new_keys_form_suffix = true;
bool seen_new_key = false;
for (auto it = target.cbegin(); it != target.cend(); ++it)
{
if (source.find(it.key()) == source.end())
{
seen_new_key = true;
diff_add(added_ops, detail::concat<string_t>(path, '/', detail::escape(it.key())), it.value());
}
else
{
common_keys_target_order.push_back(it.key());
if (seen_new_key)
{
new_keys_form_suffix = false;
}
}
}
if (common_keys_source_order == common_keys_target_order && new_keys_form_suffix)
{
// fast path: order of common keys already matches (or the
// object_t's iteration order does not depend on
// insertion history), so a plain per-key diff is correct
// and minimal, as before
common_keys = std::move(common_keys_source_order);
return true;
}
// slow path: the common keys are in a different relative
// order in source and target (only possible for a
// reorderable object_t like ordered_map). Building a
// minimal reordering patch is a nontrivial (LCS-like)
// problem; instead, remove every source key -- both
// deleted keys (which must be removed regardless) and
// common keys (removed so they can be re-added in
// target's order) -- and re-add every key that should
// remain, with its final target value, in target's
// order. basic_json::patch()'s "add" operation on an
// object uses operator[], which appends at the end for a
// vector-backed insertion-ordered map when the key does
// not already exist -- so removing a key and then adding
// it moves it to the end, fixing its position.
for (auto it = source.cbegin(); it != source.cend(); ++it)
{
diff_remove(result, detail::concat<string_t>(path, '/', detail::escape(it.key())));
}
// add every key that is either common (just removed
// above) or brand new, in target's iteration order, so
// that the final order after applying the patch matches
// target exactly
for (auto it = target.cbegin(); it != target.cend(); ++it)
{
diff_add(result, detail::concat<string_t>(path, '/', detail::escape(it.key())), it.value());
}
return false;
}
/*!
@brief @ref diff, for values at nesting level @a depth, appending the
operations to @a result
Diffing two arrays or objects calls this function again, once per nesting
level, so values nested deeply enough used to exhaust the call stack and
terminate the process. The descent is bounded here: once @ref
detail::recursion_depth_limit levels have been entered, @ref
diff_iteratively diffs what is left without the call stack.
*/
static void diff_recursively(basic_json& result, const basic_json& source, const basic_json& target,
const string_t& path, const std::size_t depth)
{
// if the values are the same, there is nothing to do
if (source == target) if (source == target)
{ {
return; return result;
}
if (JSON_HEDLEY_UNLIKELY(depth >= detail::recursion_depth_limit()))
{
diff_iteratively(result, source, target, path);
return;
} }
if (source.type() != target.type()) if (source.type() != target.type())
{ {
// different types: replace value // different types: replace value
diff_replace(result, path, target); result.push_back(
return; {
{"op", "replace"}, {"path", path}, {"value", target}
});
return result;
} }
switch (source.type()) switch (source.type())
@@ -6306,50 +6118,185 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
while (i < source.size() && i < target.size()) while (i < source.size() && i < target.size())
{ {
// recursive call to compare array values at index i // recursive call to compare array values at index i
diff_recursively(result, source[i], target[i], detail::concat<string_t>(path, '/', detail::to_string<string_t>(i)), depth + 1); auto temp_diff = diff(source[i], target[i], detail::concat<string_t>(path, '/', detail::to_string<string_t>(i)));
result.insert(result.end(), temp_diff.begin(), temp_diff.end());
++i; ++i;
} }
// We now reached the end of at least one array // We now reached the end of at least one array
// in a second pass, traverse the remaining elements // in a second pass, traverse the remaining elements
diff_array_tails(result, source, target, path, i);
// remove my remaining elements, highest index first; appending
// in that order avoids the quadratic reinsertion done before
for (std::size_t j = source.size(); j > i; --j)
{
result.push_back(object(
{
{"op", "remove"},
{"path", detail::concat<string_t>(path, '/', detail::to_string<string_t>(j - 1))}
}));
}
i = source.size();
// add other remaining elements
while (i < target.size())
{
result.push_back(
{
{"op", "add"},
{"path", detail::concat<string_t>(path, "/-")},
{"value", target[i]}
});
++i;
}
break; break;
} }
case value_t::object: case value_t::object:
{ {
std::vector<typename object_t::key_type> common_keys; // first pass: record, for every source key, whether it is
basic_json added_ops(value_t::array); // common to both objects (in source's iteration order) or
if (diff_object_keys(result, source, target, path, common_keys, added_ops)) // was deleted (i.e., in source but not in target) -- this is
// a by-product of the target.find() call already needed to
// tell the two cases apart, so it adds no extra lookups. The
// "remove" ops themselves are emitted later, interleaved
// with the recursive per-key diffs in the fast path below,
// to match source's original iteration order (as the
// original, pre-reordering-aware implementation did) instead
// of grouping all removes before all recursive diffs.
std::vector<typename object_t::key_type> common_keys_source_order;
for (auto it = source.cbegin(); it != source.cend(); ++it)
{ {
// fast path: common_keys is, by construction, the if (target.find(it.key()) != target.end())
// subsequence of source's keys that are common to both {
// objects, in source's iteration order -- so it can be common_keys_source_order.push_back(it.key());
// walked in lockstep with `source` using a cheap key }
// comparison instead of another lookup. Deleted keys }
// (those source keys not in common_keys) are interleaved
// here too, in source's original order, to match the // second pass: find keys that were added (i.e., in target but
// historical (pre-reordering-aware) output order. // not in source), and record the keys common to both, in
auto common_it = common_keys.cbegin(); // target's iteration order -- again a by-product of the
// source.find() call already needed to detect added keys. At
// the same time, determine whether every added key comes
// after every common key in target's order (a precondition
// for the fast path below, which only ever appends new keys
// at the very end): for an object_t whose iteration order is
// a pure function of the key set (e.g. the default std::map,
// which always iterates in sorted key order), the order
// check further below is always true and this whole
// mechanism is effectively a no-op; it only matters for a
// reorderable object_t such as the one backing `ordered_json`.
// patch ops for keys that were added (i.e., in target but not
// in source); built here so the fast path below can reuse
// them without a second source.find() per target key. Only
// used by the fast path -- the slow (reordering) path
// rebuilds "add" ops for every key itself.
std::vector<typename object_t::key_type> common_keys_target_order;
basic_json added_ops(value_t::array);
bool new_keys_form_suffix = true;
bool seen_new_key = false;
for (auto it = target.cbegin(); it != target.cend(); ++it)
{
if (source.find(it.key()) == source.end())
{
seen_new_key = true;
const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
added_ops.push_back(
{
{"op", "add"}, {"path", path_key},
{"value", it.value()}
});
}
else
{
common_keys_target_order.push_back(it.key());
if (seen_new_key)
{
new_keys_form_suffix = false;
}
}
}
if (common_keys_source_order == common_keys_target_order && new_keys_form_suffix)
{
// fast path: order of common keys already matches (or the
// object_t's iteration order does not depend on
// insertion history), so a plain per-key recursive diff
// is correct and minimal, as before. common_keys_source_order
// is, by construction, the subsequence of source's keys
// that are common to both objects, in source's iteration
// order -- so it can be walked in lockstep with `source`
// using a cheap key comparison instead of another lookup.
// Deleted keys (those source keys not in common_keys_source_order)
// are interleaved here too, in source's original order, to
// match the historical (pre-reordering-aware) output order.
auto common_it = common_keys_source_order.cbegin();
for (auto it = source.cbegin(); it != source.cend(); ++it) for (auto it = source.cbegin(); it != source.cend(); ++it)
{ {
if (common_it != common_keys.cend() && it.key() == *common_it) if (common_it != common_keys_source_order.cend() && it.key() == *common_it)
{ {
diff_recursively(result, it.value(), target[it.key()], detail::concat<string_t>(path, '/', detail::escape(it.key())), depth + 1); const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
auto temp_diff = diff(it.value(), target[it.key()], path_key);
result.insert(result.end(), temp_diff.begin(), temp_diff.end());
++common_it; ++common_it;
} }
else else
{ {
// found a key that is not in target -> remove it // found a key that is not in target -> remove it
diff_remove(result, detail::concat<string_t>(path, '/', detail::escape(it.key()))); const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
result.push_back(object(
{
{"op", "remove"}, {"path", path_key}
}));
} }
} }
// append the "add" ops for brand-new keys collected by // append the "add" ops for brand-new keys collected above
// diff_object_keys -- no second source.find() per target // during the pass over target -- no second source.find()
// key needed // per target key needed
result.insert(result.end(), added_ops.begin(), added_ops.end()); result.insert(result.end(), added_ops.begin(), added_ops.end());
} }
else
{
// slow path: the common keys are in a different relative
// order in source and target (only possible for a
// reorderable object_t like ordered_map). Building a
// minimal reordering patch is a nontrivial (LCS-like)
// problem; instead, remove every source key -- both
// deleted keys (which must be removed regardless) and
// common keys (removed so they can be re-added in
// target's order) -- and re-add every key that should
// remain, with its final target value, in target's
// order. basic_json::patch()'s "add" operation on an
// object uses operator[], which appends at the end for a
// vector-backed insertion-ordered map when the key does
// not already exist -- so removing a key and then adding
// it moves it to the end, fixing its position.
for (auto it = source.cbegin(); it != source.cend(); ++it)
{
const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
result.push_back(object(
{
{"op", "remove"}, {"path", path_key}
}));
}
// add every key that is either common (just removed
// above) or brand new, in target's iteration order, so
// that the final order after applying the patch matches
// target exactly
for (auto it = target.cbegin(); it != target.cend(); ++it)
{
const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
result.push_back(
{
{"op", "add"}, {"path", path_key},
{"value", it.value()}
});
}
}
break; break;
} }
@@ -6364,170 +6311,16 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
default: default:
{ {
// both primitive types: replace value // both primitive types: replace value
diff_replace(result, path, target); result.push_back(
{
{"op", "replace"}, {"path", path}, {"value", target}
});
break; break;
} }
} }
return result;
} }
/*!
@brief @ref diff without the call stack, appending the operations to
@a result
Produces the same operations as @ref diff_recursively. Only reached for
values nested more deeply than @ref detail::recursion_depth_limit.
*/
static void diff_iteratively(basic_json& result, const basic_json& source, const basic_json& target,
const string_t& path)
{
// The arrays and objects being diffed are kept on an explicit stack,
// and every pair of elements is still diffed completely before the
// next one, so the operations come out in the same order as in
// diff_recursively. The path of the values being diffed is kept in
// one buffer that grows and shrinks with the stack, rather than in a
// new string per level.
std::vector<diff_frame> stack;
string_t current_path = path;
// diff `s` against `t`, whose path is current_path: primitives,
// values of different types, and objects whose members were reordered
// are handled right away; arrays and other objects get a frame
const auto enter = [&result, &stack, &current_path](const basic_json & s, const basic_json & t)
{
// if the values are the same, there is nothing to do. Arrays and
// objects are not compared up front: comparing them visits
// everything below them, so doing that at every level would take
// quadratic time in the nesting depth - equal ones yield no
// operations anyway.
if ((!s.is_structured() || !t.is_structured()) && s == t)
{
return;
}
if (s.type() != t.type())
{
// different types: replace value
diff_replace(result, current_path, t);
return;
}
switch (s.type())
{
case value_t::array:
{
stack.emplace_back(&s, &t, current_path.size());
return;
}
case value_t::object:
{
std::vector<typename object_t::key_type> common_keys;
basic_json added_ops(value_t::array);
if (diff_object_keys(result, s, t, current_path, common_keys, added_ops))
{
// fast path: the frame walks source in lockstep with
// common_keys, as diff_recursively does, and appends
// added_ops once all members are done
stack.emplace_back(&s, &t, current_path.size());
stack.back().member = s.cbegin();
stack.back().common_keys = std::move(common_keys);
stack.back().added_ops = std::move(added_ops);
}
return;
}
case value_t::null:
case value_t::string:
case value_t::boolean:
case value_t::number_integer:
case value_t::number_unsigned:
case value_t::number_float:
case value_t::binary:
case value_t::discarded:
default:
{
// both primitive types: replace value
diff_replace(result, current_path, t);
return;
}
}
};
enter(source, target);
while (!stack.empty())
{
// the frame is copied out member by member and changed through
// stack.back(): enter() may push a frame and the end of the loop
// pops it, either of which would invalidate a reference to it
const basic_json* const s = stack.back().source;
const basic_json* const t = stack.back().target;
const std::size_t path_length = stack.back().path_length;
const std::size_t depth = stack.size();
if (s->is_array())
{
const auto& source_array = *s->m_data.m_value.array;
const auto& target_array = *t->m_data.m_value.array;
// first pass: traverse common elements
const std::size_t i = stack.back().index;
if (i < source_array.size() && i < target_array.size())
{
++stack.back().index;
detail::concat_into(current_path, '/', detail::to_string<string_t>(i));
enter(source_array[i], target_array[i]);
if (stack.size() == depth)
{
current_path.resize(path_length);
}
continue;
}
// We now reached the end of at least one array
// in a second pass, traverse the remaining elements
diff_array_tails(result, *s, *t, current_path, i);
}
else
{
const const_iterator it = stack.back().member;
if (it != s->cend())
{
++stack.back().member;
const std::size_t next_common = stack.back().next_common;
if (next_common < stack.back().common_keys.size() && it.key() == stack.back().common_keys[next_common])
{
++stack.back().next_common;
const basic_json& target_value = (*t)[it.key()];
detail::concat_into(current_path, '/', detail::escape(it.key()));
enter(it.value(), target_value);
if (stack.size() == depth)
{
current_path.resize(path_length);
}
}
else
{
// found a key that is not in target -> remove it
diff_remove(result, detail::concat<string_t>(current_path, '/', detail::escape(it.key())));
}
continue;
}
// append the "add" ops for brand-new keys collected when the
// object was entered
result.insert(result.end(), stack.back().added_ops.begin(), stack.back().added_ops.end());
}
// this array or object is done: continue with the one it is in
stack.pop_back();
if (!stack.empty())
{
current_path.resize(stack.back().path_length);
}
}
}
public:
/// @} /// @}
//////////////////////////////// ////////////////////////////////
+205 -410
View File
@@ -6820,7 +6820,9 @@ void to_json(BasicJsonType& j, const std::optional<T>& opt) noexcept
{ {
if (opt.has_value()) if (opt.has_value())
{ {
j = *opt; // explicit construction, as the conversion from a basic_json with a different
// string type is explicit if JSON_USE_IMPLICIT_CONVERSIONS is 0 (#2649)
j = BasicJsonType(*opt);
} }
else else
{ {
@@ -27488,12 +27490,42 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
assert_invariant(); assert_invariant();
} }
private:
/// whether a basic_json specialization can be converted implicitly into this one;
/// with JSON_USE_IMPLICIT_CONVERSIONS set to 0, this is only the case if both share
/// the same string type (see https://github.com/nlohmann/json/issues/2649)
template<typename BasicJsonType>
using is_implicitly_convertible_basic_json = std::integral_constant < bool,
(JSON_USE_IMPLICIT_CONVERSIONS != 0)
|| std::is_same<typename BasicJsonType::string_t, string_t>::value >;
/// tag to select the constructor that performs the conversion from another basic_json specialization
struct convert_basic_json_tag {};
public:
/// @brief create a JSON value from an existing one /// @brief create a JSON value from an existing one
/// @sa https://json.nlohmann.me/api/basic_json/basic_json/ /// @sa https://json.nlohmann.me/api/basic_json/basic_json/
template < typename BasicJsonType, template < typename BasicJsonType,
detail::enable_if_t < detail::enable_if_t <
detail::is_basic_json<BasicJsonType>::value&& !std::is_same<basic_json, BasicJsonType>::value, int > = 0 > detail::is_basic_json<BasicJsonType>::value&& !std::is_same<basic_json, BasicJsonType>::value
&& is_implicitly_convertible_basic_json<BasicJsonType>::value, int > = 0 >
basic_json(const BasicJsonType& val) basic_json(const BasicJsonType& val)
: basic_json(val, convert_basic_json_tag{})
{}
/// @brief create a JSON value from an existing one
/// @sa https://json.nlohmann.me/api/basic_json/basic_json/
template < typename BasicJsonType,
detail::enable_if_t <
detail::is_basic_json<BasicJsonType>::value&& !std::is_same<basic_json, BasicJsonType>::value
&& !is_implicitly_convertible_basic_json<BasicJsonType>::value, int > = 0 >
explicit basic_json(const BasicJsonType& val)
: basic_json(val, convert_basic_json_tag{})
{}
private:
template<typename BasicJsonType>
basic_json(const BasicJsonType& val, convert_basic_json_tag /*unused*/)
#if JSON_DIAGNOSTIC_POSITIONS #if JSON_DIAGNOSTIC_POSITIONS
: start_position(val.start_pos()), : start_position(val.start_pos()),
end_position(val.end_pos()) end_position(val.end_pos())
@@ -27549,6 +27581,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
assert_invariant(); assert_invariant();
} }
public:
/// @brief create a container (array or object) from an initializer list /// @brief create a container (array or object) from an initializer list
/// @sa https://json.nlohmann.me/api/basic_json/basic_json/ /// @sa https://json.nlohmann.me/api/basic_json/basic_json/
basic_json(initializer_list_t init, basic_json(initializer_list_t init,
@@ -28315,7 +28348,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
int > = 0 > int > = 0 >
BasicJsonType get_impl(detail::priority_tag<2> /*unused*/) const BasicJsonType get_impl(detail::priority_tag<2> /*unused*/) const
{ {
return *this; return BasicJsonType(*this);
} }
/*! /*!
@@ -28454,7 +28487,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
int> = 0> int> = 0>
ValueType & get_to(ValueType& v) const ValueType & get_to(ValueType& v) const
{ {
v = *this; v = ValueType(*this);
return v; return v;
} }
@@ -31944,240 +31977,21 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
{ {
// the patch // the patch
basic_json result(value_t::array); basic_json result(value_t::array);
diff_recursively(result, source, target, path, 0);
return result;
}
private: // if the values are the same, return an empty patch
/// @brief two arrays or two objects @ref diff_iteratively is diffing
struct diff_frame
{
diff_frame(const basic_json* source_, const basic_json* target_, const std::size_t path_length_) noexcept
: source(source_), target(target_), path_length(path_length_)
{}
// declared for GCC's -Weffc++, which asks for them in a class with
// pointer members and a non-trivial destructor; the exception
// specifications are left implicit, as GCC 4.8 rejects explicit ones
// that differ from them
diff_frame(const diff_frame&) = default;
diff_frame(diff_frame&&) = default;
diff_frame& operator=(const diff_frame&) = default;
diff_frame& operator=(diff_frame&&) = default;
~diff_frame() = default;
/// the values being diffed, both arrays or both objects
const basic_json* source;
const basic_json* target;
/// the length of their path in `current_path`
std::size_t path_length;
/// arrays: the next index to diff
std::size_t index = 0;
/// objects: the next member of source to look at
const_iterator member{}; // NOLINT(readability-redundant-member-init)
/// objects: the keys common to both, in source's order
std::vector<typename object_t::key_type> common_keys{}; // NOLINT(readability-redundant-member-init)
/// objects: the next entry of common_keys
std::size_t next_common = 0;
/// objects: the "add" operations for keys only target has
basic_json added_ops{}; // NOLINT(readability-redundant-member-init)
};
// The operations of a diff are built by the functions below rather than
// where they are needed: building one takes several temporaries, and
// unoptimized builds give each temporary a stack slot of its own in the
// function it appears in. In diff_recursively, which is on the call stack
// once per nesting level, that made every level cost kilobytes of stack.
/// @brief append a "replace" operation for @a path with @a value to @a result
static void diff_replace(basic_json& result, const string_t& path, const basic_json& value)
{
result.push_back(
{
{"op", "replace"}, {"path", path}, {"value", value}
});
}
/// @brief append a "remove" operation for @a path to @a result
static void diff_remove(basic_json& result, const string_t& path)
{
result.push_back(object(
{
{"op", "remove"}, {"path", path}
}));
}
/// @brief append an "add" operation for @a path with @a value to @a result
static void diff_add(basic_json& result, const string_t& path, const basic_json& value)
{
result.push_back(
{
{"op", "add"}, {"path", path}, {"value", value}
});
}
/// @brief append the "remove" operations for the elements of array
/// @a source from @a index on, and the "add" operations for the
/// elements of array @a target from source's size on, to @a result
static void diff_array_tails(basic_json& result, const basic_json& source, const basic_json& target,
const string_t& path, const std::size_t index)
{
// remove my remaining elements, highest index first; appending
// in that order avoids the quadratic reinsertion done before
for (std::size_t j = source.size(); j > index; --j)
{
diff_remove(result, detail::concat<string_t>(path, '/', detail::to_string<string_t>(j - 1)));
}
// add other remaining elements
for (std::size_t i = source.size(); i < target.size(); ++i)
{
diff_add(result, detail::concat<string_t>(path, "/-"), target[i]);
}
}
/*!
@brief compare the keys of objects @a source and @a target
If the keys both objects have are in the same order in both, and the keys
only @a target has come after them, stores the keys common to both in
source's order in @a common_keys, stores the "add" operations for the keys
only @a target has in @a added_ops, and returns true: the caller then diffs
the objects member by member. Otherwise, appends operations that remove
every member of @a source and add every member of @a target to @a result,
and returns false.
*/
static bool diff_object_keys(basic_json& result, const basic_json& source, const basic_json& target,
const string_t& path, std::vector<typename object_t::key_type>& common_keys,
basic_json& added_ops)
{
// first pass: record, for every source key, whether it is
// common to both objects (in source's iteration order) or
// was deleted (i.e., in source but not in target) -- this is
// a by-product of the target.find() call already needed to
// tell the two cases apart, so it adds no extra lookups. The
// "remove" ops themselves are emitted later, interleaved
// with the per-key diffs in the caller's fast path, to match
// source's original iteration order (as the original,
// pre-reordering-aware implementation did) instead of
// grouping all removes before all per-key diffs.
std::vector<typename object_t::key_type> common_keys_source_order;
for (auto it = source.cbegin(); it != source.cend(); ++it)
{
if (target.find(it.key()) != target.end())
{
common_keys_source_order.push_back(it.key());
}
}
// second pass: find keys that were added (i.e., in target but
// not in source), and record the keys common to both, in
// target's iteration order -- again a by-product of the
// source.find() call already needed to detect added keys. At
// the same time, determine whether every added key comes
// after every common key in target's order (a precondition
// for the fast path, which only ever appends new keys
// at the very end): for an object_t whose iteration order is
// a pure function of the key set (e.g. the default std::map,
// which always iterates in sorted key order), the order
// check further below is always true and this whole
// mechanism is effectively a no-op; it only matters for a
// reorderable object_t such as the one backing `ordered_json`.
// The patch ops for keys that were added (i.e., in target but not
// in source) are built here so the fast path can reuse
// them without a second source.find() per target key. Only
// used by the fast path -- the slow (reordering) path
// rebuilds "add" ops for every key itself.
std::vector<typename object_t::key_type> common_keys_target_order;
bool new_keys_form_suffix = true;
bool seen_new_key = false;
for (auto it = target.cbegin(); it != target.cend(); ++it)
{
if (source.find(it.key()) == source.end())
{
seen_new_key = true;
diff_add(added_ops, detail::concat<string_t>(path, '/', detail::escape(it.key())), it.value());
}
else
{
common_keys_target_order.push_back(it.key());
if (seen_new_key)
{
new_keys_form_suffix = false;
}
}
}
if (common_keys_source_order == common_keys_target_order && new_keys_form_suffix)
{
// fast path: order of common keys already matches (or the
// object_t's iteration order does not depend on
// insertion history), so a plain per-key diff is correct
// and minimal, as before
common_keys = std::move(common_keys_source_order);
return true;
}
// slow path: the common keys are in a different relative
// order in source and target (only possible for a
// reorderable object_t like ordered_map). Building a
// minimal reordering patch is a nontrivial (LCS-like)
// problem; instead, remove every source key -- both
// deleted keys (which must be removed regardless) and
// common keys (removed so they can be re-added in
// target's order) -- and re-add every key that should
// remain, with its final target value, in target's
// order. basic_json::patch()'s "add" operation on an
// object uses operator[], which appends at the end for a
// vector-backed insertion-ordered map when the key does
// not already exist -- so removing a key and then adding
// it moves it to the end, fixing its position.
for (auto it = source.cbegin(); it != source.cend(); ++it)
{
diff_remove(result, detail::concat<string_t>(path, '/', detail::escape(it.key())));
}
// add every key that is either common (just removed
// above) or brand new, in target's iteration order, so
// that the final order after applying the patch matches
// target exactly
for (auto it = target.cbegin(); it != target.cend(); ++it)
{
diff_add(result, detail::concat<string_t>(path, '/', detail::escape(it.key())), it.value());
}
return false;
}
/*!
@brief @ref diff, for values at nesting level @a depth, appending the
operations to @a result
Diffing two arrays or objects calls this function again, once per nesting
level, so values nested deeply enough used to exhaust the call stack and
terminate the process. The descent is bounded here: once @ref
detail::recursion_depth_limit levels have been entered, @ref
diff_iteratively diffs what is left without the call stack.
*/
static void diff_recursively(basic_json& result, const basic_json& source, const basic_json& target,
const string_t& path, const std::size_t depth)
{
// if the values are the same, there is nothing to do
if (source == target) if (source == target)
{ {
return; return result;
}
if (JSON_HEDLEY_UNLIKELY(depth >= detail::recursion_depth_limit()))
{
diff_iteratively(result, source, target, path);
return;
} }
if (source.type() != target.type()) if (source.type() != target.type())
{ {
// different types: replace value // different types: replace value
diff_replace(result, path, target); result.push_back(
return; {
{"op", "replace"}, {"path", path}, {"value", target}
});
return result;
} }
switch (source.type()) switch (source.type())
@@ -32189,50 +32003,185 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
while (i < source.size() && i < target.size()) while (i < source.size() && i < target.size())
{ {
// recursive call to compare array values at index i // recursive call to compare array values at index i
diff_recursively(result, source[i], target[i], detail::concat<string_t>(path, '/', detail::to_string<string_t>(i)), depth + 1); auto temp_diff = diff(source[i], target[i], detail::concat<string_t>(path, '/', detail::to_string<string_t>(i)));
result.insert(result.end(), temp_diff.begin(), temp_diff.end());
++i; ++i;
} }
// We now reached the end of at least one array // We now reached the end of at least one array
// in a second pass, traverse the remaining elements // in a second pass, traverse the remaining elements
diff_array_tails(result, source, target, path, i);
// remove my remaining elements, highest index first; appending
// in that order avoids the quadratic reinsertion done before
for (std::size_t j = source.size(); j > i; --j)
{
result.push_back(object(
{
{"op", "remove"},
{"path", detail::concat<string_t>(path, '/', detail::to_string<string_t>(j - 1))}
}));
}
i = source.size();
// add other remaining elements
while (i < target.size())
{
result.push_back(
{
{"op", "add"},
{"path", detail::concat<string_t>(path, "/-")},
{"value", target[i]}
});
++i;
}
break; break;
} }
case value_t::object: case value_t::object:
{ {
std::vector<typename object_t::key_type> common_keys; // first pass: record, for every source key, whether it is
basic_json added_ops(value_t::array); // common to both objects (in source's iteration order) or
if (diff_object_keys(result, source, target, path, common_keys, added_ops)) // was deleted (i.e., in source but not in target) -- this is
// a by-product of the target.find() call already needed to
// tell the two cases apart, so it adds no extra lookups. The
// "remove" ops themselves are emitted later, interleaved
// with the recursive per-key diffs in the fast path below,
// to match source's original iteration order (as the
// original, pre-reordering-aware implementation did) instead
// of grouping all removes before all recursive diffs.
std::vector<typename object_t::key_type> common_keys_source_order;
for (auto it = source.cbegin(); it != source.cend(); ++it)
{ {
// fast path: common_keys is, by construction, the if (target.find(it.key()) != target.end())
// subsequence of source's keys that are common to both {
// objects, in source's iteration order -- so it can be common_keys_source_order.push_back(it.key());
// walked in lockstep with `source` using a cheap key }
// comparison instead of another lookup. Deleted keys }
// (those source keys not in common_keys) are interleaved
// here too, in source's original order, to match the // second pass: find keys that were added (i.e., in target but
// historical (pre-reordering-aware) output order. // not in source), and record the keys common to both, in
auto common_it = common_keys.cbegin(); // target's iteration order -- again a by-product of the
// source.find() call already needed to detect added keys. At
// the same time, determine whether every added key comes
// after every common key in target's order (a precondition
// for the fast path below, which only ever appends new keys
// at the very end): for an object_t whose iteration order is
// a pure function of the key set (e.g. the default std::map,
// which always iterates in sorted key order), the order
// check further below is always true and this whole
// mechanism is effectively a no-op; it only matters for a
// reorderable object_t such as the one backing `ordered_json`.
// patch ops for keys that were added (i.e., in target but not
// in source); built here so the fast path below can reuse
// them without a second source.find() per target key. Only
// used by the fast path -- the slow (reordering) path
// rebuilds "add" ops for every key itself.
std::vector<typename object_t::key_type> common_keys_target_order;
basic_json added_ops(value_t::array);
bool new_keys_form_suffix = true;
bool seen_new_key = false;
for (auto it = target.cbegin(); it != target.cend(); ++it)
{
if (source.find(it.key()) == source.end())
{
seen_new_key = true;
const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
added_ops.push_back(
{
{"op", "add"}, {"path", path_key},
{"value", it.value()}
});
}
else
{
common_keys_target_order.push_back(it.key());
if (seen_new_key)
{
new_keys_form_suffix = false;
}
}
}
if (common_keys_source_order == common_keys_target_order && new_keys_form_suffix)
{
// fast path: order of common keys already matches (or the
// object_t's iteration order does not depend on
// insertion history), so a plain per-key recursive diff
// is correct and minimal, as before. common_keys_source_order
// is, by construction, the subsequence of source's keys
// that are common to both objects, in source's iteration
// order -- so it can be walked in lockstep with `source`
// using a cheap key comparison instead of another lookup.
// Deleted keys (those source keys not in common_keys_source_order)
// are interleaved here too, in source's original order, to
// match the historical (pre-reordering-aware) output order.
auto common_it = common_keys_source_order.cbegin();
for (auto it = source.cbegin(); it != source.cend(); ++it) for (auto it = source.cbegin(); it != source.cend(); ++it)
{ {
if (common_it != common_keys.cend() && it.key() == *common_it) if (common_it != common_keys_source_order.cend() && it.key() == *common_it)
{ {
diff_recursively(result, it.value(), target[it.key()], detail::concat<string_t>(path, '/', detail::escape(it.key())), depth + 1); const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
auto temp_diff = diff(it.value(), target[it.key()], path_key);
result.insert(result.end(), temp_diff.begin(), temp_diff.end());
++common_it; ++common_it;
} }
else else
{ {
// found a key that is not in target -> remove it // found a key that is not in target -> remove it
diff_remove(result, detail::concat<string_t>(path, '/', detail::escape(it.key()))); const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
result.push_back(object(
{
{"op", "remove"}, {"path", path_key}
}));
} }
} }
// append the "add" ops for brand-new keys collected by // append the "add" ops for brand-new keys collected above
// diff_object_keys -- no second source.find() per target // during the pass over target -- no second source.find()
// key needed // per target key needed
result.insert(result.end(), added_ops.begin(), added_ops.end()); result.insert(result.end(), added_ops.begin(), added_ops.end());
} }
else
{
// slow path: the common keys are in a different relative
// order in source and target (only possible for a
// reorderable object_t like ordered_map). Building a
// minimal reordering patch is a nontrivial (LCS-like)
// problem; instead, remove every source key -- both
// deleted keys (which must be removed regardless) and
// common keys (removed so they can be re-added in
// target's order) -- and re-add every key that should
// remain, with its final target value, in target's
// order. basic_json::patch()'s "add" operation on an
// object uses operator[], which appends at the end for a
// vector-backed insertion-ordered map when the key does
// not already exist -- so removing a key and then adding
// it moves it to the end, fixing its position.
for (auto it = source.cbegin(); it != source.cend(); ++it)
{
const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
result.push_back(object(
{
{"op", "remove"}, {"path", path_key}
}));
}
// add every key that is either common (just removed
// above) or brand new, in target's iteration order, so
// that the final order after applying the patch matches
// target exactly
for (auto it = target.cbegin(); it != target.cend(); ++it)
{
const auto path_key = detail::concat<string_t>(path, '/', detail::escape(it.key()));
result.push_back(
{
{"op", "add"}, {"path", path_key},
{"value", it.value()}
});
}
}
break; break;
} }
@@ -32247,170 +32196,16 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
default: default:
{ {
// both primitive types: replace value // both primitive types: replace value
diff_replace(result, path, target); result.push_back(
{
{"op", "replace"}, {"path", path}, {"value", target}
});
break; break;
} }
} }
return result;
} }
/*!
@brief @ref diff without the call stack, appending the operations to
@a result
Produces the same operations as @ref diff_recursively. Only reached for
values nested more deeply than @ref detail::recursion_depth_limit.
*/
static void diff_iteratively(basic_json& result, const basic_json& source, const basic_json& target,
const string_t& path)
{
// The arrays and objects being diffed are kept on an explicit stack,
// and every pair of elements is still diffed completely before the
// next one, so the operations come out in the same order as in
// diff_recursively. The path of the values being diffed is kept in
// one buffer that grows and shrinks with the stack, rather than in a
// new string per level.
std::vector<diff_frame> stack;
string_t current_path = path;
// diff `s` against `t`, whose path is current_path: primitives,
// values of different types, and objects whose members were reordered
// are handled right away; arrays and other objects get a frame
const auto enter = [&result, &stack, &current_path](const basic_json & s, const basic_json & t)
{
// if the values are the same, there is nothing to do. Arrays and
// objects are not compared up front: comparing them visits
// everything below them, so doing that at every level would take
// quadratic time in the nesting depth - equal ones yield no
// operations anyway.
if ((!s.is_structured() || !t.is_structured()) && s == t)
{
return;
}
if (s.type() != t.type())
{
// different types: replace value
diff_replace(result, current_path, t);
return;
}
switch (s.type())
{
case value_t::array:
{
stack.emplace_back(&s, &t, current_path.size());
return;
}
case value_t::object:
{
std::vector<typename object_t::key_type> common_keys;
basic_json added_ops(value_t::array);
if (diff_object_keys(result, s, t, current_path, common_keys, added_ops))
{
// fast path: the frame walks source in lockstep with
// common_keys, as diff_recursively does, and appends
// added_ops once all members are done
stack.emplace_back(&s, &t, current_path.size());
stack.back().member = s.cbegin();
stack.back().common_keys = std::move(common_keys);
stack.back().added_ops = std::move(added_ops);
}
return;
}
case value_t::null:
case value_t::string:
case value_t::boolean:
case value_t::number_integer:
case value_t::number_unsigned:
case value_t::number_float:
case value_t::binary:
case value_t::discarded:
default:
{
// both primitive types: replace value
diff_replace(result, current_path, t);
return;
}
}
};
enter(source, target);
while (!stack.empty())
{
// the frame is copied out member by member and changed through
// stack.back(): enter() may push a frame and the end of the loop
// pops it, either of which would invalidate a reference to it
const basic_json* const s = stack.back().source;
const basic_json* const t = stack.back().target;
const std::size_t path_length = stack.back().path_length;
const std::size_t depth = stack.size();
if (s->is_array())
{
const auto& source_array = *s->m_data.m_value.array;
const auto& target_array = *t->m_data.m_value.array;
// first pass: traverse common elements
const std::size_t i = stack.back().index;
if (i < source_array.size() && i < target_array.size())
{
++stack.back().index;
detail::concat_into(current_path, '/', detail::to_string<string_t>(i));
enter(source_array[i], target_array[i]);
if (stack.size() == depth)
{
current_path.resize(path_length);
}
continue;
}
// We now reached the end of at least one array
// in a second pass, traverse the remaining elements
diff_array_tails(result, *s, *t, current_path, i);
}
else
{
const const_iterator it = stack.back().member;
if (it != s->cend())
{
++stack.back().member;
const std::size_t next_common = stack.back().next_common;
if (next_common < stack.back().common_keys.size() && it.key() == stack.back().common_keys[next_common])
{
++stack.back().next_common;
const basic_json& target_value = (*t)[it.key()];
detail::concat_into(current_path, '/', detail::escape(it.key()));
enter(it.value(), target_value);
if (stack.size() == depth)
{
current_path.resize(path_length);
}
}
else
{
// found a key that is not in target -> remove it
diff_remove(result, detail::concat<string_t>(current_path, '/', detail::escape(it.key())));
}
continue;
}
// append the "add" ops for brand-new keys collected when the
// object was entered
result.insert(result.end(), stack.back().added_ops.begin(), stack.back().added_ops.end());
}
// this array or object is done: continue with the one it is in
stack.pop_back();
if (!stack.empty())
{
current_path.resize(stack.back().path_length);
}
}
}
public:
/// @} /// @}
//////////////////////////////// ////////////////////////////////
+36
View File
@@ -13,6 +13,7 @@
#include <cstdint> #include <cstdint>
#include <string> #include <string>
#include <type_traits>
#include <utility> #include <utility>
#include <vector> #include <vector>
@@ -374,4 +375,39 @@ TEST_CASE("alternative string type")
const auto j2 = j.flatten(); const auto j2 = j.flatten();
CHECK(j2.dump() == R"({"/foo/0":"bar","/foo/1":"baz"})"); CHECK(j2.dump() == R"({"/foo/0":"bar","/foo/1":"baz"})");
} }
SECTION("conversion between basic_json specializations (#2649)")
{
// explicit conversions are always possible
CHECK(std::is_constructible<nlohmann::json, alt_json>::value);
CHECK(std::is_constructible<alt_json, nlohmann::json>::value);
CHECK(std::is_constructible<nlohmann::json, nlohmann::ordered_json>::value);
CHECK(std::is_constructible<nlohmann::ordered_json, nlohmann::json>::value);
// specializations with the same string type are implicitly convertible
CHECK(std::is_convertible<nlohmann::ordered_json, nlohmann::json>::value);
CHECK(std::is_convertible<nlohmann::json, nlohmann::ordered_json>::value);
// specializations with different string types are only implicitly convertible
// if implicit conversions are enabled
#if JSON_USE_IMPLICIT_CONVERSIONS
CHECK(std::is_convertible<alt_json, nlohmann::json>::value);
CHECK(std::is_convertible<nlohmann::json, alt_json>::value);
#else
CHECK_FALSE(std::is_convertible<alt_json, nlohmann::json>::value);
CHECK_FALSE(std::is_convertible<nlohmann::json, alt_json>::value);
#endif
// get<BasicJsonType>() works in either case
const nlohmann::json j = {{"foo", 1}, {"bar", true}};
CHECK(j.get<nlohmann::ordered_json>() == nlohmann::ordered_json(j));
// (only a number is converted here, as objects and strings are affected by #3425)
CHECK(nlohmann::json(42).get<alt_json>() == 42);
CHECK(alt_json(nlohmann::json(42)) == 42);
// get_to() also works in either case
alt_json a;
nlohmann::json(42).get_to(a);
CHECK(a == 42);
}
} }
-153
View File
@@ -15,65 +15,8 @@ using nlohmann::json;
#endif #endif
#include <fstream> #include <fstream>
#include <string>
#include <vector>
#include "make_test_data_available.hpp" #include "make_test_data_available.hpp"
namespace
{
// alternating objects and arrays nested `depth` levels deep, with members that
// depend on `variant` at some levels, so diffing two variants yields
// operations on many levels: replacing the innermost value, adding, removing,
// and (for ordered_json) reordering members, and changing array lengths
template<typename BasicJsonType>
BasicJsonType nested(const std::size_t depth, const int variant)
{
BasicJsonType value = variant;
for (std::size_t i = 0; i < depth; ++i)
{
if (i % 2 == 0)
{
BasicJsonType object = BasicJsonType::object();
if ((i + static_cast<std::size_t>(variant)) % 7 == 0)
{
object["x"] = i;
}
if (variant == 2 && i % 11 == 0)
{
object["z"] = "z";
}
object["a"] = std::move(value);
if (variant == 1 && i % 5 == 0)
{
object["y"] = 1;
}
value = std::move(object);
}
else
{
BasicJsonType array = BasicJsonType::array({std::move(value)});
if ((i + static_cast<std::size_t>(variant)) % 3 == 0)
{
array.push_back(i);
}
value = std::move(array);
}
}
return value;
}
// a path of `depth` reference tokens, as nested() nests its values
std::string nested_path(const std::size_t depth)
{
std::string path;
for (std::size_t i = depth; i > 0; --i)
{
path += (i - 1) % 2 == 0 ? "/a" : "/0";
}
return path;
}
} // namespace
TEST_CASE("JSON patch") TEST_CASE("JSON patch")
{ {
SECTION("examples from RFC 6902") SECTION("examples from RFC 6902")
@@ -1809,102 +1752,6 @@ TEST_CASE("JSON patch - diff emits array removals in descending index order")
} }
} }
TEST_CASE("JSON patch: diff of deeply nested values")
{
SECTION("the diff reproduces the target at every depth")
{
// depths on either side of the nesting depth up to which diff()
// recurses (detail::recursion_depth_limit(), 128); not every depth up
// to 300, as the test would then time out under Valgrind
std::vector<std::size_t> depths;
for (std::size_t depth = 0; depth <= 16; ++depth)
{
depths.push_back(depth);
}
for (std::size_t depth = 120; depth <= 136; ++depth)
{
depths.push_back(depth);
}
depths.push_back(300);
for (const auto depth : depths)
{
CAPTURE(depth);
for (int from = 0; from < 3; ++from)
{
for (int to = 0; to < 3; ++to)
{
CAPTURE(from);
CAPTURE(to);
const auto source = nested<json>(depth, from);
const auto target = nested<json>(depth, to);
const auto patch = json::diff(source, target);
CHECK(source.patch(patch) == target);
CHECK(patch.empty() == (from == to));
const auto ordered_source = nested<nlohmann::ordered_json>(depth, from);
const auto ordered_target = nested<nlohmann::ordered_json>(depth, to);
CHECK(ordered_source.patch(nlohmann::ordered_json::diff(ordered_source, ordered_target)) == ordered_target);
}
}
}
}
SECTION("a difference only in the innermost value is one replace operation")
{
for (std::size_t depth = 0; depth <= 300; ++depth)
{
CAPTURE(depth);
json source = 1;
json target = 2;
for (std::size_t i = 0; i < depth; ++i)
{
source = i % 2 == 0 ? json::object({{"a", std::move(source)}}) : json::array({std::move(source)});
target = i % 2 == 0 ? json::object({{"a", std::move(target)}}) : json::array({std::move(target)});
}
CHECK(json::diff(source, target, "/root") == json::array({{{"op", "replace"}, {"path", "/root" + nested_path(depth)}, {"value", 2}}}));
}
}
SECTION("values nested too deeply for the call stack (#5393)")
{
// diff() used to recurse once per nesting level, and compared the
// values with operator== on every level. The values are only
// parsed and diffed, never copied or compared, since those recurse
// too.
const std::size_t depth = 100000;
for (const bool objects :
{
false, true
})
{
CAPTURE(objects);
std::string source_text;
std::string target_text;
std::string equal_text;
std::string path;
for (std::size_t i = 0; i < depth; ++i)
{
source_text += objects ? "{\"a\":" : "[";
path += objects ? "/a" : "/0";
}
target_text = source_text + "2";
equal_text = source_text + "1";
source_text += "1";
const std::string closing(depth, objects ? '}' : ']');
const auto source = json::parse(source_text + closing);
const auto patch = json::diff(source, json::parse(target_text + closing));
REQUIRE(patch.size() == 1);
CHECK(patch[0]["op"] == "replace");
CHECK(patch[0]["path"] == path);
CHECK(patch[0]["value"] == 2);
CHECK(json::diff(source, json::parse(equal_text + closing)).empty());
}
}
}
TEST_CASE("JSON patch - every operation on ordered_json") TEST_CASE("JSON patch - every operation on ordered_json")
{ {
using nlohmann::ordered_json; using nlohmann::ordered_json;