amorfo77/json
1
0
Fork 0
mirror of https://github.com/nlohmann/json.git synced 2026-10-03 05:00:30 +00:00
Code Issues Packages Projects Releases Wiki Activity

Fix stack overflow converting deep values between specializations (#5723)

Browse Source 
* Fix stack overflow converting deep values between specializations

Constructing a basic_json from another specialization (json to
ordered_json or back, also via get<ordered_json>()) converted every
container with its range constructor, which calls the converting
constructor for each element. The call stack therefore grew with every
nesting level, and a value nested some 30,000 levels deep overflowed it.

The conversion now bounds its descent the way the copy constructor does
since #5387: the first 128 levels are converted exactly as before, and
below that convert_iteratively() finishes the value with an explicit
stack. It builds each container bottom-up from its converted elements
with the container's range constructor, so member order and keys that
become equal are handled as before, and it gives a value its type only
once its container exists, so an exception leaves nothing behind that
cannot be destroyed. Parents (JSON_DIAGNOSTICS) and positions
(JSON_DIAGNOSTIC_POSITIONS) are set for every value.

Converting a null value no longer resets its positions: the constructor
assigned null to a value that already was null, which swapped in the
positions of the temporary.

Fixes #5650.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Explain why converting null keeps positions and why next is a reference

Review feedback on #5723 (gregmarr): clarify in comments that the
converting constructor has already copied the positions of val, which
the null case keeps like every other case, and that next must be a
reference into pending so that ++next advances the stored iterator.

Comments only; no code change.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Refer to recursion_depth_limit() in the convert_structured() docs

The comment still named nesting_depth_limit, which #5637 removed on
develop in favor of detail::recursion_depth_limit().

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Advance the pending iterator through pending.back() and shorten the null comment

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

---------

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
Niels Lohmann
2026-09-30 21:36:04 +02:00
committed by GitHub
parent 3b6ae43c53
commit 7d7055ec50
6 changed files with 735 additions and 94 deletions
Download Patch File Download Diff File Expand all files Collapse all files
single_include/nlohmann/json.hpp
+227 -47
View File
@@ -27850,11 +27850,11 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
/*!
@brief how many levels the operation going on in this thread has descended into
Copying a value and comparing two values share this count. The library never
nests one inside the other - copying a value does not compare one, and
comparing two values does not copy them - and where user code nests them
anyway, sharing the count only ends a descent sooner than it had to, which
costs a little speed and is never wrong.
Copying a value, converting one from another specialization, and comparing
two values share this count. The library never nests one of them inside
another - none of them does either of the other two on the way - and where
user code nests them anyway, sharing the count only ends a descent sooner
than it had to, which costs a little speed and is never wrong.
A byte is enough: the count never exceeds the limit by more than the single
level that notices the limit has been reached.
@@ -28211,6 +28211,222 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
copy_iteratively(src);
}
/*!
@brief convert the value @a val of another specialization into this null
value; @a val must be neither an object nor an array
Converting such a value never descends, so both ways of converting an
object or an array (@ref convert_structured) leave their elements of this
kind to the converting constructor, which leaves them to this.
*/
template<typename BasicJsonType>
void convert_leaf(const BasicJsonType& val)
{
using other_boolean_t = typename BasicJsonType::boolean_t;
using other_number_float_t = typename BasicJsonType::number_float_t;
using other_number_integer_t = typename BasicJsonType::number_integer_t;
using other_number_unsigned_t = typename BasicJsonType::number_unsigned_t;
using other_string_t = typename BasicJsonType::string_t;
using other_binary_t = typename BasicJsonType::binary_t;
switch (val.type())
{
case value_t::boolean:
JSONSerializer<other_boolean_t>::to_json(*this, val.template get<other_boolean_t>());
break;
case value_t::number_float:
JSONSerializer<other_number_float_t>::to_json(*this, val.template get<other_number_float_t>());
break;
case value_t::number_integer:
JSONSerializer<other_number_integer_t>::to_json(*this, val.template get<other_number_integer_t>());
break;
case value_t::number_unsigned:
JSONSerializer<other_number_unsigned_t>::to_json(*this, val.template get<other_number_unsigned_t>());
break;
case value_t::string:
JSONSerializer<other_string_t>::to_json(*this, val.template get_ref<const other_string_t&>());
break;
case value_t::binary:
JSONSerializer<other_binary_t>::to_json(*this, val.template get_ref<const other_binary_t&>());
break;
case value_t::null:
// m_data.m_type is already value_t::null
break;
case value_t::discarded:
m_data.m_type = value_t::discarded;
break;
case value_t::object: // LCOV_EXCL_LINE
case value_t::array: // LCOV_EXCL_LINE
default: // LCOV_EXCL_LINE
JSON_ASSERT(false); // NOLINT(cert-dcl03-c,hicpp-static-assert,misc-static-assert) LCOV_EXCL_LINE
}
}
/// scratch space for the converted elements of the arrays that
/// @ref convert_iteratively has yet to create
using convert_scratch_t = std::vector<basic_json, AllocatorType<basic_json>>;
/*!
@brief create the object or array @a val converted into this null value
Its converted elements are the last `val.size()` entries of @a elements (an
array) or of @a members (an object); they are moved into the container in
one go and then removed.
*/
template<typename BasicJsonType>
void convert_level(const BasicJsonType& val, convert_scratch_t& elements, copy_scratch_t& members)
{
if (val.is_object())
{
const auto first = members.end() - static_cast<typename copy_scratch_t::difference_type>(val.size());
m_data.m_value.object = create<object_t>(std::make_move_iterator(first),
std::make_move_iterator(members.end()));
// only now that the object exists may this stop being a null value
m_data.m_type = value_t::object;
members.erase(first, members.end());
}
else
{
const auto first = elements.end() - static_cast<typename convert_scratch_t::difference_type>(val.size());
m_data.m_value.array = create<array_t>(std::make_move_iterator(first),
std::make_move_iterator(elements.end()));
// only now that the array exists may this stop being a null value
m_data.m_type = value_t::array;
elements.erase(first, elements.end());
}
set_parents();
}
/*!
@brief convert the object or array @a val of another specialization into
this null value without recursing
The containers whose conversion has begun are kept on an explicit stack
rather than on the call stack. Unlike @ref copy_iteratively, this builds
every container from the bottom up: all its elements are converted first,
and the container is then created from them in one go, the way the range
constructor that converts the levels above the bound does. The two object
types need not enumerate their members in the same order, so the members
could not be paired up by position anyway, and building from a range keeps
what the range constructor does with keys that become equal on conversion.
Every value is complete before it is handed on, and a container gets its
type only once it exists, so whatever throws, every value left behind can
be destroyed.
*/
template<typename BasicJsonType>
void convert_iteratively(const BasicJsonType& val)
{
using other_const_iterator = typename BasicJsonType::const_iterator;
// the containers whose conversion has begun, innermost last, each with
// its element to convert next
std::vector<std::pair<const BasicJsonType*, other_const_iterator>> pending;
// the converted elements of the pending arrays and the converted
// members of the pending objects, those of the innermost one last
convert_scratch_t elements;
copy_scratch_t members;
pending.emplace_back(&val, val.cbegin());
for (;;)
{
const BasicJsonType& container = *pending.back().first;
// a copy, as descending below can reallocate pending; the
// iterator kept in pending is only advanced through pending.back()
const other_const_iterator next = pending.back().second;
if (next != container.cend())
{
if (next->is_structured())
{
// convert its elements first; next stays where it is until
// the converted container is handed back to this one
pending.emplace_back(&*next, next->cbegin());
continue;
}
// the converting constructor does not descend into this value
if (container.is_object())
{
members.emplace_back(next.key(), *next);
}
else
{
elements.emplace_back(*next);
}
++pending.back().second;
continue;
}
// all elements of the container are converted: create it
pending.pop_back();
if (pending.empty())
{
convert_level(container, elements, members);
return;
}
basic_json converted;
converted.convert_level(container, elements, members);
#if JSON_DIAGNOSTIC_POSITIONS
converted.start_position = container.start_pos();
converted.end_position = container.end_pos();
#endif
// hand it to the container it is an element of
if (pending.back().first->is_object())
{
members.emplace_back(pending.back().second.key(), std::move(converted));
}
else
{
elements.push_back(std::move(converted));
}
++pending.back().second;
}
}
/*!
@brief convert the object or array @a val of another specialization into
this null value
Converting a container converts its elements, so a value nested deeply
enough used to exhaust the call stack. The descent is bounded here as in
@ref copy_structured: the first `detail::recursion_depth_limit()` levels
are converted by the containers' range constructors, just as they always were,
and anything below that is converted without the call stack by
@ref convert_iteratively.
@sa https://github.com/nlohmann/json/issues/5650
*/
template<typename BasicJsonType>
void convert_structured(const BasicJsonType& val)
{
const nesting_depth_guard guard;
if (JSON_HEDLEY_LIKELY(guard.okay()))
{
// every element comes back to the converting constructor
if (val.is_object())
{
using other_object_t = typename BasicJsonType::object_t;
JSONSerializer<other_object_t>::to_json(*this, val.template get_ref<const other_object_t&>());
}
else
{
using other_array_t = typename BasicJsonType::array_t;
JSONSerializer<other_array_t>::to_json(*this, val.template get_ref<const other_array_t&>());
}
return;
}
convert_iteratively(val);
}
/// the result of comparing two values, including values that cannot be
/// ordered at all, such as a discarded value or a NaN
@@ -28566,49 +28782,13 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
end_position(val.end_pos())
#endif
{
using other_boolean_t = typename BasicJsonType::boolean_t;
using other_number_float_t = typename BasicJsonType::number_float_t;
using other_number_integer_t = typename BasicJsonType::number_integer_t;
using other_number_unsigned_t = typename BasicJsonType::number_unsigned_t;
using other_string_t = typename BasicJsonType::string_t;
using other_object_t = typename BasicJsonType::object_t;
using other_array_t = typename BasicJsonType::array_t;
using other_binary_t = typename BasicJsonType::binary_t;
switch (val.type())
if (val.is_structured())
{
case value_t::boolean:
JSONSerializer<other_boolean_t>::to_json(*this, val.template get<other_boolean_t>());
break;
case value_t::number_float:
JSONSerializer<other_number_float_t>::to_json(*this, val.template get<other_number_float_t>());
break;
case value_t::number_integer:
JSONSerializer<other_number_integer_t>::to_json(*this, val.template get<other_number_integer_t>());
break;
case value_t::number_unsigned:
JSONSerializer<other_number_unsigned_t>::to_json(*this, val.template get<other_number_unsigned_t>());
break;
case value_t::string:
JSONSerializer<other_string_t>::to_json(*this, val.template get_ref<const other_string_t&>());
break;
case value_t::object:
JSONSerializer<other_object_t>::to_json(*this, val.template get_ref<const other_object_t&>());
break;
case value_t::array:
JSONSerializer<other_array_t>::to_json(*this, val.template get_ref<const other_array_t&>());
break;
case value_t::binary:
JSONSerializer<other_binary_t>::to_json(*this, val.template get_ref<const other_binary_t&>());
break;
case value_t::null:
*this = nullptr;
break;
case value_t::discarded:
m_data.m_type = value_t::discarded;
break;
default: // LCOV_EXCL_LINE
JSON_ASSERT(false); // NOLINT(cert-dcl03-c,hicpp-static-assert,misc-static-assert) LCOV_EXCL_LINE
convert_structured(val);
}
else
{
convert_leaf(val);
}
JSON_ASSERT(m_data.m_type == val.type());