Fix rollback of pointer operator[] for unusual object and string types

Follow-up to the strong exception guarantee of the non-const
operator[](json_pointer):

- get_unchecked_create assumed that emplace() always inserts the key that
  the walk did not find. For object types whose lookup and emplace()
  disagree (e.g. a copied fifo_map), nothing was inserted, the rollback
  erased the existing member, and the walk continued with a value of the
  wrong type (assertion, type_error.305, or a crash). It now checks the
  result of emplace(), arms the rollback only if a key was inserted, and
  otherwise continues the walk with the member found, as basic_json's
  operator[] does: such pointers throw out_of_range.404 and leave the value
  unchanged, as before the strong guarantee.
- The rollback erased the inserted key by key, which converts string_t to
  object_t::key_type and may allocate inside a noexcept function
  (std::terminate on bad_alloc with custom key types). It now erases by the
  iterator emplace() returned, and shrinks a grown array with erase() at
  the end instead of resize(). The documentation states the remaining
  requirement for custom template parameters.
- Creating values no longer calls string_t::begin(), so the non-const
  operator[](json_pointer) compiles with string types that lack it (as the
  alternative string type of the tests).
- Correct the @throw lists of get_unchecked and get_unchecked_create.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This commit is contained in:
Niels Lohmann committed 2026-10-26 15:59:03 +01:00
1 parent 447df43650
commit 0157e9c035
5 files changed
+325 -73

No files matched your search

@@ -94,7 +94,9 @@ Strong exception safety: if an exception occurs, the original value stays intact
If the **non-const** version throws, the value is left unchanged: values it created for earlier reference tokens
of `ptr` are removed again. This also holds for exceptions such as `#!cpp std::bad_alloc` or
`#!cpp std::length_error` thrown while padding an array up to a very large index.
`#!cpp std::length_error` thrown while padding an array up to a very large index. With custom
[template parameters](../../features/types/template_parameters.md), this requires that `erase()` of the object and
array types and the move operations of the custom base class do not throw.
## Complexity
+144 -36
View File
@@ -8,7 +8,7 @@
#pragma once
#include <algorithm> // all_of, copy
#include <algorithm> // copy
#include <array> // array
#include <cctype> // isdigit
#include <cerrno> // errno, ERANGE
@@ -20,6 +20,7 @@
#ifndef JSON_NO_IO
#include <iosfwd> // ostream
#endif // JSON_NO_IO
#include <iterator> // next
#include <limits> // max
#include <string> // string
#include <utility> // move
@@ -732,15 +733,32 @@ class json_pointer
removes everything created after it, so the value is left as it was
(strong exception safety). The guard needs no try/catch and therefore also
compiles with JSON_NOEXCEPTION.
Reverting the change must not throw, as it happens in a destructor. It
erases the inserted member by its iterator (no key is converted or
compared, nothing is allocated), erases the added elements at the end of
the array, or swaps the null value back. This holds for std::map,
std::vector, ordered_map, and every object_t, array_t, and
json_base_class_t whose erase() and move operations do not throw.
Between recording and reverting, nothing is inserted into or erased from
the object or array, so the iterator stays valid.
*/
template<typename BasicJsonType>
class get_unchecked_rollback
{
public:
/// record the change that resolving the reference token @a token makes to @a value
get_unchecked_rollback(BasicJsonType* value, const string_t* token)
using object_iterator = typename BasicJsonType::object_t::iterator;
/// record that @a member was inserted into the object @a value
get_unchecked_rollback(BasicJsonType* value, const object_iterator& member)
: m_value(value)
, m_member(member)
, m_change(change::key_inserted)
{}
/// record the change that the next reference token makes to the null value or array @a value
explicit get_unchecked_rollback(BasicJsonType* value)
: m_value(value)
, m_key(token)
{
if (value->is_null())
{
@@ -750,10 +768,6 @@ class json_pointer
m_saved_null.swap(*value);
m_change = change::null_converted;
}
else if (value->is_object())
{
m_change = change::key_inserted;
}
else
{
JSON_ASSERT(value->is_array());
@@ -791,11 +805,14 @@ class json_pointer
m_value->swap(m_saved_null);
break;
case change::key_inserted:
m_value->m_data.m_value.object->erase(*m_key);
m_value->m_data.m_value.object->erase(m_member);
break;
case change::array_grown:
m_value->m_data.m_value.array->resize(m_old_size);
{
auto& elements = *m_value->m_data.m_value.array;
elements.erase(std::next(elements.begin(), static_cast<typename BasicJsonType::difference_type>(m_old_size)), elements.end());
break;
}
case change::none:
default:
break;
@@ -811,7 +828,7 @@ class json_pointer
};
BasicJsonType* m_value;
const string_t* m_key;
object_iterator m_member {};
typename BasicJsonType::size_type m_old_size = 0;
change m_change = change::none;
BasicJsonType m_saved_null {};
@@ -1206,9 +1223,13 @@ class json_pointer
array is strongly exception safe by itself; any other change is made by
@ref get_unchecked_create, which undoes it if a later step throws.
@throw parse_error.106 if an array index begins with '0'
@throw parse_error.109 if an array index was not a number
@throw out_of_range.404 if the JSON pointer can not be resolved
@throw parse_error.106 if an array index of an existing array begins with '0'
@throw parse_error.109 if an array index of an existing array was not a number
@throw out_of_range.404 if the JSON pointer can not be resolved, or if an
array index is too large for strtoull
@throw out_of_range.410 if an array index exceeds size_type
@throw std::length_error if padding an array exceeds its maximal size
@throw std::bad_alloc if an allocation fails
*/
template<typename BasicJsonType>
BasicJsonType& get_unchecked(BasicJsonType* ptr) const
@@ -1299,24 +1320,104 @@ class json_pointer
@ref get_unchecked_rollback); all later tokens only change values created
here.
If the emplace() of the object type finds a member that the lookup in @ref
get_unchecked did not find (the two disagree, e.g. for fifo_map), nothing
is inserted, and the walk continues with that member as in @ref
get_unchecked, which is what basic_json::operator[] does.
@param[in] ptr the value the reference token @a first is applied to
@param[in] first the first reference token to resolve (an element of
@ref reference_tokens)
@param[in] end the end of @ref reference_tokens
@throw parse_error.106 if an array index begins with '0'
@throw parse_error.109 if an array index was not a number
@throw parse_error.106 if an array index of an existing array begins with
'0' (only after emplace() found a member, see above)
@throw parse_error.109 if an array index of an existing array was not a
number (only after emplace() found a member)
@throw out_of_range.404 if an array index is too large for strtoull, or (only
after emplace() found a member) if the JSON pointer
can not be resolved
@throw out_of_range.410 if an array index exceeds size_type
@throw std::length_error if padding an array exceeds its maximal size
@throw std::bad_alloc if an allocation fails
*/
template<typename BasicJsonType>
JSON_HEDLEY_NEVER_INLINE static BasicJsonType& get_unchecked_create(BasicJsonType* ptr, const string_t* first, const string_t* end)
{
get_unchecked_rollback<BasicJsonType> rollback(ptr, first);
BasicJsonType& result = create_values(ptr, first, end);
const auto* token = first;
while (ptr->is_object())
{
const auto old_capacity = ptr->container_capacity();
const auto inserted = ptr->m_data.m_value.object->emplace(*token, nullptr);
if (JSON_HEDLEY_LIKELY(inserted.second))
{
get_unchecked_rollback<BasicJsonType> rollback(ptr, inserted.first);
BasicJsonType& result = create_values(&ptr->set_parent_after_insert(inserted.first->second, old_capacity), token + 1, end);
rollback.commit();
return result;
}
// nothing was inserted: walk existing values from the member
// emplace() found on, like get_unchecked
ptr = &inserted.first->second;
while (++token != end)
{
if (ptr->is_object())
{
auto& members = *ptr->m_data.m_value.object;
const auto it = object_search(members, *token);
if (!object_search_found(members, it, *token))
{
break;
}
ptr = &it->second;
}
else if (ptr->is_array())
{
if (is_dash(*token))
{
break;
}
const auto idx = array_index<BasicJsonType>(*token);
if (idx >= ptr->m_data.m_value.array->size())
{
break;
}
ptr = &(*ptr->m_data.m_value.array)[idx];
}
else if (ptr->is_null())
{
break;
}
else
{
// primitive value with a reference token left
throw_unresolved(*token, ptr);
}
}
if (token == end)
{
return *ptr;
}
}
get_unchecked_rollback<BasicJsonType> rollback(ptr);
BasicJsonType& result = create_values(ptr, token, end);
rollback.commit();
return result;
}
/// the loop of @ref get_unchecked_create
/*!
@brief the loop of @ref get_unchecked_create
@a ptr is a null value or an array. Every value created here is a fresh
one, so every object a key is added to is empty.
@throw out_of_range.404 if an array index is too large for strtoull
@throw out_of_range.410 if an array index exceeds size_type
@throw std::length_error if padding an array exceeds its maximal size
@throw std::bad_alloc if an allocation fails
*/
template<typename BasicJsonType>
static BasicJsonType& create_values(BasicJsonType* ptr, const string_t* first, const string_t* end)
{
@@ -1327,23 +1428,11 @@ class json_pointer
// convert null values to arrays or objects before continuing
if (ptr->is_null())
{
// check if the reference token is a valid array index, that is
// a nonempty sequence of digits without a leading '0'
// (cf. RFC 6901, Sect. 4); tokens that could never be a valid
// array index (such as "01" or "") are treated as object keys
const bool nums = !reference_token.empty()
&& (reference_token.size() == 1 || reference_token[0] != '0')
&& std::all_of(reference_token.begin(), reference_token.end(),
[](const unsigned char x)
{
return std::isdigit(x);
});
// change value to an array for array indices or "-" or to an
// object otherwise; every null value here is a fresh one (see
// get_unchecked_rollback), so converting it in place leaves
// the same value as assigning an array or object to it
if (nums || is_dash(reference_token))
if (is_array_index_token(reference_token) || is_dash(reference_token))
{
ptr->template convert_null_to<typename BasicJsonType::array_t>();
}
@@ -1355,14 +1444,13 @@ class json_pointer
if (ptr->is_object())
{
// the key is missing: the object is new, or get_unchecked did
// not find the key
// the object is new and empty
ptr = &object_add(*ptr, reference_token);
}
else
{
// only the first value is not created here, and it is no
// primitive value (see get_unchecked)
// only the first value is not created here, and it is a null
// value or an array (see get_unchecked_create)
JSON_ASSERT(ptr->is_array());
if (is_dash(reference_token))
{
@@ -1380,6 +1468,26 @@ class json_pointer
return *ptr;
}
/// whether @a s is a valid array index, that is a nonempty sequence of
/// digits without a leading '0' (cf. RFC 6901, Sect. 4); tokens that could
/// never be a valid array index (such as "01" or "") are object keys
static bool is_array_index_token(const string_t& s)
{
const std::size_t size = s.size();
if (size == 0 || (size > 1 && s[0] == '0'))
{
return false;
}
for (std::size_t i = 0; i < size; ++i)
{
if (std::isdigit(static_cast<unsigned char>(s[i])) == 0)
{
return false;
}
}
return true;
}
/*!
@throw parse_error.106 if an array index begins with '0'
@throw parse_error.109 if an array index was not a number
+144 -36
View File
@@ -23585,7 +23585,7 @@ NLOHMANN_JSON_NAMESPACE_END
#include <algorithm> // all_of, copy
#include <algorithm> // copy
#include <array> // array
#include <cctype> // isdigit
#include <cerrno> // errno, ERANGE
@@ -23597,6 +23597,7 @@ NLOHMANN_JSON_NAMESPACE_END
#ifndef JSON_NO_IO
#include <iosfwd> // ostream
#endif // JSON_NO_IO
#include <iterator> // next
#include <limits> // max
#include <string> // string
#include <utility> // move
@@ -24318,15 +24319,32 @@ class json_pointer
removes everything created after it, so the value is left as it was
(strong exception safety). The guard needs no try/catch and therefore also
compiles with JSON_NOEXCEPTION.
Reverting the change must not throw, as it happens in a destructor. It
erases the inserted member by its iterator (no key is converted or
compared, nothing is allocated), erases the added elements at the end of
the array, or swaps the null value back. This holds for std::map,
std::vector, ordered_map, and every object_t, array_t, and
json_base_class_t whose erase() and move operations do not throw.
Between recording and reverting, nothing is inserted into or erased from
the object or array, so the iterator stays valid.
*/
template<typename BasicJsonType>
class get_unchecked_rollback
{
public:
/// record the change that resolving the reference token @a token makes to @a value
get_unchecked_rollback(BasicJsonType* value, const string_t* token)
using object_iterator = typename BasicJsonType::object_t::iterator;
/// record that @a member was inserted into the object @a value
get_unchecked_rollback(BasicJsonType* value, const object_iterator& member)
: m_value(value)
, m_member(member)
, m_change(change::key_inserted)
{}
/// record the change that the next reference token makes to the null value or array @a value
explicit get_unchecked_rollback(BasicJsonType* value)
: m_value(value)
, m_key(token)
{
if (value->is_null())
{
@@ -24336,10 +24354,6 @@ class json_pointer
m_saved_null.swap(*value);
m_change = change::null_converted;
}
else if (value->is_object())
{
m_change = change::key_inserted;
}
else
{
JSON_ASSERT(value->is_array());
@@ -24377,11 +24391,14 @@ class json_pointer
m_value->swap(m_saved_null);
break;
case change::key_inserted:
m_value->m_data.m_value.object->erase(*m_key);
m_value->m_data.m_value.object->erase(m_member);
break;
case change::array_grown:
m_value->m_data.m_value.array->resize(m_old_size);
{
auto& elements = *m_value->m_data.m_value.array;
elements.erase(std::next(elements.begin(), static_cast<typename BasicJsonType::difference_type>(m_old_size)), elements.end());
break;
}
case change::none:
default:
break;
@@ -24397,7 +24414,7 @@ class json_pointer
};
BasicJsonType* m_value;
const string_t* m_key;
object_iterator m_member {};
typename BasicJsonType::size_type m_old_size = 0;
change m_change = change::none;
BasicJsonType m_saved_null {};
@@ -24792,9 +24809,13 @@ class json_pointer
array is strongly exception safe by itself; any other change is made by
@ref get_unchecked_create, which undoes it if a later step throws.
@throw parse_error.106 if an array index begins with '0'
@throw parse_error.109 if an array index was not a number
@throw out_of_range.404 if the JSON pointer can not be resolved
@throw parse_error.106 if an array index of an existing array begins with '0'
@throw parse_error.109 if an array index of an existing array was not a number
@throw out_of_range.404 if the JSON pointer can not be resolved, or if an
array index is too large for strtoull
@throw out_of_range.410 if an array index exceeds size_type
@throw std::length_error if padding an array exceeds its maximal size
@throw std::bad_alloc if an allocation fails
*/
template<typename BasicJsonType>
BasicJsonType& get_unchecked(BasicJsonType* ptr) const
@@ -24885,24 +24906,104 @@ class json_pointer
@ref get_unchecked_rollback); all later tokens only change values created
here.
If the emplace() of the object type finds a member that the lookup in @ref
get_unchecked did not find (the two disagree, e.g. for fifo_map), nothing
is inserted, and the walk continues with that member as in @ref
get_unchecked, which is what basic_json::operator[] does.
@param[in] ptr the value the reference token @a first is applied to
@param[in] first the first reference token to resolve (an element of
@ref reference_tokens)
@param[in] end the end of @ref reference_tokens
@throw parse_error.106 if an array index begins with '0'
@throw parse_error.109 if an array index was not a number
@throw parse_error.106 if an array index of an existing array begins with
'0' (only after emplace() found a member, see above)
@throw parse_error.109 if an array index of an existing array was not a
number (only after emplace() found a member)
@throw out_of_range.404 if an array index is too large for strtoull, or (only
after emplace() found a member) if the JSON pointer
can not be resolved
@throw out_of_range.410 if an array index exceeds size_type
@throw std::length_error if padding an array exceeds its maximal size
@throw std::bad_alloc if an allocation fails
*/
template<typename BasicJsonType>
JSON_HEDLEY_NEVER_INLINE static BasicJsonType& get_unchecked_create(BasicJsonType* ptr, const string_t* first, const string_t* end)
{
get_unchecked_rollback<BasicJsonType> rollback(ptr, first);
BasicJsonType& result = create_values(ptr, first, end);
const auto* token = first;
while (ptr->is_object())
{
const auto old_capacity = ptr->container_capacity();
const auto inserted = ptr->m_data.m_value.object->emplace(*token, nullptr);
if (JSON_HEDLEY_LIKELY(inserted.second))
{
get_unchecked_rollback<BasicJsonType> rollback(ptr, inserted.first);
BasicJsonType& result = create_values(&ptr->set_parent_after_insert(inserted.first->second, old_capacity), token + 1, end);
rollback.commit();
return result;
}
// nothing was inserted: walk existing values from the member
// emplace() found on, like get_unchecked
ptr = &inserted.first->second;
while (++token != end)
{
if (ptr->is_object())
{
auto& members = *ptr->m_data.m_value.object;
const auto it = object_search(members, *token);
if (!object_search_found(members, it, *token))
{
break;
}
ptr = &it->second;
}
else if (ptr->is_array())
{
if (is_dash(*token))
{
break;
}
const auto idx = array_index<BasicJsonType>(*token);
if (idx >= ptr->m_data.m_value.array->size())
{
break;
}
ptr = &(*ptr->m_data.m_value.array)[idx];
}
else if (ptr->is_null())
{
break;
}
else
{
// primitive value with a reference token left
throw_unresolved(*token, ptr);
}
}
if (token == end)
{
return *ptr;
}
}
get_unchecked_rollback<BasicJsonType> rollback(ptr);
BasicJsonType& result = create_values(ptr, token, end);
rollback.commit();
return result;
}
/// the loop of @ref get_unchecked_create
/*!
@brief the loop of @ref get_unchecked_create
@a ptr is a null value or an array. Every value created here is a fresh
one, so every object a key is added to is empty.
@throw out_of_range.404 if an array index is too large for strtoull
@throw out_of_range.410 if an array index exceeds size_type
@throw std::length_error if padding an array exceeds its maximal size
@throw std::bad_alloc if an allocation fails
*/
template<typename BasicJsonType>
static BasicJsonType& create_values(BasicJsonType* ptr, const string_t* first, const string_t* end)
{
@@ -24913,23 +25014,11 @@ class json_pointer
// convert null values to arrays or objects before continuing
if (ptr->is_null())
{
// check if the reference token is a valid array index, that is
// a nonempty sequence of digits without a leading '0'
// (cf. RFC 6901, Sect. 4); tokens that could never be a valid
// array index (such as "01" or "") are treated as object keys
const bool nums = !reference_token.empty()
&& (reference_token.size() == 1 || reference_token[0] != '0')
&& std::all_of(reference_token.begin(), reference_token.end(),
[](const unsigned char x)
{
return std::isdigit(x);
});
// change value to an array for array indices or "-" or to an
// object otherwise; every null value here is a fresh one (see
// get_unchecked_rollback), so converting it in place leaves
// the same value as assigning an array or object to it
if (nums || is_dash(reference_token))
if (is_array_index_token(reference_token) || is_dash(reference_token))
{
ptr->template convert_null_to<typename BasicJsonType::array_t>();
}
@@ -24941,14 +25030,13 @@ class json_pointer
if (ptr->is_object())
{
// the key is missing: the object is new, or get_unchecked did
// not find the key
// the object is new and empty
ptr = &object_add(*ptr, reference_token);
}
else
{
// only the first value is not created here, and it is no
// primitive value (see get_unchecked)
// only the first value is not created here, and it is a null
// value or an array (see get_unchecked_create)
JSON_ASSERT(ptr->is_array());
if (is_dash(reference_token))
{
@@ -24966,6 +25054,26 @@ class json_pointer
return *ptr;
}
/// whether @a s is a valid array index, that is a nonempty sequence of
/// digits without a leading '0' (cf. RFC 6901, Sect. 4); tokens that could
/// never be a valid array index (such as "01" or "") are object keys
static bool is_array_index_token(const string_t& s)
{
const std::size_t size = s.size();
if (size == 0 || (size > 1 && s[0] == '0'))
{
return false;
}
for (std::size_t i = 0; i < size; ++i)
{
if (std::isdigit(static_cast<unsigned char>(s[i])) == 0)
{
return false;
}
}
return true;
}
/*!
@throw parse_error.106 if an array index begins with '0'
@throw parse_error.109 if an array index was not a number
+17
View File
@@ -373,6 +373,23 @@ TEST_CASE("alternative string type")
CHECK_FALSE(j.contains(alt_json::json_pointer("/foo/bar")));
}
SECTION("non-const operator[](json_pointer)")
{
// creating values must compile without string_t::begin()
alt_json j = alt_json::parse(R"({"a":{"b":[1]},"c":null})");
const alt_json before = j;
j[alt_json::json_pointer("/a/b/3/x/-/y")] = 1; // pad, append, create
j[alt_json::json_pointer("/a/z")] = 2; // insert a key
j[alt_json::json_pointer("/c/0/q")] = 3; // convert null
j[alt_json::json_pointer("/a/b/-")] = 4; // append
CHECK(j.dump() == R"({"a":{"b":[1,null,null,{"x":[{"y":1}]},4],"z":2},"c":[{"q":3}]})");
// a failing access leaves the value unchanged
alt_json k = before;
CHECK_THROWS_AS(k[alt_json::json_pointer("/a/n/99999999999999999999999")], alt_json::out_of_range&);
CHECK(k == before);
}
SECTION("operator/(std::size_t)")
{
// json_pointer::operator/=(std::size_t) must compile without string_t
+17
View File
@@ -1499,6 +1499,23 @@ TEST_CASE("regression tests 1")
CHECK(order.dump() == R"({"a":2,"m":{"y":1,"b":2},"new_key":3})");
}
SECTION("operator[](json_pointer) when emplace() finds a key the lookup does not")
{
// a copied fifo_map restarts its timestamps, so a new key "k" gets
// the timestamp of "x": the lookup does not find "k", but emplace()
// finds "x" and inserts nothing; the pointer then continues with the
// member of "x", as basic_json::operator[] does
const my_json orig = my_json::parse(R"({"x":1})");
const std::array<const char*, 3> pointers = {{"/k/0", "/k/a/b", "/k/-"}};
for (const char* pointer : pointers)
{
CAPTURE(pointer)
my_json j = orig;
CHECK_THROWS_AS(j[my_json::json_pointer(pointer)], my_json::out_of_range&);
CHECK(j.dump() == R"({"x":1})");
}
}
SECTION("issue #977 - Assigning between different json types")
{
foo_json lj = ns::foo{3};