Merge branch 'develop' into claude/copy-without-assignable-base-5674

Conflicts:
- include/nlohmann/json.hpp: copy_array_level() keeps develop's #5721
  "set the array type only once the container exists" and then builds the
  elements with the PR's copy_construct_tag emplace_back loop
- single_include/nlohmann/json.hpp: regenerated with make amalgamate

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

docs/mkdocs/docs/api/basic_json/operator[].md

@@ -89,9 +89,6 @@ Strong exception safety: if an exception occurs, the original value stays intact
     - Throws [`out_of_range.410`](../../home/exceptions.md#jsonexceptionout_of_range410) if an array index in the passed
       JSON pointer `ptr` exceeds the range of `size_type` (e.g., on 32-bit platforms).
 
-    For the **const** version, an object key or array index in `ptr` that does not exist is not reported by an
-    exception, but is undefined behavior (see the notes below). Use [`at`](at.md) for checked access.
-
 ## Complexity
 
 1. Constant if `idx` is in the range of the array. Otherwise, linear in `idx - size()`.
@@ -106,12 +103,9 @@ Strong exception safety: if an exception occurs, the original value stays intact
     The following cases apply to the **const** overloads; the non-const overloads instead insert the missing element
     (see the notes below).
 
-    1. If the element at index `idx` does not exist, the behavior is undefined and is **guarded by a
-       [runtime assertion](../../features/assertions.md)**!
+    1. If the element at index `idx` does not exist, the behavior is undefined.
     2. If the element with key `key` does not exist, the behavior is undefined and is **guarded by a
        [runtime assertion](../../features/assertions.md)**!
-    3. If the JSON pointer `ptr` refers to an object key or an array index that does not exist, the behavior is
-       undefined and is **guarded by a [runtime assertion](../../features/assertions.md)**!
 
 1. The non-const version may add values: If `idx` is beyond the range of the array (i.e., `idx >= size()`), then the
    array is silently filled up with `#!json null` values to make `idx` a valid reference to the last stored element. In
@@ -267,11 +261,9 @@ Strong exception safety: if an exception occurs, the original value stays intact
 ## Version history
 
 1. Added in version 1.0.0. Fixed in version 3.13.0 to throw `#!cpp std::length_error` instead of emptying the array and
-   accessing it out of bounds when `idx` equals the maximum value of `size_type`. A missing index in the const version
-   is guarded by a runtime assertion since version 3.13.0.
+   accessing it out of bounds when `idx` equals the maximum value of `size_type`.
 2. Added in version 1.0.0. Added overloads for `T* key` in version 1.1.0. Removed overloads for `T* key` (replaced by 3)
    in version 3.11.0.
 3. Added in version 3.11.0. Fixed in version 3.13.0 to consistently accept `std::string_view`-convertible keys, as
    already supported by [`at`](at.md), [`value`](value.md), [`find`](find.md), and other lookup functions.
-4. Added in version 2.0.0. A missing array index in the const version is guarded by a runtime assertion since
-   version 3.13.0.
+4. Added in version 2.0.0.

docs/mkdocs/docs/features/assertions.md

@@ -16,15 +16,14 @@ before including the `json.hpp` header.
 
 ## Function with runtime assertions
 
-### Unchecked access to a const value
+### Unchecked object access to a const value
 
-Function [`operator[]`](../api/basic_json/operator%5B%5D.md) implements unchecked access for arrays and objects. Whereas
-a missing element is added in the case of non-const values, accessing a const value with a missing object key or an
-invalid array index is undefined behavior (think of a dereferenced null pointer) and yields a runtime assertion. This
-also applies to a [JSON pointer](json_pointer.md) that refers to a missing key or an invalid index.
+Function [`operator[]`](../api/basic_json/operator%5B%5D.md) implements unchecked access for objects. Whereas a missing
+key is added in the case of non-const objects, accessing a const object with a missing key is undefined behavior (think
+of a dereferenced null pointer) and yields a runtime assertion.
 
-If you are not sure whether an element exists, use checked access with the [`at` function](../api/basic_json/at.md)
-or call the [`contains` function](../api/basic_json/contains.md) before.
+If you are not sure whether an element in an object exists, use checked access with the
+[`at` function](../api/basic_json/at.md) or call the [`contains` function](../api/basic_json/contains.md) before.
 
 See also the documentation on [element access](element_access/index.md).
 
@@ -47,30 +46,7 @@ See also the documentation on [element access](element_access/index.md).
     Output:
 
     ```
-    Assertion failed: (it != m_data.m_value.object->end()), function operator[], file json.hpp, line 28795.
-    ```
-
-??? example "Example 2: Invalid array index in a JSON pointer"
-
-    The following code will trigger an assertion at runtime:
-
-    ```cpp
-    #include <nlohmann/json.hpp>
-    
-    using json = nlohmann::json;
-    using namespace nlohmann::literals;
-    
-    int main()
-    {
-        const json j = {{"array", {1, 2, 3}}};
-        auto v = j["/array/5"_json_pointer];
-    }
-    ```
-
-    Output:
-
-    ```
-    Assertion failed: (idx < m_data.m_value.array->size()), function operator[], file json.hpp, line 28758.
+    Assertion failed: (m_value.object->find(key) != m_value.object->end()), function operator[], file json.hpp, line 2144.
     ```
 
 ### Constructing from an uninitialized iterator range

include/nlohmann/detail/json_pointer.hpp

@@ -536,10 +536,6 @@ class json_pointer
     @return const reference to the JSON value pointed to by the JSON
     pointer
 
-    @pre Every object key and array index the pointer refers to exists.
-         Like the const operator[] for keys and indices, a missing one is
-         undefined behavior, guarded by a runtime assertion.
-
     @throw parse_error.106   if an array index begins with '0'
     @throw parse_error.109   if an array index was not a number
     @throw out_of_range.402  if the array index '-' is used
@@ -554,8 +550,7 @@ class json_pointer
             {
                 case detail::value_t::object:
                 {
-                    // use unchecked object access; the const operator[]
-                    // asserts that the key exists
+                    // use unchecked object access
                     ptr = &ptr->operator[](reference_token);
                     break;
                 }
@@ -568,8 +563,7 @@ class json_pointer
                         JSON_THROW(detail::out_of_range::create(402, detail::concat("array index '-' (", std::to_string(ptr->m_data.m_value.array->size()), ") is out of range"), ptr));
                     }
 
-                    // use unchecked array access; the const operator[]
-                    // asserts that the index exists
+                    // use unchecked array access
                     ptr = &ptr->operator[](array_index<BasicJsonType>(reference_token));
                     break;
                 }

include/nlohmann/json.hpp

@@ -1004,19 +1004,39 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
     using copy_scratch_value_t = std::pair<typename object_t::key_type, basic_json>;
     using copy_scratch_t = std::vector<copy_scratch_value_t, AllocatorType<copy_scratch_value_t>>;
 
-    /// @brief copy everything of @a src into @a dst but its type and value
-    static void copy_metadata(const basic_json& src, basic_json& dst)
-    {
-        // a custom base class is only required to be copy-constructible and
-        // move-assignable, so the copy has to go through a temporary
-        static_cast<json_base_class_t&>(dst) = json_base_class_t(static_cast<const json_base_class_t&>(src));
+    /// @brief tag selecting the constructor below; used only to build the
+    /// elements of a deep copy (@ref copy_array_level, @ref copy_object_level)
+    struct copy_construct_tag {};
 
+  public:
+    /*!
+    @brief construct a null value whose base class - and, with @ref
+           JSON_DIAGNOSTIC_POSITIONS, positions - are copied from @a src
+
+    Copy-constructing @ref json_base_class_t here, rather than default-
+    constructing the element and assigning its base class afterwards, means
+    that copying a @ref basic_json only ever requires a copy-constructible
+    base class, and never a move-assignable one as well.
+
+    @note this constructor has to be public: @ref copy_array_level and
+          @ref copy_object_level reach it through @ref array_t's or @ref
+          object_t's own emplace_back(), which constructs the element from
+          outside @ref basic_json and so cannot call a private constructor.
+          @ref copy_construct_tag is private, though, and nothing in the
+          public interface hands out a value of it, so outside code can still
+          never name it to call this constructor itself.
+    */
+    basic_json(copy_construct_tag /*unused*/, const basic_json& src)
+        : json_base_class_t(src)
 #if JSON_DIAGNOSTIC_POSITIONS
-        dst.start_position = src.start_position;
-        dst.end_position = src.end_position;
+        , start_position(src.start_position)
+        , end_position(src.end_position)
 #endif
+    {
     }
 
+  private:
+
     /*!
     @brief copy the value of @a src into @a dst, which must not be structured
 
@@ -1079,8 +1099,11 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
     }
 
     /*!
-    @brief copy everything of @a src into the null value @a dst but the children
+    @brief finish the copy @a dst of @a src that a @ref copy_construct_tag
+           constructor started, other than the children of an object or array
 
+    @a dst already has @a src's base class and, with @ref
+    JSON_DIAGNOSTIC_POSITIONS, positions; only its value is still missing.
     Objects and arrays are not copied here; they are appended to @a worklist to
     be created later by @ref copy_iteratively. Until that happens, @a dst remains
     a null value, so that a partially built copy can be destroyed at any point
@@ -1088,8 +1111,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
     */
     static void copy_shallow(const basic_json& src, basic_json& dst, copy_worklist_t& worklist)
     {
-        copy_metadata(src, dst);
-
         if (src.m_data.m_type == value_t::object || src.m_data.m_type == value_t::array)
         {
             // defer: dst stays a null value until its container exists
@@ -1110,15 +1131,19 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
     {
         const array_t& src_array = *src.m_data.m_value.array;
 
-        // create all elements up front: growing the array afterwards could
-        // invalidate the pointers that are handed to the worklist; resize()
-        // rather than the fill constructor, because not every array type
-        // provides the latter (e.g., ones without a matching allocator-aware
-        // fill constructor)
         dst.m_data.m_value.array = create<array_t>();
         // only now that the array exists may dst stop being a null value
         dst.m_data.m_type = value_t::array;
-        dst.m_data.m_value.array->resize(src_array.size());
+
+        // create every element - its base class already copy-constructed from
+        // its counterpart in src, via the copy_construct_tag constructor -
+        // before any of their addresses are handed to worklist below: growing
+        // the array while that is going on could reallocate it and invalidate
+        // addresses taken from an earlier iteration
+        for (const auto& src_element : src_array)
+        {
+            dst.m_data.m_value.array->emplace_back(copy_construct_tag{}, src_element);
+        }
 
         auto dst_it = dst.m_data.m_value.array->begin();
         for (auto src_it = src_array.cbegin(); src_it != src_array.cend(); ++src_it, ++dst_it)
@@ -1136,12 +1161,14 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
 
         // build the complete key skeleton and hand it to the object's range
         // constructor: adding the keys one by one would be quadratic for object
-        // types that are backed by a vector, such as nlohmann::ordered_map
+        // types that are backed by a vector, such as nlohmann::ordered_map; each
+        // value's base class is already copy-constructed from its counterpart
+        // in src, via the copy_construct_tag constructor
         scratch.clear();
         scratch.reserve(src_object.size());
         for (const auto& element : src_object)
         {
-            scratch.emplace_back(element.first, basic_json());
+            scratch.emplace_back(element.first, basic_json(copy_construct_tag{}, element.second));
         }
 
         dst.m_data.m_value.object = create<object_t>(std::make_move_iterator(scratch.begin()),
@@ -2874,7 +2901,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
         // const operator[] only works for arrays
         if (JSON_HEDLEY_LIKELY(is_array()))
         {
-            JSON_ASSERT(idx < m_data.m_value.array->size());
             return m_data.m_value.array->operator[](idx);
         }
 

single_include/nlohmann/json.hpp

@@ -20148,10 +20148,6 @@ class json_pointer
     @return const reference to the JSON value pointed to by the JSON
     pointer
 
-    @pre Every object key and array index the pointer refers to exists.
-         Like the const operator[] for keys and indices, a missing one is
-         undefined behavior, guarded by a runtime assertion.
-
     @throw parse_error.106   if an array index begins with '0'
     @throw parse_error.109   if an array index was not a number
     @throw out_of_range.402  if the array index '-' is used
@@ -20166,8 +20162,7 @@ class json_pointer
             {
                 case detail::value_t::object:
                 {
-                    // use unchecked object access; the const operator[]
-                    // asserts that the key exists
+                    // use unchecked object access
                     ptr = &ptr->operator[](reference_token);
                     break;
                 }
@@ -20180,8 +20175,7 @@ class json_pointer
                         JSON_THROW(detail::out_of_range::create(402, detail::concat("array index '-' (", std::to_string(ptr->m_data.m_value.array->size()), ") is out of range"), ptr));
                     }
 
-                    // use unchecked array access; the const operator[]
-                    // asserts that the index exists
+                    // use unchecked array access
                     ptr = &ptr->operator[](array_index<BasicJsonType>(reference_token));
                     break;
                 }
@@ -27937,19 +27931,39 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
     using copy_scratch_value_t = std::pair<typename object_t::key_type, basic_json>;
     using copy_scratch_t = std::vector<copy_scratch_value_t, AllocatorType<copy_scratch_value_t>>;
 
-    /// @brief copy everything of @a src into @a dst but its type and value
-    static void copy_metadata(const basic_json& src, basic_json& dst)
-    {
-        // a custom base class is only required to be copy-constructible and
-        // move-assignable, so the copy has to go through a temporary
-        static_cast<json_base_class_t&>(dst) = json_base_class_t(static_cast<const json_base_class_t&>(src));
+    /// @brief tag selecting the constructor below; used only to build the
+    /// elements of a deep copy (@ref copy_array_level, @ref copy_object_level)
+    struct copy_construct_tag {};
 
+  public:
+    /*!
+    @brief construct a null value whose base class - and, with @ref
+           JSON_DIAGNOSTIC_POSITIONS, positions - are copied from @a src
+
+    Copy-constructing @ref json_base_class_t here, rather than default-
+    constructing the element and assigning its base class afterwards, means
+    that copying a @ref basic_json only ever requires a copy-constructible
+    base class, and never a move-assignable one as well.
+
+    @note this constructor has to be public: @ref copy_array_level and
+          @ref copy_object_level reach it through @ref array_t's or @ref
+          object_t's own emplace_back(), which constructs the element from
+          outside @ref basic_json and so cannot call a private constructor.
+          @ref copy_construct_tag is private, though, and nothing in the
+          public interface hands out a value of it, so outside code can still
+          never name it to call this constructor itself.
+    */
+    basic_json(copy_construct_tag /*unused*/, const basic_json& src)
+        : json_base_class_t(src)
 #if JSON_DIAGNOSTIC_POSITIONS
-        dst.start_position = src.start_position;
-        dst.end_position = src.end_position;
+        , start_position(src.start_position)
+        , end_position(src.end_position)
 #endif
+    {
     }
 
+  private:
+
     /*!
     @brief copy the value of @a src into @a dst, which must not be structured
 
@@ -28012,8 +28026,11 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
     }
 
     /*!
-    @brief copy everything of @a src into the null value @a dst but the children
+    @brief finish the copy @a dst of @a src that a @ref copy_construct_tag
+           constructor started, other than the children of an object or array
 
+    @a dst already has @a src's base class and, with @ref
+    JSON_DIAGNOSTIC_POSITIONS, positions; only its value is still missing.
     Objects and arrays are not copied here; they are appended to @a worklist to
     be created later by @ref copy_iteratively. Until that happens, @a dst remains
     a null value, so that a partially built copy can be destroyed at any point
@@ -28021,8 +28038,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
     */
     static void copy_shallow(const basic_json& src, basic_json& dst, copy_worklist_t& worklist)
     {
-        copy_metadata(src, dst);
-
         if (src.m_data.m_type == value_t::object || src.m_data.m_type == value_t::array)
         {
             // defer: dst stays a null value until its container exists
@@ -28043,15 +28058,19 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
     {
         const array_t& src_array = *src.m_data.m_value.array;
 
-        // create all elements up front: growing the array afterwards could
-        // invalidate the pointers that are handed to the worklist; resize()
-        // rather than the fill constructor, because not every array type
-        // provides the latter (e.g., ones without a matching allocator-aware
-        // fill constructor)
         dst.m_data.m_value.array = create<array_t>();
         // only now that the array exists may dst stop being a null value
         dst.m_data.m_type = value_t::array;
-        dst.m_data.m_value.array->resize(src_array.size());
+
+        // create every element - its base class already copy-constructed from
+        // its counterpart in src, via the copy_construct_tag constructor -
+        // before any of their addresses are handed to worklist below: growing
+        // the array while that is going on could reallocate it and invalidate
+        // addresses taken from an earlier iteration
+        for (const auto& src_element : src_array)
+        {
+            dst.m_data.m_value.array->emplace_back(copy_construct_tag{}, src_element);
+        }
 
         auto dst_it = dst.m_data.m_value.array->begin();
         for (auto src_it = src_array.cbegin(); src_it != src_array.cend(); ++src_it, ++dst_it)
@@ -28069,12 +28088,14 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
 
         // build the complete key skeleton and hand it to the object's range
         // constructor: adding the keys one by one would be quadratic for object
-        // types that are backed by a vector, such as nlohmann::ordered_map
+        // types that are backed by a vector, such as nlohmann::ordered_map; each
+        // value's base class is already copy-constructed from its counterpart
+        // in src, via the copy_construct_tag constructor
         scratch.clear();
         scratch.reserve(src_object.size());
         for (const auto& element : src_object)
         {
-            scratch.emplace_back(element.first, basic_json());
+            scratch.emplace_back(element.first, basic_json(copy_construct_tag{}, element.second));
         }
 
         dst.m_data.m_value.object = create<object_t>(std::make_move_iterator(scratch.begin()),
@@ -29807,7 +29828,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
         // const operator[] only works for arrays
         if (JSON_HEDLEY_LIKELY(is_array()))
         {
-            JSON_ASSERT(idx < m_data.m_value.array->size());
             return m_data.m_value.array->operator[](idx);
         }
 

tests/src/unit-custom-base-class.cpp

@@ -405,3 +405,73 @@ TEST_CASE("JSON Visit Node")
         );
     CHECK(expected.empty());
 }
+
+// A custom base class with a const member: copy-constructible (initializing a
+// const member works fine), but not copy-/move-assignable (assigning one does
+// not). Used to check that copy construction never requires more than that.
+struct const_member_base
+{
+    const int id = 7; // NOLINT(misc-non-private-member-variables-in-classes)
+};
+
+using json_with_const_base = nlohmann::basic_json <
+                             std::map,
+                             std::vector,
+                             std::string,
+                             bool,
+                             std::int64_t,
+                             std::uint64_t,
+                             double,
+                             std::allocator,
+                             nlohmann::adl_serializer,
+                             std::vector<std::uint8_t>,
+                             const_member_base
+                             >;
+
+// build an array nested @a depth levels deep, with the innermost value 1;
+// every level is constructed (never assigned), since const_member_base does
+// not support assignment
+static json_with_const_base make_nested_array(std::size_t depth)
+{
+    if (depth == 0)
+    {
+        return json_with_const_base(1);
+    }
+    return json_with_const_base::array({make_nested_array(depth - 1)});
+}
+
+TEST_CASE("Regression test for issue #5674 - copy construction must not require an assignable base class")
+{
+    SECTION("depth 0")
+    {
+        // as in the original bug report: copy construction only, no assignment
+        const json_with_const_base j = {1, 2};
+        const json_with_const_base copy = j; // NOLINT(performance-unnecessary-copy-initialization)
+
+        CHECK(copy.size() == 2);
+        CHECK(copy.id == 7);
+    }
+
+    SECTION("nested deeper than the copy constructor's descent bound")
+    {
+        // beyond nesting_depth_limit() (128) levels, the copy constructor
+        // copies without the call stack (copy_iteratively / copy_array_level),
+        // which used to assign the base class of every element it created
+        const std::size_t depth = 300;
+
+        const json_with_const_base j = make_nested_array(depth);
+        const json_with_const_base copy = j; // NOLINT(performance-unnecessary-copy-initialization)
+
+        const json_with_const_base* c = &copy;
+        for (std::size_t level = 0; level <= depth; ++level)
+        {
+            CAPTURE(level)
+            REQUIRE(c->id == 7);
+            if (level < depth)
+            {
+                c = &c->at(0);
+            }
+        }
+        CHECK(*c == 1);
+    }
+}