Merge branch 'develop' into claude/const-json-pointer-assertion

Conflicts:
- docs/mkdocs/docs/api/basic_json/operator[].md: version history item 1 keeps develop's std::length_error fix note and appends the PR's runtime assertion note

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

docs/mkdocs/docs/api/basic_json/operator[].md

@@ -89,6 +89,9 @@ Strong exception safety: if an exception occurs, the original value stays intact
     - Throws [`out_of_range.410`](../../home/exceptions.md#jsonexceptionout_of_range410) if an array index in the passed
       JSON pointer `ptr` exceeds the range of `size_type` (e.g., on 32-bit platforms).
 
+    For the **const** version, an object key or array index in `ptr` that does not exist is not reported by an
+    exception, but is undefined behavior (see the notes below). Use [`at`](at.md) for checked access.
+
 ## Complexity
 
 1. Constant if `idx` is in the range of the array. Otherwise, linear in `idx - size()`.
@@ -103,9 +106,12 @@ Strong exception safety: if an exception occurs, the original value stays intact
     The following cases apply to the **const** overloads; the non-const overloads instead insert the missing element
     (see the notes below).
 
-    1. If the element at index `idx` does not exist, the behavior is undefined.
+    1. If the element at index `idx` does not exist, the behavior is undefined and is **guarded by a
+       [runtime assertion](../../features/assertions.md)**!
     2. If the element with key `key` does not exist, the behavior is undefined and is **guarded by a
        [runtime assertion](../../features/assertions.md)**!
+    3. If the JSON pointer `ptr` refers to an object key or an array index that does not exist, the behavior is
+       undefined and is **guarded by a [runtime assertion](../../features/assertions.md)**!
 
 1. The non-const version may add values: If `idx` is beyond the range of the array (i.e., `idx >= size()`), then the
    array is silently filled up with `#!json null` values to make `idx` a valid reference to the last stored element. In
@@ -261,9 +267,11 @@ Strong exception safety: if an exception occurs, the original value stays intact
 ## Version history
 
 1. Added in version 1.0.0. Fixed in version 3.13.0 to throw `#!cpp std::length_error` instead of emptying the array and
-   accessing it out of bounds when `idx` equals the maximum value of `size_type`.
+   accessing it out of bounds when `idx` equals the maximum value of `size_type`. A missing index in the const version
+   is guarded by a runtime assertion since version 3.13.0.
 2. Added in version 1.0.0. Added overloads for `T* key` in version 1.1.0. Removed overloads for `T* key` (replaced by 3)
    in version 3.11.0.
 3. Added in version 3.11.0. Fixed in version 3.13.0 to consistently accept `std::string_view`-convertible keys, as
    already supported by [`at`](at.md), [`value`](value.md), [`find`](find.md), and other lookup functions.
-4. Added in version 2.0.0.
+4. Added in version 2.0.0. A missing array index in the const version is guarded by a runtime assertion since
+   version 3.13.0.

docs/mkdocs/docs/features/assertions.md

@@ -16,14 +16,15 @@ before including the `json.hpp` header.
 
 ## Function with runtime assertions
 
-### Unchecked object access to a const value
+### Unchecked access to a const value
 
-Function [`operator[]`](../api/basic_json/operator%5B%5D.md) implements unchecked access for objects. Whereas a missing
+Function [`operator[]`](../api/basic_json/operator%5B%5D.md) implements unchecked access for arrays and objects. Whereas
-key is added in the case of non-const objects, accessing a const object with a missing key is undefined behavior (think
+a missing element is added in the case of non-const values, accessing a const value with a missing object key or an
-of a dereferenced null pointer) and yields a runtime assertion.
+invalid array index is undefined behavior (think of a dereferenced null pointer) and yields a runtime assertion. This
+also applies to a [JSON pointer](json_pointer.md) that refers to a missing key or an invalid index.
 
-If you are not sure whether an element in an object exists, use checked access with the
+If you are not sure whether an element exists, use checked access with the [`at` function](../api/basic_json/at.md)
-[`at` function](../api/basic_json/at.md) or call the [`contains` function](../api/basic_json/contains.md) before.
+or call the [`contains` function](../api/basic_json/contains.md) before.
 
 See also the documentation on [element access](element_access/index.md).
 
@@ -46,7 +47,30 @@ See also the documentation on [element access](element_access/index.md).
     Output:
 
     ```
-    Assertion failed: (m_value.object->find(key) != m_value.object->end()), function operator[], file json.hpp, line 2144.
+    Assertion failed: (it != m_data.m_value.object->end()), function operator[], file json.hpp, line 28795.
+    ```
+
+??? example "Example 2: Invalid array index in a JSON pointer"
+
+    The following code will trigger an assertion at runtime:
+
+    ```cpp
+    #include <nlohmann/json.hpp>
+    
+    using json = nlohmann::json;
+    using namespace nlohmann::literals;
+    
+    int main()
+    {
+        const json j = {{"array", {1, 2, 3}}};
+        auto v = j["/array/5"_json_pointer];
+    }
+    ```
+
+    Output:
+
+    ```
+    Assertion failed: (idx < m_data.m_value.array->size()), function operator[], file json.hpp, line 28758.
     ```
 
 ### Constructing from an uninitialized iterator range

include/nlohmann/detail/json_pointer.hpp

@@ -536,6 +536,10 @@ class json_pointer
     @return const reference to the JSON value pointed to by the JSON
     pointer
 
+    @pre Every object key and array index the pointer refers to exists.
+         Like the const operator[] for keys and indices, a missing one is
+         undefined behavior, guarded by a runtime assertion.
+
     @throw parse_error.106   if an array index begins with '0'
     @throw parse_error.109   if an array index was not a number
     @throw out_of_range.402  if the array index '-' is used
@@ -550,7 +554,8 @@ class json_pointer
             {
                 case detail::value_t::object:
                 {
-                    // use unchecked object access
+                    // use unchecked object access; the const operator[]
+                    // asserts that the key exists
                     ptr = &ptr->operator[](reference_token);
                     break;
                 }
@@ -563,7 +568,8 @@ class json_pointer
                         JSON_THROW(detail::out_of_range::create(402, detail::concat("array index '-' (", std::to_string(ptr->m_data.m_value.array->size()), ") is out of range"), ptr));
                     }
 
-                    // use unchecked array access
+                    // use unchecked array access; the const operator[]
+                    // asserts that the index exists
                     ptr = &ptr->operator[](array_index<BasicJsonType>(reference_token));
                     break;
                 }

include/nlohmann/detail/meta/type_traits.hpp

@@ -189,37 +189,6 @@ struct actual_object_comparator
 template<typename BasicJsonType>
 using actual_object_comparator_t = typename actual_object_comparator<BasicJsonType>::type;
 
-template<typename T>
-using detect_key_comp = decltype(std::declval<const T&>().key_comp());
-
-// whether ObjectType can be constructed from a pair of Iterator together with
-// a copy of its own comparator, the way std::map can: it needs a nested
-// key_compare, a const key_comp() convertible to it, and a matching
-// (Iterator, Iterator, const key_compare&) constructor.
-//
-// used to preserve a stateful comparator when a copy is built from a range
-// past the iterative deep copy's nesting bound (see copy_object_level); an
-// object type that does not satisfy this, such as nlohmann::ordered_map
-// (which has key_compare for its std::map-like interface, but no key_comp()),
-// keeps default-constructing its comparator, just as it always has
-template<typename ObjectType, typename Iterator, typename = void>
-struct is_comparator_constructible_object_type_impl : std::false_type {};
-
-template<typename ObjectType, typename Iterator>
-struct is_comparator_constructible_object_type_impl <
-    ObjectType, Iterator, enable_if_t<is_detected<detect_key_compare, ObjectType>::value >>
-{
-    using key_compare = typename ObjectType::key_compare;
-
-    static constexpr bool value =
-        is_detected_convertible<key_compare, detect_key_comp, ObjectType>::value &&
-        std::is_constructible<ObjectType, Iterator, Iterator, const key_compare&>::value;
-};
-
-template<typename ObjectType, typename Iterator>
-struct is_comparator_constructible_object_type
-    : is_comparator_constructible_object_type_impl<ObjectType, Iterator> {};
-
 /////////////////
 // char_traits //
 /////////////////

include/nlohmann/json.hpp

@@ -1127,27 +1127,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
         }
     }
 
-    /// @brief create the object type from a range, preserving @a src_object's
-    /// comparator when the object type supports it
-    /// Enabled for object types that provide a key_comp() and a matching
-    /// range-plus-comparator constructor, such as std::map. Other object
-    /// types, such as nlohmann::ordered_map, fall back to the plain range
-    /// constructor and default-construct their comparator, just as they
-    /// always have (@ref detail::is_comparator_constructible_object_type).
-    template<typename Iterator, detail::enable_if_t<
-                 detail::is_comparator_constructible_object_type<object_t, Iterator>::value, int> = 0>
-    static object_t* create_object_with_comparator(const object_t& src_object, Iterator first, Iterator last)
-    {
-        return create<object_t>(first, last, src_object.key_comp());
-    }
-
-    template<typename Iterator, detail::enable_if_t<
-                 detail::negation<detail::is_comparator_constructible_object_type<object_t, Iterator>>::value, int> = 0>
-    static object_t* create_object_with_comparator(const object_t& /*src_object*/, Iterator first, Iterator last)
-    {
-        return create<object_t>(first, last);
-    }
-
     /// @brief create the copy of the object @a src in @a dst
     /// @note structured values are appended to @a worklist instead
     static void copy_object_level(const basic_json& src, basic_json& dst,
@@ -1165,8 +1144,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
             scratch.emplace_back(element.first, basic_json());
         }
 
-        dst.m_data.m_value.object = create_object_with_comparator(src_object,
+        dst.m_data.m_value.object = create<object_t>(std::make_move_iterator(scratch.begin()),
-                                    std::make_move_iterator(scratch.begin()),
                                     std::make_move_iterator(scratch.end()));
         // only now that the object exists may dst stop being a null value
         dst.m_data.m_type = value_t::object;
@@ -2896,6 +2874,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
         // const operator[] only works for arrays
         if (JSON_HEDLEY_LIKELY(is_array()))
         {
+            JSON_ASSERT(idx < m_data.m_value.array->size());
             return m_data.m_value.array->operator[](idx);
         }
 

single_include/nlohmann/json.hpp

@@ -4197,37 +4197,6 @@ struct actual_object_comparator
 template<typename BasicJsonType>
 using actual_object_comparator_t = typename actual_object_comparator<BasicJsonType>::type;
 
-template<typename T>
-using detect_key_comp = decltype(std::declval<const T&>().key_comp());
-
-// whether ObjectType can be constructed from a pair of Iterator together with
-// a copy of its own comparator, the way std::map can: it needs a nested
-// key_compare, a const key_comp() convertible to it, and a matching
-// (Iterator, Iterator, const key_compare&) constructor.
-//
-// used to preserve a stateful comparator when a copy is built from a range
-// past the iterative deep copy's nesting bound (see copy_object_level); an
-// object type that does not satisfy this, such as nlohmann::ordered_map
-// (which has key_compare for its std::map-like interface, but no key_comp()),
-// keeps default-constructing its comparator, just as it always has
-template<typename ObjectType, typename Iterator, typename = void>
-struct is_comparator_constructible_object_type_impl : std::false_type {};
-
-template<typename ObjectType, typename Iterator>
-struct is_comparator_constructible_object_type_impl <
-    ObjectType, Iterator, enable_if_t<is_detected<detect_key_compare, ObjectType>::value >>
-{
-    using key_compare = typename ObjectType::key_compare;
-
-    static constexpr bool value =
-        is_detected_convertible<key_compare, detect_key_comp, ObjectType>::value &&
-        std::is_constructible<ObjectType, Iterator, Iterator, const key_compare&>::value;
-};
-
-template<typename ObjectType, typename Iterator>
-struct is_comparator_constructible_object_type
-    : is_comparator_constructible_object_type_impl<ObjectType, Iterator> {};
-
 /////////////////
 // char_traits //
 /////////////////
@@ -20179,6 +20148,10 @@ class json_pointer
     @return const reference to the JSON value pointed to by the JSON
     pointer
 
+    @pre Every object key and array index the pointer refers to exists.
+         Like the const operator[] for keys and indices, a missing one is
+         undefined behavior, guarded by a runtime assertion.
+
     @throw parse_error.106   if an array index begins with '0'
     @throw parse_error.109   if an array index was not a number
     @throw out_of_range.402  if the array index '-' is used
@@ -20193,7 +20166,8 @@ class json_pointer
             {
                 case detail::value_t::object:
                 {
-                    // use unchecked object access
+                    // use unchecked object access; the const operator[]
+                    // asserts that the key exists
                     ptr = &ptr->operator[](reference_token);
                     break;
                 }
@@ -20206,7 +20180,8 @@ class json_pointer
                         JSON_THROW(detail::out_of_range::create(402, detail::concat("array index '-' (", std::to_string(ptr->m_data.m_value.array->size()), ") is out of range"), ptr));
                     }
 
-                    // use unchecked array access
+                    // use unchecked array access; the const operator[]
+                    // asserts that the index exists
                     ptr = &ptr->operator[](array_index<BasicJsonType>(reference_token));
                     break;
                 }
@@ -28085,27 +28060,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
         }
     }
 
-    /// @brief create the object type from a range, preserving @a src_object's
-    /// comparator when the object type supports it
-    /// Enabled for object types that provide a key_comp() and a matching
-    /// range-plus-comparator constructor, such as std::map. Other object
-    /// types, such as nlohmann::ordered_map, fall back to the plain range
-    /// constructor and default-construct their comparator, just as they
-    /// always have (@ref detail::is_comparator_constructible_object_type).
-    template<typename Iterator, detail::enable_if_t<
-                 detail::is_comparator_constructible_object_type<object_t, Iterator>::value, int> = 0>
-    static object_t* create_object_with_comparator(const object_t& src_object, Iterator first, Iterator last)
-    {
-        return create<object_t>(first, last, src_object.key_comp());
-    }
-
-    template<typename Iterator, detail::enable_if_t<
-                 detail::negation<detail::is_comparator_constructible_object_type<object_t, Iterator>>::value, int> = 0>
-    static object_t* create_object_with_comparator(const object_t& /*src_object*/, Iterator first, Iterator last)
-    {
-        return create<object_t>(first, last);
-    }
-
     /// @brief create the copy of the object @a src in @a dst
     /// @note structured values are appended to @a worklist instead
     static void copy_object_level(const basic_json& src, basic_json& dst,
@@ -28123,8 +28077,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
             scratch.emplace_back(element.first, basic_json());
         }
 
-        dst.m_data.m_value.object = create_object_with_comparator(src_object,
+        dst.m_data.m_value.object = create<object_t>(std::make_move_iterator(scratch.begin()),
-                                    std::make_move_iterator(scratch.begin()),
                                     std::make_move_iterator(scratch.end()));
         // only now that the object exists may dst stop being a null value
         dst.m_data.m_type = value_t::object;
@@ -29854,6 +29807,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
         // const operator[] only works for arrays
         if (JSON_HEDLEY_LIKELY(is_array()))
         {
+            JSON_ASSERT(idx < m_data.m_value.array->size());
             return m_data.m_value.array->operator[](idx);
         }
 

tests/src/unit-comparison.cpp

@@ -827,46 +827,6 @@ Json nest(Json j, const std::size_t depth)
     return j;
 }
 
-// a std::map comparator with state: case-insensitive, unless constructed
-// case-sensitive. Used to check that copying an object copies the original's
-// comparator rather than default-constructing a new one (see #5649).
-struct key_case_less
-{
-    key_case_less() = default;
-    explicit key_case_less(const bool cs) noexcept : case_sensitive(cs) {}
-
-    bool operator()(const std::string& a, const std::string& b) const
-    {
-        if (case_sensitive)
-        {
-            return a < b;
-        }
-        return std::lexicographical_compare(a.begin(), a.end(), b.begin(), b.end(),
-                                            [](unsigned char x, unsigned char y)
-        {
-            return std::tolower(x) < std::tolower(y);
-        });
-    }
-
-    bool case_sensitive = false;
-};
-
-template<class Key, class Value, class /*Compare*/, class Allocator>
-using key_case_map = std::map<Key, Value, key_case_less, Allocator>;
-using key_case_json = nlohmann::basic_json<key_case_map>;
-
-// the innermost value of a chain of single-element arrays
-template<typename Json>
-const Json& innermost(const Json& j)
-{
-    const Json* p = &j;
-    while (p->is_array())
-    {
-        p = &(*p)[0];
-    }
-    return *p;
-}
-
 // orders keys case-insensitively, so "key" and "KEY" compare equivalent
 // (neither less than the other) although they are not equal
 struct case_insensitive_less
@@ -931,47 +891,6 @@ TEST_CASE("equality of objects whose entries have no fixed order")
     }
 }
 
-TEST_CASE("copying an object preserves its comparator's state")
-{
-    // Past the iterative deep copy's nesting bound, an object copy used to be
-    // built with a default-constructed comparator instead of a copy of the
-    // original's. For an object type whose comparator carries state - here, a
-    // std::map that compares keys case-sensitively only when created that way
-    // - this reordered the copy's keys and could even drop entries that the
-    // original's comparator kept distinct (see #5649).
-    key_case_json object = key_case_json::object_t(key_case_less(true)); // case-sensitive
-    object["b"] = 1;
-    object["B"] = 2;
-    object["a"] = 3;
-    REQUIRE(object.dump() == R"({"B":2,"a":3,"b":1})");
-
-    for (const std::size_t depth : std::vector<std::size_t> {0, 127, 128, 200})
-    {
-        CAPTURE(depth);
-
-        key_case_json original = object;
-        for (std::size_t i = 0; i < depth; ++i)
-        {
-            original = key_case_json::array({std::move(original)});
-        }
-
-        {
-            const key_case_json copy = original; // NOLINT(performance-unnecessary-copy-initialization)
-            CHECK(innermost(copy).size() == 3);
-            CHECK(innermost(copy).dump() == R"({"B":2,"a":3,"b":1})");
-            CHECK(copy == original);
-        }
-
-        {
-            key_case_json copy = key_case_json::array();
-            copy = original;
-            CHECK(innermost(copy).size() == 3);
-            CHECK(innermost(copy).dump() == R"({"B":2,"a":3,"b":1})");
-            CHECK(copy == original);
-        }
-    }
-}
-
 TEST_CASE("equality of an object whose comparator treats different keys as equivalent")
 {
     // https://github.com/nlohmann/json/issues/5655: past the nesting bound,