Assert on missing array indices in const operator[] and document the JSON pointer case

The const operator[] overloads are unchecked by design, and a missing key
or index is undefined behavior. The key overload guards this with a
runtime assertion, but the index overload did not, although the element
access documentation says an assertion fires in both cases. The const
JSON pointer overload inherits both through json_pointer::get_unchecked(),
so a pointer to a missing array index read out of bounds even in debug
builds, and its documentation promised out_of_range.404 for any pointer
that cannot be resolved.

Add JSON_ASSERT(idx < size()) to const operator[](size_type), which also
covers the index leg of the const JSON pointer overload. Document the
undefined behavior for the const JSON pointer overload in operator[].md
and in the runtime assertions page. Release builds are unchanged.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

docs/mkdocs/docs/api/basic_json/operator[].md

@@ -86,6 +86,9 @@ Strong exception safety: if an exception occurs, the original value stays intact
     - Throws [`out_of_range.410`](../../home/exceptions.md#jsonexceptionout_of_range410) if an array index in the passed
       JSON pointer `ptr` exceeds the range of `size_type` (e.g., on 32-bit platforms).
 
+    For the **const** version, an object key or array index in `ptr` that does not exist is not reported by an
+    exception, but is undefined behavior (see the notes below). Use [`at`](at.md) for checked access.
+
 ## Complexity
 
 1. Constant if `idx` is in the range of the array. Otherwise, linear in `idx - size()`.
@@ -100,9 +103,12 @@ Strong exception safety: if an exception occurs, the original value stays intact
     The following cases apply to the **const** overloads; the non-const overloads instead insert the missing element
     (see the notes below).
 
-    1. If the element at index `idx` does not exist, the behavior is undefined.
+    1. If the element at index `idx` does not exist, the behavior is undefined and is **guarded by a
+       [runtime assertion](../../features/assertions.md)**!
     2. If the element with key `key` does not exist, the behavior is undefined and is **guarded by a
        [runtime assertion](../../features/assertions.md)**!
+    3. If the JSON pointer `ptr` refers to an object key or an array index that does not exist, the behavior is
+       undefined and is **guarded by a [runtime assertion](../../features/assertions.md)**!
 
 1. The non-const version may add values: If `idx` is beyond the range of the array (i.e., `idx >= size()`), then the
    array is silently filled up with `#!json null` values to make `idx` a valid reference to the last stored element. In
@@ -128,13 +134,10 @@ Strong exception safety: if an exception occurs, the original value stays intact
 
         When the JSON pointer traverses intermediate levels that don't exist at all yet (not just a missing
         leaf), each missing level is created as an array or an object depending on whether the corresponding
-        pointer token is a valid array index: the token `0`, a sequence of digits that does not begin with `0`,
-        or the token `-` creates an array, and every other token creates an object. For example, on an
-        initially `#!json null` value, `/foo/0/0/0` creates nested arrays, while `/foo/one/one/one` creates
-        nested objects. Tokens such as `01` or the empty token cannot be array indices (cf. RFC 6901, Sect. 4)
-        and therefore create objects, just as they would if the level already existed as an object. This is not
-        specified by the JSON Pointer RFC; it is this library's own, intentional disambiguation rule. See also
-        [JSON Pointer](../../features/json_pointer.md).
+        pointer token parses as a non-negative integer: a numeric token creates an array, a non-numeric token
+        creates an object. For example, on an initially `#!json null` value, `/foo/0/0/0` creates nested arrays,
+        while `/foo/one/one/one` creates nested objects. This is not specified by the JSON Pointer RFC; it is
+        this library's own, intentional disambiguation rule. See also [JSON Pointer](../../features/json_pointer.md).
 
 ## Examples
 
@@ -260,9 +263,11 @@ Strong exception safety: if an exception occurs, the original value stays intact
 
 ## Version history
 
-1. Added in version 1.0.0.
+1. Added in version 1.0.0. A missing index in the const version is guarded by a runtime assertion since version
+   3.13.0.
 2. Added in version 1.0.0. Added overloads for `T* key` in version 1.1.0. Removed overloads for `T* key` (replaced by 3)
    in version 3.11.0.
 3. Added in version 3.11.0. Fixed in version 3.13.0 to consistently accept `std::string_view`-convertible keys, as
    already supported by [`at`](at.md), [`value`](value.md), [`find`](find.md), and other lookup functions.
-4. Added in version 2.0.0.
+4. Added in version 2.0.0. A missing array index in the const version is guarded by a runtime assertion since
+   version 3.13.0.

docs/mkdocs/docs/features/assertions.md

@@ -16,14 +16,15 @@ before including the `json.hpp` header.
 
 ## Function with runtime assertions
 
-### Unchecked object access to a const value
+### Unchecked access to a const value
 
-Function [`operator[]`](../api/basic_json/operator%5B%5D.md) implements unchecked access for objects. Whereas a missing
-key is added in the case of non-const objects, accessing a const object with a missing key is undefined behavior (think
-of a dereferenced null pointer) and yields a runtime assertion.
+Function [`operator[]`](../api/basic_json/operator%5B%5D.md) implements unchecked access for arrays and objects. Whereas
+a missing element is added in the case of non-const values, accessing a const value with a missing object key or an
+invalid array index is undefined behavior (think of a dereferenced null pointer) and yields a runtime assertion. This
+also applies to a [JSON pointer](json_pointer.md) that refers to a missing key or an invalid index.
 
-If you are not sure whether an element in an object exists, use checked access with the
-[`at` function](../api/basic_json/at.md) or call the [`contains` function](../api/basic_json/contains.md) before.
+If you are not sure whether an element exists, use checked access with the [`at` function](../api/basic_json/at.md)
+or call the [`contains` function](../api/basic_json/contains.md) before.
 
 See also the documentation on [element access](element_access/index.md).
 
@@ -46,7 +47,30 @@ See also the documentation on [element access](element_access/index.md).
     Output:
 
     ```
-    Assertion failed: (m_value.object->find(key) != m_value.object->end()), function operator[], file json.hpp, line 2144.
+    Assertion failed: (it != m_data.m_value.object->end()), function operator[], file json.hpp, line 28795.
+    ```
+
+??? example "Example 2: Invalid array index in a JSON pointer"
+
+    The following code will trigger an assertion at runtime:
+
+    ```cpp
+    #include <nlohmann/json.hpp>
+    
+    using json = nlohmann::json;
+    using namespace nlohmann::literals;
+    
+    int main()
+    {
+        const json j = {{"array", {1, 2, 3}}};
+        auto v = j["/array/5"_json_pointer];
+    }
+    ```
+
+    Output:
+
+    ```
+    Assertion failed: (idx < m_data.m_value.array->size()), function operator[], file json.hpp, line 28758.
     ```
 
 ### Constructing from an uninitialized iterator range

include/nlohmann/detail/json_pointer.hpp

@@ -416,19 +416,15 @@ class json_pointer
             // convert null values to arrays or objects before continuing
             if (ptr->is_null())
             {
-                // check if the reference token is a valid array index, that is
-                // a nonempty sequence of digits without a leading '0'
-                // (cf. RFC 6901, Sect. 4); tokens that could never be a valid
-                // array index (such as "01" or "") are treated as object keys
-                const bool nums = !reference_token.empty()
-                                  && (reference_token.size() == 1 || reference_token[0] != '0')
-                                  && std::all_of(reference_token.begin(), reference_token.end(),
-                                                 [](const unsigned char x)
+                // check if the reference token is a number
+                const bool nums =
+                    std::all_of(reference_token.begin(), reference_token.end(),
+                                [](const unsigned char x)
                 {
                     return std::isdigit(x);
                 });
 
-                // change value to an array for array indices or "-" or to object otherwise
+                // change value to an array for numbers or "-" or to object otherwise
                 *ptr = (nums || reference_token == "-")
                        ? detail::value_t::array
                        : detail::value_t::object;
@@ -539,6 +535,10 @@ class json_pointer
     @return const reference to the JSON value pointed to by the JSON
     pointer
 
+    @pre Every object key and array index the pointer refers to exists.
+         Like the const operator[] for keys and indices, a missing one is
+         undefined behavior, guarded by a runtime assertion.
+
     @throw parse_error.106   if an array index begins with '0'
     @throw parse_error.109   if an array index was not a number
     @throw out_of_range.402  if the array index '-' is used
@@ -553,7 +553,8 @@ class json_pointer
             {
                 case detail::value_t::object:
                 {
-                    // use unchecked object access
+                    // use unchecked object access; the const operator[]
+                    // asserts that the key exists
                     ptr = &ptr->operator[](reference_token);
                     break;
                 }
@@ -566,7 +567,8 @@ class json_pointer
                         JSON_THROW(detail::out_of_range::create(402, detail::concat("array index '-' (", std::to_string(ptr->m_data.m_value.array->size()), ") is out of range"), ptr));
                     }
 
-                    // use unchecked array access
+                    // use unchecked array access; the const operator[]
+                    // asserts that the index exists
                     ptr = &ptr->operator[](array_index<BasicJsonType>(reference_token));
                     break;
                 }

include/nlohmann/json.hpp

@@ -2866,6 +2866,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
         // const operator[] only works for arrays
         if (JSON_HEDLEY_LIKELY(is_array()))
         {
+            JSON_ASSERT(idx < m_data.m_value.array->size());
             return m_data.m_value.array->operator[](idx);
         }
 

single_include/nlohmann/json.hpp

@@ -19060,19 +19060,15 @@ class json_pointer
             // convert null values to arrays or objects before continuing
             if (ptr->is_null())
             {
-                // check if the reference token is a valid array index, that is
-                // a nonempty sequence of digits without a leading '0'
-                // (cf. RFC 6901, Sect. 4); tokens that could never be a valid
-                // array index (such as "01" or "") are treated as object keys
-                const bool nums = !reference_token.empty()
-                                  && (reference_token.size() == 1 || reference_token[0] != '0')
-                                  && std::all_of(reference_token.begin(), reference_token.end(),
-                                                 [](const unsigned char x)
+                // check if the reference token is a number
+                const bool nums =
+                    std::all_of(reference_token.begin(), reference_token.end(),
+                                [](const unsigned char x)
                 {
                     return std::isdigit(x);
                 });
 
-                // change value to an array for array indices or "-" or to object otherwise
+                // change value to an array for numbers or "-" or to object otherwise
                 *ptr = (nums || reference_token == "-")
                        ? detail::value_t::array
                        : detail::value_t::object;
@@ -19183,6 +19179,10 @@ class json_pointer
     @return const reference to the JSON value pointed to by the JSON
     pointer
 
+    @pre Every object key and array index the pointer refers to exists.
+         Like the const operator[] for keys and indices, a missing one is
+         undefined behavior, guarded by a runtime assertion.
+
     @throw parse_error.106   if an array index begins with '0'
     @throw parse_error.109   if an array index was not a number
     @throw out_of_range.402  if the array index '-' is used
@@ -19197,7 +19197,8 @@ class json_pointer
             {
                 case detail::value_t::object:
                 {
-                    // use unchecked object access
+                    // use unchecked object access; the const operator[]
+                    // asserts that the key exists
                     ptr = &ptr->operator[](reference_token);
                     break;
                 }
@@ -19210,7 +19211,8 @@ class json_pointer
                         JSON_THROW(detail::out_of_range::create(402, detail::concat("array index '-' (", std::to_string(ptr->m_data.m_value.array->size()), ") is out of range"), ptr));
                     }
 
-                    // use unchecked array access
+                    // use unchecked array access; the const operator[]
+                    // asserts that the index exists
                     ptr = &ptr->operator[](array_index<BasicJsonType>(reference_token));
                     break;
                 }
@@ -28753,6 +28755,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
         // const operator[] only works for arrays
         if (JSON_HEDLEY_LIKELY(is_array()))
         {
+            JSON_ASSERT(idx < m_data.m_value.array->size());
             return m_data.m_value.array->operator[](idx);
         }
 

tests/src/unit-json_pointer.cpp

@@ -438,72 +438,6 @@ TEST_CASE("JSON pointers")
         }
     }
 
-    SECTION("creating intermediate levels")
-    {
-        SECTION("tokens that are valid array indices create arrays")
-        {
-            json j;
-            j["/0"_json_pointer] = 1;
-            CHECK(j == json({1}));
-
-            json j2;
-            j2["/2"_json_pointer] = 1;
-            CHECK(j2 == json({nullptr, nullptr, 1}));
-
-            json j3;
-            j3["/-"_json_pointer] = 1;
-            CHECK(j3 == json({1}));
-
-            json j4;
-            j4["/foo/0/0"_json_pointer] = 1;
-            CHECK(j4 == json({{"foo", {{1}}}}));
-        }
-
-        SECTION("tokens that are no valid array indices create objects")
-        {
-            json j;
-            j["/one"_json_pointer] = 1;
-            CHECK(j == json({{"one", 1}}));
-
-            // leading '0' can never be a valid array index (RFC 6901, Sect. 4)
-            json j2;
-            j2["/01"_json_pointer] = 1;
-            CHECK(j2 == json({{"01", 1}}));
-
-            // the empty token is a valid object key, but no valid array index
-            json j3;
-            j3["/"_json_pointer] = 1;
-            CHECK(j3 == json({{"", 1}}));
-        }
-
-        SECTION("creating a level yields the same result as reusing it (#5357)")
-        {
-            json j;
-            j["/a/b/01/d"_json_pointer] = "value";
-
-            json j_init = json::object();
-            j_init["/a/b"_json_pointer] = json::object();
-            j_init["/a/b/01/d"_json_pointer] = "value";
-
-            const json expected = json::parse(R"({"a":{"b":{"01":{"d":"value"}}}})");
-            CHECK(j == expected);
-            CHECK(j_init == expected);
-
-            // unflatten uses the same key
-            const json flat = {{"/a/b/01/d", "value"}};
-            CHECK(flat.unflatten() == expected);
-        }
-
-        SECTION("existing arrays still reject invalid indices")
-        {
-            json j = {1, 2, 3};
-            CHECK_THROWS_WITH_AS(j["/01"_json_pointer],
-                                 "[json.exception.parse_error.106] parse error: array index '01' must not begin with '0'", json::parse_error&);
-            CHECK_THROWS_WITH_AS(j.at("/01"_json_pointer),
-                                 "[json.exception.parse_error.106] parse error: array index '01' must not begin with '0'", json::parse_error&);
-        }
-    }
-
     SECTION("flatten")
     {
         json j =