From 2cca04ae6f1a63ed65843a466b95d70f218885f5 Mon Sep 17 00:00:00 2001 From: Niels Lohmann Date: Sun, 11 Oct 2026 08:21:41 +0200 Subject: [PATCH] Report one-character non-numeric array indices like longer ones (#5800) * Report one-character non-numeric array indices like longer ones A JSON pointer reference token that is not a number but has only one character (e.g. "/a/x") was reported as out_of_range.404 ("unresolved reference token"), because the "is not a number" check only ran for tokens longer than one character; "/a/xy" got parse_error.109. Both now throw parse_error.109. "-" and the empty token are still reported as out_of_range.404. As a consequence, value(json_pointer, default) on an array now throws for "/x" as it already did for "/xy". Also document why ordered_map::erase's destroy/placement-new loop on pair is kept despite [basic.life]/8 before C++20. Signed-off-by: Niels Lohmann * Document the one-character array index change Add 3.13.0 version-history entries to at, operator[], value, patch, patch_inplace, and unflatten, and describe in exceptions.md which array indices throw parse_error.109 and which out_of_range.404. Signed-off-by: Niels Lohmann --------- Signed-off-by: Niels Lohmann --- docs/mkdocs/docs/api/basic_json/at.md | 4 +++- docs/mkdocs/docs/api/basic_json/operator[].md | 4 +++- docs/mkdocs/docs/api/basic_json/patch.md | 3 +++ .../docs/api/basic_json/patch_inplace.md | 3 +++ docs/mkdocs/docs/api/basic_json/unflatten.md | 8 +++++-- docs/mkdocs/docs/api/basic_json/value.md | 4 +++- docs/mkdocs/docs/home/exceptions.md | 12 +++++++++-- include/nlohmann/detail/json_pointer.hpp | 8 ++++--- include/nlohmann/ordered_map.hpp | 8 +++++++ single_include/nlohmann/json.hpp | 16 +++++++++++--- tests/src/unit-element_access2.cpp | 2 ++ tests/src/unit-json_patch.cpp | 15 +++++++++++++ tests/src/unit-json_pointer.cpp | 21 +++++++++++++++++++ 13 files changed, 95 insertions(+), 13 deletions(-) diff --git a/docs/mkdocs/docs/api/basic_json/at.md b/docs/mkdocs/docs/api/basic_json/at.md index 18f108964..ad851c77e 100644 --- a/docs/mkdocs/docs/api/basic_json/at.md +++ b/docs/mkdocs/docs/api/basic_json/at.md @@ -241,4 +241,6 @@ Strong exception safety: if an exception occurs, the original value stays intact 3. Added in version 3.11.0. Fixed in version 3.13.0 to consistently accept `std::string_view`-convertible keys, as already supported by [`operator[]`](operator[].md), [`value`](value.md), [`find`](find.md), and other lookup functions. -4. Added in version 2.0.0. +4. Added in version 2.0.0. Throws [`parse_error.109`](../../home/exceptions.md#jsonexceptionparse_error109) instead of +[`out_of_range.404`](../../home/exceptions.md#jsonexceptionout_of_range404) for a one-character array index that is not +a digit (e.g., `/x`) in version 3.13.0, as it already did for longer ones. diff --git a/docs/mkdocs/docs/api/basic_json/operator[].md b/docs/mkdocs/docs/api/basic_json/operator[].md index 74996d629..f31d90072 100644 --- a/docs/mkdocs/docs/api/basic_json/operator[].md +++ b/docs/mkdocs/docs/api/basic_json/operator[].md @@ -286,4 +286,6 @@ Strong exception safety: if an exception occurs, the original value stays intact 3. Added in version 3.11.0. Fixed in version 3.13.0 to consistently accept `std::string_view`-convertible keys, as already supported by [`at`](at.md), [`value`](value.md), [`find`](find.md), and other lookup functions. 4. Added in version 2.0.0. A missing array index in the const version is guarded by a runtime assertion since - version 3.13.0. + version 3.13.0. Throws [`parse_error.109`](../../home/exceptions.md#jsonexceptionparse_error109) instead of +[`out_of_range.404`](../../home/exceptions.md#jsonexceptionout_of_range404) for a one-character array index that is not +a digit (e.g., `/x`) in version 3.13.0, as it already did for longer ones. diff --git a/docs/mkdocs/docs/api/basic_json/patch.md b/docs/mkdocs/docs/api/basic_json/patch.md index 8c2d7f231..951ee46e6 100644 --- a/docs/mkdocs/docs/api/basic_json/patch.md +++ b/docs/mkdocs/docs/api/basic_json/patch.md @@ -112,3 +112,6 @@ is thrown. In any case, the original value is not changed: the patch is applied location has a non-object/non-array parent in version 3.13.0. - Added [`out_of_range.414`](../../home/exceptions.md#jsonexceptionout_of_range414) and rejected a "move" operation whose "from" location is a proper prefix of its "path" location instead of silently producing a corrupted result in version 3.13.0. +- Throws [`parse_error.109`](../../home/exceptions.md#jsonexceptionparse_error109) instead of + [`out_of_range.404`](../../home/exceptions.md#jsonexceptionout_of_range404) for a one-character array index that is + not a digit (e.g., `/x`) in version 3.13.0, as it already did for longer ones. diff --git a/docs/mkdocs/docs/api/basic_json/patch_inplace.md b/docs/mkdocs/docs/api/basic_json/patch_inplace.md index 2d46a2cc8..dff8b095b 100644 --- a/docs/mkdocs/docs/api/basic_json/patch_inplace.md +++ b/docs/mkdocs/docs/api/basic_json/patch_inplace.md @@ -110,3 +110,6 @@ function throws an exception. location has a non-object/non-array parent in version 3.13.0. - Added [`out_of_range.414`](../../home/exceptions.md#jsonexceptionout_of_range414) and rejected a "move" operation whose "from" location is a proper prefix of its "path" location instead of silently producing a corrupted result in version 3.13.0. +- Throws [`parse_error.109`](../../home/exceptions.md#jsonexceptionparse_error109) instead of + [`out_of_range.404`](../../home/exceptions.md#jsonexceptionout_of_range404) for a one-character array index that is + not a digit (e.g., `/x`) in version 3.13.0, as it already did for longer ones. diff --git a/docs/mkdocs/docs/api/basic_json/unflatten.md b/docs/mkdocs/docs/api/basic_json/unflatten.md index a8f0a60b0..661ffa3e8 100644 --- a/docs/mkdocs/docs/api/basic_json/unflatten.md +++ b/docs/mkdocs/docs/api/basic_json/unflatten.md @@ -36,8 +36,9 @@ The function can throw the following exceptions: - Throws [`parse_error.109`](../../home/exceptions.md#jsonexceptionparse_error109) if an array index in a key is not a number; example: `"array index 'one' is not a number"` - Throws [`out_of_range.404`](../../home/exceptions.md#jsonexceptionout_of_range404) if a level becomes an array - (because one of its keys is `0`) and another key at that level cannot be an array index; example: - `"unresolved reference token 'x'"` + (because one of its keys is `0`) and another key at that level begins with a digit but is not a valid array index + (such as `1a`), or is `-`; example: + `"unresolved reference token '-'"` ## Complexity @@ -80,3 +81,6 @@ Apart from these two cases, for a JSON value `j`, the following is always true: - Added in version 2.0.0. - Made the array/object decision independent of the object's iteration order in version 3.13.0. +- Throws [`parse_error.109`](../../home/exceptions.md#jsonexceptionparse_error109) instead of + [`out_of_range.404`](../../home/exceptions.md#jsonexceptionout_of_range404) for a one-character array index that is + not a digit (e.g., `/x`) in version 3.13.0, as it already did for longer ones. diff --git a/docs/mkdocs/docs/api/basic_json/value.md b/docs/mkdocs/docs/api/basic_json/value.md index 80f5691dc..f8a1c5528 100644 --- a/docs/mkdocs/docs/api/basic_json/value.md +++ b/docs/mkdocs/docs/api/basic_json/value.md @@ -227,4 +227,6 @@ changes to any JSON value. [`operator[]`](operator[].md), [`at`](at.md), [`find`](find.md), and other lookup functions. 3. Added in version 2.0.2. Extended to work with arrays in version 3.13.0, including fixing an issue where resolving `ptr` through an array unexpectedly threw `out_of_range` instead of returning the resolved element (or - `default_value`, as documented). + `default_value`, as documented). Throws [`parse_error.109`](../../home/exceptions.md#jsonexceptionparse_error109) + instead of returning `default_value` for a one-character array index that is not a digit (e.g., `/x`) in version + 3.13.0, as it already did for longer ones. diff --git a/docs/mkdocs/docs/home/exceptions.md b/docs/mkdocs/docs/home/exceptions.md index d6fdce37a..32312334a 100644 --- a/docs/mkdocs/docs/home/exceptions.md +++ b/docs/mkdocs/docs/home/exceptions.md @@ -278,7 +278,9 @@ In a JSON Pointer, only `~0` and `~1` are valid escape sequences. ### json.exception.parse_error.109 -A JSON Pointer array index must be a number. +A JSON Pointer array index must be a number. This exception is thrown for an array index that does not begin with a +digit, except for `-` and the empty reference token, which throw [`out_of_range.404`](#jsonexceptionout_of_range404) +where they cannot be resolved. !!! failure "Example messages" @@ -289,6 +291,11 @@ A JSON Pointer array index must be a number. [json.exception.parse_error.109] parse error: array index '+1' is not a number ``` +!!! note + + Before version 3.13.0, a one-character array index that is not a digit (e.g., `x`) threw + [`out_of_range.404`](#jsonexceptionout_of_range404) instead. + ### json.exception.parse_error.110 When parsing a [binary format](../features/binary_formats/index.md), the byte vector ends before the complete value has @@ -871,7 +878,8 @@ The provided key was not found in the JSON object. ### json.exception.out_of_range.404 -A reference token in a JSON Pointer could not be resolved. +A reference token in a JSON Pointer could not be resolved, for instance an array index that begins with a digit but +contains other characters (e.g., `1a`), or `-` where it cannot be used. !!! failure "Example message" diff --git a/include/nlohmann/detail/json_pointer.hpp b/include/nlohmann/detail/json_pointer.hpp index cff8420be..1b439f9bf 100644 --- a/include/nlohmann/detail/json_pointer.hpp +++ b/include/nlohmann/detail/json_pointer.hpp @@ -270,7 +270,7 @@ class json_pointer { ok, ///< @a s is a valid, representable array index leading_zero, ///< @a s begins with '0' but has more than one character - not_a_number, ///< @a s does not begin with a digit + not_a_number, ///< @a s is neither empty nor "-" and does not begin with a digit unresolved, ///< @a s could not be converted to an integer exceeds_size_type ///< @a s converts to an integer that exceeds size_type }; @@ -297,8 +297,10 @@ class json_pointer return array_index_status::leading_zero; } - // error condition (cf. RFC 6901, Sect. 4) - if (JSON_HEDLEY_UNLIKELY(s.size() > 1 && !(s[0] >= '1' && s[0] <= '9'))) + // error condition (cf. RFC 6901, Sect. 4); this also covers single- + // character tokens, so "/x" and "/xy" fail alike; "-" and the empty + // token are left to the conversion below and are reported as unresolved + if (JSON_HEDLEY_UNLIKELY(!s.empty() && s != "-" && !(s[0] >= '0' && s[0] <= '9'))) { return array_index_status::not_a_number; } diff --git a/include/nlohmann/ordered_map.hpp b/include/nlohmann/ordered_map.hpp index 5f0be7d28..fdcf4a5c0 100644 --- a/include/nlohmann/ordered_map.hpp +++ b/include/nlohmann/ordered_map.hpp @@ -240,6 +240,14 @@ public: // ^ ^ // first last + // Note on conformance: before C++20, [basic.life]/8 did not allow an + // object of a type with a const member (like value_type's const Key) + // to transparently replace the destroyed one, so strictly, accessing + // it through the vector's existing pointers would have required + // std::launder (which does not exist before C++17). C++20 dropped that + // condition (P1971R0, NB comment US 041). Compilers have always treated + // this pattern as intended, so it is kept deliberately. + // Since we cannot move const Keys, we re-construct them in place. // We start at first and re-construct (viz. copy) the elements from // the back of the vector. Example for the first iteration: diff --git a/single_include/nlohmann/json.hpp b/single_include/nlohmann/json.hpp index a8a9bab3e..c4f7ab348 100644 --- a/single_include/nlohmann/json.hpp +++ b/single_include/nlohmann/json.hpp @@ -20283,7 +20283,7 @@ class json_pointer { ok, ///< @a s is a valid, representable array index leading_zero, ///< @a s begins with '0' but has more than one character - not_a_number, ///< @a s does not begin with a digit + not_a_number, ///< @a s is neither empty nor "-" and does not begin with a digit unresolved, ///< @a s could not be converted to an integer exceeds_size_type ///< @a s converts to an integer that exceeds size_type }; @@ -20310,8 +20310,10 @@ class json_pointer return array_index_status::leading_zero; } - // error condition (cf. RFC 6901, Sect. 4) - if (JSON_HEDLEY_UNLIKELY(s.size() > 1 && !(s[0] >= '1' && s[0] <= '9'))) + // error condition (cf. RFC 6901, Sect. 4); this also covers single- + // character tokens, so "/x" and "/xy" fail alike; "-" and the empty + // token are left to the conversion below and are reported as unresolved + if (JSON_HEDLEY_UNLIKELY(!s.empty() && s != "-" && !(s[0] >= '0' && s[0] <= '9'))) { return array_index_status::not_a_number; } @@ -27883,6 +27885,14 @@ public: // ^ ^ // first last + // Note on conformance: before C++20, [basic.life]/8 did not allow an + // object of a type with a const member (like value_type's const Key) + // to transparently replace the destroyed one, so strictly, accessing + // it through the vector's existing pointers would have required + // std::launder (which does not exist before C++17). C++20 dropped that + // condition (P1971R0, NB comment US 041). Compilers have always treated + // this pattern as intended, so it is kept deliberately. + // Since we cannot move const Keys, we re-construct them in place. // We start at first and re-construct (viz. copy) the elements from // the back of the vector. Example for the first iteration: diff --git a/tests/src/unit-element_access2.cpp b/tests/src/unit-element_access2.cpp index 1b59d8add..7765c0601 100644 --- a/tests/src/unit-element_access2.cpp +++ b/tests/src/unit-element_access2.cpp @@ -535,6 +535,8 @@ TEST_CASE_TEMPLATE("element access 2", Json, nlohmann::json, nlohmann::ordered_j // Test malformed index (non-numeric) throws parse_error CHECK_THROWS_WITH_AS(j_array.value("/foo"_json_pointer, 1), "[json.exception.parse_error.109] parse error: array index 'foo' is not a number", typename Json::parse_error&); CHECK_THROWS_WITH_AS(j_array_const.value("/foo"_json_pointer, 1), "[json.exception.parse_error.109] parse error: array index 'foo' is not a number", typename Json::parse_error&); + CHECK_THROWS_WITH_AS(j_array.value("/x"_json_pointer, 1), "[json.exception.parse_error.109] parse error: array index 'x' is not a number", typename Json::parse_error&); + CHECK_THROWS_WITH_AS(j_array_const.value("/x"_json_pointer, 1), "[json.exception.parse_error.109] parse error: array index 'x' is not a number", typename Json::parse_error&); // Test leading-zero index throws parse_error CHECK_THROWS_WITH_AS(j_array.value("/01"_json_pointer, 1), "[json.exception.parse_error.106] parse error: array index '01' must not begin with '0'", typename Json::parse_error&); diff --git a/tests/src/unit-json_patch.cpp b/tests/src/unit-json_patch.cpp index 55775c0b6..9e83edbcf 100644 --- a/tests/src/unit-json_patch.cpp +++ b/tests/src/unit-json_patch.cpp @@ -1713,6 +1713,21 @@ TEST_CASE("JSON patch - move where 'from' is a proper prefix of 'path' (regressi CHECK(doc.patch(patch) == R"({"b": 1})"_json); } + SECTION("array index tokens that are not a number") + { + json const doc = R"({"a": [1, 2]})"_json; + + // "-" cannot be removed and is reported as unresolved + json const patch_dash = {{{"op", "remove"}, {"path", "/a/-"}}}; + CHECK_THROWS_WITH_AS(doc.patch(patch_dash), "[json.exception.out_of_range.404] unresolved reference token '-'", json::out_of_range&); + + // a single character is reported like a longer token + json const patch_x = {{{"op", "remove"}, {"path", "/a/x"}}}; + CHECK_THROWS_WITH_AS(doc.patch(patch_x), "[json.exception.parse_error.109] parse error: array index 'x' is not a number", json::parse_error&); + json const patch_xy = {{{"op", "remove"}, {"path", "/a/xy"}}}; + CHECK_THROWS_WITH_AS(doc.patch(patch_xy), "[json.exception.parse_error.109] parse error: array index 'xy' is not a number", json::parse_error&); + } + SECTION("the array-append token '-' is an ordinary child token") { // "-" (append-to-array) addresses a location *inside* the array, diff --git a/tests/src/unit-json_pointer.cpp b/tests/src/unit-json_pointer.cpp index 1faf511f4..658a7fa8d 100644 --- a/tests/src/unit-json_pointer.cpp +++ b/tests/src/unit-json_pointer.cpp @@ -421,6 +421,27 @@ TEST_CASE("JSON pointers") CHECK_THROWS_WITH_AS(json({{"/list/0", 1}, {"/list/1", 2}, {"/list/three", 3}}).unflatten(), "[json.exception.parse_error.109] parse error: array index 'three' is not a number", json::parse_error&); + // a single-character token that is not a digit is reported like a + // longer one (parse_error.109), not as unresolved (out_of_range.404) + CHECK_THROWS_WITH_AS(j["/x"_json_pointer] = 1, + "[json.exception.parse_error.109] parse error: array index 'x' is not a number", json::parse_error&); + CHECK_THROWS_WITH_AS(j_const["/x"_json_pointer] == 1, + "[json.exception.parse_error.109] parse error: array index 'x' is not a number", json::parse_error&); + CHECK_THROWS_WITH_AS(j.at("/x"_json_pointer) = 1, + "[json.exception.parse_error.109] parse error: array index 'x' is not a number", json::parse_error&); + CHECK_THROWS_WITH_AS(j_const.at("/x"_json_pointer) == 1, + "[json.exception.parse_error.109] parse error: array index 'x' is not a number", json::parse_error&); + CHECK_THROWS_WITH_AS(j.at("/+"_json_pointer), + "[json.exception.parse_error.109] parse error: array index '+' is not a number", json::parse_error&); + CHECK(!j.contains("/x"_json_pointer)); + CHECK(!j_const.contains("/x"_json_pointer)); + CHECK_THROWS_WITH_AS(json({{"/list/0", 1}, {"/list/x", 2}}).unflatten(), + "[json.exception.parse_error.109] parse error: array index 'x' is not a number", json::parse_error&); + + // "-" is a valid reference token, so it is still reported as unresolved + CHECK_THROWS_WITH_AS(json({{"/list/0", 1}, {"/list/-", 2}}).unflatten(), + "[json.exception.out_of_range.404] unresolved reference token '-'", json::out_of_range&); + // assign to "-" j["/-"_json_pointer] = 99; CHECK(j == json({1, 13, 3, 33, nullptr, 55, 99}));