Compare dump()s instead of json values in the BJData fuzzer's round-trip check

The value-stability assertion added to fix the earlier OSS-Fuzz crash
(json::from_bjdata(to_bjdata(j2)) == j2) itself broke on a NaN payload:
IEEE 754 NaN is never equal to itself, so operator== reports two
structurally-identical trees containing a non-finite double as
different -- not a round-trip bug, just NaN's ordinary
non-reflexivity. dump() serializes any non-finite double the same
deterministic way (as JSON null, since JSON cannot represent NaN or
Infinity), so comparing dumps is stable under exactly the values that
break operator==.

Verified against both the original OSS-Fuzz crash input and the new
one (0x68 0x68 0x7c, which decodes to a NaN), plus a local 2.5M-case
random-input sweep with no failures.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

docs/mkdocs/docs/api/basic_json/patch.md

@@ -34,8 +34,6 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
   ("add", "remove", "move")
 - Throws [`out_of_range.411`](../../home/exceptions.md#jsonexceptionout_of_range411) if an "add" operation's target
   location has a parent that is neither an object nor an array.
-- Throws [`out_of_range.413`](../../home/exceptions.md#jsonexceptionout_of_range413) if a "remove" operation's target
-  location has a parent that is neither an object nor an array.
 - Throws [`other_error.501`](../../home/exceptions.md#jsonexceptionother_error501) if "test" operation was
   unsuccessful.
 
@@ -77,5 +75,3 @@ is thrown. In any case, the original value is not changed: the patch is applied
 - Added in version 2.0.0.
 - Added [`out_of_range.411`](../../home/exceptions.md#jsonexceptionout_of_range411) and stopped relying on an internal assertion when an "add" operation's
   target location has a non-object/non-array parent in version 3.13.0.
-- Added [`out_of_range.413`](../../home/exceptions.md#jsonexceptionout_of_range413) and stopped silently ignoring a "remove" operation whose target
-  location has a non-object/non-array parent in version 3.13.0.

docs/mkdocs/docs/api/basic_json/patch_inplace.md

@@ -30,8 +30,6 @@ No guarantees, value may be corrupted by an unsuccessful patch operation.
   ("add", "remove", "move")
 - Throws [`out_of_range.411`](../../home/exceptions.md#jsonexceptionout_of_range411) if an "add" operation's target
   location has a parent that is neither an object nor an array.
-- Throws [`out_of_range.413`](../../home/exceptions.md#jsonexceptionout_of_range413) if a "remove" operation's target
-  location has a parent that is neither an object nor an array.
 - Throws [`other_error.501`](../../home/exceptions.md#jsonexceptionother_error501) if "test" operation was
   unsuccessful.
 
@@ -74,5 +72,3 @@ function throws an exception.
 - Added in version 3.11.0.
 - Added [`out_of_range.411`](../../home/exceptions.md#jsonexceptionout_of_range411) and stopped relying on an internal assertion when an "add" operation's
   target location has a non-object/non-array parent in version 3.13.0.
-- Added [`out_of_range.413`](../../home/exceptions.md#jsonexceptionout_of_range413) and stopped silently ignoring a "remove" operation whose target
-  location has a non-object/non-array parent in version 3.13.0.

docs/mkdocs/docs/home/exceptions.md

@@ -933,20 +933,6 @@ BSON stores the length of documents, arrays, strings, and binary values in a sig
     [`to_bson`](../api/basic_json/to_bson.md) produced documents with negative length prefixes that
     [`from_bson`](../api/basic_json/from_bson.md) rejected.
 
-### json.exception.out_of_range.413
-
-A JSON Patch `remove` operation cannot be applied because the target location's parent is neither an object nor an array. Per [RFC 6902](https://datatracker.ietf.org/doc/html/rfc6902), a `remove` target must reference a member of an existing object or an element of an existing array; a primitive value (string, number, boolean, etc.) or `null` has no members or elements to remove.
-
-!!! failure "Example message"
-
-    ```
-    cannot remove value: the JSON Patch 'remove' target's parent is of type number, but must be an object or array
-    ```
-
-!!! note
-
-    This exception was added in version 3.13.0. Before that, this situation was silently ignored (the `remove` operation had no effect).
-
 ## Further exceptions
 
 This exception is thrown in case of errors that cannot be classified with the

include/nlohmann/detail/output/binary_writer.hpp

@@ -1647,6 +1647,20 @@ class binary_writer
         return 'D';  // float 64
     }
 
+    /*!
+    @brief checks whether a JSON number fits into @a TargetType
+    @param[in] el a JSON number of either the signed or unsigned integer kind
+    @return whether @a el's value can be represented by @a TargetType without
+            wrapping, regardless of which of the two kinds it is stored as
+    */
+    template<typename TargetType>
+    static bool bjdata_ndarray_value_in_range(const BasicJsonType& el)
+    {
+        return el.is_number_unsigned()
+               ? value_in_range_of<TargetType>(el.template get<std::uint64_t>())
+               : value_in_range_of<TargetType>(el.template get<std::int64_t>());
+    }
+
     /*!
     @return false if the object is successfully converted to a bjdata ndarray, true if the type or size is invalid
     */
@@ -1658,6 +1672,16 @@ class binary_writer
         };
 
         string_t key = "_ArrayType_";
+        // the type name is looked up as a string below; a non-string
+        // annotation (e.g. a number, null, or an array) cannot name a known
+        // dtype, so it is treated the same as an unrecognized type name and
+        // falls back to a plain object encoding instead of throwing
+        // type_error.302 out of get<string_t>()
+        if (!value.at(key).is_string())
+        {
+            return true;
+        }
+
         // use get<string_t>() instead of static_cast<string_t> to avoid an
         // ambiguous conversion under explicit instantiation on C++17 (see #4825)
         auto it = bjdtype.find(value.at(key).template get<string_t>());
@@ -1667,6 +1691,16 @@ class binary_writer
         }
         CharType dtype = it->second;
 
+        // the 'B' (byte) marker is only defined by BJData Draft 3; emitting it
+        // under the default Draft 2 mode would produce a stream that Draft 2
+        // readers reject, so such an object falls back to a plain object
+        // encoding instead (see the "Binary values" section of the BJData
+        // documentation)
+        if (dtype == 'B' && bjdata_version != bjdata_version_t::draft3)
+        {
+            return true;
+        }
+
         key = "_ArraySize_";
         // the dimensions are written verbatim as the header length below, so a
         // value that is not an array cannot produce a valid one: null emits 'Z'
@@ -1731,6 +1765,60 @@ class binary_writer
             }
         }
 
+        // every element is cast to the (possibly narrower) C++ type matching
+        // dtype below; a value that does not fit that type would silently
+        // wrap (integers) or overflow to infinity (the "single" precision
+        // float) instead of being reported, so such an object falls back to
+        // a plain object encoding as well
+        for (const auto& el : value.at(key))
+        {
+            bool in_range = true;
+            switch (dtype)
+            {
+                case 'U':
+                case 'C':
+                case 'B':
+                    in_range = bjdata_ndarray_value_in_range<std::uint8_t>(el);
+                    break;
+                case 'i':
+                    in_range = bjdata_ndarray_value_in_range<std::int8_t>(el);
+                    break;
+                case 'u':
+                    in_range = bjdata_ndarray_value_in_range<std::uint16_t>(el);
+                    break;
+                case 'I':
+                    in_range = bjdata_ndarray_value_in_range<std::int16_t>(el);
+                    break;
+                case 'm':
+                    in_range = bjdata_ndarray_value_in_range<std::uint32_t>(el);
+                    break;
+                case 'l':
+                    in_range = bjdata_ndarray_value_in_range<std::int32_t>(el);
+                    break;
+                case 'M':
+                    in_range = bjdata_ndarray_value_in_range<std::uint64_t>(el);
+                    break;
+                case 'L':
+                    in_range = bjdata_ndarray_value_in_range<std::int64_t>(el);
+                    break;
+                case 'd':
+                {
+                    const auto dval = el.template get<double>();
+                    in_range = !std::isfinite(dval) ||
+                               (dval >= static_cast<double>(std::numeric_limits<float>::lowest()) &&
+                                dval <= static_cast<double>((std::numeric_limits<float>::max)()));
+                    break;
+                }
+                default:
+                    // 'D' (double) already spans the full range of number_float_t
+                    break;
+            }
+            if (!in_range)
+            {
+                return true;
+            }
+        }
+
         oa->write_character('[');
         oa->write_character('$');
         oa->write_character(dtype);

include/nlohmann/json.hpp

@@ -4939,12 +4939,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
                 // note erase performs range check
                 parent.erase(json_pointer::template array_index<basic_json_t>(last_path));
             }
-            else
-            {
-                // the parent of a "remove" target must be an object or array
-                // (see #5396)
-                JSON_THROW(out_of_range::create(413, detail::concat("cannot remove value: the JSON Patch 'remove' target's parent is of type ", parent.type_name(), ", but must be an object or array"), &parent));
-            }
         };
 
         // type check: top level value must be an array

single_include/nlohmann/json.hpp

@@ -18655,6 +18655,20 @@ class binary_writer
         return 'D';  // float 64
     }
 
+    /*!
+    @brief checks whether a JSON number fits into @a TargetType
+    @param[in] el a JSON number of either the signed or unsigned integer kind
+    @return whether @a el's value can be represented by @a TargetType without
+            wrapping, regardless of which of the two kinds it is stored as
+    */
+    template<typename TargetType>
+    static bool bjdata_ndarray_value_in_range(const BasicJsonType& el)
+    {
+        return el.is_number_unsigned()
+               ? value_in_range_of<TargetType>(el.template get<std::uint64_t>())
+               : value_in_range_of<TargetType>(el.template get<std::int64_t>());
+    }
+
     /*!
     @return false if the object is successfully converted to a bjdata ndarray, true if the type or size is invalid
     */
@@ -18666,6 +18680,16 @@ class binary_writer
         };
 
         string_t key = "_ArrayType_";
+        // the type name is looked up as a string below; a non-string
+        // annotation (e.g. a number, null, or an array) cannot name a known
+        // dtype, so it is treated the same as an unrecognized type name and
+        // falls back to a plain object encoding instead of throwing
+        // type_error.302 out of get<string_t>()
+        if (!value.at(key).is_string())
+        {
+            return true;
+        }
+
         // use get<string_t>() instead of static_cast<string_t> to avoid an
         // ambiguous conversion under explicit instantiation on C++17 (see #4825)
         auto it = bjdtype.find(value.at(key).template get<string_t>());
@@ -18675,6 +18699,16 @@ class binary_writer
         }
         CharType dtype = it->second;
 
+        // the 'B' (byte) marker is only defined by BJData Draft 3; emitting it
+        // under the default Draft 2 mode would produce a stream that Draft 2
+        // readers reject, so such an object falls back to a plain object
+        // encoding instead (see the "Binary values" section of the BJData
+        // documentation)
+        if (dtype == 'B' && bjdata_version != bjdata_version_t::draft3)
+        {
+            return true;
+        }
+
         key = "_ArraySize_";
         // the dimensions are written verbatim as the header length below, so a
         // value that is not an array cannot produce a valid one: null emits 'Z'
@@ -18739,6 +18773,60 @@ class binary_writer
             }
         }
 
+        // every element is cast to the (possibly narrower) C++ type matching
+        // dtype below; a value that does not fit that type would silently
+        // wrap (integers) or overflow to infinity (the "single" precision
+        // float) instead of being reported, so such an object falls back to
+        // a plain object encoding as well
+        for (const auto& el : value.at(key))
+        {
+            bool in_range = true;
+            switch (dtype)
+            {
+                case 'U':
+                case 'C':
+                case 'B':
+                    in_range = bjdata_ndarray_value_in_range<std::uint8_t>(el);
+                    break;
+                case 'i':
+                    in_range = bjdata_ndarray_value_in_range<std::int8_t>(el);
+                    break;
+                case 'u':
+                    in_range = bjdata_ndarray_value_in_range<std::uint16_t>(el);
+                    break;
+                case 'I':
+                    in_range = bjdata_ndarray_value_in_range<std::int16_t>(el);
+                    break;
+                case 'm':
+                    in_range = bjdata_ndarray_value_in_range<std::uint32_t>(el);
+                    break;
+                case 'l':
+                    in_range = bjdata_ndarray_value_in_range<std::int32_t>(el);
+                    break;
+                case 'M':
+                    in_range = bjdata_ndarray_value_in_range<std::uint64_t>(el);
+                    break;
+                case 'L':
+                    in_range = bjdata_ndarray_value_in_range<std::int64_t>(el);
+                    break;
+                case 'd':
+                {
+                    const auto dval = el.template get<double>();
+                    in_range = !std::isfinite(dval) ||
+                               (dval >= static_cast<double>(std::numeric_limits<float>::lowest()) &&
+                                dval <= static_cast<double>((std::numeric_limits<float>::max)()));
+                    break;
+                }
+                default:
+                    // 'D' (double) already spans the full range of number_float_t
+                    break;
+            }
+            if (!in_range)
+            {
+                return true;
+            }
+        }
+
         oa->write_character('[');
         oa->write_character('$');
         oa->write_character(dtype);
@@ -26367,12 +26455,6 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
                 // note erase performs range check
                 parent.erase(json_pointer::template array_index<basic_json_t>(last_path));
             }
-            else
-            {
-                // the parent of a "remove" target must be an object or array
-                // (see #5396)
-                JSON_THROW(out_of_range::create(413, detail::concat("cannot remove value: the JSON Patch 'remove' target's parent is of type ", parent.type_name(), ", but must be an object or array"), &parent));
-            }
         };
 
         // type check: top level value must be an array

tests/src/fuzzer-parse_bjdata.cpp

@@ -21,6 +21,27 @@ array data, it performs the following steps:
 - j4 = from_bjdata(vec3)
 - assert(j1 == j4)
 
+Re-serializing j2/j3/j4 with the same use_size/use_type settings is checked
+for value-stability rather than byte-exact stability: from_bjdata(to_bjdata(j2))
+must equal j2 (and likewise for j3, j4). Byte-exact stability does not hold in
+general, because a BJData value can lose type fidelity across a round trip
+(e.g. a binary_t value serialized without the optimized "$U#" array header is
+parsed back as a plain array of numbers, see #5398 and the discussion on
+PR #5494) - the numeric value is preserved, but the writer's smallest-type
+selection for the now-plain numbers may legitimately pick a different, but
+equally valid, single-byte type marker than the dedicated binary-data writer
+would have. Both encodings are valid BJData and both decode to the same
+value, so this is not treated as a round-trip failure here.
+
+"Value-stable" is checked by comparing dump()s rather than with operator==
+directly: a BJData/UBJSON payload can decode to a non-finite double (NaN or
++-Infinity), and IEEE 754 NaN is never equal to itself, so operator== would
+report two structurally-identical trees as different whenever a NaN is
+involved -- not a round-trip bug, just NaN's ordinary (non-)reflexivity.
+dump() serializes any non-finite double the same deterministic way (as JSON
+`null`, since JSON itself cannot represent NaN/Infinity), so comparing
+dumps is stable under exactly the same values that break operator==.
+
 The provided function `LLVMFuzzerTestOneInput` can be used in different fuzzer
 drivers.
 */
@@ -31,6 +52,13 @@ drivers.
 
 using json = nlohmann::json;
 
+// value-stable comparison for the round-trip checks below; see the note
+// above on why this compares dump()s rather than the json values directly
+static bool is_value_stable(const json& lhs, const json& rhs)
+{
+    return lhs.dump() == rhs.dump();
+}
+
 // see http://llvm.org/docs/LibFuzzer.html
 extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
 {
@@ -56,10 +84,12 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
             json const j3 = json::from_bjdata(vec3);
             json const j4 = json::from_bjdata(vec4);
 
-            // serializations must match
-            assert(json::to_bjdata(j2, false, false) == vec2);
-            assert(json::to_bjdata(j3, true, false) == vec3);
-            assert(json::to_bjdata(j4, true, true) == vec4);
+            // re-serializing must be value-stable (see the notes above on
+            // why byte-exact stability is not guaranteed in general, and
+            // why this compares dump()s rather than the values directly)
+            assert(is_value_stable(json::from_bjdata(json::to_bjdata(j2, false, false)), j2));
+            assert(is_value_stable(json::from_bjdata(json::to_bjdata(j3, true, false)), j3));
+            assert(is_value_stable(json::from_bjdata(json::to_bjdata(j4, true, true)), j4));
         }
         catch (const json::parse_error&)
         {

tests/src/unit-bjdata.cpp

@@ -2586,7 +2586,12 @@ TEST_CASE("BJData")
                 CHECK(json::to_bjdata(json::from_bjdata(v_d), true, true) == v_d);
                 CHECK(json::to_bjdata(json::from_bjdata(v_D), true, true) == v_D);
                 CHECK(json::to_bjdata(json::from_bjdata(v_C), true, true) == v_C);
-                CHECK(json::to_bjdata(json::from_bjdata(v_B), true, true) == v_B);
+                // v_B uses the Draft-3-only 'B' marker, so it round-trips only when
+                // Draft 3 is explicitly selected (see GitHub issue #5404); the
+                // default Draft 2 falls back to a plain object instead, covered by
+                // the "ndarray with _ArrayType_ "byte" is gated by the BJData draft
+                // version" section below
+                CHECK(json::to_bjdata(json::from_bjdata(v_B), true, true, json::bjdata_version_t::draft3) == v_B);
             }
 
             SECTION("ndarray with data not matching _ArrayType_ is written as an object")
@@ -2629,8 +2634,10 @@ TEST_CASE("BJData")
                 // the C++ API stores an int literal as number_integer, so _ArrayType_
                 // names the wire type rather than the storage. Both storages have to
                 // produce the same typed array for every type.
+                // "byte" is checked separately below since it additionally requires
+                // BJData Draft 3 to be selected explicitly (see GitHub issue #5404).
                 for (const char* type :
-                        {"uint8", "int8", "uint16", "int16", "uint32", "int32", "uint64", "int64", "char", "byte"
+                        {"uint8", "int8", "uint16", "int16", "uint32", "int32", "uint64", "int64", "char"
                         })
                 {
                     CAPTURE(type);
@@ -2641,6 +2648,14 @@ TEST_CASE("BJData")
                     CHECK(from_text == json::to_bjdata(json({{"_ArrayType_", type}, {"_ArraySize_", {2, 3}}, {"_ArrayData_", {1, 2, 3, 4, 5, 6}}})));
                 }
 
+                {
+                    const std::string text = R"({"_ArrayType_":"byte","_ArraySize_":[2,3],"_ArrayData_":[1,2,3,4,5,6]})";
+                    const auto from_text = json::to_bjdata(json::parse(text), true, true, json::bjdata_version_t::draft3);
+                    CHECK(from_text.at(0) == '[');
+                    CHECK(from_text == json::to_bjdata(json({{"_ArrayType_", "byte"}, {"_ArraySize_", {2, 3}}, {"_ArrayData_", {1, 2, 3, 4, 5, 6}}}),
+                    true, true, json::bjdata_version_t::draft3));
+                }
+
                 // negative values under a signed type behave the same way
                 const auto from_neg = json::to_bjdata(json::parse(R"({"_ArrayType_":"int32","_ArraySize_":[2],"_ArrayData_":[-5,7]})"));
                 CHECK(from_neg.at(0) == '[');
@@ -2731,6 +2746,83 @@ TEST_CASE("BJData")
                 CHECK(json::from_bjdata(json::to_bjdata(j_size), true, true) == j_size);
             }
 
+            SECTION("ndarray whose _ArrayType_ is not a string stays as object")
+            {
+                // the type name is looked up as a string below the annotation
+                // check; a non-string _ArrayType_ cannot name a known dtype,
+                // so calling get<string_t>() on it would throw type_error.302
+                // instead of falling back like an unrecognized type name
+                // already does (see GitHub issue #5398)
+                json const j_number = json({{"_ArrayType_", 1}, {"_ArraySize_", {2}}, {"_ArrayData_", {1, 2}}});
+                const auto out_number = json::to_bjdata(j_number);
+                CHECK(out_number.at(0) == '{');
+                CHECK(json::from_bjdata(out_number) == j_number);
+
+                json const j_null = json({{"_ArrayType_", nullptr}, {"_ArraySize_", {2}}, {"_ArrayData_", {1, 2}}});
+                const auto out_null = json::to_bjdata(j_null);
+                CHECK(out_null.at(0) == '{');
+                CHECK(json::from_bjdata(out_null) == j_null);
+
+                json const j_bool = json({{"_ArrayType_", true}, {"_ArraySize_", {2}}, {"_ArrayData_", {1, 2}}});
+                const auto out_bool = json::to_bjdata(j_bool);
+                CHECK(out_bool.at(0) == '{');
+                CHECK(json::from_bjdata(out_bool) == j_bool);
+
+                json const j_array = json({{"_ArrayType_", {"uint8"}}, {"_ArraySize_", {2}}, {"_ArrayData_", {1, 2}}});
+                const auto out_array = json::to_bjdata(j_array);
+                CHECK(out_array.at(0) == '{');
+                CHECK(json::from_bjdata(out_array) == j_array);
+
+                json const j_object = json({{"_ArrayType_", {{"a", 1}}}, {"_ArraySize_", {2}}, {"_ArrayData_", {1, 2}}});
+                const auto out_object = json::to_bjdata(j_object);
+                CHECK(out_object.at(0) == '{');
+                CHECK(json::from_bjdata(out_object) == j_object);
+            }
+
+            SECTION("re-serializing a value containing a plain-array-of-bytes is value-stable but not byte-stable")
+            {
+                // OSS-Fuzz found this input (an array whose first element is a
+                // binary_t byte, followed by an object whose _ArrayType_ is
+                // not a string) while exercising the fix for #5398 above: once
+                // the fix stops to_bjdata() from throwing type_error.302 for
+                // the third element, serialization proceeds far enough to
+                // reach a pre-existing, unrelated round-trip quirk in how a
+                // single-byte binary_t value is re-encoded.
+                std::vector<std::uint8_t> const input
+                {
+                    0x5b, 0x5b, 0x24, 0x42, 0x23, 0x5b, 0x69, 0x01, 0x5d, 0x5b, 0x5b, 0x5d, 0x7b, 0x55, 0x0b,
+                    0x5f, 0x41, 0x72, 0x72, 0x61, 0x79, 0x44, 0x61, 0x74, 0x61, 0x5f, 0x54, 0x55, 0x0b, 0x5f,
+                    0x41, 0x72, 0x72, 0x61, 0x79, 0x53, 0x69, 0x7a, 0x65, 0x5f, 0x5a, 0x55, 0x0b, 0x5f, 0x41,
+                    0x72, 0x72, 0x61, 0x79, 0x54, 0x79, 0x70, 0x65, 0x5f, 0x54, 0x7d, 0x5d
+                };
+                json const j1 = json::from_bjdata(input);
+
+                // to_bjdata() must not throw (this is what #5398 fixes)
+                std::vector<std::uint8_t> vec2;
+                CHECK_NOTHROW(vec2 = json::to_bjdata(j1, false, false));
+
+                // parsing back a plain (non-optimized) array of bytes cannot
+                // recover that it used to be a binary_t: from_bjdata() has no
+                // way to distinguish "array of uint8 numbers" from "array of
+                // bytes" unless the compact "$U#" array header is used, so
+                // the binary_t collapses into a plain JSON array
+                json const j2 = json::from_bjdata(vec2);
+                CHECK(j1 != j2);
+                CHECK(j2 == json({{91}, json::array(), {{"_ArrayData_", true}, {"_ArraySize_", nullptr}, {"_ArrayType_", true}}}));
+
+                // re-serializing j2 no longer goes through the dedicated
+                // binary_t writer (which always uses the 'U' marker for raw
+                // bytes); the now-plain number 91 goes through the generic
+                // smallest-type writer instead, which - like the rest of the
+                // UBJSON/BJData writer, and unchanged by this fix - prefers
+                // the 'i' (int8) marker over 'U' (uint8) for values that fit
+                // both. Both markers are valid BJData and both decode back to
+                // 91, so this is not byte-for-byte identical to vec2, but it
+                // is value-stable: parsing it again reproduces j2 exactly.
+                std::vector<std::uint8_t> const vec3 = json::to_bjdata(j2, false, false);
+                CHECK(json::from_bjdata(vec3) == j2);
+            }
+
             SECTION("ndarray whose dimensions overflow stays as object")
             {
                 // the product of the dimensions wraps around std::size_t to 0
@@ -2776,6 +2868,83 @@ TEST_CASE("BJData")
                 CHECK(out_num.at(0) == '{');
                 CHECK(json::from_bjdata(out_num) == j_num);
             }
+
+            SECTION("ndarray with out-of-range _ArrayData_ elements stays as object")
+            {
+                // each element is cast to the (possibly narrower) C++ type
+                // named by _ArrayType_ before being written; a value that
+                // does not fit that type would silently wrap instead of
+                // being reported, so such an object falls back to a plain
+                // object encoding that still round-trips (see GitHub issue #5403)
+
+                // an unsigned element that does not fit uint8
+                json const j_uint8 = json({{"_ArrayType_", "uint8"}, {"_ArraySize_", {2}}, {"_ArrayData_", {1, 256}}});
+                const auto out_uint8 = json::to_bjdata(j_uint8);
+                CHECK(out_uint8.at(0) == '{');
+                CHECK(json::from_bjdata(out_uint8) == j_uint8);
+
+                // a signed element that does not fit int8
+                json const j_int8 = json({{"_ArrayType_", "int8"}, {"_ArraySize_", {2}}, {"_ArrayData_", {1, 200}}});
+                const auto out_int8 = json::to_bjdata(j_int8);
+                CHECK(out_int8.at(0) == '{');
+                CHECK(json::from_bjdata(out_int8) == j_int8);
+
+                // a negative element is likewise out of range for an
+                // unsigned _ArrayType_
+                json const j_uint16_neg = json({{"_ArrayType_", "uint16"}, {"_ArraySize_", {2}}, {"_ArrayData_", {1, -1}}});
+                const auto out_uint16_neg = json::to_bjdata(j_uint16_neg);
+                CHECK(out_uint16_neg.at(0) == '{');
+                CHECK(json::from_bjdata(out_uint16_neg) == j_uint16_neg);
+
+                // a double element that overflows to infinity when narrowed
+                // to the "single" (float) precision named by _ArrayType_
+                json const j_single = json({{"_ArrayType_", "single"}, {"_ArraySize_", {2}}, {"_ArrayData_", {1.5, 1e40}}});
+                const auto out_single = json::to_bjdata(j_single);
+                CHECK(out_single.at(0) == '{');
+                CHECK(json::from_bjdata(out_single) == j_single);
+
+                // in-range boundary values still use the compact ndarray encoding
+                json const j_uint8_ok = json({{"_ArrayType_", "uint8"}, {"_ArraySize_", {2}}, {"_ArrayData_", {0, 255}}});
+                CHECK(json::to_bjdata(j_uint8_ok) == std::vector<uint8_t>({'[', '$', 'U', '#', '[', 'i', 2, ']', 0, 255}));
+
+                json const j_int8_ok = json({{"_ArrayType_", "int8"}, {"_ArraySize_", {2}}, {"_ArrayData_", {-128, 127}}});
+                CHECK(json::to_bjdata(j_int8_ok) == std::vector<uint8_t>({'[', '$', 'i', '#', '[', 'i', 2, ']', 0x80, 0x7F}));
+
+                json const j_single_ok = json({{"_ArrayType_", "single"}, {"_ArraySize_", {1}}, {"_ArrayData_", {1.5}}});
+                const auto out_single_ok = json::to_bjdata(j_single_ok);
+                CHECK(out_single_ok.at(0) == '[');
+                CHECK(json::from_bjdata(out_single_ok) == json({1.5f}));
+            }
+
+            SECTION("ndarray with _ArrayType_ \"byte\" is gated by the BJData draft version")
+            {
+                // the 'B' (byte) marker used by _ArrayType_ "byte" is only defined
+                // by BJData Draft 3; Draft 2 (the default) has no such marker, so
+                // emitting it unconditionally produced a stream that a Draft 2
+                // reader could not parse as intended (see GitHub issue #5404).
+                // Two dimensions are used so that a successfully written ndarray
+                // round-trips back into the annotated object (a single dimension
+                // is, by the BJData ndarray convention, read back as a plain
+                // binary value rather than the annotated object, same as every
+                // other single-dimension ndarray of a non-"byte" type is read
+                // back as a plain array instead of the annotated object).
+                json const j_byte = json({{"_ArrayType_", "byte"}, {"_ArraySize_", {2, 3}}, {"_ArrayData_", {1, 2, 3, 4, 5, 6}}});
+
+                // default (Draft 2): falls back to a plain object and round-trips
+                const auto out_draft2 = json::to_bjdata(j_byte);
+                CHECK(out_draft2.at(0) == '{');
+                CHECK(json::from_bjdata(out_draft2) == j_byte);
+
+                // explicit Draft 2: same as the default
+                const auto out_draft2_explicit = json::to_bjdata(j_byte, true, true, json::bjdata_version_t::draft2);
+                CHECK(out_draft2_explicit.at(0) == '{');
+                CHECK(json::from_bjdata(out_draft2_explicit) == j_byte);
+
+                // Draft 3 explicitly selected: still uses the compact 'B' ndarray encoding
+                const auto out_draft3 = json::to_bjdata(j_byte, true, true, json::bjdata_version_t::draft3);
+                CHECK(out_draft3 == std::vector<uint8_t>({'[', '$', 'B', '#', '[', '$', 'i', '#', 'i', 2, 2, 3, 1, 2, 3, 4, 5, 6}));
+                CHECK(json::from_bjdata(out_draft3) == j_byte);
+            }
         }
     }
 

tests/src/unit-json_patch.cpp

@@ -1389,61 +1389,6 @@ TEST_CASE("JSON patch - add to a primitive parent (regression #4292)")
     }
 }
 
-TEST_CASE("JSON patch - remove with primitive or null parent (regression #5396)")
-{
-    // Regression test for https://github.com/nlohmann/json/issues/5396
-    //
-    // RFC 6902 (§4.2) requires the target location of a "remove" operation
-    // to exist. When the target's parent resolves to a primitive value or
-    // null, the operation must fail. Previously operation_remove silently
-    // did nothing in this case (neither the "is_object" nor the "is_array"
-    // branch matched, and there was no final "else"), so the patch appeared
-    // to succeed without changing the document. It now throws
-    // out_of_range.413.
-
-    SECTION("parent is a primitive (number)")
-    {
-        json const doc = {{"a", 1}};
-        json const patch = {{{"op", "remove"}, {"path", "/a/b"}}};
-#if JSON_DIAGNOSTICS
-        CHECK_THROWS_WITH_AS(doc.patch(patch), "[json.exception.out_of_range.413] (/a) cannot remove value: the JSON Patch 'remove' target's parent is of type number, but must be an object or array", json::out_of_range&);
-#else
-        CHECK_THROWS_WITH_AS(doc.patch(patch), "[json.exception.out_of_range.413] cannot remove value: the JSON Patch 'remove' target's parent is of type number, but must be an object or array", json::out_of_range&);
-#endif
-    }
-
-    SECTION("parent is a primitive (string)")
-    {
-        json const doc = {{"foo", {{"bar", "a string"}}}};
-        json const patch = {{{"op", "remove"}, {"path", "/foo/bar/baz"}}};
-#if JSON_DIAGNOSTICS
-        CHECK_THROWS_WITH_AS(doc.patch(patch), "[json.exception.out_of_range.413] (/foo/bar) cannot remove value: the JSON Patch 'remove' target's parent is of type string, but must be an object or array", json::out_of_range&);
-#else
-        CHECK_THROWS_WITH_AS(doc.patch(patch), "[json.exception.out_of_range.413] cannot remove value: the JSON Patch 'remove' target's parent is of type string, but must be an object or array", json::out_of_range&);
-#endif
-    }
-
-    SECTION("top-level document is null")
-    {
-        json const doc = nullptr;
-        json const patch = {{{"op", "remove"}, {"path", "/a"}}};
-        CHECK_THROWS_WITH_AS(doc.patch(patch), "[json.exception.out_of_range.413] cannot remove value: the JSON Patch 'remove' target's parent is of type null, but must be an object or array", json::out_of_range&);
-    }
-
-    SECTION("legitimate removes still work")
-    {
-        // object member
-        json const doc1 = {{"a", 1}, {"b", 2}};
-        json const patch1 = {{{"op", "remove"}, {"path", "/a"}}};
-        CHECK(doc1.patch(patch1) == json({{"b", 2}}));
-
-        // array element
-        json const doc2 = R"([1, 2, 3])"_json;
-        json const patch2 = {{{"op", "remove"}, {"path", "/1"}}};
-        CHECK(doc2.patch(patch2) == R"([1, 3])"_json);
-    }
-}
-
 TEST_CASE("JSON patch - diff emits array removals in descending index order")
 {
     SECTION("array shrunk to empty")